To prevent pornography from spreading on the Internet effectively, we propose a novel method of adult image detection which combines bag-of-visual-words (BoVW) based on region of interest (ROI) and color moments (CM). The goal of BoVW is to automatically mine the local patterns of adult contents, called visual words. The usual BoVW method clusters visual words from the patches in the whole image and adopts the weighting schemes of hard assignment. However, there are many background noises in the whole image and soft-weighting scheme is better than hard assignment. Therefore, we propose the method of BoVW based on ROI, which includes two perspectives. Firstly, we propose to create visual words in ROI for adult image detection. The representative power of visual words can be improved because the patches in ROI are more indicative to adult contents than those in the whole image. Secondly, soft-weighting scheme is adopted to detect adult images. Moreover, CM is selected by evaluating some commonly-used global features to be combined with BoVW based on ROI. The experiments and the comparison with the state-of-the-art methods show that our method is able to remarkably improve the performance of adult image detection.