Not Suitable for Work (NSFW) classification using deep neural network Caffe models.

Overview

Open nsfw model

This repo contains code for running Not Suitable for Work (NSFW) classification deep neural network Caffe models. Please refer our blog post which describes this work and experiments in more detail.

Not suitable for work classifier

Detecting offensive / adult images is an important problem which researchers have tackled for decades. With the evolution of computer vision and deep learning the algorithms have matured and we are now able to classify an image as not suitable for work with greater precision.

Defining NSFW material is subjective and the task of identifying these images is non-trivial. Moreover, what may be objectionable in one context can be suitable in another. For this reason, the model we describe below focuses only on one type of NSFW content: pornographic images. The identification of NSFW sketches, cartoons, text, images of graphic violence, or other types of unsuitable content is not addressed with this model.

Since images and user generated content dominate the internet today, filtering nudity and other not suitable for work images becomes an important problem. In this repository we opensource a Caffe deep neural network for preliminary filtering of NSFW images.

Demo Image

Usage

  • The network takes in an image and gives output a probability (score between 0-1) which can be used to filter not suitable for work images. Scores < 0.2 indicate that the image is likely to be safe with high probability. Scores > 0.8 indicate that the image is highly probable to be NSFW. Scores in middle range may be binned for different NSFW levels.
  • Depending on the dataset, usecase and types of images, we advise developers to choose suitable thresholds. Due to difficult nature of problem, there will be errors, which depend on use-cases / definition / tolerance of NSFW. Ideally developers should create an evaluation set according to the definition of what is safe for their application, then fit a ROC curve to choose a suitable threshold if they are using the model as it is.
  • Results can be improved by fine-tuning the model for your dataset/ use case / definition of NSFW. We do not provide any guarantees of accuracy of results. Please read the disclaimer below.
  • Using human moderation for edge cases in combination with the machine learned solution will help improve performance.

Description of model

We trained the model on the dataset with NSFW images as positive and SFW(suitable for work) images as negative. These images were editorially labelled. We cannot release the dataset or other details due to the nature of the data.

We use CaffeOnSpark which is a wonderful framework for distributed learning that brings deep learning to Hadoop and Spark clusters for training models for our experiments. Big thanks to the CaffeOnSpark team!

The deep model was first pretrained on ImageNet 1000 class dataset. Then we finetuned the weights on the NSFW dataset. We used the thin resnet 50 1by2 architecture as the pretrained network. The model was generated using pynetbuilder tool and replicates the residual network paper's 50 layer network (with half number of filters in each layer). You can find more details on how the model was generated and trained here

Please note that deeper networks, or networks with more filters can improve accuracy. We train the model using a thin residual network architecture, since it provides good tradeoff in terms of accuracy, and the model is light-weight in terms of runtime (or flops) and memory (or number of parameters).

Docker Quickstart

This Docker quickstart guide can be used for evaluating the model quickly with minimal dependency installation.

Install Docker Engine

Build a caffe docker image (CPU)

docker build -t caffe:cpu https://raw.githubusercontent.com/BVLC/caffe/master/docker/cpu/Dockerfile

Check the caffe installation

docker run caffe:cpu caffe --version
caffe version 1.0.0-rc3

Run the docker image with a volume mapped to your open_nsfw repository. Your test_image.jpg should be located in this same directory.

cd open_nsfw
docker run --volume=$(pwd):/workspace caffe:cpu \
python ./classify_nsfw.py \
--model_def nsfw_model/deploy.prototxt \
--pretrained_model nsfw_model/resnet_50_1by2_nsfw.caffemodel \
test_image.jpg

We will get the NSFW score returned:

NSFW score:   0.14057905972

Running the model

To run this model, please install Caffe and its python extension and make sure pycaffe is available in your PYTHONPATH.

We can use the classify.py script to run the NSFW model. For convenience, we have provided the script in this repo as well, and it prints the NSFW score.

python ./classify_nsfw.py \
--model_def nsfw_model/deploy.prototxt \
--pretrained_model nsfw_model/resnet_50_1by2_nsfw.caffemodel \
INPUT_IMAGE_PATH 

Disclaimer

The definition of NSFW is subjective and contextual. This model is a general purpose reference model, which can be used for the preliminary filtering of pornographic images. We do not provide guarantees of accuracy of output, rather we make this available for developers to explore and enhance as an open source project. Results can be improved by fine-tuning the model for your dataset.

License

Code licensed under the [BSD 2 clause license] (https://github.com/BVLC/caffe/blob/master/LICENSE). See LICENSE file for terms.

Contact

The model was trained by Jay Mahadeokar, in collaboration with Sachin Farfade , Amar Ramesh Kamat, Armin Kappeler and others. Special thanks to Gerry Pesavento for taking the initiative for open-sourcing this model. If you have any queries, please raise an issue and we will get back ASAP.

Comments
  • Releasing Other Models?

    Releasing Other Models?

    First of all, thanks for the release! Second: its a pity that the released model (Resnet) has a highly non-standard layer ("Scale", which has a learnable bias parameter). That means that this model cannot be converted to other frameworks (Torch, MXNet, etc).

    Would it be possible to release one of your other models as well, such as the Googlenet model?

    Thanks!

    opened by sbodenstein 10
  • Add Dockerfile and Docker quickstart guide

    Add Dockerfile and Docker quickstart guide

    I've created a simple Dockerfile and guide to be considered for inclusion in this repo. The goal in mind is to lower the entry barrier for users that want to quickly get started and explore this repo without having to install all of the projects dependencies (Caffe).

    opened by neufeldtech 9
  • ImportError: No module named skimage.io

    ImportError: No module named skimage.io

    hi, I install docker and Build a caffe docker image sucessfully, but is run the programm, it report the error. please help me.

    root@iZ25q8sqgi7Z:~/open_nsfw# docker run caffe:cpu caffe --version libdc1394 error: Failed to initialize libdc1394 caffe version 1.0.0-rc3

    root@iZ25q8sqgi7Z:~/open_nsfw# docker run --volume=$(pwd):/workspace caffe:cpu \

    python ./classify_nsfw.py
    --model_def nsfw_model/deploy.prototxt
    --pretrained_model nsfw_model/resnet_50_1by2_nsfw.caffemodel
    test_image.jpg libdc1394 error: Failed to initialize libdc1394 Traceback (most recent call last): File "./classify_nsfw.py", line 16, in import caffe File "/opt/caffe/python/caffe/init.py", line 1, in from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver File "/opt/caffe/python/caffe/pycaffe.py", line 15, in import caffe.io File "/opt/caffe/python/caffe/io.py", line 2, in import skimage.io ImportError: No module named skimage.io

    opened by fengjinhai 8
  • train_val.prototxt file for fine-tuning

    train_val.prototxt file for fine-tuning

    Hello. Thank you for sharing this project. The readme suggests fine-tuning for improved performance, and I'm writing to request direction on how to convert the deploy.prototxt file into a train_val.prototxt file for fine-tuning. Or, more simply, to request a sample train_val.prototxt file.

    Thank you.

    opened by michaelholm-ce 5
  • OpenNSFW with Docker

    OpenNSFW with Docker

    I would like to share how to run NSFT with docker

    1. Build a caffe docker image (GPU)
    $ cd open_nsfw
    $ wget https://raw.githubusercontent.com/BVLC/caffe/master/docker/standalone/gpu/Dockerfile
    $ docker build -t caffe:cpu ./
    

    To build for cpu only get the Dockerfile for cpu from Caffe repo

    $ wget https://raw.githubusercontent.com/BVLC/caffe/master/docker/standalone/cpu/Dockerfile
    
    1. When the image it's ready, we check caffe installation
    $ docker run -ti caffe:cpu caffe --version
    caffe version 1.0.0-rc3
    
    1. We run bash in this folder as mapped volume with the option --volume in order to run the training manually
    docker run -ti --volume=$(pwd):/workspace caffe:cpu bash
    

    so we are in the docker caffe image

    root@8855e6188183:/workspace#
    
    1. Run nsfw in the current container volume
    root@8855e6188183:/workspace#python ./classify_nsfw.py --model_def nsfw_model/deploy.prototxt --pretrained_model nsfw_model/resnet_50_1by2_nsfw.caffemodel anderson_210.jpg
    

    and we are done

    NSFW score:   0.14057905972
    
    opened by loretoparisi 5
  • Building docker image fails

    Building docker image fails

    $> docker build -t caffe:cpu https://raw.githubusercontent.com/BVLC/caffe/master/docker/cpu/Dockerfile
    [...]
    Get:469 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 libopencv2.4-jni amd64 2.4.9.1+dfsg-1.5ubuntu1.1 [148 kB]
    Get:470 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 libopencv2.4-java all 2.4.9.1+dfsg-1.5ubuntu1.1 [433 kB]
    Get:471 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 libopencv-dev amd64 2.4.9.1+dfsg-1.5ubuntu1.1 [169 kB]
    Get:472 http://archive.ubuntu.com/ubuntu xenial/main amd64 libprotobuf-lite9v5 amd64 2.6.1-1.3 [58.4 kB]
    Get:473 http://archive.ubuntu.com/ubuntu xenial/main amd64 libprotobuf9v5 amd64 2.6.1-1.3 [326 kB]
    Get:474 http://archive.ubuntu.com/ubuntu xenial/main amd64 libprotoc9v5 amd64 2.6.1-1.3 [273 kB]
    Get:475 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-decorator all 4.0.6-1 [9326 B]
    Get:476 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-numpy amd64 1:1.11.0-1ubuntu1 [1763 kB]
    Get:477 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 python-pip-whl all 8.1.1-2ubuntu0.4 [1110 kB]
    Get:478 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 python-pip all 8.1.1-2ubuntu0.4 [144 kB]
    Get:479 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-pkg-resources all 20.7.0-1 [108 kB]
    Get:480 http://archive.ubuntu.com/ubuntu xenial/main amd64 python-setuptools all 20.7.0-1 [169 kB]
    Get:481 http://archive.ubuntu.com/ubuntu xenial/main amd64 libprotobuf-dev amd64 2.6.1-1.3 [473 kB]
    Get:482 http://archive.ubuntu.com/ubuntu xenial/main amd64 libsnappy-dev amd64 1.1.3-2 [24.0 kB]
    Get:483 http://archive.ubuntu.com/ubuntu xenial/main amd64 protobuf-compiler amd64 2.6.1-1.3 [20.4 kB]
    Get:484 http://archive.ubuntu.com/ubuntu xenial/universe amd64 python-scipy amd64 0.17.0-1 [8733 kB]
    debconf: delaying package configuration, since apt-utils is not installed
    Fetched 234 MB in 10s (23.3 MB/s)
    (Reading database ... 4768 files and directories currently installed.)
    Preparing to unpack .../perl-base_5.22.1-9ubuntu0.6_amd64.deb ...
    Unpacking perl-base (5.22.1-9ubuntu0.6) over (5.22.1-9ubuntu0.5) ...
    dpkg: error processing archive /var/cache/apt/archives/perl-base_5.22.1-9ubuntu0.6_amd64.deb (--unpack):
     unable to make backup link of './usr/share/lintian/overrides/perl-base' before installing new version: Invalid cross-device link
    dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
    dpkg: error: error creating new backup file '/var/lib/dpkg/status-old': Invalid cross-device link
    E: Sub-process /usr/bin/dpkg returned an error code (2)
    The command '/bin/sh -c apt-get update && apt-get install -y --no-install-recommends         build-essential         cmake         git         wget         libatlas-base-dev         libboost-all-dev         libgflags-dev         libgoogle-glog-dev         libhdf5-serial-dev         libleveldb-dev         liblmdb-dev         libopencv-dev         libprotobuf-dev         libsnappy-dev         protobuf-compiler         python-dev         python-numpy         python-pip         python-setuptools         python-scipy &&     rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100
    
    opened by 4goettma 4
  • Some images marked as nsfw are wrong

    Some images marked as nsfw are wrong

    Hi guys, I am using open_nsfw to classify images, but some images like the attachment(a avatar's wall, open_nsfw score:0.993217) is marked as nsfw. Anyone knows how to solve this? Thank you very much! 1523765279290

    opened by zhaoxy2850 4
  • there is a RuntimeError: Pickling of

    there is a RuntimeError: Pickling of "caffe._caffe.Net" instances is not enabled (http://www.boost.org/libs/python/doc/v2/pickle.html) when I try run open_nsfw in multi-process

    when I try run open_nsfw in multi-process , a runtimeError raised: Traceback (most recent call last): File "./verify_concurrency.py", line 152, in main(sys.argv) File "./verify_concurrency.py", line 149, in main print res.get()
    File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get raise self._value RuntimeError: Pickling of "caffe._caffe.Net" instances is not enabled (http://www.boost.org/libs/python/doc/v2/pickle.html)

    could some expert help check where is wrong as i'm rookie my command is : ./verify_concurrency.py --model_def ./nsfw_model/deploy.prototxt --pretrained_model ./nsfw_model/resnet_50_1by2_nsfw.caffemodel /home/harold/nsfw/pic/1

    below is my script verify_concurrency.py : #!/usr/bin/env python """ Copyright 2016 Yahoo Inc. Licensed under the terms of the 2 clause BSD license. Please see LICENSE file in the project root for terms. """

    import numpy as np import os import sys import argparse import glob import time from PIL import Image from StringIO import StringIO import caffe from multiprocessing import Pool

    def resize_image(data, sz=(256, 256)): """ Resize image. Please use this resize logic for best results instead of the caffe, since it was used to generate training dataset :param str data: The image data :param sz tuple: The resized image dimensions :returns bytearray: A byte array with the resized image """ img_data = str(data) im = Image.open(StringIO(img_data)) if im.mode != "RGB": im = im.convert('RGB') imr = im.resize(sz, resample=Image.BILINEAR) fh_im = StringIO() imr.save(fh_im, format='JPEG') fh_im.seek(0) return bytearray(fh_im.read())

    def caffe_preprocess_and_compute(file_name, caffe_transformer=None, caffe_net=None, output_layers=None): """ Run a Caffe network on an input image after preprocessing it to prepare it for Caffe. :param file name file_name: PIL image to be input into Caffe. :param caffe.Net caffe_net: A Caffe network with which to process pimg afrer preprocessing. :param list output_layers: A list of the names of the layers from caffe_net whose outputs are to to be returned. If this is None, the default outputs for the network are returned. :return: Returns the requested outputs from the Caffe net. """ if caffe_net is not None:

        # Grab the default output names if none were requested specifically.
        if output_layers is None:
            output_layers = caffe_net.outputs
        pimg = open(file_name).read()
        start_time = time.time()
        img_data_rs = resize_image(pimg, sz=(256, 256))
        image = caffe.io.load_image(StringIO(img_data_rs))
    
        H, W, _ = image.shape
        _, _, h, w = caffe_net.blobs['data'].data.shape
        h_off = max((H - h) / 2, 0)
        w_off = max((W - w) / 2, 0)
        crop = image[h_off:h_off + h, w_off:w_off + w, :]
        transformed_image = caffe_transformer.preprocess('data', crop)
        transformed_image.shape = (1,) + transformed_image.shape
    
        input_name = caffe_net.inputs[0]
        all_outputs = caffe_net.forward_all(blobs=output_layers,
                    **{input_name: transformed_image})
    
        outputs = all_outputs[output_layers[0]][0].astype(float)
        
        return outputs
    else:
        return []
    
    end_time = time.time()
    print("deal image:%s;start time:%.3f;end time:%.3f;score:%.3f;cost time:%.3f" % (file_name,start_time, end_time , outputs[1] , (end_time - start_time)) )
    

    def main(argv): pycaffe_dir = os.path.dirname(file)

    parser = argparse.ArgumentParser()
    # Required arguments: input file.
    parser.add_argument(
        "input_dir",
        help="Directory of the input image file"
    )
    
    # Optional arguments.
    parser.add_argument(
        "--model_def",
        help="Model definition file."
    )
    parser.add_argument(
        "--pretrained_model",
        help="Trained model weights file."
    )
    
    args = parser.parse_args()
    # Pre-load caffe model.
    nsfw_net = caffe.Net(args.model_def,  # pylint: disable=invalid-name
        args.pretrained_model, caffe.TEST)
    
    # Load transformer
    # Note that the parameters are hard-coded for best results
    caffe_transformer = caffe.io.Transformer({'data': nsfw_net.blobs['data'].data.shape})
    caffe_transformer.set_transpose('data', (2, 0, 1))  # move image channels to outermost
    caffe_transformer.set_mean('data', np.array([104, 117, 123]))  # subtract the dataset-mean value in each channel
    caffe_transformer.set_raw_scale('data', 255)  # rescale from [0, 1] to [0, 255]
    caffe_transformer.set_channel_swap('data', (2, 1, 0))  # swap channels from RGB to BGR
    
    start_time  = time.time()
    print("The process starttime: %.3f" % start_time) 
    file_list = os.listdir(args.input_dir)
    multiprocess_pool = Pool(10)
    result = []
    for file in file_list:
    
        # Classify.
    
        p = args.input_dir + '/' + file
        #print("The image %s deal startat %.3f" % (p, time.time()) )
        result.append(multiprocess_pool.apply_async(caffe_preprocess_and_compute, args=(p, caffe_transformer, nsfw_net, ['prob'] ,)))
        #scores = caffe_preprocess_and_compute(p, caffe_transformer=caffe_transformer, caffe_net=nsfw_net, output_layers=['prob'])
    
        # Scores is the array containing SFW / NSFW image probabilities
        # scores[1] indicates the NSFW probability
        #print "NSFW score:  " , scores[1]
        
        #print("The image %s got score: %.3f at %.3f" % (p, scores[1] ,time.time()) )
    
    print 'Waiting for all subprocesses done...'
    multiprocess_pool.close()
    multiprocess_pool.join()
    print 'All subprocesses done.'
    end_time  = time.time()  
    print("The process endtime: %.3f" % end_time) 
    
    print("The process cost time is %.3f for %d images" % ( (end_time - start_time), len(file_list)) )
    for res in result:
        print res.get()    
    

    if name == 'main': main(sys.argv)

    opened by kylezhang1983 4
  • The python script does not run on Python 3 - Probably an issue related to StringIO and PIL

    The python script does not run on Python 3 - Probably an issue related to StringIO and PIL

    After changing some lines, it is still not working (however the includes and the syntax seem to be ok):

      File "classify_nsfw.py", line 128, in <module>
        main(sys.argv)
      File "classify_nsfw.py", line 104, in main
        image_data = open(args.input_file).read()
      File "/usr/lib/python3.5/codecs.py", line 321, in decode
        (result, consumed) = self._buffer_decode(data, self.errors, final)
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
    

    Changed

    from StringIO import StringIO
    

    to

    from io import StringIO
    

    and

    print "NSFW score:  " , scores[1]
    

    to

    print("NSFW score:  %f" %  float(scores[1]))
    
    opened by fabianfrz 4
  • Build error

    Build error

    I got a build error (using the new URL from #51)

    docker build -t caffe:cpu https://raw.githubusercontent.com/BVLC/caffe/master/docker/cpu/Dockerfile

    Downloading build context from remote url: https://raw.githubusercontent.com/BVLC/caffe/master/docker/cpu/Dockerfile  1.349kB
    Sending build context to Docker daemon  3.072kB
    Step 1/12 : FROM ubuntu:16.04
     ---> 0b1edfbffd27
    Step 2/12 : LABEL maintainer [email protected]
     ---> Using cache
     ---> d8b42b3a353c
    Step 3/12 : RUN apt-get update && apt-get install -y --no-install-recommends         build-essential         cmake         git         wget         libatlas-base-dev         libboost-all-dev         libgflags-dev         libgoogle-glog-dev         libhdf5-serial-dev         libleveldb-dev         liblmdb-dev         libopencv-dev         libprotobuf-dev         libsnappy-dev         protobuf-compiler         python-dev         python-numpy         python-pip         python-setuptools         python-scipy &&     rm -rf /var/lib/apt/lists/*
     ---> Using cache
     ---> 2fb926bad41f
    Step 4/12 : ENV CAFFE_ROOT=/opt/caffe
     ---> Using cache
     ---> 7538610c87fa
    Step 5/12 : WORKDIR $CAFFE_ROOT
     ---> Using cache
     ---> 46d10669c5cf
    Step 6/12 : ENV CLONE_TAG=1.0
     ---> Using cache
     ---> 302ad52eb00c
    Step 7/12 : RUN git clone -b ${CLONE_TAG} --depth 1 https://github.com/BVLC/caffe.git . &&     pip install --upgrade pip &&     cd python && for req in $(cat requirements.txt) pydot; do pip install $req; done && cd .. &&     mkdir build && cd build &&     cmake -DCPU_ONLY=1 .. &&     make -j"$(nproc)"
     ---> Running in 4ed26361a7c6
    Cloning into '.'...
    Note: checking out 'eeebdab16155d34ff8f5f42137da7df4d1c7eab0'.
    
    You are in 'detached HEAD' state. You can look around, make experimental
    changes and commit them, and you can discard any commits you make in this
    state without impacting any branches by performing another checkout.
    
    If you want to create a new branch to retain commits you create, you may
    do so (now or later) by using -b with the checkout command again. Example:
    
      git checkout -b <new-branch-name>
    
    Collecting pip
      Downloading https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl (1.3MB)
    Installing collected packages: pip
      Found existing installation: pip 8.1.1
        Not uninstalling pip at /usr/lib/python2.7/dist-packages, outside environment /usr
    Successfully installed pip-10.0.1
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    Traceback (most recent call last):
      File "/usr/bin/pip", line 9, in <module>
        from pip import main
    ImportError: cannot import name main
    The command '/bin/sh -c git clone -b ${CLONE_TAG} --depth 1 https://github.com/BVLC/caffe.git . &&     pip install --upgrade pip &&     cd python && for req in $(cat requirements.txt) pydot; do pip install $req; done && cd .. &&     mkdir build && cd build &&     cmake -DCPU_ONLY=1 .. &&     make -j"$(nproc)"' returned a non-zero code: 1
    
    opened by brentadamson 3
  • System reboot when calling 'caffe_preprocess_and_compute'

    System reboot when calling 'caffe_preprocess_and_compute'

    My PC gets rebooted automatically.

    I notice that caffe model was loaded well, but it crashed at scores = caffe_preprocess_and_compute(image_data, caffe_transformer=caffe_transformer, caffe_net=nsfw_net, output_layers=['prob']).

    It still reboots when I switch to caffe.set_mode_cpu().

    Is there anything that I can do? Thank you.

    opened by LiberiFatali 3
  • Detached head during image building

    Detached head during image building

    Running

    docker build -t caffe:cpu https://raw.githubusercontent.com/BVLC/caffe/master/docker/cpu/Dockerfile

    get

    `Step 7/12 : RUN git clone -b ${CLONE_TAG} --depth 1 https://github.com/BVLC/caffe.git . && pip install --upgrade pip && cd python && for req in $(cat requirements.txt) pydot; do pip install $req; done && cd .. && mkdir build && cd build && cmake -DCPU_ONLY=1 .. && make -j"$(nproc)" ---> Running in 0a91bffbd189 Cloning into '.'... Note: checking out 'eeebdab16155d34ff8f5f42137da7df4d1c7eab0'.

    You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout.

    If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example:

    git checkout -b `

    opened by parserpro 0
  • ImportError: cannot import name main

    ImportError: cannot import name main

    Hi , while running : docker build -t caffe:cpu https://raw.githubusercontent.com/BVLC/caffe/master/docker/cpu/Dockerfile This command , i am getting error at your step 7 , when you are updating pip version.

    File "/usr/bin/pip", line 9, in from pip import main ImportError: cannot import name main

    As i have already installed latest pip version but when you update in it's giving error because of pip.

    opened by akhanivishant 3
  • My NSFW model

    My NSFW model

    Hi Yahoo people.

    Your blog post about the release of this model asks that if people develop their own model, to share it here in a bug ticket.

    In the same blog post, you mention that you built a squeezenet 1.0 model that performed better, but released the ResNet-thin model (based on fine-tuning imagenet models).

    I have trained a squeezenet 1.1 model from-scratch and have achieved, in my tests, slightly better overall accuracy than your released model, so I'd like to share it here and hopefully, as the blog post describes, get it included in your readme.

    The model can be found here:

    https://github.com/TechnikEmpire/NsfwSqueezenet

    I achieved, again in my tests, ~99.1% overall accuracy on ~24,000 random test images, ~12K from each class. The best part is, using OpenCV's DNN module along with Intel's now open-source Inference Engine as the back-end, evaluation on the CPU on modern hardware*, including JPEG decoding and preprocessing, can be executed in ~3msec, or ~333fps. This is more than sufficient for real-time video classification on the CPU, and even ~13x realtime. With the same module, your model executes on the same hardware at ~8msec.

    A cutoff (described in the project wiki) was determined through a process of averaging positive classification probability.

    This work was sponsored in part by CloudVeil Technologies so I'd like to give them a nod there.

    Thank you for open sourcing your model. It inspired me to to develop my model, because it was a clear demonstration of the possibilities in tackling this problem with deep learning.

    • "Modern Hardware" here is Intel i7-6700 @ 3.4 ghz. Also of note, this model executes at ~30msec on a piddly little Intel Atom processor.
    opened by TechnikEmpire 0
  • Fixed Dockerfile not found

    Fixed Dockerfile not found

    fixed unable to prepare context: unable to download remote context https://raw.githubusercontent.com/BVLC/caffe/master/docker/standalone/cpu/Dockerfile: failed to GET https://raw.githubusercontent.com/BVLC/caffe/master/docker/standalone/cpu/Dockerfile with status 404 Not Found: 404: Not Found

    opened by xinzhanguo 1
  • Python3 Support

    Python3 Support

    Hi,

    Most modern Linux distributions are packaging python-caffe only for Python3 and hence getting this up and running has been a pain for me, it would be super awesome if the sample script could be made Python3 friendly. I will probably tidy up what I have locally here to fix this and raise a PR.

    Best Regards,

    Rob

    opened by robputt 8
Owner
Yahoo
This organization is the home to many of the active open source projects published by engineers at Yahoo Inc.
Yahoo
MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

MMdnn MMdnn is a comprehensive and cross-framework tool to convert, visualize and diagnose deep learning (DL) models. The "MM" stands for model manage

Microsoft 5.7k Jan 9, 2023
A static analysis library for computing graph representations of Python programs suitable for use with graph neural networks.

python_graphs This package is for computing graph representations of Python programs for machine learning applications. It includes the following modu

Google Research 258 Dec 29, 2022
The original weights of some Caffe models, ported to PyTorch.

pytorch-caffe-models This repo contains the original weights of some Caffe models, ported to PyTorch. Currently there are: GoogLeNet (Going Deeper wit

Katherine Crowson 9 Nov 4, 2022
Caffe: a fast open framework for deep learning.

Caffe Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by Berkeley AI Research (BAIR)/The Berke

Berkeley Vision and Learning Center 33k Dec 28, 2022
Face Mask Detection is a project to determine whether someone is wearing mask or not, using deep neural network.

face-mask-detection Face Mask Detection is a project to determine whether someone is wearing mask or not, using deep neural network. It contains 3 scr

amirsalar 13 Jan 18, 2022
Rest API Written In Python To Classify NSFW Images.

Rest API Written In Python To Classify NSFW Images.

Wahyusaputra 2 Dec 23, 2021
Python project to take sound as input and output as RGB + Brightness values suitable for DMX

sound-to-light Python project to take sound as input and output as RGB + Brightness values suitable for DMX Current goals: Get one pixel working: Vary

Bobby Cox 1 Nov 17, 2021
hipCaffe: the HIP port of Caffe

Caffe Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by the Berkeley Vision and Learning Cent

ROCm Software Platform 126 Dec 5, 2022
a reimplementation of LiteFlowNet in PyTorch that matches the official Caffe version

pytorch-liteflownet This is a personal reimplementation of LiteFlowNet [1] using PyTorch. Should you be making use of this work, please cite the paper

Simon Niklaus 365 Dec 31, 2022
Caffe-like explicit model constructor. C(onfig)Model

cmodel Caffe-like explicit model constructor. C(onfig)Model Installation pip install git+https://github.com/bonlime/cmodel Usage In order to allow usi

null 1 Feb 18, 2022
Hl classification bc - A Network-Based High-Level Data Classification Algorithm Using Betweenness Centrality

A Network-Based High-Level Data Classification Algorithm Using Betweenness Centr

Esteban Vilca 3 Dec 1, 2022
Pytorch Implementation of Adversarial Deep Network Embedding for Cross-Network Node Classification

Pytorch Implementation of Adversarial Deep Network Embedding for Cross-Network Node Classification (ACDNE) This is a pytorch implementation of the Adv

陈志豪 8 Oct 13, 2022
The official implementation of the IEEE S&P`22 paper "SoK: How Robust is Deep Neural Network Image Classification Watermarking".

Watermark-Robustness-Toolbox - Official PyTorch Implementation This repository contains the official PyTorch implementation of the following paper to

null 49 Dec 19, 2022
BasicNeuralNetwork - This project looks over the basic structure of a neural network and how machine learning training algorithms work

BasicNeuralNetwork - This project looks over the basic structure of a neural network and how machine learning training algorithms work. For this project, I used the sigmoid function as an activation function along with stochastic gradient descent to adjust the weights and biases.

Manas Bommakanti 1 Jan 22, 2022
Deep learning (neural network) based remote photoplethysmography: how to extract pulse signal from video using deep learning tools

Deep-rPPG: Camera-based pulse estimation using deep learning tools Deep learning (neural network) based remote photoplethysmography: how to extract pu

Terbe Dániel 138 Dec 17, 2022
Deep Q-Learning Network in pytorch (not actively maintained)

pytoch-dqn This project is pytorch implementation of Human-level control through deep reinforcement learning and I also plan to implement the followin

Hung-Tu Chen 342 Jan 1, 2023
Scripts for training an AI to play the endless runner Subway Surfers using a supervised machine learning approach by imitation and a convolutional neural network (CNN) for image classification

About subwAI subwAI - a project for training an AI to play the endless runner Subway Surfers using a supervised machine learning approach by imitation

null 82 Jan 1, 2023
paper: Hyperspectral Remote Sensing Image Classification Using Deep Convolutional Capsule Network

DC-CapsNet This is a tensorflow and keras based implementation of DC-CapsNet for HSI in the Remote Sensing Letters R. Lei et al., "Hyperspectral Remot

LEI 7 Nov 29, 2022