LSUN Dataset Documentation and Demo Code

Related tags

Deep Learning lsun
Overview

LSUN

Please check LSUN webpage for more information about the dataset.

Data Release

All the images in one category are stored in one lmdb database file. The value of each entry is the jpg binary data. We resize all the images so that the smaller dimension is 256 and compress the images in jpeg with quality 75.

Citing LSUN

If you find LSUN dataset useful in your research, please consider citing:

@article{yu15lsun,
    Author = {Yu, Fisher and Zhang, Yinda and Song, Shuran and Seff, Ari and Xiao, Jianxiong},
    Title = {LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop},
    Journal = {arXiv preprint arXiv:1506.03365},
    Year = {2015}
}

Download data

Please make sure you have cURL installed

# Download the whole latest data set
python3 download.py
# Download the whole latest data set to <data_dir>
python3 download.py -o <data_dir>
# Download data for bedroom
python3 download.py -c bedroom
# Download testing set
python3 download.py -c test

Demo code

Dependency

Install Python

Install Python dependency: numpy, lmdb, opencv

Usage:

View the lmdb content

python3 data.py view <image db path>

Export the images to a folder

python3 data.py export <image db path> --out_dir <output directory>

Example:

Export all the images in valuation sets in the current folder to a "data" subfolder.

python3 data.py export *_val_lmdb --out_dir data

Submission

We expect one category prediction for each image in the testing set. The name of each image is the key value in the LMDB database. Each category has an index as listed in index list. The submitted results on the testing set will be stored in a text file with one line per image. In each line, there are two fields separated by a whitespace. The first is the image key and the second is the predicted category index. For example:

0001c44e5f5175a7e6358d207660f971d90abaf4 0
000319b73404935eec40ac49d1865ce197b3a553 1
00038e8b13a97577ada8a884702d607220ce6d15 2
00039ba1bf659c30e50b757280efd5eba6fc2fe1 3
...

The score for the submission is the percentage of correctly predicted labels. In our evaluation, we will double check our ground truth labels for the testing images and we may remove some images with controversial labels in the final evaluation.

Comments
  • Download URL not working again again

    Download URL not working again again

    python3 download.py -c bedroom Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 1254, in do_open h.request(req.get_method(), req.selector, req.data, headers) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1107, in request self._send_request(method, url, body, headers) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1152, in _send_request self.endheaders(body) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1103, in endheaders self._send_output(message_body) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 934, in _send_output self.send(msg) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 877, in send self.connect() File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 849, in connect (self.host,self.port), self.timeout, self.source_address) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 712, in create_connection raise err File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/socket.py", line 703, in create_connection sock.connect(sa) OSError: [Errno 51] Network is unreachable

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "download.py", line 59, in main() File "download.py", line 41, in main categories = list_categories() File "download.py", line 17, in list_categories with urlopen(Request(url)) as response: File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 163, in urlopen return opener.open(url, data, timeout) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 1282, in http_open return self.do_open(http.client.HTTPConnection, req) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/request.py", line 1256, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [Errno 51] Network is unreachable>

    opened by stm233 5
  • Where is the LSUN-room layout dataset?

    Where is the LSUN-room layout dataset?

    I can't find layout dataset in this project. When i export image with data.py. The data only include some .webp image.Where is the layout label? Thank you.

    opened by ChrisHRZ 4
  • Cannot download data

    Cannot download data

    When i run

    python download.py
    

    I get the error

    Traceback (most recent call last):
      File "download.py", line 62, in <module>
        main()
      File "download.py", line 43, in main
        categories = list_categories(args.tag)
      File "download.py", line 20, in list_categories
        return json.loads(f.read())
      File "/Users/sorensonderby/anaconda/lib/python2.7/json/__init__.py", line 338, in loads
        return _default_decoder.decode(s)
      File "/Users/sorensonderby/anaconda/lib/python2.7/json/decoder.py", line 366, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/Users/sorensonderby/anaconda/lib/python2.7/json/decoder.py", line 384, in raw_decode
        raise ValueError("No JSON object could be decoded")
    ValueError: No JSON object could be decoded
    

    How do i solve this?

    opened by skaae 4
  • curl http://dl.yf.io/lsun/scenes/classroom_train_lmdb.zip timed out

    curl http://dl.yf.io/lsun/scenes/classroom_train_lmdb.zip timed out

    Hello,

    in https://github.com/fyu/lsun/blob/7fea09db0dff99b3c58d7e00a884c3e0ffd8018f/download.py#L21, I'm getting a curl timeout issue

      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
      0     0    0     0    0     0      0      0 --:--:--  0:00:12 --:--:--     0
    
    
    
     91 3127M   91 2873M    0     0  2208k      0  0:24:09  0:22:12  0:01:57 2473k
    curl: (18) transfer closed with 266626630 bytes remaining to read
    

    The partially downloaded classroom zip file has caused a subsequent unzip exception. Maybe it can be solved by adding the -C - flag?

    Thank you

    opened by ShiboXing 2
  • Fix export errors caused by lmdb keys being binaries

    Fix export errors caused by lmdb keys being binaries

    Currently in python3, lmdb gives the keys as a binary, not string which causes the export to error with:

    File "data.py", line 47, in export_images
        image_out_path = join(image_out_dir, key + '.webp')
    TypeError: can't concat str to bytes
    

    You can see a Colab without and with the fix here - https://colab.research.google.com/drive/1TUULlY7FkPQXdwN7iFddjAmqwFvaz-h3

    opened by Tenoke 2
  • Some datasets are not accessible for permission denied.

    Some datasets are not accessible for permission denied.

    At website http://complexity.cecs.ucf.edu/lsun/ I notice that there is a car dataset. I followed the guidance on the website. But there is no permission for ordinary prople.

    The dataset can be downloaded either from GitHub https://github.com/fyu/lsun or the categories lists on http://tigress-web.princeton.edu/~fy/lsun/public/release/. More details are available on the dataset page http://www.yf.io/p/lsun.

    opened by ZaneCode6574 2
  • No module named request

    No module named request

    Although requests has been installed via pip, I get the error that no module is named as request.

    $ pip install --user request
    DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
    Collecting request
      Downloading https://files.pythonhosted.org/packages/f1/27/7cbde262d854aedf217061a97020d66a63163c5c04e0ec02ff98c5d8f44e/request-2019.4.13.tar.gz
    Collecting get (from request)
      Downloading https://files.pythonhosted.org/packages/3f/ef/bb46f77f7220ac1b7edba0c76d810c89fddb24ddd8c08f337b9b4a618db7/get-2019.4.13.tar.gz
    Collecting post (from request)
      Downloading https://files.pythonhosted.org/packages/0f/05/bd79da5849ea6a92485ed7029ef97b1b75e55c26bc0ed3a7ec769af666f3/post-2019.4.13.tar.gz
    Requirement already satisfied: setuptools in /home/mahmood/.local/lib/python2.7/site-packages (from request) (41.0.1)
    Collecting query_string (from get->request)
      Downloading https://files.pythonhosted.org/packages/12/3c/412a45daf5bea9b1d06d7de41787ec4168001dfa418db7ec8723356b119f/query-string-2019.4.13.tar.gz
    Collecting public (from query_string->get->request)
      Downloading https://files.pythonhosted.org/packages/54/4d/b40004cc6c07665e48af22cfe1e631f219bf4282e15fa76a5b6364f6885c/public-2019.4.13.tar.gz
    Building wheels for collected packages: request, get, post, query-string, public
      Building wheel for request (setup.py) ... done
      Created wheel for request: filename=request-2019.4.13-cp27-none-any.whl size=1677 sha256=635c9a974941fd509fd7dfe5f47bf12a96489c47c67f748a8672ff858c40ff52
      Stored in directory: /home/mahmood/.cache/pip/wheels/30/84/5f/484cfba678967ef58c16fce6890925d5c7172622f20111fbfd
      Building wheel for get (setup.py) ... done
      Created wheel for get: filename=get-2019.4.13-cp27-none-any.whl size=1691 sha256=6db08500aa1c1620b31332e0ca0c167927bd6dd2b0e4ec25039cfe13fa232e15
      Stored in directory: /home/mahmood/.cache/pip/wheels/c1/e3/c1/d02c8c58538853e4c9b78cadb74f6d5c5c370b48a69a7271aa
      Building wheel for post (setup.py) ... done
      Created wheel for post: filename=post-2019.4.13-cp27-none-any.whl size=1661 sha256=079dee3764bce7c1f6a44eae59940c4a529c0cd3a0ab10342be4697697fb020c
      Stored in directory: /home/mahmood/.cache/pip/wheels/c3/c3/24/b5c132b537ab380c02d69e6bd4dec1f5db56b5fe19030473d7
      Building wheel for query-string (setup.py) ... done
      Created wheel for query-string: filename=query_string-2019.4.13-cp27-none-any.whl size=2048 sha256=d91766531ceb491430edddb1fd8be7629137904c0c3b0814e1a5709618e75c91
      Stored in directory: /home/mahmood/.cache/pip/wheels/d6/a4/78/01b20a9dc224dcc009fab669f7f27b943b8889c5150bd68d8a
      Building wheel for public (setup.py) ... done
      Created wheel for public: filename=public-2019.4.13-cp27-none-any.whl size=2537 sha256=226deaaa53d129077c1778b2902044b4efd00e67fa5d97113338abdd630ba285
      Stored in directory: /home/mahmood/.cache/pip/wheels/23/7c/6e/f5b4e09d6596c8b8802b347e48f149031e2363368048f1347a
    Successfully built request get post query-string public
    Installing collected packages: public, query-string, get, post, request
    Successfully installed get-2019.4.13 post-2019.4.13 public-2019.4.13 query-string-2019.4.13 request-2019.4.13
    
    
    $ pip install --user requests
    DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
    Requirement already satisfied: requests in /home/mahmood/.local/lib/python2.7/site-packages (2.22.0)
    Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/mahmood/.local/lib/python2.7/site-packages (from requests) (1.25.3)
    Requirement already satisfied: certifi>=2017.4.17 in /home/mahmood/.local/lib/python2.7/site-packages (from requests) (2019.6.16)
    Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/mahmood/.local/lib/python2.7/site-packages (from requests) (3.0.4)
    Requirement already satisfied: idna<2.9,>=2.5 in /usr/lib/python2.7/dist-packages (from requests) (2.6)
    WARNING: You are using pip version 19.2.1, however version 19.3 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.
    $ python download.py -c bedroom
    Traceback (most recent call last):
      File "download.py", line 8, in <module>
        from urllib.request import Request, urlopen
    ImportError: No module named request
    

    Any thought?

    opened by mahmoodn 2
  • urlopen error  No route to host

    urlopen error No route to host

    running python3 download.py -c church_outdoor but getting an error

    Traceback (most recent call last):
      File "/usr/lib/python3.8/urllib/request.py", line 1354, in do_open
        h.request(req.get_method(), req.selector, req.data, headers,
      File "/usr/lib/python3.8/http/client.py", line 1256, in request
        self._send_request(method, url, body, headers, encode_chunked)
      File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
        self.endheaders(body, encode_chunked=encode_chunked)
      File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
        self._send_output(message_body, encode_chunked=encode_chunked)
      File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
        self.send(msg)
      File "/usr/lib/python3.8/http/client.py", line 951, in send
        self.connect()
      File "/usr/lib/python3.8/http/client.py", line 922, in connect
        self.sock = self._create_connection(
      File "/usr/lib/python3.8/socket.py", line 808, in create_connection
        raise err
      File "/usr/lib/python3.8/socket.py", line 796, in create_connection
        sock.connect(sa)
    OSError: [Errno 113] No route to host
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "download.py", line 59, in <module>
        main()
      File "download.py", line 41, in main
        categories = list_categories()
      File "download.py", line 17, in list_categories
        with urlopen(Request(url)) as response:
      File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
        return opener.open(url, data, timeout)
      File "/usr/lib/python3.8/urllib/request.py", line 525, in open
        response = self._open(req, data)
      File "/usr/lib/python3.8/urllib/request.py", line 542, in _open
        result = self._call_chain(self.handle_open, protocol, protocol +
      File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
        result = func(*args)
      File "/usr/lib/python3.8/urllib/request.py", line 1383, in http_open
        return self.do_open(http.client.HTTPConnection, req)
      File "/usr/lib/python3.8/urllib/request.py", line 1357, in do_open
        raise URLError(err)
    urllib.error.URLError: <urlopen error [Errno 113] No route to host>
    
    opened by ssusie 1
  • LSUN Objects Dataset

    LSUN Objects Dataset

    Hi @fyu ,

    I think the link for downloading the object categories dl.yf.io/lsun/objects/ is down. I wanted to know if the URL is changed?

    Thank you, Shenoy

    opened by ps48 1
  • Pictures exported not in expected manner

    Pictures exported not in expected manner

    The pictures export from the lmdb seems to have some decoding problem. Does anyone have any idea to solve it? image

    Platform: Window 10, Anaconda 3, python 2.7

    opened by louiszen 1
  • The directory structure is not onsistent with meta list

    The directory structure is not onsistent with meta list

    Whe I export churches_lmdb to churches by cmd:

    python3 data.py export churches_lmdb --out_dir churches

    The directory structure like this: image

    but the meta list is just only show the filename of the images: image I get the filelist from https://ommer-lab.com/files/lsun.zip

    Is there any filelist that include the right subpath of images?

    opened by GxjGit 2
  • curl: (18) transfer closed with (1729112674) bytes remaining to read

    curl: (18) transfer closed with (1729112674) bytes remaining to read

    After multiple attempts to run python3 download.py -c church_outdoor, I keep getting this error after few hours of downloading

    Downloading church_outdoor train set
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
     29 2335M   29  686M    0     0   180k      0  3:41:07  1:04:57  2:36:10  186k
    curl: (18) transfer closed with 1729112674 bytes remaining to read
    Downloading church_outdoor val set
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                     Dload  Upload   Total   Spent    Left  Speed
    100 5817k  100 5817k    0     0   118k      0  0:00:48  0:00:48 --:--:--  173k
    
    opened by ssusie 0
  • Can this dataset be used for commercial purposes ?

    Can this dataset be used for commercial purposes ?

    Hi

    I haven't been able to find any information on the license attached to this dataset. Am I allowed to use this data for my professional work? It would mostly revolve around testing out data augmentation methods and / or directly training a classifier using some of the dataset ?

    Thanks

    opened by yk287 0
  • HTTP 404 Error

    HTTP 404 Error

    I get a 404 error when I tried downloading following the instructions., therefore cannot download datasets. I think the server is down now. Can you please address the problem?

    opened by gwang-kim 0
Owner
Fisher Yu
Fisher Yu
This is the official source code for SLATE. We provide the code for the model, the training code, and a dataset loader for the 3D Shapes dataset. This code is implemented in Pytorch.

SLATE This is the official source code for SLATE. We provide the code for the model, the training code and a dataset loader for the 3D Shapes dataset.

Gautam Singh 66 Dec 26, 2022
This is the dataset and code release of the OpenRooms Dataset.

This is the dataset and code release of the OpenRooms Dataset.

Visual Intelligence Lab of UCSD 95 Jan 8, 2023
Code, Data and Demo for Paper: Controllable Generation from Pre-trained Language Models via Inverse Prompting

InversePrompting Paper: Controllable Generation from Pre-trained Language Models via Inverse Prompting Code: The code is provided in the "chinese_ip"

THUDM 101 Dec 16, 2022
Annotated notes and summaries of the TensorFlow white paper, along with SVG figures and links to documentation

TensorFlow White Paper Notes Features Notes broken down section by section, as well as subsection by subsection Relevant links to documentation, resou

Sam Abrahams 437 Oct 9, 2022
Official Implementation and Dataset of "PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask and Group-Level Consistency", CVPR 2021

Portrait Photo Retouching with PPR10K Paper | Supplementary Material PPR10K: A Large-Scale Portrait Photo Retouching Dataset with Human-Region Mask an

null 184 Dec 11, 2022
Demo code for paper "Learning optical flow from still images", CVPR 2021.

Depthstillation Demo code for "Learning optical flow from still images", CVPR 2021. [Project page] - [Paper] - [Supplementary] This code is provided t

null 130 Dec 25, 2022
Demo code for ICCV 2021 paper "Sensor-Guided Optical Flow"

Sensor-Guided Optical Flow Demo code for "Sensor-Guided Optical Flow", ICCV 2021 This code is provided to replicate results with flow hints obtained f

null 10 Mar 16, 2022
Fast image augmentation library and easy to use wrapper around other libraries. Documentation: https://albumentations.ai/docs/ Paper about library: https://www.mdpi.com/2078-2489/11/2/125

Albumentations Albumentations is a Python library for image augmentation. Image augmentation is used in deep learning and computer vision tasks to inc

null 11.4k Jan 9, 2023
A large dataset of 100k Google Satellite and matching Map images, resembling pix2pix's Google Maps dataset.

Larger Google Sat2Map dataset This dataset extends the aerial ⟷ Maps dataset used in pix2pix (Isola et al., CVPR17). The provide script download_sat2m

null 34 Dec 28, 2022
LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation (NeurIPS2021 Benchmark and Dataset Track)

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation by Junjue Wang, Zhuo Zheng, Ailong Ma, Xiaoyan Lu, and Yanfei Zh

Kingdrone 174 Dec 22, 2022
PyStan, a Python interface to Stan, a platform for statistical modeling. Documentation: https://pystan.readthedocs.io

PyStan NOTE: This documentation describes a BETA release of PyStan 3. PyStan is a Python interface to Stan, a package for Bayesian inference. Stan® is

Stan 229 Dec 29, 2022
Genshin-assets - 👧 Public documentation & static assets for Genshin Impact data.

genshin-assets This repo provides easy access to the Genshin Impact assets, primarily for use on static sites. Sources Genshin Optimizer - An Artifact

Zerite Development 5 Nov 22, 2022
Dataset used in "PlantDoc: A Dataset for Visual Plant Disease Detection" accepted in CODS-COMAD 2020

PlantDoc: A Dataset for Visual Plant Disease Detection This repository contains the Cropped-PlantDoc dataset used for benchmarking classification mode

Pratik Kayal 109 Dec 29, 2022
EMNLP 2021: Single-dataset Experts for Multi-dataset Question-Answering

MADE (Multi-Adapter Dataset Experts) This repository contains the implementation of MADE (Multi-adapter dataset experts), which is described in the pa

Princeton Natural Language Processing 68 Jul 18, 2022
EMNLP 2021: Single-dataset Experts for Multi-dataset Question-Answering

MADE (Multi-Adapter Dataset Experts) This repository contains the implementation of MADE (Multi-adapter dataset experts), which is described in the pa

Princeton Natural Language Processing 39 Oct 5, 2021
The Habitat-Matterport 3D Research Dataset - the largest-ever dataset of 3D indoor spaces.

Habitat-Matterport 3D Dataset (HM3D) The Habitat-Matterport 3D Research Dataset is the largest-ever dataset of 3D indoor spaces. It consists of 1,000

Meta Research 62 Dec 27, 2022
OCTIS: Comparing Topic Models is Simple! A python package to optimize and evaluate topic models (accepted at EACL2021 demo track)

OCTIS : Optimizing and Comparing Topic Models is Simple! OCTIS (Optimizing and Comparing Topic models Is Simple) aims at training, analyzing and compa

MIND 478 Jan 1, 2023