In the unsplash_photos.photo_location_country and unsplash_photos.photo_location_city the values appear to be freeform text that was probably direct user input, with effectively duplicate entries for example,

unsplash_lite=# select '>' || photo_location_city || '<' as city, '>' || photo_location_country || '<' as country,  count(*) from unsplash_photos where lower(photo_location_city) like '%london%' group by 1,2;
 ?column?  |       ?column?       | count
-----------+----------------------+-------
 >LONDON < | >United Kingdom <    |     1
 >London<  | >Canada<             |     7
 >London<  | >Egyesült Királyság< |     1
 >London<  | >England<            |     1
 >London<  | >U.K.<               |     2
 >London<  | >U.K<                |     1
 >London<  | >United Kingdom <    |     1
 >London<  | >United Kingdom<     |    73
 >London<  |                      |     3
(9 rows)```

It looks like there needs to be some data cleaning on these fields, definitely some stripping white space and such. Is it assumed that we should do our own location normalization on this and possibly add in a normalized_photo_location_country and normalized_photo_location_city ?

Also - over in unsplash_conversion.conversion_country this appears to be ISO 2 letter country codes. Is this guaranteed to be a valid ISO 2 letter country code? And was this data created based upon a maxmind geoip lookup or something similar?

Thanks so much for this dataset, I think it is going to be quite useful for demonstrational purposes. I hope these questions help increase the quality of what is a already great dataset.

I love your API and would like to integrate your commercial images into our product but through your API, do you consider creating an API Endpoint?

It could work the same way as your existing API, just for the usage of those datasets.

Creative Commons Images from Flickr is the same, but I like your API more. 👍

Overview

• Closes #10
• Closes #13
• Closes #21
• Closes #22
• Closes #29

A 1.1.0 release tag will be created with more details about what's new in this version:

New:

• #10: Included user descriptions of the photos
• #21: Included width, height and aspect ratio of photos
• #22: Included colors data from photos, coming from a 3rd party AI

Fix:

• #13: Trimmed some fields.
• #29: Replaced newlines in keywords file by spaces to avoid CSV importation issues

Ticket #21 mentions additional image metadata that should be in the dataset. Are there other image analysis things that unsplash calculates that could be added? I'm in particular thinking of color value statistics, mean/median pixel value, min/max pixel values etc.

Overview

• Closes #12
• Fix: All landmark columns are null
• Fix: Some landmark names showing as empty strings instead of null
• Fix: Duplicate entry for one photo in the Full Dataset

New release

• Upon merging, create a 1.0.1 tag and release for the dataset.
• /lite/latest API link updated to the 1.0.1 dataset.

Describe the bug Hello ,

I was looking on the data from the lite dataset this morning and I noticed something weird in the column 'ai_service_2_confidence' from the keywords.tsv000 file.

when I applied some stats on the columns about ai_service the column 'ai_service_2_confidence' seems to have extreme value that are exceeding 100 that is for me the expected max (if I take the ai_service_1_confidence as reference for exemple)

To Reproduce

There is the code to build the stats

import pandas as pd
dfp_keywords_raw = pd.read_csv('keywords.tsv000', sep='\t', header=0)
dfp_keywords_raw[['ai_service_1_confidence', 'ai_service_2_confidence']].describe()

Steps to reproduce the behavior: Having a python environment (3.6.13) with pandas 1.1.5 installed

Expected behavior I am expecting to have a value in the column 'ai_service_2_confidence' in keywords.tsv000 file between 0 and 100 or if it's not the case having a more precise description of the value for the 'ai_service_2_confidence' in the description (like the range)

Additional context I have a list of the keywords that seems to be impacted by these extreme values unsplash_extreme_value.zip

Hope that it will help on your investigation 🕵️‍♀️ (and I hope that is not just me that is missing something)

PS: your dataset is great by the way (really hope to have access to the full version soon)👍

I am trying to download unsplash dataset lite version and the link for the lite version download doesn't give me images. Is image in the download link or do I need to download it by using API?

Hey, I'm trying to calculate the average amount of images a user downloads.

As I know from my own photo stats, a lot of downloads are generated via API requests from external applications. You state in your API doc that external applications don't need to authenticate on a user level.

My question: Is for an external application like Trello one anonymous user id generated or do you guys have a better approach to distinguish users "behind" the external application?

Example from the test dataset Could the user from the first row (942 downloads) really be one person or also a whole logical entity like Trello?

anonymous_user_id | downloads ---- | --- 5a055748-57d2-45c1-a882-5b9bb9313509 | 942 beb0923e-c17d-4a90-a8db-47b0f45fb0fc |897 85e5db9c-07c7-49bf-9e08-5cbd1603dd74 | 546 ... | ...

When reviewing the data in the lite dataset, all of the following fields are null in all records.

• unsplash_photos.ai_primary_landmark_name
• unsplash_photos.ai_primary_landmark_latitude
• unsplash_photos.ai_primary_landmark_longitude
• unsplash_photos.ai_primary_landmark_confidence

If all of these are supposed to be null all the time - It may be useful to drop those columns from the dataset completely.

Although if these columns do have data in the full dataset it makes sense to have them exist. If this is the case, it may be useful to update the documentation to note that these fields are null in the lite dataset and have values in the full dataset.

In any case, just checking to make sure that this is the expected behavior.

Hello,

I'm an English Learning Educator and I have a registered project set up already.

I would like to get random images, but ONLY the ones that have a clear or detailed description field, not null or empty values.

Is there a way to do this with your current api or can it be implemented by your team as a feature request?

something like this:

https://api.unsplash.com/photos/random?description=true&description_min_chars=10&client_id=XXXXXX

params

description = true (required)

description_min_chars = int (required) minimum description characters length or char count

Regards

Hugo Barbosa

I've applied the full dataset and it was approved. Now I get the 47 GB dataset. Will I be banned if I download the pictures? I have a 1Gb/s server and I'd download the pictures in 'photos.tsv000', the url begins in 'images.unsplash.com', I wonder if I could get banned when I download them too quickly? (For a 1Gb server of mine, about 5-6 pictures per second.)

Describe the bug URL is bad

To Reproduce Try downloading it

Expected behavior Good URL

Additional context Add any other context about the problem here.

[1] photo_featured does this include the featured topic categories at the top?
[1.1] Could this field include which topics it is featured in, but also if the photo was submitted and rejected from a topic? [1.2] Is there historic data for which images were not included as 'searchable' before the search system was replaced for everything being searchable?

[2] suggested_by_user the description mentions 'a user (human)'. At some point (maybe?) unsplash was adding tags or keywords to its approved/moderated photos, does (could) this distinguish who added it (uploader or staff)?

[4] keyword Is this referring to the search terms used to find the photo? [4.1] assuming it is the searching keywords, can we add a field for the position it was displayed on the website (i.e. if it was the first row first column, or it was an image that was 30 photos down that a searcher scrolled down to find and pick)

Thanks for releasing the dataset, its a great contribution to the research community!

Describe the bug Photo with id sEDzxW4NhL4 has errors. While it can be accessed via photo_url https://unsplash.com/photos/sEDzxW4NhL4 it cannot get accessed in photo_img_url https://images.unsplash.com/photo-1586019496196-bdbea65add07

To Reproduce https://images.unsplash.com/photo-1586019496196-bdbea65add07

Expected behavior

Additional context

Is your feature request related to a problem? Please describe. Since the Unsplash Slack server as the only connection point of the community isn't as populated as it could be, I think there could be another way of bringing the people of Unsplash together.

Describe the solution you'd like With a dataset of all Unsplash contributors it would be possible to create a map, giving them a chance to find other motivated photographers nearby. The dataset should contain the name, the location, the number of photos, the URL and maybe the linked website.

Describe alternatives you've considered Before, there have been local Slack channels on the server to connect with other people from the same country, but afaik they have been shutdown.

Additional context Nothing to add

Hello,

Thanks for sharing / publishing this dataset. Are there any plans for mirroring this dataset on Kaggle? If not, can I publish the Lite Dataset as a public dataset on Kaggle linking it back to Unsplash as the source and using the same licensing terms as here?

It would be great to make this data available on Kaggle where a lot of ML research and models can be built.