👻🟡 Download all Snapchat video & photo memories from a data export.

Todd Birchard

Last update: Dec 26, 2022

Related tags

Downloader python automation snapchat asynchronous concurrency aiohttp asyncio snapchat-stories aiofiles snapchat-video

Overview

Snapchat "Memories" Fetcher

In compliance with the California Consumer Privacy Act of 2018 (“CCPA”), businesses which collect and store user data must allow customers the ability to request data the company has collected about the requesting user:

"... you may ask businesses to disclose what personal information they have about you and what they do with that information, to delete your personal information and not to sell your personal information. You also have the right to be notified, before or at the point businesses collect your personal information, of the types of personal information they are collecting and what they may do with that information."

Many businesses (such as Snapchat) begrudgingly comply by "technically" allowing users to do this, but in such as way that is impossible for a human to parse. While Snapchat technically allows users to request photos and videos Snapchat has stored (dubbed "Memories"), they do so in a way which is unusable by human beings. Sparing the technical details, exporting your account's memories is hidden behind a convoluted process of clicking individual URLs, which serve as proxies to reveal actual URLs containing your data.

The workflow Snapchat is intending to implement is unreasonable. To claim a single media file which belongs to you, users are expected to individually click URLs one-by-one. In reality, these URLs simply link to other URls, which will eventually fetch a single media file of the user. For context, by account has over 3500 media files.

Snapchat's practice of intentionally complicating this process is of questionable legality, and surely an act of defiance. This script automates the process of max-exporting media from Snapchat data exports to resolve this issue in the meantime while the questionable legality of Snapchat's practices are persued.

Getting Started

Requesting your data

User data can be requested from Snapchat here.
Under Manage My Account, select My Data.
Your data export will be emailed to you as a .zip file containing the following:

/mydata_1618592678039
├── index.html
├── /html
│   └── *.html
└── /json
    └── *.json

You can explore your exported data via the index.html page:

Running this script

Clone this repo (git clone https://github.com/toddbirchard/snapchat-memory-fetcher.git).
Drop the contents of your export to the /export directory of this repo.
Initialize project with requirements via make install.
Run script via make run.

Result

The first time this script is run, it will export decoded URLs extracted from your data into JSON files as /urls/photos.json and /urls/videos.json. Urls only need to be decoded once and can be reused for future fetching.
The script will then save all photos and videos associated with your account to /downloads/photos and /downloads/videos, respectively.

This script utilizes Python concurrency in both decoding URLS and fetching media, and therefore should run quickly. Note your hard drive space available as you may be downloading thousands of uncompressed videos.

Comments

Update dependency flake8 to v4
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | flake8 (changelog) | ^3.9.0 -> ^4.0.0 | | | | |

Release Notes

pycqa/flake8

v4.0.1

Compare Source

v4.0.0

Compare Source

Configuration

📅 Schedule: At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box.

This PR has been generated by WhiteSource Renovate. View repository job log here.
opened by renovate[bot] 0
Update dependency aiofiles to v0.7.0
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | aiofiles | ^0.6.0 -> ^0.7.0 | | | | | | aiofiles | ==0.6.0 -> ==0.7.0 | | | | |

Configuration

📅 Schedule: At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻️ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about these updates again.

[x] If you want to rebase/retry this PR, check this box.

This PR has been generated by WhiteSource Renovate. View repository job log here.
opened by renovate[bot] 0
[ImgBot] Optimize images

Beep boop. Your images are optimized!

Your image file size has been reduced by 40% 🎉

Details

| File | Before | After | Percent reduction | |:--|:--|:--|:--| | /.github/[email protected] | 252.13kb | 151.38kb | 39.96% |

Black Lives Matter | 💰 donate | 🎓 learn | ✍🏾 sign

📝 docs | :octocat: repo | 🙋🏾 issues | 🏅 swag | 🏪 marketplace

opened by imgbot[bot] 0
Configure Renovate
Welcome to Renovate! This is an onboarding PR to help you understand and configure settings before regular Pull Requests begin.

:vertical_traffic_light: To activate Renovate, merge this Pull Request. To disable Renovate, simply close this Pull Request unmerged.

Detected Package Files

requirements.txt (pip_requirements)

pyproject.toml (poetry)

Configuration Summary

Based on the default config's presets, Renovate will:

Start dependency updates only once this onboarding PR is merged

Separate major versions of dependencies into individual branches/PRs

Do not separate patch and minor upgrades into separate PRs for the same dependency

Upgrade to unstable versions only if the existing version is unstable

Raise PRs immediately (after branch is created)

If semantic commits detected, use semantic commit type fix for dependencies and chore for all others

Keep existing branches updated even when not scheduled

Disable automerging feature - wait for humans to merge all PRs

Ignore node_modules, bower_components, vendor and various test/tests directories

Autodetect whether to pin dependencies or maintain ranges

Rate limit PR creation to a maximum of two per hour

Limit to maximum 20 open PRs at any time

Group known monorepo packages together

Use curated list of recommended non-monorepo package groupings

Ignore spring cloud 1.x releases

Ignore http4s digest-based 1.x milestones

:abcd: Would you like to change the way Renovate is upgrading your dependencies? Simply edit the renovate.json in this branch with your custom config and the list of Pull Requests in the "What to Expect" section below will be updated the next time Renovate runs.

What to Expect

It looks like your repository dependencies are already up-to-date and no Pull Requests will be necessary right away.

:question: Got questions? Check out Renovate's Docs, particularly the Getting Started section. If you need any further assistance then you can also request help here.

This PR has been generated by WhiteSource Renovate. View repository job log here.
opened by renovate[bot] 0
Dependency Dashboard
This issue provides visibility into Renovate updates and their statuses. Learn more

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

[ ] Update dependency aiofiles to v0.7.0

[ ] Update dependency flake8 to v4

[ ] Check this box to trigger a request for Renovate to run again on this repository
opened by renovate[bot] 0

make install error in Ubuntu

Please help! What is the problem?

e@ubuntu:~/Downloads/snapchat-memory-fetcher-master$ sudo make install
[sudo] password for e: 
make clean
make[1]: Entering directory '/home/e/Downloads/snapchat-memory-fetcher-master'
find . -name '*.pyc' -delete
find . -name '__pycache__' -delete
find . -name 'poetry.lock' -delete
find . -name '*.log' -delete
find . -wholename 'logs/*.json' -delete
find . -wholename '.pytest_cache' -delete
find . -wholename '**/.pytest_cache' -delete
find . -wholename '*/logs/*.json' -delete
find . -wholename '.webassets-cache/*' -delete
find . -wholename '.urls/*json' -delete
find . -wholename '**/logs' -delete
make[1]: Leaving directory '/home/e/Downloads/snapchat-memory-fetcher-master'
python3 -m venv /home/e/Downloads/snapchat-memory-fetcher-master/.venv
. /home/e/Downloads/snapchat-memory-fetcher-master/.venv/bin/activate
/home/e/Downloads/snapchat-memory-fetcher-master/.venv/bin/python3 -m pip install --upgrade pip setuptools wheel
Requirement already satisfied: pip in ./.venv/lib/python3.8/site-packages (21.3)
Requirement already satisfied: setuptools in ./.venv/lib/python3.8/site-packages (58.2.0)
Requirement already satisfied: wheel in ./.venv/lib/python3.8/site-packages (0.37.0)
make requirements
make[1]: Entering directory '/home/e/Downloads/snapchat-memory-fetcher-master'
Ignoring colorama: markers 'python_version >= "3.5" and python_full_version < "3.0.0" and sys_platform == "win32" or sys_platform == "win32" and python_version >= "3.5" and python_full_version >= "3.5.0"' don't match your environment Ignoring win32-setctime: markers 'sys_platform == "win32" and python_version >= "3.5"' don't match your environment Requirement already satisfied: aiofiles==0.6.0 in ./.venv/lib/python3.8/site-packages (from -r requirements.txt (line 1)) (0.6.0) Requirement already satisfied: aiohttp==3.7.4.post0 in ./.venv/lib/python3.8/site-packages (from -r requirements.txt (line 2)) (3.7.4.post0) Requirement already satisfied: async-timeout==3.0.1 in ./.venv/lib/python3.8/site-packages (from -r requirements.txt (line 3)) (3.0.1) Requirement already satisfied: asyncio==3.4.3 in ./.venv/lib/python3.8/site-packages (from -r requirements.txt (line 4)) (3.4.3) Requirement already satisfied: attrs==21.2.0 in ./.venv/lib/python3.8/site-packages (from -r requirements.txt (line 5)) (21.2.0) Requirement already satisfied: chardet==4.0.0 in ./.venv/lib/python3.8/site-packages (from -r requirements.txt (line 6)) (4.0.0) Requirement already satisfied: idna==3.1 in ./.venv/lib/python3.8/site-packages (from -r requirements.txt (line 8)) (3.1) Requirement already satisfied: loguru==0.5.3 in ./.venv/lib/python3.8/site-packages (from -r requirements.txt (line 9)) (0.5.3) Requirement already satisfied: multidict==5.1.0 in ./.venv/lib/python3.8/site-packages (from -r requirements.txt (line 10)) (5.1.0) Requirement already satisfied: simplejson==3.17.2 in ./.venv/lib/python3.8/site-packages (from -r requirements.txt (line 11)) (3.17.2) Requirement already satisfied: typing-extensions==3.10.0.0 in ./.venv/lib/python3.8/site-packages (from -r requirements.txt (line 12)) (3.10.0.0) Requirement already satisfied: yarl==1.6.3 in ./.venv/lib/python3.8/site-packages (from -r requirements.txt (line 14)) (1.6.3)
/bin/sh: 1: Syntax error: "(" unexpected
make[1]: *** [Makefile:24: .requirements.txt] Error 2
make[1]: Leaving directory '/home/e/Downloads/snapchat-memory-fetcher-master'
make: *** [Makefile:42: install] Error 2
e@ubuntu:~/Downloads/snapchat-memory-fetcher-master$

opened by Keki1337 0