Export your data from Xiami

Related tags

Web Content Extracting xiami_exporter
Overview

Xiami Exporter

导出虾米音乐的个人数据，功能：

  • 导出歌曲为 json
    • 收藏歌曲
    • 收藏专辑
    • 播放列表
  • 导出收藏艺人为 json
  • 导出收藏专辑为 json
  • 导出播放列表为 json (个人和收藏)
  • 将导出的数据整理至 sqlite 数据库
    • 收藏歌曲
    • 收藏艺人
    • 收藏专辑
    • 播放列表
  • 下载已导出歌曲的 MP3 文件
  • 下载已导出专辑的 MP3 文件
  • 下载已导出播放列表的 MP3 文件
  • 根据导出信息为 MP3 添加完整的 ID3 tag
  • 下载已导出歌曲的专辑、艺人的封面图片

Getting Started

  1. Clone 项目，创建 Python 3 虚拟环境，安装所有依赖

    $ python -m venv venv
$ source venv/bin/activate
$ pip install -r requirements.txt

  2. 在 Chrome 中登录虾米，点击 “我的音乐”，从 URL 中获取 user_id，例如 https://www.xiami.com/user/932367, user_id 即为 932367.

  3. 运行 python -m xiami_exporter.cli init，根据提示，输入配置项，其中包括刚刚获取的 user_id.

  4. 回到 Chrome “我的音乐” 页面，右键选择 “审查页面” (Inspect)，点击 “网络” (Network) 并在过滤器中选择 XHR，刷新页面，在最后一条带有 _s 的网络请求上点击右键，选择 “Copy - Copy as Node.js fetch”

  5. 在项目目录下创建 fetch.py 文件，将刚才拷贝的内容粘贴进去并保存 (pbpaste > fetch.py)

  6. 运行 python -m xiami_exporter.cli check, 显示成功表示可以使用导出功能，否则请重试上一步，或联系开发者

  7. 根据想要导出的数据，运行相应指令，如 python -m xiami_exporter.cli export-songs 即导出收藏歌曲为 json

Usage

运行方式为 python -m xiami_exporter.cli COMMAND, 可通过 python -m xiami_exporter.cli --help 查看指令列表。

COMMAND: init

初始化配置文件

COMMAND: check

检查 fetch.py 是否可以通过虾米 API 的验证，成功时输出如下:

test fetch() ok
recal token correct: be5bb12dbb135066f4cb282706019bc8
Success, you can now use the export commands

COMMAND: export

Previously export-songs

导出数据为 json 文件，需要指定子命令 fav_type:

  • SONGS: 收藏歌曲
  • ALBUMS: 收藏专辑
  • ARTISTS: 收藏艺人
  • PLAYLISTS: 收藏歌单
  • MY_PLAYLISTS: 我创建的歌单

json 文件每页一个，文件名为 -.json，存放于 XiamiExports/json/ 目录下。

对于专辑和歌单 (ALBUMS, PLAYLISTS, MY_PLAYLISTS)，在完成首次导出后，需要额外使用 -c, --complete-songs 参数运行， 来获取包含歌曲的详细信息，每个专辑/歌单的详细信息 json 文件会以 .json 为名存放于 XiamiExports/json//details 目录下。

此指令是后续创建数据库、下载音乐的基础。

COMMAND: create-songs-db

将收藏歌曲导入数据库中记录，此指令是 download-music 的基础。

COMMAND: download-music

下载所有收藏歌曲，下载状态 (download_status) 会同步保存到数据库中。

中断后可重新运行，只会从数据库中筛选 download_status = NOT_SET 的歌曲进行下载。

注: 此指令后续会支持下载 album 和 playlist

COMMAND: download-covers

下载所有歌曲的专辑封面 (album cover)，支持如下选项：

  • -f, --force: 强制重新下载即使文件存在
  • -l, --artist-logos: 下载艺人图片而非专辑封面

COMMAND: tag-music

为所有已下载的歌曲添加 ID3 tags。

若专辑封面文件存在，则会将其添加到 tags 中，因此建议先运行 download-covers。

COMMAND: trim-json

对已导出的 json 文件进行修剪，去掉不必要的数据。

此命令用于维护已导出的 json 数据，若使用最新版重新导出， 则导出时已对各类数据进行自动修剪，无需之后运行此命令。

Hierarchy

Xiami Exporter 保存的数据有如下几类：

  • json 数据文件: 歌曲、专辑、艺人、播放列表
  • MP3 音频文件: 歌曲、专辑、播放列表
  • jpg/png 图片文件: 专辑封面
  • sqlite3 数据库文件: 歌曲、专辑、艺人、播放列表

默认在项目路径下创建 XiamiExports 目录，其文件系统结构大体如下:

XiamiExports/
  db.sqlite3
  json/
    songs/
      song-1.json
      song-2.json
      ...
    albums/
      details/
        ALBUM_ID.json
      albums-1.json
      ...
    playlists/
      details/
        PLAYLIST_ID.json
      playlists-1.json
      ...
    my_playlists/
      | same as `playlists/`
    artists/
  music/
    NUM-SONG_ID.mp3
  covers/
    ALBUM_ID.jpg
  artist_logos/
    ARTIST_ID.jpg

Development

Development notes, check if you have interests.

TODOs

  • download_covers: handle songs from albums/playlists details

tag problems

  • arrangement -> TIPL, tried to save but cannot be displayed

  • [solved] comment -> COMM, easyid3 writes as:

    COMM==XXX=artist_alias: あーりーれい

    which is not recognized by Meta.app.

    Meta.app writes as:

    COMM==ENG=first line
second line
COMM=ID3v1 Comment=eng=first line
second line

    solution:

    follows how itunes mp3 writes comment:

    COMM==eng=artist_alias: あーりーれい

  • performers should be TMCL, but is written as TXXX:

    TXXX=PERFORMER=陽花
You might also like...
Poetry plugin to export the dependencies to various formats

Poetry export plugin This package is a plugin that allows the export of locked packages to various formats. Note: For now, only the requirements.txt f

Poetry 90 Jan 5, 2023
Just some scripts to export vector tiles to geojson.

Vector tiles to GeoJSON Nowadays modern web maps are usually based on vector tiles. The great thing about vector tiles is, that they are not just imag

Lilith Wittmann 77 Jul 26, 2022
Export CenterPoint PonintPillars ONNX Model For TensorRT
Export CenterPoint PonintPillars ONNX Model For TensorRT

CenterPoint-PonintPillars Pytroch model convert to ONNX and TensorRT Welcome to CenterPoint! This project is fork from tianweiy/CenterPoint. I impleme

CarkusL 149 Dec 13, 2022
Export solved codewars kata challenges to a text file.

Codewars Kata Exporter Note:this is not totally my work.i've edited the project to make more easier and faster for me.you can find the original work h

Oussama Ben Sassi 4 Aug 13, 2021
A simple desktop application to scan and export Genshin Impact Artifacts.
A simple desktop application to scan and export Genshin Impact Artifacts.

「天目」 -- Amenoma 简体中文 | English 「天目流的诀窍就是滴水穿石的耐心和全力以赴的意志」 扫描背包中的圣遗物，并导出至 json 格式。之后可导入圣遗物分析工具( 莫娜占卜铺 、 MingyuLab 、 Genshin Optimizer 进行计算与规划等。 已支持 原神2.

夏至 475 Dec 30, 2022
Simple script to export contacts from telegram into vCard file

Telegram Contacts Exporter Simple script to export contacts from telegram into vCard file Getting Started Prerequisites You must to put your Telegram

Pere Antoni 1 Oct 17, 2021
Program to export all new icons from the latest Fortnite patch

Assets Exporter This program allows you to generate all new icons of a patch in png! Requierements Python =3.8 (installed on your computer) If you wa

ᴅᴊʟᴏʀ3xᴢᴏ 6 Jun 24, 2022
DICexport is a GUI (PyQt5) to export digital image correlation videos

DIC Video Exporter DICexport is a GUI (PyQt5) to export digital image correlation videos. It offers the flexibility to choose a selected range of a vi

Chaoyi Zhu 0 Jun 23, 2022
PowerShell module to import/export Excel spreadsheets, without Excel
PowerShell module to import/export Excel spreadsheets, without Excel

PowerShell + Excel = Better Together Automate Excel via PowerShell without having Excel installed. Runs on Windows, Linux and MAC. Creating Tables, Pi

Doug Finke 2k Dec 30, 2022
New program to export a Blender model to the LBA2 model format.
New program to export a Blender model to the LBA2 model format.

LBA2 Blender to Model 2 This is a new program to export a Blender model to the LBA2 model format. This is also the first publicly released version of

null 2 Nov 30, 2022
Export Statistics for a Telegram Group Chat

Telegram Statistics Export Statistics for a Telegram Group Chat How to Run First, in main repo directory, run the following code to add src to your PY

Ali Hejazizo 22 Dec 5, 2022
Addon for Blender 2.8+ that automatically creates NLA tracks for all animations. Useful for GLTF export.
Addon for Blender 2.8+ that automatically creates NLA tracks for all animations. Useful for GLTF export.

PushDownAll An addon for Blender 2.8+ that runs Push Down on all animations, creating NLA tracks for each. This is useful if you have an object with m

Cory Petkovsek 16 Oct 6, 2022
GIMP script to export bitmap as GRAPHICS 4 file (aka SCREEN 5)
GIMP script to export bitmap as GRAPHICS 4 file (aka SCREEN 5)

gimpfu-msx-gr4.py GIMP script to export bitmap as GRAPHICS 4 file (aka SCREEN 5). GRAPHICS 4 specs are: 256x212 (or 256x192); 16 color palette (from 5

Pedro de Medeiros 4 Oct 17, 2022
🐝 ℹ️ Honeybee extension for export to IES-VE gem file format
🐝 ℹ️ Honeybee extension for export to IES-VE gem file format

honeybee-ies Honeybee extension for export a HBJSON file to IES-VE GEM file format Installation pip install honeybee-ies QuickStart import pathlib fro

Ladybug Tools 4 Jul 12, 2022
This is a script to export logs from AWS CloudTrail to a local file.

cloudtrail-export-logs This is a script to export logs from AWS CloudTrail to a local file. Getting Started Prerequisites python 3 boto3 pip Installin

Claick Assunção de Oliveira 2 Jan 2, 2022
ViberExport - Export messages from Viber messenger using viber.db file

📲 ViberExport Export messages from Viber messenger using viber.db file ⚡ Usage:

null 7 Nov 23, 2022
Skype export archive to text converter for python

Skype export archive to text converter This software utility extracts chat logs

Roland Pihlakas open source projects 2 Jun 30, 2022
Sheet2export - FreeCAD macro to export spreadsheet
Sheet2export - FreeCAD macro to export spreadsheet

Description This is FreeCAD macro to export spreadsheet to file.

Darek L 3 Jul 9, 2022
Export watched content from Tautulli to the Letterboxd CSV Import Format

Export watched content from Tautulli to the Letterboxd CSV Import Format

Evan J 5 Aug 31, 2022
Comments
  • 收藏的歌曲只能获取100页

    收藏的歌曲只能获取100页

    首先感谢作者在百忙之中写出此救急工具，但是使用新版虾米接口的话收藏的歌曲只能获取到100页，如果收藏歌曲超过3000曲的话将会获取不到。

    可以使用旧版收藏的歌曲页面进行获取，https://emumo.xiami.com/space/lib-song/u/{id}/page/1 ，不知道您是否有时间更新或使用其他方法解决此问题。

    opened by GhostStar 4
  • run with error

    run with error

    after running 'python -m xiami_exporter.cli create-songs-db'

    Traceback (most recent call last):
  File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/opt/xiami_exporter/xiami_exporter/cli.py", line 258, in <module>
    cli()
  File "/opt/xiami_exporter/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/xiami_exporter/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/xiami_exporter/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/xiami_exporter/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/xiami_exporter/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/xiami_exporter/xiami_exporter/cli.py", line 127, in create_songs_db
    prepare_db()
  File "/opt/xiami_exporter/xiami_exporter/cli.py", line 50, in prepare_db
    db.create_tables(all_models)
  File "/opt/xiami_exporter/lib/python3.6/site-packages/peewee.py", line 3317, in create_tables
    model.create_table(**options)
  File "/opt/xiami_exporter/lib/python3.6/site-packages/peewee.py", line 6648, in create_table
    cls._schema.create_all(safe, **options)
  File "/opt/xiami_exporter/lib/python3.6/site-packages/peewee.py", line 5775, in create_all
    self.create_table(safe, **table_options)
  File "/opt/xiami_exporter/lib/python3.6/site-packages/peewee.py", line 5630, in create_table
    self.database.execute(self._create_table(safe=safe, **options))
  File "/opt/xiami_exporter/lib/python3.6/site-packages/peewee.py", line 3142, in execute
    return self.execute_sql(sql, params, commit=commit)
  File "/opt/xiami_exporter/lib/python3.6/site-packages/peewee.py", line 3127, in execute_sql
    cursor = self.cursor(commit)
  File "/opt/xiami_exporter/lib/python3.6/site-packages/peewee.py", line 3111, in cursor
    self.connect()
  File "/opt/xiami_exporter/lib/python3.6/site-packages/peewee.py", line 3065, in connect
    self._state.set_connection(self._connect())
  File "/opt/xiami_exporter/lib/python3.6/site-packages/peewee.py", line 3402, in _connect
    isolation_level=None, **self.connect_params)
TypeError: argument 1 must be str, not PosixPath
    opened by mangopaq 2
Owner
Xiao Meng
Craft and smelt.
Xiao Meng
GitHub
CKAN is an open-source DMS (data management system) for powering data hubs and data portals. CKAN makes it easy to publish, share and use data. It powers catalog.data.gov, open.canada.ca/data, data.humdata.org among many other sites.

CKAN: The Open Source Data Portal Software CKAN is the world’s leading open-source data portal platform. CKAN makes it easy to publish, share and work

ckan 3.6k Dec 27, 2022
A way to export your saved reddit posts to a Notion table.

reddit-saved-to-notion A way to export your saved reddit posts and comments to a Notion table.Uses notion-sdk-py and praw for interacting with Notion

null 19 Sep 12, 2022
Notion4ever - Python tool for export all your content of Notion page using official Notion API

NOTION4EVER Notion4ever is a small python tool that allows you to free your cont

null 50 Dec 30, 2022
This is django-import-export module that exports data into many formats

django-import-export This is django-import-export module which exports data into many formats, you can implement this in your admin panel. -> Dehydrat

Shivam Rohilla 3 Jun 3, 2021
CLI utility for updating the EVE Online static data export in a postgres database

EVE SDE Postgres updater CLI utility for updating the EVE Online static data export postgres database. This has been tested with the Fuzzwork postgres

Markus Juopperi 1 Oct 29, 2021
👻🟡 Download all Snapchat video & photo memories from a data export.

Snapchat "Memories" Fetcher In compliance with the California Consumer Privacy Act of 2018 (“CCPA”), businesses which collect and store user data must

Todd Birchard 18 Dec 26, 2022
A command line utility to export Google Keep notes to markdown.

Keep-Exporter A command line utility to export Google Keep notes to markdown files with metadata stored as a frontmatter header. Supports exporting: S

Nathan Beals 85 Dec 17, 2022
Django Smuggler is a pluggable application for Django Web Framework that helps you to import/export fixtures via the automatically-generated administration interface.

Django Smuggler Django Smuggler is a pluggable application for Django Web Framework to easily dump/load fixtures via the automatically-generated admin

semente 373 Dec 26, 2022
An MkDocs plugin to export content pages as PDF files

MkDocs PDF Export Plugin An MkDocs plugin to export content pages as PDF files The pdf-export plugin will export all markdown pages in your MkDocs rep

Terry Zhao 266 Dec 13, 2022
Deploy tensorflow graphs for fast evaluation and export to tensorflow-less environments running numpy.

Deploy tensorflow graphs for fast evaluation and export to tensorflow-less environments running numpy. Now with tensorflow 1.0 support. Evaluation usa

Marcel R. 349 Aug 6, 2022