Music source separation is a task to separate audio recordings into individual sources

Bytedance Inc.

Last update: Jan 3, 2023

Related tags

Deep Learning music_source_separation

Overview

Music Source Separation

Music source separation is a task to separate audio recordings into individual sources. This repository is an PyTorch implmementation of music source separation. Users can separate their favorite songs into different sources by installing this repository. In addition, users can train their own music source separation systems using this repository. This repository also includes speech enhancement, instruments separation, etc.

Demos

Vocals and accompaniment separation: https://www.youtube.com/watch?v=WH4m5HYzHsg

Separation

Users can easily separate their favorite audio recordings into vocals and accompaniment using the pretrained checkpoints.

Method 1. Separate by installing the package

python3 setup.py install

python3 separate_scripts/separate.py 
    --audio_path="./resources/vocals_accompaniment_10s.mp3" 
    --source_type="accompaniment"

Method 2. Separate by using the source code

1. Install dependencies

pip install -r requirements.txt

2. Download checkpoints

./separate_scripts/download_checkpoints.sh

3. Separate vocals and accompaniment

./separate_scripts/separate_vocals.sh "resources/vocals_accompaniment_10s.mp3" "sep_vocals.mp3"
./separate_scripts/separate_accompaniment.sh "resources/vocals_accompaniment_10s.mp3" "sep_accompaniment.mp3"

Train a music source separation system from scratch

1. Download dataset

We use the MUSDB18 dataset to train music source separation systems. The trained system can be used to separate vocals, accompaniments, bass, and other sources. Execute the following script to download and decompress the MUSDB18 dataset:

./scripts/0_download_datasets/musdb18.sh

The dataset looks like:

./datasets/musdb18
├── train (100 files)
│   ├── 'A Classic Education - NightOwl.stem.mp4'
│   └── ...
├── test (50 files)
│   ├── 'Al James - Schoolboy Facination.stem.mp4'
│   └── ...
└── README.md

2. Pack audio files into hdf5 files

We pack audio waveforms into hdf5 files to speed up training.

."/scripts/1_pack_audios_to_hdf5s/musdb18/sr=44100,chn=2.sh"

3. Create indexes for training

./scripts/2_create_indexes/musdb18/create_indexes.sh

3. Create evaluation audios

./scripts/3_create_evaluation_audios/musdb18/create_evaluation_audios.sh

4. Train & evaluate & save checkpoints

./scripts/4_train/musdb18/train.sh

5. Inference

./scripts/5_inference/musdb18/inference.sh

##

Results

Model	Size (MB)	SDR (dB)	process 1s time (GPU Tesla V100)	process 1s time (CPU Core i7)
ResUNet143 vocals	461	8.9	0.036	2.513
ResUNet143 acc.	461	16.8	0.036	2.513
ResUNet143 Subband vocals	414	8.8	0.012	0.614
ResUNet143 Subband acc.	414	16.4	0.012	0.614

Reference

[1] Qiuqiang Kong, Yin Cao, Haohe Liu, Keunwoo Choi, Yuxuan Wang, Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separation, International Society for Music Information Retrieval (ISMIR), 2021.

@inproceedings{kong2021decoupling,
  title={Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separation.},
  author={Kong, Qiuqiang and Cao, Yin and Liu, Haohe and Choi, Keunwoo and Wang, Yuxuan },
  booktitle={ISMIR},
  year={2021},
  organization={Citeseer}
}

FAQ

On Mac OSX, if users met "ModuleNotFoundError: No module named ..." error, then execute the following commands:

PYTHONPATH="./"
export PYTHONPATH

Comments

WE ARE NOT ABLE TO (DOWNLOAD_CHECKPOINTS)
Hi, I'm trying to use bytedance;

But I'm facing a problem when executing this command:

python -m bytesep download_checkpoints

But I'm getting the following error:

C:\Users\lucas\MSS>python -m bytesep download_checkpoints usage: __main__.py [-h] {download-checkpoints,separate} ... __main__.py: error: argument mode: invalid choice: 'download_checkpoints' (choose from 'download-checkpoints', 'separate')

I would like to know how I can solve this problem and can use bytedance in my songs,

Note, I was able to use it normally until October 2021, but now I'm not doing it anymore;

My thanks in advance, regards, Lucas.
opened by lucasbr15 6
ModuleNotFoundError
Hello, I have done all the installations and downloaded the templates, but when using the command

"python3 separate_scripts/separate.py" or "./separate_scripts/separate_accompaniment.sh "resources/vocals_accompaniment_10s.mp3" "sep_accompaniment.mp3"

I get the following error:

Traceback (most recent call last): File "bytesep/inference.py", line 13, in <module> from bytesep.models.lightning_modules import get_model_class ModuleNotFoundError: No module named 'bytesep

How can I fix this? Thanks in advance.
opened by lucasdobr15 6
CUDA out of memory

Hi! First, thanks for your contribution to music source separation! Since I'd like to separate vocals & accompaniment using only 1 model, and the pre-trained model you provided has only one target, I decided to train it from scratch. I also modify the code so that a validation set is added.

However, I cannot train it on 3 GTX 1080 with small batch size (<12), and I thought that the ResUNet_ismir doesn't require that much memory? Or does it really need a lot of GPU memory? Could you provide some insight, like how long it takes for you to train a model and how many GPU memory it requires?

Thank you a lot!

opened by sophia1488 4
Readlink & Optimizer_Type errors when preparing the audio and running training script

Thanks for sharing your great work!

Following the steps in the ReadMe, I have these error messages when preparing the audio and running the training script:

How do I solve the issues?

I am running the code on a machine with macOS Big Sur.

Also do you have ONNX version of your models to share? The main reason I was running the training code is trying to convert your PyTorch models to ONNX models, it'd help us a lot if you already have them available!

Thanks!

opened by qunliu 4
Run Example failed in windows10
Hi Team, When I install bytesep according to the readdme, execute the demo command.

python3 -m bytesep separate \ --source_type="vocals" \ --audio_path="./resources/vocals_accompaniment_10s.mp3" \ --output_path="separated_results/output.mp3"

However, I can't get the result and any error info.

My env: Windows10 Python 3.7.5 (installed by conda)

I tried to use command line, PowerShell and git bash, and the results were the same. Does it lack support for the windows ?

Appreciate if you can help this !
opened by BigCat20196 3
Evaluation scores do not seem to improve during training
Hi, I modified this config so that there'd be 2 targets (vocals & accompaniment). (https://github.com/bytedance/music_source_separation/blob/master/scripts/4_train/musdb18/configs/accompaniment-vocals%2Cresunet_subbandtime.yaml) I also changed batch size from 16 to 12, and the MUSDB I use is HQ dataset (.wav) Those are the only modifications I did.

But given the evaluation scores during training, I'm not sure it'll reach 16.x & 8.x for accompaniment and vocals at step 500001

Step: 0, accompaniment: -0.606, vocals: -2.908

Step: 10000, accompaniment: 2.662, vocals: 0.399

Step: 20000, accompaniment: 2.680, vocals: 0.451

Step: 30000, accompaniment: 2.702, vocals: 0.498

Step: 40000, accompaniment: 2.726, vocals: 0.518

Step: 50000, accompaniment: 2.719, vocals: 0.539

Thanks in advance!
opened by sophia1488 2

无法提取人声（背景音可以正常提取）

--2021-10-21 23:15:39--  https://zenodo.org/record/5513378/files/resunet143_subbtandtime_vocals_8.8dB_350k_steps?download=1
正在解析主机 zenodo.org (zenodo.org)... 137.138.76.77
正在连接 zenodo.org (zenodo.org)|137.138.76.77|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 404 NOT FOUND
2021-10-21 23:15:40 错误 404：NOT FOUND。

↑ 看起来是因为模型文件404了......

命令为 python separate_scripts/separate.py --audio_path="（我的用户目录）/Desktop/test.mp3" --source_type="vocals" 全部信息如下

/Users/shihongsuo/opt/anaconda3/lib/python3.8/site-packages/librosa/core/audio.py:162: UserWarning: PySoundFile failed. Trying audioread instead.
  warnings.warn("PySoundFile failed. Trying audioread instead.")
Checkpoint path: /Users/shihongsuo/bytesep_data/resunet143_subbtandtime_vocals_8.8dB_350k_steps.pth
--2021-10-21 23:15:39--  https://zenodo.org/record/5513378/files/resunet143_subbtandtime_vocals_8.8dB_350k_steps?download=1
正在解析主机 zenodo.org (zenodo.org)... 137.138.76.77
正在连接 zenodo.org (zenodo.org)|137.138.76.77|:443... 已连接。
已发出 HTTP 请求，正在等待回应... 404 NOT FOUND
2021-10-21 23:15:40 错误 404：NOT FOUND。

Traceback (most recent call last):
  File "separate_scripts/separate.py", line 67, in <module>
    separate(args)
  File "separate_scripts/separate.py", line 26, in separate
    separator = SeparatorWrapper(
  File "/Users/shihongsuo/opt/anaconda3/lib/python3.8/site-packages/bytesep/inference.py", line 255, in __init__
    checkpoint = torch.load(self.checkpoint_path, map_location='cpu')
  File "/Users/shihongsuo/opt/anaconda3/lib/python3.8/site-packages/torch/serialization.py", line 595, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/Users/shihongsuo/opt/anaconda3/lib/python3.8/site-packages/torch/serialization.py", line 764, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
EOFError: Ran out of input

opened by xisang233 2

训练和预测用的默认模型不一致

在预测时用的默认模型是resunet143_subbtandtime_vocals_8.8dB_350k_steps/resunet143_subbtandtime_accompaniment_16.4dB_350k_steps.pth。训练时默认是vocals-accompaniment.unet,并且缺少optimizer_type，请问是为什么呢，如果想自己训练的话，应该用哪个呢？谢谢

opened by hihei 2

ffprobe error

Hi, I created a new Anaconda3 environment and installed the requirements and downloaded musdb18 using the command line in the README. But I could not pack them into hdf5:

$ PYTHONPATH=. ./scripts/1_pack_audios_to_hdf5s/musdb18/sr=44100,chn=2.sh
MUSDB18_DATASET_DIR=./datasets/musdb18
WORKSPACE=./workspaces/bytesep
Traceback (most recent call last):
  File "bytesep/dataset_creation/pack_audios_to_hdf5s/musdb18.py", line 195, in <module>
    pack_audios_to_hdf5s(args)
  File "bytesep/dataset_creation/pack_audios_to_hdf5s/musdb18.py", line 37, in pack_audios_to_hdf5s
    mus = musdb.DB(root=dataset_dir, subsets=[subset], split=split)
  File "/data1/howard/anaconda3/lib/python3.8/site-packages/musdb/__init__.py", line 115, in __init__
    self.tracks = self.load_mus_tracks(subsets=subsets, split=split)
  File "/data1/howard/anaconda3/lib/python3.8/site-packages/musdb/__init__.py", line 256, in load_mus_tracks
    track = MultiTrack(
  File "/data1/howard/anaconda3/lib/python3.8/site-packages/musdb/audio_classes.py", line 133, in __init__
    super(MultiTrack, self).__init__(path=path, *args, **kwargs)
  File "/data1/howard/anaconda3/lib/python3.8/site-packages/musdb/audio_classes.py", line 52, in __init__
    self.info = stempeg.Info(self.path)
  File "/data1/howard/anaconda3/lib/python3.8/site-packages/stempeg/read.py", line 328, in __init__
    self.info = ffmpeg.probe(filename)
  File "/data1/howard/anaconda3/lib/python3.8/site-packages/ffmpeg/_probe.py", line 23, in probe
    raise Error('ffprobe', out, err)
ffmpeg._run.Error: ffprobe error (see stderr output for detail)

I have installed ffmpeg using conda install -c conda-forge ffmpeg

opened by howardlau1999 2

NEW GOOGLE COLAB FOR BYTEDANCE 2022

For those who are facing problems with the previously distributed colab, you can use the one I wrote some time ago, it works perfectly, good use!

https://colab.research.google.com/drive/1AQlrzo9UOwAUrOpCyUYRZ8fRwHTqBeNl?usp=sharing

opened by lucasbr15 0

adjust the order of dependent packages

adjust the torch installation sequence

(venv3_test) ➜  music_source_separation git:(master) ✗ pip install torch
Collecting torch
  Downloading torch-1.9.1-cp38-none-macosx_10_9_x86_64.whl (218.8 MB)
     |████████████████████████████████| 218.8 MB 325 kB/s
Collecting typing-extensions
  Downloading typing_extensions-3.10.0.2-py3-none-any.whl (26 kB)
Installing collected packages: typing-extensions, torch
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
bytesep 0.0.1 requires h5py==2.10.0, which is not installed.
bytesep 0.0.1 requires inplace-abn==1.1.0, which is not installed.
bytesep 0.0.1 requires librosa==0.8.0, which is not installed.
bytesep 0.0.1 requires matplotlib==3.3.4, which is not installed.
bytesep 0.0.1 requires musdb==0.4.0, which is not installed.
bytesep 0.0.1 requires museval==0.4.0, which is not installed.
bytesep 0.0.1 requires numpy==1.18.5, which is not installed.
bytesep 0.0.1 requires pytorch_lightning==1.2.1, which is not installed.
bytesep 0.0.1 requires torchlibrosa==0.0.9, which is not installed.
bytesep 0.0.1 requires torch==1.7.1, but you have torch 1.9.1 which is incompatible.

adjustment completed

(venv3_test) ➜  music_source_separation git:(master) ✗ pip install -r requirements.txt
Collecting h5py==2.10.0
  Downloading h5py-2.10.0-cp38-cp38-macosx_10_9_x86_64.whl (3.0 MB)
     |████████████████████████████████| 3.0 MB 373 kB/s
Collecting librosa==0.8.1
  Downloading librosa-0.8.1-py3-none-any.whl (203 kB)
     |████████████████████████████████| 203 kB 237 kB/s
Collecting numpy==1.20.3
  Downloading numpy-1.20.3-cp38-cp38-macosx_10_9_x86_64.whl (16.0 MB)
     |████████████████████████████████| 16.0 MB 1.3 MB/s
Collecting numba==0.54.0
  Downloading numba-0.54.0-cp38-cp38-macosx_10_14_x86_64.whl (2.3 MB)
     |████████████████████████████████| 2.3 MB 1.8 MB/s
Collecting musdb==0.4.0
  Downloading musdb-0.4.0-py2.py3-none-any.whl (29 kB)
Collecting pytorch_lightning==1.2.1
  Downloading pytorch_lightning-1.2.1-py3-none-any.whl (814 kB)
     |████████████████████████████████| 814 kB 1.7 MB/s
Collecting torchlibrosa==0.0.9
  Downloading torchlibrosa-0.0.9-py3-none-any.whl (11 kB)
Collecting matplotlib==3.3.4
  Downloading matplotlib-3.3.4-cp38-cp38-macosx_10_9_x86_64.whl (8.5 MB)
     |████████████████████████████████| 8.5 MB 2.2 MB/s
Collecting museval==0.4.0
  Downloading museval-0.4.0-py2.py3-none-any.whl (21 kB)
Collecting inplace-abn==1.1.0
  Downloading inplace-abn-1.1.0.tar.gz (137 kB)
     |████████████████████████████████| 137 kB 1.1 MB/s
Collecting torch==1.7.1
  Downloading torch-1.7.1-cp38-none-macosx_10_9_x86_64.whl (108.9 MB)
     |████████████████████████████████| 108.9 MB 682 kB/s
Collecting six
  Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting packaging>=20.0
  Using cached packaging-21.0-py3-none-any.whl (40 kB)
Collecting resampy>=0.2.2
  Downloading resampy-0.2.2.tar.gz (323 kB)
     |████████████████████████████████| 323 kB 2.8 MB/s
Collecting soundfile>=0.10.2
  Downloading SoundFile-0.10.3.post1-py2.py3.cp26.cp27.cp32.cp33.cp34.cp35.cp36.pp27.pp32.pp33-none-macosx_10_5_x86_64.macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.whl (613 kB)
     |████████████████████████████████| 613 kB 2.9 MB/s
Collecting audioread>=2.0.0
  Downloading audioread-2.1.9.tar.gz (377 kB)
     |████████████████████████████████| 377 kB 3.0 MB/s
Collecting joblib>=0.14
  Downloading joblib-1.0.1-py3-none-any.whl (303 kB)
     |████████████████████████████████| 303 kB 897 kB/s
Collecting decorator>=3.0.0
  Downloading decorator-5.1.0-py3-none-any.whl (9.1 kB)
Collecting pooch>=1.0
  Downloading pooch-1.5.1-py3-none-any.whl (57 kB)
     |████████████████████████████████| 57 kB 2.2 MB/s
Collecting scipy>=1.0.0
  Downloading scipy-1.7.1-cp38-cp38-macosx_10_9_x86_64.whl (32.6 MB)
     |████████████████████████████████| 32.6 MB 695 kB/s
Collecting scikit-learn!=0.19.0,>=0.14.0
  Downloading scikit_learn-0.24.2-cp38-cp38-macosx_10_13_x86_64.whl (7.2 MB)
     |████████████████████████████████| 7.2 MB 480 kB/s
Collecting llvmlite<0.38,>=0.37.0rc1
  Downloading llvmlite-0.37.0-cp38-cp38-macosx_10_9_x86_64.whl (19.1 MB)
     |████████████████████████████████| 19.1 MB 2.4 MB/s
Requirement already satisfied: setuptools in /Users/songming/venv3_test/lib/python3.8/site-packages (from numba==0.54.0->-r requirements.txt (line 5)) (56.0.0)
Collecting pyaml
  Downloading pyaml-21.8.3-py2.py3-none-any.whl (17 kB)
Collecting stempeg>=0.2.3
  Downloading stempeg-0.2.3-py3-none-any.whl (963 kB)
     |████████████████████████████████| 963 kB 3.3 MB/s
Collecting tqdm
  Downloading tqdm-4.62.3-py2.py3-none-any.whl (76 kB)
     |████████████████████████████████| 76 kB 3.4 MB/s
Collecting PyYAML!=5.4.*,>=5.1
  Downloading PyYAML-5.3.1.tar.gz (269 kB)
     |████████████████████████████████| 269 kB 2.8 MB/s
Collecting fsspec[http]>=0.8.1
  Downloading fsspec-2021.8.1-py3-none-any.whl (119 kB)
     |████████████████████████████████| 119 kB 911 kB/s
Collecting future>=0.17.1
  Downloading future-0.18.2.tar.gz (829 kB)
     |████████████████████████████████| 829 kB 2.7 MB/s
Collecting tensorboard>=2.2.0
  Downloading tensorboard-2.6.0-py3-none-any.whl (5.6 MB)
     |████████████████████████████████| 5.6 MB 2.1 MB/s
Collecting python-dateutil>=2.1
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     |████████████████████████████████| 247 kB 2.7 MB/s
Collecting cycler>=0.10
  Downloading cycler-0.10.0-py2.py3-none-any.whl (6.5 kB)
Collecting pillow>=6.2.0
  Downloading Pillow-8.3.2-cp38-cp38-macosx_10_10_x86_64.whl (3.0 MB)
     |████████████████████████████████| 3.0 MB 2.5 MB/s
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.3
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting kiwisolver>=1.0.1
  Downloading kiwisolver-1.3.2-cp38-cp38-macosx_10_9_x86_64.whl (61 kB)
     |████████████████████████████████| 61 kB 435 kB/s
Collecting pandas>=1.0.1
  Downloading pandas-1.3.3-cp38-cp38-macosx_10_9_x86_64.whl (11.4 MB)
     |████████████████████████████████| 11.4 MB 249 kB/s
Collecting jsonschema
  Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
     |████████████████████████████████| 56 kB 310 kB/s
Collecting simplejson
  Downloading simplejson-3.17.5-cp38-cp38-macosx_10_9_x86_64.whl (74 kB)
     |████████████████████████████████| 74 kB 340 kB/s
Requirement already satisfied: typing-extensions in /Users/songming/venv3_test/lib/python3.8/site-packages (from torch==1.7.1->-r requirements.txt (line 12)) (3.10.0.2)
Collecting aiohttp
  Downloading aiohttp-3.7.4.post0-cp38-cp38-macosx_10_14_x86_64.whl (648 kB)
     |████████████████████████████████| 648 kB 186 kB/s
Collecting requests
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
     |████████████████████████████████| 62 kB 417 kB/s
Collecting pytz>=2017.3
  Downloading pytz-2021.1-py2.py3-none-any.whl (510 kB)
     |████████████████████████████████| 510 kB 454 kB/s
Collecting appdirs
  Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting threadpoolctl>=2.0.0
  Downloading threadpoolctl-2.2.0-py3-none-any.whl (12 kB)
Collecting cffi>=1.0
  Downloading cffi-1.14.6-cp38-cp38-macosx_10_9_x86_64.whl (176 kB)
     |████████████████████████████████| 176 kB 579 kB/s
Collecting pycparser
  Downloading pycparser-2.20-py2.py3-none-any.whl (112 kB)
     |████████████████████████████████| 112 kB 624 kB/s
Collecting ffmpeg-python>=0.2.0
  Downloading ffmpeg_python-0.2.0-py3-none-any.whl (25 kB)
Collecting tensorboard-data-server<0.7.0,>=0.6.0
  Downloading tensorboard_data_server-0.6.1-py3-none-macosx_10_9_x86_64.whl (3.5 MB)
     |████████████████████████████████| 3.5 MB 957 kB/s
Collecting tensorboard-plugin-wit>=1.6.0
  Downloading tensorboard_plugin_wit-1.8.0-py3-none-any.whl (781 kB)
     |████████████████████████████████| 781 kB 1.3 MB/s
Collecting werkzeug>=0.11.15
  Downloading Werkzeug-2.0.1-py3-none-any.whl (288 kB)
     |████████████████████████████████| 288 kB 1.6 MB/s
Collecting markdown>=2.6.8
  Downloading Markdown-3.3.4-py3-none-any.whl (97 kB)
     |████████████████████████████████| 97 kB 2.2 MB/s
Collecting google-auth<2,>=1.6.3
  Downloading google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
     |████████████████████████████████| 152 kB 2.3 MB/s
Collecting absl-py>=0.4
  Downloading absl_py-0.14.0-py3-none-any.whl (131 kB)
     |████████████████████████████████| 131 kB 2.4 MB/s
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting grpcio>=1.24.3
  Using cached grpcio-1.40.0-cp38-cp38-macosx_10_10_x86_64.whl (4.0 MB)
Collecting protobuf>=3.6.0
  Downloading protobuf-3.18.0-cp38-cp38-macosx_10_9_x86_64.whl (1.0 MB)
     |████████████████████████████████| 1.0 MB 2.3 MB/s
Collecting wheel>=0.26
  Downloading wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting cachetools<5.0,>=2.0.0
  Downloading cachetools-4.2.2-py3-none-any.whl (11 kB)
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
     |████████████████████████████████| 155 kB 2.9 MB/s
Collecting rsa<5,>=3.1.4
  Downloading rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting requests-oauthlib>=0.7.0
  Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
     |████████████████████████████████| 77 kB 2.5 MB/s
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.7-py2.py3-none-any.whl (138 kB)
     |████████████████████████████████| 138 kB 3.1 MB/s
Collecting idna<4,>=2.5
  Downloading idna-3.2-py3-none-any.whl (59 kB)
     |████████████████████████████████| 59 kB 3.3 MB/s
Collecting charset-normalizer~=2.0.0
  Downloading charset_normalizer-2.0.6-py3-none-any.whl (37 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
     |████████████████████████████████| 145 kB 2.7 MB/s
Collecting oauthlib>=3.0.0
  Downloading oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
     |████████████████████████████████| 146 kB 2.6 MB/s
Collecting multidict<7.0,>=4.5
  Downloading multidict-5.1.0-cp38-cp38-macosx_10_14_x86_64.whl (49 kB)
     |████████████████████████████████| 49 kB 3.4 MB/s
Collecting chardet<5.0,>=2.0
  Downloading chardet-4.0.0-py2.py3-none-any.whl (178 kB)
     |████████████████████████████████| 178 kB 2.6 MB/s
Collecting attrs>=17.3.0
  Downloading attrs-21.2.0-py2.py3-none-any.whl (53 kB)
     |████████████████████████████████| 53 kB 2.5 MB/s
Collecting yarl<2.0,>=1.0
  Downloading yarl-1.6.3-cp38-cp38-macosx_10_14_x86_64.whl (124 kB)
     |████████████████████████████████| 124 kB 2.3 MB/s
Collecting async-timeout<4.0,>=3.0
  Downloading async_timeout-3.0.1-py3-none-any.whl (8.2 kB)
Collecting pyrsistent>=0.14.0
  Downloading pyrsistent-0.18.0-cp38-cp38-macosx_10_9_x86_64.whl (68 kB)
     |████████████████████████████████| 68 kB 2.7 MB/s
Using legacy 'setup.py install' for inplace-abn, since package 'wheel' is not installed.
Using legacy 'setup.py install' for audioread, since package 'wheel' is not installed.
Using legacy 'setup.py install' for future, since package 'wheel' is not installed.
Using legacy 'setup.py install' for PyYAML, since package 'wheel' is not installed.
Using legacy 'setup.py install' for resampy, since package 'wheel' is not installed.
Installing collected packages: urllib3, pyasn1, idna, charset-normalizer, certifi, six, rsa, requests, pyparsing, pycparser, pyasn1-modules, oauthlib, numpy, multidict, llvmlite, future, cachetools, yarl, threadpoolctl, scipy, requests-oauthlib, PyYAML, packaging, numba, joblib, google-auth, ffmpeg-python, chardet, cffi, attrs, async-timeout, appdirs, wheel, werkzeug, tqdm, tensorboard-plugin-wit, tensorboard-data-server, stempeg, soundfile, scikit-learn, resampy, pytz, python-dateutil, pyrsistent, pyaml, protobuf, pooch, markdown, grpcio, google-auth-oauthlib, fsspec, decorator, audioread, aiohttp, absl-py, torch, tensorboard, simplejson, pillow, pandas, musdb, librosa, kiwisolver, jsonschema, cycler, torchlibrosa, pytorch-lightning, museval, matplotlib, inplace-abn, h5py
    Running setup.py install for future ... done
    Running setup.py install for PyYAML ... done
    Running setup.py install for resampy ... done
    Running setup.py install for audioread ... done
  Attempting uninstall: torch
    Found existing installation: torch 1.9.1
    Uninstalling torch-1.9.1:
      Successfully uninstalled torch-1.9.1
    Running setup.py install for inplace-abn ... done
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
bytesep 0.0.1 requires librosa==0.8.0, but you have librosa 0.8.1 which is incompatible.
bytesep 0.0.1 requires numpy==1.18.5, but you have numpy 1.20.3 which is incompatible.
Successfully installed PyYAML-5.3.1 absl-py-0.14.0 aiohttp-3.7.4.post0 appdirs-1.4.4 async-timeout-3.0.1 attrs-21.2.0 audioread-2.1.9 cachetools-4.2.2 certifi-2021.5.30 cffi-1.14.6 chardet-4.0.0 charset-normalizer-2.0.6 cycler-0.10.0 decorator-5.1.0 ffmpeg-python-0.2.0 fsspec-2021.8.1 future-0.18.2 google-auth-1.35.0 google-auth-oauthlib-0.4.6 grpcio-1.40.0 h5py-2.10.0 idna-3.2 inplace-abn-1.1.0 joblib-1.0.1 jsonschema-3.2.0 kiwisolver-1.3.2 librosa-0.8.1 llvmlite-0.37.0 markdown-3.3.4 matplotlib-3.3.4 multidict-5.1.0 musdb-0.4.0 museval-0.4.0 numba-0.54.0 numpy-1.20.3 oauthlib-3.1.1 packaging-21.0 pandas-1.3.3 pillow-8.3.2 pooch-1.5.1 protobuf-3.18.0 pyaml-21.8.3 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.20 pyparsing-2.4.7 pyrsistent-0.18.0 python-dateutil-2.8.2 pytorch-lightning-1.2.1 pytz-2021.1 requests-2.26.0 requests-oauthlib-1.3.0 resampy-0.2.2 rsa-4.7.2 scikit-learn-0.24.2 scipy-1.7.1 simplejson-3.17.5 six-1.16.0 soundfile-0.10.3.post1 stempeg-0.2.3 tensorboard-2.6.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.0 threadpoolctl-2.2.0 torch-1.7.1 torchlibrosa-0.0.9 tqdm-4.62.3 urllib3-1.26.7 werkzeug-2.0.1 wheel-0.37.0 yarl-1.6.3
WARNING: You are using pip version 21.1.1; however, version 21.2.4 is available.
You should consider upgrading via the '/Users/songming/venv3_test/bin/python -m pip install --upgrade pip' command.

opened by songminger 0

主分支使用发现两个命令和参数问题

1、readme.md记录的命令python3 -m bytesep download_checkpoints，最后不是_而是-，即python3 -m bytesep download-checkpoints 2、去伴奏的参数检查没问题，但是去人声的命令及类型为accompaniment时，检查家在的resunet143_subbtandtime_accompaniment_16.4dB_500k_steps_v2.pth文件大小时候，代码里是和414046363判断相等，实际文件大小是414036369，需要修改下__main__.py 127:65 ，414046363改为414036369 我本地这么修改是运行没有问题

opened by baozezhong 0
现在完全无法转换

在colab 上面現在完全無法使用了 !python3 /content/music_source_separation/bytesep/separate.py 執行這個指令都會出錯

OSError: /usr/local/lib/python3.7/dist-packages/torchtext/_torchtext.so: undefined symbol: _ZNK3c104Type14isSubtypeOfExtESt10shared_ptrIS0_EPSo

希望能解决问题,谢谢！

opened by seq2193 0
poor peformance using short segment <1s

Hi,

Thanks for you grant work! The repo achieve good performance for musice separation offline.

However, for online setting(using short segment <1s), it achieve poor performance. There are some short accompaniment on separated vocals. And about -2 SDR is got. Could you provide some suggestion about how to achieve better performance.

Thanks!

opened by John-Yao 0

模型权重链接是失效了吗？

Hi~ 当我运行 python3 -m bytesep download-checkpoints 的时候，输出如下。想问下是我的操作错误还是模型权重链接失效了呢？感谢回复！

--2022-06-25 00:07:44--  https://zenodo.org/record/5804160/files%5Cmobilenet_subbtandtime_vocals_7.2dB_500k_steps_v2.pth?download=1
Connecting to 127.0.0.1:7890... connected.
Proxy request sent, awaiting response... 404 NOT FOUND
2022-06-25 00:07:46 ERROR 404: NOT FOUND.

--2022-06-25 00:07:46--  https://zenodo.org/record/5804160/files%5Cmobilenet_subbtandtime_accompaniment_14.6dB_500k_steps_v2.pth?download=1
Connecting to 127.0.0.1:7890... connected.
Proxy request sent, awaiting response... 404 NOT FOUND
2022-06-25 00:07:47 ERROR 404: NOT FOUND.

--2022-06-25 00:07:47--  https://zenodo.org/record/5804160/files%5Cresunet143_subbtandtime_vocals_8.7dB_500k_steps_v2.pth?download=1
Connecting to 127.0.0.1:7890... connected.
Proxy request sent, awaiting response... 404 NOT FOUND
2022-06-25 00:07:49 ERROR 404: NOT FOUND.

--2022-06-25 00:07:49--  https://zenodo.org/record/5804160/files%5Cresunet143_subbtandtime_accompaniment_16.4dB_500k_steps_v2.pth?download=1
Connecting to 127.0.0.1:7890... connected.
Proxy request sent, awaiting response... 404 NOT FOUND
2022-06-25 00:07:50 ERROR 404: NOT FOUND.

--2022-06-25 00:07:50--  https://zenodo.org/record/5804160/files%5Ctrain_scripts.zip?download=1
Connecting to 127.0.0.1:7890... connected.
Proxy request sent, awaiting response... 404 NOT FOUND
2022-06-25 00:07:52 ERROR 404: NOT FOUND.

Archive:  C:/Users/Administrator/bytesep_data/train_scripts.zip
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in C:/Users/Administrator/bytesep_data/train_scripts.zip,
        and cannot find C:/Users/Administrator/bytesep_data/train_scripts.zip.zip, period.

opened by hemath1001 4

Music source separation is a task to separate audio recordings into individual sources

Related tags

Overview

Music Source Separation

Demos

Separation

Method 1. Separate by installing the package

Method 2. Separate by using the source code

1. Install dependencies

2. Download checkpoints

3. Separate vocals and accompaniment

Train a music source separation system from scratch

1. Download dataset

2. Pack audio files into hdf5 files

3. Create indexes for training

3. Create evaluation audios

4. Train & evaluate & save checkpoints

5. Inference

Results

Reference

FAQ

Comments

Owner

Bytedance Inc.

a dnn ai project to classify which food people are eating on audio recordings

Facestar dataset. High quality audio-visual recordings of human conversational speech.

Music Source Separation; Train & Eval & Inference piplines and pretrained models we used for 2021 ISMIR MDX Challenge.

A PyTorch Implementation of the paper - Choi, Woosung, et al. "Investigating u-nets with various intermediate blocks for spectrogram-based singing voice separation." 21th International Society for Music Information Retrieval Conference, ISMIR. 2020.

audioLIME: Listenable Explanations Using Source Separation

A two-stage U-Net for high-fidelity denoising of historical recordings

Real-time analysis of intracranial neurophysiology recordings.

Experimenting with computer vision techniques to generate annotated image datasets from gameplay recordings automatically.

A very simple tool for situations where optimization with onnx-simplifier would exceed the Protocol Buffers upper file size limit of 2GB, or simply to separate onnx files to any size you want.

Source code for "MusCaps: Generating Captions for Music Audio" (IJCNN 2021)

Code for the ICASSP-2021 paper: Continuous Speech Separation with Conformer.

harmonic-percussive-residual separation algorithm wrapped as a VST3 plugin (iPlug2)

Speech Separation Using an Asynchronous Fully Recurrent Convolutional Neural Network

Offical implementation for "Trash or Treasure? An Interactive Dual-Stream Strategy for Single Image Reflection Separation".

BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation

Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learning

Code for Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding

Compute execution plan: A DAG representation of work that you want to get done. Individual nodes of the DAG could be simple python or shell tasks or complex deeply nested parallel branches or embedded DAGs themselves.