commonfate 📦commonfate 📦 - Common Fate Model and Transform.

Related tags

Audio commonfate
Overview

Common Fate Transform and Model for Python

This package is a python implementation of the Common Fate Transform and Model to be used for audio source separation as described in an ICASSP 2016 paper "Common Fate Model for Unison source Separation".

Common Fate Transform

cft

The Common Fate Transform is based on a signal representation that divides a complex spectrogram into a grid of patches of arbitrary size. These complex patches are then processed by a two-dimensional discrete Fourier transform, forming a tensor representation which reveals spectral and temporal modulation textures.

Common Fate Model

cfm

An adapted factorization model similar to the PARAFAC/CANDECOMP factorisation allows to decompose the common fate transform tesnor into different time-varying harmonic sources based on their particular common modulation profile: hence the name Common Fate Model.

Usage

See the full API documentation at http://aliutkus.github.io/commonfate.

Applying the Common Fate Transform

import commonfate

# # forward transform

# STFT Parameters

framelength = 1024
hopsize = 256
X = commonfate.transform.forward(signal, framelength, hopsize)

# Patch Parameters
W = (32, 48)
mhop = (16, 24)

Z = commonfate.transform.forward(X, W, mhop, real=False)

# inverse transform of cft
Y = commonfate.transform.inverse(
    Z, fdim=2, hop=mhop, shape=X.shape, real=False
)
# back to time domain
y = commonfate.transform.inverse(
    Y, fdim=1, hop=hopsize, shape=x.shape
)

Fitting the Common Fate Model

import commonfate

# initialiase and fit the common fate model
cfm = commonfate.model.CFM(z, nb_components=10, nb_iter=100).fit()

# get the fitted factors
(A, H, C) = cfm.factors

# returns the of z approximation using the fitted factors
z_hat = cfm.approx()

Decompose an audio signal using CFT and CFM

commonfate has a built-in wrapper which computes the Common Fate Transform, fits the model according to the Common Fate Model and return the synthesised time domain signal components obtained through wiener / soft mask filtering.

The following example requires to install pysoundfile.

import commonfate
import soundfile as sf

# loading signal
(audio, fs) = sf.read(filename, always_2d=True)

# decomposes the audio signal into
# (nb_components, nb_samples, nb_channels)
components = decompose.process(
    audio,
    nb_iter=100,
    nb_components=10,
    n_fft=1024,
    n_hop=256,
    cft_patch=(32, 48),
    cft_hop=(16, 24)
)

# write out the third component to wave file
sf.write(
    "comp_3.wav",
    components[2, ...],
    fs
)

Optimisations

The current common fate model implementation makes heavily use of the Einstein Notation. We use the numpy einsum module which can be slow on large tensors. To speed up the computation time we recommend to install Daniel Smith's opt_einsum package.

Installation via pip
pip install -e 'git+https://github.com/dgasmith/opt_einsum.git#egg=opt_einsum'

commonfate automatically detects if the package is installed.

References

You can download and read the paper here. If you use this package, please reference to the following publication:

@inproceedings{stoeter2016cfm,
  TITLE = {{Common Fate Model for Unison source Separation}},
  AUTHOR = {St{\"o}ter, Fabian-Robert and Liutkus, Antoine and Badeau, Roland and Edler, Bernd and Magron, Paul},
  BOOKTITLE = {{41st International Conference on Acoustics, Speech and Signal Processing (ICASSP)}},
  ADDRESS = {Shanghai, China},
  PUBLISHER = {{IEEE}},
  SERIES = {Proceedings of the 41st International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  YEAR = {2016},
  KEYWORDS = {Non-Negative tensor factorization ; Sound source separation ; Common Fate Model},
}
You might also like...
C++ library for audio and music analysis, description and synthesis, including Python bindings

Essentia Essentia is an open-source C++ library for audio analysis and audio-based music information retrieval released under the Affero GPL license.

An app made in Python using the PyTube and Tkinter libraries to download videos and MP3 audio.

yt-dl (GUI Edition) An app made in Python using the PyTube and Tkinter libraries to download videos and MP3 audio. How do I download this? Windows: Fi

Small Python application that links a Digico console and Reaper, handling automatic marker insertion and tracking.
Small Python application that links a Digico console and Reaper, handling automatic marker insertion and tracking.

Digico-Reaper-Link This is a small GUI based helper application designed to help with using Digico's Copy Audio function with a Reaper DAW used for re

Anki vector Music ❤ is the best and only Telegram VC player with playlists, Multi Playback, Channel play and more
Anki vector Music ❤ is the best and only Telegram VC player with playlists, Multi Playback, Channel play and more

Anki Vector Music 🎵 A bot that can play music on Telegram Group and Channel Voice Chats Available on telegram as @Anki Vector Music Features 🔥 Thumb

Just-Music - Spotify API Driven Music Web app, that allows to listen and control and share songs

Just Music... Just Music Is A Web APP That Allows Users To Play Song Using Spoti

Audio fingerprinting and recognition in Python
Audio fingerprinting and recognition in Python

dejavu Audio fingerprinting and recognition algorithm implemented in Python, see the explanation here: How it works Dejavu can memorize audio by liste

Python library for audio and music analysis

librosa A python package for music and audio analysis. Documentation See https://librosa.org/doc/ for a complete reference manual and introductory tut

?️ Open Source Audio Matching and Mastering
?️ Open Source Audio Matching and Mastering

Matching + Mastering = ❤️ Matchering 2.0 is a novel Containerized Web Application and Python Library for audio matching and mastering. It follows a si

Python Audio Analysis Library: Feature Extraction, Classification, Segmentation and Applications
Python Audio Analysis Library: Feature Extraction, Classification, Segmentation and Applications

A Python library for audio feature extraction, classification, segmentation and applications This doc contains general info. Click here for the comple

Comments
  • Change factorisation notation so that it matches the paper

    Change factorisation notation so that it matches the paper

    https://github.com/aliutkus/commonfate/blob/master/commonfate/model.py#L121

    is different to paper where we state:

    notation

    the output should therefore be changed to (A, H, C)

    enhancement 
    opened by faroit 1
  • Raised a MemoryError after called decompose.process

    Raised a MemoryError after called decompose.process

    code: (audio, fs) = sf.read('1.wav', always_2d=True) components = commonfate.decompose.process( audio, nb_components=10, ) sf.write( "comp_3.wav", components[2, ...], fs ) raise errors: Traceback (most recent call last): File "F:/py_project/independent-project.git/music_ctl_lettin/music_feature_engineering/process_music.py", line 190, in nb_components=10, File "C:\Users\Besitzer\AppData\Local\Programs\Python\Python36-32\lib\site-packages\commonfate\decompose.py", line 63, in process n_hop, File "C:\Users\Besitzer\AppData\Local\Programs\Python\Python36-32\lib\site-packages\commonfate\transform.py", line 325, in forward stft = fftFunction(stft, frameShape, axes=range(len(frameShape))) File "C:\Users\Besitzer\AppData\Local\Programs\Python\Python36-32\lib\site-packages\numpy\fft\fftpack.py", line 1099, in rfftn a = rfft(a, s[-1], axes[-1], norm) File "C:\Users\Besitzer\AppData\Local\Programs\Python\Python36-32\lib\site-packages\numpy\fft\fftpack.py", line 372, in rfft _real_fft_cache) File "C:\Users\Besitzer\AppData\Local\Programs\Python\Python36-32\lib\site-packages\numpy\fft\fftpack.py", line 83, in _raw_fft r = work_function(a, wsave) MemoryError

    Any ideas?

    opened by okideal 1
Releases(0.1.3)
Owner
Fabian-Robert Stöter
Audio-ML researcher
Fabian-Robert Stöter
This library provides common speech features for ASR including MFCCs and filterbank energies.

python_speech_features This library provides common speech features for ASR including MFCCs and filterbank energies. If you are not sure what MFCCs ar

James Lyons 2.2k Jan 4, 2023
This is an AI that runs in the terminal. It is a voice assistant that can do common activities and can also help in your coding doubts like

This is an AI that runs in the terminal. It is a voice assistant that can do common activities and can also help in your coding doubts like

OneBit 1 Nov 5, 2021
Using python to generate a bat script of repetitive lines of code that differ in some way but can sort out a group of audio files according to their common names

Batch Sorting Using python to generate a bat script of repetitive lines of code that differ in some way but can sort out a group of audio files accord

David Mainoo 1 Oct 29, 2021
Deep learning transformer model that generates unique music sequences.

music-ai Deep learning transformer model that generates unique music sequences. Abstract In 2017, a new state-of-the-art was published for natural lan

xacer 6 Nov 19, 2022
A small project where I identify notes and key harmonies in a piece of music and use them further to recreate and generate the same piece of music through Python

A small project where I identify notes and key harmonies in a piece of music and use them further to recreate and generate the same piece of music through Python

null 5 Oct 7, 2022
nicfit 425 Jan 1, 2023
Read music meta data and length of MP3, OGG, OPUS, MP4, M4A, FLAC, WMA and Wave files with python 2 or 3

tinytag tinytag is a library for reading music meta data of MP3, OGG, OPUS, MP4, M4A, FLAC, WMA and Wave files with python Install pip install tinytag

Tom Wallroth 577 Dec 26, 2022
Read music meta data and length of MP3, OGG, OPUS, MP4, M4A, FLAC, WMA and Wave files with python 2 or 3

tinytag tinytag is a library for reading music meta data of MP3, OGG, OPUS, MP4, M4A, FLAC, WMA and Wave files with python Install pip install tinytag

Tom Wallroth 435 Feb 17, 2021
Speech recognition module for Python, supporting several engines and APIs, online and offline.

SpeechRecognition Library for performing speech recognition, with support for several engines and APIs, online and offline. Speech recognition engine/

Anthony Zhang 6.7k Jan 8, 2023
Music player and music library manager for Linux, Windows, and macOS

Ex Falso / Quod Libet - A Music Library / Editor / Player Quod Libet is a music management program. It provides several different ways to view your au

Quod Libet 1.2k Jan 7, 2023