Steerable discovery of neural audio effects

Overview

Steerable discovery of neural audio effects

Christian J. Steinmetz and Joshua D. Reiss

Abstract

Applications of deep learning for audio effects often focus on modeling analog effects or learning to control effects to emulate a trained audio engineer. However, deep learning approaches also have the potential to expand creativity through neural audio effects that enable new sound transformations. While recent work demonstrated that neural networks with random weights produce compelling audio effects, control of these effects is limited and unintuitive. To address this, we introduce a method for the steerable discovery of neural audio effects. This method enables the design of effects using example recordings provided by the user. We demonstrate how this method produces an effect similar to the target effect, along with interesting inaccuracies, while also providing perceptually relevant controls.

Watch the demo video.

Accepted to the NeurIPS 2021 Workshop for Machine Learning for Creativity and Design

source code for https://arxiv.org/abs/2005.11248 "Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics"

Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics This work will be published in Nature Biomedical

71 Nov 15, 2022

Unsupervised Discovery of Object Radiance Fields

Unsupervised Discovery of Object Radiance Fields by Hong-Xing Yu, Leonidas J. Guibas and Jiajun Wu from Stanford University. arXiv link: https://arxiv

148 Nov 30, 2022

Official implementation of "A Unified Objective for Novel Class Discovery", ICCV2021 (Oral)

A Unified Objective for Novel Class Discovery This is the official repository for the paper: A Unified Objective for Novel Class Discovery Enrico Fini

118 Dec 26, 2022

Pytorch implementation of the unsupervised object discovery method LOST.

LOST Pytorch implementation of the unsupervised object discovery method LOST. More details can be found in the paper: Localizing Objects with Self-Sup

189 Dec 25, 2022

A python implementation of Physics-informed Spline Learning for nonlinear dynamics discovery

PiSL A python implementation of Physics-informed Spline Learning for nonlinear dynamics discovery. Sun, F., Liu, Y. and Sun, H., 2021. Physics-informe

8 Jul 13, 2022

This is the implementation of "SELF SUPERVISED REPRESENTATION LEARNING WITH DEEP CLUSTERING FOR ACOUSTIC UNIT DISCOVERY FROM RAW SPEECH" submitted to ICASSP 2022

CPC_DeepCluster This is the implementation of "SELF SUPERVISED REPRESENTATION LEARNING WITH DEEP CLUSTERING FOR ACOUSTIC UNIT DISCOVERY FROM RAW SPEEC

2 Sep 15, 2022

Comments

Converting models to VST3 plugin

Thank you for working on this project! I have been learning a lot related to Machine Learning thanks to you. Any ideas how to convert the PT file I get after training to a VST3 plugin ? Also where can I get more info on how to tweak the TCN model training parameters (what each parameter means) ?

opened by creativeguitar 2
pip install google.colab causes error: failed building wheel for pandas

Thank you for your code. I need download a google.colab module, but it shows the error: failed building wheel for pandas, and it also tries to update my pandas, but finally it shows another error: legacy-install-failure, like below.

opened by UNdeng 0
Thank you!

I just saw this on Hacker News. I can't thank you enough!

I can finally DEcompress audio! I will train a compressor in reverse, and I will re-remaster all the crappily-remastered tracks!

BTW, I only buy DRM-free music, so I can do stuff like this on my personal copies.

opened by danuker 0

Steerable discovery of neural audio effects

Related tags

Overview

Steerable discovery of neural audio effects

Abstract

You might also like...

source code for https://arxiv.org/abs/2005.11248 "Accelerating Antimicrobial Discovery with Controllable Deep Generative Models and Molecular Dynamics"

Unsupervised Discovery of Object Radiance Fields

Official implementation of "A Unified Objective for Novel Class Discovery", ICCV2021 (Oral)

Pytorch implementation of the unsupervised object discovery method LOST.

A python implementation of Physics-informed Spline Learning for nonlinear dynamics discovery

This is the implementation of "SELF SUPERVISED REPRESENTATION LEARNING WITH DEEP CLUSTERING FOR ACOUSTIC UNIT DISCOVERY FROM RAW SPEECH" submitted to ICASSP 2022

Pytorch implementation of paper "Learning Co-segmentation by Segment Swapping for Retrieval and Discovery"

This repository holds code and data for our PETS'22 article 'From "Onion Not Found" to Guard Discovery'.

Systemic Evolutionary Chemical Space Exploration for Drug Discovery

Comments

Converting models to VST3 plugin

pip install google.colab causes error: failed building wheel for pandas

Thank you!

Owner

Christian J. Steinmetz

BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation

This repository contains a re-implementation of the code for the CVPR 2021 paper "Omnimatte: Associating Objects and Their Effects in Video."

Code for "On the Effects of Batch and Weight Normalization in Generative Adversarial Networks"

SurvITE: Learning Heterogeneous Treatment Effects from Time-to-Event Data

An Image compression simulator that uses Source Extractor and Monte Carlo methods to examine the post compressive effects different compression algorithms have.

This repository contains notebook implementations of the following Neural Process variants: Conditional Neural Processes (CNPs), Neural Processes (NPs), Attentive Neural Processes (ANPs).

Seach Losses of our paper 'Loss Function Discovery for Object Detection via Convergence-Simulation Driven Search', accepted by ICLR 2021.

TraND: Transferable Neighborhood Discovery for Unsupervised Cross-domain Gait Recognition.

KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control

Neighborhood Contrastive Learning for Novel Class Discovery