User-friendly Voice Cloning Application

@Dannypeja @Aatularyan @AsterTheWanderer @Fzr2k @CrazyPlaysHD @lavarith @Andredenise @wotulong @cryptoCommunitiesCONTROLfuture-RAOKindx @HirabayashiCallie @DimensionWarped @Archviz360 @francisco-renteria @AlexSteveChungAlvarez @mithisha @brcisna @PabloSandovalxtxt @alexpeattie @matheusfillipe @vanpelt @cforcomputer @cclauss @mathigatti @CorentinJ @akaitsuki-ii @LeonardsonCC @greatabel @Jinksi @sebiglesias @akozdev @musikalkemist @aydodo @SynapticSage @noahyoda @vidalmaxime @jason-h-35 @thecodeboss @iamgroot42 @leopard627 @mruettgers @berdakh @adilsoybali @EdwardELeininger @DanielLin94144 @PatentLobster

Dear fans of the Voice Cloning Community,

since we are all interested in Voice Cloning or Audio Engineering overall and all have either used or co-developed the Real-Time-Voice-Cloning repo, we are well aware, that the repository is not maintained anymore and lacks structure, meaning that many errors stay, issues cannot get handled and the documentation is unsatisfying.

For those and more reasons, I have decided to begin a new project, aiming to provide the functionality of Real-Time-Voice-Cloning but enhancing it by adding support for other languages, taking multiple developers/maintainers abord and writing a good and clear documentation that explains both the theory behind and the application of the code.

Since I wanted to wait for an initial response, I have not yet begun structuring the repository (except for some basics) but I have a few things in mind that need to be taken care of if we want to improve the original repo:

1. Writing a documentation that can actually help beginners (tricky, needs to be done by someone who really understands the theory of pre-processing, spectrograms, etc.). First, a wiki structure needs to be discussed/drafted before actually writing it with all its sections (Theory, Prerequesites, Download, Learning, Training, Using GUI, etc.)
2. Structuring the directory hierarchy (lots of loose scripts in the old repo, maybe structuring it differently/cleaner -> would mean code adaption though)
3. Adding pre-trained models for different languages (I am working on a German model myself at the moment, could take some time though)
4. Developing a well-oiled work routine where issues get handled, new ideas discussed and things done.

It would be really sad to see a good idea end on the siding just for lack-of-maintenance reasons.

I am waiting for responses...

Greetings Sven

I have created a Link for the next 100 members: https://discord.gg/asym7axdgd After that we can renew but I wanted to avoid getting trolled. Feel free to add it to the readme @sveneschlbeck

Attempt to resolve error on multi-GPU system where vocoder preprocess fails because "data_parallel_workaround" is not recognized in mlrtvc/src/core/synthesizer/synthesize.py.

I have refactored the code from the original RTVC into the new repo structure that we developed in https://github.com/sveneschlbeck/Multi-Language-RTVC/discussions/2 .

Everything is functional, including the toolbox, CLI, preprocess and training. The training scripts save into the appropriate folder in the new saved_models folder structure, including the language code.

Some documentation on preprocess and training is provided. It is not enough, but can be expanded upon.

I did not implement a unified mel spectrogram definition for the encoder, synth, and vocoder. This is no longer necessary as the repo directory structure has been simplified without it.

A downloadable release including pretrained models can be obtained here: https://github.com/raccoonML/Multi-Language-RTVC/releases/tag/v1.0

Hi devs. I'm not a programmer, but I suggest making the windows app accessible to screen readers and assistive technologies. The other thing I suggest is making the app able to export the models as a sapi5 voice so assistive techs can use them. The most crucial requirement is responsiveness and realtime output, which means that the voice shouldn't lag before the speech or in the middle of inference. Please impliment these suggestions. What does this app use for guis? Wxpython is the most accessible one in my opinion, as it uses native controles which are accessible to assistive technologies. Thanks and I hope you impliment them.

As many of you know, hifi-gan is a recent neural vocoder that is very popular for its speed and quality. I will replace the current WaveRNN vocoder with hifi-gan.

Making this announcement to avoid duplication of effort.

#11 @raccoonML @Dannypeja @Aatularyan @AsterTheWanderer @Fzr2k @CrazyPlaysHD

I just pushed some changes to the new branch ml_support (https://github.com/sveneschlbeck/Multi-Language-RTVC/tree/ml_support).

I updated the following files:

• mlrtvc/src/core/synthesizer/utils/symbols.py
• mlrtvc/src/core/synthesizer/utils/cleaners.py
• mlrtvc/src/core/synthesizer/utils/numbers.py

The changes are quite complicated as they reveal further problems:

1. The language_code parameter now appears very often. I haven't yet thought about what needs to be done with it or where it can get called from, I just implemented it in several functions as an argument.
2. While adding abbreviations for other languages was quite easy, the number thing is quite heavy. Differences between decimal points (USA) and decimal commas (EU & more) make it quite harsh to deal with.
3. All methods imported from inflect are designed for the English language (plurals, declinations, etc.). I don't know how to solve this for other languages. This is still a big flaw in the code.

The checklist for the next steps involves:

• [ ] Adding abbreviations (from native speakers if possible) for English, German, Spanish & French in the module mlrtvc/src/core/synthesizer/utils/cleaners.py
• [ ] Checking symbols/characters/letters (from native speakers if possible) for English, German, Spanish & French in the module mlrtvc/src/core/synthesizer/utils/symbols.py
• [ ] Solutions/substitutes for inflect
• [ ] Embedding the changed files into the context of all other files (especially thinking about language_code parameter that appears in different functions but isn't yet defined/called/imported/read out)

The requirements.txt in the old repo was inaccurate and not very well-written. We need to fill the environment.yml in this repository with the essential packages needed, specifying versions of packages only if it really needs to be that very version.

i added a Dockerfile for setting up this repo. Some guides recommend venv, some virtualenv some conda, and all of them mostly don't work the same way. I created this dockerfile to run everything inside docker enviornment, so just using a few docker build and docker run command should get you this environment working.

I'm not sure about the location for documentation, since the complete repository is empty as of now. I have added how do use the dockerfile under ## Using Docker section in the main README.md.

To bring GUI outside this docker you need to forward X like for arch linux its

xhost + command. This will change depending on the OS, so feel free to add to it if you know about other OSes.