Hi,

First of all, thank you for this nice implementation.

I trained the network with default settings and data (~500k iteration), but the results are really unnaturalistic (eg.: link) and far from the samples provided by the author of the paper: http://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/maskcyclegan-vc/index.html

Why is this? Did you experience the same or you got nice results?

Hi!

Thanks for sharing code.

Why did you use vocoder() funtion to extract features? These features are mel-spectrograms, right? Why didn't use just librosa? I tried and differences between extracted featurs are not negligible. Ranges are so different.

Thank you for your code! I found a confusion: According to the original paper, the mask is used in the source mel-spectrongram instead of target mel-spectrongram, while both Mel-spectrongrams use mask in your scripts. Why does target side also uses mask? Looking forward to your reply!

I was wondering that if I could use 2 sets of non-parallel voice datasets, where each speaker utters completely different sentences, and there is no transcript of the data.

also, does this program use end-end conversion, or does it train a speech-to-text model and then recreates the speech in the converted voice from the text?

Hi, I was very surprised to find this project. Would you mind telling me what the configuration of your experimental hardware environment is? Such as how many RAM is needed for the graphics card. And, how long it will take after several hundred epochs training to get excellent results? Thank you.

The original article says:

We used L_id only for the first 1e4 iterations to guide the learning direction. 
We kept the same learning rate for the first 2e5 iterations and linearly decay over the next 2e5 iterations.

But in the code decay_after = 1e4 which also controls the identity constraint removal.

I corrected this issue with adding a new argument: stop_identity_after

There are also minor updates with no effects and an experimental update regarding a spectral norm in the discriminator model which is to be discarded.

Hi,

Thanks for the well-documented implementation and the outputs from the model are really good. I know the original paper was built for 1 to 1 conversion but I'm wondering if there's any possibility to control the output by manipulating the z vector input for the generator by using a pre-trained classifier. For example, if I train one source speaker to multiple target speakers(recordings of multiple speakers in one dataset acting as one target) and train a classifier to differentiate each speaker, will it be possible to control the output for the target we want. I'm just wondering whether the structure of this model will allow it? A reply would be highly appreciated since I'm trying to use this implementation for my undergraduate final year project in a couple of months. Thanks!

train.py: error: unrecognized arguments: --lr 5e-4

I've done everything as said in the description but this Error happens. I'm running Ubuntu 20.0.4 2080ti

Hello, What is the optimal duration? Is there any advantage to using 4 hour for a target and 2 hours for a source? or 4 hours for the source? I can definitely hear differences when the duration is longer, e.g 30 minutes vs 2 hours.

Thanks

In the inference script the scaling back with speaker statistics is wrong:

            wav_fake_B = decode_melspectrogram(self.vocoder, fake_B[0].detach(
            ).cpu(), self.dataset_A_mean, self.dataset_A_std).cpu()

This should be scaled with B speaker statistics.

This correction largely improves the performance of the inference.

Solves issue #3 .

If I want to continue the train after it have train more than 1,000 epochs for example 1,200, set the --continue_train and set --load_epoch as 1200 in train.py, it will have a error path. In model_saver.py line 76, it set the path to model to be loaded have only three digits.

HIi @hikaruhotta @KarlHajal @terbed, Thank you so much for your excellent work. very nice paper.

I tried High pitch dialogues for the source to convert Target audio. but it didn't work.

May I know any reason for this? and How can we overcome this issue?

Kindly request, Please give your suggestions.

Thanks

Has this been used for genre style conversion or emotion conversion in music? Also, does this work for arbitrary lengths of audio or will one have to adjust the network for spectrograms that were not from the dataset used in the paper?

Hi everyone so I started testing after the debug @HikaruHotta did .and i reached at gloss 7.5 and dloss 0.29 so i decided to convert an output and see if the result is good . but there is a problem here .my output wav file is from speaker s1 to s2 but its the same evaluation file i have for s1 added some noise .its no VC its just the same file with some noise. can u tell me whats the problem?