The performance of speaking conversion is good, and the singing conversion is not ideal. If I do singing voice conversion, can you teach me how to use hifigan, hififan also has a pre-model with the same parameters. Do you have any plans to upgrade the singing conversion next?

Thank you for share this great repository.

I tried some song to convert voice, but there is problem with high pitch.

The pretrained vocoder can not express high pitch..

So If you have the pretrained hifi pkl file with proper config, please share..

Thanks.

I noticed that the torchaudio.transforms.MelSpectrogram you used is 16000 sampling rate, but the wav read is 24000 sampling rate. In other words, you use a mel with a sampling rate of 16,000 for audio with a sampling rate of 24,000 and use it as the target.Will this affect？

Greetings!

I've tried to replace your supplied pretrained ParallelWaveGAN checkpoint with a different one I trained (Using the implementation over at https://github.com/kan-bayashi/ParallelWaveGAN ), to go along with a custom StarGANv2-VC checkpoint.

I copied the parameters from the config.yml that you supply with your pretrained checkpoint exactly, and used your checkpoint for finetuning.

However, the resulting vocoder checkpoint cannot use the output of StarGANv2-VC correctly - it produces near-clipping, way too low-frequency-centric output, even when running over the original wave files for testing.

After some investigation (and a lot of headache), it seems that the mel spectrograms produced by StarGANv2-VC use a different log base and are not compatible. So I trained a PWG vocoder with log() instead of the default log10(), but this also did not yield acceptable results.

It seems that the normalization you use for StarGANv2-VC is different, also. (?) Your own vocoder checkpoint was trained with those changes implemented, since it works fine out of the box. But they're not documented.

If possible, could you please share details about what changes you made when training your ParallelWaveGAN checkpoint, so that other vocoder checkpoints may be correctly trained for use with StarGANv2-VC? That would be great.

Great production! For the live implementation of reasoning, is there any detailed code? I have tried it, but unfortunately, I still lack some experience in it. Any help will be appreciated.

I am training the F0 model on 160 mels with the pre-trained f0 model that you provided as a base model. training data is Librispeech-clean-100. is the model overfitting only in 10-20 epochs?

When I fine-tuned the model, I had 20 speakers and the model was epoch_00300.pth, now I want to add 1 person, how should I set up? I changed the pretrained_model in config.yml, and then num_domains=21? can you tell me how to do it,thanks

Hi, I am trying to training starganv2 vc , one question about dataset for training ASR and vc, can I use different dataset for ASR training and starggan ?

hi, I want to train a new model using mixed open source data. I would like to know, How much loss model will converge And the required gpu resources and time-consuming

{'max_lr': 0.0001, 'pct_start': 0.0, 'epochs': 150, 'steps_per_epoch': 1346} {'max_lr': 2e-06, 'pct_start': 0.0, 'epochs': 150, 'steps_per_epoch': 1346} {'max_lr': 0.0001, 'pct_start': 0.0, 'epochs': 150, 'steps_per_epoch': 1346} {'max_lr': 0.0001, 'pct_start': 0.0, 'epochs': 150, 'steps_per_epoch': 1346} {'max_lr': 0.0001, 'pct_start': 0.0, 'epochs': 150, 'steps_per_epoch': 1346} {'max_lr': 0.0001, 'pct_start': 0.0, 'epochs': 150, 'steps_per_epoch': 1346} [train]: 100%|█████████████████████████████████████████████████████████| 1346/1346 [15:04<00:00, 1.49it/s] [eval]: 99%|█████████████████████████████████████████████████████████████▎| 92/93 [00:31<00:00, 2.88it/s] Traceback (most recent call last): File "train.py", line 156, in main() File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1130, in call return self.main(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1055, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1404, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 760, in invoke return __callback(*args, **kwargs) File "train.py", line 126, in main eval_results = trainer._eval_epoch() File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "/mnt/e/ai/src/StarGANv2-VC/trainer.py", line 255, in _eval_epoch g_loss, g_losses_latent = compute_g_loss( File "/mnt/e/ai/src/StarGANv2-VC/losses.py", line 106, in compute_g_loss loss_f0 = f0_loss(F0_fake, F0_real) File "/mnt/e/ai/src/StarGANv2-VC/losses.py", line 212, in f0_loss x_mean = compute_mean_f0(x_f0) File "/mnt/e/ai/src/StarGANv2-VC/losses.py", line 203, in compute_mean_f0 f0_mean = f0_mean.expand(f0.shape[-1], f0_mean.shape[0]).transpose(0, 1) # (B, M) IndexError: tuple index out of range

Hi，thanks for this project.It is amazing.When I was thinking about converting between male and female voices, I didn't know how to tune the pitch. Now I'm changing the male voice to a female voice, and the pitch is too high. Can you tell me where to adjust it? Thank you very much.

@yl4579 Thank you so much for this wonderful work

Support needed for Teacher Forcing HiFiGAN Vocoder Finetuning. Couldn't training converted Mel spectrogram as input to HiFiGAN fine-tuning.

Thanks

    Using multiple discriminators is effective, and when the model converges, the sound quality on the unseen speaker is better, and the similarity to the target speaker is better than the original one.

Originally posted by @980202006 in https://github.com/yl4579/StarGANv2-VC/issues/6#issuecomment-945602695

Hello! It's great to have such an interesting and great project open source, I've read some articles and would like to try to train a model myself!

This is probably a pretty simple problem to solve, but I'm not very familiar with python and this is my first attempt at it, so I'm hoping for some help!

I followed the readme on Colab and everything seems to be working fine, but I'm stuck here

python train.py --config_path ./Configs/config.yml

To reduce the probability of errors, I currently chose to reproduce your VCTK model (step 4 in Pre-requisites)

After running '!python train.py --config_path . /Configs/config.yml'

returned the following error

[libprotobuf FATAL google/protobuf/stubs/common.cc:87] This program was compiled against version 3.9.2 of the Protocol Buffer runtime library, which is not compatible with the installed version (3.17.3).  Contact the program author for an update.  If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library.  (Version verification failed in "bazel-out/k8-opt/bin/tensorflow/core/framework/tensor_shape.pb.cc".)
terminate called after throwing an instance of 'google::protobuf::FatalException'
  what():  This program was compiled against version 3.9.2 of the Protocol Buffer runtime library, which is not compatible with the installed version (3.17.3).  Contact the program author for an update.  If you compiled the program yourself, make sure that your headers are from the same version of Protocol Buffers as your link-time library.  (Version verification failed in "bazel-out/k8-opt/bin/tensorflow/core/framework/tensor_shape.pb.cc".)

How can I solve this problem? Thank you!

Hello and thank you sharing your great work, but I have some questions.

1. For song vc with Madarian, I tried train a new starganv2vc model with pretrained ASR and F0 model, but the result sound not well, do you have some advice ?
2. In song vc with Madarian, do i need to retrain a ASR or F0 model ? I'm looking forward for your reply, and thank you again.

Posting this as a new issue in case anybody else has the problem.

There might be hacky workarounds for this but it seems to be a pytorch issue: https://github.com/pytorch/pytorch/issues/40497

So I guess the answer is to train with fp16_run: false for now. Might want to note this in the README to save people time. Also the config file default is set to true, could probably set that to false.