HI@keonlee9420, Thank you for your suggestions these days, I successfully integrated model PortaSpeech on the basis of this model. These are some questions to ask you! Thank you!

1. In the DiffGAN-TTS, the return of get_mask from length is mask. And the return of get_mask from length in PortaSpeech is ~mask. I want to know the difference between them,
2. In DiffGAN-TTS, about def diffuse_trace(self, x_start, mask). I want to know how do the ~ aims to do in def diffuse_trace. In my integrated model, I set the return of get_mask from length is ~mask. If I delete the ~ in diffuse_trace, the synthesis mel is error and the voice likes to the voice of water. While If I preserve the ~ in diffuse_trace, the mel is also error and the voice likes to electric voice. Thank you very much!

• Deng Yan
• 2022.5.9
• GuangXi University

File "train.py", line 320, in 3.24s/it] main(args, configs) File "train.py", line 196, in main figs, wav_reconstruction, wav_prediction, tag = synth_one_sample( File "/data/workspace/liukaiyang/TTS/DiffGAN-TTS-main/utils/tools.py", line 227, in synth_one_sample mels = [mel_pred[0, :mel_len].float().detach().transpose(0, 1) for mel_pred in diffusion.sampling()] File "/root/anaconda3/envs/LKYBase/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context return func(*args, **kwargs) File "/data/workspace/liukaiyang/TTS/DiffGAN-TTS-main/model/diffusion.py", line 157, in sampling b, *_, device = *self.cond.shape, self.cond.device File "/root/anaconda3/envs/LKYBase/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1177, in getattr raise AttributeError("'{}' object has no attribute '{}'".format( AttributeError: 'GaussianDiffusion' object has no attribute 'cond'

Thank for your work!I seem to get a Error……

raceback (most recent call last): | 0/5468 [00:00<?, ?it/s] File "train.py", line 307, in main(args, configs) File "train.py", line 99, in main output = model(*(batch[2:])) TypeError: 'NoneType' object is not subscriptable

How can I solve this problem? Thank You!

Hello, thank you very much for your brilliant open-source project. I have been able to do single and batch generations using the LJSpeech dataset. However, when I try to replicate the results for the VCTK dataset, it fails.

I run the following command, !python3 synthesize.py --text "Hello World" --model naive --restore_step 300000 --mode single --dataset VCTK

I obtain the following output:

[nltk_data] Downloading package averaged_perceptron_tagger to
[nltk_data]     /root/nltk_data...
[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.
[nltk_data] Downloading package cmudict to /root/nltk_data...
[nltk_data]   Unzipping corpora/cmudict.zip.

==================================== Inference Configuration ====================================
 ---> Type of Modeling: naive
 ---> Total Batch Size: 32
 ---> Path of ckpt: ./output/ckpt/VCTK_naive
 ---> Path of log: ./output/log/VCTK_naive
 ---> Path of result: ./output/result/VCTK_naive
================================================================================================
Removing weight norm...
Traceback (most recent call last):
  File "synthesize.py", line 264, in <module>
    )) if load_spker_embed else None
  File "/usr/local/lib/python3.7/dist-packages/numpy/lib/npyio.py", line 416, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
FileNotFoundError: [Errno 2] No such file or directory: './preprocessed_data/VCTK/spker_embed/p225-spker_embed.npy' 

I tried to investigate further and discovered that the specific speaker embedding folder and file did not exist in my directory. Any pointer to how I can solve the issue will be appreciated.

HI, I wanna run "python3 preprocess.py --dataset VCTK" after "python3 prepare_align.py --dataset VCTK", but in ./preprocessor/preprocessor.py
line :115 tg_path = os.path.join(self.out_dir, "TextGrid", speaker, "{}.TextGrid".format(basename) I cannot find file named "*TextGrid", I want to know when it created?

After step "python3 prepare_align.py --dataset VCTK" I only get files name ".lab" and ".wav", no files named ".TextGrid"

Thanks

self.mlp = nn.Sequential( LinearNorm(residual_channels, residual_channels * 4), Mish(), # return x * torch.tanh(F.softplus(x)) LinearNorm(residual_channels * 4, residual_channels) )

class Mish(nn.Module): def forward(self, x): return x * torch.tanh(F.softplus(x))

Hello, thank you very much for the open source project. I ran into a problem: the model successfully converged during training, but after generating the mel spectrum (which looked very good), I put the mel spectrum into my own hifigan vocoder, and the resulting wav was murmur, I could be sure that the parameters of the hifigan's sample radio, hoplength and winlength were consistent with the diffgan model, and I guessed that the problem was in the process of processing the audio of the data into a mel spectrum. I noticed that you used pytorch-stft to implement it, which is very different from the processing result of librosa.stft?

Hi@keonlee9420, I encountered some problems during the training stage. I often have loss functions that occasionally fluctuate a lot during training, even from around 3 to tens or hundreds. After I set the training set shuffle, sometimes I have this problem, sometimes but not this problem. This problem was encountered in the naive, aux and shallow stages. Thank you for my friend！Best wish to you!

Hi, keonlee. Thanks for sharing code! I found that when training aux model, we get \hat{x_0} from G, then diffuse it to \hat{x_1}, finally get a prediciton list [ \hat{x_0}, \hat{x_1}]. When calculating mel loss, add l1 loss of them with target. It confuse me. I understand l1(x_0, \hat{x_0}). But why not l1(x_1, \hat{x_1}).

I realise that when I remove adversarial loss and feature match loss, it still works well and has no degeneration of performance. This makes me question the role of adversarial training in reduction of inference steps, or this this task is simple enough to learn directly with denoise model. Here are samples from two models https://drive.google.com/drive/folders/1uvURiQkOrP9n1jJsKyNe9NcSO4AfdFID?usp=sharing

HI@keonlee9420, I have some questions to ask you about the mel-spectrogram. In the picture, The above mel-spectrogram alignment has been generated, but the horizontal details have not been released yet. What problem do you think caused it

Hi Keon, thanks so much for sharing this wonderful project. I am wondering can we just use the FastSpeech part for inference? Looking forward to your reply