https://github.com/NVIDIA/waveglow/issues/54 In this issue, they were talking about lower some parameters to maximize inference speed. But I dont know how to do it properly, what can be reduced and what need to remain. Anyone did this before? Please send me your hparams configuration.

if I trained my model using fp32, can it run inference in fp16 and vice versa? in this case, will it impove inference speed? I am using RTX 2080ti, my model run 7 times faster than real-time, and I am pretty sure it can be improved

and one more thing, is there any benefit of running inference using multi-GPUs?

Training on Russian dataset, output says I have less than 0.2 loss, the default is 500 epoch. Now I'm on 1333 epoch and still get Warning! Reached max decoder steps Should I counting or it screwed up? http://puu.sh/CgXpt/41886048cd.jpg

Hi, My alignment plot has a little gap between words.

batch size - 64 filter_length=2048, hop_length=275, win_length=1100,

And it sounds bit like reading separated words. Is there anyone who had same issue before.

Thanks.

I would like to know if it possible to train a Tacotron 2 model for another language, using another dataset which have the same structure as LJ Speech dataset? And if it is possible, is there any tutorial to do so?

Very cool work, this! However, it would be ideal to also provide examples of input text + output audio from a trained system, alongside held-out examples from the database. This will give an impression of what kind of results the code is capable of generating with the LJSpeech data, and is standard practise in the text-to-speech field.

Aside from synthesising held-out sentences from LJSpeech, Google's speech examples for Tacotron 2 provide another set of challenging text prompts to generate.

Are there any plans to do this? Or are synthesised speech examples already available somewhere?

Hello, I have a question.

I'm using a dataset with > 17k sentences (about 30 hours audio), 90% for training and 10% for validating. It's been training for 3 days (using batch_size 8) and reaching Epoch 56. Plz see training info below [Grad norm] [Training loss] [Validation loss] I thought it looks good. But when I tested it, the output audio was wrong and Attention looks awful.

And the loss seems can not decrease any more. Do I have to train for more Epoch or there was something wrong with my dataset, or something else? Plz help me, thank u guys so much.

Hi, so we have been training both tacotron2 and waveglow models on clean male speech (Hindi language) of about 10+ hours @ 16kHz sampling rate using phonemic text. We keep the window parameters, and mel_band parameters unchanged from the 22.5kHz setup in the original repo. Both the models were trained from scratch in a distributed manner using 8-V100 for over 2 days. The resulting voice synthesis produces a hoarse voice. We tried synthesizing using models from different iterations. The hoarseness remains the same irrespective. From visualizing the spectrogram of the original (left-side of the image) and synthesized audio (right-side of the image) of the same text we observe that the spectrogram of synthesized audio is smudged in the middle frequencies in comparison to crisp harmonics of the original audio.

Any suggestions on how to train better?

Left: Original (Audio) , Right: Synthesized (Audio)

I have trained the model for only few steps (8000) for testing.

At this checkpoint, the synthesized mel spectrogram's frames always bigger than max frames (in hparams config)

As I have checked, the problem is gate_output always less than configured gate_threshold (it's around 0.0022 or 0.0023), so the frames had been generated forever.

Is this problem because I was not trained model much enough, or what should I have to check?

Please help, thank you so much.

I had no such problem before But today used pretrained models And sometimes spectrogram is bugged

here is notebook for reproduce https://colab.research.google.com/drive/1jR12cEKdkg0hlDUHGhf2fPb0RwqPwEYj?#scrollTo=CyBu2F7eisFM

Hello, I am experimenting with 3.5 hours of data set. (16kbit, 22khz, 9.5s av.duration, 1300 audio files) I am using pretrained models which are provided by this repo.

My hparams; p_attention_dropout=0.5, p_decoder_dropout=0.5, learning_rate=4e-5, batch_size=16,

After 15k iteraton: Train loss 0.25. It is producing almost original like outputs.

The only thing I couldn't improve is a validation loss, it stuck since 4000 iteration.

All suggestions are welcome! Thanks beforehand.

Hello,

First of all thanks for the nice Tacotron 2 implementation.

I'm trying to use the trained Tacotron 2 outputs as inputs to r9r9's Wavenet vocoder. However his pre-trained wavenet works on scaled Mel Spectrogram between [0, 1].

What is the range for this tacotron 2 implementation, I'm having a hard time finding this out to use it for scaling.

For reference, this is r9r9's normalization function that he applies to the Mel Spectrogram before using it for training, which scales it between 0 and 1:

def _normalize(S): return np.clip((S - hparams.min_level_db) / -hparams.min_level_db, 0, 1)

• numpy==1.13.3
  • conflicts with tensorflow and matplotlib
• scipy==1.0.0
  • function that are used are missing in this version
• torch
  • isn't even there
• numba==0.48
  • missing functions in new version required by librosa
• resampy==0.3.1
  • I forgot which module requires this but it works

when I run the inference code ,it has an error as followed

waveglow_path = '/home/zhonghuihang/tacotron2-master/waveglow_256channels_universal_v5.pt' waveglow = torch.load(waveglow_path)['model'] waveglow.cuda().eval().half() for k in waveglow.convinv: k.float() denoiser = Denoiser(waveglow)

InvalidArgumentsError Traceback (most recent call last) /tmp/ipykernel_1534266/1318601042.py in 1 waveglow_path = '/home/zhonghuihang/tacotron2-master/waveglow_256channels_universal_v5.pt' ----> 2 waveglow = torch.load(waveglow_path)['model'] 3 waveglow.cuda().eval().half() 4 for k in waveglow.convinv: 5 k.float()

~/miniconda3/envs/torch/lib/python3.7/site-packages/torch/serialization.py in load(f, map_location, pickle_module, weights_only, **pickle_load_args)

~/miniconda3/envs/torch/lib/python3.7/site-packages/torch/serialization.py in _legacy_load(f, map_location, pickle_module, **pickle_load_args)

~/miniconda3/envs/torch/lib/python3.7/site-packages/torch/serialization.py in find_class(self, mod_name, name)

~/miniconda3/envs/torch/lib/python3.7/site-packages/glow/init.py in 28 extend_all(functions) 29 ---> 30 from .wgr import * # For backwards compatibility. Avoid showing this import in docs. 31 32 from . import wgr

~/miniconda3/envs/torch/lib/python3.7/site-packages/glow/wgr/init.py in 13 # limitations under the License. 14 ---> 15 from glow.wgr.ridge_reduction import * 16 from glow.wgr.ridge_regression import * 17 from glow.wgr.logistic_ridge_regression import *

~/miniconda3/envs/torch/lib/python3.7/site-packages/glow/wgr/ridge_reduction.py in 13 # limitations under the License. 14 ---> 15 from .ridge_udfs import * 16 from .model_functions import _is_binary, _prepare_covariates, _prepare_labels_and_warn, _check_model 17 from nptyping import Float, NDArray

~/miniconda3/envs/torch/lib/python3.7/site-packages/glow/wgr/ridge_udfs.py in 13 # limitations under the License. 14 ---> 15 from glow.wgr.model_functions import * 16 from nptyping import Float 17 import pandas as pd

~/miniconda3/envs/torch/lib/python3.7/site-packages/glow/wgr/model_functions.py in 108 # @typechecked -- typeguard does not support numpy array 109 def assemble_block(n_rows: Int, n_cols: Int, pdf: pd.DataFrame, cov_matrix: NDArray[(Any, Any), --> 110 Float], 111 row_mask: NDArray[Any]) -> NDArray[Float]: 112 """

~/miniconda3/envs/torch/lib/python3.7/site-packages/nptyping/base_meta_classes.py in getitem(cls, item) 136 raise NPTypingError(f"Type nptyping.{cls} is already parameterized.") 137 --> 138 args = cls._get_item(item) 139 additional_values = cls._get_additional_values(item) 140 assert hasattr(cls, "args"), "A SubscriptableMeta must have args."

~/miniconda3/envs/torch/lib/python3.7/site-packages/nptyping/ndarray.py in _get_item(cls, item) 67 def _get_item(cls, item: Any) -> Tuple[Any, ...]: 68 cls._check_item(item) ---> 69 shape, dtype = cls._get_from_tuple(item) 70 return shape, dtype 71

~/miniconda3/envs/torch/lib/python3.7/site-packages/nptyping/ndarray.py in _get_from_tuple(cls, item) 108 def _get_from_tuple(cls, item: Tuple[Any, ...]) -> Tuple[Shape, DType]: 109 # Return the Shape Expression and DType from a tuple. --> 110 shape = cls._get_shape(item[0]) 111 dtype = cls._get_dtype(item[1]) 112 return shape, dtype

~/miniconda3/envs/torch/lib/python3.7/site-packages/nptyping/ndarray.py in _get_shape(cls, dtype_candidate) 122 else: 123 raise InvalidArgumentsError( --> 124 f"Unexpected argument '{dtype_candidate}', expecting" 125 " Shape[]" 126 " or Literal[]"

InvalidArgumentsError: Unexpected argument '(typing.Any, typing.Any)', expecting Shape[] or Literal[] or typing.Any.

Since Google colab no longer functions with Tensorflow 1, it has corrupted Tacotron 2 training and synthesis notebooks. Even when I use the Tensorflow 2, it still corrupts Tacotron 2 by not recognizing the child directories. Like: os.chdir('tacotron2') no longer functions. Is there a way to fix?

Bumps tensorflow from 1.15.2 to 2.9.3.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Hello, as the title says I'm getting the AssertionError on line 85 of train.py when I try to run the code with

python J:\tacotron2\train.py --output_directory="J:\tacotron\output" --log_directory="J:\tacotron\log" -c tacotron2_statedict.pt --warm_start

I've installed Anaconda3 with Python 3.9, Tensorflow (GPU) 2.6.0 and all the required libraries (librosa, inflect, unidecode, keras). I've also made the modifications to the hparams.py file in order to make it work with Tensorflow2 as suggested by @v-nhandt21 at https://github.com/NVIDIA/tacotron2/issues/278.

What could be the problem? I have a list of 100 WAV files, 22050 Hz 16-bit.

Hi NVIDIA/tacotron2!

This is a one-off automatically generated pull request from LGTM.com :robot:. You might have heard that we’ve integrated LGTM’s underlying CodeQL analysis engine natively into GitHub. The result is GitHub code scanning!

With LGTM fully integrated into code scanning, we are focused on improving CodeQL within the native GitHub code scanning experience. In order to take advantage of current and future improvements to our analysis capabilities, we suggest you enable code scanning on your repository. Please take a look at our blog post for more information.

This pull request enables code scanning by adding an auto-generated codeql.yml workflow file for GitHub Actions to your repository — take a look! We tested it before opening this pull request, so all should be working :heavy_check_mark:. In fact, you might already have seen some alerts appear on this pull request!

Where needed and if possible, we’ve adjusted the configuration to the needs of your particular repository. But of course, you should feel free to tweak it further! Check this page for detailed documentation.

Questions? Check out the FAQ below!

FAQ