Pytorch implementation of Tacotron

Related tags

Text Data & NLP text-to-speech pytorch tts tacotron
Overview

Tacotron-pytorch

A pytorch implementation of Tacotron: A Fully End-to-End Text-To-Speech Synthesis Model.

Requirements

  • Install python 3
  • Install pytorch == 0.2.0
  • Install requirements: 
    pip install -r requirements.txt

Data

I used LJSpeech dataset which consists of pairs of text script and wav files. The complete dataset (13,100 pairs) can be downloaded here. I referred https://github.com/keithito/tacotron for the preprocessing code.

File description

  • hyperparams.py includes all hyper parameters that are needed.
  • data.py loads training data and preprocess text to index and wav files to spectrogram. Preprocessing codes for text is in text/ directory.
  • module.py contains all methods, including CBHG, highway, prenet, and so on.
  • network.py contains networks including encoder, decoder and post-processing network.
  • train.py is for training.
  • synthesis.py is for generating TTS sample.

Training the network

  • STEP 1. Download and extract LJSpeech data at any directory you want.
  • STEP 2. Adjust hyperparameters in hyperparams.py, especially 'data_path' which is a directory that you extract files, and the others if necessary.
  • STEP 3. Run train.py.

Generate TTS wav file

  • STEP 1. Run synthesis.py. Make sure the restore step.

Samples

  • You can check the generated samples in 'samples/' directory. Training step was only 60K, so the performance is not good yet.

Reference

Comments

  • Any comments for the codes are always welcome.
Comments
  • No module named 'Tacotron'

    No module named 'Tacotron'

    Hi I'm running into the below error - has anyone else encountered this and know a fix?

    File ".../tacotron/text/init.py", line 4, in from Tacotron.text import cleaners ModuleNotFoundError: No module named 'Tacotron'

    Many thanks in advance.

    opened by taronling 1
  • Fix path bugs

    Fix path bugs

    Thanks for the great tacotron implementation! Howerver, 'Tacotron.text' is a little path bug, it should be 'text'. Also, to be more compatible with new pytorch, we should use loss.item(). Finally, I change the log format to be more readable.

    opened by ttaoREtw 0
  • ValueError: operands could not be broadcast together with shapes (1,1024) (0,)

    ValueError: operands could not be broadcast together with shapes (1,1024) (0,)

    Hello!When I run the code, I meet with this error (0,)ValueError: operands could not be broadcast together with shapes (1,1024) (0,). Could you help me solve it?

    opened by zhexian949 0
  • Plot the spectrogram

    Plot the spectrogram

    How would you plot the spectrogram? I tried to create an image from inside of inv_spectrogram(spectrogram) definition after the # Convert back to linear line but still it is black image generated of (1024, 1000) dimension. I am looking for such methodology because I want to convert an image [of spectrogram] back into audio wav file. Can you please help for that?

    opened by RahulBhalley 0
  • memory problem

    memory problem

    Such a wonderful work, I learned a lot from it. thanks @soobinseo I have encountered a problem when running the program. My GPU has 12G memory, but when the batch size larger than one, out of memory error will occur. I tried to use inplace=True for all relu activation function, but it seems useless. all the remaining hyperparameter stay the same with the origin provided. Any suggestions for this problem, Thanks!

    opened by keithyin 1
  • Would you please provide some speech wave file results?

    Would you please provide some speech wave file results?

    Hi soobinseo, I am new to TTS. I found that there were two versions of tacotron implemented by tensorflow and pytorch. I think you have already tested your pytorch version. Would you please share some speech wav file results synthesised by this project?

    opened by youngsuenXMLY 0
Owner
soobin seo
Department of Industrial Engineering, Seoul National University
soobin seo
GitHub
Multispeaker & Emotional TTS based on Tacotron 2 and Waveglow

This Repository contains a sample code for Tacotron 2, WaveGlow with multi-speaker, emotion embeddings together with a script for data preprocessing.

Ivan Didur 106 Jan 1, 2023
German Text-To-Speech Engine using Tacotron and Griffin-Lim

jotts JoTTS is a German text-to-speech engine using tacotron and griffin-lim. The synthesizer model has been trained on my voice using Tacotron1. Due

padmalcom 6 Aug 28, 2022
pytorch-kaldi is a project for developing state-of-the-art DNN/RNN hybrid speech recognition systems. The DNN part is managed by pytorch, while feature extraction, label computation, and decoding are performed with the kaldi toolkit.

The PyTorch-Kaldi Speech Recognition Toolkit PyTorch-Kaldi is an open-source repository for developing state-of-the-art DNN/HMM speech recognition sys

Mirco Ravanelli 2.3k Dec 27, 2022
Pytorch-version BERT-flow: One can apply BERT-flow to any PLM within Pytorch framework.

Pytorch-version BERT-flow: One can apply BERT-flow to any PLM within Pytorch framework.

Ubiquitous Knowledge Processing Lab 59 Dec 1, 2022
SAINT PyTorch implementation

SAINT-pytorch A Simple pyTorch implementation of "Towards an Appropriate Query, Key, and Value Computation for Knowledge Tracing" based on https://arx

Arshad Shaikh 63 Dec 25, 2022
Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorch

COCO LM Pretraining (wip) Implementation of COCO-LM, Correcting and Contrasting Text Sequences for Language Model Pretraining, in Pytorch. They were a

Phil Wang 44 Jul 28, 2022
Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

Phil Wang 5k Jan 2, 2023
A fast and easy implementation of Transformer with PyTorch.

FasySeq FasySeq is a shorthand as a Fast and easy sequential modeling toolkit. It aims to provide a seq2seq model to researchers and developers, which

宁羽 7 Jul 18, 2022
A PyTorch Implementation of End-to-End Models for Speech-to-Text

speech Speech is an open-source package to build end-to-end models for automatic speech recognition. Sequence-to-sequence models with attention, Conne

Awni Hannun 647 Dec 25, 2022
Google AI 2018 BERT pytorch implementation

BERT-pytorch Pytorch implementation of Google AI's 2018 BERT, with simple annotation BERT 2018 BERT: Pre-training of Deep Bidirectional Transformers f

Junseong Kim 5.3k Jan 7, 2023
Unofficial PyTorch implementation of Google AI's VoiceFilter system

VoiceFilter Note from Seung-won (2020.10.25) Hi everyone! It's Seung-won from MINDs Lab, Inc. It's been a long time since I've released this open-sour

MINDs Lab 881 Jan 3, 2023
Implementation of ProteinBERT in Pytorch

ProteinBERT - Pytorch (wip) Implementation of ProteinBERT in Pytorch. Original Repository Install $ pip install protein-bert-pytorch Usage import torc

Phil Wang 92 Dec 25, 2022
A PyTorch implementation of paper "Learning Shared Semantic Space for Speech-to-Text Translation", ACL (Findings) 2021

Chimera: Learning Shared Semantic Space for Speech-to-Text Translation This is a Pytorch implementation for the "Chimera" paper Learning Shared Semant

Chi Han 43 Dec 28, 2022
PyTorch Implementation of Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation

StyleSpeech - PyTorch Implementation PyTorch Implementation of Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation. Status (2021.06.09

Keon Lee 142 Jan 6, 2023
PyTorch implementation and pretrained models for XCiT models. See XCiT: Cross-Covariance Image Transformer

Cross-Covariance Image Transformer (XCiT) PyTorch implementation and pretrained models for XCiT models. See XCiT: Cross-Covariance Image Transformer L

Facebook Research 605 Jan 2, 2023
A pytorch implementation of the ACL2019 paper "Simple and Effective Text Matching with Richer Alignment Features".

RE2 This is a pytorch implementation of the ACL 2019 paper "Simple and Effective Text Matching with Richer Alignment Features". The original Tensorflo

null 286 Jan 2, 2023
PyTorch Implementation of VAENAR-TTS: Variational Auto-Encoder based Non-AutoRegressive Text-to-Speech Synthesis.

VAENAR-TTS - PyTorch Implementation PyTorch Implementation of VAENAR-TTS: Variational Auto-Encoder based Non-AutoRegressive Text-to-Speech Synthesis.

Keon Lee 67 Nov 14, 2022
A Pytorch implementation of "Splitter: Learning Node Representations that Capture Multiple Social Contexts" (WWW 2019).

Splitter ⠀⠀ A PyTorch implementation of Splitter: Learning Node Representations that Capture Multiple Social Contexts (WWW 2019). Abstract Recent inte

Benedek Rozemberczki 201 Nov 9, 2022
Simple Text-Generator with OpenAI gpt-2 Pytorch Implementation

GPT2-Pytorch with Text-Generator Better Language Models and Their Implications Our model, called GPT-2 (a successor to GPT), was trained simply to pre

Tae-Hwan Jung 775 Jan 8, 2023