PyTorch tutorials.

Last update: Jan 2, 2023

Related tags

PyTorch Learning Resources tutorials

Overview

PyTorch Tutorials

All the tutorials are now presented as sphinx style documentation at:

https://pytorch.org/tutorials

Contributing

We use sphinx-gallery's notebook styled examples to create the tutorials. Syntax is very simple. In essence, you write a slightly well formatted python file and it shows up as documentation page.

Here's how to create a new tutorial or recipe:

Create a notebook styled python file. If you want it executed while inserted into documentation, save the file with suffix tutorial so that file name is your_tutorial.py.
Put it in one of the beginner_source, intermediate_source, advanced_source based on the level. If it is a recipe, add to recipes_source.
For Tutorials (except if it is a prototype feature), include it in the TOC tree at index.rst
For Tutorials (except if it is a prototype feature), create a thumbnail in the index.rst file using a command like .. customcarditem:: beginner/your_tutorial.html. For Recipes, create a thumbnail in the recipes_index.rst

In case you prefer to write your tutorial in jupyter, you can use this script to convert the notebook to python file. After conversion and addition to the project, please make sure the sections headings etc are in logical order.

Building

Start with installing torch, torchvision, and your GPUs latest drivers. Install other requirements using pip install -r requirements.txt

If you want to use virtualenv, make your environment in a venv directory like: virtualenv ./venv, then source ./venv/bin/activate.

Then you can build using make docs. This will download the data, execute the tutorials and build the documentation to docs/ directory. This will take about 60-120 min for systems with GPUs. If you do not have a GPU installed on your system, then see next step.
You can skip the computationally intensive graph generation by running make html-noplot to build basic html documentation to _build/html. This way, you can quickly preview your tutorial.

If you get ModuleNotFoundError: No module named 'pytorch_sphinx_theme' make: *** [html-noplot] Error 2 from /tutorials/src/pytorch-sphinx-theme or /venv/src/pytorch-sphinx-theme (while using virtualenv), run python setup.py install.

About contributing to PyTorch Documentation and Tutorials

You can find information about contributing to PyTorch documentation in the PyTorch Repo README.md file.
Additional information can be found in PyTorch CONTRIBUTING.md.

Comments

[HELP WANTED]Tutorial for nn.Transformer with nn.TransformerDecoder

Hi, The existing tutorials for nn.Transformer e.g. https://pytorch.org/tutorials/beginner/transformer_tutorial.html and https://github.com/pytorch/examples/tree/master/word_language_model both are only use nn.TransformerEncoder. There is no tutorial using nn.TransformerDecoder. I think it is better to add one example that include nn.TransformerDecoder so that users can easy get started with. As for my own experience, I have some troubles using nn.TransfomerDecoder in inference process.

opened by xdwang0726 104
[WIP] Improve training of DQN tutorial

Following up the discussion from https://github.com/pytorch/tutorials/pull/2026

I still need to do multiple runs to get a semblance of the statistics of # episodes vs duration for both the original and my changes. The slight increase in model capacity still only uses ~1.5 GB of VRAM, so it should be pretty accessible and training is still relatively quick.

Here's the reward history for one run of these tweaks when I was doing a bunch of trial and error (spent an embarrassing amount of time tweaking hyperparameters and rewards).

@vmoens Feel free to change (or completely discard) anything based on your findings. I haven't tried tweaking anything else in the training pipeline.
cla signed rl

opened by SiftingSands 58
Neural-Style tutorial

PyTorch tutorial explaining how to implement the neural transfer of artistic style developed by Leon Gatys and AL. The code parts of the tutorial can be tested in a whole script available here: https://github.com/alexis-jacq/Pytorch-Tutorials/blob/master/Neural_Style.py.

opened by alexis-jacq 16
Add nvFuser tutorial for PyTorch 1.12.0

Adds an nvFuser tutorial for the PyTorch 1.12.0 release. A separate blog post will be published with this tutorial and we'll add cross references. If preferred, I can remove the placeholder in this PR and follow up with the right link once the blog post is published.

CC @csarofeen @kevinstephano @mruberry
cla signed

opened by ptrblck 15
Language Translation with Transformers

This tutorial show how train a language translation model from scratch using Transformers. The data processing code is taken from RNN translation tutorial. Resolves #719
cla signed

opened by iamsimha 15
Add new speech command recognition tutorial
Add command recognition tutorial, see colab.

[x] This tutorial runs in about ~~1h~~ ~~10 min using this~~ 5 min.

[x] Tutorial is faster on GPU, comment.

[x] ~~SpeechCommands as it is currently in torchaudio doesn't have the train/valid/test split. We could decide to add that in before the release to simplify the tutorial. pytorch/audio#966.~~

[x] ~~Experiment with MFCC instead of resample only.~~

[x] ~~Replace SPEECHCOMMANDS by YESNO dataset, and use torchaudio.transforms.vad to segment audio into utterances.~~

This pull request complements #572, see also deprecated tutorial.

cc @brianjo
cla signed
opened by vincentqb 15
Add a new tutorial: Building a Flask API server with PyTorch

I was writing this for my blog, however, I noticed that PyTorch Tutorials does not have one on building an API. So I am submitting here. My apologies, since I did not discuss this here first. I think PyTorch Tutorial could really use this 😄

opened by avinassh 15
Adding model parallel tutorial

I feel model parallel itself already has enough meat to make a separate tutorial page. The DistributedDataParallel wrapping ModelParallel tutorial will come in my next PR. Let me know if I should merge them into one instead.

opened by mrshenli 15
Update audio preprocessing tutorial
Updating the audio tutorial for the Jan 7 release

[x] support for batching in transforms. (pytorch/audio#337, pytorch/audio#327)

[x] torchaudio.functional.gain and torchaudio.functional.dither. (pytorch/audio#319, pytorch/audio#360)

[x] data augmentations in torchaudio.augmentation: TimeStretch, FrequencyMasking, TimeMasking. (pytorch/audio#285, pytorch/audio#333, pytorch/audio#348)

Going to add example in next iteration of tutorial

[x] compute deltas: torchaudio.functional.compute_deltas. (pytorch/audio#268, pytorch/audio#326)

[x] torchaudio.functional.equalizer_biquad. (pytorch/audio#315, pytorch/audio#340)

[x] torchaudio.functional.lowpass_biquad and torchaudio.functional.highpass_biquad. (pytorch/audio#275)

[x] torchaudio.kaldi.mfcc (https://pytorch.org/audio/functional.html#mfcc ). (pytorch/audio#228)

In a following update of the tutorial, we will want:

[x] We introduce a backend switch mechanism to replace sox by something else. We currently support soundfile along with sox. The backends are loaded lazily when needed. (pytorch/audio#355)

[x] We now have a unified datasets interface with lazy loading of files to memory, new download and extract functions, and new datasets: LibriSpeech and CommonVoice. (pytorch/audio#303, pytorch/audio#316, pytorch/audio#330)

Lower priority items:

[x] We make jit compilation optional for function and use nn.Module where possible. (pytorch/audio#314, pytorch/audio#326, pytorch/audio#342)
opened by CamiWilliams 13
Web page for Tutorials
Hi,

I've been working on beautifying/integrating all the tutorials on pytorch into one. see https://github.com/pytorch/pytorch/pull/778. These tutorials are based on sphinx-gallery and tutorials are executed during build time.

I've created a separate repo for the tutorials and used gh-pages to host them: http://chsasank.github.io/pytorch-tutorials. I also added my own transfer learning tutorial

After a discussion with @soumith, he suggested we should host these tutorials at tutorials.pytorch.org. He also requested a change:

[x] Categorize tutorials by level instead of source

If indeed this is to be front face of tutorials, we'll need to figure out

[x] how to modify the repo

[x] how to build the sources and host

For hosting, we shouldn't probably use github pages as it will mess up the git history with all the html files.

Since these tutorials are executed at build, we might need a decently powered build environment. Most tutorials take may be 5 min on my macbook air. Except seq2seq tutorial, which took 40 min on CPU/25 min on GPU. Note that a tutorial is re-excuted only if changes are made to the tutorial file.

Thanks, Sasank.
opened by chsasank 13
Reinstate torchaudio tutorial

Let's see if the tutorial loads correctly with current binaries. pytorch/audio#307

This partly reverts commit 9cee82507629c5f557dcef0ed508dc973935ad92 from #685.

CC @SethHWeidman @ezyang

opened by vincentqb 12
Use Main branch in the docs instead of old Master branch

While referring to the CONTRIBUTING.md as a new contributor, I noticed that the guidelines still reference outdated Master branch and its docs. This PR updates Contributing guidelines and other docs to use the latest Main branch of the Tutorial repo instead of the Master branch.
cla signed

opened by spzala 5
Quantize weights to unisgned 8 bit

I am trying to quantize the weights of the BERT model to unsigned 8 bits. Am using the 'dynamic_quantize' function for the same.

quantized_model = torch.quantization.quantize_dynamic( model, {torch.nn.Linear}, dtype=torch.quint8 )

But it throws the error 'AssertionError: The only supported dtypes for dynamic quantized linear are qint8 and float16 got: torch.quint8'.

Is there any specific reason for this not to be supported? Could I use any other method to quantize the weights to unsigned int8 bits?

Here is a link to the colab sheet: https://colab.research.google.com/drive/14G_jdLuZD5846jUDZUdGNf1x0DMz8GKK?usp=sharing

Thanks!

cc @jerryzh168 @z-a-f @vkuzo
question arch-optimization

opened by rohanjuneja 0
missing parameter in WikiText2

train_iter = WikiText2(split='train')

TypeError: init() missing 2 required positional arguments: 'path' and 'text_field'

cc @pytorch/team-text-core @Nayef211
module: torchtext

opened by joepareti54 1

Owner

GitHub https://pytorch.org/tutorials/

Minimal tutorials for PyTorch

Minimal tutorials for PyTorch adapted from Alec Radford's Theano tutorials. Tensor multiplication Linear Regression Logistic Regression Neural Network

321 Oct 25, 2022

Simple PyTorch Tutorials Zero to ALL!

PyTorchZeroToAll Quick 3~4 day lecture materials for HKUST students. Video Lectures: (RNN TBA) Youtube Bilibili Slides Lecture Slides @GoogleDrive If

3.7k Dec 30, 2022

PyTorch tutorials and best practices.

Effective PyTorch Table of Contents Part I: PyTorch Fundamentals PyTorch basics Encapsulate your model with Modules Broadcasting the good and the ugly

1.5k Jan 4, 2023

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.

PyTorch Examples WARNING: if you fork this repo, github actions will run daily on it. To disable this, go to /examples/settings/actions and Disable Ac

19.4k Jan 1, 2023

Deep Learning (with PyTorch)

Deep Learning (with PyTorch) This notebook repository now has a companion website, where all the course material can be found in video and textual for

6.2k Jan 2, 2023

Open source guides/codes for mastering deep learning to deploying deep learning in production in PyTorch, Python, C++ and more.

Deep Learning Materials by Deep Learning Wizard Start Learning Now Please head to www.deeplearningwizard.com to start learning! It is mobile/tablet fr

572 Dec 28, 2022

Simple examples to introduce PyTorch

This repository introduces the fundamental concepts of PyTorch through self-contained examples. At its core, PyTorch provides two main features: An n-

4.4k Jan 7, 2023

PyTorch Tutorial for Deep Learning Researchers

This repository provides tutorial code for deep learning researchers to learn PyTorch. In the tutorial, most of the models were implemented with less

25.4k Jan 5, 2023

PyTorch Implementation of Fully Convolutional Networks. (Training code to reproduce the original result is available.)

pytorch-fcn PyTorch implementation of Fully Convolutional Networks. Requirements pytorch >= 0.2.0 torchvision >= 0.1.8 fcn >= 6.1.5 Pillow scipy tqdm

1.6k Jan 4, 2023

Pytorch implementations of various Deep NLP models in cs-224n(Stanford Univ)

DeepNLP-models-Pytorch Pytorch implementations of various Deep NLP models in cs-224n(Stanford Univ: NLP with Deep Learning) This is not for Pytorch be

2.9k Dec 24, 2022

A scalable template for PyTorch projects, with examples in Image Segmentation, Object classification, GANs and Reinforcement Learning.

PyTorch Project Template is being sponsored by the following tool; please help to support us by taking a look and signing up to a free trial PyTorch P