# Machine Learning Collection

In this repository you will find tutorials and projects related to Machine Learning. I try to make the code as clear as possible, and the goal is be to used as a learning resource and a way to lookup problems to solve specific problems. For most I have also done video explanations on YouTube if you want a walkthrough for the code. If you got any questions or suggestions for future videos I prefer if you ask it on YouTube. This repository is contribution friendly, so if you feel you want to add something then I'd happily merge a PR 😃

## PyTorch Tutorials

If you have any specific video suggestion please make a comment on YouTube :)

### Object Detection

Object Detection Playlist

GAN Playlist

## TensorFlow Tutorials

If you have any specific video suggestion please make a comment on YouTube :)

### CNN Architectures

• #### ProGAN Pretrained weights link is broken!

opened by extremety1989 11
• #### ProGan RuntimeError

i downloaded celeba_hq image dataset,modified config.py (DATASET = 'celeba_hq') , modified train.py( at main()
# import sys # sys.exit()) then when i run python train.py i got this error

`return F.conv_transpose2d( RuntimeError: Expected 4-dimensional input for 4-dimensional weight [512, 512, 4, 4], but got 2-dimensional input of size [256, 512] instead`

opened by extremety1989 9
• #### Transformer Question, and Request

Question/Bug?: In SelfAttention you split values, keys, and query by the number of heads. Then pass this into Linear with same input and output dimension. Why not keep the full dimension (ie: not split) and let the Linear do the reduction? This would allow linear to learn what to take out of the input rather?

btw, https://github.com/tunz/transformer-pytorch/blob/master/model/transformer.py, class MultiHeadAttention(nn.Module) does this (if I interpret their code correctly).

The paper https://arxiv.org/pdf/1706.03762.pdf indicates "learned linear projections to dk, dk and dv dimensions".

If I'm all wrong, would love to be corrected as I learning. If I'm right, would also love to know that I'm starting to understand this stuff.

Request: Starting to understand torch.einsum power but I am sure I am missing a bunch. Can you do a video on this?

Regards, John

opened by johngrabner 4
• #### Getting error while executing Sementic segmentation w. UNET in pytorch

Hi, I watched your recent tutorial on sementic segmentation with pytorch. Being new to pytorch I was looking for some tutorial with good explanations especially in segmentation module and your tutorial came as a great help. I tried to implement your way on a UNet network for segmentation on google-colab but getting an error. I tried to fix it but no luck. Can you please help me in fixing the error. The error I am getting is:

TypeError Traceback (most recent call last) in () 85 86 if name == "main": ---> 87 main()

7 frames in main() 67 68 for epoch in range(Num_epochs): ---> 69 train_fn(train_loader, model, optimizer, loss_fn, scaler) 70 71

in train_fn(loader, model, optimizer, loss_fn, scaler) 2 loop = tqdm(loader) 3 ----> 4 for batch_idx, (data, targets) in enumerate(loop): 5 data= data.to(device=device) 6 targets= targets.float().unsqueeze(1).to(device=device)

/usr/local/lib/python3.6/dist-packages/tqdm/std.py in iter(self) 1102 fp_write=getattr(self.fp, 'write', sys.stderr.write)) 1103 -> 1104 for obj in iterable: 1105 yield obj 1106 # Update and possibly print the progressbar.

/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py in next(self) 433 if self._sampler_iter is None: 434 self._reset() --> 435 data = self._next_data() 436 self._num_yielded += 1 437 if self._dataset_kind == _DatasetKind.Iterable and \

/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py in _next_data(self) 473 def _next_data(self): 474 index = self._next_index() # may raise StopIteration --> 475 data = self._dataset_fetcher.fetch(index) # may raise StopIteration 476 if self._pin_memory: 477 data = _utils.pin_memory.pin_memory(data)

/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py in fetch(self, possibly_batched_index) 42 def fetch(self, possibly_batched_index): 43 if self.auto_collation: ---> 44 data = [self.dataset[idx] for idx in possibly_batched_index] 45 else: 46 data = self.dataset[possibly_batched_index]

/usr/local/lib/python3.6/dist-packages/torch/utils/data/_utils/fetch.py in (.0) 42 def fetch(self, possibly_batched_index): 43 if self.auto_collation: ---> 44 data = [self.dataset[idx] for idx in possibly_batched_index] 45 else: 46 data = self.dataset[possibly_batched_index]

in getitem(self, index) 17 18 if self.transform is not None: ---> 19 augmentations= self.transform(image=image, mask=mask) 20 image = augmentations["image"] 21 mask = augmentations["mask"]

TypeError: 'int' object is not callable

opened by sautami26 3
• #### Expected object of scalar type Long but got scalar type Float for sequence element 1 in sequence argument at position #1 'tensors'

Hi,

I rewrote the code along with watching your tutorial. When I run the training procedure, I get the following error:

``````Traceback (most recent call last):
File "/home/niko/programs/pycharm-community-2019.2.1/helpers/pydev/pydevd.py", line 1415, in _exec
pydev_imports.execfile(file, globals, locals)  # execute the script
File "/home/niko/programs/pycharm-community-2019.2.1/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/niko/workspace/pytorch-and-lightning-tutorials/yolo/train_original.py", line 147, in <module>
main()
File "/home/niko/workspace/pytorch-and-lightning-tutorials/yolo/train_original.py", line 126, in main
File "/home/niko/workspace/pytorch-and-lightning-tutorials/yolo/utils.py", line 255, in get_bboxes
true_bboxes = cellboxes_to_boxes(labels)
File "/home/niko/workspace/pytorch-and-lightning-tutorials/yolo/utils.py", line 322, in cellboxes_to_boxes
converted_pred = convert_cellboxes(out).reshape(out.shape[0], S * S, -1)
File "/home/niko/workspace/pytorch-and-lightning-tutorials/yolo/utils.py", line 315, in convert_cellboxes
(predicted_class, best_confidence, converted_bboxes), dim=-1
RuntimeError: Expected object of scalar type Long but got scalar type Float for sequence element 1 in sequence argument at position #1 'tensors'
``````

Then I tried to copy the exact same code from your train.py and dataset.py file but the error still persisted. I guess getitem in dataset.py should return long instead of float types for bounding boxes. Do you know what might be the cause of the error above?

opened by nikogamulin 3
• #### CNN Architecture implementations in Tensorflow

Your YT contents are awesome!! Especially those CNN architectures from scratch videos. I myself is trying to learn DL and your videos helped me understand the concept better when I was reading the academic papers.

Not very long ago, I started implementing some of the popular CNN architectures with Tensorflow 2.0 in my repo and I think it would be good to PR those to here so the rest can checkout both PyTorch and Tensorflow implementations.

I am not super good with Tensorflow, so if there's something that can be improved, feel free to give comments.

I have implemented

• AlexNet
• LeNet5
• ResNet
• VGGNet
opened by the-robot 3
• #### error when running code

when running your code i get this error:

RuntimeError: Expected object of device type cuda but got device type cpu for argument #3 'index' in call to _th_index_select

any tips? thank you

opened by javismiles 3
• #### Why do you need to slice captions?

Hello, thank you for your version of the image captioning solution! However, one thing is not clear to me. Why would you do that slice? If I correctly understood the captions, in that case, is a padded batch of captions, so it looks like: 1 1 1 1 1 2 1 1 1 2 0 0 1 1 1 1 2 0

and if you make a slice [:,: - 1] that would be: 1 1 1 1 1 1 1 1 2 0 1 1 1 1 2 (1 is any token, 2 is and 0 is padding)

So if you want to get rid of tokens that would not work.

opened by concrete13377 3
• #### Image captioning: all training example output is

When training for image captioning, in the first epoch, the `print_examples` function returns the following

``````Example 1 CORRECT: Dog on a beach by the ocean
Example 1 OUTPUT: chasing stores mossy participates player brush museum phone handle drops native punk buried alongside cellphones very bags hairy paintball mouths mats markings volleyball backpacker dressed backpacks legos light bitten various pillow singing attempt superman weather try gnawing ceiling shaped tree someone phone scarf crouching courtyard cows indoors seeds hits hits
Example 2 CORRECT: Child holding red frisbee outdoors
Example 2 OUTPUT: chasing stores mossy bushes tags hardwood tulips chin lining gnawing taken tinkerbell both kind cable tile colorfully shepherd dangling skinny cake scene tattooed swimmer beverage come points come 23 wheels puppy scenic ring snake one piggy snowboard camera slightly fireworks nature try gnawing ceiling shaped tree someone phone scarf crouching
Example 3 CORRECT: Bus driving by parked cars
Example 3 OUTPUT: trucks each that cheerleader hawk jeeps formal ring skeleton forested various plastic goofy snowmobile dances very wearing seaweed cards kick works baseman past daughter football waterfalls bathroom motorcycle bar bikers phone following kid ring past converse nose nose college wide skyscraper rough holding bending seeds broken kissing follows pouring pouring
Example 4 CORRECT: A small boat in the ocean
Example 4 OUTPUT: chasing stores mossy bushes tags hardwood tulips chin lining gnawing taken tinkerbell both kind cable tile colorfully shepherd dangling skinny cake scene tattooed swimmer beverage come points come 23 wheels puppy scenic ring snake one piggy snowboard camera slightly fireworks nature try gnawing ceiling shaped tree someone phone scarf crouching
Example 5 CORRECT: A cowboy riding a horse in the desert
Example 5 OUTPUT: avoid windsurfing alongside roof between enjoys dimly artists artists others biting upon holding silhouette ascending apples curve tennis o leaves gives dinner chasing picnic pack ceremony kayak kayak office festive hikes covered visible signs dancing construction construction when hiking pillow foot leotard about all pit between stool ear sports cigarette
``````

however, after the first epoch and later, the `print_examples` function returns:

``````Example 1 CORRECT: Dog on a beach by the ocean
Example 1 OUTPUT: <SOS> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK>
Example 2 CORRECT: Child holding red frisbee outdoors
Example 2 OUTPUT: <SOS> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK>
Example 3 CORRECT: Bus driving by parked cars
Example 3 OUTPUT: <SOS> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK>
Example 4 CORRECT: A small boat in the ocean
Example 4 OUTPUT: <SOS> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK>
Example 5 CORRECT: A cowboy riding a horse in the desert
Example 5 OUTPUT: <SOS> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK> <UNK>
``````

Im not sure what's going on

opened by tbass134 3
• #### Train set up Yolov3

Hey Aladdin, Thank you for your great tutorial! I have a question about your Yolov3 training. I've tried training Yolo on Pascal VOC with my own settings, but I'm stuck at a mAP of 57.5. How did you get 78.2 mAP? Can you tell me your settings?

My settings:

``````BATCH_SIZE = 16
IMAGE_SIZE = 416
LEARNING_RATE = 1e-5
WEIGHT_DECAY = 1e-4
NUM_EPOCHS = 500
CONF_THRESHOLD = 0.2
MAP_IOU_THRESH = 0.5
NMS_IOU_THRESH = 0.45
``````
opened by salokin1997 2
• #### Inplace Operation in Yolo Loss function for width and height coordinates.

There is a file loss.py file in YOLO object_detection, where inplace operation is performed for width and height coordinates in the YoloLoss class.

opened by Devanshu-singh-VR 0
• #### where to get the "gen.path.tar" and "disc.pth.tar".

Hello thank you for very nice code i search quite for CHECKPOINTS file (CHECKPOINT_GEN = "gen.pth" CHECKPOINT_DISC = "disc.pth") but i did't find in your repository can you please given me the links of these two files.

opened by niazahamd89 0
• #### AttributeError - semantic_segmentation_unet

Receiving an AttributeError. Screen short attached below.

opened by iampaigedlamini 1
• #### The `lr` is now deprecated, we use `learning_rate` instead.

The msg:

``````...\lib\site-packages\keras\optimizer_v2\adam.py:105: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
``````

opened by bitsnaps 0
• #### source mask shape in transformer from scratch

in the code "Transformers from Scratch " in defining the function for source mask why did change the shape from N,src_len to N,1,1,src_len? I know its related to line "energy = energy.masked_fill(mask == 0, float("-1e20"))" but can u explain more?

opened by FarhangAmaji 0
• #### Error File "train.py" line 59 in enumerate(tqdm(dataset))

Please help me with an error in the code when executed, I don't know how to fix the error... I've already scanned the internet for the solution

opened by naruto112 0
• #### CycleGAN

Great repo! I have two questions about CycleGAN though:

1. why `down_blocks` and `up_blocks` in the generator are nn.ModuleList and not nn.Sequential?
2. trying to get the horse/zebra thing to work but the quality of the generated images is not great with "dither"-like artefacts, no idea what could be going on
opened by bosmart 0
• #### how to inference

Hi, for cycle gan, if I have the checkpoint .pth file, how to inference only for the specific test set ?

opened by zhou-rui1 0
• #### Fix EfficientNet expand_conv

Fixed the kernel_size of expand_conv in regard to https://github.com/aladdinpersson/Machine-Learning-Collection/issues/77 tested all efficientnet versions with the foward passes and all return torch.size([4, 10]) as they should

opened by MaxVanDijck 0
• #### EfficientNet Error

nn.Conv2d use a kernel size of 3 which is creating models 2-3x larger than the parameter size defined in the EfficientNet: Rethinking Model Structure paper https://arxiv.org/abs/1905.11946v5

Line 451: https://github.com/keras-team/keras/blob/v2.6.0/keras/applications/efficientnet.py#L526-L549 Official Keras repo uses a kernel size of 1 here

opened by MaxVanDijck 0
• #### 1.0(Mar 6, 2021)

###### Owner
I'm a math geek who likes programming. Particularly interested in machine learning, algorithms and software development.
###### PyTorch Implementation of [1611.06440] Pruning Convolutional Neural Networks for Resource Efficient Inference

PyTorch implementation of [1611.06440 Pruning Convolutional Neural Networks for Resource Efficient Inference] This demonstrates pruning a VGG16 based

792 Nov 26, 2021
###### Code for "Graph-Evolving Meta-Learning for Low-Resource Medical Dialogue Generation". [AAAI 2021]

Graph Evolving Meta-Learning for Low-resource Medical Dialogue Generation Code to be further cleaned... This repo contains the code of the following p

22 Nov 29, 2021
###### Meta Representation Transformation for Low-resource Cross-lingual Learning

MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning This repo hosts the code for MetaXL, published at NAACL 2021. [Meta

34 Nov 14, 2021
###### Pose Detection and Machine Learning for real-time body posture analysis during exercise to provide audiovisual feedback on improvement of form.

Posture: Pose Tracking and Machine Learning for prescribing corrective suggestions to improve posture and form while exercising. This repository conta

4 Nov 25, 2021
###### Implementation of Feedback Transformer in Pytorch

Feedback Transformer - Pytorch Simple implementation of Feedback Transformer in Pytorch. They improve on Transformer-XL by having each token have acce

90 Nov 21, 2021
###### "Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback"

This is code repo for our EMNLP 2017 paper "Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback", which implements the A2C algorithm on top of a neural encoder-decoder model and benchmarks the combination under simulated noisy rewards.

127 Nov 4, 2021
###### OpenDILab RL Kubernetes Custom Resource and Operator Lib

DI Orchestrator DI Orchestrator is designed to manage DI (Decision Intelligence) jobs using Kubernetes Custom Resource and Operator. Prerequisites A w

45 Nov 1, 2021
###### Punctuation Restoration using Transformer Models for High-and Low-Resource Languages

Punctuation Restoration using Transformer Models This repository contins official implementation of the paper Punctuation Restoration using Transforme

75 Dec 1, 2021
###### Byte-based multilingual transformer TTS for low-resource/few-shot language adaptation.

One model to speak them all ?? Audio Language Text ▷ Chinese 人人生而自由，在尊严和权利上一律平等。 ▷ English All human beings are born free and equal in dignity and rig

39 Nov 17, 2021
###### Code for "3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop"

PyMAF This repository contains the code for the following paper: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop Hongwe

194 Nov 24, 2021
###### The code for two papers: Feedback Transformer and Expire-Span.

transformer-sequential This repo contains the code for two papers: Feedback Transformer Expire-Span The training code is structured for long sequentia

110 Nov 11, 2021
###### Emulation and Feedback Fuzzing of Firmware with Memory Sanitization

BaseSAFE This repository contains the BaseSAFE Rust APIs, introduced by "BaseSAFE: Baseband SAnitized Fuzzing through Emulation". The example/ directo

118 Oct 31, 2021
###### Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

4.7k Nov 28, 2021
###### Robust Video Matting in PyTorch, TensorFlow, TensorFlow.js, ONNX, CoreML!

Robust Video Matting (RVM) English | 中文 Official repository for the paper Robust High-Resolution Video Matting with Temporal Guidance. RVM is specific

0 Sep 28, 2021
###### Release of SPLASH: Dataset for semantic parse correction with natural language feedback in the context of text-to-SQL parsing

SPLASH: Semantic Parsing with Language Assistance from Humans SPLASH is dataset for the task of semantic parse correction with natural language feedba

23 Oct 4, 2021