CS583: Deep Learning

Related tags

Deep Learning DeepLearning
Overview

CS583: Deep Learning

  1. Machine learning basics. This part briefly introduces the fundamental ML problems-- regression, classification, dimensionality reduction, and clustering-- and the traditional ML models and numerical algorithms for solving the problems.

  2. Neural network basics. This part covers the multilayer perceptron, backpropagation, and deep learning libraries, with focus on Keras.

  3. Convolutional neural networks (CNNs). This part is focused on CNNs and its application to computer vision problems.

    • CNN basics [slides].

    • Tricks for improving test accuracy [slides].

    • Feature scaling and batch normalization [slides].

    • Advanced topics on CNNs [slides].

    • Popular CNN architectures [slides].

    • Further reading:

      • [style transfer (Section 8.1, Chollet's book)]

      • [visualize CNN (Section 5.4, Chollet's book)]

  4. Recurrent neural networks (RNNs). This part introduces RNNs and its applications in natural language processing (NLP).

  5. Transformer Models.

  6. Autoencoders. This part introduces autoencoders for dimensionality reduction and image generation.

    • Autoencoder for dimensionality reduction [slides].

    • Variational Autoencoders (VAEs) for image generation [slides].

  7. Generative Adversarial Networks (GANs).

  8. Deep Reinforcement Learning.

  9. Parallel Computing.

  10. Adversarial Robustness. This part introduces how to attack neural networks using adversarial examples and how to defend from the attack.

  11. Meta Learning.

  12. Neural Architecture Search (NAS).

You might also like...
PyKale is a PyTorch library for multimodal learning and transfer learning as well as deep learning and dimensionality reduction on graphs, images, texts, and videos
PyKale is a PyTorch library for multimodal learning and transfer learning as well as deep learning and dimensionality reduction on graphs, images, texts, and videos

PyKale is a PyTorch library for multimodal learning and transfer learning as well as deep learning and dimensionality reduction on graphs, images, texts, and videos. By adopting a unified pipeline-based API design, PyKale enforces standardization and minimalism, via reusing existing resources, reducing repetitions and redundancy, and recycling learning models across areas.

PyKale 370 Dec 27, 2022
🧠 A PyTorch implementation of 'Deep CORAL: Correlation Alignment for Deep Domain Adaptation.', ECCV 2016
🧠 A PyTorch implementation of 'Deep CORAL: Correlation Alignment for Deep Domain Adaptation.', ECCV 2016

Deep CORAL A PyTorch implementation of 'Deep CORAL: Correlation Alignment for Deep Domain Adaptation. B Sun, K Saenko, ECCV 2016' Deep CORAL can learn

Andy Hsu 200 Dec 25, 2022
Awesome Deep Graph Clustering is a collection of SOTA, novel deep graph clustering methods

ADGC: Awesome Deep Graph Clustering ADGC is a collection of state-of-the-art (SOTA), novel deep graph clustering methods (papers, codes and datasets).

yueliu1999 297 Dec 27, 2022
🔥 Cogitare - A Modern, Fast, and Modular Deep Learning and Machine Learning framework for Python
🔥 Cogitare - A Modern, Fast, and Modular Deep Learning and Machine Learning framework for Python

Cogitare is a Modern, Fast, and Modular Deep Learning and Machine Learning framework for Python. A friendly interface for beginners and a powerful too

Cogitare - Modern and Easy Deep Learning with Python 76 Sep 30, 2022
Visualizer for neural network, deep learning, and machine learning models
Visualizer for neural network, deep learning, and machine learning models

Netron is a viewer for neural network, deep learning and machine learning models. Netron supports ONNX (.onnx, .pb, .pbtxt), Keras (.h5, .keras), Tens

Lutz Roeder 21k Jan 6, 2023
PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）
PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

English | 简体中文 Welcome to the PaddlePaddle GitHub. PaddlePaddle, as the only independent R&D deep learning platform in China, has been officially open

null 19.4k Jan 4, 2023
Machine learning framework for both deep learning and traditional algorithms
Machine learning framework for both deep learning and traditional algorithms

NeoML is an end-to-end machine learning framework that allows you to build, train, and deploy ML models. This framework is used by ABBYY engineers for

NeoML 704 Dec 27, 2022
Deep Learning and Reinforcement Learning Library for Scientists and Engineers 🔥
Deep Learning and Reinforcement Learning Library for Scientists and Engineers 🔥

TensorLayer is a novel TensorFlow-based deep learning and reinforcement learning library designed for researchers and engineers. It provides an extens

TensorLayer Community 7.1k Dec 27, 2022
Predict stock movement with Machine Learning and Deep Learning algorithms

Project Overview Stock market movement prediction using LSTM Deep Neural Networks and machine learning algorithms Software and Library Requirements Th

Naz Delam 46 Sep 13, 2022
Comments
  • Questions about Differentiable Neural Architecture Search

    Questions about Differentiable Neural Architecture Search

    image

    问题 1: https://www.bilibili.com/video/BV1C64y127Fv 时间: 10:26

    θ 不是张量 (看时间：7:00)，所以卷积层block 的输出 不等于 θ

    那么 卷积层block 的输出 怎么生成 θ 呢 ？

    Question 1: https://youtu.be/D9m9-CXw_HY?t=626

    θ is not a tensor (check video at t=7:00), so the convolutional block output is not equal to θ

    So, how does the convolutional block generate θ ?

    image

    问题 2: 在 https://github.com/D-X-Y/AutoDL-Projects/issues/99#issuecomment-835802887 和 GDAS 文献 的式子 (7) 里，如何在两个节点之间的多个平行 的连接线 进行反向传播的运作 ？

    Question 2: For https://github.com/D-X-Y/AutoDL-Projects/issues/99#issuecomment-835802887 and equation (7) of GDAS paper , how to do backpropagation across multiple parallel edges between two nodes ?

    opened by buttercutter 2
  • FL联邦学习中，server对client恶意行为的侦测（方式的猜想）

    FL联邦学习中，server对client恶意行为的侦测（方式的猜想）

    机器学习是希望使用一个矩阵，高维空间，对数据特征特征进行模拟。 server如 google 维护一个全局的矩阵w_global。客户端的数据不满足独立同分布，其中本地数据记为 w_local_i, 其中i 为用户名。 给定一个标量 trust_Max 超参数，定义最大偏离的距离，定义某个客户 与 w 的”距离“的矩阵 distance_to_global_i = w - w_local_i (对应元素相减)，检验一下distance_to_global_i 矩阵 L2 范数与 trust_Max 的关系。

    大于阈值，代表该用户和大多数用户的使用习惯偏差过大，有蓄意攻击的可能。 但世界上还会存在少量其他用户 w_local_k 和他的 w_local_i 的分布近似，体现在两个“距离矩阵”的差 distance_to_global_i - distance_to_global_k 的新矩阵的L2 很小。意味着利用 google 提供的高维空间 w 拟合两个用户的习惯类似。

    ---以下，猜测的部分----

    在i k中分别训练另一个网络 m_i 和 m_k，m 使用不同于 w 的网络结构、参数，但 m_i 和 m_k参数相同。训练相同epoch。 方法一：得到本地的训练评价acc_i 和 acc_k，交换 m，分别在异地验证m，得到对方m 在本地的评价。与先前的评价进行比较。 方法二：直接对 m_i 和 m_k 差的 L2进行度量，判断 m_1 和 m_k 是否相近。

    // 猜测基于假设： // 如果两个客户的本地 w_local_ 和 w_global 的 对应元素差相近，它们的数据更接近。 // 蓄意攻击者通过加噪声等方式修改本地数据，意图污染 w_global，会让本地数据变得很奇怪，和其他的善良的偏离者在 w 网络下表现类似，却在另一个高纬空间的投影，新的网络 m 中表现不同。（我的猜测，不知道有没有可能。恶意用户 i 修改数据会让数据变得和谁都不像，在 w 中呈现和善良的用户 k 相近的表现，只是偶然。并不会在 m 中 依然接近 k）

    还有一种通用方式（应对某些恶意用户 i 他的 w_local_i 并不过分偏离 w_global）： 找一个用户k，他的 w_local_k 和 w_local_i 距离很近。server命令他们根据参数列表 temporary_table(临时生成的一组网络参数，包括层数、行列数量)。i 和 k 依据 table 训练网络 m， 交叉验证异地表现。

    opened by David-Dingle 0
  • 并行机器学习中（ring—structure），efficient all—reduce，可否在 O( number_of(node) / Network_bandwidth ) 复杂度下实现？

    并行机器学习中（ring—structure），efficient all—reduce，可否在 O( number_of(node) / Network_bandwidth ) 复杂度下实现？

    您的视频中，四个节点，六轮（每轮通讯四条链路同时利用）通讯，共24次通讯，复杂度是（number_of_parameter / wide——bandwidth）

    有A B C D四台机器，分别在本地维护维护一个 w 矩阵，和一个同纬度的sum_local 矩阵，一个计数器 i 初始化为1。

    sum_local = w // 根据本地 w 的拷贝初始化 sum_local i = 1 // sum_local 已收集了一个矩阵 while( i != number_of_Node_in_ring ) { throw w to next node get new w from previous node // 每次接到上个节点传来的 w，下次循环时只将接收到的w再传递出去 ++i sum_local += w }

    这个方法可行么，共计四轮（每轮，四条链路同时利用）通讯，共16 次通讯（复杂度是 number_of_node / bandwidth）

    opened by David-Dingle 0
Owner
Shusen Wang
Assistant Professor@Stevens Institute of Technology
Shusen Wang
GitHub
Deep learning (neural network) based remote photoplethysmography: how to extract pulse signal from video using deep learning tools

Deep-rPPG: Camera-based pulse estimation using deep learning tools Deep learning (neural network) based remote photoplethysmography: how to extract pu

Terbe Dániel 138 Dec 17, 2022
deep-table implements various state-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch.

deep-table implements various state-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch.

null 63 Oct 17, 2022
Time-series-deep-learning - Developing Deep learning LSTM, BiLSTM models, and NeuralProphet for multi-step time-series forecasting of stock price.

Stock Price Prediction Using Deep Learning Univariate Time Series Predicting stock price using historical data of a company using Neural networks for

Abdultawwab Safarji 7 Nov 27, 2022
FTIR-Deep Learning - FTIR Deep Learning With Python

CANDIY-spectrum Human analyis of chemical spectra such as Mass Spectra (MS), Inf

Wei Mei 1 Jan 3, 2022
Deep Learning: Architectures & Methods Project: Deep Learning for Audio Super-Resolution

Deep Learning: Architectures & Methods Project: Deep Learning for Audio Super-Resolution Figure: Example visualization of the method and baseline as a

Oliver Hahn 16 Dec 23, 2022
PyTorch implementation of the Deep SLDA method from our CVPRW-2020 paper "Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis"

Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis This is a PyTorch implementation of the Deep Streaming Linear Discriminant

Tyler Hayes 41 Dec 25, 2022
Deep Image Search is an AI-based image search engine that includes deep transfor learning features Extraction and tree-based vectorized search.

Deep Image Search - AI-Based Image Search Engine Deep Image Search is an AI-based image search engine that includes deep transfer learning features Ex

null 139 Jan 1, 2023
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.

H2O H2O is an in-memory platform for distributed, scalable machine learning. H2O uses familiar interfaces like R, Python, Scala, Java, JSON and the Fl

H2O.ai 6.1k Jan 5, 2023
Machine Learning From Scratch. Bare bones NumPy implementations of machine learning models and algorithms with a focus on accessibility. Aims to cover everything from linear regression to deep learning.

Machine Learning From Scratch About Python implementations of some of the fundamental Machine Learning models and algorithms from scratch. The purpose

Erik Linder-Norén 21.8k Jan 9, 2023
A resource for learning about deep learning techniques from regression to LSTM and Reinforcement Learning using financial data and the fitness functions of algorithmic trading

A tour through tensorflow with financial data I present several models ranging in complexity from simple regression to LSTM and policy networks. The s

null 195 Dec 7, 2022