Statistics and Mathematics for Machine Learning, Deep Learning , Deep NLP

Omid Safarzadeh

Last update: Dec 29, 2022

Related tags

Text Data & NLP Stat4ML

Overview

Stat4ML

Statistics and Mathematics for Machine Learning, Deep Learning , Deep NLP

This is the first course from our trio courses:

Statistics Foundation for ML

https://github.com/Bellman281/Stat4ML/

Introduction to Statistical Learning https://github.com/Bellman281/Intro_Statistical_Learning
Advanced Statistical Learning for DL ( to be anounced)

Registration Form for cohort 2 of STAT4ML:

https://forms.gle/ZqLJLmv1K5nGVx3m7

Notes about the course:

Instructor : Omid Safarzadeh,

LinkedIn: https://www.linkedin.com/in/omidsafarzadeh/

IG : @deepdatascientists

Course Text Book: Statistical Inference 2nd Edition by George Casella (Author), Roger L. Berger (Author) :

https://www.amazon.com/Statistical-Inference-George-Casella-dp-0534243126/dp/0534243126/ref=mt_other?_encoding=UTF8&me=&qid=

Pre Requisitives

Recall from Calculus:

    Derivative
          Chain rule
    Integral
          Techniques of Integration
          Substitution
    Integration by parts

Matrix Algebra Review:

    Matrix operations
    Matrix Multiplication
       Properties of determinants
       Inverse Matrix
       Matrix Transpose
       Properties of transpose
    Partioned Matrices
    Eigenvalues and Eigenvectors
    Matrix decomposition
       LU decomposition
       Cholesky decomposition
       QR decomposition
       SVD
    Matrix Differentiation

Course 1 :

Slide 1 : Probability Theory Foundation

 Sample Space
 Probability Theory Foundation
    Axiomatic Foundations
    The Calculus of Probabilities
 Independence
 Conditional Probability
    Bayes Theorem
 Random Variables
 Probability Function
    Distribution Functions
    Density function

Slide 2: Moments

   Moments
       Expected Value
       Variance
       Covariance and Correlation
   Moment Generating Functions
       Normal mgf
   Matrix Notation for Moments

Slide 3: Distribution Functions

   Distributions
     Discrete Distribution
       Discrete Uniform Distribution
       Binomial Distribution
       Poisson Distribution
     Continuous Distribution
       Uniform Distribution
       Exponential Distribution
       Normal Distribution
       Lognormal Distribution
       Laplace Distribution
       Beta Distribution

Slide 4: Conditional and Multivariate Distributions

Joint and Marginal Distribution
Conditional Distributions and Independence
Bivariate Transformations
Hierarchical Models and Mixture Distribution
Bivariate Normal Distribution
Multivariate Distribution

Slide 5: Convergence Concepts

Random Samples
   Sums of Random Variable from a Random Sample
Inequalities
Convergence Concepts:
   Almost Sure Convergence
   Convergence in Probability
   Convergence in Distribution
The Delta Method

Slide 6: Maximum Likelihood Estimation

Maximum Likelihood Estimation
  Motivation and the Main Ideas
  Properties of the Maximum Likelihood Estimator

Slide 7: Bayesian and posterior distribution Estimation

   Computing the posterior
   Maximum likelihood estimation (MLE)
Maximum a posteriori (MAP) estimation
   Posterior mean
   MAP properties
Bayesian linear regression

Kashgari is a production-level NLP Transfer learning framework built on top of tf.keras for text-labeling and text-classification, includes Word2Vec, BERT, and GPT2 Language Embedding.

Kashgari Overview | Performance | Installation | Documentation | Contributing 🎉 🎉 🎉 We released the 2.0.0 version with TF2 Support. 🎉 🎉 🎉 If you

2k Feb 9, 2021

Data loaders and abstractions for text and NLP

torchtext This repository consists of: torchtext.data: Generic data loaders, abstractions, and iterators for text (including vocabulary and word vecto

3.2k Dec 30, 2022

Data loaders and abstractions for text and NLP

torchtext This repository consists of: torchtext.data: Generic data loaders, abstractions, and iterators for text (including vocabulary and word vecto

2.6k Feb 18, 2021

:mag: End-to-End Framework for building natural language search interfaces to data by utilizing Transformers and the State-of-the-Art of NLP. Supporting DPR, Elasticsearch, HuggingFace’s Modelhub and much more!

Haystack is an end-to-end framework that enables you to build powerful and production-ready pipelines for different search use cases. Whether you want

1.4k Feb 18, 2021

nlabel is a library for generating, storing and retrieving tagging information and embedding vectors from various nlp libraries through a unified interface.

2 Jun 10, 2022

:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.

(Framework for Adapting Representation Models) What is it? FARM makes Transfer Learning with BERT & Co simple, fast and enterprise-ready. It's built u

1.6k Dec 27, 2022

:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.

(Framework for Adapting Representation Models) What is it? FARM makes Transfer Learning with BERT & Co simple, fast and enterprise-ready. It's built u

1.1k Feb 14, 2021

Prompt-learning is the latest paradigm to adapt pre-trained language models (PLMs) to downstream NLP tasks

Prompt-learning is the latest paradigm to adapt pre-trained language models (PLMs) to downstream NLP tasks, which modifies the input text with a textual template and directly uses PLMs to conduct pre-trained tasks. This library provides a standard, flexible and extensible framework to deploy the prompt-learning pipeline. OpenPrompt supports loading PLMs directly from huggingface transformers. In the future, we will also support PLMs implemented by other libraries.

2.3k Jan 8, 2023

NLP, before and after spaCy

textacy: NLP, before and after spaCy textacy is a Python library for performing a variety of natural language processing (NLP) tasks, built on the hig

2k Jan 4, 2023

Statistics and Mathematics for Machine Learning, Deep Learning , Deep NLP

Related tags

Overview

Stat4ML

Registration Form for cohort 2 of STAT4ML:

Pre Requisitives

Recall from Calculus:

Matrix Algebra Review:

Course 1 :

Slide 1 : Probability Theory Foundation

Slide 2: Moments

Slide 3: Distribution Functions

Slide 4: Conditional and Multivariate Distributions

Slide 5: Convergence Concepts

Slide 6: Maximum Likelihood Estimation

Slide 7: Bayesian and posterior distribution Estimation

You might also like...

Kashgari is a production-level NLP Transfer learning framework built on top of tf.keras for text-labeling and text-classification, includes Word2Vec, BERT, and GPT2 Language Embedding.

Data loaders and abstractions for text and NLP

Data loaders and abstractions for text and NLP

:mag: End-to-End Framework for building natural language search interfaces to data by utilizing Transformers and the State-of-the-Art of NLP. Supporting DPR, Elasticsearch, HuggingFace’s Modelhub and much more!

nlabel is a library for generating, storing and retrieving tagging information and embedding vectors from various nlp libraries through a unified interface.

:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.

:house_with_garden: Fast & easy transfer learning for NLP. Harvesting language models for the industry. Focus on Question Answering.

Prompt-learning is the latest paradigm to adapt pre-trained language models (PLMs) to downstream NLP tasks

NLP, before and after spaCy

Owner

Omid Safarzadeh

Grading tools for Advanced NLP (11-711)Grading tools for Advanced NLP (11-711)

Machine learning models from Singapore's NLP research community

NLP, Machine learning

NLP - Machine learning

A program that uses real statistics to choose the best times to bet on BloxFlip's crash gamemode

This is my reading list for my PhD in AI, NLP, Deep Learning and more.

This repo is to provide a list of literature regarding Deep Learning on Graphs for NLP

A NLP program: tokenize method, PoS Tagging with deep learning

Kashgari is a production-level NLP Transfer learning framework built on top of tf.keras for text-labeling and text-classification, includes Word2Vec, BERT, and GPT2 Language Embedding.