Advance Machine Learning Program

1. Mathematical Foundations of Data Science:

A. Linear Algebra:

1. Vectors, Matrices
2. Tensors
3. Matrix Operations
4. Projections
5. Eigenvalue decomposition of a matrix
6. LU Decomposition
7. QR Decomposition/Factorization
8. Symmetric Matrices
9. Orthogonalization & Orthonormalization
10. Real and Complex Analysis (Sets and Sequences, Topology, Metric Spaces, Single-Valued and Continuous Functions, Limits, Cauchy Kernel, Fourier Transforms)
11. Information Theory (Entropy, Information Gain)
12. Function Spaces and Manifolds
13. Relational Algebra and SQL

B. Multivariate Calculus

1. Differential and Integral Calculus
2. Partial Derivatives
3. Vector-Values Functions
4. Directional Gradient
5. Hessian
6. Jacobian
7. Laplacian and Lagragian Distribution

2: Probability for Data Scientists

1. Probability Theory and Statistics
2. Combinatorics
3. Random Variables
4. Probability Rules & Axioms
5. Bayes' Theorem
6. Variance and Expectation
7. Conditional and Joint Distributions
8. Standard Distributions (Bernoulli, Binomial, Multinomial, Uniform and Gaussian)
9. Moment Generating Functions
10. Maximum Likelihood Estimation (MLE)
11. Prior and Posterior
12. Maximum a Posteriori Estimation (MAP) and Sampling Methods
13. Descriptive Statistics
14. Hypothesis Testing
15. Goodness of Fit
16. Analysis of Variance
17. Correlation
18. Chi2 test
19. Design of Experiments

3: Algorithms and Data Structures:

A. Graph Theory: Basic Concepts and Algorithms
B. Algorithmic Complexity

1. Algorithm Analysis
2. Greedy Algorithms
3. Divide and Conquer and Dynamic Programming

C. Data Structures

1. Array, List, Hashing, Binary Trees, Hashing, Heap, Stack etc
2. Dynamic Programming
3. Randomized & Sublinear Algorithm
4. Graphs

4: R and Python

A. R PROGRAMMING LANGUAGE

1. Vectors
2. Matrices
3. Lists
4. Data frame
5. Basic Syntax
6. Basic Statistics
7. Data Manipulation (dplyr)
8. Visualization (ggplot2)
9. Connecting to databases (RJDBC)

B. Python Programming Language

1. Python language fundamentals
2. Data Structures
3. Beautiful Soup
4. Regular Expressions
5. JSON
6. Restful Web Services (Flask)
7. NumPy
8. Plots in matplotlib, seaborn
9. Pandas

5: Numerical and Combinatorial Optimization

Conjugate Gradient Methods, Quasi-Newton Methods, Constrained Optimization, Linear Programming, Nonlinear Constrained optimization, Quadratic Programming, Integer Programming, Knapsack problem, travelling salesman, vehicle routing, job shop scheduling, Gradient/Stochastic Descents and Primal-Dual methods.

In this course, DankoNikolic brain scientists and AI inventor explains the fundamental of intelligence needed for everyone interested in creating ambitious AI solutions. What you do when things get tough? In the course, you will learn the differences between machine intelligence and human intelligence. You will understand why AI fails and when. AI does not have a narrowly limited working memory (a.k.a., short-term memory) but we humans do. How does our working memory make us more intelligent than machines? Why do we understand the world and machines don't? You will also learn fundamental theorems for machine learning and see how they apply to machine intelligence and human intelligence. After having learned that, you will be able to judge whether an ML project is too ambitious or is likely to succeed. You will be able to identify fundamental problems that plagued some of the ambitious AI projects in the past. You will understand why it is nearly impossible for machines to reach human levels of intelligence. Also, you will learn why some of the tricks in machine learning sometimes work and other times not. You will understand why it is so difficult to build self-driving cars.

The course offers fundamentals that you cannot find in any other course or a book. These fundamentals will be invaluable for your future work on ML and AI.

1. This course explores the nature of intelligence, ranging from machines to the biological brain. Information provided in the course is useful when undergoing ambitious projects in machine learning and AI. It will help you avoid pitfalls in those projects.
2. What are the differences between the real brain and machine intelligence and how can you use this knowledge to prevent failures in your work? What are the limits of today's AI technology? How to assess early in your AI project whether it has chances of success?
3. What are the most fundamental mathematical theorems in machine learning and how they are relevant for your everyday work?

1. Data Exploration And Preprocessing

• Basic Plotting of Data
• Outlier Detection
• Dimensionality Reduction: Principal Component Analysis, Multidimensional Scaling
• Data Transformation
• Dealing with Missing Values

2. Feature Engineering

• Feature extraction and feature engineering,
• Feature transformation
• Feature selection
• Grid search
• Automatically create features
• Aggregations and transformations
• Introduction about Feature tools
• Introduction about Entities & Entity Sets, table Relationships, Feature Primitives, Deep Feature synthesis

1. Learning from Data:

• The appeal of learning from examples, Motivational Case Studies, A formal definition of learning, Key Components of Learning, Population vs. Sample, Decision Boundary, Types of data, Typical Issues with Data, Types of Learning
• Learning as search: Instance and Hypothesis Space, Introductions to Search Algorithms, Cost Functions
• Version Spaces/Perceptron/Linear Regression /Nearest Neighbor
• Overfitting/Regularization, Worst case performance: VC dimension, Bias Variance Tradeoff, Non Linear Embedding, Outlier Detection, Minimum description length
• Estimating Accuracy: Train/test split, Cross Validation, Bootstrap Hypothesis testing, Confusion Matrix, Sensitivity and Specificity, Precision and Recall, ROC curves and AUC, MAPE, Kappa Statistic, AIC, BIC
• Data Science Process

Introduction to Bayesian Learning:

• Probability review
• Bayes rule
• Conjugate priors
• Bayesian Inference I (coin flipping)
• Bayesian Inference II (hypothesis testing and summarizing distributions)
• Bayesian Inference III (decision theory)
• Bayesian linear regression
• Bayes classifiers
• Statistical Estimation a. Maximum Likelihood Estimation c. Bayes error rate e. Curse of dimensionality
• Laplace approximation, Naïve Bayes, Introduction to Bayesian Belief Networks, Introduction to Sampling, Gibbs sampling, Logistic regression

1. Classification Algorithms: Decision Trees, Rule Induction, SVM
2. Dealing with Skewed Class Distribution and Cost-based Classification: Resampling
3. Regression: Lasso and Ridge Regression, MARS, OLS, PLS, GLM
4. Survival Analysis: Cox’s Regression, Weibull Distribution, Parametric Survival Models
5. Ensemble Models: Bagging, Boosting, Stacking, Random Forests, XGBoost, GBDT
6. Neural Networks and Introduction to Deep Learning: Multi Layered Perceptron, Back propagation, Convolutional Neural Networks, Encode-Decoders, Recurrent Neural Networks, Long Short Term Memory (LSTM)
7. Time Series Forecasting: Time Series Decomposition, Holt-Winters, ARIMA, Intermittent Models, Time-frequency domain, Fourier transforms, wavelet transforms, Dynamic Regression Models, Neural Networks, Demand Forecasting

1. Case Study: Network Intrusion detection
2. Case Study: Weather Forecasting
3. Case Study: Image Classification
4. Case Study: Predictive Text Generation
5. Case Study: Customer Lifetime Modelling
6. Case Study: Churn Prediction
7. Case Study: Speech synthesis

In this course, you will learn to analyse and solve competitively such predictive modelling tasks. When you finish this class, you will: Understand how to solve predictive modelling competitions efficiently and learn which of the skills obtained can be applicable to real-world tasks.

1. Preprocess the data and generate new features from various sources text and images.
2. Advance feature engineering techniques Generating mean-encodings, aggregated statistical measures, nearest neighbors as a means to improve your predictions.
3. Cross-validation methodologies benchmark your solutions.
4. Analyzing and interpreting the data. Inconsistencies, high noise levels, errors and other data-related issues such as leakages.
5. Efficiently tune hyperparameters of Algorithms and achieve top performance.
6. Master the art of combining different machine learning models and learn how to ensemble.
7. Get exposed to past (winning) solutions and codes and learn how to read them.

1. Clustering Methods: Expectation Maximization, K-means, k-medoids, Agglomerative and Divisive Hierarchical clustering, Birch, DBScan, Spectral Clustering, Self-organizing maps
2. Community detection in graphs
3. Association Rules and Sequence Pattern Discovery
4. Deviation Detection
5. Semi-supervised and Active Learning
6. Sequential Data Models: Markov Models and Hidden Markov Models, Kalman Filters
7. Model Selection: Model Comparisons, Analysis Considerations

1. Case Study: Viral Marketing
2. Case Study: Driving for Fuel Efficiency

1. Expectation Maximization algorithm
2. Probit regression
3. Expectation Maximization to variational inference
4. Variational inference
5. Finding optimal distributions
6. Exponential families
7. Conjugate exponential family models
8. Scalable inference
9. Bayesian nonparametric clustering
10. Markov Models, Hidden Markov models
11. Conditional Random Fields
12. Monte Carlo, Sampling, Rejection Sampling
13. Poisson matrix factorization
14. Decision Networks
15. Bayesian Optimization
16. Bayesian State-Space Models and Kalman Filtering
17. Probabilistic Numerics and Bayesian Quadrature

1. Case Study: Named Entity Extraction
2. Case Study: Car navigation
3. Case Study: Diagnosis

1. Big Data Stack

• The Map reduce paradigm
• Hadoop,: HDFS, Hive (SQL), Pig, Sqoop Flume, Avro
• NoSQL: Big Table, HBase, Document stores, Graph stores, Key-Value stores
• Spark and Introduction to PySpark

2. Functional Programming in Scala
Functions, Data and Abstractions, collections, pattern matching and functions, Lazy Evaluation, Functions and State, Observer Pattern, reactive programming,Parallel Programming, Data Parallelism, Data Structures for Parallel Computing, RDDs,pair RDDs, Reduction Operations, Partitioning and Shuffling,Dataframes, Spark SQL

3. Design Thinking and Storytelling
Preparing your mind for innovation, Idea Generation and Experimentation

1. Statistical Language Modeling
2. Computational Linguistics
3. Statistical Decision Making and the Source-Channel Paradigm
4. Sparseness; Smoothing
5. Measuring Success: Information Theory, Entropy and Perplexity Maximum Entropy Models, Whole-Sentence Models, Semantic Modeling
6. EM for sound separation
7. Probabilistic Context Free Grammars (PCFG), the Inside-Outside Algorithm
8. Syntactic Language Models
9. Decision Tree Language Models

1. Multitask Learning
2. Asynchronous Gradient Decent: Hogwild!, Momentum, Gibbs Sampling, Cyclades
3. Hyperparameter Optimization
4. Low-Precision Training
5. Matrix Completion and Approximation
6. Tensor Factorization
7. On-device Inference
8. Performance Optimizers
9. Local Learning
10. Feature Selection
11. Reinforcement Learning Optimization
12. Distributed Reinforcement Learning
13. Scalable Variational Inference
14. Interpretable ML
15. ADMM and its connections to belief propagation
16. The nonconvex landscape of neural networks

1. Data Streams Computational Model
2. Change Detection
3. Classification
4. Regression
5. Clustering Data Streams
6. Frequent Pattern Mining

1. Case Study: IOT Data Processing

1. Deep Neural Networks for Natural Language Processing. (8 Hours)

A. Introduction

• Introduction to Neural Networks
• Example Tasks and Their Difficulties
• What Neural Nets Can Do To Help

B. Predicting the Next Word in a Sentence

• Computational Graphs
• Feed-forward Neural Network Language Models
• Measuring Model Performance: Likelihood and Perplexity

C. Distributional Semantics and Word Vectors

• Describing a word by the company that it keeps
• Counting and predicting
• Skip-grams and CBOW
• Evaluating/Visualizing Word Vectors
• Advanced Methods for Word Vectors

D. Why is word2vec So Fast?: Speed Tricks for Neural Nets

• Softmax Approximations: Negative Sampling, Hierarchical Softmax
• Parallel Training
• Tips for Training on GPUs

E. Convolutional Networks for Text

• Bag of Words, Bag of n-grams, and Convolution
• Applications of Convolution: Context Windows and Sentence Modeling
• Stacked and Dilated Convolutions
• Structured Convolution
• Convolutional Models of Sentence Pairs
• Visualization for CNNs

F. Recurrent Networks for Sentence or Language Modeling

• Recurrent Networks
• Vanishing Gradient and LSTMs
• Strengths and Weaknesses of Recurrence in Sentence Modeling
• Pre-training for RNNs

G. Using/Evaluating Sentence Representations

• Sentence Similarity
• Textual Entailment
• Paraphrase Identification
• Retrieval

H. Conditioned Generation

• Encoder-Decoder Models
• Conditional Generation and Search
• Ensembling
• Evaluation
• Types of Data to Condition On

I. Attention

• Attention
• What do We Attend To?
• Improvements to Attention
• Specialized Attention Varieties

1. A Case Study: "Attention is All You Need"

2. Foundations of Natural language Processing

• Word embedding
• Named entity recognition
• Parts-of-Speech tagging
• Language modeling
• Segmentation
• Paraphrasing
• Machine translation
• Information Extraction
• Text Summarization
• Conditional Random Fields
• Dimensionality Reduction: Matrix Factorization, Topic Models

1. Case Study: Q & A Systems
2. Case Study: Spam Detector

3. Text Classification

• Tokenization
• Lemmatization
• Vectorization
• Bag of Words representation
• Language Models
• Tfidf
• Singular Value Decomposition
• Topic Models
• Discourse Modelling
• Coreference Resolution
• Question Answering Systems
• Visualizing complex and high dimensional data
• Sentiment Analysis

1. Case Study: Web Personalization
2. Case Study: Text Classification

1. Content Based Filtering
2. User and Item based Collaborative Filtering
3. New Item Problem
4. ALS
5. Conversational Recommenders
6. Diversity in Recommendation

1. Frequent Itemset Mining
2. Locality-Sensitive Hashing
3. Dimensionality Reduction
4. Algorithms on Large Graphs
5. Large-Scale Machine Learning
6. Computational Advertising
7. Learning through Experimentation
8. Optimizing Submodular Functions

1. Introduction and Structure of Graphs
2. Web as a Graph and the Random Graph Model
3. The Small World Phenomena
4. Decentralized search in small-world and P2P networks
5. Applications of Social Network Analysis
6. Networks with Signed Edges
7. Cascading Behavior: Decision Based Models of Cascades
8. Cascading Behavior: Probabilistic Models of Information Flow
9. Influence Maximization
10. Outbreak Detection
11. Power-laws and Preferential attachment
12. Models of evolving networks
13. Kronecker graphs
14. Link Analysis: HITS and PageRank
15. Strength of weak ties and Community structure in networks
16. Network community detection: Spectral Clustering
17. Biological networks, Overlapping communities in networks
18. Representation Learning on Graphs