Applied Machine Learning Program

1. Mathematical Foundations of Data Science:

A. Linear Algebra:

1. Vectors, Matrices
2. Tensors
3. Matrix Operations
4. Projections
5. Eigenvalue decomposition of a matrix
6. LU Decomposition
7. QR Decomposition/Factorization
8. Symmetric Matrices
9. Orthogonalization & Orthonormalization
10. Real and Complex Analysis (Sets and Sequences, Topology, Metric Spaces, Single-Valued and Continuous Functions, Limits, Cauchy Kernel, Fourier Transforms)
11. Information Theory (Entropy, Information Gain)
12. Function Spaces and Manifolds
13. Relational Algebra and SQL

B. Multivariate Calculus

1. Differential and Integral Calculus
2. Partial Derivatives
3. Vector-Values Functions
4. Directional Gradient
5. Hessian
6. Jacobian
7. Laplacian and Lagragian Distribution

2: Probability for Data Scientists

1. Probability Theory and Statistics
2. Combinatorics
3. Random Variables
4. Probability Rules & Axioms
5. Bayes' Theorem
6. Variance and Expectation
7. Conditional and Joint Distributions
8. Standard Distributions (Bernoulli, Binomial, Multinomial, Uniform and Gaussian)
9. Moment Generating Functions
10. Maximum Likelihood Estimation (MLE)
11. Prior and Posterior
12. Maximum a Posteriori Estimation (MAP) and Sampling Methods
13. Descriptive Statistics
14. Hypothesis Testing
15. Goodness of Fit
16. Analysis of Variance
17. Correlation
18. Chi2 test
19. Design of Experiments

3: Algorithms and Data Structures:

A. Graph Theory: Basic Concepts and Algorithms
B. Algorithmic Complexity

1. Algorithm Analysis
2. Greedy Algorithms
3. Divide and Conquer and Dynamic Programming

C. Data Structures

1. Array, List, Hashing, Binary Trees, Hashing, Heap, Stack etc
2. Dynamic Programming
3. Randomized & Sublinear Algorithm
4. Graphs

4: R and Python

A. R PROGRAMMING LANGUAGE

1. Vectors
2. Matrices
3. Lists
4. Data frame
5. Basic Syntax
6. Basic Statistics
7. Data Manipulation (dplyr)
8. Visualization (ggplot2)
9. Connecting to databases (RJDBC)

B. Python Programming Language

1. Python language fundamentals
2. Data Structures
3. Beautiful Soup
4. Regular Expressions
5. JSON
6. Restful Web Services (Flask)
7. NumPy
8. Plots in matplotlib, seaborn
9. Pandas

5: Numerical and Combinatorial Optimization

Conjugate Gradient Methods, Quasi-Newton Methods, Constrained Optimization, Linear Programming, Nonlinear Constrained optimization, Quadratic Programming, Integer Programming, Knapsack problem, travelling salesman, vehicle routing, job shop scheduling, Gradient/Stochastic Descents and Primal-Dual methods.

In this course, DankoNikolic brain scientists and AI inventor explains the fundamental of intelligence needed for everyone interested in creating ambitious AI solutions. What you do when things get tough? In the course, you will learn the differences between machine intelligence and human intelligence. You will understand why AI fails and when. AI does not have a narrowly limited working memory (a.k.a., short-term memory) but we humans do. How does our working memory make us more intelligent than machines? Why do we understand the world and machines don't? You will also learn fundamental theorems for machine learning and see how they apply to machine intelligence and human intelligence. After having learned that, you will be able to judge whether an ML project is too ambitious or is likely to succeed. You will be able to identify fundamental problems that plagued some of the ambitious AI projects in the past. You will understand why it is nearly impossible for machines to reach human levels of intelligence. Also, you will learn why some of the tricks in machine learning sometimes work and other times not. You will understand why it is so difficult to build self-driving cars.

The course offers fundamentals that you cannot find in any other course or a book. These fundamentals will be invaluable for your future work on ML and AI.

1. This course explores the nature of intelligence, ranging from machines to the biological brain. Information provided in the course is useful when undergoing ambitious projects in machine learning and AI. It will help you avoid pitfalls in those projects.
2. What are the differences between the real brain and machine intelligence and how can you use this knowledge to prevent failures in your work? What are the limits of today's AI technology? How to assess early in your AI project whether it has chances of success?
3. What are the most fundamental mathematical theorems in machine learning and how they are relevant for your everyday work?

1. Data Exploration And Preprocessing

• Basic Plotting of Data
• Outlier Detection
• Dimensionality Reduction: Principal Component Analysis, Multidimensional Scaling
• Data Transformation
• Dealing with Missing Values

2. Feature Engineering

• Feature extraction and feature engineering,
• Feature transformation
• Feature selection
• Grid search
• Automatically create features
• Aggregations and transformations
• Introduction about Feature tools
• Introduction about Entities & Entity Sets, table Relationships, Feature Primitives, Deep Feature synthesis

1. Learning from Data:

• The appeal of learning from examples, Motivational Case Studies, A formal definition of learning, Key Components of Learning, Population vs. Sample, Decision Boundary, Types of data, Typical Issues with Data, Types of Learning
• Learning as search: Instance and Hypothesis Space, Introductions to Search Algorithms, Cost Functions
• Version Spaces/Perceptron/Linear Regression /Nearest Neighbor
• Overfitting/Regularization, Worst case performance: VC dimension, Bias Variance Tradeoff, Non Linear Embedding, Outlier Detection, Minimum description length
• Estimating Accuracy: Train/test split, Cross Validation, Bootstrap Hypothesis testing, Confusion Matrix, Sensitivity and Specificity, Precision and Recall, ROC curves and AUC, MAPE, Kappa Statistic, AIC, BIC
• Data Science Process

Introduction to Bayesian Learning:

• Probability review
• Bayes rule
• Conjugate priors
• Bayesian Inference I (coin flipping)
• Bayesian Inference II (hypothesis testing and summarizing distributions)
• Bayesian Inference III (decision theory)
• Bayesian linear regression
• Bayes classifiers
• Statistical Estimation a. Maximum Likelihood Estimation c. Bayes error rate e. Curse of dimensionality
• Laplace approximation, Naïve Bayes, Introduction to Bayesian Belief Networks, Introduction to Sampling, Gibbs sampling, Logistic regression

1. Classification Algorithms: Decision Trees, Rule Induction, SVM
2. Dealing with Skewed Class Distribution and Cost-based Classification: Resampling
3. Regression: Lasso and Ridge Regression, MARS, OLS, PLS, GLM
4. Survival Analysis: Cox’s Regression, Weibull Distribution, Parametric Survival Models
5. Ensemble Models: Bagging, Boosting, Stacking, Random Forests, XGBoost, GBDT
6. Neural Networks and Introduction to Deep Learning: Multi Layered Perceptron, Back propagation, Convolutional Neural Networks, Encode-Decoders, Recurrent Neural Networks, Long Short Term Memory (LSTM)
7. Time Series Forecasting: Time Series Decomposition, Holt-Winters, ARIMA, Intermittent Models, Time-frequency domain, Fourier transforms, wavelet transforms, Dynamic Regression Models, Neural Networks, Demand Forecasting

1. Case Study: Network Intrusion detection
2. Case Study: Weather Forecasting
3. Case Study: Image Classification
4. Case Study: Predictive Text Generation
5. Case Study: Customer Lifetime Modelling
6. Case Study: Churn Prediction
7. Case Study: Speech synthesis

In this course, you will learn to analyse and solve competitively such predictive modelling tasks. When you finish this class, you will: Understand how to solve predictive modelling competitions efficiently and learn which of the skills obtained can be applicable to real-world tasks.

1. Preprocess the data and generate new features from various sources text and images.
2. Advance feature engineering techniques Generating mean-encodings, aggregated statistical measures, nearest neighbors as a means to improve your predictions.
3. Cross-validation methodologies benchmark your solutions.
4. Analyzing and interpreting the data. Inconsistencies, high noise levels, errors and other data-related issues such as leakages.
5. Efficiently tune hyperparameters of Algorithms and achieve top performance.
6. Master the art of combining different machine learning models and learn how to ensemble.
7. Get exposed to past (winning) solutions and codes and learn how to read them.

1. Clustering Methods: Expectation Maximization, K-means, k-medoids, Agglomerative and Divisive Hierarchical clustering, Birch, DBScan, Spectral Clustering, Self-organizing maps
2. Community detection in graphs
3. Association Rules and Sequence Pattern Discovery
4. Deviation Detection
5. Semi-supervised and Active Learning
6. Sequential Data Models: Markov Models and Hidden Markov Models, Kalman Filters
7. Model Selection: Model Comparisons, Analysis Considerations

1. Case Study: Viral Marketing
2. Case Study: Driving for Fuel Efficiency

1. Expectation Maximization algorithm
2. Probit regression
3. Expectation Maximization to variational inference
4. Variational inference
5. Finding optimal distributions
6. Exponential families
7. Conjugate exponential family models
8. Scalable inference
9. Bayesian nonparametric clustering
10. Markov Models, Hidden Markov models
11. Conditional Random Fields
12. Monte Carlo, Sampling, Rejection Sampling
13. Poisson matrix factorization
14. Decision Networks
15. Bayesian Optimization
16. Bayesian State-Space Models and Kalman Filtering
17. Probabilistic Numerics and Bayesian Quadrature

1. Case Study: Named Entity Extraction
2. Case Study: Car navigation
3. Case Study: Diagnosis