Applied Deep Learning & Artificial Intelligence Program

1. Mathematical Foundations of Data Science:

A. Linear Algebra:

1. Vectors, Matrices
2. Tensors
3. Matrix Operations
4. Projections
5. Eigenvalue decomposition of a matrix
6. LU Decomposition
7. QR Decomposition/Factorization
8. Symmetric Matrices
9. Orthogonalization & Orthonormalization
10. Real and Complex Analysis (Sets and Sequences, Topology, Metric Spaces, Single-Valued and Continuous Functions, Limits, Cauchy Kernel, Fourier Transforms)
11. Information Theory (Entropy, Information Gain)
12. Function Spaces and Manifolds
13. Relational Algebra and SQL

B. Multivariate Calculus

1. Differential and Integral Calculus
2. Partial Derivatives
3. Vector-Values Functions
4. Directional Gradient
5. Hessian
6. Jacobian
7. Laplacian and Lagragian Distribution

2: Probability for Data Scientists

1. Probability Theory and Statistics
2. Combinatorics
3. Random Variables
4. Probability Rules & Axioms
5. Bayes' Theorem
6. Variance and Expectation
7. Conditional and Joint Distributions
8. Standard Distributions (Bernoulli, Binomial, Multinomial, Uniform and Gaussian)
9. Moment Generating Functions
10. Maximum Likelihood Estimation (MLE)
11. Prior and Posterior
12. Maximum a Posteriori Estimation (MAP) and Sampling Methods
13. Descriptive Statistics
14. Hypothesis Testing
15. Goodness of Fit
16. Analysis of Variance
17. Correlation
18. Chi2 test
19. Design of Experiments

3: Algorithms and Data Structures:

A. Graph Theory: Basic Concepts and Algorithms
B. Algorithmic Complexity

1. Algorithm Analysis
2. Greedy Algorithms
3. Divide and Conquer and Dynamic Programming

C. Data Structures

1. Array, List, Hashing, Binary Trees, Hashing, Heap, Stack etc
2. Dynamic Programming
3. Randomized & Sublinear Algorithm
4. Graphs

4: R and Python

A. R Programming Language

1. Vectors
2. Matrices
3. Lists
4. Data frame
5. Basic Syntax
6. Basic Statistics
7. Data Manipulation (dplyr)
8. Visualization (ggplot2)
9. Connecting to databases (RJDBC)

Python Programming Language

1. Python language fundamentals
2. Data Structures
3. Beautiful Soup
4. Regular Expressions
5. JSON
6. Restful Web Services (Flask)
7. NumPy
8. Plots in matplotlib, seaborn
9. Pandas

In this course, DankoNikolic brain scientists and AI inventor explains the fundamental of intelligence needed for everyone interested in creating ambitious AI solutions. What you do when things get tough? In the course, you will learn the differences between machine intelligence and human intelligence. You will understand why AI fails and when. AI does not have a narrowly limited working memory (a.k.a., short-term memory) but we humans do. How does our working memory make us more intelligent than machines? Why do we understand the world and machines don't? You will also learn fundamental theorems for machine learning and see how they apply to machine intelligence and human intelligence. After having learned that, you will be able to judge whether an ML project is too ambitious or is likely to succeed. You will be able to identify fundamental problems that plagued some of the ambitious AI projects in the past. You will understand why it is nearly impossible for machines to reach human levels of intelligence. Also, you will learn why some of the tricks in machine learning sometimes work and other times not. You will understand why it is so difficult to build self-driving cars.

The course offers fundamentals that you cannot find in any other course or a book. These fundamentals will be invaluable for your future work on ML and AI.

1. This course explores the nature of intelligence, ranging from machines to the biological brain. Information provided in the course is useful when undergoing ambitious projects in machine learning and AI. It will help you avoid pitfalls in those projects.
2. What are the differences between the real brain and machine intelligence and how can you use this knowledge to prevent failures in your work? What are the limits of today's AI technology? How to assess early in your AI project whether it has chances of success?
3. What are the most fundamental mathematical theorems in machine learning and how they are relevant for your everyday work?

UNIT 1:

1. Introduction to deep learning
2. Course logistics
3. History and cognitive basis of neural computation
4. The perceptron / multi-layer perceptron

UNIT 2:

1. The neural net as a universal approximator

UNIT 3A:

1. Training a neural network
2. Perceptron learning rule
3. Empirical Risk Minimization
4. Optimization by gradient descent

UNIT 3B:

1. Single hidden layered networks and universal approximation
2. Motivation for more than 1 hidden layer
3. Feature engineering versus co-learning features and estimation

UNIT 4:

1. Back propagation
2. Calculus of back propagation

UNIT 5:

1. Convergence in neural networks
2. Rates of convergence
3. Loss surfaces, Learning rates, and optimization methods
4. RMSProp, Adagrad, Momentum

UNIT 6:

1. Stochastic gradient descent
2. Acceleration
3. Overfitting and regularization
4. Tricks of the trade:
5. Choosing a divergence (loss) function
6. Batch normalization
7. Dropout

UNIT 7:

1. Recap of Training Q&A session for students

UNIT 8:

1. Optimization continued

UNIT 9A:

1. Models of vision
2. Neocognitron
3. Mathematical details of CNNs, Alexnet, Inception, VGG

UNIT 9 B:

1. Architecture
2. Convolution, Pooling, Normalization
3. Training strategies
4. Visualizing and understanding convolutional networks
5. Some MORE well-known convolution networks (LeNet/ZFNet/GoogleNet)

UNIT 10:

1. Recurrent Neural Networks (RNNs)
2. Modeling series
3. Back propogation through time
4. Bidirectional RNNs

UNIT 11:

1. Stability
2. Exploding/vanishing gradients
3. Long Short-Term Memory Units (LSTMs) and variants
4. Resnets

UNIT 12:

1. Loss functions for recurrent networks
2. Sequence Prediction

UNIT 13:

1. Sequence To Sequence Methods
2. Connectionist Temporal Classification (CTC)

UNIT 14:

1. Sequence-to-sequence models
2. Attention models, examples from speech and language

UNIT 15:

1. What to networks represent
2. Autoencoders and dimensionality reduction
3. Learning representations

UNIT 16:

1. Variational Autoencoders (VAEs)

UNIT 17:

1. Generative Adversarial Networks (GANs) Part 1

UNIT 18:

1. Generative Adversarial Networks (GANs) Part 2

UNIT 19:

1. Regularization

UNIT 20:

1. Transfer Learning

UNIT 21:

1. Hopfield Networks
2. Boltzmann Machines

UNIT 22:

1. Training Hopfield Networks
2. Stochastic Hopfield Networks

UNIT 23:

1. Restricted Boltzmann Machines
2. Deep Boltzmann Machines

UNIT 24:

1. Reinforcement Learning 1

UNIT 25:

1. Reinforcement Learning 2

UNIT 26:

1. Reinforcement Learning 3

UNIT 27:

1. Reinforcement Learning 4

UNIT 28:

1. Q Learning Deep Q Learning

1. Case Study:Computer Vision (auto-colorization of b/w images; attention)
2. Case Study:Natural Language Processing (caption generation, Word2Vec)

1. Image Processing:
   • Projects: Medical image analysis -- pathology, tracking suspicious movement for security.
2. Natural Language Processing:
   • Projects: Spam detector, Q&A/Conversational systems.
3. Speech:
   • Project: Conversational systems.
4. Deep Learning in Cyber Security