Deep Learning Pipeline

-Example – Titanic

o Text

-Example – Classifying IMDB Movie Reviews o Images
-Type of images (2D, 3D, 4D)
-Example – CIFAR-10

· Quick recap and check
· Summary

Chapter 5:​ Data Wrangling and Preprocessing
After understanding the data, readers now choose the approaches and methodologies for preparing it.

-The deep learning pipeline revisited
-Data loading and preprocessing

o Data Loading with Numpy
o Data Loading with Pandas

-Missing and Noisy Data
-Dealing with big datasets
-Accessing other data formats
-Data preprocessing
-Data augmentation
-Categorical to Text data
-Methods can not use in text data
-Tokenization, Steaming, and Stop-words
-The Bag-of-Words Model (BoW)

o CountVectorizer and Term Frequency-Inverse Document Frequency (TF-IDF)

-Summary



Chapter 6:​ Data Resampling
After cleaning and preparing the dataset, the reader should know how to sample this dataset in the right way. Choosing wrong samples from your data may influence the result of your models. So in this chapter, we will illustrate all techniques and approaches needed to sample your dataset in the right way.

-Creating training and test sets
-Cross-Validation

o Validation set technique
oLeave-One-Out Cross-Validation (LOOCV) o K-Fold Cross-Validation

-Bootstrap
-Case Study -- Generators (modify)
-Summary



Chapter 7:​ Feature Selection and Feature Engineering
In this chapter, we will describe a very important topic in the pipeline, which is feature selection and engineering. The audience should know how to select and choose the most important input feature that contributes most to your output feature. We will cover the process of using domain knowledge of the data to create features that make machine learning algorithms work. And also feature selection and engineering, which are fundamental to the application of machine and deep learning.
-Datasets used in this chapter
-Feature selection and filtering
-Principal component analysis
o Non-Negative Matrix Factorization (NMF) o Sparse PCA
oKernel PCA

-ZCA Whitening
-Topic Model and Latent Dirichlet Allocation (LDA)
-Atom extraction and dictionary learning
-Summary



Part Three: ​TensorFlow
In this part, first we illustrate the fundamental concepts of deep learning. Then we deep dive to the core of neural networks and the types of neural networks, describing each type and showing the important concepts of the equations of deep learning. Finally we present real-life examples of each type.



Chapter 8:​ Deep Learning Fundamentals
In this chapter, we will describe the basic functions that deep learning is built on. Then we will try to build layers from these functions, and combine these layers together to get more complex models that will help us solve more complex problems. All that will be described by TensorFlow examples.

-Perceptron
oSingle Perceptron

o Multi Perceptron

-Difference neural network layers

o input layer
o hidden layer
o output layer
▪sigmoid
▪Softmax

-Shallow vs. Deep Neural Networks
-Activation Functions
-Gradient Descent
-Loss Functions and Backpropagation
-Vanishing and Exploding Gradients
-TensorFlow Basics

o Placeholder vs. Variable vs. Constant
oGradient-Descent Optimization Methods from a Deep-Learning Perspective
o Learning Rate in Mini-batch Approach to Stochastic Gradient Descent
oOptimizers in TensorFlow

-Full-Batch vs. Stochastic vs. Mini-Batch Gradient Descent
-Summary



Chapter 9:​ Improving Deep Neural Networks
After building the deep learning model, the improvement starts. This chapter covers optimization, tuning, and choosing hyperparameter techniques. The reader will also learn to evaluate, optimize, and tune their models parameters to reach optimal solutions and accuracy.

-Gradient-Descent Optimization Methods
-Choosing the Learning Rate
-Dropout Layers and Regularization
-Optimizers in TensorFlow
-Normalization Techniques

o Batch Normalization
o Weight Normalization
o Group Normalization
oInstance Normalization
o Layer Normalization

-● Summary



Chapter 10:​ Convolutional Neural Networks
One of the important classes of deep learning is the convolutional neural network. In this chapter, we will illustrate everything about CNN from the one-dimensional mask to the advanced stuff like weight sharing and the difference between equivariance and invariance. We will illustrate a case study using the famous dataset CIFAR-10
● What is ConvNet

-Convolution Operation

o 1D Convolution
o 2D Convolution

-Two-dimensional Unit Step Function
-Common Image-Processing Filters

o Mean and Median Filters
oGaussian Filter
oGradient-based Filters
oSobel Edge-Detection Filter o Identity Transform

-Convolution Neural Networks
-Layers of Convolution Neural Networks

o Input Layer
oConvolution Layer oPooling Layer

Backpropagation Through the Convolutional and Pooling Layer -Weight Sharing Through Convolution and Its Advantages
-Translation Equivariance and Invariance
-Case Study -- Digit Recognition on the CIFAR-10 Dataset
-Summary



Chapter 11:​ Sequential Models
Another class of deep learning is the sequential models. In this chapter, we will describe the problem of sequential data and the rise of recurrent neural networks as well as the problem and also the evolution of the GRU and LSTM.

-Recurrent Neural Networks (RNN)
-Language Modeling
-Backpropagation Through Time (BPTT)
-Case Study -- Predicting the Next Word in a Sentence
-Vanishing and Exploding Gradient Problem in RNN
-Solution to Vanishing and Exploding Gradients Problem in RNNs
-Long Short-Term Memory (LSTM)
-Exploding and Vanishing Gradient Reduction in LSTM
-Case Study -- Digit Identification on the MNIST Dataset
-Gated Recurrent Unit (GRU)
-Bidirectional RNN (Bi-RNN)
-Summary



Part Four:​ Applications and Appendix
We will give the readers this part to make sure that they practice by using TensorFlow and build the pipeline.



Chapter 12:​ Selected Topics in Computer Vision
After finishing CNN in part three, it’s good to add some extra knowledge that makes it easier for readers when they work, like using pre-built architectures and transfer learning.

-● Different Architectures in Convolutional Neural Networks

o LeNet
o AlexNet
o VGG16
o ResNet

-Transfer Learning
-Guidelines for Using Transfer Learning

o InceptionV3
o Pre-trained VGG16
● Summary



Chapter 13:​ Selected Topics in Natural Language Processing
In this chapter, we fill the gaps that the audience needs for working with text. Giving the readers all the advanced approaches and techniques of natural language processing.

-Vector Space Model
Vector Representation of Words -Word2Vec

o Continuous Bag of Words (CBOW)
oSkip-Gram Model for Word Embedding
o GloVe
● Summary



Chapter 14:​ Model Deployment
Here we will show the audience how model deployment works. It’s one last important step needed to communicate with others such as stakeholders and decision makers.

-Introduction to Model Deployment.
-Type and Architecture of deployments.
-Introduction to Flask.

-Installing Flask
-Building a small REST Service

-Real-life train and test.
-Case Study -- tiny predictors
-Summary



Chapter 15:​ Applications
Here we will show some case studies to make sure that the reader gets the full knowledge and understanding of how to build pipelines with real-life examples.

-Case Study -- Tabular Datasets
-Case Study -- IMDB Movie Review Data with Paragraph2Vec
-Case Study -- Image Segmentation
-Conclusion