Progressive Deep Learning with Keras in Practice

This video provides an overview of the entire course.

Before going into the concepts of deep learning, it is important we understand what deep learning is.

• Introduce AI and deep learning

• Understand how deep learning became so popular

• Learn the main successes of deep learning till date

Like all other technologies, Machine Learning also has some basics concepts which are important.

• Understand linear regression, cost function, classification

• Learn about overfitting and underfitting

• Understand hyper parameters, and cross validation

In this video, we will learn essential concepts of fully connected architecture.

• Define neural networks

• Show empirical explanation of how neural network works

• Demonstrate how neural network sees an image

Optimization of models is very important and this video discusses about it.

• Learn about gradients and optimization

• Build our own first optimizer

• Use gradients to create a more robust optimizer

To get going with Keras, the first step would be to set up a GPU in an Amazon instance or a computer, and install Keras.

• Set up the environment by updating packages, installing Python, NVidia driver

• Clone GitHub master branch

• Install Keras

What is a model in Keras and how do we design, train, evaluate and predict a model?

• Define a model in Keras with various operations and core layers

• Learn to implement loss functions and optimizers

• Explore few important activation-functions in Keras

In this video, as the name suggests, we will go through the designing, training and visualization phase with Keras.

• Design a simple neural network in Keras

• Learn the important steps to train a network

• Explore Hualos, a visualization tool to help train your model

Effective regularization is important for proper functioning of our models with real inputs. This video deals with some regularization techniques.

• Discuss ‘Early stopping’ technique

• Introduce the popular regularization technique - ‘Dropout’

• Get familiar with two other regularization techniques, ‘Batch Normalization’ and ‘L1,L2 Regularization’

Computer vision was the first field revolutionized by deep learning.

• Discover computer vision

• Learn about ImageNet dataset

• Explore the main challenges of computer vision

Convolution networks are very good at image classification.

• Learn about the capabilities of convolutional networks

• Understand how convolutional networks work

In this video, we will learn about CNN architectures and research trends.

• Learn about the most common CNN architectures in the literature

• Follow the trend towards extremely deep networks

In this video, we will implement convolutional networks in Keras.

• Know that MNIST is a very popular database for training an image classification system

• See how it works on the MNIST dataset

This video discusses what segmentation in deep learning is

• Introduction to segmentation

• See an example that uses image segmentation task

This video introduces recurrent network and sequential data

• Understand the concepts of image captioning, sentiment analysis, machine translation

• Explore different types of recurrent networks

What makes neural network recurrent?

• Define recurrent neural network

• See an example of a recurrent network in Keras

What is one to many architecture?

• Define one to many architecture in Keras

• Go through an image captioning demonstration

What is a many to one architecture?

• Define many to one architecture in Keras

• Run a sentiment analysis

What is a many to many architecture?

• Define many to many architecture in Keras

• Implement a text generation task in Keras

Embedding layer is widely used in deep learning.

• Go through a famous algorithm word to vector

• Explore how this algorithm is used in embedding layers of Keras

We get to know what is behind the term recommender system and its applications.

• Define recommendation system

• Introduce the concepts of collaborative filtering and item-based filtering

• Contrast between collaborative filtering and content based filtering

Here we further extend our knowledge of collaborative filtering with a Keras example.

• Predict similar movies based on posters in a Keras example with content based filtering

In this video we go through user based collaborative filtering with a Keras example.

• Try to predict the user rating for a specific movie, based on collaborative filtering

In this video we will talk about hybrid systems and extend the Keras example from the previous video.

• Define Hybrid systems

• Predict user rating based on collaborative and content based filtering (hybrid)

This video is about neural style transfer and how it works. We also cover a brief overview of the optimization phase.

• Define neural style transfer

• Explain how style transfer works

• Explain how the optimization phase works

This video is a hands-on demonstration of style transfer.

• Go through the most popular style transfer framework written in Keras

• Going through the installation of the framework

• Demonstrate style transfer from the command line

In this video, we will go through some of the advanced techniques of style transfer.

• Define concepts like color preservation, style interpolation to only name a few

• Get hands-on with these techniques

This video explains style transfer and its underlying mechanism in depth.

• Implement style transfer step by step

• Get a deep understanding on how style transfer networks really work, especially from a backend perspective

In this video, we discuss data augmentation and its main features.

• Define data augmentation

• Go through data augmentation techniques

• Go through the official Keras image augmentation API

In this video, we will discuss transfer learning and its applications.

• Define transfer learning

• Understand why knowledge transfer is important

• Get an overview of how transfer learning takes place

We have used the term hyper parameter search a number of times in the past videos; here we formally introduce the term and how to effectively use it in Keras

• Define the most common algorithms to search the best hyper parameters

• Go through a Keras Hyper Parameter Search API

This video explains what natural language is processing and what purpose does it solve.

• Define Natural Language Processing (NLP)

• Learn on a research example where NLP was successfully applied

This video is an introduction to GAN.

• Define GAN

• Learn the new possibilities that GAN offer

• Discover some successful applications of GANs

This video demonstrates how to train and run a basic generative adversarial network.

• Get an overview of the Keras GAN framework

• Learn how to build, train and run a generative adversarial network

This video explains what a DCGAN is, and why is it so popular.

• Define DCGAN

• Understand its underlying structure, based on convolutions and de-convolutions

• Explore de-convolution and how it differs from convolution

GANs are still at a research stage and so it is important that we learn empirical techniques to make GANs work better.

• Learn tricks that may help run GANs better

• Track failure early during training

• Understand the problems faced by GANs

This video provides an overview of the entire course.

Get acquainted the basics of Jupyter Notebooks, required for the course.

• Learn how the cells are executed

• Learn how we can use Jupyter Notebooks to save time when creating deep learning models

Understand data shapes and data reshaping.

• Explore the mental concepts of the different types of data shapes

• Learn how to reshape data

• Learn how to check the current shape of our data

Learn the concepts and applications of deep learning, perceptron and artificial neural networks, and how they are implemented in Keras.

• Get a general idea of what deep learning is

• Learn about perceptions

• Learn about artificial neural networks

Learn about fully-connected layers and activation functions. Learn how to apply them in Keras.

• Add a dense layer to the model

• Choose number of neuron units and kernel initializer

• Choose an activation function

Learn about gradient descent, backprogation, loss functions and optimizers. Learn how to apply them with Keras.

• Use the model’s compile() method

• Choose a loss functions

• Choose and optimizers and optionally specify a learning rate

Improve our basic neural network model.

• Increase the number of neurons

• Add more hidden layers

• Use an adaptive learning rate optimizer

Learn how to use our data to train the neural network model. Learn about underfitting and overfitting.

• Provide our training data in the fit() method

• Choose number of epochs and batch size

• Choose validation set percentage

Learn how to test our neural network’s performance.

• Choose metrics in the compile() method

• Provide our test data in the evaluate() method

• Check metrics from the method’s returned variable

Learn which metrics can be useful for our model and how to further improve performance.

• Try changing various hyperparameters

• Check validation loss each time and adjust the model

• Check the training loss

Explore the concepts and applications of convolutional neural networks.

• Learn the concepts of convolutional neural networks

• Learn the applications of convolutional neural networks

• Learn how convolutional layers work

Get familiar with filters, strides, padding and pooling layers.

• Understand filters, strides, padding

• Understand pooling layers

• Learn how to implement about filters, strides, padding and pooling layers in Keras

Build a basic CNN model in Keras.

• Add convolutional layers

• Choose number of filters, strides and padding

• Add pooling layers

Check out how Leaky Rectified Linear Units work and how to use them in Keras.

• Understand Leaky Rectified Linear Units

• Learn to use them in Keras

Know how dropout works and why it is useful.

• Understand dropout

• Understand why we want to use dropout

• Learn to implement dropout in Keras

Build an advanced, more complex CNN model in Keras.

• Add a few convolutional layers with increasing number of filters

• Add pooling layers

• Add leaky ReLU activation functions and dropout

Train our CNN model.

• Use the train method

• Provide training data and training labels, possibly set validation split

• Set optimizer, loss function, number of epochs and batch size

Learn how to test our model and what accuracy measures.

• Use the evaluate method

• Provide test data and labels

• Check the accuracy

Know why transfer learning is useful, its use cases and how to do it in Keras.

• Select a pretrained model and load it in Keras

• Build a model on top of it and train it, or freeze early layers and train the chosen transfer learning model

• Check the results and test different hyperparameters to improve them

Learn how autoencoders work and some of their more common applications.

• Get a general idea about autoencoders

• Learn some of their properties

• Get acquainted with some autoencoder applications

Build a basic autoencoder model in Keras.

• Add input and encoding layers

• Add decoding layer

• Create and compile the model

Build a deep autoencoder in Keras.

• Add encoding layers with decreasing number of units

• Add decoding layers with the same number of units, but in reverse

• Create and compile the model

Build a deep convolutional autoencoder for image denoising in Keras.

• Add convolutional layers, followed by pooling layers in the encoder

• Add convolutional layers, followed by upsampling layers in the decoder.

• Use the same number of feature in the decoder as in the encoder, but in reverse

Train our network to denoise images.

• Add noise to the training images

• Use the fit() method providing the noisy and clear images as parameters

• Provide the noisy and clear images as the first and second parameters

Test how well our autoencoder can denoise images.

• Display noise images

• Pass noisy images through the autoencoder with the predict() method

• Display denoised images

Learn the concepts and applications of RNNs, LSTM, and GRU cells.

• Learn about RNNs

• Learn about LSTM

• Learn about GRUs

Learn how to preprocess data for RNN networks.

• Tokenize the data

• Pad the data

• Reshape the labels

Learn how to build a simple RNN model.

• Add a GRU cell

• Add a Time Distributed layer

• Apply a sparse cross entropy loss function

Improve our basic RNN model.

• Add bidirectional layers

• Add a repeat vector

• Add a second cell

Learn how to use our data to train the RNN model.

• Provide our training data in the fit() method

• Provide the max length of our labels as a parameter

• Provide the length of the tokenizer dictionaries as parameters

Learn how to test RNN model’s performance.

• Check the loss and validation loss

• Check the accuracy

• Try testing how the model translates a sentence

Learn the concepts and applications of Generative Adversarial Networks.

• Know how the generator works

• Learn how the discriminator works

• Learn how the GAN model combines the generator and discriminator

Understand why batch normalization is used, how and where to use it.

• Learn the concept of batch normalization

• Learn how to implement batch normalization

• Learn where to implement batch normalization

Learn how to build a Convolutional GAN model with Keras.

• Implement the generator model

• Implement the discriminator model

• Combine the generator and discriminator into a GAN model

Learn how to use our data to train the GAN model.

• Train the generator to create images from random, data

• Train the discriminator to distinguish real from fake images

• Train the GAN to create images, which are close enough to the real ones

Learn how to test GAN network.

• Create random input in the correct shape

• Feed it to the GAN network

• Compare the output images to the real ones