Introduction To GANs

A free video tutorial from Rajeev D. Ratan
Data Scientist, Computer Vision Expert & Electrical Engineer
Rating: 4.3 out of 5Instructor rating
7 courses
70,571 students
Introduction To GANs

Learn more from the full course

Deep Learning Computer Vision™ CNN, OpenCV, YOLO, SSD & GANs

2020 Update with TensorFlow 2.0 Support. Become a Pro at Deep Learning Computer Vision! Includes 20+ Real World Projects

14:01:07 of on-demand video • Updated June 2020

Learn by completing 26 advanced computer vision projects including Emotion, Age & Gender Classification, London Underground Sign Detection, Monkey Breed, Flowers, Fruits , Simpsons Characters and many more!
Advanced Deep Learning Computer Vision Techniques such as Transfer Learning and using pre-trained models (VGG, MobileNet, InceptionV3, ResNet50) on ImageNet and re-create popular CNNs such as AlexNet, LeNet, VGG and U-Net.
Understand how Neural Networks, Convolutional Neural Networks, R-CNNs , SSDs, YOLO & GANs with my easy to follow explanations
Become familiar with other frameworks (PyTorch, Caffe, MXNET, CV APIs), Cloud GPUs and get an overview of the Computer Vision World
How to use the Python library Keras to build complex Deep Learning Networks (using Tensorflow backend)
How to do Neural Style Transfer, DeepDream and use GANs to Age Faces up to 60+
How to create, label, annotate, train your own Image Datasets, perfect for University Projects and Startups
How to use OpenCV with a FREE Optional course with almost 4 hours of video
How to use CNNs like U-Net to perform Image Segmentation which is extremely useful in Medical Imaging application
How to use TensorFlow's Object Detection API and Create A Custom Object Detector in YOLO
Facial Recognition with VGGFace
Use Cloud GPUs on PaperSpace for 100X Speed Increase vs CPU
Build a Computer Vision API and Web App and host it on AWS using an EC2 Instance
English [Auto]
Hi and welcome back to chapter 24.1, where we do an introduction to Gans or also known as Generative adversarial networks. So let's do a quick recap on what we've done so far. We've used pre-trained neural nets such as Vgg, Inception and Resnet to create some awesome, deep dream images like this. And then we did some awesome neural style transfers like this row below. But what else is possible? Remember when applying deep dream or neural style transfer, we needed an input image to manipulate. But what if our network could think up or create something on its own? Hmm. Well, that's the brilliance of Ganz. Ganz Allow us to generate images completely without any input. It basically just uses pre-trained information and thinks up. Yes, to create beautiful images like this. Now, I'm actually just kidding. It's like this. These are actually celebrity faces that, again, has generated based on a data set of celebrity faces. So that's pretty cool. So let's continue. So what else can we do with Gans? Let's look at some cool Gan projects. You can actually turn horses into zebras just like this. Now, this type of Gan project actually does require an input image, but what the Gan is doing, it's actually finding the horse and changing the horse to a zebra based on how it envisions a zebra or a horse to look like. So that's pretty cool. What else can Gans do? We can actually create anime characters based on an anime character datasets. These are all Gan generated anime characters. These weren't drawn by human artists. What about the fake celebrity photos like we saw before? These were done by Nvidia's research group and basically it was done by training itself on a celebrity image dataset. So these are all fake people, by the way, even though they have a resemblance to a lot of actors that I can kind of figure out. Let's see what else Gans can do now. You can actually create a smiling face. This was the original picture. And then we used a Gan to actually create some extreme smiles here. I think the first two here would probably be better. Next we can do super image resolution. What is this? This is basically taking a low resolution image like this and using a Gan to basically interpolate and basically create almost a replica of the original. This is what the Gan generated here. And this is the original pretty close. So you can see this has a lot of potential in image compression. What about pics to pics? This is a very cool gun. You actually take images like segmented images like this and generate an entire city view like this, or with maps or with a building facade. Black and white to color, which is super cool. And these examples here, day to night, drawing to a purse. Wow. And this is we're going to this is going to be our project at the end of this Gan chapter. We are going to age a face, maybe even up to 40 or 50 years. So we take an original picture like this and we pass it through our Gan and we get an old looking perception of us that's going to be at 60 plus. It's going to be a bit scary, but stay tuned. Also going to be quite fun. So let's talk about the history of Gans. Generative adversarial networks were first introduced by Ian Goodfellow. He is also known as the grandfather, but technically he should be the father unless his gan actually made another gan, which would be kind of weird. Anyway, you can read his paper. It's quite awesome. Uh, it was proposed as an alternative to variational auto encoders, which also learn the latent spaces of images which are also used to generate synthetic images. And its aim to create an artificial or realistic artificial image that can be almost indistinguishable from a real one. This is a snapshot. I mean, a screenshot of his paper. I really encourage you to read it. It's quite entertaining. So this is a quote from Richard Feynman What I cannot create, I do not understand and this is actually lends itself perfectly to Gans, because you'll now understand how Gans basically learn what the image dataset is supposed to look like. And that's Richard Feynman, in case you didn't know. Famous physicist from Caltech. So this is again, analogy. My high level intuitive explanation of Gans, also known as analogy. Okay. I really should stop it with those cheesy Gan jokes. Okay, so here's my high level explanation. Imagine there's an ambitious young criminal, this guy who wants to learn to counterfeit money so he can sell it to a mobster who specializes in handling counterfeit money. This guy. So now, at first, our young counterfeiter, he isn't very good. He's making money. That looks like Monopoly money. So our expert mobster. So this guy is an expert who deals with handling counterfeit money, basically tells him, take this back and start over. It doesn't look real at all. So the kid, oh, a young counterfeiter, he says he'll try harder next time. So he goes back and tries to figure out how to make his money better. So slowly he gets better and he even makes a good copy now and then. So the mobster is going to be like, okay, you're getting better. Keep trying. You fooled me. So the kid is like, Ah, I see what he likes. I see what fools him. I'm going to make more of those. And that's basically this one. The real money, not the fake money. So after some time, both our forger, this is this guy and the expert mobster. This guy, they're both getting better at their jobs. He is creating better money that looks legit and he can't. He's even though he's learned, learned to tell which money is fake and real, it's getting harder for him. So he has to learn the skill even more. So he has to learn how to distinguish between this and real money. Even better now. So now he's like, I can't tell the fake money from real. You're quite good. So the kid is basically become an expert. So this is basically the analogy for how Gans work. Gans comprise of two types of networks a generator and a discriminator. And the purpose of the generator network is to basically take a random image initialization and decode it into a synthetic image. The purpose of the discriminator is to take this input and predict whether this image came from a real data set or a synthetic data set. So let's go over this one more time. The generator, who is our counterfeiter? He basically starts off making random money, which is why his early money sucked and tried to decode that into basically an image which was his money. So you just start from scratch and just takes something and says, Is this money? And obviously the discriminator, who is a bit of an expert in the beginning, says, No, this is totally fake. So let's go on. So as we just saw, this is effectively what Gans are two antagonistic networks that are contesting against each other. The two components are called, as we just described, the generator network, and that's our counterfeiter and the discriminator network, which was our mobster. So how would we train our guns now? Training guns is notoriously difficult. Previously, we know gradient descent was used to basically change our weights to reduce our loss. And that was fairly simple to do. Even though it was time consuming, it was a fairly understandable algorithm. However, in Gans there's a problem here. Every weight change changes the entire balance of our system, and that's because it's a dynamic system. You have two networks sort of contesting against each other, so we can't simply just reduce a loss and hope for the best. It's not exactly how it works and you'll see how it works shortly, because this is what I just said in Gans. We're not seeking to minimize the loss, but to find equilibrium between two opposing networks. So now let's take a step by step look on how we actually start training. Gans. The first step is that we input randomly generated noisy images into our generator network, and that is used to generate some sample images. Secondly, we take some of those sample images. Those are these images that were produced from some random randomly generated noise. We mix this with some of our real data, some some of our images from our real dataset. And what do we do with this mix of images that are real and fake? Well, then we pass these images to our discriminator who will then be trained on this mixed dataset and he will update his weights accordingly. So what the discriminator does, he receives these inputs and he knows well, the labels tell it who is real and who is fake. So he now he trains on this using those labels to update his weights accordingly. So this is the part where he gets better at actually figuring out what is fake and what is not not fake. Now, this is the important part. We then make some more fake images and input it into discriminator. However, we label all as real and this is now done. So we train the generator. This time the discriminator learning stops. At this stage we freeze weights here and then we use the feedback from the discriminator now because remember the discriminator, he just learnt a bit more on what's real and what's fake. So now generator input some images into him and the generator is going to figure out from his feedback what are good fake images and what are the bad fake images. And that is how we teach both the generator here and the discriminator to get better up here at spotting fakes. So this is a diagram that helps you figure out how we did this. Here's our latent space and our noise. So we use these here to generate our fake images. Then we mix some of those fake images with our real images in datasets and discriminator. That's him here. He basically gets boats and he tells us whether it's correct or not, whether fake or real, basically. So then this feedback goes to the generator, who does the same thing again? And basically the discriminator, his part is done right here. So he trains here and then he trains here. So hopefully this diagram is clear to you. So this is a summary of our chapter. We understand what Gans are and what their capabilities are, and we've gone through some cool projects on what Gans can actually do. We do understand how Gans intuitively work. Remember, there are antagonistic network of a generator and a discriminator, and then we've just learned on how we train the generator and the discriminator. So hopefully you found this Gan chapter entertaining. I thought it was fantastic. Up next, we're going to learn a bit about the mathematics involved in training Gans and how we actually implement a simple Gan in Keras.