Federated Learning

Name: Federated Learning
Rating: 4.6 (108 reviews)

Federated Learning Using PyTorch

Highest Rated

Created byMohamed Gharibi

Last updated 5/2021

English

What you'll learn

Introduction to Deep Learning and Neural Networks
Introduction to Federated Learning
Build Neural Networks from scratch using PyTorch
Load your datasets in IID, non-IID, and non-IID unbalanced settings
Introduction to PySyft
Federated Learning techniques (FedAvg, FedSGD, FedProx, FedDANE)
Build your custom optimizer using PyTorch
Introduction to Differential Privacy
Implement FedAvg using Differential Privacy
Federated Learning on cloud
Implement FedAvg on cloud

Course content

6 sections • 32 lectures • 7h 56m total length

Welcome0:04
Course Contents4:44
Acknowledgement3:14
Acknowledge the libraries and papers that enable federated learning, including decentralized data and average-based methods, and thank the authors, code repositories, and cloud resources that support this course.
Introduction to Deep Learning41:55
Explore the core deep learning concepts, including neural networks, parameters, weights, bias, activation functions, and forward and backward propagation, with practical notes on gradient descent, mini-batches, normalization, and dropout.
Introduction to Federated Learning32:01

Welcome0:17
Welcome0:21
PyTorch and PySyft installtion2:19
Basic operations on tensors4:25
Building single layer with a single neuron NN using PyTorch6:06
Build a single-layer neural network in PyTorch with one neuron by computing the dot product of inputs and weights, adding bias, and applying the sigmoid activation.
Using matmul and mm in PyTorch3:32
Building a NN using multiple neurons6:23
Introduction to PyTorch Autograd4:06
Explore how PyTorch autograd builds a computational graph, performs forward propagation, then backward propagation to compute gradients and guide loss minimization.
Loading MNIST Dataset using PyTorch9:28
Load the MNIST dataset using PyTorch and torchvision, apply transforms including normalization, configure a data loader with batch size 64, and iterate over images and labels.
Run a single epoch of training using PyTorch13:31
Automate the training phase and run multiple epochs of training7:02
Automate the training phase by looping over multiple epochs and all batches, performing forward propagation, computing loss, and updating weights via backward propagation and optimizer steps.
Inference using PyTorch8:43

Loading MNIST in IID setting25:06
Learn to prepare MNIST data for federated learning in an IID setting by randomly distributing 60k images across clients with non-overlapping indices and a fed data set for training.
Loading MNIST in non-IID setting15:59
Loading MNIST in non-IID and unbalanced setting12:18
Explore how to load MNIST in a non-IID, unbalanced federated setting, allocate class indices to clients, balance distributions, and handle edge cases with random class allocation.
PySyft10:28

Requirements

Python Programming Language

Description

The course starts by introducing you to the main concepts in Neural Networks (NN) and how do they work. Then we will implement a NN from scratch using Pytorch. After that, a quick introduction to Federated Learning architecture. Then, we will start by loading the dataset on the devices in IID, non-IID, and non-IID and unbalanced settings followed by a quick tutorial on PySyft to show you how to send and receive the models and the datasets between the clients and the server.

This course will teach you Federated Learning (FL) by looking at the original papers' techniques and algorithms then implement them line by line. In particular, we will implement FedAvg, FedSGD, FedProx, and FedDANE. You will learn about Differential Privacy (DP) and how to add it to FL, then we will implement FedAvg using DP. In this course, you will learn how to implement FL techniques locally and on the cloud. For the cloud setting, we will use Google Cloud Platform to create and configure all the instances that we will use in our experiments. By the end of this course, you will be able to implement different FL techniques and even build your own optimizer and technique. You will be able to run your experiments locally and on the cloud.

Who this course is for:

Federated Learning enthusiasts

Federated Learning

What you'll learn

Explore related topics

Course content

Introduction5 lectures • 1hr 22min

Introduction to PyTorch12 lectures • 1hr 6min

Setting the environment4 lectures • 1hr 4min