Federated Learning

Learn how to set up an isolated conda environment for a federated learning project, install PyTorch and PySyft, and launch a notebook to begin experimentation.

Explore basic tensor operations in federated learning, including creating tensors, indexing, shape and dimensions, broadcasting, and using a random function to create multidimensional tensors.

Build a single-layer neural network in PyTorch with one neuron by computing the dot product of inputs and weights, adding bias, and applying the sigmoid activation.

Learn how to perform matrix multiplication in PyTorch using mm and matmul, handle broadcasting with shape adjustments, and reshape tensors to align dimensions for efficient GPU or CPU execution.

Learn to build a neural network with multiple neurons by adding weight sets and biases, tracking input and output dimensions, and performing multiple matrix multiplications with an activation function.

Explore how PyTorch autograd builds a computational graph, performs forward propagation, then backward propagation to compute gradients and guide loss minimization.

Load the MNIST dataset using PyTorch and torchvision, apply transforms including normalization, configure a data loader with batch size 64, and iterate over images and labels.

Build a reusable neural network class in PyTorch to perform forward propagation, compute negative log likelihood loss with softmax, and update weights via SGD for a single training epoch.

Automate the training phase by looping over multiple epochs and all batches, performing forward propagation, computing loss, and updating weights via backward propagation and optimizer steps.

Train a PyTorch model for inference, comparing sigmoid and relu activations and reshaping inputs, then perform no-gradient single-image inference to obtain logits, softmax probabilities, and predicted class.

Learn to prepare MNIST data for federated learning in an IID setting by randomly distributing 60k images across clients with non-overlapping indices and a fed data set for training.

Explore loading MNIST in non-iid settings by distributing data across clients by classes, preserving indices with labels using stacking, and comparing local models to a more accurate global model.

Explore how to load MNIST in a non-IID, unbalanced federated setting, allocate class indices to clients, balance distributions, and handle edge cases with random class allocation.

Explore how to build a federated learning setup with PySyft, using virtual clients to simulate data on a server and synchronize global and local models.

Implement federated averaging (fedavg) by loading decentralized data, distributing models to virtual clients, training locally, and averaging weights toward the global model. Explore server-side orchestration, client updates, and weighted averaging.

Explore fedsgd in federated learning by updating global weights via gradient averages from client models, and compare two equivalent update methods.

Tackle federated optimization in heterogeneous networks using FedProx, adding a proximal term to updates to handle system heterogeneity and non-identically distributed data, improving accuracy beyond FedAvg.

Explore FedDANE, a federated learning optimization that adds a gradient correction term to a proximal framework, contrasting with Fed average and Fed approx, and detailing subset selection and update rules.

Discover differential privacy in federated learning, applying FedAvg with gradient clipping and Gaussian noise to bound privacy loss and protect model weights.

Set up a two-machine cloud environment to run federated averaging, with algorithm provider and data owner, static ip, port 8888 firewall, and jupyter notebooks in a Python 3.7 conda environment.

Explore implementing FedAvg on cloud with multiple machines by enabling multi-client training, local model updates, and server-side averaging to update the global model.