Federated Learning: Theory and Practical

Explain how traditional programming uses rules to map inputs to outputs, while machine learning uses a trained model to approximate f(x). Focus on supervised learning and neural networks.

Explore the neural network architecture, detailing input, hidden, and output layers, with neurons per layer tied to input features and output classes, and discuss trade-offs of depth.

Understand neural network model parameters as weight matrices and bias vectors across layers that map inputs to outputs, with dimensions tied to layer sizes and training objectives to optimize them.

Train a neural network by forward propagation to predict outputs, compute loss against actual values, and use backward propagation to optimize weights and biases until the loss is minimized.

Learn how forward propagation combines linear matrix multiplications and nonlinear activations like ReLU and sigmoid across a four-layer neural network to produce predictions and guide loss-based parameter optimization.

Compute the loss between predicted outputs and actual outputs using the last-layer sigmoid output a3. Apply binary cross-entropy loss to guide loss minimization in binary classification.

Learn how gradient descent optimizes model parameters by computing the loss gradient and updating theta, weights, and biases with a learning rate alpha.

Explore backward propagation from the output layer to the first layer, deriving partial derivatives with sigmoid activation and binary cross-entropy loss to update parameters like w3 and b3.

Recap of neural network training steps: define architecture, initialize parameters, and forward propagate. Then compute accuracy and loss, backpropagate gradients, and update parameters across iterations to minimize loss in Python.

Federated learning trains a global model across distributed clients by exchanging only model parameters, preserving privacy, as local models are averaged to update the global model.

simulate federated learning on a single machine, training a four-layer neural network with bank note data across ten clients, using per-client csv files and average parameter aggregation.

Explore horizontal and vertical federated learning, defined by data partitioning across clients, where horizontal uses same features with different patients and vertical uses same patients with different features.

We explore horizontal data partitioning and the horizontal federated learning algorithm, with a server initializing a global model and hospitals sharing identical features.

Explore vertical data partitioning where different clients hold features for the same patients, with the output feature y, and learn how the vertical federated learning algorithm coordinates partial neural-network computations.

Explore vertical federated learning theory, where multiple clients compute the first hidden layer activations z1 from their local features, share partial derivatives, and achieve centralized-like results.

Implement vertical federated learning in Python distributing four input features and one binary output across two clients and a server, using banknote authentication data 1372 samples.

Present the combined data partitioning in federated learning with horizontal and vertical partitioning across hospitals and a registry in Australia, and outline the two cases by output feature location.

Explore federated learning with a combined data partitioning scenario, where horizontal and vertical clients train locally, exchange z1 and ds1 updates, and aggregate to improve the global model.

Explore a practical federated learning setup using banknote authentication data, distributing data across ten horizontal and one vertical client, training a neural network through CSV-based parameter sharing and global averaging.

explains how a vertical client centric federated learning case operates, where horizontal clients exchange z1 matrices with a server acting vertical client, concatenate them, and update and average local weights.

Explore a combined federated learning setup that distributes banknote authentication data across ten horizontal clients and a server, detailing z1 computation, concatenation, and gradient updates.

Analyze the performance of horizontal and vertical federated learning, comparing accuracy with centralized models and exploring non-iid data effects. Examine privacy-accuracy tradeoffs in a split neural network with cut-off layers.

This lecture compares the accuracy of two combined federated learning algorithms with centralized machine learning, detailing horizontal and vertical client setups and their impact on non-iid data and communication time.

Evaluate the impact of communication time in federated learning, comparing horizontal and vertical flows, noting exchanges of Z1 and DS1 matrices, rounds to converge, and practical efficiency concerns.

Explore how federated learning's gradients can reveal client data through gradient inversion attacks, challenging the assumption of inherent privacy. Learn why strong privacy measures and defense strategies are essential.

Explore gradient inversion attacks and how differential privacy, quantization, and compression defend privacy in federated learning, balancing privacy with model accuracy.

Demonstrates additive homomorphic encryption in federated learning, where clients encrypt gradients, the server aggregates encrypted gradients without decrypting, and clients decrypt to obtain the average gradients.

Explore homomorphic encryption in federated learning by implementing key generation, encryption, decryption, and homomorphic addition to securely aggregate client gradients while balancing privacy and performance.

Select participating clients in horizontal, vertical, and combined federated learning based on resources and data quality; balance differential privacy impacts to improve model accuracy.

Explore aggregation methods in federated learning, from simple and weighted averages to quantized gradients and hierarchical aggregation, and learn deployment considerations, data preprocessing, and normalization strategies.