Federated Learning
- Introduced by
- McMahan et al.
- Year
- 2017
- Type
- Distributed privacy-preserving machine learning
- Family
- Privacy-preserving analysis
What does it do?
Federated Learning is a distributed machine learning paradigm introduced by McMahan et al. in 2017 in which a global model is trained collaboratively across multiple decentralized clients — such as mobile devices or hospital systems — without ever transferring raw data to a central server.
Each participant computes model updates locally using its private data; only those updates, not the underlying data, are communicated and aggregated by the server to improve the shared model.
The intuition behind the method
Imagine a group of researchers at different hospitals each studying patient records they cannot share due to privacy laws. Instead of pooling records into one database, each hospital trains a local version of the model on its own patients, then mails only a summary of what it learned — not the records themselves — to a coordinator who blends all summaries into one improved model. No patient data ever leaves its source institution, yet the collective intelligence of every hospital is captured.
History
The term and framework were introduced by McMahan et al. at Google in 2017, motivated by the need to train language models on mobile devices without uploading sensitive text. The foundational FedAvg algorithm demonstrated that averaging locally trained weights is both communication-efficient and competitive with centralized training. Subsequent research expanded federated learning to handle non-IID data, add formal differential-privacy guarantees, incorporate secure multi-party computation for aggregation, and apply the paradigm to cross-silo settings such as hospital networks.
When to use it
Use Federated Learning when training data is distributed across many clients and cannot be centralized due to privacy regulations, data-ownership constraints, or communication costs — for example, mobile keyboards, medical imaging across hospitals, or IoT sensor networks. Key assumptions include sufficient local data per client and a shared task across all participants. It is less suitable when client data distributions are highly heterogeneous (non-IID), communication bandwidth is severely limited, or when strong formal privacy guarantees beyond data locality are required (consider combining with differential privacy in that case).
Limitations and considerations
- Convergence can be slow and unstable when client data distributions are highly non-IID.
- Communication overhead across many rounds can be significant on low-bandwidth networks.
- Model updates can still leak information about local data through gradient inversion or membership inference attacks.
- Stragglers and dropped clients complicate synchronous aggregation and can bias the global model.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Preserves data privacy by design: raw data never leaves the client device or institution.
- Scales to millions of heterogeneous edge devices without centralizing storage.
- Reduces data-transfer costs because only model updates, not datasets, are communicated.
- Enables learning from sensitive domains (healthcare, finance) where data sharing is legally restricted.
Applications
- Next-word prediction and keyboard personalization on mobile devices without uploading user text.
- Medical imaging model training across hospitals that cannot share patient records.
- Fraud detection across banks sharing behavioral patterns but not transaction histories.
- Autonomous vehicle perception models trained across a fleet without centralizing sensor logs.
- Wearable health monitoring that learns personal baselines while keeping biometric data on-device.
How it works
How it works
At the start of each communication round the server selects a random subset of available clients and transmits the current global model weights to them. This broadcast ensures that all participating clients begin local computation from a common starting point, enabling coherent aggregation later. Selecting only a subset rather than all clients keeps each round tractable even when thousands of devices are registered.
Formula
w_t \rightarrow \text{subset } S_t \subset \{1,\ldots,N\}Each selected client runs one or more steps of stochastic gradient descent on its private local dataset using the received global weights as a starting point. The client computes the gradient of its local loss function and updates the weights accordingly. Only these updated weights — not the raw gradients or the training data — are prepared for transmission back to the server.
Formula
w_{t+1}^k = w_t - \eta \nabla \mathcal{L}_k(w_t)The server collects updated weights from all participating clients and computes a weighted average, where each client's contribution is proportional to its local dataset size relative to the total data across all participants. This Federated Averaging (FedAvg) step produces a new global model that incorporates information from every client's private data without any raw data leaving the devices.
Formula
w_{t+1} = \sum_{k \in S_t} \frac{n_k}{n} w_{t+1}^kThe aggregated global model is evaluated against a convergence criterion, typically a bound on the expected loss relative to the optimal loss. If the criterion is not yet satisfied, the server increments the round counter and broadcasts the updated global model to a new subset of clients, repeating the cycle until the model converges or a maximum number of rounds is reached.
Formula
\mathbb{E}[\mathcal{L}(w_T)] \leq \mathcal{L}^* + \epsilonCommon pitfalls
- Assuming federated training alone is sufficient for privacy: without additional mechanisms such as differential privacy or secure aggregation, model updates remain vulnerable.
- Ignoring data heterogeneity: applying vanilla FedAvg to severely non-IID data often leads to client drift and poor global model performance.
- Overlooking Byzantine or malicious clients: a small fraction of adversarial participants can corrupt the aggregated model if robust aggregation is not applied.
- Treating communication rounds as free: the cumulative cost of many rounds over slow links can outweigh the benefit of decentralization.
Frequently asked questions
Does Federated Learning guarantee complete privacy?
Not by itself. While raw data never leaves client devices, model updates (gradients) can still leak information through gradient inversion or membership inference attacks. Combining federated learning with differential privacy noise addition and secure aggregation protocols substantially strengthens privacy guarantees, but each layer adds its own accuracy or communication trade-offs.
How does Federated Learning handle non-IID data across clients?
Standard FedAvg can diverge when client data distributions differ sharply from one another — a phenomenon called client drift. Remedies include increasing local batch sizes, reducing local epochs, applying personalized federated learning variants (such as FedProx or SCAFFOLD), or using server-side momentum. The severity depends on the degree of statistical heterogeneity across clients.
How many clients and rounds are typically needed for convergence?
McMahan et al. showed that even with a small fraction of clients sampled per round (e.g., 10 out of 100), FedAvg converges within tens to hundreds of rounds for standard tasks. In practice the number depends on dataset size per client, degree of non-IID-ness, learning rate, and the number of local gradient steps performed before each aggregation.
Related methods
Sources
McMahan, B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. (2017). Communication-efficient learning of deep networks from decentralized data. Artificial Intelligence and Statistics, 1273–1282.
Open publication referenceCopied.