Skip to content
Back to search results

Active Learning Federated Learning

Introduced by
Multiple authors (federated active learning emerged ~2020)
Year
2020s
Type
Hybrid paradigm (active querying within distributed training)
Family
Machine learning
Font size

What does it do?

Federated Active Learning combines the annotation-efficiency of active learning with the privacy-preserving decentralization of federated learning.

A shared global model is trained across distributed clients, each of which independently ranks its unlabeled local data and requests labels only for the most informative examples, keeping raw data on-device throughout.

The intuition behind the method

In standard federated learning every client trains on whatever labeled data it already has, while in active learning a central oracle picks which examples to label next. Federated Active Learning merges both ideas: each client scores its own unlabeled pool locally, selects the samples where the model is most uncertain, and sends only those to a labeler. No raw data leaves the client. The global server then aggregates the locally updated models just as in standard federated learning, so the global model improves from the most informative new labels without ever collecting the underlying data centrally.

History

Federated learning was formalized by McMahan et al. in 2017 with the FedAvg algorithm, enabling privacy-preserving decentralized model training. Active learning has roots in the 1990s query-by-committee and uncertainty sampling literature. Their combination — Federated Active Learning — emerged as an active research topic in the early 2020s, driven by the need to reduce labeling costs in healthcare, mobile, and IoT applications where data cannot be centralized. Multiple groups independently proposed frameworks combining local uncertainty estimation with federated aggregation, and the area remains an open research frontier.

When to use it

Use Federated Active Learning when labeled data is scarce across distributed clients, labeling is expensive or requires domain expertise, and raw data must remain on-device for privacy, regulatory, or bandwidth reasons — for example, medical records across hospitals, user data on mobile devices, or IoT sensor streams. It is particularly powerful when client data distributions are heterogeneous (non-IID) and annotation budgets are small. Do not use it when all data can be centralized freely (plain active learning suffices), when clients lack the compute to score their own pool, when label noise is high (uncertainty sampling can exploit noisy labels), or when latency between labeling rounds is a binding constraint.

Limitations and considerations

  • Communication overhead per round is multiplied by the number of clients reporting query selections.
  • Non-IID client distributions can cause the active-learning selection to be biased toward local minima of the global loss.
  • Synchronizing annotation rounds across distributed clients introduces latency and coordination complexity.
  • Evaluating model performance is harder without a centralized labeled test set.
  • The annotation budget allocation across clients requires careful tuning to avoid starving small or minority clients.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Preserves data privacy by keeping raw samples on-device and only sharing model updates.
  • Dramatically reduces labeling cost by querying only the most informative samples per round.
  • Handles heterogeneous client distributions that would mislead a centralized active learner.
  • Scales horizontally — more clients typically improve coverage of the unlabeled space.
  • Compatible with differential privacy and secure aggregation for stronger privacy guarantees.

Applications

  • Medical imaging annotation across hospitals where patient data cannot be shared.
  • On-device personalization for mobile keyboards or speech recognition with minimal user labeling.
  • Industrial IoT anomaly detection where sensor data must remain at edge nodes.
  • Distributed clinical trial monitoring requiring federated, privacy-preserving model updates.
  • Cross-silo NLP fine-tuning across organizations with proprietary corpora.
  • Remote sensing classification where satellite imagery is stored in geographically distributed data centers.

How it works

How it works

1

A global model is initialized and broadcast to all participating clients. This can be a randomly initialized model or a pre-trained checkpoint, and it defines the shared starting point for the round.

2

Each client applies the current global model to its own unlabeled local pool and computes an informativeness score for each sample — for example, prediction entropy, margin sampling, or query-by-committee disagreement — without transmitting the data.

3

Based on the informativeness scores, each client selects a budget of the most uncertain or most representative unlabeled examples to be annotated. The selection strategy balances exploration with the annotation budget available per round.

4

The selected examples (or only their identifiers plus features necessary for labeling) are sent to a human oracle or automated labeler. Labels are returned to the client. This is the only step at which any data may leave the device, and in privacy-sensitive deployments only the oracle receives them.

5

Each client fine-tunes the global model on its newly labeled examples — and optionally on its existing labeled set — using standard local SGD or Adam optimization for a fixed number of local epochs.

6

Clients upload their model updates (gradients or weight deltas, not data) to the server. The server aggregates these updates, typically via FedAvg (weighted averaging), to form the updated global model, which is then broadcast to start the next round.

Common pitfalls

  • Applying a centralized active learning acquisition function directly without accounting for distribution shift between clients — local uncertainty estimates may not reflect global model uncertainty.
  • Ignoring communication costs when designing the query budget: requesting labels for too many samples per client per round can overwhelm annotation pipelines.
  • Treating the global model as converged after too few federation rounds before querying, leading to poorly calibrated uncertainty scores.
  • Neglecting stragglers — slow or intermittently available clients — which can bias aggregation toward faster clients' distributions.
  • Assuming IID client data when applying FedAvg aggregation, causing divergence when client distributions are highly heterogeneous.

Frequently asked questions

How is Federated Active Learning different from standard Federated Learning?

Standard Federated Learning trains only on the labeled data clients already possess. Federated Active Learning adds an active querying step: each client scores its unlabeled pool and selects the most informative examples to be annotated, allowing the global model to improve with far fewer total labels.

Which acquisition function should I use for local scoring?

Entropy sampling (choosing examples with highest predictive entropy) is the most common starting point and works well when the local model is reasonably calibrated. Margin sampling and BALD (Bayesian Active Learning by Disagreement) are good alternatives when uncertainty calibration across the federation is a concern.

Can Federated Active Learning be combined with differential privacy?

Yes. Differential privacy mechanisms such as DP-SGD can be applied during local training, and secure aggregation can be used server-side. The privacy budget must account for both the model updates and any metadata about which samples were selected for querying.

What happens when clients have very different amounts of unlabeled data?

Clients with large unlabeled pools will dominate the query budget if each receives an equal absolute quota. Proportional or adaptive budget allocation — scaling each client's annotation budget by the size or estimated informativeness of its pool — mitigates this imbalance.

How many federation rounds are typically needed before active querying helps?

It depends on initial model quality. Starting with at least a few rounds of standard federated training with available labeled data improves the calibration of uncertainty estimates before switching on active querying. Cold-start querying with a random initial model can waste the annotation budget on uninformative samples.

Related methods

Sources

  1. Ro, J. Y., Ali, A., Lin, Z., & Suresh, A. T. (2021). Scaling Federated Learning for Fine-tuning of Large Language Models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP).

    Open publication reference
    Copied.
  2. Federated learning. Wikipedia.

    Open publication reference
    Copied.