LLM on a Budget: How Federal Reserve Researchers Solved the $10,000 Classification Problem With M-RARU Active Learning

The $10,000 Classification Problem

Large Language Models have transformed text classification, achieving unprecedented accuracy in understanding nuanced language and context. But there’s a catch that prevents widespread adoption: cost. When you need to classify hundreds of thousands of documents—a routine requirement for financial institutions—LLM API calls quickly escalate into prohibitive expenses.

Consider a typical scenario at the Federal Reserve: analyzing 125,000 public comments on regulatory proposals. Using GPT-4 or Claude to classify each document could easily cost $10,000 or more in API fees. Multiply this across multiple analysis projects, and the expense becomes unsustainable for even well-funded institutions.

This creates a fundamental tension in AI deployment. Large language models deliver superior accuracy but carry prohibitive computational and financial costs. Traditional machine learning models like Random Forest or SVM are efficient and interpretable—crucial for regulated industries—but lack the nuanced language understanding that makes LLMs so powerful.

Federal Reserve researchers tackled this exact problem, asking: How can we capture LLM intelligence while maintaining the efficiency and interpretability of traditional models? Their answer: a revolutionary approach called M-RARU that achieves up to 80% cost reduction while preserving classification performance.

Knowledge Distillation — Teaching Small Models to Think Like LLMs

The foundation of the Fed’s approach rests on knowledge distillation, a machine learning technique where an expensive “teacher” model trains a more efficient “student” model. Think of it as an expert professor teaching a class: the professor (LLM) has deep knowledge, but can’t teach every student individually. Instead, the professor labels carefully chosen examples that the class (traditional ML model) learns from.

In practice, this means using an LLM to label a subset of your data, then training a traditional classifier on those labels. The student model learns to mimic the teacher’s decision-making patterns while operating at a fraction of the cost. For financial institutions, this is particularly valuable because traditional models support explainability techniques like SHAP and LIME—essential for regulatory compliance.

However, knowledge distillation still faces a critical bottleneck: the teacher must label thousands of samples to train an effective student. For large datasets, this process alone can cost thousands of dollars and require substantial time. The breakthrough insight was realizing that not all data points are equally valuable for training.

Active Learning Meets Knowledge Distillation

Active learning operates on a simple but powerful principle: not all data points are equally informative for training. Rather than randomly selecting samples for the teacher to label, an intelligent system identifies and prioritizes the most valuable unlabeled examples—those that will teach the student model the most.

The Federal Reserve team combined active learning with knowledge distillation in an innovative hybrid framework. Their system converts text documents into numerical embeddings using efficient transformer models, then builds curated training sets by iteratively selecting high-value samples for LLM labeling.

Here’s how the process works:

• Start with a small randomly labeled seed set to train an initial student model
• Use the student model to predict uncertainty scores for all unlabeled documents
• Select the most uncertain predictions for expensive LLM labeling
• Add the newly labeled data to the training set and retrain the student
• Repeat until performance targets are met

This approach replaces passive random sampling with intelligent, uncertainty-driven data selection. Instead of labeling 10,000 random documents, the system might achieve equivalent performance by strategically labeling just 2,000 carefully chosen examples.

M-RARU — The Technical Innovation Behind 80% Cost Reduction

The heart of the Fed’s breakthrough is M-RARU: Multi-class Randomized Accept/Reject Uncertainty Sampling. While that’s a mouthful, the concept elegantly solves two critical problems with traditional active learning: computational scalability and selection bias.

Traditional uncertainty sampling examines every unlabeled document to find the most uncertain examples—a computationally expensive process that becomes impractical with datasets containing hundreds of thousands of items. M-RARU introduces a randomized acceptance mechanism that eliminates this exhaustive search.

The technical innovation centers on acceptance probability: p(accepted) = 1 − max_k Pr(C_k|x). In plain English, documents with low confidence (high uncertainty) are more likely to be selected for labeling. A document the model predicts with 95% confidence has only a 5% chance of selection, while one predicted with 60% confidence has a 40% chance.

The randomization component serves multiple purposes:

• Eliminates costly sorting and ranking of entire unlabeled datasets
• Maintains exploration-exploitation balance to avoid bias toward noisy regions
• Scales to datasets of any size with constant-time selection decisions
• Extends seamlessly from binary to multi-class classification tasks

This combination of uncertainty-based selection with efficient randomized sampling delivers remarkable performance gains. The researchers achieved up to 1,788× speedup over exhaustive uncertainty sampling while maintaining superior sample efficiency.

Experimental Design — Five Models, Two Datasets

The Fed team validated M-RARU across diverse configurations to ensure robust real-world applicability. They tested five different student models representing the spectrum of traditional machine learning approaches:

• Support Vector Machine (SVM): Linear classifier with geometric decision boundaries
• Linear Discriminant Analysis (LDA): Probabilistic linear model
• Random Forest (RF): Ensemble of decision trees
• Gradient Boosted Decision Trees (GBDT): Sequential boosting approach
• DistilBERT: Lightweight transformer as neural baseline

Two real-world datasets provided the testing ground:

Federal Reserve Public Comments (125,179 documents, 5 classes): Public responses to Fed announcements and regulations, classified as Banks/Trades, Consumer, Government, General Public, or Other. This dataset represents the exact use case driving the research.

GNAD Financial News Headlines (12,288 documents, 3 classes): German financial news headlines classified for economic trend analysis—rising, falling, or flat GDP signals. This dataset tests cross-domain and multilingual applicability.

The experimental setup used locally deployed Gemma-3-27b as the teacher model to ensure consistent, controlled conditions. Text embeddings came from all-MiniLM-L6-v2, producing 384-dimensional representations that capture semantic similarity while maintaining computational efficiency.

Results — Up to 80% Reduction in Labeling Requirements

The results exceeded expectations, with M-RARU consistently outperforming random sampling across all model-dataset combinations. The improvements weren’t marginal—they represented game-changing cost reductions that make LLM-powered classification economically viable at scale.

Gradient Boosted Decision Trees delivered the most impressive results: On the Public Comments dataset, GBDT achieved 71% sample reduction to reach 90% accuracy, requiring only 1,825 samples compared to 6,275+ samples with random sampling. This translates directly to cost savings—what previously required $6,000+ in API calls now costs under $2,000.

Consistent improvements across model types: SVM achieved 58% sample reduction, LDA reached 64% reduction, and Random Forest delivered similar gains. Even DistilBERT, despite architectural limitations, showed 10-20% improvements.

Balanced accuracy gains were even more pronounced: For balanced accuracy metrics that account for class imbalances, Random Forest achieved 81% sample reduction to reach 80% performance thresholds. This is crucial for real-world datasets where some classes are naturally rare.

The GNAD dataset results were particularly striking. M-RARU enabled models to reach performance thresholds that random sampling couldn’t achieve within practical budget constraints. This suggests the approach becomes more valuable as task complexity increases.

Model-Specific Performance — Why Uncertainty Quality Matters

The research revealed an important insight: M-RARU’s effectiveness depends heavily on how well each model architecture estimates prediction uncertainty. This explains why some models benefited more than others from the active learning approach.

Tree-based models performed best because ensemble methods like Random Forest and GBDT naturally produce well-calibrated uncertainty estimates through voting mechanisms. When trees disagree about a prediction, the model correctly identifies genuine uncertainty that benefits from additional labeled examples.

Linear models showed substantial improvements with 50-70% sample reductions. SVM and LDA uncertainty estimation relies on geometric interpretations—distance from decision boundaries and probabilistic confidence intervals—which provide reasonably reliable uncertainty signals for active learning.

Neural networks underperformed because models like DistilBERT tend to produce overconfident softmax outputs that don’t accurately reflect true prediction uncertainty. This is a well-known limitation of neural networks that affects their compatibility with uncertainty-based active learning.

This analysis has practical implications for organizations implementing similar systems. Choose your student model architecture based not just on accuracy, but also on how well it estimates uncertainty. Tree-based ensembles offer the best combination of interpretability, efficiency, and active learning compatibility.

Computational Efficiency — Speed and Cost Multipliers

Beyond sample efficiency, the Fed’s approach delivers remarkable computational performance gains. These speed improvements enable rapid experimentation and hyperparameter tuning that would be impractical with slower alternatives.

Training and inference speed multipliers:

• GBDT: 44× training speedup, 35× inference speedup vs. DistilBERT
• Random Forest: 23× faster inference
• LDA: 13× faster inference
• SVM: 5× faster inference

M-RARU’s accept/reject mechanism eliminates the computational bottleneck of exhaustive uncertainty sampling, delivering up to 1,788× speedup over traditional approaches. This enables real-time active learning on datasets of any size.

The combined sample efficiency and computational speed create a virtuous cycle. Faster models enable rapid experimentation with different uncertainty thresholds, acceptance parameters, and ensemble configurations. This accelerated iteration leads to better-tuned systems that extract maximum value from each expensive LLM query.

Federal Reserve Applications and Real-World Impact

The Federal Reserve’s motivation for this research stems from practical needs that many organizations share. Central banks process vast quantities of unstructured text that require sophisticated analysis while maintaining strict interpretability requirements for regulatory compliance.

Public Comments Analysis: The Fed regularly solicits public input on proposed regulations, receiving tens of thousands of comments that must be systematically analyzed. M-RARU enables classification into stakeholder categories (Banks/Trades, Consumer, Government, General Public, Other) at scale while preserving the ability to explain classification decisions to oversight bodies.

Economic Sentiment Analysis: Financial news headlines provide early indicators of economic trends. The GNAD dataset experiment demonstrates how the Fed can analyze thousands of news sources to identify rising, falling, or stable economic signals without prohibitive costs.

Regulatory Compliance: Financial regulators require institutions to justify algorithmic decisions affecting consumers or markets. Traditional machine learning models support explainability techniques like SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations) that provide clear decision rationale—something black-box LLMs cannot offer.

The framework addresses a critical challenge in regulated industries: how to leverage cutting-edge AI capabilities while meeting transparency and interpretability requirements. By transferring LLM intelligence to interpretable models, institutions can have both advanced performance and regulatory compliance.

Resource-Constrained Environments: Not every analysis can wait for expensive LLM inference or tolerate high-latency API calls. The trained student models enable rapid, offline classification that supports time-sensitive decision-making in resource-constrained settings.

Future Directions — Democratizing LLM Intelligence

The Fed’s research opens multiple avenues for extending and improving intelligent knowledge distillation. The M-RARU framework’s flexibility suggests broad applicability beyond financial text classification.

Model Architecture Extensions: The approach works with any student model providing predictive uncertainty. Future work could explore uncertainty calibration techniques for transformer models, multi-round iterative distillation, and ensemble distillation methods that combine multiple teacher models.

Domain Adaptation: While demonstrated on financial texts, the framework should transfer to healthcare (clinical notes), legal (case analysis), and scientific literature (paper classification). Each domain may benefit from specialized uncertainty estimation techniques tailored to field-specific language patterns.

Multi-Modal Extensions: The uncertainty-based selection principle could extend beyond text to images, audio, and multi-modal content. Computer vision applications might use visual uncertainty to select images for expensive human annotation or advanced model labeling.

Real-Time Learning: Current batch-based approaches could evolve into streaming systems that continuously learn from new data as it arrives. This would enable adaptive classification systems that improve over time without periodic retraining costs.

The broader impact extends beyond technical improvements. By making LLM-powered classification affordable, this research democratizes access to advanced AI capabilities. Organizations with limited budgets can now deploy sophisticated text understanding systems that were previously accessible only to tech giants with unlimited API budgets.

As recent research in efficient AI demonstrates, the future belongs to systems that achieve maximum intelligence per dollar spent. The Federal Reserve’s M-RARU framework represents a significant step toward that future, proving that strategic data selection, not data volume, drives efficient knowledge transfer from large language models to practical applications.