Meta uses idle AI capacity to improve ad recommendations

Meta has deployed a new system that converts locally idle AI inference capacity into improved advertisement recommendations. Published on October 12, 2026, the engineering team details how they aggregate fragmented resources across their global fleet to enhance ad relevance without purchasing additional hardware.

What happened

Inference fleets at large technology companies often face a paradox where they are globally constrained by peak demand yet hold significant pockets of locally idle capacity. This mismatch occurs due to variations in traffic patterns, timing differences between model development and hardware delivery, and regional imbalances in resource ratios like CPUs and accelerators. Meta identified that these stranded resources could be pooled to provide substantial computational power if managed correctly.

To address this, Meta built a system that treats this volatile, preemptible capacity as a shared opportunistic pool. The system allows optional advertising workloads to expand into this space when available and contract instantly when the capacity is reclaimed for baseline tasks. This approach ensures that the core infrastructure remains stable while extracting extra value from hardware that would otherwise sit unused.

The implementation relies on two specific mechanisms to consume this excess capacity safely. By isolating optional work from critical baseline traffic, the system prevents latency spikes or reliability issues during sudden capacity preemptions. This design enables Meta to improve advertiser outcomes and user experience through better ad matching without increasing their overall infrastructure footprint.

How it works

The system employs two complementary elasticity levers to utilize idle resources: Rank More Ads (RMA) and Serve Better Models (SBM). RMA increases the number of candidate advertisements evaluated during the ranking stages when surplus CPU and accelerator capacity is detected. This process requires both processor types because it involves upstream feature processing and downstream inference. By evaluating a larger pool of candidates, the system increases the probability of selecting highly relevant ads for each user request.

SBM focuses specifically on accelerator surplus by routing a higher share of requests to more powerful, complex machine learning models. Unlike RMA, SBM keeps the number of candidates constant, which stabilizes upstream CPU demand. A model-router component uses value signals to prioritize requests that are likely to benefit most from the enhanced model. This allows the system to consume accelerator-only capacity that RMA cannot utilize, ensuring a broader range of idle resources are put to work.

Safety is maintained through strict physical and logical isolation. SBM uses separate routing targets for baseline and powerful models, ensuring that if opportunistic capacity vanishes, requests simply fall back to the baseline model. RMA uses logical isolation by calculating a global budget for additional work based on available replicas. Feedback loops monitor tail latency and request value, adjusting the workload in minutes, while millisecond-level load shedding protects against sudden preemptions by dropping lower-value optional work immediately.

Key details

• The system addresses the inference capacity paradox where fleets are globally constrained but locally idle due to peak-provisioning and resource mismatches.
• Rank More Ads (RMA) expands the candidate set for ranking, requiring both CPU and accelerator surplus to function effectively.
• Serve Better Models (SBM) routes more traffic to complex models using only accelerator surplus, keeping upstream CPU load stable.
• Physical isolation protects SBM baseline traffic, while logical isolation via global budgets protects RMA baseline performance.
• Multi-layer load shedding reacts within milliseconds to capacity preemption, dropping low-value optional work before central controllers adjust.
• Future plans include expanding model portability and automating value-based capacity allocation across regions and hardware types.

Why it matters

For engineering teams managing large-scale ML infrastructure, this architecture offers a blueprint for improving efficiency without capital expenditure. It demonstrates how to decouple optional quality improvements from critical service level objectives. By treating idle capacity as a volatile but valuable resource, organizations can enhance product quality, such as ad relevance, while maintaining strict latency guarantees for core user interactions.

The separation of concerns between baseline and opportunistic workloads is particularly instructive for systems dealing with heterogeneous hardware. As AI models grow more complex, the ability to dynamically route requests to different model tiers based on real-time resource availability becomes crucial. This approach minimizes waste from hardware mismatches and ensures that expensive accelerators are utilized even when primary demand fluctuates.

Furthermore, the use of value-aware load shedding provides a robust safety net for distributed systems. It allows engineers to design aggressive optimization strategies that can gracefully degrade under pressure. This pattern is applicable beyond advertising, extending to any service where incremental computational effort yields diminishing but positive returns, such as search ranking or content recommendation engines.

What you can do

• Audit your inference fleet for local idle capacity caused by regional resource ratios or peak-provisioning gaps.
• Identify workloads with high elasticity that can scale up or down quickly without affecting core service reliability.
• Implement logical or physical isolation between baseline traffic and optional computational tasks to prevent contention.
• Deploy feedback loops that monitor latency and resource usage to dynamically adjust the volume of opportunistic work.
• Add millisecond-level load shedding at the server and client levels to handle sudden capacity preemptions gracefully.
• Use value signals to prioritize which optional tasks receive extra resources, ensuring the highest impact per compute unit.