One AI module faked 86% of a pipeline's accuracy gains by feeding another the answers

A retrieval-augmented generation (RAG) system is built to answer strictly from the documents it retrieves. But when engineers optimize these AI pipelines end-to-end, the reader module can learn a shortcut: instead of relying on retrieved evidence, it starts answering from its own internal memory — while the system's overall accuracy keeps climbing. This is the hidden challenge of "role drift," a failure mode in compound AI systems where individual modules learn to bypass their assigned tasks even as end-to-end performance improves. To address this, researchers at MIT and Harvard introduce Role Anchor, a technique that forces modules to stay in their lanes during training. When applied, the technique mitigates role drift. For example, it forces the RAG reader to rely on retrieved evidence instead of answering based on its internal knowledge. The primary takeaway for practitioners is that end-to-end accuracy alone can overstate how much a compound AI system has genuinely learned. Engineers must evaluate individual components and ensure they work as intended. Role Anchor serves as both a guardrail and a diagnostic tool when optimizing multi-step LLM pipelines. It can be essential for real-world AI applications that require a strict division of labor between modules. Why terminal accuracy hides the problem Compound LLM systems divide complex tasks among specialized modules. For example, a system designed for multi-hop reasoning might split a task between a "Decomposer" and a "Solver." The Decomposer breaks a large problem down into manageable sub-tasks, while the Solver computes the answers to those sub-questions. This division of labor allows AI engineers to delegate execution to smaller, cheaper models, and makes it possible to process sub-tasks in parallel where possibl

A retrieval-augmented generation (RAG) system is built to answer strictly from the documents it retrieves. But when engineers optimize these AI pipelines end-to-end, the reader module can learn a shortcut: instead of relying on retrieved evidence, it starts answering from its own internal memory — while the system's overall accuracy keeps climbing. This is the hidden challenge of "role drift," a failure mode in compound AI systems where individual modules learn to bypass their assigned tasks even as end-to-end performance improves. To address this, researchers at MIT and Harvard introduce Role Anchor, a technique that forces modules to stay in their lanes during training. When applied, the technique mitigates role drift. For example, it forces the RAG reader to rely on retrieved evidence instead of answering based on its internal knowledge. The primary takeaway for practitioners is that end-to-end accuracy alone can overstate how much a compound AI system has genuinely learned. Engineers must evaluate individual components and ensure they work as intended. Role Anchor serves as both a guardrail and a diagnostic tool when optimizing multi-step LLM pipelines. It can be essential for real-world AI applications that require a strict division of labor between modules. Why terminal accuracy hides the problem Compound LLM systems divide complex tasks among specialized modules. For example, a system designed for multi-hop reasoning might split a task between a "Decomposer" and a "Solver." The Decomposer breaks a large problem down into manageable sub-tasks, while the Solver computes the answers to those sub-questions. This division of labor allows AI engineers to delegate execution to smaller, cheaper models, and makes it possible to process sub-tasks in parallel where possibl