The Agent That Can't Keep a Secret
AI research agents are leaking enterprise secrets one web query at a time. A new benchmark—and a smarter training method—show why prompting won't fix it.
By The ai-compute Desk
A research agent at a healthcare firm is working through a routine question. Along the way it fires off a handful of ordinary-looking web searches—one references a cloud-migration milestone, one a January 2024 security disclosure, one narrows down which vendor got hit. No single query necessarily gives away the whole secret. But anyone watching the agent's outbound traffic can reassemble the fragments: MediConn had migrated 70% of its infrastructure to the cloud by January 2025, a fact that lived only in private documents. [1]
That scenario is the opening illustration in MosaicLeaks, a new benchmark from ServiceNow Research designed to measure exactly this failure mode. Deep research agents increasingly combine private local documents with external tools like web retrieval, creating a privacy risk: an agent's external queries may leak sensitive information. [1] The benchmark's name comes from the underlying dynamic: no individual tile reveals a picture, but the mosaic does.
The tension is almost architectural. An agent built to be useful must reach outward—toward search engines, APIs, public corpora. But every reach outward is a potential transmission, and transmissions can be intercepted. The more capable the agent, the richer those transmissions tend to be.
MosaicLeaks treats web queries as the leakage channel. The adversary never sees the private documents or the agent's reasoning—only the cumulative query log—and tries to infer private enterprise information from it. [1] The benchmark formalises three distinct threat levels. Intent leakage reveals what the agent is investigating. Answer leakage means the query log holds enough to answer a private question someone already has in hand. Full-information leakage is the strongest case: the observer can discover and state private facts without being told what to look for. [1]
The dataset itself contains 1,001 multi-hop research chains over local enterprise documents and a controlled web corpus, split into 559 training chains, 98 validation chains, and 344 held-out-company test chains. [1] Each chain interleaves local and web sub-questions, and the answer to one sub-question becomes a bridge entity in the next, so the agent must retrieve local information before it can form the next useful web query. [1] The goal is to create tasks with a high likelihood of inducing privacy leakage from enterprise documents, but that can still be solved without leaking. [1]
A worked example illustrates how quickly fragment accumulation escalates. The agent searches twice about Lee's Market's 2020 traffic growth, leaking its intent, then issues a third query to answer a follow-up. Each query looks benign alone, but seen together they let an observer deduce that the answer was 15%, and so claim that Lee's online traffic grew 15% in 2020. [1] The individual queries are innocuous. The log is damning.
The instinct when confronted with a systems problem is to reach for instructions—add a rule, append a warning, tell the thing to behave. It is the cheapest possible intervention, and in this domain, it is nearly useless.
The researchers tested that instinct directly. They added a line to the planning prompt telling the agent not to issue web queries that leak local information. The prompt helps slightly for some models, but its effect is inconsistent and significant leakage remains. It also often has a negative effect on task performance. [1] For Qwen3-4B, the prompt lowers answer and full-information leakage from 34.0% to 25.5%, but strict chain success drops from 48.7% to 44.5%. [1] The primary behavioral change, they found, was fewer web queries—not consistently safer query construction.
Training for raw performance made things worse still. Strict chain success rose from 48.7% to 59.3% when the agent was trained only to solve more chains correctly—but answer and full-information leakage climbed from 34.0% to 51.7%. [1] The model had learned to pack more context into its web queries, which helped it retrieve the right document but amplified privacy risk. Each richer query hands the observer another fragment.
This is the core irony that MosaicLeaks surfaces: a more competent agent is, by default, a more leaky one. Capability and confidentiality pull in opposite directions unless something actively reconciles them.
The proposed reconciliation is Privacy-Aware Deep Research, or PA-DR—a reinforcement learning training method built around two novel reward signals. The first is a situational task reward. Rather than scoring an entire research trajectory with a single outcome signal, PA-DR judges each model call against other calls made at the same stage and hop, with the same information available. A planning call is rewarded for searching the correct source and retrieving the right document; if that document is already in hand, it is rewarded for not searching again. [1]
The second signal is a learned privacy reward. Whenever the agent produces web queries, a Qwen3-4B classifier estimates two risks: whether the current queries leak private information directly, and whether adding them to the existing query log creates a new mosaic leak. PA-DR penalises the larger of the two, so the privacy cost lands on the exact planning decision that made the query log more revealing. [1]
The results are striking. PA-DR raises strict chain success—the share of chains where every hop is answered correctly—from 48.7% to 58.7%, while reducing answer and full-information leakage from 34.0% to 9.9%. [1] That 9.9% is lower than the untrained base model's own 34.0%, meaning training for privacy did not simply cancel the leakage that training for performance introduced—it left the agent leaking less than it did at the start. [1]
Critically, the agent did not get safer by simply searching less. PA-DR actually issues more web queries than the base model, but those queries drop the revealing details: specific metrics like "15%" or "2024", and clues about the kind of answer it is looking for. The agent still finds the right public documents. It just stops carrying private fragments along in the query text. [1]
Situational rewards also pay off during training itself. Because they compare matching calls rather than scoring a whole rollout once, they assign credit far more precisely—with no separate value model and no need to align step indices across rollouts. The situational task reward reaches the same task performance as outcome-only reinforcement learning with roughly 5–6x fewer generated training samples, and PA-DR keeps that efficiency while adding the privacy gain. [1]
None of this is a finished answer—a controlled benchmark is not a deployed system, and synthetic enterprise documents are not the sprawling, contested archives that real organisations carry. But the direction of travel is clear.
The researchers are direct about scope. MosaicLeaks is a controlled benchmark, not a measurement of leakage in deployed systems. The enterprise documents are synthetic, the web corpus is fixed, the chains span three company contexts, and every result comes from a single agent harness running multi-hop question answering rather than open-ended research. Broader tasks, real deployments, and other agent designs still need their own study. [1]
The conclusion lands with the flatness of a settled argument: you can't prompt privacy in—you have to train it in. [1] Telling an agent to be careful barely moves the needle, while rewarding how it constructs each query cuts leakage by more than 3x and leaves task success essentially intact. The mosaic effect comes from how an agent searches over time, and that turns out to be something you can measure, assign credit to, and train down. [1] For any organisation already running deep-research agents across private document stores, that is not a theoretical finding. It is an audit waiting to happen.
You can't prompt privacy in. You have to train it in. Telling an agent to be careful barely moves the needle, while rewarding how it constructs each query cuts leakage by more than 3x and leaves task success essentially intact.MosaicLeaks research blog