// agents
When one agent quietly changes.
We build monitoring for AI agents. It notices when one agent's behavior changes, names which one, and measures how often its own alerts are wrong. The same method runs inside NVIDIA FLARE and Flower.
// on this page · 6 sections
Teams now run many agents at once. A model update, an edited prompt or a tool that starts returning bad data can shift one of them, and nothing errors. Dashboards average the agents together, so the overall numbers look healthy while one agent drifts.
Tracing tools show what an agent did on a given run. This answers a different question: has this agent's behavior changed since last week, and does it still behave like its peers?
How it works
- Every agent answers the same fixed set of probe tasks on a schedule. The tasks never change, so a change in the answers comes from the agent.
- Each agent's answers are compared with its own history and with the other agents' answers from the same run. History catches an agent that changed. Peers catch one that was off from the start.
- Before any flag is trusted, the monitor runs on healthy agents to count its false alarms. Every flag comes with that error rate attached.
What it is built on
telltale
The same method for federated learning, where many machines train one shared model without moving their data. Each machine answers a fixed quiz, and telltale flags the one whose answers drift from its peers or from its own past. It runs inside Flower and NVIDIA FLARE.
NVIDIA merged three of our pull requests into FLARE in October 2026 and lists telltale in the framework's research index. Measured results are on measurement.
Limits, stated up front
It can show that an agent's answers moved. It cannot say the new answers are wrong. A person still decides that.
If every agent goes wrong the same way at the same time, there is nothing left to compare against, and it will miss it.
An agent's answers vary far more than a number does, so every flag comes with the false-alarm rate measured on healthy agents.
Where the discipline comes from
We run automated trading systems on our own capital. Our profit screen once showed less than a tenth of the real loss, because one kind of bet was silently dropped. Nothing errored. Checking every number against the exchange's own records caught it. That check is open source as kalshi-truth.
telltale came from the same habit, and so does this. Thresholds are set before the data exists, and failures are published with the results.
Design partners
We are taking on a few teams running several agents in production. Write to ryan@anchor163.com.
Common questions
Yes. Write to ryan@anchor163.com and we will set it up on your agents. telltale, the federated-learning version, is open source.
Those tools trace runs and evaluate outputs, and some offer regression checks. This watches whether each agent stays consistent over time and with its peers, and it reports its own false-alarm rate. It is meant to sit beside them.
Each agent answers a fixed set of probe tasks on a schedule. The monitor keeps only the answers.
No. telltale is our own code, built on each framework's public API. NVIDIA FLARE lists it in its research index as externally maintained. We are not affiliated with NVIDIA or Flower Labs.
studio@anchor163.com
// copied