// measurement
Whether the number is real.
A model gets approved on a measured figure — an accuracy, a detection rate, a false-alarm rate. Everything downstream rests on that figure being an honest estimate of field performance. Often it is not, and nothing in the reported metrics says so.
We are the second set of eyes on the measurement rather than on the model. Not whether the architecture is fashionable — whether the number that justified the decision would survive being re-derived by someone who did not want it to be true.
What gets audited
information about the label, or information that will not exist at decision time, reaching the model through the training or validation path. Temporal splits on time series, features populated downstream of the outcome, preprocessing fitted before the split, the same entity in both partitions under different identifiers. Each biases the reported figure upward and none of them announce themselves. The four classes are written up in our leakage guide.
a probability that was honest at training time and is not honest now. We measure whether the confidence still means what it claims as the input distribution moves, which is the failure that survives every accuracy check.
where a shared model trains across nodes and one node's data is quietly degrading. The aggregate score barely moves while the damage accumulates; catching it needs a lightweight probe rather than a bigger dashboard. Written up in the drift-detection guide.
re-deriving the headline result from the rawest data reachable, by a different path than the one that produced it. When the two paths disagree, the disagreement is the finding.
What an engagement delivers
A written evaluation protocol with thresholds fixed before the numbers are looked at, so the result cannot be chosen after the fact. A leakage audit of the training and validation path, class by class. A calibration measurement under the shift your deployment actually faces. And a findings report that says plainly which claims survived re-derivation, which did not, and which could not be tested with the data available.
Where a fault is found, we do not stop at naming it — the protocol comes with the harness that reproduces it, so your team can re-run the check after they fix it.
When you do not need this
If a wrong answer is cheap — internal drafts, ranking that a human reviews anyway, anything where the cost of being wrong is a shrug — the audit costs more than the failure. Ship it and watch it.
This work earns its price when the number is load-bearing: a model that gates a decision about money, safety, eligibility, or a deployment nobody can easily reverse, or one being handed to an evaluator who will ask how you know. If that is not your situation, we will say so in the first call.
Where the discipline came from
We did not learn this from a standards document. We run automated systems that trade company capital, where a number that does not reconcile to the venue's own records is a story rather than a number, and where believing a dashboard over the fills costs real money. The same practice — canonical source, pre-registered thresholds, a verdict ledger that records when a conclusion was reversed — is what gets pointed at your measurement.
Engagements from $5,000, scoped to a defined claim rather than an open-ended review. Confidential. Anchor 163 LLC is registered for federal work — details and identifiers on the capability page. studio@anchor163.com
Common questions
An independent check of whether a model's reported performance is an honest estimate of field performance. It examines the measurement procedure — leakage in the training and validation path, calibration under distribution shift, and whether the headline result reproduces from source data — rather than reviewing the architecture.
In-house validation is run by the team whose result it is, usually against thresholds that can move after the numbers arrive. An independent evaluation fixes thresholds before looking, re-derives the claim by a different path, and reports what did not survive.
Engagements start at $5,000 and are scoped to a defined claim rather than an open-ended review. The deliverable is a protocol, a findings report, and the harness that reproduces any fault found.
studio@anchor163.com
// copied