The reviewer is asking for a record, not a screen
When a governance reviewer asks you to approve a release, a dashboard link answers the wrong question. A dashboard shows the current state of your evaluation system. The reviewer needs the state that existed when the decision was made: which tests ran, against which version, under which rule, and who accepted the result. NIST's Generative AI Profile points the same way, calling for retained test, evaluation, verification, and validation history, deployment approval thresholds, and pre-deployment measurement results shared with the people who authorize release.[1]
Provenance vocabularies already model that shape. W3C PROV-DM describes a bundle as a named set of provenance descriptions, with the agent responsible for it and the time it was generated.[2] That is the trio a reviewer otherwise reconstructs by hand: what was recorded, who stood behind it, and when.
UK guidance on AI assurance draws a further line worth internalizing, separating assurance activities from the mechanisms used to communicate their results, including dashboards and reports.[3] A dashboard is a presentation layer over evidence. If the evidence itself is not preserved and addressable, the presentation is all you have, and it changes without telling anyone. Field names below illustrate record shape; they are not measured results.
- release_evidence_id, so the decision can be cited in a later review.
- system_version and eval_suite_version, recorded together on the same record.
- run_completed_at and decision_at, kept as separate timestamps.
- decision_maker and applied_policy_version, named rather than implied.
Assemble one bundle per approval decision
Give every approval candidate a bundle identifier before the review, not after it. The bundle should name the model or release version under test, the evaluation-suite version that produced the results, and the time the run finished. Those three fields let a later reader tell quickly whether the evidence predates the change they are worried about.
Pair the machine-readable manifest with a short reviewer summary: what was found, what the run could not cover, which exceptions were granted, and what was decided. NIST's profile treats pre-deployment measurement results as something shared with release authorities rather than left inside a tool, and treats the surrounding documentation as part of the evidence.[1] The summary is what a reviewer reads. The manifest is what an engineer replays.
Record the decision itself as structured provenance: the agent who approved or rejected, the policy or test plan applied, and the decision time, which in PROV terms is an activity with an associated agent and a generation time.[2] Then keep superseded bundles when dashboard data, benchmark definitions, or alert thresholds move, so last quarter's decision stays reviewable on the terms it was actually made.
- Bundle contents: run manifest, gate results, artifacts index, reviewer summary.
- Reviewer summary sections: findings, limitations, exceptions, decision.
- One identifier per decision, reused by later incidents and re-evaluations.
- Retention rule: never overwrite a bundle when thresholds or test sets change.
Record the whole input surface of the run
Scores and prompts alone do not reproduce a result. NIST's profile identifies model details, data provenance, evaluation data, and the provenance of test and evaluation data as documentation needs, and Regulation 2024/1689, the EU AI Act, addresses system versioning, development methods, use of third-party systems, data provenance, and testing and validation information.[1][4] Both point at the same practical unit: a manifest describing the run's inputs, not a summary of its outputs.
Software build provenance already solved the framing problem. The SLSA build provenance specification separates the definition of a process from a specific execution of it, and its execution fields include resolved dependency digests, the builder identity, an invocation identifier, timestamps, and byproducts.[5] An evaluation run has the same two layers: the suite as declared, and the one execution you are asking a reviewer to trust.
Experiment tracking shows the granularity that works day to day. MLflow records parameters, code version, metrics, artifacts, start and end times, and dataset metadata against an individual run.[6] Treat that as the floor rather than the ceiling. For an LLM or agent evaluation, the judge model, retrieval configuration, tool set, and policy version belong in the same record.
- System under test: deployed model or provider version, application code, runtime settings.
- Evaluation machinery: evaluator code, judge model, dependencies, prompts, tools, policies.
- Test definition: test-set version, case identifiers, transformations, metric implementation.
- Decision inputs: thresholds, aggregation method, random seed or sampling settings.
Resolve mutable references before you store them
A manifest that says model: latest or dataset: main stops describing the run as soon as either reference moves. Resolve every mutable reference at execution time: model aliases to the specific provider version, branches to commits, dataset labels to a version or digest. SLSA's provenance model does exactly this by recording resolved dependency digests rather than names, so a verifier can check what was actually consumed.[5]
Apply the same discipline to per-case evidence. Store outputs and annotations as artifacts where you can retain them, and where you cannot, record their identifiers and digests in the manifest so a reviewer can tell whether the file in front of them is the file that was graded. MLflow's split between run-level metrics and stored artifacts with dataset metadata is a workable model for that boundary.[6]
Keep identities distinct. The execution environment and the evaluator that ran the suite are not the same as the person who authored the release, and a bundle that collapses them removes the separation a reviewer is looking for. PROV's distinction between entity, activity, and agent is the vocabulary for keeping those roles apart in the record.[2]
- Store the resolved provider model version, not the alias you called.
- Store commit SHAs for application, evaluator, and prompt repositories.
- Store a digest for the test set and for each stored artifact.
- Record evaluator identity and environment separately from the release author.
Declare the gate before you read the results
The reviewer's real question is whether the release passed a rule you wrote in advance or a rule chosen after the aggregate looked acceptable. NIST recommends risk-based minimum performance or assurance thresholds in go/no-go deployment policies, asks teams to document risks that cannot be measured, and cautions against drawing conclusions from narrow or anecdotal assessment.[1] A versioned release policy answers all three by naming in-scope claims and risks, required test categories, thresholds, confidence expectations, allowed regressions, required slices, blocking conditions, and who may approve.
Then emit one structured result per criterion with an explicit state: pass, fail, not-measured, waived, or not-applicable. Not-measured is the state a dashboard cannot express, because absent data looks like empty space. In a bundle it reads as a stated coverage gap the reviewer must accept or refuse. Every waiver needs a rationale, an accountable owner, a link to the evidence it rests on, and an expiry or retest condition.
Report the distribution, not only the headline. The Model Cards framework recommends documenting evaluation procedures, decision thresholds, variability or uncertainty, evaluation-data details, and disaggregated results across relevant groups and conditions.[7] UK assurance guidance makes a parallel distinction between measuring a system and evaluating it against agreed benchmarks or guidance to inform a decision.[3] The gate result is the evaluation. The metrics are only the measurement.
- gate_result states: pass, fail, not-measured, waived, not-applicable.
- waiver record: rationale, owner, linked evidence, expiry or retest condition.
- Report scenario and subgroup results, sample sizes, and uncertainty where used.
- Treat not-measured as risk evidence requiring a decision, not missing data.
Bind the evidence to the deployment it authorizes
A passing suite is not a claim about your product unless the bundle states what the product is meant to do. NIST's profile calls for documenting intended purpose, settings, users, assumptions, limitations, and impacts alongside relevant test and evaluation metrics, and notes that benchmark-based testing may not extrapolate to deployment conditions.[1] So intended use, out-of-scope uses, deployment conditions, and known limitations belong inside the release package rather than in a separate slide.
Then map each test group to something a reviewer can act on: the product claim it supports, the foreseeable failure or misuse mode it probes, the deployment condition it assumes, the policy threshold it feeds, and the risk owner accountable for it. A test group that cannot be mapped that way is a score looking for a decision.
State coverage explicitly, including languages, user populations, modalities, geographies, retrieval and tool configurations, operational constraints, and adversarial conditions, and label benchmark-only results separately from deployment-like results.[1] The two are not interchangeable evidence, and a reviewer who cannot tell them apart will either over-trust the bundle or discount all of it.
- Include intended use, out-of-scope uses, and deployment conditions in the bundle.
- Map every test group to a claim, a failure mode, a threshold, and an owner.
- Publish a coverage statement that names what was not exercised.
- Tag each result set as benchmark-only or deployment-like.
Say what invalidates the bundle
Evidence expires when the system changes. Define change categories in advance for model, prompt, guardrail, tool, retrieval data, evaluator, test data, and policy changes, and attach a minimum re-evaluation set to each, so a model upgrade next week triggers a known run instead of a debate. NIST recommends reassessing risks after adaptations and tracking changes, monitoring results, incidents, errors, near misses, and negative impacts, and Regulation 2024/1689 describes risk management as a continuous, iterative process across the lifecycle with post-market monitoring after deployment.[1][4]
Link revisions rather than replacing them. PROV-DM provides derivation and revision relationships that state how a revised artifact relates to an earlier one, which is what lets a reviewer follow the chain from the original approval to the version running today without guessing.[2] Model Cards makes the same point from the reporting side by tying documented evaluation procedures, thresholds, and evaluation-data details to the model version they describe.[7]
Close the loop from production back into evidence. Every incident, escalation, override, and near miss should reference the production version, the bundle that authorized it, the affected risk or scenario, the corrective action, and the re-evaluation run that resulted. That linkage is what turns an evaluation suite into release evidence instead of a reporting habit.
- change_category to minimum re-evaluation set, decided before the change lands.
- revision_of, pointing at the bundle the new evidence supersedes.
- incident record fields: production version, bundle, risk, action, re-evaluation run.
- Re-open the gate when the evaluator or test data changes, not only the model.