Two levels of evaluation
Anthropic's September 18 announcement describes a partnership with Accenture on embedded evaluation and acknowledges that standards for access, reporting, and funding remain unsettled. At the application level, OpenAI's current agent evaluation guidance describes traces, graders, datasets, and evaluation runs for inspecting workflow behavior. These address different scales of accountability. Our view is that business buyers should ask for evidence at the scale they can act on: whether the actual workflow handles their representative tasks, respects its instructions, and produces an output another employee can use.
Source: Partnering with Accenture on embedded evaluation · 2026-09-18
Source: Evaluate agent workflows
Write acceptance in the buyer's language
An illustrative invoice-processing pilot should be judged by whether it identifies the correct supplier, preserves amounts and dates, flags missing information, and prepares the record in the required format. A broad benchmark score cannot answer those questions on its own. Have the receiving team describe a good output and the defects that would make it unusable. Translate that description into a small review rubric. Keep consequential requirements explicit. A system should not earn a passing score by producing attractive formatting while quietly omitting the business rule that mattered most.
Include difficult ordinary cases
Use examples that resemble daily work, including scanned documents, contradictory terms, duplicates, and missing fields. Add a few cases that should stop or escalate rather than complete automatically. The evaluation should reward appropriate uncertainty, because a false sense of completeness can be costly. Keep a portion of the examples separate from the implementation team until review. This provides a modest check against tuning everything to a familiar demonstration. The goal is an honest view of how the workflow behaves across the situations staff will actually encounter.
Inspect the path as well as the answer
A correct final answer can conceal a wrong or overbroad tool call. Review whether the system accessed the intended source, preserved the right customer context, and obtained approval before a consequential action. A useful trace lets the reviewer locate where an error entered the process. It can also reveal duplicated work and unexpected cost. Ask for a concise evidence package rather than an overwhelming dump of every message. The reviewer needs enough context to reproduce the finding, understand the consequence, and distinguish a model error from a source or integration problem.
Keep review independent enough to matter
The person who built the workflow should help explain it, but should not be the only person deciding whether it passes. Give a separate reviewer the claim, acceptance criteria, constraints, and evidence. Let them inspect original sources or rerun the relevant checks. Record disagreements and unresolved items without rewriting the criteria to fit the result. Independence can be proportionate: a colleague from the receiving team may be enough for a small internal pilot. More consequential workflows need stronger separation, clearer records, and a release decision by the accountable owner.
Ask for evaluation in the scope
Treat the evaluation record as part of the deliverable when commissioning AI work. It should include the tested version, examples, criteria, results, known limitations, and checks required after a change. This makes later improvements easier to assess and reduces dependence on persuasive demonstrations. A demo shows what happened once under chosen conditions. An evaluation shows what the team examined and where confidence remains limited. Our opinion is that buyers will gain more from demanding this practical evidence than from seeking a universal assurance that an AI product is safe or reliable in every circumstance.