Writing Pilot Acceptance Criteria That Can Actually Be Checked

A goal like "30% more efficient" cannot be accepted. Three formats for checkable criteria, the baseline data to capture up front, and the questions to ask at the review.

Pilots most often end with "it seems fine, let us keep using it". That conclusion cannot support a decision to invest further, nor locate the cause when results later degrade. Acceptance criteria must be written before work starts, not assembled at the end.

Three checkable formats

Rewrite a vague goal into any of these and acceptance has a clear basis:

State-based: a given action no longer requires manual execution, for example "weekly report data is no longer compiled by hand". Easiest to judge, and the recommended default.
Metric-based: a measurable indicator reaches an agreed range. This only works if baseline data already exists.
Deadline-based: first response to a class of request stays within an agreed time. Suits routing and support processes.

Baseline data to capture first

Without a baseline no metric-based goal can be accepted. Before starting, record at least:

Current effort: who performs the action, how long each occurrence takes, and how often per week.
Current quality: today’s error or rework rate, even as a rough estimate.
Measurement definition: who records it and by what rule. Inconsistent definitions make before-and-after figures incomparable.

Four questions for the review

Beyond the metrics, probe whether the result will last:

Can this hold up without the project team watching over it.
When something goes wrong, does the team know what to do and whom to contact.
Are the relevant prompts, rules and process notes written into the knowledge base.
If it were handed to another colleague today, how long would the handover explanation take.

A note on percentages

Unless the baseline definition is explicit and the sample sufficient, percentages easily become numbers each side interprets differently. In a small pilot, state-based criteria are usually more reliable and far less disputable.

Summary

Acceptance criteria do more than settle success or failure — they force both sides to make expectations explicit before work begins. A project with no checkable criteria usually has an undefined requirement.