An AI pilot should produce evidence, not activity

Many AI pilots are easy to start and surprisingly difficult to conclude. A team integrates a model, builds a demonstration and gathers positive reactions, but leadership still cannot make the next decision with confidence.

The problem usually begins before implementation. The pilot was defined as a collection of activities rather than as an experiment.

Start from the decision

Before selecting a model or building an interface, write down:

1. The decision the organisation expects to make after the pilot.
2. The assumption that currently prevents that decision.
3. The evidence that would support or contradict the assumption.
4. The smallest credible experiment capable of producing that evidence.
5. The consequence of each possible result.

This logic is familiar from continuous experimentation, but it becomes especially important in AI because technical novelty can easily distract from the decision the system is meant to support.

A convincing demonstration may still fail to answer whether data quality is sustainable, whether uncertainty can be managed, whether users will act on the output or whether the architecture can satisfy the deployment context.

The exit criteria matter

Every pilot should have an explicit path for at least three outcomes:

• proceed and invest;
• revise the assumption or design;
• stop the initiative.

If “stop” is not a legitimate result, the pilot is probably theatre rather than experimentation.