Decide what failure looks like before a pilot
Your objectiveSet a useful acceptance boundary without hiding degraded results.
Understand the idea.
A pilot should define successful behavior and unacceptable failure before the first run. Include incomplete work, incorrect output, duplicate actions and recovery time. A test that measures only fast successful responses can overlook the cases most likely to create operational trouble.
A worked example.
A job completes quickly in nine runs but produces duplicate records in the tenth. Reporting only the median completion time conceals a correctness problem. The evaluation needs both timing and outcome criteria.
Try the reasoning.
Write three failure cases for a proposed workflow and a corresponding observation for each. State which result would stop rollout and which would allow a limited retry.
Carry it into practice.
Keep failed cases in the regression suite and rerun them after the correction.
Read the reference: IETF · HTTP semantics ↗