The short answer
The best first AI pilot scores high on four dimensions: measurable economic impact (over $250K/yr or 20%+ time savings), available structured data, a clear human-approval point that bounds risk, and reversibility if the pilot fails. Most failed pilots score well on enthusiasm and poorly on data readiness or reversibility. Score 5–7 candidate workflows on this rubric, pick the highest-scoring one with executive sponsorship, and write the success criteria before any code is written.
Why first-pilot selection is the single highest-leverage decision
AI pilots fail less from technology than from misallocation. A pilot picked because someone in marketing read a McKinsey post will not deliver economics that justify a second pilot. A pilot picked with a rubric — and with success criteria agreed in writing before the build — establishes the governance, evaluation, and team patterns that every subsequent agent reuses. The rubric below is what we run in every Workflow Triage.
The four-factor rubric
Score each candidate workflow 1–5 on each factor. Anything below 14/20 is a deferred pilot, not a no-go. Anything above 17/20 is a strong first pilot. Tiebreakers go to the workflow with the clearest executive sponsor.
- ›Economic impact: cost savings, revenue acceleration, or risk reduction expressible in dollars per year.
- ›Data readiness: the data the agent needs to read and write is already in systems with API or MCP access.
- ›Governance fit: there is a natural human-approval point where a reviewer can catch agent errors before customer impact.
- ›Reversibility: if the pilot underperforms, the cost of rolling back is bounded and known.
Common high-scoring first pilots
Across our client base, the workflows that consistently score 17+ are proposal-to-SOW drafting, Tier-1 support deflection, sales discovery-to-CRM hygiene, invoice and collections drafting, and cross-system reconciliation. Each has clear dollar impact, accessible data, a natural review point, and bounded rollback. Each is also our packaged solution set — that is not coincidence.
Common low-scoring 'looks great' pilots
Open-ended copilots for executive research, autonomous outbound prospecting, and unattended financial forecasting consistently score below 14. They look attractive (high ambition, board-friendly demo) but fail on governance fit or reversibility. A failed proposal draft is a non-event; a hallucinated forecast that hit the board is a career event.
Write the success criteria first
Before any code, agree in writing: the one or two KPIs that define success, the baseline measurement, the target after pilot, the review cadence, and the decision rule for proceed/iterate/stop. This document survives every leadership change and prevents the 'we are not sure if it worked' conversation that kills second pilots.
Key takeaways
- Score candidates on impact, data readiness, governance fit, and reversibility — not on enthusiasm.
- First pilots in the 17+/20 range almost always succeed; below 14/20 almost always struggle.
- Write the success criteria, baseline, and decision rule before the build kicks off.
- Open-ended autonomous workflows are almost never the right first pilot.
Frequently asked questions
How long should the pilot selection process take?+
One to two weeks. A Workflow Triage call surfaces 5–7 candidates; scoring them takes a couple of hours; the written success criteria is one document. Longer than two weeks usually means the executive sponsor is not engaged.
What if the highest-scoring pilot is not the most exciting one?+
Pick the highest-scoring one. The exciting one becomes pilot two, funded by the credibility the first one earns.
Can we run multiple first pilots in parallel?+
Only if you have separate executive sponsors and separate evaluation capacity. Most teams should sequence them — pilot one establishes the memory, governance, and eval scaffold that pilot two reuses.