Measuring Impact
Purpose
A scenario for recording and reporting the economic impact of an AI initiative after it goes into production (the "Business Impact" stage).
Core Ideas
- Impact is measured according to a predefined methodology and metrics (agreed upon at the idea/prototype stage).
- Saved hours are not counted as money automatically: first, the organization must define how released capacity will be captured.
- Attribution matters: how to attribute the achieved result specifically to the AI solution (A/B tests, control groups, calculation models).
- The results are used for portfolio reporting, scaling decisions, and case studies.
How It Works
- Baseline measurement plan: which metrics, data source, recalculation frequency; attribution method (see impact confirmation, economic-impact-model).
- Baseline and counterfactual: what existed before AI and what would have happened without the initiative: hiring, backlog growth, SLA breach, contractor expansion.
- Value path: how the task change links to an operational metric, business metric, and financial result.
- Impact capture mechanism: where released capacity goes: additional volume, avoided cost, risk reduction, or end-to-end process acceleration.
- Data collection: obtaining actual figures for business metrics and technical KPIs.
- Impact calculation: applying the economic impact model, comparing against the target value, and accounting for full TCO.
- Documentation: recording actual impact, Hypothesis / Gray / Green status, and confirmation owner in the initiatives registry.
- Gate 5: decision to close the initiative as having met its goal, to scale, or to adjust.
Portfolio metrics are updated in line with portfolio-kpis.
See also: Saved hours are not ROI.