Skip to main content

Measuring Impact

Purpose

A scenario for recording and reporting the economic impact of an AI initiative after it goes into production (the "Business Impact" stage).

Core Ideas

  • Impact is measured according to a predefined methodology and metrics (agreed upon at the idea/prototype stage).
  • Saved hours are not counted as money automatically: first, the organization must define how released capacity will be captured.
  • Attribution matters: how to attribute the achieved result specifically to the AI solution (A/B tests, control groups, calculation models).
  • The results are used for portfolio reporting, scaling decisions, and case studies.

How It Works

  1. Baseline measurement plan: which metrics, data source, recalculation frequency; attribution method (see impact confirmation, economic-impact-model).
  2. Baseline and counterfactual: what existed before AI and what would have happened without the initiative: hiring, backlog growth, SLA breach, contractor expansion.
  3. Value path: how the task change links to an operational metric, business metric, and financial result.
  4. Impact capture mechanism: where released capacity goes: additional volume, avoided cost, risk reduction, or end-to-end process acceleration.
  5. Data collection: obtaining actual figures for business metrics and technical KPIs.
  6. Impact calculation: applying the economic impact model, comparing against the target value, and accounting for full TCO.
  7. Documentation: recording actual impact, Hypothesis / Gray / Green status, and confirmation owner in the initiatives registry.
  8. Gate 5: decision to close the initiative as having met its goal, to scale, or to adjust.

Portfolio metrics are updated in line with portfolio-kpis.

See also: Saved hours are not ROI.