Story Points Stopped Working the Day AI Started Writing Code

Why estimates now mispredict in both directions, and a framework that measures what actually constrains delivery

A few sprints ago I sat in a planning session where the team spent twenty minutes debating whether a story was a 5 or an 8. It was a genuinely tricky piece of matching logic, the kind of story that would have earned its 8 points in any planning poker session over the last two decades. After the meeting, the developer pointed Claude at it and had a working implementation before lunch.

The story still took four days to finish, not because the code was hard to write, but because the change touched three services, the diff was large enough that two of us had to review it carefully, and the testing needed a full regression run plus a security sign-off. The 8 points we argued about measured none of that.

That planning session convinced me of something I had been suspecting for months: our estimates were not wrong, they were measuring something that no longer costs anything. Coding effort has become cheap, almost free, and yet our entire estimation system is still built around how hard the code is to write. This post is my attempt to work out what we should be estimating instead, and to propose a concrete framework you can run in your next sprint.

Why story points worked for twenty years

To see what broke, it helps to remember why story points worked in the first place.

Classic agile estimation asks the team to size a story relative to other stories, usually on the Fibonacci scale of 1, 2, 3, 5, 8, 13. Most of us were taught the mantra that points measure complexity rather than time. The real history is more interesting. Story points began on the first XP team as ideal days, literally time, multiplied by a load factor, and the time label was filed off only so that managers would stop hearing estimates as commitments (Ron Jeffries has told that story himself). Mike Cohn, whose book taught a generation of teams to estimate, spent years correcting the complexity myth: points estimate effort, and complexity matters only to the degree that it adds effort. His memorable example compares licking a thousand stamps with performing a simple brain surgery: wildly different complexity, similar time, same points.

So whether your team practiced the folklore (complexity) or the canon (effort), the estimate tracked the same underlying thing, which was the work of building the software. Complexity became the everyday shorthand for one specific reason: writing the code was the dominant share of a story's effort, so the harder the code, the bigger the story. A rough split of where that effort went looked like this.

When coding is more than half the work, and complex code takes proportionally longer to write, complexity and effort rise and fall together. A 5-point story really did take more time than a 2-point story, because the thing that made it a 5 (trickier logic, more code) was exactly the thing that consumed the hours. The Fibonacci scale made sense too: implementation effort grows roughly exponentially with scope, so the gaps between the numbers widened as the stories grew.

It is worth being honest about what the estimate never really counted, even then. Testing, review, and integration were real work in every sprint, but they entered the estimate only implicitly, folded silently into the coding effort: a complex story was assumed to need proportionally more testing, so nobody sized testing separately. Because coding dominated, that shortcut rarely hurt. Keep it in mind, though, because the shortcut is exactly what breaks next.

The whole system rested on one quiet assumption: the expensive part of a story is writing the code. Nobody stated it, because for twenty years nobody needed to.

Then AI rewrote the split

Once your team is using Claude or a similar coding assistant seriously, that quiet assumption collapses. The code that took three days now takes an afternoon, and whether the logic is clever or boring barely changes the wall-clock time, because the AI generates fifty lines and five thousand lines at roughly the same speed.

But nothing else about delivering the story got faster. A human still has to review the diff, and there is now more diff to review than ever. A human still has to test the behaviour, coordinate the rollout across services, and clarify what the story actually meant when the acceptance criteria turn out to be fuzzy. The effort didn't disappear; it moved.

The exact percentages will vary by team, and I should be clear that the splits above are illustrative rather than measured. But the direction is showing up in industry data at scale. Faros AI's 2025 telemetry study of more than ten thousand developers across 1,255 teams found that developers with high AI adoption completed 21 percent more tasks and merged 98 percent more pull requests, while average PR size grew by 154 percent and review time grew by 91 percent (Antonio Pagano has a good discussion of the review-side findings). Read those numbers together and the story is unmistakable: far more code is shipping, there is far more of it to read per change, and the hours available to read it have not grown at all.

In my experience the review slice is the one that surprises teams the most. The more code the AI generates, the more a human has to read, and reading code you didn't write is slow, careful work.

A faster plane does not shorten the journey

Here is a metaphor that captured it for me.

Think about the last flight you took. The flight itself might have been an hour, but the journey was not. You reached the airport early, queued for check-in, queued again at security, waited at the gate, and on the other side waited for your baggage and then for a cab. Door to door, the one-hour flight cost you five.

Now imagine the airline replaces the plane with a supersonic jet that does the hop in ten minutes. Almost nothing changes, because check-in, security, immigration, and baggage do not care how fast the plane is. Your five-hour journey becomes four hours and fifty minutes. And anyone who still estimates journeys by flight duration is now confidently wrong in both directions: the short international hop with two immigration queues takes far longer than its flight time suggests, while the long domestic flight with web check-in and hand luggage takes barely more than its flight time at all.

AI is the supersonic jet, and coding is the flight. Story points estimate the flight, but the sprint pays for the journey.

A story with a clever algorithm confined to one file gets a big number, but the AI finishes it in a day, so the team under-commits and sandbagging creeps in. A story with a trivial code change that spans four repositories and touches authentication gets a small number, but the review, the security sign-off, and the cross-team testing take a week, so the team over-commits and the sprint spills. Both errors come from the same root: the things that actually consume the sprint (testing, code review, cognitive load on the reviewer, integration effort, coordination between teams) appear nowhere in the estimate.

And the Fibonacci scale quietly lost its justification along the way. The fine-grained ladder existed because implementation effort was decomposable and grew exponentially. The human work that remains does not follow that curve, and it resists fine decomposition. Debating 2 versus 3 is now noise dressed up as rigour.

Estimate the work AI cannot eliminate

So what should we estimate instead? The principle I have landed on is simple to state:

Estimate the work AI cannot eliminate.

AI has commoditised implementation, but it has not commoditised human judgment. What remains expensive is attention: clarifying intent, choosing between valid strategies, reviewing diffs, testing, and coordinating integration. That residual human attention is the scarce resource on an AI-assisted team, so that is what the estimate should measure. I call the approach Residual Attention Estimation, and it has five moving parts.

The clearest way to see the proposal is to compare the question being asked, before and after.

Classic points asked about the code, put all their weight on coding complexity, and let testing, review, and integration ride along uncounted. Residual attention asks about the humans instead: six drivers carry the estimate, coding complexity fades out as absorbed by AI, and the answer coarsens from a Fibonacci ladder to three honest sizes.

Before the detail, here is the whole framework in one breath. Ambiguous stories get no estimate at all; they get gated back to refinement or into a timeboxed spike. Everything that passes the gate is sized S, M, or L on a single question about human attention rather than code complexity, and disagreements are settled by arguing over six concrete drivers rather than gut feel. Sprints are planned from net person-days and a trailing rate, with a WIP limit where the real constraint now sits: verification. Milestones are forecast from throughput history and reported as confidence levels rather than dates. And once the habit sticks, the AI itself pre-fills the whole assessment during refinement while humans audit it.

That is the entire proposal. What follows is the detail of each part, and if the sketch is enough for you, feel free to jump to the adoption plan at the end.

1. A readiness gate instead of padded estimates

Before any story gets a size, it must pass two questions. Are the acceptance criteria testable and agreed? And has the team converged on one implementation strategy? A useful operational test for the AI era: the story is ready when the AI could begin implementing it without asking a human another question.

If a story fails the gate, it does not get a bigger number; it gets no number at all. Either it goes back for refinement, or it becomes a timeboxed spike of one to three days that answers the open question. Basecamp's Shape Up calls this fixed budget an appetite, and the word captures the stance well: you decide what the exploration is worth instead of estimating how long discovery takes. This matters because letting ambiguity inflate estimates is how vague stories sneak into sprints. The right response to ambiguity is to resolve it; putting a number on it only hides it.

2. One question, three sizes

For each ready story, the team answers a single question: how much focused human attention will this consume end to end, across review, testing, integration, and coordination? Everyone votes S, M, or L, and that is the whole ceremony.

• S (1 point): one repo, a small diff, existing tests cover it. The AI does nearly everything and a human skims and merges.
• M (3 points): a diff that needs a real review, new tests to write, or a second service touched.
• L (8 points): multiple repos or teams, regression or security sign-off, or a high-risk area like auth, payments, or a data migration.
• XL: not allowed. AI makes thin slicing cheap, so split it.

If it seems inconsistent to bury the Fibonacci ladder and then assign 1, 3, and 8 points, the choice is deliberate. The capacity arithmetic later in this post needs sizes with nonlinear weights, because an L genuinely consumes several times the attention of an S, and reusing numbers your existing tooling and velocity charts already understand makes the migration free. What goes away is the fine-grained ladder and the debates between its adjacent rungs, not the idea that sizes should sit far apart.

Just as deliberately, the sizes carry no day equivalents. How many days an M costs is a property of your team, of its combined experience and its familiarity with the system, so pinning sizes to durations would bake one team's abilities into every team's estimates. The trailing rate in the capacity math learns that conversion from your own sprints, which is exactly where it belongs.

Notice what is missing from the question: how hard the code is to write. Algorithmic difficulty, boilerplate volume, and refactoring scale are all absorbed by the AI, so they no longer predict anything.

So what does drive the size? Six things, and they are worth spelling out because they are the real content of the estimate. When votes split, the team does not average; it walks these drivers, and each strong answer pushes the size up one step.

• Review load. How large and how critical is the diff a human must actually read? Five hundred generated lines in a payments service are not five hundred lines of CRUD; the former gets read line by line, the latter gets skimmed.
• Cognitive load. How much system context must a reviewer hold in their head before they can trust the change? A 50-line edit to a payment service can be a three-hour review, because the reviewer has to reconstruct how retries, refunds, and reconciliation interact before approving anything. The cost lives in that context, and the line count barely hints at it.
• Blast radius. How many repos, services, and teams does the change touch? Every additional system multiplies the review conversations and the ways a deploy can go sideways.
• Test burden. Does existing automation already cover the change, do new automated tests need to be written, or does it demand manual, exploratory, or environment-dependent testing? Each step up that ladder is a real jump in human attention.
• Coordination. Does shipping require another team's review, a contract change between services, or a release window? Calendar time spent waiting on other people is invisible to complexity but very visible in the sprint.
• Risk and irreversibility. Data migrations, public APIs, anything security-facing. What does it cost if this is wrong in production? A change you cannot easily roll back earns a slower, more careful verification pass, no matter how simple its code is. A feature toggle to enable disable the feature or to switch between alternative flows.

"I said L because it touches three repos and needs a security review" is a resolvable argument; "it feels bigger" is not. These drivers are what your team should internalise, and they belong in refinement discussions and tiebreaks, never in a live scoring matrix, because nobody should be computing six dimensions per story in planning poker. The single attention question is the shortcut; the six drivers are what the shortcut compresses.

An aside: how to think while estimating residual attention

The greatest challenge in embracing this approach lies not in the process itself, but in overriding an ingrained habit. For twenty years, the voice in a developer's head during planning asked one thing: how complex is this story? That voice needs replacing, and retraining it takes a few sprints of deliberately asking different questions before the new reflex sets in. When a story comes up, instead of imagining yourself writing the code, imagine yourself shepherding the finished code to production, and ask:

• How much of the generated code will I actually have to read, and how carefully?
• How much of the system will I need to page into my head before I can trust this change?
• In how many places does this change land: repos, services, other teams' backyards?
• How will we prove it works, and does that proof already exist as automation, or does someone have to build or run it?
• Who will I have to wait for? A reviewer, another team, a release window?
• If this goes wrong in production, what does it cost, and how fast can we undo it?

Notice that not one of these questions asks how clever the code is. Here is the shift in a single story. "Add rate limiting to our public API" would once have earned a comfortable 3: known algorithm, one service, a solved problem. Ask the new questions and it reads very differently. The code is boring, but it lands on a public surface, needs a load-test environment to prove the limits behave, requires client teams to be informed of the new error responses, and cannot be rolled back quietly once clients start seeing 429s. That story is an L, and the old reflex would have sailed it into the sprint as a 3. The complexity voice is not wrong about the code; it is answering a question nobody is asking anymore.

3. Sprint planning: familiar math, one new constraint

Here is the honest part: once stories carry the right sizes, sprint planning does not change dramatically, and that is by design. You still sum points against a capacity number, and the capacity number is still person-day arithmetic:

net person-days = (team size × sprint days) − leave days − holiday days
trailing rate   = points verified-done over last 3 sprints ÷ person-days
capacity        = net person-days × trailing rate × 0.85

Normalising by net person-days means leaves and holidays fall out of the math automatically, and counting only verified-done work keeps the rate honest. The 0.85 is a plain focus factor, a buffer for the meetings, incidents, and support interruptions that consume attention without ever appearing on the board; tune it to your team's reality after a few sprints. But if your team already does disciplined capacity planning, this will feel familiar, and it should: the point of fixing the estimate is that the downstream arithmetic gets to survive. That is what makes this adoptable without a big-bang process migration.

What is genuinely new is one constraint the point totals cannot see. AI generation will outrun human verification if you let it; a faster plane only makes the security queue longer. So cap the "awaiting review and testing" column with a WIP limit, and before committing, sanity-check the specialists: if three L stories all need the one person qualified to review auth changes, the sprint fails on calendar math even when the points fit. Scrum spent twenty years limiting development WIP; AI-assisted teams need to limit verification WIP, because that is where the constraint now lives. This is Goldratt's Theory of Constraints applied to a sprint board: put the limit at the bottleneck and the whole system's throughput follows. In my experience this single guardrail changes sprint outcomes more than any estimation tweak, because it stops the sprint from ending with ten stories that are done except for review.

4. Milestones: forecast in confidence levels

Milestone tracking similarly keeps its classic skeleton (size the remaining backlog coarsely, apply a scope buffer, divide by weekly burn), so I will not walk through what you already do. The one new tool here is Monte Carlo forecasting, which runs on throughput counts and works best when stories are consistently right-sized, which is exactly what the readiness gate and the no-XL rule produce. Given five or six sprints of throughput history, you can simulate thousands of possible futures and report confidence instead of a date: "50 percent confidence for March 3, 85 percent for March 24." None of this needs inventing, because the Kanban community has forecast this way for a decade. Daniel Vacanti's When Will It Be Done? is the standard text, and Troy Magennis maintains free spreadsheets that will run the simulation against your own throughput data this afternoon.

A percentile forecast is honest in a way that "our velocity is 34" never was, and it has a property that matters in this era: it self-corrects as AI tooling improves. When better tooling turns M-work into S-work, the throughput history shifts and the forecast follows, without anyone re-estimating anything.

5. Let the AI run the rubric

There is one more step, and I think it is where this ends up for all of us. Look back at the six drivers and notice something: most of them are not judgments at all, they are facts about the codebase. Which repos and services a change would touch, how large the diff is likely to be, whether existing tests cover the affected paths, which specialists will need to sign off. A human in planning poker can only guess at these. An AI agent with access to the repositories does not have to guess; it can check.

So once the manual process is habitual, move the assessment itself to the AI. During refinement, an agent (Claude with repo access, in our case) takes each story and does a dry run: it drafts an implementation sketch against the actual codebase and reports back against the rubric. It names the repos and files in the predicted blast radius, weighs the expected review load by how critical the touched areas are, lists which tests already cover the change and which would have to be written or run by hand, and identifies the reviewers and sign-offs the change would need. Any open questions it could not resolve from the story or the code feed straight back into the readiness gate, because a story the AI cannot start is not ready. At the end comes a suggested size with a one-line rationale: "M: two services, moderate diff, existing integration tests cover the main path."

Planning then stops being an estimation session and becomes an audit. An undisputed suggestion stands in seconds. A dispute means a human knows context the agent could not see, perhaps that this module is politically sensitive or that the stakeholder has quietly changed their mind, and that dispute is exactly the conversation estimation was always meant to surface. The judgment stays human while the arithmetic and the archaeology move to the machine. Estimation becomes like a compiler warning, silent when everything is normal and loud precisely when human attention is needed.

This is also the deepest shift in the whole framework, so it is worth saying plainly: humans stop estimating and start auditing estimates. Grounded in a dry run against the real code, the AI's numbers sit closer to a measurement than a guess, and the humans add the one thing a measurement cannot contain, which is context.

Estimate the current constraint, not yesterday's

I want to be honest about this framework's shelf life. Nothing here claims that review and testing will be the bottleneck forever. If AI absorbs most of verification next, the right move will be to re-derive the estimation unit again and estimate whatever human work remains, probably requirements, architecture, and business judgment. The durable principle is the meta-rule: estimate the current constraint, not yesterday's. Story points were the right instrument when implementation was scarce, and they have become the wrong instrument today for the simple reason that the constraint they were designed to measure has moved.

If you want to try this, you do not need a big-bang migration. In sprint one, keep your ceremony and just change the question: size S, M, or L on human attention, enforce the readiness gate, and start logging net person-days. In sprint two, replace the capacity math. In sprint three, add the AI annotation pass. Three sprints is enough to know whether your estimates have started matching your reality again.

Here is the thought I want to leave you with. Agile was always estimating the team's constraint, whatever we wrote on the cards. For twenty years that constraint was writing software, so we sized coding complexity, called it planning, and it honestly worked. AI has moved the constraint, while the need to estimate remains. If we keep estimating implementation, we will keep planning against yesterday's bottleneck. Size the journey instead of the flight, and estimates start matching reality again.

If you try this with your team, I would love to hear how it goes, especially where it breaks. That is how the framework got to this shape in the first place.