Estimation: Precision and Accuracy (and Economics)

I was listening to a episode of the NPR show Planet Money on the accuracy of economic forecasts and how economic forecasts often have precision, but are often inaccurate. Consider this exchange between two of the show’s hosts:

Alex Blumberg: On average, the leading forecasters, like Prakken’s Macroeconomic Advisors, have around a one percentage point margin of error, which might sound pretty good. But let’s say you forecast a two percent growth rate for the year. That means you’re saying the actual rate will be somewhere between one percent and three percent.

Adam Davisdon: Now those are two totally different economies. One percent, that doesn’t even keep up with population growth. That’s a bad year. Unemployment will go up. It’ll feel bad in the U.S. Three percent is the opposite, a pretty good year.

In one of the interviews, Simon Johnson, (Economist, MIT, Peterson Institute for International Economics) said this about economic estimates:

I would call it a necessary evil. There’s so much imprecision. There’s so much, you know, lagging in terms of our updating, that in some sense, we’d be better off without forecasts. We’d better off, you know, making up our minds afresh every day. But the problem is the businesses, the institutions, all involve thinking about the future and planning for the future. And you can’t do that without taking a view of the future.

Which sounds quite analogous to estimates on software projects. One of the reasons that Planning Poker estimation uses a non-linear scale (for example, a Fibonacci sequence)) is that, in general, as tasks get larger, you are less accurate. While it might be meaningful to talk about a 1 hour task as opposed to a 2 hour task,  it’s less likely to be useful to estimate 6 hours as opposed to 5 hours for a task. (A Planning Poker desk might have 1,2,3,5,8, 13, etc, hours as possible values.)

Planning Poker is intuitive, and often matches experience, yet many people insist on estimating a task that takes more than 3 hours as 4 rather than 5. This is probably because everyone forgets that estimates are just that: “estimates.” Estimation gets better as a team works together, but the estimates are only one part of the planning process. If the goal of the team is to make a commitment to deliver functionality, then the non-linear jump in estimates is a measure of risk. The best way to reduce risk is to work in smaller pieces. When faced with a task that is bigger than an 8, and 13 seems too big, see if you can decompose the task in to smaller tasks, thus improving your ability to estimate accurately.

Estimation is difficult, not just for software teams, but we need something to help us plan, so just be mindful to give estimates the appropriate precision. And reestimate work remaining as you go.