Software Estimation: Traditional and Agile approaches

Software estimation has historically been a difficult task for numerous reasons, yet it's also one of the most important steps in a software development project that typically garners nowhere near the attention or thought that it deserves.

Curious as to the academic consensus on software estimation, I surveyed a variety of software texts on the topic; from classics: Mythical Man Month (MMM), Code Complete (CC), and Rapid Development (RD), to more current texts: Practices of an Agile Developer (POAAD), The Pragmatic Programmer (PP), and Agile Software Requirements (ASR).

Surprisingly, and not so surprisingly, there is quite a bit of overlap between older and newer approaches to software estimation.

Why is software estimation so hard?

Code Complete (CC) begins its chapter on software estimation with, "Managing a software project is one the formidable challenges of the late 20th century. . .the average large software project is 1 year late and 100% over budget." Consoling words, and they resonate with most developers who have done professional software development. But why is software estimation so hard?

• The simplest answer is: software is extremely complex. Software has lots of moving parts, and these parts are often constantly changing or being updated. Not just small parts either (like a new library of module), but large parts as well - new languages, new frameworks, new operating systems, new hardware - all affect the complexity of the task - and add to the difficulty of coming up with an accurate estimate. It is not unusual for a developer to use a few, if not many, modules that she's never used before during a project.
  
• Software quality varies. Writing a program vs. writing a "programming product" can vary by as much as 3X (MMM).
• Developer skill varies. One study indicates that programmer productivity can vary as much as 10:1 (MMM). Some say as much as 20:1.
• The client's understanding of his product varies. I haven't read a lot of authors writing about this, but, in my experience the client's grasp of his own product definition hugely affects the pace of development. The clearer the client's understanding of his product's details and design, not surprisingly, the faster the pace of development.
• Developer optimism. "The first false assumption that underlies the scheduling of systems programming is that all will go well." (MMM)

Why do we want good estimates?
"Rapid Development" makes a salient point: (1) good estimates are more than a means to make the client happy. Good estimates also: (2) reduce development costs internally (e.g. reducing scheduling inefficiencies: overlap and overrun), and (3) they provide a sustainable pace that helps developers avoid burnout, making them more productive in the long-term.

When estimation fails. . .what can you do?
There are a few common fixes to a software schedule that's falling behind. It turns out that some of them, aren't really 'fixes' at all:

• MYTH: We'll fix it in the end. "One survey of over 300 software projects concluded that delays and overhangs generally increase toward the end of the project. Projects don't make up lost time later; they fall further behind." (CC)
• MYTH (largely): Add more people. While it may seem counter-intuitive, Brook's Law (MMM) asserts that "Adding manpower to a late software project makes it later." In the words of Code Complete: "New people need time to familiarize themselves with a project before they can become productive. Their training takes up the time of the people who have already been trained. And merely increasing the number of people increases the complexity and amount of project communication." In reality, for a large project adding people during the project may help - less likely so, for smaller projects.
  
• Reduce scope. This is probably the most sensible and easiest solution of all. It can take many forms: dropping a feature, delaying performance tuning, implementing a crude version of a feature, to be fleshed out in a future release.

I. Traditional approaches to improving estimates:
The following are tips and suggestions for improving the accuracy of estimates. Labeling these approaches "traditional" is not to imply that these approaches are outdated or somehow lacking; in fact, a lot of the following techniques have been absorbed by Agile estimation techniques.

• Formalize requirements. This may sound like a 'duh!' statement - but it is often overlooked. (CC) Why? Oftentimes, a client wants to build something without knowing the outlying details of what she's having built; a client can have a solid sense of her product definition from a high-level, but be fuzzy on the details; she may expect that development will help flesh out the details. In this case, rapid prototyping, may be the best approach - so that the client and developer can both know, with greater detail, what they are building, as quickly as possible.
• Make time for estimates: "Rushed estimates are inaccurate estimates. . ." (CC). Software estimation can be a mini-project unto itself and, if time permits, should be treated as such. While this may sound like a luxury and an 'unnecessary' upfront cost, its cost can pale in comparison to the potentially costly overruns that will likely arise due to poor estimation. "Until each feature is understood in detail, you can't estimate the cost of a program precisely. Software development is a process of making increasingly detailed decisions." (RD)
• Allow developers to make the estimates. (RD) "We have to mention a basic estimating trick that always gives good answers: ask someone who's already done it." (PP)
• Use several estimation techniques, and compare the results. (RD)
• Re-estimate periodically: for long-term projects it might make sense to schedule re-estimation periods, factoring in newly determined velocity. (RD)
• Track historical data: documenting estimates and actual completion times, while often seen as an annoyance, can provide sharper metrics for estimating a future set of tasks. A set of historical data is required to accurately define future estimates. (RD)

II. The Agile approach to estimation (see "Agile Software Requirements")
Yes, all these tips and suggestions may sound daunting, maybe even other-worldly. Given a tight schedule - who has time to do all this? To the rescue, in walks the Agile approach. . .

To the outsider, Agile methods can sound like a justification for no documentation, no planning, no coordination; not true. It is true that Agile teams often abhor long-term estimation. . .not for fear of commitment, but because they understand that long-term estimates are most often inaccurate, so - why bother? One of the main reasons that Agilists rejects traditional project estimating (e.g: identify tasks, estimate tasks, sum tasks, then build a Gantt chart) is that it "never actually worked for software projects." (ASR)

Basic Concepts of Agile estimation:

• User Stories: a brief statement that explains a feature, usually based upon a cause and effect statement. Example: User X does Action Y and Result Z happens.
• Story points: numbers (integers) that indicate the 'bigness' of a single User Story - the higher the number the bigger the task. Using techniques below, each User Story, is assigned Story Points. There are four aspects to consider when assigning Story Points to a User Story:

1. Knowledge and comprehension of the User Story.
2. Complexity: how hard it will be to implement
3. Volume: how much actual work there is to do
4. Uncertainty: unknowns that might effect the estimate

Any numerical scale (1-5 or 1-100) can be used as a metric for Story Points - what matters is that they are 'numerically relevant': a two-point story should take 2x as long as a one-point estimate. Another approach to Story Points: perhaps more suited for larger projects, is to use IDDs (Ideal Developer Days - a theoretical perfect day of work) as Story Points.

A few approaches to Agile estimating:

• Work first, estimate after. "Let the team actually work on the project, with the current client, to get realistic estimates." This may seem backwards, but it is actually a very practical way to obtain an estimate grounded in reality (not some potentially flawed algorithm). (POAAD) Understandably, not all clients may not be open to starting out a project without any scheduling estimates.
• Planning Poker. This is an online game in which developers bid on User Stories, until an entire backlog has been estimated. These estimates can then be used to define a developer's estimation accuracy as well as help developers' hone their estimation skills over time. http://www.planningpoker.com/ (ASR)
• Tabletop relative estimation. This is a simple approach to estimation whereby a development teams orders index cards (each with a single User Story on them) on a table by Story Point value. The team can discuss each story and place smaller stories to the left and larger stories to the right, in order of complexity. Each column has a different Story Point value. By the end of the estimation session (pretty quick), there can be a good relative sense of the scale regarding the User Stories. (ASR)

Advice: Keep estimating brief. . .
Good news, according to ASR, there are diminishing returns with repeated attempts at estimation: after having a team provide three estimates, the estimates do not gain any accuracy. More is not better, here.

Velocity:
After estimations have been generated by assigning Story Points to User Stories, a sense of scope is acquired, but still no sense of how long development will take. How is a sense of development speed (aka Velocity) determined?

Answer: By determining how many Story Points a team or person can complete during a timeboxed iteration of User Stories.

Example: Have two teams work on the same set of User Stories (after having defined their Story Points) for a 2 hour timebox (iteration). Most likely, when the Story Points for each team are summed at the end of the iteration, there will be two different scores. These scores indicate the Velocity of each team.

III. Conclusion:
Having surveyed a few classic and current approaches to software estimation, it's clear that the problems surrounding software estimation have not changed: software estimation has and will always remain a complex task, most likely, as long as writing software remains a complex task.

There are many techniques to minimize the problems associated with poor software estimation: traditional methods focus on defining tasks, estimating tasks and summing the estimates; Agile methods avoid a waterfall technique and assert that the only way to gain an accurate estimate is to begin the work and estimate from there, and re-estimate as the project continues.

Agile estimation embraces the complexity of software and tackles the difficulties of estimation with a lightweight real-world process:

1. write User Stories
2. have developers estimate with Story Points
3. closely track actual iterations of development tasks (User Stories) to determine developers' Velocity
4. finally, arrive at estimates that are based on real-world development; they should become more accurate over the course of the project as work history progresses