Our PMOs tell us everything is on time, until suddenly it isn't. However, there is a group of engineers delivering new technology and bringing in their deliverables on time with a predictable degree of confidence.
How often have we seen projects come in late? It sometimes seems they are late more often than not. Bizarrely, while it is easy to be cynical about the quality of estimates things look different when we look at individual projects. Our PMOs tell us everything is on time, until suddenly it isn't.
However, there is a group of engineers delivering new technology and bringing in their deliverables on time with a predictable degree of confidence.
They can give their customers estimates upon which to make budgeting decisions, and upon which they can rely for making related plans, such as marketing for a new product.
This is remarkable for a few reasons. Nearly everything these engineers are delivering has never been done quite the same way before. It is also delivered consistently in very small increments, such that new benefits are enjoyed by one stakeholder or another sometimes more than once a day.
Tasks are grouped into batches to be estimated, but the batches are small. Each one takes no longer than a month to deliver (because they are doing Scrum ).
Estimates are known to be hard to get right, so it would be understandable if the cost of doing so much estimation overwhelmed the value of the work, but in fact estimation overheads can be as low as 0.4% of effort . It seems like magic.
Who is creating these amazing estimates and how?
The industry we're talking about is software, particularly web services and apps. It certainly helps that the product is ephemeral, delivering a new building every day is a different challenge, but there is much to admire about their system.
The system goes under different names. The overall scheme for producing a forecasted delivery date is usually called relative estimation. A popular method for gathering raw estimates is a card game known as Planning Poker or Estimation Poker . In Scrum teams, it is often called Scrum Poker. The technique is often seen in Scrum teams but is popular with anyone using Agile methods.
The raw estimates are rarely captured in units of time. In fact, it is debatable if estimate is even the right word. The two most popular units that are used are T-Shirt Sizes (Small, Medium, Large etc) and story points which are numbers that mean precisely nothing at all. Instead of having a precise meaning, the estimates are meaningful in relative terms and they reflect not time, but the complexity of the technology involved.
Capturing estimates cheaply and effectively
Cross-functional, closely collaborating teams get together to discuss a business problem. The requirement is described, and a solution is discussed for a few minutes.
Time allowing, there is an optional task break down, typically recorded on sticky notes. At the end of the conversation comes the poker game. T-Shirt Sizing cards, or Estimation Poker cards, are held discretely by each estimator.
They must independently assign a size or points score to the solution they just discussed. The cards are revealed simultaneously, to avoid anchoring, and the conversation continues (now in a stricter time-box) until the team comes to a consensus.
The poker game has a few interesting dynamics. Players are forced to stay engaged as they will be asked questions about their estimates. The post-reveal conversation can be focused on the highest and lowest estimates, quickly revealing why there might be reasons to be cheerful, or pessimistic, about the work needed.
It is very good and getting engineers to make a quick decision, keeping in mind all the information and involving the whole team.
Using estimates to produce a forecast
Estimates are turned into a forecast by the magic of maths, statistics, and educated guessing. At first, the team will have no data to justify its forecast, so finger-in-the-air guessing is as good as any other method, but that is used for just the first month after the team is formed. After the first month, everything is based on statistics.
The numbers you need to get to statistically vary a little. For T-Shirt sizes, you will track how long the team takes to deliver items of each size. If you used story points (where more complex work is given more points) then the points acquire an average time to deliver, which is known as your velocity - the ratio between points and time. Regardless, you will eventually have hard evidence to tell you what any new estimate is likely to mean.
Armed with estimates and statistics, producing the forecast is a straightforward mathematical process, usually done in a spreadsheet. If you know, for example, that 75% of Medium solutions take a month to deliver then four such items will probably take four months to arrive, and you have a 25% chance of a delay. You may even know in advance what the likely maximum delay is going to be by looking at your team's record of delivering in the face of adversity.
With teams able to estimate 4 to 8 deliverables in an hour, with a decent amount of technical conversation around each item, you can see that the cost of producing the estimates is capped at a low level.
If you have good records about what teams are then able to deliver, then you can get all the answers you need: average elapsed time (cycle time), how often teams deliver to their estimates, and the degree of variation between deliverables of the same size (confidence metrics).
If you keep the size of items small, ideally a week's work per item, you will be able to forecast key milestones with confidence.
Everything you need for effective relative estimation is contained in your beginner's Scrum Team Kit, available from Hotbox Storage. Get a free Scrum Team Kit with every £500 spent.