Saturday, October 31, 2009

Simple metrics for Scrum

After my talk yesterday at the Barcelona PHP Conference I received a lot of questions on the metrics we're using -- how we're building our burn-down charts, etc.

Here's what we're doing as of today. It's a mixture of ideas from "Scrum and XP from the Trenches", "Agile Estimation and Planning", and our own:
  • The Product Backlog (PBL) is estimated in dimension-less Story Points via Planning Poker. For that, we run a planning poker session at least once a sprint. Note that we only use these for long-term planning.
  • Before each Sprint Planning Meeting, we compute how many man·hours we will have available for the next sprint. We multiply by a focus factor (currently 55%). Until a couple of sprints ago, we used the focus factor obtained from the previous sprint, but that didn't work well and caused alternating sprints of too high/too low workload. So we decided to reset the value to sthg in the low-ish range and slowly bring it up as we feel confortable we can. I don't think we're done experimenting in this area.
  • At each Sprint Planning Meeting, we (meaning Team+PO) choose a few items from the top of the PBL, break them down into tasks (in yellow post-its), and estimate each in hours ("ideal" hours, to be precise). We decide how many (and which) items to take based on these hours: the total must not exceed the product computed in the previous step. The post-its for the items taken go to the "Pendiente" (english: "pending") column on the whiteboard.
  • The sum of the estimates for all the tasks taken into the sprint becomes the first point in the burn-down chart.
  • The team updates the whiteboard daily by:
    • Writing two numbers into each task they've worked on: hours worked on it since the previous sprint & estimated hours remaining to completion.
    • Moving any completed tasks into the "completed" column.
    • Adding a blue post-it for each task that has been "willingly" taken into the sprint. Of course we only do this when we're clearly ahead of plan. ("Willingly" meaning we had the choice of taking it or not, in contrast with "forced" below).
    • Adding a red post-it (with an estimate) in the top row (labelled "Fuera de sprint", english: "out of sprint") for each unplanned task that has been forced into the sprint. We would prefer to avoid those, but the business requires that we take some of these -- they are mostly value-added tasks which just can't be planned because they depend on external events such as the sale of an advertising campaign. Nasty bugs are handled this way, too.
    • Tasks that should belong to the sprint but which we failed to identify during planning also get a red post-it, but that one goes in the row for the corresponding product item.
  • I then add up the latest estimate available for each task on the "live" part of the whiteboard (that is: all except completed tasks) to obtain an estimate of work remaining. That is the value charted in the burn-down chart for the day. The manual addition is annoying, but it only takes me a couple of minutes -- which I happily pay in exchange for the many advantages of a real whiteboard.
  • At the end of each sprint I produce a small set of statistics which I publish in the wiki page for that sprint (I also write the previous sprint's value by them to compare -- graphs might be better):

    INITIAL STATS (computed at the start of the sprint)

    A. Target velocity: sum of story points for all items initially taken into the sprint

    B. Estimated hours: sum of estimates in all yellow post-its at the start of the sprint -- as said, this is the first value plotted in the burn-down chart.
    C. Hours available: total man·hours available for the sprint work.
    D. Target focus factor (B/C): as a %

    FINAL STATS (computed only at the end)

    E
    . Real velocity: total story points actually completed in the sprint

    F. Real hours: actual man·hours available for the sprint work (i.e. C +/- sickness and alikes)
    G. Total estimate of completed planned tasks: sum of the initial estimates of the yellow and blue stickers which were actually completed. That is: B + blue stickers - descoped work.
    H. Final focus factor (G/F): as a %

    J. Hours charged to planned tasks: count of all hours written against all yellow and blue stickers completed in the sprint. This is the most painful to compute, but it's not more than two or three of minutes per sprint, so stop whining. Even my 9-year-old daughter can add that without a computer.
    K. Hours charged to unplanned tasks "inside the sprint"
    L. Estimation error ((J+K)/G-1): as a %
    M. Real focus factor ((J+K)/F): as %

    N. Hours charged to unplanned tasks "out of sprint": I often also decompose these in categories such as bugfixes, support, operations, advertising.

    P. Total hours charged (J+K+N)
    Q. Reporting error (1-P/F)
It's a lot of values, and the naming of the different "focus factors" can get confusing, but they allow us to tell, in the event of a planning problem, whether it was caused by excessive unplanned work, invalid estimation (missing tasks), or inaccurate estimation. The last two are just a sort of "checksum" so we can notice if we ever become lazy and stop reporting properly.

No comments:

Post a Comment