Misleading Metrics
Monday, May 24th, 2010Statistics may be defined as “a body of methods for making wise decisions in the face of uncertainty.” W.A. Wallis
Here’s the Situation. Jim needs to release software on a specific date for a big trade show. It’s currently Monday and ship is Friday. Everyone is pumped! Marketing has their message for Friday morning ready. Brochures are printed. Sales team is pumped. Support and services have been trained. Everything is set!
Jim is the project manager for this release. From the information he has there doesn’t appear to be any issue with hitting the deadline. During the mid-morning bug triage the test manager Neil reported that 45 bugs out of the 74 for this project need to be fixed for release.
Jim doesn’t see a problem. “Seem like a reasonable goal – 45 bugs in 5 days.. with all our coding resources dedicated to fixing this it should be no problem. We’ll work on them in priority order – P1′s first, P2′s second, P3′s third. With 6 developers working full time we will fix them in no time.”
“This is how humans are: we question all our beliefs, except for the ones we really believe, and those we never think to question.” -Orson Scott Card
In the last week of software projects things can get pretty heated. Lots of buzz. Lots of teamwork and camaraderie. Lots of great questions from management too! How long will it take to fix these bugs? Will we ship on time? Can we do it? What will it take? Can we put more people on the project and get more of these bugs fixed? There can also be an insane amount of pressure
Neil isn’t so sure of what Jim is saying. Of the 45 must fix bugs the priorities were as 4 P1, 21 P2, 8 P3, 12 P4. Jim kept getting hounded by upper management – “are we going to make the date?”. To answer Jim did something to get a quick and easy response that not even Neil was expecting.. He applied a linear trend analysis to the bug counts and predicted the release would be ok. ”Well – it’s now Wednesday morning and we are fixing 10 bugs a day. We have 25 bugs left so we will be done in 2.5 days. That should leave us with a half of day buffer.” Jim proudly noted!
What do you think happened to the project and to Jim?