Misleading Metrics

Monday, May 24th, 2010

Statistics may be defined as “a body of methods for making wise decisions in the face of uncertainty.”  W.A. Wallis

Here’s the Situation. Jim needs to release software on a specific date for a big trade show. It’s currently Monday and ship is Friday. Everyone is pumped! Marketing has their message for Friday morning ready. Brochures are printed. Sales team is pumped. Support and services have been trained. Everything is set!

Jim is the project manager for this release. From the information he has there doesn’t appear to be any  issue with hitting the deadline. During the mid-morning bug triage the test manager Neil reported that 45 bugs out of the 74 for this project need to be fixed for release.

Jim doesn’t see a problem. “Seem like a reasonable goal – 45 bugs in 5 days.. with all our coding resources dedicated to fixing this it should be no problem. We’ll work on them in priority order – P1′s first, P2′s second, P3′s third. With 6 developers working full time we will fix them in no time.”

“This is how humans are: we question all our beliefs, except for the ones we really believe, and those we never think to question.” -Orson Scott Card

In the last week of software projects things can get pretty heated. Lots of buzz. Lots of teamwork and camaraderie. Lots of great questions from management too! How long will it take to fix these bugs? Will we ship on time? Can we do it? What will it take? Can we put more people on the project and get more of these bugs fixed? There can also be an insane amount of pressure

Neil isn’t so sure of what Jim is saying. Of the 45 must fix bugs the priorities were as 4 P1, 21 P2, 8 P3, 12 P4. Jim kept getting hounded by upper management – “are we going to make the date?”. To answer Jim did something to get a quick and easy response that not even Neil was expecting.. He applied a linear trend analysis to the bug counts and predicted the release would be ok.   ”Well – it’s now Wednesday morning and we are fixing 10 bugs a day. We have 25 bugs left so we will be done in 2.5 days. That should leave us with a half of day buffer.” Jim proudly noted!

What do you think happened to the project and to Jim?

(more…)

The Quality Secret

Friday, March 13th, 2009

 

I’m about to let the cat out of the bag. Some of you know it already – some of you don’t. 

If you really care about quality work on improving people’s relationship to your software. Don’t stop there – take it to the next level. Work on improving how the people who work on your software interact with each other.

Get support and test working together – really working together. Have people switch roles for a couple of days – walk a mile in the other guys shoes. Have them shadow each other. Get your testers and developers together (if they aren’t already). And GAASSP.

Get your developers and your support reps working together.

Don’t just get them exchanging emails. That’s not going to help. Sitting in on meetings together doesn’t do much either. Have developers start taking support calls. Writing KB articles, reviewing calls with support, dealing with irate customers.

I bet you might have a team of services guys out “in the real world”, flying around to customers, helping them out, fixing issues, working with support. Why not hook them up with the testers and developers too? Don’t tell me they don’t have time to help improve Quality.

Like magic better people interactions make quality problems disappear

If these teams really work together I can promise you something magical will start to happen. Your quality problems will start to disappear. Magically. Poof!! Gone.

Why? Because you have increased the value you provide to your customers. Quality is NOT an attribute of software. You can’t touch it, you can’ count it. Quality is a relationship between the user and the software. So keep working on it.

Here’s the thing. If your customers find a bug and you deal with it in a timely and appropriately manner – your customers will more than likely not be upset. If you have a skilled support person they will take ownership and let the customer know their issue is being looked into. They can follow up with the customer – even if you don’t have a fix yet. That’s crazy talk man!! Make the customer feel like their issue is important and follow up even if we don’t have fix!! Pffft.

Does this mean that customers won’t get mad? No.

Does this mean your support guys won’t want to hang up on rude customers? No.

But it does mean your customers might be happier when it comes to renew their support and services contract because they got value out of it. They aren’t just a number anymore – the are a real, living human being that someone at your company cares about.

The metrics game

Once you start you are going to run into a problem   (more…)