Comments on: Misleading Metrics http://www.adamkwhite.com/2010/05/24/misleading-metrics/ Leadership, Management and Software Thu, 30 Sep 2010 05:05:19 +0000 hourly 1 http://wordpress.org/?v=3.1 By: Adam White » Blog Archive » InsideSpin – Contributor of the month http://www.adamkwhite.com/2010/05/24/misleading-metrics/comment-page-1/#comment-11834 Adam White » Blog Archive » InsideSpin – Contributor of the month Sun, 13 Jun 2010 22:58:53 +0000 http://www.adamkwhite.com/?p=58#comment-11834 [...] Strictly by the number of posts in the discussion forum. Maybe I should look at my own post about misleading metrics What is [...] [...] Strictly by the number of posts in the discussion forum. Maybe I should look at my own post about misleading metrics What is [...]

]]> By: John Allen http://www.adamkwhite.com/2010/05/24/misleading-metrics/comment-page-1/#comment-11799 John Allen Fri, 04 Jun 2010 14:41:13 +0000 http://www.adamkwhite.com/?p=58#comment-11799 Who is Nick and what is his side? Who is Nick and what is his side?

]]> By: Glory Leung http://www.adamkwhite.com/2010/05/24/misleading-metrics/comment-page-1/#comment-11796 Glory Leung Wed, 02 Jun 2010 23:21:16 +0000 http://www.adamkwhite.com/?p=58#comment-11796 Nick probably should have just told Jim that his metrics are wrong. I wouldn't totally say it's Nick's fault, but he should take some blame in not convincing Jim enough to delay the release or push more bugs. Nick should play all the cards to the Jim and say "Here is the situation. blah blah blah". Whether or not Jim takes it to heart is another question but at least Nick covered for the Test team. Nick probably should have just told Jim that his metrics are wrong. I wouldn’t totally say it’s Nick’s fault, but he should take some blame in not convincing Jim enough to delay the release or push more bugs.

Nick should play all the cards to the Jim and say “Here is the situation. blah blah blah”. Whether or not Jim takes it to heart is another question but at least Nick covered for the Test team.

]]> By: Glory Leung http://www.adamkwhite.com/2010/05/24/misleading-metrics/comment-page-1/#comment-11795 Glory Leung Wed, 02 Jun 2010 22:59:21 +0000 http://www.adamkwhite.com/?p=58#comment-11795 Jim could have perhaps assigned Risk numbers to each P value. Where P is the "observed" value of time it takes for development on average to fix that type of issue. (say.. in hours.. but you can use anything you want) For Example, if it takes roughly 10 hours days to fix a P1. Then assign a factor of 10. If it takes roughly 7 hours to fix a P2, Then assign a factor of 7. We'll use 5 for P3s and 3 for P4s. Therefore we have 263 man hours divided by 6 hours a day divided by N number of Developers. (In this case, if there is 1 developer, it will take 44 days!!) By no means this is the real solution, but I think it can give you a more accurate account of the situation at hand. There are too many external values to get an absolute estimate (people getting sick, delays, acts of god) I'll get to Nick's side later. Jim could have perhaps assigned Risk numbers to each P value. Where P is the “observed” value of time it takes for development on average to fix that type of issue. (say.. in hours.. but you can use anything you want)

For Example, if it takes roughly 10 hours days to fix a P1. Then assign a factor of 10. If it takes roughly 7 hours to fix a P2, Then assign a factor of 7. We’ll use 5 for P3s and 3 for P4s.

Therefore we have 263 man hours divided by 6 hours a day divided by N number of Developers. (In this case, if there is 1 developer, it will take 44 days!!)

By no means this is the real solution, but I think it can give you a more accurate account of the situation at hand. There are too many external values to get an absolute estimate (people getting sick, delays, acts of god)

I’ll get to Nick’s side later.

]]> By: Joseph Kubik http://www.adamkwhite.com/2010/05/24/misleading-metrics/comment-page-1/#comment-11793 Joseph Kubik Tue, 01 Jun 2010 18:37:22 +0000 http://www.adamkwhite.com/?p=58#comment-11793 Is the problem with a linear trend line (or any type of trend line) that we fail to use statistics like statisticians do? For other math / stats we would use a trend line and a P value (mathematically, how consistent is our rate, or is the trend an average of very random data) and then say +/- N units. We would use the P value to decide how accurate our trend is. example: We have fixed an average of 5 bugs per day over 10 weeks with a 95th percentile of 0 and 25 leaving us a P value that means a very bad match. so we will ship on March 15th, +/- a LARGE variance. OR, We have fixed an average of 5 bugs per day over 10 weeks with a 95th percentile of 4 and 6 leaving us a P value that means a very good match. so we will ship on March 15th, +/- a very small variance. -Joseph- Is the problem with a linear trend line (or any type of trend line) that we fail to use statistics like statisticians do?
For other math / stats we would use a trend line and a P value (mathematically, how consistent is our rate, or is the trend an average of very random data) and then say +/- N units. We would use the P value to decide how accurate our trend is.

example:
We have fixed an average of 5 bugs per day over 10 weeks with a 95th percentile of 0 and 25 leaving us a P value that means a very bad match. so we will ship on March 15th, +/- a LARGE variance.
OR,
We have fixed an average of 5 bugs per day over 10 weeks with a 95th percentile of 4 and 6 leaving us a P value that means a very good match. so we will ship on March 15th, +/- a very small variance.

-Joseph-

]]> By: Tweets that mention Adam White » Blog Archive » Misleading Metrics -- Topsy.com http://www.adamkwhite.com/2010/05/24/misleading-metrics/comment-page-1/#comment-11773 Tweets that mention Adam White » Blog Archive » Misleading Metrics -- Topsy.com Tue, 25 May 2010 10:11:31 +0000 http://www.adamkwhite.com/?p=58#comment-11773 [...] This post was mentioned on Twitter by Bob Marshall, Rob Lambert. Rob Lambert said: Nice post by Adam White on the dangers of metrics. http://bit.ly/9jX2pP [...] [...] This post was mentioned on Twitter by Bob Marshall, Rob Lambert. Rob Lambert said: Nice post by Adam White on the dangers of metrics. http://bit.ly/9jX2pP [...]

]]>