Comments on: When to Start Logging Bugs http://www.adamkwhite.com/2007/09/14/when-to-start-logging-bugs/ Leadership, Management and Software Thu, 30 Sep 2010 05:05:19 +0000 hourly 1 http://wordpress.org/?v=3.1 By: Eric Jacobson http://www.adamkwhite.com/2007/09/14/when-to-start-logging-bugs/comment-page-1/#comment-43 Eric Jacobson Thu, 01 Nov 2007 19:31:23 +0000 http://blog.adamkwhite.com/?p=5#comment-43 I think the real problem here is the insistence on testing a feature before the dev is finished writing it. Sure, it’s possible, fun, and even valuable. But when there are an infinite number of tests to execute, it may be more valuable to execute those tests in areas getting no attention from other testers. The other testers I speak of are the devs trying to complete their code on said feature. Yes, devs are testers too! Your metaphor about “using a tornado to knock over a straw house” is great! I think as testers, we provide more value to the project by hitting the stone houses (i.e., code complete features) with tornados. This is where the skilled tester provides the most value. “Follow-on bugs” is a term I think I’ve seen Michael Hunter use to describe bugs that occur downstream due to an upstream “blocking bug”. In many cases follow-on bugs get fixed indirectly when the upstream bug is fixed. I’m a firm believer in this heuristic, and it hopefully keeps my focus in the right place. Logging 200 bugs on a broken feature seems a little extreme…I guess that may be what you are saying. If someone (who matters) on your project thinks certain chunks of bugs are time wasters, they are probably right. I agree with you that logging these bugs outside your bug tracking system is also a time waster. This is completely silly, and I have seen it as well. But if you focus on more valuable bugs, the whole notion of the extra dumbed-down bug tracking system is no longer necessary. I think the real problem here is the insistence on testing a feature before the dev is finished writing it. Sure, it’s possible, fun, and even valuable. But when there are an infinite number of tests to execute, it may be more valuable to execute those tests in areas getting no attention from other testers. The other testers I speak of are the devs trying to complete their code on said feature. Yes, devs are testers too!

Your metaphor about “using a tornado to knock over a straw house” is great! I think as testers, we provide more value to the project by hitting the stone houses (i.e., code complete features) with tornados. This is where the skilled tester provides the most value.

“Follow-on bugs” is a term I think I’ve seen Michael Hunter use to describe bugs that occur downstream due to an upstream “blocking bug”. In many cases follow-on bugs get fixed indirectly when the upstream bug is fixed. I’m a firm believer in this heuristic, and it hopefully keeps my focus in the right place. Logging 200 bugs on a broken feature seems a little extreme…I guess that may be what you are saying.

If someone (who matters) on your project thinks certain chunks of bugs are time wasters, they are probably right. I agree with you that logging these bugs outside your bug tracking system is also a time waster. This is completely silly, and I have seen it as well. But if you focus on more valuable bugs, the whole notion of the extra dumbed-down bug tracking system is no longer necessary.

]]> By: Rikard Edgren http://www.adamkwhite.com/2007/09/14/when-to-start-logging-bugs/comment-page-1/#comment-6 Rikard Edgren Mon, 24 Sep 2007 08:56:43 +0000 http://blog.adamkwhite.com/?p=5#comment-6 Some more possible reasons to use the bug system: * It is easier to use, train and maintain one system instead of several. * The testers will probably describe the issues better in a bug system, since it is more official. In a Wiki or spread sheet, it is easier to enter a oneliner that is difficult to understand for developers, or impossible to reproduce. * If developers/project managers don’t use the unofficial list, the testers eventually needs to do the tracking for them. * If metrics are interesting, it could be easier with one system, or more difficult (a new field for phase of testing might be needed.) On the other hand, besides the overhead argument, there are communication and feelings that often points to not using the bug system. I usually let each developer decide in which way to report and when to start testing for their functionality, until we reach Feature Complete. /Rikard Some more possible reasons to use the bug system:

* It is easier to use, train and maintain one system instead of several.

* The testers will probably describe the issues better in a bug system, since it is more official. In a Wiki or spread sheet, it is easier to enter a oneliner that is difficult to understand for developers, or impossible to reproduce.

* If developers/project managers don’t use the unofficial list, the testers eventually needs to do the tracking for them.

* If metrics are interesting, it could be easier with one system, or more difficult (a new field for phase of testing might be needed.)

On the other hand, besides the overhead argument, there are communication and feelings that often points to not using the bug system.

I usually let each developer decide in which way to report and when to start testing for their functionality, until we reach Feature Complete.

/Rikard

]]> By: Rikard Edgren http://www.adamkwhite.com/2007/09/14/when-to-start-logging-bugs/comment-page-1/#comment-20442 Rikard Edgren Tue, 18 Sep 2007 09:23:01 +0000 http://blog.adamkwhite.com/?p=5#comment-20442 Some more possible reasons to use the bug system: * It is easier to use, train and maintain one system instead of several. * The testers will probably describe the issues better in a bug system, since it is more official. In a Wiki or spread sheet, it is easier to enter a oneliner that is difficult to understand for developers, or impossible to reproduce. * If developers/project managers don't use the unofficial list, the testers eventually needs to do the tracking for them. * If metrics are interesting, it could be easier with one system, or more difficult (a new field for phase of testing might be needed.) On the other hand, besides the overhead argument, there are communication and feelings that often points to not using the bug system. I usually let each developer decide in which way to report and when to start testing for their functionality, until we reach Feature Complete. /Rikard Some more possible reasons to use the bug system:

* It is easier to use, train and maintain one system instead of several.

* The testers will probably describe the issues better in a bug system, since it is more official. In a Wiki or spread sheet, it is easier to enter a oneliner that is difficult to understand for developers, or impossible to reproduce.

* If developers/project managers don’t use the unofficial list, the testers eventually needs to do the tracking for them.

* If metrics are interesting, it could be easier with one system, or more difficult (a new field for phase of testing might be needed.)

On the other hand, besides the overhead argument, there are communication and feelings that often points to not using the bug system.

I usually let each developer decide in which way to report and when to start testing for their functionality, until we reach Feature Complete.

/Rikard

]]> By: Adam Goucher http://www.adamkwhite.com/2007/09/14/when-to-start-logging-bugs/comment-page-1/#comment-2 Adam Goucher Fri, 14 Sep 2007 12:41:34 +0000 http://blog.adamkwhite.com/?p=5#comment-2 "Testing starts as soon as we have a build to test." - Okay, you know I disagree with this one. Testing should begin as soon as the developer starts working on the code. Are not their unit tests considered, erm, testing? How about when the CI build kicks off? The integration tests get run? The inspection tools run? Only *then* do you have a 'build' to test. But you know this sales pitch already. :) As for logging bugs before the feature is complete, when I ran the shop at points this is what we did - once the developer had 'something' to test, we would poke it, but not log bugs. - issues would be recorded in the wiki for future reference. we were there to assist the developer in identifying any gaps they missed - once the feature was 'complete', we then we would unleash hell on the feature and log bugs Now, this worked well because the apps were built in layers which helped facilitate this. - first the gui was written, with the backend logic was stubbed out so we could check validation rules, etc - after that the database stuff was readied so we could test the upload / integrity of it - now the stubs were replaced with real logic -adam “Testing starts as soon as we have a build to test.” – Okay, you know I disagree with this one. Testing should begin as soon as the developer starts working on the code. Are not their unit tests considered, erm, testing? How about when the CI build kicks off? The integration tests get run? The inspection tools run? Only *then* do you have a ‘build’ to test. But you know this sales pitch already. :)

As for logging bugs before the feature is complete, when I ran the shop at points this is what we did
- once the developer had ‘something’ to test, we would poke it, but not log bugs.
- issues would be recorded in the wiki for future reference. we were there to assist the developer in identifying any gaps they missed
- once the feature was ‘complete’, we then we would unleash hell on the feature and log bugs

Now, this worked well because the apps were built in layers which helped facilitate this.
- first the gui was written, with the backend logic was stubbed out so we could check validation rules, etc
- after that the database stuff was readied so we could test the upload / integrity of it
- now the stubs were replaced with real logic

-adam

]]>