Adam White

I was impressed by what Cong had done. I thought his summary was thoughtful and showed understanding of the concepts. I went through bugs and reviewed the answers to each of the 4 questions. I learned a few things about the details of our product that I didn’t know before.

I picked up the phone and called him to talk to him about his findings. I gave him credit for doing what he did. When discussing the bugs I asked him if he could do this process for every bug or if it would be too time consuming. His answer was something to this effect “Well even for bugs that I thought were trivial I still learned something from talking with the developer so I’ll have to use judgement in the future”

In discussion of one particular bug I asked him to think about whether it was something we should write down as a test idea for future release and also to help future testers learn about the product. I didn’t give explicit directions/instructions on whether this is an expctation/requirement for him. He will have to figure that out for himself and decide where the info would go.

One point we discussed was that there was no response to my initial request until it was done. Throughout the week I wasn’t sure if he was putting any effort into it. I’m also at fault for not setting expectations for an initial response. We both learned a lesson from this and can now communicate more effectively in the future.

I feel like now when I respond to his daily updates we can speak a common language and have a common understanding.

Here is my summary of this interaction

What went well

• Attempting to make Cong feel safe and that I wasn’t threatening or unhappy with his work. I am his boss’ boss afterall and I had never given him a direct request before
• Clearly set completion date
• Set out expectations on how the end result should be delivered
• Gave some local options for getting help
• Follow up via phone to discuss findings and do a little bit more coaching

What could be improved

• I could have picked up the phone in the beginning and explained the exercise, following up with the email writeup
• In my initial request I could have asked Cong to respond and either accept, reject or negotiate my request and give him a date to do this by
• I could have followed up with him mid-week to see how things were going with the task
• I could have set up a skype call with him

Things to remember when coaching remotely

1. Look for things that will teach you about something of interest to you – not only the person who you are coaching
2. It’s important to set two deadlines – one for the initial response/commitment and one for the actual deadline
3. If you’ve not met the person or had very little interaction with them make sure they don’t feel threatened
4. Give the person options for getting local help
5. Follow up via phone/skype, give feedback watching for anything from #1

I’d love to hear any comments on this situation or thoughts/experiences you have with remote coaching/training

Cong’s response  (only edited formatting)

<begin response>

Summary

One of the most important things in testing, is to have a sufficient number of perspectives. We can either look at a product by its elements: Structure, Function, Data, Platform and Operations-SFDPO, or by quality criteria from customer’s point of view, CRUSPIC-Capability, Reliability, Usability, Scalability, Performance, Installability, and Compatiblity. Note that scalability and performance are closely related, and usability, what might annoy or frustrate the person or program that uses the product, and installability should not be forgotten.

In test design, risk-based testing will try to anticipate a problem and test for it. Rapid Testers model the risk in four parts. There is a risk when a victim may be affected by some problems caused by a vulnerability in the product that is triggered by some threat. The model itself is a good application to the exploratory testing we do here at Platespin. When deciding how and whether to test the risks on the risk list, where risks are assessed via a combination of the four parts, tasks are prioritized and used to guide exploratory testing.

As a tester, especially a tester in exploratory testing, we examine the criteria (CRUSPIC) from a user’s point of view. Surprised, annoyed, puzzled, frustrated, or maybe satisfied, emotions are generated when user experiencing our product. Emotions, “arousals”, should guide us in testing and work as oracles that trigger heuristics and wake us up to the discovery of a new problem, a problem that might be function-wise correct but interfere customer’s goals.

In addition, there are several key points that can lead to successful testing: A diversity of workflows tends to expose sequence-related bugs; A scriptable or programmable API to a product makes it more testible (which is the case that we use vmware ESX and vCenter) ; shortening the feedback loop between developer and end-user is a powerful force for beneficial change; real (realistic) circumstances tends to reveal problems and to help identify the kinds of problems that are important to real users.

5 Bugs discussed with developers

Edit – removed bugs that were discussed

Please point out anything you like.

Thanks,
Cong
<end response>

I thought I would share something I did recently regarding remote coaching/training. My inspiration for sharing comes from here and here

I have always enjoyed training new software testers. Lately it’s been more difficult for me to do since I’m in a different office from the bulk of the testers on my team. See my linked in profile for an explanation as to why I’m away.

Given that I’m away I started looking for ways to continue engaging with my team. One opportunity came from reading Cong Zhao’s daily updates.

About Cong: Cong joined us from Waterloo for his second summer work term. His first work term was with ACRP. They provide testing fixtures to Rim to test blackberry. You can find out more about him on linked in

About daily updates: I have everyone on my team send a little summary of the interesting things they learned that day. It’s meant to resemble newspaper headlines that catch your eye, make you want to read further and hopefully learn something of value to you.

Note: I’ve asked Cong for permission to share the process we went through.

Here is the initial correspondence I sent to Cong based on something I read in his daily update.

Cong,

Mike and Herb are out so I’m going to do some remote mentoring with you. I reviewed your list of bugs. You did a good job at re-running the original steps and documenting that in the bug. I want to challenge you to figure out what other areas of the product might have been affected by the changes you are testing.

I want you to ask yourself (and others) “What else could have been broken as a result of this fix?” “What areas of the code where changed and what other areas depend on/use this area” This will cause you to become familiar with the code and will probably require talking with the developer who fixed it.

Questions you should be able to answer for every bug

• What area of the code was changed?
• Why was it changed (explain why the change fixes the bug).
• What needs to be done to verify that the fix resolves the original problem.
• What needs to be tested to prove that the fix does not cause the product to break in some related but different area.

For some guidance on different approaches to thinking about the software check out these articles

• More Than One Answer More Than One Question
• Is There A Problem Here
• Test Design With Risk In Mind
• The Proof Of The Pudding

Action items I expect you to complete by next Friday.

For Articles listed above

• Read these articles over the next week
• Write a summary of your learning from them
• Send summary to Mike, Herb and myself

Look for and document your exploration of bugs beyond the original description.

• Pick 5 issues that had code changes that you are assigned to close.
• Speak with the developer and get the questions from above answered. Document the answers in the bug when closing.
• Show evidence of the above to Mike, Herb and myself.

Given that Herb and Mike are out and I’m remote – you could also rely on Joseph for som
e guidance as well. Let me know if you have any questions or if you cannot complete this assignment by next Friday.

Thanks,

Adam

What I think I did well with this communication

• Explained to Cong what I was intending. I attempted to make Cong feel safe and that I wasn’t threatening or unhappy with his work. I am his boss’ boss afterall and I emailed him fairly out of the blue.
• Clearly set completion date
• Set out my expectations on how the end result of the work should be delivered
• Gave him some local options for getting help

What could have made this communication stronger

• I could have asked Cong to respond and either accept, reject or negotiate my request and give him a date to do this by

I’d love to hear any comments on this situation or thoughts/experiences you have with remote coaching/training

Statistics may be defined as “a body of methods for making wise decisions in the face of uncertainty.”  W.A. Wallis

Here’s the Situation. Jim needs to release software on a specific date for a big trade show. It’s currently Monday and ship is Friday. Everyone is pumped! Marketing has their message for Friday morning ready. Brochures are printed. Sales team is pumped. Support and services have been trained. Everything is set!

Jim is the project manager for this release. From the information he has there doesn’t appear to be any  issue with hitting the deadline. During the mid-morning bug triage the test manager Neil reported that 45 bugs out of the 74 for this project need to be fixed for release.

Jim doesn’t see a problem. “Seem like a reasonable goal – 45 bugs in 5 days.. with all our coding resources dedicated to fixing this it should be no problem. We’ll work on them in priority order – P1′s first, P2′s second, P3′s third. With 6 developers working full time we will fix them in no time.”

“This is how humans are: we question all our beliefs, except for the ones we really believe, and those we never think to question.” -Orson Scott Card

In the last week of software projects things can get pretty heated. Lots of buzz. Lots of teamwork and camaraderie. Lots of great questions from management too! How long will it take to fix these bugs? Will we ship on time? Can we do it? What will it take? Can we put more people on the project and get more of these bugs fixed? There can also be an insane amount of pressure

Neil isn’t so sure of what Jim is saying. Of the 45 must fix bugs the priorities were as 4 P1, 21 P2, 8 P3, 12 P4. Jim kept getting hounded by upper management – “are we going to make the date?”. To answer Jim did something to get a quick and easy response that not even Neil was expecting.. He applied a linear trend analysis to the bug counts and predicted the release would be ok.   ”Well – it’s now Wednesday morning and we are fixing 10 bugs a day. We have 25 bugs left so we will be done in 2.5 days. That should leave us with a half of day buffer.” Jim proudly noted!

What do you think happened to the project and to Jim?

(more…)

I was using a mind mapping tool called BrainMine. I found a bug that exhibited itself if you printed the mind map and then edited and/or moved a node afterwards. What do you think is going on underneath the covers here?  Would your automation have caught this? How would you automate this test?

The video is here

I wasn’t up on my BB Test assitant skills when this video was taken so you’ll have to infer my troubleshooting process from what you see in the video.

I need YOUR

help in providing a better value to YOU

. I started my recent posts with many goals – one of which was to demonstrate how I might go about providing value to Rypple and Idée as a tester using the Heuristic Test Strategy Guide as a tool in my process.

I got a few comments on my early posts and then participation tapered off. This got me wondering – am I providing value to you as a reader? Before I go ahead with more posts I’d like to know what I can do better to serve you. My hope is to turn this series of blog posts into a springboard for demonstrating one way to provide value.

How am I doing with demonstrating one of my stated goals – providing value using HSTM? I’d appreciate hearing from you.

We could even use Rypple. It would be a great test and I might get some valuable feedback and so would they!

Adam

I found a company called Idée inc.

 who makes really neat products for searching images. Their products are  cool because they don’t rely soley on tags to find images. Instead they use attribuates of the actual image. I must say it seems rather magical and mysterious to me.

Here is how one product, TinEye

, it works according to their website

“Every day TinEye’s spiders crawl the web for additional images. Using sophisticated pattern recognition algorithms, TinEye creates a unique and compact digital signature or ‘fingerprint’ for each one and adds it to the index.

When you want to find out where an image is being used on the web, you submit it to TinEye. The attributes of the image are analyzed instantly, and its fingerprint is compared to the fingerprint of every single image in the TinEye search index. The result? A detailed list of any websites using that image, worldwide”

They also have something called Idée Lab where you can try out their technologies yourself

http://labs.ideeinc.com/

I wondered a couple of things about this

1) Exactly what attributes are they looking at? What goes into the making of these picture signatures or “digital fingerprints”?

2) If I were a tester working for Idée how would I test their applications?

I will post some of my thoughts and notes.

I have to give hats off to John Creson – a tester at Autodesk. When he arrived at the testers lunch today he brought a person from his support team! WHAT!!! A support person hanging out with a bunch of testers?? Can’t be. Really?

It’s true – it’s a real life story of teams working together. My last post because was too close to lunch for him to gaming the system

Hats of to John for expanding our testing circle to include people from his support team.

Way to go John!

I’m about to let the cat out of the bag. Some of you know it already – some of you don’t. 

If you really care about quality work on improving people’s relationship to your software. Don’t stop there – take it to the next level. Work on improving how the people who work on your software interact with each other.

Get support and test working together – really working together. Have people switch roles for a couple of days – walk a mile in the other guys shoes. Have them shadow each other. Get your testers and developers together (if they aren’t already). And GAASSP.

Get your developers and your support reps working together.

Don’t just get them exchanging emails. That’s not going to help. Sitting in on meetings together doesn’t do much either. Have developers start taking support calls. Writing KB articles, reviewing calls with support, dealing with irate customers.

I bet you might have a team of services guys out “in the real world”, flying around to customers, helping them out, fixing issues, working with support. Why not hook them up with the testers and developers too? Don’t tell me they don’t have time to help improve Quality.

Like magic better people interactions make quality problems disappear

If these teams really work together I can promise you something magical will start to happen. Your quality problems will start to disappear. Magically. Poof!! Gone.

Why? Because you have increased the value you provide to your customers. Quality is NOT an attribute of software. You can’t touch it, you can’ count it. Quality is a relationship between the user and the software. So keep working on it.

Here’s the thing. If your customers find a bug and you deal with it in a timely and appropriately manner – your customers will more than likely not be upset. If you have a skilled support person they will take ownership and let the customer know their issue is being looked into. They can follow up with the customer – even if you don’t have a fix yet. That’s crazy talk man!! Make the customer feel like their issue is important and follow up even if we don’t have fix!! Pffft.

Does this mean that customers won’t get mad? No.

Does this mean your support guys won’t want to hang up on rude customers? No.

But it does mean your customers might be happier when it comes to renew their support and services contract because they got value out of it. They aren’t just a number anymore – the are a real, living human being that someone at your company cares about.

The metrics game

Once you start you are going to run into a problem   (more…)

So here’s the thing. I’m on vacation for a month. My boss told me to disconnect from work completely – no emails, no blackberry, no nothing. So I did. I’ve been trying not chat with team members on msn. etc, etc, etc. And I’ve been successful at that (for the most part).

I failed at one thing though. I can’t disconnect from testing. It’s in my blood. It’s like a drug. I crave it. I get a rush when I’m learning, when I get in that tester mindset, and I’m interact with intellectually stimulating people.  People who think differently and don’t follow the herd.  Those who say wait a minute let’s look at what’s going on here. Testing is so much a part of what I do and what I’m so passionate about.

Ok Boss I can disconnect from day to day project demands and release schedules – Fine. But disconnect from testing? Forget it!! I can’t. Even with our new house I’m being a tester all the time if you take Weinberg’s definition “A tester is someone who knows things can be different”. House ownership has definitely taught me that things can be different.

Here are some examples of how I can’t get away from testing and i’m learning how things can be different.

I spent most of yesterday reading “The Gift of Time”. Then I spent some time thinking about James Bach’s

recent post on Quality is Dead. Do I agree with him or not? At first yes – but then no.  Still deciding. I can argue it either way. Maybe there is a blog post there for me.

Last night I had dinner with Rob Sabourin who is in town teaching his course for a client. If you ever need someone to get you pumped about testing just give Rob a shout! Having dinner with Rob is just so damn interesting. I know the conversation is interesting and valuable when I’ve started taking notes on the table. It’s even more interesting when I tear that paper off and bring it home with me for later reference.  Who else do you know of that can link software testing to Dr. Seuss. Who else can stand up on a stage and do a presentation called “What baseball taught me about software testing” or “What the the looney tunes taught me about software testing”. Rob can! He inspired me to write about Curious George as a tester. I got rejected by a magazine so I gave up on it.  I’m going to dust that piece off and put it on my blog.

Last year I was telling Rob about the link between my training at second city improv and testing. He told me I should write about it – which I did. I wrote up some thoughts but my inner critic got in my way and I didn’t put any more effort into it. I think it’s time to kick that critic in the ass – thanks Rob!

The energy Rob brings to the testing community is great. I’ve had the priviledge of attending his JIT (Just in Time testing) course as an auditor and also to bring him in to my company to do work on visual modeling. Great stuff. If you want to give your testers a kick of energy – have them do visual modeling with Rob. I promise they won’t look at things the same way afterwards.  My team definitely did not.

I found Michele Smith’s blog  from a link on James’ site. I liked what I saw so I read her recent posts. I found a link to pre-order Secrets of a Buccaneer Scholar, James’ new book from Amazon.  So I put that in my basket. Thanks Michele!

From her site I also found Michael Bolton ‘s article/presentation called “T he Two Futures Of Software Testing“.  I spent the whole morning devouring it, thinking about what he is saying.  I know Michael is at Novell this week at their Quality Summit 2009 and teaching RST afterwards. I hope it’s going well for him. 

I’ve been dusting off my testing skills with the help of Rypple and TinEye.

I’m having lunch tomorrow with a bunch of testers in Toronto. I love these conversations!!!

I’m so pumped about testing or should I say knowing things can be different. I often get this way when I’m at conferences where I can think about testing without day to day project demands.  I’ve got the better part of a whole month left to work on my testing skills! I feel like I am in a very fortunate place