Thought we could actually use the application under test to get exchange feedback
Are these blogs posts providing value to you? Comment here.
https://www.rypple.com/adamkwhite/valuetoyou
You’ll notice in my email exchanges with Rypple I’m digging out more project related information without even trying. The heuristic test strategy model for the project environment has another mnemonic - CIDTESTD.
An easy way to remember this is to remember the phrase “Kid tested - mother approved.” or in this case “CIDTESTD - mother approved”
It stands for
So what are some of the nuggest I have discovered about the project from my interactions with Rypple?
Customers
Information
Developer Relations
Test Team
Equipment and Tools
Schedule
Test Items
Deliverables
Discovering these things about the project environment and then probing them more might lead me to better information and charters later on.
I need YOUR
help in providing a better value to YOU
. I started my recent posts with many goals - one of which was to demonstrate how I might go about providing value to Rypple and Idée as a tester using the Heuristic Test Strategy Guide as a tool in my process.
I got a few comments on my early posts and then participation tapered off. This got me wondering - am I providing value to you as a reader? Before I go ahead with more posts I’d like to know what I can do better to serve you. My hope is to turn this series of blog posts into a springboard for demonstrating one way to provide value.
How am I doing with demonstrating one of my stated goals - providing value using HSTM? I’d appreciate hearing from you.
We could even use Rypple. It would be a great test and I might get some valuable feedback and so would they!
Adam
Let’s recap what’s been done so far:
Here is how I could approach Rypple going forward with what I’ve learned so far
So have I provided value to Rypple? In response to my last post I got this from Rypple.
I’m going to jump back to Rypple for a bit. From my last Rypple post I’ve got multiple jumping off points for a deeper discussion with Tiho. I’m sure there are things that I’ve missed that would be important to them. I’d love to sit down with him and brainstorm. I would go more into the capabilities with him and see what is important.
For reliability/robustness I would have to clarify with Tiho what this means to him. If I were to follow the heuristic test strategy model it guide me to ask if the product will work well and resist failure in all required situations. It goes deeper to talk about error messages and data integrity. I would explore the error messages in the program and ways for invoking them. If I had an architecture diagram I might be able to find ways to invoke these error messages individually and make my reliability testing more thorough. I could even make a mock up diagram based on what I know and ask him to fill it in. Then I could start asking what would happen if certain things failed or didn’t work properly.
I find it interesting that reliability and robustness got combined together. I take reliability to be the way a system performs over time. Robustness is resilience to failure and when failure does happen how the product handles it. A quick idea might explore doing some kind of denial of service attack on Rypple or corrupt the database in some way to see how it handles failure. Another idea would be to use perlclip to insert some really long strings and see if I can past their validation.
Usability is also important. For this I would have to focus on my opinion and present it. I can also poll the people I send message to through Rypple to see what they think of the usability of the program.
On the security side I’ve discovered something I didn’t know - they have an SSL site. There is also mention of an enterprise version. They take steps to prevent sql injection, cross site scripting, and use a state of the art facility to store the data. This is a gold mine for more test ideas. I also have to do more research into security exploits and ways to test SaaS apps for this.
Installability is not perceived to be a problem since they are SaaS. I found this intriguing as I’m not familiar with installation processes for SaaS. They will have to have some method/process of getting the code from dev to production.
I need more info on Testability. Tiho mentions that there are testing harness in place. This is encouraging. What would help me more effectively is to know what kind of logging they have for error messages. Are the logs primarily on the web server? Are there any other types of hooks/harnesses? What special tools do the developers use when debugging?
Maintainability in the test strategy model talks about how economical it is to build, fix and enhance the product. For me this ties closely into the support process and patching process. How easily can the support team, or at Rypple’s stage in the game probably the developers directly, diagnose and fix problems. What’s the turn around time? How readable is the code? If Rypple experiences a growth spurt will the developers coming in say “Wow - this is great code and it’s so easy to understand” or will they say “Wow - this is the worst code I’ve ever seen”.
For portability Tiho mentions that they picked a stack that would allow them to easily move to another hosting provider. I wonder what easy means to them and how “easy” it really is. I wonder how important this might be in the grand scheme of things. If relationships with the provider go sour then it will probably be a pretty big deal.
Localization. There isn’t much to explore since it’s not a focus right now. I think this was perfectly acceptable especially given the phase they are in. If I was a tester at Rypple it would be something I would be heavily encouraging the development team to integrate in a way that allows all the static text strings to be in separate files - especially error messages. This has the potential to save them lots of time and pain in terms of making the product support different languages. We went through this ordeal at PlateSpin and it required a significant re-work and re-test of the product.
At this point to help keep me focused I’m starting a spreadsheet of possible charters. Stay tuned for that.
To review the first session - I explored the product and took free-style notes while doing it. I came up with a list of ideas that i can explore further. I also came up with an issues list that I would like to speak to someone about.
For my second session I uncovered a lot of information but I didn’t operate their product. How can this be testing? What did I do?
I emailed one the folks at Rypple and asked what kind of information they would be interested in. I used the quality related information from James’ Heuristic Test Strategy Model. The quality heuristic can be remembered with the mnemonic CRUSPIC STMPL. What does that stand for?
It’s most of the ility type words that people talk about in the context of software. This mnemonic has become stuck in my head - when James and Michael present it they often tell stories that might help you remember it. They also encourage people to make up their own mnemonic as well.
CRUSPIC STMPL stands for:
Capability
Reliability
Usability
Security
Performance
Installability
Compatibility
Supportability
Testability
Maintainability
Portability
Localizability
I wasn’t expecting much of a response. I was pleasantly surprised. I got a response from Tiho who is listed on Linked In as Star Software Developer at Rypple. Here is his response.
I’m about to let the cat out of the bag. Some of you know it already - some of you don’t.
If you really care about quality work on improving people’s relationship to your software. Don’t stop there - take it to the next level. Work on improving how the people who work on your software interact with each other.
Get support and test working together - really working together. Have people switch roles for a couple of days - walk a mile in the other guys shoes. Have them shadow each other. Get your testers and developers together (if they aren’t already). And GAASSP.
Get your developers and your support reps working together.
Don’t just get them exchanging emails. That’s not going to help. Sitting in on meetings together doesn’t do much either. Have developers start taking support calls. Writing KB articles, reviewing calls with support, dealing with irate customers.
I bet you might have a team of services guys out “in the real world”, flying around to customers, helping them out, fixing issues, working with support. Why not hook them up with the testers and developers too? Don’t tell me they don’t have time to help improve Quality.
If these teams really work together I can promise you something magical will start to happen. Your quality problems will start to disappear. Magically. Poof!! Gone.
Why? Because you have increased the value you provide to your customers. Quality is NOT an attribute of software. You can’t touch it, you can’ count it. Quality is a relationship between the user and the software. So keep working on it.
Here’s the thing. If your customers find a bug and you deal with it in a timely and appropriately manner - your customers will more than likely not be upset. If you have a skilled support person they will take ownership and let the customer know their issue is being looked into. They can follow up with the customer - even if you don’t have a fix yet. That’s crazy talk man!! Make the customer feel like their issue is important and follow up even if we don’t have fix!! Pffft.
Does this mean that customers won’t get mad? No.
Does this mean your support guys won’t want to hang up on rude customers? No.
But it does mean your customers might be happier when it comes to renew their support and services contract because they got value out of it. They aren’t just a number anymore - the are a real, living human being that someone at your company cares about.
Once you start you are going to run into a problem Read the rest…
So here’s the thing. I’m on vacation for a month. My boss told me to disconnect from work completely - no emails, no blackberry, no nothing. So I did. I’ve been trying not chat with team members on msn. etc, etc, etc. And I’ve been successful at that (for the most part).
I failed at one thing though. I can’t disconnect from testing. It’s in my blood. It’s like a drug. I crave it. I get a rush when I’m learning, when I get in that tester mindset, and I’m interact with intellectually stimulating people. People who think differently and don’t follow the herd. Those who say wait a minute let’s look at what’s going on here. Testing is so much a part of what I do and what I’m so passionate about.
Ok Boss I can disconnect from day to day project demands and release schedules - Fine. But disconnect from testing? Forget it!! I can’t. Even with our new house I’m being a tester all the time if you take Weinberg’s definition “A tester is someone who knows things can be different”. House ownership has definitely taught me that things can be different.
Here are some examples of how I can’t get away from testing and i’m learning how things can be different.
I spent most of yesterday reading “The Gift of Time”. Then I spent some time thinking about James Bach’s
recent post on Quality is Dead. Do I agree with him or not? At first yes - but then no. Still deciding. I can argue it either way. Maybe there is a blog post there for me.
Last night I had dinner with Rob Sabourin who is in town teaching his course for a client. If you ever need someone to get you pumped about testing just give Rob a shout! Having dinner with Rob is just so damn interesting. I know the conversation is interesting and valuable when I’ve started taking notes on the table. It’s even more interesting when I tear that paper off and bring it home with me for later reference. Who else do you know of that can link software testing to Dr. Seuss. Who else can stand up on a stage and do a presentation called “What baseball taught me about software testing” or “What the the looney tunes taught me about software testing”. Rob can! He inspired me to write about Curious George as a tester. I got rejected by a magazine so I gave up on it. I’m going to dust that piece off and put it on my blog.
Last year I was telling Rob about the link between my training at second city improv and testing. He told me I should write about it - which I did. I wrote up some thoughts but my inner critic got in my way and I didn’t put any more effort into it. I think it’s time to kick that critic in the ass - thanks Rob!
The energy Rob brings to the testing community is great. I’ve had the priviledge of attending his JIT (Just in Time testing) course as an auditor and also to bring him in to my company to do work on visual modeling. Great stuff. If you want to give your testers a kick of energy - have them do visual modeling with Rob. I promise they won’t look at things the same way afterwards. My team definitely did not.
I found Michele Smith’s blog from a link on James’ site. I liked what I saw so I read her recent posts. I found a link to pre-order Secrets of a Buccaneer Scholar, James’ new book from Amazon. So I put that in my basket. Thanks Michele!
From her site I also found Michael Bolton ’s article/presentation called “T he Two Futures Of Software Testing“. I spent the whole morning devouring it, thinking about what he is saying. I know Michael is at Novell this week at their Quality Summit 2009 and teaching RST afterwards. I hope it’s going well for him.
I’ve been dusting off my testing skills with the help of Rypple and TinEye.
I’m having lunch tomorrow with a bunch of testers in Toronto. I love these conversations!!!
I’m so pumped about testing or should I say knowing things can be different. I often get this way when I’m at conferences where I can think about testing without day to day project demands. I’ve got the better part of a whole month left to work on my testing skills! I feel like I am in a very fortunate place
The purpose of this entire exercise is to show how I would apply the heuristic test strategy model developed by James Bach
What follows in the post is the notes from my first testing session intended to show show my thought process during my exploration.
*Warning - I’m stepping out on a limb here. I’m putting my work out there for review. Keep in mind they are my notes and may not make sense to you. They aren’t edited for spelling mistakes, grammar or anything else. The notes were taking as a stream of consciousness while I was testing.
If you want to know more about session based testing check out the this write up
To accompany my notes I have a 60 minute video taken with BB Test assistant. I also have a few screenshots that I took. I will try to find a way to post these so that i’m mindful of bandwidth constraints.
# Charter: Explore Rypple functionality
# Tester: Adam K White
# Length: Normal
# Charter vs opportunity
100% charter
# Set-up
0
# Notes
Clicked link in email for account activation. Typed in a 52 character password. After putting in my password again to verify I was asked to invite contacts I was prompted with a page that asked me to watch a video to get started. It showed import contacts on this page as well. I would think import contacts should have been on the invite page for those who are familar with the functionality.
The main functionality seeems to be
- Get feedback
- Review feedback
- Give Feedback
- My Network
In the top frame we have
- Welcome, Adam
- Help
- Give Us feedback
- Logout
I will explore the functionalities within the get feedback area.
Get feedback has 3 sections
Section 1
What is your question?
Max Characters appear to be 140. The training video said 100. This is inconsistent. I would imagine they found that 100 characters is not enough. The text box that appears to allow 2 lines of text to accomodate 140 characters. I would assume 70 characters per line. Will test later.
There is an icon of lightbulb below the “What is your question text box” with a + sign at the bottom right of it. A mouse over shows a tool top of suggested questions. Click the button expands a list of canned questions.
Section 2
What is your question about?
Text box has the following text
<Select up to 3 attibutes, seperated by a comma. E.g. leadership, negotiating, communication>
There is a similar light bulb icon which upon clicking reveals key words such as
leadership, integrity, presentation, attitude, communication, more attributes
Clicking more attibutes reveals
delegation, vision, strategy, listening, reliability less attributes
Noticed the the L in Less attibutes in capitilized. Problem? Don’t remember if the m in more attributes is capitalized. Clicking less attributes confirms that it is. Should it be? Something says to me that is should be. It’s consistent within itself but it violates something else. Will think about it.
No character max displayed
Section 3
How do you want to ask?
Two radio buttons
- Ask advisers through email
- Get a unique URL
There are two blue icons that seem to indicate information.
Just noticed them in the other sections as well.
Clicking gives a tooltip/fly out of information.
Text box is bigger than previous two. Appears to be 3 lines. No character max displayed
Can createa group from these advisors as well. Interesting functionality. I wonder how many people can be in a group of advisers.
—–
At the bottom of the 3 main sections there is a personalize your request, preview and create feedback request.
Personalize your request
- Shows a 3 line text box. Max appears to be 440 characters
Below the text box there is an arrow that reminds me of the undo arro in word or something similar to a back button in a browser. Mouse over shows a fly out saying “Reset personal message”. Seems to fit my expectations.
Preview
- Click preview says that need to enter at least one attribute. Thought this was strange. I expected it to ask me to enter a question first. Scrolling up I found that I had a question in there.
Removed question and clicked preview. Program prompted me to type a question. That meets my expecations.
Going to sumbit a question now.
Typed question “How would you test this program?”
Attributes “testing, listening, delegation”
I wonder if there is character max on the attributes - test for later
Unchecked rate these attributes
Started typing the emails of people i added earlier during invite session. They show up in a drop down while typing. Meets my expecations. If they didn’t show up I’d be disappointed.
Typed a personal message “I’d like your help in exploring this program. I’m testing it out for fun - not for profit. :)”
Click Preview Shows a dialog. First line is a red circle with a white exclamation point in it. Text to the right of it says “Heads up. You’re one of two people being asked”
That’s odd - since I didn’t typ in my own email. Investigate later
Sumbitted my question. Dialog shows submitting and then success I think. Don’t rememeber and wasn’t taking a video.
Final page shows the date, my question, advisers asked, attributes, labels.
Not sure what this label business is about - investigate later.
Drop down box with “select action” is visible. text in box says add advisors, close request, send reminder.
There is a button called share feedback - will investigate later
There is a page divider and below it shows
No responses so far with links Download CSV, Expand All, Collapse all. Then button with Share feedback and back to listing.
These buttons seem to be duplicates of the ones from the top of the frame. Might make more sense once I’ve asked more questions and gotten feedback.
Going to stop this session now, clean up my notes and organize my findings
While typing up notes I noticed that I’m now in the review my feedback section.
# Issues/Things to follow up on
- Not being given the option to import contact right away after activating. I might have missed it. I have no way to reset my account to “new”. As a tester this would a useful tool to have. Often times I want to set my account back to a known state. Unfortunetly this doesn’t always work properly either.
Click Preview shows red circle with a white exclamation point in it. Text to the right of it says “Heads up. You’re one of two people being asked”. I wasn’t one of the two people being asked
# Test ideas
- Test for max characters in text boxes
- Research share feedback functionality
- Research labels under review my feedback.
- Explore download CSV functionality
- Explore Expand all and collapse all
- Create a workflow diagram of all the dialogs and their functionality.
- Find a way to do some automatic data input and exercise the workflow.
From my test ideas I have a set of new charters to explore. It took me roughly 60 minutes to get this basic list of ideas. From there there are a few different directions I can take. I can discuss the issues I found and I could prioritize my test ideas with them. Before I run off and operate the product more I can also jump off on a different thread. Stay tuned….
I did some exploring on their site and liked their premise “Get fast, honest, feedback from people you trust” 3 easy steps 1) Ask a question of people you trust 2) Get honest, private feedback 3) Use their feedback to improve. Sounds easy enough. I noticed that they are in beta so I thought I would sign up and try their product out.
Sometimes it pays to ask.