Skip to main content

Quality in a Jar

Its an oft chanted mantra that software quality can be 'baked in'. Like flour or eggs you just mix in the right ingredients and out comes perfection. If my wife and I were ever to take part in our own bake-off, then you'd soon see that hypothesis undermined. We'd both begin with the same ingredients, oven, spoons etc. My better half would no-doubt deliver another scrumptious banana cake and I - a new and interesting exterior wall sealant [not my intention].
Batteries and LEDs, but no quality.

The problems here are manifold, There's a Reification error, for quality is not a tangible entity (A favourite issue for Michael Bolton). There's the idea that once built software and its use is immutable. If you've ever had to use one of those unmaintained applications (typically a time sheet/billing system) thats gone to seed you'll agree that it hasn't exactly aged like fine Claret. But what's most naive is the assumption that the simple use of a tool or ingredient will make some thing better - it'll make the bugs go away and stay away.

People and consequently software is more complicated than that. A problem with a piece of software can be almost anything from an incorrect conditional operator, to the use of an incorrect language dialect  e.g.: I had the pleasure of testing some navigation/directions software, developed in the USA, as it was being 'tweaked' to work in the UK. A good example of an issue was an instruction along the lines of: "Take the off ramp, then rotary 1st exit". To a British English speaker that sounds more like instruction on how to skate-board while donning a wristwatch than how to navigate a junction (sorry 'intersection')

The tool, is just that, a tool and is nothing without the artisan's skill behind it. In testing, it is the testing skill involved that matters over the particular software tool or library installed.  To highlight the oddness of the proposition, Transplant the situation to another team member and see how it fits: Does the programmer with the most feature rich-IDE automatically produce the best code? or wider afield, Would you trust a bad doctor with space-age tools more than a skilled and experienced doctor and his judgement as to what tools and instruments to use?

Just adding a test tool, a Jar file or executable, and robotically implementing checks in its prescribed format does not automatically give you good software. Further-more, approaching a testing problem with the tool in-hand, biases you greatly as to what problems you are going to find as well as possibly miss-directing you away from the problems.

   "When all you have is a hammer, all problems start to look like nails." - Mark Twain

For example over-using Selenium or WATIR for testing a feature rich GUI is a common mistake. Many companies standardise on such a tool for GUI test automation. If your application is heavily GUI driven, then that's important to bear in mind. Your feature-changes are human user oriented - so testing could well benefit from well errh more 'human user' interaction.

The most powerful tools are your eyes, brain and hands. Let these take a look first, get an idea of where to apply the tools and what those tools might be. Don't expect that just installing that Jar will bring automatic improvements.

PS: You can learn more about those 'Sun Jars' here: http://www.comfycamping.co.uk/Gear/Lighting.aspx

Comments

Popular posts from this blog

The gamification of Software Testing

A while back, I sat in on a planning meeting. Many planning meetings slide awkwardly into a sort of ad-hoc technical analysis discussion, and this was no exception. With a little prompting, the team started to draw up what they wanted to build on a whiteboard.

The picture spoke its thousand words, and I could feel that the team now understood what needed to be done. The right questions were being asked, and initial development guesstimates were approaching common sense levels.

The discussion came around to testing, skipping over how they might test the feature, the team focused immediately on how long testing would take.

When probed as to how the testing would be performed? How we might find out what the team did wrong? Confused faces stared back at me. During our ensuing chat, I realised that they had been using BDD scenarios [only] as a metric of what testing needs to be done and when they are ready to ship. (Now I knew why I was hired to help)



There is nothing wrong with checking t…

A h̶i̶t̶c̶h̶h̶i̶k̶e̶r̶'s̶ software tester's guide to randomised testing - Part 1

Mostly Harmless, I've talked and written about randomisation as a technique in software testing several times over the last few years. It's great to see people's eyes light up when they grok the concept and its potential. 
The idea that they can create random test data on the fly and pour this into the app step back and see what happens is exciting to people looking to find new blockers on their apps path to reliability.
But it's not long before a cloud appears in their sunny demeanour and they start to conceive of the possible pitfalls. Here are a few tips on how to avert the common apparent blockers. (Part 1) Problem: I've created loads of random numbers as input data, but how will I know the answer the software returns, is correct? - Do I have to re-implement the whole app logic in my test code?
Do you remember going to the fun-fair as a kid? Or maybe you recall taking your kids now as an adult? If so then you no doubt are familiar with the height restriction -…

Betting in Testing

“I’ve completed my testing of this feature, and I think it's ready to ship”
“Are you willing to bet on that?”
No, Don't worry, I’m not going to list various ways you could test the feature better or things you might have forgotten.
Instead, I recommend you to ask yourself that question next time you believe you are finished. 
Why? It might cause you to analyse your belief more critically. We arrive at a decision usually by means of a mixture of emotion, convention and reason. Considering the question of whether the feature and the app are good enough as a bet is likely to make you use a more evidence-based approach.

Why do I think I am done here? Would I bet money/reputation on it? I have a checklist stuck to one of my screens, that I read and contemplate when I get to this point. When you have considered the options, you may decide to check some more things or ship the app. Either could be the right decision.
Then the app fails…
The next day you log on and find that the feature is b…