investigating software

Testing doesn't complete, it might end, it might finish, but it doesn't complete. There's too much to test. If you ever need confirmation of this, test something, something that's been tested already. Better still test a piece of software, you know has been tested by someone you think is a brilliant tester. A good tester like you, will still find new issues, ambiguities and bugs. That's because the complexity of modern software is huge: as well as all the potential code paths of your code, there's all the other underlying code's paths and the near infinite domain of data it might process. Thats part of the beauty of testing, you have to be able to get a handle on this vast test space. That is, review a near infinite test-space in a [very] finite time-frame. We are unable to give a complete picture of the product to our clients. But we are also free to find out new issues, that have so far eluded others. In fact the consequences are potentially more drama

In my last post I discussed how test automation could be used to do things that I couldn't easily do unaided. In that example, execute thousands of news 'content searches' and help me sort through them. With the help of some simple test automation I found some potential issues with the results returned by the REST API. In that case, I started out with the aim of implementing a tool. But your testing might not lead you that way, often your own hands-on investigation can find an issue. But you don't know how widespread it is, is it a one-off curiosity? or a sign of something more widespread.? Again, this is where test automation can help, and if done well, without being an implementation or maintenance burden. Many test automation efforts are blind to the very Agile idea of YAGNI or You Ain't Gonna Need it . They often presume to know all that needs to be tested in advance, deciding to invest most of their time writing 'tests' blindly against a specific

Have you ever had to test an API that's accessible over the internet? or even one thats available internally within your organisation? They often take the form of a REST service (or similar) through which other software can easily access information in a machine readable form. Even if you are not familiar with these APIs, you've probably heard-of or seen the results of them. Some examples of APIs are the Twitter API ,   Flickr  and the Guardian's Open Platform . Some examples of what people have built using the Flickr API are published on the flickr site. Despite being 'machine-readable' they are often human readable, greatly helping you test and debug them. Companies use these APIs to ease the distribution of their content, encourage community and commercial development around their content or to simply provide a clear and documentable line between their role as data-provider and where the consumer's role begins. When testing an API like the above, many

I recently blogged about some of the tools I use , and how some are so useful I keep using them. As I mentioned, randomness is pretty useful, and I have tools to help me generate random text. A few of my readers requested a copy of my simple random text generating script, so I've decided to open it up for everyone to use and test. It will have bugs, like all software, please send details and I'll try and fix them. If you are interested in what UTF-8 is and what all that Unicode stuff is about, there is a great article by Joel Spolsky that explains all, and the wikipedia page is ok . To use it... First download the script, its on GitHub . The script is fairly short and is all in one file. You don't have to 'install it', its not a GEM. Second, make sure you have Ruby version 1.9 or greater. You need version 1.9, because Ruby didn't handle UTF-8 well in older versions. Thirdly run the script like this: ruby fuzzutf8.rb That will give you some usag

Consistency, is one thing I test for in software. For example, if software refers to something by a particular name, then [usually] it should always refer to it by that name. Furthermore, when it uses that name e.g. 'London Tube Map' I would expect to see such a map, when I click to view it, and not another kind of map e.g.: a street map. Conventions, These are also an important part of software. People will [usually] expect your software to use conventions that are appropriate for the field. For example, The traditional London Tube map is a schematic diagram, designed to show the relative positions of the stations rather than their geographic location. Though, sometimes it's actually useful to have geographic information, e.g.: is Queensway (Central line) station very close to Bayswater (Circle line)? So if a map isn't using the schematic form, then the geographic form also has it uses. I would be surprised if I received a London Tube map that was neither schemati

Once upon a time there was a young and naive tester, he was new to the world of software testing. He often felt he didn't have what it took to be a tester. Sure, he found the odd bug, and he enjoyed his work, but he also often missed bugs, issues or problems. After a while, he admitted to himself that this was a problem, and decided to seek help. He stood up from his desk and walked over to his test manager's desk. His manager was wise and experienced. He was the Mr Miyagi of testing, and as such was always offering zen-like advice for his team. A simple question about where the stapler had escaped to could turn into a somewhat baffling series of Haiku , leaving our young tester baffled. Our novice explained his problem, and his concerns about how maybe he wasn't cut out for testing. The wise test manager smiled, thought for a moment and then opened his little Moleskine notebook. He turned carefully through the pages, settled on a page, looked up and said: "I over

Do you ever examine what you carry around with you every day, and wonder if you actually use it? For example, in my pockets I've got a 'smart' phone, wallet (credit & debit cards, cash and ID), keys, Travelcard ( Oyster ) and some coins. Every now and then something gets added, if its unlucky it stays. Over the years, I've noticed, that the criteria for being kept is usually convenience or enablement. That is, the items that don't get chucked or deposited somewhere about my home are usually 'tools' that make other 'things' easier like a smart-phone - I can just look up something or text someone at any time. I could just wait until I got back to my office, or see the person later but it can be easier to just act in the moment, and do it there and then. Enablement items, are things that mean I -can- do things, that without, I'm stuck. For example: door keys. The smart phone fits into this category also, if I want to meet up with someone at sh