Skip to main content

Testing, Testing, 1, 2, 3.

When I have a spare moment, I usually try and think about how to test something. In fact thats not true, what I do is actually test something. It might be an app on my phone, an online tool, parking-ticket machine or search engine. Usually it is what-ever is to hand, at the time. This is a good way to practice my skills, and can take as long as I have free. In fact having only moments is beneficial, you soon get better at finding out more issues - more quickly.

For example, a few moments ago I thought I'd test Google's currency converter. If you haven't seen it, it looks like this:


You enter a value and two currencies in the format shown, and Google will give you an answer with great precision. (I haven't examined the accuracy.)

Starting from this I varied the text slightly, using "euro" instead of "EUR", also swapping "gbp" and "euro" to see how precedence affected the results. This seemed to behave as expected, but it did make me think about how Google was parsing the query. How might I confuse Google? Could I get it to misinterpret the order of the currencies?

Inspired by this question I tried typing:



This was actually the result of me pausing while typing, and observing the automatically updated search results presented by Google. I had [probably] confused the parser into trying to convert the result of my currency conversion into metric length measurements. This seemed like odd behaviour, but possibly acceptable to Google.

Next I checked how the search engine handles the reverse...



Rather than converting to Imperial measurements, Google has stayed metric, but displayed the result in millimetres. This started me thinking that this area had not been heavily tested - or at least had not been a focus for bug fixing. I'd found two unexpected behaviours, albeit slight, in seconds.

So how could I use this to highlight something that may confuse users or undermine the confidence that a user might have in this product. If this was my product, I'd want to know about such issues, as they might be bad for business.

The next thing I tried was deliberately aimed at being typical if not a semantically perfect query, that Google might misinterpret. I used the '/' character. Commonly used informally to mean English OR.




This result was interesting. I can imagine a user typing this query, or copying similar text into Google. This seems like it would be a problem for some users, If only because its confusing. For many users it would seem Google is 'broken', especially those unfamiliar with imperial measurements. I'll stop documenting the process there, but the leads generated in these quick tests suggest more avenues of investigation. Its clearly easy to confuse the search engine's parser.

If I'd created these tests in advance, be that in a spreadsheet or test automation, how would I have jumped from one result to the next? By taking out the feedback-loop, I would have unlikely known to try those tests. I also would probably be still writing the tests, long after the point in time I had found the above information, and was on my way to finding more.

Comments

  1. Pete, thanks for sharing, not only it is interesting and made me laugh... but finally I know how much a centimeter costs.

    ReplyDelete
  2. Awesome! this is a great blog post

    ReplyDelete

Post a Comment

Popular posts from this blog

The gamification of Software Testing

A while back, I sat in on a planning meeting. Many planning meetings slide awkwardly into a sort of ad-hoc technical analysis discussion, and this was no exception. With a little prompting, the team started to draw up what they wanted to build on a whiteboard.

The picture spoke its thousand words, and I could feel that the team now understood what needed to be done. The right questions were being asked, and initial development guesstimates were approaching common sense levels.

The discussion came around to testing, skipping over how they might test the feature, the team focused immediately on how long testing would take.

When probed as to how the testing would be performed? How we might find out what the team did wrong? Confused faces stared back at me. During our ensuing chat, I realised that they had been using BDD scenarios [only] as a metric of what testing needs to be done and when they are ready to ship. (Now I knew why I was hired to help)



There is nothing wrong with checking t…

A h̶i̶t̶c̶h̶h̶i̶k̶e̶r̶'s̶ software tester's guide to randomised testing - Part 1

Mostly Harmless, I've talked and written about randomisation as a technique in software testing several times over the last few years. It's great to see people's eyes light up when they grok the concept and its potential. 
The idea that they can create random test data on the fly and pour this into the app step back and see what happens is exciting to people looking to find new blockers on their apps path to reliability.
But it's not long before a cloud appears in their sunny demeanour and they start to conceive of the possible pitfalls. Here are a few tips on how to avert the common apparent blockers. (Part 1) Problem: I've created loads of random numbers as input data, but how will I know the answer the software returns, is correct? - Do I have to re-implement the whole app logic in my test code?
Do you remember going to the fun-fair as a kid? Or maybe you recall taking your kids now as an adult? If so then you no doubt are familiar with the height restriction -…

Software development is in the Doldrums

"Don't get off the boat."

"Seriously, never get off the boat," The instructor said, leaning forward and looking at each of us in turn.

"But surely if it's sinking..." We reply, somewhat confused and slightly incredulous. We've seen Titanic, we think to ourselves, we know how this sea survival stuff works...

"OK" He concedes, If things get really bad, "Get on the life raft if you can step-up from the boat to the life raft".

"But, But... the yacht is like 37ft long, Do we want to wait until that whole boat is lower than the life-raft? When less than 1ft of the yacht is above the surface? Meanwhile all the time the life raft is just there... floating happily alongside."

"Pretty much, yes," he said nodding.


That was about 15 years ago. Not much has changed since. The reasons are manifold. Firstly, the yacht is a decent shelter. The thin plastic of a legal minimum life-raft isn't going to protect you fro…