Skip to main content

Testing, Testing, 1, 2, 3.

When I have a spare moment, I usually try and think about how to test something. In fact thats not true, what I do is actually test something. It might be an app on my phone, an online tool, parking-ticket machine or search engine. Usually it is what-ever is to hand, at the time. This is a good way to practice my skills, and can take as long as I have free. In fact having only moments is beneficial, you soon get better at finding out more issues - more quickly.

For example, a few moments ago I thought I'd test Google's currency converter. If you haven't seen it, it looks like this:


You enter a value and two currencies in the format shown, and Google will give you an answer with great precision. (I haven't examined the accuracy.)

Starting from this I varied the text slightly, using "euro" instead of "EUR", also swapping "gbp" and "euro" to see how precedence affected the results. This seemed to behave as expected, but it did make me think about how Google was parsing the query. How might I confuse Google? Could I get it to misinterpret the order of the currencies?

Inspired by this question I tried typing:



This was actually the result of me pausing while typing, and observing the automatically updated search results presented by Google. I had [probably] confused the parser into trying to convert the result of my currency conversion into metric length measurements. This seemed like odd behaviour, but possibly acceptable to Google.

Next I checked how the search engine handles the reverse...



Rather than converting to Imperial measurements, Google has stayed metric, but displayed the result in millimetres. This started me thinking that this area had not been heavily tested - or at least had not been a focus for bug fixing. I'd found two unexpected behaviours, albeit slight, in seconds.

So how could I use this to highlight something that may confuse users or undermine the confidence that a user might have in this product. If this was my product, I'd want to know about such issues, as they might be bad for business.

The next thing I tried was deliberately aimed at being typical if not a semantically perfect query, that Google might misinterpret. I used the '/' character. Commonly used informally to mean English OR.




This result was interesting. I can imagine a user typing this query, or copying similar text into Google. This seems like it would be a problem for some users, If only because its confusing. For many users it would seem Google is 'broken', especially those unfamiliar with imperial measurements. I'll stop documenting the process there, but the leads generated in these quick tests suggest more avenues of investigation. Its clearly easy to confuse the search engine's parser.

If I'd created these tests in advance, be that in a spreadsheet or test automation, how would I have jumped from one result to the next? By taking out the feedback-loop, I would have unlikely known to try those tests. I also would probably be still writing the tests, long after the point in time I had found the above information, and was on my way to finding more.

Comments

  1. Pete, thanks for sharing, not only it is interesting and made me laugh... but finally I know how much a centimeter costs.

    ReplyDelete
  2. Awesome! this is a great blog post

    ReplyDelete

Post a Comment

Popular posts from this blog

Why you might need testers

I remember teaching my son to ride his bike. No, Strike that, Helping him to learn to ride his bike. It’s that way round – if we are honest – he was changing his brain so it could adapt to the mechanism and behaviour of the bike. I was just holding the bike, pushing and showering him with praise and tips.
If he fell, I didn’t and couldn’t change the way he was riding the bike. I suggested things, rubbed his sore knee and pointed out that he had just cycled more in that last attempt – than he had ever managed before - Son this is working, you’re getting it.
I had help of course, Gravity being one. When he lost balance, it hurt. Not a lot, but enough for his brain to get the feedback it needed to rewire a few neurons. If the mistakes were subtler, advice might help – try going faster – that will make the bike less wobbly. The excitement of going faster and better helped rewire a few more neurons.
When we have this sort of immediate feedback we learn quicker, we improve our game. When the f…

Thank you for finding the bug I missed.

Thank you to the colleague/customer/product owner, who found the bug I missed. That oversight, was (at least in part) my mistake. I've been thinking about what happened and what that means to me and my team.

I'm happy you told me about the issue you found, because you...

1) Opened my eyes to a situation I'd never have thought to investigate.

2) Gave me another item for my checklist of things to check in future.

3) Made me remember, that we are never done testing.

4) Are never sure if the application 'works' well enough.

5) Reminded me to explore more and build less.

6) To request that we may wish to assign more time to finding these issues.

7) Let me experience the hindsight bias, so that the edge-case now seems obvious!

Google Maps Queue Jumps.

Google Maps directs me to and from my client sites. I've saved the location of the client's car parks, when I start the app in the morning - it knows where I want to go. When I start it at the end of the day, Google knows where I want to go.
This is great! It guides me around traffic jams, adjusts when I miss a turn and even offers faster routes en-route as they become available.
But sometimes Google Maps does something wrong. I don't mean incorrect, like how it sometimes gets a street name wrong (typically in a rural area). I don't mean how its GPS fix might put me in a neighbouring street (10m to my left - when there are trees overhead).
I mean wrong - As in something unfair and socially unacceptable. An action, that if a person did it, would be frowned upon.
Example:
Let’s assume a road has a traffic jam, so instead of the cars doing around 60 mph, we are crawling at <10 mph.
In the middle of this traffic jam, the road has a junction, an example is shown here: