Skip to main content

Posts

Showing posts with the label bias

AI Muggins

I play a card game called cribbage. I often play it with my son . One interesting part of the game is the muggins rule. This means that you can claim points from other players turns, if they miscount the score.  The scoring is slightly nerve racking, with each of us double and triple checking our scores, to avoid falling foul of ‘muggins’, that’s part of the fun.  But my son and I also find ourselves discussing other hands of cards, in a sort of alternate history version of the game. “So if I had a 7 instead of a 2 of hearts, then I’d get a double run and score at least 8 more points”.   “Yes Dad, if you had different cards then you would likely have a different score, but you don’t” he says while rolling his eyes.  This sort of bitter-sweet history rewriting is a convenient tool for us to swallow the awkward truth of the real world. We often create alternate things to object to.  Take Chat GPT 4 and tools like Copilot X. These are powerful tools, capable o...

How to avoid testing in circles.

I once had an interesting conversation with a colleague who worked in a company selling hotel room bookings. The problem was interesting. Their profits depended on many factors. Firstly, fluctuating demand e.g.: Holidays, Weekends, Local events etc. Secondly, varying types of demand e.g. Business customers, Tourists, Single night bookings or e.g.: 11 day holidays. They also had multiple types of contracts on the rooms. For some, they might have had the exclusive right to sell [as they had pre-paid], for others they had an option to sell [at a lower profit] etc. My naive view had been they priced the room bookings at a suitable mark up, upping that markup for known busy times etc. For example a tourist hotel hotel near the Olympics would be a high mark up, the tourist hotel room in winter would have incurred less of a markup. Better to get some money than none at all). He smiled and said some places do that, but he didn't. He had realised his team had a bias towards making a hea...

Wrong in front of you.

In 2008 I attended GTAC in Seattle , a conference devoted to the use of automation in software testing. Since their first in London in 2006 , Google have been running about one a year, in various locations around the world. This post isn't really about the conference, its about a realisation that I had the day after the conference. After the conference I went sight-seeing in Seattle. I rode the short Simpsons -like Monorail and took a lift up the Jetsons -like space needle. I enjoyed my time there, and found the people very friendly. The conference had been very technology focused, many (but granted, not all) speakers focused on tools and how to use tools. While useful, the tools are only part of testing - and even then they typically just support testing rather than "do" testing. The Seattle Monorail. While I was at the top of the Space needle, I took out my phone and like a good tourist started taking pictures. I'd typically take a couple of pictures then ...

Fishing for bugs.

You probably don't know, but I'm keen on fishing (Honest! Ok, maybe not, but bare with me...) I spend my free time, by the river bank or out on the sea searching for 'the big one'. The big catch that'll stand as tall as me, and feed my family for a week. My dream is to be the guy standing next to his prize-fish on that black and white picture behind the bar. Over the years, I've become reasonably skilled. I usually find a fish or two when ever I'm out on the imaginary water. I've learned where they live, where they spawn, and of course where's best to catch them. For example, There's a little bend in the river upstream from my home, that has some great fishing spots. The overhanging rocks protect small fish from the predatory eyes of birds, and people. Of course where there's small fish, there's usually the odd big fish or two. Ok, lets imagine that fishing had a profitable side too, and wasn't just a [fictitious] hobby. For exam...

Into the testing hinterland.

Why do we refer to our ancestors as Cavemen? The evidence of course! The cave paintings, the rubbish piles found in caves all round the world. It's simple, Cavemen lived in caves, they painted on the walls and threw rubbish into the corner of the cave. Thousands of years later we find the evidence, demonstrating they lived in caves. Hence the moniker 'caveman'. How many caves have you seen? Seriously, How many have you seen or even heard of? Now I'm lucky, as former resident of Nottingham [in the UK], I've at least heard of a few . But if you think about it, you probably haven't seen that many. Even assuming you've seen a fair-few, how many were dry, spacious and safe enough for human habitation? As you can guess, my point is: there probably isn't a great selection of prime cave real-estate available. It doesn't add up: The whole of mankind descended from cave [dwelling] men? Before you roll your eyes, and think I'm some sort of Creationist ,...

ID Skeptic

At a client site, a few years ago, I had an interesting discussion with a 'senior programmer'. Our discussion centred on a configurable home-page. A user could decide what news or other information etc, they wanted to display on their home page. They'd start off by being given a generic page - and the customer could add or remove certain types of content to customise their page. Once they saved the 'new look' site, their choices would be saved on the web-server. The company didn't want to force people to login, or even make them sign-up for an account. The goal was to keep it simple for the user. But they needed a way to uniquely identify the users, so they used an existing feature of the website. The first time a user came to the site, they were cookied with a 'unique' id 'code'. We could then use this identifier as a key in our database to store the details of what the users had configured their homepage to look like. The testers reading th...

Conspicuous in their absence

If you're a tester then you'll no doubt of heard phrases to the effect of "That's pretty unlikely", "Our users don't do that" or "Thats a fairly minor browser". Its been blogged about before , and elsewhere. The argument is many users are niche, novice, confused or from different backgrounds / viewpoints / languages. These are realistic and probably correct hypotheses, for many situations. From my experience, thats often where the discussion ends, someone makes a judgement call, and the issue is fixed, mitigated or ignored. More often, than not, its ignored. That decision should probably be a business decision, its their money. But can they make such a decision safely? We are asking for consent to 'not operate' or 'operate' on their software. To come to the right decision, they need to be fully informed. i.e.: Are we sure that the issue is indeed rare? Are they making a properly informed decision? For example what if th...

The arrogance of regression testing

Lets assume we know that our software is not perfect. How can it be? Its complex, mortals created it and we don’t have enough time to test every execution path & environment – so we could never be sure anyway. This is Ok - this is normal, testers deal with this situation every day. This tends to be a typical scenario... Our team has been working on some new features. They’re looking good, initial teething issues have been fixed and the new features are considered worthwhile enough and bug-sparse enough to be released into the wild. This is where things can get a little awkward. The team member’s opinions are often split across a wide spectrum. The relatively minor perceived impact of the work leads some to conclude that the work is ready for release as is. Other team members, who are possibly twice shy from previous ‘minor change’ induced problems, argue for a comprehensive ‘regression test’ of the software. There is usually a range of views in between suggesting for example only ‘...