investigating software

It's August 1941, during the opening years of WW2, the axis powers had conquered much of mainland Europe -and the Allies were desperately trying to limit the capacity of the axis military industrial complex. They did this with little feedback on their success - much like some modern teams using AI tools. The Royal Air Force's (RAF) strategic bombing campaign had applied considerable resources to the task of diminishing continental targets such as occupied French ports or the factories of the Ruhr valley. Much as the northwest and northeast of England were the industrial centres of Britain, the Ruhr was where a large part of the Axis industrial might lay, and so was an important military target.  The air crews reported back after their sorties (missions), stating how successful their missions had been. If such reports were to be believed, Germany and the axis powers should be decimated, be ‘on their knees’, and would fall or at least falter in short order. Wisely the air min...

Sometimes people discuss test engineers and QA as if they were a sort of police force, patrolling the streets of code looking for offences and offenders. While I can see the parallels, the investigation, checking the veracity of claims and a belief that we are making things safer. The simile soon falls down. But testers are not on the other side of the problem, we work alongside core developers, we often write code and follow all the same procedures (pull requests, planning, requirements analysis etc) they do. We also have the same goals, the delivery of working software that fulfills the team’s/company's goals and avoids harm. "A few good men" a great courtroom drama, all about finding the truth. Software quality, whatever that means for you and your company is helped by Test Engineers. Test Engineers approach the problem from another vantage point. We are the lawyers (& their investigators) in the court-room, sifting the evidence, questioning the facts and viewing t...

"Don't get off the boat." "Seriously, never get off the boat," The instructor said, leaning forward and looking at each of us in turn. "But surely if it's sinking..." We reply, somewhat confused and slightly incredulous. We've seen Titanic, we think to ourselves, we know how this sea survival stuff works... "OK" He concedes, If things get really bad, "Get on the life raft if you can step- up from the boat to the life raft". "But, But... the yacht is like 37ft long, Do we want to wait until that whole boat is lower than the life-raft? When less than 1ft of the yacht is above the surface? Meanwhile all the time the life raft is just there... floating happily alongside." "Pretty much, yes," he said nodding. The movie Apocalypse Now speaks the truth. That was about 15 years ago. Not much has changed since. The reasons are manifold. Firstly, the yacht is a decent shelter. The thin plastic ...

Thank you to the colleague/customer/product owner, who found the bug I missed. That oversight, was (at least in part) my mistake. I've been thinking about what happened and what that means to me and my team. I'm happy you told me about the issue you found, because you... 1) Opened my eyes to a situation I'd never have thought to investigate. 2) Gave me another item for my checklist of things to check in future. 3) Made me remember, that we are never done testing. 4) Are never sure if the application 'works' well enough. 5) Reminded me to explore more and build less. 6) To request that we may wish to assign more time to finding these issues. 7) Let me experience the hindsight bias, so that the edge-case now seems obvious!

Google Maps directs me to and from my client sites. I've saved the location of the client's car parks, when I start the app in the morning - it knows where I want to go. When I start it at the end of the day, Google knows where I want to go. This is great! It guides me around traffic jams, adjusts when I miss a turn and even offers faster routes en-route as they become available. But sometimes Google Maps does something wrong. I don't mean incorrect, like how it sometimes gets a street name wrong (typically in a rural area). I don't mean how its GPS fix might put me in a neighbouring street (10m to my left - when there are trees overhead). I mean wrong - As in something unfair and socially unacceptable. An action, that if a person did it, would be frowned upon. Example: Let’s assume a road has a traffic jam, so instead of the cars doing around 60 mph, we are crawling at <10 mph. In the middle of this traffic jam, the road has a junction, an...

After a weary few hours sorting through, re-running and manually double checking the "automated test" results, the team decide they need to "run the tests again!", that's a problem to the team. Why? because they are too slow. The 'test' runs take too long and they won't have the results until tomorrow. How does our team intend to fix the problem? ... make the tests run faster. Maybe use a new framework, get better hardware or some other cool trick. The team get busy, update the test tools and soon find them selves in a similar position. Now of course they need to rewrite them in language X or using a new [A-Z]+DD methodology. I can't believe you are still using technology Z , Luddites! Updating your tooling, and using a methodology appropriate to your context makes sense and should be factored into your workflow and estimates. But the above approach to solving the problem, starts with the wrong problem. As such, its not likely to find ...

Earlier this year Boeing 787 maintenance engineers were given some new instructions by the FAA (The US government's: Federal Aviation Authority). They were informed that if the airplane's electrical generators were left running for 248 days, they would enter the fail-safe mode.  In plain English: they will stop producing electrical power. This short video looks into why that might be and how this information can help us to test our software. The FAA directive is available on their website . A Guardian article: Boeing 787 bug could cause 'loss of control' on Dreamliner

I now cover this issue in more detail in my podcast ! The EPA (The US government's Environmental Protection Agency) recently issued Notice of Violations regarding the emissions from Volkswagen cars. Volkswagen is actually a group of brands, therefore the Notice affects other cars such as Audi, Porsche and Skoda. A lot of the focus has been on what was going on in Volkswagen, for example who knew what was being done? Did the VW testers know? Did they pass the details on etc. What interests me is the wider issue of how this could have been possible for so long?  ( Since 2009 )  If so many cars were affected and for so long, why didn’t we hear about this sooner? Why isn’t there a team of people assigned to finding this stuff out... Oh wait, there is... In the UK these emissions tests are governed by the Vehicle Certification Agency , answering to the Department of Transport. One might expect the manufacturer to be less inclined to investigate the cars emissions, after-a...

When you read a blog post like this, or an article on a website, can you be sure its the 'real thing'? How would you know if it had been doctored? Lets assume the 'server' is fairly secure and hasn't been hacked into. So the content is going to be OK isn't it?, it looks OK..? So we've checked the location bar at the top of our web browser and it definitely has the right website/company name. No funny-looking misspelled names, possibly meaning I'm reading a fake site. And to be doubly sure, the browsers location bar states its using HTTPS and even has that reassuring little padlock we've come to look for and trust. OK, so to recap: The website's server is secured. (Well - for the the purposes of this, lets give them the benefit of the doubt) The logo, words, content and layout all appear to be kosher. We are using the correct website address. (No unusual spellings e.g.: www.goole.com etc) The page is secured using HTTPS. (Warm glow from ...

Monday 3rd September 1827, A man steps off the road at the corner of Fifth and Elm, and walks into a store. He's frequented the store a few times since it opened, and he's starting to get to know the owner and his range of merchandise. In fact, like many of people in town he's becoming a regular customer. He steps up to the counter, both he and the store owner glance at the large clock hanging on the wall and nod in unison. The shop-keeper makes a note of the time, the two then begin a rapid discussion of requirements and how the shop keeper might be able to help. When they've agreed what's needed, the shop keeper prepares the various items, bringing them to the counter, weighed, measured and packaged ready for transport to the customers nearby holding. The store keeper then presents the bill to the customer, who glances at the clock again, and the prices listed on each of the items arranged around the store's shelves and then pays. The customer smiles as he l...

Someone once asked me if "Testing a login-process was a good thing to 'automate'?". We discussed the actual testing and checking they were concerned with. Their real concern was that their product's 'login' feature was a fundamental requirement, if that was 'broken' they wanted the team to know quick and to get it fixed quicker. A failure to login was probably going to be a show-stopping defect in the product. Another hope was that they could 'liberate' the testers from testing this functionality laboriously in every build/release etc. At this point the context becomes relevant, the answers can change depending the team, company and application involved. We have an idea of what the team are thinking - we need to think about why they have those ideas. For example, do we host or own the login/authentication service? if not, how much value is their in testing the actual login-process? Would a mock of that service suffice for our automated c...

Do your hear phrases like "All our users use [insert some technology]" spoken in your office? or possibly "We have a 'corporate standard' desktop". I have a lot. I have since my first job, back in the '90s. It's a commonly held belief in most of the client companies I've worked with. Programmers, testers, project managers and product owners frequently hold faith in the standard build. It -is- a matter of faith. Often based on little more than wishful thinking or at best very loose 'standards'. The problem isn't purely one of client machines or end-users. I've often seen servers defined as 'clones' that in fact have quite different properties. e.g. different versions of java or application servers or even different time. The blind faith on these standard systems has caught myself and colleagues out so many times that I now find myself instantly questioning the assumption, and encouraging others to do the same. Even in thi...