Skip to main content

Posts

Showing posts from 2015

Your software sucks (any data you give it)

At 1524h, On the afternoon of January 15th 2009, US Airways Flight 1549 was cleared for takeoff from Runway 4 at New York's La Guardia airport. The airplane carried 150 passengers and 5 flight crew, on a flight to Charlotte Douglas, North Carolina. The Airbus A320's twin CFM56 engines had been serviced just over a month prior to the flight. The plane climbed to a height of 859m (2818 feet) before disaster struck.

Passengers reported hearing several loud bangs and then flames being visible from the engines' exhaust. Shortly thereafter the 2 engines shut-down, robbing the Airbus of thrust and its primary source of electrical power.

At this point the Captain took over from the First officer and between them they spent the next 3 minutes both looking for somewhere to land, while also desperately trying to restart their aircraft's engines.

What Happened? A flock of birds had crossed the path of the Airbus and several had struck the plane. Both engines had ingested birds an…

Even the errors are broken!

An amused but slightly exasperated developer once turned to me and said "I not only have to get all the features correct, I have to get the errors correct too!". He was referring to the need to implement graceful and useful failure behaviour for his application.

Rather than present the customer or user with an error message or stack trace - give them a route to succeed in their goal. E.g. Find the product they seek or even buy it.

Graceful failure can take several forms, take a look at this Bing [search] Suggestions bug in Internet Explorer 11.

As you can see, the user is presented with a useful feature, most of the time. But should they paste a long URL into the location bar - They get hit with an error message.

There are multiple issues here. What else is allowing this to happen to the user? The user is presented with an error message - Why? What could the user possibly do with it? Bing Suggestions does not fail gracefully.
I not only have to get all the features correct, I…

Counting Images, a FireFox Add-on

Many of my clients ask me to test their content management and processing systems. Often this involves investigating how the software handles images of various sizes as well as text of various lengths or types.

To help create test-images, I created this little FireFox Add-on. The Counting Images add-on starts with one click and can be used to create an image of a custom size.

For example: if you need a 300x250 MPU advert image - just enter 300 and 250 into the panel and click Create Image. To download the image, just click on it - as you would a a link and choose Save.

The image files are named widthxheight.png, and include markings to help identify if they have been truncated e.g.:



The marked numbers refer to the size in pixels of the rectangle they are in. E.g.: the blue rectangle (always the outermost one) is 150x100 pixels in size.

Another example:
As you can see the rectangles start at the defined size and count down in steps of 20 pixels.

What could go wrong? Well a good example…

Learning from the Boeing 787's broken software.

Earlier this year Boeing 787 engineers were given some new instructions by the FAA (The US government's: Federal Aviation Authority). They were informed that if the aeroplane's electrical generators were left running for 248 days, they would enter fail-safe mode. 
In plain English: they will stop producing electrical power. This short video looks into why that might be and how this information can hep us test our software.



The FAA directive is available on their website.A Guardian article: Boeing 787 bug could cause 'loss of control' eamliner-bug-could-cause-loss-of-control

Bug Automation

In many of my clients, more effort is spent on 'test automation' than on other forms of testing or quality assurance. That can be the right choice, for example, I worked on a Data Warehousing project where we needed to write some test automation before we could test the data and its processing.

Many other projects in different technology areas also spend a lot of time on their test automation. To be precise, they spend an increasing amount of time fixing & maintaining old 'tests' and 'frameworks'.

There are great tools around to help us write these automated checks quickly. But as with many software systems: maintenance, in the long term, is where the time and money goes. That is why I'm surprised we don't use short term automation more. We have the skills.

One good example of short term automation is Bug Automation. A simple script / executable that recreates or demonstrates a bug. This isn't a new idea, I've been doing it for years and I …

VW behaving badly.

The EPA (The US government's Environmental Protection Agency) recently issued Notice of Violations regarding the emissions from Volkswagen cars. Volkswagen is actually a group of brands, therefore the Notice affects other cars such as Audi, Porsche and Skoda.

A lot of the focus has been on what was going on in Volkswagen, for example who knew what was being done? Did the VW testers know? Did they pass the details on etc.

What interests me is the wider issue of how this could have been possible for so long?  (Since 2009)  If so many cars were affected and for so long, why didn’t we hear about this sooner? Why isn’t there a team of people assigned to finding this stuff out... Oh wait, there is...

In the UK these emissions tests are governed by the Vehicle Certification Agency, answering to the Department of Transport.

One might expect the manufacturer to be less inclined to investigate the cars emissions, after-all testing costs money (less profit). I might also expect them to exp…

'No More ASCII' Firefox Add-on

Many of my clients have a multi-national (and multi-lingual) user base, and their software receives input from a range of devices, not just those configured to UK or US locales. The sites may also need to process and publish content that is 'non-ASCII'.

So when I'm quickly testing a website or web application, I need to investigate how they handle inputs from a multitude of locales, quickly.

That's why I created the No More ASCII, a Firefox Add-on, it has a set of stock text strings from a range of languages and scripts. These have been chosen for their widespread use around the world, as well as their ability to highlight deficiencies in many web-sites. For example these features of the scripts can cause problems for ASCII/poor-Unicode implementations:
Right To Left text  - HebrewDiacritics - SwedishNon-Roman - Mandarin, Hindi etc. The text strings may not make 'sense' as some are partial sentences or Monty Python quotes. They are aimed to have a selection of c…

Counting Strings Firefox Addon

A while back I created a simple web based tool that helped you create text strings of a specified length. The text strings are created to make it easy to tell their length even if they are truncated.

The tool was based on a similar tool by James Bach, called perlclip.

I've now updated my Counting Strings script to be a free Firefox add-on. So you can now have it with you where ever you test online. You don't even need to restart your browser.



Build, Test, Ship, Learn, Rinse & repeat.

Ever feel like your team is in a deadlock? The product owner wants Gizmo+ to be shipped, your senior engineers are split between grokking Gizmo+ and fixing Widget++. Meanwhile the SDETs are franticly updating automated checks/BDD scripts and exploratory testers are uncovering that Widget+ and Gizmo+ should have been named ...+10 given the number of surprise bonus features they are finding. As a consequence feature delivery can start to slow and quality is inevitably hit as difficult decisions are made on what to fix.

The typical reactions to such a situation can depend on your project's context, but to highlight a few common ones:
Ramp up team size. Push back on deadlines.Push back on new features.Delay releases until 'it all gets sorted' ... I don't have to break it to you that these options are 'far from optimal'. In summary they all revolve around costing more and delivering less (from my time as a programme manager I can tell you - thats what we call a hard …