investigating software

Shutter sync is an interesting artefact generated when we video moving objects. Take a look at this video of a Helicopter taking off: Notice how the boats are moving as normal, but the rotors appear to be barely moving at all. This isn’t a ‘Photoshop’. It’s an effect of video camera’s frame rate matching the speed/position of the rotors. Each time the camera takes a picture or ‘frame’ the rotors happen to be in approximately the same relative position. The regular and deterministic behaviour of both machines enables the helicopter to appear to be both broken and flying. The rotors don’t appear to be working, while other evidence suggests its rotors are providing all the lift required. What's so exciting is that this tells us something useful, as well as apparently being a flaw or fail. We could both assume the rotors move with a constant rotation, and estimate a series of possible values for the speed of the rotors, given this video. Your automated checks/tests can ...

Google Maps directs me to and from my client sites. I've saved the location of the client's car parks, when I start the app in the morning - it knows where I want to go. When I start it at the end of the day, Google knows where I want to go. This is great! It guides me around traffic jams, adjusts when I miss a turn and even offers faster routes en-route as they become available. But sometimes Google Maps does something wrong. I don't mean incorrect, like how it sometimes gets a street name wrong (typically in a rural area). I don't mean how its GPS fix might put me in a neighbouring street (10m to my left - when there are trees overhead). I mean wrong - As in something unfair and socially unacceptable. An action, that if a person did it, would be frowned upon. Example: Let’s assume a road has a traffic jam, so instead of the cars doing around 60 mph, we are crawling at <10 mph. In the middle of this traffic jam, the road has a junction, an...

After a weary few hours sorting through, re-running and manually double checking the "automated test" results, the team decide they need to "run the tests again!", that's a problem to the team. Why? because they are too slow. The 'test' runs take too long and they won't have the results until tomorrow. How does our team intend to fix the problem? ... make the tests run faster. Maybe use a new framework, get better hardware or some other cool trick. The team get busy, update the test tools and soon find them selves in a similar position. Now of course they need to rewrite them in language X or using a new [A-Z]+DD methodology. I can't believe you are still using technology Z , Luddites! Updating your tooling, and using a methodology appropriate to your context makes sense and should be factored into your workflow and estimates. But the above approach to solving the problem, starts with the wrong problem. As such, its not likely to find ...

“We got a good run from the tests” the tester stated. “So what’s the story?” the scrum master asked. “85% Pass” comes the reply, meekly. “OK, just need to fix that 5% then.” The scrum master announces before striding off to announce that the team is only a couple of % away from success. Our tester takes a moment to try and process the exchange… Firstly , their own words: “We got a good run” Why had they said that? Well - in a sense - it was true. They had executed the tests before, and they had returned a much higher failure rate. But the code being checked was the same... OK, so there were at least 3 obvious ways to interpret the data. The app code meets the criteria checked by the tests. ( Based on test run 2 ) The app code does not meet the criteria checked by the tests. ( Based on test run 1 ) The tests are as reliable a the toss of the coin. ( Based on both test runs ) Its surprising how unlikely people are to choose (3). Secondly , the scrum...

In many of my clients, more effort is spent on 'test automation' than on other forms of testing or quality assurance. That can be the right choice, for example, I worked on a Data Warehousing project where we needed to write some test automation before we could test the data and its processing. Many other projects in different technology areas also spend a lot of time on their test automation. To be precise, they spend an increasing amount of time fixing & maintaining old 'tests' and 'frameworks'. There are great tools around to help us write these automated checks quickly. But as with many software systems: maintenance, in the long term, is where the time and money goes. That is why I'm surprised we don't use short term automation more. We have the skills. One good example of short term automation is Bug Automation . A simple script / executable that recreates or demonstrates a bug. This isn't a new idea, I've been doing it for year...

Many of my clients have a multi-national (and multi-lingual) user base, and their software receives input from a range of devices, not just those configured to UK or US locales. The sites may also need to process and publish content that is 'non-ASCII'. So when I'm quickly testing a website or web application, I need to investigate how they handle inputs from a multitude of locales, quickly. That's why I created the No More ASCII, a Firefox Add-on , it has a set of stock text strings from a range of languages and scripts. These have been chosen for their widespread use around the world, as well as their ability to highlight deficiencies in many web-sites. For example these features of the scripts can cause problems for ASCII/poor-Unicode implementations: Right To Left text  - Hebrew Diacritics - Swedish Non-Roman - Mandarin, Hindi etc. The text strings may not make 'sense' as some are partial sentences or Monty Python quotes. They are aimed to have ...

A while back I created a simple web based tool that helped you create text strings of a specified length. The text strings are created to make it easy to tell their length even if they are truncated. The tool was based on a similar tool by James Bach, called perlclip . I've now updated my Counting Strings script to be a free Firefox add-on . So you can now have it with you where ever you test online. You don't even need to restart your browser. Counting Strings opens right in your browser, without affecting your website.

(This issue is now raised in OSVDB .) On the 20th October 2013, The SymphonyCMS project released version 2.3.4 of their Content Management System. The release included a security fix for an issue I’d found in their software. The bug made it much easier for people to gain unauthorised access to the SymphonyCMS administration pages. More about that in a moment. The date of the release is also relevant, its a couple of days shy of 60 days after I had informed the development team of the issue. When I’d informed the team of the bug, I’d mentioned that I’d blog about the issue, sometime on or after the 60 days had elapsed. (That was in line with my Responsible Disclosure policy at the time) Which product had the bug? Symphony CMS is a web content management system, built in PHP. It appears to be used by several larger companies & organisations, learn more here .  What was the bug? The forgotten password functionality in v2.3.3 had a weakness, This meant an at...

Can you see the 74? This is an Ishihara Color Test. Its used to help diagnose colour blindness, people with certain forms of colour blindness would not be able to read the text contained in the image. The full set of 38 plates would allow a doctor to accurately diagnose the colour-perception deficiencies affecting a patient. The test is ingenious in its concept, yet remarkably simple in its execution. No complicated lenses, lighting, tools or measuring devices are required. The doctor or nurse can quickly administer the test with a simple and portable pack of cards. The Ishihara test is an end to end test. Anything, from lighting in the room, to the brain of the patient can influence the result. The examiner will endeavour minimise many of the controllable factors, such as switching off the disco lights, asking the patient to remove their blue tinted sun-glasses and maybe checking they can read normal cards (e.g. your patient might be a child.). End to end tests like this are messy...

Someone once asked me if "Testing a login-process was a good thing to 'automate'?". We discussed the actual testing and checking they were concerned with. Their real concern was that their product's 'login' feature was a fundamental requirement, if that was 'broken' they wanted the team to know quick and to get it fixed quicker. A failure to login was probably going to be a show-stopping defect in the product. Another hope was that they could 'liberate' the testers from testing this functionality laboriously in every build/release etc. At this point the context becomes relevant, the answers can change depending the team, company and application involved. We have an idea of what the team are thinking - we need to think about why they have those ideas. For example, do we host or own the login/authentication service? if not, how much value is their in testing the actual login-process? Would a mock of that service suffice for our automated c...

I recently purchased a Samsung Galaxy Tab and an iPad2. Unlike many of my previous gadget purchases, these new gadgets have become very much part of the way I now work and play. One thing I like about them, is their tactile nature. You have a real sense that their is less barrier between you and what you want to do. If you want to do something - you touch it - and it 'just' does it. I don't have to look at a different device, click a couple of keys or move a box on a string  to get access to what I can see right in front of me. Features such as the haptic feedback provide a greater feeling that you are actually working with a tool, rather than herding unresponsive 'icons' or typing magic incantations into a typing device, originally conceived 300 years ago . The underlying software systems used in these devices is a UNIX variant, just like the computer systems that underpin the majority of real world systems from the internet to a developer's shiny Apple Ma...