Skip to main content

Posts

Showing posts with the label tools

Software development is in the Doldrums

"Don't get off the boat." "Seriously, never get off the boat," The instructor said, leaning forward and looking at each of us in turn. "But surely if it's sinking..." We reply, somewhat confused and slightly incredulous. We've seen Titanic, we think to ourselves, we know how this sea survival stuff works... "OK" He concedes, If things get really bad, "Get on the life raft if you can step- up from the boat to the life raft". "But, But... the yacht is like 37ft long, Do we want to wait until that whole boat is lower than the life-raft? When less than 1ft of the yacht is above the surface? Meanwhile all the time the life raft is just there... floating happily alongside." "Pretty much, yes," he said nodding. The movie Apocalypse Now speaks the truth. That was about 15 years ago. Not much has changed since. The reasons are manifold. Firstly, the yacht is a decent shelter. The thin plastic ...

A h̶i̶t̶c̶h̶h̶i̶k̶e̶r̶'s̶ software tester's guide to randomised testing - Part 1

Mostly Harmless, I've talked and written about randomisation as a technique in software testing several times over the last few years. It's great to see people's eyes light up when they grok the concept and its potential.  The idea that they can create random test data on the fly and pour this into the app step back and see what happens is exciting to people looking to find new blockers on their apps path to reliability. But it's not long before a cloud appears in their sunny demeanour and they start to conceive of the possible pitfalls. Here are a few tips on how to avert the common apparent blockers. (Part 1) A good motto for software testing, as well as pan-galactic hitchhiking. Problem: I've created loads of random numbers as input data, but how will I know the answer the software returns, is correct? - Do I have to re-implement the whole app logic in my test code? Do you remember going to the fun-fair as a kid? Or maybe you recall tak...

Your software sucks (any data you give it)

At 1524h, On the afternoon of January 15th 2009, US Airways Flight 1549 was cleared for takeoff from Runway 4 at New York's La Guardia airport. The airplane carried 150 passengers and 5 flight crew, on a flight to Charlotte Douglas, North Carolina. The Airbus A320's twin CFM56 engines had been serviced just over a month prior to the flight. The plane climbed to a height of 859m (2818 feet) before disaster struck. Passengers reported hearing several loud bangs and then flames being visible from the engines' exhaust. Shortly thereafter the 2 engines shut-down, robbing the Airbus of thrust and its primary source of electrical power. At this point the Captain took over from the First officer and between them they spent the next 3 minutes both looking for somewhere to land, while also desperately trying to restart their aircraft's engines. What Happened? A flock of birds had crossed the path of the Airbus and several had struck the plane. Both engines had ingested bi...

Counting Images, a FireFox Add-on

Many of my clients ask me to test their content management and processing systems. Often this involves investigating how the software handles images of various sizes as well as text of various lengths or types . To help create test-images, I created this little FireFox Add-on . The Counting Images add-on starts with one click and can be used to create an image of a custom size. For example: if you need a 300x250 MPU advert image - just enter 300 and 250 into the panel and click Create Image . To download the image, just click on it - as you would a a link and choose Save. The image files are named width x height .png , and include markings to help identify if they have been truncated e.g.: The marked numbers refer to the size in pixels of the rectangle they are in. E.g.: the blue rectangle (always the outermost one) is 150x100 pixels in size. Another example: As you can see the rectangles start at the defined size and count down in steps of 20 pixels. What could...

Bug Automation

In many of my clients, more effort is spent on 'test automation' than on other forms of testing or quality assurance. That can be the right choice, for example, I worked on a Data Warehousing project where we needed to write some test automation before we could test the data and its processing. Many other projects in different technology areas also spend a lot of time on their test automation. To be precise, they spend an increasing amount of time fixing & maintaining old 'tests' and 'frameworks'. There are great tools around to help us write these automated checks quickly. But as with many software systems: maintenance, in the long term, is where the time and money goes. That is why I'm surprised we don't use short term automation more. We have the skills. One good example of short term automation is Bug Automation . A simple script / executable that recreates or demonstrates a bug. This isn't a new idea, I've been doing it for year...

'No More ASCII' Firefox Add-on

Many of my clients have a multi-national (and multi-lingual) user base, and their software receives input from a range of devices, not just those configured to UK or US locales. The sites may also need to process and publish content that is 'non-ASCII'. So when I'm quickly testing a website or web application, I need to investigate how they handle inputs from a multitude of locales, quickly. That's why I created the No More ASCII, a Firefox Add-on , it has a set of stock text strings from a range of languages and scripts. These have been chosen for their widespread use around the world, as well as their ability to highlight deficiencies in many web-sites. For example these features of the scripts can cause problems for ASCII/poor-Unicode implementations: Right To Left text  - Hebrew Diacritics - Swedish Non-Roman - Mandarin, Hindi etc. The text strings may not make 'sense' as some are partial sentences or Monty Python quotes. They are aimed to have ...

Counting Strings Firefox Addon

A while back I created a simple web based tool that helped you create text strings of a specified length. The text strings are created to make it easy to tell their length even if they are truncated. The tool was based on a similar tool by James Bach, called perlclip . I've now updated my Counting Strings script to be a free Firefox add-on . So you can now have it with you where ever you test online. You don't even need to restart your browser. Counting Strings opens right in your browser, without affecting your website.

Are you sure you've "completed" testing? A Guardian Content API example.

Testing doesn't complete, it might end, it might finish, but it doesn't complete. There's too much to test. If you ever need confirmation of this, test something, something that's been tested already. Better still test a piece of software, you know has been tested by someone you think is a brilliant tester. A good tester like you, will still find new issues, ambiguities and bugs. That's because the complexity of modern software is huge: as well as all the potential code paths of your code, there's all the other underlying code's paths and the near infinite domain of data it might process. Thats part of the beauty of testing, you have to be able to get a handle on this vast test space. That is, review a near infinite test-space in a [very] finite time-frame. We are unable to give a complete picture of the product to our clients. But we are also free to find out new issues, that have so far eluded others. In fact the consequences are potentially more drama...

Is your test automation actually agile? A Guardian Content API example.

In my last post I discussed how test automation could be used to do things that I couldn't easily do unaided. In that example, execute thousands of news 'content searches' and help me sort through them. With the help of some simple test automation I found some potential issues with the results returned by the REST API. In that case, I started out with the aim of implementing a tool. But your testing might not lead you that way, often your own hands-on investigation can find an issue. But you don't know how widespread it is, is it a one-off curiosity? or a sign of something more widespread.? Again, this is where test automation can help, and if done well, without being an implementation or maintenance burden. Many test automation efforts are blind to the very Agile idea of YAGNI or You Ain't Gonna Need it . They often presume to know all that needs to be tested in advance, deciding to invest most of their time writing 'tests' blindly against a specific...

Random text tool

I recently blogged about some of the tools I use , and how some are so useful I keep using them. As I mentioned, randomness is pretty useful, and I have tools to help me generate random text. A few of my readers requested a copy of my simple random text generating script, so I've decided to open it up for everyone to use and test. It will have bugs, like all software, please send details and I'll try and fix them. If you are interested in what UTF-8 is and what all that Unicode stuff is about, there is a great article by Joel Spolsky that explains all, and the wikipedia page is ok . To use it... First download the script, its on GitHub . The script is fairly short and is all in one file. You don't have to 'install it', its not a GEM. Second, make sure you have Ruby version 1.9 or greater. You need version 1.9, because Ruby didn't handle UTF-8 well in older versions. Thirdly run the script like this: ruby fuzzutf8.rb That will give you some usag...

Tools

Do you ever examine what you carry around with you every day, and wonder if you actually use it? For example, in my pockets I've got a 'smart' phone, wallet (credit & debit cards, cash and ID), keys, Travelcard ( Oyster ) and some coins. Every now and then something gets added, if its unlucky it stays. Over the years, I've noticed, that the criteria for being kept is usually convenience or enablement. That is, the items that don't get chucked or deposited somewhere about my home are usually 'tools' that make other 'things' easier like a smart-phone - I can just look up something or text someone at any time. I could just wait until I got back to my office, or see the person later but it can be easier to just act in the moment, and do it there and then. Enablement items, are things that mean I -can- do things, that without, I'm stuck. For example: door keys. The smart phone fits into this category also, if I want to meet up with someone at sh...

2.2250738585072012e-308

Meet my new friend 2.2250738585072012e-308, We've been hanging out recently. If you've not heard of him, he's about ten years old but thats pretty old in [dog and in] software years. He's getting pretty famous in his old age, but he had humble beginnings as a lowly bug report on a Sun Microsystems website. It's rumoured he was first discovered back in 2001, but his big break didn't come until recently , when it was realised that he has the potential to be a key component of a Denial of Service attack that could bring down many java based systems [that accept floating point numbers  as input]. This includes commonplace application servers like Tomcat, who accept floating point numbers as part of the HTTP protocol. 2.2250738585072012e-308 has now been placed firmly in my mental bag of tricks along with divide by zero, 2^32, null, imaginary numbers, localised floats and all the others that routinely get brought out to help me test and investigate software. Bu...

Counting Strings

If you've done the Rapid Software Testing course , then you'll probably be familiar with the Perlclip tool, from James Bach . If not, its a useful tool for generating strings of test text. In particular I find the Perlclip Counter-string function to be pretty useful. Counterstring builds a string that indicates its own length. E.g.: "*3*5*7*10*" The last asterisk is the 10th character. Now available as a Firefox Add-on ! I've taken the counterstring functionality and implemented it in HTML and Javascript. While this form does not do everything the original [and the best] does, It might be useful for it to be accessible anywhere via a web page. All credit for the usefulness of this goes to James Bach, all the bugs are probably my doing. Thats right - its got bugs, like you don't have to enter a character for the 'mark' or it lets you use numbers for the 'mark'. Older versions did odd things in IE6 etc. There are no doubt many more bugs......