Skip to main content

Posts

Showing posts with the label random

A h̶i̶t̶c̶h̶h̶i̶k̶e̶r̶'s̶ software tester's guide to randomised testing - Part 2

How would test a water sac? (Wow there, calm that tester brain... I know what you are thinking, Whats it used for? Who / what uses it? how long does it need to last? Does the temperature of the water matter?  Is it single use? etc. But let's assume a generic hiking or camping water sac for now) I'm guessing one of your suggestions includes filling it with water, shaking it a bit and checking for leaks. Seems kind of obvious right? but when it comes to software, we often do away with old-fashioned techniques such as filling something up and looking at it. Where's the machine learning test algorithm? Call this a BDD scenario? Can Selenium check for H₂0? I have to run this past the B.A... This is your software. We can treat randomly generated test data and inputs in much the same way as water . Data files or other inputs like user interactions are the ever-moving parts of our applications. Think about it, the code is entirely static - it's the state or data that i

The Obscure One

Heraclitus wrote these words 2500 years ago: "Ever-newer waters flow on those who step into the same rivers." or paraphrased in more colloquial English: You never stand in the same river twice. Known as the "The obscure one" to some of his contemporaries, he was known to make statements that were considered paradoxical and sometimes unhelpfully contradictory. I don't know about you  - but sometimes when discussing testing feedback - I feel like I am channeling the ghost of Heraclitus. His comments regarding walking through rivers are an apt description of our work with software and its versioning. Do we ever play with the same app twice? On a trivial level, we do. When we widen our view we can see that the waters have moved on. For example,  The time has changed. It may even have gone back to a previous date and time.  The code is probably located in a different memory location.  The app and operating system are probably facing different types of

Pick a card...

Take a pack of cards, shuffle them well, and place them on the desk in front of you. Could you accurately tell me what the order of the cards would be, without looking?   By Rosapicci - Own work, CC BY-SA 4.0 Now spend 2 weeks in a software development team, writing code & using services, and deploy that code to your cloud server environments. Could you tell me where the bugs would be, before looking? In both cases we have a rough idea of what’s in the product at the end. But the detail, how its actually going to play out? we have no realistic idea. Skeptical? Take the playing cards… If we lay the cards out one card at a time. Then the order in which they are laid out, has probably never been seen before. Ever.The number of permutations of the well-known 52 playing card pack is 80,658,175,170,943,878,571,660,636,856,403,766,975,289,505,440,883,277,824,000,000,000,000. OK, now let’s get back to our code. Even trivial apps, include dozens of code libr

A security bug in SymphonyCMS ( Predictable Forgotten Password Token Generation )

(This issue is now raised in OSVDB .) On the 20th October 2013, The SymphonyCMS project released version 2.3.4 of their Content Management System. The release included a security fix for an issue I’d found in their software. The bug made it much easier for people to gain unauthorised access to the SymphonyCMS administration pages. More about that in a moment. The date of the release is also relevant, its a couple of days shy of 60 days after I had informed the development team of the issue. When I’d informed the team of the bug, I’d mentioned that I’d blog about the issue, sometime on or after the 60 days had elapsed. (That was in line with my Responsible Disclosure policy at the time) Which product had the bug? Symphony CMS is a web content management system, built in PHP. It appears to be used by several larger companies & organisations, learn more here .  What was the bug? The forgotten password functionality in v2.3.3 had a weakness, This meant an attack

Nobody expects the...

In a previous post I discussed one method I use to improve my testing skills, spending spare minutes testing a machine or website that is readily at hand. The example I used was Google's search, in particular its currency conversion feature. This is useful for getting practice, and trying to speed up my testing, that is - finding information more quickly. Another activity I perform is watching someone else test something. As testers, we are often asked to be a second pair of eyes, as its assumed that a programmer might not notice some issues in their own code. The idea being that you will not be blinded by the same assumptions, and will hopefully find new issues with the software. Using the same logic, by watching someone else test, I can examine their successes and failures more easily. I've asked many people to test a variety of objects, usually things to hand, like a wristwatch or something I've recently bought. One recurring pattern I have noticed is how programmer

Is your test automation actually agile? A Guardian Content API example.

In my last post I discussed how test automation could be used to do things that I couldn't easily do unaided. In that example, execute thousands of news 'content searches' and help me sort through them. With the help of some simple test automation I found some potential issues with the results returned by the REST API. In that case, I started out with the aim of implementing a tool. But your testing might not lead you that way, often your own hands-on investigation can find an issue. But you don't know how widespread it is, is it a one-off curiosity? or a sign of something more widespread.? Again, this is where test automation can help, and if done well, without being an implementation or maintenance burden. Many test automation efforts are blind to the very Agile idea of YAGNI or You Ain't Gonna Need it . They often presume to know all that needs to be tested in advance, deciding to invest most of their time writing 'tests' blindly against a specific

Random text tool

I recently blogged about some of the tools I use , and how some are so useful I keep using them. As I mentioned, randomness is pretty useful, and I have tools to help me generate random text. A few of my readers requested a copy of my simple random text generating script, so I've decided to open it up for everyone to use and test. It will have bugs, like all software, please send details and I'll try and fix them. If you are interested in what UTF-8 is and what all that Unicode stuff is about, there is a great article by Joel Spolsky that explains all, and the wikipedia page is ok . To use it... First download the script, its on GitHub . The script is fairly short and is all in one file. You don't have to 'install it', its not a GEM. Second, make sure you have Ruby version 1.9 or greater. You need version 1.9, because Ruby didn't handle UTF-8 well in older versions. Thirdly run the script like this: ruby fuzzutf8.rb That will give you some usag

The arrogance of regression testing

Lets assume we know that our software is not perfect. How can it be? Its complex, mortals created it and we don’t have enough time to test every execution path & environment – so we could never be sure anyway. This is Ok - this is normal, testers deal with this situation every day. This tends to be a typical scenario... Our team has been working on some new features. They’re looking good, initial teething issues have been fixed and the new features are considered worthwhile enough and bug-sparse enough to be released into the wild. This is where things can get a little awkward. The team member’s opinions are often split across a wide spectrum. The relatively minor perceived impact of the work leads some to conclude that the work is ready for release as is. Other team members, who are possibly twice shy from previous ‘minor change’ induced problems, argue for a comprehensive ‘regression test’ of the software. There is usually a range of views in between suggesting for example only ‘