Skip to main content

Posts

Showing posts with the label regression testing

Being a square keeps you from going around in circles.

After a weary few hours sorting through, re-running and manually double checking the "automated test" results, the team decide they need to "run the tests again!", that's a problem to the team. Why? because they are too slow. The 'test' runs take too long and they won't have the results until tomorrow. How does our team intend to fix the problem? ... make the tests run faster. Maybe use a new framework, get better hardware or some other cool trick. The team get busy, update the test tools and soon find them selves in a similar position. Now of course they need to rewrite them in language X or using a new [A-Z]+DD methodology. I can't believe you are still using technology Z , Luddites! Updating your tooling, and using a methodology appropriate to your context makes sense and should be factored into your workflow and estimates. But the above approach to solving the problem, starts with the wrong problem. As such, its not likely to find

A Good Run!

“We got a good run from the tests” the tester stated. “So what’s the story?” the scrum master asked. “85% Pass” comes the reply, meekly. “OK, just need to fix that 5% then.” The scrum master announces before striding off to announce that the team is only a couple of % away from success. Our tester takes a moment to try and process the exchange… Firstly , their own words: “We got a good run” Why had they said that? Well - in a sense - it was true. They had executed the tests before, and they had returned a much higher failure rate. But the code being checked was the same... OK, so there were at least 3 obvious ways to interpret the data. The app code meets the criteria checked by the tests. ( Based on test run 2 ) The app code does not meet the criteria checked by the tests. ( Based on test run 1 ) The tests are as reliable a the toss of the coin. ( Based on both test runs ) Its surprising how unlikely people are to choose (3). Secondly , the scrum

How to avoid testing in circles.

I once had an interesting conversation with a colleague who worked in a company selling hotel room bookings. The problem was interesting. Their profits depended on many factors. Firstly, fluctuating demand e.g.: Holidays, Weekends, Local events etc. Secondly, varying types of demand e.g. Business customers, Tourists, Single night bookings or e.g.: 11 day holidays. They also had multiple types of contracts on the rooms. For some, they might have had the exclusive right to sell [as they had pre-paid], for others they had an option to sell [at a lower profit] etc. My naive view had been they priced the room bookings at a suitable mark up, upping that markup for known busy times etc. For example a tourist hotel hotel near the Olympics would be a high mark up, the tourist hotel room in winter would have incurred less of a markup. Better to get some money than none at all). He smiled and said some places do that, but he didn't. He had realised his team had a bias towards making a hea

If it's not good testing, it's not good regression testing either.

Pick a coin from your pocket, and hold it at arms length. Take a good look. Now take another one, of the same denomination and hold it out at arms length as before. Based on your observations alone - can you say they are the identical? Lets go a step further. If someone had given you one coin to look at, then exchanged it for another, could you have determined whether they are the same or different coins? Maybe, yes? If the differences had been large enough e.g. one coin was heavily tarnished or scratched, then the different coins would be identifiable. Or if you'd been given the opportunity to examine the coin using magnifying equipment, you probably could of found differences. But lets assume our only test was a standard set of checks i.e.: viewing at arms length and comparing what we see with our notes/records. It's better than nothing, I would see some differences, some might be important ones. For example if my next coin was blank: I might have suspected an issue with

Just ban just!

Office meetings are interesting events. Seriously, even the most boring ones. There's usually an important reason for the meeting, even if that meeting has been lost in the sands of Outlook repeat-booking, or the present attendees feel strongly otherwise. There's usually some kind of agenda, although as the meeting progresses, you may suspect the real agenda is hidden. But none of that is what captures my imagination. What sticks in my mind is the perspectives of the people in the room, especially when they are talking about a subject close to my heart like testing. We can't help but give away our positions and perceived hierarchies when discussing what work we need to do, and how. For example have you heard someone utter, in a meeting, words to the effect of: "Just create a fully automated framework for our regression test pack, to test the future releases." Then continue on, as if they had asked to "borrow a pen", or if you could "just adjust t

Saving Time?

For those of you that don't know, I'm somewhat of an amateur horologist. I love clocks, watches and all sorts of time and date keeping gadgets. To feed my passion I've decided to invest in my own custom made timepiece. This device will be my first custom made high value item, in what I hope will one day be a great collection. To ensure I get what I want, I've taken some time and documented the following requirements for my new clock. They list what I want from the timepiece, and also what I don't want or need. Have a read through, and hopefully you'll see what it is I'm after. A new clock should be developed for my new home in London. The clock will need to: - Display the time. - Display in roman numerals and modern  'arabic' numerals . - Be accurate enough for household use - approx' to with in a few minutes. - Be ornamental - preferably with a intricately styled clock face. - Have a traditional square brass clock face - Be construct