Skip to main content

Posts

Showing posts with the label experiment

I for one welcome our new AI helper.

 I was lucky enough to have started my career in a small company and then in a start-up. Both provided me with an environment perfect for learning. I sat with experts who took time out of their day to help answer my questions. From them, I learned the basics of what I still use today.  I’ve built on those foundations, but things would have been much harder if I didn’t have those foundational moments of my career. I’m not just talking about technical skills, the mentoring on how companies work, consulting and how to be better generally.  But those technical skills were also a big part of it – and a part many people miss out on in their careers. The rise of Large Language Models like ChatGPT4 is rapidly helping to fill that gap – where people don’t have a technical mentor who can explain and help work through those technical problems.  I’m no longer that junior team member – asking the dumb questions (OK, well usually I’m not) but even I find Chat GPT excellent at consolidating a broad s

Avoiding Wild Goose Chases While Debugging.

When I’m debugging a complex system, I’m constantly looking for patterns. I just ran this test code... What did I see in the log? I just processed a metric $^&*-load of data, did our memory footprint blip? I’m probably using every freedom-unit of screen space to tail logs, run a memory usage tool, run an IDE & debugger, watch a trace of API calls, run test code… And I’m doing this over and over. Then I see it. Bingo, that spike in API calls hits only when that process over there jumps to 20% processor usage when the app also throws that error. Unfortunately, I may have been mistaken. On a sufficiently complex system, the emergent behaviour can approach the appearance of randomness. Combinatorial explosions are for real, and they are happening constantly in your shiny new MacBook. My bug isn’t what I think it is. I’m examining so many variables in a system with dozens of subsystems at play, it's inevitable I will see a correlation. We know this more formally as

Was there a test for that? No, and there shouldn't be.

The release shipped. For a while, the team felt good. The work was done, the team had achieved something, and that was rewarding.  Unfortunately for the team, it wasn't long before a problem was found. The Product Owner wasn't happy and had asked was going on down there in the galley, do we need new coders? Better ones? Hipster coders? The release shipped... After an investigation, some blushes, raised eyebrows and a couple of "Oh... Yeeeah's" they found the cause. A confusion had collided with a bodge, and the result was a mess. Should they write an automated - test for this problem?  An embarrassing mistake or a misstep can make us feel we have to do something . An action greater than a fix is needed. A penitance needs to be performed, to redeem ourselves, to make us right again. Sometimes the penitence is best spent adding a test for that issue. Especially if writing that test has a low cost, the frequency of the problem occurring is high or

The Like-Live Paradox

I was recently struck by a glaring difference between how I and a programmer prepared for testing. Unlike the majority of the testing I am involved in, this particular testing 'phase' had to be scheduled in advance and we couldn't "just do it". This also meant we had more time to prepare and plan than we typically do. This 'waiting period' had its uses. We had time to create tools that might be useful and check the configuration of the systems we would be testing. The team, familiar with the concepts of exploratory testing, were comfortable with an approach that meant we did not spend the majority of the time pre-scripting tests [be they coded or in a spreadsheet etc]. We did however build a high-level checklist of areas to test and used this to drive our program of tools and configuration checking/fixing/building. The key difference I observed was the absolute nature of the programmers comparisons between our test systems and our live-production syste