An Observation on the challenge of being a tester

Last night, Monday night, was shopping night, so I met my girlfriend in Tesco’s after work and we set about our weekly shop. I noticed recently that Tesco’s have altered the setup of their tills slightly and they’ve changed the arrangement so that customers are now able to scan their clubcards without the cashier having to do it and the card payment device was also moved toward the end of the till, in the area where you place your shopping in to bags – whether this is to make the whole process quicker, or just remove tasks from the cashier I’m not sure.

Anyway, as we went through the tills and completed bagging up our goods I took my Tesco clubcard from my wallet and went to scan it through the clubcard-scanner, which is situated at adult waist height, pointing slightly downward. “You can’t use that”, declared the cashier, “We’re not using them anymore.” “You’ve only just had them installed haven’t you?”, said I. “Yeah, but they realised that the laser beams which scan the barcode can damage children’s eyes so we can’t use them”, “you’d have thought they’d have tested that”, she then added. I wasn’t sure how to reply to that, as a hundred reasons why it might not have been tested flew through my head and I almost told her I was a tester, but then I’d imagine that information would have fallen on deaf ears – what did she care what I did for a living? Instead, I just laughed, thanked her, and wandered off thinking that this was a good topic for a blog post.

So what makes it a good topic for a blog post? Well, for me it highlights the difficulties that we have as testers. On the one hand, it sounds like whoever manufactured and tested this product didn’t consider the types of users that would be using or impacted by this device and for that there is no excuse. But there’s also the consideration that the barcode reader may have been through tens of test cycles, with masses of defects found & fixed, or maybe the test team only had a week to test the damn thing before someone told them it had to be released and so they spent the whole week ensuring it could actually read barcodes and transmit that data to the client, rather than test or understand how dangerous the rays were from the device.

As testers we have an almost impossible job because there will always be untested situations, scenarios or environments at the point of release and we never truly know if those untested areas will be the very areas which throw up a customer critical defect upon release. With risk assessments, and good judgement, we can have a very good go and ensuring we’ve removed most of the risk ahead of release, but there will always be that percentage of risk we release with, and that’s what we’ll always worry about.

1st day of work in 2009 and straight into a coverage reporting conundrum

I’m working on something called Early Life Monitoring (as mentioned in a blog below) whereby I review any defects found after project release and review them to understand if they were found prior to release and if not, whether they should have been found by our test effort.

Today I’ve been looking through the list of issues to try and I’m attempting to give each missed defect a catchy category which highlights why those defects weren’t found prior to release which lead me to this problem; summarising defects which sit outside of your testing visibility? I.e. you didn’t test for it, and you didn’t communicate that you weren’t testing it.

To further detail, when I write a test strategy (or, when I did before a move into a management role) I’d define what I was testing, I’d define what I wasn’t testing but there’s still this huge area of the product which isn’t covered by those two; what is that area called? It’s like a testing abyss, those tests you don’t consider, or communicate; how do you communicate that to your departmental manager? Or your technical support representative who’s wondering why we didn’t test for that?

Currently, I’m at a loss. Maybe there isn’t a word to summarise it, maybe I shouldn’t need to look for that word, maybe my brain’s not sharp enough to find an answer to this question today…

Happy new year to all.