I’m working on something called Early Life Monitoring (as mentioned in a blog below) whereby I review any defects found after project release and review them to understand if they were found prior to release and if not, whether they should have been found by our test effort.
Today I’ve been looking through the list of issues to try and I’m attempting to give each missed defect a catchy category which highlights why those defects weren’t found prior to release which lead me to this problem; summarising defects which sit outside of your testing visibility? I.e. you didn’t test for it, and you didn’t communicate that you weren’t testing it.
To further detail, when I write a test strategy (or, when I did before a move into a management role) I’d define what I was testing, I’d define what I wasn’t testing but there’s still this huge area of the product which isn’t covered by those two; what is that area called? It’s like a testing abyss, those tests you don’t consider, or communicate; how do you communicate that to your departmental manager? Or your technical support representative who’s wondering why we didn’t test for that?
Currently, I’m at a loss. Maybe there isn’t a word to summarise it, maybe I shouldn’t need to look for that word, maybe my brain’s not sharp enough to find an answer to this question today…
Happy new year to all.
Subscribe to:
Post Comments (Atom)
1 comment:
Hi Simon,
How do you determine what you will be testing and what you wont be testing? I think if you can answer that question, part of the answer to the "test abyss" will reveal itself.
"I'd define what I was testing, I'd define what I wasn't testing but there's still this huge area of the product which isn't covered by these two." Do you mean something similar to this: I tested a, b, and c features, but only in a WinXP SP2 system. Therefore the features were tested, but not in all supported operating systems.
If this is what you are referring to as the "testing abyss", perhaps recalling the "Impossibility of Complete Testing" lesson from the BBST Foundations course will help you to sort this out.
I think you already hold most of the answers to the questions you are pondering. Sometimes the intended audience of our documents makes writing them more difficult because the recipients don't really have
knowledge/understanding of software testing. Maybe you can use this document, in part, as a tool to help educate the audience.
Post a Comment