I’ve worked in this test team for almost 8 years now and in that time we’ve had numerous cycles where we’ve tried to make automation work for us but each time we’ve failed for a number of reasons…and now we’re trying again.
Originally we tried with WinRunner, then we tried TestPartner, then we tried QTP and now we’re trying with Selenium. To clarify “automation”, I’m talking about record & playback here as we’ve successfully used code to make other aspects of testing easier but for the purposes of this blog I’m talking about record & playback.
There are numerous reasons why I don’t think we’ve managed to make automation work for us in the past, and these are a few of them:
• Test leads not being aware of the benefits of automation and therefore not thinking about automation early in the product development and ensuring the product’s designed with automation/testability in mind
• Skillset – i.e. testers having the appropriate knowledge, or interest, to do the coding around any R&P
• Interest
• Thinking too big, too soon
• Products not working with automation tools
We’re now coming back round to another automation cycle and this time the weapon of choice is Selenium. Selenium’s a hugely popular tool in the world of automation at the moment and our development team are actually using it to automation their dev tests and they’re building a large range of tests which they run as part of nightly builds.
I’m really hoping we can finally get to a point where our team understand what opportunities a tool like Selenium can offer them and how they can use it to their benefit. I want to see tools being used to automate the mundane, checking/verification type tests that we use people to perform so our testers can concentrate on thinking up complex tests which requires them to spend time gaining a deeper level of understanding about the product – something a tool will never accomplish.
Hopefully I won’t be writing the same blog in 12 months time.
Thursday, 21 May 2009
Wednesday, 13 May 2009
We ain’t got no SMART requirements
As a manager I write objectives for people in my team and I’m told these objectives have to be SMART; that’s Specific, Measurable, Achievable, Realistic and Timebound. And, as a manager, I have people within my team complaining about a lack of quality requirements across projects – they lack clarity, they don’t detail expectations, they’re ambiguous etc. So why don’t people write SMART requirements?
Requirements should be specific because an ambiguous requirement isn’t going to be a testable requirement and the more open to interpretation a requirement is the poorer it is. More importantly, your interpretation of a requirement may be an incorrect interpretation and that means you could end up testing and providing information which is of no value to your project team.
Requirements should be measurable, and by measurable I mean testable. If you can’t measure whether the requirement’s been met, or whether it performs as expected, it’s not testable.
Requirements should be achievable. Ok, there is scope for highly desirable requirements which may make it in to the product if time allows (ever seen that happen?!) but this should serve as a warning. Does it sound achievable to you? If it doesn’t you probably want to question it before you spend time designing tests for it…
Requirements should be realistic. Well, yes, but sometimes we don’t know until we try to implement a requirement that we can’t actually implement it in the given timescale, or because of timescales, and therefore it’s not realistic. But again, if it doesn’t sound realistic it should probably be questioned before you start spending time designing those tests.
Requirements should be timebound. Think performance. “Requirement 1: Function A must do Y in N seconds” Is that realistic? What if it doesn’t? Would we still release if it took an extra 10 seconds, 30 seconds or 10 minutes? How good are your performance requirements?
The SMART method of reviewing requirements has it’s flaws but it does highlight the need for good requirements. You might use SMART as a guide to review requirements as it’ll raise some questions for you to ask or another method might be to try and write early high level test cases against requirements as a way of teasing out ambiguity or a lack of testability. Whatever your method, it’s key that we understand product requirements and more importantly, it’s key we ensure our understanding is aligned with that of the project team.
Requirements should be specific because an ambiguous requirement isn’t going to be a testable requirement and the more open to interpretation a requirement is the poorer it is. More importantly, your interpretation of a requirement may be an incorrect interpretation and that means you could end up testing and providing information which is of no value to your project team.
Requirements should be measurable, and by measurable I mean testable. If you can’t measure whether the requirement’s been met, or whether it performs as expected, it’s not testable.
Requirements should be achievable. Ok, there is scope for highly desirable requirements which may make it in to the product if time allows (ever seen that happen?!) but this should serve as a warning. Does it sound achievable to you? If it doesn’t you probably want to question it before you spend time designing tests for it…
Requirements should be realistic. Well, yes, but sometimes we don’t know until we try to implement a requirement that we can’t actually implement it in the given timescale, or because of timescales, and therefore it’s not realistic. But again, if it doesn’t sound realistic it should probably be questioned before you start spending time designing those tests.
Requirements should be timebound. Think performance. “Requirement 1: Function A must do Y in N seconds” Is that realistic? What if it doesn’t? Would we still release if it took an extra 10 seconds, 30 seconds or 10 minutes? How good are your performance requirements?
The SMART method of reviewing requirements has it’s flaws but it does highlight the need for good requirements. You might use SMART as a guide to review requirements as it’ll raise some questions for you to ask or another method might be to try and write early high level test cases against requirements as a way of teasing out ambiguity or a lack of testability. Whatever your method, it’s key that we understand product requirements and more importantly, it’s key we ensure our understanding is aligned with that of the project team.
Friday, 1 May 2009
Get out of the office and be inspired
This isn’t particularly pertaining to software testing, but I thought I’d blog about it anyway. Yesterday I went down to London and spent the day at two schools, one primary, on secondary, in one of the more deprived parts of London. Working at an Education company it’s important that the employees who work here take the time to visit schools, our customers, and build an empathy for our end users, something which can be quite difficult when you develop and test market products.
The day in itself was absolutely fascinating. Schools have moved on dramatically since I were there (not that long ago) and it’s great to see students using web-based products to collaborate on assignments with students of other schools around the region, country, or world.
It’s been far to long since I’ve been to a school, probably a couple of years, long enough, but it was an inspiring day and I’ve come back to the office enthused about what we do as a company and how I can play my part in ensuring we meet the demands of our customers.
Similarly, I’ve felt the same when returning from something like SIGIST, or Eurostar. That time away from the office allows you to regain perspective, to think new thoughts, and to be inspired.
So, to my point! Getting out of the office to meet customers, go on a training course or attend work-related conferences is a great way to inspire and energise your self. We can all become tired, or bogged down in our day-to-day work and responsibilities and it’s important to know you don’t always need two weeks in the Maldives, however nice that’d be, to re-charge your batteries!
The day in itself was absolutely fascinating. Schools have moved on dramatically since I were there (not that long ago) and it’s great to see students using web-based products to collaborate on assignments with students of other schools around the region, country, or world.
It’s been far to long since I’ve been to a school, probably a couple of years, long enough, but it was an inspiring day and I’ve come back to the office enthused about what we do as a company and how I can play my part in ensuring we meet the demands of our customers.
Similarly, I’ve felt the same when returning from something like SIGIST, or Eurostar. That time away from the office allows you to regain perspective, to think new thoughts, and to be inspired.
So, to my point! Getting out of the office to meet customers, go on a training course or attend work-related conferences is a great way to inspire and energise your self. We can all become tired, or bogged down in our day-to-day work and responsibilities and it’s important to know you don’t always need two weeks in the Maldives, however nice that’d be, to re-charge your batteries!
Subscribe to:
Posts (Atom)
