Thursday 21 May 2009

Breaking the back of automation

I’ve worked in this test team for almost 8 years now and in that time we’ve had numerous cycles where we’ve tried to make automation work for us but each time we’ve failed for a number of reasons…and now we’re trying again.

Originally we tried with WinRunner, then we tried TestPartner, then we tried QTP and now we’re trying with Selenium. To clarify “automation”, I’m talking about record & playback here as we’ve successfully used code to make other aspects of testing easier but for the purposes of this blog I’m talking about record & playback.

There are numerous reasons why I don’t think we’ve managed to make automation work for us in the past, and these are a few of them:

• Test leads not being aware of the benefits of automation and therefore not thinking about automation early in the product development and ensuring the product’s designed with automation/testability in mind
• Skillset – i.e. testers having the appropriate knowledge, or interest, to do the coding around any R&P
• Interest
• Thinking too big, too soon
• Products not working with automation tools

We’re now coming back round to another automation cycle and this time the weapon of choice is Selenium. Selenium’s a hugely popular tool in the world of automation at the moment and our development team are actually using it to automation their dev tests and they’re building a large range of tests which they run as part of nightly builds.

I’m really hoping we can finally get to a point where our team understand what opportunities a tool like Selenium can offer them and how they can use it to their benefit. I want to see tools being used to automate the mundane, checking/verification type tests that we use people to perform so our testers can concentrate on thinking up complex tests which requires them to spend time gaining a deeper level of understanding about the product – something a tool will never accomplish.

Hopefully I won’t be writing the same blog in 12 months time.

4 comments:

Unknown said...

Cloud Testing (http://www.cloudtesting.com/) offer a service where you can upload scripts that you have captured in Selenium IDE to be run either manually or scheduled as part of a build and test process.

Joe said...

"Originally we tried with WinRunner, then we tried TestPartner, then we tried QTP and now we’re trying with Selenium."

I think that pretty much indicates that the tool isn't the problem, right?

Unless the other issues are solved, I see you writing this post again in 12 months.

Ray Claridge said...

Your points are valid ones - especially when it comes to tester skill sets. Because of this, I remind my team to keep it simple. Doing this keeps the maintenance to a minimal and ensures there's a cost payback.

Having used Rational Robot and Test Complete at previous companies, I see Selenium as an excellent free alternative.

Read my thoughts on automation at http://www.testertroubles.com/2008/12/when-to-automate.html

I think you will find my views very similar : - )

T.J. said...

Now, after this blog entry is more than 1 years old, I am excited to hear from you whether or not, this time, your automation approach was successful or failure ++1.

Cheers