Monday, December 04, 2006

The Temptation of Free and Easy to Use Test Tools

&otAfter recommending those nice open source test tools such as Watir and Selenium to my colleagues @work (and to my boss of course) for quite some time, I begin to see certain dangers which come along with the availabilty of free high quality test tools. This fact combined with all those success stories all over the net, which state how easy to use those tools are or how fast a newbie could learn Ruby (which would be required in order to use Watir) seem to lead people to the wrong conclusion test automation was a simple task.

In the last view weeks I have seen several posts on the Watir mailing list from people asking how to read/write data in/from external files, how to get all the links on page (and after getting the answer they ask the same for all buttons...), asking what happens if you do $some_ruby_statement_here instead of trying it in irb or reading the (very fine) documentation. The attack of the newbies has begun ;-)

Don't get me wrong, I like the idea of test automation becoming more popular and I'd like to see more testers trying to improve their work with some automated testing. The point is, test automation has always been a serious engineering task. we all have seen test automation projects fail because people were not aware of that fact. Where is the difference in the vendors telling you record&playback is easy and works perfectly for your needs and "us" telling open source test tools and languages like Ruby would be well suited for beginners?

Many organisations didn't use test automation because of the high costs of a commercial test tool and lack of skilled personel to use them. Now with one of those restrictions gone some people seem to think of open source tools as the new and improved record&replay. Test automation with a free tool is still the same challenging task as it was with a commercial one.

There are some points which people who are new to test automation and start using Watir for example should keep in mind:

  • Ruby is not a good language for beginners! (im curious how Brian Marrick's forthcoming book Everyday Scripting with Ruby will prove me wrong on this one) It may be easier to learn than C or Perl but there are some very advanced concepts involved in Ruby and its syntax is huge compared to that of Python or VB. On the other hand if you are an experienced programmer Ruby is a very nice toy to play with - That's for sure.

  • You have to learn a few things about software engineering in order to be a good test automation engineer. Not to mention reading the documentation, asking your friend Google and finally use your own brain ;-)
The examples from the Watir mailing list I described before are very similar to experiences we have made with offshore test automation engineers in India. Many of them (not all, but many of them) dont have a deeper understanding of how test automation should be done and somehow they prefer to ask their team lead everything they could have googled or found in the documentation of the test tool on their own.

I believe the point im trying to make is: no matter how nice, sophisticated or "easy" those tools are and how much we like to encourage others to try them, we should never forget: a fool with a tool is still a fool...

People like Cem Kaner, James Bach, Bret Pettichord, Elisabeth Hendrickson and others have done a terrific job in educating testers and test managers about "good" test automation - Testers who take the new free test tools as the new record&playback will fail and we test automation engineers can start education our test managers and customers again.

Tuesday, November 07, 2006

how Selenium got its name

Today I found the a hint on how the nice Selenium tool got its name on some slides by Neal Ford:

– Selenium is the key mineral which protects the body from mercury toxicity (from
– Selenium compounds medicate mercury poisoning

funny ;-)

Sunday, November 05, 2006

yeah, finally some nice toys, ehm tools

as Elisabeth Hendrickson sums it up in a recent post on her blog: there has been a big change in the acceptance test automation tool landscape. With frameworks like Fit and FitNesse and tools like Selenium and Watir test automation engineers finally got some of the powerful solutions they always dreamed of.

Have you ever made a feature request for a commercial test tool or fought with the support of a test tool vendor? That's no fun. Now with very powerful tools (and each one of them has a very alive community) written in general purpose languages such as Ruby or Python. Yes, there are some nice tools written in Python as well. With all that Ruby hype lately we must not forget how perfectly well suited Python is for scripting your test cases (a clean and very simple syntax, many 3rd party libs and unicode support for example is way better than in Ruby).

The main advantage of these open source tools over the commercial tools is not the price if you ask me. It's the fun people have using them. If a commercial test tool gives you a hard time testing your software, you have to live with it or have to try the vendor's support. If Watir for example would not exactly behave the way you would need it for testing your special application you can easily change the code yourself (maybe with some help from the community - but it's definitely possible).

Hello World

"Hello World". This is my first post on blogger :-) While the adventures of my real life still will be blogged on the nice blueblog (in german) I would like to share my thoughts on software testing, test automation and related topics here.