"The Selenium Guidebook" and Thoughts on my Learning Process

"The Selenium Guidebook: How To Use Selenium, successfully" by Dave Haeffner does not specifically set out to teach you how to automate elements on web pages with the WebDriver API. What it does set out to do - as the full title suggests - is show how you can practically implement WebDriver automation to add business value, avoiding common pitfalls in the process.

And in that vein, this post doesn't exactly review the book. It does a bit of that, but it's more about my personal experience of working through it, and reflects on how I might improve my learning process in the "code" domain going forward.

Some basics about the book
I worked on the Ruby version of the book. A version for Java is also now available.

Topics the book covers include:

• Formulating an automation strategy
• Understanding what constitutes the right approach to writing your checks
• Using Page Objects/abstraction to make a more robust suite that your team will trust
• Running your checks against multiple system and browser configurations on Sauce Labs
• Parallelisation to keep large suites running in sensible time-frames
• How to set up suites which run automatically as part of continuous integration
• Multiple examples of how to approach specific automation challenges

What's great about the book:

• Haeffner's style is ultra-clear
• He provides not just a download of code from the book to compare yours with, but a real website to automate and practice on
• There is also additional supporting material, eg. videos, available to purchase

There were points when I was working through The Selenium Guidebook that I felt frustrated - not with the book, which is very good - but with my own lack of progress.

The frustrations came when something wasn't working for me and I felt I didn't have the knowledge to understand why or fix it.  I tried to think about why I was allowing myself to get frustrated.

Coping with a lack of coding knowledge
First, a slightly boring aside to explain where my knowledge stood when I started the book. 

I had studied WebDriver using Alan Richardson's excellent Java-based course about a year before working through this book. However, in the intervening time I had taken a job that didn't involve web testing and so my knowledge had gone stale. In resurrecting it, I decided to go back to coding with Ruby - which I had been learning the basics of previously - because I felt the less intuitive syntax of Java hadn't helped my learning of WebDriver.

Haeffner advises that an ability to code is not necessarily needed upfront. Whilst that is certainly true, in my experience learning is slower if you don't have a lot of coding knowledge.

I think the biggest problem caused by my lack of coding experience was not always being able to make good guesses about the cause/nature of errors - and error messages - I encountered, and therefore struggling to correct them.

Troubleshooting hints for the examples in the book could be helpful, but are probably impractical given the variety of problems students might face.

It might have been useful to know exactly which versions of particular Ruby gems Dave had tested his examples with. I'm not sure if it was ever really the cause of a problem I hit (there are still a few examples that I haven't managed to get working) but I did wonder at times whether an issue might relate to my having installed the latest version of gem X whereas Dave possibly had been working with an earlier one.

Putting in the hours
Dave Haeffner does very generously offer up his own contact details in the book to answer questions. I deliberately resisted that because I didn't think it was fair on the author; and not helpful to me to have the answers provided too easily.

Mostly I got failing examples working by putting in the hours and using the routes you might expect: - Google and StackOverflow to look up error messages
- seeing if the provided sample files ran on my setup and, if so, comparing that sample file with what I had written to find the difference.

And in one extreme case using a file comparison tool to try and find what the hell was different between my failing project and the provided one. (The difference turned out to be not in my Ruby code but with gems I was using not being listed in my gemfile.)

Of course, this "pain" is actually good for learning and I need to remember that when the frustration bites. When I eventually managed to get the HTTP status codes example (with browsermob proxy) working there was a real sense of achievement because I had had to do research/thinking/work on my own to get there.

By the time I had gone through all the material in the book I felt it had been a really good investment and I had stopped worrying about whether I should have been able to get through it more smoothly. I shall certainly be keeping The Selenium Guidebook on hand and coming back to it.

Finding the right IDE/Editor
Something practical that I think would have helped me, and that I still need to sort out, was either a better IDE or better knowledge of the one I was using.  (I suppose this too falls under the heading of a lack of coding experience.)

After looking for free Ruby IDEs, I went with Aptana Studio. Quite possibly I don't have Aptana set up correctly - I struggled even to find useful documentation - but I found it of limited use beyond picking up syntax errors.

For Alan Richardson's Java course I had used the suggested IDE, Jetbrains' IntelliJ.  And I missed its extensive code completion suggestions here, and its ability to pick up basic typos on top of syntax errors.  Sadly, Jetbrains' "Rubymine" IDE is not free.

I also found that running commands with Aptana Studio's built-in terminal (running Git Bash) wasn't always helpful. Like the time when I could not get my first Sauce Labs checks to run and wasted an hour or more trying to figure out what the syntax/structural error was that Aptana seemed to report. When I ran the check in a simple Windows command prompt instead I straight away saw more useful feedback that I simply had my Saucelabs credentials wrong.

Give myself a chance
But the simplest way I could have improved the process for me was to relax a bit. Not to feel I was working against the clock to get through the material. Not to overly criticise myself when I might spend a whole evening on a few pages but still not get the example working by bedtime.

And to give myself more credit when I eventually did chase down, and correct, my errors.

This is a generic flaw I sometimes show in my learning process - unrealistic expectations of how quickly and how fully I can process something.  It's something I will blog on separately in my next post .....

---------------

"The Selenium Guidebook: How to use Selenium, successfully" by Dave Haeffner is available now as an e-book in both Ruby and Java versions. A range of packages with extra supporting material are also available at the same link. 

Haeffner also produces a free weekly Selenium tips email .

Webinar: Practical Tips + Tricks for Selenium Test Automation (Dave Haeffner)

Notes on this webinar: presented by Dave Haeffner, and hosted by Sauce Labs
---------------------

Another Dave Haeffner webinar! This time rather than considering the broad why and how of introducing automation, he talks about what is certainly an area of expertise for him: specific ideas to extend, and/or overcome challenges in, Selenium browser automation.

The amount of good, free learning that Haeffner offers around Selenium is commendable. He also offers his own "the-internet" site/application which you can use to practice on - and for which the code is open-sourced on Github.

The format of the webinar was in line with Haeffner's free weekly Selenium tips email; and with the second half of his Selenium Guidebook.

(At least that's what I think the second half of the book does ... I'm still working through it!)

The learning here is not really about the details of using the WebDriver API to find, and interact with, elements in web pages. Rather it's about ways you can add value to your automation efforts.
My notes aren't extensive but here's what I took from it:

Headless automation

"Headless" automation, ie. performed without launching an actual browser instance, is especially useful for running on a Continuous Integration server providing feedback every time there is a build.

Haeffner mentioned:

-- Xvfb. Which is limited to running on linux, and which he suggests offers no real speed benefits over launching a browser

-- Ghostdriver (PhantomJS).  Which is multi-platform and does offer faster performance.

Visual Testing

I only became aware of the existence of "visual testing" recently and know very little about it. It sounded kind of counter-intuitive because checking for small changes in the UI was always exactly the kind of thing that automation couldn't sensibly do.  (I thought maybe this was just another example of over-promising by excitable automation tool vendors!)

However, there are now open-source tools that will work alongside WebDriver to look for problems like layout or font issues that WD on its own won't really handle. In effect giving "hundreds of assertions for a few lines of code".

This looked like interesting stuff although, without having tried it, it still sounds a little too good to be true and I would need convincing before I trusted it. As you'd expect, Haeffner does talk about the inherent complexity and that false positives are more of a problem than usual.  It seems like some human intervention is still necessary to confirm "failures" that the automation reports.

Proxy Tools

Configuring automation to run through a proxy tool (like Browsermob or Fiddler) opens up a range of extra options:

- Checking returned HTTP status codes at URLs

- "Blacklisting". An idea which appealed to me from past experience. Here you manipulate the traffic so that third-party elements like ads .... which are slow to load thus affecting your checks ... can be excluded.

- Load testing by capturing traffic and then converting it into a Jmeter file

- Checking for broken images by examining the status code of all IMG elements.  (Haeffner also talks about other ways to check for broken images without needing a proxy.)

Forgotten Password Scenarios

Difficult to fully automate something like forgotten password workflow when that typically involves the generation and receipt of email. At least I thought so.

But Haeffner describes how you can use Gmail's API to look for and scan an email rather than attempting the madness of automating your way in and out of the Gmail web front end.

A/B testing

Pesky A/B tests running on your site (Marketing! *shakes fist* ) can make automation checks fail because the page may not be as expected when the driver gets there. Haeffner shows ways to opt out of A/B tests when running your automation eg. by creating an opt-out cookie, or by using URLs which bypass the page variants.

File Management Scenarios

Scenarios involving file upload buttons are tricky because they will generally involve a standard system/OS file management dialog - which WebDriver can't automate. But get WebDriver to submit the full file path and you may be able to bypass system dialogs entirely.

Additional Explanatory Output

Haeffner showed how to add explicit messaging around the assertions in your checks by highlighting located web elements using Javascript. Having captured the original styling you can add your own styling with Javascript - and revert back to the original when done.


There was, of course, more information - and more detail in the webinar itself.
A recording of it is available here: http://sauceio.com/index.php/2015/04/recap-practical-tips-tricks-for-selenium-test-automation/