Author Archive

Hacking Mouse Move Events Into Safari Driver The Nasty Way

Posted by on Sunday, 2 December, 2012

I thought long and hard before posting this entry because while it works, it’s not the real solution (Which is adding the code to enable the Actions classes into Safari driver, something I’m looking at right now to see if I can contribute something useful back to the Selenium community). I follow the Selenium/WebDriver mailing lists quite closley and regularly see people overcomplicating and hacking things, usually due to the fact that they did not get instant gratification when trying to do things the right way because they didn’t quite get it working the first time. I really hope people will not use this example to “try and get things working” in other browsers because they don’t understand how to use the Actions class, or they can’t be bothered to learn how to do things the right way.

Let me be very clear, this will work but it’s a hack and there are consequences:

  • You will no longer be able to run multiple safari instances on your local machine because there is only one mouse cursor and we will be using it (bye bye threading to speed your tests up in safari)
  • You will have messy code because you are going to have to put in code branches specifically for Safari mouse events.
  • You can’t run the tests in the background, Safari will need to be in the foreground and have focus while tests are running

If you can live with these issues and you really need to get Safari working with mouse events in the short term, this may just work for you.

Now that the warnings are out of the way let’s have a look at the problem…

Lot of modern websites are using hover events to make customised tooltips (well they aren’t tooltips in the HTML sense, but everybody still calls them tooltips) that appear when hovering your mouse over something like a chart point, or maybe there is some drag and drop functionality you need to test. The problem with Safari is that the Actions class hasn’t yet been implemented and the Safari driver object does not have an underlying mouse object. Game over? Not quite there is a way to get around the current limitations of the Safari Driver that will enable you to start clicking and hovering away like a mad man, the Java AWT Robot class (I’ll just call it the robot for the rest of this post).

The Robot is cross platform compliant so this solution could potentially work on any OS, but since Safari 6 is only available on OSX that is all we are interested in at the moment. The solution I have still uses Selenium for the majority of the heavy lifting, it is simply using the robot in place of the actions class to manipulate the mouse.

For all of these examples I have created a class called RobotPowered with the following private variables and constructor:

private final Robot mouseObject;
private final WebDriver driver;
private final JavascriptExecutor executor;
 
public RobotPowered(WebDriver driver) throws AWTException {
  this.mouseObject = new Robot();
  this.driver = driver;
  this.executor = (JavascriptExecutor) driver;
}

Now we know what is available to us let’s start creating the code to move the mouse in safari. First of all we have a basic robot implementation that will allow you to move the mouse to a specific X/Y coordinate on the screen

public void robotPoweredMoveMouseToAbsoluteCoordinates(int xCoordinates, int yCoordinates) {
  mouseObject.mouseMove(xCoordinates, yCoordinates);
  mouseObject.waitForIdle();
}

It really doesn’t get much easier than this, the code is self-explanatory and it will just work. We do have a problem however; we don’t know where the browser was loaded on the screen so if we tried to click on some coordinates we would effectively be clicking blind. On to part two:

public void robotPoweredMoveMouseToCoordinatesOnPage(int xCoordinates, int yCoordinates) {
  //Get Browser dimensions
  int browserWidth = driver.manage().window().getSize().width;
  int browserHeight = driver.manage().window().getSize().height;
 
  //Get dimensions of the window displaying the web page
  int pageWidth = Integer.parseInt(executor.executeScript("return document.documentElement.clientWidth").toString());
  int pageHeight = Integer.parseInt(executor.executeScript("return document.documentElement.clientHeight").toString());
 
  //Calculate the space the browser is using for toolbars
  int browserFurnitureOffsetX = browserWidth - pageWidth;
  int browserFurnitureOffsetY = browserHeight - pageHeight;
 
  //Calculate the correct X/Y coordinates based upon the browser furniture offset and the position of the browser on the desktop
  int xPosition = driver.manage().window().getPosition().x + browserFurnitureOffsetX + xCoordinates;
  int yPosition = driver.manage().window().getPosition().y + browserFurnitureOffsetY + yCoordinates;
 
  //Move the mouse to the calculated X/Y coordinates
  mouseObject.mouseMove(xPosition, yPosition);
  mouseObject.waitForIdle();
}

This now calculates where the browser is on the screen, the size of the browser and how much space is used up by browser toolbars. There is one caveat to the above code; you will need to disable the status bar. I haven’t found an easy way to work out how much space is used by the status bar, and how much is used by the rest of the browser so the simple solution is to just remove it.

This code is getting better, we now know that when we pass X/Y coordinates into the function it will click on the page, but that still leaves us guessing where on the page a specific WebElement is, well it’s not a problem Selenium actually knows the coordinates of the elements on the page and it can tell you where they are:

public void robotPoweredMoveMouseToWebElementCoordinates(WebElement element) {
  //Get Browser dimensions
  int browserWidth = driver.manage().window().getSize().width;
  int browserHeight = driver.manage().window().getSize().height;
 
  //Get dimensions of the window displaying the web page
  int pageWidth = Integer.parseInt(executor.executeScript("return document.documentElement.clientWidth").toString());
  int pageHeight = Integer.parseInt(executor.executeScript("return document.documentElement.clientHeight").toString());
 
  //Calculate the space the browser is using for toolbars
  int browserFurnitureOffsetX = browserWidth - pageWidth;
  int browserFurnitureOffsetY = browserHeight - pageHeight;
 
  //Get the coordinates of the WebElement on the page and calculate the centre point
  int webElementX = ((Locatable) element).getCoordinates().getLocationOnScreen().x + Math.round(element.getSize().width / 2);
  int webElementY = ((Locatable) element).getCoordinates().getLocationOnScreen().y + Math.round(element.getSize().height / 2);
 
  //Calculate the correct X/Y coordinates based upon the browser furniture offset and the position of the browser on the desktop
  int xPosition = driver.manage().window().getPosition().x + browserFurnitureOffsetX + webElementX;
  int yPosition = driver.manage().window().getPosition().y + browserFurnitureOffsetY + webElementY;
 
  //Move the mouse to the calculated X/Y coordinates
  mouseObject.mouseMove(xPosition, yPosition);
  mouseObject.waitForIdle();
}

We are now taking in a WebElement and getting the coordinates of its top left point. We then use the size and height of the element to work out its centre point which is where we move the mouse.

What about drag and drop? Well that’s easy we can use the robot to hold and release the mouse button as well.

public void robotPoweredMouseDown() {
  mouseObject.mousePress(InputEvent.BUTTON1_DOWN_MASK);
  mouseObject.waitForIdle();
}
public void robotPoweredMouseUp() {
  mouseObject.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);
  mouseObject.waitForIdle();
}

You now have everything you need to perform mouse actions with the Safari driver, unfortunately there is another caveat. I have found that when safari initially loads it doesn’t always have focus and when it doesn’t have focus it ignores the mouse events, the solution is to make the robot click on the window to set focus at the start of your test, here’s a quick click function to let you do just that:

public void robotPoweredClick() {
  mouseObject.mousePress(InputEvent.BUTTON1_DOWN_MASK);
  mouseObject.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);
  mouseObject.waitForIdle();
}

Finally here’s a little trick to find out what browser the current driver object is driving, you can use this to add in some specific code branches for safari:

((RemoteWebDriver) driver).getCapabilities().getBrowserName();

Once again I must reiterate that the above code is a hack and has its limitations, it does however get you out of a tight spot if you need to run your hover/drag and drop automated tests against Safari.

All of the above code is available on github at https://github.com/Ardesco/Powder-Monkey/blob/master/src/main/java/com/lazerycode/selenium/tools/RobotPowered.java

Introducing The Driver Binary Downloader Maven Plugin For Selenium

Posted by on Tuesday, 7 August, 2012

I’m a regular Maven user and one thing that has annoyed me for a while with Selenium is the standalone server binaries.  Not because I think the concept is a bad thing, but because it adds a dependency that has to be manually downloaded, in my mind this kind of defeats the object  of using Maven.  It wasn’t so bad when it was just the googlechrome executable, but recently the IEDriver has gone the same way and from Jim Evans’ presentation at at the Selenium conference  this year, it looks like everybody else will be following the trend at some point.

Well this left me with an itch to scratch and I have finally got around to doing something about it, introducing the driver-binary-downloader-maven-plugin.  It’s a bit of a mouthful but I didn’t want to use server in the name as people would think it was going to download the selenium server binaries, anyway it’s just a name, how bad can it be?

I have released it today (and it’s now available in the central Maven repository) at version 0.9.0 as it’s fully functional, but in my mind still in beta stage.

Basic Usage

The basic configuration is very simple, you tell it where to download things and it will go off and do it.

<plugins>
    <plugin>
        <groupId>com.lazerycode.selenium</groupId>
        <artifactId>driver-binary-downloader-maven-plugin</artifactId>
        <version>0.9.1</version>
        <configuration>
            <!-- root directory that downloaded driver binaries will be stored in -->
            <rootStandaloneServerDirectory>/my/location/for/driver/binaries</rootStandaloneServerDirectory>
            <!-- Where you want to store downloaded zip files -->
            <downloadedZipFileDirectory>/my/location/for/downloaded/zip/files</downloadedZipFileDirectory>
        </configuration>
        <executions>
            <execution>
                <goals>
                    <goal>selenium</goal>
                </goals>
            </execution>
        </executions>
    </plugin>
</plugins>

The way the plugin works is by downloading the zipped binaries into the <downloadedZipFileDirectory>and then performing a hash check on them to ensure that the zip files are valid.  If the files are valid it extracts the contents of the file in a directory structure based upon your &lt;rootStandaloneServerDirectory&gt;.  The file structure will be ${root.server.directory}/${driver.name}/${operating.system}/${bit.rate}/${version}/$driver.binary}.  So for example if you have downloaded the 64bit googlechrome driver binary version 22 for linux and your &lt;rootStandaloneServerDirectory&gt; is “/binaries” the absolute path to it will be “/binaries/googlechrome/linux/64bit/22/googlechrome”.

By default the plugin is run in the ‘test-compile’ phase so binaries will always be downloaded before your tests start to run.

What Do I Do With It?

Your mavenised test suite can now automatically download any required standalone server binaries without you needing to manually download them and place then in a shared area. If you had them checked into your source control repository (it happens) you can now remove them.

Since you now know where the binaries are going to exist once they have been downloaded you can use them in your selenium tests secure in the knowledge that if they don’t exist they will don’t once you have run a mvn verify

Known Problems

This is only a beta and as a result may not be perfect, there is currently one problem that I currently know about

  1. If the latest version of a driver does not have binaries for all the specified operating systems (all are specified by default) it will go back to the previous version and download all of those binaries as well.

Advanced Usage

While the above is useful, it’s not exactly extensible and doesn’t really help if you don’t want to download things from the internet every time. With this in mind there is a more advanced set of configuration options available.

<plugins>
    <plugin>
        <groupId>com.lazerycode.selenium</groupId>
        <artifactId>driver-binary-downloader-maven-plugin</artifactId>
        <version>0.9.1</version>
        <configuration>
            <!-- root directory that downloaded driver binaries will be stored in -->
            <rootStandaloneServerDirectory>/tmp/binaries</rootStandaloneServerDirectory>
            <!-- Where you want to store downloaded zip files -->
            <downloadedZipFileDirectory>/tmp/zips</downloadedZipFileDirectory>
            <!-- Location of a custom repository map -->
            <customRepositoryMap>/tmp/repo.xml</customRepositoryMap>
            <!-- Operating systems you want to download binaries for (Only valid options are: windows, linux, osx) -->
            <operatingSystems>
                <windows>true</windows>
                <linux>true</linux>
                <osx>true</osx>
            </operatingSystems>
            <!-- Download 32bit binaries -->
            <thirtyTwoBitBinaries>true</thirtyTwoBitBinaries>
            <!-- Download 64bit binaries -->
            <sixtyFourBitBinaries>true</sixtyFourBitBinaries>
            <!-- If set to false will download every version available (Other filters will be taken into account -->
            <onlyGetLatestVersions>false</onlyGetLatestVersions>
            <!-- Provide a list of drivers and binary versions to download (this is a map so only one version can be specified per driver) -->
            <getSpecificExecutableVersions>
                <googlechrome>18</googlechrome>
            </getSpecificExecutableVersions>
            <!-- Number of times to attempt to download each file -->
            <fileDownloadRetryAttempts>2</fileDownloadRetryAttempts>
            <!-- Number of ms to wait before timing out when trying to connect to remote server to download file -->
            <fileDownloadConnectTimeout>20000</fileDownloadConnectTimeout>
            <!-- Number of ms to wait before timing out when trying to read file from remote server -->
            <fileDownloadReadTimeout>10000</fileDownloadReadTimeout>
            <!-- Overwrite any existing binaries that have been downloaded and extracted -->
            <overwriteFilesThatExist>true</overwriteFilesThatExist>
        </configuration>
        <executions>
            <execution>
                <goals>
                    <goal>selenium</goal>
                </goals>
            </execution>
        </executions>
    </plugin>
</plugins>

The above options are fairly self explanatory so I’m not going to go into all of them in detail here, the most useful option however is probably <customRepositoryMap>. This option will allow you to specify your own RepositoryMap.xml (the file that specifies the drivers versions and download locations). This can enable you to have a local mirror for the driver binary zip files that you connect to, to download your Selenium server binaries to your local machine.

The repository map is checked using an xsd when it is loaded in so if you are planning on rolling your own you will probably find it useful to look at the current RepositoryMap.xml and the RepositoryMap.xsd that it is checked against.

If you want to have a look at the code for this plugin it is all publicly available here: https://github.com/Ardesco/selenium-standalone-server-plugin

Feedback

The one thing I really want now is feedback. Does it work the way you expect? Can you think of any improvements?

If you find any bugs please raise them on the Issues Page. The only way it will improve is if I know there are problems.

*Edit*

Just pushed out 0.9.1 with a fix for the issue highlighted in the comments below. The plugin now explicitly makes the files it extracts executable. I have updated the example POM files above with the new version number.

How To Download Files With Selenium And Why You Shouldn’t

Posted by on Wednesday, 25 July, 2012

In this blog post I will try and make you think why you are performing automated file download tests, and I will provide some Java code that will enable you to perform file downloads in a cross platform way without resorting to hacks like AutoIT.

First things first, don’t do file download tests!

Let’s start off with a scenario. You and the BA have talked to the product owner and they have said that they want to give the users some cool functionality that enables them to download some PDF’s with useful information in them. Everybody agrees that this is easy to implement and that you can very quickly run an exploratory test to check that it works by clicking on the download link in your browser. It downloads the file, you open it in PDF reader, all looks good and everybody is happy.

Now comes the tricky bit, you are asked to automate this scenario. After all, we want the build to go red on the CI server if some changes the developers make break your shiny new PDF download functionality. So, you load up Selenium and start replicating the actions you would take if you were playing the scenario out manually:

  • You load the page with the download link.
  • You find the <a> element on the page.
  • You click on it…

You have just fallen into a trap, the trap being that Selenium can’t deal with OS level dialogues so as soon as you click on the download link your test stops, you do not pass go and you don’t collect £200.

You go and have a look at the Selenium mailing lists and see lots of posts about AutoIT or maybe a post about a Java robot class and start looking at implementing one of these to interact with your OS level dialogue box…

STOP RIGHT THERE!

Now is the time to take a step backward and work out exactly what you want to test.

Do you really need to download that file?

I’m guessing your initial reaction is “Yes, I do. I need to make sure that the download functionality continues to work”. Sounds pretty reasonable so far; let’s go a further down this rabbit hole:

  • How many files are you planning to download?
  • How big are these files?
  • Do you have disk space to hold all of these files?
  • Do you have network capacity to continually download these files?
  • What are you planning to do with the downloaded file?

The last questions is where people usually stop and realise that they aren’t actually planning to do anything with the downloaded file. They are just planning to download the file and as long as a file has been downloaded they are happy that the test has passed. Now ask yourself, do you really need to download a file to perform this test. All you are actually doing is checking that when you click on a link you are getting a valid response from the server. You aren’t checking that you can download the file, you are checking for broken links. This is a worthwhile test, but it doesn’t require you to actually download anything. So let’s put AutoIT back in its little box and give you some code that can check to see if the link is valid.

Checking that links are valid

It’s actually pretty simple, all you need to do is find the link on the page, extract a URL from its href attribute and then check to see if sending an HTTP GET request to that URL results in a valid response. To do this I have a URLStatusChecker class:

package com.lazerycode.selenium.urlstatuschecker;
 
import org.apache.http.client.methods.*;
 
public enum RequestMethod {
    OPTIONS(new HttpOptions()),
    GET(new HttpGet()),
    HEAD(new HttpHead()),
    POST(new HttpPost()),
    PUT(new HttpPut()),
    DELETE(new HttpDelete()),
    TRACE(new HttpTrace());
 
    private final HttpRequestBase requestMethod;
 
    RequestMethod(HttpRequestBase requestMethod) {
        this.requestMethod = requestMethod;
    }
 
    public HttpRequestBase getRequestMethod() {
        return this.requestMethod;
    }
}
package com.lazerycode.selenium.urlstatuschecker;
 
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpRequestBase;
import org.apache.http.client.params.ClientPNames;
import org.apache.http.client.protocol.ClientContext;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.cookie.BasicClientCookie;
import org.apache.http.params.HttpParams;
import org.apache.http.protocol.BasicHttpContext;
import org.apache.log4j.Logger;
import org.openqa.selenium.Cookie;
import org.openqa.selenium.WebDriver;
 
import java.io.IOException;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
import java.util.Set;
 
public class URLStatusChecker {
 
    private static final Logger LOG = Logger.getLogger(URLStatusChecker.class);
    private URI linkToCheck;
    private WebDriver driver;
    private boolean mimicWebDriverCookieState = true;
    private boolean followRedirects = false;
    private RequestMethod httpRequestMethod = RequestMethod.GET;
 
    public URLStatusChecker(WebDriver driverObject) throws MalformedURLException, URISyntaxException {
        this.driver = driverObject;
    }
 
    /**
     * Specify a URL that you want to perform an HTTP Status Check upon
     *
     * @param linkToCheck
     * @throws MalformedURLException
     * @throws URISyntaxException
     */
    public void setURIToCheck(String linkToCheck) throws MalformedURLException, URISyntaxException {
        this.linkToCheck = new URI(linkToCheck);
    }
 
    /**
     * Specify a URL that you want to perform an HTTP Status Check upon
     *
     * @param linkToCheck
     * @throws MalformedURLException
     */
    public void setURIToCheck(URI linkToCheck) throws MalformedURLException {
        this.linkToCheck = linkToCheck;
    }
 
    /**
     * Specify a URL that you want to perform an HTTP Status Check upon
     *
     * @param linkToCheck
     */
    public void setURIToCheck(URL linkToCheck) throws URISyntaxException {
        this.linkToCheck = linkToCheck.toURI();
    }
 
    /**
     * Set the HTTP Request Method (Defaults to 'GET')
     *
     * @param requestMethod
     */
    public void setHTTPRequestMethod(RequestMethod requestMethod) {
        this.httpRequestMethod = requestMethod;
    }
 
    /**
     * Should redirects be followed before returning status code?
     * If set to true a 302 will not be returned, instead you will get the status code after the redirect has been followed
     * DEFAULT: false
     *
     * @param value
     */
    public void followRedirects(Boolean value) {
        this.followRedirects = value;
    }
 
    /**
     * Perform an HTTP Status check and return the response code
     *
     * @return
     * @throws IOException
     */
    public int getHTTPStatusCode() throws IOException {
 
        HttpClient client = new DefaultHttpClient();
        BasicHttpContext localContext = new BasicHttpContext();
 
        LOG.info("Mimic WebDriver cookie state: " + this.mimicWebDriverCookieState);
        if (this.mimicWebDriverCookieState) {
            localContext.setAttribute(ClientContext.COOKIE_STORE, mimicCookieState(this.driver.manage().getCookies()));
        }
        HttpRequestBase requestMethod = this.httpRequestMethod.getRequestMethod();
        requestMethod.setURI(this.linkToCheck);
        HttpParams httpRequestParameters = requestMethod.getParams();
        httpRequestParameters.setParameter(ClientPNames.HANDLE_REDIRECTS, this.followRedirects);
        requestMethod.setParams(httpRequestParameters);
 
        LOG.info("Sending " + requestMethod.getMethod() + " request for: " + requestMethod.getURI());
        HttpResponse response = client.execute(requestMethod, localContext);
        LOG.info("HTTP " + requestMethod.getMethod() + " request status: " + response.getStatusLine().getStatusCode());
 
        return response.getStatusLine().getStatusCode();
    }
 
    /**
     * Mimic the cookie state of WebDriver (Defaults to true)
     * This will enable you to access files that are only available when logged in.
     * If set to false the connection will be made as an anonymouse user
     *
     * @param value
     */
    public void mimicWebDriverCookieState(boolean value) {
        this.mimicWebDriverCookieState = value;
    }
 
    /**
     * Load in all the cookies WebDriver currently knows about so that we can mimic the browser cookie state
     *
     * @param seleniumCookieSet
     * @return
     */
    private BasicCookieStore mimicCookieState(Set seleniumCookieSet) {
        BasicCookieStore mimicWebDriverCookieStore = new BasicCookieStore();
        for (Cookie seleniumCookie : seleniumCookieSet) {
            BasicClientCookie duplicateCookie = new BasicClientCookie(seleniumCookie.getName(), seleniumCookie.getValue());
            duplicateCookie.setDomain(seleniumCookie.getDomain());
            duplicateCookie.setSecure(seleniumCookie.isSecure());
            duplicateCookie.setExpiryDate(seleniumCookie.getExpiry());
            duplicateCookie.setPath(seleniumCookie.getPath());
            mimicWebDriverCookieStore.addCookie(duplicateCookie);
        }
 
        return mimicWebDriverCookieStore;
    }
}

This will take a URL supplied to it and then return an HTTP status code. If it’s there I would expect a 200 (OK) or maybe even a 302 (Redirect). If it’s not there, I would expect a 404 (Not found) or if things really went badly a 500 (Server Error). It’s up to you to define which HTTP status code is a pass or a fail, the above code will simply tell you what the HTTP status code is. The above code is a little more complex than just performing a HTTP GET, it also mirrors your WebDriver session so that you can access the same resources as the user you are currently logged in as.
To use it you would simply do the following:

@Test
public void statusCode404FromString() throws Exception {
    urlChecker.setURIToCheck(webServerURL + ":" + webServerPort + "/doesNotExist.html");
    urlChecker.setHTTPRequestMethod(RequestMethod.GET);
    assertThat(urlChecker.getHTTPStatusCode(), is(equalTo(404)));
}

That’s nice but I really do want to download the file
I know that there are some people who really do want to download the actual file and perform checks on it, so how should we do it?

Everybody raves about AutoIT, that’s a good solution right?

Well, no actually it’s not. AutoIT will only work on Windows so you can kiss goodbye to your cross platform testing. AutoIT will also be looking for a specific window name so you are going to need to have an AutoIT script for every different download dialogue that you trigger and if you are not calling it programmatically, but leaving it running in the background, it is going to automatically click on every download box that appears, not just the ones you want to interact with during your testing. Oh, did I also mention that you are going to have problems renaming the file you download?

OK that doesn’t sound so good, how about a Java robot class? Lots of people talk about them as well

That’s better; it can be cross platform compliant and you can rename files that you download, but it still has issues. With a Java robot class you will be either blindly clicking at a specific location, or trying to send keystrokes to the pop up dialogue in the hope that it is in the state you expect it to be in. This again means that you are probably going to have to have different robot classes for different operating systems and maybe for different browsers. It’s a lot of work and not guaranteed to be successful.

So what do I do then? Forget about it? Use Sikuli?

There is another option, you can use the information provided by Selenium to programmatically download the file and completely bypass the OS level dialogue:

package com.lazerycode.selenium.filedownloader;
 
import org.apache.commons.io.FileUtils;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.params.ClientPNames;
import org.apache.http.client.protocol.ClientContext;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.cookie.BasicClientCookie;
import org.apache.http.params.HttpParams;
import org.apache.http.protocol.BasicHttpContext;
import org.apache.log4j.Logger;
import org.openqa.selenium.Cookie;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
 
import java.io.File;
import java.io.IOException;
import java.net.URISyntaxException;
import java.net.URL;
import java.util.Set;
 
public class FileDownloader {
 
    private static final Logger LOG = Logger.getLogger(FileDownloader.class);
    private WebDriver driver;
    private String localDownloadPath = System.getProperty("java.io.tmpdir");
    private boolean followRedirects = true;
    private boolean mimicWebDriverCookieState = true;
    private int httpStatusOfLastDownloadAttempt = 0;
 
    public FileDownloader(WebDriver driverObject) {
        this.driver = driverObject;
    }
 
    /**
     * Specify if the FileDownloader class should follow redirects when trying to download a file
     *
     * @param value
     */
    public void followRedirectsWhenDownloading(boolean value) {
        this.followRedirects = value;
    }
 
    /**
     * Get the current location that files will be downloaded to.
     *
     * @return The filepath that the file will be downloaded to.
     */
    public String localDownloadPath() {
        return this.localDownloadPath;
    }
 
    /**
     * Set the path that files will be downloaded to.
     *
     * @param filePath The filepath that the file will be downloaded to.
     */
    public void localDownloadPath(String filePath) {
        this.localDownloadPath = filePath;
    }
 
    /**
     * Download the file specified in the href attribute of a WebElement
     *
     * @param element
     * @return
     * @throws Exception
     */
    public String downloadFile(WebElement element) throws Exception {
        return downloader(element, "href");
    }
 
    /**
     * Download the image specified in the src attribute of a WebElement
     *
     * @param element
     * @return
     * @throws Exception
     */
    public String downloadImage(WebElement element) throws Exception {
        return downloader(element, "src");
    }
 
    /**
     * Gets the HTTP status code of the last download file attempt
     *
     * @return
     */
    public int getHTTPStatusOfLastDownloadAttempt() {
        return this.httpStatusOfLastDownloadAttempt;
    }
 
    /**
     * Mimic the cookie state of WebDriver (Defaults to true)
     * This will enable you to access files that are only available when logged in.
     * If set to false the connection will be made as an anonymouse user
     *
     * @param value
     */
    public void mimicWebDriverCookieState(boolean value) {
        this.mimicWebDriverCookieState = value;
    }
 
    /**
     * Load in all the cookies WebDriver currently knows about so that we can mimic the browser cookie state
     *
     * @param seleniumCookieSet
     * @return
     */
    private BasicCookieStore mimicCookieState(Set seleniumCookieSet) {
        BasicCookieStore mimicWebDriverCookieStore = new BasicCookieStore();
        for (Cookie seleniumCookie : seleniumCookieSet) {
            BasicClientCookie duplicateCookie = new BasicClientCookie(seleniumCookie.getName(), seleniumCookie.getValue());
            duplicateCookie.setDomain(seleniumCookie.getDomain());
            duplicateCookie.setSecure(seleniumCookie.isSecure());
            duplicateCookie.setExpiryDate(seleniumCookie.getExpiry());
            duplicateCookie.setPath(seleniumCookie.getPath());
            mimicWebDriverCookieStore.addCookie(duplicateCookie);
        }
 
        return mimicWebDriverCookieStore;
    }
 
    /**
     * Perform the file/image download.
     *
     * @param element
     * @param attribute
     * @return
     * @throws IOException
     * @throws NullPointerException
     */
    private String downloader(WebElement element, String attribute) throws IOException, NullPointerException, URISyntaxException {
        String fileToDownloadLocation = element.getAttribute(attribute);
        if (fileToDownloadLocation.trim().equals("")) throw new NullPointerException("The element you have specified does not link to anything!");
 
        URL fileToDownload = new URL(fileToDownloadLocation);
        File downloadedFile = new File(this.localDownloadPath + fileToDownload.getFile().replaceFirst("/|\\\\", ""));
        if (downloadedFile.canWrite() == false) downloadedFile.setWritable(true);
 
        HttpClient client = new DefaultHttpClient();
        BasicHttpContext localContext = new BasicHttpContext();
 
        LOG.info("Mimic WebDriver cookie state: " + this.mimicWebDriverCookieState);
        if (this.mimicWebDriverCookieState) {
            localContext.setAttribute(ClientContext.COOKIE_STORE, mimicCookieState(this.driver.manage().getCookies()));
        }
 
        HttpGet httpget = new HttpGet(fileToDownload.toURI());
        HttpParams httpRequestParameters = httpget.getParams();
        httpRequestParameters.setParameter(ClientPNames.HANDLE_REDIRECTS, this.followRedirects);
        httpget.setParams(httpRequestParameters);
 
        LOG.info("Sending GET request for: " + httpget.getURI());
        HttpResponse response = client.execute(httpget, localContext);
        this.httpStatusOfLastDownloadAttempt = response.getStatusLine().getStatusCode();
        LOG.info("HTTP GET request status: " + this.httpStatusOfLastDownloadAttempt);
        LOG.info("Downloading file: " + downloadedFile.getName());
        FileUtils.copyInputStreamToFile(response.getEntity().getContent(), downloadedFile);
        response.getEntity().getContent().close();
 
        String downloadedFileAbsolutePath = downloadedFile.getAbsolutePath();
        LOG.info("File downloaded to '" + downloadedFileAbsolutePath + "'");
 
        return downloadedFileAbsolutePath;
    }
 
}

The above code will mimic your current WebDriver session and programmatically download your file to the system temp directory where you can perform further checks upon it (It tells you where it downloaded it to). It’s relatively simple to use, a couple of basic examples are shown below:

@Test
public void downloadAFile() throws Exception {
    FileDownloader downloadTestFile = new FileDownloader(driver);
    driver.get("http://www.localhost.com/downloadTest.html");
    WebElement downloadLink = driver.findElement(By.id("fileToDownload"));
    String downloadedFileAbsoluteLocation = downloadTestFile.downloadFile(downloadLink);
 
    assertThat(new File(downloadedFileAbsoluteLocation).exists(), is(equalTo(true)));
    assertThat(downloadTestFile.getHTTPStatusOfLastDownloadAttempt(), is(equalTo(200)));
}
 
@Test
public void downloadAnImage() throws Exception {
    FileDownloader downloadTestFile = new FileDownloader(driver);
    driver.get("http://www.localhost.com//downloadTest.html");
    WebElement image = driver.findElement(By.id("ebselenImage"));
    String downloadedImageAbsoluteLocation = downloadTestFile.downloadImage(image);
 
    assertThat(new File(downloadedImageAbsoluteLocation).exists(), is(equalTo(true)));
    assertThat(downloadTestFile.getHTTPStatusOfLastDownloadAttempt(), is(equalTo(200)));
}

But that’s not the same as clicking on a link and downloading the file…


Well, actually it is. When you click on the link your browser sends a HTTP GET request over to the webserver and then downloads the file to a temporary location, then it hands the file over to the operating system which then pops up a dialogue asking you where you really want to save it. All you are doing is taking the browser and the operating system out of the equation. Let’s face it, if the browsers download mechanism doesn’t work, there isn’t anything much you can do about it anyway (apart from raise a bug with the browser vendor).

I’ve tried it and it works, but how do I know I have the right file?

The most simple and obvious way to check that the file is correct is to compare it to a known good copy of the file. If the file we have downloaded matches the original file it must be the correct file. No doubt you are now thinking “But hang on a moment, that means I need to keep a copy of every file that I download and some of them are massive…”. Not quite, there is another option you can just store a hash of the known good file. Taking an unsalted MD5/SHA1 hash of a file will always produce the same hash for the same file. So all you need to do is take a hash of the file you have downloaded and compare it to a known good hash of the file. If the hash doesn’t match you can fail the test and then examine the file manually later to find out what went wrong.
The final bit of code I have to offer is a class that will perform a hash check for you:

package com.lazerycode.selenium.filedownloader;
 
public enum HashType {
    MD5,
    SHA1;
}
package com.lazerycode.selenium.filedownloader;
 
import org.apache.commons.codec.digest.DigestUtils;
import org.apache.log4j.Logger;
 
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
 
public class CheckFileHash {
 
    private static final Logger LOG = Logger.getLogger(CheckFileHash.class);
    private HashType typeOfHash = null;
    private String expectedFileHash = null;
    private File fileToCheck = null;
 
    /**
     * The File to perform a Hash check upon
     *
     * @param fileToCheck
     * @throws FileNotFoundException
     */
    public void fileToCheck(File fileToCheck) throws FileNotFoundException {
        if (!fileToCheck.exists()) throw new FileNotFoundException(fileToCheck + " does not exist!");
 
        this.fileToCheck = fileToCheck;
    }
 
    /**
     * Hash details used to perform the Hash check
     *
     * @param hash
     * @param hashType
     */
    public void hashDetails(String hash, HashType hashType) {
        this.expectedFileHash = hash;
        this.typeOfHash = hashType;
    }
 
    /**
     * Performs a expectedFileHash check on a File.
     *
     * @return
     * @throws IOException
     */
    public boolean hasAValidHash() throws IOException {
        if (this.fileToCheck == null) throw new FileNotFoundException("File to check has not been set!");
        if (this.expectedFileHash == null || this.typeOfHash == null) throw new NullPointerException("Hash details have not been set!");
 
        String actualFileHash = "";
        boolean isHashValid = false;
 
        switch (this.typeOfHash) {
            case MD5:
                actualFileHash = DigestUtils.md5Hex(new FileInputStream(this.fileToCheck));
                if (this.expectedFileHash.equals(actualFileHash)) isHashValid = true;
                break;
            case SHA1:
                actualFileHash = DigestUtils.shaHex(new FileInputStream(this.fileToCheck));
                if (this.expectedFileHash.equals(actualFileHash)) isHashValid = true;
                break;
        }
 
        LOG.info("Filename = '" + this.fileToCheck.getName() + "'");
        LOG.info("Expected Hash = '" + this.expectedFileHash + "'");
        LOG.info("Actual Hash = '" + actualFileHash + "'");
 
        return isHashValid;
    }
 
}

You can then use it in a test like the one below:

private final URL testFile = this.getClass().getResource("/download.zip");
 
@Test
public void checkValidMD5Hash() throws Exception {
    CheckFileHash fileToCheck = new CheckFileHash();
    fileToCheck.fileToCheck(new File(testFile.toURI()));
    fileToCheck.hashDetails("def3a66650822363f9e0ae6b9fbdbd6f", MD5);
    assertThat(fileToCheck.hasAValidHash(), is(equalTo(true)));
}

Hopefully I have managed to make you think twice about downloading files in your automated tests and provided a good cross platform/cross browser solution that will remove the need to add in yet another application to your test framework. The code above is a snapshot in time and will continue to be tweaked and updated as I’m made aware of problems, or think of better ways to do things.

One thing you may have noticed is that the code performing a status check and a file download is very similar.  I’m aware of this but wanted to keep it separate to reinforce the idea that you do not need to do file downloads, I will be merging both parts together in the near future.

If you want to have a look at the latest revision it’s available on Github as part of https://github.com/Ardesco/Powder-Monkey.

All feedback appreciated :)