Windmill 1.0!

Just in time for PyCon we’re releasing [Windmill 1.0]( It’s been almost 3 years of development and I’m both excited and relieved to have something we’re comfortable calling 1.0.

The latest RCs have seen bigger improvements than we thought would make it. A new wave of contributions has given us some great new features in Django support and test serialization and [Adam]( has done some great UI work as well.

Big thanks to everyone who contributed to the release and I hope to see many of you at PyCon. Adam is doing a [talk on windmill]( which I’ll be helping out with and Adam and I will also be on a [functional testing tools panel]( which should be a lot of fun. And if you’re staying around for the sprint days of the conference we’ll be leading a [windmill sprint](

Skinning Windmill with JQuery UI Themes

I have been doing a lot of UI work on the Windmill trunk, and over the past few months I have had multiple requests for the ability to apply skins. Of course my reaction up until this point has been… alter the CSS! Which is not exactly the answer people were looking for.

Here is your answer, and it is now easier than ever.

1. Go to: and create your theme (or pick a pre-defined one)
2. Download and unzip the file. (it doesn’t matter what you select for jQuery components)
- On a mac you will find something like this: Folder
3. Open the contained CSS folder there will be another folder, currently windmill uses “smoothness”
4. Find windmill/html/css folder in the windmill source directory
5. Copy the specified folder in step 3 into this folder
6. Edit windmill/html/remote.html, line 10 to read like the following:

<link type="text/css" href="css/*your folder name*/jquery-ui-1.7.1.custom.css" rel="stylesheet" />

7. Load windmill!!

Here are a couple examples of pre-defined themes I tested out.

Windmill Theme
UI Darkness
Windmill Theme

Happy skinning.

PyCon Chicago 2009

I’ll be attending PyCon this year (finally!).

Somehow I haven’t attended the last 3 years that I tentatively planned to do so. 

Windmill will be well represented. Adam and I will be on a “Functional Testing Tools in Python” panel, and Adam will be giving a full blown Windmill talk.

There will also be a Windmill Sprint for anyone interested. If you want some hands on time figuring out how to test crazy webapp stuff we’ll be helping anyone who comes by the first few days of the Sprint. Our plan is to eventually transition the Sprint to working on the windmill2 codebase.

Feel free to stop by the Sprint or grab me during the conference if you want to discuss any of the other projects I work on (mozmill, jsbridge, pouch, etc) or if you want to discuss Mozilla testing and tools in general.

Meet lookupNode

A few months ago I did a re-write of the DOM access functionality used by Windmill and various other projects of mine and the result was a wrapper around ElementsLib called lookupNode. I use this all over internally but until today didn’t realize that it could be hugely helpful for test developers and people trying to debug their applications.

The following list are the different options you have for looking up your node:

  • link
  • xpath
  • id
  • name
  • value
  • classname
  • tagname
  • label

Keep in mind that we iterate all iframes we can access in the window if the node isn’t found in the page document. lookupNode takes an object, like so:

var myNode = lookupNode({id:'myNodeId'});

The very interesting use case we had today was the need to access and parse data from a link on a page in a Python test, I believe the final result looked something like the following:

url = client.commands.execJS(code="lookupNode({link:'Link Text'}).href")["result"]
#url parsing in python goes here into needed_value
client.asserts.assertValue({id:'mycheckbox', validator=needed_value})

lookupNode in Action

lookupNode in Action

I hope someone finds this useful, the abstracted standalone ElementsLib for the content space can be found here!

Windmill Gets a Facelift for 1.0Beta1

Working up to the Windmill 1.0 Beta 1 Release, I finally had the opportunity to put some time into making the IDE (that a lot of you live in when in test writing mode) a little bit nicer to look at.

The IDE has been growing organically since 0.1 and there was a lot of functionality hacked into it that wasn’t in the original game plan, so I did what I could to improve the beauty of the CSS/Layout as well as the whole mess of code behind it.

If you have used our latest release, or are running trunk you know that we have significantly improved the load times for the Windmill IDE. By compressing the JavaScript when the service is instantiated we can simply hand the IDE window one file that contains the vast majority of the required code.

The reason that this makes such a huge performance difference is because we are loading the source via the local windmill proxy and the data size size had very little impact, the overhead was in the browser two connection limit. When you have to pull down ~30 files two at a time it takes its toll and made the IDE feel very sluggish and more like a web page loading than an IDE.

In the process of figuring out exactly what was slowing down the launch time we added some more informative messages and output so you don’t sit there staring at a twirling circle graphic wondering if anything is happening. And to make the experience even more fun, I couldn’t help but implement a progress bar.

General Layout
I removed the toolbar at the bottom of the screen, which I felt it was an irritation for test editing (especially with the drag and droppable actions). It is now in a drop down menu at the top right of the screen, with the rest of the UI access to IDE functions.

Settings and Firebug Lite Improvements
The settings dialog has continued to improve by implementing more useful defaults, adding new options, removing deprecated options and simply making it just look better. Thanks jQuery UI!

Firebug Lite has been a very popular feature since we first announced it, which has led to a handful of bug fixes over the last month. The most major of these was that the initial Windmill implementation of Firebug Lite required you to have Internet access as it was using resources that were hosted elsewhere.

These have since been copied to our source tree and are made available by the Windmill server so you can happily introspect your Web Apps JavaScript while writing tests on your Intranet.

Output and Performance
Instead of writing all the raw windmill output to the output and performance tabs there is now an array called windmill.errorArr, where all terrible errors and warnings about technical details are pushed in the case you are interested to see all that data. However, it’s more likely that you aren’t and scrolling through all that output data becomes tedious.

This is why we have implemented output in blocks with the background color representing pass/failure with green/red (white for performance). These blocks are expandable, clicking them will reveal all output (or performance information) we know about the action that was executed. This should give you a faster general overview of your results and allow you to quickly see the details you care about.

Other Worthwhile Mentions
We moved our XPath implementation from Ajax-Slt to JS-XPath, which has proven to be more accurate when it comes to resolving XPath generated in Firefox (or using Firebug) against non XPath native browsers such as IE.

Many bugs and improvements have been made to the DOM Explorer, which should now feel a lot more like the Firebug DOM inspector, but should work in any browser.

We have also put a lot of effort into improving the communication between the JavaScript Controller and the Python Service so that when a test fails you get as much detailed information in the service as you do in the IDE.

Timing and MozMill
The timing has lined up nicely as we are working on both a 1.0 release for Windmill and MozMill. MozMill is geared towards automated testing of all applications on the Mozilla Platform and functions in the trusted space providing lots of very useful flexibility.

You can currently try out MozMill 1.0rc1 as a Firefox Add-on, and keep your eyes pealed as some exciting new MozMill feature work is around the corner.

We are always trying to make life easier for the test writer, so please log your bugs and feel free to come chat with us in #windmill on FreeNode.

Running Windmill tests from Django

It’s usually a good idea to document cool features you write and tell people about them. I totally forgot about this one, which I wrote during DjangoCon a few months back.

A few years ago I wrote a patch for live server support in Django. The patch fell out of sync with trunk and was picked up a year or so later and substantially improved. But the patch didn’t end up making it in for 1.0 :( . Luckily the code is simple enough that I was able to use it windmill and add live server support to any Django 1.0 install by dynamically overriding and changing the classes that we need to start and stop a live test server for you django project.

After it’s all said and done this makes it dirt simple to bootstrap windmill tests from your Django unittests, all you need to do is define a single test class along side your other django unittests pointing to the directory of windmill tests you want to run and which browser you want to run them in and windmill’s django support will automatically start a live django server and run your windmill tests against it.

from os import path
from windmill.authoring import djangotest
class WindmillTests(djangotest.WindmillDjangoUnitTest):
    test_dir = path.join(path.dirname(path.abspath(__file__)), 'windmilltests')
    browser = 'firefox'

This is also all written up on a wiki page along with some of the caveats.

Windmill 0.9.1 Released

I didn’t think we’d be doing any notable windmill releases until 1.0. Boy was I wrong!

Seriously Faster

On Wednesday Adam messaged me and said that the windmill startup time was too slow. He was right, we’ve know about this for a while but hadn’t put a lot of serious thought in to how we could reduce it.
The issue here was was that we have about 50 JavaScript files that need to get loaded for windmill to start.

Enter windmill-compressor, a new url namespace we added that concats all the js windmill needs in to one file and minifies it. We do this dynamically when windmill starts up, because adding a “build” step would just be too….. Java.

That reduced the startup time from 5-10 seconds to around 2 seconds. But that wasn’t good enough. We saw that most of that startup time was blocking waiting for the compressor to finish, so I threaded it when we start the windmill service and it does the compression while we wait for the browser to start up.

In all, windmill startup times are about 10x faster than 0.9!

Adam also decided to make our page load wait code a bit more aggressive which made not only startup times faster but all of our tests!

Native JavaScript Test Framework

We tied up the loose ends on the JavaScript testing framework and it can now shutdown all of windmill in it’s own teardown, hello continuous integration for native JavaScript tests )

This was the last thing holding us back from promoting this along side the Python test authoring library.

We Love Firebug

In 0.9 we introduced Firebug Lite integration for windmill on all browsers. But when you’re using Firefox you probably still want to use the full Firebug extension, which is easy enough to integrate since we use mozrunner for Firefox launching.

Windmill now has a “firebug” command line argument that installs the full Firebug
extension when launching Firefox.

Zero to Continuous Integration with Windmill

Following ‘automation’ and ‘continuous integration’ in the micro blogging world I have seen a major influx in people being super interested in functionally automating their web apps. I have seen a slew of things about Grid, and Selenium, and people hacking on Watir so I decided to show you from the ground up how incredibly easy it is to get automated test running setup using Windmill and Hudson. I am not going to walk you through every detail, this is much more high level but I do plan to start a ‘continuous integration’ page on in the near future for those kinds of details.

The first step is to get a couple machines that you want use as slaves and a machine to run Hudson, our setup looks like this:

Each of the machines with a different OS has Windmill installed. To make them slaves you simply bring up the Hudson web page on the machine, and run the launcher.. now it’s a slave — crazy easy right?

Now to setup test runs for the machines, in Hudson you click: “New Job” on the left hand side and do something like the following:

Tie this job to the slave you want it to run on (we can’t have IE runs happening on MacOSX):

Tell this job to run 10 and 30 minutes after the hour:

The build steps to actually run the tests, the first kills any straggling processes (more details below):

On the Mac for the Safari job, I want to make sure there aren’t any instances of Safari left hanging, or windmill processes sitting around so we do:
ps -ax | grep windmill | awk '{ print $1 }' | xargs kill | true
ps -ax | grep Safari | awk '{ print $1 }' | xargs kill | true

Then we want to grab the latest test code from svn and launch the windmill test:
svn up /Users/adam/Documents/main_bt/windmill/
python /usr/local/bin/windmill safari test=/Users/adam/Documents/main_bt/windmill/fb password=pass report=true exit
rm /Users/adam/Library/Cookies/Cookies.plist

I am telling windmill to run a test against, with the test hierarchy in the windmill/fb directory in Safari, with the provided email and password, then to report it’s results and exit.

The only thing different on our windows test runs is the way we kill the processes:
taskkill /F /T /IM windmill.exe
taskkill /F /T /IM firefox.exe

You might be asking how do I use those variables, check it out in my setup module:

def setup_module(module):
    client = WindmillTestClient(__name__)
    client.type(text=functest.registry['email'], id=u'email')
    client.type(text=functest.registry['password'], id=u'pass')'doquicklogin')

You can also read a great entry about adding reporting to your tests on Mikeal Rogers blog, here.

And that last line removing Cookies.plist makes sure that the next test run starts without any cookies set to cause problems.

Have Hudson keep you updated on Jabber:

Grab the generated XML output so you can view the test results in Hudson:

Do this for each of the test runs you would like to have, and boom — continuous integration:

This is obviously a simple scenario, and you can do way, way more customization.. but this should get you off the ground. Happy testing!


Firebug Lite in Windmill

Everyone knows how awesome Firebug is. But for those times you have the unfortunate task of debugging in a browser that isn’t Firefox the Firebug team recently put out a new release of Firebug Lite.

For some time now we’ve always had a shell in Windmill for debugging but it was never as nice as the Firebug shell and maintaining that code was kind of a pain. In the recent 0.8.2 release of Windmill we dropped our old shell code and fully integrated Firebug Lite so that whenever you need to debug a test in any browser you can open Firebug Lite in the target application by clicking a single link in the Windmill IDE.

I can’t even begin to explain how much easier this is going to make writing and debugging tests in Windmill.

DjangoCon 2008

I had the pleasure of attending the very first DjangoCon this last weekend.

I’ve attended a lot of conferences; big conferences, small conferences, very targetted conferences and very broad conferences. I think DjangoCon may have been the best conference I’ve ever been to.

Over the years what I’ve learned to truly value about a conference is; meeting with people in the community, learning and understanding things I care about that would be harder to learn from reading than listening to a speaker, and just generally having a good time. DjangoCon was fantastic on all fronts. Being so targeted (people who use Django) there weren’t many talks that I didn’t care about and the people speaking were in all cases the best person to be speaking about a particular topic. The Django community is my favorite open source community, beating out the larger Python community by a small margin. Everyone is nice, helpful, and rarely suffer from even a marginal amount of smugness. Add food, decent wifi, and Cal’s talk “Why I hate Django” and you definitely cover the “having a good time” bit.


I messed with AppEngine for a couple weeks when it was first released. There were some serious rough edges that I decided I couldn’t deal with and I really didn’t completely get my head around the datastore.

Guido on AppEngine

Seeing Guido’s talk I noticed that those rough edges have been smoothed out and the development environment seems nicer, not to mention the documentation seems to have significantly improved. I’ve also been doing a lot of CouchDB work lately so I’ve internalized the “Document Store” concept and once I saw the Expando class provided by AppEngine I felt like I was now in a place where I can really understand Google’s datastore. It’s also not documented very well but Google’s datastore supports some geo data, which has always been a concern of mine putting too many eggs in the CouchDB basket right now. Add the fact that I absolutely hate managing infrastructure and I made a quick decision to invest once again in AppEngine or an upcoming project.

Why I Hate Django

I don’t think anyone will disagree that the highlight of Day One was Cal Henderson talk “Why I Hate Django”, he was absolutely hilarious. The video will be up eventually and I highly suggest everyone watch because it’s just too damn entertaining. I pulled a series of CC pictures from flickr in the meantime that illustrate the humor of the talk.

Cal Hates Django
Cal Hates Django
Cal Hates Django
Cal Hates Django
Cal Hates Django
Cal Hates Django
Cal Hates Django

Live Testing

Something like two years ago I wrote a patch to add live server support to the Django Test Framework (based on unittest). That patch fell out of sync with trunk and on the floor at some point. Eventually an intern at Disqus came along and got it working again and made some really nice improvements.

Although the patch didn’t make it in to Django 1.0 I’m told it should land soon and will be in 3.1, but Disqus has been running with it all summer, using it for some Selenium tests. Well I couldn’t stand idle while Selenium test were running with Django and Windmill tests weren’t so I put my head down for about an hour during one of the talks and committed Django support to Windmill trunk. I ported the current patch for live server support in Django’s Trac so that it dynamically adds what is needed to Django and you can use this with Django 1.0 unmodified, and created a TestCase class that will launch your Windmill tests from within the normal Django python test runner. I wrote up a bit more about this in a post to the windmill-dev list.