Lately I’ve been experimenting with a nifty little tool that has allowed me to write integration/acceptance tests for both Desktop & web applications. Since I haven’t seen very much about this online, I thought I would share some of my experiences in case it might be useful to anybody else.
For over a year now I’ve been using SikuliX to automate some of my day-to-day desktop interactions in Windows. For those not familiar with Sikuli, here’s a brief description from their front page:
Sikuli automates anything you see on the screen. It uses image recognition to identify and control GUI components. It is useful when there is no easy access to a GUI’s internal or source code.
I can use Sikuli to automate just about anything I can do with a mouse and keyboard. Recently I’ve been experimenting with using this tool for acceptance/integration testing purposes. I work for company that makes education games for schools, a lot of our software has to deal with very dynamic 3D environments with all sorts of crazy physics computations going on in the background that aren’t easily testable. User interaction interaction is easily testable, however. For instance, installing the software should create a desktop shortcut icon that presents a splash screen when opened, hitting a keystroke should make some menu appear, or saving a project should create some file with a specific file extension on the users file system, etc. These are all tests that can easily be automated in Sikuli, and with a little assistance from JRuby and the excellent RSpec testing framework I have all the tools I need to write some really readable, and easy to maintain acceptance tests.
The SikuliX install comes with a very simple IDE that supports interactions such as clicks, doubleClicks, rightClicking, hovering, waiting for elements to appear/dissapear, simplying checking if elements exist, drag and dropping, and more. I found a nifty JRuby gem that exposes this api to any ruby program, including rspec tests.
Here’s an example of a simple interaction I’m testing for in my acceptance test suite to drive a virtual robot in one of our games, these tests are simply ensuring that the robot can move in various directions when supplied with a pre-pared program:
describe "Robot" do
describe "actuators" do
describe "servo motor" do
describe "linear motion" do
before(:each) do
open_project("linear_motion_test/linear_motion_test.rsproj")
click image("ortho-view.png")
end
after(:each) { close_program }
it "can move forward" do
execute_script("move-forward.rxe", "multi-bot.png")
expect { wait(image("fw-result.png"), 5) }.to_not raise_exception,
"Expected the robot to move forward."
end
it "can move in reverse" do
execute_script("move-reverse.rxe", "multi-bot.png")
expect { wait(image("rev-result.png"), 5) }.to_not raise_exception,
"Expected the robot to move in reverse."
end
end
end
end
describe "some other tests..." do
it "can do some fancy UI interaction" do
type("n", Key.CTRL)
hover image("some-menu-on-the-screen.png")
dragDrop image("draggy-thing.png"),
image("place-to-drop-draggy-things.png")
end
end
end
The above tests aren’t meant to demonstrate my testing prowess or anything, just to give you a quick and dirty example of some of the tests I’ve been writing. To explain what’s going on here a bit, I have an open_project helper that’s simply opening a premade project for test purposes via the command line, and an execute_script helper to interact with the UI such that a script gets loaded into the robot for execution. I’m using rspecs expect matchers here to generate a prettier error message than what’s given out of the box. If any one of these sikuli api calls fails, a simple StandardError gets raised with a marginally useful error message (e.g., “such and such image was not located on the screen”.).
How I envision myself using this tool going forward is to write a simple user scenario story as new features are added to my companies software, and possibly some form of ATDD-style tests for bugs and requested features.
If this sounds useful to anybody, I got all of this setup in a matter of only a couple of hours. Simply install the the sikuli-setup.jar here, double-click to run the installer ( I believe I selected both options 1 and 2 to get both the IDE and the API. If the API is all you need simply select option 2). Since Sikuli is written in Java, you will need JRuby in order to interact with it using Ruby. I’ve tested this with Linux and Windows only. Supposedly, it SHOULD work on Linux, but I had some trouble getting some features working on Ubuntu (see my launchpad question for details), YMMV. It works fantastic on Windows, however, which worked out for me because my companies software is all running on Windows anyways. Haven’t tested on mac.