Automated Testing in iOS

When we send an app out into the world we want to be sure it’s bulletproof. I’m sure you write excellent code and you unit-test like a mad man, but how can you be sure? Wouldn't it be great to plug in a device and have Instruments run an exhaustive UI test while checking for leaks and logging total memory and cpu usage? Yes it would.

“I've got a fever and the only cure is more testing.”

For the last week iOS UI Automation has consumed my days. UI Automation uses JavaScript to navigate through your iOS app. Take a look at Apple’s documentation (scroll to the UI Automation section), if you haven’t already. Elements objects return arrays of everything on your screen. You interact with your UI by referencing an Accessibility label or array index. The best and easiest way to script interactions is to leverage Accessibility labels in your project (BONUS: this will also make your app more accessible). Be sure to use a short label, like "Favorites" or "Add". VoiceOver will read these as "Favorites list" or "Add button".

Here are two short simple examples:

This line will tap a cell labeled "Favorites" in the first tableview in your mainWindow:

UIATarget.localTarget().frontMostApp().mainWindow().tableview[0].cells["Favorites"].tap();

The following will randomly tap 1000 times all over your app. Think of it like buckshot testing.

var target = UIATarget.localTarget();
//random between x320 and y460
for (i=0;i<=1000;i++) {
 xPoint = Math.floor(Math.random()*319+1)
 yPoint = Math.floor(Math.random()*479+1)
 target.tap( { x:xPoint, y:yPoint } );
}

Alex Vollmer has a great introduction to UI Automation on his site. He explains how to set Accessibility labels in Xcode and he covers all the basics of scripting UI Automation. He also created some excellent JavaScript to make scripting much easier. Kind users on StackOverflow collected more introductory resources here: Best Resources for UI Automation Testing for iOS Applications.

This is very exciting, but it gets even better: iOS 5 added the ability to record interaction straight to JavaScript. Armed with the record button you can write more accurate scripts faster. You’ll have a flexible script ready in no time. The screenshot below shows the output. Every element is returned with every possible way to reference it. A forward-thinking scripter could use predicates to ensure the script works even as your interface is reorganized in the future.

Record lacks just two important features: delay timing and text entry. Since scripts often run faster than the iPhone Simulator’s UI you’ll need to set a long timeout or insert a few target.delay() to ensure your UI can catch up. In practice your test writing workflow will probably go like:

  1. Plan the UI flow for the test
  2. Record the activity
  3. Add timing/delays, text entry, and assertions
  4. Run the test
  5. Repeat

The future of UI Automation testing is bright. My next step will be integrating UI testing in to our build server’s existing tests so every new build will be automatically tested for functionality, memory leaks, and crashes.

In parting, I’ll leave you with a short example using tuneup.js

#import "/path/to/tuneup/tuneup.js"
test("Log In", function(target, app) {
    app.mainWindow().activityIndicators()["In progress"].waitForInvalid();
    target.delay(1);
    app.navigationBar().elements()["Settings"].tapAndWaitForInvalid();  
    app.navigationBar().elements()["Login"].tapAndWaitForInvalid();
    // expect failure - User Does Not Enter Username or Password  
    app.mainWindow().scrollViews()[0].secureTextFields()["Password"].setValue("notcorrect");
    target.delay(1);
    app.keyboard().buttons()["Done"].tap();
    target.delay(1);
    UIATarget.onAlert = function onAlert(alert) {
        UIALogger.logMessage("expect alert occured");
        alert.buttons()["OK"].tap();
        return true;
    }
}