The Importance of Early User Testing

Since we subscribe to agile development principles here at Vokal Interactive, one maxim we try to impress upon our clients is to "ship early, and ship often." Sometimes its not always easy to get clients on board with this approach; it's easy to get into a trap of thinking that your app has to have every cool feature you thought of before or since development began, and that every feature has to be "perfect." However, such thinking can lead to roadblocks and delays, pushing an app's release date further and further away while competitors are busy building a user base.

The conventional thinking—which makes sense for traditional manufactured goods like a car, or a computer—goes something like this: develop a concept, refine the concept, brainstorm a feature list, implement all those features, refine, manufacture, package, and put on sale. Using that same process to develop software, however, practically guarantees failure. While software can be faster and sometimes less expensive to develop than physical goods, the time and cost of development increase significantly with complexity. Brainstorm a huge list of features, and you could end up spending a year or more getting your idea to market.

Instead, we typically work with a client to define an MVP, or minimum viable product. An MVP will include the minimum set features in order to get a working implementation of an app out the door and in the hands real users. This sometimes means making hard choices about which features are "nice to have", and which features are truly "must have". Making those hard choices brings focus to a project. "If you decide to ship a piece of software 'early,'" notes software developer Matthew Strickland, "you must identify and focus on core features."

But even if that 1.0 has flaws—and believe us when we say most 1.0 software has flaws—developer Jeff Atwood insists you should ship it anyway.

You could regroup and spend a few extra months fixing up this version before releasing it. You might even feel good about yourself for making the hard call to get the engineering right before unleashing yet another buggy, incomplete chunk of software on the world.

Unfortunately, this is an even bigger mistake than shipping a flawed version. Instead of spending three months fixing up this version in a sterile, isolated lab, you could be spending that same three month period listening to feedback from real live, honest-to-god, dedicated users of your software. Not the software as you imagined it, and the users as you imagined them, but as they exist in the real world. You can turn around and use that directed, real world feedback to not only fix all the sucky parts of version 1.0, but spend your whole development budget more efficiently, predicated on hard usage data from your users.

In other words, you will be wasting time and money grinding on features or details you "believe" are important when you don't actually have any hard data to back that up.

Smart Chicago recently detailed the results of getting an early version of a web app that helps parents find daycare centers and preschools for their children called Chicago Early Learning. The first version was released after a "short development process", but the next several months were spent revising or completely redesigning its features.

"We actively listened to regular Chicago residents, dutifully noted their feedback, and directly changed the site so that it worked better for our target audience of Chicago parents and guardians looking for early childhood education," wrote Smart Chicago Executive Director Daniel X O'Neil.

That included an overhaul of the design language, "softening" its look, as well as a complete re-think of location-based searching, which let's users pick a neighborhood instead of searching by address.

"One thing we heard loud and clear from parents was that they wanted to be able to compare more than two locations," O'Neil noted. "In response, we completely changed the comparison system—changing it to a more recognizable star/favorite system, displaying starred items in a grid, and giving the user the flexibility to easily add and remove locations."

Even if you don't plan to ship early versions of your app to the masses in the Apple App Store or the Google Play Store, however, that doesn't mean you shouldn't get the app in front of users as soon as possible. You can still "ship" a beta version of your app to a manageable pool of representative "real-world" users, and use their feedback to help polish the user experience and guide feature additions as your app winds its way from a 0.1 alpha release to a full-fledged 1.0.