Just-in-Time Usability Testing
If you've ever worked in a state-of-the-art usability lab, with skilled user experience staff, you probably find it hard to consider a product finished until it's been tested for usability.
Unfortunately, most companies don't have the infrastructure, budget or time for this sort of testing. While I believe it's too expensive not to do usability testing, I know that it's still very difficult for a PM to insist when deadlines are tight and there's no in-house usability infrastructure.
Not doing usability testing costs money and time - the later a usability problem is discovered, the more it costs to fix. Find a problem in the spec/prototype stage, and it's trivial. Find it during beta testing, and it's a lot more expensive to fix. Find it after shipping, and the cost can be measured in extra tech support and lost sales, and then it may still have to be fixed in the next release.
Clever user experience designers (you have some of them, right?) have done cognitive walkthroughs and paper prototypes for years. These are good tools for evaluating early-stage designs. But in the past few years it's become practical to do real, live, full scale usability testing without a lab. The equivalent of a $100k lab can run on a laptop, and anyone who knows testing methodology can run it - even a Product Manager.
A usability lab consists of equipment for recording a user's on-screen activities, as well as a synchronized video image of the user - usually a face and hands view. Traditionally, a one-way mirror was used to permit spectators, but live views can now be achieved via webcast.
The key to "miniaturizing" this setup is a product called Morae, from TechSmith (the Camtesia/Snagit people). Morae is a software app that runs on any modern PC. It has a module for running the test, which records a video of the user and everything on their screen as they use the being-tested app or web site. It has an administrator's module to record, annotate and edit the screen and live-video that's captured. And there's an observer module that can run on any remote PC to enable live observation of the test over the web. Morae can be used virtually anywhere, even at a customer's site, with very little setup time.
Of course, useful usability testing requires some skill. You have to pick the right people to test, design tests carefully to test the right tasks, and talk the participant through the test in a way that produces actionable information. Finally, the results must be analyzed in a way that is useful to the PM, the development team, and, often, executives.
A novice tester might not get this right unaided, but can learn by working with a good contract user experience expert. There's a skill in guiding the user, observing carefully, responding appropriated, and being very, very patient. It's not that hard, but it takes practice. And it's worth it.
The goal should be to have someone in house who owns usability testing, and can run tests when needed. They don't need to be dedicated to this, and in a small/midsized company this will be out of the question. But they need to understand the value of usability testing, and they need to be able to roll out the "lab" and run a few tests. The result is a "just-in-time" testing capability. I've successfully set up and run usability tests with as little as a few hours notice.
It can be done by a product manager (not that your plate isn't already full), a QA person, or someone else who's interested. The best non-dedicated usability tester I've worked with was a Tech Pubs manager.
With this capability, you can evaluate the usability of your product with very little lead time, simply by inviting in appropriate testees and having them do a few tasks that will confirm that a design is working…or not. The tests can be observed by developers or executives, or they can be distilled into a video/slideshow summary that illustrates where the application is or isn't working. Seeing is believing. There is no substitute for watching a user succeed or struggle - it instantly brings home a problem that might otherwise become just another bug report.
This may not be in your budget or schedule, but you can't afford not to do it. User interface mistakes cost money and can drive away customers. In long development projects, major development efforts can be based on flawed user interfaces. In an agile development environment it's crucial to evaluate the results of a sprint before moving on to the next one.
If you rely on people who know an application intimately to validate the user experience on behalf of first-time users, you're headed for trouble. There's no longer a reason to leave usability testing to big companies - it can and should be part of every software/web development project.
Unfortunately, most companies don't have the infrastructure, budget or time for this sort of testing. While I believe it's too expensive not to do usability testing, I know that it's still very difficult for a PM to insist when deadlines are tight and there's no in-house usability infrastructure.
Not doing usability testing costs money and time - the later a usability problem is discovered, the more it costs to fix. Find a problem in the spec/prototype stage, and it's trivial. Find it during beta testing, and it's a lot more expensive to fix. Find it after shipping, and the cost can be measured in extra tech support and lost sales, and then it may still have to be fixed in the next release.
Clever user experience designers (you have some of them, right?) have done cognitive walkthroughs and paper prototypes for years. These are good tools for evaluating early-stage designs. But in the past few years it's become practical to do real, live, full scale usability testing without a lab. The equivalent of a $100k lab can run on a laptop, and anyone who knows testing methodology can run it - even a Product Manager.
A usability lab consists of equipment for recording a user's on-screen activities, as well as a synchronized video image of the user - usually a face and hands view. Traditionally, a one-way mirror was used to permit spectators, but live views can now be achieved via webcast.
The key to "miniaturizing" this setup is a product called Morae, from TechSmith (the Camtesia/Snagit people). Morae is a software app that runs on any modern PC. It has a module for running the test, which records a video of the user and everything on their screen as they use the being-tested app or web site. It has an administrator's module to record, annotate and edit the screen and live-video that's captured. And there's an observer module that can run on any remote PC to enable live observation of the test over the web. Morae can be used virtually anywhere, even at a customer's site, with very little setup time.
Of course, useful usability testing requires some skill. You have to pick the right people to test, design tests carefully to test the right tasks, and talk the participant through the test in a way that produces actionable information. Finally, the results must be analyzed in a way that is useful to the PM, the development team, and, often, executives.
A novice tester might not get this right unaided, but can learn by working with a good contract user experience expert. There's a skill in guiding the user, observing carefully, responding appropriated, and being very, very patient. It's not that hard, but it takes practice. And it's worth it.
The goal should be to have someone in house who owns usability testing, and can run tests when needed. They don't need to be dedicated to this, and in a small/midsized company this will be out of the question. But they need to understand the value of usability testing, and they need to be able to roll out the "lab" and run a few tests. The result is a "just-in-time" testing capability. I've successfully set up and run usability tests with as little as a few hours notice.
It can be done by a product manager (not that your plate isn't already full), a QA person, or someone else who's interested. The best non-dedicated usability tester I've worked with was a Tech Pubs manager.
With this capability, you can evaluate the usability of your product with very little lead time, simply by inviting in appropriate testees and having them do a few tasks that will confirm that a design is working…or not. The tests can be observed by developers or executives, or they can be distilled into a video/slideshow summary that illustrates where the application is or isn't working. Seeing is believing. There is no substitute for watching a user succeed or struggle - it instantly brings home a problem that might otherwise become just another bug report.
This may not be in your budget or schedule, but you can't afford not to do it. User interface mistakes cost money and can drive away customers. In long development projects, major development efforts can be based on flawed user interfaces. In an agile development environment it's crucial to evaluate the results of a sprint before moving on to the next one.
If you rely on people who know an application intimately to validate the user experience on behalf of first-time users, you're headed for trouble. There's no longer a reason to leave usability testing to big companies - it can and should be part of every software/web development project.