Quality: What's Good Enough?
Everyone has a different idea of what's "good enough." What's the best criteria?
Getting quality right is important: products (and whole companies) have failed because a product was not good enough, or, because it was good enough and didn't get to market while it was being "perfected." It's the Product Manager's job to identify the appropriate level of quality for a product, and to see that this gets to market promptly.
I once built furniture and cabinetry with a guy named Robert. He was a terrific craftsman, and had very high standards. Our woodworking shop was next to a river, and whenever Robert made a mistake, he would go outside and throw the less-than perfect piece into the river. Robert was good enough that not a lot of stuff went into the river, but he threw away things that other people would have used. And his finished work was gorgeous.
I learned a lot about making things from Robert. I learned that sometimes there is a right way and a wrong way to do things, and little in between. But later, when I was working on my own, I learned another important lesson: the "right" level of quality depends on the intended use and the intended user. Context is important.
For example, a "perfect" saw cut is different if you're making a cabinet or a stud wall. Likewise, a "perfect" user interface is different if you're making a multi-million-user social networking website or a console for an expert system administrator. The different audiences expect different levels of design and "polish", as well as different levels of control and access. Quality is very much in the eye of the beholder, and it's up to the PM to understand what the beholder (the customer) wants and needs.
What makes this so hard is that there are a number of criteria to consider. For example:
- Is the product sufficiently functional that the user can successfully complete the intended task?
- Is the product sufficiently reliable for the intended use?
- Is the product sufficiently discoverable that people can figure out how to use it?
- Is the product sufficiently usable so that the intended user can use it effectively and with satisfaction?
- Is the product sufficiently attractive that users will be drawn to it? (Especially important for web services.)
The answer to these (and other) questions might vary for different target customers. A minor error message might be trivial for one audience, but might keep another from using the product.
It's crucial that product manager identify the expectations of the users at the beginning of a product, and make this clear to the development team. The more that the team understands about the needs and experience level of the users, the more likely they are to produce something with an appropriate level of quality.
There are ways to quantify problems: those responsible for Quality Assurance will likely identify problems by severity, frequency, and perhaps by customer impact. The PM can add an important level of realism to this process by helping to establish a clear vision of who the user is, typically by using personas and use cases, but also by encouraging engineers and QA staff to watch usability tests or even visit customers. QA people love to refer back to real customers, if they've had the chance to meet them, when assessing the impact of a bug. This not only empowers the team, it reduces the need for the PM to be involved in every quality decision.
When it's time to decide when (or if) the product is good enough to go to market, the product manager is in a unique position to contribute to this decision. In addition to assessing the functionality, usability and reliability of the product, then weighing customer expectations, the PM has to consider these things too:
- Will customers be able to accomplish primary tasks well, even though secondary tasks might not work as well, and is this acceptable?
- How much support will the product require?
- What effect will any problems have on the reputation of the product? The company?
Software is never perfect. There are always fix/defer decisions to make. There are times when it's OK to ship or go live with something that has a few glitches, and there are other times when it's better to throw some code in the river and do it again. The better you know the customer and business objectives, and can communicate these to a development team, the easier these decisions will be.
It's probably not a coincidence that Robert and I both went on to found successful software companies. We both loved to create things that people used, things that were functional and beautiful. It's not such a big leap from creating a nice piece of furniture to making an elegant and useful application or web service. It helps to have a strong desire to create things of "quality", but also to keep a healthy perspective on what's appropriate.
Getting quality right is important: products (and whole companies) have failed because a product was not good enough, or, because it was good enough and didn't get to market while it was being "perfected." It's the Product Manager's job to identify the appropriate level of quality for a product, and to see that this gets to market promptly.
I once built furniture and cabinetry with a guy named Robert. He was a terrific craftsman, and had very high standards. Our woodworking shop was next to a river, and whenever Robert made a mistake, he would go outside and throw the less-than perfect piece into the river. Robert was good enough that not a lot of stuff went into the river, but he threw away things that other people would have used. And his finished work was gorgeous.
I learned a lot about making things from Robert. I learned that sometimes there is a right way and a wrong way to do things, and little in between. But later, when I was working on my own, I learned another important lesson: the "right" level of quality depends on the intended use and the intended user. Context is important.
For example, a "perfect" saw cut is different if you're making a cabinet or a stud wall. Likewise, a "perfect" user interface is different if you're making a multi-million-user social networking website or a console for an expert system administrator. The different audiences expect different levels of design and "polish", as well as different levels of control and access. Quality is very much in the eye of the beholder, and it's up to the PM to understand what the beholder (the customer) wants and needs.
What makes this so hard is that there are a number of criteria to consider. For example:
- Is the product sufficiently functional that the user can successfully complete the intended task?
- Is the product sufficiently reliable for the intended use?
- Is the product sufficiently discoverable that people can figure out how to use it?
- Is the product sufficiently usable so that the intended user can use it effectively and with satisfaction?
- Is the product sufficiently attractive that users will be drawn to it? (Especially important for web services.)
The answer to these (and other) questions might vary for different target customers. A minor error message might be trivial for one audience, but might keep another from using the product.
It's crucial that product manager identify the expectations of the users at the beginning of a product, and make this clear to the development team. The more that the team understands about the needs and experience level of the users, the more likely they are to produce something with an appropriate level of quality.
There are ways to quantify problems: those responsible for Quality Assurance will likely identify problems by severity, frequency, and perhaps by customer impact. The PM can add an important level of realism to this process by helping to establish a clear vision of who the user is, typically by using personas and use cases, but also by encouraging engineers and QA staff to watch usability tests or even visit customers. QA people love to refer back to real customers, if they've had the chance to meet them, when assessing the impact of a bug. This not only empowers the team, it reduces the need for the PM to be involved in every quality decision.
When it's time to decide when (or if) the product is good enough to go to market, the product manager is in a unique position to contribute to this decision. In addition to assessing the functionality, usability and reliability of the product, then weighing customer expectations, the PM has to consider these things too:
- Will customers be able to accomplish primary tasks well, even though secondary tasks might not work as well, and is this acceptable?
- How much support will the product require?
- What effect will any problems have on the reputation of the product? The company?
Software is never perfect. There are always fix/defer decisions to make. There are times when it's OK to ship or go live with something that has a few glitches, and there are other times when it's better to throw some code in the river and do it again. The better you know the customer and business objectives, and can communicate these to a development team, the easier these decisions will be.
It's probably not a coincidence that Robert and I both went on to found successful software companies. We both loved to create things that people used, things that were functional and beautiful. It's not such a big leap from creating a nice piece of furniture to making an elegant and useful application or web service. It helps to have a strong desire to create things of "quality", but also to keep a healthy perspective on what's appropriate.