Step-by-step guide to effective product testing

According to research, around 30,000 new products are launched every year, and of those 95% end up failing.

All too often, in a bid to recoup investment or to beat off the competition,  brands rush new products to market with testing whether it resonates with consumers, or indeed whether the product is something that people actually want. But, even the most experienced decision-makers struggle to make the right calls without input from their target audience.

Whether you’re looking for the safest bet or wondering if a risky concept might actually pay off, product testing can help you find a concept that meets your goals and resonates with your audience. With a product concept test, you can gauge customer sentiment and find the proven winners in a pool of different product ideas.

Product testing gives your target audience a range of options and asks them to provide feedback on each one. With product testing, you can compare metrics like purchase intent, quality, and value to find out which product concepts customers like best.

In this article, we’ll take a closer look at why you should test your product concepts, how to conduct an effective product assessment, and what you can do to get accurate feedback from your respondents.

Product testing gives your target audience a range of options and asks them to provide feedback on each one. With product testing, you can compare metrics like purchase intent, quality, and value to find out which product concepts customers like best. Drawing on those insights, you can craft products that will resonate, sell and engender loyal customers. Let’s take a closer look at what product testing entails, and why you need to do it. 

Sometimes described as consumer testing or comparative testing, product testing is the process of measuring the properties or likely performance of products among a target audience. While a great product can recover from a less than stellar launch, an impressive launch can’t save a product that nobody wants to buy. By asking for audience feedback on your product concepts before moving forward with a launch, you can identify successful product ideas right off the bat. This ensures that  you commit money and resources to the right projects.

You can use product testing to reach a number of different goals:

Determine whether a new product development program is on track

If you’re developing a new product, one of the worst things you can do is to wait until the end of the development stage to test whether your product lands with those people you want to market it to. The most successful products are tested regularly throughout the development process in order to demonstrate proof of concept and to make sure that the final product is exactly what the customer wants.

Decide suitability for end use

Product marketing doesn’t end when the customer leaves the store or gets the product to their home. If you want people to come back time and time again, or to shout about the product to prospective new customers, you’ll need to validate the suitability of your product for the end user. As a bonus, if you test your product with different segments of consumers, you’ll not only learn more about how the product works, but how different end categories of customers use it.

Introduce your product to the customer

One of the biggest causes of product failure is the failure to adequately communicate the purpose of the product to the market. Product testing can help ensure that consumers understand what your products will do for them, and which products offer them the best value.

Solve problems with current products

Too often, new products are recalled or have to be developed because of problems only recognized once they’re sold. Product testing can help to avoid costly errors or oversights by having the end user try out the product and send that all important feedback.

Increase consumer protection

If you’re distributing a new product or service, it's in your best interest to check the claims made during marketing campaigns. So, before you advertise your product as the cheapest, fastest or best in class, test it with your target audience. 

Identify potential cost savings

The cost of developing or redeveloping products can be tremendous. Oftentimes, features are added that customers don’t really value, or tweaks are made that make little difference to the bottom line. By testing your product before it reaches the market, you can make sure that every penny of your research and development adds value.

When you use product testing, you take on a product launch with 4 major advantages:

Now that you know why it’s important to test your product concepts before settling on one, you’re ready to put together your test survey. For accurate and actionable results, follow these 4 steps:

While you can test any product concept you want, it’s best to choose stimuli that you’re already somewhat confident in based on an informal survey of coworkers and friends. To make sure your survey results are useful, choose products in a similar stage of development. If you’re testing a finished product against a product you’re still working on, the winner is obvious.

To make the testing process more manageable for you and your respondents, limit the number of stimuli in your survey. The maximum number of stimuli you should include in your test depends on whether you plan to use a monadic survey design or a sequential monadic survey design.

A monadic survey design divides up your respondent pool and presents each respondent with a survey asking for feedback on a single stimulus. When you’ve gathered enough feedback on each stimulus, you can pick a winning concept by aggregating the feedback from all your respondents.

So, if you have a sample size of 500 respondents, you ask 250 of them about one product option and 250 of them about the other.

Here’s an example of a product testing survey with a monadic design.

This survey design lets you ask more questions about each stimulus, which lets you figure out more about how  individual features resonate, and which shortcomings might prove difficult in the future. It’s also more likely to result in a relatively short questionnaire, which benefits your survey’s quality of data and completion rate. However, since you’re only showing each respondent one stimulus, you’ll need to target a larger audience. This can prove expensive and it might not be as easy to execute.

A sequential monadic survey design presents every respondent with the same survey that asks for feedback on multiple stimuli. Once you’ve collected responses from a statistically significant number of respondents, you can pick a winning concept by looking at the most common reactions to the stimuli.

In this type of survey, if you have a sample size of 500, you’d ask all 500 about both product ideas. 

Here’s what your product testing survey might look like if you use a sequential monadic design.

This type of design lets you target a smaller audience than a monadic design, which makes it more cost-effective. However, if you want to keep your survey to a manageable length, you won’t be able to ask as many questions about each stimulus—meaning fewer insights about the specifics of each product.

Learn more about the pros and cons of each design, and how you can set them up in SurveyMonkey.

What makes a good product? The metrics you choose to measure will help you decide.

The questions you ask in your product testing survey can be as specific as you’d like. If you want to know whether your audience finds a certain feature useful, for example, you might ask about the innovativeness, relevance, and value of that feature. In general, these are some key metrics you might want to include in your survey:

The relative value of each metric you measure depends on your goals for the product. If you want a product that will sell well, purchase intent may be the most important metric for you. If your main goal is to win loyal customers, you’ll likely place more importance on quality.

We suggest using the Likert scale to test your metrics. This scale is simple and intuitive for respondents, offers consistent choices, and makes it easy for you to analyze results. If you choose this scale, your questions can follow the formula, “How (metric) is the product?” where your answer choices range from “Extremely (metric)” to “Not at all (metric).”

For example, here’s how the question might look if your metric is innovativeness:

How innovative is this product?

Sometimes, it isn’t clear from a respondent’s answers which concept they like best. Resolve this issue by including a forced-answer question that asks respondents to choose their favorite product from all the options at the end of the survey. The results of this question will also help you rank the importance of each metric to the overall likeability of your product. For example, if respondents consistently choose the product with the highest value as their favorite, you can assume value is a key value driver for your target market.

You might also want to include screener, category, and demographic questions in your survey. These questions tell you more about the people you’re collecting feedback from and let you segment your results by group. 

Your target audience can help you decide which product concepts are worth pursuing and which you should set aside. There are 2 ways to get feedback from this group of people:

Once you’ve collected your survey responses, you’re ready to compare your product concepts. You can use Top 2 Box scores to make your data clearer and your findings more pronounced. It groups together positive and negative responses to each question into a single percentage, which is easier to compare.

You also need to look at open-ended responses. Our word cloud feature can make them easier to grock.

Product testing belongs to the development and growth stages of the product life cycle, when you’re testing new product ideas and adding new inventory.

During the development stage, you’ll probably be looking for broad feedback that’ll help you make major decisions about your products. This is the time to ask respondents to choose between several very different product concepts, for example. Because everything about your brand is new at this point, it’s the best time to field broad, directional feedback on your products.

During the growth stage, you may be looking for more nuanced feedback on your product concepts. This is the time to ask respondents focused questions about new product features or niche alternatives to current products, for example. It’s also the time to think about sampling your loyal customers to see if they’re on board with your new ideas.

By following a set of best practices for product testing, you can get better feedback from your respondents. Here are 5 things to keep in mind:

Since you understand why testing your products is important, know how to run your own test, and have some useful testing tricks up your sleeve. 
Now it’s time to jump into your own product testing survey. If you want results fast, we can get you some: in less than an hour.