How We Find the Right People for Product Validation

Finding participants for product validation isn’t easy. Finding the right participants is even more difficult.

There are far too many variables in any given market and for any given product for me to recommend a step-by-step process for recruiting participants for a study. However, defining the parameters of the project and using the right channels to approach potential participants will help you find the right people. With feedback from the right people, you can validate your feature/product with increased accuracy. Here’s the steps we follow at UserVoice to find the right people for product validation.

1. We Determine the Scope of the Project

Defining the scope of the project helps us find a jumping-off point for gathering data. As a product manager, you might be inclined to only talk to people that fit the buyer persona, but it’s often not as simple as that. In most cases, it is okay to look beyond buyer persona. You have to consider factors like your deadline, whether it’s a new feature or product, and your goal. For example, your project scope will be different if you’re expanding into a new market as opposed to helping your current market with a problem.

At UserVoice, we look at three factors when determining the scope of our project:

• Deadline — We always start here because our timeframe will dictate how many people we can interview/survey.
• New feature vs. new product — Launching an entirely new product requires a much bigger sample size than adding a feature because you’ll need to find product-market fit.
• New market vs. current market — Trying to expand into a new market through a new feature means having to formulate a completely different set of questions. That’s because you’ll have to talk to people who don’t use your product.

Once we understand what we’re launching and why, we’re in a position to set the parameters for our data collection.

2. We Choose the Sample Size and Method of Data Collection

Sample size and method of data collection are often interdependent variables for us because we have to make sure we have enough time to collect data. A smaller sample size means we can do interviews, while a sample size of a thousand people or more limits us to surveys. When introducing a new feature to a current product, we usually go with a smaller sample size. That’s because, with a current product, we have an established userbase, and the new feature is usually a result of the userbase’s feedback. Larger sample sizes are usually good for new products because we want to be extra thorough in the first stage of product validation.

When validating a new feature, we prefer to start with a smaller sample size and conduct interviews through video calls. Video calls allow us to collect in-depth feedback, have conversations with customers, and develop a better understanding of their problems.

When doing interviews, we usually start with 5-10 participants. We then analyze the data we’ve gathered, schedule more interviews if we need more data, and keep interviewing more people until we can identify trends and patterns in their answers. The smallest sample size I’ve ever worked with was 10 people. In a recent project, I interviewed 30 people. It all depends on the quality of the answers you get.You should go for bigger sample sizes if:

• There are no clearly identifiable patterns in the data collected
• You don’t have clear answers to your questions after your first round of interviews

Let’s say you want to introduce a feature that allows users to post updates to social media sites directly from your app. You interview 5-10 people to see what they think. It looks like a couple were for it, a couple were against it, and the rest were indifferent. That’s not really enough data to confidently validate or invalidate your idea, so you know you need more data.

With surveys, the rule of thumb is a sample size of 400 for a potential userbase of 100,000, but if you do six-week development cycles and roll out new features every quarter, you won’t have the luxury to collect data from hundreds or thousands of people every time. So we save surveys for much larger projects.

‍