Best Practices for Assumption Testing

Running quick landing page tests or A/B tests without a clear hypothesis may seem like experimentation, but it doesn’t provide true validation of your assumptions. For example, imagine you're launching a fitness app and create a landing page to see if people are interested. You get some sign-ups, but without a clear goal, it’s hard to know what those sign-ups mean. Did users sign up because of your app’s core features, the design, or just curiosity?

Instead, a better approach is to identify a key assumption that needs testing, like "Users will sign up for the app because they value personalized workout plans." Now, you can design an experiment focused on this assumption — perhaps by offering two versions of a landing page, one highlighting personalized plans and another promoting general workouts. With clear data and goals, you’ll get more meaningful insights, helping you validate your assumption and guiding your next steps.^[1]

Start by generating a list of assumptions that align with your product or business goals. Once you have your list, prioritize these assumptions by thinking about how much they will help you achieve your objectives. Consider the potential impact of each assumption — how much could it drive progress if validated? For example, if you believe a new feature will increase user engagement, that assumption may have a high impact.

Next, think about how feasible it is to test each assumption. Can it be tested quickly and easily, or will it take more time and resources? Assumptions that are both high-impact and easy to test should be prioritized first, as they can bring the quickest results with minimal effort.

In this approach, you prioritize assumptions based on how they support your goals, rather than focusing on certainty and risk like in assumption mapping.

Preparing to be wrong means accepting that some of your ideas or assumptions may not work out as planned, and that’s okay! In product development, it’s important to stay flexible and open to feedback. Testing your assumptions is key because it helps you discover what works and what doesn’t, without wasting too much time or resources.

Start by viewing every test or experiment as a learning opportunity. If something doesn’t go as expected, ask yourself, “What can I learn from this?” Instead of being discouraged, use the results to make improvements.

Being ready to be wrong also means having backup plans in case an assumption fails. This mindset helps you stay adaptable, learn faster, and ultimately build better products that truly meet your users' needs.

Every idea you have is built on underlying assumptions. For example, let’s say you have an idea for a food delivery app that sends healthy meals to users. Testing the idea might involve asking people if they like the concept, and most will probably say yes. However, this doesn’t tell you whether people will actually use it.

Instead, test the assumptions behind the idea. For instance:

• Assumption 1: Users will pay extra for healthier meals.
• Assumption 2: Users prefer curated healthy meal options over choosing individual ingredients.

By testing these assumptions, such as through surveys, prototypes, or A/B tests, you can validate whether the core beliefs behind your idea are correct. This helps you avoid building a product that sounds good in theory but doesn’t meet real user needs or behaviors.

Different assumptions require different types of experiments. For example, if you need to understand why users are not engaging with a feature, conducting user interviews or usability tests might be the best approach. For evaluating different design layouts, A/B testing would be more appropriate. To choose the right method, start by asking what you need to learn from the assumption. Do you need user opinions, behavior insights, or performance data? Pick a method that gives you the clearest, most direct answer.

Also, understand the strengths and limitations of various testing methods. Use qualitative methods like interviews and observations for deeper insights into user behavior, and quantitative methods like surveys and A/B tests for broader, statistically significant data. Having a diverse set of tools in your experimentation toolkit ensures you can approach each assumption with the most effective method.

Defining what success looks like before you start any test is important because it helps you know when an assumption has been validated. Success should be clear and measurable. For example, if you're testing a pricing assumption, success might mean that 50% of users are willing to pay for a new premium subscription plan.

To define success, ask yourself: what outcome would prove this assumption right? Set clear goals, like how many users need to subscribe or how much they are willing to pay. Make sure to define success before you run the test. This way, you know exactly what to look for in the results and can make better decisions based on data.

When you run a test, it's important to stick to the success criteria or thresholds you set ahead of time. It can be tempting to bend the rules if the results aren’t exactly what you hoped for. For example, if you decided that success means 70% of users complete a task, but only 60% did, you might feel tempted to say, “That’s close enough.” But doing this can lead to poor decisions.

Instead, stick to the thresholds you set. If the results fall short, take action based on the data. This might mean tweaking the feature, running another test, or rethinking the idea. Your brain naturally wants to see what it hopes for, so working with a partner or team can help you stay objective. By holding yourself accountable to the original criteria, you'll make better, more evidence-based decisions.

After you get the results from an assumption test, the next step is to decide what actions to take. If the results support your assumption, you can move forward with confidence, knowing you're on the right track. For example, if users respond well to a new onboarding process and complete it quickly, you can continue developing that process.

If the results don’t support your assumption, adjustments are needed. You might need to modify the idea, test a different approach, or rethink the strategy. For instance, if users drop out halfway through the onboarding process, you might need to simplify the steps or make the instructions clearer.

Decide on your next steps based on whether the data confirms, challenges, or contradicts your assumptions. This way, you're making informed decisions and constantly improving your product.