Making scientific experimentation the fabric of your Product Management decision making: The Building Blocks.

Experiments are essential to test Hypothesis.

A Hypothesis is an untested assumption.

It is human nature to state assumptions as facts and be biased towards making them work- the biggest threat to a good product-market fit.

An example of a Hypothesis that can be construed as a fact could be “Launching Apple Pay in my app would lead to customer happiness and acquisition.”

H1 is the outcome of the hypothesis that you want to prove. In short, H1 is what you’re betting on.

H0 (H-naught) is the unfavorable outcome f your hypothesis and you’d want to disprove this.

There are primarily three types oh hypothesis that product managers test frequently:
1. Value Hypothesis- Will your customers find a feature valuable?
2. Scalability Hypothesis- Is the feature going to find takers when I cast a wider net or is the feature solving the problems of only a handful of my customers?
3. Usability Hypothesis: Is the user easily able to complete the JTBD and reach the end of their journey easily without dropping off or getting frustrated?

Experiments test for specific outcomes and are formulated with specific thresholds.

“Will users have a drop off rate below 5% as a result of this new call to action” is an experiment that you can test and prove your hypothesis.

Hypothesis for this experiment would be something like “ If I make the Call to Action Sticky, the drop off rates on my landing page would reduce”

MVPs are vehicles to test for hypothesis and to create measurable experiments. A Minimum Viable let’s you experiment before you invest a boat load of money into building a feature. By definition, an MVP is a cost effective way of testing stuff out. MVPs and experiments help us in removing our biases through a scientific method.

Minimum Viables have different connotations for different people; for organizations that are not used to a culture of experimentation; an MVP usually means the smallest slice of the product that can be released to customers. This is a Minimum Marketable Product.

An MVP, at it’s core, is either a thin slice of the functionality or a simple experiment that tests hypothesis before investing Dollars into development.

“It shouldn’t take more than a week to formulate and test the hypothesis it is meant to test.”

Below, I have highlighted 4 of the many MVP types that test measurable outcomes. This by no means is an exhaustive list, but it gives you an idea of what forms the construct of an MVP:

1. Concierge MVP- Focuses on the JTBD and not the usability. Unearths whether there are any problems worth solving when a customer performs a journey. Eg: Taking food orders via a whatsapp group manually instead of an app.
2. Wizard of Oz- tests for usability. Involved users playing with a prototype/front end/fake product without a backend to observe how they go about doing the job and whether they feel particularly frustrated or delighted at any step.

3. Landing Page and

4. Email MVP- both 3 and 4 testing for smoke testing

To summarize; Hypothesis, Experiments and MVPs are the building blocks needed to cultivate a culture of experimentation. These help to:

1. Shorten the feedback loop
2. Vehicles of testing demand, usability and scalability hypothesis
3. Let you know what NOT to build
4. Are great ways of attracting investment

Credits: The Lean Startup, Design thinking by Alex Cowan