3 Ways to Validate Your Product Idea by Testing a Hypothesis

Having a product idea is great… But how do we know if the problem we are trying to solve actually exists? Learn the ways you can do this! 

A/B testing is the most common approach used to validate your hypothesis. While this is a useful and powerful approach, there are some limitations and it doesn’t always result in the highest ROI. Personalization is an alternative approach to A/B testing which can maximize value for specific audiences but can be costly to implement at scale. We’ll guide you through the strengths and weaknesses of each approach that are critical to creating a winning optimization strategy.

Online Mortgage Lead Gen Flow Overview

    • Bigger focus on a single transaction (‘lead’) 
    • Sensitive to market conditions (Ex. Increase in interest rates)
    • Impacted by government regulations (Ex. HARP extension)
    • Marketing-driven (Ex. changes in marketing costs)
    • Highly competitive and dynamicLead buyers exert great influence on the quality and price for leads they accept

alphabet-abc-test-testing

3 Tactics to Test Your Product Idea

1. A/B testing

A way to compare two versions of something to see which one performs better.

  • Benefits:
    • Allows for distinction between causation and correlation
    • Streamlined and easy to use tools available (ex. Optimizely)
    • Can yield significant improvements to KPI goals
  • Cons:
    • Does not maximize value across audiences
    • Not easily adaptable to changes in user behavior or market conditions
    • Can be time-consuming to get statistically significant results and business value
  • Use Cases:

Appropriate for testing fundamental changes in the online experience.

      • Test fundamentally different designs or user experiences (Ex. LMB)
      • Test difference in a number of steps and grouping of questions for online forms (Ex. LMB)
      • Longer forms and moving deeper in the funnel (Ex. LT)

personalization-product-test-testing

2. Personalization

Customizing a service or a product to meet the specific needs of a segment of consumers.

  • Benefits:
    • Maximizes the business value from specific consumers’ groups
    • Presents the best customer experience to specific consumers’ groups
  • Cons:
    • Difficult to discover differences in audiences
    • Costly to implement personalized experiences at scale

light-bulb-test-testing

3. Predictive Personalization

Use predictive technologies to determine the variations of the product or service which performs best for specific consumer groups.

  • Benefits:
    • Maximizes the business value across all consumer groups
    • Presents the best customer experience across all consumer groups
  • Cons: 
    • Fewer commercial tools are available
    • Costly or complex to implement technology in-house

Personalization Examples

The ideal approach for sustainable ongoing optimizations, capturing incremental value and maximizing value.

  • Examples:
    • Engaging headlines on landing pages
    • Convenient ‘common’ answers
    • Consistent CTA placements
    • Setting expectations and progress indicators
    • Effects of branding and site logos

Summary

    • Use A/B testing to find the best base site experience
    • Use personalization to optimize for different audiences
    • Leverage predictive personalization if available to:
      • Maximize business value and consumer satisfaction
      • Adapt to changes in market and consumer behavior
      • Streamline ongoing optimizations

This article is based on Ali Daftarian’s talk “Optimization Tactics”

Have you ever applied any of these approaches? Which one worked for you? Share your comments with us!

Enjoyed the article? You may like this too: