Updated: January 24, 2024 - 11 min read
If you've ever wondered how the best fintech companies develop groundbreaking products that are perfectly in tune with user needs, the secret lies in their dedication to experimentation. Beyond just a trendy buzzword, product discovery experiments in fintech ensure firms venture in directions that promise genuine returns. Dive into our exploration below as we uncover the importance, advantages, and best practices for conducting effective product discovery experiments in the fintech sector.
Editorial note: This post is based on a talk on How to Run Product Discovery Experiments in FinTech by former Lead Product Owner at Lloyds Banking Group, and contains additional insights and examples from the Product School team. You can watch the webinar in full below.
The importance of experimentation in fintech
In the fast-evolving FinTech sector, experimentation is vital. Rather than banking on an idea solely because it's innovative, product discovery experiments offer a structured way to determine if the concept holds genuine promise in the complex financial market. They also bring organizations the following advantages:
1. Risk mitigation: By testing on a small scale, fintech firms can identify potential pitfalls or challenges and address them before a full-scale rollout, minimizing losses. For example, introducing a new loan product without understanding its potential impact can be risky. To mitigate this risk, a fintech company can run a pilot program, offering the loan to a select group of users to gauge its success and identify potential pitfalls before a full-scale launch.
2. Enhanced user experience: By experimenting with different user interfaces or features, companies can identify what their users prefer, ensuring a more seamless and user-friendly experience.
3. More data-driven decisions: For example, imagine a fintech company that’s contemplating adding cryptocurrency as an investment option on their platform. Rather than making this decision based on industry buzz, the company conducts experiments, studying user behavior and preferences, and analyzing market data. This way, they can make a more informed decision and increase the chances of developing a feature that serves genuine user needs.
Pssst! Are you struggling to translate data into decisions? Take our free Product Analytics Micro-Certification and become the data-savvy Product Manager every team needs.
4. Regulatory compliance: The fintech sector is heavily regulated. Experimentation allows companies to ensure that new products or features are compliant with local and international regulations.
5. Increased efficiency: Imagine you’re a Product Manager for a bank and you’ve noticed a decline in usage of their banking app. Instead of hastily revamping the entire app, you can experiment to identify the root problem. By doing this, you might discover that it's not the entire app but just one feature, like a complicated money transfer process, that's causing user dissatisfaction. What’s more, it's cheaper to test and fail on a small scale than to implement a full-scale project that doesn't meet expectations.
6. Faster time-to-market: With the insights gained from experiments, FinTech companies can accelerate product development cycles and get to market faster than competitors.
7. Innovation: Continuous experimentation fosters a culture of innovation. It encourages teams to think outside the box and come up with groundbreaking solutions.
Understanding the experimentation framework
As you can see, experimentation has several compelling benefits, but how do you go about conducting product discovery experiments in fintech? Here are some pointers to get you started.
Focus on problem identification
Every groundbreaking fintech innovation starts with identifying a pressing problem. For instance, many fintech companies saw that traditional banking services were often not accessible or user-friendly, leading to innovations like mobile banking apps or peer-to-peer lending platforms. Entrepreneurs and Product Managers in this sector must deeply immerse themselves in these problems, continually seeking the most effective solutions.
Solve your customers’ pain points
To truly serve your customers, you should deeply understand them. For example, when developing a digital wallet, understanding who uses it, why they prefer it over traditional payment methods, and what challenges they face during transactions is crucial.
Empathy maps and free tools like the Value Proposition Canvas allow you to gain insights into your customer's world, ensuring you can offer them a solution based on genuine user needs.
Prioritize
The fast-paced nature of technology and finance demands an entrepreneurial approach. As a Product Manager in a fintech company, you might see numerous challenges, from regulatory issues to customer pain points.
Just like with features, when you have multiple problems and hypotheses, it's essential to prioritize them. To do this you can ask yourself the following questions: Does the customer want or need this? Does the organization have the skills or technology to do this? Is it financially viable? The cost of development shouldn't outweigh potential returns.
By focusing primarily on the most significant problems, you can prioritize solutions that have the most substantial impact. For instance, upon realizing that cross-border transactions were costly and slow, fintech startups like TransferWise (now Wise) emerged to tackle this very issue.
Make experimentation a continuous process
Experimentation is an ongoing process, not just a phase in product development. While more experiments might be conducted during the product discovery phase, they should continue throughout the product lifecycle. This ensures that products are relevant, effective, and user-centric.
Make data-driven decisions
In fintech, data is king. Whether it's a startup trying to understand user behavior on their new budgeting app, or a large-scale digital bank aiming to improve its customer service, there are several tools and methodologies you can use to leverage insights from data. These include the following:
Direct interactions with customers can offer first-hand insights into their needs and preferences.
Engaging with stakeholders can shed light on larger industry trends. For instance, if you’re a Product Manager developing a product for a B2B audience, discussing it with financial experts can provide invaluable feedback.
Analyzing customer complaints can identify gaps in the current offerings. If a significant number of users complain about hidden fees in a digital payment service, there's a clear problem that needs addressing.
Existing data, such as transaction histories, app usage stats, or customer feedback, can be goldmines of information. If a digital lending platform notices that loan applications spike at the end of the month, they can tailor their marketing efforts accordingly.
Design the team structure for experimentation
Before jumping into building a product or solution based on the identified problem and hypothesis, it's vital to have a structured team in place. This team should be cross-functional in nature, comprising different specialties to ensure a holistic approach. This usually includes:
Product professionals who understand the product's nuances and its market fit.
Engineers who can build and execute the technical aspects of the product.
User Experience (UX) experts who ensure the product is user-friendly and meets user needs.
Data Science or Analytic Teams who can analyze metrics and drive data-driven decisions
Ideally, this cross-functional team will be customer-centric, analytic, and focused solely on experimentation.
Apart from the core team that handles the primary responsibilities of experimentation, it's very useful to include other interested parties from the organization. This can be leadership, subject matter experts, and others who might have a vested interest in the product or the experimentation process.
By involving these parties, they get firsthand experience in the experimentation process, learn from it, and can further advocate for such practices in the organization. This is truly pivotal to driving a more inclusive, informed, and product-led culture of experimentation within the broader organization.
Choose the right type of experiment
There are two primary categories of experiment you can run – Low Fidelity and High Fidelity. Which type you run will be largely influenced by the nature of your hypothesis.
Low-fidelity testing uses the initial, basic representations of the product which resemble the final product but don't function like them. Examples include wireframes or prototypes. While low-fidelity tests are good for initial feedback and understanding user preferences, interaction with users is usually limited, and the feedback tends to be more qualitative.
High-fidelity experiments on the other hand offer detailed insights into user behavior and system performance as they are closer to the final product and are functional. High-fidelity testing allows for detailed user interaction and captures both qualitative and quantitative data thereby collecting more comprehensive feedback.
It’s also important to identify whether you will need front-end or back-end testing to test your hypothesis.
If your hypothesis revolves around user behavior on a website or an application interface, you might opt for A/B testing or multivariate testing. These tests don't usually require changes in the underlying system (backend), so front-end testing would be your best bet.
In contrast, for hypotheses that involve core functionalities or data handling, backend experiments are necessary. This could involve creating new databases or altering the system's architecture.
Segment your users
Before running any tests or experiments, you need to determine the target audience or users who will test your product. By correctly segmenting your users, you ensure that products and services are tailored to meet their distinct needs and preferences, enhancing user satisfaction and adoption rates. It also allows businesses to deliver more personalized experiences, increasing the effectiveness of their offerings.
What’s more, segmentation allows for more accurate feedback during testing phases, ensuring that potential issues or improvements are identified early on. This targeted approach also optimizes marketing efforts, ensuring that resources are directed towards the most receptive audiences, ultimately leading to better return on investment and more efficient product development cycles.
There are two Main Categories of User Groups:
1. Model Office
Relevant mainly in a B2B (Business-to-Business) setting.
If you're developing a product or tool for internal use within an organization, the "model office" refers to a select group from the intended users, like the marketing team or customer support. They test and provide feedback as they represent the end-users within that business setting.
2. General User Group
Pertinent in a B2C (Business-to-Consumer) context.
If your product is for a vast consumer base (like a new app for the general public), the "user group" refers to a segment of your intended broader audience. You'd want feedback from this segment to gauge how well your product might be received by millions of potential users.
The dos and don’ts of running product discovery experiments in fintech
As you’ve seen, in the fast-paced and highly competitive world of fintech, running product discovery experiments effectively is crucial. As you embark on this exploratory journey, it's essential to have a clear roadmap to maximize insights and ensure meaningful outcomes.
To help you run successful experiments, we’ll close this post by offering you a guide to the dos and don’ts of conducting product discovery experiments in the fintech sector. By steering clear of common pitfalls and embracing best practices, you can pave the way for success by gaining more insightful and impactful results.
Do collect data: When running experiments, it's vital to have mechanisms to gather data. The primary goal of the experiment is to gain insights from user interactions. Collaborative analysis involving data science or analytics teams can ensure more accurate interpretations.
Do keep your team on standby: Even after initiating the experiment as there might be a need for quick fixes or modifications based on real-time feedback. Immediate action can save an experiment from failing due to small, correctable issues.
Do collaborate: Before finalizing any findings, sharing a draft version with a select group can help validate your interpretations. Different people might see data differently, so getting a consensus is essential.
Do prepare for worst-case scenarios: Having an exit strategy is essential in case an experiment needs to be terminated early.
Don’t over-extend experiments: Keep your experiments concise. Running lengthy experiments can consume a lot of resources, from time to funding. The objective is to learn quickly and iterate, not to linger.
Don’t misjudge the scale: Don't design your experiments for large scale right from the start. Begin with the minimum viable product (MVP) or minimum testable product. This allows for faster iterations and reduces the risk of major setbacks.
Don’t stop at one experiment: One experiment is rarely conclusive. If you're basing significant decisions on a single experiment, you might be heading down the wrong path. Multiple experiments ensure you're working with robust and diverse data.
Don’t see failure if you disprove your hypothesis: Disproving a hypothesis is not a sign of failure but a learning opportunity. It provides clarity on what doesn't work, allowing you to pivot towards more promising directions.
Don’t choose the wrong metrics: Metrics are the compass of your experiments. Choosing the wrong metrics from the outset can send you on a wild goose chase. It's crucial to identify and monitor metrics that directly relate to your hypothesis and goals.
Don’t choose too many metrics: While secondary metrics can provide additional insights, avoid fragmenting your primary metric excessively. Too much data can clutter the analysis, making it challenging to derive meaningful conclusions.
Don’t miss out on analysis support: If you're diving deep into secondary metrics and amassing a large volume of data, ensure you have the analytical expertise on hand. A data expert can help you navigate through the maze of numbers and provide actionable insights.
Don't just hand over the experiment and disappear: Maintain communication with your users throughout the testing phase. This ensures timely feedback and helps in spotting potential issues.
Updated: January 24, 2024
