You just received the call; company X has selected you as a candidate for the open Product Management position. Now comes the challenging part, nailing the interview. One of the most important questions you will be asked will involve discussing metrics, (confidently). Before you go in, review this method using Dave McClure’s Pirate Metrics, AARRR!.
- The Product Question – “What metrics would you use to measure the success of a new feature?“
- Acquisition: “Users come to the site from various channels.”
- Activation: “Users enjoy first visit: “happy” user experience.”
- Retention: “Users come back, visit site multiple times.”
- Referral: “Users like product enough to refer others.”
- Revenue: “Users conduct some monetization behavior.”
The Product Question – “What metrics would you use to measure the success of a new feature?“
By asking this question, the interviewer wants to know if you have a solid grasp on actionable metrics and have the mindfulness to use data effectively in making decisions and understanding how to tell when you need to say “no.”
Acquisition: “Users come to the site from various channels.”
One of the most important ways of knowing if your product or feature is going well is looking at how many new people are using it and from which channels are they finding and signing up for your product. Avoid talking about using vanity metrics in tracking acquisitions; monthly visitors don’t tell you how many people converted and subscribed, downloaded information or did any other action that showed a higher level of interest.
Activation: “Users enjoy first visit: “happy” user experience.”
Depending on the product or feature, this would include an active download, account signup, or anything that makes your users come back to your site or app. Activation also includes reviewing how long users stay inside the app. Discuss what elements could have a n affect on acquisition, such as onboarding strategy, straightforward UI etc… this will demonstrate you understand the importance of this data and how to apply it.
Retention: “Users come back, visit site multiple times.”
Dave measures retention as a user who returns to your site at least three times within the first 30 days of acquisition. Review these rates from the beginning and continue to look at them in comparison to the week before, and then three months and even six months later. Are people still using the feature? Should you get rid of it or update it?
Referral: “Users like product enough to refer others.”
A referral is a huge indicator of a product’s success because this means people are talking about your product, and getting their friends and family on-board for you. Referrals can be through Facebook, Twitter or an active referral program you implemented, (and A/B tested). When you discuss how to track this data, you will want to include the number of people referrals have brought to your site and how many of those people converted.
Revenue: “Users conduct some monetization behavior.”
Knowing how many paying customers you have, means tracking the number of transactions and where they came from, a blog post, a Tweet or Adwords. What’s important to note here is not only mentioning the most recent data but also checking the information monthly and even annually.
Additional metrics you can add to your response include feedback from social media, forums or looking at the amount of support tickets for your feature so you can decide what you need to fix and how can you create a better user experience.
For more practice with interview questions check out 7 Steps to Ace Quantitative Questions (With a Sample)