Metrics Sense: Troubleshooting a Metric

Question

You are the TPM for a mobile application. You notice a sudden drop in the number of in-app purchases. What could be the problem?

Background

Often, metrics will have seemingly unexplainable spikes and dips. As a metrics-focused TPM, being able to investigate and explain these occurrences is a core analytical skill. This question is quite open-ended, so the important thing to do here is to apply a structured and rigorous approach.

Solution approach

A common pitfall is to simply list off a set of possible reasons without any structure. While this sounds like the type of question where you need to brainstorm many ideas (which is absolutely true), it is just as important to make sure your ideas are well-organized and easy to follow. We’ll use the following structured approach:

• Propose a potential breakdown of the data flow for the metric.
• Dive deep into each step in the data flow.
• Enumerate potential reasons for the root cause of the metrics issue.

Sample answer

This is a high-level diagram of how the in-app purchases’ metrics data could flow from the client mobile app to being visualized in an internal dashboard.