Analyze the Data
Try out different metrics to see if the feature change request met its business goals.
We'll cover the following
Now that we’ve defined the context of the problem and how the feature might work, we have to dig into the data to understand what pieces of data and information we can brainstorm that could prove our hypothesis that this feature is needed and necessary.
The framework would be to go through each feature goal and analyze:
- What data and metrics could we look at to help us determine if the goals align with the feature?
- What considerations do we have to make when introducing the feature to our goals?
Acquire more price-sensitive riders
How do we know that launching this feature will help us acquire more price-sensitive riders? Well, one thing we could immediately measure would be the number of riders that open the app, type in a location, and then don’t request the ride due to the price or the estimated wait time. We can define price sensitivity by looking at each user’s average ride request rate completion, given different price thresholds.
Since this metric is dependent on wait times as well, we would probably need to segment wait times into this analysis as well. Let’s assume we keep distance and other geographic metrics constant.
Price | Wait Time | Request Completion Rate |
---|---|---|
<$10 | <2 minutes | 90% |
$10-20 | <2 minutes | 80% |
$20+ | <2 minutes | 50% |
<$10 | 2-5 minutes | 88% |
$10-20 | 2-5 minutes | 78% |
… | … | … |
Given this analysis, we can understand how big the portion of price-sensitive users is in our marketplace. We know what proportion of our users we could potentially increase the number of riders if we decreased pricing and increased wait times.
If we run this analysis and see that increasing wait times by an average of 10% decreases the request rate by 5%, but a decrease in price by 10% increases the request rate by 15%, then the tradeoff would be worth implementing in terms of a feature change.
Wait times -> +10%
Request Rate -> -5%
Decrease price -> -10%
Request Rate -> +15%
We could do this for multiple different prices and wait time thresholds.
Increase in rider usage
We can run this same analysis for price sensitivity but now change the target metric to accurately reflect this particular goal. In this case, we want to increase rider usage where riders are price-sensitive. This boils down to increasing the number of rides per price-sensitive user, increasing the frequency that users use Uber, and generally increasing the lifetime value of our price-sensitive users as well.
Let’s look at the price-sensitive users that we defined in our previous example. We can imagine that for certain users, there exists a drop-off point from which they don’t use Uber anymore when dealt an extremely high price or an extremely high wait time after making a request.
In either scenario, we want to understand what the retention curves look like for each user to then launch a feature that can communicate both the decrease in price and an increase in estimated wait time.
Level up your interview prep. Join Educative to access 80+ hands-on prep courses.