Define Success Metrics

Success metrics are the main metrics that we need to measure and think about for the product to actually measure success. We’ll explore some strategies for formulating these metrics in this lesson.

Define objects

Defining the objects of the problem helps structure a thought process for creating metrics. In our example scenario, our objects are attendees, events, and hosts. Hosts must create events, which then get attended by our users or attendees. These relationships are the basis for Facebook Events and give us everything we need to create metrics off of them.

Within complex products, there can be even more granularity that is reached, but we want to start from the top and then work our way down. For example, if users post on an event, maybe we want to track posts per event. However, this metric is a proxy for something bigger, which is just an attribute of event activity. Therefore, we can slowly work our way down by starting at the top.

This is where the MECE framework comes in. If we’re analyzing a problem, we don’t want to immediately jump into one of the bottom nodes in a tree versus slowly traversing through each branch/problem and deciding how deep we should investigate.

How the product works

Within Facebook Events, we must note exactly how the product works. Facebook Events allows users to create an event with a date, description, and public or private permission. Each event will allow a host to invite friends or make it open to the public to attend, which the invite list and attendee list can be public or private. Each event will also hold posts and comments, and attendees/invitees will be able to share these posts and pictures.

User journey flows

Mapping out the users journey helps us understand how to formulate metrics that will inevitably measure the core health of the product. Like the Investigative Metrics chapter, user journeys allow us to understand the product a little deeper and give us more contextual knowledge about how it relates to the larger platform as a whole.

Each individual behavior and user flow helps account for edge cases we might not have thought about in the beginning and helps with discovering which behaviors matter more than others in order to prioritize metrics.

We can subdivide user flows into the following parts:

Acquisition

Creating an understanding of how users discover the feature and land on the feature is important from the perspective of each object. In this case, it would be:

Attendees

• Browsing events on the events page
• Being invited to a Facebook event
• Seeing a public event on the newsfeed

Hosts

• Where can organizers create events? On the events page, or is it scattered across different platforms on Facebook?

Funnels and conversion

After the acquisition stage, how are the users converting into the product or feature? What is the drop-off in each step of the funnel? For our example, we want to know:

How many users convert to attendees from

• The newsfeed
• Browsing the events page
• Seeing that their friends will attend the event
• Increasing the number of friends to determine whether this increases the likelihood of the user converting to an attendee
• Being invited to attend the event

How many users convert to becoming hosts from

• The different parts of the platform where you’re able to create events

At this stage, it is worth considering the following things: what is the drop-off at each step of creating an event? Are users getting stuck on some part of the UI that is difficult to use?

Moreover, we can also consider the question of engagement here: what level of detail is expected in the event for it to be engaging enough for other attendees to commit to attending?

Product engagement

Engagement behaviors allow us to understand how users interact with the product after converting through the funnels. How do user and product interactions eventually affect the top-level goals that we’re trying to achieve?

Example engagement behaviors for Facebook Events are looking at what users do once they’ve seen an event or created an event. These behaviors can all be encapsulated into metrics that we can list down to understand the engagement between the user and the product.

For example, let’s consider the following questions:

• What percentage of users are attending an event?
• How much time do users spend browsing Facebook events?
• What is the total number of events that a user will confirm attendance out of the number they browse?
• What’s the average number of events a user will create and host?
• What’s the percentage of friends that a user will invite to their event that’s public versus private?

List the top-level metrics

Given these user journeys and how we’ve mapped out the relationship between the user and the product, we can now try to suggest top-level metrics that we want to measure. These will be the initial core metrics that represent the health of the product most of the time. We can imagine them as the metrics or graphs we would want to see on a daily dashboard to understand how the product is performing.

These metrics usually depend on three factors: engagement, revenue, or growth. For the Facebook Events example, we have to be sure to understand the core metrics around general user growth and engagement. Generally, these metrics showcase the product is still functioning more than anything else.