Defining Product Success: Metrics and Goals

Data Science Team

Goals are important in defining and monitoring success. When a goal is in place, the destination is clear—even though the route may change. Goals help connect your mission to your strategy, roadmaps, initiatives and tactics by tying the single metric you care about most with a target and a time frame during which it can be achieved. More than anything else, this process will help your team define success. In this post, we’ll explain how to define success for your product by identifying the one key “metric that matters” and then, how to set the right goals.

DEFINING THE RIGHT METRIC

What is the one metric that matters most to the success of your company and that you can rally your team around? For Facebook, it is active users; for WhatsApp, it is number of sends; for eBay, it is gross merchandise; for PayPal, it is total payment volume. Once you identify this “top-line” metric, you can set success criteria around it, monitor it, understand what drives changes in it, obsessively push it in the right direction—and properly evaluate and manage the health of your product.

For growth, a simple top-line metric might be number of users: for engagement, time spent; for monetization, revenue or number of advertisers. Fundamentally, your choice of metric should be driven by your vision for your product and the mission of your company. If you dream years into the future and visualize your product and company, how would you qualitatively describe them?

A vision statement should be aspirational, inspiring and future-focused. For example, eBay’s vision for commerce is “enabled by people, powered by technology and open to everyone.” eBay’s mission is “to be the world’s favorite destination for discovering great value and unique selection.” Taken together, these two statements point toward eBay’s dream of a world where everyone can find whatever they want, however obscure, at a good price.

What top-line metric would best encapsulate this goal? It is not the number of active users shopping on the site; that metric doesn’t measure whether buyers are actually finding what they want, and at the right price. How about the number of active buyers? While that and similar buyer-side metrics can tell us whether users are finding what they want at the right price, they cannot address the “unique selection” criteria of eBay’s mission statement.

Maybe a seller-side metric would work—how about the number of sellers? This metric can be helpful, but it doesn’t measure the inventory sellers are listing. How about the number of listings? That metric doesn’t tell us whether the listings are unique or whether the inventory is selling.

Ideally, the metric we choose would measure the number of unique items sold. A large number would suggest buyers are likely getting what they want, and at the right price. Therefore, the “right metric” in this case would be the market share of the sale of unique selections. However, this metric is complex and difficult to define, measure and grow. Instead, eBay chose as their top-line metric gross merchandise volume (GMV), or the total value of merchandise sold over a period of time. While this metric measures value to users, it does not guarantee unique selection. Nevertheless, it captures much of the spirit of eBay’s vision and mission. Picking the right metric can be as much art as it is science.

Some additional guidance on choosing a top-line metric for your company:

data_04_01

SETTING GOALS

Teams often think about metrics and goals simultaneously, as they cannot easily be separated. Once you have identified the right metric and goal, you will be prepared to define a strategy and roadmap against which your product team can execute. Goals should highlight what you hope to accomplish and are often stepping stones to accelerating business growth. They can unify your company around a common objective and hold your team accountable for its promises.

As an example, let’s assume you want to grow your number of active users. A goal statement could be “grow MAU to 10M by Q4 2018.” This goal connects the metric (MAU) to a target (10M) and a time frame (Q4 2018), clearly describing what the product team wants to achieve and providing a purpose for the organization. Goals should be simple, actionable, achievable and most important, easy to measure and track.

You can choose goals based on:

  1. Product or business aspirations: Most long-term goals are based on the company’s mission. For example, if your company is in the video space, you might aspire to have the fastest-growing share of time spent on video. If you want to achieve that goal in x years, you can then break it into chunks to determine the growth you’ll need over the next y months in order to stay on track.

  2. Product metrics: If your product has been around for a while, you can do a “bottom-up” forecasting exercise to determine your goal for your top-line metric over a given period of time. For example, a forecast of MAUs could consider historical data on seasonality, platform, country, penetration and product changes. (Future blog posts will offer in-depth guidance on forecasting.)

  3. New products: If your product is completely new, it will be useful to look at external benchmarks and set “top-down” goals. For example, if the product is a Messenger-style communication app, you may choose to study growth at similar companies and let that inform your goals. You may also consider postponing goal-setting for a completely new product for a couple of months, until you see how it performs.

Some general guidance on choosing goals:

NEXT STEPS: STRATEGY AND BEYOND

Imagine a consumer company with a 50-50 goal to increase monthly active users 10 percent by Q4. This goal meets many of the criteria mentioned above: it is specific, quantitative, time-bound and includes an appropriate top-line growth metric. The next step of the product-building process is to establish a product strategy, such as “resurrect users,” that specifies how to achieve the goal. This strategy can then help drive a roadmap, such as “resurrect stale India users.” The company could generate specific initiatives within this roadmap that would contribute most toward moving their top-line metric. Such initiatives might include “improve SMS notifications in India,” and would in turn lead to engineering tasks such as “build SMS notifications for low-end Android phones.” This example shows how a company’s vision and mission can drive its goals, strategies, roadmaps, initiatives and tasks. Future blog posts will offer in-depth guidance on setting data-informed roadmaps and strategy.

TAKEAWAYS

Follow us on Medium for weekly updates.


This work is a product of Sequoia Capital's Data Science team. Jamie Cuffe, Avanika Narayan, Chandra Narayanan, Hem Wadhar and Jenny Wang contributed to this post. Please email data-science@sequoiacap.com with questions, comments and other feedback.