For a product to succeed over a long period of time, several conditions must be present: product-market fit, positive unit economics, and the ability to scale and grow. Product-market fit requires a deeply engaging product to which people return because they find genuine value. Positive unit economics require careful attention to the fundamental financial building blocks of the business. To effectively scale requires, among other factors, a sustainable organization. In this post, we offer guidance on building both sustainable products and scalable organizations—specifically, by developing frameworks.
FRAMEWORKS: WHAT THEY ARE AND WHY WE NEED THEM
A framework is a way of thinking about a problem—a way to understand something complex. Frameworks help us simplify; most problems seem complex until we create frameworks to help us think through them. Frameworks make problem-solving tractable.
Frameworks are a constant presence not only in businesses but in our personal lives. Whether consciously or not, we use them to simplify and guide many decisions—for example, which school to attend and which major to choose. By determining key dimensions (such as schools, programs, earning potential and distance from home) and setting criteria (such as top five school, top five program, earning potential greater than $100,000 and West Coast location) we create a framework for decision-making. The same process can help us understand how well a product is performing, how to build an effective organization, how to retain users, how to activate more users, etc.
Without conceptual frameworks, the process of identifying and solving problems is difficult to scale, repeat, extend and reuse. Frameworks are necessary to effectively diagnose and understand the root causes and associated drivers of a problem, to obtain actionable insights and to communicate findings.
BUILDING FRAMEWORKS
Frameworks are developed by breaking a problem into parts, each of which can then be broken down further. The examples below are designed to help you understand this process.
The key takeaways below are not unique to each example; rather, you can apply them across many situations.
Example 1: Data-informed company building At the company level, it is often valuable to establish a single framework that can connect your entire business. By helping employees understand what matters most to the company, such a framework can motivate the team, unifying individuals team around one metric and helping them connect the company’s goals to its mission. A single company-level framework can also help you resolve conflict and decide how to structure the team.
As an example of how frameworks can help you build a data-informed company, consider Facebook. Facebook’s mission is “to give people the power to build community and bring the world closer together.” The company dreams of a world in which everyone is on its platform—where we are all plugged into and deeply engaged with the communities around us and therefore closer together.
From a metrics perspective, Facebook’s mission implies a focus on user growth, meaning the company would like every single person on the planet to use the platform. The metric that best encapsulates this goal would be either monthly active users or daily active users (DAU), as building strong communities and bringing people closer requires users to deeply engage with the product. To encourage this behavior, Facebook recently began prioritizing "meaningful" (rather than total) time spent on the platform; users now see more posts from family and close friends and fewer viral videos.
To continue to grow its product and engage its users, Facebook must also monetize by running ads on the platform. These three factors—growth, engagement and monetization—are the primary levers by which Facebook can grow revenue, as shown in the simple framework in Figure 1.
By organizing its business units around these levers, a company can transform itself into a more data-informed and metrics-focused organization. In the case of Facebook, the Growth team could focus on growing DAU as its “north star” metric; such alignment could help resolve conflict about the value of specific initiatives. For example, we know that if everything else were equal, shipping products that increase DAU would be the right thing to do.
Similarly, the Engagement team could focus on increasing time spent per user, while the Monetization team could work to grow revenue per minute of time spent.
Effective pursuit of these goals also requires checks and balances to ensure improvement of these key metrics does not negatively affect other company-level metrics. For example, while attempting to increase DAU, the Growth team should also monitor revenue per unit of time spent.
While metrics such as those in Figure 1 are also useful at the team or business unit level, developing a simple company-level framework can significantly improve your ability to monitor your business at all levels. At the highest order of aggregation, executive dashboards shed light on how each business unit is performing overall.
Dashboards can also be tailored for the leaders of each business unit, to give them a deeper understanding of how their team is doing, what’s driving their business and the strategies they should consider. These dashboards can then be further broken down into subunits of each business unit and monitored by mid-level executives to determine how they can affect a top-level business goal. In this way, a single company-level framework can be “broken up” into parts, empowering each business unit to monitor its own metrics and providing clarity, ownership and accountability at every level.
Example 2: Activating users In this example, we examine how a growth team might build a framework for user activation. Growing users sustainably (a key goal for every product) requires retaining existing users, resurrecting churned users and attracting new users. Growth teams focus on getting new users at the top of the funnel and then converting them at each subsequent step.
New users can be reached through multiple organic and paid marketing channels; Figure 2 shows a funnel for outreach via email. “Users Reached” includes all users who receive the email, but only some will read it. Of those who do, a small percentage will click on a call to action, such as a link to a page where they can install an app. Among those who click on the link, only some will actually complete installation. Another drop-off occurs among people who install the app but do not open it and reach a landing page. From there, only a small fraction of users will reach the signup page, create a new account and successfully login. A simple formula for the number of successful logins is:
This framework can be used to set new goals and initiatives and a new product roadmap, with each term in the formula above as a potential lever to increase the overall number of logins. To improve activation, it is paramount that you understand every part of the funnel; as in the first example, monitoring each step of the process is critical. You should optimize at all levels, making the terms in the formula part of your monitoring dashboard; understanding them deeply and in the context of relevant benchmarks; “diagnosing with data” to hypothesize what would move them; and “treating with design” by setting your goals, initiatives and product roadmap against them.
Example 3: Retaining users To improve overall growth, it is valuable to develop a framework for understanding retention. Below, we provide a formula that combines different levels of a funnel of user cohorts (similar to the framework in Example 2).
Retention is the best indicator of whether your product is valuable and whether you have found product-market fit, because retention tells you whether people who tried your product liked it enough to use it again. The best way to enable healthy retention is to build a product your core users—the ones who are most engaged—will love. This involves both creating a magical moment where users first “get” the product (see Example 4 below) and identifying the tipping point that establishes user retention.
Figure 3 shows the retention funnel for a cohort of users as they go through the signup flow and become activated or “new” users.
For many products, the first day or week is critical for user retention. Do users have enough content to consume? Do they have a few close friends to chat with? Do they have a clear understanding of what the product is?
Knowing the retention of a cohort after 1, 7, 28, 84 and 364 days can help your Product team prioritize and focus its efforts, as well as create specific strategies, roadmaps and initiatives. For example, ifD364/ D84=1 (that is, the number of people who came to the product stayed constant between days 84 and 364), you know all net churn happened before day 84. If you also find that D84/ D28=1, you know all net churn occured in the first 28 days.
In this scenario, you should focus on what you can do in the first 0, 1, 7 and 28 days to create the best possible experience for a new user. To investigate options, you can perform an exploratory analysis—hypothesizing the reason or reasons for each drop-off, analyzing those potential causes, identifying issues and opportunities, and offering product roadmap recommendations based on the levers you have and the actions you can take. In a future post, we will describe details of how to conduct exploratory analysis.
As in the examples above, monitoring each level of the retention funnel is critical to improving your product. We strongly recommend creating a dashboard that captures retention at the highest level, as well as additional dashboards that address retention on various days of the user’s journey.
Example 4: First experience and the magical moment For many products, the first-day experience is the most important for engaging and retaining users. When a user first experiences a product, they don’t know what to expect, don’t understand what the product is about and don’t know how to navigate it. Therefore, it is critical that you offer users a fantastic first-day experience and offer them a “magical moment” that makes them want to come back each day, leaving them with an understanding of the value that the product provides.
This is particularly important in gaming, as there is typically a sharp drop-off between the first and second day. The more time users spend with a gaming product on the first day, the more likely they are to return the following day; therefore, most games offer tutorials, levels, missions, bonuses or awards to new users.
For example, imagine the fictional game shown in Figure 4, in which a user who activates their account, continues to the home screen, completes a tutorial and reaches the first level then gets a bonus. This serves as the “magical moment”; users are encouraged to start Mission 1, and some will eventually complete it.
In this example, just five percent of users who do not reach the home screen come back the second day (D1), while users who finish the tutorial have a 40 percent chance of returning the next day and users who finish both the tutorial and their first mission return 80 percent of the time. This suggests the game’s Product team should focus on encouraging users to complete Mission 1. However, if most people drop off at the tutorial stage and a very small percentage of people complete Mission 1, the team should instead focus on the user experience before the tutorial stage, where the largest drop-off occurs.
By understanding both user retention at each part of the funnel and the relationship between retention rates after one day, one week, etc., you will better understand where to focus efforts to increase engagement. If the drop-off at a given stage is negligible, that stage should not be prioritized. On the other hand, a “broken” experience that significantly affects the overall conversion rate should be addressed. In many cases, the fix is as simple as changing the design flow or fixing a bug.
Ultimately, the goal in this example would be to get most users to reach Mission 1 and to create magical moments along the way, ensuring those users are retained in the long term. By breaking this seemingly intractable problem into smaller parts and creating a framework, a Product team can divide and conquer.
TIPS FOR CREATING FRAMEWORKS
Know the outcome To create a framework, you must understand the outcome you are trying to achieve. In Example 1, the desired outcome is to develop a formula that logically connects different parts of the organization. In Example 2, the goal is to improve activation of users and get them to login. In Example 3, the desired outcome is to improve retention. And in Example 4, it is improving the first-day experience, which will eventually help achieve product-market fit.
Identify building blocks To develop a useful product roadmap, you must carefully choose levels of your funnel. They should be actionable and should clearly lead to the desired outcome. In Example 3, this outcome is retaining as much of a cohort as possible over the long term. Because we have different levers to pull on a user’s first day, first week and later on, it makes sense to break this retention problem into parts based on time. Hypothetically, one roadmap would be to improve retention in the first week by increasing the number of friends.
Create a formula A formula can connect each step or stage of your process and help you understand which of them matter most. It can also help you move tractable metrics that align with your “north star metric.” You can create such a formula by converting the funnel into a series of ratio metrics, thereby identifying levers to help you achieve a desired outcome.
CLOSING THOUGHTS
Frameworks are a valuable tool in building a data-informed company and can guide your roadmap to help you build both scalable organizations and sustainable products. As shown in the examples above, you can use frameworks to align your company on key metrics; to break down growth, activation and retention; and to help improve engagement by offering users magical moments and amazing first-day experiences.
However, despite their many benefits, frameworks are not silver bullets. Determining how to construct a funnel can be difficult, and measuring various parts of the funnel may be impossible. Ultimately, there is no substitute for human creativity, experience, discipline, diligence and judgement.
TAKEAWAYS
- Design a framework by breaking your problem into parts (such as levels of a funnel) and deriving a formula that connects those parts.
- Ensure your frameworks logically connect an overall desired outcome to its levers and the actions you can take.
- Excellence in analysis, design, implementation, validation and optimization is necessary to reap the benefits of frameworks.
Follow us on Medium for weekly updates.
This work is a product of Sequoia Capital's Data Science team. Jamie Cuffe, Avanika Narayan, Chandra Narayanan, Hem Wadhar and Jenny Wang contributed to this post. Please email data-science@sequoiacap.com with questions, comments and other feedback.
