Our first series on engagement discussed activity feed and social product environments, which offer a mix of user-generated and professional content. In this series, we explore engagement on platforms that offer purely professional content. In the first part, we described how production of evergreen professional content makes your product truly engaging. In part two of this series, we focused on how to make content relevant by making the right recommendations. In the final part of this series, we focus on content consumption and feedback as well as providing guidance on product implications.
UNDERSTANDING CONTENT CONSUMPTION
A user’s connectivity and device significantly influence their consumption experience for any product, including — if not especially — one that offers professional content. Thus, it’s important to keep the following factors in mind:
Consumption device
How does your product appear on different devices (iPad vs. smartphone vs. TV, etc.)? How will appear to users on low-end devices?
Optimization function
What do you want to optimize your product toward? Depending on your vision, the answer may be more time spent, more valuable time spent, more original content, or another factor. How do you make the tradeoffs between each?
Low-end devices and lower connectivity
What types of content should be recommended to users in regions where poor connectivity is common?
Order of content served
The order of content served and the recommendations also have strong implications of what content gets viewed and watched.
Search
Search is becoming an increasingly important tool for people to find relevant content. Having a great search engine that can recognize the intent of the people is very valuable.
METRICS TO TRACK
Time spent/DAU
Time spent is generally a strong indicator of whether your product is engaging. Market share of time spent is also a useful indicator.
Number of sessions/week
Do people keep coming back to the product? This is the earliest indicator of product-market fit, and a decrease in number of sessions is the earliest warning sign that things are going wrong.
Time spent/session
Like number of sessions, this metric will boost overall time spent. Determine which is the most powerful lever for your product.
Number of videos started, completed, added to watchlist
Total number of videos watched is a good indicator of whether a product is engaging.
Also, understanding the waterfall from videos started to completed are useful measures for engagement.
PRODUCT IMPLICATIONS
Explore versus exploit
Should you optimize for (exploit) what you already know about your users’ behavior, or try to learn more (explore) what you don’t know? That is, to what extent should you highlight the kinds of content they’re likely to value, and to what extent should you highlight the kinds of videos they haven’t tried? This is a fundamental question for all ranking algorithms, and there is no simple answer. How much of content should your product show that is popular versus personalized for the user?
Not enough data
No matter how meticulously you construct your algorithm, there will always be data you don’t have. This is especially true when people have watched very few videos.
No optimization function is perfect
Prediction algorithms are designed to optimize toward a given metric or metrics. However, such metrics can never fully capture the spirit of a company’s goals and mission — and predictions and relevancy scores will thus never be entirely sufficient. Ranking algorithms can help predict whether and how a user will interact with a piece of content, but not whether that interaction truly serves your mission.
Shared account implications
Because each account for a Netflix-like product is often shared by multiple people, it may not be clear exactly who watched a given piece of content. This means your product must strike a balance between personalized recommendations, and recommendations that reflect the entire account’s viewing history.
Seasonality
Delivering the right content to the right users in the right order is in part a function of time, and will thus consumption will be affected by variations in user behavior based on the day of the week, the season of the year, summer, holidays, etc.
Signals used for ranking
The content type and diversity, binge-ability, recency, type of user and other signals all impact the order in which the content is served which in turn affects the consumption.
TAKEAWAYS
- Consumption of content is strongly affected by device, connectivity, order of content served and search.
- Type of content, Recommendation algorithms, Explore versus Exploit choices, Seasonality, Shared accounts, Optimization function etc. strongly influence the engagement of professional content.
This work is a product of Sequoia Capital’s Data Science team and was originally published on Medium. Chandra Narayanan and Hem Wadhar wrote this post. See the full data science series here. Please email data-science@sequoiacap.com with questions, comments and other feedback.
