How To Keep Your Users | Startup School

• Core Thesis: High cohort retention is the single most quantitative indicator that a startup has "made something people want."
• Speaker Background: David Leib (YC Group Partner) co-founded Bump (100M+ users, acquired by Google), which formed the basis of Google Photos (serving >1 billion users).
• Origin of Insight: Leib realized the critical nature of cohort retention only during a Series A pitch where his "hand-wavy" answer about user retention revealed his lack of understanding until post-meeting research.

Defining Cohort Retention Metrics

• Cohort Definition: Groups of users are isolated based on their start date (e.g., weekly or monthly), though advanced analysis can slice by country, region, acquisition channel, or device.
• Action Definition: The specific user behavior counted as "active" must correlate with real product value; shallow actions (e.g., app opens) are insufficient.
  • Instagram Example: "Viewed three or more posts" filters out users who open the app but get no value.
  • Uber Example: "Completed a ride" ensures the user achieved their intent.
  • Google Photos Example: "Tapped and viewed a photo full-screen" signals genuine engagement with content.
• Time Period Granularity: The measurement interval must match the product's intended usage frequency.
  • Social/Entertainment: Daily (e.g., TikTok, Instagram).
  • Utility: Weekly (e.g., Google Photos, Uber).
  • Low-Frequency: Quarterly, semi-annually, or annually (e.g., Airbnb).

Interpreting Cohort Curves

• The "Flatness" Rule: The shape of the curve matters more than the absolute height; retention must flatten out to indicate sustainable accumulation of users.
• Growth Implication: Flat curves allow a startup to add net new users over time; non-flat curves result in a "treadmill" where acquiring new users is immediately offset by churn.
• Google Photos Validation: Early curves dropped quickly but flattened at 20-40% retention; this stability gave confidence in eventual billion-user scale within four years.
• The "Holy Grail": The ideal curve not only flattens but trends upward over time, indicating increasing usage density and network effects.

Common Pitfalls and Misinterpretations

• Incorrect Time Periods: Using overly long intervals (e.g., quarterly for a weekly app) artificially inflates retention numbers, masking poor product-market fit.
  • Leib's Mistake: Bump initially showed terrible weekly retention; extending the window to monthly and quarterly created a false impression of success.
• Weak Action Definitions: Defining "active" too loosely (e.g., viewing a notification bell) allows for gaming metrics without actual value delivery.
  • Google+ Case: Counting users who merely clicked a notification bell inflated active user numbers while true cohort retention remained poor.
• Payment vs. Usage: Relying solely on "paying" as an action is counterintuitive; users often stop using a product before they stop paying (e.g., unused streaming subscriptions).
• Single-Point Analysis: Focusing on a single retention week (e.g., "80% week-over-week retention") without analyzing the trend line or full curve shape leads to incorrect conclusions.
• Analytics Tool Limitations: Built-in dashboard metrics often measure "rolling retention" or cumulative return rather than specific cohort behavior; founders should verify definitions against raw logs.

Strategies for Improving Cohort Retention

• Product Iteration: Making product improvements (speed, simpler flows, new use cases) will cause curves to become flatter and higher.
  • Observation: Improvements made in mid-year cohorts often result in visibly better retention trends compared to earlier cohorts.
• User Acquisition Targeting: Acquiring users who match the product's value proposition improves retention; mismatched audiences (e.g., Gen Z for a memory-archiving tool) show poor retention.
• Cohort Slicing: Segmenting cohorts by demographics or channels often reveals high-performing segments, guiding product focus and acquisition strategy.
• Onboarding Optimization: Investing in activation and teaching users how to reach a "good state" of product usage is a high-leverage, low-cost intervention.
• Network Effects: Products where more users increase value for existing users (social networks, texting apps) should see retention curves rise as the network densifies.

Visualizing Success: The "Layer Cake" Chart

• Construction: Aggregating users by calendar month rather than relative time, where each layer represents a specific cohort's contribution to total active users.
• Significance: A growing top line composed of thick, stable layers from old cohorts indicates a sustainable, scalable business trajectory.
• Qualitative Integration: While cohort data identifies if a problem exists, founders must combine this with direct user interviews to determine what to change.