Conference Presentation, Fireside Chat
Suhail Doshi - How to Measure Your Product
- Market Position and Scale: The speaker projects Mixpanel operates with approximately 300 employees and 7,000 paying customers, generating annual recurring revenue near $100 million.
- Audience Composition: An initial assessment suggests the majority of the current audience consists of companies with zero users, highlighting a gap between the audience's scale and the speaker's established enterprise.
- Strategic Philosophy on Metrics: The presentation advocates for simplicity, warning that overcomplicating metrics causes immediate paralysis and decision-making issues even in mid-sized or large organizations, predicting that companies with dashboards of "30 panes" or "mission control" become confusing for their workforce.
- North Star Metric Commitment: Founders are advised to commit to a single North Star metric for at least six months, accepting that the initial choice will likely be incorrect, while monitoring only three to five additional metrics to avoid discombobulating the workforce.
- Growth and Retention Risks: A "shark fin effect" is predicted where high user churn rates, such as 7% monthly, result in losing 58% of annual revenue, leading to business failure within four or five years if not corrected.
- B2B Conversion Benchmarks: Conversion rates from visit to sign-up for B2B companies are expected to hover around 4% to 5%, with significant variance in retention benchmarks across sectors like gaming, e-commerce, and social platforms.
- Product Optimization Strategy: The speaker predicts that companies must optimize every step of the user flow continuously, as even complex products like Airbnb require roughly 50 iterations to appear simple, and neglecting the first step eliminates chances for success in subsequent stages.
- User Behavior and Loss: Users will leave immediately if a product is not easy to understand, indicated by a mouse hovering over the back button, and reactivation campaigns generally fail because lost users rarely return without a specific trigger.
- Feature Prioritization: Resources should not be spent on fraud protection or features like "forgot password" until demand necessitates it, though a clever workaround (automated emails after three failed attempts) can provide significant retention benefits.
- Data Collection Methods: For companies with fewer than 50 users or zero users, talking to customers directly is the only reliable method to measure product understanding, often providing more depth than data graphs.
- Experimentation Constraints: B2B companies face harder traffic volumes for A-B testing compared to consumer companies, requiring them to conduct as many experiments as possible, though some decisions can be made quickly by asking a small group (e.g., 11 people) via instant messaging.
- Historical Product Lessons: The original tagline "Metrics that'll make you drool" is predicted to have failed by lacking differentiation from Google Analytics, whereas "actions speak louder than pages" effectively secured market focus for the following five years.
- Hardware and Subscription Models: Startups with hardware components or subscription models, such as Jawbone and Fitbit, are expected to successfully apply these same engagement and retention metrics.
- Long-term Forecasting: The speaker predicts that ignoring retention leads to a long arc of decline where the company cannot acquire new users fast enough to replace churn, and that complex flows can be simplified but require significant work to perfect.