Conference Presentation, Fireside Chat
Suhail Doshi - How to Measure Your Product
- Suhail Doshi, co-founder of Mixpanel (founded 2008), presented a framework for product measurement centered on solving three core problems: product understanding, ease of getting started, and retention.
- Mixpanel currently employs 300 people, serves 7,000 paying customers, and is approaching $100 million in annual recurring revenue.
- Doshi identified the primary goal of analytics as identifying "levers" for growth rather than creating complex dashboards, warning that over-complicating metrics leads to decision paralysis even at large companies.
- The growth formula consists of five sequential levers: (1) visitors to page, (2) visitors who sign up, (3) visitors who perform a valuable action, (4) visitors who return later, and (5) visitors who refer others.
- Doshi emphasized that even thousand-person companies often fail because they track too many metrics; he recommends selecting a single "North Star Metric" to bet the company on for six-month increments.
- In the first 18 months, Mixpanel struggled to communicate its value, initially using the tagline "Metrics that'll make you drool" which failed to differentiate them from Google Analytics.
- Mixpanel pivoted its messaging to "Actions speak louder than pages," explicitly differentiating their engagement tracking from Google Analytics' page-view focus.
- Key metrics for determining if a product is understood include conversion from landing page to sign-up and, for B2B, the ratio of users clicking the pricing page.
- For startups with fewer than 50 users, Doshi advises against relying on A/B testing and recommends direct qualitative feedback from a small group of users to validate hypotheses.
- Mixpanel's early landing page (2009) contained grammatical errors and targeted everyone, resulting in zero users before they refined their positioning toward startups.
- A critical insight for "ease of getting started" is that if the first step is difficult, no amount of subsequent feature development will succeed; early prioritization should ruthlessly focus on the initial user hurdle.
- Mixpanel delayed building "Forgot Password" functionality for 18 months to prioritize core value, noting that users often re-attempted login multiple times rather than using recovery options.
- To solve login frustration without building a dedicated feature, Mixpanel implemented a trick where users were emailed a magic link to log in after three failed attempts, significantly improving retention.
- Doshi warns against early optimization for fraud prevention (e.g., email confirmations) unless the startup faces actual spam issues, citing that confirmations can cause a 60–70% drop-off in user activation.
- The "Shark Fin Effect" describes a growth pattern where rapid viral acquisition is followed by a collapse because retention is ignored; users are lost faster than they can be acquired.
- LivingSocial is cited as a case study where a viral Facebook app had to pivot entirely to a Groupon-style business because its original product lacked retention.
- Doshi recommends measuring retention over longer periods (e.g., 30 days) rather than one week, as short-term spikes do not accurately reflect long-term product value.
- Daily Active Users (DAU) is advocated over Monthly Active Users (MAU), which Doshi views as a "BS metric" similar to registered user counts; DAU strongly correlates with reduced churn.
- For subscription businesses, even a 7% monthly revenue churn results in losing 58% of revenue annually, creating a "flat death" scenario where growth must outpace massive attrition.
- In the Q&A, Doshi noted that B2B conversion rates from visit to sign-up typically range between 4% and 5%.
- Doshi stated that social login options generally convert better than standard email/password sign-ups but emphasized the need for companies to test benchmarks specific to their industry.
- Regarding product-market fit, Doshi suggests using harsh retention benchmarks (e.g., ensuring users perform multiple valuable actions to count as "active") rather than loose definitions.
- Hardware startups with software components (e.g., Fitbit, Jawbone) can and should use the same digital metrics as software companies, provided the hardware is connected.
- Doshi concluded that the most common founder mistake is over-complicating products and metrics with every version; founders tend to get distracted by "next features" before solving fundamental retention issues.