Topic
Best Ab testing Podcast Episodes
Ab testing is covered across 1 podcast episode in our library — including My First Million. Conversations explore core themes like liquid vs. crystallized intelligence, resulting, no permission tech, drawing on firsthand experience and research from leading practitioners.
Below you'll find key insights, core concepts, and actionable advice aggregated from the top episodes — followed by a ranked list of the best ab testing discussions to explore next.
Key Insights on Ab testing
- 1.DHH believes that operating as a private company, free from investor constraints and the need to disclose full revenues, offers a significant privilege for building a business based on personal philosophy [00:40, 05:08].
- 2.Instead of outspending competitors, companies can "outteach" and "outinteresting" them by consistently sharing their unique philosophy and observations, a strategy DHH implemented at 37signals due to their lack of VC funding [03:00-04:00].
- 3.The "liquid intelligence" of younger individuals, characterized by smart, fast, yet sometimes ignorant thinking, is often the source of novel, world-changing ideas because it lacks the baggage of established paradigms [10:00-11:00].
- 4.Significant constraints, such as those faced by 37signals after the dot-com bust, can paradoxically fuel an "explosion of creativity" and lead to groundbreaking innovations like Ruby on Rails [14:00-15:00].
- 5.Having substantial profit margins allows founders the "luxury of margins" to prioritize taste and intuition over exhaustive A/B testing for marginal gains, even if it appears "reckless" by conventional business standards [25:31, 29:33].
- 6.The concept of "resulting" warns against evaluating a decision solely by its outcome; a statistically sound decision might still fail, and a poor decision might accidentally succeed [19:30-20:30].
Key Concepts in Ab testing
Liquid vs. crystallized intelligence
Liquid intelligence involves fast, quick reasoning and problem-solving, peaking in one's 20s, while crystallized intelligence involves accumulated knowledge and connections, developing later in life. DHH argues that "liquid intelligence" drives new ideas, often from young individuals who are quick, smart, and ignorant of existing limitations, allowing them to challenge established norms.
Resulting
A poker term referring to the error of evaluating the quality of a decision based solely on its outcome. DHH uses this to explain why his "wrong" Facebook valuation prediction was still based on a sound (at the time) analysis of its monetization strategy, rather than just the eventual outcome.
No permission tech
A philosophy advocating for technologies and platforms where users do not need to ask for permission from a central authority to create or publish. DHH champions this principle, citing the internet and open source software as prime examples, and highlights how Apple's App Store policies on Hey.com directly contradicted this ideal.
Luxury of margins
The ability of a highly profitable company to prioritize intuition, taste, and founder satisfaction over strict data-driven optimization for marginal gains. DHH attributes 37signals' ability to eschew extensive A/B testing and focus on what "felt good" to their consistently high profit margins, which afforded them significant freedom.
Actionable Takeaways
- ✓Prioritize sharing your unique philosophy and insights consistently to earn attention and build an audience, rather than relying solely on paid advertising to outspend competitors [03:00-04:00].
- ✓When starting a venture, embrace constraints (e.g., limited capital) as opportunities to foster creativity and find novel, more productive solutions, much like the genesis of Ruby on Rails [14:00-15:00].
- ✓Consider whether your business's profit margins allow you to prioritize intuition and good taste in product and marketing decisions, rather than feeling obligated to perform endless A/B tests for marginal gains [25:31-29:33].
- ✓Reflect on past decisions by separating the quality of the decision from its outcome, using the "resulting" concept to avoid misinterpreting success or failure solely based on a single result [19:30-20:30].
- ✓Explore the "no permission tech" ethos by building on open platforms and leveraging open source software to minimize reliance on gatekeepers and maintain creative freedom [71:48-72:50].
Top Episodes — Ranked by Insight (1)
My First Million
$100M+ Advice That'll Piss Off Every Business Guru (ft. DHH)
DHH believes that operating as a private company, free from investor constraints and the need to disclose full revenues, offers a significant privilege for building a business based on personal philosophy [00:40, 05:08].
Episodes ranked by insight density — scored on key takeaways, concepts explained, and actionable advice. AI-generated summaries; listen to full episodes for complete context.
