Invest Like the Best
GPUs, TPUs, & The Economics of AI Explained | Gavin Baker Interview
Guest: Gavin BakerDecember 9, 2025
Episode Summary
AI-generated · Mar 2026AI-generated summary — may contain inaccuracies. Not a substitute for the full episode or professional advice.
This episode features Gavin Baker, a renowned technology investor with an encyclopedic understanding of the tech landscape, known for his infectious curiosity about markets. Baker and host Patrick O'Shaughnessy dive deep into the competitive dynamics of the artificial intelligence (AI) ecosystem, focusing on the intense battle between Nvidia's GPUs and Google's TPUs, and the profound economic shifts unfolding in the AI industry. The central thesis is that AI's underlying economics, particularly cost per token and infrastructure, are poised for dramatic changes, creating existential challenges and opportunities for every major player.
👤 Who Should Listen
- Technology investors and venture capitalists interested in the competitive landscape of AI infrastructure.
- Executives and strategists at major tech companies (Google, Nvidia, Microsoft, Meta) looking to understand market dynamics and potential shifts.
- SaaS company founders and leaders grappling with AI integration and its impact on business models and gross margins.
- Engineers and researchers in the AI and semiconductor industries seeking insights into chip development, scaling laws, and future compute architectures.
- Entrepreneurs and innovators exploring novel solutions for AI power and cooling, including space-based data centers.
- Anyone curious about the long-term economic and societal implications of artificial intelligence and its accelerating progress.
🔑 Key Takeaways
- 1.To truly understand AI's capabilities, investors and researchers must use the highest paid tiers of frontier models like Gemini Ultra or Super Grock, as free versions are analogous to judging an adult's potential based on a 10-year-old's abilities.
- 2.Scaling laws for AI pre-training are empirically intact, as reaffirmed by Gemini 3, but post-training progress has been driven by new scaling laws: reinforcement learning with verified rewards (RLVR) and test-time compute, which bridged an 18-month gap in hardware development.
- 3.Google currently holds a temporary advantage as the "lowest cost producer of tokens" due to its advanced TPUs (v6/v7) and vertically integrated design process, allowing it to strategically undercut competitors and "suck the economic oxygen" out of the AI ecosystem.
- 4.Nvidia's next-generation Blackwell chips, particularly the GB300, are anticipated to shift the cost advantage, making companies utilizing them (especially XAI, which builds data centers fastest) the new low-cost producers of tokens by early 2026.
- 5.Many large tech companies like Meta, Microsoft, and Amazon have struggled to build competitive frontier models, indicating that creating and maintaining a leading AI lab is far more complex than widely perceived, requiring not just capital but also sophisticated infrastructure management and research "taste."
- 6.The "flywheel effect" of user data feeding back into model improvement, absent in early AI, is now beginning to spin with reasoning models, creating more separation among leading labs (OpenAI, Gemini, Anthropic, XAI) that possess advanced internal checkpoints.
- 7.Traditional enterprise software (SaaS) companies are making a fundamental mistake by resisting lower gross margins for AI products, akin to brick-and-mortar retailers' slow adoption of e-commerce, which could lead to their platforms being displaced by AI-native competitors.
- 8.Data centers in space, leveraging continuous solar power and free cooling (near absolute zero on the dark side of a satellite), offer a first-principles superior solution for AI compute, potentially solving Earth-bound power and cooling constraints long-term.
💡 Key Concepts Explained
Scaling Laws for Pre-training
These are empirical observations that predict how model performance improves with increased compute, data, and model size during the initial training phase. Gemini 3 notably confirmed these laws remain intact, despite researchers not fully understanding the underlying 'how' or 'why' they work.
Scaling Laws for Post-training
Two new scaling laws driving recent AI progress: Reinforcement Learning with Verified Rewards (RLVR) and test-time compute. RLVR involves training AI models using outcomes that can be objectively verified (e.g., did a sale convert, did the model balance the books), while test-time compute refers to allowing models to 'think' or process for longer during inference. These laws enabled significant progress even when pre-training hardware was stalled.
Low-Cost Producer of Tokens
In the AI industry, this refers to the entity that can generate AI output (tokens) at the lowest computational cost. Gavin Baker highlights that Google's TPUs have given them this advantage, allowing them to exert economic pressure on competitors. This metric is uniquely important in AI, unlike most traditional tech industries where low-cost production hasn't been the primary driver of market value.
Reasoning (in AI)
Refers to an AI model's ability to 'think' or chain together logical steps to solve problems or generate more sophisticated outputs. The advent of reasoning models (like the first from OpenAI) significantly accelerated AI intelligence levels and enabled a 'flywheel' where user interactions and verifiable outcomes can be fed back to continuously improve the model.
Data Centers in Space
A visionary concept for AI infrastructure, proposing orbiting satellites equipped with chips for compute. From first principles, they offer superior conditions: constant, intense solar power (eliminating batteries) and free cooling to near absolute zero in the vacuum of space (reducing complex cooling systems), potentially leading to faster, lower-cost inference and training via laser communication.
SaaS AI Margin Mistake
The error made by incumbent Software-as-a-Service (SaaS) companies who are reluctant to adopt AI agents because it would reduce their traditional 70-90% gross margins to 35-40%. This is compared to brick-and-mortar retailers ignoring e-commerce. Gavin argues this will leave them vulnerable to AI-native startups willing to operate at these lower margins, eventually leading to their displacement.
⚡ Actionable Takeaways
- →Subscribe to the highest paid tiers of frontier AI models (e.g., Gemini Ultra, Super Grock) to accurately assess their capabilities and track progress.
- →Follow leading AI researchers and engineers on platforms like X (formerly Twitter) and read their papers (e.g., Andrej Karpathy's work) to stay at the cutting edge of AI developments.
- →Listen to podcasts featuring engineers and researchers from the four leading AI labs (OpenAI, Gemini, Anthropic, XAI) to gain direct insights into frontier model progress.
- →Utilize AI tools to manage information overload from the rapidly evolving AI landscape, like asking an AI to summarize podcasts or research papers you've consumed.
- →For SaaS companies: immediately embrace lower gross margins (e.g., 35-40%) for AI-driven agent strategies to avoid obsolescence and capitalize on existing customer data and distribution, as Microsoft has done with Copilot.
- →Keep an eye on companies that excel at applying AI-driven productivity in traditional industries (like C.H. Robinson matching freight loads), as these indicate tangible ROI beyond the immediate tech giants.
- →Consider the long-term implications of power constraints for compute on Earth, recognizing that advancements in natural gas, solar, and potentially data centers in space are critical drivers for AI growth.
⏱ Timeline Breakdown
00:00Introduction to Gavin Baker and the episode's focus on Nvidia, Google, and AI economics.
01:01Gavin's process for staying updated on AI, emphasizing using paid tiers of models and following top researchers on X.
02:01Critique of investors making AI conclusions based on free tiers; importance of high-tier access ($200/month).
03:02AI developments happen on X, following the 500-1000 cutting-edge researchers, and listening to podcasts from top labs.
04:04Gavin discusses using AI to keep up with AI; Gemini 3's significance for scaling laws.
05:05Explanation of scaling laws for pre-training as empirical observations, not fully understood mechanisms, like ancient astronomy.
06:05Misunderstanding about 2024/2025 progress; the need for next-gen chips (Blackwell) and the concept of coherent GPUs.
07:06Blackwell's complex product transition (air to liquid cooled, weight, power increase), delaying its deployment.
08:06How 'reasoning' (OpenAI's first model) saved AI by allowing progress without new pre-training chips, bridging an 18-month gap.
09:07Introduction of two new scaling laws: reinforcement learning with verified rewards (RLVR) and test-time compute.
10:10Google's TPU v6/v7 compared to Hopper/Blackwell, indicating Google's current temporary advantage in compute.
11:10Google's strategic advantage as the 'low-cost producer of tokens' and its impact on competitors; first time low-cost matters in tech.
12:11Prediction of XAI releasing the first Blackwell model in early 2026 due to fast data center deployment by Elon Musk.
13:12The GB300's compatibility with GB200 racks will make GB300 users the new low-cost producers of tokens, especially if vertically integrated.
14:14Profound implications of this cost shift for Google's strategy and the economics of AI; the 'prisoners dilemma' of AI spending.
15:14The impact on Google's stock if it becomes a higher-cost producer; the widening gap with Reuben vs. TPUs.
16:14Reasons for TPU slowdown: Google's conservative design decisions and paying Broadcom a high margin for backend semiconductor services.
17:16Apple's in-house ASIC strategy contrasts with Google's reliance on Broadcom, leading to high costs for Google.
18:16Google bringing in MediaTek as a warning shot to Broadcom; the challenge of making a full stack of chips beyond just an accelerator.
19:19It takes at least three generations for a chip to become truly competitive; comparing TPU V1 to V3/V4.
20:19Amazon's ASIC team (Graviton, Nitro, Supernick) as the best, and the limited expected success of other custom ASICs.
21:21The 'global human dividend' of AI progress: what the infrastructure war unlocks.
22:21Near-term AI applications: restaurant/hotel reservations, personal assistants, customer support (already 50%+ AI for some tech-forward companies), and sales.
23:21AI's strength in 'persuasion' (sales, support) and automating anything that can be verified (accounting, robotics).
24:22The economic returns to artificial super intelligence (ASI) and the prisoners' dilemma driving relentless AI development.
25:23Empirical evidence of positive ROI on AI from public company financials, including opex savings and efficiency gains in recommender systems.
26:25Internal company fights over GPUs between revenue-driving teams and research, and the religious belief in ASI as a driver.
27:26The most plausible 'bear case' for AI compute demand: Edge AI, where powerful models run on local devices (like Apple's strategy).
28:28The power of long context windows to hold vast amounts of information, enabling more complex tasks for AI models.
29:31Assessing the S-curve of frontier models; hard to see differences in performance unless you are an expert using top tiers.
30:33AI's improvements in sophisticated decision-making (e.g., fantasy football) and the need to shift from intelligence to usefulness.
31:33Building blocks of usefulness: consistency, reliability, and expanded context windows (e.g., planning complex vacations).
32:36The importance of expanding 'task length' for economic utility, from booking a restaurant to planning a full family vacation.
33:37AI accelerating product design (e.g., Forell hearing aid), manufacturing, and distribution (supply chain, vision systems).
34:39Fortune 500 companies historically slow to adopt new tech; venture capitalists see real productivity gains in startups already using AI.
35:40Q3 as the first quarter with Fortune 500s (e.g., C.H. Robinson with 20% earnings uplift) giving specific quantitative examples of AI benefits.
36:40C.H. Robinson's AI-driven efficiency in quoting freight (100% quotes in seconds vs. 60% in 15-45 minutes), boosting revenue and ROI.
37:40Worry about a potential 'ROI air gap' if Blackwells are used only for training with no immediate inference ROI; Meta's ROIC decline example.
38:40The '490 other companies' beyond the top 10 tech giants; those with innovation culture will succeed with AI.
39:41Critique of VCs forming AI holding companies to fix traditional businesses, arguing private equity is already proficient at this.
39:41Discussion of Meta, Anthropic, and OpenAI in the context of infrastructure developments.
40:42Eric Vishria's quote: 'foundation models are the fastest appreciating assets in history' – modified to 'without unique data and internet-scale distribution'.
40:42How reasoning fundamentally changed the industry dynamics by enabling a 'flywheel' for frontier labs through user data and feedback.
42:44Mark Zuckerberg's failed prediction about Meta's AI performance; Microsoft and Amazon's struggles in developing top-tier models.
43:45The difficulty of running large, coherent GPU clusters at high utilization rates, and the variance in company capabilities.
44:47The importance of 'taste' (intuitive sense for experiments) in AI research, given the high opportunity cost of large-scale experiments.
45:50The creation of separation among leading labs (XAI, Gemini, OpenAI, Anthropic) through better internal models and continuous iteration.
46:51Chinese open source as a 'gift from God to Meta' for bootstrapping models; China's 'terrible mistake' in forcing use of their own chips.
47:51The widening gap between American frontier labs and Chinese open source due to Blackwell, creating geopolitical leverage for America.
48:51XAI's market share on OpenRouter and its projected early lead with Blackwell models and inference at scale.
49:52OpenAI's 'code red' due to high per-token costs and reliance on external compute providers, explaining their large spending commitments.
50:53Anthropic's success in burning less cash and growing faster, benefiting from relationships with Google/Amazon (TPUs/Trainiums).
51:53Anthropic signing a $5 billion deal with Nvidia, signaling understanding of Blackwell/Rubin's superiority over TPUs; Nvidia's expanded partnerships.
51:53Discussion of data centers in space as the most important development in 3-4 years for power and cooling.
52:53First principles argument for space data centers: 24/7 solar power (6x irradiance), no batteries, free cooling to absolute zero, faster laser networking.
54:56Superior user experience for inference with direct-to-phone satellite communication, bypassing terrestrial infrastructure.
55:58Frictions to space data centers: need for many Starships (launch cost/availability), and the convergence of Tesla, SpaceX, and XAI.
57:00XAI as the intelligence for Tesla's Optimus, powered by SpaceX's space data centers, creating competitive advantages.
58:01The 'iron law of history' that gluts follow shortages; why AI compute demand might defy this, as every company could use 10x more compute.
58:01Monetization of AI through ads in free tiers and commissions on bookings (OpenAI, actions) as sources of ROI.
59:02Taiwan Semi's cautious capacity expansion as a 'governor' preventing an overbuild, despite Intel's empty fabs and Leapoo's strategy.
61:03Power as a critical constraint and 'governor': rewards advanced compute (more tokens per watt) irrespective of price.
62:04Solutions for power in the US: natural gas and solar, located anywhere; Caterpillar increasing turbine capacity by 75%.
64:06AI's impact on young entrepreneurs and CEOs: becoming more polished and effective faster by using AI for advice (pitching, HR, sales).
65:06The accessibility of knowledge through podcasts and the internet is creating incredibly proficient young investment talent.
66:08Semiconductor venture capital resurgence, driven by Nvidia's success, funding an ecosystem of specialized parts needed to keep pace with annual chip cadences.
67:08Nvidia and AMD cannot accelerate alone; they need the entire ecosystem of suppliers (transceivers, wires, lasers) to keep up.
68:09Innovation in memory as a critical gating factor; risk of a true DRAM capacity cycle and extreme price increases.
69:11Application SaaS companies making the same mistake as brick-and-mortar retailers with e-commerce: not embracing lower gross margins for AI.
70:11AI requires recomputing answers, leading to lower gross margins (e.g., 40%) compared to traditional software (70-90%).
71:12SaaS companies generate cash earlier with AI due to fewer human employees, despite lower gross margins.
72:13Refusal to accept sub-35% gross margins for AI agents guarantees failure for SaaS companies; historical parallels with Adobe and Microsoft's cloud transitions.
73:14A playbook for SaaS companies: show AI revenues/low gross margins, leverage cash-generative business to outcompete venture-funded AI natives.
74:18Salesforce, ServiceNow, HubSpot, GitLab, Atlassian are examples of companies that could run successful AI agent strategies by automating core functions.
75:19This is a 'life-or-death decision' for SaaS companies, with Microsoft as the rare exception of success.
75:19Other 'off-the-wall' market observations: rolling bubbles in nuclear/quantum (not good public investment expressions).
76:20Quantum supremacy is misunderstood; it enables specific calculations, not universal superiority over classical computers.
77:20Fascination with how AI consistently gets what it needs to grow (e.g., sudden public acceptance of nuclear, data centers in space).
78:21Kevin Kelly's 'technium' concept: technology's inherent drive to grow more powerful, supplied by humans.
79:21Gavin's passion for investing: the search for hidden truths, applied history, current events, and the competitive game of skill and chance.
80:21Early life interest in history and current events, and his parents' encouragement of argumentative skill.
81:22His original plan to be a ski bum, river guide, wildlife photographer, and novelist; a pivotal internship at Donaldson Lufkin & Jenrette.
82:23The impact of working as a housekeeper at Alta on his worldview and how he treats people.
83:23His parents' support for his unconventional plans, but request for one professional internship leading to his discovery of investing.
84:24Conceptualizing investing as a game of skill and chance, like poker, where thorough knowledge of history and current events provides an edge.
85:25Rapid immersion into investing literature (Peter Lynch, Warren Buffett, Market Wizards), self-teaching accounting, and changing college majors.
86:25His dedication to learning about stocks, reading the Motley Fool, and the appeal of finally being competitive at something.
💬 Notable Quotes
“"The free tier is like you're dealing with a 10-year-old and you're making conclusions about the 10-year-old's capabilities as an adult. And you could just pay and I do think actually you do need to pay for the highest tier whether it's Gemini Ultra, you know, um, Super Grock, whatever it is, you have to pay the $200 per per month ti whereas those are like a fullyfledged 30 35y old." – Gavin Baker [02:01]”
“"With software, anything you can specify, you can automate. With AI, anything you can verify, you can automate. It's such an important concept and I think an important distinction." – Gavin Baker [09:07]”
“"AI is the first time in my career as a tech investor that being the lowcost producer has ever mattered. Apple is not worth trillions because they're the lowcost producer of phones... It's never mattered." – Gavin Baker [11:10]”
“"This is a life ordeath decision. And essentially everyone except Microsoft is failing it. To quote that memo from that um Noia guy long ago, like their their platforms are burning." – Gavin Baker [75:19]”
More from this guest
Gavin Baker
Listen to Full Episode
📬 Get weekly summaries like this one
No spam. Unsubscribe anytime. By subscribing you agree to our Privacy Policy.
Continue Exploring
