The Economic Reality of AI: Friction, Talent, and the Future of the Firm
Steve Tadelis
Hugo Bowne-Anderson
Duncan Gilchrist
Steve Tadelis, Professor of Economics at UC Berkeley and former senior economist at eBay and Amazon, joins High Signal to bridge the gap between economic theory and the high-stakes reality of data science and AI. Drawing on his experience at the forefront of the world's largest marketplaces, Steve discusses the "invisible friction" that prevents organizations from acting on data: a combination of misaligned incentives, organizational inertia, and the "Upton Sinclair problem," where leaders are effectively paid not to understand new paradigms. The conversation moves from the "frustratingly obvious" opportunities left on the floor during eBay's early years to the relentlessly scientific culture of Amazon. Steve explains why surface-level metrics like conversion rates often mask underlying rot in user retention and how rigorous experimentation, such as his famous $20 million search-ad experiment, can expose the difference between genuine growth and mere navigational intent. We also explore the structural shifts of the AI era, where Steve offers an important counter-narrative: rather than leveling the playing field, AI may act as an "unequalizer" that exponentially rewards those with the deepest critical thinking skills.
Guest
Steve Tadelis
Professor of Economics at UC Berkeley
Key Takeaways
Incentives override insights.
Organizational structures can render even the most brilliant data insights useless. Tadelis observes that when leaders are "paid not to understand something," they won't. If a company values consensus over single-thread ownership (as eBay did during its "kumbaya" era), clear opportunities for value — like fixing a 13-click checkout process — will be ignored for years in favor of cultural harmony.
Conversion metrics often mask rot.
High-frequency metrics like "conversion" or "reputation scores" create a false sense of security. At eBay, 99% of sellers had positive ratings, but Tadelis's cohort analysis revealed that nearly two-thirds of new users never returned after two purchases. Data leaders must look past surface-level success to find "hidden" friction, such as using NLP on customer-to-seller messages to find complaints that never made it into formal feedback.
Performance marketing frequently measures intent, not incrementality.
Large-scale organizations often waste millions of dollars on ads that capture users who were already coming to the site. By experimenting with eBay's $20 million spend on the keyword "eBay," Tadelis proved that "high ROI" metrics were merely capturing navigational intent. Without rigorous experimental verification of incrementality, marketing models will continue to credit ads for sales that would have happened anyway.
AI is a skill-biased complement, not an equalizer.
While many assume AI will level the playing field, Tadelis argues it will do the opposite. Because AI requires deep critical thinking to refine prompts and evaluate "hallucinated" or "slop" output, it acts as a complement to high-skill workers rather than a substitute for them. This creates a widening gap where elite talent becomes exponentially more productive, while low-skill output remains mediocre.
Synthetic experiments bypass short-term revenue blockers.
Business owners often reject experiments because they fear losing quarterly revenue. Tadelis circumvented this at eBay Motors by designing a "two-part tariff" experiment: charging dealers their historical average as a flat fee while making marginal listings free. This revenue-neutral design allowed him to gather the demand-curve data needed for a $25 million pricing overhaul without triggering the business's defensive short-termism.
AI drives a "barbell" firm distribution.
The future of the economy likely favors the extremes rather than the middle. By lowering both internal coordination costs (favoring massive scale) and external transaction costs (allowing 3-person firms to outsource legal and HR via AI), the middle-market firm is at risk. Leaders should prepare for a world where incumbents get much larger and "one-person billion-dollar companies" become viable.
Trust is the precursor to experimental runway.
Technical brilliance is insufficient for influence; trust is the actual currency. Tadelis notes that building trust — often by ensuring stakeholders don't "look like idiots" when historical mistakes are uncovered — is the only way to get permission to "play in the sandbox." Data scientists only move from cost centers to value drivers once they have the emotional intelligence to navigate organizational psychology.
AI automates engineering, not goal-setting.
Technology is rapidly collapsing the time required for "how" (engineering and coding) but has not touched "what" (strategy and priorities). The human bottleneck has shifted from technical execution to the ability to set the right goals and exercise critical discernment. In an AI-augmented world, the most valuable leaders are those who can navigate the mystery of human values and community, not just optimize for "bigger, better, faster."