Dan Mac Weekly Podcast Roundup: November 19 - 26
The Age of Research Returns: This Week’s Most Mind-Bending AI Conversations
Something big happened this week. Ilya Sutskever—the guy who helped create the deep learning revolution—went on Dwarkesh Patel’s podcast and essentially said: “The age of scaling is over.”
Let that sink in.
The man who pioneered the idea that you could just throw more compute at neural networks and watch them get smarter is now telling us we’ve hit a wall. And he’s not alone. Across a dozen conversations this week, I heard a consistent thread: the easy gains are done. What comes next requires actual ideas.
Here’s what caught my attention.
Ilya Sutskever: We’re Back to the Age of Research
Ilya dropped maybe the most important framing I’ve heard all year. He breaks AI history into three eras:
2012-2020: The Age of Research
2020-2025: The Age of Scaling
2025-onwards: Back to Research
His argument is elegant. When everyone discovered “scaling works,” it became this single powerful word that told labs exactly what to do: get more data, get more compute, repeat. Low risk. Predictable returns. But now? Data is running out. Compute is astronomically expensive. And a 100x increase from here won’t fundamentally transform capabilities the way the last 100x did.
The analogy that stuck with me: imagine two competitive programming students. One practices 10,000 hours, memorizes every technique, and becomes technically flawless. The other practices 100 hours but has “the it factor.” Which one has the better career?
Current AI models are like that first student—incredibly competent at things they’ve seen, but brittle at genuine generalization. They lack what Ilya calls the “it factor.”
His solution at SSI? Stop chasing scale. Start chasing understanding. He wants AI that generalizes like humans—not through brute force memorization but through something more fundamental. The teenager learning to drive doesn’t need 10,000 hours. They need 10 hours and an internal sense of “how am I doing?” That self-correcting value function is what we’re missing.
Most provocative take: maybe we shouldn’t build AI aligned to human values specifically. Maybe we should align it to care about sentient life—because the AI itself will be sentient, making that alignment more natural than trying to subordinate its interests to ours alone.
Llion Jones: “I Invented the Transformer. Now I’m Replacing It.”
When one of the eight authors of “Attention Is All You Need” tells you Transformers aren’t the final architecture, you pay attention.
Llion Jones is at Sakana AI now, working on what comes next. His observation is sharp: before Transformers, everyone was obsessing over tiny modifications to RNNs. Different gate placements, layered structures, identity initialization tricks. All that research became instantly obsolete when attention arrived.
We’re in the same situation now. Endless papers about where to put the normalization layer. Slightly different training schedules. And Llion believes a breakthrough will make all of this seem like wasted time—just like RNN research looks in retrospect.
The phrase he’s drawn to is “jagged intelligence.” Current models can solve PhD-level problems in one sentence and make obviously wrong claims in the next. This isn’t a data problem. It’s an architecture problem. The models are too powerful—they can be forced to approximate anything with enough compute and data, but they don’t want to represent information the way humans do.
His example is perfect: given a spiral classification problem, standard neural networks solve it with ugly piecewise linear boundaries. But a different architecture in a research paper actually learned to represent the spiral as a spiral. If the data is a spiral, shouldn’t the model see a spiral?
The Continuous Thought Machine work coming out of Sakana rethinks neurons entirely. Instead of ReLU on/off switches, each neuron is a small model itself with internal dynamics. And instead of reading the state at any moment, they measure how neurons synchronize over time. A thought isn’t a snapshot—it’s a pattern that unfolds.
Mike Knoop: Two Breakthroughs in Ten Years
TBPN - Gemini 3 Launch Episode
Mike Knoop from Zapier offered the cleanest framework for understanding where we are. In the last decade, we’ve had exactly two major conceptual breakthroughs:
The Transformer (2017)
Chain of Thought / Reasoning (2022 → scaled into o1/o3)
Everything else has been compute scaling—necessary but not sufficient. The breakthroughs were the sufficient conditions.
His thesis: AI reasoning systems with no new innovation from here can enable mass automation. Any problem where you can generate lots of training examples and verify feedback can be solved. Full stop.
But mass innovation? That’s still an AGI-complete problem. Current systems lack fluid intelligence—the ability to adapt reasoning to genuinely novel situations. They’re excellent within their training distribution and fall apart outside it.
What’s working now is the merger of deep learning with symbolic program synthesis. The IMO Gold results, ICPC wins, Gemini 3’s visual understanding—all of these use language models with symbolic recomposition systems layered on top. That intersection is underexplored, and there’s gold to mine.
For builders, the takeaway is blunt: update your mental model. What’s possible in the last 12 months wasn’t possible before. Stop thinking about LLMs based on your 2023 experience.
TBPN’s John Coogan & Jordi Hays: X as the Internet’s Dive Bar
This one’s different—not about AI directly, but about how information spreads through tech. John and Jordi run TBPN, the show that’s essentially productized the group chat.
The insight I loved: X (Twitter) is tech’s Bloomberg terminal. The 200,000 most important people in private markets are there. Every tier-one fund is glued to it, even if they claim otherwise. (John’s test: tweet someone’s name without tagging them—if they’re “not really on Twitter,” why do they text you within five seconds?)
Their growth hack was the “super quote tweet”—a 4K video of two guys in suits holding a printed version of your tweet. In a sea of likes and anonymous engagement, it became the highest honor a poster could receive.
The broader point: advertising enables price discrimination across audiences that range from billionaires to college students. A great sponsor can extract $1 billion from a massive company founder and $50 from a college student, and both get value. That’s the power of reaching the right distribution.
Dwarkesh Patel’s advice for tweeting well: “Tweet like you’re lobbing texts in a group chat.” Don’t open a doc and craft something—just fire off what you’d send to friends.
All-In Pod: AI Output Tokens Have Different Values
All-In - Epstein Files Fallout Episode
Quick hit from the All-In crew. Jason Calacanis made an underrated observation: not all output tokens are equal. Their value depends on the revenue they generate.
His wife hit the end of the internet this week—voice mode on Grok said “you’re out of tokens” after 50 minutes of conversation. Why? Because XAI is tracking the revenue potential of those outputs. They’re not going to generate negative-revenue tokens just for the sake of helpfulness.
The decoder infrastructure at every major lab has been rebuilt to understand value per token. There are manipulations happening before and after the user interaction—all invisible, all designed to optimize the economics.
Also from this episode: David Friedberg’s path back to CEO. He spent years as a chairman, funding other CEOs, watching them ignore his advice. The frustration built until Ohalo—a research project with $40M invested—started producing game-changing results. Then he watched Oppenheimer in IMAX, cried, and realized he wasn’t doing what he should be doing with his life. Being a board member who couldn’t drive outcomes was “useless.” Now he’s all-in.
TBPN: Tech’s Water Cooler
TBPN - NVIDIA Beats Earnings Episode
The NVIDIA earnings call was treated like a 1995 internet moment—and for good reason. But the insight I found most interesting was about NVIDIA’s strategic investments in OpenAI and Anthropic.
One guest framed those massive checks as R&D OpEx, not investments. Being close to the frontier labs tells NVIDIA where technology is heading. They get to see inside the data centers. They learn what bottlenecks matter. That intelligence is worth billions, regardless of financial returns.
Also this week: David Chang on the state of food in America. Culinary knowledge among younger generations is higher than ever. You can find great restaurants in Oklahoma City and tertiary cities that would’ve been food deserts a decade ago. The excellence has broadened and flattened across the country. Maybe not as many titans, but the floor has risen dramatically.
What This Means for You
The theme across all these conversations is convergence. We’re exiting the era where “throw more compute at it” was the answer to every question. The playbook that worked from 2020-2025 is exhausting itself.
What comes next requires actual ideas. New architectures. Novel training paradigms. A deeper understanding of what makes intelligence general rather than merely large.
If you’re building with AI, the opportunity isn’t to wait for GPT-6. It’s to recognize that the paradigm shift has already happened in the last 12 months with reasoning models. Update your priors. What was unreliable is now reliable. What needed human oversight can now run autonomously.
And if you’re thinking about the longer arc? The people who matter are asking different questions now. Not “how big can we scale?” but “what are we missing about generalization?” Not “how do we align AI to human values?” but “how do we align AI to something it actually wants to be aligned to?”
The easy part is over. The interesting part begins.
Subscribe for weekly podcast roundups that cut through the noise and find the signal.
Couldn't agree more. I've been skeptical about the scaling-only approach for a while, you even touched on it in a previous write-up. But do you think the industry will really pivot away from scaling entirly?