Attention Heads

Attention Heads

3 Necessities for Full AI Adoption ✅

Hurdles to leap over to maximize AI adoption the global economy

Dan McAteer's avatar
Dan McAteer
Jun 04, 2025
∙ Paid

1. Infinite Context

Context is the amount of attention that an AI model can give to a collection of thoughts at a single time. Everything that the model “knows” about and can apply its knowledge to needs to fit into the context.

Imagine having a conversation with someone and then can only remember the last few sentences of what you said. AI models with limited context are like that.

Google has the lead on this front and has had it for quite a while. The most capable Gemini models have a very strong 1 million token context window. With GPT-4.1, OpenAI now offers a 1 million token context model but GPT-4.1 isn’t as overall capable as the strongest Gemini models and it’s specialized for coding.

User's avatar

Continue reading this post for free, courtesy of Dan McAteer.

Or purchase a paid subscription.
© 2026 Daniel McAteer · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture