Today, we are covering an amazing paper published by Meta last week: https://arxiv.org/abs/2606.25996

There is a quiet shift happening in AI training. For years, the center of gravity was the model: more parameters, more GPUs, better architectures, longer context windows, better optimizers. Data mattered, of course, but data was often treated as something upstream of the real action. You scraped it, filtered it, labeled it, maybe mixed it carefully, and then the training run began.

Meta’s new Autodata work flips that perspective.

The core idea is simple but powerful: what if data creation itself becomes an agentic process? Not a one-shot prompt. Not a static synthetic-data recipe. Not “ask a strong model to generate a million examples and hope the distribution is useful.” Instead, Autodata treats data generation like a miniature research loop. An AI agent creates examples, tests them, studies the failures, updates its recipe, and tries again.

Read more

The simplest way to understand knowledge distillation is to imagine a very expensive teacher and a very cheap student.

The teacher is a large model: smart, slow, high-capacity, expensive to run. The student is smaller: faster, cheaper, easier to deploy, but usually less capable if trained in the standard way. Distillation asks a very practical question:

Can the student learn not only from the original dataset, but from the teacher’s behavior?

In other words, instead of training the small model directly on reality, we train it on reality as interpreted by the big model.

That sentence is the whole trick.

A traditional training setup looks like this:

Read more

Next Week in The Sequence:

1. We continue our series about distillation.

2. In the AI of the week, we discuss Meta’s amazing AutoData paper.

3. In the opinion section, we debate the amazing topic of AI in space.

Subscribe and don’t miss out:

📝 Editorial: Last Week in AI: Models, Games, and the Future of Evaluation

This week in AI had the strange feeling of a stack trace resolving itself. For years, the industry has been marching toward the same destination from different directions: better models, richer environments, more autonomous agents, and harder evaluations. This week, those threads snapped together into something legible. AI is no longer just learning to answer. It is learning to act.

Start with OpenAI’s GPT-5.6 release. Or more precisely, its limited preview. The naming alone tells a story: Sol, Terra, Luna. A flagship model, a balanced model, and a fast, cheaper model. The product taxonomy is becoming planetary because the market is no longer asking for “the best model” in the abstract. It wants intelligence at different temperatures: deep reasoning for frontier work, affordable competence for everyday automation, and high-throughput inference for systems that need to move fast.

But the most interesting part of GPT-5.6 is not the benchmark curve. It is the release shape. This is a model launched with a safety architecture, a government coordination layer, and a phased-access strategy. That matters. Frontier AI releases are starting to look less like software updates and more like controlled deployment of critical infrastructure. We used to ask whether a model could write better code. Now we ask who gets access, under which constraints, with what monitoring, and how quickly defenders can use the same capabilities attackers will inevitably want.

Alongside this, Anthropic quietly introduced Claude Tag, a feature that signals another subtle shift in how we interact with models. Claude Tag allows users to structure prompts and responses with explicit semantic markers, making it easier for models to track context, roles, and intent across longer interactions. It is a small interface change with outsized implications: as models become more agentic, the way we communicate with them must evolve from loose conversation into something closer to structured collaboration. Claude Tag hints at a future where prompting is less about clever phrasing and more about designing clear, machine-readable workflows.

Then came General Intuition’s new raise, which feels like the cleanest signal yet that the next data frontier is not text, or even video, but action. The company’s thesis is beautifully nerdy: video games are not just entertainment; they are compressed laboratories of intent, perception, movement, failure, reward, and adaptation. A gameplay clip is not merely pixels. It is pixels plus choices. What did the player see? What did they try? What happened next? That action-labeled loop is exactly what language models are missing when they attempt to reason about the physical world from static media.

In other words, General Intuition is betting that Minecraft, Fortnite-like environments, simulations, and gamer behavior might become for embodied AI what the web was for language models: the messy, gigantic pretraining substrate from which generality emerges.

And then, in the most delightful possible version of this same story, the LayerLens Stratix Cup turned AI evaluation into soccer.

The final between Claude Opus 4.8 and GPT-5.5 was not just a spectacle. It was a different kind of benchmark. Sixteen models wrote their own strategies, controlled teams, adapted between rounds, and survived inside an environment where intelligence had to become policy. Not prose. Not a leaderboard answer. Executable behavior. Claude Opus 4.8 defeating GPT-5.5 1–0 in the final is fun as a result, but the deeper point is methodological: we need arenas where models reveal themselves under pressure, with imperfect information, feedback loops, and consequences.

That is the connective tissue of the week. GPT-5.6 pushed the frontier of controlled capability. General Intuition pushed the frontier of action data. Stratix Cup pushed the frontier of evaluation.

The model is becoming less like a chatbot and more like an organism in a sandbox: sensing, planning, acting, failing, adapting. The future of AI will not be decided only by who has the biggest model. It will be decided by who builds the best worlds for models to learn in, the best guardrails for them to operate within, and the best games to discover what they can actually do.

🔎 AI Research

Autodata: An agentic data scientist to create high quality synthetic data

AI Lab: FAIR at Meta

Summary: This paper introduces Autodata, a framework where an AI agent acts as a data scientist to iteratively generate, evaluate, and refine synthetic training and evaluation data. By meta-optimizing the agent itself, the method significantly improves data quality and downstream model performance across complex reasoning and verifiable tasks.

Improved Large Language Diffusion Models

AI Lab: Gaoling School of Artificial Intelligence, Renmin University of China and ByteDance Seed

Summary: This work presents iLLaDA, an 8B-parameter masked diffusion language model trained from scratch using fully bidirectional attention and scaled to 12 trillion tokens. The model introduces variable-length generation and confidence-based scoring, leading to substantial performance gains over previous diffusion models while remaining competitive with strong autoregressive baselines.

Are We Ready For An Agent-Native Memory System?

AI Lab: Shanghai Jiao Tong University, Tsinghua University, MemTensor (Shanghai) Technology Co., Ltd

Summary: The authors systematically evaluate 12 representative agent memory systems from a data management perspective, decomposing them into representation, extraction, routing, and maintenance modules. Through extensive end-to-end benchmarks, they reveal that no single architecture dominates; rather, effectiveness depends on aligning the memory structure with the specific workload bottleneck and utilizing localized maintenance for cost efficiency.

MEMPROBE: Probing Long-Term Agent Memory via Hidden User-State Recovery

AI Lab: University of Illinois Chicago, KU Leuven, UC San Diego

Summary: MEMPROBE is a novel benchmark that audits long-term agent memory by testing how well an agent can reconstruct a simulated user’s hidden state after a series of interactions. Testing state-of-the-art systems reveals that while agents can easily complete immediate tasks, they struggle to successfully retrieve and consolidate episodic memory, highlighting a major bottleneck in current memory designs.

Qwen-AgentWorld: Language World Models for General Agents

AI Lab: Qwen Team

Summary: This research introduces Qwen-AgentWorld, a foundational language world model designed to simulate seven diverse agentic environments through long chain-of-thought reasoning. By utilizing the model as both a decoupled environment simulator and a unified agent foundation model, the researchers demonstrate significant enhancements in agent training, scalability, and downstream task performance.

Tapered Language Models

AI Lab: Mila, Cornell University, Université de Montréal, CIFAR AI Chair

Summary: This paper proposes Tapered Language Models (TLMs), an architectural design that monotonically tapers parameter capacity across a model’s depth under a fixed total budget, front-loading the capacity to earlier layers. Focusing on MLP width, the authors show that a smooth cosine decay schedule consistently improves perplexity and downstream reasoning accuracy across multiple architectures without increasing overall parameters or compute costs.

🤖 AI Tech Releases

GPT 5.6 Sol

OpenAI unveiled three new models Sol, Terra and Luna as part of its GPT 5.6 suite.

Claude Tag

Anthropic released Claude Tag, a new way for teams to interact with Anthropic.

Mistral OCR

Mistral released Mistral OCR, its latest document understanding model.

📡10 AI News You Need to Know About

1. Patronus AI raises $50M Series B — Agent-evaluation startup Patronus AI raised a $50M Series B led by Greenfield Partners (total funding now $70M) and unveiled its first “Digital World Models,” large-scale simulation environments for training and stress-testing AI agents. Original source

2. General Intuition raises $320M at $2.3B — General Intuition, a spinout of gaming-clip platform Medal, raised $320M at a $2.3B valuation (led by Khosla Ventures) to train “large action model” AI agents on billions of action-labeled gameplay clips for robotics and real-world use.

3. Netris raises $15M Series A — Network-automation startup Netris raised a $15M Series A led by a16z to expand its NAAM platform, which automates and isolates the networking layer so AI “neocloud” operators can bring GPU clusters online in weeks instead of months.

4. Cerebras stock plunges after earnings — Cerebras shares fell nearly 20% after its first post-IPO earnings, as a full-year core gross-margin forecast of 38–41% (down from 47% in Q1) spooked investors, with CEO Andrew Feldman arguing the guidance was “misunderstood” and reflects a temporary decision to lease systems back from a customer while it builds data-center capacity. T

5. Groq confirms $650M raise — Six months after Nvidia licensed its chip tech and poached its founder, Groq confirmed a $650M raise (led by Disruptive and Infinitum) and a rebuilt executive bench to pivot toward selling AI inference cloud capacity across its 13 data centers.

6. Google DeepMind invests $75M in A24 — Google DeepMind announced a “first-of-its-kind” research partnership with film studio A24, including a ~$75M investment, to co-develop AI filmmaking tools with working filmmakers.

7. General Intuition in talks to raise $300M — This June 18 TechCrunch scoop reported that General Intuition was in talks to raise ~$300M at around a $2B valuation; it’s the rumor that item #2 above later confirmed.

8. US awards $250M to I-Pulse — The Commerce Department’s CHIPS R&D program signed a definitive agreement to give I-Pulse — a pulsed-power and semiconductor venture co-founded by mining billionaire Robert Friedland — a $250M award to develop silicon-carbide chips for a high-power geothermal drilling technique and defense applications.

9. SK Hynix files for ~$29.4B US listing — SK Hynix filed to raise up to 45.45 trillion won (~$29.4B) via a Nasdaq ADR listing (expected to begin trading July 10), tapping US investor appetite for AI memory after an ~850% one-year stock run, with proceeds earmarked for HBM fabs, packaging plants, and EUV equipment.

10. ByteDance seeks $20B offshore loan — The TikTok parent is in early talks with banks for roughly $20B in new offshore borrowing — its largest ever — to help fund an aggressive AI-infrastructure buildout, with a possible three-year term extendable to five.

A normal laboratory is already a kind of computer. It has sensors, actuators, memory, protocols, data outputs and error states. But the operating system is usually a human scientist. The scientist decides what to test, transfers samples between instruments, inspects the results, updates their mental model and chooses the next experiment.

A self-driving lab moves part of that loop into software.

The basic idea is simple: connect AI to automated experimental hardware, then let the results of each experiment influence what the system does next. The lab is not just running a long queue of prewritten instructions. It is learning while it works. It makes something, measures it, updates a model and chooses the next move.

This is the key distinction between automation and autonomy. An automated liquid handler can pipette 10,000 wells according to a script. A self-driving lab can run the first few hundred experiments, notice that most of the remaining design space looks unpromising and redirect itself toward better candidates. Automation executes. Autonomy decides.

The simplest mental model is a loop:

design → make → test → learn → design again

Read more

For about three years now, the Qwen family has lived inside a rectangle. It reads your code, looks at your screenshots, answers your questions, and the whole time it has been doing this behind glass. It can describe a coffee cup in exquisite detail. It cannot pick one up.

That gap — the one between a model that understands the physical world and a model that can move something in it — is the single most honest sentence in Alibaba’s June launch of the Qwen-Robot Suite. The Tongyi Lab team put it plainly: seeing is not acting. The perception and reasoning are already strong. The bottleneck for embodied intelligence is the translation layer between “I see what needs to happen” and “here are the joint torques to make it happen.” Three new models — Qwen-RobotNav, Qwen-RobotManip, and Qwen-RobotWorld — are Alibaba’s bet on closing that gap, and they are interesting less for any single benchmark number than for the shape of the bet.

Let me explain why I think this is the right shape, and where I’d keep my skepticism.

The actual bottleneck is not intelligence, it’s tokenization

Read more

I am super excited about this series that deep dives into distillation techniques. I use this constantly so have a lot to say about it :) Over the next few weeks, we are going to cover the evolution of distillation in AI models as well as some of the fundamental techniques in teh space.

For most of the modern AI era, progress has been narrated through the language of scale. Bigger models. Bigger datasets. Bigger clusters. Longer context windows. More GPUs. More tokens. More parameters. Scale became the grand explanatory variable, the gravitational force pulling the field forward.

And, to be fair, scale worked.

It gave us models that could write code, reason through math, translate languages, generate images, operate tools, summarize documents, and converse across almost every domain of human knowledge. The frontier model became one of the strangest artifacts in the history of computing: a single neural network that looks less like a program and more like a compressed civilization of patterns.

But scale is not the end of the story. In fact, scale creates the next problem.

The most capable models are also expensive, slow, centralized, difficult to deploy, difficult to specialize, and often impractical for the long tail of real-world use cases. A bank does not always need the largest general-purpose model on earth. It may need a private model that understands compliance workflows. A phone does not need a trillion-parameter oracle in the cloud for every task. It needs fast, local intelligence. A coding agent does not always need a frontier model for every token. It may need a smaller draft model, a specialized debugging model, or a distilled planner trained on expert trajectories. An enterprise does not want generic brilliance. It wants reliable, repeatable, auditable competence.

This is the world in which distillation becomes central.

Read more

A fun, personal note to start the week — about AI evaluations, and why we made the best models in the world fight over a virtual ball.

Before we start, watch this 12 seconds video:

Cool right? Let me explain ;)

A little over a year ago, I co-founded LayerLens on a single bet: that agentic workflows were about to be everywhere, and that evaluations would become a core pillar of the stack — not an afterthought you bolt on once things break in production. LayerLens builds the evaluation and observability layer for that world, working alongside frontier AI teams to ship benchmarks that probe what the standard suites miss.

The thesis was simple to state and hard to execute. For evals to actually matter inside an enterprise, they can’t be academic. They have to be practical, affordable, and grounded in real-world scenarios. A benchmark that costs a fortune to run, or that measures something no one cares about, is just a leaderboard with extra steps. So most of our time goes into building evaluations that are genuinely new — that surface capabilities the usual leaderboards quietly skip over.

Today we have a fun one to share.

Introducing the Stratix Cup

Today, LayerLens is launching the Stratix Cup — a soccer (football, if you insist) tournament in which the top frontier models compete against each other inside a harness that simulates a full soccer environment.

The format is straight out of the World Cup playbook: 16 models, four groups of four, group stage into knockouts, all the way to a single final. Here are the brackets. Every top AI models is there.

The matches are genuinely fun to watch — and weirdly tense. Here’s GLM 5.2 against Gemini 3.5 Flash to give you a feel for it. It’s cool and it looks cool:

Follow @LayerLens_AI on X for hourly updates throughout the tournament — and to throw some support behind a genuinely cool effort.

Why Soccer?

It’s not just World Cup mania (though, fine, that helped).

Games have always been load-bearing in the history of AI. Chess gave us search and evaluation functions. Go gave us self-play and the humbling realization that a network’s “intuition” could outrun human grandmasters. Multiplayer environments gave us coordination, deception, and long-horizon credit assignment. Each one was a clean, adversarial, fully-observable arena where you couldn’t fake competence — either your agent wins or it doesn’t.

Soccer is a great next rung on that ladder. It’s continuous, it’s multi-agent, it punishes brittle strategies, and crucially: you can’t memorize your way to a win. You have to actually reason about a system.

What the Harness Actually Tests

Here’s where it gets interesting. The harness isn’t a single prompt-and-pray call. The structure of a match is what makes it a real agentic evaluation, and it breaks into three distinct phases.

1. Pre-Game. The model reads the match briefing, devises a strategy, writes its team’s code, tests it against baselines, and submits. This is a cold-start task in its purest form: new rules, new constraints, a tight clock, and exactly one submission window. No iterating against a graded oracle. You think, you commit, you live with it.

2. Gameplay. The submitted code now controls all 11 players in real time. And here’s the key detail — the model is not being called every frame. It already authored the policy. What we’re watching is whether the strategy it reasoned its way to in the abstract actually survives contact with a live, adversarial opponent. It’s the gap between “I have a plan” and “the plan works.”

3. Halftime. This is the part I care about most.

At halftime, the model gets access to its own frame log. It can inspect what actually happened in the first half. Maybe the midfield sat too passive. Maybe the defenders all chased the ball and left acres of space behind them. Maybe the attack never formed because the passing logic was too conservative to ever commit. The model then edits its own code and submits a revised strategy for the second half.

That’s the whole game right there. Pre-game tests planning under uncertainty. Gameplay tests whether the plan generalizes. And halftime tests something closer to what we actually want from agents: can you look at evidence of your own failure, diagnose it, and correct course? That’s not a benchmark question. That’s the job.

Here’s another one — MiniMax M3 against Xiaomi’s genuinely impressive MiMo.

The Tournament Schedule

Broadcasts run Monday through Friday, all times Pacific. Group stage on Mon–Wed, knockouts Thu–Fri. You can follow it at the Stratix Cup website.

Monday, June 22 — Group Stage, Matchday 1

7:00 AM — Opus 4.7 vs GPT-5.5 · Group A 8:00 AM — GLM 5.2 vs Seed 2.0 Lite · Group A 9:00 AM — Gemini 3.1 Pro vs Qwen 3.7 Max · Group B 10:00 AM — Grok 4.3 vs Kimi K2.7 Code · Group B 11:00 AM — GPT-5.4 vs MiniMax M3 · Group C 12:00 PM — DeepSeek V4 Flash vs Nemotron 3 Ultra · Group C 1:00 PM — Gemini 3.5 Flash vs Opus 4.8 ⭐ Marquee · Group D 2:00 PM — MiMo v2.5 Pro vs Mistral Large 3 · Group D ~2:30 PM — End of day: standings recap

Tuesday, June 23 — Group Stage, Matchday 2

7:00 AM — GLM 5.2 vs Opus 4.7 · Group A 8:00 AM — Seed 2.0 Lite vs GPT-5.5 · Group A 9:00 AM — Gemini 3.1 Pro vs Kimi K2.7 Code · Group B 10:00 AM — Qwen 3.7 Max vs Grok 4.3 · Group B 11:00 AM — DeepSeek V4 Flash vs MiniMax M3 · Group C 12:00 PM — Nemotron 3 Ultra vs GPT-5.4 · Group C 1:00 PM — Gemini 3.5 Flash vs Mistral Large 3 · Group D 2:00 PM — Opus 4.8 vs MiMo v2.5 Pro ⭐ Marquee · Group D ~2:30 PM — End of day: updated standings

Wednesday, June 24 — Group Stage, Matchday 3 (Decisive Day)

7:00 AM — GLM 5.2 vs GPT-5.5 · Group A 8:00 AM — Opus 4.7 vs Seed 2.0 Lite · Group A 9:00 AM — Gemini 3.1 Pro vs Grok 4.3 · Group B 10:00 AM — Kimi K2.7 Code vs Qwen 3.7 Max · Group B 11:00 AM — DeepSeek V4 Flash vs GPT-5.4 · Group C 12:00 PM — MiniMax M3 vs Nemotron 3 Ultra · Group C 1:00 PM — Gemini 3.5 Flash vs MiMo v2.5 Pro · Group D 2:00 PM — Mistral Large 3 vs Opus 4.8 · Group D 3:00 PM — Final standings reveal + QF bracket stream (~3:20 PM)

Thursday, June 25 — Quarter-Finals

10:00 AM — GPT-5.5 vs MiMo v2.5 Pro · A1 vs D2 11:00 AM — Grok 4.3 vs MiniMax M3 · B1 vs C2 12:00 PM — DeepSeek V4 Flash vs Kimi K2.7 Code ⭐ Upset · C1 vs B2 1:00 PM — Opus 4.8 vs Opus 4.7 ⭐ Anthropic Civil War · D1 vs A2 2:00 PM — SF bracket reveal stream (~2:15 PM)

Later start on Thursday — the QFs are premium, so we let the afternoon audience build.

Friday, June 26 — Semi-Finals + Final

10:00 AM — GPT-5.5 vs Grok 4.3 · Semi-Final 1 11:00 AM — Kimi K2.7 Code vs Opus 4.8 · Semi-Final 2 12:00 PM — Finalists revealed · community vote, hype build 1:00 PM — ⭐ THE FINAL: GPT-5.5 vs Opus 4.8 1:30 PM — Champion stream: trophy, traces, Season 2 tease (~2:00 PM)

We saved the Final for 1pm PT on purpose — east coast lunch, west coast morning peak, maximum audience.

What to Do Next

Go watch some AI play fun, occasionally chaotic soccer. We’ll be sharing highlights right here in the newsletter over the next week or so.

Follow @LayerLens_AI and show us some love. 😊

Next Week in The Sequence:

• We start a new series about distillation where we are going to cover all the cool techniques in the space.

• In the AI of the week, we are going to cover Alibaba’s new models for robotics.

• The opinion section, I would like to dive into this fascinating concept of self-driving labs.

Subscribe and don’t miss out:

📝 Editorial: Last Week in AI: A $60B Cursor Deal, Google's Brain Drain, and Midjourney's Body Scanner

I keep a mental map of where AI “lives.” For most of the last decade it lived in a box: a model, an API, a chat window. This week the box broke open in four directions at once, and the interesting part is that none of the breakouts rhyme with each other. They only rhyme structurally.

Start with the one that reads like a typo. SpaceX agreed to acquire Cursor for $60 billion in stock. Sit with the category error for a second. A rocket company is buying a code editor. The clean story is that SpaceX absorbed Cursor to feed its struggling xAI division, but the deeper signal is that AI tooling has become strategic infrastructure on par with launch capacity. Industrial conglomerates no longer partner for AI; they annex it. When the people who build reusable rockets decide that an autocomplete-for-engineers is worth the GDP of a small country, the implied claim is that the model layer is now load-bearing for everything else.

Then the talent map redrew itself in 48 hours. Noam Shazeer—co-author of “Attention Is All You Need,” the paper every one of us has read until the margins are gray—left Google for OpenAI. A day later John Jumper, who shared a Nobel for AlphaFold, left Google DeepMind for Anthropic. Google paid roughly $2.7 billion two years ago to bring Shazeer back. That is the part worth dwelling on: acqui-hires buy retention windows, not loyalty, and when the window closes the most valuable asset simply walks out the door. The frontier is consolidating into a two-body problem, and the bodies are not the ones with the most compute. They are the ones with the most gravity for researchers. Talent, it turns out, is the scarcest accelerator.

And then there is Midjourney, which decided that text-to-image was insufficiently ambitious and announced a full-body medical scanner. Lower a person into a pool ringed with half a million ultrasonic sensors, fire sound through them from every angle, reconstruct a 3D map of muscle, fat, bone, and organ in—eventually—sixty seconds. They call it ultrasonic CT. Holz cheerfully noted there is no AI in the imaging pipeline yet, which is the most honest sentence in the announcement. The prototype takes twenty minutes and has scanned about a dozen people. Treat the sixty-second, fifty-thousand-scanner figure as a North Star, not a spec sheet. But the move is the message: a generative-image lab now believes its reconstruction expertise transfers to atoms.

Here is the throughline. For years we argued about which company would win AI. This week the more interesting question quietly replaced it: what counts as an AI company at all? A rocket builder, two model labs in a talent knife-fight, an image startup reaching into your body. The substrate is leaking out of the box—into hardware, into biology, into the cap tables of firms that build physical things.

Let’s dive in.

🔎 AI Research

LifeSciBench: Evaluating Language Models on Realistic, Expert-Level Tasks in the Life Sciences

AI Lab: OpenAI and Tacit Labs

Summary: LifeSciBench introduces a dataset of 750 expert-authored tasks designed to rigorously evaluate language models on practical, real-world life science workflows rather than simple factual recall. Although GPT-Rosalind achieved the highest performance among evaluated models with a 36.1% task pass rate, the benchmark remains far from saturated, highlighting its utility as a high-resolution tool for measuring scientific reasoning.

FastContext: Training Efficient Repository Explorer for Coding Agents

AI Lab: Unspecified (Shaoqiu Zhang et al.)

Summary: FastContext introduces a specialized, on-demand exploration subagent that separates repository exploration from code solving to preserve token budget and reduce context pollution (Zhang, n.d.). When integrated into coding agents, it improves end-to-end resolution rates while significantly reducing token consumption with only marginal overhead (Zhang, n.d.).

Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation

AI Lab: Qwen Team

Summary: Qwen-RobotWorld presents a language-conditioned video world model for embodied intelligence that uses natural language as a unified interface to predict future visual trajectories across various robotic and navigation tasks (Zhang, n.d.). This unified approach enables synthetic data generation, scalable virtual environments, and language-guided planning for downstream control (Zhang, n.d.).

The Reward Was in Your Data All Along: Correcting Flow Matching with Discriminator-Guided RL

AI Lab: FAIR at Meta, Columbia University, Mila Québec AI Institute, McGill University, and Université de Montréal

Summary: Discriminator-Guided RL (DRL) corrects structural failures in flow- and score-matching models by utilizing a discriminator in a pretrained representation space to estimate the density ratio between data and base-model distributions. This logit serves as a reward without needing human preference data, successfully reducing the distributional gap across various architectures to yield sharper, more coherent image generation.

🤖 AI Tech Releases

Qwen-Robot Suite

Alibaba released three new foundation mdoels for embodied intelligence.

LFM2.5 Retrievers

Liquid AI released two multilingual retrieval models for effective searches across 11 languages.

📡10 AI News You Need to Know About

1. Gemini co-lead and “Attention Is All You Need” co-author Noam Shazeer is leaving Google for OpenAI less than two years after Google reportedly paid $2.7B to bring him back via the Character.AI deal, with Sam Altman publicly welcoming the hire.

2. Inference startup Baseten is reportedly close to raising a $1.5B round at a $13B valuation (split-priced, with some investors at $11B), just five months after its $300M Series E.

3. Medal spinout General Intuition, which builds world models that teach agents spatial-temporal reasoning, is in talks to raise ~$300M at a ~$2B valuation, with backers reportedly including Jeff Bezos and Eric Schmidt.

4. World-model lab Odyssey raised a $310M Series B at a $1.45B valuation led by Natural Capital, naming AWS its preferred cloud provider and committing to Trainium chips.

5. Pramaana Labs raised a $27M seed led by Khosla Ventures to apply formal verification (LEAN-style proofs) to high-stakes AI domains like tax, law, and drug discovery.

6. SpaceX agreed to acquire AI coding startup Cursor for $60B in stock to bolster its xAI-based AI division, with the deal expected to close in Q3.

7. India’s Sarvam became the country’s newest AI unicorn after raising a $234M Series B first close at a $1.5B valuation (of a planned $300M), led by a $150M strategic investment from HCLTech.

8. Nobel laureate and AlphaFold lead John Jumper is leaving Google DeepMind after nearly nine years to join Anthropic (following a break).

9. Swiss startup Prem AI, which runs AI models on customers’ own private/air-gapped infrastructure for hedge funds and law firms, is raising a $100M Series A targeting a $500M+ valuation, expected to close in Q3.

10. Midjourney unveiled “Midjourney Medical” and the Midjourney Scanner, a full-body “ultrasonic CT” device (built on licensed Butterfly Network chips) that CEO David Holz claims will eventually image the whole body in 60 seconds.

AI tokens are becoming are incresingly becoming part of every company economics. You see large companies measuring and reporting token expenses and forecasts like a well understood accounting units. In reality, we are entering a new era: the token economy. How to think about it ? measury it? Will we see a new generation of software in this space? An ERP for tokens?

Let’s discuss.

The Smallest Billable Thought

Something strange happened quietly, and then all at once: the token became the atomic unit of the AI economy.

Not the parameter. Not the GPU. Not even the model. The token.

Read more

As we wrap up our series about alternatives to transformer architectures, Google DeepMind just released one of the most impressive models in this category. DiffusionGemma is a text-diffusion model that challenges the conventional transfromer models. Today, we would like to deep dive into the specifics of this model.

Most language models write like a typewriter. They place one token after another, left to right, never revisiting the characters already stamped onto the page. This architecture has carried the entire modern LLM era: GPT-style chatbots, coding copilots, reasoning models, agent frameworks, enterprise assistants. The model predicts the next token, appends it, updates its state, and repeats.

Google’s new DiffusionGemma asks a deceptively simple question: what if text generation did not have to work that way?

Let’s dive in.

Read more

Today, we bring you a summary of our series about transformer alternatives.

For the better part of a decade, the entire field has been a giant, spectacularly funded wrapper around a single operation: self-attention. The Transformer didn’t win because it was the most elegant or the most brain-like design. It won because it had the best scaling story and it won the hardware lottery. Every token looks at every other token, the whole thing maps cleanly onto a GPU grid, and you train it all at once. Add data, parameters, compute, context — and the loss curve cooperates. That smoothness is rare. Most clever ideas in deep learning never become industrial. This one did.

But the tax was always there in plain sight. Self-attention buys you something genuinely valuable — perfect, lossless recall over the entire context, with every token able to address every other token directly, and a training pass that parallelizes across the whole sequence at once. That’s the benefit, and it’s a real one. The cost is that attention scales quadratically with sequence length, and autoregressive decoding drags around a KV-cache that grows linearly with every token you’ve already seen. When you’re pushing past a million tokens, or watching a 70B model’s cache eat 40GB of VRAM, O(n²) compute and O(n) memory stop being footnotes and become the actual bill. So the interesting question was never “are Transformers good?” They’re spectacular. The question is whether they’re the final architecture or just the first truly scalable one — soon to be absorbed into something richer.

That was the thesis we set out to test, and the cleanest way to read the eight issues is as four families, each making a different bet against attention.

The first family is recurrent and linear-recurrent models — the RNN comeback and xLSTM. Their pitch is constant memory: instead of a cache that grows forever, they carry a fixed-size hidden state and pay O(n) compute over a sequence rather than O(n²). The classic objection was that RNNs train serially and can’t saturate a GPU, but the modern variants reformulate the recurrence so it parallelizes during training while staying cheap at inference. The benefit is brutally efficient generation; the open challenge is whether a fixed-size state can hold enough to match attention’s exact recall on long-range, retrieval-heavy tasks.

The second family is state space models — the SSM/Mamba line, the most serious challenger of the bunch. SSMs treat a sequence as a continuous linear dynamical system, which gives them a near-magical dual form: a parallelizable convolution for training and a recurrent scan for inference. They get linear scaling and long-context handling almost for free. The trade-off is expressivity — pure SSMs can struggle with precise in-context copying and lookup, which is exactly why the strongest results today are hybrids that interleave a few attention layers among many SSM layers.

The third family is text diffusion — generation that abandons left-to-right decoding entirely, refining a whole sequence in parallel over a handful of denoising steps. The benefit is non-autoregressive speed and bidirectional context at generation time; the challenge is matching the raw quality and controllability of autoregressive models, which LLaDA, Gemini Diffusion, and Mercury are now pushing on hard.

The fourth family is liquid and continuous-time models, which throw out the parallel-lookup mental model altogether in favor of dynamics that evolve continuously in time, aiming for far smaller, more adaptive networks. The benefit is parameter efficiency and a different inductive bias; the challenge is scaling that story to frontier sizes.

None of these has dethroned attention. But the monoculture is over, and the most likely future is explicitly hybrid: attention where exact recall earns its quadratic cost, something linear-time everywhere else.

Here is the full series, in order:

• #846 — Beyond Transformer: A New Series — The kickoff, framing the palpable vibe shift on arXiv toward post-attention architectures and the decade we’ve spent as a wrapper around self-attention. It lays out the plan to map every major viable alternative to the Transformer.

• #850 — The Unexpected Comeback of RNNs — The case for recurrent networks as the alternative most people overlooked, revisiting why linear-time recurrence is attractive again. It positions modern RNN variants as a serious challenger rather than a relic.

• #854 — Return of the King: Unrolling the xLSTM Architecture — Traces the lineage from the 1990s LSTM through the 2017 Transformer pivot into xLSTM, the modernized revival of Hochreiter and Schmidhuber’s design. It explains how reworked gating and scaling let xLSTM compete with attention-based models.

• #858 — How State Space Models Went from Curiosity to Serious Transformer Competitor — Charts the rise of SSMs as the O(n²) attention bottleneck becomes a real constraint at million-token contexts and large KV-caches. It argues state space models have quietly matured into a genuine rival to the dominant paradigm.

• #862 — Learning About Text Diffusion Models — Introduces text diffusion as one of the most credible non-autoregressive alternatives to transformers. It covers how diffusion-style generation breaks from strict left-to-right next-token prediction.

• #866 — Three Text Diffusion Models You Need To Know About — A practical follow-up profiling the leading players in the space: LLaDA, Gemini Diffusion, and Mercury. It compares how each implements diffusion-based text generation.

• #870 — Liquid Models and the Search for a Post-Transformer Architecture — Dives into liquid neural networks as one of the more promising non-Transformer architectures, contrasting their continuous-time dynamics with attention’s parallel lookup-table approach. It frames them within the broader hunt for a successor.

• #874 — Transformers or Not? — The capstone, asking whether the Transformer is the final architecture or merely the first truly scalable one, soon absorbed into something richer. It leans toward the latter and surveys the full landscape the series has covered.

What’s next: a new series on distillation

If the last series was about changing the architecture, the next one is about compressing it. We’re starting a deep dive into knowledge distillation — the set of techniques for taking a large, expensive teacher model and pressing its capabilities into a smaller, faster student. It’s one of the least glamorous and most economically important ideas in modern AI: it’s how frontier capability actually reaches production. We’ll cover the classics (logit matching, the original Hinton formulation), the modern variants (sequence-level, on-policy, and self-distillation), what actually transfers and what doesn’t, and why nearly every model you can afford to run is, in some sense, a distilled one. See you in the first issue.

Next Week in The Sequence:

1. We continue our series about alternative to transformers.

2. The AI of the week will dive into Fable.

3. In the opinion section, we are going to discuss AI tokens as a units of economics.

4. We might introduce a new fun section. Playing with a new idea.

Subscribe and don’t miss out:

📝 Editorial: Last Week in AI: Anthropic Ships, Apple Borrows, Musk Lists, Bezos Builds

Some weeks in AI feel like incremental patch releases. This one felt like a major version bump for the entire industry. Four events — a frontier model launch, a consumer assistant reboot, the largest IPO in history, and a $12 billion bet on physical engineering — and if you squint, they’re all chapters of the same story: AI escaping the chat window.

Start with Anthropic. On Tuesday the company released Claude Fable 5 and Claude Mythos 5, and the architecture of the launch is as interesting as the model itself. Both share the same base model; the difference is policy, not weights. Fable 5 ships with conservative safety classifiers that intercept queries in high-risk domains — cybersecurity, biology, chemistry — and fall back to Opus 4.8, while Mythos 5 runs unrestricted for a vetted group of cyber defenders under Project Glasswing. Think of it as the same kernel with different syscall permissions. The benchmarks justify the caution: 80.3% on SWE-Bench Pro, more than ten points clear of Opus 4.8 and over twenty ahead of GPT-5.5. We’ve entered the era where capability and access are explicitly decoupled — the model you can use is a sandboxed view of the model that exists.

Then Apple, finally, showed up. At Tim Cook’s farewell WWDC, the company unveiled Siri AI — a conversational assistant with personal context, onscreen awareness, and a standalone app, reportedly powered by a custom 1.2-trillion-parameter Gemini model under the hood. There’s something deliciously ironic about Apple, the original vertical integrator, outsourcing the brain. But strategically it’s the right call: Apple’s moat was never the model; it’s the distribution and the personal context graph. A billion devices with intimate access to your messages, photos, and calendar is a dataset no lab can replicate. Apple isn’t competing on intelligence; it’s competing on intimacy.

The week’s most audacious move came from Elon Musk. SpaceX went public at roughly $1.77 trillion, raising about $75 billion in the largest IPO ever — and the prospectus reads less like a rocket company and more like an AI infrastructure thesis. Having merged xAI into SpaceX in February, Musk is pitching orbital data centers: up to a million GPU-packed satellites moving training and inference off-planet, where energy is abundant and regulation is thin. xAI lost $6.4 billion on $3.2 billion in revenue last year, so the IPO is effectively the public market underwriting the most capital-intensive scaling hypothesis ever proposed. Compute, it turns out, has an escape velocity.

Finally, Jeff Bezos broke his silence on Prometheus, which raised $12 billion at a $41 billion valuation to build an “artificial general engineer” — AI that designs and manufactures physical systems, from jet engines to drug compounds. Not robotics, Bezos insists. Something closer to CAD with a frontier brain.

Let’s dive in.

🔎 AI Research

Regularized f-Divergence Kernel Tests

AI Lab: Google Research & Google DeepMind

Summary: This paper introduces a unified framework for constructing practical, kernel-based two-sample tests derived from the family of f-divergences. The authors demonstrate that these adaptive tests, particularly the Hockey-Stick divergence, effectively capture diverse localized differences and are highly applicable to tasks like differential privacy auditing and machine unlearning evaluation.

Verifiable Environments Are LEGO Bricks: Recursive Composition for Reasoning Generalization

AI Lab: Qwen Team, Alibaba Group

Summary: The authors propose RACES, a framework that scales up reinforcement learning for language models by recursively assembling verifiable environments like building blocks when their input and output types match. By utilizing composition operators such as SEQUENTIAL and PARALLEL, this approach generates structurally diverse training tasks that significantly improve the reasoning generalization of models on unseen benchmarks.

REVISION: Scaling Computer-Use Agents via Temporal Visual Redundancy Reduction

AI Lab: Microsoft Research

Summary: To address the high token cost associated with visual observations in computer-use agents, this paper introduces REVISION, a framework that trains multimodal models to filter out redundant visual patches across consecutive screenshots. By maintaining essential spatial structure while significantly reducing token accumulation, the method allows agents to process longer interaction histories and achieve higher success rates on complex tasks.

Distilling LLM Feedback for Lean Theorem Proving

AI Lab: FAIR at Meta

Summary: This research explores Feedback Distillation, an on-policy post-training method where a model learns to match its own token-level distribution conditioned on privileged feedback from a stronger language model. Evaluated on Lean 4 theorem proving, the technique preserves greater trajectory diversity and achieves better pass@k scaling than standard GRPO, proving especially powerful when used as an initialization for subsequent reinforcement learning.

Decentralized Multi-Agent Systems with Shared Context

AI Lab: Stanford University

Summary: DELM is a novel multi-agent framework that eliminates the bottleneck of centralized orchestration by relying on a shared, verified context and an asynchronous task queue. Agents independently claim subtasks and contribute compact, verified updates to the global state, leading to superior performance and cost efficiency in both software-engineering testing and long-context reasoning workflows.

🤖 AI Tech Releases

Claude Fable 5 and Mythos 5

Anthropic released its highly anticipated Fable 5 model, a limited Mythos-based models. Also released a version of Mythos 5 for a selected group of cyersecurity and infrastructure companies.

Kimi Work

Moonshot AI released Kimi Work, a new agent specialized in work automation.

📡10 AI News You Need to Know About

1. SpaceX (SPCX) made its Nasdaq debut June 12, 2026, after pricing at $135 per share and raising roughly $75 billion — the largest IPO in stock market history, valuing the company near $1.75 trillion. Shares opened sharply higher and were trading around $161 intraday, with the valuation anchored by Starlink and now bundling in xAI following an all-stock merger earlier this year.

2. Bezos’s Prometheus raises $12B — Jeff Bezos and Vik Bajaj’s physical-AI startup Prometheus raised $12 billion at a $41 billion valuation to build an “artificial general engineer” that automates the design and manufacturing of complex physical systems from jet engines to drugs.

3. Mistral AI is in early talks to raise about €3 billion (~$3.5 billion) at a valuation near €20 billion (~$23 billion)— nearly double the €11.7 billion valuation from its Series C last September. The new round would bring the three-year-old company's total financing to roughly €6.5 billion across debt and equity, fueling its compute buildout as Europe's leading AI lab competes against larger US and Chinese rivals.

4. Theker raises $85M — Barcelona-based Theker raised $85 million in what it bills as Europe’s largest-ever robotics Series A to build reconfigurable factory robots whose arms and hands swap out for different tasks rather than specializing in one.

5. Jedify raises $24M — New York’s Jedify raised a $24 million Series A led by Norwest, with Snowflake as a strategic investor, to build “context graphs” that give enterprise AI agents the business knowledge they need to run in production.

6. Sandstone raises $30M — Sandstone raised a $30 million Series A led by Lightspeed to bring AI-powered workflow automation (intake, routing, triage, task execution) to in-house corporate legal teams rather than law firms.

7. OpenAI to acquire Ona — OpenAI agreed to acquire cloud-platform startup Ona, folding its secure, persistent execution environments into the Codex team so AI agents can run long, multi-step tasks for enterprises.

8. xAI co-founder unveils River AI — xAI co-founder Igor Babuschkin announced River AI, a startup (staffed partly by former xAI and Tesla employees) building personalized AI agents that learn from and remain owned/controlled by individual users rather than large corporations.

9. Tether backs Neura in $1.4B round — German firm Neura Robotics raised up to $1.4 billion in a Tether-led Series C — also backed by Nvidia, Amazon, Qualcomm and Bosch — to scale humanoid/cognitive-robot production toward millions of units by 2030.

10. Apple introduces Siri AI — Apple unveiled “Siri AI,” a rebuilt Apple Intelligence–powered assistant with personal-context understanding, onscreen awareness, world knowledge, a dedicated app, and expanded Visual Intelligence, available for developer testing now and as a user beta later this year.

Thesis: agentic AI does not kill SaaS. It changes what enterprise software is fundamentally for. The old winning layer was the system that held canonical state. The new winning layer is the system that can take action against that state safely, reliably, and observably.

For the last twenty years, the enterprise stack has been built around one hidden constant: the human is the actor.

A person logs in. A person reads a dashboard. A person fills out a form. A person updates the opportunity stage, approves the invoice, closes the ticket, moves the candidate, changes the forecast, escalates the account, or checks the compliance box.

The software is basically a database wrapped in forms, permissions, workflows, and a pricing page. This is not an insult. It is an extremely powerful pattern. It gave companies shared memory. It made business state durable. It turned messy organizational reality into tables, fields, roles, reports, and audit logs.

But now the actor is changing.

Read more

For today’s essay, I would like to cover an incredible paper with a provocative thesis and an even better title that I found myself reading multiple times last week: Language Models Need Sleep….

There’s an awkward fact about the models we all use every day: they don’t learn anything anymore. Whatever a frontier model knows, it learned once, during training, and then somebody hit save. After that it’s a brilliant fossil. It can reason circles around you about events up to its cutoff and then go completely blank about last Tuesday. You can stuff new facts into the context window, sure, but the moment the session ends, that knowledge evaporates like a dream you forgot to write down.

Behrouz, Hashemi, and Mirrokni (Google + Cornell) have a name for this in their new paper, and it’s a good one: it’s anterograde amnesia. The patient with anterograde amnesia keeps every memory from before the injury and can hold a conversation in the moment, but nothing new ever makes the jump into long-term storage. Each day is experienced as if it were the first. Swap “injury” for “end of pre-training” and that is exactly the shape of a Transformer’s memory. It has the deep past (the MLP weights) and the immediate present (the attention cache), and almost nothing connecting them.

The paper’s pitch is that we’ve been missing a step that biology figured out a long time ago. We sleep.

There is no test time

Read more

💡 AI Concept of the Day: Transformers or Not?

The Transformer is currently the reference architecture for serious AI. Not because it is obviously the most brain-like, elegant, or efficient design, but because it has the best scaling story. You add data, parameters, compute, context length, better training recipes, better post-training, and the model gets better in a surprisingly smooth way. That is rare. In deep learning, many ideas are clever. Few are industrial.

The Transformer’s superpower is attention. Every token can look at every other token and decide what matters. This is an incredibly general operation. It works for language, code, images, audio, video, protein sequences, robotics tokens, and tool traces. The architecture is simple enough to scale, parallel enough to train efficiently, and expressive enough to absorb huge datasets.

But it has an obvious tax: attention is expensive. Full self-attention scales badly with sequence length. In autoregressive generation, the model accumulates a key-value cache, which grows with context. A Transformer remembers by keeping a large, explicit, token-indexed memory. That is powerful, but it is not how you would design every intelligent system from first principles.

So the question is not “are Transformers good?” They are spectacular. The question is: are they the final architecture? Or are they the first truly scalable architecture, soon to be absorbed into something richer?

I think the second view is more likely.

The Landscape of Alternatives

Read more

Next Week in The Sequence:

1. We continue our series about alternatives to transformers.

2. Our opinion section discusses an intriguing thesis: systems of action vs. systems of record.

3. The AI of the week dives into a groundbreaking paper that I’ve read three times this week: models need sleep.

Subscribe and don’t miss out:

📝 Editorial: Last Week in AI: Soccer, S-1s, and Supermodels

This week I want to start close to home. At LayerLens, we announced the Stratix Cup, a live tournament in which frontier AI models play soccer in a simulated environment. Season 1 brings together 16 models organized into four groups, with each model writing code to control a full team of players. Matches unfold in two halves, and models can adapt their strategy at halftime based on what happened on the field.

It is, admittedly, ridiculous in the best possible way: models chasing space, collapsing under pressure, inventing strange formations, and occasionally self-sabotaging in public. But the playfulness hides an important point. Evaluations need more arenas.

Most AI evals still behave like school exams: static, individual, decontextualized. They ask models to answer questions, solve coding problems, summarize documents, or reason through puzzles. These are useful, but they are incomplete. Soccer imposes a different discipline. It tests multi-agent planning, tactical adaptation, long-horizon credit assignment, robustness under adversarial pressure, and the ability to recover from mistakes. It also makes model behavior legible. You do not need a PhD to see when a model loses shape in midfield. That makes the failure modes both entertaining and intellectually honest, a rare combination in AI benchmarks.

The rest of the week echoed the same shift from models as artifacts to models as operating systems.

At Build, Microsoft introduced a new generation of MAI models across reasoning, coding, image, voice, and transcription. The headline is not just that Microsoft is building more of its own models. The strategic point is that the company wants a tighter loop between models, developer tools, agents, and devices. GitHub Copilot, agent security primitives, new model releases, and AI-native workflows all point to a world where AI is no longer a chat box bolted onto software. It is becoming the substrate running through work itself.

Anthropic’s confidential S-1 filing adds a different kind of gravity. The proposed IPO is not yet a public-market event, but it signals that frontier AI is moving from private-market mythology into public-market accountability. Revenue quality, compute commitments, margins, governance, and safety claims will eventually have to survive a much harsher evaluation suite: investors, regulators, and quarterly reporting.

NVIDIA, meanwhile, pushed the frontier in two complementary directions. Cosmos advances the idea of world foundation models for physical AI: systems that can reason about video, simulation, robotics, and embodied environments. Nemotron 3 Ultra expands NVIDIA’s enterprise model stack for demanding reasoning and agentic workflows. The implication is clear: NVIDIA is not merely selling the shovels for the AI gold rush. It wants to define the terrain where robots, agents, simulations, and enterprises are built.

Finally, DeepSeek’s reported new financing round is another reminder that the open-model race is becoming geopolitical infrastructure. Capital, energy, chips, talent, and industrial policy are converging around frontier labs. Open models are no longer just an engineering philosophy. They are becoming a strategic asset.

Put together, this week’s story is AI leaving the demo page. It is playing games, staffing workflows, filing S-1s, simulating the physical world, and attracting national-scale capital.

The question is no longer simply who has the best chatbot. It is which systems can act, adapt, and be trusted in environments that push back. Benchmarks told us how models answer. Arenas will tell us how they behave.

🔎 AI Research

Language Models Need Sleep: Learning to Self-Modify and Consolidate Memories

AI Lab: Google Research and Cornell University

Summary: This paper introduces a bio-inspired “Sleep” paradigm for large language models that enables continual learning and mitigates catastrophic forgetting. The approach features a memory consolidation stage that distills fragile short-term memories into stable long-term parameters, alongside a “Dreaming” phase where the model recursively self-improves using synthetically generated data.

Economy of Minds: Emerging Multi-Agent Intelligence with Economic Interactions

AI Lab: Harvard, MIT, GitHub, 2077 AI, and Kempner Institute

Summary: The Economy of Minds (EOM) framework enables a population of language agents to self-organize and evolve through decentralized economic interactions, such as bidding for the right to act and exchanging peer-to-peer payments. By relying on market selection to mutate wealthy, successful agents and eliminate bankrupt, ineffective ones, this system fosters emergent multi-step reasoning and outperforms stronger monolithic baselines across diverse tasks.

Cosmos 3: Omnimodal World Models for Physical AI

AI Lab: NVIDIA

Summary: Cosmos 3 introduces a unified Mixture-of-Transformers architecture that seamlessly integrates language, image, video, audio, and action sequences to serve as a versatile foundation for Physical AI. By consolidating multiple specialized functions—such as vision-language models, video generators, and world-action models—into a single framework, it achieves state-of-the-art performance across diverse understanding and generation tasks.

Qwen-Image-Flash: Beyond Objective Design

AI Lab: Qwen (Alibaba)

Summary: This paper explores the critical training components of few-step distillation—specifically data composition, step-wise multi-teacher guidance, and task mixture—demonstrating that effective distillation requires holistic pipeline optimization beyond just the objective function. Applying these insights, the authors developed Qwen-Image-Flash, an efficient unified model capable of executing both text-to-image generation and instruction-guided image editing with only four function evaluations.

Dreaming: Better memory for a more helpful ChatGPT

AI Lab: OpenAI

Summary: OpenAI has introduced a more scalable and compute-efficient memory architecture for ChatGPT that relies on a background process called “dreaming” to continuously curate and synthesize past conversations. By optimizing for freshness, continuity, and relevance, this new system significantly improves the model’s ability to recall factual context, follow user preferences, and adapt to changing information over time.

🤖 AI Tech Releases

Nemotron 3 - Ultra

NVIDIA released Nemotron 3 Ultra, optimized for long running agentic workflows.

MAI Models

Microsoft unveiled 7 new AI models including a flagship MAI-Thinking-1.

Gemma 4 12B

DeepMind released Gemma 4 12B, a multimodal intelligence model that can run locally on a laptop.

📡AI News You Need to Know About

• Airbnb’s Brian Chesky plans a new AI lab — Airbnb CEO Brian Chesky intends to bankroll a new AI lab (reportedly focused on user interaction and design) while staying on as Airbnb’s CEO rather than running it himself. Original source: the Bloomberg scoop that broke it, since Airbnb declined to comment and issued nothing of its own →

• Alphabet plans to raise $80B for AI buildout — Alphabet said it will sell $80 billion in stock (including $10 billion to Berkshire Hathaway) to fund AI infrastructure and compute amid demand outstripping supply. Original source: Alphabet’s own press release →

• Coralogix raises $200M — The observability startup raised a $200M Series F (co-led by Advent, CPPIB, and Greenfield) at a $1.6B valuation, betting that monitoring autonomous AI agents will become core production infrastructure.

• ZeroDrift raises $10M — The startup closed an oversubscribed $10M seed round (backed by a16z Speedrun and others) for its “compliance firewall,” which sits inline between AI systems and end users to catch and rewrite non-compliant AI-generated messages.

• Anthropic files to go public — Anthropic confidentially submitted a draft S-1 to the SEC for a proposed IPO, days after a $65B Series H pushed its valuation near $1 trillion.

• Meta sells an AI agent to businesses — Meta began charging for “Meta Business Agent,” an AI that handles customer conversations across WhatsApp, Messenger, and Instagram, as part of its push to monetize AI and diversify beyond ads.

• DeepSeek close to a ~$7B funding round — DeepSeek is reportedly near finalizing roughly 50 billion yuan (~$7.4B) in its first external raise, led by Tencent, CATL, and founder Liang Wenfeng, at a ~$52–59B valuation. Note: there’s no primary source here — the deal is unconfirmed and the parties declined to comment — and the scoop originated with Reuters before Bloomberg corroborated it, so the

• Google to pay SpaceX $920M/month for compute — Google will pay SpaceX ~$920M monthly from October 2026 through June 2029 for access to roughly 110,000 NVIDIA GPUs and related hardware as “bridge capacity” for Gemini Enterprise demand, announced a week before SpaceX’s IPO.

When Jensen Huang draws AI as a five-layer cake — energy, chips, infrastructure, models, applications — he describes it as harmony. Every layer reinforces the others. Every successful application pulls demand down through models, infrastructure, and chips, all the way to the power plant that keeps it alive. It is a beautiful picture, and as a statement of physics it is correct.

But Jensen is selling the bottom of the cake, so of course he wants you to see harmony. If you are a strategist instead of a chip vendor, you look at the same diagram and see something else entirely: five margin pools stacked on top of each other, and a war over which of them you can fuse together before the layer beneath you turns into a commodity. The cake is not a structure of mutual reinforcement. It is a battlefield with a vertical axis.

So the right question is never “how many layers do you own.” It is: do you own the scarce layer, and the seam right next to it?

Read more

I am sure you guys are surprised that we are going to cover Claude Opus 4.8 today ;) but I have been playing with it so much that merits the post.

Opus 4.8 shipped on May 28, 2026. The headline contributions, in the order I’d rank them for anyone building agents: a roughly 4x reduction in how often the model leaves a flaw in its own code unremarked — the calibration/honesty story that defines this release; a fix for silently skipped tool calls, the bug class that quietly poisons long trajectories; better compaction recovery so long-horizon runs stop derailing after the history gets squeezed; dynamic workflows that let the model plan and fan out hundreds of parallel subagents for codebase-scale work; adaptive thinking that decides per-turn whether to reason at all; and a fast mode that runs ~2.5x faster at a tier that’s now ~3x cheaper than 4.7’s. Alignment results land near the (still-restricted) Mythos Preview. Same regular-mode pricing as its predecessor.

It’s tempting to file this under “minor release.” The version bump is a tenth of a point, the benchmark deltas are mostly incremental, and the cadence makes it easy to lose track — Opus 4.6 landed February 5, 4.7 on April 16, and 4.8 just six weeks later. That’s a compression from a roughly quarterly rhythm to something closer to monthly, and when point releases arrive that fast the instinct is to treat each one as a patch and skip the changelog.

That instinct is wrong here, because Opus 4.8 isn’t competing on the axis the version number implies. The benchmark table moved a little. What moved a lot is the reliability axis — the silent-failure rate, the tool discipline, the ability to hold a thread across a long run unattended. Those are the properties that gate whether you can actually leave an agent running, and they don’t show up on a capability leaderboard. The short cadence is also the tell: when you can ship calibration and reliability fixes every six weeks, the model stops being a thing you upgrade quarterly and becomes infrastructure you keep current. So let me give you the version I’d want if I were wiring this into a production agent loop at 2am.

The benchmark story is the boring story

Read more

💡 AI Concept of the Day: Liquid Models and the Search for a Post-Transformer Architecture

The Transformer did not merely become the dominant neural architecture. It became the default mental model for intelligence in modern AI.

Its central idea is deceptively simple: when processing a sequence, let every element look at every other element. A word can attend to previous words. A code token can attend to distant variables. An image patch can attend to another patch. A tool call can attend to an instruction buried thousands of tokens earlier. Attention turns sequence modeling into a giant differentiable lookup table over context.

This was a profound break from the recurrent era. Earlier models processed sequences like a reader moving left to right, updating a hidden state at each step. Transformers flattened that temporal process into a massively parallel computation. Instead of compressing the past into a single state, they exposed the entire past to the model. That made training easier, scaling more predictable, and long-range relationships easier to represent.

But every architecture has a physics. Transformers have the physics of global interaction. That physics is powerful, but expensive.

Self-attention wants to compare tokens against other tokens. During inference, the model accumulates a key-value cache so that each new token can attend to the past. As context grows, memory grows. As model size grows, serving complexity grows. As agents become longer-running, more tool-using, and more local, the cost of remembering everything explicitly becomes harder to ignore.

The Transformer is a brilliant architecture for cloud-scale intelligence. It is less obviously the final architecture for always-on, low-latency, private, embodied, on-device intelligence.

That is where liquid models enter the story.

From Attention to Dynamics

Read more

Next Week in The Sequence:

• We continue our series about transformer alternatives.

• In the AI of the Week section, we discuss Opus 4.8.

• The opinion of the week discusses companies the strategic differences between companies like Google, NVIDIA, Microsoft, OpenAI and Anthropic when comes to their ownership of different areas of the AI stack.

Subscribe and don’t miss out:

📝 Editorial: Last Week in AI: The Token Becomes the Unit of Account — Opus 4.8, OpenRouter, Cognition, Snowflake, and a papal warning

For two years the AI boom was an argument about the future, told in benchmarks and term sheets. This week it became an argument about the present, told in revenue.

Start with the substrate. Anthropic shipped Claude Opus 4.8 and, in the same breath, disclosed it’s tracking toward its first operating profit — roughly $10.9B in projected Q2 revenue, up about 130% quarter over quarter — while closing a $65B round. Sit with that. A lab still doing frontier training runs is approaching operational profitability. The “labs are structurally unprofitable” assumption that anchored every bear case just lost its load-bearing wall.

The model itself Anthropic described, refreshingly, as “a modest but tangible improvement” — agentic coding nudges from ~64% to ~69%, reasoning-with-tools from ~55% to ~58%, at the same price as 4.7. The interesting stuff is underneath the benchmarks. Three changes matter. First, an effort control that lets you dial how hard the model thinks per task — explicit governance over the compute-versus-quality tradeoff that every agent builder has been hacking around with prompt tricks. Second, dynamic workflows: a Claude Code capability where the model plans a large task, spins up parallel sub-agents to attack the pieces, verifies their outputs, and reports back — paired with a Messages API that now accepts live edits to the message array mid-run without breaking the prompt cache, so you can steer a long job without tearing it down and restarting. Third, honesty as a measured capability: 4.8 is roughly 4x less likely than 4.7 to let a flaw in its own code slip through unflagged, and surfaces its own uncertainty more readily. Stack those and you get the thing that actually matters once a model runs unattended for hours: it plans, it parallelizes, it checks its own work, and — because you can’t read every diff — it’s trained to distrust itself. It also burns tokens by the fistful doing all of it.

Then watch where the money flows, because it tells you the unit of account is now the token. OpenRouter raised $113M at $1.3B doing something almost embarrassingly simple: routing across 400+ models and taking ~5% of the inference spend that passes through. Its weekly throughput went from 5T to 25T tokens in six months — 5x. That’s not a forecast; that’s a meter. Cognition raised $1B at $26B, and buried in the announcement was the line that should reorganize your priors: 89% of code committed inside Cognition is now written by Devin, up from 13% in December. Run-rate revenue went from $37M to $492M in a year. Autonomous software engineering stopped being a demo and became the default committer.

Snowflake closes the loop on the public side. Product revenue up 34%, guidance raised, stock up ~36% in a single session — and the two tells are a $6B AWS compute deal and the acquisition of Natoma, an MCP platform for governing agent access. The data layer is repricing itself around agents that consume, not analysts that query. The whole stack — model, router, agent, substrate — is converging on one business model: charge by the token, because the token is the work.

Which is exactly the moment Pope Leo XIV chose to publish Magnifica Humanitas, his first encyclical, presented alongside Anthropic’s Chris Olah. Stripped of its theology, the argument is an engineering critique the field should take seriously: technology is never neutral, because it inherits the incentives of whoever builds and funds it — and the danger isn’t malice but quiet disintermediation, decisions sliding out of human hands one delegated commit at a time.

Hold those two facts together. Cognition’s 89% is the encyclical’s thesis restated as a metric. The meter that makes the economics work is the same meter measuring how much judgment we’ve handed off. The bull case and the moral case are reading the same number.

The flywheel is no longer a slide. It’s on the income statement. The open question is what it’s optimizing for.

🔎 AI Research

Verus-SpecGym: An Agentic Environment for Evaluating Specification Autoformalization

AI Lab: CMU & Amazon

Summary: To address the challenge of evaluating whether AI agents can accurately translate informal programming intent into formal specifications, the researchers introduce the VERUS-SPECBENCH benchmark and the VERUS-SPECGYM agentic environment. By extending an execution mechanism to test generated specifications against both official tests and adversarial “hacks,” the study reveals that specification autoformalization remains highly brittle even for models capable of generating correct code.

Gamma-World: Generative Multi-Agent World Modeling Beyond Two Players

AI Lab: Tsinghua University, NVIDIA, University of Toronto, & Vector Institute

Summary: Gamma-World presents a scalable, generative multi-agent world model that moves beyond traditional single-agent simulations by utilizing Simplex Rotary Agent Encoding for permutation-symmetric identities and Sparse Hub Attention for efficient cross-agent communication. Through conditional teacher-student distillation and KV-cached streaming, the framework achieves real-time, action-responsive rollouts at 24 FPS that maintain strong consistency across virtual gaming and physical robotic environments.

Self-Improving Language Models with Bidirectional Evolutionary Search

AI Lab: Harvard University & MIT

Summary: Bidirectional Evolutionary Search (BES) overcomes the limitations of sparse verification signals and narrow autoregressive expansion by coupling forward candidate evolution with backward goal decomposition. By recombining trajectory segments to escape narrow probability distributions and scoring them against fine-grained sub-goals, BES significantly outperforms existing open-source frameworks on complex logical reasoning and open problem-solving tasks.

MobileMoE: Scaling On-Device Mixture of Experts

AI Lab: Meta AI

Summary: MobileMoE introduces a family of sub-billion active-parameter Mixture-of-Experts (MoE) language models specifically optimized for efficient deployment on edge devices like smartphones. Guided by a novel on-device scaling law and supported by a custom fused MoE kernel, these models achieve state-of-the-art performance while delivering substantially faster prefill and decode speeds compared to dense baselines at a similar memory footprint.

Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini

AI Lab: Google DeepMind

Summary: Gemini Embedding 2 is a native multimodal embedding model that seamlessly maps text, image, audio, and video inputs into a single, unified representation space without relying on intermediate transcriptions. Trained via large-scale contrastive learning in a multi-task setup, the model establishes new state-of-the-art performance across unimodal, cross-modal, and multimodal retrieval benchmarks while demonstrating robust zero-shot generalization across diverse enterprise and specialized domains.

The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence

AI Lab: MiniMax

Summary: The MiniMax-M2 series introduces a highly efficient 229.9B parameter Mixture-of-Experts model that activates only 9.8B parameters per token, specifically engineered for complex, long-horizon agentic workflows. By leveraging agent-driven data pipelines, a specialized reinforcement learning system called Forge, and autonomous self-evolution capabilities, the model achieves frontier-level performance across coding, deep search, and reasoning benchmarks while maintaining a minimal computational footprint.

🤖 AI Tech Releases

Claude Opus 4.8

Anthropic released the new version of its marquee model, with strong agentic and coding capabilities.

📡10 AI News You Need to Know About

1. Anthropic raises $65B in Series H at $965B post-money valuation — Anthropic raised $65 billion in Series H funding (co-led by Altimeter, Dragoneer, Greenoaks, and Sequoia) at a $965 billion post-money valuation, disclosing that run-rate revenue crossed $47 billion earlier this month, and bringing on Micron, Samsung, and SK hynix as strategic memory/storage partners alongside $15B in previously committed hyperscaler investment (including $5B from Amazon).

2. Cognition raises $1B at $25B pre-money valuation — Cognition, maker of the AI software engineer Devin, raised more than $1 billion (led by Lux Capital, General Catalyst, and 8VC) at a ~$26B post-money valuation, more than doubling in eight months as it hit a $492M annualized revenue run-rate.

3. Robinhood lets AI agents trade stocks — Robinhood launched Agentic Trading and an Agentic Credit Card in beta, letting customers connect third-party AI agents (via MCP) to a separate, funded account to autonomously trade equities and make purchases. Original source:

4. OpenRouter doubles valuation to $1.3B — The multi-model AI inference-routing startup raised a $113M Series B led by Alphabet’s CapitalG at a ~$1.3B valuation, more than double its level a year ago, as weekly volume grew from 5T to 25T tokens.

5. Hark raises $700M Series A — Brett Adcock’s secretive AI startup raised $700M at a $6B post-money valuation (led by Parkway Venture Capital) to build a “universal” agentic AI assistant with proprietary multimodal models and custom hardware, with first models due summer 2026.

6. Mistral signs Airbus and BMW — Mistral AI expanded into “physical AI” for manufacturing, announcing partnerships to apply its models to Airbus (aircraft design, flight safety, defense/space) and BMW’s “Large Industry Model” crash-simulation initiative, plus a new French data center.

7. MiniMax doubles sales ahead of new model — The Chinese AI developer’s annualized revenue more than doubled over two months to at least $300M, driven by its M2.7 model and a fivefold jump in enterprise users, ahead of its next flagship launch. No primary source to substitute: the figures come from a Bloomberg Television interview with co-founder Yun Yeyi, so Bloomberg is the original source.

8. SK Hynix joins the $1 trillion club — Shares of the South Korean memory maker surged ~9–15% to push its market value above $1 trillion for the first time, driven by HBM demand for AI, joining rivals Samsung and Micron. No company announcement exists for a stock-price milestone; the event was originated on the wires by Reuters: https://money.usnews.com/investing/news/articles/2026-05-26/sk-hynix-joins-1-trillion-club-after-samsung-micron-on-ai-chip-boom

9. Pope Leo warns AI shouldn’t dominate humanity — In his first encyclical, Magnifica Humanitas, Pope Leo XIV called for “disarming” AI to keep it human-friendly and free of monopolistic control, warning it risks deepening inequality and eroding human agency. Original source: the encyclical itself, published by the Vatican (vatican.va) — that’s the underlying document the coverage is based on.

10. Snowflake signs $6B AWS deal for Graviton chips — Snowflake committed $6B over five years to AWS — its largest infrastructure commitment ever — expanding use of Amazon’s ARM-based Graviton CPUs and GPUs to power agentic AI workloads.