JEPA is not an LLM.
It's built for the real world.

In the last note we unpacked what actually sits behind the "AI" label: the large language model, and how the prediction game gives it its remarkable power over anything made of words. This note is about the most serious contender for what comes next. JEPA, the architecture Yann LeCun has championed for years, is not a language model at all. It is not trained on text. It learns from images and video of the real world, and that single difference explains a lot, including why today's LLMs are so much weaker with images than with words.

This piece is about that difference: what JEPA (the Joint-Embedding Predictive Architecture) actually is, and why the "world model" school of AI it belongs to starts from pictures of reality rather than sentences about it. Not because LLMs are finished (they aren't), but because knowing where the field is heading is part of architecting systems that are still standing when it gets there.

Why LLMs struggle with the real world, images included

The last note ended on the LLM's defining trait: everything it knows, it learned from text, by predicting text. Language is its native substance. That is precisely the problem when you point one at an image: an LLM doesn't see a picture, it reads one. The image is chopped into patches, converted into token-like vectors, and pushed through a machine built for text — description bolted onto a language engine, not perception. It's why even frontier models miscount the objects on a table, misread an analogue gauge, mangle spatial relationships in a diagram, and confidently describe things that aren't there. The failure isn't a missing feature waiting on the next release. It's the architecture:

• No working model of the world. An LLM predicts what text tends to follow other text. It has read every physics textbook and still has no internal simulation of objects, causes, or consequences, which is why it can explain gravity flawlessly and then reason as if a full glass can be tipped without spilling.
• Planning is not its native motion. Generating one token at a time is a poor fit for problems that need looking several moves ahead, backtracking, and weighing alternatives: the shape of most real operational decisions.
• Staggering sample inefficiency. A four-year-old learns how the physical world works from a few years of watching and touching it. Frontier LLMs consume a meaningful fraction of all text ever written and still don't have that model. That gap is a signal about the architecture, not the compute budget.

None of this makes LLMs a bad investment. It makes them a specific investment: brilliant at language-shaped work, structurally weak at world-shaped work. The research frontier's response to that weakness is to stop treating vision as a bolt-on and start from the visual world itself. That's JEPA.

What JEPA actually does

JEPA is the architecture Yann LeCun, Turing Award winner and for over a decade Meta's chief AI scientist (a role he left at the end of 2025 to pursue world models full time), has been advancing as the path beyond token prediction. It inverts the LLM's relationship with reality: where a language model treats images as an afterthought grafted onto a text engine, JEPA is trained on images and video from day one. The visual world isn't an add-on; it's the curriculum. And the core idea fits in a sentence: instead of predicting the next word or the next pixel, predict the next state of the world, in an abstract representation space that ignores the details that don't matter.

When you catch a ball, your brain doesn't render every future pixel of the ball's flight. It predicts the trajectory (a compact, abstract summary) and discards the irrelevant detail: the exact glint of light, the precise spin of the logo. JEPA formalises that. Two encoders turn raw input (an image, a video clip) into compact representations; a predictor learns to fill in the representation of the missing or future part. The model is never asked to reproduce reality pixel-perfectly — only to get the gist of what happens next right. That single design choice is why the approach is efficient, and why its advocates believe it scales toward genuine understanding: it spends its capacity on structure, not surface.

This isn't a whiteboard theory. The milestones have been landing steadily, and every one of them is visual: I-JEPA (2023) proved the idea on real images, learning visual structure without labels and without reconstructing pixels. V-JEPA (2024) extended it to video, learning intuitive physics by watching the world move. V-JEPA 2 (2025) turned that into a working world model: after pre-training on over a million hours of video, it can drive a robot arm through pick-and-place tasks in labs it has never seen, planning by imagining the outcomes of candidate actions before taking them — zero-shot, in Meta's published results. Where an LLM struggles to genuinely read one image, this line of work learns physics from a million hours of them. Same pixels, opposite architecture.

Why this matters to a business, not just a lab

The honest answer for most companies in 2026: nothing on your roadmap should be rebuilt around JEPA this quarter. World models are where LLMs were around 2019: real, working, and not yet productised. But three implications are worth acting on now.

1. Today's LLMs are a chapter, not the whole book

The industry's biggest bets are already diversifying beyond pure next-token prediction: world models, reasoning systems that search before they answer, agents that act rather than reply. The systems that eventually plan warehouse movements, drive machinery, and run physical operations will need to understand the world those operations happen in, not just the language describing them. If your AI strategy begins and ends with one model wired into one feature, you have a feature, not a strategy.

2. Match the architecture to the shape of the problem

Language-shaped problems (drafting, summarising, extraction, code assistance, support triage) are LLM territory today and will stay that way for years. World-shaped problems are exactly where LLMs are structurally weakest and where world-model approaches are aimed: logistics simulation, operational planning, anything where the cost of a confidently wrong answer is physical. Knowing which kind of problem you're holding is the difference between an AI feature that compounds and one that gets quietly turned off.

3. Build so the model is swappable, architecture included

We wrote about surviving the day your AI vendor pulls the model. The same discipline covers the bigger shift: if your product treats "the model" as a replaceable component behind your own interface, with prompts, context, and evaluation sets versioned in your repo and capabilities measured against your real tasks, then an architecture change arrives as an upgrade, not a rewrite. Teams that welded a single vendor's API into their core logic will be doing surgery. Teams that built the boundary will change a config.

The calm read

Every wave of this technology has punished two groups: the ones who dismissed it and the ones who bet everything on the current shape of it. The LLM changed what software can do, and it is still, today, the most useful AI machine a business can deploy. JEPA and the world-model school are a credible view of what comes after: systems that understand and plan rather than complete. You don't need to bet on the winner. You need software built so it doesn't matter who wins.

Don't build for this year's architecture. Build so the architecture is a detail. LittleMan · Digital Architects

That's the standing advice, and it's the same whether the next shift is JEPA, a reasoning system, or something no lab has published yet: keep the intelligence replaceable, keep your context portable, and keep a senior engineer between the hype and your production system.