A neural network architecture, introduced in 2017, that uses attention mechanisms to process sequences of tokens in parallel and powers every modern large language model.

Transformer | The Wise Operator

What It Is

A transformer is a specific kind of neural network architecture, introduced in the 2017 paper “Attention Is All You Need” by a team at Google, that learned to do something the previous generation of models could not. It processes every token in a sequence at the same time and lets each token attend to every other token directly. Before the transformer, language models read sentences left to right, one word at a time, and forgot what they had seen by the end of a long paragraph. The transformer’s attention mechanism replaced that left-to-right pipeline with a global view. Every token sees every other token, weighted by relevance.

That sounds like a footnote in AI history. It is not. Every model an operator is likely to use this year is built on the transformer architecture. Every general-purpose chat model, every coding assistant, every translation engine, every multilingual document tool. The shift from “neural net research project” to “language model that can write a contract” runs through this single design choice. When a non-technical reader hears the phrase large language model, the transformer is the engine under the hood.

How It Actually Works

The transformer breaks input text into tokens and processes them through stacked blocks of two operations: self-attention and a feed-forward step. Self-attention is the heart of the design. For each token in the sequence, the model computes how relevant every other token is to it, then weights its understanding of that token accordingly. A pronoun like “it” in a sentence can directly attend to the noun it refers to, no matter how many words separate them. A multilingual model trained on dozens of languages can learn that the Spanish word for “house” and the English word for “house” sit near each other in concept space, because attention exposes those alignments during training.

The architecture is parallelizable in a way the previous generation of recurrent networks was not. That parallelism is why training runs on tens of thousands of GPUs can finish in months instead of years, and why a single model can serve millions of users at once at inference time without grinding to a stop.

Why It Matters Right Now

Almost everything an operator actually touches when working with AI in 2026 is a transformer descendant. The decision to use one model over another is partly a decision about which lab’s particular transformer variant best fits the workflow at hand. Differences in context window length, in cost per million tokens, in multilingual coverage, in tool-use reliability, in how well a model follows instructions, all flow from architectural choices made on top of the same transformer foundation. Knowing the term means recognizing that those choices are real engineering decisions, not magic, and that the lab that ships the most expensive frontier model is still publishing papers that reference the same 2017 design.

How TWO Uses It

The reason this word lives in TWO’s dictionary is that operators tend to talk about “AI” as if it were one technology and one product category. It is not. AI in 2026 is, almost entirely, a family of transformer architecture language models with different training data, different fine tuning, and different commercial wrappers. When a vendor at a conference says their assistant “uses our proprietary AI,” the right follow up question is which transformer derived model is underneath, and the answer is almost always Claude, GPT, Gemini, Llama, or a fine tune of one of them. There is no mysterious fifth category. Knowing the term sharpens that conversation in five seconds.

The operator decision looks like this. Scott is evaluating two competing scheduling assistants for a small services business, both pitched as “purpose built.” The cheaper one runs a Llama fine tune. The more expensive one routes calls to Claude. The architectures are in the same family. The cost gap is the markup on inference, plus the wrapper, plus the brand. Knowing the term lets him price the wrapper honestly rather than treating the more expensive option as inherently smarter. Sometimes the wrapper is worth the markup. Often it is not. Either way the question is now answerable, not mystical.

Common Misconceptions

The most common misread is that the transformer is the model. It is not. The transformer is an architecture, a blueprint. A model is a specific set of trained weights running on that blueprint. Claude Opus, GPT, Gemini, Llama are all different models, all transformers. Equating the architecture with a single product is like calling every car a Toyota because Toyota has a lot of them on the road.

The second misread is that the transformer is finished. It is not. Researchers are actively publishing variants: mixture of experts routing, state space hybrids, sub-quadratic attention. None has displaced the core design yet. If one does, the dictionary will say so. For now, every model an operator uses this year is a transformer.

The transformer connects directly to tokens, the units it operates on, to context window, the upper bound on how many tokens it can attend to at once, and to large language model, the product category that sits on top.