Write a PREreview

Softmax attention and nodal analysis are the same arithmetic. For each query, the exponential of an attention score is a conductance, the softmax normaliser is the total conductance at a node, and the attention output is the node voltage that satisfies Kirchhoff’s current law when every key is a voltage source. The identity composes: an attention-only transformer with two heads per layer and width linear in the number of nodes performs, layer by layer and up to an explicit exponentially small term, the Jacobi relaxation of nodal analysis on any passive resistor network with voltage-source boundaries supplied in context, at the circuit’s convergence rate. Does learning find it? Transformers trained on random twelve-node circuits separate three things the identity ties together: they predict voltages well (one layer is worth 5 sweeps) without storing sweep iterates; in the attention-only family they grow a last-layer Kirchhoff head whose attention matches the normalised conductances (correlation 0.89) and whose ablation returns the model to the zero predictor; and at matched voltage error they violate the current law several times more than the sweep, because their error is rough where the sweep’s is smooth. Training from the exact weights keeps the head (at depth four and eight) but abandons the sweep. The identity gives transformers a physical vocabulary and a provable capability; the experiments measure, in sweeps and in current, the gap between what the architecture can express and what learning chooses.