Saltar al contenido principal

Escribe una PREreview

Semantic Thermodynamics of Transformer Architectures: A Framework for Understanding Hallucination Constraints

Publicada
Servidor
Preprints.org
DOI
10.20944/preprints202602.1962.v1

We develop \emph{Semantic Thermodynamics}, an information-theoretic framework for analyzing hallucinations in transformer systems under finite resources. The central object is mutual information between latent facts and model outputs, together with Fano-style lower bounds on semantic error. We clarify the stochastic assumptions required for non-degenerate information measures, distinguish true data-generating uncertainty from model-implied uncertainty, and replace unsupported hard capacity formulas with explicit capacity surrogates tied to precision, context budget, and effective representational rank. Under standard identification assumptions, we derive a baseline bound \begin{equation*} H_R \geq \max\left\{0,\,1-\frac{I(F;Y)+1}{\log M}\right\}, \end{equation*} where HRH_R is hallucination rate, FF is the latent semantic fact, YY is model output, and MM is semantic cardinality. We also provide a distribution-dependent variant and a bottleneck-aware extension for retrieval-augmented generation. This paper contributes a mathematically consistent formulation, a tighter assumptions section, and concrete empirical protocols for estimation and falsification.

Puedes escribir una PREreview de Semantic Thermodynamics of Transformer Architectures: A Framework for Understanding Hallucination Constraints. Una PREreview es una revisión de un preprint y puede variar desde unas pocas oraciones hasta un extenso informe, similar a un informe de revisión por pares organizado por una revista.

Antes de comenzar

Te pediremos que inicies sesión con tu ORCID iD. Si no tienes un iD, puedes crear uno.

¿Qué es un ORCID iD?

Un ORCID iD es un identificador único que te distingue de otros/as con tu mismo nombre o uno similar.

Comenzar ahora