Saltar al contenido principal

Escribe una PREreview

Does Linguistic Register Affect Faithfulness in Spanish Cache-Augmented Generation? An Exploratory Study with Open-Source LLMs

Publicada
Servidor
Preprints.org
DOI
10.20944/preprints202609.0733.v1

Cache-Augmented Generation (CAG) provides an alternative to retrieval-based knowledge augmentation by preloading external knowledge into a reusable key–value (KV) cache. However, the extent to which CAG remains stable when semantically equivalent queries are expressed through different linguistic registers remains insufficiently understood, particularly in Spanish. This exploratory study investigates whether formal–colloquial register variation affects the Faithfulness of CAG responses. A controlled paired design was constructed from 10 Spanish concepts and two complementary tasks—definition generation and concept identification—yielding 20 semantic cases and 40 formal–colloquial queries per model. Three open-source instruction-tuned LLMs (Qwen2.5-1.5B-Instruct, SmolLM2-1.7B-Instruct, and Ministral-3B-Instruct) were evaluated under deterministic generation conditions using a fixed preloaded knowledge context. Faithfulness was assessed with RAGAS using Granite 3.2 as the LLM judge. Of 120 planned generations, 113 produced valid Faithfulness evaluations, yielding 53 complete formal–colloquial pairs. Neither paired parametric nor non-parametric analyses identified statistically significant register effects for any individual model, and the between-model analysis did not detect systematic differences in register sensitivity. Descriptively, however, SmolLM2 tended to favor formal formulations, Ministral showed the opposite tendency, and Qwen2.5 exhibited the most stable profile, combining comparatively high Faithfulness, limited formal–colloquial separation, and relatively low within-condition variability. These findings provide initial evidence that formal–colloquial variation does not produce a detectable systematic effect on CAG Faithfulness under the conditions examined, while suggesting that descriptive sensitivity to linguistic register may depend on the underlying model and task configuration.

Puedes escribir una PREreview de Does Linguistic Register Affect Faithfulness in Spanish Cache-Augmented Generation? An Exploratory Study with Open-Source LLMs. Una PREreview es una revisión de un preprint y puede variar desde unas pocas oraciones hasta un extenso informe, similar a un informe de revisión por pares organizado por una revista.

Antes de comenzar

We will ask you to log in with your ORCID iD. If you don’t have an iD, you can create one.

What is an ORCID iD?

An ORCID iD is a unique identifier that distinguishes you from everyone with the same or similar name.

Comenzar ahora