Skip to main content

Write a PREreview

Does Linguistic Register Affect Faithfulness in Spanish Cache-Augmented Generation? An Exploratory Study with Open-Source LLMs

Posted
Server
Preprints.org
DOI
10.20944/preprints202609.0733.v1

Cache-Augmented Generation (CAG) provides an alternative to retrieval-based knowledge augmentation by preloading external knowledge into a reusable key–value (KV) cache. However, the extent to which CAG remains stable when semantically equivalent queries are expressed through different linguistic registers remains insufficiently understood, particularly in Spanish. This exploratory study investigates whether formal–colloquial register variation affects the Faithfulness of CAG responses. A controlled paired design was constructed from 10 Spanish concepts and two complementary tasks—definition generation and concept identification—yielding 20 semantic cases and 40 formal–colloquial queries per model. Three open-source instruction-tuned LLMs (Qwen2.5-1.5B-Instruct, SmolLM2-1.7B-Instruct, and Ministral-3B-Instruct) were evaluated under deterministic generation conditions using a fixed preloaded knowledge context. Faithfulness was assessed with RAGAS using Granite 3.2 as the LLM judge. Of 120 planned generations, 113 produced valid Faithfulness evaluations, yielding 53 complete formal–colloquial pairs. Neither paired parametric nor non-parametric analyses identified statistically significant register effects for any individual model, and the between-model analysis did not detect systematic differences in register sensitivity. Descriptively, however, SmolLM2 tended to favor formal formulations, Ministral showed the opposite tendency, and Qwen2.5 exhibited the most stable profile, combining comparatively high Faithfulness, limited formal–colloquial separation, and relatively low within-condition variability. These findings provide initial evidence that formal–colloquial variation does not produce a detectable systematic effect on CAG Faithfulness under the conditions examined, while suggesting that descriptive sensitivity to linguistic register may depend on the underlying model and task configuration.

You can write a PREreview of Does Linguistic Register Affect Faithfulness in Spanish Cache-Augmented Generation? An Exploratory Study with Open-Source LLMs. A PREreview is a review of a preprint and can vary from a few sentences to a lengthy report, similar to a journal-organized peer-review report.

Before you start

We will ask you to log in with your ORCID iD. If you don’t have an iD, you can create one.

What is an ORCID iD?

An ORCID iD is a unique identifier that distinguishes you from everyone with the same or similar name.

Start now