Avalilação PREreview de Exploring Similarity between Neural and LLM Trajectories in Language Processing

Summary

The authors studied the similarity between large language models (LLMs) and human brain activity to advance AI and cognitive neuroscience. They compared text embeddings from 16 publicly available pretrained LLMs, such as Mistral-7B-v0.1, Llama-2-7b-hf, and Qwen2.5-7B, with EEG-extracted neural features from human brain responses during natural language processing tasks in English and Chinese. Using ridge regression, they assessed representational similarity between LLM embeddings and EEG signals, and analyzed similarities between "neural trajectories" and "LLM latent trajectories," capturing dynamic patterns like magnitude, angle, uncertainty, and confidence. This work reveals how brain activity over time during language processing relates to LLM internal representations, extending beyond static outputs to dynamic trajectories.

Major issues

• Major Concern: Limited explanation of why ridge regression was chosen over alternatives like linear regression or other encoding methods.

  Suggestion: Please add a justification in the methodology section, discussing why ridge regression was adapted in comparison to other encoders for transparency and reproducibility.

Minor issues

• Results emphasize English alignment over Chinese; discuss potential biases from training data.

  Suggestion: Expand discussion on implications for non-English languages and suggest future multilingual training evaluations.

Competing interests

The author declares that they have no competing interests.

Use of Artificial Intelligence (AI)

The author declares that they did not use generative AI to come up with new ideas for their review.