Write a PREreview

We introduce Kernel Geometry Divergence (KGD), a Hilbert-Schmidt metric for comparing Mercer kernels induced by distinct random-feature (RF) constructions in efffficient attention mechanisms. KGD measures the L 2 distance between kernels via their Funk-Hecke eigenvalue spec tra under the uniform probability measure on the sphere. We estab lish Mercer decompositions for independent Gaussian RF and Gram Schmidt orthogonal RF (GS-ORF), revealing distinct Gaussian RBF versus spherical hypergeometric kernel limits. We prove that KGD con trols performance gaps in kernel ridge regression and attention-layer output through operator-theoretic bounds, and derive the dimension scaling law showing that KGD = Θ(d −α) with α ≈ 0.88 in the unit sphere regime. We characterize the three-way trade-off among inde pendent RF, GS-ORF, and random Hadamard features (RHF) through KGD-induced hierarchy. Numerical simulations on synthetic spherical data and a sequence prediction task validate the predicted scaling laws and confirm that KGD upper-bounds empirical performance gaps.