Preprint Open access
Scaling Influence Functions in LLMs through Eigenbasis-Corrected One-Bit Gradient Projection
Influence functions estimate how individual training examples affect the behavior of large language models (LLMs). Analyzing how training data influence different behaviors of an LLM involves repeated influence computation. Reusing stored training gradients reduces the computational cost, but storing full gradients is …