Authors

Hinrich Schütze

Publications 2

Preprint Open access

Orthogonal Yet Coupled: Decoupling Geometric Components for Model Merging

Zijing Wang, Yongkang Liu, Mingyang Wang et al. · 2026

Merging pretrained models has emerged as an effective approach for consolidating diverse capabilities into a single unified model. However, prevailing merging methods typically treat each task vector as an indivisible merging unit, overlooking the heterogeneous geometric changes encoded within it. This treatment can in …

Preprint Open access

Weakening Neurons: An Input-Output Functionality in Transformers with Outsize Influence

Sebastian Gerstner, Hilal AlQuabeh, Kentaro Inui et al. · 2026

We analyze the learned input-output behavior of GLU-based neurons in large language models (LLMs). We propose a simple analysis method: For each neuron, we compute the cosine similarities between its input (reading) and output (writing) weight vectors. In this scheme, a strong negative cosine similarity indicates the n …

Co-authors