Preprint Open access
The Missing Coefficients: Bayesian Pairwise Merging for Model Personalization
How can we personalize a shared expert library from a user's pairwise choices? Prior work can realize different reward trade-offs by merging reward-specialized experts, given a vector of trade-off weights. In practice, users can more naturally choose between outputs than specify numerical weights. The challenge is ther …