Preprint Open access
Same Scores, Different Decisions: Evaluating JEV and Language Models for Legal Document Understanding
Contract inference requires multiple judgments about a shared document, but aggregate accuracy can conceal changes in the individual decisions. Repeated agreement is also insufficient: a model may consistently return the wrong answer. In this paper, we compare Jev with nine language models on ContractNLI, evaluating in …