Authors

Aline Villavicencio

Publications 2

Preprint Open access

Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior

Atsuki Yamaguchi, Tatsuro Inaba, Joel Niklaus et al. · 2026

Pre-pretraining (PPT) on synthetic non-natural language data improves token efficiency during language model pre-training (PT). Prior work attributes this gain to a grammatical prior, i.e., a structural inductive bias learned during PPT that transfers to natural language grammar. However, PPT has only been tested on mo …

Preprint Open access

MWE-ECL: Recoverable Long-Range Context Does Not Always Override Local Lexical Priors

Wei He, Aline Villavicencio, Rodrigo Wilkens et al. · 2026

Long-context evaluations often test whether a model can recover distant evidence, but recoverability does not guarantee behavioral influence. We test the prediction that a distant discourse anchor can remain explicitly recoverable yet fail to change the locally preferred reading of a familiar multiword expression; such …

Co-authors