Preprint Open access
Learning Decision-Stump Thresholds in Context: Dynamics of Softmax Attention
Estimating a decision threshold requires locating observations near an unknown boundary. We study how gradient-based pretraining learns this statistical rule in a two-parameter softmax-attention model with a fixed feature and inequality direction. Pretraining uses labeled contexts and their true thresholds; a fresh thr …