Preprint Open access
Rethinking the Tradeoff Between Temporal Encoding and Nonlinear Computation in Spiking Language Models
Spiking language models face a tradeoff between representing continuous semantic features over short temporal windows and retaining costly nonlinear attention operations. We introduce Spora, which jointly designs spike encodings and attention operators. Binary temporal weights let $T$ spikes represent compositional val …