الملخص
Analog in-memory computing (AIMC) offers an alternative for model training by executing matrix operations directly where weights are stored. However, scaling AIMC to train modern deep models remains an open challenge due to severe hardware non-idealities, including physical weights with finite dynamic range and write granularity, analog-digital converters with finite resolution, and noisy and asymmetric updates. Guided by the insight that gradient accumulation is sensitive to precision and rounding errors, we adopt a mixed-precision training paradigm: executing forward and backward matrix multiplications in the analog domain while computing weight gradients in the digital domain. To enable scalable training, we present a holistic system-algorithm co-design that co-optimizes weight mapping to ensure well-conditioned physical and logical weight profiles, couples a preconditioned optimizer with threshold-triggered open-loop pulsing to stabilize training trajectories, and aligns converter dynamic ranges to suppress quantization errors. Evaluated via hardware-calibrated architectural simulations calibrated with electrochemical RAM measurements, our framework scales Transformer training up to $123\text{M}$ parameters with validation loss scaling as $L\propto N^{-0.231}$, where $N$ is the parameter count, comparable to $L\propto N^{-0.238}$ for digital training.
الكلمات المفتاحية
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Wu, Z., Gokmen, T., Fagbohungbe, O., Xiao, T. P., & Chen, T. (2026). Making Analog Training Scale: Co-Designing Mapping, Optimizer, and Converters. https://omanscience.com/ar/articles/making-analog-training-scale-co-designing-mapping-optimizer-and-converters
MLA 9
Wu, Zhaoxian, et al. "Making Analog Training Scale: Co-Designing Mapping, Optimizer, and Converters." https://omanscience.com/ar/articles/making-analog-training-scale-co-designing-mapping-optimizer-and-converters.
شيكاغو (المؤلف–التاريخ)
Wu, Zhaoxian, Tayfun Gokmen, Omobayode Fagbohungbe, T. Patrick Xiao, and Tianyi Chen. 2026. "Making Analog Training Scale: Co-Designing Mapping, Optimizer, and Converters." https://omanscience.com/ar/articles/making-analog-training-scale-co-designing-mapping-optimizer-and-converters.
هارفارد
Wu, Z., Gokmen, T., Fagbohungbe, O., Xiao, T. P. and Chen, T. (2026) 'Making Analog Training Scale: Co-Designing Mapping, Optimizer, and Converters', Available at: https://omanscience.com/ar/articles/making-analog-training-scale-co-designing-mapping-optimizer-and-converters.
فانكوفر
Wu Z, Gokmen T, Fagbohungbe O, Xiao TP, Chen T. Making Analog Training Scale: Co-Designing Mapping, Optimizer, and Converters. https://omanscience.com/ar/articles/making-analog-training-scale-co-designing-mapping-optimizer-and-converters
IEEE
Z. Wu, T. Gokmen, O. Fagbohungbe, T. P. Xiao, and T. Chen, "Making Analog Training Scale: Co-Designing Mapping, Optimizer, and Converters," https://omanscience.com/ar/articles/making-analog-training-scale-co-designing-mapping-optimizer-and-converters.