الملخص
تمت ترجمة أجزاء من هذه الصفحة آلياً وقد تحتوي على أخطاء.
Neural networks trained on modular addition tasks often develop Fourier-structured representations that support exact generalization. While prior work has identified these Fourier circuits, the mechanism by which gradient-based training selects them from the data distribution remains unclear. We address this question using probability signatures, which express leading gradient interactions through conditional statistics of the training distribution. For modular addition, these signatures are cyclic shift operators and are diagonalized by the discrete Fourier transform, yielding approximately decoupled Fourier-mode dynamics. This explains the emergence of Fourier sparsity, frequency matching, and phase alignment. The same framework resolves a puzzle under label noise: corrupted examples can show faster early loss decrease than clean examples, despite lacking a coherent generalization rule. We show that noise increases conditional label collisions, strengthening early shared-coordinate reinforcement. Finally, this method can be applied to other operators. Taking XOR as an example, we observed the predicted frequency in experiments.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Wang, Y., Yao, J., Liu, L., Lu, P., & Xu, Z. Q. J. (2026). ديناميكيات بصمة الاحتمال: كشف تعلم الجمع المعياري داخل الشبكات ذات الطبقتين. https://omanscience.com/ar/articles/probability-signature-dynamics-unpacking-modular-addition-learning-within-two-layer-networks
MLA 9
Wang, Yunji, et al. "ديناميكيات بصمة الاحتمال: كشف تعلم الجمع المعياري داخل الشبكات ذات الطبقتين." https://omanscience.com/ar/articles/probability-signature-dynamics-unpacking-modular-addition-learning-within-two-layer-networks.
شيكاغو (المؤلف–التاريخ)
Wang, Yunji, Junjie Yao, Linyu Liu, Pinyan Lu, and Zhi-Qin John Xu. 2026. "ديناميكيات بصمة الاحتمال: كشف تعلم الجمع المعياري داخل الشبكات ذات الطبقتين." https://omanscience.com/ar/articles/probability-signature-dynamics-unpacking-modular-addition-learning-within-two-layer-networks.
هارفارد
Wang, Y., Yao, J., Liu, L., Lu, P. and Xu, Z. Q. J. (2026) 'ديناميكيات بصمة الاحتمال: كشف تعلم الجمع المعياري داخل الشبكات ذات الطبقتين', Available at: https://omanscience.com/ar/articles/probability-signature-dynamics-unpacking-modular-addition-learning-within-two-layer-networks.
فانكوفر
Wang Y, Yao J, Liu L, Lu P, Xu ZQJ. ديناميكيات بصمة الاحتمال: كشف تعلم الجمع المعياري داخل الشبكات ذات الطبقتين. https://omanscience.com/ar/articles/probability-signature-dynamics-unpacking-modular-addition-learning-within-two-layer-networks
IEEE
Y. Wang, J. Yao, L. Liu, P. Lu, and Z. Q. J. Xu, "ديناميكيات بصمة الاحتمال: كشف تعلم الجمع المعياري داخل الشبكات ذات الطبقتين," https://omanscience.com/ar/articles/probability-signature-dynamics-unpacking-modular-addition-learning-within-two-layer-networks.