الباحثون

Ziye Ma

المنشورات 2

نسخة أولية وصول مفتوح

A Fine-Grained Analysis of the LoRA Fine-Tuning Landscape with Implications for Data Selection

Bowen Zhang, Changrui Fang, Xinsong Ma وآخرون · 2026

Low-Rank Adaptation (LoRA) has become a standard approach for parameter-efficient fine-tuning, yet a fundamental practical question remains unresolved: how should the adapter rank be chosen? An overly small rank may lead to a poorly conditioned optimization landscape, whereas an unnecessarily large rank sacrifices the …

نسخة أولية وصول مفتوح

Understanding the Weight Averaging Mechanism in LLM Training for Post-Training Quantization

Hanzhang Wang, Tianqi Shen, Zonglin Liu وآخرون · 2026

Large language models (LLMs) are typically pretrained in high precision but increasingly deployed with low-precision post-training quantization (PTQ). Recent studies have shown that using weight averaging during pretraining can improve PTQ performance compared with learning-rate decay, suggesting that it might provide …

المؤلفون المشاركون