الملخص
Reinforcement learning based post training of diffusion models, such as Denoising Diffusion Policy Optimization (DDPO), optimizes a reverse diffusion process under a reward function. However, current approaches to reward optimizations do so at the cost of diversity and quality. In this paper, we provide better tradeoffs through careful theoretical considerations and method design. We analyze the theoretical framework and mathematically demonstrate that \emph{only-latter timestep} updates of diffusion model may be harmful for diversity contrary to the conclusions presented in a previous work. Additionally, we propose an incremental Feynman-Kac training based on strong theoretical foundations in order to achieve the best-yet alignment-diversity tradeoffs. We perform extensive experiments and compare our method against related diffusion policy optimization approaches in three different tasks and also provide strong ablations for each component, thus validating strong performance gains in both alignment and diversity.
الكلمات المفتاحية
الموضوع
بيانات النشر
- المجلة
- غير متاح
- وصول مفتوح
- وصول مفتوح أخضر
اقتبس هذه المقالة
APA 7
Neupane, A. P., Adhikari, S., Paudel, P., Chhatkuli, A., & Paudel, D. P. (2026). iADD: Improving Alignment and Diversity in Diffusion Policy Optimization. https://omanscience.com/ar/articles/iadd-improving-alignment-and-diversity-in-diffusion-policy-optimization
MLA 9
Neupane, Ashok Prasad, et al. "iADD: Improving Alignment and Diversity in Diffusion Policy Optimization." https://omanscience.com/ar/articles/iadd-improving-alignment-and-diversity-in-diffusion-policy-optimization.
شيكاغو (المؤلف–التاريخ)
Neupane, Ashok Prasad, Saugat Adhikari, Pramish Paudel, Ajad Chhatkuli, and Danda Pani Paudel. 2026. "iADD: Improving Alignment and Diversity in Diffusion Policy Optimization." https://omanscience.com/ar/articles/iadd-improving-alignment-and-diversity-in-diffusion-policy-optimization.
هارفارد
Neupane, A. P., Adhikari, S., Paudel, P., Chhatkuli, A. and Paudel, D. P. (2026) 'iADD: Improving Alignment and Diversity in Diffusion Policy Optimization', Available at: https://omanscience.com/ar/articles/iadd-improving-alignment-and-diversity-in-diffusion-policy-optimization.
فانكوفر
Neupane AP, Adhikari S, Paudel P, Chhatkuli A, Paudel DP. iADD: Improving Alignment and Diversity in Diffusion Policy Optimization. https://omanscience.com/ar/articles/iadd-improving-alignment-and-diversity-in-diffusion-policy-optimization
IEEE
A. P. Neupane, S. Adhikari, P. Paudel, A. Chhatkuli, and D. P. Paudel, "iADD: Improving Alignment and Diversity in Diffusion Policy Optimization," https://omanscience.com/ar/articles/iadd-improving-alignment-and-diversity-in-diffusion-policy-optimization.