Abstract
Deploying large language models (LLMs) on sensitive data via Retrieval-Augmented Generation (RAG) introduces severe privacy risks. Recent studies apply Differential Privacy (DP) to LLMs with RAG for formal privacy guarantees. However, existing DP-RAG frameworks rapidly exhaust the privacy budget. Although recent efforts attempt to save the budget by narrowing the retrieval scope or sparsifying private generation, these methods themselves cumulatively consume the budget, whereas they could actually rely merely on public information or at a negligible one-time privacy cost. This mismatch fails to align budget expenditure with the model's actual reliance on private data, causing substantial waste on operations that require no private access. To address this, we propose SparsePay-RAG, adopting "only pay what you must spend" as its core principle. Using public information as a zero-privacy prior, it charges the privacy budget only for the private increment. Specifically, SparsePay-RAG narrows the retrieval scope via public topic-guided clustering, adaptively controls private access frequency without privacy cost through isotonic cross-layer trajectory fitting, and compresses per-access budget via DP contrastive decoding. Under strong privacy constraints, experiments show SparsePay-RAG achieves superior privacy-utility trade-offs over baselines.
Keywords
Subject
Publication details
- Journal
- Not available
- Open access
- Green open access
Cite this article
APA 7
Sun, Z., Tian, Z., Fang, X., Ma, S., Zhang, J., Song, Y., & Li, D. (2026). Only Pay What You Must Spend: On-Demand Privacy Budget Payment for Differentially Private RAG. https://omanscience.com/en/articles/only-pay-what-you-must-spend-on-demand-privacy-budget-payment-for-differentially-private-rag
MLA 9
Sun, Zhonghao, et al. "Only Pay What You Must Spend: On-Demand Privacy Budget Payment for Differentially Private RAG." https://omanscience.com/en/articles/only-pay-what-you-must-spend-on-demand-privacy-budget-payment-for-differentially-private-rag.
Chicago (author–date)
Sun, Zhonghao, Zhiliang Tian, Xinyue Fang, Shuo Ma, Juhua Zhang, Yiping Song, and Dongsheng Li. 2026. "Only Pay What You Must Spend: On-Demand Privacy Budget Payment for Differentially Private RAG." https://omanscience.com/en/articles/only-pay-what-you-must-spend-on-demand-privacy-budget-payment-for-differentially-private-rag.
Harvard
Sun, Z., Tian, Z., Fang, X., Ma, S., Zhang, J., Song, Y. and Li, D. (2026) 'Only Pay What You Must Spend: On-Demand Privacy Budget Payment for Differentially Private RAG', Available at: https://omanscience.com/en/articles/only-pay-what-you-must-spend-on-demand-privacy-budget-payment-for-differentially-private-rag.
Vancouver
Sun Z, Tian Z, Fang X, Ma S, Zhang J, Song Y, et al. Only Pay What You Must Spend: On-Demand Privacy Budget Payment for Differentially Private RAG. https://omanscience.com/en/articles/only-pay-what-you-must-spend-on-demand-privacy-budget-payment-for-differentially-private-rag
IEEE
Z. Sun, Z. Tian, X. Fang, S. Ma, J. Zhang, Y. Song, and D. Li, "Only Pay What You Must Spend: On-Demand Privacy Budget Payment for Differentially Private RAG," https://omanscience.com/en/articles/only-pay-what-you-must-spend-on-demand-privacy-budget-payment-for-differentially-private-rag.