Preprint Open access
A Single-Loop, Constant-Batch First-Order Penalty Method for Stochastic Bilevel Optimization
Recent advances in penalty-based methods for stochastic bilevel optimization (SBO) have eliminated the need for second-order derivative oracles. However, for stochastic nonconvex-strongly convex bilevel problems, existing first-order methods typically rely on nested loops and/or large batch sizes for attaining $O(ε^{-6 …