A Single Time-Scale Stochastic Approximation Method for Nested Stochastic Optimization
In this talk, we consider constrained nested stochastic optimization problems in which the objective function is a composition of two smooth functions whose exact values and derivatives are not available. We propose a single time-scale stochastic approximation algorithm, which we call the Nested Averaged Stochastic Approximation (NASA), to find an approximate stationary point of the
problem. We show that NASA achieves the sample complexity of O(1/\epsilon^2) for finding an \epsilon-approximate stationary point, thus outperforming all extant methods for nested stochastic approximation. Our method and its analysis are the same for both unconstrained and constrained problems, without any need of batch samples makes the algorithm attractive for online learning where the samples are received one by one.