First-Order Methods
Subgradient, proximal, accelerated, and stochastic gradient algorithms.
First-Order Methods. Subgradient, proximal, accelerated, and stochastic gradient algorithms.
Foundations and canonical references
The standard treatments of first-order methods approach the subject from complementary angles. Nesterov, Lectures on Convex Optimization (2018) is the anchor reference for the subject and lays out the core definitions, theorems, and worked examples that practitioners return to. Beck, First-Order Methods in Optimization (2017) gives a parallel, more proof-oriented exposition of the same material and is widely used as a graduate text.
Open methodological questions for first-order methods include sharpening the bridges between foundational theory and computational practice, extending classical results to broader or more structured settings, and integrating the techniques surveyed above with adjacent mathematical disciplines. The references listed in this page are the entry points that current work builds on.
Prerequisites
Sources
- textbook · primary · 2018Lectures on Convex Optimizationnesterov-2018
- textbook · primary · 2017First-Order Methods in Optimizationbeck-2017
In context
Where this topic sits in the prerequisite graph. Click any node to jump.
Review this topic
This page was drafted by an agent and is waiting on expert review. Spotted a wrong prerequisite, a missing concept, a misattributed source, or a factual slip? Tell us — your review opens a tracked issue maintainers act on.