Edit on GitHub

Adaptive Proximal Gradient Descent Methods

This package implements several variants of adaptive proximal gradient descent methods. Their primary use is internal to FrankWolfe.jl, specifically for the Blended Conditional Gradients algorithm, but they can also be used as standalone algorithms.

FrankWolfe.adaptive_gradient_descent — Method
adaptive_gradient_descent(f, grad!, x0; kwargs...)

Adaptive gradient descent algorithm that automatically adjusts the step size based on local Lipschitz estimates.

Arguments

  • f: Objective function
  • grad!: In-place gradient function
  • x0: Initial point
  • step0: Initial step size (default: 1.0)
  • max_iteration: Maximum number of iterations (default: 10000)
  • epsilon: Tolerance for stopping criterion (default: 1e-7)
  • callback: Optional callback function (default: nothing)
  • verbose: Whether to print progress (default: false)
  • memory_mode: Memory emphasis mode (default: InplaceEmphasis())

Returns

Tuple containing:

  • Final iterate
  • Final objective value
  • Vector of states from callback
source
FrankWolfe.adaptive_gradient_descent2 — Method
adaptive_gradient_descent2(f, grad!, x0; kwargs...)

Second variant of adaptive gradient descent with modified step size adaptation.

Takes the same arguments as adaptive_gradient_descent.

source
FrankWolfe.proximal_adaptive_gradient_descent — Function
proximal_adaptive_gradient_descent(f, grad!, prox, x0; kwargs...)

Proximal variant of adaptive gradient descent that includes a proximal operator in the update step.

Arguments

  • f: Objective function
  • grad!: In-place gradient function
  • prox: Proximal operator (default: identity)
  • x0: Initial point
  • step0: Initial step size (default: 1.0)
  • max_iteration: Maximum number of iterations (default: 10000)
  • epsilon: Tolerance for stopping criterion (default: 1e-7)
  • callback: Optional callback function (default: nothing)
  • verbose: Whether to print progress (default: false)
  • memory_mode: Memory emphasis mode (default: InplaceEmphasis())

Returns

Tuple containing:

  • Final iterate
  • Final objective value
  • Vector of states from callback
source