GitHub

Adaptive Proximal Gradient Descent Methods

This package implements several variants of adaptive proximal gradient descent methods. Their primary use is internal to FrankWolfe.jl, specifically for the Blended Conditional Gradients algorithm, but they can also be used as standalone algorithms.

For now however, they remain in the Experimental submodule to highlight that they may be subject to breaking changes without changing the major version.

FrankWolfe.Experimental.adaptive_gradient_descent — Method
adaptive_gradient_descent(f, grad!, x0; kwargs...)

Adaptive gradient descent algorithm that automatically adjusts the step size based on local Lipschitz estimates.

Arguments

  • f: Objective function
  • grad!: In-place gradient function
  • x0: Initial point
  • step0: Initial step size (default: 1.0)
  • max_iteration: Maximum number of iterations (default: 10000)
  • epsilon: Tolerance for stopping criterion (default: 1e-7)
  • callback: Optional callback function (default: nothing)
  • verbose: Whether to print progress (default: false)
  • memory_mode: Memory emphasis mode (default: InplaceEmphasis())

Returns

Named tuple containing:

  • x: final iterate
  • primal: final objective value
  • status: ExecutionStatus with which the algorithm terminated
  • traj_data: vector of states from callback
source
FrankWolfe.Experimental.adaptive_gradient_descent2 — Method
adaptive_gradient_descent2(f, grad!, x0; kwargs...)

Second variant of adaptive gradient descent with modified step size adaptation.

Takes the same arguments as adaptive_gradient_descent.

Returns

Named tuple containing:

  • x: final iterate
  • primal: final objective value
  • status: ExecutionStatus with which the algorithm terminated
  • traj_data: vector of states from callback
source
FrankWolfe.Experimental.proximal_adaptive_gradient_descent — Function
proximal_adaptive_gradient_descent(f, grad!, prox, x0; kwargs...)

Proximal variant of adaptive gradient descent that includes a proximal operator in the update step.

Arguments

  • f: Objective function
  • grad!: In-place gradient function
  • prox: Proximal operator (default: identity)
  • x0: Initial point
  • step0: Initial step size (default: 1.0)
  • max_iteration: Maximum number of iterations (default: 10000)
  • epsilon: Tolerance for stopping criterion (default: 1e-7)
  • callback: Optional callback function (default: nothing)
  • verbose: Whether to print progress (default: false)
  • memory_mode: Memory emphasis mode (default: InplaceEmphasis())

Returns

Named tuple containing:

  • x: final iterate
  • primal: final objective value
  • status: ExecutionStatus with which the algorithm terminated
  • traj_data: vector of states from callback
source