Optimizers Supported

The table below contains a list of different optimzers that DESC is interfaced to. New optimizers can be added easily, see Adding new optimizers.

• Name : Name of the optimizer method. Pass this string to desc.optimze.Optimizer to use the method.

• Scalar : Whether the method assumes a scalar residual, or a vector of residuals for least squares.

• Equality constraints : Whether the method handles equality constraints.

• Inequality constraints : Whether the method handles inequality constraints.

• Stochastic : Whether the method can handle noisy objectives.

• Hessian : Whether the method requires calculation of the full hessian matrix.

• GPU : Whether the method supports running on GPU

• Description : Short description of the optimizer method.

List of Optimizers

Name	Scalar	Equality Constraints	Inequality Constraints	Stochastic	Hessian	GPU	Description
fmin-auglag	True	True	True	False	True	True	Augmented Lagrangian method with trust region subproblem.
fmin-auglag-bfgs	True	True	True	False	False	True	Augmented Lagrangian method with trust region subproblem. Uses BFGS to approximate hessian
lsq-auglag	False	True	True	False	False	True	Least Squares Augmented Lagrangian approach to constrained optimization
lsq-exact	False	False	False	False	False	True	Trust region least squares method, similar to the trf method in scipy
fmin-exact	True	False	False	False	True	True	Trust region method using iterative cholesky method to exactly solve the trust region subproblem.
fmin-dogleg	True	False	False	False	True	True	Trust region method using Powell’s dogleg method to approximately solve the trust region subproblem.
fmin-subspace	True	False	False	False	True	True	Trust region method solving the subproblem over the 2d subspace spanned by the gradient and newton direction.
fmin-exact-bfgs	True	False	False	False	False	True	Trust region method using iterative cholesky method to exactly solve the trust region subproblem. Uses BFGS to approximate hessian
fmin-dogleg-bfgs	True	False	False	False	False	True	Trust region method using Powell’s dogleg method to approximately solve the trust region subproblem. Uses BFGS to approximate hessian
fmin-subspace-bfgs	True	False	False	False	False	True	Trust region method solving the subproblem over the 2d subspace spanned by the gradient and newton direction. Uses BFGS to approximate hessian
sgd	True	False	False	True	False	True	Stochastic gradient descent with Nesterov momentum
scipy-bfgs	True	False	False	False	False	False	BFGS quasi-newton method with line search. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-bfgs.html
scipy-CG	True	False	False	False	False	False	Nonlinear conjugate gradient method. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-cg.html
scipy-Newton-CG	True	False	False	False	True	False	Newton conjugate gradient method. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-newtoncg.html
scipy-dogleg	True	False	False	False	True	False	Trust region method with dogleg step. Requires the hessian to be positive definite. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-dogleg.html
scipy-trust-exact	True	False	False	False	True	False	Trust region method using ‘exact’ method to solve subproblem. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-trustexact.html
scipy-trust-ncg	True	False	False	False	True	False	Trust region method using conjugate gradient to solve subproblem. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-trustncg.html
scipy-trust-krylov	True	False	False	False	True	False	Trust region method using Kyrlov iterations to solve subproblem. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-trustkrylov.html
scipy-trf	False	False	False	False	False	False	Trust region least squares method. See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html
scipy-lm	False	False	False	False	False	False	Levenberg-Marquardt implicit trust region method. See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html
scipy-dogbox	False	False	False	False	False	False	Dogleg method with box shaped trust region. See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.least_squares.html
scipy-trust-constr	True	True	True	False	True	False	Trust region interior point method. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-trustconstr.html
scipy-SLSQP	True	True	True	False	False	False	Sequential least squares programming method. See https://docs.scipy.org/doc/scipy/reference/optimize.minimize-slsqp.html