Thanks to visit codestin.com
Credit goes to github.com

Skip to content

BUG: optimize: trust-constr gives not much more than single-precision accuracy on many simple well-conditioned problems #26130

Open
Open
BUG: optimize: trust-constr gives not much more than single-precision accuracy on many simple well-conditioned problems#26130
Labels
defectA clear bug or issue that prevents SciPy from being installed or used as expectedscipy.optimize

Description

@donhatch
donhatch
opened on Sep 8, 2026

Describe your issue.

[I originally asked about this on stackoverflow here]

I've been unable to get trust-constr to give much better than single-precision accuracy on many problems that are well-conditioned, well-scaled, and easily solvable to double precision.

This is even when I set gtol,xtol,barrier_tol aggressively, and I provide analytic jac and hess functions. (And I've fiddled with hundreds of other less-likely combinations of options as well, without success.)

A simple from-scratch solver using Newton's method and https://en.wikipedia.org/wiki/Lagrange_multiplier solves such problems reliably, with double-precision accuracy. I expect trust-constr to do the same, given analytic derivatives and tight tolerance settings.

The program below shows the simplest example problem I know which demonstrates this issue: find the local minimum of y=cos(x) with initial guess near x=3, with zero constraints. That local minimum occurs at x=pi, y=-1.

trust-constr finds the correct answer math.pi, with error -1.053671194739536e-08 (relative error -3.3539395807268046e-09), which is surprisingly poor (barely more than single precision).
I expect the relative error to be on the order of 1e-15 (double-precision accuracy) instead. Given the same information, a simple 3-line Newton's-method solver reliably produces exactly math.pi (i.e. pi correctly rounded to double-precision) on every initial guess in the vicinity, as expected.

Reproducing Code Example

#!/usr/bin/python3

import math
import scipy

# Find the local minimum of y=cos(x) near x=3, with zero constraints.
# This local minimum occurs at x=pi, y=-1.
minimize_result = scipy.optimize.minimize(
  fun=lambda x: math.cos(x[0]),
  jac=lambda x: [-math.sin(x[0])],
  hess=lambda x: [[-math.cos(x[0])]],
  x0=[2.931166344],  # varying x0 gives varying errors; this is the worst I found
  method='trust-constr',
  options={
    'gtol': 1e-20,  # aggressive gradient norm and constraint tolerance (default is 1e-8)
    'xtol': 1e-20,  # aggressive trust region radius tolerance (default is 1e-8)
    'barrier_tol': 1e-20,  # aggressive barrier parameter tolerance (default is 1e-8)
    'maxiter': 5000,  # default is 1000
    'verbose': 3,  # "display progress during iterations (more complete report)."
  },
)
print(f"minimize_result =\n{minimize_result}")
solution = float(minimize_result.x[0])
print(f"solution = {solution}")           # 3.141592643053081
print(f"expected = {math.pi} = math.pi")  # 3.141592653589793
error = solution - math.pi
print(f"error          = {error}")            # -1.053671194739536e-08 (poor accuracy)
print(f"relative error = {error / math.pi}")  # -3.3539395807268046e-09 (poor accuracy)

Output:

| niter |f evals|CG iter|  obj func   |tr radius |   opt    |  c viol  | penalty  |CG stop|
|-------|-------|-------|-------------|----------|----------|----------|----------|-------|
|   1   |   1   |   0   | -9.7794e-01 | 1.00e+00 | 2.09e-01 | 0.00e+00 | 1.00e+00 |   0   |
|   2   |   2   |   1   | -1.0000e+00 | 1.50e+00 | 3.16e-03 | 0.00e+00 | 1.00e+00 |   1   |
|   3   |   3   |   2   | -1.0000e+00 | 1.50e+00 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   1   |
|   4   |   4   |   3   | -1.0000e+00 | 1.50e-01 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   1   |
|   5   |   4   |   4   | -1.0000e+00 | 1.50e-02 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   1   |
|   6   |   4   |   5   | -1.0000e+00 | 1.50e-03 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   1   |
|   7   |   4   |   6   | -1.0000e+00 | 1.50e-04 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   1   |
|   8   |   4   |   7   | -1.0000e+00 | 1.50e-05 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   1   |
|   9   |   4   |   8   | -1.0000e+00 | 1.50e-06 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   1   |
|  10   |   4   |   9   | -1.0000e+00 | 1.50e-07 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   1   |
|  11   |   4   |  10   | -1.0000e+00 | 1.50e-08 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   1   |
|  12   |   4   |  11   | -1.0000e+00 | 7.48e-09 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   1   |
|  13   |   5   |  12   | -1.0000e+00 | 3.74e-09 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  14   |   6   |  13   | -1.0000e+00 | 1.87e-09 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  15   |   7   |  14   | -1.0000e+00 | 9.34e-10 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  16   |   8   |  15   | -1.0000e+00 | 4.67e-10 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  17   |   9   |  16   | -1.0000e+00 | 2.34e-10 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  18   |  10   |  17   | -1.0000e+00 | 1.17e-10 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  19   |  11   |  18   | -1.0000e+00 | 5.84e-11 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  20   |  12   |  19   | -1.0000e+00 | 2.92e-11 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  21   |  13   |  20   | -1.0000e+00 | 1.46e-11 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  22   |  14   |  21   | -1.0000e+00 | 7.30e-12 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  23   |  15   |  22   | -1.0000e+00 | 3.65e-12 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  24   |  16   |  23   | -1.0000e+00 | 1.83e-12 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  25   |  17   |  24   | -1.0000e+00 | 9.13e-13 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  26   |  18   |  25   | -1.0000e+00 | 4.56e-13 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  27   |  19   |  26   | -1.0000e+00 | 2.28e-13 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  28   |  20   |  27   | -1.0000e+00 | 1.14e-13 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  29   |  21   |  28   | -1.0000e+00 | 5.70e-14 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  30   |  22   |  29   | -1.0000e+00 | 2.85e-14 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  31   |  23   |  30   | -1.0000e+00 | 1.43e-14 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  32   |  24   |  31   | -1.0000e+00 | 7.13e-15 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  33   |  25   |  32   | -1.0000e+00 | 3.56e-15 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  34   |  26   |  33   | -1.0000e+00 | 1.78e-15 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  35   |  27   |  34   | -1.0000e+00 | 8.91e-16 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  36   |  28   |  35   | -1.0000e+00 | 4.46e-16 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  37   |  29   |  36   | -1.0000e+00 | 2.23e-16 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  38   |  29   |  37   | -1.0000e+00 | 1.11e-16 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  39   |  30   |  38   | -1.0000e+00 | 5.57e-17 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  40   |  30   |  39   | -1.0000e+00 | 2.78e-17 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  41   |  30   |  40   | -1.0000e+00 | 1.39e-17 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  42   |  30   |  41   | -1.0000e+00 | 6.96e-18 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  43   |  30   |  42   | -1.0000e+00 | 3.48e-18 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  44   |  30   |  43   | -1.0000e+00 | 1.74e-18 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  45   |  30   |  44   | -1.0000e+00 | 8.70e-19 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  46   |  30   |  45   | -1.0000e+00 | 4.35e-19 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  47   |  30   |  46   | -1.0000e+00 | 2.18e-19 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  48   |  30   |  47   | -1.0000e+00 | 1.09e-19 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  49   |  30   |  48   | -1.0000e+00 | 5.44e-20 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  50   |  30   |  49   | -1.0000e+00 | 2.72e-20 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  51   |  30   |  50   | -1.0000e+00 | 1.36e-20 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |
|  52   |  30   |  51   | -1.0000e+00 | 6.80e-21 | 1.05e-08 | 0.00e+00 | 1.00e+00 |   2   |

`xtol` termination condition is satisfied.
Number of iterations: 52, function evaluations: 30, CG iterations: 51, optimality: 1.05e-08, constraint violation: 0.00e+00, execution time: 0.0086 s.
minimize_result =
          message: `xtol` termination condition is satisfied.
          success: True
           status: 2
              fun: -1.0
                x: [ 3.142e+00]
              nit: 52
             nfev: 30
             njev: 3
             nhev: 3
         cg_niter: 51
     cg_stop_cond: 2
             grad: [-1.054e-08]
  lagrangian_grad: [-1.054e-08]
           constr: []
              jac: []
      constr_nfev: []
      constr_njev: []
      constr_nhev: []
                v: []
           method: equality_constrained_sqp
       optimality: 1.0536712069860041e-08
 constr_violation: 0
   execution_time: 0.008605480194091797
        tr_radius: 6.799005547317789e-21
   constr_penalty: 1.0
            niter: 52
solution = 3.141592643053081
expected = 3.141592653589793 = math.pi
error          = -1.053671194739536e-08
relative error = -3.3539395807268046e-09

SciPy/NumPy/Python version and system information

1.16.3 2.3.5 sys.version_info(major=3, minor=14, micro=4, releaselevel='final', serial=0)
Build Dependencies:
  blas:
    detection method: pkgconfig
    found: true
    include directory: /usr/include/x86_64-linux-gnu
    lib directory: /usr/lib/x86_64-linux-gnu
    name: blas
    openblas configuration: unknown
    pc file directory: /usr/lib/x86_64-linux-gnu/pkgconfig
    version: 3.12.1
  lapack:
    detection method: pkgconfig
    found: true
    include directory: /usr/include/x86_64-linux-gnu
    lib directory: /usr/lib/x86_64-linux-gnu
    name: lapack
    openblas configuration: unknown
    pc file directory: /usr/lib/x86_64-linux-gnu/pkgconfig
    version: 3.12.1
  pybind11:
    detection method: pkgconfig
    include directory: /include
    name: pybind11
    version: 3.0.1
Compilers:
  c:
    args: -g, -O2, -Werror=implicit-function-declaration, -fno-omit-frame-pointer,
      -mno-omit-leaf-frame-pointer, -ffile-prefix-map=/build/scipy-2N9o1u/scipy-1.16.3=.,
      -flto=auto, -ffat-lto-objects, -fstack-protector-strong, -fstack-clash-protection,
      -Wformat, -Werror=format-security, -fcf-protection, -fdebug-prefix-map=/build/scipy-2N9o1u/scipy-1.16.3=/usr/src/scipy-1.16.3-4build1,
      -Wdate-time, -D_FORTIFY_SOURCE=3
    commands: cc
    linker: ld.bfd
    linker args: -Wl,-Bsymbolic-functions, -Wl,--package-metadata=%7B%22type%22:%22deb%22%2C%22os%22:%22ubuntu%22%2C%22name%22:%22scipy%22%2C%22version%22:%221.16.3-4build1%22%2C%22architecture%22:%22amd64%22%7D,
      -flto=auto, -ffat-lto-objects, -Wl,-z,relro, -g, -O2, -Werror=implicit-function-declaration,
      -fno-omit-frame-pointer, -mno-omit-leaf-frame-pointer, -ffile-prefix-map=/build/scipy-2N9o1u/scipy-1.16.3=.,
      -flto=auto, -ffat-lto-objects, -fstack-protector-strong, -fstack-clash-protection,
      -Wformat, -Werror=format-security, -fcf-protection, -fdebug-prefix-map=/build/scipy-2N9o1u/scipy-1.16.3=/usr/src/scipy-1.16.3-4build1,
      -Wdate-time, -D_FORTIFY_SOURCE=3
    name: gcc
    version: 15.2.0
  c++:
    args: -g, -O2, -fno-omit-frame-pointer, -mno-omit-leaf-frame-pointer, -ffile-prefix-map=/build/scipy-2N9o1u/scipy-1.16.3=.,
      -flto=auto, -ffat-lto-objects, -fstack-protector-strong, -fstack-clash-protection,
      -Wformat, -Werror=format-security, -fcf-protection, -fdebug-prefix-map=/build/scipy-2N9o1u/scipy-1.16.3=/usr/src/scipy-1.16.3-4build1,
      -Wdate-time, -D_FORTIFY_SOURCE=3
    commands: c++
    linker: ld.bfd
    linker args: -Wl,-Bsymbolic-functions, -Wl,--package-metadata=%7B%22type%22:%22deb%22%2C%22os%22:%22ubuntu%22%2C%22name%22:%22scipy%22%2C%22version%22:%221.16.3-4build1%22%2C%22architecture%22:%22amd64%22%7D,
      -flto=auto, -ffat-lto-objects, -Wl,-z,relro, -g, -O2, -fno-omit-frame-pointer,
      -mno-omit-leaf-frame-pointer, -ffile-prefix-map=/build/scipy-2N9o1u/scipy-1.16.3=.,
      -flto=auto, -ffat-lto-objects, -fstack-protector-strong, -fstack-clash-protection,
      -Wformat, -Werror=format-security, -fcf-protection, -fdebug-prefix-map=/build/scipy-2N9o1u/scipy-1.16.3=/usr/src/scipy-1.16.3-4build1,
      -Wdate-time, -D_FORTIFY_SOURCE=3
    name: gcc
    version: 15.2.0
  cython:
    commands: cython
    linker: cython
    name: cython
    version: 3.1.6
  fortran:
    args: -fPIC
    commands: gfortran
    linker: ld.bfd
    linker args: -Wl,-Bsymbolic-functions, -Wl,--package-metadata=%7B%22type%22:%22deb%22%2C%22os%22:%22ubuntu%22%2C%22name%22:%22scipy%22%2C%22version%22:%221.16.3-4build1%22%2C%22architecture%22:%22amd64%22%7D,
      -flto=auto, -ffat-lto-objects, -Wl,-z,relro, -fPIC
    name: gcc
    version: 15.2.0
  pythran:
    include directory: ../../../../usr/lib/python3/dist-packages/pythran
    version: 0.18.1
Machine Information:
  build:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
  cross-compiled: false
  host:
    cpu: x86_64
    endian: little
    family: x86_64
    system: linux
Python Information:
  path: /usr/bin/python3.14
  version: '3.14'

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    defectA clear bug or issue that prevents SciPy from being installed or used as expectedscipy.optimize

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions