Unreproducible and likely wrong total energies with SFX2C PBC #2655

pavel-po · 2024-05-04T19:12:14Z

pavel-po
May 4, 2024

This issue may or may not be the same as #2199.

The example below gives very different energies for the initial guess and the first iterations for the newer (6d3b24b) and older (6eb7542) versions of PySCF. What is even worse, the energies of the newest version are not reproducible if the calculation is restarted.

The total energies of the newer version for the first few iterations (first run):

init E= 8192.07436114727
cycle= 1 E= -112084.929979568  delta_E= -1.2e+05  |g|= 8.77e+04  |ddm|= 3.08e+06
cycle= 2 E= -112086.904447416  delta_E= -1.97  |g|= 8.77e+04  |ddm|= 1.73e+05
cycle= 3 E= -112104.010516455  delta_E= -17.1  |g|= 8.77e+04  |ddm|= 1.24e+05
cycle= 4 E= -112096.119746549  delta_E= 7.89  |g|= 8.77e+04  |ddm|= 1.24e+05

The total energies of the newer version for the first few iterations (second run):

init E= 95887.7078129938
cycle= 1 E= -169952.657780885  delta_E= -2.66e+05  |g|= 6.69e+05  |ddm|= 3.16e+06
cycle= 2 E= -169948.000695332  delta_E= 4.66  |g|= 6.69e+05  |ddm|= 4.14e+05
cycle= 3 E= -169918.536572383  delta_E= 29.5  |g|= 6.69e+05  |ddm|= 5.76e+05
cycle= 4 E= -169921.588612998  delta_E= -3.05  |g|= 6.69e+05  |ddm|= 3.26e+05

The total energies of the older version for the first few iterations (first run):

init E= -11351.6141270972
cycle= 1 E= -11266.8858217425  delta_E= 84.7  |g|= 21.5  |ddm|= 2e+04
cycle= 2 E= -11191.9620108872  delta_E= 74.9  |g|= 52.7  |ddm|= 6.51e+03
cycle= 3 E= -11265.8576434756  delta_E= -73.9  |g|=   21  |ddm|= 7.07e+03
cycle= 4 E= -11278.9507825114  delta_E= -13.1  |g|= 4.72  |ddm|=  162

The total energies of the older version for the first few iterations (second run):

init E= -11351.6141270972
cycle= 1 E= -11266.8858217425  delta_E= 84.7  |g|= 21.5  |ddm|= 2e+04
cycle= 2 E= -11191.9620108872  delta_E= 74.9  |g|= 52.7  |ddm|= 6.51e+03
cycle= 3 E= -11265.8576434756  delta_E= -73.9  |g|=   21  |ddm|= 7.07e+03
cycle= 4 E= -11278.9507825114  delta_E= -13.1  |g|= 4.72  |ddm|=  162

The full example:

import numpy as np
from pyscf.pbc import gto, df, tools
from pyscf import gto as mgto
from pyscf.pbc import scf
import scipy.linalg as LA
from pyscf import lib
from pyscf.df import addons
import pyscf.lib.chkfile as chk
import os
import sys
import argparse

# Setup the cell with cc-pvdz-x2c (with removed g-functions) for Nd and cc-pvdz for Ni,O
cell = gto.M(
    a = '''
    3.92 0.0 0.0
    0.0 3.92 0.0
    0.0 0.0 3.28
    ''',
    atom = '''
    Nd 1.96 1.96 1.64
    Ni 0.0 0.0 0.0
    O  0.0 1.96 0.0
    O  1.96 0.0 0.0
    ''',
    unit = 'A',
    basis = {'Nd': mgto.parse('''
Nd    S
      5.919604E+07           3.1606400E-04         -1.7273200E-04          9.3794000E-05         -2.1204100E-05          1.1448100E-05         -6.2716200E-06          1.0492500E-05          0.0000000E+00
      1.302884E+07           4.4759500E-04         -2.4471600E-04          1.3292800E-04         -3.0051800E-05          1.6225200E-05         -8.8888000E-06          1.4883100E-05          0.0000000E+00
      3.571596E+06           1.1421100E-03         -6.2476300E-04          3.3947400E-04         -7.6768200E-05          4.1445900E-05         -2.2704900E-05          3.7959200E-05          0.0000000E+00
      1.087391E+06           2.2077100E-03         -1.2097500E-03          6.5826500E-04         -1.4888200E-04          8.0378800E-05         -4.4036300E-05          7.3816500E-05          0.0000000E+00
      3.612535E+05           4.6566000E-03         -2.5579000E-03          1.3940000E-03         -3.1561700E-04          1.7037400E-04         -9.3330100E-05          1.5571200E-04          0.0000000E+00
      1.286881E+05           9.4032000E-03         -5.1913300E-03          2.8408600E-03         -6.4345300E-04          3.4736200E-04         -1.9031800E-04          3.2013100E-04          0.0000000E+00
      4.871343E+04           1.9504100E-02         -1.0844900E-02          5.9613100E-03         -1.3543000E-03          7.3073700E-04         -4.0027100E-04          6.6426300E-04          0.0000000E+00
      1.944012E+04           4.0112500E-02         -2.2608700E-02          1.2563900E-02         -2.8574300E-03          1.5423500E-03         -8.4509000E-04          1.4324400E-03          0.0000000E+00
      8.127617E+03           8.1320500E-02         -4.6847500E-02          2.6394200E-02         -6.0520500E-03          3.2612900E-03         -1.7864400E-03          2.9342400E-03          0.0000000E+00
      3.540105E+03           1.5371100E-01         -9.2648200E-02          5.3961100E-02         -1.2443800E-02          6.7121900E-03         -3.6777600E-03          6.3212400E-03          0.0000000E+00
      1.598656E+03           2.4857600E-01         -1.6299000E-01          9.9806500E-02         -2.3594800E-02          1.2666900E-02         -6.9396700E-03          1.1166300E-02          0.0000000E+00
      7.451854E+02           2.8547700E-01         -2.2154000E-01          1.5008500E-01         -3.6305900E-02          1.9534500E-02         -1.0703600E-02          1.9129900E-02          0.0000000E+00
      3.554914E+02           1.7788500E-01         -1.4598600E-01          1.0190800E-01         -2.6126200E-02          1.3822000E-02         -7.5790100E-03          9.8994600E-03          0.0000000E+00
      1.619886E+02           7.8596800E-02          1.7943700E-01         -2.3796200E-01          7.5995600E-02         -3.9645700E-02          2.1775400E-02         -2.9349000E-02          0.0000000E+00
      8.121741E+01           8.7276000E-02          3.9542300E-01         -7.0467100E-01          2.5176600E-01         -1.3287400E-01          7.3000900E-02         -1.4195100E-01          0.0000000E+00
      4.122633E+01           5.0918500E-02          2.1540800E-01         -4.0962600E-01          1.7990700E-01         -9.3371000E-02          5.1692100E-02         -5.3824000E-02          0.0000000E+00
      1.924267E+01           3.7614000E-02          2.1802400E-01          6.8631400E-01         -5.7892800E-01          3.0962800E-01         -1.7355600E-01          2.3748400E-01          0.0000000E+00
      9.935612E+00           2.9955300E-02          1.7392000E-01          6.7067000E-01         -7.3358100E-01          4.2178400E-01         -2.3872600E-01          5.7817400E-01          0.0000000E+00
      4.236188E+00           2.7484400E-03          1.6291500E-02          2.7048600E-02          6.8401300E-01         -5.1253500E-01          3.0600700E-01         -1.0383500E+00          0.0000000E+00
      2.071560E+00          -8.3807300E-04         -6.1993300E-03         -5.2208200E-02          7.1901400E-01         -7.0622400E-01          4.5170900E-01         -3.9300200E-01          0.0000000E+00
      6.139873E-01           1.9802100E-05          1.7271800E-04         -3.1871800E-03          1.0688100E-01          6.7109400E-01         -5.5785100E-01          2.6542200E+00          0.0000000E+00
      2.782967E-01          -4.1095700E-05         -3.3276300E-04         -1.6764100E-03          3.3849100E-02          4.3685900E-01         -5.2093300E-01         -2.0978800E+00          0.0000000E+00
      4.831604E-02           4.8920100E-06          4.0837500E-05         -1.1948000E-04          9.8079600E-03          2.5010100E-01          7.8344200E-01         -6.4550500E-01          0.0000000E+00
      2.052529E-02          -3.9317300E-06         -3.2305100E-05         -9.3696400E-05          1.6541000E-03          9.8178500E-02          3.4748200E-01          1.0676700E+00          1.0000000E+00
Nd    P
      1.560092E+06           3.8761200E-05         -3.1949800E-05          8.7275600E-06         -4.2683500E-06          2.6061900E-06          9.5501500E-07          0.0000000E+00
      2.246696E+05           1.6986200E-04         -1.4020300E-04          3.8345300E-05         -1.8762800E-05          1.1470400E-05          4.1995500E-06          0.0000000E+00
      4.712701E+04           7.0585800E-04         -5.8402300E-04          1.5993400E-04         -7.8155800E-05          4.7651700E-05          1.7481400E-05          0.0000000E+00
      1.251877E+04           2.8391400E-03         -2.3567000E-03          6.4738700E-04         -3.1701300E-04          1.9418900E-04          7.0983000E-05          0.0000000E+00
      3.999338E+03           1.0858700E-02         -9.0839600E-03          2.5069900E-03         -1.2236100E-03          7.4472300E-04          2.7363700E-04          0.0000000E+00
      1.470515E+03           3.7794600E-02         -3.2036600E-02          8.9363900E-03         -4.3773900E-03          2.6868600E-03          9.8053900E-04          0.0000000E+00
      5.994803E+02           1.1078200E-01         -9.6765700E-02          2.7529200E-02         -1.3410700E-02          8.1532000E-03          2.9994900E-03          0.0000000E+00
      2.631965E+02           2.4369100E-01         -2.2255500E-01          6.5428500E-02         -3.2052600E-02          1.9757600E-02          7.1881300E-03          0.0000000E+00
      1.218109E+02           3.3769900E-01         -3.2293100E-01          9.7592900E-02         -4.7239300E-02          2.8483700E-02          1.0558500E-02          0.0000000E+00
      5.831698E+01           2.4287100E-01         -1.1110100E-01          6.3998000E-03         -4.1430300E-03          3.4883400E-03          9.3940000E-04          0.0000000E+00
      2.809497E+01           1.4722800E-01          4.1558000E-01         -2.7586100E-01          1.3861000E-01         -9.0023100E-02         -3.1684700E-02          0.0000000E+00
      1.400584E+01           1.2627900E-01          5.4458800E-01         -3.8299800E-01          1.8495600E-01         -1.1144300E-01         -4.1877500E-02          0.0000000E+00
      6.759591E+00           3.4091400E-02          1.3851700E-01          2.3696200E-01         -1.4190300E-01          9.2467800E-02          3.4252300E-02          0.0000000E+00
      3.309964E+00          -1.6608200E-03         -2.6553300E-02          6.6637000E-01         -4.5845200E-01          3.6289100E-01          1.1560700E-01          0.0000000E+00
      1.585821E+00          -2.9140600E-04         -9.0536100E-03          2.7513800E-01         -8.8417600E-02         -5.3179600E-02          1.3940500E-02          0.0000000E+00
      6.532364E-01          -2.7456500E-04         -2.3354300E-03          5.4555900E-02          6.1122800E-01         -7.8410000E-01         -2.0704700E-01          0.0000000E+00
      2.724449E-01           1.9746700E-05         -4.0707600E-04          3.2581800E-02          5.1320500E-01          2.3663900E-01         -2.1524700E-01          0.0000000E+00
      7.578313E-02          -2.4200200E-05         -1.6664200E-04          3.2865100E-03          5.2663300E-02          8.2101100E-01          4.3582800E-01          0.0000000E+00
      2.774380E-02           8.1265900E-06          4.6478300E-05         -4.7214500E-04         -7.4643800E-03          1.0752500E-01          6.9866800E-01          1.0000000E+00
Nd    D
      3.211514E+03           5.5872700E-04         -2.5216300E-04          6.6031700E-05          9.9071700E-05          0.0000000E+00
      8.869724E+02           4.7299000E-03         -2.1564600E-03          5.6912000E-04          9.1620400E-04          0.0000000E+00
      3.289876E+02           2.5628500E-02         -1.1716700E-02          3.0738300E-03          4.6250900E-03          0.0000000E+00
      1.403983E+02           9.4989500E-02         -4.4352900E-02          1.1740600E-02          1.8842700E-02          0.0000000E+00
      6.500852E+01           2.4025100E-01         -1.1310900E-01          2.9748000E-02          4.4332700E-02          0.0000000E+00
      3.139662E+01           3.8875600E-01         -1.7883100E-01          4.7480100E-02          8.0508900E-02          0.0000000E+00
      1.554173E+01           3.4617400E-01         -9.8094500E-02          2.2043800E-02          2.4502600E-02          0.0000000E+00
      7.598468E+00           1.1791300E-01          2.5538500E-01         -8.0075600E-02         -9.8228100E-02          0.0000000E+00
      3.638691E+00           4.2643300E-03          5.1821200E-01         -1.6479900E-01         -3.4884600E-01          0.0000000E+00
      1.676986E+00          -1.9997900E-03          3.3922100E-01         -4.7368700E-02          1.5852900E-03          0.0000000E+00
      6.639806E-01          -6.8299100E-04          5.6147500E-02          3.2523200E-01          1.0036200E+00          0.0000000E+00
      2.422670E-01           1.1367700E-04         -2.0324500E-03          5.6721500E-01         -3.6806500E-01          0.0000000E+00
      7.848798E-02          -3.0514900E-05          9.1825500E-04          3.2259600E-01         -5.8238500E-01          1.0000000E+00
Nd    F
      9.254739E+01           9.4011400E-03         -9.3255900E-03          0.0000000E+00
      3.085202E+01           5.7244000E-02         -5.7499200E-02          0.0000000E+00
      1.206868E+01           1.8383000E-01         -1.8220900E-01          0.0000000E+00
      5.055417E+00           3.4039600E-01         -3.1859500E-01          0.0000000E+00
      2.092477E+00           4.0491500E-01         -2.1296100E-01          0.0000000E+00
      8.127031E-01           3.0201400E-01          4.8926300E-01          0.0000000E+00
      2.996266E-01           1.2014000E-01          4.7963300E-01          0.0000000E+00
      1.054974E-01           1.7627200E-02          1.3410800E-01          1.0000000E+00
'''),
'Ni': 'cc-pvdz', 'O': 'cc-pvdz'},
    verbose = 7,
    spin = 32
)

cell.exp_to_discard = 0.1
cell.build()

kmesh = cell.make_kpts([2,2,2])


mydf   = df.GDF(cell)
mydf._prefer_ccdf = True
mydf.auxbasis = df.aug_etb(cell, beta=2.0)
mydf.kpts = kmesh
mydf._cderi_to_save = "cderi.h5"
mydf.build()
mydf._cderi = "cderi.h5"

mf = scf.KUHF(cell,kmesh).density_fit().sfx2c1e()
mf.kpts = mydf.kpts
mf.with_df = mydf
mf.diis_space = 20
mf.max_cycle = 100

mf.kernel()

Answered by sunqm

Aug 10, 2024

The reproduciblity in pyscf-2.5 may be due to the unconverged pnucp integrals under PBC lattice summation. You can try to adjust the cell.precision, to a very tight value, such as 1e-14.

The PBC-X2C may have a theoretical issue I'm not 100% sure how to identify at the moment. Roughly, the coulomb integrals in PBC rely on neutral charge of the system, i.e. a cancellation between the positive charge of nucleus and negative charge of electron density. When constructing the pnucp integrals in X2C 1-electron Hamiltonian, it does not have an appropriate electron density to "neutralize" this potential. It may be interpreted in a different way. To remove the singularity, the nuclear attraction in…

View full answer

sunqm · 2024-08-10T20:35:15Z

sunqm
Aug 10, 2024
Maintainer

The reproduciblity in pyscf-2.5 may be due to the unconverged pnucp integrals under PBC lattice summation. You can try to adjust the cell.precision, to a very tight value, such as 1e-14.

The PBC-X2C may have a theoretical issue I'm not 100% sure how to identify at the moment. Roughly, the coulomb integrals in PBC rely on neutral charge of the system, i.e. a cancellation between the positive charge of nucleus and negative charge of electron density. When constructing the pnucp integrals in X2C 1-electron Hamiltonian, it does not have an appropriate electron density to "neutralize" this potential. It may be interpreted in a different way. To remove the singularity, the nuclear attraction integrals in Hcore are computed with a uniform background negative charge. This background charge is also involved in the X2C transformation, leading to some ad hoc effects in the relativistic corrections. I think setting mf.with_x2c.approx = 'ATOM1E' would give a more reasonable and stable results.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Uh oh!

Unreproducible and likely wrong total energies with SFX2C PBC #2655

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Select a reply

Uh oh!

Uh oh!

Uh oh!

Unreproducible and likely wrong total energies with SFX2C PBC #2655

Uh oh!

pavel-po May 4, 2024

Replies: 1 comment

Uh oh!

Uh oh!

sunqm Aug 10, 2024 Maintainer

pavel-po
May 4, 2024

sunqm
Aug 10, 2024
Maintainer