Merge branch 'develop' into adjust-test

docs/installation.rst

@@ -1178,7 +1178,7 @@ options and different versions of the application. A full list can be displayed
     ppconvert [off]         on, off                 Install with pseudopotential
                                                     converter.
     qe [on]                 on, off                 Install with patched Quantum
-                                                    Espresso 6.4.0
+                                                    ESPRESSO 6.4.0
     timers [off]            on, off                 Build with support for timers
 
   Installation Phases:

nexus/examples/pyscf/01_h2o_hf/README.rst

@@ -8,7 +8,7 @@ The Nexus script for this example is ``h2o_ae_hf.py``.
 
 Similar to Nexus, the user interface to PySCF is in the form of pure 
 Python scripts.  Because a PySCF script can come in a wide variety of 
-custom forms (far exceeding that of a Quantum Espresso input file for 
+custom forms (far exceeding that of a Quantum ESPRESSO input file for 
 example), Nexus takes the conservative approach of only adding portions 
 to an existing user-written PySCF template script.
 
@@ -67,7 +67,7 @@ The Nexus script, ``h2o_ae_hf.py``, is shown below:
     
     run_project()
 
-The overall flow of the script should be familiar from the Quantum Espresso 
+The overall flow of the script should be familiar from the Quantum ESPRESSO 
 examples presented earlier.  If it is not, please revisit these examples before 
 proceeding.
 

nexus/examples/qmcpack/rsqmc_misc/estimators/README.rst

@@ -1,7 +1,7 @@
 Nexus Inputs for QMCPACK Estimators
 =====================================
 This example aims to illustrate Nexus inputs for all available QMCPACK estimators.
-``iron_ldaU_dmc.py`` file uses bcc-Fe structure and Quantum Espresso code for demonstration.
+``iron_ldaU_dmc.py`` file uses bcc-Fe structure and Quantum ESPRESSO code for demonstration.
 Only minor changes will be needed depending on the boundary conditions and the code generating the trial wave function.
 For each estimator, important information is provided in the Nexus workflow as comments.
 In addition, ``scripts_postproc`` directory includes some post-processing tools for these estimators.

nexus/examples/qmcpack/rsqmc_misc/estimators/iron_ldaU_dmc.py

@@ -94,7 +94,6 @@
     )
 
 optJ12 = generate_qmcpack(
-    driver               = 'legacy',
     identifier           = 'optJ12',
     path                 = basepath + 'optJ12',
     job                  = job(cores=16,threads=4,app='qmcpack'),
@@ -117,7 +116,6 @@
 
 J3_rcut = system.structure.rwigner()
 optJ123 = generate_qmcpack(
-    driver               = 'legacy',
     identifier           = 'optJ123',
     path                 = basepath + 'optJ123',
     job                  = job(cores=16,threads=4,app='qmcpack'),
@@ -184,7 +182,6 @@
 # Finally, note that for large numbers of blocks, the stat.h5 file's disk footprint can get considerably large.
 # This is especially true for dense grids, say spindensity with (300x300x300).
 qmc = generate_qmcpack(
-    driver               = 'legacy',
     #skip_submit          = True,
     identifier           = 'qmc',
     path                 = basepath + 'qmc',

nexus/examples/qmcpack/rsqmc_quantum_espresso/04_iron_dft_dmc_gcta/README.rst

@@ -63,3 +63,4 @@ Specifically, without this keyword, the twist averaged DMC energy is ``-123.9042
 An alternative way to run GCTA is to use ``gcta = 'afl'``, which uses an "adapted Fermi level" that guarantees charge neutrality but does not target the SCF magnetization.
 This can be useful in noncollinear calculations where a single total magnetization is not defined.
 Detailed description and performance of ``'safl'`` and ``'afl'`` can be found in `this article <https://pubs.acs.org/doi/10.1021/acs.jctc.4c00058>`_.
+Also see the Nexus documentation for the details of the ``gcta`` implementation.

nexus/examples/qmcpack/rsqmc_quantum_espresso/04_iron_dft_dmc_gcta/iron_ldaU_dmc_gcta.py

@@ -51,12 +51,9 @@
     pseudos        = pseudos,
     )
 
-# read the primitive cell
-structure = read_structure('Fe_bcc.xsf')
-structure.change_units('B')
-
+# define the physical system
 system = generate_physical_system(
-    structure  = structure.copy(),
+    structure  = 'Fe_bcc.xsf',
     kgrid      = (6,6,6),
     kshift     = (0,0,0),
     net_spin   = 6, # The net_spin to be used in Jastrow optimizations is chosen as an integer close to the SCF value of 5.66.
@@ -80,7 +77,7 @@
 
 nscf = generate_pwscf(
     identifier        = 'nscf',
-    path              = basepath + 'nscf',
+    path              = basepath + 'nscf', # A different path is currently required for nscf since gcta-safl requires scf magnetization, which gets overwritten otherwise
     job               = job(cores=16, app='pw.x'),
     calculation       = 'nscf',
     system            = system,
@@ -98,7 +95,6 @@
     )
 
 optJ12 = generate_qmcpack(
-    driver               = 'legacy',
     identifier           = 'optJ12',
     path                 = basepath + 'optJ12',
     job                  = job(cores=16,threads=4,app='qmcpack'),
@@ -107,11 +103,12 @@
     qmc                  = 'opt',
     J2                   = True,
     minmethod            = 'oneshift',
-    warmupsteps          = 10,
     init_cycles          = 5,
     cycles               = 5,
-    blocks               = 10,
-    steps                = 50,
+    walkers_per_rank     = 8,
+    warmupsteps          = 10,
+    samples              = 2e5,
+    blocks               = 100,
     minwalkers           = 0.3,
     corrections          = [],
     dependencies         = (p2q,'orbitals'),
@@ -120,7 +117,6 @@
 
 J3_rcut = system.structure.rwigner()
 optJ123 = generate_qmcpack(
-    driver               = 'legacy',
     identifier           = 'optJ123',
     path                 = basepath + 'optJ123',
     job                  = job(cores=16,threads=4,app='qmcpack'),
@@ -131,11 +127,12 @@
     J3                   = True,
     J3_rcut              = J3_rcut,
     minmethod            = 'oneshift',
-    warmupsteps          = 10,
     init_cycles          = 0,
     cycles               = 10,
-    blocks               = 10,
-    steps                = 50,
+    walkers_per_rank     = 8,
+    warmupsteps          = 10,
+    samples              = 2e5,
+    blocks               = 100,
     minwalkers           = 0.5,
     corrections          = [],
     dependencies         = [(p2q,'orbitals'), (optJ12,'jastrow')],
@@ -148,7 +145,6 @@
 
 qmc = generate_qmcpack(
     #skip_submit          = True,
-    driver               = 'legacy',
     identifier           = 'qmc',
     path                 = basepath + 'qmc',
     job                  = job(cores=12,threads=4,app='qmcpack_complex'),

nexus/examples/quantum_espresso/01_diamond_scf/README.rst

@@ -2,7 +2,7 @@ Nexus QE+QMCPACK Example 1: Diamond primitive cell (DFT only)
 =============================================================
 
 In this example, we give an introduction to Nexus by using it to run a simple 
-system (diamond) with Quantum Espresso.  The Nexus script we will use is 
+system (diamond) with Quantum ESPRESSO.  The Nexus script we will use is 
 ``diamond_lda.py`` in the current directory.
 
 If you have not used Nexus before, we recommend you briefly read the 

nexus/examples/quantum_package/01_h2o_hf/README.rst

@@ -56,7 +56,7 @@ the script are shown below:
     
     run_project()
 
-The overall flow of the script should be familiar from the Quantum Espresso 
+The overall flow of the script should be familiar from the Quantum ESPRESSO 
 examples presented earlier.  If it is not, please revisit these examples before 
 proceeding.
 

nexus/sphinx_docs/bibs/methods.bib

@@ -0,0 +1,7 @@
+@article{annaberdiyev_enhanced_2024,
+  author = {Annaberdiyev, Abdulgani and Ganesh, Panchapakesan and Krogel, Jaron T.},
+  journal = {Journal of Chemical Theory and Computation},
+  title = {Enhanced {{Twist-Averaging Technique}} for {{Magnetic Metals}}: {{Applications Using Quantum Monte Carlo}}},
+  year = {2024},
+  doi = {10.1021/acs.jctc.4c00058},
+}

nexus/sphinx_docs/conf.py

@@ -36,7 +36,7 @@
 import sphinx_rtd_theme
 
 extensions = ['sphinxcontrib.bibtex', "sphinx_rtd_theme"]
-bibtex_bibfiles = []
+bibtex_bibfiles = ['bibs/methods.bib']
 
 numfig = True
 

nexus/sphinx_docs/installation.rst

@@ -113,7 +113,7 @@ excited state calculations with QMCPACK.
 
 Of course, to run full calculations, the simulation codes and converters
 involved must be installed as well. These include a modified version of
-Quantum Espresso (``pw.x``, ``pw2qmcpack.x``, optionally
+Quantum ESPRESSO (``pw.x``, ``pw2qmcpack.x``, optionally
 ``pw2casino.x``), QMCPACK (``qmcpack``, ``qmcpack_complex``,
 ``convert4qmc``, ``wfconvert``, ``ppconvert``), SQD (``sqd``, packaged
 with QMCPACK), VASP, and/or GAMESS. Complete coverage of this task is

nexus/sphinx_docs/overview.rst

@@ -9,7 +9,7 @@ What Nexus is
 Nexus is a collection of tools, written in Python, to perform
 complex electronic structure calculations and analyze the results.  The main
 focus is currently on performing arbitrary Quantum Monte Carlo (QMC)
-calculations with QMCPACK; however, VASP, Quantum Espresso, and GAMESS are
+calculations with QMCPACK; however, VASP, Quantum ESPRESSO, and GAMESS are
 also supported.  A single QMC calculation typically requires several
 previous calculations with other codes to produce a starting guess for the
 many-body wavefunction and convert it into a form that QMCPACK understands.
@@ -24,7 +24,7 @@ What Nexus can do
 
 The capabilities of Nexus currently include crystal structure
 generation, standalone Density Functional Theory (DFT) calculations with
-PWSCF (Quantum Espresso) or VASP,  quantum chemical calculations with GAMESS,
+PWSCF (Quantum ESPRESSO) or VASP,  quantum chemical calculations with GAMESS,
 Hartree-Fock (HF) calculations of atoms with the SQD code (packaged with
 QMCPACK), complete QMC calculations with QMCPACK (including wavefunction
 optimization, Variational Monte Carlo (VMC), and Diffusion Monte Carlo (DMC) in

nexus/sphinx_docs/reading.rst

@@ -44,7 +44,7 @@ PWSCF: pw.x, pw2qmcpack.x, pw2casino.x
   http://qmcpack.org/downloads/
 | Installation instructions: See the ``README`` file in
   ``qmcpack/external_codes/quantum_espresso`` for instructions on
-  patching Quantum Espresso version 5.1.
+  patching Quantum ESPRESSO version 5.1.
 
 QMCPACK: qmcpack, qmcpack_complex, convert4qmc, ppconvert, sqd
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

nexus/sphinx_docs/theory.rst

@@ -97,10 +97,10 @@ potentially poor starting guess for particle positions). In DMC, the
 walker population is a dynamic feature of the calculation and must be
 large enough to avoid introducing bias in expectation values.
 
-The tradeoff of moving to a the sampling procedure for the integration
+The tradeoff of moving to a sampling procedure for the integration
 is that it introduces statistical error into the calculation which
 diminishes slowly with the number of samples (it falls off like
-:math:`1/(\# of samples)` by the Central Limit Theorem). The good news
+:math:`1/\sqrt{\text{# of samples}}` by the Central Limit Theorem). The good news
 for ground state QMC is that this error can be reduced more rapidly
 through the discovery of better guesses at the detailed nature of the
 many-body wavefunction.

nexus/sphinx_docs/user-scripts.rst

@@ -263,7 +263,7 @@ atomic/electronic structure to simulate:
 
 The simulation objects created in this way are just data.  They represent requests for particular simulations to be carried out at a later time.  No simulation runs are actually performed during the creation of these objects. A basic example of generation input for each of the four major codes currently supported by Nexus is given below.
 
-Quantum Espresso (PWSCF) generation:
+Quantum ESPRESSO (PWSCF) generation:
 ************************************
 
 ::
@@ -588,3 +588,92 @@ manner.
     mean            = -75.0484800012
     sample_variance = 0.00645881103012
     variance        = 0.850521272106
+
+
+
+.. _user-occupations:
+
+Twist occupation specification
+------------------------------
+
+Gapped systems
+^^^^^^^^^^^^^^
+
+For QMC calculations involving twist-averaging, Nexus can set the desired twist occupations for the spin up and down electron channels.
+By default, the spin up and down occupations for each twist will be equal (non-magnetic phase) and each twist will be charge neutral.
+If a ferromagnetic type phase is required in gapped systems, ``net_spin`` argument can be specified in the ``generate_physical_system()`` function.
+This will result in each twist having magnetization equal to ``net_spin`` while still preserving charge neutrality in each twist.
+Specifying ``net_spin`` is sufficient for gapped systems since the charge and the net spin should not vary from twist to twist.
+
+Metallic systems
+^^^^^^^^^^^^^^^^
+
+The charge and the net spin are expected to vary from twist to twist in metallic systems in general.
+In Nexus, this can be handled via grand-canonical twist-averaging (GCTA), by specifying the ``gcta`` argument in the ``generate_qmcpack()`` function.
+The ``gcta`` argument can take the values given in :ref:`Table 2 <table2>`.
+Currently, only workflows that use Quantum ESPRESSO (PWSCF) are supported by ``gcta``.
+
+.. _table2:
+
+.. table:: Available GCTA flavors in Nexus and their descriptions. The "charge-neutrality" indicates if the GCTA flavor guarantees :math:`\sum_{i}^{N} q_i = 0` where :math:`i` is the twist index.
+
+   +------------+-----------------------+----------------------+---------------+--------------------------+
+   | **gcta**   | **charge-neutrality** | **k-point symmetry** | **spinors**   | **description**          |
+   +============+=======================+======================+===============+==========================+
+   | ``'nscf'`` |          No           | Supported            | Supported     | NSCF Fermi level         |
+   +------------+-----------------------+----------------------+---------------+--------------------------+
+   | ``'scf'``  |          No           | Supported            | Supported     | SCF Fermi level          |
+   +------------+-----------------------+----------------------+---------------+--------------------------+
+   | ``'afl'``  |          Yes          | Not supported        | Supported     | Adapted Fermi level      |
+   +------------+-----------------------+----------------------+---------------+--------------------------+
+   | ``'safl'`` |          Yes          | Not supported        | Not supported | Spin-adapted Fermi level |
+   +------------+-----------------------+----------------------+---------------+--------------------------+
+
+**Additional information**:
+
+- ``'nscf'``: Use the non-self-consistent-filed (NSCF) Fermi level to determine the twist occupations.
+  Nexus will attempt to traceback one level in PWSCF dependencies to read the NSCF Fermi level from the ``pwscf_output/pwscf.xml`` file.
+- ``'scf'``: Use the self-consistent-filed (SCF) Fermi level to determine the twist occupations.
+  Nexus will attempt to traceback two levels in PWSCF dependencies to read the SCF Fermi level from the ``pwscf_output/pwscf.xml`` file.
+  Due to this, the NSCF simulation should be in a separate path from the SCF simulation to avoid overwriting.
+- ``'afl'``: Use an adapted Fermi level determined from the available sorted eigenvalues that will give a net charge-neutral system :cite:`annaberdiyev_enhanced_2024`.
+  The Fermi level is determined solely from the eigenvalues in ``pwscf.pwscf.h5``.
+  Therefore, Nexus will not attempt to traceback anything in dependencies in this case.
+- ``'safl'``: Use a spin-adapted Fermi level determined by sorting each spin channel separately and using the SCF magnetization as target magnetization, while achieving net charge-neutrality :cite:`annaberdiyev_enhanced_2024`.
+  Since it requires the SCF magnetization, Nexus will attempt to traceback two levels in PWSCF dependencies to read the total magnetization from the ``pwscf_output/pwscf.xml`` file.
+  Due to this, the NSCF simulation should be in a separate path from the SCF simulation to avoid overwriting.
+
+**Caution**:
+
+- Note that the ``net_spin`` will be overwritten by the ``gcta`` for the specified QMC simulation object.
+  This is because the net magnetization of the system is now defined by the Fermi level.
+  For example, in ``'afl'``, the net magnetization is uniquely determined by the single-particle eigenvalues.
+  In ``'safl'``, the net magnetization is chosen as close as possible to the SCF magnetization.
+- Note that ``gcta`` flavors ``'scf'`` and ``'nscf'`` do not guarantee a net charge-neutrality.
+  They were implemented for comparison and research purposes only.
+  Therefore, these are not likely to be useful in production runs.
+
+**Current limitations**:
+
+- The ``'afl'`` and ``'safl'`` options require a twist-averaging without k-point symmetry, i.e., using the full **k**-points with equal weights instead of the reduced **k**-points with varying weights.
+  The reason is that the existence of weights in **k**-points can prevent determining a Fermi level that will result in a charge-neutral system.
+  Effectively, it requires fractional charges in twists, which is not nominally possible in QMCPACK.
+  However, it could be possible to achieve net charge-neutrality by adding extra **k**-points with certain required charges.
+  This is currently not implemented.
+- Currently, the twist occupations are calculated by first determining the Fermi level requested by ``gcta`` flavor and then occupying all single-particle eigenvalues below this energy (:math:`e_i < E_F`).
+  Some initial uses of ``gcta`` showed issues with eigenvalue degeneracies near the Fermi level, leading to a net non-zero charge.
+  In most cases, this is not an issue due to the residual convergence error in NSCF, which effectively acts as a small randomizer of eigenvalues.
+  However, it is possible to avoid this issue altogether, which requires re-implementation of ``gcta``.
+  In this new potential version, a table of sorted eigenvalues and the **k**-points they folded from are kept together.
+  Then, since the number of electrons that needs to be occupied is known, the occupations can be set for each **k**-point and their folded twists.
+  It avoids the (:math:`e_i < E_F`) comparison which is numerically problematic if there are eigenvalue degeneracies.
+- Due to the hard-coded nature of the limited dependency traceback capability implemented in ``gcta``, hybrid functionals in PWSCF will currently not work with ``safl`` and ``scf``.
+  This is because NSCF calculations are not possible with the hybrid functionals, requiring a direct (``scf`` -> ``pw2qmcpack``) instead of (``scf`` -> ``nscf`` -> ``pw2qmcpack``).
+- ``gcta`` argument will currently not work if there is a single twist in the system (it only activates when there are multiple twists).
+- ``gcta`` currently supports only Quantum ESPRESSO (PWSCF).
+
+Please contact the developers if any of these issues are critical for research.
+
+
+
+.. bibliography:: bibs/methods.bib

tests/solids/diamondC_2x1x1-Gaussian_pp_MSD/CMakeLists.txt

@@ -27,7 +27,7 @@ list(APPEND diamondC_2x1x1-Gaussian_pp_MSD_SHORT_SCALARS "totenergy" "-21.341584
 list(APPEND diamondC_2x1x1-Gaussian_pp_MSD_SHORT_SCALARS "kinetic" "19.99861447 0.207280984944")
 list(APPEND diamondC_2x1x1-Gaussian_pp_MSD_SHORT_SCALARS "potential" "-41.34019869 0.234973849712")
 list(APPEND diamondC_2x1x1-Gaussian_pp_MSD_SHORT_SCALARS "eeenergy" "-4.48998595 0.1")
-list(APPEND diamondC_2x1x1-Gaussian_pp_MSD_SHORT_SCALARS "localecp" "-13.06854246 0.15")
+list(APPEND diamondC_2x1x1-Gaussian_pp_MSD_SHORT_SCALARS "localecp" "-13.06854246 0.2")
 list(APPEND diamondC_2x1x1-Gaussian_pp_MSD_SHORT_SCALARS "nonlocalecp" "2.05740964 0.077946495184")
 list(APPEND diamondC_2x1x1-Gaussian_pp_MSD_SHORT_SCALARS "ionion" "-25.55132719 0.000003")
 qmc_run_and_check(