Machine Learning in Electronic-Structure Theory

Metal-organic frameworks (MOFs) are crystalline materials consisting of a metal node and an organic linker.¹ By combining different metal nodes and organic linkers, chemists can synthesize infinite materials² for applications ranging from gas separation, gas storage, sensing, catalysis, etc. This makes MOFs the ideal playground for data science. In this presentation we show how data science can support help designing MOFs for carbon capture applications. In  particular, we show how data-science methods^3,4 can be used to obtain insights into questions for which conventional theory does not have an answer, such as the oxidation state of the metal⁵ and the heat capacity of a MOF.⁶ In addition, we show how data science can help us identifying the characteristics of the top-performing materials for a carbon capture process. ⁷

References

¹                  H. Furukawa, K. E. Cordova, M. O’Keeffe, and O. M. Yaghi, The Chemistry and Applications of Metal-Organic Frameworks Science 341 (6149), 974 (2013) http://dx.doi.org/10.1126/Science.1230444

²                  S. Lee, B. Kim, H. Cho, H. Lee, S. Y. Lee, E. S. Cho, and J. Kim, Computational Screening of Trillions of Metal-Organic Frameworks for High-Performance Methane Storage Acs Appl Mater Inter 13 (20), 23647 (2021) http://dx.doi.org/10.1021/acsami.1c02471

³                  K. M. Jablonka, D. Ongari, S. M. Moosavi, and B. Smit, Big-Data Science in Porous Materials: Materials Genomics and Machine Learning Chem. Rev. 120 (16), 8066 (2020) http://dx.doi.org/10.1021/acs.chemrev.0c00004

⁴                  S. M. Moosavi, K. M. Jablonka, and B. Smit, The Role of Machine Learning in the Understanding and Design of Materials J. Am. Chem. Soc. 142 (48), 20273 (2020) http://dx.doi.org/10.1021/jacs.0c09105

⁵                  K. M. Jablonka, D. Ongari, S. M. Moosavi, and B. Smit, Using collective knowledge to assign oxidation states of metal cations in metal-organic frameworks Nat. Chem. 13, 771 (2021) http://dx.doi.org/10.1038/s41557-021-00717-y

⁶                  S. M. Moosavi, B. Á. Novotny, D. Ongari, E. Moubarak, M. Asgari, Ö. Kadioglu, C. Charalambous, A. Ortega-Guerrero, A. H. Farmahini, L. Sarkisov, S. Garcia, F. Noé, and B. Smit, A data-science approach to predict the heat capacity of nanoporous materials Nat Mater 21, 1419 (2022) http://dx.doi.org/10.1038/s41563-022-01374-3

⁷                  C. Charalambous, E. Moubarak, J. Schilling, E. Sanchez Fernandez, J.-Y. Wang, L. Herraiz, F. Mcilwaine, K. M. Jablonka, S. M. Moosavi, J. Van Herck, G. Mouchaham, C. Serre, A. Bardow, B. Smit, and S. Garcia, Shedding Light on the Stakeholders’ Perspectives for Carbon Capture. ChemRxiv  (2023) http://dx.doi.org/10.26434/chemrxiv-2023-sn90q

Electronic structure calculations usually begin with the choice of a truncated basis set for functions of a single space variable, followed by a Galerkin projection onto the associated antisymmetrized tensor product basis. This projection determines a second-quantized “quantum chemistry Hamiltonian” in which the problem details are encoded into one matrix and one four-index tensor, which respectively encode the single-electron and two-electron contributions to the energy. In particular, the four-index tensor of electron repulsion integrals (ERI) can be difficult to manage computationally. Diagonal basis sets—smooth, orthogonal basis sets that behave like delta functions—can be constructed to yield ERI that are highly structured in a way that simplifies many downstream calculations. However, these constructions are grid-based and can require a large basis set to yield acceptable accuracy. We describe two new approaches for constructing adaptive diagonal basis sets that conform to individual problem geometries. The first of these computes a highly accurate solution of a mass transport problem to define a grid deformation and then constructs a quantum chemistry Hamiltonian using pseudospectral insights that sidestep expensive numerical integrations. The second defines a nested generalization of the recently introduced gausslet basis set. This latter construction relies on new mathematical insights into diagonal 1D basis sets. (The approaches are based on collaborations with Sandeep Sharma and Steve White, respectively.)

I will discuss the details and challenges of using machine-learning to create new density functional approximations.   I plan to focus on the distinction between materials and molecules, the role of exact conditions, and the importance of using the density.  Relevant references are:

 The difference between molecules and materials: Reassessing the role of exact conditions in density functional theory Ryan Pederson and Kieron Burke, The Journal of Chemical Physics 159, 214113 (2023).
 Extending density functional theory with near chemical accuracy beyond pure water Suhwan Song, Stefan Vuckovic, Youngsam Kim, Hayoung Yu, Eunji Sim, and Kieron Burke, Nature Communications 14, 799 (2023).
 Machine learning and density functional theory Ryan Pederson, Bhupalee Kalita, and Kieron Burke, Nature Reviews Physics (2022).
 Improving Results by Improving Densities: Density-Corrected Density Functional Theory Eunji Sim, Suhwan Song, Stefan Vuckovic, and Kieron Burke, Journal of the American Chemical Society 144, 6625-6639 (2022).
 Calculation and interpretation of classical turning surfaces in solids Aaron D. Kaplan, Stewart J. Clark, Kieron Burke, and John P. Perdew, npj Computational Materials 7, 2057-3960 (2021)  

Discovery and understanding of next-generation materials requires a challenging combination of the high accuracy of first-principles calculations with the ability to reach large size and time scales. We pursue a multi-tier method development strategy in which machine learning (ML) algorithms are combined with exact physical symmetries and constraints to significantly accelerate computations of electronic structure and atomistic dynamics. First, density functional theory (DFT) is the cornerstone of modern computational materials science, but its current approximations fall short of the required accuracy and efficiency for predictive calculations of defect properties, band gaps, stability and electrochemical potentials of materials for energy storage and conversion. To advance the capability of DFT we introduce non-local charge density descriptors that satisfy exact scaling constraints and learn exchange functionals called CIDER [1]. These models are orders of magnitude faster in self-consistent calculations for solids than hybrid functionals but similar in accuracy. On a different level, we accelerate molecular dynamics (MD) simulations by using machine learning to capture the potential energy surfaces obtained from quantum calculations. We developed NequIP [2] and Allegro [3], the first deep equivariant neural network interatomic potential models, whose Euclidean symmetry-preserving layer architecture achieves state-of-the-art data efficiency and accuracy for simulating dynamics of molecules and materials. The machine learning architectures can also be combined with physical constraints to learn electronic polarization and Born effective charges which can be used to predict dielectric and spectroscopic properties of materials.

Transition metal systems and radical species pose a unique challenge in quantum chemistry due to the near-degeneracy of various electronic states, often defying description by a single electronic configuration. Multiconfigurational treatments, such as the complete active space self-consistent-field (CASSCF) method, are crucial for accurate results in these scenarios, particularly in systems with multiple interacting transition metals.

However, computational demands escalate dramatically when dealing with more than one metal center. In this talk, I will describe recent breakthroughs in overcoming the exponential scaling associated with the CASSCF method, enabling efficient treatment of complex systems with multiple transition metals.[1]

Moreover, multiconfigurational methods require careful user intervention in defining active spaces and orbital localization, precluding their use as black-box approaches, and limiting the availability of comprehensive datasets in the literature. To address this limitation, I will delve into our efforts to automate multireference methods for generating extensive datasets of excitation energies[2] and reactivity.[3] These datasets hold promise in training machine learning algorithms for enhanced predictive modeling in the challenging domain of strongly correlated systems. In the final part of the talk, I will outline our ongoing endeavor to automate the labeling of crucial features such as orbitals, which play a pivotal role in understanding chemical phenomena.

Neural network wavefunctions optimized using the variational Monte Carlo method have been shown to produce highly accurate results for the electronic structure of atoms and small molecules, but the high cost of optimizing such wavefunctions prevents their application to larger systems. We propose the Subsampled Projected-Increment Natural Gradient Descent (SPRING) optimizer to reduce this bottleneck. SPRING combines ideas from the recently introduced minimum-step stochastic reconfiguration optimizer (MinSR) and the classical randomized Kaczmarz method for solving linear least-squares problems. We demonstrate that SPRING outperforms both MinSR and the popular Kronecker-Factored Approximate Curvature method (KFAC) across a number of small atoms and molecules, given that the learning rates of all methods are optimally tuned. For example, on the oxygen atom, SPRING attains chemical accuracy after forty thousand training iterations, whereas both MinSR and KFAC fail to do so even after one hundred thousand iterations.

Based on work by Gil Goldshlager, N.A., Lin Lin

Cluster Expansion (CE) enables the modeling of the complex relationship, governed by quantum mechanics, between atomic structure and material properties in substitutional systems such as alloys or surface-adsorbate complexes. These models are crucial for the thermodynamic description of such systems, given the exponential increase in possible configurations with system size. In this talk, I will introduce two novel methods that leverage and extend CE for modelling temperature dependent properties and non-linear structure-property relationships. The first method enables the determination of the temperature-dependent electronic structure of complex alloys by fully accounting for the effects of short-range order, thereby going beyond traditional effective-medium theories like the virtual-crystal and coherent-potential approximations. I will showcase results demonstrating the insulator-to-metal transition in a complex clathrate alloy for thermoelectricity. The second method addresses a long-standing problem in CE: the lack of convergence for properties with non-linear dependence on the concentration of substituents. We introduce a straightforward approach that makes CE useful for the nonlinear modeling of materials properties. We do so by leveraging the data analytics aspects of CE, which can be interpreted as a machine learning problem, thereby extending its use also to classification tasks. The effectiveness of this method is demonstrated through its application in modeling highly non-linear properties of clathrates and perovskites, such as the band-gap and the formation energy.

Physical systems at all scales (from atoms to the cosmos) are naturally represented by tensor fields, i.e. tensor features at specific coordinates. This poses a challenge for machine learning due to the sensitivity of coordinates and features to the symmetries of the space the physical system inhabits (e.g. 3D rotations, translations, and inversions in the case of 3D Euclidean space).

Symmetry-equivariant Neural Networks (ENNs) are specifically designed to address this issue. They faithfully capture the symmetries of physical systems and operate on the scalar, vector, and tensor fields that characterize these systems. ENNs avoid brute-force data augmentation that traditional neural networks require to understand these transformations and are extremely data-efficient, they make more accurate predictions and need less data to do so.

In this talk, I will give an overview of recent advances in ENNs and their application to physical systems, describe challenges in training and scaling these methods, and highlight unexplored opportunities in ENN techniques including applying these methods more broadly. I will focus on ENNs for 3D Euclidean Symmetry, E(3)NNs, and their application to atomistic systems to motivate this discussion.

E(3)NNs have been successfully used to accelerate and scale ab initio calculations of atomistic systems and are emerging as effective generative models of molecules, crystals, and proteins.  These results are promising — yet there is still much work to do to achieve the generalizability necessary for E(3)NNs to be a robust tool in chemical and materials discovery workflows. Furthermore, this is far from an exhaustive list of potential applications of ENNs, e.g. domains such as fluid dynamics and astrophysics are under-explored.