Representing Born effective charges with equivariant graph convolutional neural networks

Representing Born effective charges with equivariant graph convolutional neural networks | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Representing Born effective charges with equivariant graph convolutional neural networks Ryoji Asahi, Koji Shimizu, Satoshi Watanabe, Alex Kutana This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5727930/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 14 May, 2025 Read the published version in Scientific Reports → Version 1 posted 8 You are reading this latest preprint version Abstract Graph convolutional neural networks have been instrumental in machine learning of material properties. When representing tensorial properties, weights and descriptors of a physics-informed network must obey certain transformation rules to ensure the independence of the property on the choice of the reference frame. Here we explicitly encode such properties using an equivariant graph convolutional neural network. The network respects rotational symmetries of the crystal throughout by using equivariant weights and descriptors and provides a tensorial output of the target value. Applications to tensors of atomic Born effective charges in diverse materials including perovskite oxides, Li 3 PO 4 , and ZrO 2 , are demonstrated, and good performance and generalization ability is obtained. Physical sciences/Mathematics and computing/Computational science Physical sciences/Mathematics and computing/Information technology Equivariant graph convolutional neural networks physics-informed neural networks tensor of atomic Born effective charges linear response oxides Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 1. Introduction Since their introduction in materials science and chemistry, graph convolutional neural networks (GCNNs) have shown remarkable results 1 . GCNNs are being used to represent various molecular and materials properties 1 – 3 , with typical tasks of predicting scalar values for electronic band gaps, ionization potentials/electron affinities, formation energies, as well as performing classification. In particular, in interatomic potentials and force fields applications, GCNNs presently achieve state of the art performance 4 – 6 . While most applications have been limited to scalar target values, many of the fundamental properties of materials are represented by tensors. When constructing a neural network with tensorial outputs, it is essential to incorporate relevant physics-informed 7 constraints and inductive biases, thus greatly improving the model performance in comparison with data augmentation approaches 8 , 9 . Tensors must obey certain transformation rules to ensure the independence of physical properties on the choice of the reference frame 10 . In order for the network features and outputs to follow these rules, it is necessary to encode their equivariance explicitly into the network architecture. Stemming from the original idea of steerable filters for vision and image processing 11 , feature equivariance ensures their proper transformations in convolutional neural networks 12 , 8 , 13 – 15 . In physics and materials science, equivariance with respect to translations and rotations in three-dimensional Euclidean space is most essential. Utilization of Euclidean equivariant networks 9 has greatly increased the accuracy of interatomic potentials 5 , 6 , and benefited other tasks, including prediction of optical, phonon, and neutron scattering spectra 16 – 18 , density functional Hamiltonians 19 , 20 , ab initio wavefunctions 21 , reaction activation barriers 22 , and spin dynamics 23 . At the same time, direct, end-to-end applications of machine learning to tensorial atomic quantities have been scarce. Recent developments include predictions of NMR chemical shift 24 and elasticity 25 , 26 tensors. Our earlier efforts utilized a symmetry-restricted network to predict part of the tensor of atomic Born effective charges 27 . The network was designed for a special case of the external field directed along one of the Cartesian axes, and accommodating the general case would have been substantially more complex. Recently, tensors of atomic Born charges of liquids, dielectrics, and ferroelectrics have also been represented by derivative learning using deep potentials 28 , kernel-based regression 29 , as well as differentiable deep NNs 30 and equivariant GCNNs 31 . Here, we report Equivar, a simple equivariant GCNN (EGCNN) for direct prediction of atomic tensorial quantities, which observes the symmetries of the physical three-dimensional Euclidean space, and operates with geometric quantities throughout. It utilizes local message passing based on the geometric input data. As only equivariant operations are performed on the input geometric data, all constraints imposed by the point group symmetry are automatically satisfied by the outputs. We demonstrate the application of Equivar to end-to-end machine learning of the target property of full tensors of atomic Born effective charges, achieving good performance. Due to equivariance constraints, high accuracy is achieved, while keeping the number of parameters small. 2. Network architecture The atomic structure is represented by a graph 32 , with atoms mapped to graph nodes and neighbor connections to edges, each equipped with attributes/features. Node features are updated via message passing from the neighbor nodes in the interaction block. The basic architecture of Equivar is shown in Fig. 1 . It has a simple layout first employed in SchNet 33 , consisting of the main unit with inputs/embeddings, interaction layers, and outputs. Each of the interaction layers contains the interaction and convolution blocks. The simple architecture helps maximize the transferability and generalizability of the model. The input attributes are scalars (atomic numbers Z i , interatomic distances r ij ), and vectors (bond directions \(\:{\widehat{r}}_{ij}\) ), while the output features are the node values of tensors of atomic Born effective charges Z * i , αβ . Node embedding is a lookup dictionary with the mapping of the atomic numbers into an array of learnable scalar weights. The embedding dimension is a hyperparameter. Edge attributes represent the length and direction of the vector connecting neighbors. Edges are indexed by the bond vector r ab connecting the central atom a with neighbor atom b . One-hot edge encodings δ( r - r ab ) are projected onto a fixed basis of products of radial and angular functions, yielding the mapping ℝ 3 →ℝ×ℝ 2 l +1 for a radial basis function R n ( r ) and a set of angular basis functions of real spherical harmonics Y l , m ( θ , φ ). A similar expansion is employed to obtain the rotational power spectrum and smooth overlap of atomic positions (SOAP) similarity kernel 34 . Uniformly shifted Gaussians R n ( r ) = exp(- ɣ ( r - µ n ) 2 ) 33 were used as radial basis functions 35 , and a 3 Å radial cutoff was used for neighbor lists. The set of spherical harmonics Y l , m of degree l , | m |≤ l , produces equivariant geometric edge features enabling the "information flow" 13 within the channels of a geometric object upon rotation. The projections are passed through linear and activation blocks, producing a trainable tensor product convolution kernel. To provide a nonlinearity, a shifted softplus activation function 33 , log(1 + e x )-log(2), is applied to the scalar edge features. After the embedding, an equivariant linear layer introduces geometric node features, initialized to invariant (zero) values. The geometric (equivariant) features are irreducible representations of O (3), a three-dimensional orthogonal group. Unlike the invariant (scalar) features, the equivariant features are “space-aware” and transform with rotations according to certain rules. Encoding these rules into the weights of the neural network increases its accuracy significantly 7 , as the target quantities share same transformation properties as tensors. The features are stored sequentially in conventional arrays of floats, and typed according to how they transform under O (3) 9 . The equivariant linear layer acts by taking linear combinations of irreducible representations with the same rotational order l and parity p , and storing them in each of the matching outputs. The scalar coefficients of the linear combinations are learnable parameters. The equivariant node features are then updated via message passing in a sequence of interaction blocks, with the number of interactions being a hyperparameter. Within the interaction block, the message passed from a node to its neighbor is a tensor product of the node features with the features of the edge connecting the nodes, and the messages are aggregated by target nodes in a ResNet-style update 36 . The tensor product is calculated using Clebsch-Gordan coefficients 15 . For convenience, identical descriptor types and sizes for node and edge are used in the interaction block. After the interactions, the final linear layer yields a decomposition of a general tensor of the second rank into irreducible representations of the O (3) group, namely 0e + 2e + 1e, which has 9 components that are converted to Cartesian form, accounting for full anisotropy. The equivariance of outputs ensures that the transformation rules for tensor components are always satisfied exactly upon coordinate system rotations. For equivariant operations with geometric features, the primitives implemented in e3nn 9 are used. Charge neutrality acoustic sum rule constraint is applied to model outputs using Eq. (49) from Ref. 37 . 3. Datasets The equivariant model was trained with full tensors of Born effective charges, calculated from first principles using density functional perturbation theory (DFPT) 37 , as implemented in the VASP package 38 . PBEsol functional 39 was used for perovskites, and PBE functional 40 for Li 3 PO 4 and ZrO 2 . Ion-electron interactions were represented by all-electron projector augmented wave potentials 41 , and a plane wave basis was used with cutoff energy of 500 eV ( Li 3 PO 4 ) and 520 eV (perovskites and ZrO 2 ). Tensors of atomic Born effective charges are fundamental quantities determining the long-range long-wavelength part of the force constants, LO-TO gamma point optical phonon splitting, and static dielectric response 37 , 42 . They are defined as the linear part of the change in polarization due to gamma point ion displacement, and given by the mixed second derivatives of the total energy E with respect to atomic positions and electric field 37 , 43 – 46 : $$\:{Z*}_{i,\alpha\:\beta\:}=-\frac{1}{\left|e\right|}\frac{{\partial\:}^{2}E}{\partial\:{\mathcal{E}}_{\alpha\:}\partial\:{u}_{i,\beta\:}}$$ 1 Here, Z * i , αβ is the tensor of the Born effective charge of atom i , u i = u i ( q = 0) is the gamma point displacement of atom i , e is elementary charge, and 𝓔 is the electric field. A definition with swapped Cartesian indices α and β has also been used 47 – 49 . In the DFPT approach used here, the ground state and first derivatives of the wavefunction are calculated to obtain the Born effective charges. Alternatively, Born charges can be obtained using the finite electric field method. Both methods yield similar values, as seen from the comparison in Fig. S1 for CaTiO 3 . After training with DFPT results, the equivariant end-to-end ML model quickly predicts the Born charges tensors from the structural input. We use three datasets representing different systems of interest for training. The datasets were generated in-house by performing substitutions or creating vacancy defects in pristine bulk structures obtained from the Materials Project 50 database. This approach is complementary to other high throughput efforts to building databases of Born effective charges and other response properties based on the structures in the Materials Project database, e.g., JARVIS-DFT 51 . Structure ids of the structures used are given in the Supplementary Information. The first dataset contains substituted perovskite oxides, relevant to electric energy storage applications 52 . Substitutions are a primary method for boosting the dielectric permittivity of transition metal oxides 53 – 55 , leading to greater energy storage capacity. Model validation with this dataset also demonstrates its capability for a wide range of chemical elements. The dataset was generated via cation substitutions in mineral perovskite, Pnma CaTiO 3 . Ca 2+ , Sr 2+ , Ba 2+ , and Pb 2+ isovalent substitutions were performed on the alkaline earth metal site, and Ti 4+ , Zr 4+ , and Hf 4+ on the transition metal site 54 , 55 to generate a dataset of 1,224 materials. Born charges of optimized structures were used for training. The second dataset contains various structures of the Li-ion battery material, Li 3 PO 4 , one of the most widely used solid electrolytes 56 – 58 . The dataset of MD snapshots including defects may test sensitivity of local configurations, and provides a more realistic and diverse distribution, compared to, e.g. employing random displacements sampled from the normal distributions centered on equilibrium positions 30 . This dataset was used to elucidate the Li ion conduction behaviors 27 , 58 . Pristine Li 3 PO 4 (Li 12 P 4 O 16 in the adopted supercell), as well as systems with Li and Li 2 O vacancy defects (Li 11 P 4 O 16 and Li 22 P 8 O 31 , respectively) 27 were used. The pristine subset consists of snapshots from NVT-ensemble ab initio molecular dynamics (AIMD) simulations at 300 and 2,000 K. A time step of 1 fs was used. The Li vacancy structure set contains images from nudged elastic band (NEB) calculations, whereas Li 2 O vacancy structures are snapshots of AIMD at 2,000 K. All frames from the ab initio MD calculations were used. The total number of systems is 17,991; additionally, a dataset with 1,870 larger structures (Li 46 P 16 O 63 ) with a Li 2 O vacancy was used for testing. The third dataset has structures of zirconia (ZrO 2 ), a high permittivity material 59 , with applications in microelectronics, energy storage, and as structural ceramics 60 . The dataset with three different crystal structures — cubic, tetragonal, and monoclinic — contains the subtle effects of these long-range orders. It is planned to use this dataset to analyze the mechanism of plastic deformation enhancement under electric field application 61 . Materials Project structure ids of the structures used are given in the Supplementary Information. The dataset consists of 10,103 NVT-ensemble empirical potential simulation 62 snapshots of cubic, tetragonal, and monoclinic zirconia (Zr 16 O 32 in the supercell for the pristine models) at 1,300, 1,500, 1,700, and 1,900 K, with isotropic lattice constant changes (-2%, -1%, 0%, + 1%, + 2%), and with some of the systems hosting an oxygen vacancy. For the oxygen vacancy structures, + 2 charge state was considered. The snapshots were taken every 1000 fs. 4. Results We benchmarked the model while tuning the hyperparameters using a small dataset of 224 substituted perovskites. The model was trained for 250 epochs using L 1 -norm loss function and AdamW optimizer 63 with an initial learning rate of 0.005, and best validation mean absolute error (MAE) on a held-out subset of 45 structures was recorded. The results are given in Table S1 . The model with ~ 131k trainable parameters (6 interaction layers and feature size of 32 in the interaction block) has the MAE of 0.0117 for the small set validation, compared to the cross-validation MAE of 0.0088 for the full set of 1,224 perovskites. The model demonstrates a good generalization with a small dataset of substituted perovskites, and stable performance with respect to hyperparameter variation. The performance can be further improved by increasing the model size and the number of interactions. Here, we use a 510k-parameter model, ‘base model 1’ (BM1), with 6 interaction layers and feature size of 64 in the interaction block, which provides a good tradeoff among the accuracy, speed, and size, and compares with the smaller/faster 131k-parameter model (BM2). The two models were trained for 500 epochs with a combined dataset of three systems types (perovskites, Li 3 PO 4 , and ZrO 2 , 29,318 structures), as well as several subsets, using a 0.8/0.2 train/validation split. The training history with validation loss is shown in Fig. 2 . BM1 shows consistently better performance, due to larger size of the descriptor in the interaction block. Further increase in the descriptor size leads to even better performance, but is not pursued due to memory constraints. Another important hyperparameter is the number of interactions. Iterative interactions allow the nodes to exchange messages beyond the nearest neighbor cutoff, which allows capturing long-range interactions. In case of 3 Å cutoff radius and 6 interactions, the effective cutoff radius for the total receptive field of each node is 18 Å, sufficient to represent most long-range interactions. Here, 6 interactions were used, although the accuracy kept improving even at 12 interactions, the largest value tested. While a small charge drift MAE of 0.154 and 0.173 were recorded for the BM1 and BM2 models, respectively, subsequent correction is applied to the model outputs to ensure that the sum rule is satisfied exactly. The definition and further details are provided in the Supplementary Information. The scalings of the training and inference times with the dataset size are shown in Figs. S2 and S3, respectively. Calculations were performed on a single NVIDIA RTX A6000 GPU. Training and inference times of Equivar are found to scale approximately linearly with both the number of atoms and number of model parameters. The training times are 0.493 ms/atom/epoch for BM1, and 0.156 ms/atom/epoch for BM2, whereas the inference times are 0.340 ms/atom for BM1, and 0.117 ms/atom for BM2. These timings can be compared with the result of 1.05 ms/atom/epoch for SchNet on the Nvidia GTX 1080 GPU 64 . The model performance summary is given in Table I. The results for the models BM1 and BM2 trained with all three datasets are compared with those for “specialized” models, which were trained with only one dataset. It is seen that a specialized model can be trained to achieve a better performance with a particular dataset. The results of five-fold cross-validation for the tensor of Born effective charges of 1,224 perovskites are shown in Fig. 3 . For cross-validation, the dataset was randomly split into five approximately equal-sized subsets. Each of the subsets was sequentially used for validation, while the remaining subsets for training the model, and validation results were recorded in each run. Cross-validation MAEs for diagonal and off-diagonal tensor components and individual atomic species are shown in Fig. S4. The obtained diagonal and off-diagonal values for these elements are typical for the ABO 3 compounds 48 , 45 . Born effective charges of Ca, Sr, and Ba are rather similar, with average values for diagonal components of 2.5–2.7 and standard deviations of 0.15–0.17, whereas Pb has a larger average Born charge of 3.5 with a standard deviation of 0.26. Ti, Zr, and Hf have the average values of 6.5, 5.7, and 5.6, and standard deviations of 0.56, 0.41, and 0.42 respectively. The diagonal components for O range between − 1.3 and − 6.5, with the average of -2.9 and standard deviation of 0.91. Larger standard deviations in B-type cations and O as compared to A-type cations reflect the bimodal distribution of their Born charges, corresponding to two distinct oxidations states. Of note are the large variations of the off-diagonal components in O, ranging from − 2.2 to 2.1, while in Ca the range is much smaller, from − 0.26 to 0.34. The larger variability of the O values does not result in the larger error for the model prediction. Validation results for training with a Li 3 PO 4 dataset are shown in Fig. 4 . In contrast with the perovskites, the O Born charges in Li 3 PO 4 show smaller values and variability, with the average being − 1.53 and standard deviation 0.33 for the diagonal components. The average charge of Li is 1.06, corresponding to its nominal valence state, whereas that of P is 2.97. The prediction error is worse than for the perovskites dataset that only contains optimized structures without any defects. Thus, the larger prediction MAE in Li 3 PO 4 is most likely due to the presence of anomalous values for a small number of structures in the dataset. The noise in the data could appear when atoms are displaced far from equilibrium in the high temperature MD. Cross-validation results for the model trained with Born effective charges of ZrO 2 are shown in Fig. 5 . In ZrO 2 , the average O charge is -2.78, and std is 0.45, Zr average charge is 5.50 and std is 0.40. The error is larger in ZrO 2 , which we surmised was due to the presence of a small number of anomalous data. We also trained an Equivar model using a cutoff radius of 5 Å and 3 interactions, to check whether a larger cutoff would better account for the nearest neighbor coordination and improve the performance. As seen from the validation plot in Fig. S5, the model performance in ZrO 2 is similar to that of the original model, showing that the cutoff radius of 3 Å is adequate. 5. Discussion The Equivar model demonstrates good performance and generalization ability while retaining simplicity and small size. The model is comparable in size to SchNet 33 , which has ~ 121k parameters (taking radial basis size of 32, same as used here, rather than 300 in the original SchNet). The larger size of Equivar is mainly due to its use of equivariant vector/tensor weights and features, which have larger dimensions than SchNet’s scalars. Overall, while showing good performance, Equivar implements a rather minimalistic EGCNN for tensor representations, and further improvements are possible by employing more complex architectures. Recently, an equivariant model for regression of atomic Born effective charges based on λ-SOAP descriptors 65 has been reported 29 . The model archives a root mean square percentage error (RMSPE) of 3% for Born effective charges isolated water dimers. Equivar shows a better or similar accuracy for bulk systems, with RMSPE of 0.5%, 3%, and 4% for the diagonal elements of effective charge tensors of perovskite, Li 3 PO 4 , and ZrO 2 datasets, respectively. Note that here we do not consider another popular mechanism of attention/transformers 66 , although it may further improve the performance. Based on the size scaling, rather large systems become readily accessible, e.g., evaluation times of ~ 1 s could be attained for a structure with ~ 3000 atoms. This presents a prospect for studying a variety of local doping/substitution configurations quite efficiently, enabling the design of new materials. We now reiterate some of the important aspects contributing to the high accuracy of the model, making it an efficient approach for “second-principles” 67 calculations of tensors of effective charge in oxides. The underlying approach is that of a convolution on a graph, whereby the target values can be gradually learned from the node and edge attributes by performing a sequence of interactions/convolutions with neighbor nodes. Message passing at each interaction allows the information to accumulate and propagate beyond nearest neighbors. Continuous filters are assigned to graph edges, enabling smooth modulation of trainable filter weights according to edge type. Unlike hand-designed representations of the local atomic environments used in the SOAP/λ-SOAP approach 34 , 65 , hidden node features of a GCNN are determined automatically through backpropagation. In SOAP/λ-SOAP, the atomic neighbor density function plays a central role and is a starting point for constructing SOAP descriptors. It is computed by taking the sum of atomic positions over all neighbors around the central atom. Power spectrum/bispectrum descriptors and similarity kernel in SOAP/λ-SOAP are based on the atomic neighbor density function having a fixed predetermined form, in contrast with hidden equivariant representations of an EGCNN. Graph edge descriptors are created from bond vectors individually for each pair of neighbors, and sum is taken at the last step after being passed through a neural network. These important distinctions allow greater flexibility for the automatically determined descriptors, endowing GCNN models with greater regressive power. Equivariant edge features and descriptors allow continuous one-hot encoding and modulation of the convolution weights not only by the neighbor-neighbor distance, but also the direction of the interatomic vector, while equivariant node features naturally represent tensorial target values. The empirically established capability for regressing tensors of Born effective charges suggests sufficient expressiveness of the EGCNN architecture. In the future, the analysis of model limitations and components that are essential for performing efficient target tensor decomposition 68 should be performed. Relevant information is expected to propagate from further neighbors over multiple interaction/message passing steps, but the efficiency of this path has not been tested yet, and more efficient pathways could be possibly designed. While the model shows good accuracy for our datasets, its extrapolation ability to new structures and compositions is still largely untested. Thus, among the current challenges that need to be addressed in the future are interpretability and strong generalization ability to diverse materials. Establishing causality 69 , 70 in the training data could improve interpretability and help models make better predictions. Other possible strategies for improving model performance include using more diverse training data and employing regularization. Fundamentally, Born effective charges define the coupling of atomic vibrations to electric fields, and are responsible for long-range Coulomb interactions, which determine the TO-LO phonon mode splitting in polar crystals 48 , 71 . An early empirical model 72 yielded reasonable values for Born charges of various ABO 3 compounds from the fitting to experimental phonon mode oscillator strengths. With the advent of ab initio approaches, theoretical values have been calculated for many systems and found in good agreement with experiments based on comparison with infrared and inelastic neutron scattering spectra 73 – 76 . Recent developments include the extension of Born charges to metals 77 , 78 , where they can be used to probe electron-phonon interactions. Besides their utility for calculating physical observables, Born effective charges can also provide insights into the nature of microscopic interactions. Qualitatively, one can consider the two limits, one of purely electrostatic (or ionic), and the other of purely electronic interactions. In the former limit, a rigid ion model gives a good approximation of the interaction energy and effective charges. This ionic model of interactions is most applicable in Li 3 PO 4 , where the variation of the effective charges is smallest, indicating mostly ionic nature of bonding. This is especially true for Li, where the effective charge corresponds to the nominal value and shows small variability, whereas variability is somewhat larger in P and O. On the other hand, perovskites and ZrO 2 display a mixture of covalent and ionic bonding. “Semicovalent” bonding is one of the characteristic features of perovskites, giving rise to several interesting phenomena, including the indirect (double) exchange coupling of cations 79 . The mixture of ionic and covalent bonding is responsible for anomalously high Born effective charges, as well as other disparities with the simple rigid ion model, such as the inequivalence of the charges of O anions 44 , 80 . This enhancement of Born effective charges has been traced to dynamical changes of O 2 p and metal d orbital hybridization 81 during ion displacement. The resulting flow of electrons augments the polarization change. Anomalous Born charges are also responsible for the large LO-TO gamma point optical phonon splitting 73 and appearance of the soft ferroelectric mode 82 , and are central for achieving colossal permittivity materials 54 , 55 . We note that these effects are present in the studied systems and should be well captured by our EGCNN model. 6. Conclusion In conclusion, we apply equivariant graph convolutional neural networks to learning the tensors of atomic Born effective charges. Good performance is achieved with three diverse systems: ABO 3 perovskites, Li 3 PO 4 , and ZrO 2 . The model possibly demonstrates a good generalization ability by effectively learning from a small dataset, although a broader dataset 51 would provide a more rigorous test. Performance is further improved when using a larger dataset. The important hyperparameters affecting the model performance are the number of interactions/convolutions and the size of the descriptor in the interaction block. The training and inference times scale linearly with the number of atoms, and model architecture allows treating a wide range of combinations of chemical elements, enabling future creation of universal foundation models trained on a vast chemical space. Our model represents an important step towards fast and accurate modeling of microscopic quantum phenomena in response to the electric field. Table I. Model performance summary. perovskites Li 3 PO 4 ZrO 2 base model BM1 (510k) MAE 0.0207 0.0196 0.0323 base model BM1 (510k) RMSE 0.0300 0.0319 0.0510 base model BM2 (140k) MAE 0.0260 0.0227 0.0370 base model BM2 (140k) RMSE 0.0377 0.0364 0.0569 specialized model (131k) MAE 0.0088 0.0199 0.0325 specialized model (131k) RMSE 0.0135 0.0325 0.0493 Declarations Data availability statement The datasets and weights of the pretrained models BM1 and BM2 supporting the findings of this study are openly available at the following URL/DOI: https://doi.org/10.17632/hx8kcpxh84.1 The python scripts for running evaluations with the model can also be downloaded from https://github.com/equivar/equivar_eval/ Acknowledgement The work was supported by the JSPS Grant-in-Aid for Transformative Research Areas (A) (23H04105), and JST CREST “Nanomechanics” (JPMJCR1996). The computation was carried out using the general project on supercomputer "Flow" at Information Technology Center, Nagoya University. References Reiser, P. et al. Graph neural networks for materials science and chemistry. Commun. Mater. 3 , 93 (2022). Wu, Z. et al. A Comprehensive Survey on Graph Neural Networks. IEEE Trans. Neural Netw. Learn. Syst. 32 , 4–24 (2021). Corso, G., Stark, H., Jegelka, S., Jaakkola, T. & Barzilay, R. Graph neural networks. Nat. Rev. Methods Primer 4 , 17 (2024). Gasteiger, J., Becker, F. & Günnemann, S. GemNet: Universal Directional Graph Neural Networks for Molecules. in Advances in Neural Information Processing Systems (eds. Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P. S. & Vaughan, J. W.) vol. 34 6790–6802 (Curran Associates, Inc., 2021). Batzner, S. et al. E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. Nat. Commun. 13 , 2453 (2022). Batatia, I. et al. A foundation model for atomistic materials chemistry. Preprint at https://arxiv.org/abs/2401.00096 (2023). Raissi, M., Perdikaris, P. & Karniadakis, G. E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378 , 686–707 (2019). Worrall, D. E., Garbin, S. J., Turmukhambetov, D. & Brostow, G. J. Harmonic Networks: Deep Translation and Rotation Equivariance. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017). Geiger, M. & Smidt, T. e3nn: Euclidean Neural Networks. Preprint at https://arxiv.org/abs/2207.09453 (2022). Arfken, G. B. Mathematical Methods for Physicists . (Academic Press, 1985). Freeman, W. T. & Adelson, E. H. The design and use of steerable filters. IEEE Trans. Pattern Anal. Mach. Intell. 13 , 891–906 (1991). Cohen, T. & Welling, M. Group Equivariant Convolutional Networks. in Proceedings of The 33rd International Conference on Machine Learning (eds. Balcan, M. F. & Weinberger, K. Q.) vol. 48 2990–2999 (PMLR, New York, New York, USA, 2016). Weiler, M., Geiger, M., Welling, M., Boomsma, W. & Cohen, T. 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data. Preprint at https://arxiv.org/abs/1807.02547 (2018). Kondor, R. & Trivedi, S. On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups. in Proceedings of the 35th International Conference on Machine Learning (eds. Dy, J. & Krause, A.) vol. 80 2747–2755 (PMLR, 2018). Thomas, N. et al. Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds. Preprint at https://arxiv.org/abs/1802.08219 (2018). Schütt, K., Unke, O. & Gastegger, M. Equivariant message passing for the prediction of tensorial properties and molecular spectra. in Proceedings of the 38th International Conference on Machine Learning (eds. Meila, M. & Zhang, T.) vol. 139 9377–9388 (PMLR, 2021). Okabe, R. et al. Virtual Node Graph Neural Network for Full Phonon Prediction. Preprint at https://arxiv.org/abs/2301.02197 (2023). Cheng, Y. et al. Direct prediction of inelastic neutron scattering spectra from the crystal structure. Mach. Learn. Sci. Technol. 4 , 015010 (2023). Gong, X. et al. General framework for E(3)-equivariant neural network representation of density functional theory Hamiltonian. Nat. Commun. 14 , 2848 (2023). Remme, R., Kaczun, T., Scheurer, M., Dreuw, A. & Hamprecht, F. A. KineticNet: Deep learning a transferable kinetic energy functional for orbital-free density functional theory. J. Chem. Phys. 159 , 144113 (2023). Hermann, J. et al. Ab initio quantum chemistry with neural-network wavefunctions. Nat. Rev. Chem. 7 , 692–709 (2023). Vijay, S. et al. CoeffNet: predicting activation barriers through a chemically-interpretable, equivariant and physically constrained graph neural network. Chem Sci 15 , 2923–2936 (2024). Miyazaki, Y. Equivariant neural networks for spin dynamics simulations of itinerant magnets. Mach. Learn. Sci. Technol. 4 , 045006 (2023). Venetos, M. C., Wen, M. & Persson, K. A. Machine Learning Full NMR Chemical Shift Tensors of Silicon Oxides with Equivariant Graph Neural Networks. J. Phys. Chem. A 127 , 2388–2398 (2023). Pakornchote, T., Ektarawong, A. & Chotibut, T. StrainTensorNet: Predicting crystal structure elastic properties using SE(3)-equivariant graph neural networks. Phys Rev Res 5 , 043198 (2023). Wen, M., Horton, M. K., Munro, J. M., Huck, P. & Persson, K. A. An equivariant graph neural network for the elasticity tensors of all seven crystal systems. Digit. Discov. 3 , 869–882 (2024). Koji Shimizu, E. M., Ryuji Otsuka, Masahiro Hara & Watanabe, S. Prediction of Born effective charges using neural network to study ion migration under electric fields: applications to crystalline and amorphous Li3PO4. Sci. Technol. Adv. Mater. Methods 3 , 2253135 (2023). Zhang, L. et al. A deep potential model with long-range electrostatic interactions. J. Chem. Phys. 156 , 124107 (2022). Schmiedmayer, B. & Kresse, G. Derivative learning of tensorial quantities—Predicting finite temperature infrared spectra from first principles. J. Chem. Phys. 161 , 084703 (2024). Malenfant-Thuot, O., Ryczko, K., Tamblyn, I. & Côté, M. Efficient determination of Born-effective charges, LO-TO splitting, and Raman tensors of solids with a real-space atom-centered deep learning approach. J. Phys. Condens. Matter 36 , 425901 (2024). Falletta, S. et al. Unified Differentiable Learning of Electric Response. Preprint at https://arxiv.org/abs/2403.17207 (2024). F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, & G. Monfardini. The Graph Neural Network Model. IEEE Trans. Neural Netw. 20 , 61–80 (2009). Schütt, K. T., Sauceda, H. E., Kindermans, P.-J., Tkatchenko, A. & Müller, K.-R. SchNet – A deep learning architecture for molecules and materials. J. Chem. Phys. 148 , 241722 (2018). Bartók, A. P., Kondor, R. & Csányi, G. On representing chemical environments. Phys Rev B 87 , 184115 (2013). Buhmann, M. D. Radial basis functions. Acta Numer. 9 , 1–38 (2000). He, K., Zhang, X., Ren, S. & Sun, J. Deep Residual Learning for Image Recognition. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016). Gonze, X. & Lee, C. Dynamical matrices, Born effective charges, dielectric permittivity tensors, and interatomic force constants from density-functional perturbation theory. Phys Rev B 55 , 10355–10368 (1997). Gajdos, M., Hummer, K., Kresse, G., Furthmüller, J. & Bechstedt, F. Linear optical properties in the projector-augmented wave methodology. Phys. Rev. B 73 , 045112 (2006). Perdew, J. P. et al. Restoring the Density-Gradient Expansion for Exchange in Solids and Surfaces. Phys Rev Lett 100 , 136406 (2008). Perdew, J. P., Burke, K. & Ernzerhof, M. Generalized Gradient Approximation Made Simple. Phys Rev Lett 77 , 3865–3868 (1996). Blöchl, P. E. Projector augmented-wave method. Phys Rev B 50 , 17953–17979 (1994). Pick, R. M., Cohen, M. H. & Martin, R. M. Microscopic Theory of Force Constants in the Adiabatic Approximation. Phys Rev B 1 , 910–920 (1970). Giannozzi, P., de Gironcoli, S., Pavone, P. & Baroni, S. Ab initio calculation of phonon dispersions in semiconductors. Phys Rev B 43 , 7231–7242 (1991). Resta, R., Posternak, M. & Baldereschi, A. Towards a quantum theory of polarization in ferroelectrics: The case of KNbO 3 . Phys Rev Lett 70 , 1010–1013 (1993). Cockayne, E. & Burton, B. P. Phonons and static dielectric constant in CaTiO3 from first principles. Phys Rev B 62 , 3735–3743 (2000). Baroni, S., de Gironcoli, S., Dal Corso, A. & Giannozzi, P. Phonons and related crystal properties from density-functional perturbation theory. Rev Mod Phys 73 , 515–562 (2001). Ghosez, P. First-principles study of the dielectric and dynamical properties of barium titanate. Dr. Thesis Univ. Cathol. Louvain (1997). Ghosez, P., Michenaud, J.-P. & Gonze, X. Dynamical atomic charges: The case of ABO 3 compounds. Phys Rev B 58 , 6224–6240 (1998). Wu, X., Vanderbilt, D. & Hamann, D. R. Systematic treatment of displacements, strains, and electric fields in density-functional perturbation theory. Phys Rev B 72 , 035105 (2005). Jain, A. et al. Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. APL Mater. 1 , 011002 (2013). Choudhary, K. et al. High-throughput density functional perturbation theory and machine learning predictions of infrared, piezoelectric, and dielectric responses. Npj Comput. Mater. 6 , 64 (2020). Yang, Z., Du, H., Jin, L. & Poelman, D. High-performance lead-free bulk ceramics for electrical energy storage applications: design strategies and challenges. J. Mater. Chem. A 9 , 18026–18085 (2021). Taniguchi, H., Sato, D., Nakano, A. & Terasaki, I. Permittivity boosting in “yellow” (Nb + In) co-doped TiO2. J. Mater. Chem. C 8 , 13627–13631 (2020). Kutana, A., Shimano, Y. & Asahi, R. Permittivity boosting by induced strain from local doping in titanates from first principles. Sci. Rep. 13 , 3761 (2023). Shimano, Y., Kutana, A. & Asahi, R. Machine learning and atomistic origin of high dielectric permittivity in oxides. Sci. Rep. 13 , 22236 (2023). Kuwata, N., Iwagami, N., Tanji, Y., Matsuda, Y. & Kawamura, J. Characterization of Thin-Film Lithium Batteries with Stable Thin-Film Li3PO4 Solid Electrolytes Fabricated by ArF Excimer Laser Deposition. J. Electrochem. Soc. 157 , A521 (2010). Deng, Y. et al. Structural and Mechanistic Insights into Fast Lithium-Ion Conduction in Li4SiO4–Li3PO4 Solid Electrolytes. J. Am. Chem. Soc. 137 , 9136–9145 (2015). Li, W., Ando, Y., Minamitani, E. & Watanabe, S. Study of Li atom diffusion in amorphous Li3PO4 with neural network potential. J. Chem. Phys. 147 , 214106 (2017). Zhao, X. & Vanderbilt, D. Phonons and lattice dielectric properties of zirconia. Phys Rev B 65 , 075105 (2002). Pešić, M., Hoffmann, M., Richter, C., Mikolajick, T. & Schroeder, U. Nonvolatile Random Access Memory and Energy Storage Based on Antiferroelectric Like Hysteresis in ZrO2. Adv. Funct. Mater. 26 , 7486–7494 (2016). Motomura, H., Tamao, D., Nambu, K., Masuda, H. & Yoshida, H. Athermal effect of flash event on high-temperature plastic deformation in Y2O3-stabilized tetragonal ZrO2 polycrystal. J. Eur. Ceram. Soc. 42 , 5045–5052 (2022). Wang, Y., Zahid, F., Wang, J. & Guo, H. Structure and dielectric properties of amorphous high-𝜅 oxides: HfO_2, ZrO_2, and their alloys. Phys Rev B 85 , 224110 (2012). Loshchilov, I. & Hutter, F. Decoupled Weight Decay Regularization. Preprint at https://arxiv.org/abs/1711.05101 (2017). Park, C. W. et al. Accurate and scalable graph neural network force field and molecular dynamics with direct force architecture. Npj Comput. Mater. 7 , 73 (2021). Grisafi, A., Wilkins, D. M., Csányi, G. & Ceriotti, M. Symmetry-Adapted Machine Learning for Tensorial Properties of Atomistic Systems. Phys Rev Lett 120 , 036002 (2018). Vaswani, A. et al. Attention is All you Need. in Advances in Neural Information Processing Systems (eds. Guyon, I. et al.) vol. 30 (Curran Associates, Inc., 2017). Ghosez, P. & Junquera, J. Modeling of Ferroelectric Oxide Perovskites: From First to Second Principles. Annual Review of Condensed Matter Physics vol. 13 325–364 (2022). Kolda, T. G. & Bader, B. W. Tensor Decompositions and Applications. SIAM Rev. 51 , 455–500 (2009). Ghosh, A., Palanichamy, G., Trujillo, D. P., Shaikh, M. & Ghosh, S. Insights into Cation Ordering of Double Perovskite Oxides from Machine Learning and Causal Relations. Chem. Mater. 34 , 7563–7578 (2022). Ghosh, A. Towards physics-informed explainable machine learning and causal models for materials research. Comput. Mater. Sci. 233 , 112740 (2024). Born, M. & Göppert-Mayer, M. Dynamische Gittertheorie der Kristalle. in Aufbau Der Zusammenhängenden Materie 623–794 (Springer Berlin Heidelberg, Berlin, Heidelberg, 1933). doi:10.1007/978-3-642-91116-3_4. Axe, J. D. Apparent Ionic Charges and Vibrational Eigenmodes of BaTiO 3 and Other Perovskites. Phys Rev 157 , 429–435 (1967). Zhong, W., King-Smith, R. D. & Vanderbilt, D. Giant LO-TO splittings in perovskite ferroelectrics. Phys Rev Lett 72 , 3618–3621 (1994). Öğüt, S. & Rabe, K. M. Anomalous effective charges and far-IR optical absorption of Al 2 Ru from first principles. Phys Rev B 54 , R8297–R8300 (1996). Balan, E., Saitta, A. M., Mauri, F. & Calas, G. First-principles modeling of the infrared spectrum of kaolinite. Am. Mineral. 86 , 1321–1330 (2001). Schmalzl, K., Strauch, D. & Schober, H. Lattice-dynamical and ground-state properties of CaF 2 studied by inelastic neutron scattering and density-functional methods. Phys Rev B 68 , 144301 (2003). Wang, C.-Y., Sharma, S., Gross, E. K. U. & Dewhurst, J. K. Dynamical Born effective charges. Phys Rev B 106 , L180303 (2022). Marchese, G. et al. Born effective charges and vibrational spectra in superconducting and bad conducting metals. Nat. Phys. 20 , 88–94 (2024). Goodenough, J. B. Theory of the Role of Covalence in the Perovskite-Type Manganites [La, M(II)]MnO 3 . Phys Rev 100 , 564–573 (1955). Posternak, M., Resta, R. & Baldereschi, A. Role of covalent bonding in the polarization of perovskite oxides: The case of KNbO 3 . Phys Rev B 50 , 8911–8914 (1994). Ph. Ghosez, X. G. & Michenaud, J.-P. A microscopic study of barium titanate. Ferroelectrics 164 , 113–121 (1995). Resta, R. & Vanderbilt, D. Theory of Polarization: A Modern Approach. in Physics of Ferroelectrics: A Modern Perspective 31–68 (Springer Berlin Heidelberg, Berlin, Heidelberg, 2007). doi:10.1007/978-3-540-34591-6_2. Additional Declarations No competing interests reported. Supplementary Files SupplementaryInformation202504052.pdf Cite Share Download PDF Status: Published Journal Publication published 14 May, 2025 Read the published version in Scientific Reports → Version 1 posted Editorial decision: Accepted 05 May, 2025 Reviews received at journal 04 May, 2025 Reviewers agreed at journal 04 May, 2025 Reviews received at journal 04 May, 2025 Reviewers agreed at journal 24 Apr, 2025 Reviewers invited by journal 22 Apr, 2025 Submission checks completed at journal 17 Apr, 2025 First submitted to journal 05 Apr, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5727930","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":446174598,"identity":"18b35258-816e-4db2-873c-a1c27333e860","order_by":0,"name":"Ryoji Asahi","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABg0lEQVRIie2RMWvCQBTH3xGIS6qdykmkfoULgm2xmg/S5UJAFy0tLoEWmxK4SXG19EvYrUOhkQO7+AEi7RAXuxroELHQnhjFWt1LyY+QC8f/d+9dHkBMzB+F0Pkbo1sf2atNDMmNmLxFcUikzBf8I7KpLI+V8ZqyLbLgqGO6Fz6U9NS9w6zp4+kZHPB3P3g6zsqw1ws+LJ4FTNEkhOT5Qsl4ZSoaM43OW48NW4NyzVbL2m1njDUGSVPNDLhmYyqlmyDXo3ZwlQhFouAZ7BUxLhSqOYqLEQOFqGnG0bNHQRWNGva6cqNnhVJH7EsolcD5dLEulNxMKPq8yuyXwlFXKBJirlCqmgMuNoSSTweMG0KR16soY3EX8mI8eIaTbjGzxtTq5V3TxSaTlHwBBhXTVkbspElWd0mYfS20rvRDrzKaTFmp1lYr3UnoNortRDM3DK1C0U6Y3AutfvTHYJ/KBMjWiUniUcCJBkX6NNpPuZK/Y2bzbAiN5fc13Z2LiYmJ+dd8A5E8iYZozVOJAAAAAElFTkSuQmCC","orcid":"","institution":"Nagoya University","correspondingAuthor":true,"prefix":"","firstName":"Ryoji","middleName":"","lastName":"Asahi","suffix":""},{"id":446174599,"identity":"44a82718-2001-460f-b97a-feeb9a70f525","order_by":1,"name":"Koji Shimizu","email":"","orcid":"","institution":"The University of Tokyo","correspondingAuthor":false,"prefix":"","firstName":"Koji","middleName":"","lastName":"Shimizu","suffix":""},{"id":446174600,"identity":"3fd7d394-b825-434c-a486-62272e2c0faa","order_by":2,"name":"Satoshi Watanabe","email":"","orcid":"","institution":"The University of Tokyo","correspondingAuthor":false,"prefix":"","firstName":"Satoshi","middleName":"","lastName":"Watanabe","suffix":""},{"id":446174601,"identity":"19937ba5-b9fe-4869-b873-e79f99a4ed31","order_by":3,"name":"Alex Kutana","email":"","orcid":"","institution":"Nagoya University","correspondingAuthor":false,"prefix":"","firstName":"Alex","middleName":"","lastName":"Kutana","suffix":""}],"badges":[],"createdAt":"2024-12-29 00:08:08","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5727930/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5727930/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41598-025-01250-5","type":"published","date":"2025-05-14T15:57:28+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":81235197,"identity":"7f125ed6-5aee-46f6-9eb0-bb47ce5db57c","added_by":"auto","created_at":"2025-04-23 19:21:49","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":89095,"visible":true,"origin":"","legend":"\u003cp\u003eArchitecture of the Equivar EGCNN. The inputs are atomic numbers and positions, and outputs are full tensors of atomic Born effective charges. \u003cem\u003el\u003c/em\u003e=0, \u003cem\u003el\u003c/em\u003e=1, and \u003cem\u003el\u003c/em\u003e=2 equivariant features are colored yellow, red, and blue, respectively; “e” and “o” stand for even and odd parity. Numbers indicate feature multiplicities. The values shown correspond to those used in “base model 1”.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-5727930/v1/c1fdc55d1bcbdff4feaa6846.png"},{"id":81235682,"identity":"10736ca3-0f0f-44b7-a15d-2acc19e9a1a2","added_by":"auto","created_at":"2025-04-23 19:29:49","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":71264,"visible":true,"origin":"","legend":"\u003cp\u003eTraining history for EGCNN base models BM1 and BM2 for regression of tensors of atomic Born effective charges. Models were trained for 500 epochs using combinations of dataset of three systems types - perovskites, Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e, and ZrO\u003csub\u003e2\u003c/sub\u003e. The larger BM1 model shows a consistently better performance.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-5727930/v1/22daede6cb0e35b476d67c9f.png"},{"id":81235198,"identity":"646c04c3-7f93-4c22-bdd3-44a824534587","added_by":"auto","created_at":"2025-04-23 19:21:49","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":75579,"visible":true,"origin":"","legend":"\u003cp\u003eFive-fold cross-validation results for the prediction of the tensor of Born effective charges of Pnma ABO\u003csub\u003e3\u003c/sub\u003e perovskites (A=Ca, Sr, Ba, Pb; B=Ti, Zr, Hf) by the equivariant GCNN model. Diagonal and off-diagonal tensor components are shown. The inset shows the MAE for all components for individual atomic species.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-5727930/v1/f3ef38fb2a1dbb27d9b6db21.png"},{"id":81235684,"identity":"0564fa89-8522-40ca-8123-6c1e8ece25d3","added_by":"auto","created_at":"2025-04-23 19:29:49","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":113529,"visible":true,"origin":"","legend":"\u003cp\u003eValidation results for tensors of Born effective charges of Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e. Panels show results for different atomic species: \u003cstrong\u003ea\u003c/strong\u003e Li, \u003cstrong\u003eb\u003c/strong\u003e P, and \u003cstrong\u003ec\u003c/strong\u003e O. The structures used for model training are the snapshots of ab initio molecular dynamics of pristine Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e as well as structures with single Li or Li\u003csub\u003e2\u003c/sub\u003eO vacancies at 300 K and 2,000 K. RMSPE is given for diagonal elements.\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-5727930/v1/1cb6d193bfb5925c360588e1.png"},{"id":81235820,"identity":"c3ddfd67-2f28-48d0-8ae3-86f24dc58ed2","added_by":"auto","created_at":"2025-04-23 19:37:49","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":112183,"visible":true,"origin":"","legend":"\u003cp\u003eFive-fold cross-validation results for the tensor of Born effective charges of ZrO\u003csub\u003e2\u003c/sub\u003e. The dataset consists of AIMD snapshots of cubic, tetragonal, and monoclinic zirconia, with some of the systems hosting an oxygen vacancy. Panels show results for different atomic species: \u003cstrong\u003ea\u003c/strong\u003e Zr, \u003cstrong\u003eb\u003c/strong\u003e O. RMSPE is given for diagonal elements.\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-5727930/v1/7fbdfacb2ad9bfc2bfe677d3.png"},{"id":83067932,"identity":"44d6942f-61a0-4010-b3a3-95509fac4d58","added_by":"auto","created_at":"2025-05-19 16:08:16","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1162811,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5727930/v1/ecd07291-4d01-40d8-8e62-72c68643bf52.pdf"},{"id":81235686,"identity":"12833b91-a602-4c7a-8fe9-5b1742d358dc","added_by":"auto","created_at":"2025-04-23 19:29:49","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":590446,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryInformation202504052.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5727930/v1/cf452c7aef9d927c880ddd0d.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Representing Born effective charges with equivariant graph convolutional neural networks","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eSince their introduction in materials science and chemistry, graph convolutional neural networks (GCNNs) have shown remarkable results\u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u003c/sup\u003e. GCNNs are being used to represent various molecular and materials properties\u003csup\u003e\u003cspan additionalcitationids=\"CR2\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e\u003c/sup\u003e, with typical tasks of predicting scalar values for electronic band gaps, ionization potentials/electron affinities, formation energies, as well as performing classification. In particular, in interatomic potentials and force fields applications, GCNNs presently achieve state of the art performance\u003csup\u003e\u003cspan additionalcitationids=\"CR5\" citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e. While most applications have been limited to scalar target values, many of the fundamental properties of materials are represented by tensors. When constructing a neural network with tensorial outputs, it is essential to incorporate relevant physics-informed\u003csup\u003e\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u003c/sup\u003e constraints and inductive biases, thus greatly improving the model performance in comparison with data augmentation approaches\u003csup\u003e\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e,\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eTensors must obey certain transformation rules to ensure the independence of physical properties on the choice of the reference frame\u003csup\u003e\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e\u003c/sup\u003e. In order for the network features and outputs to follow these rules, it is necessary to encode their equivariance explicitly into the network architecture. Stemming from the original idea of steerable filters for vision and image processing\u003csup\u003e\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e\u003c/sup\u003e, feature equivariance ensures their proper transformations in convolutional neural networks\u003csup\u003e\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e,\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e,\u003cspan additionalcitationids=\"CR14\" citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u003c/sup\u003e. In physics and materials science, equivariance with respect to translations and rotations in three-dimensional Euclidean space is most essential. Utilization of Euclidean equivariant networks\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e has greatly increased the accuracy of interatomic potentials\u003csup\u003e\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e,\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e\u003c/sup\u003e, and benefited other tasks, including prediction of optical, phonon, and neutron scattering spectra\u003csup\u003e\u003cspan additionalcitationids=\"CR17\" citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e\u003c/sup\u003e, density functional Hamiltonians\u003csup\u003e\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e,\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e\u003c/sup\u003e, ab initio wavefunctions\u003csup\u003e\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e\u003c/sup\u003e, reaction activation barriers\u003csup\u003e\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e\u003c/sup\u003e, and spin dynamics\u003csup\u003e\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u003c/sup\u003e. At the same time, direct, end-to-end applications of machine learning to tensorial atomic quantities have been scarce. Recent developments include predictions of NMR chemical shift\u003csup\u003e\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u003c/sup\u003e and elasticity\u003csup\u003e\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e,\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e\u003c/sup\u003e tensors. Our earlier efforts utilized a symmetry-restricted network to predict part of the tensor of atomic Born effective charges\u003csup\u003e\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e. The network was designed for a special case of the external field directed along one of the Cartesian axes, and accommodating the general case would have been substantially more complex. Recently, tensors of atomic Born charges of liquids, dielectrics, and ferroelectrics have also been represented by derivative learning using deep potentials\u003csup\u003e\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u003c/sup\u003e, kernel-based regression\u003csup\u003e\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e\u003c/sup\u003e, as well as differentiable deep NNs\u003csup\u003e\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e\u003c/sup\u003e and equivariant GCNNs\u003csup\u003e\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e\u003c/sup\u003e. Here, we report Equivar, a simple equivariant GCNN (EGCNN) for direct prediction of atomic tensorial quantities, which observes the symmetries of the physical three-dimensional Euclidean space, and operates with geometric quantities throughout. It utilizes local message passing based on the geometric input data. As only equivariant operations are performed on the input geometric data, all constraints imposed by the point group symmetry are automatically satisfied by the outputs. We demonstrate the application of Equivar to end-to-end machine learning of the target property of full tensors of atomic Born effective charges, achieving good performance. Due to equivariance constraints, high accuracy is achieved, while keeping the number of parameters small.\u003c/p\u003e"},{"header":"2. Network architecture","content":"\u003cp\u003e \u003c/p\u003e \u003cp\u003eThe atomic structure is represented by a graph\u003csup\u003e\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e\u003c/sup\u003e, with atoms mapped to graph nodes and neighbor connections to edges, each equipped with attributes/features. Node features are updated via message passing from the neighbor nodes in the interaction block. The basic architecture of Equivar is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. It has a simple layout first employed in SchNet\u003csup\u003e\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e\u003c/sup\u003e, consisting of the main unit with inputs/embeddings, interaction layers, and outputs. Each of the interaction layers contains the interaction and convolution blocks. The simple architecture helps maximize the transferability and generalizability of the model. The input attributes are scalars (atomic numbers \u003cem\u003eZ\u003c/em\u003e\u003csub\u003e\u003cem\u003ei\u003c/em\u003e\u003c/sub\u003e, interatomic distances \u003cem\u003er\u003c/em\u003e\u003csub\u003e\u003cem\u003eij\u003c/em\u003e\u003c/sub\u003e), and vectors (bond directions \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\widehat{r}}_{ij}\\)\u003c/span\u003e\u003c/span\u003e), while the output features are the node values of tensors of atomic Born effective charges Z\u003csup\u003e*\u003c/sup\u003e\u003csub\u003e\u003cem\u003ei\u003c/em\u003e,\u003cem\u003eαβ\u003c/em\u003e\u003c/sub\u003e. Node embedding is a lookup dictionary with the mapping of the atomic numbers into an array of learnable scalar weights. The embedding dimension is a hyperparameter. Edge attributes represent the length and direction of the vector connecting neighbors. Edges are indexed by the bond vector \u003cb\u003er\u003c/b\u003e\u003csub\u003e\u003cem\u003eab\u003c/em\u003e\u003c/sub\u003e connecting the central atom \u003cem\u003ea\u003c/em\u003e with neighbor atom \u003cem\u003eb\u003c/em\u003e. One-hot edge encodings δ(\u003cb\u003er\u003c/b\u003e-\u003cb\u003er\u003c/b\u003e\u003csub\u003e\u003cem\u003eab\u003c/em\u003e\u003c/sub\u003e) are projected onto a fixed basis of products of radial and angular functions, yielding the mapping ℝ\u003csup\u003e3\u003c/sup\u003e\u0026rarr;ℝ\u0026times;ℝ\u003csup\u003e2\u003cem\u003el\u003c/em\u003e+1\u003c/sup\u003e for a radial basis function \u003cem\u003eR\u003c/em\u003e\u003csub\u003e\u003cem\u003en\u003c/em\u003e\u003c/sub\u003e(\u003cem\u003er\u003c/em\u003e) and a set of angular basis functions of real spherical harmonics \u003cem\u003eY\u003c/em\u003e\u003csub\u003e\u003cem\u003el\u003c/em\u003e,\u003cem\u003em\u003c/em\u003e\u003c/sub\u003e(\u003cem\u003eθ\u003c/em\u003e,\u003cem\u003eφ\u003c/em\u003e). A similar expansion is employed to obtain the rotational power spectrum and smooth overlap of atomic positions (SOAP) similarity kernel\u003csup\u003e\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e\u003c/sup\u003e. Uniformly shifted Gaussians \u003cem\u003eR\u003c/em\u003e\u003csub\u003e\u003cem\u003en\u003c/em\u003e\u003c/sub\u003e(\u003cem\u003er\u003c/em\u003e)\u0026thinsp;=\u0026thinsp;exp(-\u003cem\u003eɣ\u003c/em\u003e(\u003cem\u003er\u003c/em\u003e-\u003cem\u003e\u0026micro;\u003c/em\u003e\u003csub\u003e\u003cem\u003en\u003c/em\u003e\u003c/sub\u003e)\u003csup\u003e2\u003c/sup\u003e) \u003csup\u003e\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e\u003c/sup\u003e were used as radial basis functions\u003csup\u003e\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e\u003c/sup\u003e, and a 3 \u0026Aring; radial cutoff was used for neighbor lists. The set of spherical harmonics \u003cem\u003eY\u003c/em\u003e\u003csub\u003e\u003cem\u003el\u003c/em\u003e,\u003cem\u003em\u003c/em\u003e\u003c/sub\u003e of degree \u003cem\u003el\u003c/em\u003e, |\u003cem\u003em\u003c/em\u003e|\u0026le;\u003cem\u003el\u003c/em\u003e, produces equivariant geometric edge features enabling the \"information flow\"\u003csup\u003e\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e\u003c/sup\u003e within the channels of a geometric object upon rotation. The projections are passed through linear and activation blocks, producing a trainable tensor product convolution kernel. To provide a nonlinearity, a shifted softplus activation function\u003csup\u003e\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e\u003c/sup\u003e, log(1\u0026thinsp;+\u0026thinsp;\u003cem\u003ee\u003c/em\u003e\u003csup\u003e\u003cem\u003ex\u003c/em\u003e\u003c/sup\u003e)-log(2), is applied to the scalar edge features.\u003c/p\u003e \u003cp\u003eAfter the embedding, an equivariant linear layer introduces geometric node features, initialized to invariant (zero) values. The geometric (equivariant) features are irreducible representations of \u003cem\u003eO\u003c/em\u003e(3), a three-dimensional orthogonal group. Unlike the invariant (scalar) features, the equivariant features are \u0026ldquo;space-aware\u0026rdquo; and transform with rotations according to certain rules. Encoding these rules into the weights of the neural network increases its accuracy significantly\u003csup\u003e\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u003c/sup\u003e, as the target quantities share same transformation properties as tensors. The features are stored sequentially in conventional arrays of floats, and typed according to how they transform under \u003cem\u003eO\u003c/em\u003e(3)\u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e. The equivariant linear layer acts by taking linear combinations of irreducible representations with the same rotational order \u003cem\u003el\u003c/em\u003e and parity \u003cem\u003ep\u003c/em\u003e, and storing them in each of the matching outputs. The scalar coefficients of the linear combinations are learnable parameters. The equivariant node features are then updated via message passing in a sequence of interaction blocks, with the number of interactions being a hyperparameter. Within the interaction block, the message passed from a node to its neighbor is a tensor product of the node features with the features of the edge connecting the nodes, and the messages are aggregated by target nodes in a ResNet-style update\u003csup\u003e\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e\u003c/sup\u003e. The tensor product is calculated using Clebsch-Gordan coefficients\u003csup\u003e\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e\u003c/sup\u003e. For convenience, identical descriptor types and sizes for node and edge are used in the interaction block.\u003c/p\u003e \u003cp\u003eAfter the interactions, the final linear layer yields a decomposition of a general tensor of the second rank into irreducible representations of the \u003cem\u003eO\u003c/em\u003e(3) group, namely 0e\u0026thinsp;+\u0026thinsp;2e\u0026thinsp;+\u0026thinsp;1e, which has 9 components that are converted to Cartesian form, accounting for full anisotropy. The equivariance of outputs ensures that the transformation rules for tensor components are always satisfied exactly upon coordinate system rotations. For equivariant operations with geometric features, the primitives implemented in \u003cspan fontcategory=\"NonProportional\" class=\"\" name=\"Emphasis\"\u003ee3nn\u003c/span\u003e \u003csup\u003e\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u003c/sup\u003e are used. Charge neutrality acoustic sum rule constraint is applied to model outputs using Eq.\u0026nbsp;(49) from Ref.\u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e"},{"header":"3. Datasets","content":"\u003cp\u003eThe equivariant model was trained with full tensors of Born effective charges, calculated from first principles using density functional perturbation theory (DFPT)\u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u003c/sup\u003e, as implemented in the VASP package\u003csup\u003e\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e\u003c/sup\u003e. PBEsol functional\u003csup\u003e\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e\u003c/sup\u003e was used for perovskites, and PBE functional\u003csup\u003e\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e\u003c/sup\u003e for Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e and ZrO\u003csub\u003e2\u003c/sub\u003e. Ion-electron interactions were represented by all-electron projector augmented wave potentials\u003csup\u003e\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e\u003c/sup\u003e, and a plane wave basis was used with cutoff energy of 500 eV ( Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e) and 520 eV (perovskites and ZrO\u003csub\u003e2\u003c/sub\u003e). Tensors of atomic Born effective charges are fundamental quantities determining the long-range long-wavelength part of the force constants, LO-TO gamma point optical phonon splitting, and static dielectric response\u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e,\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e\u003c/sup\u003e. They are defined as the linear part of the change in polarization due to gamma point ion displacement, and given by the mixed second derivatives of the total energy \u003cem\u003eE\u003c/em\u003e with respect to atomic positions and electric field\u003csup\u003e\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e,\u003cspan additionalcitationids=\"CR44 CR45\" citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e\u003c/sup\u003e:\u003cdiv id=\"Equ1\" class=\"Equation\"\u003e\u003cdiv format=\"TEX\" class=\"mathdisplay\" id=\"FileID_Equ1\" name=\"EquationSource\"\u003e\n$$\\:{Z*}_{i,\\alpha\\:\\beta\\:}=-\\frac{1}{\\left|e\\right|}\\frac{{\\partial\\:}^{2}E}{\\partial\\:{\\mathcal{E}}_{\\alpha\\:}\\partial\\:{u}_{i,\\beta\\:}}$$\u003c/div\u003e\u003cdiv class=\"EquationNumber\"\u003e1\u003c/div\u003e\u003c/div\u003e\u003c/p\u003e \u003cp\u003eHere, Z\u003csup\u003e*\u003c/sup\u003e\u003csub\u003e\u003cem\u003ei\u003c/em\u003e\u003c/sub\u003e,\u003csub\u003e\u003cem\u003eαβ\u003c/em\u003e\u003c/sub\u003e is the tensor of the Born effective charge of atom \u003cem\u003ei\u003c/em\u003e, \u003cb\u003eu\u003c/b\u003e\u003csub\u003e\u003cem\u003ei\u003c/em\u003e\u003c/sub\u003e=\u003cb\u003eu\u003c/b\u003e\u003csub\u003e\u003cem\u003ei\u003c/em\u003e\u003c/sub\u003e(\u003cem\u003eq\u003c/em\u003e\u0026thinsp;=\u0026thinsp;0) is the gamma point displacement of atom \u003cem\u003ei\u003c/em\u003e, \u003cem\u003ee\u003c/em\u003e is elementary charge, and \u003cb\u003e\u0026#120020;\u003c/b\u003e is the electric field. A definition with swapped Cartesian indices \u003cem\u003eα\u003c/em\u003e and \u003cem\u003eβ\u003c/em\u003e has also been used\u003csup\u003e\u003cspan additionalcitationids=\"CR48\" citationid=\"CR47\" class=\"CitationRef\"\u003e47\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR49\" class=\"CitationRef\"\u003e49\u003c/span\u003e\u003c/sup\u003e. In the DFPT approach used here, the ground state and first derivatives of the wavefunction are calculated to obtain the Born effective charges. Alternatively, Born charges can be obtained using the finite electric field method. Both methods yield similar values, as seen from the comparison in Fig. \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003eS1\u003c/span\u003e for CaTiO\u003csub\u003e3\u003c/sub\u003e. After training with DFPT results, the equivariant end-to-end ML model quickly predicts the Born charges tensors from the structural input.\u003c/p\u003e \u003cp\u003eWe use three datasets representing different systems of interest for training. The datasets were generated in-house by performing substitutions or creating vacancy defects in pristine bulk structures obtained from the Materials Project\u003csup\u003e\u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e\u003c/sup\u003e database. This approach is complementary to other high throughput efforts to building databases of Born effective charges and other response properties based on the structures in the Materials Project database, e.g., JARVIS-DFT\u003csup\u003e\u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e\u003c/sup\u003e. Structure ids of the structures used are given in the Supplementary Information. The first dataset contains substituted perovskite oxides, relevant to electric energy storage applications\u003csup\u003e\u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e52\u003c/span\u003e\u003c/sup\u003e. Substitutions are a primary method for boosting the dielectric permittivity of transition metal oxides\u003csup\u003e\u003cspan additionalcitationids=\"CR54\" citationid=\"CR53\" class=\"CitationRef\"\u003e53\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e55\u003c/span\u003e\u003c/sup\u003e, leading to greater energy storage capacity. Model validation with this dataset also demonstrates its capability for a wide range of chemical elements. The dataset was generated via cation substitutions in mineral perovskite, Pnma CaTiO\u003csub\u003e3\u003c/sub\u003e. Ca\u003csup\u003e2+\u003c/sup\u003e, Sr\u003csup\u003e2+\u003c/sup\u003e, Ba\u003csup\u003e2+\u003c/sup\u003e, and Pb\u003csup\u003e2+\u003c/sup\u003e isovalent substitutions were performed on the alkaline earth metal site, and Ti\u003csup\u003e4+\u003c/sup\u003e, Zr\u003csup\u003e4+\u003c/sup\u003e, and Hf\u003csup\u003e4+\u003c/sup\u003e on the transition metal site\u003csup\u003e\u003cspan citationid=\"CR54\" class=\"CitationRef\"\u003e54\u003c/span\u003e,\u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e55\u003c/span\u003e\u003c/sup\u003e to generate a dataset of 1,224 materials. Born charges of optimized structures were used for training. The second dataset contains various structures of the Li-ion battery material, Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e, one of the most widely used solid electrolytes\u003csup\u003e\u003cspan additionalcitationids=\"CR57\" citationid=\"CR56\" class=\"CitationRef\"\u003e56\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR58\" class=\"CitationRef\"\u003e58\u003c/span\u003e\u003c/sup\u003e. The dataset of MD snapshots including defects may test sensitivity of local configurations, and provides a more realistic and diverse distribution, compared to, e.g. employing random displacements sampled from the normal distributions centered on equilibrium positions\u003csup\u003e\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e\u003c/sup\u003e. This dataset was used to elucidate the Li ion conduction behaviors\u003csup\u003e\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e,\u003cspan citationid=\"CR58\" class=\"CitationRef\"\u003e58\u003c/span\u003e\u003c/sup\u003e. Pristine Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e (Li\u003csub\u003e12\u003c/sub\u003eP\u003csub\u003e4\u003c/sub\u003eO\u003csub\u003e16\u003c/sub\u003e in the adopted supercell), as well as systems with Li and Li\u003csub\u003e2\u003c/sub\u003eO vacancy defects (Li\u003csub\u003e11\u003c/sub\u003eP\u003csub\u003e4\u003c/sub\u003eO\u003csub\u003e16\u003c/sub\u003e and Li\u003csub\u003e22\u003c/sub\u003eP\u003csub\u003e8\u003c/sub\u003eO\u003csub\u003e31\u003c/sub\u003e, respectively)\u003csup\u003e\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e\u003c/sup\u003e were used. The pristine subset consists of snapshots from NVT-ensemble ab initio molecular dynamics (AIMD) simulations at 300 and 2,000 K. A time step of 1 fs was used. The Li vacancy structure set contains images from nudged elastic band (NEB) calculations, whereas Li\u003csub\u003e2\u003c/sub\u003eO vacancy structures are snapshots of AIMD at 2,000 K. All frames from the ab initio MD calculations were used. The total number of systems is 17,991; additionally, a dataset with 1,870 larger structures (Li\u003csub\u003e46\u003c/sub\u003eP\u003csub\u003e16\u003c/sub\u003eO\u003csub\u003e63\u003c/sub\u003e) with a Li\u003csub\u003e2\u003c/sub\u003eO vacancy was used for testing. The third dataset has structures of zirconia (ZrO\u003csub\u003e2\u003c/sub\u003e), a high permittivity material\u003csup\u003e\u003cspan citationid=\"CR59\" class=\"CitationRef\"\u003e59\u003c/span\u003e\u003c/sup\u003e, with applications in microelectronics, energy storage, and as structural ceramics\u003csup\u003e\u003cspan citationid=\"CR60\" class=\"CitationRef\"\u003e60\u003c/span\u003e\u003c/sup\u003e. The dataset with three different crystal structures \u0026mdash; cubic, tetragonal, and monoclinic \u0026mdash; contains the subtle effects of these long-range orders. It is planned to use this dataset to analyze the mechanism of plastic deformation enhancement under electric field application\u003csup\u003e\u003cspan citationid=\"CR61\" class=\"CitationRef\"\u003e61\u003c/span\u003e\u003c/sup\u003e. Materials Project structure ids of the structures used are given in the Supplementary Information. The dataset consists of 10,103 NVT-ensemble empirical potential simulation\u003csup\u003e\u003cspan citationid=\"CR62\" class=\"CitationRef\"\u003e62\u003c/span\u003e\u003c/sup\u003e snapshots of cubic, tetragonal, and monoclinic zirconia (Zr\u003csub\u003e16\u003c/sub\u003eO\u003csub\u003e32\u003c/sub\u003e in the supercell for the pristine models) at 1,300, 1,500, 1,700, and 1,900 K, with isotropic lattice constant changes (-2%, -1%, 0%, +\u0026thinsp;1%, +\u0026thinsp;2%), and with some of the systems hosting an oxygen vacancy. For the oxygen vacancy structures, +\u0026thinsp;2 charge state was considered. The snapshots were taken every 1000 fs.\u003c/p\u003e"},{"header":"4. Results","content":"\u003cp\u003eWe benchmarked the model while tuning the hyperparameters using a small dataset of 224 substituted perovskites. The model was trained for 250 epochs using \u003cem\u003eL\u003c/em\u003e\u003csup\u003e\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u003c/sup\u003e-norm loss function and AdamW optimizer\u003csup\u003e\u003cspan citationid=\"CR63\" class=\"CitationRef\"\u003e63\u003c/span\u003e\u003c/sup\u003e with an initial learning rate of 0.005, and best validation mean absolute error (MAE) on a held-out subset of 45 structures was recorded. The results are given in Table \u003cspan refid=\"MOESM1\" class=\"InternalRef\"\u003eS1\u003c/span\u003e. The model with ~\u0026thinsp;131k trainable parameters (6 interaction layers and feature size of 32 in the interaction block) has the MAE of 0.0117 for the small set validation, compared to the cross-validation MAE of 0.0088 for the full set of 1,224 perovskites. The model demonstrates a good generalization with a small dataset of substituted perovskites, and stable performance with respect to hyperparameter variation. The performance can be further improved by increasing the model size and the number of interactions. Here, we use a 510k-parameter model, \u0026lsquo;base model 1\u0026rsquo; (BM1), with 6 interaction layers and feature size of 64 in the interaction block, which provides a good tradeoff among the accuracy, speed, and size, and compares with the smaller/faster 131k-parameter model (BM2).\u003c/p\u003e \u003cp\u003eThe two models were trained for 500 epochs with a combined dataset of three systems types (perovskites, Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e, and ZrO\u003csub\u003e2\u003c/sub\u003e, 29,318 structures), as well as several subsets, using a 0.8/0.2 train/validation split. The training history with validation loss is shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e. BM1 shows consistently better performance, due to larger size of the descriptor in the interaction block. Further increase in the descriptor size leads to even better performance, but is not pursued due to memory constraints. Another important hyperparameter is the number of interactions. Iterative interactions allow the nodes to exchange messages beyond the nearest neighbor cutoff, which allows capturing long-range interactions. In case of 3 \u0026Aring; cutoff radius and 6 interactions, the effective cutoff radius for the total receptive field of each node is 18 \u0026Aring;, sufficient to represent most long-range interactions. Here, 6 interactions were used, although the accuracy kept improving even at 12 interactions, the largest value tested. While a small charge drift MAE of 0.154 and 0.173 were recorded for the BM1 and BM2 models, respectively, subsequent correction is applied to the model outputs to ensure that the sum rule is satisfied exactly. The definition and further details are provided in the Supplementary Information.\u003c/p\u003e \u003cp\u003eThe scalings of the training and inference times with the dataset size are shown in Figs. S2 and S3, respectively. Calculations were performed on a single NVIDIA RTX A6000 GPU. Training and inference times of Equivar are found to scale approximately linearly with both the number of atoms and number of model parameters. The training times are 0.493 ms/atom/epoch for BM1, and 0.156 ms/atom/epoch for BM2, whereas the inference times are 0.340 ms/atom for BM1, and 0.117 ms/atom for BM2. These timings can be compared with the result of 1.05 ms/atom/epoch for SchNet on the Nvidia GTX 1080 GPU\u003csup\u003e\u003cspan citationid=\"CR64\" class=\"CitationRef\"\u003e64\u003c/span\u003e\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eThe model performance summary is given in Table I. The results for the models BM1 and BM2 trained with all three datasets are compared with those for \u0026ldquo;specialized\u0026rdquo; models, which were trained with only one dataset. It is seen that a specialized model can be trained to achieve a better performance with a particular dataset. The results of five-fold cross-validation for the tensor of Born effective charges of 1,224 perovskites are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. For cross-validation, the dataset was randomly split into five approximately equal-sized subsets. Each of the subsets was sequentially used for validation, while the remaining subsets for training the model, and validation results were recorded in each run. Cross-validation MAEs for diagonal and off-diagonal tensor components and individual atomic species are shown in Fig. S4.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe obtained diagonal and off-diagonal values for these elements are typical for the ABO\u003csub\u003e3\u003c/sub\u003e compounds\u003csup\u003e\u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e48\u003c/span\u003e,\u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e45\u003c/span\u003e\u003c/sup\u003e. Born effective charges of Ca, Sr, and Ba are rather similar, with average values for diagonal components of 2.5\u0026ndash;2.7 and standard deviations of 0.15\u0026ndash;0.17, whereas Pb has a larger average Born charge of 3.5 with a standard deviation of 0.26. Ti, Zr, and Hf have the average values of 6.5, 5.7, and 5.6, and standard deviations of 0.56, 0.41, and 0.42 respectively. The diagonal components for O range between \u0026minus;\u0026thinsp;1.3 and \u0026minus;\u0026thinsp;6.5, with the average of -2.9 and standard deviation of 0.91. Larger standard deviations in B-type cations and O as compared to A-type cations reflect the bimodal distribution of their Born charges, corresponding to two distinct oxidations states. Of note are the large variations of the off-diagonal components in O, ranging from \u0026minus;\u0026thinsp;2.2 to 2.1, while in Ca the range is much smaller, from \u0026minus;\u0026thinsp;0.26 to 0.34. The larger variability of the O values does not result in the larger error for the model prediction.\u003c/p\u003e \u003cp\u003eValidation results for training with a Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e dataset are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e. In contrast with the perovskites, the O Born charges in Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e show smaller values and variability, with the average being \u0026minus;\u0026thinsp;1.53 and standard deviation 0.33 for the diagonal components. The average charge of Li is 1.06, corresponding to its nominal valence state, whereas that of P is 2.97. The prediction error is worse than for the perovskites dataset that only contains optimized structures without any defects. Thus, the larger prediction MAE in Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e is most likely due to the presence of anomalous values for a small number of structures in the dataset. The noise in the data could appear when atoms are displaced far from equilibrium in the high temperature MD.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eCross-validation results for the model trained with Born effective charges of ZrO\u003csub\u003e2\u003c/sub\u003e are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e. In ZrO\u003csub\u003e2\u003c/sub\u003e, the average O charge is -2.78, and std is 0.45, Zr average charge is 5.50 and std is 0.40. The error is larger in ZrO\u003csub\u003e2\u003c/sub\u003e, which we surmised was due to the presence of a small number of anomalous data. We also trained an Equivar model using a cutoff radius of 5 \u0026Aring; and 3 interactions, to check whether a larger cutoff would better account for the nearest neighbor coordination and improve the performance. As seen from the validation plot in Fig. S5, the model performance in ZrO\u003csub\u003e2\u003c/sub\u003e is similar to that of the original model, showing that the cutoff radius of 3 \u0026Aring; is adequate.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e"},{"header":"5. Discussion","content":"\u003cp\u003eThe Equivar model demonstrates good performance and generalization ability while retaining simplicity and small size. The model is comparable in size to SchNet\u003csup\u003e\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e\u003c/sup\u003e, which has ~\u0026thinsp;121k parameters (taking radial basis size of 32, same as used here, rather than 300 in the original SchNet). The larger size of Equivar is mainly due to its use of equivariant vector/tensor weights and features, which have larger dimensions than SchNet\u0026rsquo;s scalars. Overall, while showing good performance, Equivar implements a rather minimalistic EGCNN for tensor representations, and further improvements are possible by employing more complex architectures. Recently, an equivariant model for regression of atomic Born effective charges based on λ-SOAP descriptors\u003csup\u003e\u003cspan citationid=\"CR65\" class=\"CitationRef\"\u003e65\u003c/span\u003e\u003c/sup\u003e has been reported\u003csup\u003e\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e\u003c/sup\u003e. The model archives a root mean square percentage error (RMSPE) of 3% for Born effective charges isolated water dimers. Equivar shows a better or similar accuracy for bulk systems, with RMSPE of 0.5%, 3%, and 4% for the diagonal elements of effective charge tensors of perovskite, Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e, and ZrO\u003csub\u003e2\u003c/sub\u003e datasets, respectively. Note that here we do not consider another popular mechanism of attention/transformers\u003csup\u003e66\u003c/sup\u003e, although it may further improve the performance. Based on the size scaling, rather large systems become readily accessible, e.g., evaluation times of ~\u0026thinsp;1 s could be attained for a structure with ~\u0026thinsp;3000 atoms. This presents a prospect for studying a variety of local doping/substitution configurations quite efficiently, enabling the design of new materials.\u003c/p\u003e \u003cp\u003eWe now reiterate some of the important aspects contributing to the high accuracy of the model, making it an efficient approach for \u0026ldquo;second-principles\u0026rdquo;\u003csup\u003e\u003cspan citationid=\"CR67\" class=\"CitationRef\"\u003e67\u003c/span\u003e\u003c/sup\u003e calculations of tensors of effective charge in oxides. The underlying approach is that of a convolution on a graph, whereby the target values can be gradually learned from the node and edge attributes by performing a sequence of interactions/convolutions with neighbor nodes. Message passing at each interaction allows the information to accumulate and propagate beyond nearest neighbors. Continuous filters are assigned to graph edges, enabling smooth modulation of trainable filter weights according to edge type. Unlike hand-designed representations of the local atomic environments used in the SOAP/λ-SOAP approach\u003csup\u003e\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e,\u003cspan citationid=\"CR65\" class=\"CitationRef\"\u003e65\u003c/span\u003e\u003c/sup\u003e, hidden node features of a GCNN are determined automatically through backpropagation. In SOAP/λ-SOAP, the atomic neighbor density function plays a central role and is a starting point for constructing SOAP descriptors. It is computed by taking the sum of atomic positions over all neighbors around the central atom. Power spectrum/bispectrum descriptors and similarity kernel in SOAP/λ-SOAP are based on the atomic neighbor density function having a fixed predetermined form, in contrast with hidden equivariant representations of an EGCNN. Graph edge descriptors are created from bond vectors individually for each pair of neighbors, and sum is taken at the last step after being passed through a neural network. These important distinctions allow greater flexibility for the automatically determined descriptors, endowing GCNN models with greater regressive power. Equivariant edge features and descriptors allow continuous one-hot encoding and modulation of the convolution weights not only by the neighbor-neighbor distance, but also the direction of the interatomic vector, while equivariant node features naturally represent tensorial target values. The empirically established capability for regressing tensors of Born effective charges suggests sufficient expressiveness of the EGCNN architecture.\u003c/p\u003e \u003cp\u003eIn the future, the analysis of model limitations and components that are essential for performing efficient target tensor decomposition\u003csup\u003e\u003cspan citationid=\"CR68\" class=\"CitationRef\"\u003e68\u003c/span\u003e\u003c/sup\u003e should be performed. Relevant information is expected to propagate from further neighbors over multiple interaction/message passing steps, but the efficiency of this path has not been tested yet, and more efficient pathways could be possibly designed. While the model shows good accuracy for our datasets, its extrapolation ability to new structures and compositions is still largely untested. Thus, among the current challenges that need to be addressed in the future are interpretability and strong generalization ability to diverse materials. Establishing causality\u003csup\u003e\u003cspan citationid=\"CR69\" class=\"CitationRef\"\u003e69\u003c/span\u003e,\u003cspan citationid=\"CR70\" class=\"CitationRef\"\u003e70\u003c/span\u003e\u003c/sup\u003e in the training data could improve interpretability and help models make better predictions. Other possible strategies for improving model performance include using more diverse training data and employing regularization.\u003c/p\u003e \u003cp\u003eFundamentally, Born effective charges define the coupling of atomic vibrations to electric fields, and are responsible for long-range Coulomb interactions, which determine the TO-LO phonon mode splitting in polar crystals\u003csup\u003e\u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e48\u003c/span\u003e,\u003cspan citationid=\"CR71\" class=\"CitationRef\"\u003e71\u003c/span\u003e\u003c/sup\u003e. An early empirical model\u003csup\u003e\u003cspan citationid=\"CR72\" class=\"CitationRef\"\u003e72\u003c/span\u003e\u003c/sup\u003e yielded reasonable values for Born charges of various ABO\u003csub\u003e3\u003c/sub\u003e compounds from the fitting to experimental phonon mode oscillator strengths. With the advent of ab initio approaches, theoretical values have been calculated for many systems and found in good agreement with experiments based on comparison with infrared and inelastic neutron scattering spectra\u003csup\u003e\u003cspan additionalcitationids=\"CR74 CR75\" citationid=\"CR73\" class=\"CitationRef\"\u003e73\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR76\" class=\"CitationRef\"\u003e76\u003c/span\u003e\u003c/sup\u003e. Recent developments include the extension of Born charges to metals\u003csup\u003e\u003cspan citationid=\"CR77\" class=\"CitationRef\"\u003e77\u003c/span\u003e,\u003cspan citationid=\"CR78\" class=\"CitationRef\"\u003e78\u003c/span\u003e\u003c/sup\u003e, where they can be used to probe electron-phonon interactions.\u003c/p\u003e \u003cp\u003eBesides their utility for calculating physical observables, Born effective charges can also provide insights into the nature of microscopic interactions. Qualitatively, one can consider the two limits, one of purely electrostatic (or ionic), and the other of purely electronic interactions. In the former limit, a rigid ion model gives a good approximation of the interaction energy and effective charges. This ionic model of interactions is most applicable in Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e, where the variation of the effective charges is smallest, indicating mostly ionic nature of bonding. This is especially true for Li, where the effective charge corresponds to the nominal value and shows small variability, whereas variability is somewhat larger in P and O.\u003c/p\u003e \u003cp\u003eOn the other hand, perovskites and ZrO\u003csub\u003e2\u003c/sub\u003e display a mixture of covalent and ionic bonding. \u0026ldquo;Semicovalent\u0026rdquo; bonding is one of the characteristic features of perovskites, giving rise to several interesting phenomena, including the indirect (double) exchange coupling of cations\u003csup\u003e\u003cspan citationid=\"CR79\" class=\"CitationRef\"\u003e79\u003c/span\u003e\u003c/sup\u003e. The mixture of ionic and covalent bonding is responsible for anomalously high Born effective charges, as well as other disparities with the simple rigid ion model, such as the inequivalence of the charges of O anions\u003csup\u003e\u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e,\u003cspan citationid=\"CR80\" class=\"CitationRef\"\u003e80\u003c/span\u003e\u003c/sup\u003e. This enhancement of Born effective charges has been traced to dynamical changes of O 2\u003cem\u003ep\u003c/em\u003e and metal \u003cem\u003ed\u003c/em\u003e orbital hybridization\u003csup\u003e\u003cspan citationid=\"CR81\" class=\"CitationRef\"\u003e81\u003c/span\u003e\u003c/sup\u003e during ion displacement. The resulting flow of electrons augments the polarization change. Anomalous Born charges are also responsible for the large LO-TO gamma point optical phonon splitting\u003csup\u003e\u003cspan citationid=\"CR73\" class=\"CitationRef\"\u003e73\u003c/span\u003e\u003c/sup\u003e and appearance of the soft ferroelectric mode\u003csup\u003e\u003cspan citationid=\"CR82\" class=\"CitationRef\"\u003e82\u003c/span\u003e\u003c/sup\u003e, and are central for achieving colossal permittivity materials\u003csup\u003e\u003cspan citationid=\"CR54\" class=\"CitationRef\"\u003e54\u003c/span\u003e,\u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e55\u003c/span\u003e\u003c/sup\u003e. We note that these effects are present in the studied systems and should be well captured by our EGCNN model.\u003c/p\u003e"},{"header":"6. Conclusion","content":"\u003cp\u003eIn conclusion, we apply equivariant graph convolutional neural networks to learning the tensors of atomic Born effective charges. Good performance is achieved with three diverse systems: ABO\u003csub\u003e3\u003c/sub\u003e perovskites, Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e, and ZrO\u003csub\u003e2\u003c/sub\u003e. The model possibly demonstrates a good generalization ability by effectively learning from a small dataset, although a broader dataset\u003csup\u003e\u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e\u003c/sup\u003e would provide a more rigorous test. Performance is further improved when using a larger dataset. The important hyperparameters affecting the model performance are the number of interactions/convolutions and the size of the descriptor in the interaction block. The training and inference times scale linearly with the number of atoms, and model architecture allows treating a wide range of combinations of chemical elements, enabling future creation of universal foundation models trained on a vast chemical space. Our model represents an important step towards fast and accurate modeling of microscopic quantum phenomena in response to the electric field.\u003c/p\u003e \u003cp\u003eTable I. Model performance summary.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"No\" id=\"Taba\" border=\"1\"\u003e \u003ccolgroup cols=\"4\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eperovskites\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eLi\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eZrO\u003csub\u003e2\u003c/sub\u003e\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ebase model BM1 (510k) MAE\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.0207\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.0196\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.0323\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ebase model BM1 (510k) RMSE\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.0300\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.0319\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.0510\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ebase model BM2 (140k) MAE\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.0260\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.0227\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.0370\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003ebase model BM2 (140k) RMSE\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.0377\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.0364\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.0569\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003especialized model (131k) MAE\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.0088\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.0199\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.0325\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003especialized model (131k) RMSE\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.0135\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.0325\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.0493\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eData availability statement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe datasets and weights of the pretrained models BM1 and BM2 supporting the findings of this study are openly available at the following URL/DOI: https://doi.org/10.17632/hx8kcpxh84.1 The python scripts for running evaluations with the model can also be downloaded from https://github.com/equivar/equivar_eval/\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003cstrong\u003eAcknowledgement\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe work was supported by the JSPS Grant-in-Aid for Transformative Research Areas (A) (23H04105), and JST CREST \u0026ldquo;Nanomechanics\u0026rdquo; (JPMJCR1996). The computation was carried out using the general project on supercomputer \u0026quot;Flow\u0026quot; at Information Technology Center, Nagoya University.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eReiser, P. \u003cem\u003eet al.\u003c/em\u003e Graph neural networks for materials science and chemistry. \u003cem\u003eCommun. Mater. \u003c/em\u003e\u003cstrong\u003e3\u003c/strong\u003e, 93 (2022).\u003c/li\u003e\n\u003cli\u003eWu, Z. \u003cem\u003eet al.\u003c/em\u003e A Comprehensive Survey on Graph Neural Networks. \u003cem\u003eIEEE Trans. Neural Netw. Learn. Syst. \u003c/em\u003e\u003cstrong\u003e32\u003c/strong\u003e, 4\u0026ndash;24 (2021).\u003c/li\u003e\n\u003cli\u003eCorso, G., Stark, H., Jegelka, S., Jaakkola, T. \u0026amp; Barzilay, R. Graph neural networks. \u003cem\u003eNat. Rev. Methods Primer \u003c/em\u003e\u003cstrong\u003e4\u003c/strong\u003e, 17 (2024).\u003c/li\u003e\n\u003cli\u003eGasteiger, J., Becker, F. \u0026amp; G\u0026uuml;nnemann, S. GemNet: Universal Directional Graph Neural Networks for Molecules. in \u003cem\u003eAdvances in Neural Information Processing Systems\u003c/em\u003e (eds. Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P. S. \u0026amp; Vaughan, J. W.) vol. 34 6790\u0026ndash;6802 (Curran Associates, Inc., 2021).\u003c/li\u003e\n\u003cli\u003eBatzner, S. \u003cem\u003eet al.\u003c/em\u003e E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials. \u003cem\u003eNat. Commun. \u003c/em\u003e\u003cstrong\u003e13\u003c/strong\u003e, 2453 (2022).\u003c/li\u003e\n\u003cli\u003eBatatia, I. \u003cem\u003eet al.\u003c/em\u003e A foundation model for atomistic materials chemistry. Preprint at https://arxiv.org/abs/2401.00096 (2023).\u003c/li\u003e\n\u003cli\u003eRaissi, M., Perdikaris, P. \u0026amp; Karniadakis, G. E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. \u003cem\u003eJ. Comput. Phys. \u003c/em\u003e\u003cstrong\u003e378\u003c/strong\u003e, 686\u0026ndash;707 (2019).\u003c/li\u003e\n\u003cli\u003eWorrall, D. E., Garbin, S. J., Turmukhambetov, D. \u0026amp; Brostow, G. J. Harmonic Networks: Deep Translation and Rotation Equivariance. in \u003cem\u003eProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)\u003c/em\u003e (2017).\u003c/li\u003e\n\u003cli\u003eGeiger, M. \u0026amp; Smidt, T. e3nn: Euclidean Neural Networks. Preprint at https://arxiv.org/abs/2207.09453 (2022).\u003c/li\u003e\n\u003cli\u003eArfken, G. B. \u003cem\u003eMathematical Methods for Physicists\u003c/em\u003e. (Academic Press, 1985).\u003c/li\u003e\n\u003cli\u003eFreeman, W. T. \u0026amp; Adelson, E. H. The design and use of steerable filters. \u003cem\u003eIEEE Trans. Pattern Anal. Mach. Intell. \u003c/em\u003e\u003cstrong\u003e13\u003c/strong\u003e, 891\u0026ndash;906 (1991).\u003c/li\u003e\n\u003cli\u003eCohen, T. \u0026amp; Welling, M. Group Equivariant Convolutional Networks. in \u003cem\u003eProceedings of The 33rd International Conference on Machine Learning\u003c/em\u003e (eds. Balcan, M. F. \u0026amp; Weinberger, K. Q.) vol. 48 2990\u0026ndash;2999 (PMLR, New York, New York, USA, 2016).\u003c/li\u003e\n\u003cli\u003eWeiler, M., Geiger, M., Welling, M., Boomsma, W. \u0026amp; Cohen, T. 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data. Preprint at https://arxiv.org/abs/1807.02547 (2018).\u003c/li\u003e\n\u003cli\u003eKondor, R. \u0026amp; Trivedi, S. On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groups. in \u003cem\u003eProceedings of the 35th International Conference on Machine Learning\u003c/em\u003e (eds. Dy, J. \u0026amp; Krause, A.) vol. 80 2747\u0026ndash;2755 (PMLR, 2018).\u003c/li\u003e\n\u003cli\u003eThomas, N. \u003cem\u003eet al.\u003c/em\u003e Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds. Preprint at https://arxiv.org/abs/1802.08219 (2018).\u003c/li\u003e\n\u003cli\u003eSch\u0026uuml;tt, K., Unke, O. \u0026amp; Gastegger, M. Equivariant message passing for the prediction of tensorial properties and molecular spectra. in \u003cem\u003eProceedings of the 38th International Conference on Machine Learning\u003c/em\u003e (eds. Meila, M. \u0026amp; Zhang, T.) vol. 139 9377\u0026ndash;9388 (PMLR, 2021).\u003c/li\u003e\n\u003cli\u003eOkabe, R. \u003cem\u003eet al.\u003c/em\u003e Virtual Node Graph Neural Network for Full Phonon Prediction. Preprint at https://arxiv.org/abs/2301.02197 (2023).\u003c/li\u003e\n\u003cli\u003eCheng, Y. \u003cem\u003eet al.\u003c/em\u003e Direct prediction of inelastic neutron scattering spectra from the crystal structure. \u003cem\u003eMach. Learn. Sci. Technol. \u003c/em\u003e\u003cstrong\u003e4\u003c/strong\u003e, 015010 (2023).\u003c/li\u003e\n\u003cli\u003eGong, X. \u003cem\u003eet al.\u003c/em\u003e General framework for E(3)-equivariant neural network representation of density functional theory Hamiltonian. \u003cem\u003eNat. Commun. \u003c/em\u003e\u003cstrong\u003e14\u003c/strong\u003e, 2848 (2023).\u003c/li\u003e\n\u003cli\u003eRemme, R., Kaczun, T., Scheurer, M., Dreuw, A. \u0026amp; Hamprecht, F. A. KineticNet: Deep learning a transferable kinetic energy functional for orbital-free density functional theory. \u003cem\u003eJ. Chem. Phys. \u003c/em\u003e\u003cstrong\u003e159\u003c/strong\u003e, 144113 (2023).\u003c/li\u003e\n\u003cli\u003eHermann, J. \u003cem\u003eet al.\u003c/em\u003e Ab initio quantum chemistry with neural-network wavefunctions. \u003cem\u003eNat. Rev. Chem. \u003c/em\u003e\u003cstrong\u003e7\u003c/strong\u003e, 692\u0026ndash;709 (2023).\u003c/li\u003e\n\u003cli\u003eVijay, S. \u003cem\u003eet al.\u003c/em\u003e CoeffNet: predicting activation barriers through a chemically-interpretable, equivariant and physically constrained graph neural network. \u003cem\u003eChem Sci \u003c/em\u003e\u003cstrong\u003e15\u003c/strong\u003e, 2923\u0026ndash;2936 (2024).\u003c/li\u003e\n\u003cli\u003eMiyazaki, Y. Equivariant neural networks for spin dynamics simulations of itinerant magnets. \u003cem\u003eMach. Learn. Sci. Technol. \u003c/em\u003e\u003cstrong\u003e4\u003c/strong\u003e, 045006 (2023).\u003c/li\u003e\n\u003cli\u003eVenetos, M. C., Wen, M. \u0026amp; Persson, K. A. Machine Learning Full NMR Chemical Shift Tensors of Silicon Oxides with Equivariant Graph Neural Networks. \u003cem\u003eJ. Phys. Chem. A \u003c/em\u003e\u003cstrong\u003e127\u003c/strong\u003e, 2388\u0026ndash;2398 (2023).\u003c/li\u003e\n\u003cli\u003ePakornchote, T., Ektarawong, A. \u0026amp; Chotibut, T. StrainTensorNet: Predicting crystal structure elastic properties using SE(3)-equivariant graph neural networks. \u003cem\u003ePhys Rev Res \u003c/em\u003e\u003cstrong\u003e5\u003c/strong\u003e, 043198 (2023).\u003c/li\u003e\n\u003cli\u003eWen, M., Horton, M. K., Munro, J. M., Huck, P. \u0026amp; Persson, K. A. An equivariant graph neural network for the elasticity tensors of all seven crystal systems. \u003cem\u003eDigit. Discov. \u003c/em\u003e\u003cstrong\u003e3\u003c/strong\u003e, 869\u0026ndash;882 (2024).\u003c/li\u003e\n\u003cli\u003eKoji Shimizu, E. M., Ryuji Otsuka, Masahiro Hara \u0026amp; Watanabe, S. Prediction of Born effective charges using neural network to study ion migration under electric fields: applications to crystalline and amorphous Li3PO4. \u003cem\u003eSci. Technol. Adv. Mater. Methods \u003c/em\u003e\u003cstrong\u003e3\u003c/strong\u003e, 2253135 (2023).\u003c/li\u003e\n\u003cli\u003eZhang, L. \u003cem\u003eet al.\u003c/em\u003e A deep potential model with long-range electrostatic interactions. \u003cem\u003eJ. Chem. Phys. \u003c/em\u003e\u003cstrong\u003e156\u003c/strong\u003e, 124107 (2022).\u003c/li\u003e\n\u003cli\u003eSchmiedmayer, B. \u0026amp; Kresse, G. Derivative learning of tensorial quantities\u0026mdash;Predicting finite temperature infrared spectra from first principles. \u003cem\u003eJ. Chem. Phys. \u003c/em\u003e\u003cstrong\u003e161\u003c/strong\u003e, 084703 (2024).\u003c/li\u003e\n\u003cli\u003eMalenfant-Thuot, O., Ryczko, K., Tamblyn, I. \u0026amp; C\u0026ocirc;t\u0026eacute;, M. Efficient determination of Born-effective charges, LO-TO splitting, and Raman tensors of solids with a real-space atom-centered deep learning approach. \u003cem\u003eJ. Phys. Condens. Matter \u003c/em\u003e\u003cstrong\u003e36\u003c/strong\u003e, 425901 (2024).\u003c/li\u003e\n\u003cli\u003eFalletta, S. \u003cem\u003eet al.\u003c/em\u003e Unified Differentiable Learning of Electric Response. Preprint at https://arxiv.org/abs/2403.17207 (2024).\u003c/li\u003e\n\u003cli\u003eF. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, \u0026amp; G. Monfardini. The Graph Neural Network Model. \u003cem\u003eIEEE Trans. Neural Netw. \u003c/em\u003e\u003cstrong\u003e20\u003c/strong\u003e, 61\u0026ndash;80 (2009).\u003c/li\u003e\n\u003cli\u003eSch\u0026uuml;tt, K. T., Sauceda, H. E., Kindermans, P.-J., Tkatchenko, A. \u0026amp; M\u0026uuml;ller, K.-R. SchNet \u0026ndash; A deep learning architecture for molecules and materials. \u003cem\u003eJ. Chem. Phys. \u003c/em\u003e\u003cstrong\u003e148\u003c/strong\u003e, 241722 (2018).\u003c/li\u003e\n\u003cli\u003eBart\u0026oacute;k, A. P., Kondor, R. \u0026amp; Cs\u0026aacute;nyi, G. On representing chemical environments. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e87\u003c/strong\u003e, 184115 (2013).\u003c/li\u003e\n\u003cli\u003eBuhmann, M. D. Radial basis functions. \u003cem\u003eActa Numer. \u003c/em\u003e\u003cstrong\u003e9\u003c/strong\u003e, 1\u0026ndash;38 (2000).\u003c/li\u003e\n\u003cli\u003eHe, K., Zhang, X., Ren, S. \u0026amp; Sun, J. Deep Residual Learning for Image Recognition. in \u003cem\u003eProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)\u003c/em\u003e (2016).\u003c/li\u003e\n\u003cli\u003eGonze, X. \u0026amp; Lee, C. Dynamical matrices, Born effective charges, dielectric permittivity tensors, and interatomic force constants from density-functional perturbation theory. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e55\u003c/strong\u003e, 10355\u0026ndash;10368 (1997).\u003c/li\u003e\n\u003cli\u003eGajdos, M., Hummer, K., Kresse, G., Furthm\u0026uuml;ller, J. \u0026amp; Bechstedt, F. Linear optical properties in the projector-augmented wave methodology. \u003cem\u003ePhys. Rev. B \u003c/em\u003e\u003cstrong\u003e73\u003c/strong\u003e, 045112 (2006).\u003c/li\u003e\n\u003cli\u003ePerdew, J. P. \u003cem\u003eet al.\u003c/em\u003e Restoring the Density-Gradient Expansion for Exchange in Solids and Surfaces. \u003cem\u003ePhys Rev Lett \u003c/em\u003e\u003cstrong\u003e100\u003c/strong\u003e, 136406 (2008).\u003c/li\u003e\n\u003cli\u003ePerdew, J. P., Burke, K. \u0026amp; Ernzerhof, M. Generalized Gradient Approximation Made Simple. \u003cem\u003ePhys Rev Lett \u003c/em\u003e\u003cstrong\u003e77\u003c/strong\u003e, 3865\u0026ndash;3868 (1996).\u003c/li\u003e\n\u003cli\u003eBl\u0026ouml;chl, P. E. Projector augmented-wave method. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e50\u003c/strong\u003e, 17953\u0026ndash;17979 (1994).\u003c/li\u003e\n\u003cli\u003ePick, R. M., Cohen, M. H. \u0026amp; Martin, R. M. Microscopic Theory of Force Constants in the Adiabatic Approximation. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e1\u003c/strong\u003e, 910\u0026ndash;920 (1970).\u003c/li\u003e\n\u003cli\u003eGiannozzi, P., de Gironcoli, S., Pavone, P. \u0026amp; Baroni, S. Ab initio calculation of phonon dispersions in semiconductors. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e43\u003c/strong\u003e, 7231\u0026ndash;7242 (1991).\u003c/li\u003e\n\u003cli\u003eResta, R., Posternak, M. \u0026amp; Baldereschi, A. Towards a quantum theory of polarization in ferroelectrics: The case of KNbO\u003csub\u003e3\u003c/sub\u003e. \u003cem\u003ePhys Rev Lett \u003c/em\u003e\u003cstrong\u003e70\u003c/strong\u003e, 1010\u0026ndash;1013 (1993).\u003c/li\u003e\n\u003cli\u003eCockayne, E. \u0026amp; Burton, B. P. Phonons and static dielectric constant in CaTiO3 from first principles. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e62\u003c/strong\u003e, 3735\u0026ndash;3743 (2000).\u003c/li\u003e\n\u003cli\u003eBaroni, S., de Gironcoli, S., Dal Corso, A. \u0026amp; Giannozzi, P. Phonons and related crystal properties from density-functional perturbation theory. \u003cem\u003eRev Mod Phys \u003c/em\u003e\u003cstrong\u003e73\u003c/strong\u003e, 515\u0026ndash;562 (2001).\u003c/li\u003e\n\u003cli\u003eGhosez, P. First-principles study of the dielectric and dynamical properties of barium titanate. \u003cem\u003eDr. Thesis Univ. Cathol. Louvain\u003c/em\u003e (1997).\u003c/li\u003e\n\u003cli\u003eGhosez, P., Michenaud, J.-P. \u0026amp; Gonze, X. Dynamical atomic charges: The case of ABO\u003csub\u003e3\u003c/sub\u003e compounds. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e58\u003c/strong\u003e, 6224\u0026ndash;6240 (1998).\u003c/li\u003e\n\u003cli\u003eWu, X., Vanderbilt, D. \u0026amp; Hamann, D. R. Systematic treatment of displacements, strains, and electric fields in density-functional perturbation theory. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e72\u003c/strong\u003e, 035105 (2005).\u003c/li\u003e\n\u003cli\u003eJain, A. \u003cem\u003eet al.\u003c/em\u003e Commentary: The Materials Project: A materials genome approach to accelerating materials innovation. \u003cem\u003eAPL Mater. \u003c/em\u003e\u003cstrong\u003e1\u003c/strong\u003e, 011002 (2013).\u003c/li\u003e\n\u003cli\u003eChoudhary, K. \u003cem\u003eet al.\u003c/em\u003e High-throughput density functional perturbation theory and machine learning predictions of infrared, piezoelectric, and dielectric responses. \u003cem\u003eNpj Comput. Mater. \u003c/em\u003e\u003cstrong\u003e6\u003c/strong\u003e, 64 (2020).\u003c/li\u003e\n\u003cli\u003eYang, Z., Du, H., Jin, L. \u0026amp; Poelman, D. High-performance lead-free bulk ceramics for electrical energy storage applications: design strategies and challenges. \u003cem\u003eJ. Mater. Chem. A \u003c/em\u003e\u003cstrong\u003e9\u003c/strong\u003e, 18026\u0026ndash;18085 (2021).\u003c/li\u003e\n\u003cli\u003eTaniguchi, H., Sato, D., Nakano, A. \u0026amp; Terasaki, I. Permittivity boosting in \u0026ldquo;yellow\u0026rdquo; (Nb + In) co-doped TiO2. \u003cem\u003eJ. Mater. Chem. C \u003c/em\u003e\u003cstrong\u003e8\u003c/strong\u003e, 13627\u0026ndash;13631 (2020).\u003c/li\u003e\n\u003cli\u003eKutana, A., Shimano, Y. \u0026amp; Asahi, R. Permittivity boosting by induced strain from local doping in titanates from first principles. \u003cem\u003eSci. Rep. \u003c/em\u003e\u003cstrong\u003e13\u003c/strong\u003e, 3761 (2023).\u003c/li\u003e\n\u003cli\u003eShimano, Y., Kutana, A. \u0026amp; Asahi, R. Machine learning and atomistic origin of high dielectric permittivity in oxides. \u003cem\u003eSci. Rep. \u003c/em\u003e\u003cstrong\u003e13\u003c/strong\u003e, 22236 (2023).\u003c/li\u003e\n\u003cli\u003eKuwata, N., Iwagami, N., Tanji, Y., Matsuda, Y. \u0026amp; Kawamura, J. Characterization of Thin-Film Lithium Batteries with Stable Thin-Film Li3PO4 Solid Electrolytes Fabricated by ArF Excimer Laser Deposition. \u003cem\u003eJ. Electrochem. Soc. \u003c/em\u003e\u003cstrong\u003e157\u003c/strong\u003e, A521 (2010).\u003c/li\u003e\n\u003cli\u003eDeng, Y. \u003cem\u003eet al.\u003c/em\u003e Structural and Mechanistic Insights into Fast Lithium-Ion Conduction in Li4SiO4\u0026ndash;Li3PO4 Solid Electrolytes. \u003cem\u003eJ. Am. Chem. Soc. \u003c/em\u003e\u003cstrong\u003e137\u003c/strong\u003e, 9136\u0026ndash;9145 (2015).\u003c/li\u003e\n\u003cli\u003eLi, W., Ando, Y., Minamitani, E. \u0026amp; Watanabe, S. Study of Li atom diffusion in amorphous Li3PO4 with neural network potential. \u003cem\u003eJ. Chem. Phys. \u003c/em\u003e\u003cstrong\u003e147\u003c/strong\u003e, 214106 (2017).\u003c/li\u003e\n\u003cli\u003eZhao, X. \u0026amp; Vanderbilt, D. Phonons and lattice dielectric properties of zirconia. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e65\u003c/strong\u003e, 075105 (2002).\u003c/li\u003e\n\u003cli\u003ePe\u0026scaron;ić, M., Hoffmann, M., Richter, C., Mikolajick, T. \u0026amp; Schroeder, U. Nonvolatile Random Access Memory and Energy Storage Based on Antiferroelectric Like Hysteresis in ZrO2. \u003cem\u003eAdv. Funct. Mater. \u003c/em\u003e\u003cstrong\u003e26\u003c/strong\u003e, 7486\u0026ndash;7494 (2016).\u003c/li\u003e\n\u003cli\u003eMotomura, H., Tamao, D., Nambu, K., Masuda, H. \u0026amp; Yoshida, H. Athermal effect of flash event on high-temperature plastic deformation in Y2O3-stabilized tetragonal ZrO2 polycrystal. \u003cem\u003eJ. Eur. Ceram. Soc. \u003c/em\u003e\u003cstrong\u003e42\u003c/strong\u003e, 5045\u0026ndash;5052 (2022).\u003c/li\u003e\n\u003cli\u003eWang, Y., Zahid, F., Wang, J. \u0026amp; Guo, H. Structure and dielectric properties of amorphous high-𝜅 oxides: HfO_2, ZrO_2, and their alloys. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e85\u003c/strong\u003e, 224110 (2012).\u003c/li\u003e\n\u003cli\u003eLoshchilov, I. \u0026amp; Hutter, F. Decoupled Weight Decay Regularization. Preprint at https://arxiv.org/abs/1711.05101 (2017).\u003c/li\u003e\n\u003cli\u003ePark, C. W. \u003cem\u003eet al.\u003c/em\u003e Accurate and scalable graph neural network force field and molecular dynamics with direct force architecture. \u003cem\u003eNpj Comput. Mater. \u003c/em\u003e\u003cstrong\u003e7\u003c/strong\u003e, 73 (2021).\u003c/li\u003e\n\u003cli\u003eGrisafi, A., Wilkins, D. M., Cs\u0026aacute;nyi, G. \u0026amp; Ceriotti, M. Symmetry-Adapted Machine Learning for Tensorial Properties of Atomistic Systems. \u003cem\u003ePhys Rev Lett \u003c/em\u003e\u003cstrong\u003e120\u003c/strong\u003e, 036002 (2018).\u003c/li\u003e\n\u003cli\u003eVaswani, A. \u003cem\u003eet al.\u003c/em\u003e Attention is All you Need. in \u003cem\u003eAdvances in Neural Information Processing Systems\u003c/em\u003e (eds. Guyon, I. et al.) vol. 30 (Curran Associates, Inc., 2017).\u003c/li\u003e\n\u003cli\u003eGhosez, P. \u0026amp; Junquera, J. Modeling of Ferroelectric Oxide Perovskites: From First to Second Principles. \u003cem\u003eAnnual Review of Condensed Matter Physics\u003c/em\u003e vol. 13 325\u0026ndash;364 (2022).\u003c/li\u003e\n\u003cli\u003eKolda, T. G. \u0026amp; Bader, B. W. Tensor Decompositions and Applications. \u003cem\u003eSIAM Rev. \u003c/em\u003e\u003cstrong\u003e51\u003c/strong\u003e, 455\u0026ndash;500 (2009).\u003c/li\u003e\n\u003cli\u003eGhosh, A., Palanichamy, G., Trujillo, D. P., Shaikh, M. \u0026amp; Ghosh, S. Insights into Cation Ordering of Double Perovskite Oxides from Machine Learning and Causal Relations. \u003cem\u003eChem. Mater. \u003c/em\u003e\u003cstrong\u003e34\u003c/strong\u003e, 7563\u0026ndash;7578 (2022).\u003c/li\u003e\n\u003cli\u003eGhosh, A. Towards physics-informed explainable machine learning and causal models for materials research. \u003cem\u003eComput. Mater. Sci. \u003c/em\u003e\u003cstrong\u003e233\u003c/strong\u003e, 112740 (2024).\u003c/li\u003e\n\u003cli\u003eBorn, M. \u0026amp; G\u0026ouml;ppert-Mayer, M. Dynamische Gittertheorie der Kristalle. in \u003cem\u003eAufbau Der Zusammenh\u0026auml;ngenden Materie\u003c/em\u003e 623\u0026ndash;794 (Springer Berlin Heidelberg, Berlin, Heidelberg, 1933). doi:10.1007/978-3-642-91116-3_4.\u003c/li\u003e\n\u003cli\u003eAxe, J. D. Apparent Ionic Charges and Vibrational Eigenmodes of BaTiO\u003csub\u003e3\u003c/sub\u003e and Other Perovskites. \u003cem\u003ePhys Rev \u003c/em\u003e\u003cstrong\u003e157\u003c/strong\u003e, 429\u0026ndash;435 (1967).\u003c/li\u003e\n\u003cli\u003eZhong, W., King-Smith, R. D. \u0026amp; Vanderbilt, D. Giant LO-TO splittings in perovskite ferroelectrics. \u003cem\u003ePhys Rev Lett \u003c/em\u003e\u003cstrong\u003e72\u003c/strong\u003e, 3618\u0026ndash;3621 (1994).\u003c/li\u003e\n\u003cli\u003e\u0026Ouml;ğ\u0026uuml;t, S. \u0026amp; Rabe, K. M. Anomalous effective charges and far-IR optical absorption of Al\u003csub\u003e2\u003c/sub\u003eRu from first principles. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e54\u003c/strong\u003e, R8297\u0026ndash;R8300 (1996).\u003c/li\u003e\n\u003cli\u003eBalan, E., Saitta, A. M., Mauri, F. \u0026amp; Calas, G. First-principles modeling of the infrared spectrum of kaolinite. \u003cem\u003eAm. Mineral. \u003c/em\u003e\u003cstrong\u003e86\u003c/strong\u003e, 1321\u0026ndash;1330 (2001).\u003c/li\u003e\n\u003cli\u003eSchmalzl, K., Strauch, D. \u0026amp; Schober, H. Lattice-dynamical and ground-state properties of CaF\u003csub\u003e2\u003c/sub\u003e studied by inelastic neutron scattering and density-functional methods. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e68\u003c/strong\u003e, 144301 (2003).\u003c/li\u003e\n\u003cli\u003eWang, C.-Y., Sharma, S., Gross, E. K. U. \u0026amp; Dewhurst, J. K. Dynamical Born effective charges. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e106\u003c/strong\u003e, L180303 (2022).\u003c/li\u003e\n\u003cli\u003eMarchese, G. \u003cem\u003eet al.\u003c/em\u003e Born effective charges and vibrational spectra in superconducting and bad conducting metals. \u003cem\u003eNat. Phys. \u003c/em\u003e\u003cstrong\u003e20\u003c/strong\u003e, 88\u0026ndash;94 (2024).\u003c/li\u003e\n\u003cli\u003eGoodenough, J. B. Theory of the Role of Covalence in the Perovskite-Type Manganites [La, M(II)]MnO\u003csub\u003e3\u003c/sub\u003e. \u003cem\u003ePhys Rev \u003c/em\u003e\u003cstrong\u003e100\u003c/strong\u003e, 564\u0026ndash;573 (1955).\u003c/li\u003e\n\u003cli\u003ePosternak, M., Resta, R. \u0026amp; Baldereschi, A. Role of covalent bonding in the polarization of perovskite oxides: The case of KNbO\u003csub\u003e3\u003c/sub\u003e. \u003cem\u003ePhys Rev B \u003c/em\u003e\u003cstrong\u003e50\u003c/strong\u003e, 8911\u0026ndash;8914 (1994).\u003c/li\u003e\n\u003cli\u003ePh. Ghosez, X. G. \u0026amp; Michenaud, J.-P. A microscopic study of barium titanate. \u003cem\u003eFerroelectrics \u003c/em\u003e\u003cstrong\u003e164\u003c/strong\u003e, 113\u0026ndash;121 (1995).\u003c/li\u003e\n\u003cli\u003eResta, R. \u0026amp; Vanderbilt, D. Theory of Polarization: A Modern Approach. in \u003cem\u003ePhysics of Ferroelectrics: A Modern Perspective\u003c/em\u003e 31\u0026ndash;68 (Springer Berlin Heidelberg, Berlin, Heidelberg, 2007). doi:10.1007/978-3-540-34591-6_2.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Equivariant graph convolutional neural networks, physics-informed neural networks, tensor of atomic Born effective charges, linear response, oxides","lastPublishedDoi":"10.21203/rs.3.rs-5727930/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5727930/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eGraph convolutional neural networks have been instrumental in machine learning of material properties. When representing tensorial properties, weights and descriptors of a physics-informed network must obey certain transformation rules to ensure the independence of the property on the choice of the reference frame. Here we explicitly encode such properties using an equivariant graph convolutional neural network. The network respects rotational symmetries of the crystal throughout by using equivariant weights and descriptors and provides a tensorial output of the target value. Applications to tensors of atomic Born effective charges in diverse materials including perovskite oxides, Li\u003csub\u003e3\u003c/sub\u003ePO\u003csub\u003e4\u003c/sub\u003e, and ZrO\u003csub\u003e2\u003c/sub\u003e, are demonstrated, and good performance and generalization ability is obtained.\u003c/p\u003e","manuscriptTitle":"Representing Born effective charges with equivariant graph convolutional neural networks","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-04-23 19:21:44","doi":"10.21203/rs.3.rs-5727930/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Accepted","date":"2025-05-05T09:01:37+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-05-04T21:54:36+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"106528343546359205159631031050056272028","date":"2025-05-04T21:48:39+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-05-04T14:16:50+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"240764866290874938460974274763312562845","date":"2025-04-24T13:57:41+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-04-22T08:00:46+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-04-17T17:26:02+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2025-04-05T15:07:08+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"7b317946-69cb-4946-a7ce-f7759cd099c6","owner":[],"postedDate":"April 23rd, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":47489999,"name":"Physical sciences/Mathematics and computing/Computational science"},{"id":47490000,"name":"Physical sciences/Mathematics and computing/Information technology"}],"tags":[],"updatedAt":"2025-05-19T16:03:34+00:00","versionOfRecord":{"articleIdentity":"rs-5727930","link":"https://doi.org/10.1038/s41598-025-01250-5","journal":{"identity":"scientific-reports","isVorOnly":false,"title":"Scientific Reports"},"publishedOn":"2025-05-14 15:57:28","publishedOnDateReadable":"May 14th, 2025"},"versionCreatedAt":"2025-04-23 19:21:44","video":"","vorDoi":"10.1038/s41598-025-01250-5","vorDoiUrl":"https://doi.org/10.1038/s41598-025-01250-5","workflowStages":[]},"version":"v1","identity":"rs-5727930","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5727930","identity":"rs-5727930","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}