Reproducibility in density functional theory calculations of solids

Kurt Lejaeghere; Gustav Bihlmayer; Torbjörn Björkman; Peter Blaha; Stefan Blügel; Volker Blum; Damien Caliste; Ivano E. Castelli; Stewart J. Clark; Andrea Dal Corso; Stefano De Gironcoli; Thierry Deutsch; John Kay Dewhurst; Igor Di Marco; Claudia Draxl; Marcin Dułak; Olle Eriksson; José A. Flores-Livas; Kevin F. Garrity; Luigi Genovese; Paolo Giannozzi; Matteo Giantomassi; Stefan Goedecker; Xavier Gonze; Oscar Grånäs; E. K.U. Gross; Andris Gulans; François Gygi; D. R. Hamann; Phil J. Hasnip; N. A.W. Holzwarth; Diana Iuşan; Dominik B. Jochym; François Jollet; Daniel Jones; Georg Kresse; Klaus Koepernik; Emine Küçükbenli; Yaroslav O. Kvashnin; Inka L.M. Locht; Sven Lubeck; Martijn Marsman; Nicola Marzari; Ulrike Nitzsche; Lars Nordström; Taisuke Ozaki; Lorenzo Paulatto; Chris J. Pickard; Ward Poelmans; Matt I.J. Probert; Keith Refson; Manuel Richter; Gian Marco Rignanese; Santanu Saha; Matthias Scheffler; Martin Schlipf; Karlheinz Schwarz; Sangeeta Sharma; Francesca Tavazza; Patrik Thunström; Alexandre Tkatchenko; Marc Torrent; David Vanderbilt; Michiel J. Van Setten; Veronique Van Speybroeck; John M. Wills; Jonathan R. Yates; Guo Xu Zhang; Stefaan Cottenier

doi:https://doi.org/10.1126/science.aad3000

Reproducibility in density functional theory calculations of solids

Kurt Lejaeghere, Gustav Bihlmayer, Torbjörn Björkman, Peter Blaha, Stefan Blügel, Volker Blum, Damien Caliste, Ivano E. Castelli, Stewart J. Clark, Andrea Dal Corso, Stefano De Gironcoli, Thierry Deutsch, John Kay Dewhurst, Igor Di Marco, Claudia Draxl, Marcin Dułak, Olle Eriksson, José A. Flores-Livas, Kevin F. Garrity, Luigi GenovesePaolo Giannozzi, Matteo Giantomassi, Stefan Goedecker, Xavier Gonze, Oscar Grånäs, E. K.U. Gross, Andris Gulans, François Gygi, D. R. Hamann, Phil J. Hasnip, N. A.W. Holzwarth, Diana Iuşan, Dominik B. Jochym, François Jollet, Daniel Jones, Georg Kresse, Klaus Koepernik, Emine Küçükbenli, Yaroslav O. Kvashnin, Inka L.M. Locht, Sven Lubeck, Martijn Marsman, Nicola Marzari, Ulrike Nitzsche, Lars Nordström, Taisuke Ozaki, Lorenzo Paulatto, Chris J. Pickard, Ward Poelmans, Matt I.J. Probert, Keith Refson, Manuel Richter, Gian Marco Rignanese, Santanu Saha, Matthias Scheffler, Martin Schlipf, Karlheinz Schwarz, Sangeeta Sharma, Francesca Tavazza, Patrik Thunström, Alexandre Tkatchenko, Marc Torrent, David Vanderbilt, Michiel J. Van Setten, Veronique Van Speybroeck, John M. Wills, Jonathan R. Yates, Guo Xu Zhang, Stefaan Cottenier

Research output: Contribution to journal › Article › peer-review

Abstract

The widespread popularity of density functional theory has given rise to an extensive range of dedicated codes for predicting molecular and crystalline properties. However, each code implements the formalism in a different way, raising questions about the reproducibility of such predictions.We report the results of a community-wide effort that compared 15 solid-state codes, using 40 different potentials or basis set types, to assess the quality of the Perdew-Burke-Ernzerhof equations of state for 71 elemental crystals. We conclude that predictions from recent codes and pseudopotentials agree very well, with pairwise differences that are comparable to those between different high-precision experiments. Older methods, however, have less precise agreement. Our benchmark provides a framework for users and developers to document the precision of new applications and methodological improvements.

Original language	English
Article number	aad3000
Journal	Science
Volume	351
Issue number	6280
DOIs	https://doi.org/10.1126/science.aad3000
State	Published - 25 Mar 2016
Externally published	Yes

All Science Journal Classification (ASJC) codes

General

Access to Document

https://doi.org/10.1126/science.aad3000

Cite this

Lejaeghere, K., Bihlmayer, G., Björkman, T., Blaha, P., Blügel, S., Blum, V., Caliste, D., Castelli, I. E., Clark, S. J., Dal Corso, A., De Gironcoli, S., Deutsch, T., Dewhurst, J. K., Di Marco, I., Draxl, C., Dułak, M., Eriksson, O., Flores-Livas, J. A., Garrity, K. F., ... Cottenier, S. (2016). Reproducibility in density functional theory calculations of solids. Science, 351(6280), Article aad3000. https://doi.org/10.1126/science.aad3000

@article{e456b151e7fa403eb0105671f860dcec,

title = "Reproducibility in density functional theory calculations of solids",

abstract = "The widespread popularity of density functional theory has given rise to an extensive range of dedicated codes for predicting molecular and crystalline properties. However, each code implements the formalism in a different way, raising questions about the reproducibility of such predictions.We report the results of a community-wide effort that compared 15 solid-state codes, using 40 different potentials or basis set types, to assess the quality of the Perdew-Burke-Ernzerhof equations of state for 71 elemental crystals. We conclude that predictions from recent codes and pseudopotentials agree very well, with pairwise differences that are comparable to those between different high-precision experiments. Older methods, however, have less precise agreement. Our benchmark provides a framework for users and developers to document the precision of new applications and methodological improvements.",

author = "Kurt Lejaeghere and Gustav Bihlmayer and Torbj{\"o}rn Bj{\"o}rkman and Peter Blaha and Stefan Bl{\"u}gel and Volker Blum and Damien Caliste and Castelli, {Ivano E.} and Clark, {Stewart J.} and {Dal Corso}, Andrea and {De Gironcoli}, Stefano and Thierry Deutsch and Dewhurst, {John Kay} and {Di Marco}, Igor and Claudia Draxl and Marcin Du{\l}ak and Olle Eriksson and Flores-Livas, {Jos{\'e} A.} and Garrity, {Kevin F.} and Luigi Genovese and Paolo Giannozzi and Matteo Giantomassi and Stefan Goedecker and Xavier Gonze and Oscar Gr{\aa}n{\"a}s and Gross, {E. K.U.} and Andris Gulans and Fran{\c c}ois Gygi and Hamann, {D. R.} and Hasnip, {Phil J.} and Holzwarth, {N. A.W.} and Diana Iu{\c s}an and Jochym, {Dominik B.} and Fran{\c c}ois Jollet and Daniel Jones and Georg Kresse and Klaus Koepernik and Emine K{\"u}{\c c}{\"u}kbenli and Kvashnin, {Yaroslav O.} and Locht, {Inka L.M.} and Sven Lubeck and Martijn Marsman and Nicola Marzari and Ulrike Nitzsche and Lars Nordstr{\"o}m and Taisuke Ozaki and Lorenzo Paulatto and Pickard, {Chris J.} and Ward Poelmans and Probert, {Matt I.J.} and Keith Refson and Manuel Richter and Rignanese, {Gian Marco} and Santanu Saha and Matthias Scheffler and Martin Schlipf and Karlheinz Schwarz and Sangeeta Sharma and Francesca Tavazza and Patrik Thunstr{\"o}m and Alexandre Tkatchenko and Marc Torrent and David Vanderbilt and {Van Setten}, {Michiel J.} and {Van Speybroeck}, Veronique and Wills, {John M.} and Yates, {Jonathan R.} and Zhang, {Guo Xu} and Stefaan Cottenier",

note = "Funding Information: ACKNOWLEDGMENTS: This research benefited from financial support from the Research Board of Ghent University; the Fond de la Recherche Scientifique de Belgique (FRS-FNRS), through Projet de Recherches (PDR) grants T.0238.13-AIXPHO and T.1031.14-HiT4FiT; the Communaut{\'e} Fran{\c c}aise de Belgique, through the BATTAB project (grant ARC 14/19-057); the U.S. NSF (grant DMR-14-08838); the Swedish Research Council; the Knut and Alice Wallenberg Foundation (grants 2013.0020 and 2012.0031); the Fund for Scientific Research-Flanders (FWO) (project no. G0E0116N); and the U.S. Department of Energy (grant DOE-BES DE-SC0008938). N.A.W.H. was supported by U.S. NSF grant DMR-1105485. J.A.F.-L. acknowledges financial support from the European Union's 7th Framework Marie-Curie Scholarship Program within the ExMaMa Project (project no. 329386). I.D.M., O.E., O.G., D.I., Y.O.K., I.L.M.L., and L.N. acknowledge support from eSSENCE. T.B. was supported by the Academy of Finland (grant 263416) and the COMP Centre of Excellence. C.D., A.G., and S.L. acknowledge support from the Deutsche Forschungsgemeinschaft (DFG) and the Einstein Foundation, Berlin. M.Sche. and C.D. received funding from the European Union's Horizon 2020 research and innovation program under grant agreement no. 676580 with The Novel Materials Discovery (NOMAD) Laboratory, a European Center of Excellence. A.D.C., S.d.G., and E.K. acknowledge support from the Italian Ministry of Education, Universities, and Research (MIUR) through PRIN (Projects of National Interest) 2010-2011 (registration no. 20105ZZTSE-005). P.J.H., D.B.J., and M.I.J.P. are grateful for financial support by the Engineering and Physical Sciences Research Council (EPSRC) under UK Car-Parrinello (UKCP) grant EP/K013564/1. C.J.P. and J.R.Y. acknowledge support from the Collaborative Computational Project for NMR Crystallography under EPSRC grant EP/J010510/1. W.P. acknowledges funding by FWO. D.J. is grateful for financial support by EPSRC under grant EP/J017639/1. S.Sa. acknowledges support from the Swiss National Science Foundation (SNSF). G.-M.R. is thankful for personal financial support from FRS-FNRS. The work by I.E.C. and N.M. was supported by the SNSF's National Centre of Competence in Research MARVEL. G.K. and P.B. acknowledge support by the Austrian Science Fund, project SFB-F41 (ViCoM). S.C. acknowledges financial support from OCAS NV by an OCASendowed chair at Ghent University. Computational resources were as follows: The Ghent University contributors used the Stevin Supercomputer Infrastructure at Ghent University, which is funded by Ghent University, FWO, and the Flemish Government (Economy, Science, and Innovation Department). The Universit{\'e} Catholique de Louvain contributors used the Tier-1 supercomputer of the F{\'e}d{\'e}ration Wallonie-Bruxelles (funded by the Walloon Region under grant agreement no. 1117545), the Centre de Calcul Intensif et de Stockage de Masse-Universit{\'e} Catholique de Louvain supercomputing facilities, and the Consortium des {\'E}quipements de Calcul Intensif en F{\'e}d{\'e}ration Wallonie-Bruxelles (C{\'E}CI) (funded by the FRS-FNRS under convention 2.5020.11). The Science and Technology Facilities Council, Scientific Computing Department's SCARF (Scientific Computing Application Resource for Facilities) cluster was used for the CASTEP calculations. The Basel University and {\'E}cole Polytechnique F{\'e}d{\'e}rale de Lausanne contributors used the Swiss National Supercomputing Center in Lugano. Finland's IT Centre for Science was used for the RSPt calculations. K.L. and F.T. thank C. Becker for instructive discussions on the comparison of atomic-scale simulations. K.L. and S.C. thank W. Dewitte for drafting the summary figure. S.J.C., P.J.H., C.J.P., M.I.J.P., K.R., and J.R.Y. declare the receipt of income from commercial sales of CASTEP by Biovia. N.M. and M.Sche. are members of the Board of Trustees of the Psi-k Electronic Structure Network. P.G. is director of the Quantum ESPRESSO Foundation, and N.M. is a representative member. X.G., D.R.H., M.T., D.C., F.J., and G.-M.R. are members of the Advisory Board of ABINIT, an organization that develops and publishes open-source software related to this article. Commercial software is identified to specify procedures. Such identification does not imply recommendation by the National Institute of Standards and Technology. Atomic Simulation Environment scripts (46) for several of the codes are available online (48). All data are listed in tables S3 to S42.",

year = "2016",

month = mar,

day = "25",

doi = "https://doi.org/10.1126/science.aad3000",

language = "الإنجليزيّة",

volume = "351",

journal = "Science",

issn = "0036-8075",

publisher = "American Association for the Advancement of Science",

number = "6280",

}

Lejaeghere, K, Bihlmayer, G, Björkman, T, Blaha, P, Blügel, S, Blum, V, Caliste, D, Castelli, IE, Clark, SJ, Dal Corso, A, De Gironcoli, S, Deutsch, T, Dewhurst, JK, Di Marco, I, Draxl, C, Dułak, M, Eriksson, O, Flores-Livas, JA, Garrity, KF, Genovese, L, Giannozzi, P, Giantomassi, M, Goedecker, S, Gonze, X, Grånäs, O, Gross, EKU, Gulans, A, Gygi, F, Hamann, DR, Hasnip, PJ, Holzwarth, NAW, Iuşan, D, Jochym, DB, Jollet, F, Jones, D, Kresse, G, Koepernik, K, Küçükbenli, E, Kvashnin, YO, Locht, ILM, Lubeck, S, Marsman, M, Marzari, N, Nitzsche, U, Nordström, L, Ozaki, T, Paulatto, L, Pickard, CJ, Poelmans, W, Probert, MIJ, Refson, K, Richter, M, Rignanese, GM, Saha, S, Scheffler, M, Schlipf, M, Schwarz, K, Sharma, S, Tavazza, F, Thunström, P, Tkatchenko, A, Torrent, M, Vanderbilt, D, Van Setten, MJ, Van Speybroeck, V, Wills, JM, Yates, JR, Zhang, GX & Cottenier, S 2016, 'Reproducibility in density functional theory calculations of solids', Science, vol. 351, no. 6280, aad3000. https://doi.org/10.1126/science.aad3000

TY - JOUR

T1 - Reproducibility in density functional theory calculations of solids

AU - Lejaeghere, Kurt

AU - Bihlmayer, Gustav

AU - Björkman, Torbjörn

AU - Blaha, Peter

AU - Blügel, Stefan

AU - Blum, Volker

AU - Caliste, Damien

AU - Castelli, Ivano E.

AU - Clark, Stewart J.

AU - Dal Corso, Andrea

AU - De Gironcoli, Stefano

AU - Deutsch, Thierry

AU - Dewhurst, John Kay

AU - Di Marco, Igor

AU - Draxl, Claudia

AU - Dułak, Marcin

AU - Eriksson, Olle

AU - Flores-Livas, José A.

AU - Garrity, Kevin F.

AU - Genovese, Luigi

AU - Giannozzi, Paolo

AU - Giantomassi, Matteo

AU - Goedecker, Stefan

AU - Gonze, Xavier

AU - Grånäs, Oscar

AU - Gross, E. K.U.

AU - Gulans, Andris

AU - Gygi, François

AU - Hamann, D. R.

AU - Hasnip, Phil J.

AU - Holzwarth, N. A.W.

AU - Iuşan, Diana

AU - Jochym, Dominik B.

AU - Jollet, François

AU - Jones, Daniel

AU - Kresse, Georg

AU - Koepernik, Klaus

AU - Küçükbenli, Emine

AU - Kvashnin, Yaroslav O.

AU - Locht, Inka L.M.

AU - Lubeck, Sven

AU - Marsman, Martijn

AU - Marzari, Nicola

AU - Nitzsche, Ulrike

AU - Nordström, Lars

AU - Ozaki, Taisuke

AU - Paulatto, Lorenzo

AU - Pickard, Chris J.

AU - Poelmans, Ward

AU - Probert, Matt I.J.

AU - Refson, Keith

AU - Richter, Manuel

AU - Rignanese, Gian Marco

AU - Saha, Santanu

AU - Scheffler, Matthias

AU - Schlipf, Martin

AU - Schwarz, Karlheinz

AU - Sharma, Sangeeta

AU - Tavazza, Francesca

AU - Thunström, Patrik

AU - Tkatchenko, Alexandre

AU - Torrent, Marc

AU - Vanderbilt, David

AU - Van Setten, Michiel J.

AU - Van Speybroeck, Veronique

AU - Wills, John M.

AU - Yates, Jonathan R.

AU - Zhang, Guo Xu

AU - Cottenier, Stefaan

N1 - Funding Information: ACKNOWLEDGMENTS: This research benefited from financial support from the Research Board of Ghent University; the Fond de la Recherche Scientifique de Belgique (FRS-FNRS), through Projet de Recherches (PDR) grants T.0238.13-AIXPHO and T.1031.14-HiT4FiT; the Communauté Française de Belgique, through the BATTAB project (grant ARC 14/19-057); the U.S. NSF (grant DMR-14-08838); the Swedish Research Council; the Knut and Alice Wallenberg Foundation (grants 2013.0020 and 2012.0031); the Fund for Scientific Research-Flanders (FWO) (project no. G0E0116N); and the U.S. Department of Energy (grant DOE-BES DE-SC0008938). N.A.W.H. was supported by U.S. NSF grant DMR-1105485. J.A.F.-L. acknowledges financial support from the European Union's 7th Framework Marie-Curie Scholarship Program within the ExMaMa Project (project no. 329386). I.D.M., O.E., O.G., D.I., Y.O.K., I.L.M.L., and L.N. acknowledge support from eSSENCE. T.B. was supported by the Academy of Finland (grant 263416) and the COMP Centre of Excellence. C.D., A.G., and S.L. acknowledge support from the Deutsche Forschungsgemeinschaft (DFG) and the Einstein Foundation, Berlin. M.Sche. and C.D. received funding from the European Union's Horizon 2020 research and innovation program under grant agreement no. 676580 with The Novel Materials Discovery (NOMAD) Laboratory, a European Center of Excellence. A.D.C., S.d.G., and E.K. acknowledge support from the Italian Ministry of Education, Universities, and Research (MIUR) through PRIN (Projects of National Interest) 2010-2011 (registration no. 20105ZZTSE-005). P.J.H., D.B.J., and M.I.J.P. are grateful for financial support by the Engineering and Physical Sciences Research Council (EPSRC) under UK Car-Parrinello (UKCP) grant EP/K013564/1. C.J.P. and J.R.Y. acknowledge support from the Collaborative Computational Project for NMR Crystallography under EPSRC grant EP/J010510/1. W.P. acknowledges funding by FWO. D.J. is grateful for financial support by EPSRC under grant EP/J017639/1. S.Sa. acknowledges support from the Swiss National Science Foundation (SNSF). G.-M.R. is thankful for personal financial support from FRS-FNRS. The work by I.E.C. and N.M. was supported by the SNSF's National Centre of Competence in Research MARVEL. G.K. and P.B. acknowledge support by the Austrian Science Fund, project SFB-F41 (ViCoM). S.C. acknowledges financial support from OCAS NV by an OCASendowed chair at Ghent University. Computational resources were as follows: The Ghent University contributors used the Stevin Supercomputer Infrastructure at Ghent University, which is funded by Ghent University, FWO, and the Flemish Government (Economy, Science, and Innovation Department). The Université Catholique de Louvain contributors used the Tier-1 supercomputer of the Fédération Wallonie-Bruxelles (funded by the Walloon Region under grant agreement no. 1117545), the Centre de Calcul Intensif et de Stockage de Masse-Université Catholique de Louvain supercomputing facilities, and the Consortium des Équipements de Calcul Intensif en Fédération Wallonie-Bruxelles (CÉCI) (funded by the FRS-FNRS under convention 2.5020.11). The Science and Technology Facilities Council, Scientific Computing Department's SCARF (Scientific Computing Application Resource for Facilities) cluster was used for the CASTEP calculations. The Basel University and École Polytechnique Fédérale de Lausanne contributors used the Swiss National Supercomputing Center in Lugano. Finland's IT Centre for Science was used for the RSPt calculations. K.L. and F.T. thank C. Becker for instructive discussions on the comparison of atomic-scale simulations. K.L. and S.C. thank W. Dewitte for drafting the summary figure. S.J.C., P.J.H., C.J.P., M.I.J.P., K.R., and J.R.Y. declare the receipt of income from commercial sales of CASTEP by Biovia. N.M. and M.Sche. are members of the Board of Trustees of the Psi-k Electronic Structure Network. P.G. is director of the Quantum ESPRESSO Foundation, and N.M. is a representative member. X.G., D.R.H., M.T., D.C., F.J., and G.-M.R. are members of the Advisory Board of ABINIT, an organization that develops and publishes open-source software related to this article. Commercial software is identified to specify procedures. Such identification does not imply recommendation by the National Institute of Standards and Technology. Atomic Simulation Environment scripts (46) for several of the codes are available online (48). All data are listed in tables S3 to S42.

PY - 2016/3/25

Y1 - 2016/3/25

N2 - The widespread popularity of density functional theory has given rise to an extensive range of dedicated codes for predicting molecular and crystalline properties. However, each code implements the formalism in a different way, raising questions about the reproducibility of such predictions.We report the results of a community-wide effort that compared 15 solid-state codes, using 40 different potentials or basis set types, to assess the quality of the Perdew-Burke-Ernzerhof equations of state for 71 elemental crystals. We conclude that predictions from recent codes and pseudopotentials agree very well, with pairwise differences that are comparable to those between different high-precision experiments. Older methods, however, have less precise agreement. Our benchmark provides a framework for users and developers to document the precision of new applications and methodological improvements.

AB - The widespread popularity of density functional theory has given rise to an extensive range of dedicated codes for predicting molecular and crystalline properties. However, each code implements the formalism in a different way, raising questions about the reproducibility of such predictions.We report the results of a community-wide effort that compared 15 solid-state codes, using 40 different potentials or basis set types, to assess the quality of the Perdew-Burke-Ernzerhof equations of state for 71 elemental crystals. We conclude that predictions from recent codes and pseudopotentials agree very well, with pairwise differences that are comparable to those between different high-precision experiments. Older methods, however, have less precise agreement. Our benchmark provides a framework for users and developers to document the precision of new applications and methodological improvements.

UR - http://www.scopus.com/inward/record.url?scp=84962221807&partnerID=8YFLogxK

U2 - https://doi.org/10.1126/science.aad3000

DO - https://doi.org/10.1126/science.aad3000

M3 - مقالة

SN - 0036-8075

VL - 351

JO - Science

JF - Science

IS - 6280

M1 - aad3000

ER -

Reproducibility in density functional theory calculations of solids

Abstract

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this