Multi-Model Approach of Global Energy Model Validation: Times and EN-ROADS Models

Michael Ryder¹, Solomon Evro^1, , Caleb Brown¹ and Olusegun S. Tomomewo¹

¹Institute for Energy Studies, University of North Dakota, Grand Forks, USA

Cite this paper:
Michael Ryder, Solomon Evro, Caleb Brown and Olusegun S. Tomomewo. Multi-Model Approach of Global Energy Model Validation: Times and EN-ROADS Models. American Journal of Energy Research. 2023; 11(2):63-81. doi: 10.12691/ajer-11-2-2

Abstract

Energy System Modeling tools are becoming ever-prevalent in global society to help decide factors in energy policymaking, power production methods, and means of environmental impact assessment. Energy system engineers need to be aware of the use of energy system models due to the complexity of the systems and the demand for model use in evaluating an energy system. This literature review will cover the importance of energy system models and the most recent advances in modeling technology, the accepted methods of model evaluation and validation before the use of an energy system model, and lastly, demonstrate a comparative analysis validation technique with three case studies using a multi-model approach, by applying two widely accepted global energy models. The two global energy models evaluated are the Integrated MARKAL-EFOM System (TIMES) and Energy-Rapid Overview and Decision-Support (EN-ROADS) models. The comparative analysis will be demonstrated by reviewing three base cases, whether 2.5°C average warming is achievable within the desired timeline, the projected global energy supply, and practical climate change mitigation scenarios. The comparative analysis results show that two globally accepted energy system models still predict different outcomes with the same inputs. The comparative analysis results exemplify the necessity for energy system engineers or other model users to properly benchmark and validate any model they decide to use for decision-making before accepting model results.

Keywords:
global energy and climate models energy modeling emission and temperature forecast EN-ROADS TIMES energy policy model validation model calibration and integrated assessment models

This work is licensed under a Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

References:

[1]	NASA. (2022). Assessing the Global climate in 2021, 2021 was the sixth-warmest year on record for the globe. https://www.ncei.noaa.gov/news/global-climate-202112.
[2]	Fecht, S. (2021). How Exactly Does Carbon Dioxide Cause Global Warming. Columbia Climate School. https://lamont.
[3]	Khandekar, M. L., Murty, T. S., & Chittibabu, P. (2005). The global warming debate: A review of the state of science. Pure and Applied Geophysics, 162(8), 1557-1586.
[4]	O’Driscoll, J. (2021). The most extreme weather events in 2021. The Week. https://www.theweek.co.uk/news/environment/953574/worlds-most-extreme-weather-events-2021.
[5]	Sharma, G. (2021). Late monsoon floods kill more than 150 in India and Nepal. https://www.reuters.com/world/india/flood-deaths-india-nepal-cross-150-2021-10-21/.
[6]	National Research Council. (2014). Convergence: Facilitating transdisciplinary integration of life sciences, physical sciences, engineering, and beyond. National Academies Press.
[7]	NOAA Climate.gov. (2022). Climate Models. https://www.climate.gov/maps-data/climate-data-primer/predicting-climate/climate-models.
[8]	NOAA. (2021). National Oceanic and Atmospheric Administration, Climate Models, and How They Work. https://www.climate.gov/maps-data/climate-data-primer/predicting-climate/climate-models.
[9]	IEA. (2022). Global Energy and Climate Model. CC BY.
[10]	Weyant, J. (2017). Some contributions of integrated assessment models of global climate change. Review of Environmental Economics and Policy.
[11]	Hall, L. M., & Buckley, A. R. (2016). A review of energy systems models in the UK: Prevalent usage and categorisation. Applied Energy, 169, 607-628.
[12]	IEA-ETSAP. (2022). Technology Collaboration Programme by IEA. https://iea-etsap.org/index.php/etsap-tools/model-generators/markal.
[13]	Siberry, V., Wu, D., Wang, D., & Ma, X. (2022). Energy Storage Valuation: A Review of Use Cases and Modeling Tools.
[14]	Filippov, S. (2023). Forecasting of technological development of energy industry: Issues of methodology and practice. AIP Conference Proceedings, 2552(1), 080001.
[15]	Van De Ven, D.-J., Gambhir, A., Doukas, H., Giarola, S., Hawkes, A., Koasidis, K., Koberle, A., Lamboll, R., McJeon, H., & Perdana, S. (2022). A multi-model analysis of post-Glasgow climate action and feasibility gap.
[16]	Beek, L. van, Oomen, J., Hajer, M., Pelzer, P., & Vuuren, D. van. (2022). Navigating the political: An analysis of political calibration of integrated assessment modelling in light of the 1.5°C goal. Environmental Science & Policy, 133, 193-202.
[17]	Gardumi, F., Keppo, I., Howells, M., Pye, S., Avgerinopoulos, G., Lekavičius, V., Galinis, A., Martišauskas, L., Fahl, U., Korkmaz, P., Schmid, D., Montenegro, R. C., Syri, S., Hast, A., Mörtberg, U., Balyk, O., Karlsson, K., Pang, X., Mozgeris, G., … Mikulić, M. (2022). Carrying out a multi-model integrated assessment of European energy transition pathways: Challenges and benefits. Energy, 258, 124329.
[18]	Loulou, R., Goldstein, G., Kanudia, A., Lettila, A., Remme, U., & Noble, K. (2016). Documentation for the TIMES Model PART I-Concepts and Theory. Energy Technology Systems Analysis Programme. https://iea-etsap.org/docs/Documentation_for_the_TIMES_Model-Part-I_July-2016.pdf.
[19]	Allena-Ozolina, S., Pakere, I., Jaunzems, D., Blumberga, A., & Bazbauers, G. (2020). Integrated MARKAL-EFOM System (TIMES) model for energy sector modelling. 2020 IEEE 61th International Scientific Conference on Power and Electrical Engineering of Riga Technical University (RTUCON, 1-7.
[20]	Densing, M., Turton, H., & Bäuml, G. (2012). Conditions for the successful deployment of electric vehicles–a global energy system perspective. Energy, 47(1), 137-149.
[21]	Siegel, L., Delibas, A., Eker, S., Fiddaman, T., Frank, T., Homer, J., & al. (2022). En-Roads Simulator Reference Guide. En-Roads.
[22]	Kapmeier, F., Sterman, J., Siegel, L., Eker, S., Fiddaman, T., Homer, J., & Jones, A. (2021). En-ROADS: A Global Energy and Climate Simulator to Support Strategic Thinking and Public Outreach. EGU General Assembly Conference Abstracts, 21-7608.
[23]	Swanborough, M. (2022). A Quantification of Seven Political Parties. In Climate Proposals in the 2022 Swedish General Election.
[24]	Beniugă, R., Beniugă, O., Machidon, D., & Istrate, M. (2021). Wind Power in Romania Energy Mix Towards Sustainable Development. 2021 9th International Conference on Modern Power Systems (MPS, 14.
[25]	Edwards, P. N. (2011). History of climate modeling. Wiley Interdisciplinary Reviews: Climate Change, 2(1), 128-139.
[26]	Oreskes, N., Shrader-Frechette, K., & Belitz, K. (1994). Verification, validation, and confirmation of numerical models in the earth sciences. Science, 263, 641-646.
[27]	Gass, S. I. (1980). Validation and Assessment Issues on Energy Models: Proceedings of a Workshop Held at the National Bureau of Standards, Gaithersburg, Maryland, January 10-11, 1979 (Vol. 569). US Department of Commerce, National Bureau of Standards.
[28]	Tolbert, P. S., & Hall, R. H. (2015). Organizations: Structures, processes and outcomes. Routledge.
[29]	Pfenninger, S., Hawkes, A., & Keirstead, J. (2014). Energy Systems Modeling for twenty-first century energy challenges. Renewable and Sustainable Energy Reviews, 33, 74-86.
[30]	Hoffman, K. (2011). Perspectives on the Validation of Energy System Models. In International Energy Agency (IEA), Energy Technology Systems Analysis Program (ETSAP) Workshop.
[31]	Fattahi, A., Sijm, J., & Faaij, A. (2020). A systemic approach to analyze Integrated Energy System Modeling Tools: A review of national models. Renewable and Sustainable Energy Reviews, 133, 110195.
[32]	Sterman, J. (2010). Business dynamics: Systems thinking and modeling for a complex world. McGraw-Hill Education.
[33]	Kriechbaum, L., Scheiber, G., & Kienberger, T. (2018). Grid-based multi-energy systems—Modelling, assessment, open-source modelling frameworks and challenges. Energy, Sustainability and Society, 8(1).
[34]	Dodds, P., Keppo, I., & Strachan, N. (2015). Characterizing the Evolution of Energy System Models Using Model Archaeology. Environ Model Assess, 20, 83-102.
[35]	Morrison, R. (2018). Energy system modeling: Public transparency, scientific reproducibility, and open development. Energy Strategy Reviews, 20, 49-63.
[36]	Stoll, B., Brinkman, G., Townsend, A., & Bloom, A. (2016). Analysis of modeling assumptions used in production cost models for Renewable Integration Studies.
[37]	Binsted, M., Iyer, G., Cui, R., Khan, Z., Dorheim, K., & Clarke, L. (2020). Evaluating long-term model-based scenarios of the Energy System. Energy Strategy Reviews, 32, 100551.
[38]	Paltsev, S. (2017). Energy scenarios: The value and limits of scenario analysis: The value and limits of energy scenario analysis. Wiley Interdisciplinary Reviews: Energy and Environment, 6. e242, 10 1002 242.
[39]	Prina, M. G., Nastasi, B., Groppi, D., Misconel, S., Garcia, D. A., & Sparber, W. (2022). Comparison methods of energy system frameworks, models, and scenario results. Renewable and Sustainable Energy Reviews, 167, 112719.
[40]	Gutiérrez González, V., Ramos Ruiz, G., & Fernández Bandera, C. (2020). Empirical and comparative validation for a building energy model calibration methodology. Sensors, 20(17), 5003.
[41]	Judkoff, R., Wortman, D., O’doherty, B., & Burch, J. (2008). Methodology for Validating Building Energy Analysis Simulations; Technical Report; National Renewable Energy Lab. NREL.
[42]	Khan, Z., Linares, P., & García-González, J. (2017). Integrating water and energy models for policy driven applications. A review of contemporary work and recommendations for future developments. Renewable and Sustainable Energy Reviews, 67, 1123-1138.
[43]	Niet, T., Arianpoo, N., Kuling, K., & Wright, A. S. (2022). Increasing the reliability of energy system scenarios with integrated modelling: A Review. Environmental Research Letters, 17(4), 043006.
[44]	Ruiz, G. R., & Bandera, C. F. (2017). Validation of Calibrated Energy Models: Common Errors. Energies, 10, 1587.
[45]	DOE, U. S. (2015). M&V Guidelines: Measurement and Verification for Performance-Based Contracts Version (Vol. 4, Issue 0). FEMP (Federal Energy Management Program.
[46]	Bemp. (2022). Building Energy Modeling Professional Certification. https://www.ashrae.org/professional-development/ashrae-certification/certification-types/bemp-building-energy-modeling-professional-certification.
[47]	Heating, A. S., Refrigerating, & Engineers, A.-C. (2014). Ashrae Guideline 14-2014: Measurement of Energy, demand, and water savings.
[48]	NREL. (2002). International Performance Measurement and Verification Protocol: Concepts and options for Determining Energy and Water Savings: Vol. I (Issue revised).
[49]	Team, M. (2022). IPMVP®: Measure and verify your energy efficiency projects. Energis. https://energis.cloud/en/ipmvp-measurement-and-verification-of-energy-savings/
[50]	Eker, S., Zimmermann, N., Carnohan, S., & Davies, M. (2017). Participatory system dynamics modelling for housing, Energy and Wellbeing Interactions. Building Research & Information, 46(7), 738-754.
[51]	Bernardo, G., & D’Alessandro, S. (2019). Societal Implications of Sustainable Energy Action Plans: From Energy Modelling to stakeholder learning. Journal of Environmental Planning and Management, 62(3), 399-423.
[52]	McGookin, C., Gallachóir, Ó., B., & Byrne, E. (2021). Participatory methods in energy system modelling and Planning – A Review. Renewable and Sustainable Energy Reviews, 151, 111504.
[53]	EPA. (2022). EPAUS9rT - An Energy Systems Database for use with the TIMES Model. https://www.epa.gov/air-research/epaus9rt-energy-systems-database-use-times-model.
[54]	Tomaschek, J. (2014). Long-term optimization of the transport sector to address greenhouse gas reduction targets under rapid growth application of an energy system model for Gauteng Province, South Africa (dissertation). Technische Informationsbibliothek u. Universitätsbibliothek.
[55]	Tomaschek, J., Haasz, T., & Fahl, U. (2016). Concentrated solar power generation: Firm and dispatchable capacity for Brazil’s solar future? AIP Conference Proceedings.
[56]	Dodds, P. (2021). Review of the Scottish TIMES energy system model.
[57]	Remme, U., Labriet, M., Loulou, R., & Blesl, M. (2007). Global Energy Supply: Model-based Scenario Analysis of Resource Use and Energy Trade (p. 21).
[58]	Nakicenovic. (2000). Special Report on Emissions Scenarios. A Special Report of Working Group III of the Intergovernmental Panel on Climate Change. Cambridge.
[59]	Syri, S., Lehtilä, A., Ekholm, T., Savolainen, I., Holttinen, H., & Peltola, E. (2008). Global energy and emissions scenarios for effective climate change mitigation—Deterministic and stochastic scenarios with the TIAM model. International Journal of Greenhouse Gas Control, 2(2), 274-285.
[60]	Huntzinger, & Michalak. (2017). Uncertainty in the response of terrestrial carbon sink to environmental drivers undermines carbon-climate feedback predictions | Scientific Reports. https://www.nature.com/articles/s41598-017-03818-2.