Persistent Bias in Probabilistic Production Forecasting and Simple Methods to Overcome It
by Reidar B Bratvold with University of Stravanger
Abstract
Unbiased probabilistic production forecasts are a crucial component in making value-creating investment decisions, but typical methods result in persistent, value-destroying biases. There are quick and easy ways to make accurate, unbiased forecasts.
Increased awareness of uncertainty, combined with increasingly sophisticated tools and models for quantifying it, is causing a shift from deterministic to probabilistic production forecasting. For these forecasts to lead to better investment decisions (e.g. assessing how much it’s worth paying to reduce uncertainty, or incorporating flexibility to manage it), they need to be an accurate (unbiased) representation of the uncertainty. However, our industry has a general history of overconfidence (ranges too narrow) and, more damaging for valuecreation, optimism (consistent over-estimation).
A large dataset of historical probabilistic production forecasts was investigated for potential bias by comparing them with actual outcomes. They were found to be both optimistic and overconfident, bringing into question their usefulness for decision-making and the value of the sophisticated uncertainty modeling techniques that were used to generate them.
Fortunately, there are quick and easy methods for creating accurate probabilistic forecasts. We describe these and show that they would have produced accurate forecasts for our case study fields (they do not make use of knowing the actual outcomes!).
Finally, preliminary results from a similar analysis of renewables projects indicate that the same problems exist. This could significantly impact government and company investment decisions and policy development in renewable projects.
Biography
Reidar Bratvold is a Professor of Investment & Decision Analysis at the University of Stavanger, specializing in decision analysis, data analytics, and behavioral challenges in decision-making. After 15 years of industry experience in technical and leadership roles, Reidar transitioned to academia, where he has contributed significantly to his fields of expertise. He co-authored 'Making Good Decisions,' has served as an SPE Distinguished Lecturer, and received the 2017 SPE Management & Information Award. A Fellow of the Society of Decision Professionals and the Norwegian Academy of Technological Sciences, Reidar earned his PhD in Engineering and MSc in Mathematics from Stanford University.
