Probability management

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

The discipline of probability management communicates and calculates uncertainties as data structures that obey both the laws of arithmetic and probability. The simplest approach is to use vector arrays of simulated or historical realizations and meta data called Stochastic Information Packets (SIPs). A set of SIPs, which preserve statistical relationships between variables, is said to be coherent and is referred to as a Stochastic Library Unit with Relationships Preserved (SLURP). SIPs and SLURPs allow stochastic simulations to communicate with one another. For example, see Analytica (Wikipedia), Analytica (SIP page), Oracle Crystal Ball, Frontline Solvers, and Autobox.

The first large documented application of SIPs involved the exploration portfolio of Royal Dutch Shell in 2005 as reported by Savage, Scholtes, and Zweidler, who formalized the discipline of probability management in 2006.[1] The topic is also explored at length in.[2]

Vectors of simulated realizations of probability distributions have been used to drive stochastic optimization since at least 1991.[3] Andrew Gelman described such arrays of realizations as Random Variable Objects in 2007.[4]

A recent approach does not store the actual realizations, but delivers formulas known as Virtual SIPs that generate identical simulation trials in the host environment regardless of platform. This is accomplished through inverse transform sampling, also known as the F Inverse method, coupled to a portable pseudo random number generator, which produces the same stream of uniform random numbers across platforms. Quantile parameterized distributions (QPDs) are convenient for inverse transform sampling in this context. In particular, the Metalog distribution is a flexible continuous probability distribution that has simple closed form equations, can be directly parameterized by data, using only a handful of parameters.[5] An ideal pseudo random number generator for driving inverse transforms is the HDR generator developed by Douglas W. Hubbard. It is a counter-based generator with a four-dimensional seed plus an iteration index that runs in virtually all platforms including Microsoft Excel.[6] This allows simulation results derived in R, Python, or other readily available platforms to be delivered identically, trial by trial to a wide audience in terms of a combination of a few parameters for a Metalog distribution accompanied by the five inputs to the HDR generator.

In 2013, was incorporated as a 501(c)(3) nonprofit that supports this approach through education, tools, and open standards. Executive Director Sam Savage is the author of The Flaw of Averages: Why we Underestimate Risk in the Face of Uncertainty and is an Adjunct Professor at Stanford University. Harry Markowitz, Nobel laureate in Economics, was a co-founding board member. The nonprofit has received financial support from Chevron Corporation, General Electric, Highmark Health, Kaiser Permanente, Lockheed Martin, PG&E, and Wells Fargo Bank. The SIPmath 2.0 Standard supports XLSX, CSV, and XML Formats.[7] The SIPmath 3.0 Standard uses JSON objects to convey virtual SIPs based on the Metalog Distribution and HDR Generator.


  1. ^ Probability Management, Sam Savage, Stefan Scholtes and Daniel Zweidler, OR/MS Today, February 2006, Volume 33 Number 1
  2. ^ Savage, Sam (2009). The Flaw of Averages, Why we Underestimate Risk in the Face of Uncertainty. Hoboken: John Wiley & Sons. ISBN 978 0-471-38197-6.
  3. ^ Dembo, Ron (1991). "Scenario Optimization". Annals of Operations Research. 30: 63–80. doi:10.1007/BF02204809.
  4. ^ Gelman, Andrew (2007). "Manipulating and summarizing posterior simulations using random variable objects". Statistics and Computing. 17 (3): 235–244. doi:10.1007/s11222-007-9020-4.
  5. ^ The Metalog Distributions by Thomas W. Keelin. Introduces the metalog distributions and describes their applications. Decision Analysis Vol. 13, No. 4, December 2016.
  6. ^ A Multi-Dimensional, Counter-Based Pseudo Random Number Generator as a Standard for Monte Carlo Simulations by Doug Hubbard. Proceedings of the 2019 Winter Simulation Conference.
  7. ^ SIPmath Standard