Normalizing constant
In probability theory, a normalizing constant or normalizing factor is used to reduce any probability function to a probability density function with total probability of one.
For example, a Gaussian function can be normalized into a probability density function, which gives the standard normal distribution. In Bayes' theorem, a normalizing constant is used to ensure that the sum of all possible hypotheses equals 1. Other uses of normalizing constants include making the value of a Legendre polynomial at 1 and in the orthogonality of orthonormal functions.
A similar concept has been used in areas other than probability, such as for polynomials.
Definition[edit]
In probability theory, a normalizing constant is a constant by which an everywhere non-negative function must be multiplied so the area under its graph is 1, e.g., to make it a probability density function or a probability mass function.[1][2]
Examples[edit]
If we start from the simple Gaussian function
Now if we use the latter's reciprocal value as a normalizing constant for the former, defining a function as
And constant is the normalizing constant of function .
Similarly,
Note that if the probability density function is a function of various parameters, so too will be its normalizing constant. The parametrised normalizing constant for the Boltzmann distribution plays a central role in statistical mechanics. In that context, the normalizing constant is called the partition function.
Bayes' theorem[edit]
Bayes' theorem says that the posterior probability measure is proportional to the product of the prior probability measure and the likelihood function. Proportional to implies that one must multiply or divide by a normalizing constant to assign measure 1 to the whole space, i.e., to get a probability measure. In a simple discrete case we have
For concreteness, there are many methods of estimating the normalizing constant for practical purposes. Methods include the bridge sampling technique, the naive Monte Carlo estimator, the generalized harmonic mean estimator, and importance sampling.[6]
Non-probabilistic uses[edit]
The Legendre polynomials are characterized by orthogonality with respect to the uniform measure on the interval [−1, 1] and the fact that they are normalized so that their value at 1 is 1. The constant by which one multiplies a polynomial so its value at 1 is a normalizing constant.
Orthonormal functions are normalized such that
The constant 1/√2 is used to establish the hyperbolic functions cosh and sinh from the lengths of the adjacent and opposite sides of a hyperbolic triangle.
See also[edit]
References[edit]
- ^ Continuous Distributions at Department of Mathematical Sciences: University of Alabama in Huntsville
- ^ Feller 1968, p. 22
- ^ Feller 1968, p. 174
- ^ Feller 1968, p. 156
- ^ Feller 1968, p. 124
- ^ Gronau, Quentin (2020). "bridgesampling: An R Package for Estimating Normalizing Constants" (PDF). The Comprehensive R Archive Network. Retrieved September 11, 2021.
- Feller, William (1968). An Introduction to Probability Theory and its Applications (volume I). John Wiley & Sons. ISBN 0-471-25708-7.