subject: The characteristic function [print this page] The characteristic function The characteristic function
Characteristic function - The characteristic function is one of the most useful objects in probability and statistics. You can read a thorough introduction to the characteristic function on Statlect.com, the digital textbook on probability theory and mathematical statistics.
Let X be a random variable. The characteristic function f(t) is defined as f(t)=E[exp(itX)] where E denotes the expected value and i=sqrt(-1) is the imaginary unit.
Like the moment generating function, the characteristic function completely characterizes the distribution of a random variable (i.e. if two random variables have the same characteristic function, then they have the same distribution). However, unlike the moment generating function, the characteristic function always exists. Therefore it can be used to prove general results, that do not require existence of the moment generating function.
The characteristic function always exists because E[exp(itX)]=E[cos(tX)]+iE[sin(tX)], the trigonometric functions sin and cos are bounded between -1 and 1 and therefore the two expected values E[cos(tX)] and E[sin(tX)] always exist and are finite.
The characteristic function is used to prove important results in probability theory and statistics. Probably, the most important result that can be proved using characteristic functions is the Central Limit Theorem (the fact that a sample average tends to converge to a normal distribution when the sample size increases).
Why is the characteristic function so popular? The reason is that often we want to derive the distribution of a sum of two or more random variables. This difficult task becomes easy using the characteristic function, because the characteristic function of a sum of two random variables is just the product of the characteristic functions of the two variables. Once we know the characteristic function of the sum, this uniquely identifies a distribution. Therefore, using characteristic function, finding the distribution of a sum boils down to performing a multiplication.
The characteristic function can also be used to easily derive the moments (mean, variance, kurtosis, etc.) of a random variable. In fact, the moments of a random variable are just the derivatives of the characteristic function, evaluated at 0 and multiplied by a power of the imaginary unit.
Two classical probability theory books that describe in detail the properties of the characteristic function are:
[1] Resnick, S.I. (1999) "A Probability Path", Birkhauser;
[2] Williams, D. (1991) "Probability with martingales", Cambridge University Press.
The first of this two contains very detailed explanations while the second is more concise.
You can also find a detailed introduction to the characteristic function and its properties on Statlect.com, a website that publishes free lectures on probability, statistics and econometrics. StatLect lectures are aimed at students that want to learn probability theory at an intermediate level of mathematical sophistication and has lots of step-by-step explanations and several exercises and multiple choice verification tests.
