Lecture #9: Moment-Generating Functions

MATH 321 – Mathematical Statistics I

Torin Quinlivan
tquinlivan@knox.edu

Knox College

October 19, 2026

Moment-Generating Functions

The moment-generating function (MGF) \(M_X(t)\) of a random variable \(X\) is: \[ M_X(t) = E[e^{tX}], \] provided this expectation is finite for all \(t\) in some open interval \((-h, h)\) around \(0\).

Moment-Generating Function Example

Example:

  1. Find the moment-generating function for \(X \sim \text{Exponential}(\beta)\).
  2. Find the moment-generating function for \(X \sim \text{Bernoulli}(p)\).
  3. Find the moment-generating function for \(X \sim \text{Binomial}(n, p)\).

Properties of Moment-Generating Functions

Theorem: If the moment-generating function of \(X\) exists, then:

  1. \(M_X(0) = 1\).
  2. \(\dfrac{d}{dt} M_X(t) \bigg\rvert_{t=0} = E[X]\).
  3. \(\dfrac{d^2}{dt^2} M_X(t) \bigg\rvert_{t=0} = E[X^2]\).
  4. \(\dfrac{d^k}{dt^k} M_X(t) \bigg\rvert_{t=0} = E[X^k]\), for any positive integer \(k\).

Example: Given that the MGF of \(X \sim \text{Exponential}(\beta)\) is \(M_X(t) = (1 - \beta t)^{-1}\) for \(t < \frac{1}{\beta}\), use it to prove that \(E[X] = \beta\) and \(V(X) = \beta^2\).

Properties of Moment-Generating Functions

Theorem: Let \(X\) and \(Y\) be independent random variables. Let \(Z = X + Y\). Then \[ M_Z(t) = M_X(t) M_Y(t). \]

Corollary: Consider independent random variables \(X_1, \dots, X_N\). Let \(Z = \sum_{i=1}^N X_i\) be the sum of random variables. Then the MGF of \(Z\) is \[ M_Z(t) = \prod_{i=1}^N M_{X_i}(t). \]

What’s the Deal with the MGF?

The moment-generating function of a random variable uniquely defines the distribution.

Theorem: Let \(M_X(t)\) and \(M_Y(t)\) denote the moment-generating functions of random variables \(X\) and \(Y\). If both moment-generating functions exist and \(M_X(t) = M_Y(t)\) for all \(t\) in some open interval \((-h, h)\) around \(0\), then \(X\) and \(Y\) have the same probability distribution.

The moment-generating function does not exist for all distributions. For example, the Cauchy distribution does not have an MGF.

Moment-Generating Functions of i.i.d. Random Variables

Theorem: Let \(X_1, \dots, X_N\) be independent and identically distributed random variables with common moment-generating function \(M_X(t)\), and let \(Z = X_1 + \dots + X_N\). Then the moment-generating function of \(Z\) is \[ M_Z(t) = \Big(M_X(t)\Big)^N. \]

MGFs of Sums of Independent Random Variables

Examples:

  1. Let \(X_1, \dots, X_N\) be a sequence of i.i.d. Bernoulli random variables with parameter \(p\). Let \(Z = X_1 + \dots + X_N\). What is the distribution of \(Z\)?
  2. Let \(X_1, \dots, X_N\) be a sequence of i.i.d. binomial random variables with parameters \(n\) and \(p\). Let \(Z = X_1 + \dots + X_N\). What is the distribution of \(Z\)?
  3. Let \(X_1, \dots, X_N\) be independent random variables with \(X_i \sim \text{Poisson}(\lambda_i)\). Let \(Z = X_1 + \dots + X_N\). What is the distribution of \(Z\)?
  4. Let \(X_1, \dots, X_N\) be independent random variables with \(X_i \sim \text{Normal}(\mu_i, \sigma_i^2)\). Let \(Z = X_1 + \dots + X_N\). What is the distribution of \(Z\)?

The Characteristic Function

There is a related function that does exist for all distributions, the characteristic function \(\varphi_X(t)\).

The characteristic function \(\varphi_X(t)\) of a random variable \(X\) is: \[ \varphi_X(t) = E[e^{itX}]. \]

Properties of the Characteristic Function

Properties:

  1. \(\varphi_X(-it) = M_X(t)\), whenever the MGF exists.
  2. \(\dfrac{1}{i^k} \dfrac{d^k}{dt^k} \varphi_X(t) \bigg\rvert_{t=0} = E[X^k]\), for any positive integer \(k\) with \(E\big[\lvert X \rvert^k\big] < \infty\).
  3. Consider independent random variables \(X_1, \dots, X_N\). Let \(Z = \sum_{i=1}^N X_i\) be the sum of random variables. Then the characteristic function of \(Z\) is \[ \varphi_Z(t) = \prod_{i=1}^N \varphi_{X_i}(t). \]