Lecture #6: Continuous Probability Distributions

MATH 321 – Mathematical Statistics I

Torin Quinlivan
tquinlivan@knox.edu

Knox College

October 17, 2026

Recall: Axioms of Probability

Definition: A probability space is a triple \((\Omega, \mathcal{F} , P)\), where \(\Omega\) is the sample space, \(\mathcal{F}\) is the event space, and \(P\) is a probability law (or measure).

Definition: A probability law is a function \(P : \mathcal{F} \to [0,\, 1]\) that maps an event \(A\) to a real number in \([0,\, 1]\). The function must satisfy the axioms of probability:

  1. Non-Negativity: \(P(A) \ge 0\), for any \(A \in \mathcal{F}\).

  2. Normalization: \(P(\Omega) = 1\).

  3. Additivity: For any pairwise disjoint events \(\{A_1,\, A_2,\, A_3,\, \dots \}\), it must be true that \[ P \left( \bigcup_{i=1}^\infty A_i \right) = \sum_{i=1}^\infty P(A_i). \]

Models of Probability Laws

We have two models for a probability law:

  1. Counting: If \(\Omega\) is finite with equally likely outcomes and \(A \subseteq \Omega\), then \(P(A) = \dfrac{\lvert A \rvert}{\lvert \Omega \rvert}\).
  1. Probability Mass Functions of Discrete RVs: \(X : \Omega \to \mathbb{R}\).

Theorem: A PMF should satisfy the condition that: \[ \sum_{x \in X(\Omega)} p_X(x) = 1 \]

Let’s now generalize this to continuous random variables.

Probability Density Functions

A probability density function \(f_X\) of a random variable \(X\) is a mapping \(f_X : X(\Omega) \to \mathbb{R}\), with the properties

  1. Non-Negativity: \(f_X(x) \ge 0\) for all \(x \in X(\Omega)\).
  2. Unity: \(\displaystyle\int_{X(\Omega)} f_X(x) \, dx = 1\).
  3. Measure of a Set: \(P(X \in A) = \displaystyle\int_A f_X(x) \, dx\).

We take \(f_X(x) = 0\) for any \(x \notin X(\Omega)\).

Probability Density Functions

Any function \(f(x)\) satisfying these properties defines a probability law using integration.

Let \(X\) be a continuous random variable. The probability density function of \(X\) is a function \(f_X : X(\Omega) \to \mathbb{R}\) that, when integrated over an interval \([a,\, b]\), yields the probability of obtaining \(a \le X \le b\): \[ P[a \le X \le b] = \int_a^b f_X(x) \, dx. \]

The probability density function uniquely determines the distribution of a random variable. If \(X(\Omega) = Y(\Omega)\) and \(f_X(t) = f_Y(t)\) for all \(t \in X(\Omega)\), then \(X\) and \(Y\) have the same distribution (but need not be equal as random variables).

Example: Probability Density Function

Example: Let \(f_X(x) = 3x^2\) with \(X(\Omega) = [0, 1]\).

  1. Explain why \(f_X\) is a PDF.
  2. If \(A = [0, 0.5]\), find \(P(X \in A)\).
  3. What does this mean graphically?
  4. What is \(P(0 \le X < 0.5)\)?

Cumulative Distribution Function

Let \(X\) be a continuous random variable. The cumulative distribution function (CDF) of \(X\) is: \[ F_X(x) \overset{def}{=} P[X \le x] = \int_{-\infty}^x f_X(t) \, dt. \]

Let \(X\) be a continuous random variable. Then, for any \(a \le b\): \[ P[a \le X \le b] = F_X(b) - F_X(a) \]

The survival function of \(X\) is: \[ S_X(x) = 1 - F_X(x) \overset{def}{=} P[X > x] = \int_{x}^\infty f_X(t) \, dt. \]

The Cumulative Distribution Function vs. Probability Density Function

The probability density function (PDF) is the derivative of the cumulative distribution function (CDF): \[ f_X(x) = \frac{d}{dx} F_X(x), \] provided \(F_X\) is differentiable at \(x\).

If \(F_X\) has a jump discontinuity at \(x = x_0\), then \(f_X(x)\) contains the term \[ P[X = x_0] \, \delta(x - x_0). \]

Example: Cumulative Distribution Function

Example: Let \(f_X(x) = 3x^2\) with \(X(\Omega) = [0, 1]\).

  1. What is the CDF \(F_X(x)\) of \(X\)?
  2. Find \(P\left(X \le \dfrac{1}{4} \right)\) with the PDF \(f_X(x)\).
  3. Find \(P\left(X \le \dfrac{1}{4} \right)\) with the CDF \(F_X(x)\).

Properties of the Cumulative Distribution Function

Theorem: Let \(X\) be a random variable (either continuous or discrete), then the CDF of \(X\) has the following properties:

  1. The CDF is nondecreasing.
  2. The CDF approaches 1 as \(x\) approaches \(\infty\): \(\displaystyle\lim_{x \to \infty} F_X(x) = 1\).
  3. The CDF approaches 0 as \(x\) approaches \(-\infty\): \(\displaystyle\lim_{x \to -\infty} F_X(x) = 0\).

Continuity of the Cumulative Distribution Function

Theorem: For any random variable \(X\) (discrete or continuous), \(F_X(x)\) is always right-continuous. That is, \[ F_X(b) = F_X(b^+) \overset{def}{=} \lim_{h \to 0^+} F_X(b + h). \]

Theorem: For any random variable \(X\) (discrete or continuous), \(P(X = b)\) is: \[ P[X = b] = \begin{cases} F_X(b) - F_X(b^-), & \text{if $F_X$ is discontinuous at $x = b$,} \\ 0, & \text{otherwise}, \end{cases} \] where \(F_X(b^-) \overset{def}{=} \displaystyle\lim_{h \to 0^+} F_X(b - h)\).

The Expected Value of a Continuous Random Variable

We can generalize the definition of the expected value we developed with discrete random variables and sums to continuous random variables and integrals.

The expectation of a continuous random variable \(X\) is: \[ E[X] = \int_{X(\Omega)} x f_X(x) \, dx. \]

Theorem: Let \(g : X(\Omega) \to \mathbb{R}\) be a function and \(X\) be a continuous random variable. Then: \[ E\Big[ g(X) \Big] = \int_{X(\Omega)} g(x) f_X(x) \, dx. \]

The \(n^{th}\) Moment of a Continuous Random Variable

The \(n^{th}\) moment of a continuous random variable \(X\) is \[ E[X^n] = \int_{X(\Omega)} x^n f_X(x) \, dx. \]

The \(n^{th}\) central moment of a continuous random variable \(X\) is \[ E[(X-\mu)^n] = \int_{X(\Omega)} (x-\mu)^n f_X(x) \, dx, \] where \(\mu = E[X]\).

The Variance of a Continuous Random Variable

The same thing applies to the variance.

The variance of a continuous random variable \(X\) is: \[ V[X] = \int_{X(\Omega)} (x-\mu)^2 f_X(x) \, dx. \]

The formulas concerning the variance and the expected value hold, such as \(V(X) = E(X^2) - [E(X)]^2\).

Example: Expectation and Variance

Example: Let \(f_X(x) = 3x^2\) with \(X(\Omega) = [0, 1]\).

  1. What is the expected value of \(X\), \(E(X)\)?
  2. What is the expected value of \(X^2\), \(E(X^2)\)?
  3. What is the expected value of \(4X + 9\), \(E(4X + 9)\)?
  4. What is the variance of \(X\), \(V(X)\)?

The Uniform Distribution

Let \(X\) be a continuous uniform random variable.

The probability density function of \(X\) is: \[ f_X(x) = \begin{cases} \dfrac{1}{b-a}, & a \le x \le b, \\ 0, & \text{otherwise}, \end{cases} \] where \([a, b]\) is the interval on which \(X\) is defined.

The Uniform Distribution

We write \[ X \sim \text{Uniform}(a, b) \] to mean that \(X\) is drawn from a uniform distribution on an interval \([a, b]\).

The Uniform Distribution

The mean of a uniformly distributed random variable is: \[ E(X) = \frac{a + b}{2}. \]

Note that this is the midpoint of the interval on which \(X\) is defined.

The variance of a uniformly distributed random variable is: \[ V(X) = \frac{(b-a)^2}{12}. \]

Example: The Uniform Distribution

Example: Suppose a train has a uniform probability of arriving between 0 and 10 minutes late.

  1. What is the probability a train will arrive between 0 and three minutes late?
  2. What is the probability it will arrive exactly 3 minutes late?
  3. What is the probability that a train will arrive at least 8 minutes late?
  4. What is the probability that a train will arrive between 3 and 8 minutes late?
  5. What is the average amount of time a train will arrive past the scheduled time?
  6. What is the variance of the amount of time a train will arrive past the scheduled time?

The Exponential Distribution

Let \(X\) be an exponential random variable.

The probability density function of \(X\) is: \[ f_X(x) = \begin{cases} \dfrac{1}{\beta} e^{-\frac{x}{\beta}}, & x \ge 0, \\ 0, & \text{otherwise}, \end{cases} \] where \(\beta > 0\) is a parameter.

The Exponential Distribution

We write \[ X \sim \text{Exponential}(\beta) \] to mean that \(X\) is drawn from an exponential distribution of parameter \(\beta\).

The mean of an exponentially distributed random variable is: \[ E(X) = \beta. \]

The variance of an exponentially distributed random variable is: \[ V(X) = \beta^2. \]

Example: The Exponential Distribution

Example: Suppose a train is always late but the distribution of (late) arrival times is given by an exponential distribution with \(\beta = 5\).

  1. What is the probability a train will arrive between 0 and three minutes late?
  2. What is the probability it will arrive exactly 3 minutes late?
  3. What is the probability that a train will arrive at least 8 minutes late?
  4. What is the probability that a train will arrive between 3 and 8 minutes late?
  5. What is the average amount of time a train will arrive past the scheduled time?
  6. What is the variance of the amount of time a train will arrive past the scheduled time?

The Gamma Distribution

Let \(X\) be a gamma random variable.

The probability density function of \(X\) is: \[ f_X(x) = \begin{cases} \dfrac{1}{\Gamma(\alpha) \beta^\alpha} x^{\alpha - 1} e^{-\frac{x}{\beta}}, & x \ge 0, \\ 0, & \text{otherwise}, \end{cases} \] where \(\alpha > 0\) (shape) and \(\beta > 0\) (scale) are two parameters.

The Gamma function

\(\Gamma(z)\) is the gamma function, defined for \(z > 0\) as: \[ \Gamma(z) = \int_0^\infty t^{z - 1} e^{-t} \, dt. \]

More usefully, for \(z > 1\), this function has the property: \[ \Gamma(z) = (z - 1) \Gamma(z - 1). \]

If \(z \in \mathbb{Z}^+\), then \(\Gamma(z) = (z-1)!\). Also, \(\Gamma\left( \dfrac{1}{2} \right) = \sqrt{\pi}\).

The Gamma Distribution

We write \[ X \sim \text{Gamma}(\alpha, \beta) \] to mean that \(X\) is drawn from a Gamma distribution with parameters \(\alpha\) and \(\beta\).

If \(X \sim \text{Gamma}(\alpha = 1, \beta)\), then \(X \sim \text{Exponential}(\beta)\).

The Gamma Distribution

The mean of a Gamma distributed random variable is: \[ E(X) = \alpha \beta. \]

The variance of a Gamma distributed random variable is: \[ V(X) = \alpha \beta^2. \]

The Beta Distribution

Let \(X\) be a beta random variable.

The probability density function of \(X\) is: \[ f_X(x) = \begin{cases} \dfrac{\Gamma(\alpha + \beta)}{\Gamma(\alpha) \Gamma(\beta)} x^{\alpha - 1} (1 - x)^{\beta - 1}, & 0 \le x \le 1, \\ 0, & \text{otherwise}, \end{cases} \] where \(\alpha > 0\) and \(\beta > 0\) are two parameters.

The Beta Distribution

We write \[ X \sim \text{Beta}(\alpha, \beta) \] to mean that \(X\) is drawn from a Beta distribution with parameters \(\alpha\) and \(\beta\).

The Beta Distribution

The mean of a Beta distributed random variable is: \[ E(X) = \frac{\alpha}{\alpha + \beta}. \]

The variance of a Beta distributed random variable is: \[ V(X) = \frac{\alpha \beta}{(\alpha + \beta)^2 (\alpha + \beta + 1)}. \]