Lecture #6: Continuous Probability Distributions

MATH 321 – Mathematical Statistics I

Torin Quinlivan
tquinlivan@knox.edu

Knox College

October 6, 2025

Recall: Axioms of Probability

Definition: A probability space is a triple \((\Omega, \mathcal{F} , P)\), where \(\Omega\) is the sample space, \(\mathcal{F}\) is the event space, and \(P\) is a probability law (or measure).

Definition: A probability law is a function \(P : \mathcal{F} \to [0,\, 1]\) that maps an event \(A\) to a real number in \([0,\, 1]\). The function must satisfy the axioms of probability:

  1. Non-Negativity: \(P(A) \ge 0\), for any \(A \subseteq \Omega\).

  2. Normalization: \(P(\Omega) = 1\).

  3. Additivity: For any disjoint sets \(\{A_1,\, A_2,\, A_3,\, \dots \}\), it must be true that \[ P \left( \bigcup_{i=1}^\infty A_i \right) = \sum_{i=1}^\infty P(A_i). \]

Models of Probability Laws

We have two models for a probability law:

  1. Counting: If \(A \subseteq \Omega\), then \(P(A) = \dfrac{\lvert A \rvert}{\lvert Ω\rvert}\).
  1. Probability Mass Functions of Discrete RVs: \(X : \Omega \to \mathbb[R]\).

Theorem: A PMF should satisfy the condition that: \[ \sum_{x \in X(\Omega)} p_X(x) = 1 \]

Let’s now generalize this to continuous random variables.

Probability Density Functions

A probability density function \(f_X\) of a random variable \(X\) is a mapping \(f_X : \Omega \to \mathbb{R}\), with the properties

  1. Non-Negativity: \(f_X(x) \ge 0\) for all \(x \in \Omega\).
  2. Unity: \(\displaystyle\int_\Omega f_X(x) dx = 1\).
  3. Measure of a Set: \(P\Big[ \{ x \in A \} \Big] = \displaystyle\int_A f_X(x) dx\).

Probability Density Functions

Any function \(f(x)\) satisfying these properties defines a probability law using integration.

Let \(X\) be a continuous random variable. The probability density function of \(X\) is a funtion \(f_X : \Omega \to \mathbb{R}\) that, when integrated over an interval \([a,\, b]\), yields the probability of obtaining \(a \le X \le b\): \[ P[a \le X \le b] = \int_a^b f_X(x) dx. \]

The probability density function uniquely defines a random variable. If \(X\) and \(Y\) share a domain \(\Omega\), and \(f_X(t) = f_Y(t)\) for all \(t \in \Omega\), then \(X = Y\).

What is the probability of \(P(X = a)\)?

Example: Probability Density Function

Example: Let \(f_X(x) = 3x^2\) with \(\Omega = [0, 1]\).

  1. Explain why \(f_X\) is a PDF.
  2. If \(A = [0, 0.5]\), find \(P(X \in A)\).
  3. What does this mean graphically?
  4. What is \(P(0 \le X < 0.5)\)?

Cumulative Distribution Function

Let \(X\) be a continuous random variable with a sample space \(\Omega = \mathbb{R}\). The cumulative distribution function (CDF) of \(X\) is: \[ F_X(x) \overset{def}{=} P[X \le x] = \int_{-\infty}^x f_X(t) dt. \]

Let \(X\) be a continuous random variable. If the CDF \(F_X\) is continuous at any \(x \in [a, b]\), then: \[ P[a \le X \le b] = F_x(b) - F_X(a) \]

The survival function of \(X\) is: \[ S(x) = 1 - F_X(x) \overset{def}{=} P[X > x] = \int_{x}^\infty f_X(t) \, dt. \]

The Cumulative Distribution Function vs. Probability Density Function

The probability density function (PDF) is the derivative of the cumulative distribuion function (CDF): \[ f_X(x) = \frac{dF_X(x)}{dx} = \frac{d}{dx} \int_{-\infty}^x f_X(t)\, dt, \] provided \(F_X\) is differentiable at \(x\).

If \(F_x\) is not differentiable at \(x=x_0\), then \[ f_X(x_0) = P[X - x_0] \delta(x - x_0). \]

Example: Cumulative Density Function

Example: Let \(f_X(x) = 3x^2\) with \(\Omega = [0, 1]\).

  1. What is the CDF \(F_X(x)\) of \(X\)?
  2. Find \(P\left(X \le \dfrac{1}{4} \right)\) with the PDF \(f_X(x)\).
  3. Find \(P\left(X \le \dfrac{1}{4} \right)\) with the CDF \(F_X(x)\).

Properties of the Cumulative Distribution Function

Theorem: Let \(X\) be a random variable (either continuous or discrete), then the CDF of \(X\) has the following properties:

  1. The CDF is nondecreasing.
  2. The maximum of the CDF is when \(x = \infty\): \(F_X(\infty) = 1\).
  3. The minimum of the CDF is when \(x = -\infty\): \(F_x(-\infty) = 0\).

Continuity of the Cumulative Distribution Function

Theorem: For any random variable \(X\) (discrete or continuous), \(F_X(x)\) is always right-continuous. That is, \[ F_X(b) = F_X(b^+) \overset{def}{=} \lim_{h \to 0} F_X(b + h). \]

Theorem: For any random variable \(X\) (discrete or continuous), \(P(X = b)\) is: \[ P[X = b] = \begin{cases} F_X(b) - F_X(b^-), & \text{if $F_X$ is discontinuous ot $x = b$,} \\ 0, & \text{otherwise}. \end{cases} \]

The Expected Value of a Continuous Random Variable

We can generalize the definition of the expected value we developed with discrete random variables and sums to continuous random variables and integrals.

The expectation of a continuous random variable \(X\) is: \[ E[X] = \int_\Omega x f_X(x) \, dx. \]

Theorem: Let \(g : \Omega \to \mathbb{R}\) be a function and \(X\) be a continuous random variable. Then: \[ E\Big[ g(X) \Big] = \int_\Omega g(x) f_X(x) \, dx. \]

The \(n^th\) Moment of a Continuous Random Variable

The \(n^{th}\) moment of a continuous random variable \(X\) is \[ E[X^n] = \int_\Omega x^n f_X(x) \, dx. \]

The \(n^{th}\) central moment of a continuous random variable \(X\) is \[ E[(X-\mu)^n] = \int_\Omega (x-\mu)^n f_X(x) \, dx. \]

The Variance of a Continuous Random Variable

The same thing applies to the variance.

The variance of a continuous random variable \(X\) is: \[ V[X] = \int_\Omega (x-\mu)^2 f_X(x) \, dx. \]

The formulas concerning the variance and the expected value hold, such as \(V(X) = E(X^2) - [E(X)]^2\).

Example: Expectation and Variance

Example: Let \(f_X(x) = 3x^2\) with \(\Omega = [0, 1]\).

  1. What is the expected value of \(X\), \(E(X)\)?
  2. What is the expected value of \(X^2\), \(E(X^2)\)?
  3. What is the expected value of \(4X + 9\), \(E(4X + 9)\)?
  4. What is the variance of \(X\), \(V(X)\)?

The Uniform Distribution

Let \(X\) be a continuous uniform random variable.

The probability density function of \(X\) is: \[ f_X(x) = \begin{cases} \dfrac{1}{b-a}, & a \le x \le b, \\ 0, & \text{otherwise}, \end{cases} \] where \([a, b]\) is the interval on which \(X\) is defined.

The Uniform Distribution

We write \[ X \sim \text{Uniform}(a, b) \] to mean that \(X\) is drawn from a uniform distribution on an interval \([a, b]\).

The Uniform Distribution

The mean of a uniformly distributed random variable is: \[ E(X) = \frac{a + b}{2}. \]

Note that this is the midpoint of the interval on which \(X\) is defined.

The variance of a uniformly distributed random variable is: \[ V(X) = \frac{(b-a)^2}{12}. \]

Example: The Uniform Distribution

Example: Suppose a train has a uniform probability of arriving between 0 and 10 minutes late.

  1. What is the probability a train will arrive between 0 and three minutes late?
  2. What is the probability it will arrive exactly 3 minutes late?
  3. What is the probability that a train will arrive at least 8 minutes late?
  4. What is the probability that a train will arrive between 3 and 8 minutes late?
  5. What is the average amount of time a train will arrive past the scheduled time?
  6. What is the variance of amount of time a train will arrive past the scheduled time?

The Exponential Distribution

Let \(X\) be an exponential random variable.

The probability density function of \(X\) is: \[ f_X(x) = \begin{cases} \dfrac{1}{\beta} e^{-\frac{x}{\beta}}, & x \ge 0, \\ 0, & \text{otherwise}, \end{cases} \] where \(\beta > 0\) is a parameter.

The Exponential Distribution

We write \[ X \sim \text{Exponential}(\beta) \] to mean that \(X\) is drawn from an exponential distribution of parameter \(\beta\).

The mean of a exponentially distributed random variable is: \[ E(X) = \beta. \]

The variance of a exponentially distributed random variable is: \[ V(X) = \beta^2. \]

Example: The Exponential Distribution

Example: Suppose a train is always late but the distribution of (late) arrival times is given by an exponential distribution with \(\beta = 5\). What is the probability a train will arrive between 0 and three minutes late?

  1. What is the probability a train will arrive between 0 and three minutes late?
  2. What is the probability it will arrive exactly 3 minutes late?
  3. What is the probability that a train will arrive at least 8 minutes late?
  4. What is the probability that a train will arrive between 3 and 8 minutes late?
  5. What is the average amount of time a train will arrive past the scheduled time?
  6. What is the variance of amount of time a train will arrive past the scheduled time?

The Gamma Distribution

Let \(X\) be an gamma random variable.

The probability density function of \(X\) is: \[ f_X(x) = \begin{cases} \dfrac{1}{\Gamma(\alpha) \beta^\alpha} x^{\alpha - 1} e^{-\frac{x}{\beta}}, & x \ge 0, \\ 0, & \text{otherwise}, \end{cases} \] where \(\alpha > 0\) (shape) and \(\beta > 0\) (scale) are two parameters.

The Gamma function

\(\Gamma(z)\) is the gamma function, defined as: \[ \Gamma(z) = \int_0^\infty t^{z - 1} e^-t \, dt. \]

More usefully, this function has the property: \[ \Gamma(z) = z \Gamma(z - 1). \]

If \(z \in \mathbb{Z}^+\), then \(Gamma(z) = (z-1)!\) and \(|Gamma\left( \dfrac{1}{2} \right) = \sqrt{\pi}\).

The Gamma Distribution

We write \[ X \sim \text{Gamma}(\alpha, \beta) \] to mean that \(X\) is drawn from an Gamma distribution with parameters \(\alpha\) and \(\beta\).

If \(X \sim \text{Gamma}(\alpha = 1, \beta)\), then \(X \sim \text{Exponential}(\beta)\).

The Gamma Distribution

The mean of a Gamma distributed random variable is: \[ E(X) = \alpha \beta. \]

The variance of a Gamma distributed random variable is: \[ V(X) = \alpha \beta^2. \]

The Beta Distribution

Let \(X\) be an beta random variable.

The probability density function of \(X\) is: \[ f_X(x) = \begin{cases} \dfrac{\Gamma(\alpha + \beta)}{\Gamma(\alpha) \Gamma(\beta)} x^{\alpha - 1} (1 - x)^{\beta - 1}, & 0 \le x \le 1, \\ 0, & \text{otherwise}, \end{cases} \] where \(\alpha > 0\) and \(\beta > 0\) are two parameters.

The Beta Distribution

We write \[ X \sim \text{Beta}(\alpha, \beta) \] to mean that \(X\) is drawn from an Beta distribution with parameters \(\alpha\) and \(\beta\).

The Beta Distribution

The mean of a Beta distributed random variable is: \[ E(X) = \frac{\alpha}{\alpha + \beta}. \]

The variance of a Beta distributed random variable is: \[ V(X) = \frac{\alpha \beta}{(\alpha + \beta)^2 (\alpha + \beta + 1)}. \]