Probability Distributions and Statistical Inference

Probability models describe variation before data are collected; statistical inference uses sample data to make carefully qualified statements about a larger population. This lesson builds on counting, probability, and displays of data from the Statistics — Data, Regression, and Probability lesson. The central question is not “what did this sample say?” but “how much could samples like this vary?”

Random Variables and Their Distributions

A random variable assigns a number to the outcome of a chance process. A discrete random variable has countable values, such as the number of defective items in ten; a continuous random variable can take any value in an interval, such as a measured mass. Its probability distribution lists possible values and their probabilities. For a discrete distribution, every probability is between 0 and 1 and the probabilities add to 1.

Expected value and variance

The expected value, or mean, is the long-run average over many repetitions. For a discrete random variable X with values xi and probabilities pi,

μ = E(X) = Σ xipi.

Variance measures squared distance from the mean, and standard deviation returns to the original units:

σ2 = Σ(xi − μ)2pi, and σ = √(σ2).

Worked example: a discrete distribution

A game pays $0 with probability 0.50, $4 with probability 0.30, and $10 with probability 0.20. Find its mean and standard deviation.

μ = 0(0.50) + 4(0.30) + 10(0.20) = 3.20

σ² = (0 − 3.2)²(0.50) + (4 − 3.2)²(0.30) + (10 − 3.2)²(0.20)
   = 5.12 + 0.192 + 9.248 = 14.56
σ = √14.56 ≈ 3.82

The long-run payout is $3.20 per play, with a typical spread of about $3.82.

Expected value need not be a possible one-play outcome. It is an average over repetitions, not a prediction that a single game will pay $3.20.

The Binomial Model

A binomial random variable counts successes in n trials. It applies only when there is a fixed number of trials, each trial has two outcomes, the success probability p stays constant, and trials are independent. Write X ~ Bin(n, p).

The probability of exactly k successes is

P(X = k) = (n⁄k)pk(1 − p)n − k,

where (n⁄k) = n!/[k!(n − k)!] counts which trials succeed. For this model, μ = np and σ = √(np(1 − p)).

Worked example: exactly and at least

A component passes inspection with probability 0.90. Assuming independent inspections, let X be the number passing among 5 components.

X ~ Bin(5, 0.90)

P(X = 4) = (5 choose 4)(0.90)⁴(0.10)
         = 5(0.6561)(0.10) = 0.32805

P(X ≥ 4) = P(X = 4) + P(X = 5)
         = 0.32805 + (0.90)⁵
         = 0.91854

μ = 5(0.90) = 4.5;  σ = √[5(0.90)(0.10)] ≈ 0.671

“At least” requires several values; use a complement when that is shorter, for example P(X ≥ 1) = 1 − P(X = 0).

Normal Models and Standardization

A normal distribution is continuous, bell-shaped, and symmetric. It is specified by mean μ and standard deviation σ. Areas under its curve are probabilities; a single exact value has probability 0. The standard normal variable Z has mean 0 and standard deviation 1. Convert a value x to a z-score:

z = (x − μ)/σ.

A z-score tells how many standard deviations a value lies above (positive) or below (negative) the mean. Tables or technology give normal areas after standardizing.

Worked example: a normal probability

Adult resting heart rates in a group are approximately normal with mean 72 beats per minute and standard deviation 8. Find the probability that one randomly selected rate is above 84.

z = (84 − 72)/8 = 1.50

P(X > 84) = P(Z > 1.50)
          = 1 − P(Z ≤ 1.50)
          = 1 − 0.9332 = 0.0668

About 6.7% of rates are above 84 beats per minute.

Do not use a normal model merely because a graph “looks smooth.” Check the context and distribution shape; strong skewness, separate groups, or impossible tails can make it inappropriate.

Samples, Statistics, and Sampling Distributions

A parameter describes a population, such as population mean μ or population proportion p. A statistic describes a sample, such as sample mean x̄ or sample proportion p̂. Repeated random samples produce a sampling distribution of the statistic.

For independent observations from a population with mean μ and standard deviation σ, the sample mean has mean μ and standard error SE(x̄) = σ/√n. Increasing n reduces sampling variability, but does not repair biased sampling. The central limit theorem says that for sufficiently large, independent samples, the distribution of sample means is approximately normal even when the population itself is not normal.

Worked example: why sample size matters

A population has μ = 50 and σ = 12. Compare sample means from samples of 9 and 36.

SE for n = 9:  12/√9  = 4
SE for n = 36: 12/√36 = 2

Both sampling distributions are centered at 50.
The n = 36 sample mean varies only half as much as the n = 9 sample mean.

Confidence Intervals

A confidence interval estimates a parameter with a plausible range. For a population proportion, when data come from a random or representative sample, observations are independent (often checked with the 10% condition when sampling without replacement), and there are enough expected successes and failures, an approximate 95% interval is

p̂ ± 1.96√(p̂(1 − p̂)/n).

Worked example: estimate a proportion

In a random sample of 400 voters, 228 support a proposal. Construct an approximate 95% confidence interval for the population support proportion.

p̂ = 228/400 = 0.570
SE = √[0.570(0.430)/400] ≈ 0.0248
margin of error = 1.96(0.0248) ≈ 0.0486

95% CI: 0.570 ± 0.0486 = (0.521, 0.619)

We are 95% confident that the population support proportion lies between 52.1% and 61.9%. In repeated sampling, about 95% of intervals made by this method would contain the fixed true proportion. It does not mean there is a 95% probability that this already-fixed parameter is in this particular interval.

Hypothesis Tests: Evidence, Not Proof

A hypothesis test begins with a null hypothesis H0, usually a status-quo parameter value, and an alternative Ha describing the direction or difference of interest. A p-value is the probability, assuming H0 is true, of getting a statistic at least as inconsistent with H0 as the one observed.

Worked example: test a claimed mean

A machine is claimed to fill bottles to 500 mL on average. A random sample of 36 fills has mean 496 mL. Historical process standard deviation is 12 mL. Test H0: μ = 500 against Ha: μ ≠ 500 at the 5% level.

SE = 12/√36 = 2
z = (496 − 500)/2 = −2.00

Two-sided p-value = 2P(Z ≤ −2.00) ≈ 2(0.0228) = 0.0456

Because 0.0456 < 0.05, reject H₀.
The sample provides evidence that the mean fill differs from 500 mL.

Rejecting H0 is not proving it false, and failing to reject it is not proving it true. A Type I error rejects a true null hypothesis; a Type II error fails to reject a false null hypothesis. The significance level controls the long-run Type I error rate when assumptions hold. Report the estimated effect and its practical importance, not only the p-value.

Limitations and Responsible Conclusions

  • Random assignment supports cause-and-effect conclusions; an observational study usually supports association only.
  • A large sample can make a tiny, unimportant difference statistically significant. Consider units, costs, and consequences.
  • Nonresponse, convenience samples, leading questions, measurement error, and excluded groups create bias that a margin of error does not capture.
  • Many tests increase false-positive opportunities. A result selected after trying many comparisons needs extra caution.
  • Model assumptions matter. Independence and a representative sampling process cannot be verified by a formula alone.

Common Mistakes

  • Calling any count binomial without checking independence and constant probability.
  • Confusing the standard deviation of individuals, σ, with the standard error of a mean, σ/√n.
  • Using P(X = x) for a continuous normal variable instead of an interval probability.
  • Interpreting a confidence level as the probability that the parameter moves around.
  • Stating “the null hypothesis is true” after a large p-value, or treating a small p-value as a measure of effect size.

Practice Set

  1. A random variable has values 1, 3, and 7 with probabilities 0.20, 0.50, and 0.30. Find E(X).
  2. Let X ~ Bin(8, 0.25). Find P(X = 2), to three decimals, and find the mean.
  3. Test scores are normal with mean 70 and standard deviation 10. Find the z-score for 85 and P(X < 85), to four decimals.
  4. A population has standard deviation 15. Find the standard error of the sample mean for n = 25 and for n = 100.
  5. A random sample of 250 has 135 successes. Find the approximate 95% confidence interval for the population proportion.
  6. A two-sided test gives p-value 0.12 at significance level 0.05. State the decision and a correct conclusion.
  7. Explain why a survey posted voluntarily on a social-media account may not justify a confidence interval for all residents of a city.
  8. A study finds a statistically significant association between carrying a lighter and lung cancer. Name a likely confounding variable and state why this does not establish causation.

Answer Checks

  1. E(X) = 1(0.20) + 3(0.50) + 7(0.30) = 3.8.
  2. P(X = 2) = (8⁄2)(0.25)2(0.75)6 ≈ 0.312. The mean is np = 8(0.25) = 2.
  3. z = (85 − 70)/10 = 1.5. Thus P(X < 85) = P(Z < 1.5) ≈ 0.9332.
  4. 15/√25 = 3 and 15/√100 = 1.5. Quadrupling the sample size halves the standard error.
  5. p̂ = 135/250 = 0.540; SE ≈ √(0.54(0.46)/250) = 0.0315. The interval is 0.540 ± 1.96(0.0315), or approximately (0.478, 0.602).
  6. Fail to reject H0, because 0.12 > 0.05. The data do not provide sufficient evidence against the null hypothesis at the 5% level; this does not prove it true.
  7. People who choose to respond may differ systematically from nonresponders. The sample is self-selected rather than random or representative, so ordinary sampling-error calculations do not address the bias.
  8. Smoking is a likely confounder: smokers may be more likely to carry lighters and to develop lung cancer. The lighter does not thereby cause cancer; the observational association may reflect smoking.