Data Analytics: Foundations to Practice · Statistical Inference Basics
Hypothesis Testing Logic
Hypothesis testing has a specific, somewhat counterintuitive internal logic: it never actually proves the thing you're hoping to show, only how surprising your data would be if that thing weren't true. This chapter walks through that logic conceptually.
Hypothesis testing starts by stating a null hypothesis, typically representing 'no effect' or 'no difference' between groups being compared, and an alternative hypothesis, representing the effect or difference an analyst actually suspects or hopes to demonstrate exists; the entire testing process is structured specifically around gathering evidence against the null hypothesis, rather than directly proving the alternative hypothesis true in any absolute, positive sense.
- Hypothesis testing gathers evidence against a null hypothesis of 'no effect,' rather than directly proving an alternative hypothesis true.
- A p-value is the probability of seeing data this extreme if the null hypothesis were true; it is not the probability the null hypothesis itself is true or false.
- The conventional 0.05 significance threshold is a widely used convention, not a universal law; different fields sometimes reasonably use stricter or looser thresholds.
- A Type I error is a false positive (concluding an effect exists when it doesn't); a Type II error is a false negative; reducing one risk tends to increase the other.