Data Analytics: Foundations to Practice · Data Cleaning and Preparation
Handling Missing Data
Missing data is rarely missing for a completely random reason, and the specific reason it's missing determines which handling approach is actually appropriate. This chapter covers the main strategies and their trade-offs.
Missing data can occur for many different reasons: a form field a respondent chose to skip, a system integration error that failed to record a value, or a genuinely inapplicable question for certain respondents, such as asking about a spouse's income on a form completed by an unmarried person; the specific underlying reason data is missing matters significantly for choosing an appropriate handling approach, since a value missing for a systematic, meaningful reason behaves very differently from one missing purely at random.
- The specific reason data is missing (random, systematic, or genuinely inapplicable) determines which handling approach is actually appropriate.
- Deletion is simplest but can reduce sample size and introduce bias if missing values aren't actually missing at random.
- Imputation preserves sample size by filling missing values with an estimate, but can understate a dataset's true variability if applied too broadly.
- Missingness itself can be meaningful information; flagging that a value was originally missing can preserve this signal rather than discarding it.