types of missing data mechanisms

Types of Missing Data Mechanisms: A Comprehensive Guide

By Olivia Anderson
AD

Missing data, an omnipresent challenge in the realm of data analysis and statistics, occurs when data that should exist is absent or unavailable. This could be due to non-responses in surveys, data entry errors, or even natural phenomena like a lack of precipitation data from drought-stricken regions. The way this data is missing influences the methods used to handle or analyze it. Let's delve into the different types of missing data mechanisms.

Missing data can generally be categorized into two broad types: Missing Completely at Random (MCAR), and Missing Not at Random (MNAR). This categorization forms the basis of our exploration into the different types of missing data.

Missing Completely at Random (MCAR)

MCAR is the simplest and most innocuous form of missing data. In MCAR, the probability that a data point is missing is unrelated to any observed or unobserved variables. In other words, the data is missing by chance, similar to flipping a coin. This type of missing data is the easiest to handle and often allows for straightforward solutions like complete case analysis or imputation.

AD
Missing Data in Time-Series: Machine Learning Techniques | Towards Data Science
Missing Data in Time-Series: Machine Learning Techniques | Towards Data Science
a flow diagram with the words handling missing data in different colors and shapes, including arrows
a flow diagram with the words handling missing data in different colors and shapes, including arrows
Different types of missing data.
Different types of missing data.

However, it's crucial to note that MCAR is a rather idealistic assumption that's rarely met in reality. But when it holds, it opens the door to simple, efficient solutions for data imputation and analysis.

Missing Entirely at Random (MEAR)

MEAR, also known as Missing Completely at Random, is a special case of MCAR where each subject has a constant probability of being missing. This type of missingness is considered completely random and ignorable, meaning it won't affect the estimates of the parameters in your statistical model. Data can be handled simply by deleting or imputing the missing values, making MEAR the most manageable form of missing data.

Why It is So Important to Understand the Missing Data Mechanisms | Towards Data Science
Why It is So Important to Understand the Missing Data Mechanisms | Towards Data Science
5 Causes of Missing Data in Enterprise Systems
5 Causes of Missing Data in Enterprise Systems
three types of data structures with the text below
three types of data structures with the text below

In MEAR, the reason for the data being missing is unrelated to the data itself, making it non-informative about the underlying process. This is a critical aspect that differentiates MEAR from MNAR types.

Missing at Random (MAR)

MAR is a more complex form of missing data where the probability of an observation being missing depends on the observed data but not the unobserved data. In other words, the reason for datamissingness is related to the observed data, but not the missing data itself. This type of missingness is semiignorable, and the methods used to handle it can be a bit more complex than those used for MCAR.

AD

Some commonly used methods to handle MAR data include multiple imputation, multiple regression, and pattern mixture models. These methods aim to account for the relationship between the observed and missing data, which is the key characteristic of MAR.

Missing Data Demystified: The Absolute Primer for Data Scientists | Towards Data Science
Missing Data Demystified: The Absolute Primer for Data Scientists | Towards Data Science
Missing Data Overview: Types, Implications & Handling
Missing Data Overview: Types, Implications & Handling
a diagram showing different types of data storage
a diagram showing different types of data storage
the memory taken by different data types
the memory taken by different data types
the types of data structures are shown in this graphic below, there is an image of a
the types of data structures are shown in this graphic below, there is an image of a
the missing value and outliers in machine learning info sheet for day 8 of 7 / weeksmil
the missing value and outliers in machine learning info sheet for day 8 of 7 / weeksmil

Missing Not at Random (MNAR)

MNAR, also known as informative missingness, is the most complex and challenging type of missing data. In MNAR, the probability of an observation being missing is related to the unobserved data itself. This type of missingness is nonignorable, meaning the reason for the data being missing is relevant to the underlying data-generating process. Therefore, ignoring the missing data in MNAR can lead to biased results.

Handling MNAR data requires more complex methods, such as selection models, shared parameter models, and pattern mixture models. These methods aim to account for the relationship between the missingness and the unobserved data, which is the defining characteristic of MNAR.

AD

Missing in Completely at Random (MICAR)

MICAR is the most restrictive form of MNAR where the probability of an observation being missing depends entirely on the observed data, but also on the unobserved data. This means that the missing data mechanism is a function of the missing data itself, making it the most challenging type of missing data to handle. MICAR is informatively missing and nonignorable, requiring sophisticated methods to address.

๐—จ๐—ป๐—ฑ๐—ฒ๐—ฟ๐˜€๐˜๐—ฎ๐—ป๐—ฑ๐—ถ๐—ป๐—ด ๐——๐—ฎ๐˜๐—ฎ๐—ฏ๐—ฎ๐˜€๐—ฒ ๐—ง๐˜†๐—ฝ๐—ฒ๐˜€ Choosing storageโ€ฆ | Dr Milan Milanoviฤ‡ | 17 comments
๐—จ๐—ป๐—ฑ๐—ฒ๐—ฟ๐˜€๐˜๐—ฎ๐—ป๐—ฑ๐—ถ๐—ป๐—ด ๐——๐—ฎ๐˜๐—ฎ๐—ฏ๐—ฎ๐˜€๐—ฒ ๐—ง๐˜†๐—ฝ๐—ฒ๐˜€ Choosing storageโ€ฆ | Dr Milan Milanoviฤ‡ | 17 comments

Handling MICAR data often involves complex modeling strategies, such as Bayesian approaches or joint modeling of the missing data mechanism and the data-generating process. These methods aim to account for the intricate relationship between the observed and missing data in MICAR scenarios.

Understanding the different types of missing data mechanisms is a requisite for data analysts to choose appropriate strategies for handling missingness. While some types, like MCAR and MEAR, allow for simple solutions, others, like MNAR and MICAR, demand complex methods to ensure unbiased and accurate analyses. The key is to identify the mechanism at play and choose the most suitable method to address it.

AD
AD