How to Assess Missing Data in a Research Paper

Missing data threaten a study when the unobserved outcomes could differ systematically from those observed. To assess the problem, reconcile every denominator, compare the amount, reasons, and timing of missingness across groups, identify the assumptions behind the primary analysis, and examine whether plausible sensitivity analyses change the conclusion.

The percentage missing is only a starting point. A small amount can matter when events are rare or missingness is strongly related to outcome. A larger amount may be less damaging when reasons are unrelated to outcome and conclusions remain stable across credible assumptions.

First identify what is missing

“Missing data” can describe different failures:

• an entire eligible study was not found or published;
• a prespecified outcome was not reported;
• a summary statistic or measure of uncertainty is absent;
• individual participants lack an outcome;
• covariates needed for adjustment are incomplete;
• follow-up times differ or observations are censored;
• study characteristics needed for subgroup analysis are unavailable.

This guide focuses on missing participant-level outcomes in a research paper. Missing studies and missing outcomes create related but distinct selective-reporting concerns.

Write down the exact result being assessed: outcome, time point, intervention effect, analysis population, and effect measure. Missingness can differ across outcomes and follow-up visits, so a whole-paper attrition percentage is rarely enough.

Reconstruct the participant flow

For each group, extract:

1. randomized or enrolled participants;
2. participants who began the assigned condition;
3. participants with the outcome measured;
4. participants included in the reported analysis;
5. withdrawals and exclusions with reasons;
6. timing of each loss;
7. participants included in safety analyses.

Reconcile the abstract, flow diagram, methods, result tables, figure footnotes, and supplement. Denominators can change silently from one outcome to another.

Distinguish missing outcomes from exclusions

A participant may have an observed outcome but still be excluded because of protocol deviations, eligibility reassessment, or analysis choices. Another may remain in the randomized group definition but have no measured outcome.

Both can bias an effect estimate, but they follow different pathways. The clinical trial appraisal guide helps compare allocation, participant flow, analysis populations, and outcome reporting without collapsing them into one “dropout” label.

Compare missingness by group, reason, and time

Do not stop at total percentages. Ask:

• Is missingness balanced between comparison groups?
• Do reasons differ by group?
• Did participants leave before or after treatment response or adverse effects could appear?
• Are missing outcomes more likely among participants with worse baseline prognosis?
• Could knowledge of treatment or outcome influence follow-up?
• Were reasons collected systematically or listed only as “lost to follow-up”?
• Does the paper combine withdrawal from treatment with withdrawal from outcome measurement?

Equal percentages do not guarantee equal bias. If participants in one group leave because symptoms worsen and participants in another relocate for unrelated reasons, the missingness mechanisms differ.

Understand MCAR, MAR, and MNAR assumptions

These statistical terms describe relationships between missingness and data:

Missing completely at random

Under MCAR, the chance that an observation is missing is unrelated to both observed information and the missing value. A random administrative loss could approximate this condition. Complete-case analysis can remain unbiased under strong MCAR conditions, but it discards information and reduces precision.

Missing at random

Under MAR, after conditioning on observed variables included in the analysis, missingness no longer depends on the unseen outcome. For example, follow-up might depend on observed baseline severity, and a model that uses baseline severity may make MAR more plausible.

MAR does not mean “missing for no reason,” and it cannot usually be proven from observed data alone.

Missing not at random

Under MNAR, missingness still depends on the unseen outcome after accounting for observed information. Participants whose unobserved symptoms worsened may be less likely to return even after measured predictors are included.

MNAR models require untestable assumptions about the missing values. That is why sensitivity analysis matters.

Check the target effect and intercurrent events

Before judging a method, ask which treatment effect the study intends to estimate. Discontinuation, rescue medication, switching treatment, and death may be intercurrent events rather than mere missingness.

For example:

• A treatment-policy strategy may seek the effect of assignment regardless of discontinuation.
• A hypothetical strategy may ask what would happen if discontinuation did not occur.
• A while-on-treatment strategy may target outcomes before discontinuation.

Different questions require different data and assumptions. Treating every post-discontinuation value as irrelevant can change the estimand and undermine the benefit of randomization.

Evaluate the primary analysis method

Identify exactly how the analysis handled incomplete outcomes.

Complete-case or available-case analysis

Only participants with observed required data are analyzed. This is simple, but its validity depends on strong assumptions about why data are missing and on the analysis model. Precision falls because information is discarded.

Do not accept “complete cases were representative” without evidence. Compare baseline characteristics and reasons for missingness, while recognizing that similarity on observed variables cannot rule out differences in unseen outcomes.

Single imputation

A single value replaces each missing value. Examples include mean imputation, baseline observation carried forward, or last observation carried forward.

Single imputation often treats guessed values as if they were known, understating uncertainty. Last observation carried forward additionally assumes that the outcome would remain unchanged after the last measurement, an assumption that may be implausible in progressive, recovering, or fluctuating conditions.

Multiple imputation

Multiple imputation creates several completed datasets, analyzes each, and combines results to reflect imputation uncertainty. Assess:

• variables included in the imputation model;
• consistency with the analysis model;
• handling of treatment group, outcomes, interactions, and nonlinear terms;
• number of imputations;
• distribution used for bounded, binary, skewed, or count outcomes;
• whether the approach assumes MAR;
• sensitivity to MNAR departures.

A sophisticated procedure can still be biased if the imputation model omits predictors of missingness or outcome.

Likelihood-based repeated-measures models

Mixed models and other likelihood-based approaches can use observed repeated outcomes without explicitly filling every blank. Their validity still depends on the model, covariance structure, time effects, and assumptions such as MAR conditional on included data.

“Uses all available data” does not mean “requires no missing-data assumptions.”

Inverse-probability weighting

Observed participants receive weights related to their estimated probability of remaining observed. This can address missingness related to measured variables when the weighting model is appropriate and probabilities are estimated well.

Extreme weights, omitted predictors, and limited overlap can make results unstable. Look for diagnostics and robust uncertainty estimation.

Do not let intention-to-treat language end the audit

An intention-to-treat effect concerns randomized assignment, but the phrase is used inconsistently in papers. Check whether:

• all randomized participants were represented in the analysis target;
• participants were analyzed in assigned groups;
• post-discontinuation outcomes were sought;
• missing values were imputed or modeled;
• exclusions occurred after randomization;
• the method matches the stated estimand.

“Modified intention-to-treat” requires a precise definition. It can range from a defensible eligibility condition to an outcome-dependent exclusion that risks bias.

Read sensitivity analyses as the main stress test

Because the unseen outcomes cannot be observed, a sensitivity analysis asks whether the conclusion survives plausible alternative assumptions.

Useful approaches can include:

• delta adjustments that shift imputed values in worse or better directions;
• pattern-mixture or selection models;
• tipping-point analyses showing how extreme missing outcomes must be to change the conclusion;
• plausible event-rate assumptions for missing participants;
• analyses including and excluding questionable post-randomization exclusions;
• alternative covariance, weighting, or imputation specifications.

The Cochrane Handbook missing-data guidance recommends making assumptions explicit and assessing robustness under reasonable changes. Extreme best-case/worst-case analyses may show theoretical bounds but can be less informative than plausible scenarios.

Ask whether the sensitivity range is plausible

A sensitivity analysis is not reassuring merely because it exists. Check who chose the parameters and whether they reflect clinical knowledge, observed reasons, external data, or convenient values.

If a small plausible shift reverses the conclusion, the study is fragile to missingness. If only implausibly extreme assumptions do so, confidence may increase—but other biases still matter.

A worked missing-data example

Imagine a two-group symptom trial with 200 participants per group:

• 8 outcomes missing in the intervention group;
• 18 missing in the comparator group;
• worsening symptoms cited more often in the comparator group;
• primary mixed-model estimate favors intervention;
• no post-discontinuation outcomes were collected;
• the only sensitivity analysis carries the last observation forward.

A careful interpretation is:

1. Missingness is differential in amount and reason.
2. Missingness may relate to unseen outcomes, making simple MAR assumptions questionable.
3. Lack of post-discontinuation follow-up may conflict with a treatment-policy estimand.
4. Last observation carried forward is not a sufficient MNAR stress test.
5. The reported benefit may be real, but its robustness to plausible worse comparator outcomes or intervention discontinuation patterns remains uncertain.

Do not replace this reasoning with “attrition was below 10%.”

Assess risk of bias, not a missingness score

The current RoB 2 tool assesses whether outcome data are available for all or nearly all participants, whether evidence suggests the result is not biased, and whether missingness could depend on the true outcome in a way likely to affect the result.

The judgment is result-specific. Missingness may be low for mortality and high for quality of life in the same trial. It may also matter differently for rare harms than for a common continuous outcome.

Use the broader risk-of-bias assessment guide to apply the correct tool and retain evidence for each signaling-question answer.

What a complete report should show

Look for:

• group-specific participant flow;
• numbers analyzed for every main outcome;
• reasons and timing of missingness;
• attempts to continue outcome collection after discontinuation;
• prespecified missing-data methods in protocol and analysis plan;
• variables and implementation details for imputation or weighting;
• assumptions behind the primary model;
• sensitivity analyses under plausible departures;
• differences between planned and reported methods;
• limitations carried into the conclusion.

The CONSORT 2025 explanation provides current trial-reporting guidance, including participant flow, analysis populations, missing-data methods, and treatment-effect estimates.

Using AI to audit missing data

AI can help retrieve:

• randomized and analyzed denominators;
• participant-flow passages;
• stated reasons for withdrawal;
• missing-data assumptions;
• imputation or model descriptions;
• sensitivity-analysis results;
• discrepancies across the abstract, methods, tables, and supplement.

SinaPilot AI Review can surface missing-data and analysis questions for verification. It cannot infer unobserved outcomes, prove MAR, or decide that a sensitivity range is clinically plausible without domain judgment and complete source material.

Ask narrow questions, require evidence locations, and treat “not reported” as a reporting observation rather than proof that a method was not used.

Common missing-data mistakes

• Applying one acceptable-percentage threshold to every outcome.
• Looking only at total attrition instead of group, reason, and timing.
• Treating balanced percentages as proof of no bias.
• Assuming baseline similarity among completers establishes MCAR or MAR.
• Believing intention-to-treat wording solves missing outcomes.
• Accepting last observation carried forward as conservative by default.
• Treating multiple imputation as assumption-free.
• Ignoring missing harms or post-discontinuation outcomes.
• Confusing treatment discontinuation with withdrawal from follow-up.
• Reading a complete-case p-value without checking who was excluded.
• Accepting an extreme sensitivity analysis that avoids plausible scenarios.
• Assigning one missing-data judgment to every result in the paper.

A 10-question missing-data checklist

1. Which exact outcome and time point are being assessed?
2. How many participants were assigned, observed, and analyzed in each group?
3. Why and when did outcomes become missing?
4. Could reasons depend on the unseen outcome?
5. Were outcomes sought after treatment discontinuation or switching?
6. What effect and estimand does the analysis target?
7. Which missing-data assumptions does the primary method require?
8. Are imputation, weighting, or repeated-measures models described well enough to audit?
9. Do plausible sensitivity analyses change magnitude, precision, or direction?
10. Does the conclusion preserve the remaining uncertainty?

Missing data are not merely empty cells. They are missing outcomes from real participants, and the reasons those outcomes disappeared can change the estimated effect. Make the assumptions visible before trusting the result.

Related critical-appraisal guides

• Reconcile allocation and participant flow with the clinical trial appraisal guide.
• Apply a formal study-design-specific risk-of-bias assessment.
• Preserve analysis populations in an accurate paper summary.
• Interpret the resulting effect size and confidence interval.
• Turn a concern into a proportionate comment with the peer-review checklist.