Domain 3: Missing outcome data
10. Was data available for nearly all participants?
For meta-analyses focused on continuous outcomes: if the proportion of available data is above >95%, then domain 3 is directly rated as low risk and it is not needed to answer the next questions. Note that this refers to the proportion of participants who have endpoint data (i.e., who completed the questionnaires), and not the LOCF or imputed data.
For meta-analyses focused on dichotomous outcomes: For dichotomous outcomes, the proportion should be directly related to the risk of the event. If the observed number of events is much larger than the number of participants with missing data, the risk of bias will be less likely.
11. If “No/PN” or “NI” to #10, was an appropriate analysis used for handling the impact of missing data?
For example, the following approaches are considered appropriate for handling the impact of missing data:
- MMRM (mixed models for repeated measures, also known as mixed models, growth curve analyses) based on two or more measurements after baseline (e.g., mid-treatment, post-test, follow-ups, etc.).
- Multiple imputation, if the following criteria are met:
- “Rubin’s rules” or other methods to account for imputation uncertainty were applied.
- Important auxiliary variables were used in the imputation model, such as intermediate outcome assessments.
- Imputations were generated separately by RCT groups (“bygroup” imputation).
- Controlled/reference-based imputation (e.g., “jump-to-reference”) is used, using appropriate external information.
- Sensitivity analyses corresponding with a range of plausible reasons for missingness to confirm the primary analyses.
“Last observation carried forward” is not considered an appropriate approach for handling missing data.
If no appropriate analyses or unclear/no information, answer the next questions.
12. If “No/PN” or “NI” in #11, is the trial of moderate to large size (minimum 40 participants per arm reported data)?
13. If “No/PN” or “NI” in #11, did at least 70% of the randomized participants in each of the arms complete the assessments at the post-treatment?
14. If “No/PN” or “NI” in #11, is there indirect evidence indicating that missingness is unrelated to the outcome?
This item examines the following sources of indirect evidence that might signal risk of bias:
- whether the overall reported reasons for missing data indicate that missingness might depend on its true value (e.g., the general reason for missing data is termination of funding would not be indicative of risk of bias),
- large imbalances in the proportion of missing data between the groups: As a rule of thumb, the difference in study drop-out between the groups should be less than 20% (e.g., the intervention arm has a study drop-out of 10% and the wait-list control has a drop-out of 20%).
- differing reasons for missing data between the groups_ (e.g., more participants from the intervention group abandon the trial due to being more critically symptomatic could be an indicator of missigness being related to the outcome).
If one of the three is present, reviewers should answer with a “No/PN”, given that missing data might be related to the outcome. To answer with a “Yes/PY” there should be enough information to evaluate these possible sources. Reviewers may answer this question with “NI” if there is not enough information to assess the sources.
