Domain 2: Deviations from intended interventions
5. Were participants masked to the nature of the intervention?
In almost all psychotherapy trials, this will be No. Exceptions can be specifically studied for the trials comparing two active psychological interventions (e.g., both active interventions are presented to participants as equally effective, the trial is designed as a non-inferiority trial, etc.).
6. Were carers and people delivering the interventions masked?
In almost all psychotherapy trials, this will be No. Exceptions can be specifically studied for the trials comparing two active psychological interventions (e.g., both active interventions are presented to participants as equally effective, the trial is designed as a non-inferiority trial, etc.).
7. If “No/PN” or “NI” to #5 or #6, Did the interventions closely align with the intended plan, without any significant deviations that could affect the results?
When participants and carers are not blinded, there could be deviations from the intended interventions associated with this lack of blinding. An example of such a type of deviation is when participants who are told that they were on the waitlist seek the intervention of the experimental arm outside the trial or other interventions.
Another example could be that, due to the trial context, a substantially larger number of participants in the psychotherapy group make use of treatment as usual than the treatment as usual control group. For example, participants in the psychotherapy group have easier access to the prescription of antidepressant medication than participants in the control group. Deviations from the intended interventions that would arise even if the intervention took place outside the trial do not constitute a risk of bias with regard to the intention-to-treat analysis. An example could be drop-outs due to side effects of antidepressants: such drop-outs are deviations from the intended intervention but would occur even if the patients were taking antidepressants outside the trial context.
To pose a risk, these deviations should be influential for the outcome. For example, participants taking benzodiazepines in a depression psychotherapy trial might not be influential, but receiving the psychotherapy of interest or antidepressant medication is influential.
If there are deviations related to the outcome but they are balanced between the groups, bias will be less likely. However, if these influential deviations are unbalanced between the groups, it is more likely that the intervention effect estimate is biased. For example, when a substantially larger proportion of participants allocated to the psychotherapy group are taking antidepressants compared to the control group. Reviewers should make agreements on how an imbalance between groups is defined. In the previous example, an imbalance of 20% could be taken as a rough indication. In these cases, the trial can be judged as high risk, by giving a “No/PN” answer to this item.
8. Was an appropriate analysis used to estimate the effect of assignment to intervention?
This item evaluates whether trialists report to have adhered to the intention-to-treat principles, or to the so-called modified ITT (where the authors drop the randomized participants because their outcomes are missing).
Dropping the randomized participants because they did not receive the intervention they were allocated to, or counting the participants who received the intervention that they were not allocated to as those receiving this other intervention is inappropriate. In some cases, it might be sensible to assume that trialists adhered to ITT principles even if it is not explicitly stated (e.g., when authors used statistical analyses that are implicitly associated with the ITT principle).
9. If “No/PN” or “NI” to #8, is the inappropriate analysis unlikely to have an impact on the result?
If no appropriate analysis was conducted (or there is not enough information), reviewers should examine whether there was likely an impact of not using an appropriate analysis to estimate the effect of assignment to intervention. In cases in which excluded participants (e.g., due to not receiving a minimum number of sessions) or participants analyzed in the wrong group (e.g., participants randomized to the intervention group are analyzed in the waitlist group because in the end they did not receive the intervention) are less than 5%, this item should be rated as “Yes/PY” (leading to some concerns). Reviewers should answer “NI” if there is not enough information, and answer “No/PN” for thresholds above 5%. Missing outcome data (proportion of participants that did not answer the questionnaires at post-test) is not assessed here, but rather in domain 3 (item 10).
