Statistical Approaches To Data Monitoring For The Independent Data & Safety Monitoring Committee
Stopping rules
The data monitoring committee (IDSMC) monitors data on hazards and benefits to the study. The committee can make recommendations to the Trial Management Committee (TMC) to determine if data are sufficiently persuasive to warrant either closing recruitment to a trial or, in rare circumstances, changing the protocol. The TMC will then make decisions in terms of subgroup analyses or if major changes such as terminating one or more of the intervention arms is warranted. The ISDMC also provide oversight for broader issues such as ensuring ethical standards, quality assurance in trial conduct by monitoring recruitment and adherence to study interventions.
The scope of the activities of the ISDMC are defined in a terms of reference (toR) composed by the study investigators and statistician and the ToR are ratified and accepted by the Chair of the ISDMC prior to their first meeting. Member of the ISDMC can then review feedback with the study chair as required.
Statistical criteria (stopping rules) provide guidelines rather than rules which should be considered alongside other information which demand an element of judgement from IDSMCs:
- incorporating external evidence from other trials
- assessment of whether interim results would be persuasive enough to change clinical practice
Interim analyses
An interim analysis is an analysis comparing intervention groups at any time before the formal completion of the trial, usually before recruitment is complete (CONSORT Statement). Interim analyses are required to inform the deliberations of the DMC. These analyses do not always revolve around efficacy but can be on recruitment, study compliance and or safety. Formal interim analyses are sometimes not appropriate if the sample size is small and there is a risk of underpowering the primary outcome of the study.
Things to think about when planning interim analyses
- Frequency of meetings?
- Timing of meetings related to trial? e.g. proportion of sample size (eg 50% of patients) or by calendar
- Who decides on dates of meetings?
- Means of communication?
- What reports are required?
- Who can suggest unplanned analyses?
- What information is required?
- Who produces it?
- What is produced?
- Who sees the data and when?
- Who owns the data and analyses? Blinded or not?
- Cover benefits and risks in a balanced way
- Accessible style
- Avoid excessive detail
- Current as possible
Summary of statistical approaches
This is a brief summary of Appendix 1 of the HTA Report: Issues in data monitoring and interim analysis of trials (Grant 2005). For more information please consult that document.
Frequentist
Fixed type I error rate
Limited number of interim analyses at preset times
Pocock model –constant boundary for Type I error
Haybittle-Peto Rule
Trial should be stopped on efficacy grounds if there is both:
a) proof beyond reasonable doubt* that for all or some types of patients one particular treatment is clearly indicated
b) evidence that might reasonably be expected to influence patient management of many clinicians who are already aware of the results of other main studies *Proof beyond reasonable doubt – more than 3 results with p<0.001; these guidelines can also apply to O’Brien and Fleming and/or to subgroups of patients
O’Brien and Fleming model (USA) – much more conservative at the beginning of a trial
Likelihood approach
Informal continuous monitoring scheme
Other approaches
Other statistical approaches include Bayesian and decision-theoretic. For more information see Grant 2005.
Risk based monitoring
Central or at risk monitoring is also a way of closely monitoring your study data in real time and anticipating any developing problems. This does not need to include the ISDMC but involves working with your statistician and your data manager to review any patterns at site that may indicate deviations from protocol and increased data delinquency.
See related toolkit – Establishing data monitoring committee
This toolkit was prepared by Caroline Crowther and Rebecca Tooher. Updated by Lucille Sebastian and Adrienne Kirby, Hala Phipps
References:
Grant A, Altman D, Babiker A, Campbell M, Clemens F, Darbyshire J, Elbourne D, McLeer S, Parmar M, Pocock S, Spiegelhalter D, Sydes M, Walker A, Wallace S, and the DAMOCLES Study Group. Issues in data monitoring and interim analysis of trials. HTA 2005;9:7. Edwards S, Lilford R, Braunholtz D, Jackson J, Hewison J, Thornton J. Ethical issues in the design and conduct of randomised trials. HTA 1998;2(15).
Last revised: 9 December 2015.