Systemic Review Finds Large Differences Between Hazard Ratios of Small Events at Interim Analyses of Trials

Large differences were found between unadjusted and bias-adjusted hazard ratios (HRs) when the number of events at the interim analysis was small or when unadjusted HRs were close to the boundaries, according to a systemic review of oncology clinical trials published in JAMA Network Open.

The researchers concluded that, since bias-adjusted estimators may play an important role in the data monitoring committee’s decision, the findings suggest presenting the bias-adjusted HRs and the unadjusted hazard ratio in the data monitoring committee meeting.

“In actual clinical trials, this systematic review found relatively large differences between the unadjusted and adjusted HRs when the number of events at the interim analysis was small or unadjusted HR was close to the boundaries,” wrote the researchers. “Therefore, we recommend presenting the 2 bias-adjusted estimators with the unadjusted HR in the DMC meeting to assess whether early termination for efficacy reasons is recommended at the interim analysis.”

In total, 19 eligible clinical trials were identified as applicable to the bias-adjusted estimators in a group of 198 abstracts screened for eligibility. The unadjusted HRs ranged from 0.203 (95% CI, 0.150-0.276) to 0.71 (95% CI, 0.60-0.84). More, the number of events at the interim analysis ranged from 58 to 540 and the information time from 48% to 82%.

For each study, the HRs adjusted by CMAE and WCMAE were found to be higher than the unadjusted HR. In large trials (243 and 414 events at the interim analysis), bias-adjusted estimates were similar to the unadjusted HR. Meanwhile, small trials (eg, with 58 events at the interim analysis) found that bias-adjusted estimates were highly disparate from the unadjusted HR.

Also, trials with large treatment effects saw a small difference between unadjusted and bias-adjusted HRs even though the number of events at interim analysis was small. Overall, larger differences were found when the unadjusted HR was more than 0.5.

“The importance of the number of events is easy to interpret; data about treatment effects from small trials are generally limited, and the amount of data at the interim analysis is smaller than that at the final analysis,” wrote the researchers. “Thus, a positive result at the interim analysis tends to exaggerate the treatment effect when trials are halted for efficacy reasons.”

The researchers identified potential trials published from 2013-2017 by searching MEDLINE and Embase on February 23, 2018. The team restricted the review to oncology clinical trials using “group sequential designs with a single preplanned interim analysis as well as 2-arm randomized clinical trials that were subsequently stopped for efficacy reasons.”

The major limitation to the study pertains to bias-adjusted estimators considered in the research, which could only be applied to superiority trials with a single interim analysis. Because of this, the search for appropriate literature was limited to trials with only 1 interim analysis planned. As a result, there is no information on the bias in HR estimation for long-term trials for this group of analyses.

“The bias-adjusted estimators may play an important role in the decision of the DMC,” wrote the researchers. “In many cases, the estimate from CMAE and WCMAE becomes more conservative than the unadjusted HR. Thus, presenting them in the DMC meeting might reduce the probability of early termination. Data monitoring committees are generally wary of interim results with few events, since 1 more or 1 less event in either group could change the conclusion of the interim analysis.”

References:

Shimura M, Nomura S, Wakabayashi M, et al. Assessment of Hazard Ratios in Oncology Clinical Trials Terminated Early for Superiority. JAMA Network Open. doi:10.1001/jamanetworkopen.2020.8633.