Real-World Studies Show Insufficiency For Informing Clinical, Reimbursement Decisions of Novel Cancer Drugs

The quality of real-world data (RWD) regarding novel anticancer therapies is often low and of insufficient rigor to influence decisions and may require quality assessment upon submission for publication, according to a study published in the European Journal of Cancer.

It was found that 78% of studies within this retrospective cohort were of low quality with none found to be of high quality. There were differences in median survival between real-world studies and corresponding trials which ranged from –32 months to 21 months (interquartile range [IQR], –4.2 months to 1.6 months). Additionally, low-quality studies had 23% higher outcomes compared with higher studies at 8% (P = .020).

“This retrospective cohort study provides the first systematic evaluation of RWD [real-world] studies reporting the effectiveness of cancer drugs for the treatment of solid organ malignancies approved by the EMA and FDA between 2010 and 2015. Overall, most EMA/FDA drugs now have RWD studies available. However, their methodologic quality is generally poor, with no high-quality studies identified and approximately 80% of the 293 studies evaluated scoring 0-3 (out of 9) using the Newcastle Ottawa Scale,” investigators of the study wrote.

This study included 293 RWD studies for 45 out of 57 indications in solid organ cancer as approved by the FDA and EMA from 2010 to 2015. The most common tumor types studied were prostate cancer (29%; n = 86), melanoma (15%; n = 43), colorectal cancer (12%; n = 34), lung cancer (11%; n = 33), and kidney cancer (11%; n = 32). Each of the studies either used small molecule inhibitors (37%, n = 108), cytotoxic agents (22%; n = 65), hormonal agents (17%; n = 50), or monoclonal antibodies (16%; n = 48) as their mechanism of action.

Of the trials, only 2% (n = 6) used data from the national cancer registries and 38% (n = 113) were from single-center evaluation, whereas 27% (n = 79) were funded by the pharmaceutical industry.

On the Newcastle Ottawa Scale that scores cohort studies across domains of study group selection, group comparability, and ascertainment of exposure and outcome, patients were scored out of 9 with 9 having the lowest risk of bias. A total of 78% of studies (n = 230) were considered to be low-quality, scoring between 0 to 3, 22% (n = 63) were moderate quality and scored between 4 to 6, and there were no high-quality studies that scored between 7 to 9.

For case series, 22% were found to adequately evaluate survival, and 33% had sufficient follow-up to estimate survival; additionally, 38% met the criteria for an adequate description of participant selection.

Investigators found that studies that had funding, demonstrated a trend toward better quality compared with those that were not funded, although this difference was not statistically significant (P = .082). In terms of studies that were funded by the pharmaceutical industry, 35% (n = 28) were of medium quality compared with 17% (n = 34) that did not have industry funding.

Studies that were conducted in the Netherlands had the highest number of scores between 4 to 6 at 63% (n = 5/8), followed by 57% (n = 8)/14 of multicenter international studies, 36% (n = 15/42) in Italy, and 33% (n = 5/15) in Spain. The tumor types that had the highest scores between 4 to 6 were sarcoma (46%; n = 6/13), gastric cancer (31%; n = 5/16), and breast cancer (30%; n = 7/23). When assessing quality based on drug interactions, 44% (n = 20) did not have real-world studies that scored between 4 to 6.

In total, 224 of 293 studies were used in the survival analysis. Of the 224 studies, 37% (n = 82) had superior survival outcomes and 63% (n = 141) had inferior survival outcomes. For all the real-world studies, the median survival difference was statistically inferior by -1.2 months (95% CI, -1.7 to -0.6; P < .001).

The median overall survival (OS) across all 224 real-world studies was 13 months. At the 10% threshold, 26% (n = 58) of real world studies had superior survival, 53% (n = 119) had inferior survival, and 22% (n = 49) had comparable survival to pivotal trials. At the 20% threshold, 15% (n = 35) had superior survival, 40% (n = 90) had inferior survival, and 45% (n = 101) had comparable survival.

Real-world studies that were of lower quality were more likely to report superior survival outcomes compared with the pivotal randomized control trials. At the 10% and 30% threshold, low-quality studies reported better survival compared with 16% of moderate studies (P = .129).

In addition, real-world studies for the same indication consistently presented superior or inferior median overall results compared with the pivotal studies. Investigators identified contradictory survival benefits compared with the pivotal trial, as well as a wide range of survival outcomes (real-world survival range, -7.0 to +14.8 months).

Lastly, investigators identified that, compared with pivotal trials, moderate quality studies reported less variation in median OS (IQR -3.9-0.2) compared with low-quality studies (IQR -4.3-2.4). Across all real-world studies, there was less variation in outcomes reported of moderate quality for those of the same drug indication. This was also observed when comparing funding status, with industry funded studies showing less variation in survival outcomes compared with those without funding.

Reference

Boyle JM, Hegarty G, Frampton C, et al. Real-world outcomes associated with new cancer medicines approved by the Food and Drug Administration and European Medicines Agency: a retrospective cohort study. Eur J Cancer. Published Online August 6, 2021. doi:10.1016/j.ejca.2021.07.001