Local Investigators Significantly Overestimate Overall Response Rates Compared to Blinded Independent Central Reviews in Phase 2 Oncology Trials

Abstract

The overall response rate (ORR) is a largely adopted outcome measure in early-phase oncology trials. ORR is highly relevant in cancer drug development at the time of deciding whether to move to confirmatory phase 3 trials; moreover, ORR is gaining increasing relevance in fast-track registration procedures. No systematic analysis has been conducted so far to investigate whether a discrepancy exists between ORR assessed by local investigators and those assessed by blinded reviewers in phase 2 oncology trials. In this study, we carried out a search in the clinicaltrials.gov and EudraCT databases, looking at the trials reporting the results of both investigator-assessed and independently-assessed ORR. A discrepancy index was obtained by calculating the ratio of each investigator-assessed ORR on the corresponding independently assessed ORR, so that a discrepancy index >1 indicates that the investigator was "more optimistic," whereas a discrepancy index <1 indicates the opposite. We also analyzed different subgroups (by tumor type, by drug type, by year). Twenty trials met the search criteria; in some cases, >1 comparison was conducted in the trial, so that the total number of comparisons analyzed was 33. The estimated mean discrepancy index was 1.175 (95% confidence interval, 1.083-1.264; n = 33). In conclusion, local investigators significantly overestimate ORR compared to paired blinded reviewers in phase 2 oncology trials. This may represent a risk in drug development, when deciding whether to move to confirmatory, more expensive phase 3 trials. Blinded independent central review should be used in ORR assessment if a more conservative estimate of treatment efficacy is required, as in the case of fast-track drug developments leading to accelerated approvals of cancer therapies.

Local Investigators Significantly Overestimate Overall Response Rates Compared to Blinded Independent Central Reviews in Uncontrolled Oncology Trials: A Comprehensive Review of the Literature

“…Therefore, the RR reported by LIs was greater in 31 out of 36 evaluations, with an estimated mean difference of +4.57% (95% confidence interval, CI, 2.95–6.19%), thus suggesting that overall the LIs were more “optimistic” than the independent reviewers in the evaluation of tumor RR. These results are consistent with a more recent analysis carried out by our research group on data from 20 Phase 2 clinical trials having the ORR as primary or secondary endpoint, assessed by both LIs and through BICR ( Dello Russo et al, 2021 ). In some trials, more than one treatment group was included for a total of 33 ORR values.…”

“…When BICR is implemented, all the radiological images and selected clinical data acquired in the clinical trials are reviewed by independent radiologists who are blinded to treatment assignment and to any kind of clinical data that can influence the independent review process ( Ford et al, 2009 ). In this regard, we have recently observed that LIs tend to overestimate the ORR in comparison to BICR in Phase 2 clinical trials, whereas we did not find any significant evaluation bias between LIs and BICR when considering PFS in both Phase 2 and 3 trials ( Dello Russo et al, 2021 ; 2020 ). In the present article, we have tried to understand the reasons behind this discrepancy by reviewing the available evidence in the literature.…”

“…In an attempt to clarify the reasons behind the differences between LIs and BICR in relation to the evaluation of these two outcomes, we accessed the clinicaltrials.gov database and the clinicaltrialsregister.eu (EudraCT) database on 30 October 2021 and searched for clinical trials that included both the ORR and the PFS as endpoints and evaluations by both LIs and blinded assessors. We have already analyzed a subgroup of Phase 2 clinical trials that included both the ORR and the PFS, showing no differences between local and central reviewers with respect to PFS ( Dello Russo et al, 2021 ). We aimed to update and extend this initial analysis by including also the Phase 3 clinical trials.…”

“… Trials that were included in our initial analysis ( Dello Russo et al, 2021 ) are highlighted in gray. IQR, interquartile range; ITT, intention to treat population; ORR, objective response rate; PFS, progression-free survival; BICR, blind independent central review; LI, local investigator; rrMLBCL, relapsed or refractory primary mediastinal large B-cell lymphoma; rrRS, relapsed or refractory richter syndrome; DI, discrepancy index; NHL, non-Hodgkin’s lymphoma; CLL, chronic lymphocytic leukemia; mRECIST, modified response evaluation criteria in solid tumors; PFI/TFI, platinum-free interval/treatment-free interval; PrALKi, previous ALK inhibitors’ treatment; PrBRad, previous brain radiations; RECIST, response evaluation criteria in solid tumors.…”

“…The data included in our analysis refer to the primary analysis cutoff date for both LI and BICR ( Trněný et al, 2016 ). The same DIs are calculated by comparing the values obtained by LI at final analysis versus BICR primary analysis ( Dello Russo et al, 2021 ). b Not included in the present analysis since the specific arm/group was missing the DI for one of the outcomes (either ORR or PFS).…”