Survival-inferred fragility of statistical significance in phase III oncology trials

In phase III oncology trials, superiority is defined by statistical significance using P thresholds. However, this approach has been criticized because P is continuous. Here, we reconstruct patient-level data for 230 phase III oncology trials to model the robustness of statistical significance by estimating the survival-inferred fragility index (SIFI), defined as the smallest number of patients changing arms that alters the statistical significance interpretation. The median SIFI was 8 patients (IQR 4–19), representing 1.4% of enrollments (IQR 0.7%–3%). As a continuous statistic, P—but not the significance interpretation—was correlated with SIFI. Moreover, overall survival endpoints were more fragile than surrogate endpoints. Taken together, while phase III oncology trials are intended to robustly inform patient care, shifting the assignment of a few patients is often sufficient to upend the statistical significance interpretation. This vulnerability underscores the need for more robust strategies to identify superiority in oncology.