A Performance Appraisal Analysis of Statistical Computation in Hospital Performance Management

Abstract

This multi-institutional study evaluates the effectiveness of six statistical methods for benchmarking hospital performance in quality improvement projects. A dataset of 48,521 patients undergoing craniotomy surgeries at 180 hospitals (2019–2023) was analyzed to assess mortality and serious complication rates. Non-contraction methods included direct standardisation with fixed effects and indirect standardisation without hospital effects, while contraction techniques comprised direct and indirect random-effects standardisation and Exponential Smoothing. The study highlights substantial reductions in adjusted mortality rates (1%-2% vs. observed 0%-10%) and adjusted serious complication rates (7%-17% vs. observed 3%-35%) when using contraction methods. These approaches aligned hospital performance closer to the average. Notably, 17%-39% of hospitals experienced significant changes in quintile rankings after applying random-effects direct standardisation. Logistic and fixed-effect methods identified outliers in smaller hospitals, whereas random-effects indirect standardisation did not. The choice of statistical method significantly impacts hospital rankings and the identification of performance outliers. These findings emphasize the importance of statistical selection in evaluating quality improvement efforts and guiding healthcare benchmarking.