Are we taking a risk with risk assessment tools? Evaluating the relationship between NSQIP and the ACS risk calculator in colorectal surgery.
Gerald O. Ogola Ph.D.
Keller, D. S., J. W. Ho, A. J. Mercadel, G. O. Ogola and S. R. Steele (2018). “Are we taking a risk with risk assessment tools? Evaluating the relationship between NSQIP and the ACS risk calculator in colorectal surgery.” Am J Surg Jul 19. [Epub ahead of print].E
GOAL: To evaluate the relationship between the ACS Risk Calculator and NSQIP expected outcomes in elective colorectal resections. METHODS: The 2015 NSQIP morbidity report for elective colorectal procedures at a single institution was evaluated. Risk Calculator (RC) reports were completed for predicted risk. Correlation coefficients were calculated for the general relationship between the tools for complications. Receiver operator characteristic (ROC) curves compared the predictive accuracy of the tools to actual outcome measures of any complication, serious complications, readmissions, unplanned return to the operating room, and mortality. RESULTS: There was high correlation between NSQIP and the RC for any complication, but low correlation for serious complications. Predictive accuracy of both tools for identifying actual occurrences was poor, with area under the ROC<0.60 for all metrics with both tools, except NSQIP mortality, which had good accuracy. CONCLUSIONS: The tools were highly correlated for predicting complications in general, but neither was accurate for predicting actual outcomes. As underestimating risk and complications could have financial implications under value-based care, refinement could benefit informed consent and quality improvement at the institutional level.