Journal of Korean Academy of Nursing (대한간호학회지)

Korean Society of Nursing Science (한국간호과학회)
권호 표지
DOI QR코드

DOI QR Code

External Validation of Carbapenem-Resistant Enterobacteriaceae Acquisition Risk Prediction Model in a Medium Sized Hospital

중규모 종합병원 대상 카바페넴 내성 장내세균속균종(Carbapenem-resistant Enterobacteriaceae) 획득위험 예측모형의 외적타당도 평가

  • Received : 2020.05.27
  • Accepted : 2020.07.30
  • Published : 2020.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Purpose: This study was aimed to evaluate the external validity of a carbapenem-resistant Enterobacteriaceae (CRE) acquisition risk prediction model (the CREP-model) in a medium-sized hospital. Methods: This retrospective cohort study included 613 patients (CRE group: 69, no-CRE group: 544) admitted to the intensive care units of a 453-beds secondary referral general hospital from March 1, 2017 to September 30, 2019 in South Korea. The performance of the CREP-model was analyzed with calibration, discrimination, and clinical usefulness. Results: The results showed that those higher in age had lower presence of multidrug resistant organisms (MDROs), cephalosporin use ≥ 15 days, Acute Physiology and Chronic Health Evaluation II (APACHE II) score ≥ 21 points, and lower CRE acquisition rates than those of CREP-model development subjects. The calibration-in-the-large was 0.12 (95% CI: - 0.16~0.39), while the calibration slope was 0.87 (95% CI: 0.63~1.12), and the concordance statistic was .71 (95% CI: .63~.78). At the predicted risk of .10, the sensitivity, specificity, and correct classification rates were 43.5%, 84.2%, and 79.6%, respectively. The net true positive according to the CREP-model were 3 per 100 subjects. After adjusting the predictors' cutting points, the concordance statistic increased to .84 (95% CI: .79~.89), and the sensitivity and net true positive was improved to 75.4%. and 6 per 100 subjects, respectively. Conclusion: The CREP-model's discrimination and clinical usefulness are low in a medium sized general hospital but are improved after adjusting for the predictors. Therefore, we suggest that institutions should only use the CREP-model after assessing the distribution of the predictors and adjusting their cutting points.

Keywords

References

  1. Korea Centers for Disease Control & Prevention (KCDC). Healthcare associated infection control guideline: VRSA/CRE. Cheongju: KCDC; 2018 Jun. p. 38-51. Report No.: 11-1352159-00832-10.
  2. Centers for Disease Control and Prevention (CDC). Facility guidance for control of carbapenem-resistant Enterobacteriaceae (CRE) [Internet]. Atlanta (GA): CDC; c2015 [cited 2020 Mar 7]. Available from: https://www.cdc.gov/hai/pdfs/cre/CRE-guidance-508.pdf.
  3. European Centre for Disease Prevention and Control (ECDC). Carbapenem-resistant Enterobacteriaceae, second update - 26 September 2019. Stockholm: ECDC; 2019. p. 1-10.
  4. Marston HD, Dixon DM, Knisely JM, Palmore TN, Fauci AS. Antimicrobial Resistance. Journal of the American Medical Association. 2016;316(11):1193-1204. https://doi.org/10.1001/jama.2016.11764
  5. Centers for Disease Control and Prevention (CDC). Antibiotic resistance threats in the United States, 2019. Atlanta (GA): CDC; 2019. p. 73.
  6. Korea Centers for Disease Control & Prevention (KCDC). Infectious diseases surveillance yearbook, 2019. Cheongju: KCDC; 2019 Aug. p. 38. Report No.: 11-1352159-000048-10.
  7. Korea Centers for Disease Control & Prevention (KCDC). Infectious disease portal [Internet]. Cheongju: KCDC; c2020 [cited 2020 Mar 7]. Available from: http://www.cdc.go.kr/npt/biz/npp/ist/simple/simplePdStatsMain.do.
  8. Tischendorf J, de Avila RA, Safdar N. Risk of infection following colonization with carbapenem-resistant Enterobactericeae: A systematic review. American Journal of Infection Control. 2016;44(5):539-543. https://doi.org/10.1016/j.ajic.2015.12.005
  9. Ho KW, Ng WT, Ip M, You JH. Active surveillance of carbapenem-resistant Enterobacteriaceae in intensive care units: Is it cost-effective in a nonendemic region? American Journal of Infection Control. 2016;44(4):394-399. https://doi.org/10.1016/j.ajic.2015.10.026
  10. Diekema DJ, Pfaller MA. Rapid detection of antibiotic-resistant organism carriage for infection prevention. Clinical Infectious Diseases. 2013;56(11):1614-1620. https://doi.org/10.1093/cid/cit038
  11. Kim YA, Lee K. Active surveillance of multidrug-resistant organisms with rapid detection methods for infection control. Annals of Clinical Microbiology. 2015;18(4):103-110. https://doi.org/10.5145/ACM.2015.18.4.103
  12. Goodman KE, Simner PJ, Klein EY, Kazmi AQ, Gadala A, Rock C, et al. How frequently are hospitalized patients colonized with carbapenem-resistant Enterobacteriaceae (CRE) already on contact precautions for other indications? Infection Control & Hospital Epidemiology. 2018;39(12):1491-1493. https://doi.org/10.1017/ice.2018.236
  13. Shimasaki T, Segreti J, Tomich A, Kim J, Hayden MK, Lin MY et al. Active screening and interfacility communication of carbapenem-resistant Enterobacteriaceae (CRE) in a tertiary-care hospital. Infection Control & Hospital Epidemiology. 2018;39(9):1058-1062. https://doi.org/10.1017/ice.2018.150
  14. Goodman KE, Simner PJ, Klein EY, Kazmi AQ, Gadala A, Toerper MF, et al. Predicting probability of perirectal colonization with carbapenem-resistant Enterobacteriaceae (CRE) and other carbapenem-resistant organisms (CROs) at hospital unit admission. Infection Control & Hospital Epidemiology. 2019;40(5):541-550. https://doi.org/10.1017/ice.2019.42
  15. Song JY, Jeong IS. Development of a risk prediction model of carbapenem-resistant Enterobacteriaceae colonization among patients in intensive care units. American Journal of Infection Control. 2018;46(11):1240-1244. https://doi.org/10.1016/j.ajic.2018.05.001
  16. Bleeker SE, Moll HA, Steyerberg EW, Donders AR, Derksen-Lubsen G, Grobbee DE, et al. External validation is necessary in prediction research: A clinical example. Journal of Clinical Epidemiology. 2003;56(9):826-832. https://doi.org/10.1016/s0895-4356(03)00207-5
  17. Debray TP, Vergouwe Y, Koffijberg H, Nieboer D, Steyerberg EW, Moons KG. A new framework to enhance the interpretation of external validation studies of clinical prediction models. Journal of Clinical Epidemiology. 2015;68(3):279-289. https://doi.org/10.1016/j.jclinepi.2014.06.018
  18. Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: Seven steps for development and an ABCD for validation. European Heart Journal. 2014;35(29):1925-1931. https://doi.org/10.1093/eurheartj/ehu207
  19. Clinical and Laboratory Standards Institute (CLSI). Performance standards for antimicrobial susceptibility testing. 27th ed. Wayne (PA): CLSI; 2017. p. 32-39.
  20. Hosmer DW, Jr, Lemeshow S. Applied logistic regression. 2nd ed. New York (NY): John Wiley & Sons; 2000. p. 160-164.
  21. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29-36. https://doi.org/10.1148/radiology.143.1.7063747
  22. Vickers AJ, Elkin EB. Decision curve analysis: A novel method for evaluating prediction models. Medical Decision Making. 2006;26(6):565-574. https://doi.org/10.1177/0272989X06295361
  23. Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. British Medical Journal. 2016;352:i6. https://doi.org/10.1136/bmj.i6
  24. Van Calster B, Vickers AJ. Calibration of risk prediction models: Impact on decision-analytic performance. Medical Decision Making. 2015;35(2):162-169. https://doi.org/10.1177/0272989X14547233
  25. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology. 2010;21(1):128-138. https://doi.org/10.1097/EDE.0b013e3181c30fb2
  26. Peek N, Arts DG, Bosman RJ, van der Voort PH, de Keizer NF. External validation of prognostic models for critically ill patients required substantial sample sizes. Journal of Clinical Epidemiology. 2007;60(5):491-501. https://doi.org/10.1016/j.jclinepi.2006.08.011
  27. Austin PC, Steyerberg EW. Interpreting the concordance statistic of a logistic regression model: Relation to the variance and odds ratio of a continuous explanatory variable. BMC Medical Research Methodology. 2012;12:82. https://doi.org/10.1186/1471-2288-12-82
  28. Parikh R, Mathai A, Parikh S, Chandra Sekhar G, Thomas R. Understanding and using sensitivity, specificity and predictive values. Indian Journal of Ophthalmology. 2008;56(1):45-50. https://doi.org/10.4103/0301-4738.37595
  29. Janssen KJ, Vergouwe Y, Kalkman CJ, Grobbee DE, Moons KG. A simple method to adjust clinical prediction models to local circumstances. Canadian Journal of Anaesthesia. 2009;56(3):194-201. https://doi.org/10.1007/s12630-009-9041-x
  30. Moons KG, Kengne AP, Grobbee DE, Royston P, Vergouwe Y, Altman DG, et al. Risk prediction models: II. External validation, model updating, and impact assessment. Heart. 2012;98(9):691-698. https://doi.org/10.1136/heartjnl-2011-301247