Artificial Intelligence Classifier for Osteoporosis Detection from Chest Radiographs: Performance Evaluation and Literature Comparison with Quantitative Ultrasound

AFFILIATIONS: 

aDepartment of Convergence Medicine, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea

bDepartment of Health Screening and Promotion Center, Seoul Chuk hospital, Seoul, Republic of Korea 

cPromedius.Inc.

dDepartment of Health Screening and Promotion Center, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea 

eDivision of Endocrinology and Metabolism, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea

fDivision of Cardiology, Department of Internal Medicine, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, South Korea

gDivision of Cardiology, Department of Internal Medicine, Veterants health service medical center, Seoul, South Korea

hDepartment of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea

Background

Osteoporosis is a common condition that often goes unnoticed until it results in fractures, significantly increasing mortality rates. Recent advancements in deep learning have demonstrated the ability to detect low bone density using simple radiographs, including chest X-rays (CXR).`osteoporosis screening in real-world settings.

The initial model, OsPor-Screen(Jang et al. 2022), was validated exclusively using CXRs from the Health screening dataset, which mainly included images from relatively healthy individuals. This limitation restricted its applicability in actual clinical settings. To overcome this, we developed and implemented a robust model specifically for osteoporosis screening in diverse, real-world environments. The new model was validated across various healthcare facilities, representing different stages of the healthcare delivery system and distinct medical settings. Furthermore, we aimed to compare the performance of our model with that of osteoporosis screening using quantitative ultrasound (QUS).

Methods

The CXR:OSTEO was validated with five datasets. Asan Medical Center (AMC), which is a tertiary hospital, internal test dataset was a part of the training dataset and AMC external dataset was an external dataset from the same hospital. Seoul Chuk Hospital (SCH) specializes in spine and joint care. Veteran Health System Medical Center (VHSMC) provides comprehensive medical and specialized care to veterans and their families. Gradient Health (GH), a leading provider of global medical imaging datasets, offers access to an extensive and diverse collection of 5 million de-identified medical images.

The Advanced model, CXR:OSTEO goes further than the OsPor-Screen, which predicts osteoporosis in a single CXR image, to predict osteoporosis in multiple regions of a CXR image. To comprehensively evaluate the screening performance for the two validation datasets, the accuracy, sensitivity, specificity, and area under the ROC curve (AUC) were calculated. 

Results

Across more than seven different CXR machines and two different DXA manufacturers in various populations, CXR:OSTEO showed excellent performances in the internal and four external datasets. The AUC was near 0.95 in the internal dataset and was over 0.81  in the global region. For osteoporosis screening guidelines in USPSTF, performance of QUS is 0.77 in women and 0.80 in men.(“Osteoporosis to Prevent Fractures: Screening” 2018) Another study(Adami et al. 2024) also reported that the maximum performance of QUS in a dataset of 200 patients was an AUC of 0.81.

Conclusion

We developed the CXR-OSTEO model to screen for osteoporosis using chest X-rays and validated it with four external test datasets. These results support that screening for osteoporosis using CXR:OTSEO is the most effective method in the real clinical setting.

Figure 1. Scheme of OSTEO:CXR

Table 1. Demographic characteristics of the datasets

Table 2. The performance metrics of CXR:OSTEO in the internal and external validation datasets

Reference

Adami, Giovanni, Maurizio Rossini, Davide Gatti, Paolo Serpi, Christian Fabrizio, and Roberto Lovato. 2024. “New Point-of-Care Calcaneal Ultrasound Densitometer (Osteosys BeeTLE) Compared to Standard Dual-Energy X-Ray Absorptiometry (DXA).” Scientific Reports 14 (1): 6898.

Jang, Miso, Mingyu Kim, Sung Jin Bae, Seung Hun Lee, Jung-Min Koh, and Namkug Kim. 2022. “Opportunistic Osteoporosis Screening Using Chest Radiographs With Deep Learning: Development and External Validation With a Cohort Dataset.” Journal of Bone and Mineral Research: The Official Journal of the American Society for Bone and Mineral Research 37 (2): 369–77.

“Osteoporosis to Prevent Fractures: Screening.” 2018. US Preventive Services Taskforce. June 26, 2018. https://www.uspreventiveservicestaskforce.org/uspstf/recommendation/osteoporosis-screening.