Assessing the Performance of an AI-Powered Osteoporosis Screening System Using Chest Radiographs in Various settings

The evolution of technology and the widespread availability of medical data have led to the development of computer-aided systems for medical images, aiming to augment clinical diagnosis by offering supplementary insights to healthcare professionals. Deep learning techniques, in particular, are gaining prominence over traditional machine learning methods due to their ability to efficiently extract clinically relevant features automatically. Their effectiveness is further underscored by their remarkable performance in handling extensive, intricate, and unstructured datasets. In the realm of osteoporosis screening, plain radiographs emerge as a practical choice in hospital settings, given their cost-effectiveness, shorter examination times, and versatility in imaging various body regions, thereby facilitating opportunistic screening efforts.

This study introduces a model designed for effective and accessible osteoporosis screening using chest radiographs (CXRs), even under diverse conditions. The model proposes a framework for screening osteoporosis across various regions from CXRs, validated for robustness through performance assessments on external datasets from different demographic groups. Data from CXRs, along with dual-energy X-ray absorptiometry (DXA) results, were gathered from individuals aged 50 and above to train and validate the osteoporosis screening framework.

For training and internal validation, a dataset of 57,589 from Seoul Asan Medical Center (AMC) was utilized, with 55,600 for training and 1,989 for internal validation. External validation was conducted using datasets from Seoul Chuk Hospital (SCH), Veterans Health Service Medical Center (VHSMC), and Gradient Health platform (GH), each representing distinct healthcare settings and demographic distributions.

The framework processes CXR inputs by segmenting them into left shoulder, right shoulder, neck, spine, and analyzing the entire image. Five models are trained independently, focusing on specific regions of interest within predefined ranges. The framework combines these models' results to generate its final output, averaging scores from the top four models.

In internal validation at AMC, the framework achieved an AUC of 0.94, sensitivity of 0.87, and specificity of 0.86. For external validation sets SCH, VHSMC, and GH, AUC values were 0.92, 0.89, and 0.81, respectively. Sensitivity ranged from 0.71 to 0.85, and specificity from 0.77 to 0.94. Leveraging information from both entire CXR images and their subsets proved effective in acquiring relevant data from bones and tissues. The proposed osteoporosis screening framework demonstrated robustness across diverse external datasets, surpassing the AUC threshold of 0.8 in all cases. Notably, the framework performed well even on the challenging GH dataset, showcasing its reliability across different racial backgrounds.