Neonatal pneumothorax AI reached an AUC of 0.975, but missed 16 of 129 positive test images

A single-center neonatal chest-radiograph model reached an AUC of 0.975 in an internal held-out set, but sensitivity fell to 87.6% at its fixed threshold and 52 of 165 positive calls were false. With no external validation, reader comparison, workflow test, or patient outcome, this is a strong development signal—not a deployment result.

THE HOT TAKE

External validation and prospective evaluation are therefore required before clinical implementation.

The model ranked neonatal pneumothorax images very well, but the bedside-relevant number is the miss rate at the chosen operating point. Researchers retrospectively assembled 6,159 chest radiographs from 3,665 neonates treated at one tertiary NICU in Japan between 2011 and 2024. The patient-level split reserved 1,231 images from 732 infants for the final test. Image-level AUC was 0.975 (95% CI 0.961–0.986), sensitivity 87.6% (80.6–92.7), and specificity 95.3% (93.9–96.5). The fixed threshold missed 16 of 129 pneumothorax radiographs: a promising internal result, not evidence that the system is ready to screen, rule out, or direct treatment.