Content-based Image Retrieval by Using Deep Learning for Interstitial Lung Disease Diagnosis with Chest CT

Abstract

Background: Evaluation of interstitial lung disease (ILD) at CT is a challenging task that requires experience and is subject tosubstantial interreader variability.

Purpose: To investigate whether a proposed content-based image retrieval (CBIR) of similar chest CT images by using deep learningcan aid in the diagnosis of ILD by readers with different levels of experience.

Materials and Methods: This retrospective study included patients with confirmed ILD after multidisciplinary discussion and availableCT images identified between January 2000 and December 2015. Database was composed of four disease classes: usual interstitialpneumonia (UIP), nonspecific interstitial pneumonia (NSIP), cryptogenic organizing pneumonia, and chronic hypersensitivity pneu-monitis. Eighty patients were selected as queries from the database. The proposed CBIR retrieved the top three similar CT imageswith diagnosis from the database by comparing the extent and distribution of different regional disease patterns quantified by a deeplearning algorithm. Eight readers with varying experience interpreted the query CT images and provided their most probable diagno-sis in two reading sessions 2 weeks apart, before and after applying CBIR. Diagnostic accuracy was analyzed by using McNemar testand generalized estimating equation, and interreader agreement was analyzed by using Fleiss k.

Results: A total of 288 patients were included (mean age, 58 years 6 11 [standard deviation]; 145 women). After applying CBIR,the overall diagnostic accuracy improved in all readers (before CBIR, 46.1% [95% CI: 37.1, 55.3]; after CBIR, 60.9% [95% CI:51.8, 69.3]; P , .001). In terms of disease category, the diagnostic accuracy improved after applying CBIR in UIP (before vs afterCBIR, 52.4% vs 72.8%, respectively; P , .001) and NSIP cases (before vs after CBIR, 42.9% vs 61.6%, respectively; P , .001).Interreader agreement improved after CBIR (before vs after CBIR Fleiss k, 0.32 vs 0.47, respectively; P = .005).

Conclusion: The proposed content-based image retrieval system for chest CT images with deep learning improved the diagnosticaccuracy of interstitial lung disease and interreader agreement in readers with different levels of experience.