Inter-reader reliability of CT Liver Imaging Reporting and Data System according to imaging analysis methodology: a systematic review and meta-analysis

Abstract

Objectives To establish inter-reader reliability of CT Liver Imaging Reporting and Data System (LI-RADS) and explore factors that affect it. Methods MEDLINE and EMBASE databases were searched from January 2014 to March 2020 to identify original articles reporting the inter-reader reliability of CT LI-RADS. The imaging analysis methodology of each study was identified, and pooled intraclass correlation coefficient (ICC) or kappa values (kappa) were calculated for lesion size, major features (arterial-phase hyperenhancement [APHE], nonperipheral washout [WO], and enhancing capsule [EC]), and LI-RADS categorization (LR) using random-effects models. Subgroup analyses of pooled kappa were performed for the number of readers, average reader experience, differences in reader experience, and LI-RADS version. Results In the 12 included studies, the pooled ICC or kappa of lesion size, APHE, WO, EC, and LR were 0.99 (0.96-1.00), 0.69 (0.58-0.81), 0.67 (0.53-0.82), 0.65 (0.54-0.76), and 0.70 (0.59-0.82), respectively. The experience and number of readers varied: studies using readers with >= 10 years of experience showed significantly higher kappa for LR (0.82 vs. 0.45, p = 0.01) than those with < 10 years of reader experience. Studies with multiple readers including inexperienced readers showed significantly lower kappa for APHE (0.55 vs. 0.76, p = 0.04) and LR (0.45 vs. 0.79, p = 0.02) than those with all experienced readers. Conclusions CT LI-RADS showed substantial inter-reader reliability for major features and LR. Inter-reader reliability differed significantly according to average reader experience and differences in reader experience. Reported results for inter-reader reliability of CT LI-RADS should be understood with consideration of the imaging analysis methodology.