Role of analytic approaches paramount when using AI-based analyses

A recent study with prominent involvement of BICL leaders Ali Guermazi and Frank Roemer points out the importance of applying advanced evaluations metrics when AI approaches are used for analyses of large OA datasets as in the OAI or clinical trials that are inherently imbalanced. BICL is working with leaders in the field of biostatistics and analytics to optimize outcome evaluation based on MRI data.

Commonly the area under the Receiver Operating Characteristic (ROC) curve is utilized to assess the performance of a deep learning-based binary classifier for classifying positive (presence of MRI-defined features) and negative classes. The area under the ROC curve (ROC-AUC) ranges from 0 to 1, with 1 indicating the perfect prediction. In general, a binary classifier with a ROC-AUC value of 0.8–0.9 is considered excellent. However, ROC-AUC alone is not sufficiently informative to evaluate the performance of approaches when the underlying data have class imbalance problems. Since most of the datasets from large-scale OA studies are imbalanced, it is imperative to examine the evaluation metrics and benchmark DL methods when balanced and/or imbalanced MRI datasets are utilized.