Evaluating Performance and Interpretability of Machine Learning Methods for Predicting Delirium in Gerontopsychiatric Patients

Delirium is an acute mental disturbance that particularly occurs during hospital stay. Current clinical assessment instruments include the Delirium Observation Screening Scale (DOSS) or the Confusion Assessment Method (CAM). The aim of this work is to analyze the performance of machine learning approaches to detect delirium based on DOSS and CAM information obtained from two geropsychiatric wards in Tyrol. From a machine learning perspective, the questions of these two assessment instruments represent the features and the ICD 10 diagnoses of delirium (yes/no) is the corresponding class variable. We compare seven popular classification methods and analyze the performance and interpretability of the learning models. As our dataset is highly imbalanced, we also evaluate the effect of common sampling methods including down- and up-sampling methods as well as hybrid methods. Our results indicate a high predictive ability of advanced methods such as Random Forest that can handle even unbalanced datasets. Overall, combining a good performance of a prediction model with the ability of users to understand the prediction is challenging. However, for clinical application in fully electronic settings, a good performance seems to be more important than an easy interpretation of the prediction by the user. On the other hand, explanations of decisions are often needed to assess other criteria such as safety.