Neuroscience studies are very often tasked with identifying measurable differences between two groups of subjects, typically one group with a pathological condition and one group representing control subjects. It is often expected that the measurements acquired for comparing groups are also affected by a variety of additional patient characteristics such as sex, age, and comorbidities.Multivariable regression (MVR) is a statistical analysis technique commonly employed in neuroscience studies to "control for" or "adjust for" secondary effects (such as sex, age, and comorbidities) in order to ensure that the main study findings are focused on actual differences between the groups of interest associated with the condition under investigation. It is common practice in the neuroscience literature to utilize MVR to control for secondary effects; however, at present, it is not typically possible to assess whether the MVR adjustments correct for more error than they introduce. In common neuroscience practice, MVR models are not validated and no attempt to characterize deficiencies in the MVR model is made. In this article, we demonstrate how standard hold-out validation techniques (commonly used in machine learning analyses) that involve repeatedly randomly dividing datasets into training and testing samples can be adapted to the assessment of stability and reliability of MVR models with a publicly available neurological magnetic resonance imaging (MRI) dataset of patients with schizophrenia. Results demonstrate that MVR can introduce measurement error up to 30.06% and, on average across all considered measurements, introduce 9.84% error on this dataset. When hold-out validated MVR does not agree with the results of the standard use of MVR, the use of MVR in the given application is unstable. Thus, this paper helps evaluate the extent to which the simplistic use of MVR introduces study error in neuroscientific analyses with an analysis of patients with schizophrenia.

Supervised machine learning classification is the most common example of artificial intelligence (AI) in industry and in academic research. These technologies predict whether a series of measurements belong to one of multiple groups of examples on which the machine was previously trained. Prior to real-world deployment, all implementations need to be carefully evaluated with hold-out validation, where the algorithm is tested on different samples than it was provided for training, in order to ensure the generalizability and reliability of AI models. However, established methods for performing hold-out validation do not assess the consistency of the mistakes that the AI model makes during hold-out validation. Here, we show that in addition to standard methods, an enhanced technique for performing hold-out validation—that also assesses the consistency of the sample-wise mistakes made by the learning algorithm—can assist in the evaluation and design of reliable and predictable AI models. The technique can be applied to the validation of any supervised learning classification application, and we demonstrate the use of the technique on a variety of example biomedical diagnostic applications, which help illustrate the importance of producing reliable AI models. The validation software created is made publicly available, assisting anyone developing AI models for any supervised classification application in the creation of more reliable and predictable technologies.

We have performed a morphological analysis of patients with schizophrenia and compared them with healthy controls. Our analysis includes the use of publicly available automated extraction tools to assess regional cortical thickness (inclusive of within region cortical thickness variability) from structural magnetic resonance imaging (MRI), to characterize group-wise abnormalities associated with schizophrenia based on a publicly available dataset. We have also performed a correlation analysis between the automatically extracted biomarkers and a variety of patient clinical variables available. Finally, we also present the results of a machine learning analysis. Results demonstrate regional cortical thickness abnormalities in schizophrenia. We observed a correlation (rho = 0.474) between patients’ depression and the average cortical thickness of the right medial orbitofrontal cortex. Our leading machine learning technology evaluated was the support vector machine with stepwise feature selection, yielding a sensitivity of 92% and a specificity of 74%, based on regional brain measurements, including from the insula, superior frontal, caudate, calcarine sulcus, gyrus rectus, and rostral middle frontal regions. These results imply that advanced analytic techniques combining MRI with automated biomarker extraction can be helpful in characterizing patients with schizophrenia.

Background: Although serum lactate levels are widely accepted markers of haemodynamic instability, an alternative method to evaluate haemodynamic stability/instability continuously and non-invasively may assist in improving the standard of patient care. We hypothesise that blood lactate in paediatric ICU patients can be predicted using machine learning applied to arterial waveforms and perioperative characteristics. Methods: Forty-eight post-operative children, median age 4 months (2.9–11.8 interquartile range), mean baseline heart rate of 131 beats per minute (range 33–197), mean lactate level at admission of 22.3 mg/dL (range 6.3–71.1), were included. Morphological arterial waveform characteristics were acquired and analysed. Predicting lactate levels was accomplished using regression-based supervised learning algorithms, evaluated with hold-out cross-validation, including, basing prediction on the currently acquired physiological measurements along with those acquired at admission, as well as adding the most recent lactate measurement and the time since that measurement as prediction parameters. Algorithms were assessed with mean absolute error, the average of the absolute differences between actual and predicted lactate concentrations. Low values represent superior model performance. Results: The best performing algorithm was the tuned random forest, which yielded a mean absolute error of 3.38 mg/dL when predicting blood lactate with updated ground truth from the most recent blood draw. Conclusions: The random forest is capable of predicting serum lactate levels by analysing perioperative variables, including the arterial pressure waveform. Thus, machine learning can predict patient blood lactate levels, a proxy for haemodynamic instability, non-invasively, continuously and with accuracy that may demonstrate clinical utility.

No abstract