An App for Detecting Bullying of Nurses Using Convolutional Neural Networks and Web-Based Computerized Adaptive Testing: Development and Usability Study

Abstract

Background Workplace bullying has been measured in many studies to investigate its effects on mental health issues. However, none have used web-based computerized adaptive testing (CAT) with bully classifications and convolutional neural networks (CNN) for reporting the extent of individual bullying in the workplace. Objective This study aims to build a model using CNN to develop an app for automatic detection and classification of nurse bullying-levels, incorporated with online Rasch computerized adaptive testing, to help assess nurse bullying at an earlier stage. Methods We recruited 960 nurses working in a Taiwan Ch-Mei hospital group to fill out the 22-item Negative Acts Questionnaire-Revised (NAQ-R) in August 2012. The k-mean and the CNN were used as unsupervised and supervised learnings, respectively, for: (1) dividing nurses into three classes (n=918, 29, and 13 with suspicious mild, moderate, and severe extent of being bullied, respectively); and (2) building a bully prediction model to estimate 69 different parameters. Finally, data were separated into training and testing sets in a proportion of 70:30, where the former was used to predict the latter. We calculated the sensitivity, specificity, and receiver operating characteristic curve (area under the curve [AUC]), along with the accuracy across studies for comparison. An app predicting the respondent bullying-level was developed, involving the model’s 69 estimated parameters and the online Rasch CAT module as a website assessment. Results We observed that: (1) the 22-item model yields higher accuracy rates for three categories, with an accuracy of 94% for the total 960 cases, and accuracies of 99% (AUC 0.99; 95% CI 0.99-1.00) and 83% (AUC 0.94; 95% CI 0.82-0.99) for the lower and upper groups (cutoff points at 49 and 66 points) based on the 947 cases and 42 cases, respectively; and (2) the 700-case training set, with 95% accuracy, predicts the 260-case testing set reaching an accuracy of 97. Thus, a NAQ-R app for nurses that predicts bullying-level was successfully developed and demonstrated in this study. Conclusions The 22-item CNN model, combined with the Rasch online CAT, is recommended for improving the accuracy of the nurse NAQ-R assessment. An app developed for helping nurses self-assess workplace bullying at an early stage is required for application in the future.

An App Developed for Predicting Nurse Intention to Quit the Job (NIQJ) Using the Artificial Neural Networks(ANN) in Microsoft Excel: Population-Based Questionnaire Study

“…There are different types of algorithms for classi cation in Machine Learning [34,35], such as Logistic Regression, Support Vector Machines(SVM) [36], Naïve Bayes, Random Forest Classi cation, ANN, CNN [18,20,22], and k-nearest neighbors algorithm (KNN) [37]. ANN was shown superior with 93.2% classi cation accuracy in the previous study [36], similar to our results, although only two models(CNN and ANN) were compared.…”

An App for Classifying Personal Mental Illness at Workplace Using Fit Statistics and Convolutional Neural Networks: Survey-Based Quantitative Study

“…In this study, CNN was applied as supervised learning to build an ELMH prediction model to estimate 108 parameters (n=4×(10+17), with 4 sets of 10 parameters for featured maps and 17 parameters for pooled layers) because 4 categories are required in the CNN model ( Figure 1 and Multimedia Appendix 3 ). Detailed information about CNN [ 38 - 40 ] is available in the literature [ 19 , 20 ].…”

“…CNN in Microsoft (MS) Excel [ 19 , 20 ] was performed to estimate model parameters and compute the prediction accuracy rate (1−the number of misclassification/352). Comparisons of prediction accuracy were evaluated using discrimination analysis on (1) factor scores and outfit MNSQ and (2) factor scores alone in each subscale.…”

“…Although numerous studies have been conducted on MI apps [ 9 , 14 - 18 ], few have used the algorithm of artificial neural networks (ANNs) in classifying MI, particularly with convolutional neural networks (CNNs). Traditionally, cutoff scores are often used in the classification of MI [ 19 , 20 ].…”

“…Most of the classifications were based on the extent (or, say, strata) to which MI or other disorders are determined by summation scores or equivalent measures of a scale (eg, patients with a psychotic disorder [ 21 ] or those with 2 or 3 groups of nurse burnouts [or bullied victims] at the workplace [ 19 , 20 ]). To the best of our knowledge, none of the studies have applied the response pattern to classify the features of MI (or other disorders) with CNN modules to highlight less confident cases of MI characterized by the aberrant response pattern that deviated from normal cases.…”