Short answer scoring (SAS) is the task of grading short text written by a learner. In recent years, deep-learning-based approaches have substantially improved the performance of SAS models, but how to guarantee high-quality predictions still remains a critical issue when applying such models to the education field. Towards guaranteeing highquality predictions, we present the first study of exploring the use of human-in-the-loop framework for minimizing the grading cost while guaranteeing the grading quality by allowing a SAS model to share the grading task with a human grader. Specifically, by introducing a confidence estimation method for indicating the reliability of the model predictions, one can guarantee the scoring quality by utilizing only predictions with high reliability for the scoring results and casting predictions with low reliability to human graders. In our experiments, we investigate the feasibility of the proposed framework using multiple confidence estimation methods and multiple SAS datasets. We find that our human-in-theloop framework allows automatic scoring models and human graders to achieve the target scoring quality.

No abstract

This study sought to investigate the actual status of awareness regarding the appropriate use of antimicrobial drugs among patients of a wide age range who visit dispensing pharmacies in order to explore more effective intervention methods for improving awareness of the appropriate use of antimicrobial drugs for patients. A questionnaire survey was conducted with 1301 patients who visited different Tsuruha Holdings-operated pharmacies between 1 September 2018 and 31 October 2018. Using multiple regression analysis, we calculated scores based on the patients’ answers regarding their knowledge of antimicrobial drugs and antimicrobial resistance and examined factors related to these scores. Of the 1185 respondents who successfully completed the survey (mean age ± SD, 52.5 ± 18.2 years), 37.2% were 60 years old or older, 13.2% had never or were not sure whether they had taken antimicrobial drugs, and 73.2% did not understand the problem of antimicrobial resistance. Older age, the non-usage of drugs and self-reports of the “lack” of knowledge of antimicrobial resistance were identified as the group that needs education regarding the appropriate use of antimicrobial drugs. Intervention studies should be conducted to examine the efficacy of interventions based on these factors.