Development and testing of a text-mining approach to analyse patients’ comments on their experiences of colorectal cancer care

Wagland, Richard; Recio‐Saucedo, Alejandra; Simon, Michaël; Bracher, Michael; Hunt, Katherine; Foster, Claire; Downing, Amy; Glaser, Adam; Corner, Jessica

doi:10.1136/bmjqs-2015-004063

Cited by 49 publications

(62 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These techniques have clear potential for developing actionable insights in diverse specialties: they have also been used to classify patient-derived open-text comments in national cancer surveys [6]. A key advantage of these techniques is the possibility of deploying trained algorithms to operate on data as they are being collected, allowing real-time feedback and insight from open-text data [33], which may be used to monitor performance, and possibly safety, in the future.…”

Section: Discussionmentioning

confidence: 99%

“…The term-document matrix simply counts the instances of a word’s use within a comment and does not consider the order in which the words are presented. This approach has been used in similar studies published in the medical literature [6,8]. …”

Section: Discussionmentioning

confidence: 99%

“…This analysis will indicate the robustness of the algorithms and their suitability for application to novel data and is preferable to split-half validation or bootstrapping [6,25]. …”

Section: Methodsmentioning

confidence: 99%

“…Classification algorithms have been previously applied to patient comments about the experience of living beyond cancer [6], clinical incident reports [7,8], and sentiment analysis of digital footprints including Twitter and online blogs [9,10]. …”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Supervised Machine Learning Algorithms Can Classify Open-Text Feedback of Doctor Performance With Human-Level Accuracy

et al. 2017

View full text Add to dashboard Cite

BackgroundMachine learning techniques may be an effective and efficient way to classify open-text reports on doctor’s activity for the purposes of quality assurance, safety, and continuing professional development.ObjectiveThe objective of the study was to evaluate the accuracy of machine learning algorithms trained to classify open-text reports of doctor performance and to assess the potential for classifications to identify significant differences in doctors’ professional performance in the United Kingdom.MethodsWe used 1636 open-text comments (34,283 words) relating to the performance of 548 doctors collected from a survey of clinicians’ colleagues using the General Medical Council Colleague Questionnaire (GMC-CQ). We coded 77.75% (1272/1636) of the comments into 5 global themes (innovation, interpersonal skills, popularity, professionalism, and respect) using a qualitative framework. We trained 8 machine learning algorithms to classify comments and assessed their performance using several training samples. We evaluated doctor performance using the GMC-CQ and compared scores between doctors with different classifications using t tests.ResultsIndividual algorithm performance was high (range F score=.68 to .83). Interrater agreement between the algorithms and the human coder was highest for codes relating to “popular” (recall=.97), “innovator” (recall=.98), and “respected” (recall=.87) codes and was lower for the “interpersonal” (recall=.80) and “professional” (recall=.82) codes. A 10-fold cross-validation demonstrated similar performance in each analysis. When combined together into an ensemble of multiple algorithms, mean human-computer interrater agreement was .88. Comments that were classified as “respected,” “professional,” and “interpersonal” related to higher doctor scores on the GMC-CQ compared with comments that were not classified (P<.05). Scores did not vary between doctors who were rated as popular or innovative and those who were not rated at all (P>.05).ConclusionsMachine learning algorithms can classify open-text feedback of doctor performance into multiple themes derived by human raters with high performance. Colleague open-text comments that signal respect, professionalism, and being interpersonal may be key indicators of doctor’s performance.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

“…This analysis will indicate the robustness of the algorithms and their suitability for application to novel data and is preferable to split-half validation or bootstrapping [6,25]. …”

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Supervised Machine Learning Algorithms Can Classify Open-Text Feedback of Doctor Performance With Human-Level Accuracy

et al. 2017

View full text Add to dashboard Cite

show abstract

“…NVivo qualitative data analysis Software: version 11, 2016). Analysis will follow three phases in a way similar to a previous study conducted by members of the research team 67. First, random samples of comments will be read and coded to develop a thematic framework that comprehensively categorises issues and identifies ‘hot topics’.…”

Section: Methods and Analysismentioning

confidence: 99%

Life after prostate cancer diagnosis: protocol for a UK-wide patient-reported outcomes study

et al. 2016

Self Cite

View full text Add to dashboard Cite

BackgroundProstate cancer and its treatment may impact physically, psychologically and socially; affecting the health-related quality of life of men and their partners/spouses. The Life After Prostate Cancer Diagnosis (LAPCD) study is a UK-wide patient-reported outcomes study which will generate information to improve the health and well-being of men with prostate cancer.Methods and analysisPostal surveys will be sent to prostate cancer survivors (18–42 months postdiagnosis) in all 4 UK countries (n=∼70 000). Eligible men will be identified and/or verified through cancer registration systems. Men will be surveyed twice, 12 months apart, to explore changes in outcomes over time. Second, separate cohorts will be surveyed once and the design will include evaluation of the acceptability of online survey tools. A comprehensive patient-reported outcome measure has been developed using generic and specific instruments with proven psychometric properties and relevance in national and international studies. The outcome data will be linked with administrative health data (eg, treatment information from hospital data). To ensure detailed understanding of issues of importance, qualitative interviews will be undertaken with a sample of men who complete the survey across the UK (n=∼150) along with a small number of partners/spouses (n=∼30).Ethics and disseminationThe study has received the following approvals: Newcastle and North Tyneside 1 Research Ethics Committee (15/NE/0036), Health Research Authority Confidentiality Advisory Group (15/CAG/0110), NHS Scotland Public Benefit and Privacy Panel (0516-0364), Office of Research Ethics Northern Ireland (16/NI/0073) and NHS R&D approval from Wales, Scotland and Northern Ireland. Using traditional and innovative methods, the results will be made available to men and their partners/spouses, the funders, the NHS, social care, voluntary sector organisations and other researchers.

show abstract

Joining the dots: Can UW‐QoL free‐text data assist in understanding individual treatment experiences and QoL outcomes in head and neck cancer?

2017

View full text Add to dashboard Cite

KEYWORDShead and neck cancer, oncology, qualitative, quality of life, UW-QoL 2 | METHODS This study was a single site, prospective observational design. Participants were recruited from the maxillofacial clinic or multidisciplinary head and neck clinic of a large tertiary hospital in Brisbane, Australia, between February and July, 2013. Quality of life was measured using the UW-QoL, version 4 (1). Participants completed the UW-QoL at study enrolment (prior to treatment), 1-and 6-months post-treatment.The study eligibility criteria included a diagnosis of a cancer in the head and neck region, being aged 18 years or older and the ability to provide informed consent. Patients who were diagnosed with a recurrence of a previously treated malignancy or had started treatment at the time of recruitment were not eligible for inclusion. Demographic and clinical data were also collected.

show abstract

Development and testing of a text-mining approach to analyse patients’ comments on their experiences of colorectal cancer care

Cited by 49 publications

References 32 publications

Supervised Machine Learning Algorithms Can Classify Open-Text Feedback of Doctor Performance With Human-Level Accuracy

Supervised Machine Learning Algorithms Can Classify Open-Text Feedback of Doctor Performance With Human-Level Accuracy

Life after prostate cancer diagnosis: protocol for a UK-wide patient-reported outcomes study

Joining the dots: Can UW‐QoL free‐text data assist in understanding individual treatment experiences and QoL outcomes in head and neck cancer?

Contact Info

Product

Resources

About