Implementation of a Cohort Retrieval System for Clinical Data Repositories Using the Observational Medical Outcomes Partnership Common Data Model: Proof-of-Concept System Validation (Preprint)

Liu, Sijia; Wang, Yanshan; Wen, Andrew; Wang, Liwei; Hong, Na; Shen, Feichen; Bedrick, Steven; Hersh, William R.; Liu, Hongfang

doi:10.2196/preprints.17376

Cited by 4 publications

(5 citation statements)

References 33 publications

(37 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This not only would give us a more realistic picture of the performance of these topics, but also identify additional patients for relevance judgment for the word-based queries. After the additional judgments, we found that the structured queries had much higher recall than the word-based queries ( Figure 5) as well as much higher precision ( Figure 6), which was also been found in comparable experiments from Mayo Clinic [39]. As precision is sometimes conceptualized as "number needed to read" [40], the higher precision for the structured queries means fewer patients would need to be assessed to identify candidates for clinical studies.…”

Section: Discussionsupporting

confidence: 67%

“…was not certified by peer review) (which The copyright holder for this preprint this version posted November 12, 2019. . https://doi.org/10.1101/19005280 doi: medRxiv preprint topics that had been selected randomly for this previous research [39], while the second five topics were selected for diversity in all five of our sources for topic definitions (OHSU, Mayo, PheKP, REP, NQF). The second five were also selected based on a higher likelihood to be seen in clinical practice (based on clinician judgement), as compared to other topics in the list of 56.…”

Section: Additional Relevance Assessment For Ten Selected Topicsmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of Patient-Level Retrieval from Electronic Health Record Data for a Cohort Discovery Task

Bedrick

Cohen

Wang

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Objective Growing numbers of academic medical centers offer patient cohort discovery tools to their researchers, yet the performance of systems for this use case is not well-understood. The objective of this research was to assess patient-level information retrieval (IR) methods using electronic health records (EHR) for different types of cohort definition retrieval. Materials and Methods We developed a test collection consisting of about 100,000 patient records and 56 test topics that characterized patient cohort requests for various clinical studies. Automated IR tasks using word-based approaches were performed, varying four different parameters for a total of 48 permutations, with performance measured using B-Pref. We subsequently created structured Boolean queries for the 56 topics for performance comparisons. In addition, we performed a more detailed analysis of 10 topics. Results The best-performing word-based automated query parameter settings achieved a mean B-Pref of 0.167 across all 56 topics. The way a topic was structured (topic representation) had the largest impact on performance. Performance not only varied widely across topics, but there was also a large variance in sensitivity to parameter settings across the topics. Structured queries generally performed better than automated queries on measures of recall and precision, but were still not able to recall all relevant patients found by the automated queries. Conclusion While word-based automated methods of cohort retrieval offer an attractive solution to the labor-intensive nature of this task currently used at many medical centers, we generally found suboptimal performance in those approaches, with better performance obtained from structured Boolean queries. Insights gained in this preliminary analysis will help guide future work to develop new methods for patient-level cohort discovery with EHR data.

show abstract

Section: Discussionsupporting

confidence: 67%

Section: Additional Relevance Assessment For Ten Selected Topicsmentioning

confidence: 99%

Evaluation of Patient-Level Retrieval from Electronic Health Record Data for a Cohort Discovery Task

Bedrick

Cohen

Wang

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…After the additional judgments, we found that the structured queries had much higher recall than the word-based queries ( Figure 5 ) as well as much higher precision ( Figure 6 ), which was also been found in comparable experiments from Mayo Clinic. 39 As precision is sometimes conceptualized as “number needed to read,” 40 the higher precision for the structured queries means fewer patients would need to be assessed to identify candidates for clinical studies.…”

Section: Discussionmentioning

confidence: 99%

“…These included topics 2, 7, 9, 17, 32, 33, 42, 44, 48, and 52. To build on previous work done in our group, we used 5 topics that had been selected randomly for this previous research, 39 while the second 5 topics were selected for diversity in all 5 of our sources for topic definitions (OHSU, Mayo, PheKB, REP, and NQF). The second 5 were also selected based on a higher likelihood to be seen in clinical practice (based on clinician judgment), as compared to other topics in the list of 56.…”

Section: Methodsmentioning

confidence: 99%

Evaluation of patient-level retrieval from electronic health record data for a cohort discovery task

et al. 2020

Self Cite

View full text Add to dashboard Cite

Objective Growing numbers of academic medical centers offer patient cohort discovery tools to their researchers, yet the performance of systems for this use case is not well understood. The objective of this research was to assess patient-level information retrieval methods using electronic health records for different types of cohort definition retrieval. Materials and Methods We developed a test collection consisting of about 100 000 patient records and 56 test topics that characterized patient cohort requests for various clinical studies. Automated information retrieval tasks using word-based approaches were performed, varying 4 different parameters for a total of 48 permutations, with performance measured using B-Pref. We subsequently created structured Boolean queries for the 56 topics for performance comparisons. In addition, we performed a more detailed analysis of 10 topics. Results The best-performing word-based automated query parameter settings achieved a mean B-Pref of 0.167 across all 56 topics. The way a topic was structured (topic representation) had the largest impact on performance. Performance not only varied widely across topics, but there was also a large variance in sensitivity to parameter settings across the topics. Structured queries generally performed better than automated queries on measures of recall and precision but were still not able to recall all relevant patients found by the automated queries. Conclusion While word-based automated methods of cohort retrieval offer an attractive solution to the labor-intensive nature of this task currently used at many medical centers, we generally found suboptimal performance in those approaches, with better performance obtained from structured Boolean queries. Future work will focus on using the test collection to develop and evaluate new approaches to query structure, weighting algorithms, and application of semantic methods.

show abstract

“…In recent years, Clinical Data Warehouses (CDW) have been developed in hospitals. Various tools relying on CDW have been created in order to mine information from structured and unstructured data such as EMERSE [1], CREATE [2] or Dr. Warehouse [3].…”

Section: Introductionmentioning

confidence: 99%

Using Clinical Data Warehouse to Optimize the Vaccination Strategy Against COVID-19: A Use Case in France

Grosjean

Pressat-Laffouilhère

Ndangang

et al. 2022

MEDINFO 2021: One World, One Health – Global Partnership for Digital Innovation

View full text Add to dashboard Cite

Clinical Data Warehouses (CDW) are gold mines and may be useful to manage the COVID-19 outbreak. This article details the use of CDW in order to retrieve patients for vaccination purposes. A list of 34 diseases (or conditions) was published by French Health Authorities to target individuals at a high risk of developing a severe form of COVID. Using a multilevel search engine, 23 queries were built based on structured or unstructured data using natural language processing features. The Diagnosis Related Group coding system was used alone in three queries (13.0%), coupled with unstructured data in four queries (17.4%), and unstructured data were used alone in 16 queries (69.6%). Eleven diseases (conditions) were too broad to be translated into queries. Finally, 6,006 unique re-identified patients were retrieved. This use case demonstrates the usefulness of the Rouen University Hospital CDW in retrieving patients for other purposes than translational research.

show abstract

Implementation of a Cohort Retrieval System for Clinical Data Repositories Using the Observational Medical Outcomes Partnership Common Data Model: Proof-of-Concept System Validation (Preprint)

Cited by 4 publications

References 33 publications

Evaluation of Patient-Level Retrieval from Electronic Health Record Data for a Cohort Discovery Task

Evaluation of Patient-Level Retrieval from Electronic Health Record Data for a Cohort Discovery Task

Evaluation of patient-level retrieval from electronic health record data for a cohort discovery task

Using Clinical Data Warehouse to Optimize the Vaccination Strategy Against COVID-19: A Use Case in France

Contact Info

Product

Resources

About