Beyond Crosswalks: Reliability of Exposure Assessment Following Automated Coding of Free-Text Job Descriptions for Occupational Epidemiology

doi:10.1093/annhyg/meu006

Cited by 19 publications

(10 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Code rates were 96 to 97 percent and matched the manual code 89 percent of the time. Similarly, another study reported on manual versus automated coding of occupation data and assignment of occupational exposure in a large governmental database, finding that automated coding of occupations results in assignment of exposures in reasonable agreement with results from manual coding (Burstyn et al 2014).…”

Section: Introductionmentioning

confidence: 77%

Feasibility and Reliability of Automated Coding of Occupation in the Health and Retirement Study

Helppie-McFall

Sonnega

2018

SSRN Journal

View full text Add to dashboard Cite

Due to advances in computing power and the increase in coverage of longitudinal datasets in the Health and Retirement Study (HRS) that provide information about detailed occupations, demand has increased among researchers for improved occupation and industry data. The detailed data are currently hard to use because they were coded at different times, and the codeframes are, therefore, not consistent over time. Additionally, the HRS gathers new occupation and industry information from respondents every two years, and coding of new data at each wave is costly and time-consuming. In this project, we tested the NIOSH Industry and Occupation Computerized Coding System (NIOCCS) to see if it could improve processes for coding data from the HRS. We tested results from NIOCCS against results from a human coder for multiple datasets. NIOCCs does reasonably well compared to coding results from a highlytrained, professional occupation and industry coder, with kappa inter-rater reliability on detailed codes of just under 70 percent and agreement rates on broader codes of around 80 percent; however, code rates for NIOCCS for the datasets tested ranged from 60 percent to 72 percent, as compared to a professional coder's ability to code those same datasets that ranged from 95 percent to 100 percent. In its current form, we find that NIOCCS is a tool that might be best used to reduce the number of cases human coders must code, either in coding historical data to a consistent codeframe or in coding data from future HRS waves. However, it is not yet ready to fully replace human coders.

show abstract

Section: Introductionmentioning

confidence: 77%

Feasibility and Reliability of Automated Coding of Occupation in the Health and Retirement Study

Helppie-McFall

Sonnega

2018

SSRN Journal

View full text Add to dashboard Cite

show abstract

“…Even at the current level of accuracy, significant time and cost savings are possible over manual coding by insurers or researchers in Canada. As identified by Burstyn et al [ 9 ], it can take manual coders days to months to code a few 100 occupations to a classification. They reported that it can take 2 months to code approximately 1600 free text descriptors of lifetime occupational histories to the 2010 SOC.…”

Section: Discussionmentioning

confidence: 99%

“…In one approach, an initial match between the input data and the text description of a top-level class triggers further matchmaking at lower levels in the hierarchy. Another method is to directly compare the input data with each of the hierarchical levels [ 9 ]. In the same study, Burstyn et al [ 9 ] applied an algorithm that mixes both methods to code against the SOC (2010) system.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Occupation Coding of Job Titles: Iterative Development of an Automated Coding Algorithm for the Canadian National Occupation Classification (ACA-NOC)

Bao¹,

Baker²,

Adisesh³

2020

JMIR Form Res

View full text Add to dashboard Cite

Background In many research studies, the identification of social determinants is an important activity, in particular, information about occupations is frequently added to existing patient data. Such information is usually solicited during interviews with open-ended questions such as “What is your job?” and “What industry sector do you work in?” Before being able to use this information for further analysis, the responses need to be categorized using a coding system, such as the Canadian National Occupational Classification (NOC). Manual coding is the usual method, which is a time-consuming and error-prone activity, suitable for automation. Objective This study aims to facilitate automated coding by introducing a rigorous algorithm that will be able to identify the NOC (2016) codes using only job title and industry information as input. Using manually coded data sets, we sought to benchmark and iteratively improve the performance of the algorithm. Methods We developed the ACA-NOC algorithm based on the NOC (2016), which allowed users to match NOC codes with job and industry titles. We employed several different search strategies in the ACA-NOC algorithm to find the best match, including exact search, minor exact search, like search, near (same order) search, near (different order) search, any search, and weak match search. In addition, a filtering step based on the hierarchical structure of the NOC data was applied to the algorithm to select the best matching codes. Results The ACA-NOC was applied to over 500 manually coded job and industry titles. The accuracy rate at the four-digit NOC code level was 58.7% (332/566) and improved when broader job categories were considered (65.0% at the three-digit NOC code level, 72.3% at the two-digit NOC code level, and 81.6% at the one-digit NOC code level). Conclusions The ACA-NOC is a rigorous algorithm for automatically coding the Canadian NOC system and has been evaluated using real-world data. It allows researchers to code moderate-sized data sets with occupation in a timely and cost-efficient manner such that further analytics are possible. Initial assessments indicate that it has state-of-the-art performance and is readily extensible upon further benchmarking on larger data sets.

show abstract

“…Unstructured text fields of job title and industry were matched using a computer algorithm to 2010 Standard Occupational Classification (SOC) codes, a system used by Federal agencies to classify workers into occupational categories, at the highest level of detail available [38]. Precision of the match (percent agreement) was evaluated using a comparison to a subset of manually coded occupations and was found to be adequate, ranging in agreement from 0.72 (or “moderate”) for ”broad occupation” groups (the first 4–5 digits in SOC codes) to 0.85 (“very good”) for “minor groups” (the first 3 digits in SOC codes) [39]. Standard codes for job titles were linked to characteristics in the Occupational Information Network (O*NET), a database of occupational descriptions widely used to identify characteristics and demands of jobs [40].…”

Section: Methodsmentioning

confidence: 99%

Work Characteristics Associated with Physical Functioning in Women

Palumbo

Roos

Cannuscio

et al. 2017

IJERPH

View full text Add to dashboard Cite

Women make up almost half of the labor force with older women becoming a growing segment of the population. Work characteristics influence physical functioning and women are at particular risk for physical limitations. However, little research has explored the effects of work characteristics on women’s physical functioning. U.S. women between the ages of 50 and 79 were enrolled in the Women’s Health Initiative Observational Study between 1993 and 1998. Women provided job titles and years worked at their three longest-held jobs (n = 79,147). Jobs were linked to characteristics in the Occupational Information Network. Three categories of job characteristics related to substantive complexity, physical demand, and social collaboration emerged. The association between job characteristics and physical limitations in later life, measured using a SF-36 Physical Functioning score <25th percentile, was examined using modified Poisson regression. After controlling for confounding variables, high physical demand was positively associated with physical limitations (RR = 1.09 CI: 1.06–1.12) and substantively complex work was negatively associated (RR = 0.94, CI: 0.91–0.96). Jobs requiring complex problem solving, active learning, and critical thinking were associated with better physical functioning. Employers should explore opportunities to reduce strain from physically demanding jobs and incorporate substantively complex tasks into women’s work to improve long-term health.

show abstract

Beyond Crosswalks: Reliability of Exposure Assessment Following Automated Coding of Free-Text Job Descriptions for Occupational Epidemiology

Cited by 19 publications

References 17 publications

Feasibility and Reliability of Automated Coding of Occupation in the Health and Retirement Study

Feasibility and Reliability of Automated Coding of Occupation in the Health and Retirement Study

Occupation Coding of Job Titles: Iterative Development of an Automated Coding Algorithm for the Canadian National Occupation Classification (ACA-NOC)

Work Characteristics Associated with Physical Functioning in Women

Contact Info

Product

Resources

About