MultiTurnCleanup: A Benchmark for Multi-Turn Spoken Conversational Transcript Cleanup

Abstract

Current disfluency detection models focus on individual utterances each from a single speaker. However, numerous discontinuity phenomena in spoken conversational transcripts occur across multiple turns, which can not be identified by disfluency detection models. This study addresses these phenomena by proposing an innovative Multi-Turn Cleanup task for spoken conversational transcripts and collecting a new dataset, MultiTurnCleanup 1 . We design a data labeling schema to collect the high-quality dataset and provide extensive data analysis. Furthermore, we leverage two modeling approaches for experimental evaluation as benchmarks for future research. * This work was done when the first author was a research intern at Google Research. 1 We release the collected MultiTurnCleanup dataset at: https://github.com/huashen218/MultiTurnCleanup.git B: Just in the last little while. Because I know my father, uh I mean, my father-in-law doesn't do that much, B: I think that's changed just in the last generation. B: Just in the last little while. Because I know my father, uh I mean, my father-in-law doesn't do that much, A: Exactly B: Just in the last little while. Because I know my father-in-law doesn't do that much, B: I think that's changed just in the last generation.Because I know my father-in-law doesn't do that much, (B) Multi-Turn Cleanup Task (A) Disfluency Detection Task A: Exactly B: Just in the last little while. Because I know my father, uh I mean, my father-in-law doesn't do that much, B: I think that's changed just in the last generation. B: Just in the last little while. Because I know my father, uh I mean, my father-in-law doesn't do that much, A: Exactly B: Just in the last little while. Because I know my father-in-law doesn't do that much, B: I think that's changed just in the last generation.Because I know my father-in-law doesn't do that much, (b) Multi-Turn Cleanup Task (a) Disfluency Detection Task A: Exactly