Inclusive Learning through Real-time Tracking Display of Captions

Abstract

Gary has been teaching and directing the Center on Access Technology Innovation Laboratory at NTID for five years. He is a deaf engineer who retired from IBM after serving for 30 years. He is a development engineering and manufacturing content expert. He develops and teaches all related engineering courses. His responsibility as a director of Center on Access Technology Innovation Laboratory include the planning, implementation and dissemination of research projects that are related to the need of accessibility. He received his BS from RIT and his MS from Lehigh University. His last assignment with IBM was an Advanced Process Control project manager. He managed team members in delivering the next generation Advanced Process Control solution which replaced the legacy APC system in the 300 mm semiconductor fabricator. Behm has fifteen patents and has presented over 30 scientific and technical papers at various professional conferences worldwide. Inclusive Learning through Real-Time Display of Captions AbstractDeaf and Hard of Hearing (DHH) students cannot follow classroom lectures without accommodations such as real-time speech-to-text transcription. Current classroom transcription systems, such as C-Print improve access to classroom lectures, but still do not provide equivalent access to spoken information. These transcription systems require the DHH students to watch the transcription on a personal laptop screen, which is suitable for speeches, but not engineering lectures. Unlike speeches, most engineering lectures include use of detailed visuals such as slides or diagrams, and sequential procedures. DHH students constantly look away from their laptop display to search and study the visuals. As a result, they spend less time watching lecture visuals and gain less information than their hearing peers. However, the need to process simultaneous aural and visual information can also be taxing for hearing students, and previous studies have shown that they also benefit from real-time speech-to-text transcription.We evaluated the real-time display of captions (RTD) usability by both deaf and hearing students in an engineering class. It further examined the factors that influence hearing students' use of RTD as an alternative source of information to help with their learning process in the classroom, and the factors that influence deaf students' use of RTD.Our evaluation showed that DHH students prefer a continuously moving RTD with three lines, and that is as close as possible to the teacher. On the other hand, hearing students prefer a RTD that has 6 lines that at a fixed location. ChallengesHistorically, DHH students are an under-represented and under-served minority in higher education because they do not receive adequate information in class. As a result these students are often unprepared for traditional STEM classrooms. Most DHH students cannot understand spoken lectures without the aid of aural-to-visual access. Prior to the introduction of the earliest accessibility laws in 1974, less than 4% of DHH individu...

Enhancing participation of deaf engineering students in lab discussion

“…The basis of RTTD-MS comes from the Real-Time Text Display (RTTD) developed by Kushalnagar, et al (R. Kushalnagar et al, 2016). for classroom use.…”

“…Kushalnagar et al, (Behm, Kushalnagar, Stanislow, & Kelstone, 2015;R. Kushalnagar et al, 2016) found RTTD to be an effective captioning method in the classroom setting, improving students' ability to follow along with a lecture and to understand lecture content over traditional captioning.…”

RTTD-ID: Tracked Captions with Multiple Speakers for Deaf Students

“…RTTD-ID builds on Real-Time Text Display (RTTD) developed by Kushalnagar, et al [11]. for classroom use.…”

“…Kushalnagar et al, [11], [12] found RTTD to be an effective captioning method in the classroom setting, improving students' ability to follow along with a lecture and to understand lecture content over traditional captioning. However, the system does not indicate who is speaking.…”