A Curated Set of Labeled Code Tutorial Images for Deep Learning

Bergh, A.M.H.L. Serlier-van den; Harnack, Paul; Atchison, Abigail; Ott, Jordan; Eiroa-Lledo, Elia; Linstead, Erik

doi:10.1109/icmla51294.2020.00200

Cited by 2 publications

(1 citation statement)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our method is related to action detection in natural scene [20], [21], [22], [23], [24], which aims to detect the start and end time of action instances in untrimmed videos. Bergh et.al [25] creates a dataset comprised of 111,229 screenshots from Java and Python tutorials. Their goal is to classify four categories of code images: Visible Typeset Code, Partially Visible Typeset Code, Handwritten Code, and No Code.…”

Section: E Use Case Scenariosmentioning

confidence: 99%

ActionNet: Vision-Based Workflow Action Recognition From Programming Screencasts

Zhao

Xing

Chen

et al. 2019

2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE)

View full text Add to dashboard Cite

Programming screencasts (e.g., video tutorials on Youtube or live coding stream on Twitch) are important knowledge source for developers to learn programming knowledge, especially the workflow of completing a programming task. Nonetheless, the image nature of programming screencasts limits the accessibility of screencast content and the workflow embedded in it, resulting in a gap to access and interact with the content and workflow in programming screencasts. Existing non-intrusive methods are limited to extract either primitive human-computer interaction (HCI) actions or coarse-grained video fragments. In this work, we leverage Computer Vision (CV) techniques to build a programming screencast analysis tool which can automatically extract code-line editing steps (enter text, delete text, edit text and select text) from screencasts. Given a programming screencast, our approach outputs a sequence of coding steps and code snippets involved in each step, which we refer to as programming workflow. The proposed method is evaluated on 41 hours of tutorial videos and live coding screencasts with diverse programming environments. The results demonstrate our tool can extract code-line editing steps accurately and the extracted workflow steps can be intuitively understood by developers.

show abstract