“…Vast sources of code in open source repositories and forums make deep learning feasible for SE tasks. Code Summarization (Movshovitz-Attias and Cohen, 2013;Allamanis et al, 2016;Iyer et al, 2016;Alon et al, 2019a;Hu et al, 2018;Harer et al, 2019;Ahmad et al, 2020), Bug Detection (Ray et al, 2016;Li et al, 2018b;Russell et al, 2018;, Program Repair (Chen et al, 2019;Lutellier et al, 2020), Code Translation (Chen et al, 2018;Drissi et al, 2018;Xu et al, 2020), Clone Detection (Zhang et al, 2019;Yu et al, 2019;, Code completion (Li et al, 2018a;Hellendoorn and Devanbu, 2017;Parvez et al, 2018) are some of the tasks that are addressed with deep neural solution. While most of the prior approaches use task-specific representation learning, a few works (Alon et al, 2019b;Feng et al, 2020;Lachaux et al, 2020;Clement et al, 2020) attempted to learn transferable representations in an unsupervised fashion.…”