Cleaning StackOverflow for Machine Translation

Cited by 6 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition to the raw text data associated with messages, we expect to process the mailing lists to automatically identify text structures such as email signatures, code and quotes in replies [23]. Quotes, in particular, are potentially powerful as they provide a window into moderation norms and practices.…”

Section: Challenges Improvements and Future Workmentioning

confidence: 99%

The OCEAN mailing list data set: Network analysis spanning mailing lists and code repositories

Melanie¹,

F.²,

Young³

et al. 2022

Preprint

View full text Add to dashboard Cite

Figure 1: Layering social communication data onto the technical network structure of an open source project. The main network is produced by projecting the account-commit-file data of the CPython repository on a "graph of who collaborates with whom" which unveils the modular structure of the project and community (nodes are colored to highlight this using Louvain modularity maximization [4]). We then zoom on a particular module to highlight the role of certain nodes in the social layer of the community using our mailing list data and find that 6 of the top 10 nodes ranked by counts of toxic words are found in this single module.

show abstract

Section: Challenges Improvements and Future Workmentioning

confidence: 99%

The OCEAN mailing list data set: Network analysis spanning mailing lists and code repositories

Melanie¹,

F.²,

Young³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In [ 25 ], the author utilizes a machine learning approach to detect stack size, which is best for beam threshold runtime values for machine translation. In [ 26 ], the authors propose a sentiment analysis approach for MT.…”

Section: Background and Literature Surveymentioning

confidence: 99%

Machine Translation Utilizing the Frequent-Item Set Concept

Mahmoud

Mengash

2021

Sensors

View full text Add to dashboard Cite

In this paper, we introduce new concepts in the machine translation paradigm. We treat the corpus as a database of frequent word sets. A translation request triggers association rules joining phrases present in the source language, and phrases present in the target language. It has to be noted that a sequential scan of the corpus for such phrases will increase the response time in an unexpected manner. We introduce the pre-processing of the bilingual corpus through proposing a data structure called Corpus-Trie (CT) that renders a bilingual parallel corpus in a compact data structure representing frequent data items sets. We also present algorithms which utilize the CT to respond to translation requests and explore novel techniques in exhaustive experiments. Experiments were performed on specific language pairs, although the proposed method is not restricted to any specific language. Moreover, the proposed Corpus-Trie can be extended from bilingual corpora to accommodate multi-language corpora. Experiments indicated that the response time of a translation request is logarithmic to the count of unrepeated phrases in the original bilingual corpus (and thus, the Corpus-Trie size). In practical situations, 5–20% of the log of the number of the nodes have to be visited. The experimental results indicate that the BLEU score for the proposed CT system increases with the size of the number of phrases in the CT, for both English-Arabic and English-French translations. The proposed CT system was demonstrated to be better than both Omega-T and Apertium in quality of translation from a corpus size exceeding 1,600,000 phrases for English-Arabic translation, and 300,000 phrases for English-French translation.

show abstract

“…While we focus on bug fix identification as a single, running example, this problem is more general, spanning any classification of interest (e.g., tangled commits [3]), on any artifact of interest (e.g., bug reports [11], Stack Overflow [12]), for which simplistic heuristics are used to categorize the artifacts. We argue that using heuristics is not the problem, rather it is using not enough of them, for lack of a way to effectively gather, integrate, and combine them.…”

Section: Introductionmentioning

confidence: 99%

Mining Software Repositories with a Collaborative Heuristic Repository

Babii

Prenner

Stricker

et al. 2021

2021 IEEE/ACM 43rd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER)

View full text Add to dashboard Cite

Many software engineering studies or tasks rely on categorizing software engineering artifacts. In practice, this is done either by defining simple but often imprecise heuristics, or by manual labelling of the artifacts. Unfortunately, errors in these categorizations impact the tasks that rely on them. To improve the precision of these categorizations, we propose to gather heuristics in a collaborative heuristic repository, to which researchers can contribute a large amount of diverse heuristics for a variety of tasks on a variety of SE artifacts. These heuristics are then leveraged by state-of-the-art weak supervision techniques to train high-quality classifiers, thus improving the categorizations. We present an initial version of the heuristic repository, which we applied to the concrete task of commit classification.

show abstract

Cleaning StackOverflow for Machine Translation

Cited by 6 publications

References 31 publications

The OCEAN mailing list data set: Network analysis spanning mailing lists and code repositories

The OCEAN mailing list data set: Network analysis spanning mailing lists and code repositories

Machine Translation Utilizing the Frequent-Item Set Concept

Mining Software Repositories with a Collaborative Heuristic Repository

Contact Info

Product

Resources

About