Learning to Organize a Bag of Words into Sentences with Neural Networks: An Empirical Study

Tao, Chongyang; Gao, Shen; Li, Juntao; Feng, Yansong; Zhao, Dongyan; Yan, Rui

doi:10.18653/v1/2021.naacl-main.134

Cited by 7 publications

(2 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this way, the CNN model encapsulates the feature extractor (HoG/SIFT) [14], the encoder (bag of visual words) [15], and the classifier (support vector machines [16]) all in itself. One of the first practical and widely adopted DL models was AlexNet (Pernik, Bulgaria), which played a pivotal role in showcasing the capabilities of CNNs.…”

Section: Introductionmentioning

confidence: 99%

Reimagining Carbon Nanomaterial Analysis: Empowering Transfer Learning and Machine Vision in Scanning Electron Microscopy for High-Fidelity Identification

Gupta

2023

Materials

View full text Add to dashboard Cite

In this report, we propose a novel technique for identifying and analyzing diverse nanoscale carbon allotropes using scanning electron micrographs. By precisely controlling the quenching rates of undercooled molten carbon through laser irradiation, we achieved the formation of microdiamonds, nanodiamonds, and Q-carbon films. However, standard laser irradiation without proper undercooling control leads to the formation of sparsely located diverse carbon polymorphs, hindering their discovery and classification through manual analyses. To address this challenge, we applied transfer-learning approaches using convolutional neural networks and computer vision techniques to achieve allotrope discovery even with sparse spatial presence. Our method achieved high accuracy rates of 92% for Q-carbon identification and 94% for distinguishing it from nanodiamonds. By leveraging scanning electron micrographs and precise undercooling control, our technique enables the efficient identification and characterization of nanoscale carbon structures. This research significantly contributes to the advancement of the field, providing automated tools for Q-materials and carbon polymorph identification. It opens up new opportunities for the further exploration of these materials in various applications.

show abstract

Section: Introductionmentioning

confidence: 99%

Reimagining Carbon Nanomaterial Analysis: Empowering Transfer Learning and Machine Vision in Scanning Electron Microscopy for High-Fidelity Identification

Gupta

2023

Materials

View full text Add to dashboard Cite

show abstract

“…BERTbased LMs (Devlin et al, 2019) have demonstrated their abilities to encode various linguistic and hierarchical properties (Lin et al, 2019;Jawahar et al, 2019;Jo and Myaeng, 2020) which have a positive effect on the downstream performance (Liu et al, 2019a;Miaschi et al, 2020) and serve as an inspiration for syntax-oriented architecture improvements Bai et al, 2021;Ahmad et al, 2021;Sachan et al, 2021). Besides, a variety of pre-training objectives has been introduced (Liu et al, 2020a), with some of them modeling reconstruction of the perturbed word order (Lewis et al, 2020;Tao et al, 2021;Panda et al, 2021).…”

Section: Introductionmentioning

confidence: 99%

Shaking Syntactic Trees on the Sesame Street: Multilingual Probing with Controllable Perturbations

Taktasheva¹,

Mikhailov²,

Artemova³

2021

Preprint

View full text Add to dashboard Cite

Recent research has adopted a new experimental field centered around the concept of text perturbations which has revealed that shuffled word order has little to no impact on the downstream performance of Transformer-based language models across many NLP tasks. These findings contradict the common understanding of how the models encode hierarchical and structural information and even question if the word order is modeled with position embeddings. To this end, this paper proposes nine probing datasets organized by the type of controllable text perturbation for three Indo-European languages with a varying degree of word order flexibility: English, Swedish and Russian. Based on the probing analysis of the M-BERT and M-BART models, we report that the syntactic sensitivity depends on the language and model pre-training objectives. We also find that the sensitivity grows across layers together with the increase of the perturbation granularity. Last but not least, we show that the models barely use the positional information to induce syntactic trees from their intermediate self-attention and contextualized representations.

show abstract

An Adversarial Approach for Unsupervised Syntax-Guided Paraphrase Generation

Xue

Zhao²,

Liu³

et al. 2022

Natural Language Processing and Chinese Computing

View full text Add to dashboard Cite

Learning to Organize a Bag of Words into Sentences with Neural Networks: An Empirical Study

Cited by 7 publications

References 22 publications

Reimagining Carbon Nanomaterial Analysis: Empowering Transfer Learning and Machine Vision in Scanning Electron Microscopy for High-Fidelity Identification

Reimagining Carbon Nanomaterial Analysis: Empowering Transfer Learning and Machine Vision in Scanning Electron Microscopy for High-Fidelity Identification

Shaking Syntactic Trees on the Sesame Street: Multilingual Probing with Controllable Perturbations

An Adversarial Approach for Unsupervised Syntax-Guided Paraphrase Generation

Contact Info

Product

Resources

About