NLPHut’s Participation at WAT2021

Parida, Shantipriya; Panda, Subhadarshi; Kotwal, Ketan; Dash, Amulya Ratna; Dash, Satya Ranjan; Sharma, Yashvardhan; Motlíček, Petr; Bojar, Ondřej

doi:10.18653/v1/2021.wat-1.16

Cited by 2 publications

(4 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To generate the Hausa captions, we followed Parida et al (2021b) who proposed a region-specific image captioning method through the fusion of the encoded features of the region and the complete image. The model consists of three modules -an encoder, fusion, and decoder -as shown in Figure 3.…”

Section: Image Caption Generationmentioning

confidence: 99%

Hausa Visual Genome: A Dataset for Multi-Modal English to Hausa Machine Translation

Abdulmumin¹,

Dash²,

Dawud³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Multi-modal Machine Translation (MMT) enables the use of visual information to enhance the quality of translations. The visual information can serve as a valuable piece of context information to decrease the ambiguity of input sentences. Despite the increasing popularity of such a technique, good and sizeable datasets are scarce, limiting the full extent of their potential. Hausa, a Chadic language, is a member of the Afro-Asiatic language family. It is estimated that about 100 to 150 million people speak the language, with more than 80 million indigenous speakers. This is more than any of the other Chadic languages. Despite a large number of speakers, the Hausa language is considered low-resource in natural language processing (NLP). This is due to the absence of sufficient resources to implement most NLP tasks. While some datasets exist, they are either scarce, machine-generated, or in the religious domain. Therefore, there is a need to create training and evaluation data for implementing machine learning tasks and bridging the research gap in the language. This work presents the Hausa Visual Genome (HaVG), a dataset that contains the description of an image or a section within the image in Hausa and its equivalent in English. To prepare the dataset, we started by translating the English description of the images in the Hindi Visual Genome (HVG) into Hausa automatically. Afterward, the synthetic Hausa data was carefully post-edited considering the respective images. The dataset comprises 32,923 images and their descriptions that are divided into training, development, test, and challenge test set. The Hausa Visual Genome is the first dataset of its kind and can be used for Hausa-English machine translation, multi-modal research, and image description, among various other natural language processing and generation tasks.

show abstract

Section: Image Caption Generationmentioning

confidence: 99%

Hausa Visual Genome: A Dataset for Multi-Modal English to Hausa Machine Translation

Abdulmumin¹,

Dash²,

Dawud³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Thereafter, a similar method was adopted for English-Bengali [54]. Inspired from [23], a sequence-to-sequence model was used for MMT [55]. In this method, the input is the concatenated source and target side, and the output is object tags, which act as dummy image features on unimodal datasets.…”

Section: Multimodal Translation On Indian Languagesmentioning

confidence: 99%

“…However, BLEU decreased slightly by 0.06 points on the challenge subset for English-Hindi. Although [55] used similar methods and observed huge improvements over text-based models, their baseline was much weaker. Our text-based baseline (without fine-tuning on multimodal data) performs better than their fine-tuned model in some subsets in terms of BLEU score.…”

Section: Non-noisy Settingsmentioning

confidence: 99%

“…74 46.32 26.68 28.91 28.62 26.30 30.01 33.84 + Text None 45.79 56.72 50.08 47.78 51.38 40.76 49.08 48.42 + SelAttn Crop 46.04 56.36 48.70 47.82 51.56 39.76 48.77 47.98 + SelAttn Full 45.30 56.03 49.74 47.48 51.55 39.49 48.86 47.67 + MMtrans Crop 45.47 55.61 49.03 47.32 52.40 40.40 48.97 47.78 + MMtrans Full 45.10 56.59 50.45 47.24 52.00 39.77 49.18 47.87 Test: Evaluation Subset; Chal: Challenge Subset; Text: Text-only finetune SelAttn: Selective Attention; MMtrans: Multimodal Transformer SOTA results are from[24,55] …”

mentioning

confidence: 99%

See 1 more Smart Citation

Experiences of Adapting Multimodal Machine Translation Techniques for Hindi

Gain¹,

Bandyopadhyay²,

Ekbal³

2021

Proceedings of the FirstWorkshop on Multimodal Machine Translation for Low Resource Languages (MMTLRL 2021)

View full text Add to dashboard Cite

Multimodal Neural Machine Translation (MNMT) is an interesting task in natural language processing (NLP) where we use visual modalities along with a source sentence to aid the source to target translation process. Recently, there has been a lot of works in MNMT frameworks to boost the performance of standalone Machine Translation tasks. Most of the prior works in MNMT tried to perform translation between two widely known languages (e.g. English-to-German, English-to-French ). In this paper, We explore the effectiveness of different state-of-the-art MNMT methods, which use various data oriented techniques including multimodal pre-training, for low resource languages. Although the existing methods works well on high resource languages, usability of those methods on low-resource languages is unknown. In this paper, we evaluate the existing methods on Hindi and report our findings.

show abstract

NLPHut’s Participation at WAT2021

Cited by 2 publications

References 17 publications

Hausa Visual Genome: A Dataset for Multi-Modal English to Hausa Machine Translation

Hausa Visual Genome: A Dataset for Multi-Modal English to Hausa Machine Translation

Experiences of Adapting Multimodal Machine Translation Techniques for Hindi

Contact Info

Product

Resources

About