Poet: Product-oriented Video Captioner for E-commerce

Zhang, Shengyu; Tan, Ziqi; Yu, Jianhua; Zhao, Zhou; Kuang, Kun; Liu, Jie; Zhou, Jingren; Yang, Hongxia; Wu, Fei

doi:10.1145/3394171.3413880

Cited by 27 publications

(11 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, our study is valuable to video producers and instructors towards high-quality production and organization of educational video content. Additionally, the importance of sensemaking in video goes beyond learning and education, as videos have made significant inroads into online domains such as digital marketing [3], e-commerce [41]. The findings of this study can be beneficial to businesses and organizations interested in the sensemaking of videos on social media.…”

Section: Implications For Practicementioning

confidence: 94%

See 1 more Smart Citation

Educative Sensemaking on Social Media: An Empirical Investigation of Informal Learning on YouTube

Shen¹,

Pritchard²,

Tan³

et al. 2022

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

Educational videos on social media are widely used in informal learning. However, empirical studies hardly look into sensemaking, a key aspect in the construction of meaning and knowledge, of educational videos on social media in informal learning, despite the growing interest and practice in educative sensemaking. This study addresses this research gap. We draw upon sensemaking theories and investigate how the physical properties of educational videos affect sensemaking. Our research shows how information control, anchor, and noise are associated with committed interpretation in the learning communities to understand the scientific inquiry at hand with data from YouTube educational videos. This study makes timely contributions to the literature on the educative sensemaking in informal learning on social media. It also offers insights into the better design of educational videos to facilitate sensemaking and informal learning.

show abstract

Section: Implications For Practicementioning

confidence: 94%

“…Sensemaking has diverse theoretical routes and has been explored in a wide variety of domains and disciplines. Four perspectives on sensemaking have become very influential: cost structure of sensemaking [30], the data/frame theory [17], individual sensemaking [6], and collective sensemaking [41].…”

Section: Sensemaking Theoriesmentioning

confidence: 99%

Educative Sensemaking on Social Media: An Empirical Investigation of Informal Learning on YouTube

Shen¹,

Pritchard²,

Tan³

et al. 2022

Proceedings of the Annual Hawaii International Conference on System Sciences

View full text Add to dashboard Cite

show abstract

“…BFVD & FFVD , Large scale product-oriented video captioning datasets proposed by POET [61], for video captioning in the field of e-commerce. The videos having page views over 1,00,000 and a click-through rate of more than 5% are collected from mobile Taobao (a Chinese shopping website) and labeled as either buyer or fan-generated.…”

Section: P Bfvd and Ffvd (Buyer-generated Fashion Video Dataset And Fan-generated Fashion Video Dataset)mentioning

confidence: 99%

“…A considerable number of unique words in the datasets make it among the largest datasets for the task. Figure 6 represents the POET's [61] sample generation on FFVD test dataset in comparison to AA-Transformer [61] and AA-Recnet [61] models.…”

Section: P Bfvd and Ffvd (Buyer-generated Fashion Video Dataset And Fan-generated Fashion Video Dataset)mentioning

confidence: 99%

Video Description: Datasets & Evaluation Metrics

Rafiq

Choi

2021

IEEE Access

View full text Add to dashboard Cite

Rapid expansion and the novel phenomenon of deep learning have manifested a variety of proposals and concerns in the area of video description, particularly in the recent past. Automatic event localization and textual alternatives generation for the complex and diverse visual data supplied in a video can be articulated as video description, bridging the two leading realms of computer vision and natural language processing. Several sequence-to-sequence algorithms are being proposed by splitting the task into two segments, namely encoding, i.e., getting and learning the insights of the visual representations, and decoding, i.e., transforming the learned representations to a sequence of words, one at a time. Implemented deep learning approaches have gained a lot of recognition for the reason of their superior computing capabilities and tremendous performance. However, the accomplishment of these algorithms strongly depends on the nature, diversity, and amount of data they are trained, validated and tested on. Techniques applied on insufficient and inadequate train/test data cannot deliver promising conclusions, consequently making it complicated to evaluate the quality of generated results. This survey focuses explicitly on the benchmark datasets, and evaluation metrics developed and deployed for video description tasks and their capabilities and limitations. Finally, we concluded with the need for essential enhancements and encouraging research directions on the topic.

show abstract

“…As such, most existing approaches for (single) image captioning [9,14,16,22,25,32,53,56,66,75] cannot be directly adopted to handle this task since they neglect the sequential dependencies over the image stream when composing a story with multiple isolated sentences rather than a holistic story line. In addition, the inter-image temporal gap and visual change in an image stream are often far greater than the inter-frame variations in a video, making video captioning methods [4,13,21,29,41,49,50,54,73,[76][77][78] perform unsatisfactorily for visual storytelling either.…”

Section: Introductionmentioning

confidence: 99%

Latent Memory-augmented Graph Transformer for Visual Storytelling

Qin

Huang

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Visual storytelling aims to automatically generate a human-like short story given an image stream. Most existing works utilize either scene-level or object-level representations, neglecting the interaction among objects in each image and the sequential dependency between consecutive images. In this paper, we present a novel Latent Memory-augmented Graph Transformer (LMGT), a Transformer based framework for visual story generation. LMGT directly inherits the merits from the Transformer, which is further enhanced with two carefully designed components, i.e., a graph encoding module and a latent memory unit. Specifically, the graph encoding module exploits the semantic relationships among image regions and attentively aggregates critical visual features based on the parsed scene graphs. Furthermore, to better preserve inter-sentence coherence and topic consistency, we introduce an augmented latent memory unit that learns and records highly summarized latent information as the story line from the image stream and the sentence history. Experimental results on three widely-used datasets demonstrate the superior performance of LMGT over the state-of-the-art methods. CCS CONCEPTS• Computing methodologies → Natural language generation; Scene understanding.

show abstract

Poet: Product-oriented Video Captioner for E-commerce

Cited by 27 publications

References 37 publications

Educative Sensemaking on Social Media: An Empirical Investigation of Informal Learning on YouTube

Educative Sensemaking on Social Media: An Empirical Investigation of Informal Learning on YouTube

Video Description: Datasets & Evaluation Metrics

Latent Memory-augmented Graph Transformer for Visual Storytelling

Contact Info

Product

Resources

About