Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation

Hu, Zhiting; Shi, Haoran; Tan, Bowen; Wang, Wentao; Yang, Zichao; Zhao, Tiancheng; He, Junxian; Qin, Lianhui; Wang, Di; Ma, Xuezhe; Liu, Zhengzhong; Liang, Xiaodan; Zhu, Wangrong; Sachan, Devendra Singh; Xing, Eric P.

doi:10.48550/arxiv.1809.00794

Cited by 15 publications

(10 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our proposed GTAE achieves the best content preservation ability as expected, while maintaining highly comparable performance on naturalness and style transfer intensity with the state-of-the-arts. The model is implemented using the Texar [41] tookit for text generation based on the Tensorflow backend [42].…”

Section: Methodsmentioning

confidence: 99%

GTAE: Graph-Transformer based Auto-Encoders for Linguistic-Constrained Text Style Transfer

Shi

Zhang

Zhou

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Non-parallel text style transfer has attracted increasing research interests in recent years. Despite successes in transferring the style based on the encoder-decoder framework, current approaches still lack the ability to preserve the content and even logic of original sentences, mainly due to the large unconstrained model space or too simplified assumptions on latent embedding space. Since language itself is an intelligent product of humans with certain grammars and has a limited rulebased model space by its nature, relieving this problem requires reconciling the model capacity of deep neural networks with the intrinsic model constraints from human linguistic rules. To this end, we propose a method called Graph Transformer based Auto Encoder (GTAE), which models a sentence as a linguistic graph and performs feature extraction and style transfer at the graph level, to maximally retain the content and the linguistic structure of original sentences. Quantitative experiment results on three non-parallel text style transfer tasks show that our model outperforms state-of-the-art methods in content preservation, while achieving comparable performance on transfer accuracy and sentence naturalness.

show abstract

Section: Methodsmentioning

confidence: 99%

GTAE: Graph-Transformer based Auto-Encoders for Linguistic-Constrained Text Style Transfer

Shi

Zhang

Zhou

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We used an encoder-decoder sequence-to-sequence architecture with bidirectional forwardbackward RNN encoder and an attention-based RNN decoder [23], as implemented in PyTorch-Texar [24]. While this architecture is no longer the top performer in terms of ROUGE metric -currently, large pre-trained self-attention models such as BERT are the state-of-the-art [25] -it is much more efficient in training.…”

Section: Methodsmentioning

confidence: 99%

Differentiable Combinatorial Losses through Generalized Gradients of Linear Programs

Gao,

Zhang,

Panahi

et al. 2019

Preprint

View full text Add to dashboard Cite

When samples have internal structure, we often see a mismatch between the objective optimized during training and the model's goal during inference. For example, in sequence-to-sequence modeling we are interested in high-quality translated sentences, but training typically uses maximum likelihood at the word level. Learning to recognize individual faces from group photos, each captioned with the correct but unordered list of people in it, is another example where a mismatch between training and inference objectives occurs. In both cases, the natural training-time loss would involve a combinatorial problem -dynamic programming-based global sequence alignment and weighted bipartite graph matching, respectively -but solutions to combinatorial problems are not differentiable with respect to their input parameters, so surrogate, differentiable losses are used instead. Here, we show how to perform gradient descent over combinatorial optimization algorithms that involve continuous parameters, for example edge weights, and can be efficiently expressed as integer, linear, or mixed-integer linear programs. We demonstrate usefulness of gradient descent over combinatorial optimization in sequenceto-sequence modeling using differentiable encoder-decoder architecture with softmax or Gumbel-softmax, and in weakly supervised learning involving a convolutional, residual feed-forward network for image classification.

show abstract

“…We implement our experiments using Tensorflow through the Texar platform [15]. We use a three-layer Transformer with default eight heads in the encoder and decoder.…”

Section: Experimental Settingsmentioning

confidence: 99%

Controllable and Diverse Text Generation in E-commerce

Shao

Wang

et al. 2021

Proceedings of the Web Conference 2021

View full text Add to dashboard Cite

In E-commerce, a key challenge in text generation is to find a good trade-off between word diversity and accuracy (relevance) in order to make generated text appear more natural and human-like. In order to improve the relevance of generated results, conditional text generators were developed that use input keywords or attributes to produce the corresponding text. Prior work, however, do not finely control the diversity of automatically generated sentences. For example, it does not control the order of keywords to put more relevant ones first. Moreover, it does not explicitly control the balance between diversity and accuracy. To remedy these problems, we propose a fine-grained controllable generative model, called Apex, that uses an algorithm borrowed from automatic control (namely, a variant of the proportional, integral, and derivative (PID) controller) to precisely manipulate the diversity/accuracy trade-off of generated text. The algorithm is injected into a Conditional Variational Autoencoder (CVAE), allowing Apex to control both (i) the order of keywords in the generated sentences (conditioned on the input keywords and their order), and (ii) the trade-off between diversity and accuracy. Evaluation results on real world datasets 1 show that the proposed method outperforms existing generative models in terms of diversity and relevance. Moreover, it achieves about 97% accuracy in the control of the order of keywords.Apex is currently deployed to generate production descriptions and item recommendation reasons in Taobao 2 , the largest E-commerce platform in China. The A/B production test results show that our method improves click-through rate (CTR) by 13.17% compared * Part of work was completed at Alibaba Group.

show abstract

Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation

Cited by 15 publications

References 25 publications

GTAE: Graph-Transformer based Auto-Encoders for Linguistic-Constrained Text Style Transfer

GTAE: Graph-Transformer based Auto-Encoders for Linguistic-Constrained Text Style Transfer

Differentiable Combinatorial Losses through Generalized Gradients of Linear Programs

Controllable and Diverse Text Generation in E-commerce

Contact Info

Product

Resources

About