A Corpus for Understanding and Generating Moral Stories

Guan, Jian; Liu, Ziqi; Huang, Minlie

doi:10.18653/v1/2022.naacl-main.374

Cited by 2 publications

(20 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we transform the traditional two-stage paradigm of "pre-training + fine-tuning" into three stage by adding an intermediate step of continual pre-training that will be tested by two downstream tasks about moral understanding. We use STORAL-ZH [18], the Chinese part of STORAL [6], as the dataset for target tasks. Furthermore, LongLM-base [21] is selected as our model, that has been pre-trained on 120G Chinese long novels.…”

Section: Storymentioning

confidence: 99%

“…There have been a range of tasks proposed about story understanding and generation, including story ending prediction [2], commonsense story generation [22] and story ending generation with fine-grained sentiment [23]. A variety of attributes are considered for better story understanding, such as storylines [3], emotions [4], styles [5], and morals [6]. Different from that storylines lead the story writting, emotions describe characters ' states, styles decide the story's tone, moral understanding aims to discover the implied and abstract theme behind concrete events, that is a more challenging task.…”

Section: Story Understandingmentioning

confidence: 99%

“…Different from that storylines lead the story writting, emotions describe characters ' states, styles decide the story's tone, moral understanding aims to discover the implied and abstract theme behind concrete events, that is a more challenging task. [6] firstly proposed moral understanding and generation, and published a new moral story dataset, STORAL.…”

Section: Story Understandingmentioning

confidence: 99%

“…Moral understanding aims to comprehend the abstract concepts that hide in a story by seeing through concrete events and vivid characters, which has become a new challenging task for Natural Language Processing (NLP). Previous works related to story understanding are mainly story ending prediction [2], story completion given constraints (e.g., storylines [3], emotions [4], styles [5], morals [6]). Most of them surround concrete concepts in story itself, whereas our work concentrates on digging out the moral lesson behind it.…”

Section: Introductionmentioning

confidence: 99%

“…Most of them surround concrete concepts in story itself, whereas our work concentrates on digging out the moral lesson behind it. Table 1 shows one example of moral-story pair from the new dataset STORAL [6].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Knowledge-Enriched Moral Understanding upon Continual Pre-training

Jiang¹,

Yue²,

Atkinson³

et al. 2023

Computer Networks &Amp; Communications

View full text Add to dashboard Cite

The aim of moral understanding is to comprehend the abstract concepts that hide in a story by seeing through concrete events and vivid characters. To be specific, the story is highly summarized in one sentence without covering any characters in the original story, which requires the machine to behave more intelligently with the abilities of moral perception and commonsense reasoning. The paradigm of “pre-training + fine-tuning” is generally accepted for applying neural language models. In this paper, we suggest adding an intermediate stage to build the flow of “pre-training + continual pre-training + finetuning”. Continual pre-training refers to further training on task-relevant or domainspecific corpora with the aim of bridging the data distribution gap between pre-training and fine-tuning. Experiments are basing on a new moral story dataset, STORAL-ZH, that composes of 4,209 Chinese story-moral pairs. We collect a moral corpus about Confucius theory to enrich the T5 model with moral knowledge. Furthermore, we leverage a Chinese commonsense knowledge graph to enhance the model with commonsense knowledge. Experimental results demonstrate the effectiveness of our method, compared with several state-of-the-art models including BERT-base, RoBERTa-base and T5-base.

show abstract

Section: Storymentioning

confidence: 99%

Section: Story Understandingmentioning

confidence: 99%

Section: Story Understandingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Knowledge-Enriched Moral Understanding upon Continual Pre-training

Jiang¹,

Yue²,

Atkinson³

et al. 2023

Computer Networks &Amp; Communications

View full text Add to dashboard Cite

show abstract

Untitled

2023

IJNLC

View full text Add to dashboard Cite

The goal of moral understanding is to grasp the theoretical concepts embedded in a narrative by delving beyond the concrete occurrences and dynamic personas. Specifically, the narrative is compacted into a single statement without involving any characters within the original text, necessitating a more astute language model that can comprehend connotative morality and exhibit commonsense reasoning. The "pretraining + fine-tuning" paradigm is widely embraced in neural language models. In this paper, we propose an intermediary phase to establish an improved paradigm of "pre-training + further pre-training + finetuning". Further pre-training generally refers to continual learning on task-specific or domain-relevant corpora before being applied to target tasks, which aims at bridging the gap in data distribution between the phases of pre-training and fine-tuning. Our work is based on a Chinese dataset named STORAL-ZH that composes of 4k human-written story-moral pairs. Furthermore, we design a two-step process of domain-adaptive pre-training in the intermediary phase. The first step depends on a newly-collected Chinese dataset of Confucian moral culture. And the second step bases on the Chinese version of a frequently-used commonsense knowledge graph (i.e. ATOMIC) to enrich the backbone model with inferential knowledge besides morality. By comparison with several advanced models including BERTbase, RoBERTa-base and T5-base, experimental results on two understanding tasks demonstrate the effectiveness of our proposed three-phase paradigm.

show abstract

A Corpus for Understanding and Generating Moral Stories

Cited by 2 publications

References 15 publications

Knowledge-Enriched Moral Understanding upon Continual Pre-training

Knowledge-Enriched Moral Understanding upon Continual Pre-training

Untitled

Contact Info

Product

Resources

About