Original scientific paper

https://doi.org/10.31534/engmod.2021.2.ri.02d

Multi-turn Dialogue Model Based on the Improved Hierarchical Recurrent Attention Network

Jiawei Miao orcid.org/0000-0002-8135-6222 ; School of Computer Science and Software Engineering, University of Science and Technology Liaoning, Anshan 114044, CHINA
Jiansheng Wu orcid.org/0000-0003-0302-2734 ; School of Computer Science and Software Engineering, University of Science and Technology Liaoning, Anshan 114044, CHINA

Abstract

When considering the multi-turn dialogue systems, the model needs to generate a natural and contextual response. At present, HRAN, one of the most advanced models for multi-turn dialogue problems, uses a hierarchical recurrent encoder-decoder combined with a hierarchical attention mechanism. However, for complex conversations, the traditional attention-based RNN does not fully understand the context, which results in attention to the wrong context that generates irrelevant responses. To solve this problem, we proposed an improved hierarchical recurrent attention network, a self-attention network (HSAN), instead of RNN, to learn word representations and utterances representations. Empirical studies on both Chinese and English datasets show that the proposed model has achieved significant improvement.

Keywords

Multi-turn dialogue; hierarchical neural network; attention mechanism; self-attention

Hrčak ID:

260827

URI

https://hrcak.srce.hr/260827

Publication date:

25.11.2021.

Visits: 893 *