Original scientific paper
https://doi.org/10.31534/engmod.2021.2.ri.02d
Multi-turn Dialogue Model Based on the Improved Hierarchical Recurrent Attention Network
Jiawei Miao
orcid.org/0000-0002-8135-6222
; School of Computer Science and Software Engineering, University of Science and Technology Liaoning, Anshan 114044, CHINA
Jiansheng Wu
orcid.org/0000-0003-0302-2734
; School of Computer Science and Software Engineering, University of Science and Technology Liaoning, Anshan 114044, CHINA
Abstract
When considering the multi-turn dialogue systems, the model needs to generate a natural and contextual response. At present, HRAN, one of the most advanced models for multi-turn dialogue problems, uses a hierarchical recurrent encoder-decoder combined with a hierarchical attention mechanism. However, for complex conversations, the traditional attention-based RNN does not fully understand the context, which results in attention to the wrong context that generates irrelevant responses. To solve this problem, we proposed an improved hierarchical recurrent attention network, a self-attention network (HSAN), instead of RNN, to learn word representations and utterances representations. Empirical studies on both Chinese and English datasets show that the proposed model has achieved significant improvement.
Keywords
Multi-turn dialogue; hierarchical neural network; attention mechanism; self-attention
Hrčak ID:
260827
URI
Publication date:
25.11.2021.
Visits: 1.176 *