层次化神经网络模型下的释义识别方法

Paraphrase identification based on hierarchical neural network

摘要:

释义识别技术（Paraphrase Identification, PI）被广泛用于问答系统、抄袭检测、个性化推荐等领域.针对已有释义识别方法缺乏有效的特征提取机制问题,提出了一种新的释义识别模型.与传统“编码-匹配”模式不同,采用“编码-匹配-提取”模式,通过添加特征提取层进一步提取分类信息.所提出模型由6层组成:输入层、嵌入层、编码层、匹配层、特征提取层、输出层.在编码层,采用基于注意力机制的上下文双向长短期记忆网络对文本上下文进行编码,充分利用句子的前向和逆向两个方向的上下文信息；在匹配层,通过多种矩阵运算,从不同角度获得句子对匹配信息；在特征提取层,利用Xception网络以便更有效地从匹配结果中提取分类信息.此外,本文采用多特征融合的方法,将GloVe预训练的词向量、字符向量和附加特征向量的连接作为最终的词向量,较普通的词向量携带更丰富的语义信息.实验结果表明,所构建的模型在Quora和SemEval-2015 PIT两个公开数据集上（分别作为大型数据集和中小型数据集的代表）都达到了竞争性效果.

Abstract:

Paraphrase identification is widely used in question answering system, plagiarism detection, and personalized recommendation. Since the existing paraphrase identification techniques are lack of effective feature extraction mechanism, a new paraphrase model was proposed. Different from previous works which normally adopt the “encoding-matching” mode, the proposed model adopts the “encoding-matching-extraction” mode by adding feature extraction layer to better acquire classification information. The proposed model is consisted of six layers: input layer, embedding layer, encoding layer, matching layer, feature extraction layer, and output layer. The encoding layer utilizes contextual bi-directional long short-term memory network (BiLSTM) with self-attention to encode context of sentences, which can make full use of contextual information in both forward and reverse directions of a sentence. The matching layer uses several matrix operations to get sentence pair matching information from different angles. The extraction layer chooses Xception as the feature extractor to better extract classification information from the matching results. Moreover, this paper combines GloVe word vectors, character vectors, and additional feature vectors as the final embeddings, which carries richer information than ordinary pretrained embeddings. Results show that the proposed model achieved competitive results on two public datasets: Quora Question Pairs (as a representative of large datasets) and SemEval-2015 PIT (as a representative of small and medium datasets).