Web数据反馈的搭配抽取方法

Automatic collocation extraction using web feedback data

摘要:

为了提高搭配(Collocation)抽取的精度,提出一种新的互联网数据的搭配抽取方法.传统的搭配抽取统计方法都是基于语料库的,常受到语料库规模的影响和制约,而在互联网数据中蕴含着丰富的知识和信息,基于Web的词汇相关性度量方法,充分利用搭配在谷歌中的页面数模拟其对应语料库的词频数,并分别选取共现频率、互信息、卡方检验3种经典统计关联度量方法.实验结果表明召回率、精确率均好于对应的基于语料库的方法,这说明互联网中大量数据应用于自然语言处理各种任务的可行性.

Abstract:

To improve the precison of collocation extraction,this paper proposes a new method based on Internet data.For the constraint by the corpus scale for traditional collocation extraction approach based on linguistic corpus,we acquire collocations from Web,which contains plenty of information and knowledge.Three classical association measures of co-occurrence frequency,mutual information and χ2-test are used to automatically extract the collocation.Based on the experimental results,the benchmarks show that the performance of this new Web-based approach is superior to that of traditional approach in both precision and recall.Thus the data from Internet may be applied in many NLP applications.