论文部分内容阅读
近年来,在自然语言处理领域,基于语料库方法重新复苏并引起了广泛关注。尽管这种方法在自然语言处理的许多领域中得到了成功的应用,如:词性标注和语音识别,但是用语料库方法在汉语切分阶段还没有获得理想的结果。而且用语料库方法获取的知识一般是用统计的数据给出的,使用这些统计数据来分析、理解自然语言很不直观。对于汉语切分,本文提出了基于规则的错误驱动学习,这种方法既可以利用大规模语料库,同时也能利用语言学家总结的语言学规律,并且使用学习后得到的规则重新调整切分语料能提高切分的正确率
In recent years, in the field of natural language processing, the corpus-based approach has been revived and has drawn much attention. Although this method has been successfully applied in many areas of natural language processing, such as POS tagging and speech recognition, the corpus method has not yet achieved the desired results in the Chinese segmentation stage. And the knowledge gained by the corpus method is generally given by statistical data. It is not intuitive to use these statistics to analyze and understand the natural language. For Chinese segmentation, this paper proposes rule-based error-driven learning, which not only can make use of large-scale corpus, but also can make use of the linguistic laws summarized by linguists and readjust the segmentation corpus Can improve the correct rate of segmentation