PKUSEG: A Toolkit for Multi-Domain Chinese Word Segmentation
Published in
arXiv.org(2019)External Links:
TL
TL;DR
The new toolkit, named PKUSEG, targets at multi-domain word segmentation and provides separate models for different domains, such as web, medicine, and tourism, and supports POS tagging and model training to adapt to various application scenarios.
Abstract
Chinese word segmentation (CWS) is a fundamental step of Chinese natural language processing. In this paper, we build a new toolkit, named PKUSEG, for multi-domain word segmentation. Unlike existing single-model toolkits, PKUSEG targets at multi-domain word segmentation and provides separate models for different domains, such as web, medicine, and tourism. The new toolkit also supports POS tagging and model training to adapt to various application scenarios. Experiments show that PKUSEG achieves high performance on multiple domains. The toolkit is now freely and publicly available for the usage of research and industry.
References12 items
1
Transfer Deep Learning for Low-Resource Chinese Word Segmentation with a Novel Neural Network
2
Overview of the NLPCC-ICCPOL 2016 Shared Task: Chinese Word Segmentation for Micro-Blog Texts
3
Dependency-based Gated Recursive Neural Network for Chinese Word Segmentation
4
Neural Word Segmentation Learning for Chinese
5