HOME JOURNALS CONTACT

Journal of Applied Sciences

Year: 2013 | Volume: 13 | Issue: 16 | Page No.: 3153-3160
DOI: 10.3923/jas.2013.3153.3160
Ambiguity Analysis Model of Word Segmentation Based on Word Group
Rongliang Luo, Hongxi Zhang and Minghui Wu

Abstract: We propose a word segment algorithm based on word group. In this model, word group is used for Ambiguity analysis. At first step, statistical information is used for build information base. In the process of dealing with sentence, a small step is triggered for counting information of adjoining situation and word frequency and calculates parameters of this model according to size of window. When get different word sequences, we use Analysis Tree to find the prime sequence. Because of short in decision distance, we get a low time complexity. Algorithm analyzing and result of experiment show that segmentation algorithm based on word group has higher efficiency and accuracy.

Fulltext PDF

How to cite this article
Rongliang Luo, Hongxi Zhang and Minghui Wu, 2013. Ambiguity Analysis Model of Word Segmentation Based on Word Group. Journal of Applied Sciences, 13: 3153-3160.

Keywords: Analyses of ambiguity, analysis tree of ambiguity, word group and segmentation degree

REFERENCES
  • Zhang, Y. and S. Clark, 2010. A fast decoder for joint word segmentation and POS-tagging using a single discriminative model. Proceedings of the Conference on Empirical Methods in Natural Language Processing, October 9-11, 2010, Cambridge, MA., USA., pp: 843-852.

  • Huang, C.N. and H. Zhao, 2007. Ten-year review about Chinese segmentation. J. Chin. Inform. Process., 21: 2-16.

  • Wang, H.S., J. Zhu, S. Tang and X. Fan, 2011. A new unsupervised approach to word segmentation. Comput. Linguist., 37: 421-454.
    CrossRef    Direct Link    

  • Li, Z., 2011. Parsing the internal structure of words: A new paradigm for chinese word segmentation. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, June 19-24, 2011, Portland, ON., USA., pp: 1405-1414.

  • Song, C., S.Y. Zhao and X.Z. Zhou, 2012. Algorithm research of segmentation technology in vertical search engine. Comput. Technol. Dev., Vol. 2.

  • Zhang, C.P., L.L. Zhao and C.M. Wu, 2010. Method of Chinese word segmentation based on character-word classification. J. Comput. Appl., 30: 2034-2037.
    Direct Link    

  • Yue, J.Y., J.A. Xu and Y.Y. Zhang, 2012. Chinese word segmentation technology for patent documents. Peking University Press, Beijing.

    • © Science Alert. All Rights Reserved