Review on Hindi Chunker

Abstract

Natural Language Processing is one of the important fields of research in Artificial Intelligence.
Most of the work is done in English, but due to unavailability or scarcity of resources for Indian
languages, not much progress has been accomplished in this area. Indian languages are free
word order languages where the order of the word does not matter and this makes Indian
languages difficult to process. For processing a language, the first step is a morphological
analysis of tokens/words of the language, then Part-of-Speech is assigned using different
approaches and finally, a parser is made in the specified language. The parser specifies the
syntactic and grammatical relationships. In Hindi and other Indian languages which are free
word order, there is a requirement for two phase parsing where first local dependencies are
handled and then long distance dependencies are handled. A shallow parser or local word
grouper is the first stage of parsing Indian Languages. They divide the sentence into chunks or
groups based on their phrases. This paper gives the review of the techniques of the shallow
parser for Hindi including probabilistic/ machine learning methods, principle-based approaches
and fusion based approaches. The paper provides a review of various papers from well known
journals and conferences.