Automatic Word Clustering in Application of Open-Ended Response Categorization

Abstract

Open ended questions are an essential and important part of survey questionnaires. They provide an opportunity for researchers to discover unanticipated information regarding the domain of study. However, they are problematic for processing since they are unstructured questions to which possible answers are not suggested, and the respondent is free to answer in his or her own words. This thesis presents novel methods of categorizing such open ended survey responses. A document clustering technique is employed in this study to categorize responses to open-ended survey questions. Supervised and unsupervised methods of categorizing open ended responses are tested in the study. Initially the author proposed a hierarchical clustering based algorithm as the unsupervised method to code the open-ended responses which were not labelled at all. The algorithm employs several natural language processing techniques to extract a classification of responses automatically. Naive Bayes classification was proposed as the supervised solution. This Naive Bayes algorithm was proposed for the open ended responses which were partially labelled. Two experiments were carried out to determine the accuracy of the proposed algorithms which proved to be promising. Hierarchical clustering based algorithm shows more than 70% accuracy when compared with the manually coded responses. The proposed Naive Bayes algorithm didn’t not illustrate the results as it expected. Therefore Positive Naive Bayes algorithm was introduced and it achieved an overall performance of 80%