텍스트 네트워크 분석을 위한 대규모 텍스트의 자료정리(Data Cleaning) 방법에 관한 연구

초록

With a help from the development of text mining technology, social scientists, including those in policy science field, could use massive text data vigorously in their works. The epitome of these types of studies is policy framing analysis, in which social network analysis is conjoined with text analysis. To transform unstructured text data into structured one for empirical analysis such as framing study, various methodological issues have to be taken account for. However, most previous studies in Korea has not specified how data cleaning of text data was performed in their analysis procedure. In that regard, this study tried to formalize the text data cleaning processes, which is collected from the diverse sources such as internet newspaper articles. To ensure this end, this study proposes 1) specific factors in cleaning text data including the typology of text data coding errors, selecting criteria of stopwords and etc., 2) data cleaning process using approximately 5% sample extracted from the data set. For this small sample, comparisons were conducted between the coding done by researchers and a computer, so as to capture the errors from the computer programs’ coding. The study collected 331 newspaper articles to demonstrate the validity of the proposed methods. When the proposed methods were applied, the error rates reduced to about 30% comparing the coding done by the computer without any treatment.

키워드