Rational Text Augmentation Method with Korean Misspellings

초록

With the development of deep learning, natural language processing models have been proposed, but they may not be able to perform fully on sentences with misspellings. From this perspective, we propose a text augmentation method using Korean misspellings from online comments. Unlike previous text augmentation methods, we focus on the fact that the original meaning of Korean words can be inferred even when misspellings occur. Furthermore, Korean has a variety of misspellings even within a single character, so we analyze the frequencies of them to create augmented sentences. The experimental results show that our method can achieve an average performance improvement of 4.5% p on the NSMC and Korean Hate Speech and is a more rational and efficient method compared to previously used text augmentation methods. © 2024 IEEE.

키워드