Analysis on Log Parsing: Classical Methods to LLM-Based Approaches

초록

Logs are crucial runtime evidence in distributed systems, but as free text, they must be parsed into structured templates to power tasks like debugging and anomaly detection. This survey reviews the evolution of log parsing from rule-based methods through data-driven, neural, and recent Large Language Model (LLM) approaches. We analyze parsers based on determinism, adaptability to format drift, cost, and complexity. We also highlight LLM design patterns-such as zero/fewshot prompting, RAG, caching, and hybrids-along with their strengths and risks. A key finding is that no single paradigm dominates. Scalable deployments favor hybrid models: routing stable streams to fast online parsers, applying learned models where formats vary, and escalating novel or low-confidence cases to cached LLMs. We conclude with implementation guidance and research directions toward scalable hybrid architectures and semantically richer templates to advance observability and reliability engineering.

키워드