Year: 2025
Type: article
Abstract: Log parsing is indispensable for system maintenance, converting unstructured log data into structured formats (log templates) for further log compression and anomaly detection. The effectiveness of log parsing relies on the efficiency of two key processes: template extraction and log matching. Traditional methods, however, suffer from slow pairwise comparisons for template extraction and the tedious, non-scalable sequential approach for template matching. Our research has uncovered two opportunities for optimization based on two log locality characteristics: logs from the same template tend to cluster sequentially, and there is a limited variety of templates used within given timeframes. To exploit these opportunities, we propose the Multi-Logs Template Extraction (MLTE)-Cache framework. MLTE-Cache leverages the MLTE algorithm to enhance the efficiency of template extraction by grouping similar logs and processing them in batch mode. Furthermore, the framework utilizes a cache-assisted proximity matching algorithm to accelerate the log matching procedure. Through comprehensive experiments on open-source datasets, the MLTE-Cache framework has proven highly effective, maintaining a high level of accuracy while delivering a 37% improvement in efficiency.
Source: Tsinghua Science & Technology
Authors: Jigang Wen, Chenying Ma, Kun Xie, Shiming He, Shuyu Pei, Yuxiang Chen, Wei Liang, Quan Feng
Institution: Hunan University of Science and Technology
Language: English
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: diamond
OpenAlex ID: W7083437774