Work

LogProb: Online Parsing Evolving Logs With Complex Parameters

Year: 2025

Type: article

Abstract: To cope with the massive volume of log messages in complex systems, effective and accurate log parsers are crucial for system maintenance. However, existing log parsers have accuracy issues, particularly when handling evolving logs with complex structures. To address this, this paper proposes LogProb, an online log parsing method that incorporates a token state prediction component and a search tree-based template extraction component. The token state prediction component determines whether each token in a log message is static (template word) or dynamic (parameter value). The token sequence and predicted token states are parsed by the template extraction component to generate event templates. The extraction process includes two stages: identifying candidate templates with predicted static tokens, and using a fast matching algorithm to determine the final template. Experiments on 16 benchmark datasets show that LogProb outperforms state-of-the-art methods in accuracy while maintaining comparable efficiency.

Source: IEEE Access

DOI: 10.1109/access.2025.3569548

Authors: Yan Li, Ziying Wang, Jiaming Mao, Zhimin Gu, Haitao Jiang

Institution: State Grid Jiangsu Electric Power (China)

Language: English

Topic: Software System Performance and Reliability

Subfield: Computer Networks and Communications

Field: Computer Science

Domain: Physical Sciences

Open Access status: gold

Estimated APC paid: $2,075

Funders: State Grid Tianjin Electric Power Company, State Grid Jiangsu Electric Power