United We Stand: Towards End-to-End Log-based Fault Diagnosis via Interactive Multi-Task Learning
Year: 2025
Type: preprint
Abstract: Log-based fault diagnosis is essential for maintaining software system availability. However, existing fault diagnosis methods are built using a task-independent manner, which fails to bridge the gap between anomaly detection and root cause localization in terms of data form and diagnostic objectives, resulting in three major issues: 1) Diagnostic bias accumulates in the system; 2) System deployment relies on expensive monitoring data; 3) The collaborative relationship between diagnostic tasks is overlooked. Facing this problems, we propose a novel end-to-end log-based fault diagnosis method, Chimera, whose key idea is to achieve end-to-end fault diagnosis through bidirectional interaction and knowledge transfer between anomaly detection and root cause localization. Chimera is based on interactive multi-task learning, carefully designing interaction strategies between anomaly detection and root cause localization at the data, feature, and diagnostic result levels, thereby achieving both sub-tasks interactively within a unified end-to-end framework. Evaluation on two public datasets and one industrial dataset shows that Chimera outperforms existing methods in both anomaly detection and root cause localization, achieving improvements of over 2.92% - 5.00% and 19.01% - 37.09%, respectively. It has been successfully deployed in production, serving an industrial cloud platform.
Source: arXiv (Cornell University)
Authors: He, Minghua, Duan, Chiming, Xiao, Pei, Jia, Tong, Yu, Siyu, Zhang, Lingzhe, Hong, Weijie, Han, Jin, Wu, Yifan, Li, Ying, Huang, Gang
Language: English
Subfield: Computer Networks and Communications
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7084166495
Cites:
Cited by:
Related to: