Obfuscation undercover: Unraveling the impact of obfuscation layering on structural code patterns
Year: 2024
Type: article
Abstract: Malware often uses code obfuscation to evade detection, employing techniques such as packing, virtualization, and data encoding or encryption. Despite widespread application, the impact of combining these techniques in a particular order – so-called obfuscation layering – on code analysis remains poorly understood. This study advances previous research by examining the effects of obfuscation layering on the classification of obfuscation techniques contained in binary code, focusing on how different layering combinations alter structural code patterns. Utilizing a dataset of 85 C programs modified with various combinations of code obfuscation techniques, we analyze the impact of obfuscation layering on structural code metrics such as its control flow complexity. Our study demonstrates that obfuscation layering significantly affects the ability to classify obfuscated code and that the order of applied obfuscations is less significant for classification than previously assumed. Through explainability methodologies our work offers novel insights for malware analysts and researchers to improve their detection strategies.
Authors: Sebastian Raubitzek, Sebastian Schrittwieser, Elisabeth Wimmer, Kevin Mallinger
Institution: SBA Research
Language: English
Subfield: Signal Processing
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $3,230
OpenAlex ID: W4401628931