Biblio

Malware analysis uses debuggers to understand and manipulate the behaviors of stripped binaries. To circumvent analysis, malware applies a variety of anti-debugging techniques, such as self-modifying, checking for or removing breakpoints, hijacking keyboard and mouse events, escaping the debugger, etc. Most state-of-the-art debuggers are vulnerable to these anti-debugging techniques. In this paper, we first systematically analyze the spectrum of possible anti-debugging techniques and compile a list of 79 attack vectors. We then propose a framework, called Apate, which detects and defeats each of these attack vectors, by performing: (1) just-in-time disassembling based on single-stepping, (2) careful monitoring of the debuggee's execution and, when needed, modification of the debuggee's states to hide the debugger's presence. We implement Apate as an extension to WinDbg and extensively evaluate it using five different datasets, with known and new malware samples. Apate outperforms other debugger-hiding technologies by a wide margin, addressing 58+–465+ more attack vectors.

Existing anti-debugging protections are relatively weak. In existing self-debugger approaches, a custom debugger is attached to the main application, of which the control flow is obfuscated by redirecting it through the debugger. The coupling between the debugger and the main application is then quite loose, and not that hard to break by an attacker. In the tightly-coupled self-debugging technique proposed in this paper, full code fragments are migrated from the application to the debugger, making it harder for the attacker to reverse-engineer the program and to deconstruct it into the original unprotected program to attach a debugger or to collect traces. We evaluate a prototype implementation on three complex, real-world Android use cases and present the results of tests conducted by professional penetration testers.