|
Malware, with harmful intent to compromise computer systems, has been one of the significant challenges to the Internet. Driven by the rich profit, relentless malware developers apply various obfuscation schemes to circumvent malware detection. Binary packing is the most common obfuscation adopted by malware authors to camouflage malicious code and defeat popular signature-based malware detection. Binary packing first encrypts or compresses malware code as data, making it immune to static analysis. At run time, the attached unpacking routine writes the decoded code to memory and then resumes malicious payload execution. Over the past two decades, packed malware has been a challenge in the anti-malware landscape. This project addresses this problem from new angles and advances the state of the art in terms of better performance and stronger anti-analysis resistance. The project's novelties are new methods and efficient tools to extract packed malware payload without the prior knowledge of packers. The project's impacts are paving the way for large-scale malware analysis and helping people respond to emerging malware attacks promptly.
Existing generic binary unpacking work suffers from high runtime overhead and lack of anti-analysis resistance. This project conducts an in-depth study on an enormous variety of malware packers and reveals promising research directions to address the long-standing binary unpacking problem. Based on the investigator's encouraging preliminary results, this project goes one step further to address the unsolved challenges and pave the last mile to a complete generic unpacking solution. This project develops a novel machine learning model to extract the semantics of the original entry point. The proposed technique notably outperforms existing search heuristics. This project's hybrid de-obfuscation approaches enable unpacking tools to recover a fully functional version of the original binary, which is the ultimate goal of unpacking technique. To achieve stronger resilience to various anti-analysis attacks, the investigator advances the use of hardware supported lower-level features to detecting the end of unpacking. The proposed methods can handle a broader range of malware packers, even brand new packers.
This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
|
CRII: SaTC: Towards Paving the Way for Large-Scale Malware Analysis: New Directions in Generic Binary Unpacking