Shubham Bansal

Summer Intern at 3S Labs (India)

Shubham Bansal

Paper Title

Generic Executable Unpacking using Dynamic Binary Instrumentation



Today to hide the main purpose of an executable from reverse engineering many techniques are used like binary and source code obfuscation , control-flow obfuscation , virtualization of original code and binary packing are few of them. One technique commonly used is code packing as packed executables hinders code analysis. This problem has been researched previously but the existing solutions either needs the working of the particular packer or are vulnerable to various evasion techniques. We can use signature matching for finding which packer is used to pack the executable but this approach is only suitable for executable packed with known packer. Universal PE Unpacker or PolyUnpack extracts the packed executable using some heuristic , but a heuristic can be evaded easily.


In this paper, we propose a dynamic approach to handle this problem that captures the original hidden code as soon as it appears in the memory space of the process. This tool monitors program execution, memory read/write at run-time, determines if the currently executing code is newly-generated, and if it is, tool extracts that newly generated hidden code, and repeat the process all over again until we reach a time-out. Thus, we gets all layers of packing with each code extraction. Our tool can unpack the executable which is packed with some unknown packer or some modified packing technique. In addition to the original code, our tool also finds other information such as Original Entry Point [OEP] which is important in further analysis of the original code. Our tool finds out the exact regions where the hidden code and data resides in memory. Also, malwares are generally packed multiple times, our tool can extract each packed layer with their corresponding OEP.

Our Approach

No matter what packing technique you used or how many hidden layers are present, the original program will be present in the memory at certain point of time and the Instruction pointer should jump to the OEP of the original code. We denote the memory region which is written over as dirty and the memory region which is not written over since the start as clean. We monitor the program, if it jumps to a memory region which has dirty bytes, if it is, then we have unpacked a layer of packing and we dump only the dirty bytes and set the OEP for that layer as the jump address of that memory region. At starting, we initialised program's memory mapping as clean. If we write to memory during the program run then we change that byte as dirty. When checking newly generated instructions, we are not going to check every instruction. To optimize the performance, we are going to check each basic block that comes during execution and if that basic block has dirty byte then this block's starting address is the OEP of the original code.

Our Implementation

UndoPack(our tool) is build on Intel's PIN Framework. During analysis, our tool runs in emulated environment. We needed an instrumentation framework to track the memory read/write and execution flow change. For assigning memory as dirty or clean we implemented Shadow Memory in the process memory space at higher mapping address so that it doesn't interfere with the main program mapping and initialised it as clean. In our Shadow Memory implementation , we are mapping process's each memory byte to a bit which can take only two values(either 0 or 1).Our Algorithm maps each byte at particular byte offset in the process memory to the exact same bit offset in the shadow memory. In our implementation 0 represents clean and 1 represents dirty bit. Everytime the program maps the library in memory region, we change the status of that memory region to clean as the library is not a part of hidden original code. UndoPack instruments each memory write instruction and changes the status of that memory write address region to dirty. UndoPack also instruments each basic block to check the status of that basic block and if that basic block has dirty bytes and we are going to execute those dirty bytes then we have unpacked another layer of packing. We dump the dirty memory region at that time to get the code at that layer and at the end of that basic block we change the status of whole memory region back to clean as we assumes that we have extracted a layer of packing and starting for unpacking another layer as fresh. We also added a time-out feature in the code to stop it from going into infinite loop of unpacking.

Speaker Bio

  • I am a B.Tech(hons.) 3rd year student, Computer Science and Engineering Undergraduate from IIIT Hyderabad
  • Security Researcher
  • Current area of research includes Reverse Engineering, Exploit Development, Cryptography
  • Working on - Deobfuscation of Virtualized Malware - in my Honors Project
  • Dedicated CTF Player, Current Team - SegFault
  • My Blog -

Copyright © 2023 | Nullcon India | International Security Conference | All Rights Reserved