After five parts covering how VMProtect packs, loads, checks, virtualizes, and mutates a binary, the obvious question is whether any of that can be undone. Devirtualization is the process of recovering the original semantics from a VM-protected binary. Not the exact original code, which is gone, but something semantically equivalent that a human or a decompiler can work with. I spent time looking at the tooling and approaches, and the honest answer is that it depends heavily on which protection mode was used and how the binary was configured.
The Problem
To devirtualize a function you need to do at minimum three things. First, identify the VM interpreter loop and its handlers. Second, decode the bytecode stream: find the start of the bytecode for the function, run the decryption chain from the beginning, and map each decrypted opcode to a handler. Third, for each handler, determine what CPU operation it corresponds to, then lift the bytecode sequence to an intermediate representation that is portable and optimizable.
Each of those steps has real obstacles. The opcode table is randomized so you cannot hardcode a mapping. The stream decryption is stateful so you cannot jump into the middle. And in Ultra mode the handlers themselves are obfuscated so you cannot just read what they do.
Static Approaches
Pure static devirtualization starts by locating the handler table and identifying each entry. For plain Virtualization mode this is feasible. You find the dispatch loop, trace the table lookup, and read each handler. Each handler is short, around 10 to 30 instructions, and does something recognizable: a push, a pop, an add, a comparison. You build a map from opcode index to operation.
Then you trace the bytecode. Starting from the entry state (known crypto key) you decrypt each byte, look up the operation in your map, and emit the equivalent IR instruction. Repeat until the bytecode ends. The result is a linear IR sequence that can be lifted to pseudocode.
This works, but only in favorable conditions. Multiple VMs per function break the assumption that one opcode table covers everything. The transition between VMs invalidates your current map. The direction flag means you might be reading bytecode backward. And the multi-layer operand encryption means you also have to run the per-opcode cryptors to get the actual operand values.
VTIL
The most well-known static devirtualization effort produced VTIL, a Virtual Translation Intermediate Language specifically designed for lifting VM bytecode. The idea is to define a low-level IR that is rich enough to represent any operation a VM handler might perform but generic enough to apply optimizations that collapse mutation noise.
The workflow is: lift each handler to VTIL, then run an optimizer over the VTIL that eliminates dead writes, folds constants, and removes opaque predicates. The result is a clean VTIL sequence that, when lifted further to a high-level representation, looks like something a decompiler could produce from the original code.
In practice VTIL worked well against Virtualization-mode binaries from the VMP 3.x era. It struggled with Ultra mode because handler analysis could not get past the mutation layer without symbolic execution of each handler body.
Dynamic Analysis
Dynamic devirtualization does not try to read the handlers. It executes the protected function under a tracing framework and records every instruction. Then it groups recorded instructions by the dispatched handler sequence and infers what each group does by observing its memory and register effects.
The advantage is that it works regardless of handler obfuscation. Ultra mode handlers that are completely opaque to static analysis will still produce the correct outputs when run, and those outputs can be recorded. A handler that adds two values and pushes the result will still add two values and push the result even if the addition is buried under 200 mutated instructions.
The disadvantage is coverage. Dynamic analysis only covers the paths actually executed during the trace. Branches not taken, error paths, infrequently executed code, and anything behind a conditional the input did not trigger are all missing from the result. The recovered IR is incomplete by construction.
Symbolic Execution
Symbolic execution is the middle ground. Instead of running with concrete values, it runs with symbolic ones: variables that represent unknown inputs. The execution engine tracks what operations are applied to each symbol and builds a formula. At the end of a handler, the formula on the output variable tells you what the handler computes in terms of its inputs.
This works on obfuscated handlers because it does not care how long the handler is or how many junk operations are inserted. Dead writes to scratch registers disappear because nothing reads from them. Opaque predicates collapse because the predicate evaluates to a concrete value even when the inputs are symbolic. What remains is the formula for the actual operation.
The problem is scalability. Symbolic execution of a 200-instruction mutated handler is expensive. Path explosion at branches inside the handler can make it intractable. And the formula simplification required to go from a complex expression to "this is an add" requires a good SMT solver and a library of known operation signatures to match against.
Bytecode Recovery
Before you can devirtualize, you need the decoded bytecode. This means running the stream decryption from the initial state. For a function with a lot of bytecode this can mean thousands of decrypt-then-dispatch cycles to walk through, and the initial crypto state is baked into the entry stub with its own cryptor on top.
In practice most tools handle this by executing the entry stub under a controlled environment to get the initial state, then running a software decryption loop with the same key schedule. The key schedule is derived from the dispatch loop, which itself can be obfuscated, so extracting it is non-trivial.
For multi-VM functions the bytecode walk has to detect transitions between VM instances and re-derive the opcode table and crypto state for each new VM. There is no signal in the bytecode itself that says "you are switching VMs now." The switch is detected by recognizing that execution has jumped to a different interpreter entry point.
What Actually Works in Practice
For Virtualization-mode binaries from the 3.x era, good static tools can recover most of the function. The recovered pseudocode is not identical to the original but it is semantically equivalent and readable enough to understand what the function does. This is useful for tasks like identifying license checks or locating specific logic.
For Ultra-mode binaries, fully automated devirtualization does not reliably work. The combination of obfuscated handlers and multi-VM functions creates enough complexity that existing tools either fail entirely or produce partial results with missing branches and incorrect operation inferences.
What does work for Ultra mode is targeted analysis. Instead of trying to devirtualize the whole function, you trace execution with known inputs, observe the outputs, and infer what the function computes at a higher level. You might not be able to read the assembly but you can characterize the function as a hash function, a comparison, a decryption routine, or something else based on how it behaves on known inputs.
Rebuilding a Runnable Binary
A separate and harder problem is rebuilding a working executable from a devirtualized analysis. Even if you can read what a function does, patching the binary to replace the VM stub with native code requires knowing exactly which bytes to patch and what native code to write. The entry stub occupies a fixed area. The bytecode section has a fixed size. You cannot just add code and expect the PE to work.
Tools that attempt this typically either rewrite the binary section by section using a linker-level approach, or they patch only specific targeted locations where they can prove the native replacement fits in the available space. Neither approach generalizes to arbitrary functions.
For analysis purposes it is usually more practical to work with the recovered IR directly rather than trying to produce a runnable rebuilt binary.
Where Things Stand
VMP remains one of the harder protections to deal with at scale. The combination of randomized opcode tables, encrypted bytecode streams, multi-VM functions, and Ultra mode handler mutation creates enough independent obstacles that no single tool handles all of them well. What works is a careful, targeted approach: understand the specific protection configuration, focus on the functions that matter, and use the right tool for the specific obstacle each function presents.
The series ends here. Six parts covering how VMProtect packs a binary, loads it at runtime, checks for debuggers, virtualizes code, mutates what is left, and what options exist for undoing it. It is a well-engineered system and the protection design reflects clear knowledge of how analysis tools and analysts actually work.