Polymorphic Code: How It Works and How Analysts Catch It
Polymorphic code is code that changes its own appearance every time it runs or spreads while doing exactly the same thing. The idea is old: the first known polymorphic virus, 1260, was written in 1990. MITRE ATT&CK now tracks the technique as sub-technique T1027.014, Polymorphic Code, under Obfuscated Files or Information.
About Devirt: Devirt recovers the code that protectors such as VMProtect and Themida hide inside their own virtual machines, for files you own or are authorized to analyze. You get an IDA database and annotated pseudocode, with a ledger showing which functions were proven. If nothing is proven, you pay nothing. Start with a free assessment.
For defenders, polymorphic code is mostly a problem for one kind of detection: matching known byte patterns. This post explains how polymorphic code works, what an example looks like, how it differs from metamorphic and VM-protected code, and how analysts and detection tools see through it.
What is polymorphic code?
Polymorphic code is code that rewrites its own form for each copy or each execution while keeping its behavior identical. The usual design encrypts the main body of the code and pairs it with a small decryption routine. The body stays the same underneath, but the decryptor is regenerated with different instructions for every copy, so no two copies share the same bytes.
“Polymorphic” simply means “many forms”. In object-oriented programming the word means something unrelated (one interface, many implementations). In security it always refers to self-changing code.
How does polymorphic code work?
Polymorphic code works in two parts: an encrypted body and a decryptor that changes. As the Wikipedia article on polymorphic code puts it, encryption alone is not polymorphism. A fixed decryptor would be one stable pattern to detect. Polymorphism comes from mutating the encryptor and decryptor pair with each copy.
The component that produces those variations is called a polymorphic engine or mutation engine. It generates decryptors that are equivalent in effect but different in form: different registers, different instruction choices, different order, and filler instructions that do nothing. When a copy runs, its decryptor restores the same body in memory and hands over control.
What is an example of polymorphic code?
An example of polymorphic code is a program whose body is stored encrypted and whose small decryptor is regenerated with different but equivalent instructions in each copy. One copy might add a value to a register, another subtract its negative, a third insert do-nothing instructions in between. Every copy decrypts to the same body and behaves identically, yet a byte-pattern signature written for one copy misses the next.
The best-known historical examples are viruses. After 1260 in 1990, the virus writer known as Dark Avenger built a mutation engine (MtE) into his viruses in 1992 so that antivirus programs couldn’t recognise their signatures. The file infector Virut is another common example. MITRE lists the malware BendyBear as a procedure example that changes its runtime footprint during execution to evade signature-based defenses.
Why does signature-based detection miss polymorphic code?
Signature-based detection misses polymorphic code because signatures match fixed byte patterns, and polymorphic code removes them. Each copy has a different decryptor and a differently encrypted body, so a file hash or a byte string taken from one copy matches nothing in the next. A detection rule has to target something that does not change, and in the file on disk almost nothing does.
That is why MITRE’s mitigation for T1027.014 points to anti-malware that uses machine learning and behavior-based detection, rather than relying on indicator-based matching alone.
How is polymorphic code different from metamorphic code?
Polymorphic code and metamorphic code differ in what stays constant. Polymorphic code changes its decryptor but decrypts to the same body every time, so running it in an emulator or sandbox exposes a constant payload. Metamorphic code rewrites its entire body for each copy and has no constant decrypted form, which makes it harder to catch even after it runs. MITRE ATT&CK groups both terms under T1027.014, but analysts treat them as different problems.
Get Devirt’s stories in your inbox
Join Medium for free to get updates from this writer.
A third case is often confused with both: VM-protected code. Code virtualization, offered by protectors such as VMProtect and Themida, converts selected functions into bytecode for a private virtual machine. The bytecode can be identical in every copy, so it isn’t polymorphic. But it never turns back into native instructions, so a sandbox dump shows the virtual machine’s interpreter rather than the logic.
Put side by side:
- Polymorphic: the decryptor changes, the body is constant once decrypted. Analysts let it decrypt and read the body.
- Metamorphic: the whole body changes, with no constant form. Analysts rely on behavior and on normalizing the code.
- VM-protected: the bytecode may be constant, but it never becomes native code. Analysts need to trace the virtual machine or translate the bytecode back, which is called devirtualization.
The comparison of devirtualization, deobfuscation and unpacking covers why that third case needs its own tools.
How do analysts detect polymorphic code?
Analysts detect polymorphic code by looking at behavior and decrypted content rather than raw bytes. Running the sample in an emulator or sandbox lets it decrypt itself so the constant body can be scanned. MITRE ATT&CK’s detection strategy for T1027.014 looks for executables whose hash, entropy or memory sections change during or between executions.
Defenders usually combine all three of these:
- Emulation and sandboxing let the polymorphic layer do its work, then inspect what it produced.
- Behavioral detection keys on what the code does, which polymorphism doesn’t change.
- Detections written on the decrypted body or on behavior, not on the file’s bytes, hold up across copies.
Reverse engineers treat the decryptor as noise. The interesting analysis starts once the body is in memory, and that body is the same in every copy.
Is polymorphic code ever legitimate?
Polymorphic techniques have legitimate uses in software protection and security testing. Commercial protectors apply related transformations, such as code mutation, to make licensed software harder to tamper with. Security teams also generate variants of known samples to check whether a detection product relies too heavily on signatures. The same techniques are used by malware, which is why they are tracked in MITRE ATT&CK.
Further reading
- Devirtualization vs deobfuscation vs unpacking: three kinds of protection and what removes each
- Devirtualization methods: how VM-protected code is recovered
Written by Devirt, an automated devirtualization service for authorized analysis of protected software.
