Malware anti-VM technics

Malware analysis usually involved the use of virtual environment (VM) such as VMware, VirtualBox and plenty of other virtualisation solutions. Mentioning the main virtualisation product is great but such products are also used in sandbox and other testing environment such as Virustotal, Anubis, etc. There is a lot of reason for using a virtual environment for such analysis. In particular it give the ability to run malicious code in a control manner. You can customize your VM to meet your needs, install vulnerable software, change configuration, etc. Not to mention the ability to start from scratch and restore a previous snapshot. You can do it the “old” way by running the malicious executable directly on your operating system but you will take a little more risk not to mention the time you will lose to restore your system.

On the other side, the creator of a viruses don’t really like the researchers and other people who try to understand how their viruses are working. The more a virus is “in the shadows” the more damage it will do (or the more money it will generate…). So we can easily guess that it will be in the interest of the virus owner to be able to detect a VM environment and change the virus behavior to avoid detection. [Please read the conclusion as well].

Guess what? This is exactly what virus are doing. Not all of them, but there is definitely a number of sophisticated threat that detect their environment. Here are some references:

So the key questions is: what is the difference between a VM environment and normal environment? There is a few elements that must be unique in a computer to work properly. Using a VM will then make this statement a bit more difficult, as we are running an operating system on an operating system the VM software must emulate some part of the OS (hardware and software), use specific driver, etc. Best to identify this? Have a unique signature or artefact.

Virtualisation solution usually don’t have…real hardware. However such software implement a “virtual” version of the hardware. This imply some specific drivers and other references. One example is the MAC address. Despite being a virtual network interface it must have a unique MAC address. Therefore, and as we know that the network interface have a unique identifier, we can look for the Organizationally Unique Identifier (OUI – registered by VMware. Following a quick check on the IEEE website ( ) we obtained the following list:

  • 00:05:69
  • 00:0C:29
  • 00:1C:14
  • 00:50:56

So one simple check will be to get the MAC address, if the OUI is matching one of the above we are running a VMware software.

It’s the same for the hard drive driver, the video driver or the mouse driver. All of those have a specific registry key or a file that can be easily use for identification.

Descriptor Table Registers – aka Red Pill test
Another thing which is unique to a computer (well a processor in this case) is the Interrupt Descriptor Table Register (IDTR) ( As mentioned above virtualisation required to run an operating system on an operating system. Thus this will generate a situation where there two processors, thus two IDTR and is going to…crash due to a conflict. So to resolve this the virtualisation software will move the IDTR to a different memory location. This then generate another way to check if the environment is a “physical” one or a “virtual” one. Determine this is not difficult, there is an x86 instructions, sidt, than can be used to read the location of the IDTR table.

This apply as well for the Global Descriptor Table (GDT) and the Local Descriptor Table (LDT) along with the respective x86 instructions sgdt and sldt.

For those of you who are fan of the Matrix movies (, the test that use the IDTR address is known as: Red Pill. If you remember in the movie, Neo has to choose between a blue and a red pill. The blue one will let him in the fake reality, the red one will bring him in the real reality. So same thing here, the Red Pill test will allow the malware to find out if its running in the matrix (VM) or not.
[Blue Pill is also the name of a rootkit that use x86 virtualization ]

A Red Pill test can be written as follow :

int swallow_redpill () {
unsigned char m[2+4], rpill[] = "x0fx01x0dx00x00x00x00xc3";
*((unsigned*)&rpill[3]) = (unsigned)m;
return (m[5]>0xd0) ? 1 : 0;

Backdoor I/O port
VMware use a virtual I/O ports for communication between the guest operating system and host operating system. This allow things like copy-paste between host and guest systems, etc. Having such particular port open is definitely a very good sign the we are using a VM environment. So we should find a way to check if this port is used.

So how this virtual I/O port is working? It use the in (0xED) instruction. This instruction copies data from the I/O port specified by the source operand to a memory location specified by the destination operand. VMware actually monitored the use of the in instruction and capture the I/O destined for the communication channel port 0x5668 (VX). Then the second operand needs to be loaded with VX in order to check for VMware, which happens only when the EAX register is loaded with the magic number 0x564D5868 (VMXh). The ECX register is then loaded with a value corresponding to the action you wanted to perform. For example the value 0xA will output the VMware version type and 0x14 will get the memory size.

All this leave a pretty good number of artefact that can be identified during reverse engineering or debugging.

How to fix it?
Most of the above can be prevented by changing the settings of VMware. Here is a few things you can do:

  • You can patch the code yourself. If you debug the malware and identify some of the specific instructions (e.g. sidt, sgdt, sldt) you can replace the code with NOPs to prevent it.
  • Use a multiprocessor machine as the Red Pill test will work only on a single processor machine. Multiple processor mean multiple IDTR (one per processor) and therefor the result of the sidt instruction can vary and the signature use by Red Pill can be unreliable.
  • Don’t install the VMware tool as it will create a lot of artefact (e.g. registry key, file, folder, etc).
  • Change the configuration of your virtual machine by adding the following options to your .vmx file: = "TRUE" = "TRUE" = "TRUE" = "TRUE"
monitor_control.disable_directexec = "TRUE"
monitor_control.disable_chksimd = "TRUE"
monitor_control.disable_ntreloc = "TRUE"
monitor_control.disable_selfmod = "TRUE"
monitor_control.disable_reloc = "TRUE"
monitor_control.disable_btinout = "TRUE"
monitor_control.disable_btmemspace = "TRUE"
monitor_control.disable_btpriv = "TRUE"
monitor_control.disable_btseg = "TRUE"

The above is only three of the main way it can be possible to detect if an executable is running in a VM environment.

But in the end is it that useful for a malware to check for the existence of a VM environment? Most of the infrastructure of companies are moving to virtualized environment due to cost reduction and other optimization. Add a VM check in your malware and it won’t run in most of the company infrastructure! Not to mention about the increasing use of the cloud.

Also using one of the above technics will be a clear sign that the executable is doing malicious things! There only a few (if not none) non-system software that will use the above test. It will therefore give a clear sign about the executable intention. thus gaining some interest from the security community and will end-up under the microscope of most of the security researcher…maybe not exactly what the owner of the malware is expecting…


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s