Tracing Hardware Virtual Machines

Traditionally, understanding what is going on inside of a Hardware Virtual Machine has been a black box. With DTrace you can begin to open up that black box and understand what is going on. This chapter focuses on debugging virtual machines that utilize extensions to the x86 instruction set architecture and assumes some familiarity with hardware virtual machine architecture.

42.1. Virtualization Background

In this chapter various terms have specific meanings that they don't in other contexts. These terms are:

42.2. vmregs[] Array

The vmregs[] array enables you to access the registers of the currently active VMCS. These registers allow you to see the guest virtual machine state. The following tables list a subset of the indices into the vmregs[] array corresponding to each of the supported CPU architecture extensions. The full list is available in /usr/lib/dtrace/regs.d.

Intel VMX vmregs[] Constants

Constant

Register

VMX_GUEST_CR0

%cr0

VMX_GUEST_CR3

%cr3

VMX_GUEST_CR4

%cr4

VMX_GUEST_RSP

%rsp

VMX_GUEST_RIP

%rip

VMX_GUEST_RFLAGS

%rflags

VMX_GUEST_CS_BASE

%es base

VMX_GUEST_DS_BASE

%es base

VMX_GUEST_ES_BASE

%es base

VMX_GUEST_FS_BASE

%es base

VMX_GUEST_GS_BASE

%es base

The vmregs[] can be combined with other providers to provide insight into what a guest virtual machine without having to access it. The following script profiles what is running on a per-guest basis using the value of %cr3. On x86 systems, that register contains the root of the page table and thus is unique on a per-process basis.


profile-97hz
/execname == "qemu-kvm"/
{
	@[pid, vmregs[VMX_GUEST_CR3]] = count();
}

tick-10s
{
	printa("%05d 0x%x %@d\n", @);
	exit(0);
}

Running this yields the following output:

65197 0xffffff02d5ed3000 1
65197 0xffffff02d5ee7000 2
65197 0x77bda000 896

Looking at output like this indicate to us that we primarily had one process that was accessing running on CPU inside of the guest during the ten second windows.