Virtualization and OpenSolaris

There’s actually a decent piece over at eWeek discussing the future of Xen and LAE (the project formerly known as Janus) on OpenSolaris. Now that our marketing folks are getting the right message out there about what we’re trying to accomplish, I thought I’d follow up with a little technical background on virtualization and why we’re investing in these different technologies. Keep in mind that these are my personal beliefs based on interactions with customers and other Solaris engineers. Any resemblance to a corporate strategy is purely coincidental 😉

Before diving in, I should point out that this will be a rather broad coverage of virtualization strategies. For a more detailed comparison of Zones and Jails in particular, check out James Dickens’ Zones comparison chart.

Benefits of Virtualization

First off, virtualization is here to stay. Our customers need virtualization – it dramatically reduces the cost of deploying and maintaining multiple machines and applications. The success of companies such as VMWare is proof enough that such a market exists, though we have been hearing it from our customers for a long time. What we find, however, is that customers are often confused about exactly what they’re trying to accomplish, and companies try to pitch a single solution to virtualization problems without recognizing that more appropriate solutions may exist. The most common need for virtualization (as judged by our customer base) is application consolidation. Many of the larger apps have become so complex that they become a system in themselves – and often they don’t play nicely with other applications on the box. So “one app per machine” has become the common paradigm. The second most common need is security, either for your application administrators or your developers. Other reasons certainly exist (rapid test environment deployment, distributed system simulation, etc), but these are the two primary ones.

So what does virtualization buy you? It’s all about reducing costs, but there are really two types of cost associated with running a system:

1. Hardware costs – This includes the cost of the machine, but also the costs associated with running that machine (power, A/C).
2. Software management costs – This includes the cost of deploying new machines, and upgrading/patching software, and observing software behavior.

As we’ll see, different virtualization strategies provide different qualities of the above savings.

Hardware virtualization

One of the most well-established forms of virtualization, the most common examples today are Sun Domains and IBM Logical Partitions. In each case, the hardware is responsible for dividing existing resources in such a way as to present multiple machines to the user. This has the advantage of requiring no software layer, no performance impact, and hardware fault isolation. The downside to this is that it requires specialized hardware that is extremely expensive, and provides zero benefit for reducing software management costs.

Software machine virtualization

This approach is probably the one most commonly associated with the term
“virtualization”. In this scheme, a software layer is created which allows
multiple OS instances to run on the same hardware. The most commercialized
versions are VMware and Virtual PC,
but other projects exist (such as qemu and PearPC). Typically, they require a
“host” operating system as well as multiple “guests” (although VMware ESX server
runs a custom kernel as the host). While Xen uses a
paravitualization technique that requires changes to the guest OS, it is still
fundamentally a machine virtualization technique. And Usermode Linux takes a
radically different approach, but accomplishes the basic same task.

In the end, this approach has similar strengths and weaknesses as the hardware assisted
virtualization. You don’t have to buy expensive special-purpose hardware, but
you give up the hardware fault isolation and often sacrifice performance (Xen’s
approach lessens this impact, but its still visible). But most importantly, you
still don’t save any costs associated with software management – administering
software on 10 virtual machines is just as expensive as administering 10
separate machines. And you have no visibility into what’s happening within the
virtual machine – you may be able to tell that Xen is consuming 50% of your CPU,
but you can’t tell why unless you log into the virtual system itself.

Software application virtualization

On the grand scale of virtualization, this ranks as the “least virtualized”.
With this approach, the operating system uses various tricks and techniques to
present an alternate view of the machine. This can range from simple
chroot(1), to BSD
Jails, to Solaris
Zones. Each of these provide a more complete OS view with varying degrees
of isolation. While Zones is the most complete and the most secure, they all
use the same fundamental idea of a single operating system presenting an
“alternate reality” that appears to be a complete system at the application
level. The upcoming Linux Application Environment on OpenSolaris will take this
approach by leveraging Zones and emulating Linux at the system call layer.

The most significant downside to this approach is the fact there is a single kernel. You cannot run different operating systems (though LAE will add an interesting twist), and the “guest” environments have limited access to hardware facilities. On the other hand, this approach results in huge savings on the software management front. Because applications are still processes within the host environment, you have total visibility into what is happening within each guest, using standard operating system tools, as well as manage them as you would any other processes, using standard resource management tools. You can deploy, patch, and upgrade software from a single point without having to physically log into each machine. While not all applications will run in such a reduced environment, those that do will be able to benefit from vastly simplified software management. This approach also has the added bonus that it tends to make better use of shared resources. In Zones, for example, the most common configuration includes a shared /usr directory, so that no additional disk space is needed (and only one copy of each library needs to be resident in memory).

OpenSolaris virtualization in the future

So what does this all mean for OpenSolaris? Why are we continuing to pursue Zones, LAE, and Xen? The short answer is because “our customers want us to.” And hopefully, from what’s been said above, it’s obvious that there is no one virtualization strategy that is correct for everyone. If you want to consolidate servers running a variety of different operating systems (including older versions of Solaris), then Xen is probably the right approach. If you want to consolidate machines running Solaris applications, then Zones is probably your best bet. If you require the ability to survive hardware faults between virtual machines, then domains is the only choice. If you want to take advantage of Solaris FMA and performance, but still want to run the latest and greatest from RedHat with support, then Xen is your option. If you have 90% of your applications on Solaris, and you’re just missing that one last app, then LAE is for you. Similarly, if you have a Linux app that you want to debug with DTrace, you can leverage LAE without having to port to Solaris first.

With respect to Linux virtualization in particular, we are always going to pursue ISV certification first. No one at Sun wants you to run Oracle under LAE or Xen. Given the choice, we will always aggressively pursue ISVs to do a native port to Solaris. But we understand that there is an entire ecosystem of applications (typically in-house apps) that just won’t run on Solaris x86. We want users to have a choice between virtualization options, and we want all those options to be a fundamental part of the operating system.

I hope that helps clear up the grand strategy. There will always be people who disagree with this vision, but we honestly believe we’re making the best choices for our customers.

Tags: OpenSolaris
Zones

You may note, that I failed to mention cross-architecture virtualization. This is most common at the system level (like PearPC), but application-level solutions do exist (including Apple’s upcoming Rosetta). This type of virtualization simply doesn’t factor into our plans, yet, and still falls under the umbrella of one of the broad virtualization types.

I also apologize for any virtualization projects out there that I missed. There are undoubtedly many more, but the ones mentioned above serve to illustrate my point.