From PrgmrWiki
Being an account of the struggles and travails encountered by Our Hero in his
zealous quest for performance: In which there is brief confusion and a beginning.


Once upon a time, in the land of Armonk where-the-shadows-lie, a band of fiendish programmers were weaving their evil schemes. And it seemed that dark days were upon the earth at last, and for all time, for the programmers seemed so very clever that no one would ever be able to stand against them. And even if some hero were, through great fortune or unimaginable heroism, to bring one low, then there would still be an innumerable quantity remaining, each more fiendish and subtle than the last.

Wait. That’s not right at all. In fact, that’s the beginning of an entirely different book. Let’s try that again.

This book is about Xen. It’s not about Zen. It will not show you a path to enlightenment, expressed as a release from the floating world. We will not give you advice on the Eightfold Path, or enumerate the Four Noble Truths. Those are beyond our purview. But if all goes well, this book will make you happy.

Virtualization: A Brief History

In this case, the vehicle for happiness will be virtualization. It sounds bizarre, but people have attempted to become happy through virtualization since the Dawn Of Time. (In computing terms, that’s the 1970s.) IBM’s team of programmers in Armonk produced the first VM (virtual machine) solution that we know of, VM/370, to ensure that their new machine would be able to run programs developed for an older model. Customers loved it back in 1979, and the Xen developers credit it as a major inspiration. A similar, more modern example might be the Xbox 360’s software emulation of the original Xbox.

For a while, not much came of it. Virtualization continued to play a part in computing, mostly in the very top end of the market, but people continued to obstinately require a complete machine for most tasks until about 2001.

2001, everyone had to admit, looked very different from 1979.1 Computers had become small and ubiquitous. The great time-sharing machines had given way to PCs. Batch processing was a rarity, and fully interactive desktop applications had become the raison d’etre of computing. Most important, from our perspective, the single computer had been eclipsed by the network: Most computers worth having were connected to the Internet, and each of them required various services.

These services, in turn, were designed in such a way that they could be readily provided by even the cheapest and smallest server, often many times over.2 Suddenly the people operating these services had a terrible surplus of computing power, devouring electricity all out of proportion to the actual services they provided. Something had to be done. The stage was set for virtualization to re-emerge, this time as a means of server consolidation.

Some clever gentlemen at Cambridge decided that this idea could be extended even further—if virtualization allows an individual or company to consolidate their machines, they reasoned, shouldn’t it also enable multiple organizations to consolidate their machines and reap even bigger benefits? That’s the goal of Xen. It treats virtualization as a technology that allows people to ignore the hardware entirely. Computing, in this model, becomes a service or a commodity, “creating a world in which XenoServer execution platforms are scattered across the globe and available for any member of the public.” 3

The full XenoServer platform hasn’t been developed yet, and it’s unclear how it’ll do in the market if it's ever released. But in the meantime, the XenoServer project has produced and spun off Xen, the subject of this book. Xen is the virtualization piece of the XenoServer service. It fits into this grand picture by enabling sites associated with the project to create “nodes” that can be managed, transferred, and billed in ways that aren’t feasible with other computing-as-service mechanisms.

So What’s Xen Again? (And Why Should I Use It?)

Even if you’re not interested in this sort of grid computing thing, Xen offers some advantages to both the system administrator and the home user.

Xen is a piece of software that enables one machine to behave as if it were many virtual machines. Each of these machines can run its own operating system and exist almost independently of the other virtual machines running on the same hardware. Each virtual machine (an instance, or domain in Xen parlance) has its own apparent network interfaces, disks, and memory.

At first, this makes Xen seem no different from an emulator, such as VMware, Microsoft’s Virtual PC, or the open source QEMU.4 However, these traditional emulators work by running software on a simulated processor that is, itself, also software—a rather slow proposition. Xen actually runs all software directly on the processor at full speed, with only a very small overhead for some resource management tasks.

This leads to the first, and probably the most important, advantage of Xen: Xen runs fast in comparison with traditional emulators. Preliminary results in “Xen and the Art of Virtualization”—one of the seminal Xen papers— indicated performance degradation of less than 2 percent for a standard workload and between 10 and 20 percent for a worst-case scenario. Since then, Xen has improved. We usually just consider Xen’s performance to be “sufficient” and leave it at that. (Readers desiring a more precise answer might want to read Chapter 10, which discusses benchmarking Xen's performance with your particular application.)

Xen’s advantages also show up in contrast to a standalone machine, even beyond the consolidation argument mentioned earlier. Like a traditional emulator, Xen provides robust fault isolation—that is, any software problem that affects one virtual machine is unlikely to affect the real machine or other virtual machines running on the same hardware. This makes it especially useful in environments where you can’t be certain of the intentions or skill level of the users.

Also like traditional emulators, Xen provides an additional layer of abstraction between the machine and the user, allowing the administrator increased flexibility—suddenly the application can be decoupled from the hardware almost completely; stopped, started, moved around; made into a genuine service.

But Xen’s main advantage is, in a sense, psychological: It makes it possible to think of computer time as even more of a commodity than it already is.5 With Xen, you can run your own virtual computer for as much or as little time as you need, with resources tailored to the desired application.

Further, Xen gives you the ability to run whatever configuration you happen to need at a given time. For example, the web developer who wants to test a new page against different versions of Microsoft’s Internet Explorer doesn’t have to maintain a farm of Windows boxes, each with different Windows versions, different patch levels, and different versions of Internet Explorer. Instead, it’s possible to just keep different OS images on the hard drive and start them as needed.

Xen’s Limitations

All right, we’re getting carried away. Xen’s not perfect, nor is it any sort of computing panacea. It has both disadvantages and limitations.

Xen’s main disadvantage is that it only works with operating systems that have been specifically modified to support it. (But note that unmodified guest OSs are possible with sufficiently advanced hardware. We’ll talk about that later, in Chapter 12.)

Xen’s also more work to set up than a pure software emulator, requiring the user to work entirely in a guest domain (albeit a special, privileged guest domain) rather than simply starting an external emulation program as desired.

Additionally, the state of the Xen documentation is pretty dreadful. (That’s what we’re here for, you might say.) People are, of course, working on it, but everyone knows it’s more fun to write code than to document it. Also, Xen’s under such active development that much of the documentation that exists is out of date.

These are significant disadvantages, but they aren’t so bad that you should be discouraged from running Xen.

Finally, though there are also some situations in which Xen—and virtualization itself—simply isn’t useful. Xen isn’t especially useful to people with a constant, CPU-limited workload, for example. It’s not great in large server farms, where individual nodes are already scaled to their jobs. In these situations, Xen is probably not what you want, although the developers (and the open source community) are working on compelling features even for environments like these.

But, in the end, it’s not Xen itself that’s interesting—it’s what you can use it for.

So, Why Should I Use Xen?

The short answer is, because it will make your life easier. Don’t trust a piece of software? Spin off a virtual machine and see how you like it. Need to test a network app? Start up a few machines and see how well they talk to each other. Have a cluster that you want to test some new software on but can’t afford a second “test” cluster? Xen offers a solution. Want decent snapshot backups? Xen could be your answer, with its ability to pause and back up a running machine within, literally, seconds. Need to provide hosting for dozens of users, each of whom wants complete authority to mess with their configuration? Well, that’s what we do, and Xen’s the way we do it. (The astute reader might notice in our writing a certain bias toward that last application. That’s why.)

On a more fundamental level, Xen lets you take a machine, stop it, send it somewhere else, and resume it at will. It’s one less thing to think about— suddenly the hardware is no longer important. A good thing for both users and administrators!

Finally, there’s one last good reason to run Xen, one that’s so big and mundane it often gets ignored: Xen is simply cheaper than running multiple boxes. CPU usage in data centers ranges between 5 percent and 40 percent—a fairly unimpressive figure.6 Xen lets you put some of those unused cycles to use, without sacrificing reliability, performance, or scalability.

Unlike the virtualization technologies of a few decades ago, Xen virtualizes cheap commodity hardware; this might not make sense at first, until you realize that much of the market is very price sensitive, and power is becoming quite expensive. It’s much cheaper to run one big dual quad-core rig than it is to run eight single-core boxes, and with Xen, you can easily split that quadcore system into individual systems.

Overview of the Book

All right, enough hype. Now for nuts and bolts.

We’ve organized this book (mostly) alternating between theoretical and practical discussion. In our experience, an admin needs both practical experience and a firm theoretical grounding to effectively solve problems, and that’s what we aim to provide.

Chapter 1 is an overview of Xen and virtualization technologies in general. We try to outline how Xen works, what distinguishes it from other virtualization packages, and why you might (or might not) want to use it. This one is theory-intensive.

Chapter 2 is a step-by-step quick start based on the rationale that there’s no substitute for experience. We install Xen from base principles on a CentOS system.

Chapter 3 describes manually creating virtual machine images to use with Xen.

Chapter 4 covers storage. It sounds kind of mundane, but storage is actually a vital part of virtualization—if storage is tied to a particular machine or hardware configuration, then many of Xen’s coolest features won’t work. We cover various storage options, laying the groundwork for subsequent discussion of migration and snapshots.

We talk about networking in Chapter 5—how to set it up and what options you have when doing so. Both this chapter and the previous focus a bit more on theory.

Chapter 6 is about a couple of popular packaged frontends that can be used with the open source Xen hypervisor to automate the routine drudgery of VM administration. We also talk about scripting Xen, if you'd rather build your own frontend.

Chapter 7 goes back to the practical case studies to talk about Xen for shared hosting. It’s one of the big applications that’s driving early adoption of Xen, and we’ve got a lot of experience doing it.

Moving on from shared hosting, in Chapter 8 we discuss possible alternatives to Linux, both as a “host” and “guest” OS.

In Chapter 9 we describe migration, both in theory and practice.

Chapter 10 is about performance analysis with Xen. We discuss Xen’s robust support in this area, which doesn’t seem to get mentioned nearly as often as it deserves.

With Chapter 11 we diverge a bit to cover the commercial product that XenSource (now a division of Citrix) has built around Xen.

Chapter 12 is about Xen’s HVM support—that is to say, the hardware virtualization supported by Intel’s and AMD’s newest processors.

Chapter 13 covers Windows. We talk about using it with Xen, making it play nicely, how you can expect to access it, and what you might expect to do with it once you’ve got it working.

[[Chapter 14: Tips|Chapter 14] is a collection of extremely practical tips for Xen admins.

Chapter 15 is a troubleshooting chapter—a collection of problems that we’ve run into and how we’ve solved them.

We’ve also included appendixes on Xen’s domain configuration files and xm’s syntax.

But I Am Impatient!

If you’re really impatient to get started with Xen, skip to Chapter 2 and follow our step-by-step instructions. Then skim the rest of the book as the fancy strikes you.

If you’re planning to deploy Xen as a service provider, we suggest following the steps in Chapter 2, then reading Chapters 3, 4, 5, 6, 7, 8, and probably 11, and finally just reading the whole book.

For those of you who are contemplating a large deployment of Xen, you’ll probably be most interested in Chapters 3, 4, 5, 6, and 7, with an excursion to 13 to consider the commercial XenSource product. But again, we think we’ve put useful information throughout the book.

NOTE: We’ve tried to keep this book as distribution- and version-independent as possible, except in the tutorial sections, where we try to be extremely specific and detailed, and in the distro-specific notes, which are necessarily, er, distro-specific.

Often we will get carried away and make some ridiculous broad generalization, like “only an idiot would use Linux as an NFS server.”7 Where reasonable, we’ve tried to add footnotes that qualify and temper the occasionally strident claims we make.

1 And, to our great dismay, also very different from the movie.
2 We know, there are many applications where this is not the case—but there are still a lot of small web servers (for example) out there.
3 Hand et al., “Controlling the XenoServer Open Platform,” (University of Cambridge, England, 2003). Abstract.
4 In fact, Xen uses QEMU extensively, as we’ll see.
5 This is sort of like cell phones. People use them, not as a substitute for landlines, but as a substitute for traditional planning.
6 This is a generally held belief, but one oft-cited source is the presentation “Virtualization: Taking Charge of Your Servers” by Thomas Bittman.
7 Actually, we’ve seen morons and imbeciles do this too.