October 2008 October 2008
Virtualization: An Introduction to Hyper-V in Windows Server 2008
The introduction of Hyper-V makes virtualization an even more compelling solution for IT environments. Get an overview of today’s virtualization market and see how Hyper-V improves the manageability, reliability, and security of virtualization Rajiv Arunkundram
Virtualization: Manage Your Virtual Environments with VMM 2008
System Center Virtual Machine Manager provides a consolidated interface for managing your virtual infrastructure. The latest version adds support for Windows Server 2008 Hyper-V, as well as for VMware virtual machines. Explore the new features and get an overview of using VMM to centralize your management tasks. Edwin Yuen
Virtualization: Getting Started with Microsoft Application Virtualization
Microsoft Application Virtualization (App-V) allows you to deliver virtualized desktops to client systems throughout your organization. This simplifies system management and liberates employees from their desktops. Take a close look at how App-V works and discover how you can deploy it in your organization. Anthony Kinney
Virtualization: Achieving High Availability for Hyper-V
Consolidating servers onto fewer physical machines has many advantages, but it is extremely important that you plan for your systems to be highly available. Here’s a guide to using Windows Server 2008 Failover Clustering to bring high availability to your Hyper-V virtual machines. Steven Ekren
Virtualization: Backup and Disaster Recovery for Server Virtualization
Virtualization brings significant changes to disaster recovery. Here’s an introduction to how the Microsoft virtualization platform factors into your disaster recovery plan, as well as a deeper look into backup and restore options and considerations for Windows Server 2008 Hyper-V. Adam Fazio
Virtualization: Essential Tools for Planning Your Virtual Infrastructure
Is your infrastructure ready for virtualization? The Microsoft Assessment and Planning Toolkit, a network-wide infrastructure assessment tool, can help you better understand your IT infrastructure and determine whether your systems are ready for upgrade or migration to a variety of technologies, including virtualization. Jay Sauls and Baldwin Ng
Columns
From the Editor: It’s a Virtual(ized) World: | Letters: Readers Speak Out: | Toolbox: New Products for IT Pros:
SQL Q&A: Large Transaction Logs, When to Use Repair, and More: In this installment, Paul Randal answers questions about backing up and restoring, looks at the differences between log shipping and database mirroring, and explains why the Repair function should only be used as a last resort.Paul S. Randal | Utility Spotlight: Offline Virtual Machine Servicing Tool: Virtual machines that are stored offline don’t automatically receive the necessary updates to keep them safe and compliant. This, in turn, can pose a risk to your entire IT environment. Find out how the free Offline Virtual Machine Servicing Tool lets you automate the process of updating virtual machines.Peter Skjøtt Larsen and Suveen Kumar Reddy Vuppala | Windows PowerShell: The Power of Profiles: Ever wish Windows PowerShell would launch with a work environment tailored to your needs? Don Jones demonstrates how you can use profiles to customize the Windows PowerShell shell.Don Jones
Hey, Scripting Guy!: Famous Last Words: The Scripting Guys discuss Socrates and revisit the topic of querying an XML file . This time, however, the XML file is structured so that rather than using child nodes, additional property values are configured as attributes.The Microsoft Scripting Guys | The Desktop Files: Customizing Windows Deployment Services: Wes Miller delves into Windows Deployment Services, showing you how you can customize and configure WDS to meet the needs of your organization.Wes Miller | Security Watch: Revisiting the 10 Immutable Laws of Security, Part 1: It's been about 8 years since "The 10 Immutable Laws of Security" were first published, and a lot has changed since then. This month, Jesper Johansson kicks off a three-part series in which he analyzes the laws from today's perspective to see if they still hold true.Jesper M. Johansson
Field Notes: High-Capacity Color Bar Code: Gavin Jancke has developed a new bar code, using colors and triangles, that has much higher data capacity than traditional black and white bar codes. Take a closer look at these high-capacity color barcodes.Gavin Jancke | Windows Confidential: Work Harder, Not Smarter: Raymond Chen looks at the skewed relationship bugs have to errors, and explains why it's important that programmers suffer as well as give results.Raymond Chen


Virtualization
An Introduction to Hyper-V in Windows Server 2008
Rajiv Arunkundram
At a Glance:
  • What is virtualization?
  • Three virtualization architectures
  • Microkernelized vs. monolithic hypervisor
  • What Hyper-V does

Contents
The Server Virtualization Market
How Server Virtualization Works
Types of Virtualization Solutions
Windows Hypervisor
The Parent Partition
Device-Sharing Architecture
Integration Components
The Hyper-V Feature Set
Scalability
High Availability
Security
Manageability
Wrapping Up
There has been quite a lot of talk about virtualization recently, and most of the discussion is specifically about server virtualization. This is one of the most exciting trends in the industry and one that has the potential, over the next few years, to change the paradigm of how IT systems are deployed. But server virtualization will not only change how IT administrators and architects think about servers and system utilization, it is also going to affect the processes and tools used to manage what will certainly become an increasingly dynamic environment.
Virtualization has actually been around for some time now, but the technology is still evolving. In fact, the word itself still means different things to different people. In broad terms, however, virtualization is about abstracting one layer of the technology stack from the next layer, like storage from servers or the OS from the applications. Abstracting the different layers, in turn, enables consolidation and better manageability.
As a concept, virtualization applies to storage, networks, servers, applications, and access. When you look at storage and networks, the goal of virtualization is to aggregate a set of different devices so the total pool of resources looks and acts like a single entity. For example, you can configure a 40TB storage solution instead of a set of 20 2TB storage devices. But with other components, virtualization acts in the opposite direction, helping you to make a single system appear as though there are multiple systems. The most common example of this is server virtualization, where you host multiple OS instances and environments on a single server.
Microsoft has approached virtualization at several different levels, extending from the desktop to the datacenter with solutions for server virtualization, application virtualization, presentation virtualization, and desktop virtualization. The common thread across all of these is the management piece with Microsoft System Center. For this article, I am focusing on the server virtualization component and specifically on how Hyper-V, a key feature of Windows Server 2008, fits into the equation for a dynamic datacenter.

The Server Virtualization Market
First, I think it would be worthwhile to look at what exists in today's environment and where the overall market is headed. Depending on what research you read, some analysts estimate that 5–9 percent of all physical servers currently sold are being used as virtualization hosts. You might consider this to be a big chunk of systems in a market where more than nine million physical servers are shipped every year. But one thing is certain: there is still a huge market opportunity as more customers become comfortable with virtualization and want to employ it.
It is important to note where virtualization is being adopted. Enterprise customers have certainly led the charge with testing and being early adopters. However, there are small- and medium-sized businesses also deploying virtualization. The adoption of virtualization reaches across different types of workloads, from business applications and management to the Web and e-mail.
So why is virtualization now all the buzz? There are a few factors, not the least of which is timing. A few key industry factors have come together at the same time, helping to push for increased adoption of virtualization. These industry factors include the move to 64-bit computing, multicore processors, and even the drive of sustainable computing to improve system utilization.
Systems are becoming much bigger and they require a technology like virtualization to make full use of system power. But while it is true that core technology (and Moore's law) has had a steady lead in terms of producing more processing capacity than systems can use, we are now also more conscious of the environmental impact, power requirements, and cooling costs.
All of these factors, plus the easy justification on the return on investment (ROI) of adopting virtualization, should together accelerate the adoption of virtualization across both large and small businesses. And we, the IT professionals, can expect all the major players to continue to invest in this technology over the next few years and improve features and functionality.

How Server Virtualization Works
Server virtualization in general terms lets you take a single physical device and install (and run simultaneously) two or more OS environments that are potentially different and have different identities, application stacks, and so on. Hyper-V is a next-generation, 64-bit hypervisor-based virtualization technology that offers reliable and scalable platform capabilities. Together with System Center, it offers a single set of integrated management tools for both physical and virtual resources.
All of this works to reduce costs, improve utilization, optimize infrastructure, and allow businesses to rapidly provision new servers. In order to help you better understand how Hyper-V is architected, I want to first take a look at the different types of virtualization solutions.

Types of Virtualization Solutions
There are essentially three general architectures used for server virtualization, as shown in Figure 1. The fundamental differences have to do with the relationship between the virtualization layer and the physical hardware. By virtualization layer, I mean the layer of software called the virtual machine monitor (VMM, not to be confused with Virtual Machine Manager). It is this layer that provides the ability to create multiple isolated instances that share the same underlying hardware resources.
Figure 1 The three architectures of virtualization
The Type-2 VMM architecture is exemplified by Java Virtual Machines. Here, the goal of virtualization is to create a runtime environment within which the process can execute a set of instructions without relying on the host system. In this case, the isolation is for the different processes, and it allows a single application to run on different OSs without having to worry about OS dependencies. Server virtualization does not fall into this category.
Type-1 VMM and Hybrid VMMs are the two approaches you are most likely to find in wide use today. The Hybrid VMM is a stage where the VMM runs alongside the host OS and helps to create virtual machines on top. Examples of the Hybrid VMM are Microsoft Virtual Server, Microsoft Virtual PC, VMware Workstation, and VMware Player. You should note that while these types of solutions are excellent for a client scenario where you are only running virtual machines part of the time, the VMMs add considerable overhead and therefore are not suitable for resource-intensive workloads.
In a Type-1 VMM architecture, the VMM layer runs directly on top of the hardware. This is often called the hypervisor layer. This architecture was originally designed in the 1960s by IBM for mainframe systems and has recently been made available on the x86/x64 platforms with a variety of solutions, including Windows Server 2008 Hyper-V.
There are solutions available on which hypervisor is an embedded part of the firmware. This, however, is simply a packaging option and does not really change the underlying technology.
As you look at Type-1 VMMs, there are essentially two main ways to architect the hypervisor solutions: microkernelized and monolithic. Both of these approaches, as shown in Figure 2, are true Type-1 VMMs that have the hypervisor installed directly on the physical hardware.
Figure 2 The two ways to architect hypervisor solutions
The monolithic hypervisor approach hosts the hypervisor/VMM in a single layer that also includes most of the required components, such as the kernel, device drivers, and the I/O stack. This is the approach used by such solutions as VMware ESX and traditional mainframe systems.
The microkernelized approach uses a very thin, specialized hypervisor that only performs the core tasks of ensuring partition isolation and memory management. This layer does not include the I/O stack or device drivers. This is the approach used by Hyper-V. In this architecture, the virtualization stack and hardware-specific device drivers are located in a specialized partition called the parent partition.

Windows Hypervisor
Ensuring that there is strong separation between multiple OSs is done by creating virtual processors, memory, timers, and interrupt controllers. OSs use these virtual resources just as they would use their physical counterparts.
The Windows hypervisor, part of Hyper-V, performs the following tasks:
  • Creates logical partitions.
  • Manages memory and processor scheduling for guest OSs.
  • Provides mechanisms in order to virtualize input/output and communicate among partitions.
  • Enforces memory access rules.
  • Enforces policy for CPU usage.
  • Exposes a simple programmatic interface known as hypercalls.
Since it uses the microkernelized approach, the Windows hypervisor is fairly small—less than 1MB in size. This minimal footprint helps improve the overall security of the system.
One of the requirements for running Hyper-V is that you have an x64 system that has Intel VT or AMD-V technologies. x64 technology enables access to a larger address space and support for systems with more memory, and thus allows more virtual machines on a single host system. Intel VT and AMD-V are hardware-assisted virtualization solutions that provide an ultra-privileged layer in the ring architecture that helps to keep the execution environment of the hypervisor separate from the rest of the system. They also allow Hyper-V to run an unmodified guest OS without incurring significant emulation performance penalties.

The Parent Partition
Hyper-V consists of one parent partition, which is essentially a virtual machine that has special or privileged access. This is the only virtual machine with direct access to hardware resources. All of the other virtual machines, which are known as Guest partitions, go through the parent partition for their device access.
The existence of the parent partition is fairly transparent. When you begin to install Hyper-V, the first thing you must do is install Windows Server 2008 x64 Edition on the physical system. You then need to go to Server Manager, enable the Hyper-V role, and restart the system. Once the system has restarted, Windows hypervisor is loaded first, and then the rest of the stack is converted to be the parent partition.
The parent partition has ownership of the keyboard, mouse, video display, and other devices attached to the host server. It does not have direct control over the timers and interrupt controllers that the hypervisor uses.
The parent partition contains a Windows Management Instrumentation (WMI) provider to facilitate management of all aspects of the virtualized environment, as well as a Virtualization Stack that performs hardware-related tasks on behalf of the child partitions. In addition, any Independent Hardware Vendor (IHV) drivers needed for host system hardware are contained in the parent partition, and any drivers created for Windows Server 2008 x64 editions will also work in the parent partition.

Device-Sharing Architecture
One of the innovative architectural components in Hyper-V is the new device-sharing architecture that supports emulated and synthetic devices in each guest OS. Device emulation is quite useful for supporting older OSs with device drivers designed for older generations of hardware. For example, Hyper-V includes an emulation of the Intel 21140 network adapter, which was called the DEC 21140 network adapter at the time many older OSs were being shipped.
Generally, device emulation is slow, not easily extendable, and doesn't scale well. But emulation is still important because it allows you to run most x86 OSs on Hyper-V. Since virtualization is now moving from a niche technology primarily for testing and development to an essential technology for production environments, users require better performance in order to run larger workloads. Emulated devices no longer meet these growing demands.
An alternative solution to this is to use Hyper-V synthetic devices. Synthetic devices are virtual devices that are mapped directly to physical devices. Unlike emulated devices, synthetic devices do not emulate legacy hardware. With the Hyper-V hardware sharing model, supported guest OSs can interact directly with synthetic devices that may have no physical counterparts. These OSs use Virtual Service Clients (VSCs), which act as device drivers within the guest OS.
Instead of accessing physical hardware directly, VSCs use the VMBus, which is a high-speed, in-memory bus, in order to access Virtual Service Providers (VSPs) in the parent partition. The parent partition VSPs then manage access to the underlying physical hardware, as illustrated in Figure 3. A key benefit of synthetic devices is that performance of synthetic devices over the VMBus is closer to performance of non-virtualized hardware devices.
Figure 3 VSCs use the VMBus to access VSPs, which then manage access to underlying physical hardware

Integration Components
Hyper-V was built to provide strong boundaries between various instances running on one computer. To enable interaction between the guest OS and the host OS and to supply some additional functionality for supported guest OSs, Hyper-V provides Integration Components.
The Hyper-V Integration Components support the following features:
  • Time synchronization
  • Volume Shadow Copy Service (VSS)
  • Heartbeat functionality
  • Guest shutdown
  • Key value pair exchange (used to access the registry of a guest OS)
  • OS identification
The Hyper-V Feature Set
It goes without saying that the closer the virtualization platform comes to acting like the physical server, the easier it becomes for organizations to deploy and rely on virtual workloads. In my view, there are four key areas under which you can view the different features of the virtualization platform.
Today most hypervisor-based virtualization solutions are pretty close to each other in terms of features and functionality. As we move forward, things like total cost of ownership (TCO) and ease of use will be key differentiators. And the management solutions will see continued investments and development to bring us closer to the vision of a dynamic IT environment, where the infrastructure is flexible enough to adapt to the needs of the business, and models and policies help drive increased automation and management.

Scalability
Using the microkernelized hypervisor architecture, Hyper-V has very low CPU overhead, leaving plenty of room to virtualize workloads. By enabling virtual machines to take advantage of powerful features and hardware, such as multicore technology, improved disk access, and greater memory, Hyper-V improves scalability and performance of the virtualization platform.
Combined with the rest of the Windows Server 2008 capabilities, Hyper-V allows you to consolidate most workloads—including 32-bit and 64-bit workloads—on a single system. And it can help you balance 64-bit technology adoption with continued support for 32-bit workloads already used throughout your environment.
The fact that Hyper-V requires a 64-bit host system with hardware-assisted virtualization helps ensure that the host system will have access to a large pool of memory resources. Hyper-V can support up to 1TB of memory on the host, with up to 64GB of memory per virtual machine. This is key for those who plan to virtualize memory-intensive workloads such as Exchange Server and SQL Server.
Hyper-V also supports up to 16 logical processors on the host system, making Hyper-V scalable to most commodity two-socket and four-socket systems with multiple cores. You can also create a virtual machine with up to four virtual processors in order to support workloads that require or take advantage of multi-processor capabilities.
Consolidating servers through Hyper-V also enables those servers to make use of robust networking support, including VLAN, Network Address Translation (NAT), and Network Access Protection (NAP) policies (quarantine). And as a Windows Server 2008 feature, Hyper-V works well with other Windows Server features, such as BitLocker and Windows PowerShell.

High Availability
High availability is a scenario where Hyper-V and host clustering capabilities work together to help address business continuity and disaster recovery needs. Business continuity is the ability to minimize both scheduled and unscheduled downtime. That includes time lost to routine functions, such as maintenance and backup, as well as unanticipated outages.
Disaster recovery is an important component of business continuity. Natural disasters, malicious attacks, and even simple configuration problems such as software conflicts can cripple services and applications until administrators resolve the problems and restore data. A reliable business and disaster recovery strategy must offer minimal data loss and powerful remote management capabilities.
When looking at high availability, you should consider three different categories—planned downtime, unplanned downtime, and backups. Protection for planned downtime is typically needed to help move the virtual machines off the host system so you can either perform hardware maintenance or apply patches to the host system or the virtualization platform (which may potentially require a reboot).
Most organizations have planned maintenance windows, and what you are really looking to do here is to minimize or eliminate the period of time in which the virtual machines will not be available while the host system is down for maintenance. With the Quick Migration feature, you can rapidly migrate a running virtual machine from one physical node to another in a matter of seconds. Thus, you can keep your virtual machines available for production use while you perform maintenance on the original host. Once the maintenance is done, you can then use Quick Migration to return the virtual machines back to the original host system.
Unplanned downtime is downtime that is not foreseen. It can be catastrophic in nature or as simple as someone accidentally unplugging a power cord and bringing a server down. Although that may sound unlikely, over the years I have met quite a few administrators at Tech•Ed, VMworld, and other conferences who have stories to tell about how some server was accidentally powered off by a colleague.
With Hyper-V, you can set up a host cluster for the different host systems and configure all the virtual machines as cluster resources that can then failover to a different system in case one of the hosts fails. Meanwhile, the multi-site clustering capability of Windows Server 2008 will enable you to set up a geographically dispersed cluster so that if your primary datacenter fails, you have the ability to recover the different virtual machines to a remote datacenter.
This is also handy for protecting all your branch offices. One of the advantages of the unplanned downtime support with Hyper-V is that it is guest OS agnostic, which means you can extend its high availability benefits to Linux virtual machines and older versions of Windows Server to protect and recover those systems similarly.
As you look at unplanned downtime, it is important to note that the recovery is equivalent to powering off the system and restarting, which means you will have lost all state information. This might or might not be a problem, depending on the workload you are running in the virtual machine. That's why it is important to look at backup in the context of high availability.
Hyper-V lets you take backups of each virtual machine or use VSS to take consistent backups of all the VSS-aware virtual machines while they are still running. With VSS, you can set up backups to occur at set intervals without having an impact on the production workload availability while ensuring that you have a continuous backup plan that can help you easily recover state in the event of an unplanned downtime. For more information on high- availability solutions with Hyper-V, see the article "Achieving High Availability for Hyper-V" by Steven Ekren in this issue of TechNet Magazine (technet.microsoft.com/magazine/cc837977).

The microkernelized hypervisor architecture is designed to minimize attack surface and to enhance security, especially when Hyper-V is implemented as a Server Core role. Server Core is an installation option of Windows Server 2008. The hypervisor contains no device drivers or third-party code, promoting a more stable, thin, and secure foundation for running virtual machines. Hyper-V also delivers strong role-based security with Active Directory integration. And Hyper-V allows virtual machines to benefit from hardware-level security features, such as the execute disable (NX) bit, further helping to increase the security of virtual machines.
Hyper-V has gone through the Secure Development Lifecycle (SDL) like the rest of the Windows Server components, and there has been extensive threat modeling and analysis done to ensure that Hyper-V is a highly secure virtualization platform. As you deploy Hyper-V, be sure to follow the best practices for deploying Windows Server 2008 and also the best practices for Hyper-V. Include Active Directory as well as antivirus and anti-malware solutions as part of your plan. And use delegated administration capabilities to ensure that you use admin access privileges appropriately for Hyper-V hosts.

Manageability
It is easy to go from a slight server sprawl problem to a massive virtual machine sprawl. This is a risk due to just how easy it is to deploy virtual machines. And with the increased mobility of virtual machines, you also have the added responsibility of knowing exactly where the different virtual machines are running, keeping track of their security contexts, and so on.
Fortunately with Hyper-V, you don't have to create a separate management infrastructure for your virtual environment. It integrates with Microsoft management tools, System Center Virtual Machine Manager, and Microsoft System Center Operations Manager, as well as with third-party management tools. This lets you manage your physical and virtual resources all from one console. For details about System Center Virtual Machine Manager 2008, see Edwin Yuen's article, "Manage Your Virtual Environments with VMM 2008," in this issue of TechNet Magazine (technet.microsoft.com/magazine/cc836456). Meanwhile, support for Windows PowerShell makes it easy to automate tasks.
Hyper-V also provides virtual machines with an unprecedented ability to use available hardware. Because all Windows Hardware Quality Lab (WHQL)-certified drivers are able to run in the parent partition, Hyper-V delivers broad compatibility for drivers and devices, making it easier to manage the different drivers that are running in your environment.

Wrapping Up
As I mentioned earlier, management is going to be a key area of development and differentiation. You will certainly see a lot of activity in that area in the years to come. Overall, this is really an exciting time as virtualization crosses into a more mainstream role. If you have not yet tried Hyper-V or you simply want more information, go to microsoft.com/Hyper-V.
Rajiv Arunkundram is a Senior Product Manager at Microsoft focused on server virtualization in the Windows Server Marketing division. Rajiv's primary responsibility is to work with customers and partners to help them understand the virtualization strategy and solutions of Microsoft from a technical and business perspective.


Virtualization
Manage Your Virtual Environments with VMM 2008
Edwin Yuen

This article is based on a prerelease version of Virtual Machine Manager 2008. All information herein is subject to change.
At a Glance:
  • Support for VMware VMs
  • Windows PowerShell-based scripting and automation
  • Consolidated, centralized management
  • Installation procedures

Contents
An Introduction to Virtual Machine Manager 2008
Integration with System Center
Virtual Machine Manager 2008 Installation
Adding VMware ESX Hosts
VMotion Migration
Conclusion
Last year, Microsoft introduced System Center Virtual Machine Manager 2007, which provides a consolidated interface for managing a virtual infrastructure running Microsoft Virtual Server 2005. Now, to coincide with the release of the new Windows Server 2008 Hyper-V hypervisor, a new version of Virtual Machine Manager (VMM) has been released. System Center Virtual Machine Manager 2008 manages not only Microsoft Virtual Server 2005 and Windows Server 2008 Hyper-V installations, but support has also been extended to VMware Infrastructure 3 (ESX/ESXi) servers. With VMM, you now have an integrated management interface that works with your entire virtual infrastructure, including both Microsoft and VMware systems.
VMM supports all the traditional virtualization tasks, such as rapid virtual machine (VM) deployments from templates and pre-configured VMs, Physical-to-Virtual (P2V) conversions, a centralized library of virtual infrastructure objects, and even Virtual-to-Virtual (V2V) conversion of VMware virtual machines (VMX configuration files plus VMDK virtual disks) to Hyper-V VMs. Moreover, VMM integration with the rest of the System Center family brings the management of physical and virtual systems to a single pane of glass.
In this article, I will examine VMM 2008 and the new features it brings to IT administrators. I'll then look at the requirements and the steps involved with a VMM installation. Finally, I'll take a closer look at how to use some of the new features of VMM 2008.

An Introduction to Virtual Machine Manager 2008
System Center Virtual Machine Manager 2008 is a standalone application that provides centralized, enterprise-class management of virtual machines running under Microsoft Virtual Server 2005 R2 SP1, Windows Server 2008 Hyper-V, Microsoft Hyper-V Server 2008, and VMware Infrastructure 3 (ESX/ESXi) servers. Together with other System Center products, VMM provides end-to-end physical and virtual infrastructure management. Figure 1 shows the architectural layout of Virtual Machine Manager.
Figure 1 System Center Virtual Machine Manager 2008 Architecture
As those familiar with VMM 2007 will note, there are significantly more interfaces to the VMM system in VMM 2008 (see Figure 1). The boxes in the upper-left represent VMM 2008; the boxes in the upper-right represent System Center Operations Manager 2007, and the managed systems and interfaces are shown below. The red boxes indicate the management servers, the blue boxes show the system inputs, and the green boxes are the managed host systems with their managed objects shown in yellow.
The red box on the upper-left represents the Virtual Machine Manager 2008 server service. Above it are the different interfaces into the service; the Administrator Console, the Windows PowerShell interface, and the Self-Service Web Portal with delegated user access. Windows PowerShell forms the basis for all access to the VMM server. The VMM Admin Console is built entirely on Windows PowerShell, so you can simply create Windows PowerShell scripts that the VMM server service runs. Thus, the public API for VMM 2008 is Windows PowerShell, and any program can leverage VMM using Windows PowerShell. A great example of this is the integration of Citrix's XenDesktop Desktop Delivery Controller with Virtual Machine Manager, which is done entirely through Windows PowerShell.
Supporting VMM is a SQL Server 2005 or SQL Server 2008 database, either a full installation or SQL Server Express Edition if no other database is available. While the diagram shows both Operations Manager and Virtual Machine Manager pointing to SQL Server, the two programs do not share a database. They simply both run on the same version of SQL Server. The actual integration between Virtual Machine Manager and Operations Manager is through a connector that is available through VMM. This connector allows VMM and OpsMgr to share data, which brings virtualization information into OpsMgr and brings OpsMgr data to VMM for actions.
The bottom half of the diagram shows the managed hosts. VMM 2008 still manages Virtual Server 2005 R2 SP1 virtual machine hosts and a centralized library share, like it did in VMM 2007. It should be noted that in VMM, the library share is not a special file store or server. It's simply a Windows file share that VMM manages. You can manage the library through VMM or you can easily copy and delete files out of the file share and let VMM discover the changes. You can also have multiple libraries, which is useful for delegated sets of servers and for servers in remote offices.
New to VMM 2008 is the ability to manage the Hyper-V and ESX hypervisors. The management of Hyper-V hosts is done through Windows Remote Management (WinRM) and the Windows Management Instrumentation (WMI) interface. In fact, when a potential Hyper-V host is added, if the Hyper-V role is not enabled in Windows Server 2008, VMM can automatically enable that role for the administrator. This feature is also available to potential Virtual Server hosts for Windows Server 2003 systems.
The management of VMware VI3 ESX hosts is actually done through a VMware VirtualCenter server. VMM 2008 interfaces with VirtualCenter through the VirtualCenter Web Services API. Thus, a VirtualCenter server is required for VMware management, and most enterprise installations of VMware will have at least one VirtualCenter server. What is unique to VMM 2008 is the ability to manage multiple VirtualCenter servers. VMM can add multiple VirtualCenter servers, allowing an administrator to aggregate the entire VMware infrastructure and provide a single enterprise view that was not possible before.
As part of the System Center family of products, VMM uses the System Center or Outlook-style interface in the Administrator Console. The VMM Administrator Console is not an MMC snap-in; it's a completely new application built using the Microsoft .NET Framework and is built on top of Windows PowerShell 1.0. In fact, as noted earlier, any action in the Administrator Console calls the associated Windows PowerShell command, so the UI actually uses Windows PowerShell. Any command or function done in the Administrator Console can be done through the command line in Windows PowerShell.
Each wizard in the Administrator Console has a View Script button that can show the associated Windows PowerShell script for the command about to be run—a great starting point for learning Windows PowerShell as it relates to VMM. New in VMM 2008, the View Script action now opens these scripts in Notepad, so you can easily edit them right from VMM. You can also easily save the edited scripts, which you can then reuse right from the VMM library—another new feature in VMM 2008. The cmdlets from VMM 2007 have been updated for VMM 2008, including support for VMware servers. The same cmdlets work across virtualization platforms, so that cmdlets like New-VM work on both Microsoft and VMware systems. IT administrators can thus use one set of cmdlets to manage Virtual Server, Hyper-V, and VMware ESX hosts.
Virtual Machine Manager 2008 is designed to provide three important benefits for IT administrators:
  • Maximize resources
  • Achieve better agility
  • Leverage existing skills
A number of features and functions promote these benefits. First, VMM helps maximize your resources by integrating the management of your entire virtual infrastructure, regardless of hypervisor. By integrating Microsoft and VMware systems, VMM 2008 provides a single view of all hosts and virtual machines, as shown in Figure 2, and allows administrators to filter and view their systems by parameters such as host, operating system, owner, date added, and so forth.
Figure 2 The VMM Console shows integrated Microsoft and VMware management
VMM also helps maximize system resources through easy server consolidation with conversions such P2V and V2V. P2V lets you consolidate your existing physical machines into virtual machines without having to reinstall those machines. This is especially useful when converting machines that have low utilization for their hardware, so you can maximize resources by placing several low-utilization machines together on one piece of physical hardware. P2V is an integrated feature of VMM and doesn't require additional software or servers. For VMM 2008, support for a greater number of physical systems has been added, including Windows Vista, Windows Server 2008, and x64 hosts. The P2V functionality of VMM is a great companion to planning tools such as the Microsoft Assessment Planning (MAP) tool (discussed in the article by Baldwin Ng and Jay Sauls in this issue of TechNet Magazine), which helps identify candidates for server consolidation.
If you already have virtual machines in VMware's VMDK format, VMM allows you to use V2V to convert your existing VMDKs into the VHD format. This is an offline process—the VMware VM has to be shut down as the V2V process converts the actual VMDK file to a VHD. You can also convert a running VMware VM by using the P2V function. This may be a little confusing, but it's important to remember that virtual machines are machines first, virtual second. Since all other systems see VMs as running machines, so will VMM's P2V functionality, which lets you convert a VM running on any virtualization software to a Microsoft VHD.
VMM provides greater agility for IT administrators through several features. One of the key features that VMM enables is the migration of VMs from one host to another, which can help maintain high availability and server optimization. With Hyper-V hosts, VMM 2008 supports Quick Migration, which can move a VM between Hyper-V hosts with only seconds of downtime. This requires a Windows Server 2008 cluster and shared storage, but VMM helps ease the management and configuration.
With a VMware ESX host, VMM 2008 also supports VMware VMotion, the VMware feature that moves VMs without any user-perceptible downtime. VMotion-enabled VMware ESX hosts are required, as is a cluster and SAN back-end like Quick Migration. Later in this article, I'll walk you through a VMotion using VMM 2008.
Finally, VMM 2008 can migrate an existing Virtual Server 2005 VM onto a Hyper-V host, automatically performing all the upgrades and conversions needed for migration.
As part of any VM deployment (including the migrations described above) started through either conversions or new deployments, VMM uses a process called Intelligent Placement to deploy the VMs to the hosts. It also uses Intelligent Placement to query all the virtual machine hosts that it is managing, retrieves several parameters about the resource availability of those hosts, and then returns a weighted list of recommended hosts to which you can deploy the VM. As you can see in Figure 3, Intelligent Placement supports Microsoft and VMware hosts, along with their storage configurations. Intelligent Placement in VMM 2008 is both platform aware and configuration aware. Thus it will only recommend hosts that fit the correct host platform; only hosts that are clustered if high availability is needed, and only x64 capable hosts for x64 VMs.
Figure 3 Intelligent Placement analyzes all available hosts

Integration with System Center
One of the key features of VMM 2008 is the integration with System Center Operations Manager 2007. While VMM 2007 had some integration with Operations Manager, VMM 2008 includes a product connector for Operations Manager that greatly increases the functionality of the integration. Through its integration with Operations Manager, VMM provides consolidated operations manager reports from the admin console, as well as through a new feature called Performance and Resource Optimization (PRO).
PRO enables VMM 2008 to create alerts called PRO Tips and even perform automated action remedies based on monitoring parameters from Operations Manager. Figure 4 shows some example PRO Tips. When using a PRO-enabled management pack (PRO pack), VMM 2008 can perform automated actions such as migrations of VMs and creation of new VMs based on host-, hypervisor-, hardware-, and even workload-based performance parameters.
Figure 4 PRO Tips provide alerts and remedies for issues in the virtual infrastructure

Virtual Machine Manager 2008 Installation
To perform a System Center VMM 2008 installation, there are several prerequisites and new system requirements. First, the VMM 2008 server requires Windows Server 2008 x64. The server will not install on Windows Server 2003 or on x86 versions of Windows Server 2008. The Admin Console and other components can still be installed on Windows Server 2003 and Windows XP or Windows Vista (in the case of the console), with the same requirements and installation procedures as VMM 2007.
Second, VMM 2008 requires an installation against an Active Directory domain for its security model. VMM 2008 supports the management of perimeter network (non-domain joined) hosts, including the P2V of non-domain systems, which is new for VMM 2008. One of the advantages of using Windows Server 2008 as the installation platform is that many of the prerequisites for VMM (such as the .NET Framework 3.0, WinRM, and Windows PowerShell 1.0) are already built in. Finally, VMM 2008 requires the Windows Automated Installation Kit (WAIK) 3.0, although the VMM 2008 installer includes the installation files.
While I recommend that in a production environment each VMM component be installed on a separate production server, I'll discuss a single-server installation. The recommended hardware for a VMM server installation is an x64-based server running at 2.8GHz or higher with at least 2GB of RAM. Since this is a single-server installation with a local SQL database, 7GB of hard disk space is required. Note that if a remote database were used, only 1GB would be required for the VMM installation itself. An additional 80GB of disk space is recommended for the local library installation.
As mentioned earlier, VMM runs on a Windows Server 2008 x64 installation. Additional software requirements include the .NET Framework 3.0, Windows PowerShell 1.0, WinRM, and IIS 6.0 or later for the self-service portal. The Microsoft MSXML 6.0 Parser is also required, but it is included a part of .NET Framework 3.0.
When you start the VMM installation, you are offered five options: the Virtual Machine Manager Server, the Administrator Console, the Self-Service Portal, and the Local Agent, as well as an option to configure the Operations Manager management pack. As I mentioned, I will install all the components on a single machine, starting with the VMM Server. Once the setup starts, the first screens will check for the system prerequisites.
In addition to the requirements mentioned previously, the machine must be on a domain. The VMM server installation will automatically enable the built-in functions for you, thus eliminating the need to enable them manually.
Once you launch the installer, I recommend using the Windows Update option to make sure VMM 2008 is up to date. The installer will then review the options, mainly relating to the SQL database installation. For this example, I'll use SQL Server 2005 Express Edition.
Next, the wizard asks you to identify the location for the centralized library. This can be either a local directory on the VMM server (and a share will be created for that directory) or it can be an existing file share. The default location is C:\ProgramData\Virtual Machine Manager Library Files.
Port assignments are next. Here I configure the port for the Administrator Console's connection to the core VMM server service (8100 by default); the port used by WinRM to perform the management functions (80 by default); and the port used by the Background Intelligent Transfer Service (BITS) to move files to the virtual machine hosts (443 by default). Once you've entered this information, a summary screen is presented and the server installation can finish.
Next, I'll install the Administrator Console, which has system prerequisites similar to the VMM server. The only configuration option available other than the install directory is to confirm the communications port used to connect to the VMM server service. This port should be 8100, as configured in the VMM server installation.
At the end of the installation, you are given the options to create a desktop shortcut for the Administrator Console and to open the console when you close the setup. It's a good idea to select both. The first time you open the Administrator Console, you will be prompted to Connect to the server. With the single server install, localhost on port 8100 is correct and you can simply press Enter to start the console.
Finally, I will install the self-service portal. The prerequisites for this are IIS 6.0 or 7.0, .NET Framework 3.0, and Windows PowerShell 1.0. (Windows PowerShell is needed as the self-service portal is just another interface to the core VMM services, which use Windows PowerShell.) For Windows Server 2008, you need to enable the IIS role before starting the VMM 2008 installation. When doing so, make sure the following IIS Role Services are enabled:
  • Common HTTP Features (Static Content, Default Document, Directory Browsing, HTTP Errors)
  • Application Development (ASP.NET, .NET Extensibility, ISAPI Extensions, ISAPI Filters)
  • Security (Request Filtering)
  • IIS 6.0 Management Compatibility
If these role services are enabled, the installation will pass the prerequisite check and the installation will proceed. During installation, the configuration options deal only with the ports used by the self-service portal. The first, the port to connect to the core VMM service, is 8100, the same as for the Administrator Console. The second is the port on which to run the portal. The default is port 80, but in many configurations, that port is already taken and the VMM installer will warn you if that's the case. If necessary, change the port to one that's open, such as 8080. The installer will then create a Web site called Microsoft System Center Virtual Machine Manager 2008 Self-Service Portal Home. The configuration can be changed in the Server Manager.
When the installation of IIS has completed, you can simply open the Web site by going to localhost:8080 in Internet Explorer. If you are prompted for authentication when opening the Web site, this is because of Internet Explorer security settings. The self-service portal should run in the intranet context in Internet Explorer and you may need to add the VMM server to the list of intranet sites in Internet Explorer. Once those configurations are complete, you will be ready to create and use self-service-enabled VMs in VMM 2008.

Adding VMware ESX Hosts
While there are many new features in VMM 2008, arguably the most anticipated feature is the addition of support of VMware ESX host management. With this support, VMM is now capable of managing the virtual infrastructure of the majority of enterprise IT departments.
The process of adding ESX hosts into Virtual Machine Manager 2008 is fairly simple. As I noted earlier, VMM 2008 manages VMware hosts through a VirtualCenter server, which must be version 2.0.1 or 2.5. To add the VMware ESX servers into the management console, you simply add the VirtualCenter server using the Add VMware VirtualCenter Server action located in the upper right of the Actions pane.
After you launch the Wizard, type in the computer name of the VirtualCenter server along with the Administrative Account for the VirtualCenter server. VMM will add the ESX hosts, including the existing Host Group structure from VirtualCenter, into the VMM interface. This task may take several minutes, but you can monitor the progress from the VMM Jobs menu.
When all the servers have been imported, you should be able to see the same VMware ESX host server structure in VMM as you see in VirtualCenter. To add another VirtualCenter server, you would simply repeat the process. Once the host structure is added to VMM, you can add non-VMware servers into the same group structure. Of course, the non-VMware servers will not be shown in VirtualCenter.
While VMM 2008 uses VirtualCenter to manage ESX hosts, it needs to be able to directly access the ESX hosts themselves for certain functions, such as V2V, saving state, and cloning VMs from ESX hosts. Thus, after you add the VirtualCenter server into VMM, the hosts will show up as OK (Limited). To enable full management, go into the Properties page of the individual ESX hosts in VMM and supply the appropriate host credentials to directly manage the servers.

VMotion Migration
Whenever VMware management in Virtual Machine Manager 2008 is discussed, invariably the question asked is whether VMM can perform VMotion and other VirtualCenter functions. And since VMM uses VirtualCenter for VMware management, the answer to that question is yes!
Functionally, VMM treats VMotion like any other migration. To start a VMotion, first select the VM you want to move, then select the Migrate virtual machine option from the Actions pane on the right-hand side of VMM. When you've done that, VMM brings up the Intelligent Placement window for that VM and presents the potential destination hosts, with a transfer type of Live, as shown in Figure 5. Intelligent Placement can help you select the right destination hosts for your VMs. Note that this is the same process as performing a Quick Migration using Hyper-V hosts.
Figure 5 VMotion using Intelligent Placement
Select the destination host for the VM—in this case, it is the server called cqaesxi003.contoso.com and click Next. The final summary window, shown in Figure 6, will display the requested action.
Figure 6 VMotion Summary window
As mentioned earlier, each wizard in VMM has a View Script button that can show the Windows PowerShell script for a job in Notepad. This gives us an opportunity to see how Windows PowerShell in VMM 2008 works with VMware. Here's the script for this action:
$VM = Get-VM -VMMServer localhost -Name "MyVM" |
  where {$_.VMHost.Name -eq "cqaesxi003.contoso.com"}
$VMHost = Get-VMHost -VMMServer localhost |   where {$_.Name -eq "cqaesxi003.contoso.com"}
Move-VM -VM $VM -VMHost $VMHost –RunAsynchronously
Three simple commands perform the VMotion for this system. In my example, I am moving the VM MyVM from the VMware server cqaesxi003.contoso.com to the server cqaesxi001.contoso.com. The first command is to identify the virtual machine to be migrated via VMotion. As you can see, in this command line, I assign the virtual machine value to the variable $VM:
$VM = Get-VM -VMMServer localhost -Name "MyVM" |
  where {$_.VMHost.Name -eq " cqaesxi003.contoso.com"}
The second command is to identify the destination ESX host system for the VM. In this line, I assign the variable $VMHost to the ESX host cqaesxi003.contoso.com:
$VMHost = Get-VMHost -VMMServer localhost | where
  {$_.Name -eq " cqaesxi003.contoso.com"}
Finally, I use the Move-VM command to actually move the VM. Here are the complete command line and parameters for Move-VM:
Move-VM [-VM] [<String VM>] -VMHost [<String Host>]
  [-JobGroup <Guid>] [-JobVariable <String>] [-Path
  <String>] [-PROTipID] [-RunAsynchronously]
  [-StartVMOnTarget <Boolean>] [-UseLAN <Boolean>]
  [<CommonParameters>]
In keeping with the heterogeneous management philosophy in Virtual Machine Manager, you can see that the command is not vendor-or software-specific. To complete the VMotion, issue this command:
Move-VM -VM $VM -VMHost $VMHost –RunAsynchronously
This tells VMM to run Move-VM, against the VM in $VM, moving to the host system in $VMHost, and to run the job asynchronously. Once the job starts, the entire process can be monitored from the Jobs window in VMM. Again, this same basic command can be done for Quick Migrations on Hyper-V clusters and even non-cluster hosts, using a LAN based transfer.
As you see, System Center Virtual Machine Manager 2008 has extended the great features and functions of Virtual Machine Manager 2007 with new management platforms and new features. By reviewing the installation process and a couple of the features, I hope I've shown how easy and the management of multiple hypervisors is in VMM 2008.
VMM 2008 is designed to be the consolidated management software for virtualization, unifying both physical and virtual systems from Microsoft and VMware. With this new version of Virtual Machine Manager, VMware administrators, Microsoft Virtual Server 2005 administrators, and new Hyper-V administrators can all take advantage of a single tool, a single scripting interface, and most importantly, a single management process for their virtual infrastructure.

Edwin Yuen is the Technical Product Manager in the Windows Enterprise Management division for System Center Virtual Machine Manager. Edwin came to Microsoft with the July 2006 acquisition of Softricity. Edwin also has 13 years of technical consulting experience in both the commercial and federal space. He has a BS in electrical engineering from Johns Hopkins University.


Virtualization
Getting Started with Microsoft Application Virtualization
Anthony Kinney

This article is based on a prerelease version of App-V. All information herein is subject to change.
At a Glance:
  • App-V architecture
  • Managing virtual apps
  • Using the App-V sequencer
  • Integrating App-V with Configuration Manager

Contents
The App-V Architecture
How App-V Full Infrastructure Works
Updating Virtual Applications
Sequencing
Version 4.5
Microsoft Application Virtualization (or App-V) is near and dear to my heart. App-V was formerly known as SoftGrid, and I came to Microsoft with the acquisition of the company that created SoftGrid, called Softricity. I am very excited to have the opportunity of writing this article for TechNet Magazine now, as a lot has changed since the acquisition.
The best way to approach App-V is first to talk about the challenges IT professionals face in terms of enterprise management. Today's business desktop is awash in applications. Before an application is installed, it must go through lengthy regression testing to ensure that it can coexist with the other applications installed on the system without impacting their ability to run properly. The application must then go through a series of deployment processes before it reaches production. And because an application is essentially only available where it is installed, your users are tied to specific computers. This further complicates complex yet critical projects, such as OS and application migrations, security refreshes, and disaster recovery planning.
App-V changes all that. Rather than being a complex series of time-consuming steps that take up resources, desktop administration becomes a simpler, more automated process with App-V. You can more easily deploy, patch, update, and terminate applications with better results.
With App-V, a user can sit down at any desktop and access his applications. The applications are delivered on demand but run as if they were actually installed locally. Thus, there is no need to install the application components or alter the host device.
This use of virtualization could drastically change how IT professionals manage desktops. Not altering the host device and running virtualized applications instead introduces numerous advantages, including:
  • Fewer application conflicts
  • Faster and easier application updates
  • The ability to run multiple versions of the same app side by side
  • Flexible applications that follow users online and offline
  • Reduced application-to-application regression testing

The App-V Architecture
So now let's look at what is really happening behind the scenes of the App-V platform. The platform consists of a few main components: a sequencer, a database, clients, a management server, a streaming server, and a management console (see Figure 1).
Figure 1 Layout of an App-V environment
At the core of the App-V system is the App-V client. There are two types of clients that can be used—the Terminal Server Client and the Desktop Client. In either case, the client must be installed on every desktop and terminal server on which you plan to deploy virtual applications. The client takes up relatively little disk space. It installs a driver and has a visible user runtime component that shows up as a tray indicator.
The client gathers a list of virtual applications from the App-V Management Server and displays the available virtual apps. It handles launching those applications (when initiated by the user) and managing the client-side cache. The client also is responsible for managing creation of the virtual runtime environment and ensuring that each environment runs in its own virtual bubble. This virtual environment includes several components, including a Virtual Registry, Virtual File System, and Virtual Services Manager.
There are three infrastructure deployment options available in App-V 4.5: full infrastructure, lightweight infrastructure, and standalone mode. When you deploy a full infrastructure, the back end includes the App-V Management Server and the App-V Streaming Server (this is a new component I will discuss in a moment). The App-V Management Server hosts and delivers the centralized virtual applications, as well as updates the virtual applications when patches or updates are applied.
This management server relies on SQL Server to host the App-V database, which contains configuration and settings for virtual applications. You should use Active Directory groups as the central management tool for provisioning and controlling permissions to virtual applications.
To manage the settings and configuration, the App-V platform provides a Microsoft .NET Framework Web service that can be loaded on the same server as long as IIS is installed. This Web service acts as a liaison between the App-V Management Console—a Microsoft Management Console (MMC) Snap-in—and the App-V database. Administrators can use the console to publish and manage virtual applications, assign Active Directory groups, and control server settings, as well as run reports on usage of virtualized applications (see Figure 2).
Figure 2 Management Console
The lightweight infrastructure includes the App-V streaming server, which enables streaming capabilities such as active/package upgrade. This option does not require Active Directory or SQL Server, it has no desktop configuration service, and it lacks licensing and metering capabilities. The lightweight infrastructure does, however, allow streaming capabilities to be added to System Center Configuration Manager (SCCM) and other third-party enterprise software deployment (ESD) solutions.
In standalone mode, the App-V sequencer can create an MSI file that automates the addition of the virtual application (see Figure 3). The MSI file contains metadata that allows any ESD system to recognize it and control the virtualized application. This mode requires the client to go into standalone mode, which only allows MSI-based updates of the virtual applications, and streaming is not allowed while in standalone mode. This mode gives organizations the ability to make use of the App-V isolation capabilities.
Figure 3 App-V sequencer
MSI files are very flexible, they can run completely standalone with just an App-V client, and they don't require any server components. This means they can be deployed manually, with a disk, or by any traditional deployment tools.
In App-V 4.5, HTTP and HTTPS are now supported protocols for streaming. This enables better performance of streaming along with a more widely adopted protocol, especially for streaming across secure wide area network (WAN) environments and across the Internet.

How App-V Full Infrastructure Works
A user logs on to a device that has one of the clients (either App-V Terminal Services or Desktop Client) installed, and the client sends a request to the server for a list of applications assigned to the current user. The server communicates with Active Directory to determine which groups the user is a member of and then returns the list of applications back to the client. The client begins building advertisements for the virtual applications that have been assigned to that particular user.
In this publishing process, several actions are performed:
  • Configuration files are copied
  • Desktop icons are created
  • Send To links are created
  • Start menu folders are created
  • File types are configured
This process is very fast and, most importantly, ensures that the environment will look exactly as the user expects with no visual changes. The virtual applications act as if they are installed locally, but, of course, they don't alter the host machine. The icons, instead of pointing to executables that reside in the program files directory, point to the App-V client, which relies on a launcher file (an OSD file) for its configuration.
It's important to note that this process has very little impact on the network because, unlike traditional software deployments, nothing is being installed. This has tremendous benefits, especially in roaming user environments, since the applications are available to the user but nothing is actually delivered until an application is launched. This advertisement method is also what provides the on-demand and roaming application features of App-V.
When the user launches a virtual application, the client reads an OSD configuration file, which has been stored on the local machine. This tells the client which protocol to use when communicating with the App-V management server and on which server the application resides.
The appropriate server responds to the client by streaming down the initial launch threshold, which is typically 20–40 percent of the full application. Once the entire launch threshold has been streamed (again, just 20–40 percent of the application), the virtual application is ready to run.
The streaming really is one of the key elements of the paradigm shift being introduced with App-V. It can send just enough of an application so that it can run without wasting valuable network bandwidth. All the data being delivered to the client resides in a local cache file on the device and any subsequent launches of the application are launched from the local cache, eliminating additional network traffic.
Once the virtual application is finished streaming, the client builds an isolated environment that prevents the application from altering the local machine (in other words, the application has no client footprint). The client does, however, allow the virtual application access to the local file system when saving and editing files, and it also allows the application to interact with local services (such as printing) as long as the user has the appropriate privileges on the local system. But any changes made by a virtual application to the local system's files and registry are redirected to the virtualized environment so the host device will remain unaltered.
When the application is being run, any features that haven't been used previously are delivered as needed and cached for subsequent use. The upside to this is that only components needed by the user are loaded on initial launch and features that aren't needed don't consume network resources. (The new version provides some client-side caching enhancements that allow for smarter cache usage and background streaming.)
Consider Microsoft Office Word, for example. Almost all users use spell checker (heck, I couldn't write this article without it!); therefore, it would be part of the initial launch. But what about the Help feature in Word? Not as many users use this feature, and, thus, it wouldn't need to be delivered in the initial launch. Instead, it would be sent to a user the first time he accesses it.
When the user is finished and closes the application, the client tears down the virtual environment and stores all the user settings in a user-specific location so the environment can be retained and rebuilt upon next launch. Whatever percentage of the virtual application that had been streamed remains in the local cache and is available for the next launch. And if another user logs onto the same host system and launches the same virtual application, the new user benefits from the application already stored in the cache.
To remove the virtual application advertisements, simply remove the user from the appropriate Active Directory group. And to uninstall the virtual application from a desktop completely, you can simply clear the cache. Since the application was never really installed locally, there are no annoying prompts asking, "Do you want to remove this shared component?"
Note that even if a virtual application is stored in a cache, that doesn't mean all users have access to it. Unlike locally installed applications where users can simply search or browse for executables that they don't have rights to, there are no visual or physical representations that the virtual application exists unless the user has been given explicit rights to it through Active Directory groups.

Updating Virtual Applications
Updating is done using the sequencer. Once an application has been revised to include an update, it is placed on the App-V management server right next to the previous version. The server then notifies the client upon next launch that a change has been made. If the previous version is still being used, the user continues to have access to that version until the virtual application is closed. Upon the next launch, the deltas that make up the update are streamed down to the client and loaded into cache, resulting in an updated version of the application.
Say you have 1,000 users running Word 2000. An administrator needs to update Word 2000 (word2K.sft) to Word 2000 SP3, so she copies the word2K.sft file over to the sequencing station and selects Open for Package Upgrade in the sequencer. By selecting Open for Package Upgrade, the admin begins working from the last package state. She can then copy DLLs, run updates, or execute patches within the virtual application to update it to Word 2000 SP3. The admin then saves this updated package.
The sequencer automatically assigns a new file name, word2K_2.sft, to prevent duplicate file names and to indicate the version of sequencing. This new package is placed in the same directory as the old package on the App-V management server so that Word 2000 (word2K.sft) and Word 2000 SP3 (word2K_2.sft) end up residing in the same directory. The admin then uses the App-V management console to link these two SFT files together.
On the client side, users who have an active session of Word 2000 without SP3 open continue to function normally. Users who launch a new session of the application after the admin has done this linking will receive a message that a change has been detected. The client then begins streaming down only the delta changes between word2K.sft and word2K_2.sft, automatically updating the application to Word 2000 SP3.
Because of the dynamic nature of virtual applications, rolling back is also quite easy. Simply go back to the App-V management console and remove the newly added version. This causes the client to roll back to the previous version upon relaunch. To ensure no crossover of package data, the client automatically purges the cache and restreams the proper SFT file. This is a fair trade-off when you consider what must be done to roll back an application update that has been installed physically using more traditional software deployment tools.

To realize the benefits of App-V, you must create virtual application packages. This is where the App-V sequencer comes into play. Any knowledge and experience you have in scripting and creating packages for traditional software deployment tools will ease your transition into sequencing. (I should point out here that sequencing could be a full article on its own.)
Most software deployment solutions rely on scripts that capture the way an application installs itself and then duplicate the process on other machines, eliminating the need to visit every machine to install or update applications. Once the application has been installed, common software deployment tools wipe their hands clean of the package. You must then install any dependencies on which the application may rely, run other scripts, or perform manual steps to configure the application for your needs.
The fundamental change in App-V is that the sequencing process generates an image of an already installed app, complete with its dependencies and configurations. This can be "played" by the App-V client without altering the device on which it's being played.
The sequencer generates a variety of files, the most important of which is the SFT file, which contains all of the application assets, dependencies, and configuration information. In some cases it may also contain multiple applications. Not surprisingly, this file can be quite large. There are some compression options, but a sound knowledge of your network and device performance is essential. The Icon file (.ico) that the sequencer creates is used to advertise the virtual application so that it acts as if it is locally installed.
The OSD file is also very important, and its options are endless. By default, this is an XML-based file that is used to tell the App-V client how to launch the virtual application. The OSD file can also be modified to configure and control how the virtual application launches and runs. I highly suggest you read the Sequencing Admin Guide and the Sequencing Best Practices document to familiarize yourself with the properties and values available through the OSD file.
Lastly, the new manifest.xml file contains package-based configuration information and can be used for integration with third-party ESD solutions and MSI deployments. The sequencer can also generate an MSI file for the virtual application package. This can be used to load the virtual applications onto standalone (serverless) clients and through an ESD system.
The sequencer itself is a wizard-based tool that walks you, the packager, through the process of installing an application and transforming it into a virtual application (see Figure 4). The first step allows you to configure default properties for the package. These properties, which are stored in the OSD file, include package name and comments. Some of the advanced settings let you specify the server from which to stream, the content directory, and what operating systems the package should support.
Figure 4 The sequencing wizard
The second step is to install, configure, and test the application. During installation, the sequencer captures all the changes being made to the local system, including file system, registry, and system. There are also a few utilities in this wizard that enable, for example, integration with Windows Update.
The next step is to configure file type associations and specify where the shortcuts should be placed. Standard placements include the Start menu, the desktop, and the quick launch bar, but you can also create customized locations.
Then you need to launch the application and configure the initial launch threshold. This is the step where App-V determines the initial portion of the application that needs to be delivered to the client to allow the application to begin.
To configure this initial code (typically referred to as Feature Block 1, or FB1), simply launch the application and use the most common features needed by users. For example, launch Word and then initiate the spell checker. Any DLLs, files, or registry keys called by the application during this phase are automatically designated as part of FB1. Any files, settings, or components not used at this point in time are added to FB2. When the application is then used, the client will receive a map of the SFT file that indicates where FB1 starts and stops and where other files exist in FB2 so the client can retrieve those files when needed by the application.
The final step in the sequencing process is to ensure everything is configured properly. The sequencer displays the dialog shown in Figure 5, which represents the SFT and allows you to make any final additions or changes to the package.
Figure 5 Confirming and tweaking a final package

Version 4.5
After two years in the making, App-V 4.5 is due to be released later this year. This is the first Microsoft version of the product to be released, and it promises to elevate application virtualization by introducing several key enhancements, such as Dynamic Virtual Application Interaction, extended scalability, and improved alignment with Microsoft Internalization and Security requirements.
Dynamic Virtual Application Interaction allows virtualized applications to interact with one another. This interaction is referred to as Dynamic Suite Composition (DSC). DSC doesn't replace the ability to add multiple applications to the same package. Instead, it offers a new way to integrate dependencies, middleware, and plug-ins shared among virtual applications.
Administrators can specify which virtualized applications can interact with each other. For example, suppose you have five Web applications that require the same Java version. In App-V 4.1, you would have to add that same Java version to each of the five separate packages. And suppose that Java version needed a patch. The administrator would then have to patch the five different packages. Using DSC, Java can be packaged once and then configured as a package to be used by all five Web applications. As a result, patching Java would only require the administrator to patch the Java package once.
The same scenario applies to middleware and plug-ins. I plan to blog about other use case scenarios as Microsoft gets closer to the release and finalizes any late additions.
Enhancements to scalability involve both streaming and back-end infrastructure. Back-end components have been modified to better support clustering and failover scenarios and streaming is more WAN and LAN friendly. The enhancements come from several key additions.
First is the new Streaming Server Component, which allows for streaming without the need for a back-end infrastructure of Active Directory and SQL Server. You still get all the great benefits of on-demand delivery and centralized updating of packages, but without the heavy back-end requirements. This will be widely used in branch office scenarios and for integration with third-party ESD solutions.
The App-V client has received a few enhancements as well. For instance, the client now stores all usage information locally so this information can be tracked whether the client system is on or off the network. The client cache has also been expanded and improved for better performance in scenarios with limited disk space. There is also now support for sequencing non-English applications, running App-V on non-English operating systems, and localization into several other languages.

Integrating App-V with Configuration Manager
A number of enhancements and new features in App-V 4.5 have been designed for integrating App-V into SCCM 2007 R2. As I already discussed, virtualization and streaming offer some capabilities for delivering applications that traditional software deployment tools don't provide. I'm not implying that App-V will replace these tools, rather that App-V can complement and extend them.
With this integration, you get all the scalability, reporting, device recognition, and WAN features of SCCM with all the streaming and isolation features of App-V. Here are some of the areas that benefit from integrating these two technologies:
Application Delivery The SCCM R2 integration supports all the capabilities of on-demand delivery, roaming applications, initial launch thresholds, and deployment of applications without altering client PCs.
Updates SCCM Distribution Points (DPs) can deploy just delta changes of virtual applications when packages are updated. This introduces the centralized ability to revert virtualized applications to previous versions with a single click.
Management Version R2 introduced a new virtual application advertisement wizard that allows administrators to deploy virtualized applications as well as traditional software packages and advertisements from a single console.
Packaging There is no need to repackage applications when integrating App-V with SCCM. The initial sequencing of an application needs to be done using the App-V sequence outside of SCCM, but administrators do have the ability to update existing packages using SCCM.
Licensing Virtual applications can be tracked for licensing and metering using the existing tools in SCCM.
SCCM offers a new method for deploying virtualized applications to App-V using the industry standard BITS protocol. Though SCCM DPs can stream, there are cases when streaming isn't the ideal method for deploying virtualized applications. When deploying via SCCM, you have two options. You can use standard streaming or you can use the Quality of Service (QoS) features of BITS to deploy in a much more controlled fashion. This is also useful in scenarios where you'd like to pre-load the cache prior to users launching the virtual application.
Machine Deployments SCCM provides the ability to deploy virtualized applications to specific machines as well as continue to support the user-based targeting approach of the App-V platform. This can be beneficial when you deploy virtual applications to laptops, kiosks, and lab machines. This is also handy for assisting with license control where your software may be licensed by device, not by named users.
Scalability The need to deploy two separate tools that have a great deal of overlap is a common concern. By integrating the scalability and WAN benefits of SCCM with the Isolation and Streaming benefits of APP-V, you can use your existing SCCM, which provides a single tool to handle both management and deployment without having to add more complexity.

Anthony Kinney is a Technical Sales Professional responsible for the Microsoft Desktop Optimization Pack. Anthony came to Microsoft in 2006 with the acquisition of Softricity. While at Softricity, Anthony wrote and designed the first training program for SoftGrid (now App-V). He can be reached at Anthony.kinney@microsoft.com.


Virtualization
Achieving High Availability for Hyper-V
Steven Ekren
At a Glance:
  • Consolidating servers using Hyper-V
  • Ensuring high availability of virtual machines
  • Setting up a Windows Server 2008 Failover

Contents
High Availability
Hosts and Guests
Guest Availability
Making VMs Highly Available
Prerequisites
HA Walk-Through
Considerations
Server virtualization is poised to make a significant impact in enterprise IT departments, and Hyper-V with Windows Server 2008 can make it a reality. The consolidation of servers onto fewer physical machines has huge advantages in resource and cost savings, but two key factors need to be considered during the planning process. Users have increasing expectations regarding the availability of their software, including both line-of-business (LOB) applications and tools such as messaging and collaboration platforms. Furthermore, problems or failure on servers can have a significantly greater impact on operations. Windows Server 2008 and Hyper-V provide solutions that can be implemented to provide high availability (HA) of virtual machines (VMs) as well as to the workloads being hosted inside the VMs.

High Availability
Availability means that users can access a system to do their work. With high availability, there is a significant expectation that users will always be able to access the system, because it has been designed and implemented to ensure operational continuity.
High availability for Hyper-V is achieved through the use of the Windows Server 2008 Failover Cluster feature. High availability is impacted by both planned and unplanned downtime, and failover clustering can significantly increase availability of virtual machines in both of these categories.
Virtual machines can be managed by the Failover Cluster, and the Failover Cluster can be used inside of virtual machines to monitor and move the workloads that are hosted in the VM. I'll describe both of these configuration scenarios in more detail, but the majority of this article will be about managing virtual machines Before you get started, however, you may want to take a look at the "Useful Hyper-V Terms" sidebar.

Hosts and Guests
Because there are multiple operating systems running on a Hyper-V system, it can get challenging to keep clear which layer, or OS, is being discussed. I use the term "guest" to refer to the OS and environment within a Hyper-V VM that is running in a child partition. I use the term "host" to indicate the physical machine, which is being managed by the OS on the Hyper-V parent partition.
Host availability addresses the problems that come with the "putting all your eggs in one basket" scenario that server consolidation can cause. The Windows Server 2008 Failover Cluster can be configured on the Hyper-V parent partition (host) so that Hyper-V child partitions (virtual machines or guests) can be monitored for health and moved between nodes of the cluster. This configuration has the following key advantages:
  • If the physical machine that Hyper-V and the VM are running on needs to be updated, changed, or rebooted, then the VMs can be moved to other nodes of the cluster. The VMs can then be moved back once the physical machine is returned to service.
  • If the physical machine that Hyper-V and the VMs are running on fails (perhaps a motherboard failure) or is significantly degraded, the other members of the Windows Failover Cluster will take over the ownership of the VMs and bring them online automatically.
  • If the VM fails, it can be restarted on the same Hyper-V server or moved to another Hyper-V server. Since this is detected by the Windows Server Failover Cluster, it will automatically take recovery steps based on the settings in the VM's resource properties. Downtime is minimized due to the detection and recovery automation.
Figure 1 represents what might happen in such situations. First, VM2 resides on Host A, then VM2 is moved to Host B. Note that the node that owns the SAN storage LUN 2 changes from Host A to Host B during this move. To ensure that your high availability solution will meet your availability needs, consider carefully where the VMs will be placed. You need to think about both capacity and performance.
Figure 1 A virtual machine and its storage move to a new host
The capacity of the nodes should be sufficient to host all the VMs and to allow for x number of nodes to fail or be taken out of active cluster participation. (X represents the number of nodes you want the cluster to be able to tolerate losing and still be able to host all of the VMs. When deciding on capacity, you can choose to have some nodes that don't regularly host VMs, holding them in reserve. Alternatively, you can spread the VMs across all nodes, ensuring that each node has enough extra capacity to be able to successfully take ownership and start the VMs if any x number of nodes were to fail.
For daily performance reasons, it may be desirable to spread the VMs across all nodes of the cluster. If nodes are held in reserve and don't host any VMs, then the nodes hosting the VMs will have more resources in use and that may reduce the performance of the VMs as well as the management partition. Spreading the VMs across the nodes reduces the load that each is carrying and can provide better performance for the VMs and management partitions. It can, however, make capacity planning more challenging. Management software, such as System Center Virtual Machine Manager 2008, can help by providing calculations on capacity for node failure and VM placement.

Guest Availability
Guest Availability focuses on making the workload that is being run inside a VM highly available. Common workloads include file and print servers, IIS, and LOB applications. Analyzing high-availability needs and solutions for workloads inside VMs is very much the same as on standalone servers. The solution will depend on the specific workload.
Some workloads can achieve high availability through Windows Network Load Balance (NLB), which allows multiple servers to be part of a pool with a common network name. Clients make a connection request using that virtual network name and the connection is made to one of the nodes of the NLB cluster. A typical scenario that uses NLB clustering is building Web farms with IIS, where each individual system has IIS with the same Web pages and access to the same data. NLB provides load balancing as well as the ability to remove servers from membership for maintenance or in case of a problem with the server, and therefore provides a level of high availability. If a Hyper-V VM is running Windows Server 2008 (or an earlier version of Windows Server that includes NLB), the guest can be a member of an NLB cluster with other guests on the same or different Hyper-V host(s).
Guests that are running Windows Server 2008 can use the Windows Failover Cluster feature to provide high availability for their workloads. There are several advantages of using Windows failover clustering inside of a guest (guest clustering):
Workload Health Monitoring The Windows Failover Cluster has a resource monitor that makes calls to the resource DLL associated with the cluster. Each resource has health monitoring that tests the application or service being managed by the resource to ensure that it is operating correctly. These checks are commonly called the isAlive/looksAlive checks. If the resource fails one of these calls, the resource itself will fail. Depending on how its properties are configured, the resource may try to restart the service or application, or it may be moved to another node in the Windows Failover Cluster.
Virtual Machine Maintenance If the configuration of the VM needs to be changed, or if the OS or software needs to be updated or changed, the workload can be moved to another node of the cluster and the VM either shut down or updated with minimal interruption to the end users.
Host Machine Maintenance If the physical machine hosting a Hyper-V VM needs maintenance or software updates and other members of the Windows Failover Cluster are located on different Hyper-V hosts, the workload in the VM can be moved to another node of the cluster and that VM can be shut down to accommodate the changes or reboots of the physical server.
Virtual or Host Machine Failure If there is a failure of the physical Hyper-V host or the virtual machine guest, the other nodes of the Windows Failover Cluster will detect that the cluster member is no longer responding or participating in the cluster and the surviving nodes will bring online the applications or services that had been running on the failed VM.

Useful Hyper-V Terms
The following is a list of terms that help define components or functions of a highly available Hyper-V system (host cluster).
Parent Partition All operating systems that run on a hypervisor server have allocated hardware resources that include RAM, CPU and other system components. In Hyper-V, the partition that manages the configuration of the hypervisor and system resources is often referred to as the parent partition. When the Hyper-V role is configured and the server rebooted, the Windows Server 2008 installation that was the native OS on the system becomes the OS in the Parent Partition of the Hyper-V server.
Child Partition An isolated environment on a Hyper-V server that is configured to hold a guest operating system and to provide hardware resources to that operating system.
Hyper-V Host The physical server that is hosting Hyper-V and the operating system running inside the parent partition.
Hyper-V Virtual Machine The Hyper-V configuration information and the data that is used to boot and run a Hyper-V child partition. This includes the configuration information to create the child partition and the VHD files or passthrough disks that contain its data.
Passthrough Disk A storage device that is exposed in Disk Management as a physical disk assigned for exclusive use by a Hyper-V guest. The Hyper-V guest mounts the disk and uses it like a locally connected storage device.
Virtual Hard Disk (VHD) A file that is connected to by a Hyper-V child partition that is exposed to the operating system as a storage device (disk). The .vhd file is located on a storage device mounted by the parent partition and can be a direct attached storage device, or storage that is connected to over a SAN, NAS, or SMB.
Making VMs Highly Available
Configuring a virtual machine to be highly available is as simple going through the HA Role Wizard under Failover Cluster Management. Hyper-V virtual machines have several key components that need to be considered when they are being managed as highly available. Let's take a look some of the important concepts and general prerequisites.
Failover Cluster Nodes Each physical server that is part of a failover cluster is called a node. For host clustering, the Failover Cluster service runs in Windows Server 2008 on the parent partition of the Hyper-V system. This allows the VMs that are running in child partitions on the same physical servers to be configured as highly available virtual machines. The virtual machines that are configured for HA will be shown as resources in the Failover Cluster Management console.
HA Storage Highly available virtual machines can be configured to use Virtual Hard Disks (VHDs), passthrough disks, and differencing disks. To enable the movement of virtual machines between failover cluster nodes, there needs to be storage (appearing as disks in Disk Management) that can be accessed by any node that might host the VM and that is managed by the Failover Cluster service. Passthrough disks should be added to the failover cluster as disk resources, and VHD files must be on disks that are added to the failover cluster as disk resources.
Virtual Machine Resource This is a failover cluster resource type that represents the virtual machine. When the virtual machine resource is brought online, a child partition is created by Hyper-V and the OS in the virtual machine is started. The offline function of the virtual machine resource removes the VM from Hyper-V on the node where it was being hosted and the child partition is removed from the Hyper-V host. If the virtual machine is shut down, stopped, or put in saved state, this resource will be put in the offline state.
Virtual Machine Configuration Resource This is a failover cluster resource type that is used to manage the configuration information for a VM. There is one virtual machine configuration resource for each virtual machine. A property of this resource contains the path to the configuration file that contains all the information needed to add the virtual machine to the Hyper-V host. Access to the configuration file is required for a virtual machine resource to start. Because the configuration is managed by a separate resource, a VM resource's configuration can be modified even when the VM is offline.
Virtual Machine Services and Applications Group For a service or application to be made highly available through failover clustering, multiple resources must be hosted on the same failover cluster node. To ensure that these resources are always on the same node, and they interoperate appropriately, the resources are put into a group that the Windows Server 2008 Failover Cluster refers to as "Service or Application." The virtual machine resource and the virtual machine configuration resource for a VM are always in the same Services or Applications group. There may also be one or more physical disk (or other storage type) resources containing VHDs or configuration files or passthrough disks in a Service or Applications group.
Resource Dependencies It is important to ensure that the virtual machine configuration resource is brought online before the virtual machine resource is brought online (started), and that the virtual machine configuration resource is taken offline after the virtual machine resource is taken offline (stopped). Setting the properties of the virtual machine resource so that it is dependent on the virtual machine configuration resource ensures this online/offline order. If there is a storage resource that contains the file for the virtual machine configuration resource or virtual machine resource, then the resource should be made dependent on that storage resource(s). For example, if the virtual machine uses VHD files on disk G: and disk H:, the virtual machine resource should be dependent on the configuration file resource and the resource for disk G: and the resource for disk H:.

Prerequisites
Here are the three prerequisites for making Hyper-V virtual machines highly available using the Windows Server 2008 Failover Cluster feature:
  1. The Windows Server 2008 Failover Cluster feature must be configured for each node of the cluster. For more information on configuring and managing Failover Clusters, see the "Hyper-V Resources" sidebar.
  2. The Hyper-V role must be installed. Hyper-V updates should be installed and the role configured for each node of the failover cluster (again, see the "Hyper-V Resources" sidebar). Hyper-V has an update package that installs the Hyper-V server components and another one that installs the Hyper-V management console. Once the update is installed for the Hyper-V server components, the role can be added through Server Manager or ServerManagerCMD.
  3. You need to have shared storage available to the virtual machines. The storage could be managed by the failover cluster as a built-in physical disk resource type, or you could use a third-party solution to manage the shared storage. Of course, the third-party solution must support Windows Server 2008 Failover Clusters.

HA Walk-Through
Now let's see how a high availability solution is set up. The first step is to set up a virtual machine. On one of the nodes of the failover cluster that has the Hyper-V role installed, configure a virtual machine using the Hyper-V Manager (see Figure 2). This could be a new VM that is configured manually, or you could import a pre-existing one. The VHDs should be located on a disk that is managed by the Windows Server 2008 Failover Cluster and that is currently online on the node the VM is being configured on.
Figure 2 Configuring a virtual machine
Now put the virtual machine in the stopped state by shutting down, turning off, or saving state. Only virtual machines in the stopped state can be configured for management by the failover cluster.
Open the Failover Custer Management console (shown in Figure 3) on any server that is running the Windows Server 2008 Failover Cluster role, or on a Windows Vista client running the Remote Server Administration Tools (RSAT). Connect to the failover cluster by choosing the Manage a cluster… action and then selecting a node or the cluster name, or by selecting the option to connect to the cluster on the node that the console is running on.
Figure 3 A virtual machine in Failover Custer Management
From the Failover Custer Management console, select the Configure a Service or Application… action. This will open the High Availability Wizard, which takes you through configuring services, applications, or virtual machines to be managed by the failover cluster. In the Select Service or Application page of the wizard, choose Virtual Machine, and then select Next.
The Select Virtual Machine page will display all the virtual machines that are configured on any node of the failover cluster. Select a virtual machine, then select Next. The Confirmation page of the wizard will show any warnings or errors. In this step, the virtual machine configuration is checked to verify that it can be configured as an HA resource and that the nodes should be able to host it. Selecting Next here adds the virtual machine to the failover cluster as a highly available resource.
The Summary page provides information about the results of adding the virtual machine as a highly available resource, including any warnings. A View Report… button can show you details of the tasks that were done to make the virtual machine highly available, as well as any warnings or errors. Finally, select Finish to close the High Availability Wizard.
As the window in Figure 3 shows, the Failover Custer Management console will list an object with the default name of Virtual Machine(x) in the left pane under the cluster name and Services or Applications. Choose the virtual machine from that tree structure and the resources that are part of that Services or Applications group will show in the center pane of the console. If any other virtual machine had its files on the same storage as the one that was chosen, it will also be added to the group. Virtual machine and virtual machine configuration resources will be shown for each virtual machine that was put in the group.
The information pane for the Services or Applications group displays the Status, Alerts, Preferred Owners, and Current Owner information for the group. The Owner node is the node where the virtual machine is currently configured or running. Select the Move Virtual Machine(s) to another node action to have the virtual machine taken offline and then brought online on another node. It's generally a best practice to move a VM to each node that may host it in the failover cluster to verify that the move is successful and the VM will start and run.

Hyper-V Resources
  • Description of the Update for the Release Version of the Hyper-V Technology for Windows Server 2008
  • Hyper-V Step-by-Step Guide: Testing Hyper-V and Failover Clustering
  • Introducing Windows Server 2008 Failover Clustering (by Chuck Timon, TechNet Magazine, July 2008)
  • The NLB Host Does Not Converge as Expected on Windows Server 2008 Hyper-V Virtual Machines
  • Win2008 Cluster Webcasts
  • The Microsoft Support Policy for Windows Server 2008 Failover Clusters
Considerations
Here are some key points to keep in mind when you are setting up virtual machines for high availability:
Storage If virtual machines have VHD files on the same shared disk, even if they are located on different volumes on the same disk, they will be placed in the same Service or Application group. One of the advantages of sharing the disk is that it enables better use of available shared storage space.
However, the trade-off is that anytime a virtual machine is moved, whether due to automated recovery from a problem with the virtual machine or because of an administrator's choice, all the virtual machines in the group will be moved.
Driver Letters and GUIDs Volumes can be created without assigned drive letters. Virtual machines can use those volumes and the volumes can be managed by the failover cluster. If a disk resource has volumes that use GUIDs instead of drive letters, the GUID will be shown in Cluster Management. When you are creating virtual machines and specifying the path for the VHDs, it is very important to make sure that the GUID in the path matches the GUID that is showing in Cluster Management for the volume. If it does not match, the virtual machine may not start successfully (online) on other nodes of the failover cluster.
There are several situations that might cause the GUIDs to not match. If a volume was brought online on nodes before they were added as a failover cluster-managed disk resource, the volume can have a different GUID on each node. It's also possible for a volume to have multiple GUIDs on a single node. When a disk is added to a failover cluster as a physical disk resource, the volume GUIDs that are in use on the node that has the disks online will be noted in the properties for the disk resource.
The GUIDs for the volumes will be added to a node when the disk resource is brought online. This ensures that the particular GUID that the failover cluster has noted for the volume is a valid path on any node that brings the disk online. That node may have other GUIDs that are also associated with the same volume. Therefore, a user could find a GUID that is valid for the volume on that node, but it is not the same GUID that the failover cluster is ensuring is used for the volume on other nodes. The symptom of this problem is that a virtual machine resource, usually the Configuration resource, fails to come online and displays an error message indicating that the path is invalid. The path in the error message shows the GUID that is not the cluster-managed GUID for the volume.
Mount Points Volumes that are mounted to a folder in another volume, instead of assigning a drive letter or using a GUID, are valid for use with Hyper-V and failover clusters. Since both the volume that is mounted and the volume that is hosting the mount point must be on the same failover cluster node, you must have all the disks that are part of the mount point in the same Failover Cluster Service or Application group.
If the volumes are on the same disks, obviously this is not an issue. However, it is definitely an issue if the volumes are on different disks. It's equally obvious, but still worth mentioning, that both the volume that is mounted and the host of the mount point must be shared storage that is configured to be managed by the failover cluster.
Differencing Disks All VHD files that are part of differencing disks need to be on shared storage in the same Services or Applications group as the VM using the differencing disks. In its simplest configuration, a differencing disk involves two VHDs. One VHD is the parent, and it has a set of data that is used as a base. The other VHD is a child associated with the parent.
When first used, the differencing disk appears just like the parent. If data is located on the parent, it's read from that VHD. All writes happen on the child VHD. If data is located on the child, then a read for that data would reference the child VHD.
If a VM is somehow configured so that the child VHD is on shared storage but the parent VHD is either not on the shared storage in the same group or on a locally attached storage device, then the VM would fail to start (online) if it is moved to another node. The High Availability Wizard should check to make sure this is configured correctly in the VM and provide an error message if it detects this problem, but it's worth noting the requirement in the case where the VM configuration has changed.

Steven Ekren is a Senior Program Manager with the Windows Server Failover Cluster and High Availability team. Steven spent 12 years with Microsoft Support where he assisted enterprise customers with implementing and troubleshooting Windows Server Failover Clusters and Virtualization technologies including Windows Hyper-V, System Center Virtual Machine Manager, Microsoft Virtual Server, and Microsoft Virtual PC.


Virtualization
Backup and Disaster Recovery for Server Virtualization
Adam Fazio
At a Glance:
  • Considerations for disaster recovery planning
  • High availability solutions for disaster recovery
  • Backing up and restoring with Windows Server BackupItem

Contents
Disaster Recovery Planning 101
Disaster Recovery and Virtualization
Physical-to-Virtual Conversion
Virtual Machine Snapshots
Backing up Hyper-V
Windows Server Backup
Backing up VMs with WSB
Considerations
Restoring VMs with WSB
Data Protection Manager
Scripted Backups
DiskShadow
Wrap-Up
As server virtualization technology evolves and industry adoption increases, organizations are recognizing benefits reaching far beyond the most popular virtualization justification: reduced infrastructure costs and increased IT agility. The next frontier is using the virtualization platform as a way to enable or enhance disaster recovery (DR) strategies.
Why is DR preparedness consistently one of the hottest topics facing the IT industry? Studies suggest that companies lose an average of $80,000 to $90,000 per hour of downtime, and that very few companies suffering from a catastrophic data loss survive long-term. This article will provide an introduction to DR using the Microsoft virtualization platform, as well as a deeper look into backup and restore options and considerations for Windows Server 2008 Hyper-V.

Disaster Recovery Planning 101
DR is the process of restoring critical services in the event of an outage and should be a part of every company's business continuity plan, which defines how the company will continue to function during or following such a disaster. These plans are the cornerstone of any DR initiative.
Some vendors claim that their DR automation technology minimizes or eliminates the need for a detailed and well-rehearsed plan. While it's fair to say that automation can improve recovery times and lessen the dependence on human intervention, let's pause for a public service announcement: you cannot successfully mitigate a disaster with technology alone. The people and processes are always as important as the technologies.
In fact, you will find it nearly impossible to select the right technologies without first knowing all the constraints and objectives that arise from the DR planning process. It's outside the scope of this article to define an entire DR plan, but I do want to emphasize those elements necessary to select the right technologies and implementations. So let's quickly outline some critical technology drivers within a DR plan.
Service Definitions and Prioritization What exactly defines the entire service you are trying to protect and how critical is it to the organization? Figure 1 shows examples of company services that would likely be included in any DR plan.
Figure 1 Service definitions and prioritization example
Service | Primary Components | Dependencies | Business Use | SLA
Public Web site | Network load balancer, three Web servers, two database servers | DNS, network, firewall, directory, storage area network (SAN) storage | Product purchasing and order tracking; eCommerce; customer support portal; company information | 99.99%
Financial system | Two database servers, one application server | DNS, network | Recording company revenue, as required by laws and regulations | 99.99%
E-mail system | Three e-mail servers, one Web server | DNS, network, firewall, directory | Company communication; customer support | 99.5%
Once you've defined the services, you can begin to identify which systems and dependencies to target for what kind of DR strategies. It could be that after you look at the entire set of services and dependencies, you find you need to consider a few different levels of DR capability, as a single DR solution for all mission-critical services would be too expensive and complex.
Service Level Agreements (SLAs) An SLA is an agreement or contract between the service provider (IT) and the customer (the organization) that defines the availability targets for a given service. These can be very lengthy or quite short and sweet; for example, "The e-mail system will be available 99.95 percent of the time during general business hours and 98 percent of the time during non-general business hours, excluding scheduled maintenance windows, measured monthly." Usually SLAs are broken out into tiers that IT services can be assigned to, measured over a pre-defined period of time.
Operating Level Agreements (OLAs) OLAs basically describe the agreements between different IT groups working to support an SLA, including the process and response times for delivery of their services. Suppose you have a mission-critical Web site with an SLA target of 99.99 percent, but a database it depends on for its content has only 95 percent as its availability target. The OLA helps clarify these mismatches and align IT teams toward the same goal.
Recovery Point and Recovery Time Objectives (RPOs/RTOs) An RTO defines how long a service can be unavailable before there is a break in continuity, while an RPO defines what the organization considers to be an acceptable level of data loss. Therefore, if a service has an SLA of 99 percent measured monthly, it has an RTO of 7 hours, 18 minutes. If you combine that with an RPO of, say, 24 hours, you can now accurately define your backup techniques and schedules.
Data Retention Policies An organization's data retention polices specify exactly how long to keep the backups and where to store them. They are usually driven by legal and regulatory requirements.
Data Categorization You should also think about the nature of the data. If you put your data into categories, you can quickly see that not all data requires the same level of DR consideration. For instance, a single database may have different availability requirements than an Active Directory with multiple domain controllers, each containing a replica of the directory. Similarly, file-server data may have very different restore procedures than CRM data.
Disaster Scenarios It's important to define all the scenarios you want to plan for, since each one will have different restore procedures, business impact, and associated costs. It's helpful to look at all the possible scenarios, and then decide which ones you want to target when working on DR planning for your environment:
  • Loss of an entire site
  • Loss of a single datacenter
  • Loss of a system (operating system or hardware failure)
  • Loss of data (data deletion or corruption)
  • Loss of a critical dependency
Clearly there are very different considerations for recovering from the loss of an entire site as opposed to a single system. You will also want to define recovery thresholds based on your SLAs. For instance, let's say an entire site is offline due to a major ISP network outage. If the SLA for the affected service is 8 hours for service restoration and 48 hours for data restoration, perhaps you would perform service failover procedures to your backup site but actually not go through the data recovery process, as you would anticipate failing back to the production site fairly quickly.
Whew! All that work and we haven't even talked about technology yet! The criticality of planning is not to be underestimated. A DR implementation with no documented plan is just a "DR hope."

Disaster Recovery and Virtualization
OK, now that we have the basics of DR planning down, what does virtualization add to the picture? Many companies report service restoration times with virtualized servers in minutes as opposed to days or weeks for their physical counterparts. Because the entire server operating system is now just a set of files, abstracted from the underlying physical hardware, new doors open when considering recoverability.
A prevalent theory today is that some or all DR goals can be met with High Availability (HA) solutions. The idea behind this is that if you have cluster nodes in separate physical locations with data synchronized between the sites, in the event of a failure the passive node can resume operations and you can recover in near real time.
This is true, but if you recall the disaster scenarios defined earlier, it is clear that this is not a solution for all of them. You need a combination of technologies to prepare for all of the scenarios, and this generally includes some type of regular backups. HA does not protect against all possible outages, and it does not completely obviate the need for some type of backup strategy.
HA with Hyper-V requires that you carefully plan the storage layer because this is a critical factor in order for recoverability to take place. For example, a 2-node Hyper-V cluster with shared storage still has the storage sub-system and data as a single point of failure, even if the cluster nodes are in separate datacenters.
However, you should know that the same 2-node Hyper-V cluster with non-shared storage is able to survive storage or data loss on one or the other nodes. This does require replication technologies in order to keep the storage in sync, and it introduces complexities as well (see Figure 2).
Figure 2 Multi-site Hyper-V cluster
There are some very interesting developments in the area of data replication and synchronization, but it's not something Microsoft currently provides. On the Windows Server 2008 Multi-Site Clustering page (microsoft.com/windowsserver2008/en/us/clustering-multisite.aspx), the showcased partners are worth a look. Another resource is the Windows Server Catalog (see windowsservercatalog.com), which lists storage vendors with replication technologies certified for Windows Server 2008.
As you can see, there are many possible HA and storage configurations to consider. Again, this is why you need to have your business requirements defined and allow those to drive the technical requirements, rather than the other way around.

Physical-to-Virtual Conversion
Virtualization clearly offers some unique recovery agility, but what about physical systems that are not good virtualization candidates? Included in System Center Virtual Machine Manager (SCVMM) is the ability to perform physical-to-virtual (P2V) conversions of running Windows servers that result in a bootable Hyper-V virtual machine (VM) that is an exact replica of the physical source server. Now you can replicate this VM just like its virtualized counterparts across campus or across the country and achieve similar recovery times.
This approach is different than traditional bare-metal restore in that your recovery location no longer needs the same number or type of physical systems as your production location. So you can over-subscribe your recovery hardware and scale it out as needed depending on the impact of the disaster.
Although SCVMM does not include a scheduler for P2V conversions, since the GUI runs entirely on top of Windows PowerShell this can be easily scripted using the New-P2V cmdlet. In fact, all wizards in SCVMM will show the code they are using to execute a job, and you can copy the code from a test P2V in your environment and modify it for future automated use. Figure 3 shows some sample code; you can run the SCVMM P2V wizard in your environment to get a unique, customizable Windows PowerShell script.
Figure 3 Code produced by the SCVMM P2V wizard
$Credential = get-credential

New-MachineConfig -VMMServer <VMM SERVER> -SourceComputerName "<SOURCE P2V SERVER>"
-Credential $Credential -RunAsynchronously

$VMHost = Get-VMHost -VMMServer <VMM SERVER> | where {$_.Name -eq "<TARGET HYPER-V HOST>"}
$MachineConfig = Get-MachineConfig -VMMServer <VMM SERVER> | where {$_.Name -eq "<SOURCE P2V SERVER>"}

New-P2V -VMMServer <VMM SERVER> -VMHost $VMHost -RunAsynchronously -JobGroup
e823f50d-dbc7-4a41-9087-fb01bb44dc26 -SourceNetworkConnectionID "00:14:D1:3C:66:2F"
-PhysicalAddress "00:14:D1:3C:66:2F" -PhysicalAddressType Static -VirtualNetwork "External"
-MachineConfig $MachineConfig

$VMHost = Get-VMHost -VMMServer <VMM SERVER> | where {$_.Name -eq "<TARGET HYPER-V HOST>"}
$MachineConfig = Get-MachineConfig -VMMServer <VMM SERVER> | where {$_.Name -eq "<SOURCE P2V SERVER>"}

New-P2V -VMMServer <VMM SERVER> -VMHost $VMHost -RunAsynchronously -JobGroup
e823f50d-dbc7-4a41-9087-fb01bb44dc26 -VolumeDeviceID "C" -Dynamic -IDE -Bus 0 -LUN 0 -MachineConfig $MachineConfig

$Credential = get-credential
$VMHost = Get-VMHost -VMMServer <VMM SERVER> | where {$_.Name -eq "<TARGET HYPER-V HOST>"}
$MachineConfig = Get-MachineConfig -VMMServer <VMM SERVER> | where {$_.Name -eq "<SOURCE P2V SERVER>"}

New-P2V -Credential $Credential -VMMServer <VMM SERVER> -VMHost $VMHost -Path
"C:\ProgramData\Microsoft\Windows\Hyper-V" -Owner "DOMAIN\username" -RunAsynchronously -JobGroup
e823f50d-dbc7-4a41-9087-fb01bb44dc26 -Trigger -Name "<SOURCE P2V SERVER>" -MachineConfig
$MachineConfig -CPUCount 1 -MemoryMB 512 -RunAsSystem -StartAction NeverAutoTurnOnVM
-UseHardwareAssistedVirtualization $false -StopAction SaveVM

Virtual Machine Snapshots
Although not technically a backup, a VM snapshot provides a point in time to which you can revert back using differencing disks and a copy of the VM configuration file. If the disaster involves accidental data deletion inside the VM, this can be considered a DR feature as the VM can be rolled back to the snapshot, undoing the damage. (We'll take a look at Volume Shadow Copy Service, or VSS, snapshots later.)

Backing up Hyper-V
Host-Based Backups One exciting benefit of server virtualization is the prospect of no longer having to individually back up the virtualized systems. Now that these systems are simply files living on a host's file system, you can just back up the files and call it a day, right? Not exactly. Because these are live computers consisting of in-memory data, data on disk, system configurations, and open files, there are a few things to consider. So how do we ensure backup data consistency given all these moving parts?
A significant improvement to the Windows Server backup story came with Windows Server 2003 and the advent of VSS, which provides a standard set of extensible APIs that VSS writers (hooks in applications and services that help provide consistent shadow copies) use in order to create backups of open files and applications. With the help of the VSS service, providers, and writers, the backup application can generate a point-in-time copy of a volume very quickly, one that the application is aware of and can process appropriately.
Hyper-V comes with its own VSS writer that allows software makers to create compelling backup solutions. The writer lets backup applications achieve host-based VSS backups of running VMs. If the operating system running within the VM has the Hyper-V Integration Components installed as well as the VSS service (available in Windows XP SP1 and Windows Server 2003 and later), the host-based backup will occur as if it were run inside the guest; the backup will be performed with the VM running and the data will be consistent (see Figure 4).
Figure 4 VSS backup
However, if the guest operating system does not support the Integration Components or VSS, the backup process requires that the guest machine be put into a saved state and that a host-based VSS snapshot is taken of VM data files that can be used for point-in-time recovery. Saved-state VSS snapshots will incur some VM downtime (this can typically be limited to 5-10 minutes), with full backup-to-tape procedures taking place against the VSS copy of the data.
Guest-Based Backups In a physical environment, servers and applications need to be backed up on an individual basis, and such backups can certainly continue in a virtualized datacenter. In this situation, the same considerations need to be taken into account when backing up a VM, such as network capacity requirements for network-based backups and performance impact to the system during the backup window. With guest-based backups, you can choose to have a dedicated physical NIC in the host that is bound to a virtual network that all guests use.

Windows Server Backup
Included with Windows Server 2008 is the VSS-capable Windows Server Backup (WSB), which can be used to perform Hyper-V host- and guest-based backups of your VMs. Because it's fully VSS-capable, it can perform host-based backups of your running VMs, which of course is preferable.
But if you have VMs without the Integration Components installed, VSS will not be used. In that case, you have a couple of options from which to choose. You can still use WSB to back up a VM that does not have the Integration Components installed, which means the VM's state will be saved and then the backup will grab the VM's virtual disks and configuration files.
However, this may not be desirable with an application such as Exchange because the application will not be aware a backup has run and application logs will not be truncated. Moreover, downtime will occur on the VM, which will vary depending on how long the backup takes.
Alternatively, a backup can be run from inside the VM just as if it were a physical machine using either NTBackup or WSB, depending on the VM's OS. Let's see how to use WSB for supported guests that have the Integration Components installed.

Backing up VMs with WSB
Hyper-V does not automatically register its VSS writer for use with WSB. You must manually add the registry key and value shown in Figure 5 before WSB will support a Hyper-V backup. You can add them via the command line, like so:
reg add "HKLM\Software\Microsoft\windows nt\
  currentversion\WindowsServerBackup\Application
  Support\{66841CD4-6DED-4F4B-8F17-FD23F8DDC3DE}"
reg add "HKLM\Software\Microsoft\windows nt\
  currentversion\WindowsServerBackup\Application
  Support\{66841CD4-6DED-4F4B-8F17-FD23F8DDC3DE}" /v
  "Application Identifier" /t REG_SZ /d Hyper-v
Figure 5 Key and value for registering the Hyper-V VSS writer
Path | Registry Key or Value | Type
HKLM\Software\Microsoft\windows nt\currentversion\WindowsServerBackup\ Application Support\ {66841CD4-6DED-4F4B-8F17-FD23F8DDC3DE} | Key | n\a
HKLM\Software\Microsoft\windows nt\currentversion\WindowsServerBackup\ Application Support\{66841CD4-6DED-4F4B-8F17-FD23F8DDC3DE}\Application Identifier | Value | REG_SZ (Hyper-V, for example)
This does not require a reboot, as WSB searches for this key/value at the backup runtime. The following command will show if the entry has been set:
reg query "HKLM\Software\Microsoft\windows nt\
  currentversion\WindowsServerBackup\Application
  Support\{66841CD4-6DED-4F4B-8F17-FD23F8DDC3DE}" /s
Here's how to install WSB. Click Start | Server Manager. In the left pane, click Features and then click Add Features in the right pane. On the Select Features page, expand Windows Server Backup Features and select the checkboxes for Windows Server Backup and Command-line Tools. Now follow these steps to configure your backup:
  1. Go to Start | Administrative Tools | Windows Server Backup.
  2. If backing up a remote host, choose Connect to Another Computer and type in the Hyper-V host.
  3. Choose either Backup Once or Backup Schedule.
  4. Select Backup configuration—Full server or Custom. If you chose Custom, make sure you get all volumes that contain data related to the VM you are backing up, including VM configuration data, virtual disks, and snapshots.
  5. Choose the location to store the backup.
  6. Choose either VSS Full or Copy Backup. For host-based backups where no other backups are occurring with the VMs, choose VSS full backup.
  7. Once you have confirmed the details, select Backup.
Considerations
  • You must back up all volumes related to a VM, including Virtual Hard Disks (VHDs), VM configuration files, and snapshots.
  • If you are creating a Backup Schedule, you must use a dedicated local volume that will be formatted and used exclusively by WSB. In contrast, if you are performing a Backup Once job, you can store the backup on a non-dedicated local volume, a removable device, or a network share.
  • If the Integration Components are not installed within the VM being backed up, WSB will save the state of the running VM in order to ensure backup data consistency.
  • Once completed, the backup set is portable and can be used with any Hyper-V host.
Restoring VMs with WSB
Although WSB does have the ability to restore individual files, this feature does not use VSS and therefore may result in an inconsistent restore if the VM was running at the time of backup. To restore running VMs, you need to restore the entire volume(s).
In order to do this, you must go to Start | Administrative Tools | Windows Server Backup and, in the Actions Pane, select Recover. Choose the server from which to recover data (the one where the WSB backup data is located), then choose the date from which to restore the data. Now you can select the recovery type.
Here's where you have to make a decision. If you need the entire VM, including its configuration, snapshots, and virtual disks (in the event of a complete host failure, for example), select Application Restore and then Hyper-V, as shown in Figure 6. In this case, you do not have the option to restore individual files. You will have to restore everything included in that backup set. Note that this will not overwrite existing Hyper-V and VM configuration data that has changed since the backup.
Figure 6 Restoring a Hyper-V backup
If you need only the VHD itself and the configuration data and snapshots for the VM are healthy, you can select Files and folders and pick the individual VHD file you need. Note that this process does not use the VSS writer; the VM should have been backed up with this in mind, with its state saved first.
If you've suffered total system and data loss and need to recover the Hyper-V host itself, including the Windows Server 2008 operating system and all the VMs running therein, you must boot to the Windows Recovery Environment and perform your restore from there. This can be done from the Windows Server 2008 setup disk or a pre-configured disk partition.

Data Protection Manager
We have taken a look at the backup and restore steps and considerations for Hyper-V hosts and guests using the reliable, free, built-in WSB, but WSB is not an enterprise-class data protection solution. Where it leaves off, Data Protection Manager (DPM) 2007 SP1 picks up. Currently scheduled for release in late 2008, DPM SP1 will support Hyper-V and offers some compelling features:
  • Single management console for all Hyper-V hosts and guests.
  • Continuous data protection, which takes VSS-based snapshots at intervals of up to 15 minutes, snapping only the changed bits in the process.
  • Hyper-V cluster awareness that allows the backup to follow the VM as it moves between cluster nodes.
  • DPM server to DPM server replication.
  • Support for disk and tape media (disk to disk, disk to tape, or disk to disk to tape).
  • Backup and restore capabilities across the whole spectrum of data, which includes Hyper-V Hosts and Guests; agentless VSS backups of running Guests; support for restoring individual VMs; failover cluster data; and best-in-class application-specific features for SQL Server, Exchange, SharePoint, Hyper-V, and Virtual Server.
  • Pre- and post-backup scripting.
If you currently use a third-party backup solution, watch for updates to the application; most vendors are working hard to get Hyper-V host-based solutions to market.

Scripted Backups
WSB includes a command-line interface, WBadmin.exe, as well as a set of Windows PowerShell cmdlets for use with scripting. When using these, the same backup rules apply as outlined previously, along with the need to manually register the Hyper-V VSS writer through the registry.
Figure 7 shows some WBAdmin commands. For full WBAdmin documentation, see go.microsoft.com/fwlink/?LinkId=124380. As you can see, there is nothing in WB​Admin to configure the backup policy itself, but there is a Windows PowerShell snap-in to manage these settings. You can test to see if this snap-in is registered with the following command:
Get-PSSnapin -Registered
Figure 7 WBAdmin commands
WBAdmin Commands | Description
ENABLE BACKUP | Enables or modifies a scheduled daily backup.
DISABLE BACKUP | Disables running scheduled daily backups.
START BACKUP | Runs a backup.
STOP JOB | Stops the currently running backup or recovery.
GET VERSIONS | Lists details of backups recoverable from a specific location.
GET ITEMS | Lists items contained in the backup.
START RECOVERY | Runs a recovery.
GET STATUS | Reports the status of the currently running job.
GET DISKS | Lists the disks that are currently online.
START SYSTEMSTATERECOVERY | Runs a system state recovery.
START SYSTEMSTATEBACKUP | Runs a system state backup.
DELETE SYSTEMSTATEBACKUP | Deletes system state backup(s).
And you can use the following to load a snap-in named Windows.ServerBackup:
Add-PSSnapin windows.serverBackup
Once this is loaded, you have access to the Windows PowerShell cmdlets for WSB, as shown in Figure 8. For a verbose description of each cmdlet, run this command:
Get-Command -PSSnapin windows.serverBackup | select name | get-help –full
Figure 8 Windows Server Backup cmdlets
Cmdlet | Description
Add-WBBackupTarget | Adds a backup target to the backup policy.
Add-WBVolume | Adds a volume to the backup policy.
Get-WBBackupTarget | Gets backup targets from a policy.
Get-WBDisk | Gets all disks.
Get-WBPolicy | Gets current backup policy.
Get-WBSchedule | Gets backup schedule in policy.
Get-WBSummary | Gets backup history and summary.
Get-WBVolume | Gets all volumes.
New-WBBackupTarget | Creates a new backup target.
New-WBPolicy | Creates a new empty policy.
Remove-WBBackupTarget | Removes a backup target from the policy.
Remove-WBPolicy | Deletes the backup policy.
Remove-WBVolume | Removes a volume from the policy.
Set-WBPolicy | Saves the WBPolicy object to create a scheduled backup.
Set-WBSchedule | Sets the schedule to the backup policy.

There is another utility built-in to Windows Server 2008 that can also make use of the Hyper-V VSS writer and adds some flexibility to your scripting options. DiskShadow.exe allows a shadow copy to be made and mounted as a drive, which lets administrators make a more selective backup than is possible using WSB. And it's important for you to remember that DiskShadow does not accept Windows PowerShell pipeline input; instead, it requires commands to be passed to it through a script, which might look something like the following:
Delete Shadows Volume C:
Set Context Persistent
Begin Backup
Writer Verify {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}
Add Volume C: ALIAS MyShadow
Create
End Backup
Expose %MyShadow% X:
Exit
This script first deletes any existing shadow copies of drive C:, then ensures that the shadow copy will persist after DiskShadow has run. Next it creates a transactional block—if any of the steps fail, the whole process fails. In this block, DiskShadow verifies that the writer for Hyper-V is loaded and adds drive C: to the list of drives to be backed up.
Drive C: will get a GUID in order to identify it, and this GUID will be stored in an environment variable that is named "MyShadow." Once that is complete, the shadow copy is created.
The backup is exposed as drive X: using the environment variable. Various things can be done with the data on X:, and then DiskShadow can be run again with the command Unexpose X: to remove the drive.
Note that restoring Hyper-V VMs that have been backed up via DiskShadow is currently a manual process (the VM must be recreated, SnapShots are not preserved, and so forth). While this has some obvious disadvantages, the data is protected.
Wrap-Up
DR can be an arduous process that seemingly never ends. But server virtualization brings new possibilities, due to both the technologies and the resulting lower-cost entry point. Microsoft is providing not only virtualization but an entire ecosystem. Together, the server virtualization platform and the System Center family offer more holistic solutions to the ever-growing complex challenges organizations face, including DR.
I would like to thank James O'Neill for his contribution to this article.

Adam Fazio began in Microsoft Consulting Services more than 2 years ago and now resides within the U.S. Public Sector practice. He has serviced the IT industry for more than 10 years working on a variety of infrastructure projects and datacenter operations for Fortune 100s to Internet start-ups. He is currently the technical escalations lead for the global Virtualization Rapid Deployment Program at Microsoft.


Virtualization
Essential Tools for Planning Your Virtual Infrastructure
Jay Sauls and Baldwin Ng
At a Glance:
  • Containing costs
  • Inventorying computers
  • Determining CPU, disk I/O, and network utilization

Contents
Tool Highlights
Find Out What You've Got
Determine Current Utilizations
Server Placement Recommendations
Virtual Machine Configuration
Placement Constraints
Considering Your Options
Requirements
Wrap Up
Virtualization is not a new technology, but it's now exploding into many new areas, far beyond the original platform virtualization developed by IBM in the 1960s. A major reason for this rapid growth is the potential for virtualization technologies to help contain costs in the datacenter and across IT operations. With energy costs rising and the appetite for more IT services by different parts of your organization ever increasing, you may be struggling to keep up with the server growth while holding down operations costs. Virtualization can help. And Microsoft and other software makers have been developing new technologies, including server, desktop, and application virtualization, that will make virtual resources more widely available.
In this article we look at a toolset that can help you with your virtualization projects—the Microsoft Assessment and Planning Toolkit (MAP), which is a member of the Microsoft Solution Accelerators family. What are Solution Accelerators? Simply put, they are automation tools and guidance you need for almost every IT project. Solution Accelerators are free, authoritative resources that help you proactively plan, deliver, and operate your IT infrastructure—from desktops to servers, from physical to virtual.
Designed in collaboration with industry experts and tested in real customer environments, Solution Accelerators offer just the right level of prescriptive guidance and automation to achieve high performance and build a cross-product IT infrastructure—all without having to reinvent the wheel. These resources focus on key concerns of companies of all sizes, such as infrastructure assessment and planning, virtualization, communication and collaboration, and security and compliance.

Tool Highlights
The newly released MAP Toolkit 3.1 (see Figure 1) is a network-wide infrastructure assessment tool designed to help you quickly gain a baseline understanding of your servers and desktops, including whether they are ready for upgrade or migration to technologies such as Hyper-V, Microsoft Application Virtualization, Windows Server 2008, Windows Vista, Office 2007, and SQL Server.
Figure 1 The Microsoft Assessment and Planning Toolkit 3.1
As Figure 2 shows, MAP autogenerates readiness-assessment reports and proposals, saving you a lot of work you'd otherwise have to do yourself. MAP Toolkit 3.1 supports both server and desktop scenarios, including evaluating desktop machines for upgrade to Windows Vista and planning for the rollout of Microsoft Application Virtualization.
Figure 2 Sample auto-generated migration proposal
For server-virtualization planning, three key features of the tool can help you determine which of your physical machines are good candidates: server inventory, performance metric gathering, and virtualization placement recommendations. Let's walk through a scenario that shows how you can use these features to build a detailed picture of your current environment and plan for the transition to virtualization.

Find Out What You've Got
Many large enterprises have some form of server-management infrastructure in place, such as System Center Operations Manager. Some environments are less actively managed. If you're working in an environment where you're really not sure what servers are in place or what they're running, the Server Inventory feature can quickly help you paint a clear picture. Server Inventory is able to discover machines in your environment using Active Directory, IP address range scan, or by simply importing a list of machines from a flat file. As Figure 3 shows, you can also use multiple discovery methods together (Active Directory and IP address scan, for example) to find machines that should be in Active Directory but aren't.
Figure 3 Choosing the computer discovery method
As machines are discovered, MAP Toolkit 3.1 attempts to connect to each one and retrieve more than 50 classes of data via Windows Management Instrumentation (WMI). Several different kinds of reports can be generated based on this data. For server consolidation with virtualization, we're most interested in the reports created when you select Migrate specific roles and services to Windows Server 2008.
Even if you're not doing a migration to Windows Server 2008 right now, the two reports this scenario creates are valuable in determining the kind of hardware in your environment (including processor type and speed, disk sizes, and memory installed), as well as what software roles you have assigned to each server (DNS servers, domain controllers, and Terminal Services).
These reports are provided to you as Microsoft Excel workbooks, so it's easy to use the sorting and filtering tools in Excel to identify exactly which machines in your environment would make good candidates for virtualization.
The server inventory reports provide data on several attributes you can consider when choosing servers to virtualize. As an example, you may want to ensure that the computers you're virtualizing are running an OS supported by the Hyper-V virtualization platform. In the WS2008HardwareAssessment Excel workbook created by the server-inventory process, under the Server Inventory tab, you can examine the Current Operating System and Service Pack columns to find any systems that may need to be upgraded.

Determine Current Utilizations
The next step in finding virtualization candidates is to measure how heavily utilized the physical machines are. The Performance Metrics Wizard in MAP Toolkit 3.1 (see Figure 4) records performance counters from a list of machines you've identified as virtualization candidates. CPU utilization, disk I/O utilization, and network utilization are all captured for later review.
Figure 4 The Performance Metrics Wizard
This part of the tool was designed to watch machines over a fairly long period of time (up to a month), which gives you some flexibility in choosing how you use it. If you already have a good idea of when the highest utilizations in your environment occur (for example, you may have a weekly batch-processing window), you can set up the wizard to run for only a few hours right around that time. However, if you're unsure when your highest utilizations occur or if you'd like to observe many machines that have different times of peak use, you can set up the wizard to run for several days or weeks to record data from your computers. This lets the tool automatically identify peak utilizations from a larger set of data.
Using the information from your earlier inventory, access the sorting and filtering capabilities in Excel to create a list of machines you'd like to monitor. The Performance Metrics Wizard will accept this list as a flat file, with one machine name per line. Copy the list of machine names from Excel into a new, blank text file in Notepad, then save it. When prompted, import this file into the wizard and set the end time of the performance metrics gathering period. The wizard will go into data-collection mode and gather performance counters from the specified machines every five minutes.
When the Performance Metrics Wizard completes, it will create a detailed Excel report listing all the machines observed and all the performance metrics for different categories of utilization: CPU, physical disk, logical disk, and network. Using this information, you can further refine your list of candidates to consider for virtualization.
You can target machines with extremely low utilization and look to consolidate several of them as virtual machines (VMs) on a single host, or you can combine this information with the inventory data to find machines that have mission-critical services that need high availability. Virtualization can be used to provide enhanced disaster-recovery capabilities for high-availability environments. After your analysis is done, you can move on to the final step of generating virtual machine placement recommendations.

Server Placement Recommendations
The Server Virtualization and Consolidation Wizard allows you to specify the configuration of a hypothetical virtualization host machine in terms of CPU, network, disk subsystem, and memory. Using the performance data collected by the Performance Metrics Wizard, the Server Virtualization and Consolidation Wizard computes a good mapping of VM guests to VM hosts, ensuring that each host is well utilized without risking overutilization during peak usage.
When the Server Virtualization and Consolidation Wizard completes, it produces a detailed Excel spreadsheet with the proposed mapping of guest VMs to physical hosts. For each VM and host, the report shows projected disk I/O, network I/O, and CPU utilization. As guest VMs are added to physical host machines during the fitting process, new host machines are allocated as previous hosts reach capacity. For each new host, a message is displayed that indicates why a new host had to be allocated.
If several of your hosts get the same message, you might consider rerunning the Placement Wizard with a different host configuration to try to increase the number of guests assigned to each host. For example, if several of your hosts display the message that they reached the limit on memory, you could try rerunning the wizard and specifying a larger amount of RAM for the host configuration. This should increase the number of guests that can be assigned to each host.

Virtual Machine Configuration
Each page in the Server Virtualization and Consolidation Wizard allows you to configure a major component of the hypothetical computer configuration that will be used for modeling the capacity of the VM host computer. Models are provided for both Virtual Server 2005 R2 and for Hyper-V, and you'll be asked which virtualization technology you want to use on the first page of the wizard (see Figure 5).
Figure 5 Selecting the virtualization technology to use for placement recommendations
Models for several families of CPUs from both AMD and Intel are provided on the CPU configuration page. If a model for your exact CPU isn't available, choose the closest match. You can configure the count of physical processors, cores, and hyper-threads per core on this page. Note that if you choose Hyper-V as your virtualization platform, only 64-bit CPUs will be available to choose from (see Figure 6).
Figure 6 Host CPU configuration
The third page in the wizard lets you configure a disk subsystem in terms of number of disk drives and RAID technology to compute both total disk space available on the host and a value for the throughput of the disk subsystem. The throughput value is provided automatically based on the interface technology and the disk Reliability and Performance Monitor (RPM) speed chosen.
In the MAP Toolkit 3.1, direct support for modeling Storage Area Network (SAN) and Network Attached Storage (NAS) disk solutions is not provided. For a rough approximation, choose the SCSI disk technology and configure a RAID level that's similar to your SAN configuration, with disks that are of similar speed. This approximation should be close enough in order to help you do your initial planning.

Placement Constraints
As virtualized guest computers are mapped to physical hosts, six factors are taken into consideration to evaluate whether or not a guest will fit within the remaining capacity of a host computer:
  • CPU utilization
  • Network I/O utilization
  • Disk subsystem I/O utilization
  • Memory
  • Disk space
  • Guest OS support on the specified virtualization technology
CPU, network, and disk subsystem utilization are fairly straightforward. Measured utilizations on the physical virtualization guest candidate are converted to an equivalent utilization on the virtualization host computer, which is then added to the existing utilization of the host. If the resulting total utilization of all guests assigned to that host for that resource is less than the threshold limit, then the guest is assigned to the host. If not, the guest won't be placed and other guests are tried to see if one can be found to fit.
As an example, suppose you gathered performance metrics about a computer with a 1.2 GHz Pentium 4 CPU that showed a 32 percent CPU utilization. If you configure a virtualization host with a quad-core Intel Xeon 5300 series CPU, the original measured CPU utilization from the potential guest computer will be extrapolated to an equivalent projected utilization on the host computer. This new utilization will be lower than the original measured value due to the newer CPU architecture, different CPU clock speed, and addition of more cores.
Conversely, if you configured a host computer with a less powerful CPU than the one in the guest, the projected utilization will be higher once extrapolated to the host. In either case, the new extrapolated CPU utilization will be used in calculating the overall usage of the host computer as guests are assigned to it.
It's worth briefly discussing just how the utilization of memory and disk space is decided. The total amount of physical memory installed on each potential virtualization guest is reserved on the host machine. Additionally, a small amount of memory (512 MB) is reserved for the host itself. When the total memory on the host is exhausted due to assignment of guests plus the host overhead, no more virtualization guests may be placed on that host computer.
Disk space is handled similarly: the total amount of physical disk space on each potential virtualization guest is reserved on the host machine. Additionally, 50 GB of disk space is reserved for use by the virtualization host. Once the total disk usage of all the guests assigned to the host plus the space reserved for the host itself reaches the size specified in the host configuration, no more guests can be assigned.
Based on the virtualization platform chosen, checks will be made to ensure that the OS running on the guest computer is actually supported by the virtualization platform. For example, 64-bit OSs are not supported as guests on Virtual Server 2005 R2, so they will not be placed if you choose that as your virtualization technology.

Considering Your Options
By running the Server Virtualization and Consolidation Wizard multiple times, it's easy to compare the results of different hardware choices for the virtualization host machine. Each time you run the wizard, you can choose different CPU, disk, and memory options to find the solution that's going to be best for your environment.
Pay attention to the Last Placement Failed Reason column on the Consolidation Recommendations tab of the server consolidation Excel workbook. These reasons will give you clues as to why more guest machines could not be placed on a particular host. If several hosts are displaying the same reason, such as Memory utilization of guest exceeds available memory on host, increasing the amount of memory on the host configuration may reduce the total number of hosts that are necessary.

Requirements
One of the advantages of MAP Toolkit 3.1 is that it does not require installation of any software on the computers being inventoried. All operations are conducted remotely. However, there are a few factors that must be taken into account in order for everything to work properly.
In order for the Inventory Wizard and the Performance Metrics Wizard to complete their work, a few TCP ports must be open on the target computer and accessible from the computer where the MAP Toolkit is installed. The Getting Started Guide goes into detail about the requirements; in short, the ports for authentication, DCOM, and WMI must be open on the computers being inventoried. If there are any firewalls installed on the network between the computer running MAP Toolkit 3.1 and the inventory targets, these firewalls must be configured to allow this traffic to pass as well.
MAP Toolkit 3.1 is designed to inventory environments of up to 100,000 computers and to collect performance metrics from up to 3,000 computers at a time. By default, this tool will install a private instance of SQL Server 2005 Express Edition. Due to the 4GB database file size limitation in SQL Server Express, this configuration only allows you to inventory about 20,000 computers before hitting the file-size limit enforced by SQL Server Express. In order to work with more computers during inventory, there are some defaults that must be modified.
To work with larger environments, you'll have to use a full version of SQL Server, such as SQL Server 2005 Standard Edition. Before you install the MAP Toolkit, install a new instance of SQL Server 2005 and name the instance "MAPS." Then run the MAP Toolkit installer as usual on the same computer. The MAP Toolkit requires a local instance of SQL Server, and the installer will automatically detect a local "MAPS" instance of the database server and install into it, instead of installing SQL Server Express Edition.
For even larger environments, you can utilize the scale-out feature, which allows multiple helper computers that have the MAP Toolkit installed to execute a large inventory cooperatively and store all the data gathered into a single central database. This configuration also requires the use of SQL Server Standard Edition in order to avoid the 4 GB file size limit.
See the Getting Started Guide for details on how to install this tool in this configuration and to edit the database connection strings appropriately for this mode. The scale-out feature can also be used to put SQL Server on a machine other than where the MAP Toolkit is installed.

Wrap Up
Solution Accelerators such as the Microsoft Assessment and Planning Toolkit are all about doing the heavy lifting for your IT projects so you do not have to. The MAP Toolkit has saved many users hundreds of hours of IT planning time for Hyper-V deployments. Are you ready to give this a try?
If you would like, you can download the Microsoft Assessment and Planning Toolkit for free at microsoft.com/map, or you can get other Virtualization Solution Accelerators at microsoft.com/vsa. We also encourage you to visit the MAP Toolkit Team Blog at blogs.technet.com/mapblog to find more tips and view demo videos.

Jay Sauls is a Senior Program Manager in the Solution Accelerators group at Microsoft, where he has worked on designing and implementing tools such as Windows Vista Hardware Assessment and the Microsoft Assessment and Planning Toolkit. Previously Jay worked on other Microsoft product teams, including Microsoft Commerce Server, Microsoft BizTalk, and System Center Capacity Planner.

Baldwin Ng is the Senior Product Manager for the Microsoft Solution Accelerators Core Engineering Team. His current projects include the Microsoft Assessment and Planning Toolkit and Virtualization Solution Accelerators. Baldwin has more than 16 years of product-management and development experience in IT, e-commerce, and wireless Internet services working with Microsoft, Corbis, and Alltel. He frequently blogs at blogs.technet.com/MAPBLOG.


From the Editor It’s a Virtual(ized) World
Joshua Hoffman

Virtualization is nothing new. That may not be the most enthusiastic way to start my note to you this month, but it's true. The concept of virtualization—separating an operating system from the underlying hardware resources—has been around since 1967, with the introduction of the IBM CP/CMS operating system. Yet while the concept may be well established, a lot has changed in the last 40+ years.
In 2008, we are hitting a tipping point in computer virtualization. As the need to reduce costs, simplify management, and establish sustainable computing practices has converged with more powerful, multicore processors and new hypervisor technology, it's time for virtualization to take the main stage. That's what this issue of TechNet Magazine is all about. Inside you'll find articles detailing the exciting new Hyper-V platform in Windows Server 2008, tools for managing a virtual environment, techniques for deploying virtual apps, and more.
On a more personal note, I want to express a debt of gratitude on behalf of the entire TechNet Magazine staff to contributing editors Greg Stemp and Jean Ross of the infamous Microsoft Scripting Guys. I know it may come as a shock to you that the Microsoft Scripting Guys are not, in fact, small men in lab coats with unusually large heads running around the halls of Microsoft with a test tube in one hand and a script in the other. Rather, for the past three years, Greg and Jean have authored the Hey, Scripting Guy! column in TechNet Magazine, and they have done so to worldwide acclaim. Hey, Scripting Guy! has not only come to be the most popular column we publish, it has won awards along the way and receives more accolades and reader e-mail than we could ever count.
It is for all these reasons and many more (not the least of which is the fact that we're particularly fond of them) that we are saddened here at TechNet Magazine to say goodbye to Greg and Jean this month. Alas, as is bound to happen to the best of us, Greg and Jean are moving on to greener pastures. Fortunately, though, they will continue to wander the halls of Redmond, dispensing their talent to the benefit of Microsoft customers. So we wish them both the very best, and we can only hope that their words will grace our pages again someday.
In the meantime, Dean Tsaltas, a venerable Scripting Guys alum, has graciously agreed to pick up the Hey, Scripting Guy! mantle for now as we hunt for the next generation of Scripting Guys. Dean will join us next month to probe the depths of Windows Management Instrumentation. Be sure not to miss it! —Joshua Hoffman

Thanks to the following Microsoft technical experts: Karri Alexion-Tiernan, Carmen Asher, Scott Culp, Steve Ekren, Trina Gorman, Michael Michael, Mike Mitchell, Michael Murgolo, Rob Polly, Dan Reger, Jez Sadler, Bill Scheidel, Bryon Surace, and Jim Truher.


Letters Readers Speak Out

SQL Database Management
Kudos for the "SQL Server: Top Tips for Effective Database Maintenance" article in the August 2008 issue (available at technet.microsoft.com/magazine/cc671165). Finally, an article in the SQL section that was aimed at us "involuntary DBAs" who just need to make sure that some application called SQL Server is running and being backed up.
And on the very next page, you follow that up with "Windows Administration: Taking Your Server's Pulse" (technet.microsoft.com/magazine/cc718984). That was awesome. I ripped those articles out of the magazine for safekeeping—something that we just can't do with digital e-zines. I'll refer back to both of those articles frequently. Keep up the good work.
We're glad to hear that you enjoyed those two articles. We always like to hear that our content is useful. But while we'll never turn down effusive praise, please don't stop there! Feel free to drop us a line with any other topics you'd like to see covered in future issues. We're always looking for input from our readers.
—Joshua Hoffman, Editor-in-Chief

Image X and Windows PE
Thank you for the column on deploying older operating systems with ImageX and Windows PE ("Dual Booting with Windows XP and Windows PE 2.0," February 2008, technet.microsoft.com/magazine/cc194422). I had discovered the potential of these tools last year, and I started to experiment with them to replace the imaging tool that my company has been using (Acronis Snap Deploy). I got everything working just as I needed it to work about six months ago, and it is now actually the only imaging tool we use.
Then I read your article and noticed your advice on adding the bootsect.exe /n52 option. I had never heard of this option before. The only thing we do before applying the images is to wipe the hard drive using DiskPart with the included script (commands.txt). We haven't experienced any problems just yet, but could this present a problem in the future?
Early on, that command was needed. But I was informed by a team member after that article went live that there is no need for it any longer since the boot code _now_ has logic in it to try looking for the new style boot manager code, and if not found it tries NTLDR before failing. So it can be omitted entirely without any problems.
—Wes Miller, Columnist, The Desktop Files

Test Image Environment
I read your Desktop Files column about implementing images using the ImageX tool and enabling multicast through Windows Deployment Services, or WDS ("Dual Booting with Windows XP and Windows PE 2.0"). I want to mount a test environment and would like to know exactly what tools are necessary. Must I install Windows Server 2008 for a multicast implementation? And also, for mirror images, is it necessary to utilize ImageX for the installation of the Deployment 4 or the Windows Automated Installation Kit (WAIK) and WDS?
To utilize multicast using WDS, you must have Windows Server 2008 installed. The multicast components are not available (and will not be available) for Windows Server 2003.
Deployment 4 is not necessary to utilize multicast. In order to use it, you need to obtain Windows Server 2008, as well as the released version of the WAIK for Windows Server 2008. That version of the WAIK is the first that contains the pieces necessary for multicast to Windows PE.
—Wes Miller

WDS Multicast
I tried to use ImageX to clone images into and from computers. I soon faced a problem: ImageX doesn't support multicast transmissions. I want to do the same using WDS servers with multicast transmissions. Is it possible? It seems that the Wdsmcast command can do the trick. How can a WDS Client inform a WDS Server which image to download?
Indeed, ImageX cannot itself handle multicast. You do have to use Wdsmcast (in the Windows Server 2008 WAIK) to specify which transmission to connect to (after you've started them on the server). The white paper "Using Transport Server" has decent information on how to connect using Wdsmcast (technet.microsoft.com/library/cc725964). Read the section "How to connect a client server to a namespace using Wdsmcast.exe."
—Wes Miller

Contact Us Got something to say? E-mail us at tnmag@microsoft.com, or visit our blog forum at blogs.technet.com/tnmag.


Toolbox New Products for IT Pros
Greg Steen

All prices were confirmed August 19, 2008, and are subject to change. The opinions expressed in this column are solely those of the author and do not necessarily reflect the opinions at Microsoft.

Take Your Apps with You
PortableApps.com
PortableApps.com
As an IT professional, you probably find yourself moving from machine to machine a lot. And when you do, you also probably find yourself wishing you had your standard set of apps, tools, and documents at your fingertips. One cool way to meet those needs is to start with a launcher platform like the one provided by PortableApps.com. On its site, you will find the free and open-source PortableApps.com Platform as well as a number of portable apps that can run solely off of a USB flash drive, memory card, portable hard drive, or digital music player.
There is also a PortableApps.com Suite. This handy download includes a bundle of the available portable applications along with the base platform, all in one installer. There are two versions of the suite. Suite Light includes Mozilla Firefox, Thunderbird, and Sunbird along with ClamWin antivirus, Pidgin for instant messaging, the Sumatra PDF reader, the KeePass password manager, the AbiWord word processor, an audio player, and some simple games. Suite Standard includes all those applications, as well as the OpenOffice.org office suite, which contains Writer, Calc, Impress, Base, and Draw.
In addition, there are a number of other open-source applications that have been made "portable" on the site, including FileZilla for FTP, PuTTY for telnet and SSH, WinSCP for SFTP and SCP, PNotes and Task Coach for quick lists, 7-Zip for archiving, a command prompt, Eraser for wiping files, InfraRecorder for CD/DVD burning, WinDirStat for visualizing a disk's content, winMd5Sum for verifying files, and WinMerge for file and directory diffs. (Frequent readers of my Toolbox column might notice that I have covered a number of these apps in previous installments.)
There is also a very active user community, which is working on portable versions of such network utilities as WinPCap and Wireshark. (These two tools can really help with your administrative and troubleshooting tasks.) The forums even provide a place specifically for suggesting applications to make portable.
These applications are self-contained on the storage device you use. If they write to the registry or require any "off-disk" access, they should return everything to its pre-run state on exit. And you can run the apps without the PortableApps.com Platform—they are available wrapped in their own installers for doing just that. But, that aside, you will most likely want the features of the Platform anyway, and the Suite is a great place to start a customized environment that you can carry with you.
The PortableApps.com Platform is set up for autorun. Thus, when you insert your USB flash or portable drive into the Windows host machine, you will get the option to run the platform application. Once you launch the platform, a system tray icon appears. Clicking this icon pops up a menu that resembles the Windows Start menu. This application launcher provides quick links to all your portable applications along with links to document folders on your portable drive, an indicator of how much space is left on your drive, a backup utility for the documents on the drive, a search tool, and a link for installing new portable applications. You can customize this interface and choose one of more than 30 interface languages.
After playing with the platform for a while, you might become interested in how you can modify an application to make it portable. The Web site also has an active Portable App Development forum to get you on your way. All things considered, this is a very cool set of tools in a portable package.

Price: Free.
PortableApps lets you take your tools with you

Troubleshoot Remote Systems
TeamViewer
teamviewer.com
When you are troubleshooting application issues, giving product demos and tutorials, or just trying to understand a user's question, nothing beats sitting right down at the machine in question. But often that is just not possible. The next best thing is to use a tool to help you "be there" without really being there. TeamViewer, from TeamViewer GmbH, does just that.
This utility has a simple installation process, allowing you to connect to a remote machine quickly. There are four basic modes of operation supported by TeamViewer. There is Remote Support, which allows you to control the remote machine so you can troubleshoot, assist with operations and maintenance, and guide the user through an activity. Then there is Presentation, which lets you display your desktop on the remote user's system so you can demonstrate an activity or give a presentation. Another mode is File Transfer, which allows you to transfer files between the two machines without actually sharing desktops. And lastly, VPN, which lets you create an instant virtual private network between you and the host machine (this mode uses the TeamViewer VPN adapter installation option).
In terms of security, the application uses an RSA key exchange and 256-bit AES session encoding. The key exchange assures you that no "man in the middle" (including the TeamView routing servers) will be sniffing the data stream. In addition, when you connect to a Supporter session (the Supporter is the term used for the one hosting the session), a session ID and password is generated and the remote user must enter them to connect. In case you want to connect to a machine without a human at the other end (to manage your servers, for example), TeamViewer offers the option to be installed as a system service so you can still easily transfer files, use remote control capabilities, create a VPN, or even reboot your remote machine.
Another cool feature, especially if you plan to use the application for support or presentations, is the TeamViewer QuickSupport module (or Customer module). This is a relatively small executable that requires no software installation or installation rights on the remote machine to execute. The end user just has to double-click the exe file and enter the session ID and password, and then you can readily connect your session to the machine.
TeamViewer has some very useful configuration options. You can optimize the number of colors shared on the display, remove the remote wallpaper from the display, or choose to keep the Windows Vista Aero Glass enabled. And you can adjust the general quality/speed setting for your sessions. In addition, you can control whether to allow various features, including remote control, file transfer, VPN connections, remote keyboard and mouse input, switching control between the local and remote hosts, and viewing of the remote screen.
You can record sessions for later replay or auditing purposes. And if you don't want to route through TeamViewer's servers, perhaps due to your security configuration, you can choose to have TeamViewer only use LAN connections to route itself.
The Premium edition, which supports unlimited Supporter installations, also includes a Portable edition that you can put on a CD or USB flash drive in order to take the tool with you. The Premium edition also includes the TeamViewer Manager, which logs session details and allows you to store partner details, both of which are handy features for tracking, reporting to, and invoicing your clients. TeamViewer is also available for Mac OS X, and it supports cross-platform access so you can stay on your Windows machine while connecting to a Mac.

Price: Free for non-commercial use. Business edition starts at $699 for one Supporter installation. Premium edition is $1,399 and allows for unlimited Supporter installations.
Connect to a remote system with TeamViewer

Book Review
End-to-End Network Security: Defense-in-Depth
Network security today incorporates multiple layers of proactive technologies to reduce exposure to incidents and limit the potential for breaches from inside and outside the network infrastructure. End-to-End Network Security: Defense-in-Depth (Cisco Press, 2007) by Omar Santos can help you navigate the ever-increasing complexities of defending your network.
Although this book is, for obvious reasons, partial to Cisco gear, it still provides a good deal of generalized security information, making it a good reference for any network professional or systems administrator interested in infrastructure security.
The book begins with an overview of network security technologies, from firewalls and VPNs to Intrusion Detection/Prevention Systems and identity management. Then it details a six-step methodology for handling security incidents: Preparation, Identification (of security threats), Classification (of security threats), Traceback, Reaction, and Postmortem.
The book goes on to discuss a proactive security framework that uses a variety of devices and tools including IDS/IPS, NAC, AAA, SNMP, Syslog, and VLANs. Then it moves onto the Cisco defense-in-depth approach, covering wireless networks, IP telephony, data center security, and IPv6 security primarily as it pertains to Cisco's solutions for those areas of network security. The book ends with three case studies that discuss real-world small, medium, and large enterprise security solutions.

Price: $49.50 (direct).

Investigate Device IDs
PCIDatabase.com
PCIDatabase.com
Ever try to upgrade a system and end up with an unknown device or two whose descriptions tell you nothing other than "PCI Device" and you have no idea how to get a driver or figure out what the device is without ripping off the cover of your machine and digging around with a flashlight? Or perhaps you've tried to find out what the underlying chipset is on a known device. Well, one free service that can help is PCIDatabase.com.
This user-supported database lets you search by vendors and device names and IDs to find more information. Simply open up the Device Manager, right-click on the device in question and select Properties, pick the Details tab, and then select the Hardware IDs property. Here you will see a set of strings that contain a few hex values that are pre-pended by "VEN_" (for vendor) and "DEV_" (for device).
The site provides links to a number of useful resources to help you find out if there are any drivers available for the device you looked up (and, of course, don't forget to check the manufacturer's Web site as well).
Another cool feature you can find at PCIDatabase.com: if you look up a particular vendor, you will get a page that lists the headquarters address, phone numbers, applicable Web sites, contacts, the vendor's hex ID for PCI devices, and a listing of all the vendor's devices that the site knows about. And since the site is user supported, it also encourages you to add your knowledge to the mix.

Price: Free.
Get vendor and device information on PCIDatabase.com

Audit and Inventory Assets
CCS Network Inventory
crowcanyon.com
Being able to find out what one of your client machines has installed in it or on it without leaving your desk can be quite a time-saver. And let's face it—tracking those assets over time is part of your job. Crow Canyon Systems can help you do those things with its CCS Network Inventory solution.
The application is an agentless inventory and auditing tool that scans your LAN for workstations—gathering hardware, software, service, and user information. Since CCS Network Inventory doesn't use a remote agent, your execution context needs RPC, Computer Browser service, Remote Registry service, file and printer sharing, and ADMIN$ share access to the machines you wish to inventory (just like most other agentless inventory products).
The application does let you use alternative credentials for its queries, which is useful when your scan spans multiple domains. After installing the application, you can scan an IP address range, use Windows Network discovery, import a list of machines, or manually specify a list of Windows workstations to inventory.
In terms of what CCS Network Inventory audits, it collects software details: what applications are installed, which hotfixes have been applied, what OS and service pack the machine has installed, and even what fonts are installed. In terms of the machine configuration, you can see what services and other processes are running, what folders are shared, drive mappings, environment variables, user accounts, scheduled tasks, startup commands, and what network protocols are running. And for hardware, you can see the details of attached monitors, printers, hard disks (and their sizes and partitions), CD and DVD drive information, processors, memory, and so on. In addition, you can set up custom scans for registry values, file counts (for example, who has the most mp3s), and free space on drives.
All this information is stored in the application's custom database and can then be used the data to create reports. CCS Network Inventory provides a number of built-in reports to visualize and transport the data for auditing and reporting purposes. If the built-in reports and query information don't include exactly what you need, you can use the built-in SQL Query Builder tool, which allows you to write and run a more detailed or specific query against the data.
Over time it is usually a good idea to take environment snapshots so you can see changes. CCS Network Inventory lets you see these changes by comparing two instances of the information databases. For example, you can compare the current scan to a scan taken a year ago to see how many workstations have switched to Windows Vista, how many have had their hardware components updated in the past year, or which users have continued to run unsupported software after you already warned them.
At the time of this writing, CCS Network Inventory doesn't fully support Windows 2008 Server, but it does support the other flavors of Windows up through Windows Vista.

Price: Starts at $235 for up to 25 nodes.
Manage remote assets with CCS Network Inventory

Write to Toolbox If you have a favorite tool or utility you would like to see featured here, please write to the author at tntools@microsoft.com.

Greg Steen is a technology professional, entrepreneur, and enthusiast. He is always on the hunt for new tools to help make operations, QA, and development easier for the IT professional.


SQL Q&A Large Transaction Logs, When to Use Repair, and More
Paul S. Randal

Q I've noticed some strange behavior with backups that I'm hoping you can explain. Every so often we back up our 62GB production database to refresh the data used by our application developers. We always delete the old copy before restoring the new one. The restored copy is the same size as the production database and the data looks the same, but the restore process takes many times longer than the backup process. What's going on? Why does it take so much longer to restore than to back up?
A There's really nothing strange going on. Depending on your circumstances, this is generally expected behavior. The difference in time required by a backup versus a restore comes from the steps that each process must perform.
Backing up a database comprises two steps. Basically, it's just doing read IOs on the database and write IOs on the backup device:
Backup Step 1 Read all the allocated data in the data files and write it to the backup device.
Backup Step 2 Read some transaction log and write it to the backup devices.
The exact amount of transaction log required can vary wildly, but it's really the amount required to be able to recover the restored database to a consistent point in time. (I've given a more in-depth explanation of this in a blog post available at sqlskills.com/blogs/paul/2008/01/31/MoreOnHowMuchTransactionLogAFullBackupIncludes.aspx.)
Restoring a database, on the other hand, can comprise up to four steps. And the work involved is more complicated than just read and write IOs:
Restore Step 1 If the database files don't exist, create them.
Restore Step 2 Read all the data and transaction log from the backup and write it to the relevant database files.
Restore Step 3 Run the REDO phase of recovery on the transaction log.
Restore Step 4 Run the UNDO phase of recovery on the transaction log.
The total elapsed time for both steps of the backup should be roughly the same as the time required for Restore Step 2 (assuming it has similar hardware and a server with no user activity). Step 1 of the restore can take a long time if the data files are large and need to be zero-initialized (which is the behavior in SQL Server 2000 and the default behavior in SQL Server 2005).
To prevent this, do not delete the existing files before starting the restore. Or alternatively, enable instant initialization so the files are created very quickly (more information on this can be found at msdn.microsoft.com/library/ms175935.aspx).
Steps 3 and 4 of the restore are running recovery on the restored database to make it transactionally consistent; this is the same process a database would go through during crash recovery. The length of time that recovery will take depends on the amount of transaction log that needs to be processed. For instance, if there was a very long-running transaction active at the time the backup was taken, all the transaction log for that transaction would be in the backup and would have to be rolled back.

Q I'm trying to choose between log shipping and database mirroring to provide a redundant copy of our production database. I'm concerned by the amount of transaction log that will need to be sent between the servers, especially for the index rebuild operations we do every night. I've heard that mirroring sends the actual rebuild commands instead of the transaction log and the rebuilds are done on the mirror. Is this true? This should make mirroring a better solution than even log shipping with the BULK_LOGGED recovery model, right?
A What you've heard is not true. Database mirroring works by sending the actual transaction log records from the principal database to the mirror server, where they are "replayed" in the mirror database. There is no translation or filtering of any kind taking place, nor any kind of interception of T-SQL commands for a mirrored database.
Database mirroring only supports the FULL recovery model, which means an index rebuild operation will always be fully logged. Depending on the size of the indexes concerned, this could mean a significant amount of transaction log being generated and, in turn, a large log file on the principal database and substantial network bandwidth for sending the log records to the mirror.
You can think of database mirroring as real-time log shipping (in fact, this was a name used for the feature early on during SQL Server 2005 development). In log shipping, backups of the primary database's transaction log are shipped regularly to the secondary server and are restored on the secondary database.
Log shipping supports the FULL and BULK_LOGGED recovery models. For an index rebuild operation in a log-shipped database with the FULL recovery model, the same amount of transaction log will be generated as the amount generated for a mirrored database. However, in the log-shipped database scenario, the data is sent to the redundant database in a log backup (or a series of log backups) rather than as a continuous flow.
If the BULK_LOGGED recovery model is used in the log-shipped database while the index rebuild is done, only a minimal amount of transaction log will be generated. However, the next transaction log backup will also contain all the data file extents that were changed by the minimally logged index rebuild operation. This means the log backups covering the index rebuild in the BULK_LOGGED recovery model will be almost exactly the same size as those covering the index rebuild in the FULL recovery model.
So the amount of information that needs to be sent to the redundant database is almost exactly the same for an index rebuild on a mirrored database and on a log-shipped database. The real difference is how the information is sent—continuously or in batches.
There are many other factors to consider when choosing between these two approaches (way too many factors to discuss in just one installment of SQL Q&A). You should see how all of these factors match up to your requirements (such as acceptable data-loss limit and allowable downtime) before you make a decision.

Q I'm running SQL Server 2005 and one of my databases has a transaction log that just keeps growing. The database is in the full recovery mode and I'm doing transaction log backups. I was under the impression that this should prevent the transaction log from growing. What exactly is going wrong here?
A You're right that taking transaction log backups in the full recovery mode is critical. However, there are other factors that can contribute to transaction log growth. It all comes down to what is requiring the transaction log to be needed (or active). Other common factors (aside from a lack of transaction log backups) that may cause your problem include replication, database mirroring, and an active transaction.
Replication works by asynchronously reading your transaction log records and then loading up the transactions to replicate into a separate distribution database. Any transaction log records that haven't yet been read by the replication log reader task cannot be freed. If your workload generates a lot of transaction log records and you've set a large interval for how often the replication log reader will run, then lots of records can accumulate and cause the transaction log to grow.
If you are running asynchronous database mirroring, then there could be a backlog of transaction log records that have not been sent from the principal to the mirror (called the database mirroring SEND queue). The transaction log records cannot be freed until they have been successfully sent. With a high rate of transaction log record generation and limited bandwidth on the network (or other hardware issues), the backlog can grow quite large and cause the transaction log to grow.
Finally, if a user starts an explicit transaction (such as using a BEGIN TRAN statement) and then does a modification of some sort (such as a DDL statement or an insert/update/delete action), the transaction log records that are generated need to be kept around until the user either commits or rolls back the transaction. This means that any subsequent transaction log records generated by other transactions also cannot be freed, as the transaction log cannot be selectively freed. If that user, for example, goes home for the day without completing the transaction, the transaction log will continue to grow as more and more transaction log records are generated but can't be freed.
You can find out why the transaction log cannot be freed by querying the sys.databases system catalog view and examining the log_reuse_wait_desc column, like so:
SELECT name AS [Database],
  log_reuse_wait_desc AS [Reason]
FROM master.sys.databases;
If the cause turns out to be an active transaction, you can use the DBCC OPENTRAN statement to get more information about the transaction:
DBCC OPENTRAN ('dbname')

Q I've heard that REPAIR_ALLOW_DATA_LOSS should only be used as a last resort for recovering from corruption; restoring from backups should be used instead. Can you explain why "repair for SQL Server 2005" shouldn't be used and why it is in the product considering how "dangerous" it is?
A First off, I actually wrote repair for SQL Server 2005. The problem with REPAIR_ALLOW_DATA_LOSS (which I'll just call repair from here on out) is that it's not clear how it works. The name of the repair option was chosen to call out that running it could lead to loss of data from the database. The way the feature usually repairs a corrupt database structure is to delete the corrupt structure and then fix up everything else in the database that referenced or was referenced by the deleted structure. Repair really is intended to be a last resort way of getting the database structurally consistent—it is not focused on saving user data. Repair doesn't go out of its way to delete user data, but it doesn't go out of its way to save user data either.
This may seem like an irresponsible way to conduct repairs, but when the repair option needs to be used, it provides the fastest and most reliable method to fix corruption. Speed is of paramount importance in a disaster recovery situation and correctness is absolutely required. It is almost impossible to engineer more complex repair algorithms that can be proven to work quickly and correctly in all cases. There are some complex algorithms in the repair code to resolve the case when two indexes have the same page or extent allocated, for instance, but mostly the algorithm is repair plus fix-up.
Still, there are some issues with repair that you should be aware of:
  • Repair does not consider foreign-key constraints when deleting corrupt structures, so it may delete records from a table that has a foreign-key relationship with another table. There is no way to determine whether this has happened without running DBCC CHECKCONSTRAINTS after running repair.
  • Repair does not (and cannot) consider any inherent business logic or data relationships defined at the application level that could be broken by some data being deleted. Again, there is no way to determine whether anything is broken without running whatever custom consistency check is built into the application.
  • Some repair operations can't be replicated. Running repair on a publisher or a node in a peer-to-peer topology may introduce inconsistencies within the topology, which must be corrected manually.
For these reasons, it is always a good idea to recover from corruption using a backup rather than running the repair option. But repair is offered in the product because there is always the potential of getting in a situation where your database is corrupt, there is no backup, and you need some way to get the database back online quickly.

Q I've just moved to a new company as a DBA, and I've been given responsibility for several applications and their back-end databases. One of the applications has very poor performance doing updates. I investigated and found that the tables used by the application have an enormous number of indexes on each. After asking around, it seems that the previous DBA had a habit of adding an index of each table column, plus some combinations. I can't believe that all of these indexes are necessary—how can I tell which ones I can safely drop? We run SQL Server 2005.
A The large number of indexes is most likely a major contributor to the poor performance, as you surmise. Every time a row is inserted, updated, or deleted in the table, corresponding operations need to be done in every non-clustered index. This adds a lot of overhead in terms of I/Os, CPU utilization, and transaction log generation.
In SQL Server 2000, the only way to determine which indexes were being used was to use profiling and examine query plans. With SQL Server 2005 there is a new Dynamic Management View (DMV) that tracks index usage—sys.dm_db_index_usage_stats.
This DMV tracks every time an index has been used (and how it was used) since the database it is part of started up. The statistics for all databases are lost when SQL Server is shutdown, and the statistics for a single database are lost when it is closed or detached. The idea is that if an index does not appear in the output, then it has not been used since the database was started up.
A simple approach to tracking index usage over time is to take periodic snapshots of the DMV output and then compare the snapshots. One thing that many people miss is that you must track the index usage over an entire business cycle. If you just took a snapshot for a single day, you may find several indexes that aren't used. But if those indexes are used, say, to help the end-of-month reports run magnitudes faster, then the indexes probably shouldn't be removed. If an index really hasn't been used over a whole business cycle, then you can probably drop it and reclaim space and performance.
For some simple code you can use to take periodic snapshots of the DMV, see my blog post at sqlskills.com/blogs/paul/2007/10/05/IndexesFromEveryAngleHowCanYouTellIfAnIndexIsBeingUsed.aspx.

Paul S. Randal is the Managing Director of SQLskills.com and a SQL Server MVP. Paul wrote DBCC CHECKDB/repair for SQL Server 2005 and was responsible for the Core Storage Engine during SQL Server 2008 development. An expert on disaster recovery, high availability, and database maintenance, Paul is a regular presenter at conferences. He blogs at SQLskills.com/blogs/paul.


Utility Spotlight Offline Virtual Machine Servicing Tool
Peter Skjøtt Larsen and Suveen Kumar Reddy Vuppala

Contents
Security Nightmare
Under the Hood
Servicing Infrastructure
Using the Tool
Coming Up
Virtualizing a computer workload simply means using a virtual machine (VM) to untether the workload from the underlying hardware. Modern IT departments find VMs useful in many situations, including:
Shifting Workloads You can easily use VMs to increase or decrease the workload bandwidth of multiple setups, as your usage demand changes, without having to make a corresponding hardware change.
Developing and Testing Applications It's possible to create multiple VMs that represent each configuration an application is supposed to support without needing dedicated hardware.
Software Upgrades You can use VMs in order to bring the new version of a software package online as you take the previous version offline—all on the same hardware.
Software Distribution VMs can be used as a unit of distribution for a line-of-business application that has a consistent combination of software in a pre-tested package.

Security Nightmare
One of the advantages of using VMs is that you are able to store them in an offline state as VM images. Then when you need them, you can "wake" these VMs and deploy them much more quickly than you could deploy the equivalent hardware.
Keeping an increasing number of computing environments waiting offline presents a maintenance challenge, however. Many software update mechanisms rely on systems to be online in order to check for updates or to receive updates automatically.
When a VM is not online, it is just a file sitting in a computer, so it cannot interact with any update mechanism. A VM that has been brought online after being offline for a few months thus might become a threat to the network, or the network could threaten it.
It is not simply a matter of missing OS updates. Outdated applications or virus profiles can render the VM vulnerable or out of compliance with company standards.
To help customers address the challenge of keeping offline VMs up-to-date, the Microsoft Solution Accelerator team has created the Offline Virtual Machine Servicing Tool. This tool works with System Center Configuration Manager (SCCM) 2007, Windows Server Update Services (WSUS) 3.0, and System Center Virtual Machine Manager (VMM) 2007 to orchestrate the updating of stored VMs. Figure 1 shows a conceptual rendering of the tool and how it connects to various external components.
Figure 1 How the Offline Virtual Machine Servicing Tool works
To make VMs available for updates, the tool uses VMM to temporarily deploy them to maintenance hosts. Because a maintenance host configuration typically has the necessary CPU and memory to run multiple VMs at the same time, the tool can manage VMs in batches.
As soon as the VMs are active on the maintenance hosts, either SCCM or WSUS can supply them with the necessary updates. After the updates have been applied, the tool uses VMM to return the VMs to their offline state. (Note that the tool only supports VMs that are managed by VMM.)

Under the Hood
The Offline Virtual Machine Servicing Tool uses Windows Workflow Foundation (WF) to orchestrate the process of updating a VM. The process has a number of decision points, beginning with choosing the appropriate update management system, picking the next available maintenance host appropriate for the VM, ensuring that the update occurred, and, finally, dealing with exceptions.
Using a Windows WF-based solution gave the development team great flexibility to change and evolve the process. It also offers users a robust solution that can be tailored to meet specific needs; at critical junctures in the process, built-in pre- and post-workflow steps provide opportunities for customization.
The tool uses Windows PowerShell to implement individual tasks below the workflow level, which ties in nicely with the Windows PowerShell API offered by VMM. And the Microsoft .NET Framework-based UI looks and feels like System Center products, so new users should feel right at home.

Servicing Infrastructure
One of the basic principles of the servicing infrastructure is to configure network security to keep VMs from harm during the update process. In version 1.0 of the tool, this is accomplished using a Virtual Private Network (VPN) to which VMM and the appropriate update system (WSUS or SCCM) connects. The most appropriate infrastructure for a VMM library is a Fibre Channel-connected Storage Area Network (SAN), which provides the means for fast transfer of VM images to the maintenance hosts.
All the VMs must be members of the same domain, one that uses Active Directory and DNS. Separate servers can be dedicated to VMM, WSUS, SCCM, and the VMM library, but combinations of virtual servers can also be used for smaller environments. Needless to say, the maintenance hosts must be physical servers.

Using the Tool
After you have set up the servicing infrastructure, you need to check that certain settings are correct before the tool can start. Make sure that VMM is managing all the appropriate VMs, that each VM has the appropriate update client installed, and that the necessary update packages are configured in WSUS or SCCM. Ensuring that groups of maintenance hosts are configured in VMM is optional.
When you are ready, start the Offline Virtual Machine Servicing Tool, which has a number of configuration steps of its own. You will have to designate the VMM server and the appropriate WSUS or SCCM server, and then specify which group of maintenance hosts to use (if the maintenance hosts are grouped) and which maintenance hosts from that group to use (see Figure 2). You may want to configure groups of VMs to be managed, but this is optional.
Figure 2 Configuring the Offline Virtual Machine Servicing Tool
After you have configured the tool, you create the servicing jobs. A servicing job contains all of the information the tool uses to manage specific VMs, including whether to use WSUS or SCCM for updates; locations of the VMM server and the WSUS or SCCM server; identities of the VMs to be managed; type (and identity, as appropriate) of network to use for the process; identities of the maintenance hosts to use; account credentials needed to access the VMs, the VMM server, and the WSUS or SCCM server; and, finally, the schedule for running the servicing job (run immediately or at a specific date and time).
If you specify a date and time for the servicing job, Windows Task Scheduler determines when to start it. As the servicing job runs, the Offline Virtual Machine Servicing Tool follows this sequence for each VM:
  • Select the next VM from the VMM library.
  • Query VMM for the most appropriate maintenance host.
  • Deploy the VM onto the maintenance host.
  • Ensure that the VM connects to the correct network.
  • Start the VM.
  • Make sure the appropriate update client is installed on the VM.
  • Trigger the update process.
  • Wait for the update process to complete.
  • Shut down the VM.
  • Store the VM back in the VMM library.
The time it takes to update a library of VMs will vary greatly, depending on the number and capacity of the maintenance hosts, the access speed of the VMM library storage, and the nature of the updates.
You should note that the current version (1.0) of the Offline Virtual Machine Servicing Tool does not support network access protection (NAP), which is a really attractive way to protect VMs from the network. In addition to this, it does not support the Hyper-V technology of Windows Server 2008 or the use of Windows Server 2008 as a client OS.
Newer versions of SCCM, WSUS, and VMM will be available soon, and version 2.0 of the Offline Virtual Machine Servicing Tool will provide support for them as well as for Hyper-V and the use of Windows Server 2008 on the client. The tool is also going to support NAP for network isolation. You can download the Offline Virtual Machine Servicing Tool at technet.microsoft.com/cc501231.

Peter SkjØtt Larsen is a Senior Product Manager at Microsoft. Before joining Microsoft, Peter was involved in both architecture and development of telecom operational software systems and standardization and development of wireless services.
Suveen Kumar Reddy Vuppala is a Senior Software Development Engineer at Microsoft. He was previously involved in designing and developing the real-time monitoring tools and deployment solutions for Microsoft for the past seven years.


Windows PowerShell The Power of Profiles
Don Jones

Contents
Shells, Hosts, and Profiles
Using Your Profile?
Making a Custom Console
More Profile Tricks
Profile Beware!
If you read this column regularly, you know by now that Windows PowerShell supports a system of profiles. These are essentially shell scripts, with a .ps1 file name extension, that execute automatically when the shell runs. These provide a great way to define custom aliases, for example. You can have it so that each time the shell runs, the profile automatically defines your aliases, making them available anytime you're using the shell.
The shell defines the following four different profiles:
  • One that applies to all shells and all users of the computer.
  • One that applies to all users, but only to the Microsoft PowerShell shell.
  • One that applies to the current user and to all shells.
  • One that applies to the current user and only to the Microsoft PowerShell shell.
The concept of "all shells" can be a bit confusing. The terminology is rooted in some early development concepts of Windows PowerShell that didn't really make it into the final product. Today, the term "all hosts" would probably be more appropriate.

Video
Watch Don Jones demonstrate how you can create profiles to customize the Windows PowerShell shell.

Shells, Hosts, and Profiles
You see, Windows PowerShell itself is a set of Microsoft .NET Framework assemblies. I like to refer to this part of the shell as the Windows PowerShell engine because it contains the functionality that you generally think of as Windows PowerShell. There's no built-in way for humans to interact with this engine, though. For you to actually use the shell, the engine must be loaded by a hosting app (or host).
The powershell.exe application that you're probably used to running is one such host. The gpowershell.exe application included with the Windows PowerShell 2.0 Community Technology Preview (CTP) is another host. This relationship between hosts and shells isn't quite as simple as I've made it out to be here, but my simplified explanation covers the behavior you see.
It's important to understand that if you're interacting with the shell via a text-based command-line interface, then you're probably using the powershell.exe host. This applies even if you've launched the shell using a shortcut installed by another app, such as the Exchange Management Shell. The Exchange Management Shell isn't a different shell or hosting application. Rather, it's merely the normal powershell.exe host launched using a shortcut that specifies a set of snap-ins and scripts to be pre-loaded. Those snap-ins are what enable the Exchange management functionality within the shell. But you're still using the same shell.
The locations (on Windows Vista) of the profiles for the powershell.exe host are as follows:
  • %windir%\system32\WindowsPowerShell\v1.0\profile.ps1 This is for all users of the computer and for all shells.
  • %windir%\system32\WindowsPowerShell\v1.0\Microsoft.PowerShell_profile.ps1 This is for all users of the computer, but it is only for the Microsoft.PowerShell shell.
  • %UserProfile%\Documents\WindowsPowerShell\profile.ps1 This is for the current user only and all shells.
  • %UserProfile%\Documents\WindowsPowerShell\Microsoft.PowerShell_profile.ps1 This is for the current user only and only for the Microsoft.PowerShell shell.
These profiles aren't created by default. They exist only if you create them.
Each hosting application is responsible for loading and executing the correct profile. If a particular hosting application "decides" not to load any profiles at all, then it doesn't really have to. Profile execution is not performed by the Windows PowerShell engine automatically.
The easiest way to keep track of all this is to simply open the shell, type $profile, and hit Enter. That will give you the full path that the shell is attempting to use as what I think of as the "primary" profile (it's the per-user, shell-specific profile). You can then create or modify that script and it will execute each time the shell starts.

Using Your Profile
One common use for a profile is to define custom aliases, as I suggested earlier. Or you can add custom PSDrives (these are essentially mapped drives that exist entirely within Windows PowerShell). A less common use is to create a sort of super-shell that can do any management task you might need, based on the software products you're using in your environment. To really explain the super-shell concept, I first need to give a little bit of background information.
The product group within Microsoft that creates Windows PowerShell is responsible for many of the other core management technologies, including the Microsoft Management Console (MMC). The MMC provides a good analogy for just how Windows PowerShell works. When you run mmc.exe, you start with a blank console that isn't good for much. You add MMC snap-ins in order to create a functional console.
Once you've got your console configured the way you want, you save the console in a file with an .msc file name extension. This saved console file allows you to quickly reload your custom console at any time.
Rather than making custom MMC consoles, many admins just rely on the consoles that are installed along with the products they manage. Exchange Server, for example, creates a console that only contains the Exchange Server snap-in. Active Directory Users and Computers is another pre-created console that contains a single snap-in.
Windows PowerShell works in much the same way. The Exchange Management Shell icon installed with Exchange Server 2007 management tools is actually a shortcut to powershell.exe with instructions to load a console file. Shell console files have a .psc1 filename extension and contain a list of snap-ins to preload (much like an .msc file contains a list of snap-ins that the MMC preloads for you). However, you're not stuck using these pre-created shell consoles. You can create a custom console—just as you can with the MMC—and use that to accomplish all of your management tasks. Profiles play a key role in making this possible.

Making a Custom Console
To make a custom console, you should start by finding the full names of each snap-in you want to work with. Make sure all the necessary management tools are installed on your computer. Then, in Windows PowerShell, run Get-PSSnapin –registered. This will list all the registered, yet unloaded, snap-ins available. Then create or edit the appropriate Windows PowerShell profile. Add the Add-PSSnapin commands to load each snap-in that you always want to be available. This might include snap-ins for Exchange Server, System Center products, and third-party snap-ins, such as the PowerShell Community Extensions. Then save your profile (remember to digitally sign the profile if your Windows PowerShell execution policy requires it) and close the shell. Reopen the shell, and it will automatically load all of the snap-ins listed in your profile.
Another technique is to load all of your snap-ins into the shell (using Add-PSSnapin and the names of the snap-ins) and then run Export-Console to create a .psc1 console file that contains all the snap-ins you're currently using. You can then use this .psc1 console file to create a new Windows PowerShell shortcut that specifies the PSConsoleFile parameter and your custom .psc1 file. That shortcut would then use your console, loading up all the specified snap-ins automatically.

More Profile Tricks
I use my Windows PowerShell profile to perform a number of other useful tasks each time the shell starts. Here's some of what happens when the shell starts on my system:
  1. I run Get-Credential for my Domain Administrator account, storing the results in a variable named $cred. This makes $cred available throughout the shell. I can then pass it to the –credential parameter of various cmdlets, such as the Get-WmiObject cmdlet. Since $cred stores the password, I can use those cmdlets without being prompted for a password.
  2. I run Cd C:\ so the shell starts in the root of my computer's C: drive.
  3. I run New-Alias of Out-File to create an alias named "of." This is so I can then use "of" instead of "Out-File." I use this a lot, so having the short alias defined is very handy.
I try to create a test key in the shell's HKLM: drive, which represents the HKEY_LOCAL_MACHINE portion of the registry. If an error occurs, then I know the shell was launched without elevated privileges. I usually require elevated privileges for the work I do, and this is just a quick way for me to know up front whether I have elevated privileges before I get into serious work.
I define a custom function named Ping-Address that accepts a computer name or IP address and returns a True or False value depending on whether the computer is pingable. I use this function a lot in my work, so having it defined in my profile makes it globally available in my shell.

Cmdlet of the Month: Get-WmiObject
The astute observer might notice that I'm covering a cmdlet that I've featured before, but I want to let you in on a cool twist that it offers. I often see administrators try to retrieve Windows Management Instrumentation (WMI) information from multiple computers, whose names are listed in a text file, using this technique:
Get-Content c:\computers.txt | ForEach-Object
  { Get-WmiObject Win32_Service –comp $_ }
While this technique works, you don't really need it because Get-WmiObject can accept an array of computer names for the –computerName parameter. To achieve the same effect, you can just use this:
Get-WmiObject Win32_Service –comp (Get-Content
  c:\computers.txt)
Placing the Get-Content command in parentheses forces the shell to execute the command and place the results—an array of computer names—into the –computerName parameter.
Profile Beware!
Keep in mind that powershell.exe isn't the only application that loads the Microsoft.PowerShell profiles or the all-shells profiles. Many integrated development environments (IDEs) that provide Windows PowerShell support—including PrimalScript from SAPIEN Technologies, PowerGUI from Quest Software, and PowerShell Plus from ShellTools—also load these same profiles to provide an experience that parallels the powershell.exe host.
A large profile that loads lots of snap-ins may cause these applications to start more slowly because there are a lot of instructions for the Windows PowerShell engine to carry out before it becomes ready for the hosting application. In fact, even powershell.exe can take a while to start up if you've got a very large profile script to execute.
Snap-ins that define cmdlets with the same names can cause collisions as well. For example, if two snap-ins define a cmdlet named Get-User, then Windows PowerShell won't execute either cmdlet until you use the fully qualified name of the cmdlet to specify which one you want. That fully qualified name can get ugly, since snap-in names can be quite long. When I run into this problem, I usually just give up on the idea of having those two snap-ins loaded simultaneously. Instead, I'll load only the one I use most in my profile and then use a separate, "fresh" shell when I need to load and use the other snap-in.
Windows PowerShell profiles can provide a great deal of extra convenience to the shell. But keep in mind that if they're maliciously edited by a piece of malware, profiles can also cause a great deal of damage. You can protect your profile by digitally signing it and configuring Windows PowerShell to use its AllSigned execution policy, or modify the NTFS file permissions on your profiles—all of them—so that your normal user account can't modify them.

Don Jones is the author of Windows PowerShell: TFM and VBScript, WMI, and ADSI Unleashed. You can contact him through the PowerShellCommunity.org Web site.


Hey, Scripting Guy! Famous Last Words
The Microsoft Scripting Guys

The famous Greek philosopher Socrates (who died in the year 399 BC, shortly before the Scripting Editor was born) is perhaps best known for saying, "An unexamined life is not worth living." Most people are familiar with this saying. Remarkably enough, however, the Scripting Guys recently discovered that Socrates never said that "an unexamined life is not worth living." As it turns out, that was a mistranslation made by a medieval scribe hundreds of years ago. Instead, what Socrates actually said was this: "An unexamined XML file is not worth having."
Now there's a philosophical saying that actually makes sense! XML is becoming more and more popular these days; a quick search of one of the Scripting Guys' test computers uncovered more than 500 XML files used by various systems or applications.
Needless to say, a lot of very important data is now stored in XML format. And that's fine. Unless, of course, that data goes unexamined. And, sadly, that's often the case, if for no other reason than this: many people don't have any idea how to even go about examining an XML file. In particular, they have no idea how to query and search that XML file.
Note If you're not up on your Greek philosophers, Socrates was a big proponent of going around talking about things but not all that big on going around doing things; in fact, his wife, Xanthippe, referred to him as a "worthless idler." You can see why the Scripting Guys have a soft spot in their hearts for Socrates.
Of course, some of you who are reading this might be thinking, "Wait a second: didn't the Scripting Guys already discuss searching an XML file in a previous column ("Chasing Cars … and XML" at technet.microsoft.com/magazine/cc162506)? In that case, why are they revisiting that topic now; are the Scripting Guys so lazy that they're willing to write the same column over and over again?"
Believe it or not, we're even lazier than that. But that's not the primary reason we're revisiting this topic. In our previous column, we talked about working with an XML file that was structured like the code in Figure 1, with each and every property value representing a single node in the file.
Figure 1 XML file with nodes only
<?xml version='1.0'?>
  <INVENTORY>
    <COMPUTER>
      <os>Windows XP</os>
      <department>Human Resources
      </department>
      <name>atl-ws-001</name>
    </COMPUTER>
    <COMPUTER>
      <os>Windows XP</os>
      <department>Finance</department>
      <name>atl-ws-002</name>
    </COMPUTER>
    <COMPUTER>
      <os>Windows Vista</os>
      <department>Finance</department>
      <name>atl-ws-003</name>
    </COMPUTER>
  </INVENTORY>
That's nice, except—as people were quick to write in and point out—that's not the only way that XML files can be structured. In the preceding example, each computer in the file has its own node; in turn, each of those nodes has a number of child nodes (os, department, and name). However, it's also possible to construct an XML file in which the individual nodes don't have any child nodes; instead, additional property values are configured as attributes. You know, like the file shown in Figure 2.
Figure 2 XML file with attributes
<?xml version='1.0'?>
  <HARDWARE>
      <COMPUTER os="Windows XP" department="Human Resources">atl-ws-001</COMPUTER>
      <COMPUTER os="Windows XP" department="Finance">atl-ws-002</COMPUTER>
      <COMPUTER os="Windows Server 2003" department="IT">atl-fs-003</COMPUTER>
      <COMPUTER os="Windows Vista" department="IT">atl-ws-004</COMPUTER>
      <COMPUTER os="Windows Vista" department="Human Resources">atl-ws-005</COMPUTER>
      <COMPUTER os="Windows Vista" department="Finance">atl-ws-006</COMPUTER>
      <COMPUTER os="Windows XP" department="Sales">atl-ws-007</COMPUTER>
      <COMPUTER os="Windows Server 2008" department="IT">atl-fs-008</COMPUTER>
      <COMPUTER os="Windows XP" department="Human Resources">atl-ws-009</COMPUTER>
      <COMPUTER os="Windows Vista" department="Sales">atl-ws-010</COMPUTER>
  </HARDWARE>
Is that a problem? As a matter of fact, it is. The script we showed you a long time ago, in an issue far, far away, won't work with an XML file structured like the one shown in Figure 2. It's just not going to happen. And that is definitely a problem because many of you need to be able to read this type of XML file.
All you had to do was ask. Well, that and also wait a year and a half before we finally got around to addressing this issue.
Before we go much further, let's take a closer look at our XML file. In this case, we have a file with a primary node named HARDWARE; each individual computer exists as a child node of HARDWARE. In addition to this, each of these nodes has a pair of attributes: os (used to store the name of the operating system installed on the computer) and department (used to store the name of the department that owns the computer).
Suppose, as an example, that we would like to get a list of all the computers that are running Windows XP. Can we do that using a script? Of course we can:
Set xmlDoc = _
  CreateObject("Microsoft.XMLDOM")

xmlDoc.Async = "False"
xmlDoc.Load("HARDWARE.xml")

Set colNodes=xmlDoc.selectNodes _
  ("//HARDWARE/COMPUTER[@os='Windows XP']")

For Each objNode in colNodes
  Wscript.Echo objNode.Text
Next
Let's see what we have here. To begin with, we create an instance of the Microsoft.XMLDOM object; as the name implies, this is the object that enables us to work with XML files. Once we've created the object, we next set the Async property to False; this allows the script know that we want to load the document synchronously rather than asynchronously. Why does the script care about that? To be honest, it probably doesn't; after all, scripts (like Scripting Guys) are inanimate objects.
However, you should care. If we loaded the document asynchronously, the script would continue to run even if the document had not been fully loaded. That's not good: imagine the problems that might occur should you try to perform a task on a document that doesn't exist yet. Loading an XML file synchronously guarantees that the file will be fully loaded before our script proceeds.
By amazing coincidence, we're now at the point where we load our XML file. That's something we do by calling the Load method and opening the file C:\Scripts\HARDWARE.xml, which brings us to this line of code:
Set colNodes=xmlDoc.selectNodes _
  ("//HARDWARE/COMPUTER[@os='Windows XP']")
What we're doing here is we're using the SelectNodes method to query the XML file and to tell the script which XML nodes we want to retrieve. Notice the syntax, which, admittedly, is a little unusual.
To begin with, we need to specify the path within the XML file. Our XML file has a top-level node named HARDWARE, followed by a series of second-level nodes named COMPUTER. Each of these second-level nodes represents a single record in our XML data file. That's why our selectNodes query starts out like this:
//HARDWARE/COMPUTER
At that point we encounter this construction:
[@os='Windows XP']
This portion of the query is similar to a WHERE clause in a standard SQL query. With SQL we might use a database query similar to this to retrieve a list of all the computers running the Windows XP operating system:
SELECT Name FROM Hardware
    WHERE OS = 'Windows XP'
With the SelectNodes method we're doing something similar: we're asking to get back a list of all the computers where the os attribute (@os) is equal to Windows XP. And to indicate that this is a WHERE clause, we enclose the entire clause in square brackets.
Note This type of query is known as an XPath query. To find out more about XPath, look at msdn.microsoft.com/library/ms256115.aspx.
Like we said, it's a little weird, but it works. What if we wanted to get a list of all the computers that belong to the Finance department? That's no big deal; our call to selectNodes would look like this:
Set colNodes=xmlDoc.selectNodes _
  ("//HARDWARE/COMPUTER" & _
   "[@department='Finance']")
Again, our WHERE clause is enclosed in square brackets, we preface the attribute name (department) with the at sign (@), and then we indicate the value we're interested in.
Simple, huh?
After we have called the selectNodes method, we can then echo back the computer names simply by looping through the collection of returned values and echoing back the Text property, as shown in the following:
For Each objNode in colNodes
  Wscript.Echo objNode.Text
Next
And what sort of information can we expect to get back? To tell you the truth, we fully expect to get back information like this:
atl-ws-001
atl-ws-002
atl-ws-007
atl-ws-009
In other words, it returns the names of all the computers that are still running Windows XP.
Incidentally, you're not limited to reporting back just the computer name; because the computer name is the "default" value for each node, that's simply what we get back if we reference the Text property.
Alternatively, we could specify exactly which attribute values we want returned. For example, take a look at this modified For Each loop:
For Each objNode in colNodes
    Wscript.Echo objNode.Text
    Wscript.Echo objNode.Attributes. _
      getNamedItem("department").Text
    Wscript.Echo
Next
As you can see, in this loop we're still echoing back the value of the Text property; however, we've also tacked on this line of code:
Wscript.Echo objNode.Attributes. _
  getNamedItem("department").Text
In this case, we're using the getNamedItem method to retrieve the value of the department attribute; we then echo back the Text property for that attribute. That's how we can specify which attribute values we do (and do not) echo back to the screen. (Note too that we added on a Wscript.Echo command to put a blank line between records.) When we run this script, we should get back the following:
atl-ws-001
Human Resources

atl-ws-002
Finance

atl-ws-007
Sales

atl-ws-009
Human Resources
And yes, you can write more complex queries if you choose. For example, suppose you'd like a list of all the computers running either Windows XP or Windows Vista. In SQL, you'd do that by writing an OR query. Guess what? You do the exact same thing when querying an XML file:
Set colNodes=xmlDoc.selectNodes _
    ("//HARDWARE/COMPUTER" & _
     "[@os='Windows XP' or " & _
     "@os='Windows Vista']")
See what we did here? Inside a single set of square brackets we provided two criteria: @os='Windows XP' or @os='Windows Vista'. That's all you have to do, and it's going to give us back a report that looks like Figure 3.
Figure 3 Result of OR query
atl-ws-001
Human Resources

atl-ws-002
Finance

atl-ws-004
IT

atl-ws-005
Human Resources

atl-ws-006
Finance

atl-ws-007
Sales

atl-ws-009
Human Resources

atl-ws-010
Sales
If you double-check the XML file, you'll see that the computers running Windows XP or Windows Vista are included in the output; computers running Windows Server 2003 or Windows Server 2008 are not included in the output. Nor should they be.
You have a question? You'd like to write a more restricted query, one that returns only those computers that are running Windows XP and belong to the Finance department? As you might have guessed, that's easy; all you have to do is write an AND query like this:
Set colNodes=xmlDoc.selectNodes _
    ("//HARDWARE/COMPUTER" & _
     "[@os='Windows XP' and " & _
     "@department='Finance']")
Doing that will then return the following dataset:
atl-ws-002
Finance
Why just one item in our report? You got it: because the Finance department has only a single computer that's running Windows XP.
We hope we've helped all of you to deal with this particular type of XML file. If it turns out that there's yet another type of XML file lurking out there, well, good luck with that.
Because we started this month's column off with a famous saying, we thought it would be fitting to end the column on a similar note. With that, we'd like to close with the words of Kaiser Wilhelm II, words we Scripting Guys hold dear: "I herewith renounce for all time claims to the throne of Prussia and to the German Imperial throne connected therewith."
Dr. Scripto's Scripting Perplexer
The monthly challenge that tests not only your puzzle-solving ability but also your scripting skills.

October 2008: VBScript Fill ‘Er Up
To solve this puzzle, simply fill in each row with the name of a VBScript function. When you're done, unscramble the letters in the blue squares to find one more function name.

ANSWER:
Dr. Scripto's Scripting Perplexer
Answer: Octobe 2008: VBScript Fill ‘Er Up

The Scripting Guys—Greg Stemp and Jean Ross—work for Microsoft.


The Desktop Files Customizing Windows Deployment Services
Wes Miller

Contents
Replacing What You've Got
Looking at the Stack
Network Boot Program
WDS PXE Provider
TFTP Daemon
Boot Configuration Data Store
Windows PE
Transport Server
Custom Multicast
Wrapping Up
For three months I've been delving into Windows Deployment Services (WDS)—its origins, followed by an overview, followed by a closer look at some advanced topics such as WDSUtil and multicasting. In this final installment of the series, we're going to take a look at where and how you can customize and configure WDS to meet the needs of your organization. Most Microsoft tools are designed with some amount of configurability. But it's when the rubber hits the road that you actually find out whether the tools will meet your needs or, more often, that they require some tweaking to work in your scenarios.

Replacing What You've Got
A question I've often been asked by readers lately is, "I've got x (an existing deployment technology)—will WDS work for me, and does it have parity feature-wise with x?" With the deprecation of Automated Deployment Services (ADS), one area of particular interest is, "How can I perform high-volume rapid deployment or reprovisioning of servers—can WDS do that for me?"
While WDS wasn't designed as a 1:1 replacement for ADS, and in fact is missing a few key components to be considered a real substitute, with a bit of work you can make WDS stand in for ADS. Similarly, if some aspect of WDS isn't working for you as is, you'll find that many of its components can be replaced—with varying degrees of complexity and engineering. Let's take a look at deployment via WDS and examine the pieces you might want to customize and how you could do so.

Looking at the Stack
Figure 1 shows the components in the WDS deployment process. Each of these steps can be customized to a degree. I've color coded each step to reflect what I believe to be the complexity (generally the development skills) involved in replacing or customizing it. The darker the blue, the harder it will likely be to customize that step.
Figure 1 Complexity of customizing WDS
How difficult it is to customize each step really, of course, depends on both the skills of your team (development or scripting) and your understanding of WDS, Windows Imaging Format (WIM), Active Directory, or any other technologies you want to integrate, such as SQL or ADSI. Let's examine each of these steps; think about the ways in which you might want to customize them and the methods you would use.

Network Boot Program
It's doubtful you would need to create a custom Network Boot Program (NBP) to replace those provided with WDS, but it is possible. WDS includes NBPs for use with headless systems (generally servers) or systems that you may or may not prompt for F12, and so on. These NBPs can either be prestaged into Active Directory using WDSUtil, or you can replace Startrom.com with the NBP you want to use for all systems that aren't pre-staged (for example, if all of your systems are headless, or you never want to prompt F12).
Unfortunately, there isn't much documentation available on creating your own NBP (see msdn.microsoft.com/library/bb870970.aspx for some information). An NBP is a very small 16-bit executable with severe limitations and requires specific programming skills. I generally recommend using the existing NBPs provided with WDS unless you are trying to meet a very specific requirement and you have a development team with the appropriate skills.

WDS PXE Provider
With Remote Installation Services (RIS), common feedback we received from customers was the desire to use a data store other than Active Directory for client system information—most often, SQL Server. With WDS, the design includes a pluggable infrastructure for Pre-Boot eXecution Environment (PXE) providers. This means that with development work, you could use another backing store besides Active Directory for PXE information.
WDS comes with its own provider that uses Active Directory; System Center Configuration Manager (SCCM) now hooks in to WDS as well and implements a provider of its own. Documentation on writing your own provider is very scarce and there isn't much sample code available (though the Windows SDK provides some documentation and a couple of samples), so the task isn't for the meek. Unless you've got very specific requirements for this aspect of the boot process, I'd again recommend not attempting a custom PXE provider.

TFTP Daemon
Sometimes customers have invested in their own Trivial File Transfer Protocol Daemon (TFTPD) before WDS comes along. Since PXE servers don't play well together, this often means settling on only one.
In my experience, this usually means taking an existing, typically Linux-based TFTPD and coaxing it into booting other OSs. With the original infrastructure RIS used you couldn't do this. But when RAMDisk booting became the norm, you could, and it is still possible to do this using WDS-based boot images.
One thing to bear in mind is that you are wandering into a technically unsupported area as far as Microsoft is concerned, and one that can most certainly cause problems that are hard to diagnose. Moreover, with the enhanced TFTPD in WDS, you may well be walking away from better performance. Ideally, I recommend using the existing WDS TFTPD and trying to use PXE timeouts, prestaging, and/or network edges to define which clients boot from which PXE server instead of trying to bend an existing server to fit.

Boot Configuration Data Store
With RIS, it was never possible at the boot level to specify what should be booted—you always had to go through the OS Chooser and select an option (whether setup, Windows PE (Windows Preinstallation Environment), or something else entirely). WDS, since it uses a full loader for network booting, also supports customizization of the Boot Configuration Data (BCD) store served up to clients. The default BCD for each architecture lives under RemoteInstall\Boot\<arch>\Default.bcd, where <arch> is the specific architecture of the system being booted.
You can customize this BCD for each client, and the client will use it for booting. You could, for example, set up one BCD entry to start setup, another for running Windows Recovery Environment (WinRE), and yet another for running a memory test—or you could have a fully automated setup entry be the default and a manual (or custom setup experience) be a user-selectable option. (For more information, go to "How to Modify the BCD Store Using Bcdedit" at go.microsoft.com/fwlink/?LinkId=115267.)

Of course, the majority of WDS's heavy lifting occurs in Windows PE—so tweaking Windows PE to meet your needs can be a critical area of focus for a custom experience. WDS, by default, provides a very standard template for setup, which includes the full setup experience. Sometimes your deployment needs may mean this won't work for you. In that case, you can create your own Windows PE boot image. Let's start at the beginning.
In addition to using the BCD to indicate which image to use, you can also specify the image by customizing the machine account object (MAO) for a computer in Active Directory. In RIS, a specific MAO attribute stored each item (which Startrom and Unattend—SIF—file to use). With WDS, these are stored as name-value pairs under the entry netBootMirrorDataFile. For example, the boot image and the Unattend file to be used by a given computer are stored in this entry. The form of the entries is as a semicolon-delimited list of name-value pairs. The entries to modify the boot image and the Unattend file are BootImage and UnattendFilePath, respectively.
Of course, you may find that you want to drop the existing setup experience completely and just build your own. Perhaps you need more configurability, more automation, or a build automated by SQL Server. You might want do as some customers did with RIS and Windows PE early on and build your own front end for setup. The key tasks you need to accomplish regardless of the setup experience you write are:
  • Finding out any per-machine or per-user information. This information may be obtained from user input or from SQL Server or from a text file, for example.
  • Copying and applying a setup image to the client system. This task can be accomplished using setup directly, using ImageX to apply an image from a network share, or via multicast (simply copy the image and apply it via ImageX).
  • Provide an Unattend file (such as Unattend.xml or sysprep.inf, depending on the version of Windows being deployed) for setup to complete.
  • Automate any post-setup migration steps that you want to perform (or any steps to apply roles in the case of Windows Server 2008).
ADS initiated the concept of task sequences that allows repeatable steps to be assigned to one or more computers. Since ADS was not designed or supported for use with Windows XP, you couldn't use it to deploy the OS. But ADS task sequences are really just structured scripts, and you can perform the same steps using a custom setup process.
I've been a fan of Microsoft SQL Server for quite some time (since the SQL Server 6.5 release), so my instinct is to build such a structure using SQL. Of course, you need to add in the SQL functionality to your build of Windows PE to do so. On top of that, you could either write your own GUI—an HTML application (HTA) or compiled executable—or use Windows Script Host (WSH) to perform a minimalist command-line-only setup experience. HTA or WSH would also need to be added to Windows PE in order to utilize them.
The complexity in designing your own setup experience completely depends on your skills and your imagination. I've seen quite elegant systems defined using only SQL and WSH or HTA—which are relatively easy skills for someone to pick up. It's very important, however, to keep in mind the constraints I have mentioned in previous columns:
  • Windows PE features no Windows on Windows (WOW) subsystem, so you need to compile once for each architecture you intend to support.
  • You cannot use Visual Basic 6.0 if you need to deploy via x64 or IA64 Windows PE.
  • You can use Visual Studio 2005 or 2008 to build applications, but you must build an unmanaged Visual C++ application, as there is no Microsoft .NET Framework—any version—supported on Windows PE.
  • Without a .NET Framework, you also are unable to use Windows PowerShell for automation.
You can, of course, use a third-party imaging utility via WDS as well, if you're willing to write your own setup experience. While I think that the WIM format and ImageX can meet most deployment scenarios, there may be certain requirements your existing imaging tool meets for you.
Similarly, certain scenarios may require custom partitioning—you may be deploying Windows Vista with BitLocker, or you're building Windows XP systems and storing profile data on a second volume, or perhaps you're deploying a Windows Server system and want to create a separate volume on the same disk for logging. Any of these requires automation of DiskPart, which, as in previous versions, can be done by feeding a script (any file format) to DiskPart that contains the commands you want to execute and ends with exit—to end DiskPart.
Creating your own setup experience isn't for the faint of heart, as you are basically rebuilding the setup executable (or at least mirroring its functionality), and there can be quite a bit to design and build. But it comes down to how much functionality you want to build into it by default and what you are building it in (HTA or WSH or a compiled programming language).

Transport Server
If you aren't using most of WDS's in-box functionality (such as Active Directory) or are engineering your own complete custom solution, Transport server may meet your needs and not bring requirements you don't want. The table in Figure 2 (reproduced from "Using Transport Server" at go.microsoft.com/fwlink/?LinkID=115298) describes what is included as a part of the WDS Transport server role.
Figure 2 Deployment Server vs. Transport Server
 | Deployment Server | Transport Server
Server requirements | Requires Active Directory Domain Services (ADDS), Dynamic Host Configuration Protocol (DHCP), and Dynamic Name Services (DNS) in the environment. | Does not require other servers in the environment.
PXE | Supports PXE boot with the default PXE provider. | A PXE provider is not installed so you must create a custom PXE provider.
Image server | Includes the Windows Deployment Services image server. | Does not include the Windows Deployment Services image server.
Transmission method | Allows both unicasting and multicasting. | Allows only multicasting.
Management tools | Is managed using either the Windows Deployment Services MMC snap-in or the WDSUTIL command-line tool. | Is managed only by the WDSUTIL command-line tool.
Application on the client computer | Uses the Windows Deployment Services client (which is basically Setup.exe and supporting files), Wdsmcast.exe (which is included in the Windows AIK), or a custom multicast application. | Uses only Wdsmcast.exe or custom application.
When I say that Transport server is a complicated item to implement, it isn't the role itself that's difficult; that, of course, is easily deployed (see Figure 3). It is the custom setup implementation around Transport server that requires work. Using the Transport server role effectively removes most of the ease of use built into WDS as a role.
Figure 3 Transport Server may be useful for custom deployment scenarios

Custom Multicast
Whether or not you are using the Transport server role—but especially if you are—there is a good case to be made for using multicast if you are doing multi-system deployment. ADS featured a very powerful multicast feature, which you can duplicate using WDS with multicast. WDS features multicast on its own, but if you are building your own custom solution, you can leverage multicast using WDSMCast, as I mentioned last month (see Figure 4). Remember that you need to transfer the image file(s) to be deployed, and then they must be applied. Generally this means you need enough local disk space for the image to be stored and then applied.
Figure 4 Running WDSMCast

Wrapping Up
WDS delivers quite a bit of power on its own—most likely enough for many organizations to meet their needs. But if you're looking to build your own solution that pushes beyond the boundaries of WDS, you can certainly do so—you're limited only by your imagination, your schedule, and your skills.

Wes Miller is a Senior Technical Product Manager at CoreTrace (CoreTrace.com) in Austin, Texas. Previously, he worked at Winternals Software and as a Program Manager at Microsoft. Wes can be reached at technet@getwired.com.


Security Watch Revisiting the 10 Immutable Laws of Security, Part 1
Jesper M. Johansson

Back in 2000, Scott Culp published an essay called "10 Immutable Laws of Security." It is one of the best essays on security I've ever read. The information that he presented remains fundamental to all work in information security, and I recommend that you check it out (and even print it out) if you haven't already done so. You can find it at microsoft.com/technet/archive/community/columns/security/essays/10imlaws.mspx.
The essay was received with varying responses. Some people commented sarcastically that it was a way for Microsoft to avoid fixing what were perceived as many serious problems. Others considered it some of the most foundational writing on security and have plagiarized it at will due to its sheer importance. Some of my favorite responses, though, are those in which people have been inspired to create their own lists, such as the one that is available at edgeblog.net/2006/10-new-immutable-laws-of-it-security.
In the eight years since this essay was published, a lot has happened in the field of security. Virtually every major worm of note has been released. We have entered a state of information warfare (with organized crime, political entities, and so on). A variety of new words and phrases have become a part of common vocabulary, including phishing, pharming, botnet, spyware, and cross-site request forgery. We have some of the most sophisticated rootkits ever created running on Windows. We have new OS releases that are dedicated, to a large extent, to security and other operating systems where security is still largely ignored.
Social engineering has evolved into a major threat. Data breaches, such as when one major retailer exposed 94 million credit card numbers, have become familiar news stories (and yet people continue to shop at such stores). The United States and United Kingdom governments, collectively, have managed to misplace private information on a significant percentage of the inhabitants of the western world (and yet people still file private information with those governments). And a huge amount of security theater has entered our lives—and our airports.
I think it is time to take another look at the laws. Given all the changes we have seen come about in the first part of this century, can we still claim that these laws are immutable? If they are—if they have survived the past 8 years—it is probably safe to say they will survive the next 10.
In this three-part series, I will take a critical look at each of the 10 laws. This month I cover laws 1 through 3. In next month's installment, I will look at laws 4 through 7. Then in the final installment, I will examine laws 8 through 10 and offer some additional food for thought and comments that seem reasonable in light of what has happened since the laws were originally written in 2000.
Law 1: If a bad guy can persuade you to run his program on your computer, it's not your computer anymore.
This law states, effectively, that any software you execute on the computer can control that computer. When the immutable laws first came out, the current operating systems from Microsoft were Windows 98, Windows Me, and Windows NT 4.0. Today we have Windows Vista and Windows Server 2008.
On Windows 98 and Windows Me, any software you executed had full control over the computer. Windows NT 4.0 had a very solid underlying security model, but if you ran it as an Administrator, you effectively downgraded its isolation model to that of Windows 98 and Windows Me. It was possible to run Windows NT 4.0 as a non-administrator, but it was quite painful and very few organizations did so (you could probably count the total number of organizations that did this without taking off your shoes).
Say you did run Windows NT 4.0 as a non-administrator. Then did Law 1 even hold true when it was written? The answer is yes. First, Windows NT 4.0 had a large number of significant holes. For instance, there were permissions that could have been tighter, particularly on kernel objects and in the registry. There were also many types of attacks that hadn't been discovered yet, but that experts expected would surface. For example, in 1999 people hadn't realized that having processes running as elevated users on the interactive desktop could compromise the computer. It was not until 2002 when Chris Paget published his white paper on Shatter attacks, "Exploiting the Win32 API," that this became mainstream knowledge (seclists.org/bugtraq/2002/Aug/0102.html).
Had Microsoft foreseen the Shatter attacks when Law 1 was drafted? No, not really. Microsoft simply recognized a simple fact: there are very few true security boundaries that can keep an application executing on a computer from taking over that computer.
Windows Vista and Windows Server 2008 are two generations removed from Windows NT 4.0. Have they invalidated Law 1? Are there any other operating systems that do? It depends. There are certainly more solid security boundaries in the new OSs, and there were some experimental operating systems back in 2000 that had proper security boundaries. However, there are still only a few of those boundaries. For example, Code-Access Security in the Microsoft .NET Framework is a security boundary. It is designed specifically to keep code executing within the sandbox from affecting the underlying operating system.
Iframes in Internet Explorer provides another security boundary. But Iframes doesn't affect access to the OS itself, only access between parts of content on a Web page. Protected Mode in Internet Explorer, shown in Figure 1, is an OS-level security boundary. Its purpose is to prevent code that executes within a browser from affecting the underlying operating system—without user action. There are a few others, such as standard user accounts, that are supposed to prevent a user account from affecting the underlying OS or any other user.
Figure 1 Internet Explorer 7 uses a security boundary called Protected Mode
It is extremely important to understand what the term "security boundary" means. It does not mean there is an inviolable wall that is guaranteed to provide impermeable and indefinite isolation. What the term really means is that the software vendor, Microsoft for example, is responsible for fixing any violations of that boundary by means of a security patch. Software will always have bugs, and we will undoubtedly continue to discover more violations that the software vendors will continue to patch. Over time, they should improve their software to prevent those vulnerabilities in the first place. That may be considered as confirmation that Law 1 still holds.
There is one more crucial piece I need to consider, however. You may have previously noticed the phrase "without user action." Law 1 isn't really about shortcomings or vulnerabilities in software. It is really about vulnerabilities in people! The key phrase is "persuade you." If a bad guy can persuade you to run his program, he can probably persuade you to do so in a context that, by design, gives the program elevated privileges.
Even if you do not have administrative privileges, it may not matter. You, as a standard user, still have access to lots of juicy information: your bank files, love letters, pictures, videos, and company confidential data. All of this data is potentially interesting to an attacker, and all of it can be read without any elevated privileges. In terms of the information you manage on your computer, executing a malicious program hands over everything you do to the attacker. Therefore, if you define "your computer" as "the data you manage on your computer," you can ignore any discussions about privilege and simply conclude that Law 1 holds.
Even without splitting hairs over the definition of your computer, Law 1 still seems to have withstood the test of time. The point of Law 1 is to establish the fact that you, as the operator of a computer, must take responsibility for the software that you run on that computer. If you install a malicious driver or an evil video codec, you have handed off complete control of that computer to a criminal!
While software vendors can do a lot to prevent accidental compromise, which is really what the security boundaries are all about, intentional execution of malicious software will generally trump all those protective measures. This is why user education is critical in addition to ensuring that users do not have permission to perform administrative tasks. Therefore, it is safe to say that Law 1 holds true today, but possibly with a slight modification to the definition of your computer.
Law 2: If a bad guy can alter the OS on your computer, it's not your computer anymore.
On the surface, this law seems pretty straightforward. Obviously, if the bad guy can alter the operating system on your computer, you can't trust the computer anymore. But what is meant by OS has changed as computing needs have evolved. Many years ago I wrote a definition of the term OS for The Blackwell Encyclopedia of Management (managementencyclopedia.com). The definition said something about an OS managing access to input and output devices, access to hardware, and such things. Now, I never got a copy of the encyclopedia and my original submission has been lost to history, but I'm certain I didn't mention that an OS includes solitaire, a tablet input system, and video transcoders.
As computing has grown increasingly complex, OSs have grown to support many more capabilities. Further complicating the issue, OEMs often include their own assortment of additional software—this additional software typically ranges from somewhat useful to outright harmful. And some of this additional software duplicates functionality already built into the core operating system.
For example, a default installation of Windows Server 2008 Enterprise Edition has a disk footprint of more than 5 gigabytes. Windows Vista Ultimate Edition includes more than 58,000 files, comprising more than 10 gigabytes. And there are more than just files making up the OS. There are configuration settings, many thousands of them; and there are daemons or services.
Any and all of these things are part of the OS. It is a collective term that includes all of the files, all of the configuration settings, and all of the services, as well as all of the runtime objects—semaphores, named pipes, RPC endpoints—created by the files and configuration settings. Even such highly abstract constructs as the system time and certain types of data, such as event log contents, should be considered part of the OS.
Given how the OS has grown and evolved, is it really the case that modifying any one of those files constitutes making the computer not trustworthy? The direct answer is no. For example, Windows Vista comes equipped with edlin.exe, the old line editor from MS-DOS. I can't say for sure, but I would bet a grande, triple-shot mocha that edlin.exe has only been invoked twice across all installed copies of Windows Vista since the operating system was released. Both times were about three minutes ago, when I was trying to remember the syntax. If someone modifies edlin.exe or some other file that nobody ever uses, does it really mean that it is no longer your computer?
Edlin.exe is indisputably part of the operating system, but if nobody ever executes the file, how is a modification to it going to result in your computer being compromised? The answer, of course, is that it won't. Modifying a part of the operating system that is never used will not compromise your computer. And there are many parts of the OS that are never used.
The indirect answer, however, is yes. You cannot simply look at whether anyone executes a file to see whether its modification may result in a compromise of your computer. The problem is more subtle than that. Take a look at the access control list (ACL) for edlin.exe, shown in Figure 2.
Figure 2 The ACL for edlin.exe is very restrictive
The ACL for edlin.exe is very restrictive. Only the TrustedInstaller service has rights to modify that executable. This is very important: it means that there is an indirect effect for a bad guy modifying that file on your computer. The act of modifying edlin.exe does mean it is no longer your computer. It is the fact that the malicious user has the ability to modify edlin.exe that is key here. If the bad guy can modify that file, he can modify any file, which means you can no longer trust anything on the computer.
The OS will protect itself. Services are protected from unauthorized modification. Configuration settings are protected from unauthorized modification. Files on disk are protected against unauthorized modification. Even the semaphores and RPC endpoints used by the operating system are protected against unauthorized modification. If an attacker can modify any one of those protected objects, he can modify all of them and, quite possibly, already has.
This is a critical point. With several of the immutable laws, it is not the act of doing something that means your computer is compromised. The thing that matters only is that someone has the ability to do something. This is a point that must not be overlooked. In every aspect of computer security, you must always remember that capability is often far more important than action actually being taken.
If a computer is wide open to the Internet and goes unpatched for months, is it still trustworthy? No. That computer must be considered compromised. You just can't trust anything on a system that could have been compromised. (I said this same thing five years ago in my article "Help! I Got Hacked. Now What Do I Do?" available at technet.microsoft.com/library/cc512587.) If you are dealing with a skilled adversary, a compromised system may not even show any signs of having been compromised. The system may look perfectly normal.
Without question, Law 2 still holds. If an attacker has the ability to modify any protected object on your computer, it is not your computer any longer. Just remember, it is the ability to modify those objects that matters, not whether it has actually been attacked.
Law 3: If a bad guy has unrestricted physical access to your computer, it's not your computer anymore.
This law was critical in 2000. A large number of people didn't fully understand what you could do with physical access to a system. In fact, even some government agencies that ought to have known a whole lot better failed to grasp this fundamental point. At that time, security guidance recommended setting the Allow shut down without logon option to disabled. That causes the Shut down… button on the logon screen to be grayed out. The theory behind this was that in order to shut down the computer, the user must first log on so there would be an audit record of who shut down the system.
This is a case study in flawed thinking. To have access to the Shut down… button on the logon screen, you actually have to sit down at the console. And if you are sitting at the console and you really want to turn off the computer, you can usually use that big round button on the front of the computer—or even the power cord. System off. No audit trail.
Windows 2000 includes a security setting called Allow undock without logon, and this option is still available in Windows Vista, as shown in Figure 3. The principle was the same. In order to undock a laptop from its docking station, it's necessary to first log on to the system.
Figure 3 Why wouldn’t you steal both the computer and the dock?
The actual security value of this setting is extremely dubious. I think the theory was that if just anyone could walk up to the laptop and undock it, someone could easily steal the computer. Now, not that I would ever steal a laptop, but if I were going to do so, this preventative measure wouldn't deter me. I'd probably just grab the laptop and the docking station together as a single package. Heck, I'd just steal the network cable and the power cord as well. Talk about a meaningless security tweak!
The point about physical access really hit home when Petter Nordahl-Hagen created his Offline NT Password & Registry Editor. His creation was simply a Linux boot disk with an experimental NTFS file system driver that permitted read and write access to an NTFS volume. The software on the boot disk would mount the registry on the local machine and write a new password for the Administrator account into the SAM (software asset management) hive. All you needed was physical access to the system and one or two minutes.
Tools like this one are exactly why Law 3 was written in the first place. In fact, Nordahl-Hagen's tool was used in many demos. Unfortunately, the point never quite made it through to the majority audiences. I personally used the tool in some demos, but then I stopped using it after I tired of being asked, "How do we make sure none of our users know about tools like that?" and "What is Microsoft doing to fix this problem?" An alarmingly large portion of the IT industry just didn't want to accept or understand that physical access would trump all else.
In that environment, Law 3 was a very important statement. Yet critics assailed it without mercy. It was derided as an attempt by Microsoft to avoid having to fix any problem that could, even in the remotest way, be tied to physical access. Law 3 was actually used in several cases to dismiss vulnerability reports, including the Offline NT Password & Registry Editor hack. However, blocking attackers who have physical access to the system is really only possible in one way: by ensuring they can't get to anything.
That's where the potential chink in the armor of Law 3 lies. Since the laws were written, full disk encryption technology has become a viable solution. With full hard disk encryption, more correctly termed full-volume encryption, an entire volume (what is known as a partition in other operating systems) can be encrypted. So if the entire boot volume (in other words, the volume with the OS on it) is encrypted, the question we have to ask is does Law 3 still hold?
The answer is a firm probably. First, the decryption keys have to be stored somewhere. The simplest place to put the keys, and the default option in BitLocker, is in a Trusted Platforms Module chip in the computer. By doing so, the computer will boot unattended. Once the computer is booted, a resourceful and well-funded attacker with permanent physical control over the computer can attack it in any number of ways. Because the computer can now be connected to an arbitrary network, there may be network-related ways to attack the system.
The attacker may, for example, read or write memory by means of a direct memory access (DMA) device, such as a USB flash drive. Once the computer is running, all bets are off when it comes to an attacker with physical access to that computer.
If the keys are not stored on the computer itself, the attack then hinges on whether the attacker can obtain or guess the keys. If a PIN code is used to boot the computer, the attacker can probably guess the code with relatively little effort. If the keys are stored or derived from a separate hardware device, such as a USB flash drive or a one-time password fob, the attacker must have access to that separate device. There are certainly ways to obtain those keys or make life very uncomfortable for the person who has access to them, though the effort required is likely quite a bit higher.
You could also interpret Law 3 in a slightly different way: "If the attacker has physical access to your computer, it is likely the computer has been stolen and, therefore, it is unlikely you will get it back." From that perspective, it really is not your computer anymore. And from that perspective, it may not matter to the attacker whether he can access the data on your computer or not. However, that really is not what the spirit of Law 3 is about. It meant the attacker had access to the data on your computer and not just the computer itself.
All things considered, Law 3 does still apply. It is true that certain technologies available today go a long way towards stopping many attackers with physical access and thus minimize the number of attackers able to access data on a computer that employs a safety measure. That said, the capabilities of the attacker always define how much the attacker can actually achieve, and new technologies address many of the 10 immutable laws—to an extent. But physical access still offers ways, though more complex, into a system.
So far the immutable laws of security are proving very resilient, both to technological advances and to time. Of the first 3, Law 3 is on the shakiest ground, yet even it still holds in some cases. It is, however, the one that has the most readily available and robust mitigations in place. I'll be back in the next two issues of TechNet Magazine to continue this discussion and determine whether Laws 4 through 10 are still immutable.

Jesper M. Johansson is a Software Architect working on security software and is a contributing editor to TechNet Magazine. He holds a Ph.D. in Management Information Systems, has more than 20 years experience in security, and is a Microsoft Most Valuable Professional (MVP) in Enterprise Security. His latest book is the Windows Server 2008 Security Resource Kit.


Field Notes High-Capacity Color Bar Code
Gavin Jancke

We are all so familiar with bar codes that they often go unnoticed—they are almost ubiquitous in everyday life, appearing everywhere from food packaging to shipping labels. The original 1D bar codes that appeared in the retail industry in 1974 comprised vertical bars of variable widths and stored a 12-digit number. Bar codes evolved as computing power increased, and black-and-white 2D bar codes (matrix codes) were developed in the late '80s, marking increased data density from just a few digits to a few hundred bytes.
In 2003 I began working on a counterfeit-proof, biometric identification card as a project for Microsoft Research. I needed to store not only the cryptographic text, such as name and age, but also fingerprint or iris scan templates. In order to store all the necessary data, the existing black-and-white bar code would have been larger than that ID card itself, so clearly a new approach was needed.
I had to determine how to store more than one bit of information per symbol, which had been the standard for black-and-white 2D bar codes. By using colors rather than black and white, I could increase this to two or three bits of data using four and eight colors, respectively.
I also experimented with the symbol shapes to see if they could be packed together more tightly without compromising their registration when being scanned. Initially hexagons seemed like a good possibility, but the shapes melded together and the computer couldn't discern them clearly enough.
I eventually settled on triangles because they intersect neatly in a compressed space, yet retain their identity as distinct shapes. In using rows of triangles in multiple colors, I could increase the density capability into 3D.
There were a lot of variables with which to contend. One of the biggest challenges of working with color is variance in printing specs and in lighting conditions. Although color may be easily discernible to the human eye (at least for those who are not color blind), it is not as clearly discernible to a computer.
I used several different approaches to ensure that the software could read color accurately. I created a very small, nearly unnoticeable palette within the bar code that contained the reference colors that existed in the actual bar code. This provided a calibration mechanism that specified exactly what constitutes a particular red, for example. I also tried more advanced techniques such as customized and lightweight color-clustering algorithms that take advantage of color space theory to categorize the colors into sets. Both approaches work equally well and complement one another.
One of my goals for this project was to write a program that would enable cell phones to read the bar codes. With cell phones, one has to consider poor lighting and out-of-focus, fixed, focal-length lenses. A further challenge was to do all of this computer vision on integer-based, relatively slow cell phone CPUs, so the team needed to come up with some solid theory on how to detect the bar code in the image while touching as few pixels as possible. The scan needed to work within a range of read qualities, such as bright in-focus LCD panels to lower-light, out-of-focus physical print.
The final algorithm decodes bar code images on real-time camera video frames faster than the actual frame rate—about 10ms on a 200Mhz CPU. In an image, the smallest number of pixels a bar code can be for the smallest data payload and still be detected is about 45 pixels square.
Making all of this work under varying conditions took more than two years to refine. This technology has recently been licensed to third parties, and there will be some exciting applications of it later this year.
Gavin Jancke, Director of Engineering for Microsoft Research, runs the Advanced Development Group and Web Experience teams. Previously he worked as a Product Support Engineer, Software Engineer for SQL Server, and served on Bill Gates' executive technical advisory staff.


Windows Confidential Work Harder, Not Smarter
Raymond Chen

For software developers, one of the great benefits of Windows Error Reporting is that they can get real-world information about how their programs fail. These error reports are available to any software vendor who registers online at winqual.microsoft.com. Sure, you can run all sorts of tests in your labs, but there is always a gap between lab conditions and real-world conditions. This can result in bugs that never show up in your testing but that drive your customers crazy.
I was told a story a few years ago by a programmer who was assigned a task from his product's reliability team. The team wanted to address the top reasons for crashes and hangs in a particular component. After some negotiation, both parties agreed to set a goal of reducing the number of these types of failures by a factor of 2. They wanted to fix bugs that are responsible for at least 50 percent of the crashes and hangs in that component, as measured by Windows Error Reporting.
We all know that failures are not uniformly distributed. Some are rare, others are common, and still others are rampant (relatively speaking). Engineering is a game of trade-offs; given finite resources (time, money, and brainpower), you have to dedicate your resources to where they will have the greatest effect. Since the goal here was to reduce the number of crashes and hangs, the sensible approach was to study the most common failures reported by Windows Error Reporting, understand the root causes, and then try to fix these common failures.
Windows Error Reporting provides real-world information on program failures
When the programmer dug into the failure data, it became clear that five particular crashes and hangs accounted for more than 60 percent of the reports. If the programmer could fix the bugs that were causing these five failures, he'd reduce the number of crashes and hangs by a factor of 2.5, well above the target factor of 2.
Microsoft's own analysis of Windows Error Reporting data shows that the Pareto principle is a surprisingly good rule of thumb for Windows crashes and hangs; about 20 percent of bugs cause 80 percent of failures. The same study revealed an even more surprising result: just 1 percent of bugs cause 50 percent of errors. Not all bugs are created equal, but it's quite an eye-opener to learn how heavily skewed the distribution is.
After studying those five failures more carefully, the programmer came to the realization that all of them had the same root cause. The crashes and hangs were just different manifestations of the same underlying bug. So he developed and tested a fix and, in conjunction with the folks in quality assurance, set into motion the steps for including the fix in the next patch. With that one bug fix, the theoretical failure rate for the component instantly dropped by a factor of 2.5. His mission was accomplished with one fairly straightforward code fix.
When I heard this story, I was amazed that the error distribution curve for the component was even sharper than for Windows. Well over half of all the crashes and hangs were caused by a single bug. This is the sort of insight into your code that isn't possible without the data collected by something like Windows Error Reporting.
You'd think the reliability team would be ecstatic to have met its goal so quickly. The agreed-upon target was exceeded in just two weeks with a single fix. Instead, they were furious.
"That's unacceptable—we expected this to take two months. It was too easy!" Apparently, they wanted the programmer to work harder, not smarter.
Raymond Chen's Web site, The Old New Thing, and identically titled book deal with Windows history and Win32 programming. Twenty percent of his clothes are responsible for eighty percent of his laundry.