 TechNet Magazine October 2007

TechNet Magazine October 2007

Green Computing: Build a Green Datacenter

Companies around the world are beginning to think of ways to save energy and reduce costs, yet there is little guidance on how you can take action. Get started with our introduction to planning, designing, and implementing a green datacenter in your organization.

Dave Ohara Windows Administration: A Guide to Active Directory Replication

Active Directory provides a great deal of flexibility for supporting a large or decentralized environment, but maintaining the integrity of your data can be challenging. Active Directory replication defines how updates are communicated throughout your environment and how conflicts are resolved. Here's what you need to know to meet the challenge.

Laura E. Hunter System Center: New Software Update Management Tools

With System Center Configuration Manager 2007, you'll find a completely new mechanism for dealing with update management that ensures better compliance and enforcement on each client. Learn how this improved approach can help you get all of your machines up-to-date.

Steve Rachui Communications: Exchange Edge Transport Servers at Microsoft

How does Microsoft ensure delivery of legitimate messages to employees while keeping the flood of malicious content away from the messaging envrionment? One way is by deploying Exchange Server 2007 Edge Transport servers and Forefront Security for Exchange Server. Find out how these technologies can keep your organization safe.

Kay Unkroth SQL Server: Optimizing SQL Server CPU Performance

The key to healthy CPU utilization on your database server is to make sure that the CPU is spending its time processing what you want it to process rather than wasting cycles on poorly optimized code or sluggish hardware. Here are some tips for getting the most out of SQL Server and your hardware.

Zach Nichter From the Editor: The Best is Yet To Come

Joshua Hoffman Toolbox: New Products for IT Pros

Greg Steen SQL Q&A: Login Triggers, Data File Defrags, and More

Edited by Nancy Michell Utility Spotlight: Virtual Machine Remote Control Plus

The Virtual Server 2005 interface is a Web application, offering the conveniences of Web-based management. But sometimes you want a rich UI that behaves like a Windows application.VMRCplus gives you an alternative UI that offers easy configuration management and remote control of virtual machines, all from within a single application.

Matthijs ten Seldam The Business of IT: Calculating the Value of IT Infrastructure in Government

What's the best way to justify IT infrastructure investments? Government IT organizations use a concept of public good. Find out how they use public value to articulate their business cases.

Acer Maamoun The Cable Guy: The Authenticated Internet Protocol

The Internet Key Exchange protocol and Authenticated Internet Protocol are both used to determine keying material and negotiate security parameters for IPsec-protected communications. Get an in-depth look at how they work.

Joseph Davies Windows PowerShell: Working Without a Script

Have you put off using Windows PowerShell because you don't have time to learn how to script. Don't worry. Here's a look at how you can start using Windows PowerShell immediately, without writing a lick of code.

Don Jones Hey, Scripting Guy!: Let There Be Silverlight

Silverlight makes it easy to create cool user interfaces and multimedia presentations. And it seems to work well in HTML Applications, or HTAs. The Scripting Guys show you how to give Silverlight a try.

The Microsoft Scripting Guys Security Watch: BitLocker and the Complexities of Trust

Some people have been skeptical of the cryptographic functionality built into Windows Vista. Here's a look at the design philosophies that went into BitLocker and why it should be taken seriously.

Justin Troutman The Desktop Files: The Power User's Guide to WIM and ImageX

Dive into ImageX, finding out about changes for Windows Server 2008 and the right way to use command switches when creating or updating WIM files.

Wes Miller Field Notes: Recycle Your PC

With its relatively unknown Community Microsoft Authorized Refurbisher program, Microsoft supports projects that improve education, create jobs, and enable sustainable economic opportunities for underserved communities.

Jim Lynch Windows Confidential: Calling Dr. Watson

Why does using Dr. Watson to take a memory dump of Winlogon bring down the entire system? This isn’t an accident. Explore the world of Task Manager and critical system files to find out why.

Raymond Chen


 Green Computing: Build a Green Datacenter
Green Computing Dave Ohara

Build a Green Datacenter
At a Glance:
  • Energy monitoring systems
  • Essential traits of a green datacenter
  • Effects of H.R. 5646
  • Best practices for getting started

Green is the hot topic these days, and the concept is having an impact on the way people think about datacenters. Companies around the world are announcing ways to save energy and reduce
costs by buying new hardware and services. Yet, there is little guidance on how you can take action to control energy costs. In the past, electricity has been treated as an overhead expense, like the cost of space. But with rising power costs and issues regarding reliability, supply, and capacity, electricity requires its own specific strategy.
Projects regarding performance optimization and cost reduction are a part of everyday best practices in nearly every area of business. So why not treat energy costs in the same way?
As IT pros, many of us make decisions about the configuration and setup of servers, the specifications on the equipment our organizations purchase, and the requirements for datacenter upgrades and construction. We even provide early design input during application development. When it comes to these projects, we obviously have a golden opportunity to be green and influence the energy efficiency of any datacenter.
The first part of any strategy is to know your current energy usage. You need to know where your energy is used and by what specific equipment, as well as what usage is efficient and what is wasteful in the datacenter. Unfortunately, it's rare to find power-consumption metering in place that can break down usage to a level where people can see the results of their actions. Most organizations typically only see a monthly power bill that rolls up consumption into an overall bottom line. This offers little incentive for saving energy since individuals never see the impact of their decisions, and there is no way for them to prove that their changes have actually saved energy.
One of the first issues people confront when considering a green datacenter initiative is whether they have executive support. For the purpose of the article, I am going to assume the answer is "not yet." Executive support requires a serious commitment that provides resources and budget for your initiative. And while there is a lot of talk about green datacenters, the reality is that there is still often a lack of serious support at the executive level. If you did already have such executive support, you would probably be running a green datacenter right now.
Still, even assuming you are not getting the support you need, there is a great deal you can do to push your green datacenter initiative forward. So how do you determine effective actions to take in achieving your goals? Fortunately, energy efficiency is not a new concept and there is a lot that IT pros can learn from other industries.

Summing Up Energy Consumption
One of the biggest challenges I find when writing on this topic is making sense of the wide variance of numbers provided by different sources. Some studies say that datacenters account for between 1.2 to 2.0 percent of the electricity consumed in the United States. By some estimates, if you were to view datacenters as an industry unto themselves, U.S. datacenters would be approaching the top five industries in terms of energy use. Meanwhile, electricity can account for up to 15 to 20 percent of the operating cost in a U.S. enterprise datacenter.
A study by Lawrence Berkeley National Laboratories for the American Council for an Energy-Efficient Economy determined that datacenters can be as high as 40 times more energy-intensive than conventional office buildings. The study goes into detail about what can be done to improve energy efficiency from a facilities perspective (see eetd.lbl.gov/emills/PUBS/PDF/ACEEE-datacenters.pdf). A good starting point is to determine to what extent your datacenter uses the best practices set out in this study.
No doubt, there are a lot of perspectives, a lot of studies, and a lot of numbers regarding power consumption. One theme that remains constant, however, is that there is a lot of waste. Just think about how many servers have been purchased with the intent of getting the most bang for the buck—this means buying as much CPU power, memory, storage, and redundant power supplies as possible while staying within a budget. A lot of those servers have been running at an average of 10 to 15 percent utilization for the past 3 to 5 years. While people often focus on CPUs, other components do play an important role here—memory, disks, and network interfaces are all significant power draws and, when combined, exceed the power consumption of the CPU.
Meanwhile, excess hardware capacity—in everything from processors to redundant power supplies—can lead to significant waste. Given the risk-averse culture in most IT departments, excess capacity is the norm. The safest decision is to have excess hardware capacity, even though it may not necessarily be the best financial decision. But while energy is a key factor, it is typically overlooked when calculating lifecycle costs. Unfortunately, there are no incentives to buy efficient solutions, as there is no cost benefit to decision makers after deployment.
There's a median to be reached, however. The pendulum swings the other way when executives come to realize how little of the datacenter hardware capacity is actually being used, and they look to server consolidation as a way to reduce costs. It is much easier to size servers appropriately up front than to attempt to consolidate after servers have already been deployed.
As the IT industry reaches a new level of maturity, optimization with cost reduction becomes a priority. Energy consumption is one indicator that executives can use to measure the effectiveness of decisions made regarding the lifecycle of IT hardware. AMD, Intel, and server and component OEMs are all making references to the performance-per-watt of their platforms. The performance-per-watt numbers can be used to calculate an operating cost by using the inverse formula: watts per performance unit. With the range of people involved in developing IT solutions—from facilities, application development, architects, operations, and business units—performance-per-watt answers the question of "What am I getting in return," and it lets you see how solutions compare in terms of energy efficiency.
Why is it a common practice to ignore energy costs and overlook decisions that affect operating and facilities costs? How many datacenters have you seen that are at capacity due to power and cooling constraints? There was a time when having enough physical space for the datacenter was the problem. Today, however, the problem is whether you have enough capacity to power and cool all the equipment—and how soon until you run out of this capacity.

The Importance of Energy Monitoring Systems
Precise energy consumption can be difficult to measure. As I mentioned earlier, most companies roll up energy costs as a whole rather than breaking down power consumption among specific units. This method of treating power as general overhead creates the perception that energy costs are beyond the control of the business unit. This, however, is not true.
A datacenter can use tens of times more power per square foot than standard office space, and therefore even minor efficiency improvements in datacenters can account for substantial savings. Not surprisingly, cost savings is the primary reason for datacenters implementing energy savings programs. The problem is that the lowest cost method is usually in place to charge for energy consumption and the monthly facilities costs for datacenter operations are rolled up. Costs are then allocated to groups based on their occupied space and/or by port connection—this approach gives a far less than accurate breakdown of the true costs to run a piece of datacenter equipment.
It's rare for an organization to have a metering solution in place that measures the actual power usage by individual datacenter devices—largely due to the fact that installing an energy metering and reporting system requires a significant upfront investment. However, investing in such a system can result in cost savings further down the line.
Of course, a power monitoring system on its own isn't enough—you'll need a plan for how you will use collected data to better manage energy consumption and ultimately see a return on your investment. Consider the Canadian company Abitibi Consolidated, the world's largest producer of newspaper and paper products. In 2001, when the Ontario electric power market was to be deregulated, Abitibi Consolidated was exposed to fluctuating market pricing and availability. To optimize its energy purchases, the company implemented a monitoring and metering system. The new system paid for itself within a few months, thanks to a 1 percent savings the company realized simply by buying their energy more efficiently.
How much energy do you need to save to justify an energy monitoring upgrade in your datacenter? Bill Laing, General Manager of Windows Server® 2008 development at Microsoft, said in a recent interview that the new Windows Server release with default energy savings enabled could reduce consumption by up to 20 percent, a significant savings. (Figure 1 shows some of the Power Options available in Windows Server 2008 beta 3.) Note, though, that the savings cannot be calculated as a direct reduction in the cost of power consumed. There are a number of other factors involved, such as the cost of building a new datacenter and the number of other systems currently supporting less efficient configurations.
Figure 1 Power Options in Windows Server 2008 beta 3
Figure 1 Power Options in Windows Server 2008 beta 3
Right now, you could select the Server Balanced Processor Power and Performance option in Windows Server 2003 SP1. But without a metering system in place, you won't be able to quantify the savings, or any other factors involved in making and supporting this change.
It's also worth noting that for many datacenters, power consumption typically fluctuates only 4 percent during any 24-hour period, indicating that power consumption is not load dependent. With recent innovations by hardware vendors, this outcome is changing for new equipment and the amount of power that is being consumed decreases under lighter loads.

Defining the Green Datacenter
I could go on for pages listing the criteria or traits that make a datacenter green. Of course, I don't have room to cover everything in this article, but I want to highlight some key characteristics that you should keep in mind if you are serious about implementing a green datacenter. Even if you are only in the planning stages right now, you can measure against this list to determine where your datacenter currently stands:
  1. Meters are used to break down energy usage to the level of components (such as a 2U server, a 4U server, a switch, a SAN, and a UPS) and which business units are charged for the power being used by those components.
  2. Energy usage is continuously monitored to determine peak and low energy demands.
  3. Energy capacities are monitored on a total datacenter level all the way down to circuits to make sure all circuits are within acceptable limits.
  4. The energy savings plan is documented and rewarded.
  5. The energy savings plan is reviewed regularly and corrective action is taken to address failures.
  6. Determining how costs are charged back to business units is used to shape behavior, encouraging energy savings among independent business units. This point must be driven at the executive level.
  7. CPU throttling is enabled on the servers, and the performance lab measures the range of power consumed under a variety of loads.
  8. Thermal profiling is used to identify hot spots and overcooling.
  9. IT performance engineering includes energy efficiency measurements.
  10. Feedback of live data is available to individual organizations, allowing them to react appropriately. (California ISO goes as far as to provide real-time data to the public on its Web site, as shown in Figure 2.)
Figure 2 The California ISO Web site provides a live snapshot that includes current demand and current level of conservation being requested of its customers
Figure 2 The California ISO Web site provides a live snapshot that includes current demand and current level of conservation being requested of its customers
The more of these characteristics you have in place, the closer you are to successfully running an advanced energy monitoring system. I've included a scorecard (see Figure 3) with eight questions you can ask yourself to see the current level of your datacenter: basic, intermediate, or advanced.
Figure 3 How green is your datacenter? (Close Figure 3)

Level | Do you ...
Basic | Review your monthly electricity bill?
Check your electrical equipment on a regular basis for capacity planning?
Intermediate | Have automated metering and reporting of load and consumption for electrical circuits?
Review equipment energy efficiency when purchasing?
Advanced | Monitor energy consumption per rack, device, and circuit?
Analyze historical data monitoring and reporting?
Use real-time reporting of energy consumption for departments to understand impacts of changes and deployments?
Use data to negotiate favorable power rates with the utility company?

Knowing Is Half the Battle
Another problem you'll face is standardization throughout the organization. Making a change such as I've described in this article—one that changes the paradigm by which energy is measured, and one that involves participation by business units throughout the organization—involves alignment between facilities groups across multiple datacenters working with a broader set of IT groups. Such organizational issues are the top blockers to putting an energy savings program in place.
It's important to make sure you have a single person or team responsible for understanding power consumption across datacenters. This center of knowledge (which most companies lack when it comes to power consumption) is essential. It's also important that your tools collect data from the disparate facilities in a uniform manner and report the results back to each business unit in a consistent way (Figure 4 shows a sample dashboard for reporting data).
Figure 4 Sample global data monitoring dashboard (courtesy of OSIsoft)
Figure 4 Sample global data monitoring dashboard (courtesy of OSIsoft)
Without the right instruments, you won't know whether you are really operating efficiently—nor will the individual business units know whether their efforts are producing results. If you really want to change human behavior, you need to give users the information in real time with historical data trends so they can visualize the results of their actions. You can't effectively reward groups for achievement in energy efficiency if each business unit is on a different system, using a different set of tools, and different measurements for reporting their success.

Industry Forces
Don't feel alone in your efforts to make your company greener. Leaders in other markets—from companies like Boeing and Toyota to government agencies such as the State of California—are making serious investments in making their operations and products more energy efficient.
As for the world of IT, in 2006 the U.S. House of Representatives passed a bill—H.R. 5646, now public law 109-431—calling upon the Environmental Protection Agency (EPA) to investigate energy costs and electricity consumption by servers and datacenters and the potential effects of moving to more energy efficient systems. The study, which was carried out by ENERGY STAR, an EPA program, projected near-term growth in energy use of U.S. computer servers and datacenters, assessed potential cost and energy savings related to computer server and datacenter energy efficiency improvements, and assessed potential incentives and voluntary programs for promoting energy-efficient computer servers and datacenters. The final results of the study were presented to Congress in August 2007. You can find more information about this study at energystar.gov.
New building standards for green buildings are also being developed. These standards call for such criteria as hourly sub-metering capabilities for circuits depending on purpose and energy consumption, building meters that communicate energy consumption data to a meter data management system, and meter data management systems that store data and create user reports showing calculated hourly, daily, monthly, and annual energy consumption for each meter. Many of these criteria are required for compliance systems if you sign up for energy savings programs where you are required to prove your changes have worked (for instance, renewable energy credits, carbon credits, and energy reduction programs).
Some users have asked for specification testing, but it is difficult to represent the spectrum of uses. Attempts are being made to standardize tests, but these tests—like most other performance tests—can be used to optimize the "perceived" results, which do not represent actual usage. Consider miles per gallon (MPG), for example. The EPA ratings estimate the miles per gallon a typical driver should get under typical city and highway conditions. However, most drivers and driving environments aren't typical and the factors that affect fuel efficiency can vary significantly. This same guidance applies to energy savings in the datacenter. The only accurate results for your organization will come from measuring energy savings in your own environment.

Where Are the Savings?
There are many factors to consider when determining savings. Do you know the cost of adding more power in a given area? Do you have a strategy to minimize power constraints? Can you solve the problem by locating your datacenters somewhere that power is cheaper? Or should you focus on improving operations in your existing facilities?
For some companies, the best way to reduce costs may be to relocate the datacenters to areas with cheaper electricity sources. Server virtualization offers another approach to reducing energy consumption. In fact, some power companies, (PG&E, SoCal Edison, and SDG&E, to name a few) are offering financial incentives to customers that undertake server virtualization projects that result in a reduction of server equipment.
But there are simpler, less drastic steps you can take. Most IT departments currently have no idea how much power their systems consume, and this means there is enormous opportunity for potential savings. A first step could be to look at the energy consumption associated with individual devices and then see if there are any changes you could make on a per-device basis. Many small changes can add up to substantial savings.
For comparison, think of the typical household where the owner has no idea about how much energy each item in their house uses—they just pay a total monthly bill. You can get a simple inexpensive device, such as a Kill A Watt, that measures the energy consumption of individual devices. How much do you spend to keep your microwave powered when you're not using it? How much power is consumed when leaving your phone charger plugged in? Taking this same approach, you can add power meters to racks and collect data from various devices. Management tools can be used to collect data directly from some servers with energy consumption SNMP traps.

Getting Started
Figure 5 provides an outline of best practices that you should use to get your plan underway. The important thing to understand at this stage is that you're not limited to choosing from the two extremes. You can opt for a more balanced approach, which is probably more realistic. Start a grass-roots effort, identifying those individuals within your company who are passionate about energy savings. Start measuring some devices within a business unit to provide visibility. Getting some momentum and showing quantifiable results will help you get the backing you need to expand the company's investment in going green.
Figure 5 Keys to putting an energy savings plan in place (Close Figure 5)

Collect data manually to determine the state of your current systems. Determine how much power is consumed by devices and applications, and break down the costs.
Create a closed loop feedback system for measuring the effectiveness of changes made.
Be sure you monitor electricity consumption in the right places and to the appropriate level (by device, IT organization, business unit, application, and so on).
Create a fund for energy savings projects to provide capital funding.
Find out who within your organization can help drive energy efficiency and get them involved.
Who are the people who will drive energy efficiency in your company? Are they the top-level executives, middle management, or individuals who simply care about the impact your company has on the environment. There are passionate individuals at every company. One of your first decisions is whether to try the grass-roots method and involve these people from the start.
Ultimately, for whichever direction you choose, planning an energy efficiency program for your datacenter will require collaboration across groups in IT. Until recently, the typical approach to planning IT solutions has been to ignore power costs early on during the design phase, focusing on the hardware and software being purchased, along with the labor and hosting costs of the solution. When power is buried in the overhead cost of running solutions in a datacenter, energy efficiency is a low priority. Exposing the actual power being consumed by solutions is the first critical step in changing the behavior of your organization.

Dave Ohara has 26 years of experience in technology, working for such companies as Microsoft, Apple, and Hewlett-Packard. Working in product development, marketing, technical evangelism, infrastructure architecture, process engineering, reliability engineering, and distribution logistics, Dave is now working with multiple companies implementing green initiatives.
© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


 Windows Administration: A Guide to Active Directory Replication
Windows Administration Laura E. Hunter

A Guide to Active Directory Replication
At a Glance:
  • Transitioning to Active Directory
  • Maintaining consistency
  • Handling conflict resolution
  • Changes in Windows Server 2008

Prior to the introduction of Windows 2000 Server and Active Directory, many corporate environments relied on Windows NT for their server infrastructure and identity and access
management. By the time Windows NT® 4.0 rolled out, it was a solid offering in the network operating system (NOS) space, but it had a number of drawbacks that made it difficult to deploy across a large enterprise.
For starters, Windows NT utilized a flat namespace to store network resources, which meant that there wasn't a good way to separate resources into smaller subsets or to configure any sort of granular administration. You could not, for example, configure a departmental container for the resources in your marketing department or configure a local administrator who had rights to reset passwords for the users within that department only. In this manner, Windows NT security was largely all-or-nothing; if you wanted to delegate administrative tasks to a desktop support engineer, you often were forced to grant far more permissions than you would have otherwise.
In addition, Windows NT stored all of its information in a Security Accounts Manager (SAM) database that was not meant to grow beyond 40MB in size. If your SAM database grew beyond this recommended maximum (this happened around 25,000-40,000 objects, depending on your configuration), you needed to split your environment into multiple separate domains, which complicated both administering the network and providing resource access to users. Each NT domain contained a single Primary Domain Controller (PDC), which contained the only read/write copy of the SAM database; while you could deploy one or more Backup Domain Controllers (BDCs) for fault tolerance, these BDCs were read-only and could not perform any update, such as changing a user's password, that required a write operation.
Finally, Windows NT relied on NetBIOS name resolution, which was broadcast-based and often generated a great deal of traffic as users browsed for network resources, particularly if they needed to do so across a slow or heavily-utilized WAN link.

A New Model Emerges
In the year 2000, Microsoft released Windows® 2000, which included a substantial overhaul of its previous NOS offerings. The new NOS service, Active Directory®, was as different from Windows NT as you could imagine. Rather than relying on a flat namespace, Active Directory was built on the X.500 standard, which created a hierarchical organizational structure; you could now organize resources into multiple Organizational Units within a single domain and delegate administration of each OU at a task-based level.
Another significant departure from Windows NT was the new model of multi-master replication. Gone was the single writeable PDC and its associated BDCs; in Active Directory, each domain controller had the ability to write to the Active Directory database.
However, while this created a great deal more flexibility in terms of supporting a large and/or decentralized environment, it also generated new challenges for maintaining the integrity of Active Directory. If John Smith and Jane Dow make a change to the same object in offices at opposite ends of the country, what happens if those changes are in conflict? The Active Directory replication model defines the ways in which updates are communicated to all domain controllers within an environment, as well as how to handle any conflicts that arise as a result of this multi-master ability to make changes from practically anywhere.

The Mechanics of Active Directory Replication
For the purpose of these examples, we will be discussing only intrasite replication. Basically, intrasite replication is designed to replicate changes quickly to DCs within the same site and is performed using change notification. In the case of intrasite replication, DC1 will notify DC2 that it has changes that need to be replicated, after which DC2 will pull those changes from DC1. Likewise, DC2 will notify DC1 when DC2 has any changes, after which DC1 will pull those changes from DC2. As you can see, all Active Directory replication takes place in pull operations, not pushes.
Since Active Directory can scale to hundreds of thousands or even millions of objects, it's necessary to carve up the Active Directory database into sections, called naming contexts. At minimum, each domain controller stores three NCs in its local copy of the Active Directory database.
Schema NC This NC is replicated to every other domain controller in the forest. It contains information about the Active Directory schema, which in turn defines the different object classes and attributes within Active Directory.
Configuration NC Also replicated to every other DC in the forest, this NC contains forest-wide configuration information pertaining to the physical layout of Active Directory, as well as information about display specifiers and forest-wide Active Directory quotas.
Domain NC This NC is replicated to every other DC within a single Active Directory domain. This is the NC that contains the most commonly-accessed Active Directory data: the actual users, groups, computers, and other objects that reside within a particular Active Directory domain.
In order to better optimize replication traffic, each naming context is replicated separately so that an NC that changes infrequently, such as the Schema NC, is not taking away network bandwidth that's needed by the Domain NC, which is likely to change on a much more frequent basis.
Since directory changes can be made from any Active Directory DC, there are two types of write operations that Active Directory replication needs to track. One type is originating writes, which is when a particular change was performed directly on a particular DC. For instance, if you connect to DC1 and change a user's password, that change is considered an originating write on DC1. Active Directory must also track replicated writes—as you might imagine, this means that a particular change was replicated in from another domain controller. The change that was considered an originating write on DC1 will be considered a replicated write when that change is replicated to DC2, DC3, and any other DCs throughout the domain.
Active Directory domain controllers manage the transmission of directory changes through the use of replication metadata. This means that, in addition to communicating the actual data that has been changed from one DC to another (John Smith's description was changed to 'HR Director'), Active Directory also transmits additional information about that change to allow domain controllers to manage replication in the most efficient manner, such as the DC that the change originated from, the time at which the change was made, and other key pieces of information.
The first item of replication metadata we'll discuss is the Update Sequence Number (USN). Each domain controller maintains a USN that is specific to that domain controller. Whenever a change is made to Active Directory from that DC, the USN is incremented by 1. So if a DC has a USN of 1000 at 11:00 a.m., and 1005 at 11:30 a.m., you know that 5 changes have been made to the Active Directory database on that DC. Exactly what these changes were is unimportant as far as the USN is concerned—you could have modified 5 different objects, created 5 objects, deleted 5 objects or any combination thereof, the DC's USN will still increase by 5. Moreover, USNs are internal to a specific domain controller only, and don't have any relevance when compared with other DCs. One DC in a domain might make a change at 11:30 a.m. that it assigns a USN of 1051, and a second DC in the same domain might make a change at precisely the same moment that it assigns a USN of 5084. While these two DCs clearly have radically different USNs for a change made at roughly the same time, this fact is irrelevant to how these changes are replicated; the Update Sequence Number of one DC has no meaning to any other DCs in terms of comparing one change to another.
But this is not the only way that a DC's USN can be incremented. Remember that a change to the Active Directory database can consist of an originating write or a replicated write. The Update Sequence Number on a domain controller is incremented by both types of write operations, which means that it is incremented whenever changes are replicated in from another DC. Now, clearly each DC needs a way to keep track of which changes have already been replicated, otherwise each DC would be sending the entire Active Directory database across the wire at every replication. To prevent this, each Active Directory domain controller maintains a value called the high watermark vector (HWMV) for other domain controllers that it is replicating with. Each DC will associate this high watermark vector with the Globally Unique Identifier (GUID) of the remote DC, to prevent any confusion if a remote domain controller is renamed or removed from the directory.
Let's start with a simple example, where you have two domain controllers configured in the contoso.com domain, dc1.contoso.com and dc2.contoso.com. Since there are only two DCs in the contoso.com domain, DC1 and DC2 replicate with each other only. (Note that this is a simplified example that doesn't tell the whole story of Active Directory replication yet; we will be adding onto it as we explain further details.)
Let's further say that DC1's current USN is 3000, DC2's current USN is 4500, and that these two DCs are fully up-to-date with one another as we begin our example:
Step 1: DC1 and DC2 are current with each other. DC1 has a high watermark vector for DC2 of 4500, and DC2 has a high watermark vector for DC1 of 3000, as shown in Figure 1.
Figure 1 Current state of two DCs
Figure 1 Current state of two DCs
Step 2: An administrator creates a new object on DC1, and DC1's USN is incremented to 3001, as shown in Figure 2. Notice that the HWMV of DC1 hasn't changed on DC2, because DC1 hasn't yet notified DC2 that it has changes waiting.
Figure 2 A new object is added
Figure 2 A new object is added
Step 3: DC1 notifies DC2 that it has changes available. DC2 then initiates replication with DC1 to request any updates that are available. As a part of this request, DC2 sends DC1 the high watermark vector that it has stored for DC1, as shown in Figure 3.
Figure 3 Notification of changes
Figure 3 Notification of changes
Step 4: DC1 sends DC2 the change that corresponds with USN 3001, that is, the object that was created on DC1 in Step 2. DC2 updates its own USN to 4501 and its HWMV for DC1 to 3001, as shown in Figure 4.
Figure 4 Changes and updates
Figure 4 Changes and updates
So far, so good, right? But now there's a problem. DC2 has a change that it needs to replicate. If the only thing to go on were USNs and high watermark vectors, at this point DC2 would contact DC1 to replicate the same change back to DC1 that DC1 just replicated to DC2, which would create an endless cycle of replication and chew up progressively more and more bandwidth. To combat this, we need to a few more pieces to the puzzle, the first of which is the up-to-dateness vector (UTD vector, or just UTDV).
The UTDV is another piece of replication metadata that is used for propagation dampening; that is, its purpose is to prevent the same change from wasting bandwidth by being replicated across the network over and over again. Each DC maintains a UTDV table for every other DC that stores a replica of the naming context in question. For the Domain NC, each DC in a domain maintains a UTDV for every DC in the domain; for the Configuration and Schema NC, this is maintained for every DC in the forest. The UTDV table keeps track, not only of the highest USN that each DC has received from its replication partners, but also the highest USN value that it has received from every DC that is replicating a given NC. To allow for this, each replicated change also includes the following information:
  • The GUID of the DC that is replicating the change. This can be a change that is being replicated as an originating write or as a replicated write.
  • The USN from the DC that is replicating the change. Again, this can be from either an originating or a replicated write.
  • The GUID of the DC that originated the change. If this GUID is the same as the GUID of the DC that is replicating the change, then this is an originating write. Otherwise, the UTDV table comes into play.
  • The USN from the DC that originated the change. Again, if this USN is the same as the USN of the DC that is replicating the change, then this is an originating write. Otherwise, it's off the UTDV table.
To better illustrate this process, we'll increase the complexity of our example by adding a third domain controller, DC3. In this instance, DC1, DC2, and DC3 are all replication partners with one another; DC1 replicates with DC2 and DC3, DC2 with DC1 and DC3, and DC3 with DC1 and DC2:
Step 1: DC1, DC2 and DC3 are all current with each other.
Step 2: DC3 performs a single originating write by resetting the password for the jsmith user account. DC3 notifies DC1 and DC2 that it has changes available. DC1 and DC2 pull in the originating write from DC3, and then update their HWMV and UTDV tables for DC3 as shown in Figure 5.
Figure 5 Updating HWMV and UTDV tables
Figure 5 Updating HWMV and UTDV tables
Step 3: Here's where the up-to-dateness vector comes into play. DC2 notifies DC1 that it has changes available. DC1 then contacts DC2 requesting any new changes by sending DC2 the following information:
  • DC1's high watermark value for DC2, in this case 4501.
  • DC1's UTDV table (shown in Figure 6), indicating the highest originating USN that it has received from all DCs including DC3.
Figure 6 UTDV table (Close Figure 6)

All DCs That Replicate This NC | UTDV
<DC2's GUID> | 4501
<DC3's GUID> | 7002
Based on the HWMV of 4501, DC2 sees that it has not replicated the change in question to DC1 (see Figure 7).
Figure 7 A change needs to be replicated (Close Figure 7)

Property | Value | Local DC's GUID | Local USN | Originating DC's GUID | Originating USN
jsmith Password Property | %#FH%2rfg2 | <DC2's GUID> | 4501 | <DC3's GUID> | 7002
However, based on the UTDV table that DC1 transmitted before replication began, DC2 determines that DC1 doesn't actually require this change, since DC1 has already received this change from DC3. At this point, DC1 simply updates its HWMV entry for DC2 to reflect the incremented USN, as shown in Figure 8. However, to conserve bandwidth, the actual data is not sent over the wire.
Figure 8 Propagation dampening
Figure 8 Propagation dampening
Step 4: This same propagation dampening will take place when DC2 notifies DC3 that it has changes available, and when DC1 similarly notifies DC2 and DC3. All three DCs will update their respective HWMV entries for their replication partners as shown in Figure 8, but the actual data will not traverse the wire again since it has already been transmitted to each DC in Step 2.

Conflict Resolution in a Multi-Master Environment
Thus far our examples have existed in a perfect world, where only one administrator is making changes to a DC at a time and nobody ever steps on anyone else's toes. In the real world, we know this is seldom the case. Since updates to an Active Directory object can come from any domain controller in the domain, what happens if two administrators make conflicting updates from two different domain controllers? There are three types of conflicts that can occur in an Active Directory environment, each of which uses a different method of conflict resolution.
Conflicting property changes . This conflict occurs if two administrators update the same object in a way that conflicts: one administrator sets a user's description to "Marketing," while another administrator on a different DC sets that user's description to "Sales and Marketing."
Creating new objects that conflict . This conflict occurs if two administrators create an object with the same name at the same time, such as two users named jsmith.
Moving an object into a deleted container . This type of conflict is much rarer, and takes place if one administrator creates or moves an object within a container, such as an OU, at the same time that another administrator on a different DC deletes that container.
To address the first two types of conflicts, it's time to introduce two more pieces of replication metadata that are used primarily for conflict resolution. The versionID value is assigned to each individual attribute on an object, with a starting value of 1 when the object is first created. The versionID gets incremented by 1 whenever an individual attribute is modified from any DC. So if the description attribute of a particular user gets updated from its default value (blank or <not set>) to "Marketing Department," the description attribute will have a versionID of 2. If the description is later modified to "Sales and Marketing Department," the description attribute will have a versionID of 3, and so on. This versionID is included with every replication entry along with the other pieces of metadata that we've already introduced.
The versionID can also be used to further cut down on replication traffic. For instance, if an administrator on DC2 has made multiple changes to a single attribute (perhaps he or she mistyped the change a few times) such that DC2 has originating writes corresponding to versionIDs 2, 3, 4, and 5, DC2 will only replicate the write that corresponds to the latest one, versionID 5. Since the earlier changes would simply be overwritten anyway, this provides a shortcut to reduce unnecessary bandwidth utilization.
Every change to Active Directory also includes the second additional piece of metadata used for conflict resolution, a timestamp, as part of the replication metadata, indicating when the modification was made.
The timestamp attribute is also used as a proactive measure of Active Directory replication health. If a DC hasn't seen any changes with a relatively recent timestamp from a particular DC, it will begin to generate error messages indicating that there may be an issue with the DC in question.
So how are these two attributes used in conflict resolution? Let's examine each type of conflict in turn.

Resolving Conflicting Property Changes
Consider the example of the jsmith user object in the contoso.com domain. An administrator on DC1 changes jsmith's description to "Marketing." Nearly simultaneously, an administrator on DC3 changes the same user's description to "Sales and Marketing." At this point, DC1's and DC3's information about jsmith's description attribute compare as shown in Figure 9.
Figure 9 Near-simultaneous changes (Close Figure 9)

Server | Property | Value | VersionID | Timestamp | Local DC's GUID | Local USN | Originating DC's GUID | Originating USN
DC1 | jsmith Description Property | "Marketing" | 2 | 2007-06-07 14:03:25 | <DC1's GUID> | 3003 | <DC1's GUID> | 3003
DC3 | jsmith Description Property | "Sales and Marketing" | 2 | 2007-06-07 14:04:57 | <DC3's GUID> | 7003 | <DC3's GUID> | 7003
If DC2 receives both of these changes simultaneously, it will clearly need to determine which one is the "winning" change. The order of tiebreakers for conflict resolution is as follows:
  1. The modification that has the higher versionID will be accepted as the "winning" change; the "losing" change will be overwritten. In this case, the versionID is 2 for both records, so we need to move onto the second tiebreaker.
  2. If both records have the same versionID, the change that has the later timestamp will be accepted as the winning change; the losing change will be overwritten. In this case, the timestamp from DC3's originating write is later, so jsmith's description will be set to "Sales and Marketing." In the rare instance when both the versionID and timestamp are identical, we need a third and definitive tiebreaker:
  3. If both records have the same versionID and timestamp, whichever write was originated by the DC with the lower-numbered GUID will win; the write from the higher-numbered GUID will be overwritten. So if DC1's GUID is 1234567890 and DC3's GUID is 2345678901, the originating write from DC1 would win if both the versionID and timestamp were identical.
You are probably thinking, "Wouldn't it make more sense to have the timestamp be the first tie-breaker?" This isn't as cut-and-dried as you might think. If timestamp were the primary tie-breaker in Active Directory conflict resolution, the only thing that a malicious administrator would need to do to propagate his or her changes would be to set back the clock on one particular DC so that it would always win by way of timestamps.

Resolving Conflicting Object Creation
In cases where two objects are created with the same name, Active Directory will use the same three tie-breakers described in the previous section to determine which one is the "winning" object. Unlike the previous section, however, the "losing" object doesn't get overwritten. Instead, the losing object is renamed using the characters CNF (for conflict object), followed by a colon and the GUID of the "losing" object. This allows administrators to more methodically determine which object should be retained and which should be deleted.

Resolving an Object Move into a Deleted Container
As mentioned, resolving an object move into a deleted container is a relatively rare conflict, which only occurs in one of two scenarios. In one, an administrator on one DC creates an object within a particular container, for example the Training OU, at the same time that an administrator on another DC deletes the Training OU. The second scenario can occur when an administrator on one DC moves an object into a container at the same time that an administrator on another DC deletes that container.
Resolution in this case is fairly straightforward: Active Directory moves the "orphaned" object into a special container within Active Directory that was designed for this purpose, the LostAndFound container that exists off of the root of each Active Directory domain. For the contoso.com domain, the LostAndFound container would be found along the following LDAP path: LDAP://cn=LostAndFound,dc=contoso,dc=com. If you do not see the LostAndFound container when you open the Active Directory Users and Computers snap-in, simply click on View | Advanced Features.

Protecting Yourself from USN Rollback
One of the most grievous situations you can encounter in an Active Directory environment is also one of the simplest to avoid, once you understand its cause and how to work around it. USN rollback is an error condition that can completely shut down replication on your network, and is caused by allowing a domain controller to remain offline for too long and then returning it to service, or by restoring a domain controller using an unsupported method.
One underlying cause of USN rollback has to do with the way that object deletions are processed in the Active Directory multi-master environment. When an object is deleted on a DC, rather than simply removing the object outright, the object is tombstoned so that the tombstoned object can be replicated to other DCs, thus notifying them of the deletion. Most notably, a tombstoned object possesses an isDeleted attribute that's set to TRUE. In order to reduce the size of the tombstoned object, most of the values contained in the object, such as the description, personal information and group membership of a user object, are removed (for more information on this process, see Gil Kirkpatrick's article "Reanimating Active Directory Tombstone Objects," available online at technetmagazine.com/issues/2007/09).
USN rollback can occur because these tombstoned objects don't hang around indefinitely. They are purged completely from the Active Directory database after 60 or 180 days by default (depending on the version of Windows Server® that you were running when you first created your Active Directory environment.) This 60- or 180-day period is called the tombstone lifetime. All domain controllers need to be able to replicate at least once during this period or they become worse than useless; they create an opportunity for USN rollback. Basically, USN rollback occurs when a domain controller is so completely out-of-date that it has "missed" one or more tombstoned objects, and thus is unable to bring its local copy of the Active Directory database completely up-to-date with other DCs. To guard against this, any domain controller that has been offline for longer than the tombstone lifetime should not be returned to service; rather it should be rebuilt from scratch after removing the old DC's metadata from Active Directory using the steps found in Microsoft Knowledge Base article 216498 (support.microsoft.com/kb/216498).
The second cause of USN rollback occurs when a domain controller has been restored using an unsupported method, most frequently using a disk cloning or imaging tool. When this occurs, the restored Active Directory database does not realize that it has gone "back in time," because the restore method was not Active Directory-aware. USN rollback was difficult to detect in Windows 2000 and the initial release of Windows Server 2003, but Windows Server 2003 SP1 (and the upcoming Windows Server 2008) have built-in controls to detect when a DC has been restored improperly. In these newer OS versions, a DC will log Event ID 1115, 2095, 2103, and 2110 in the Directory Services event log; the text of these events, as well as the necessary steps to recover from USN rollback, can be found in Knowledge Base article 875495 (support.microsoft.com/kb/875495) for Windows Server 2003. (You can find information about dealing with USN rollback in Windows 2000 in Knowledge Base article 885875, available at support.microsoft.com/kb/885875.)

Updating the Multi-Master Model in 2008
With the upcoming release of Windows Server 2008, Microsoft has included a slight change to the multi-master model by introducing the Read-Only Domain Controller (RODC). The RODC is designed primarily for branch office deployments, or for any scenario where you don't have dedicated IT staff on-site at a particular location and need to take extra steps to ensure the integrity of a particular DC. While we could take up an entire article discussing all of the technical details of the RODC, let's review the key points you should be familiar with.
As the name indicates, the copy of the Active Directory database that resides on an RODC is read-only. You can connect to an RODC to read almost any information that you need, but you will not be able to perform any write operations on an RODC.
Second, an RODC doesn't perform any outbound replication. This is a fundamental change from the multi-master replication model that we've been discussing thus far. An RODC will receive inbound replication from other writeable Windows Server 2008 DCs, but it will not replicate any information whatsoever out to other DCs. This creates an additional layer of security such that, even if a malicious user were somehow able to modify Active Directory from the RODC, those modifications would not propagate out to the rest of your environment.
Perhaps most interesting of all, the RODC does not replicate user passwords by default. When the Active Directory database is replicated inbound to an RODC from a writeable DC, all user objects are replicated without the user's password information. This provides yet another layer of security in a branch environment, such that if a DC "grows legs and walks away," there is no password information resident on the DC's hard drive. Taken as a whole, these changes to the original idea of multi-master Active Directory replication create a much-improved model for securing domain controllers in branch offices or other remote locations.

Laura E. Hunter is a four-time recipient of the Microsoft MVP award in the area of Windows Server Networking. She is a 10-year veteran of the IT industry and has authored or coauthored numerous industry books, articles, and white papers, including Active Directory Cookbook, Second Edition (O'Reilly, 2006).


 System Center: New Software Update Management Tools
System Center Steve Rachui

New Software Update Management Tools
At a Glance:
  • Update management the old way
  • The new process in SCCM 2007
  • Custom configuration and flexibility
  • Finely tuned update scheduling

Software update management was first introduced in Systems Management Server (SMS) 2.0 with the security and Office updates scan tools. These tools have undergone multiple revisions: the extended
security update inventory tool was revised to support scanning of additional products; the Inventory Tool for Microsoft® Updates to combine the security and Office tools into a single product; and the Inventory Tool for Dell Updates. The Custom Updates Publish Tool was also introduced to support patching third-party products. The Inventory Tool for Microsoft Updates (ITMU) has seen three revisions of its own. Each of these tools has its own catalogs, scanning infrastructure, and unique requirements and challenges. Over time, the catalogs for these systems became notably large, so large, in fact, that older updates were removed in favor of new ones due to size constraints.
With the array of options, deploying updates with these tools could be challenging. Clearly, there was a need to simplify the process. The next release of SMS—System Center Configuration Manager 2007 (SCCM) makes great strides in that direction. With SCCM, the software updates mechanism is built to ensure better bottom-line results. With a completely different approach to update management operations, old catalogs and scan engines are a thing of the past. Update status, historically reported via hardware inventory, uses a new mechanism—the state message—to ensure better determination of compliance as well as enforcement on each client. I think you'll see that the new approach to update management in SCCM 2007 represents a significant improvement.
SCCM 2007 introduces two new components that I'll discuss before we delve into the update management functionality.
Software Update Point (SUP) As I mentioned, catalogs no longer exist in SCCM. Instead, SCCM works directly with a dedicated Windows Server® Update Services (WSUS) server, which will act as a Software Update Point (SUP) to provide metadata downloads and client scanning. In SCCM, the update binary delivery features of the WSUS are disabled to allow focus on the scanning process. With this adjustment, WSUS scales to handle scanning for up to 25,000 clients per SUP on a dedicated server. SCCM also allows configuring the SUP behind a Network Load Balancing (NLB) cluster for increased availability and scale, enabling up to four SUPs for a total of 100,000 possible clients per primary site.
Configuration Items (CI) A configuration item is a new approach used in SCCM to define specific configurations that should exist on target systems. SCCM update management now uses this infrastructure. Updates should no longer be thought of in terms of normal software distribution—although there are still similarities. Instead, updates reside in their own node in the administrator console and use this completely unique CI mechanism. On the client side, updates are selectively downloaded and stored locally.

Configuring Update Management
Setting up update management falls into three general areas—enabling the client agent, configuring the SUP and configuring updates for distribution.
The SCCM client agent (as shown in Figure 1) is similar to other client agents and is configured in the same node of the administrative console. The properties page of the Software Updates Client Agents node is where administrators enable this agent and configure update installation and Deployment Re-evaluation settings. The setting to enable the Software Updates Client Agent is site-wide, meaning every SCCM client assigned to the site where this function is turned on will have this role enabled. If some systems in an environment should not have this function enabled, you can override the site-wide setting on a per-system basis using local policy override settings.
Figure 1 Client agent properties
Figure 1 Client agent properties
The settings here also allow the administrator to specify how to handle update deadlines. Assume, for example, you have four updates pending installation, one (update A) with a deadline a day ahead of the other three. But suppose it's preferable to install all four updates at the same time—even those whose deadline hasn't yet been reached. The option on this tab to Enforce all mandatory deployments lets you configure this choice. Further, the settings here allow administrators to specify that all notifications and visible indications of the updating process should be suppressed. These options are shown in Figure 1.
The settings on the Deployment Re-evaluation tab allow administrators to schedule when deployments are reevaluated. SCCM clients will be evaluated according to this schedule to see if updates that have been previously installed are still required and, if they are missing, reinstall them.

Configuring the SUP
As mentioned previously, the SUP is the component responsible for downloading all available update information on a schedule and for interacting directly with SCCM clients to perform compliance scans based on its database of updates. Before a SUP is configured, a WSUS 3.0 or greater server must be installed as the SUP process does not actually install WSUS—it just configures WSUS for SCCM use. Additionally, the WSUS Administrator Console must be installed on the Site Server to enable connectivity with the WSUS Server. Once SCCM takes over the WSUS server for the SUP role, the WSUS server is dedicated to SCCM and can only be used by SCCM clients.
The SUP is a new site system role in SCCM. To begin configuration, navigate to the site systems node of the administrator console and add a new site system. The next step is to select Software Update Point and proceed through the wizard, which will collect the following information:
  • Proxy configuration
  • Port and SSL information
  • How this SUP should sync updates—from Microsoft Update, via an upstream SUP, or manually
  • Synchronization schedule
  • Update classifications of interest, such as Critical Updates, Drivers, Security Updates, Update Rollups, and so forth
  • Update products available—Windows®, Exchange, SQL Server™, and so on (shown in Figure 2)
  • Language
Figure 2 Product settings in the site role wizard
Figure 2 Product settings in the site role wizard
When the wizard is complete, the SUP will be installed and internal configurations will be adjusted to support SCCM requirements. It's a good idea to confirm that the installation succeeded by reviewing the SUPSetup.log located in the logs folder where SCCM was installed.
Once the SUP is installed, synchronization must be performed to cause the SUP to retrieve updates and store them in the SUP and SCCM database. The synchronization may either be run manually or on a schedule configured when the SUP setup wizard runs. Manual update synchronization may be initialized at any time and it's a good idea to ensure the synchronization happens after the SUP is installed the first time. Figure 3 shows how to do this.
Figure 3 Running synchronization manually
Figure 3 Running synchronization manually
The synchronization process has two steps. First, the SUP (WSUS) server is configured to match the SCCM settings and configured updates are synchronized to the SUP (WSUS) server itself. When this is complete, the update information from the SUP (WSUS) is synchronized into the SCCM database and made available for update deployment. This second step creates the required CIs (one for each software update) in the SCCM database and may be time-consuming. It is not unusual for the initial synchronization to take several hours. The synchronization process may be reviewed in the WSyncMgr log, an excerpt of which is shown inFigures 4 and 5. Specifically, Figure 4 shows the sync process requesting that the SUP (WSUS) server update its list of updates from Microsoft Update. Figure 5 shows the start of the process for synchronizing the SUP inventory to the SCCM database.
Figure 4 Synchronizing update list
Figure 4 Synchronizing update list
Figure 5 Synchronizing the updates as CIs in the SCCM database
Figure 5 Synchronizing the updates as CIs in the SCCM database
Once a WSUS server is configured as a SUP, any required administration can be accomplished via the SCCM Admin console. If configuration changes are made directly to the WSUS server, SCCM could be adversely affected. Further, these configurations will be reset hourly when SCCM checks its SUP to ensure configurations are correct.
In a multitiered hierarchy, you need to have at least one SUP at each primary site where software updates will be deployed. These SUPs will work together in a hierarchy with only one SUP, generally the one highest in the hierarchy, synchronizing with Microsoft Updates. The remaining SUPs will synchronize from a configured upstream SUP replication partner.

Configuring Update Deployment
Once the client agent is configured and the SUP configured and synchronized, you are ready to prepare an actual deployment. First, let's review the various nodes of the SCCM admin UI. Basically, there are five main nodes of interest—Update Repository, Update Lists, Deployment Templates, Deployment Management and Deployment Packages.
The Update Repository is where all of the synchronized updates are stored and available for deployment. Updates may be deployed by selecting one or more updates and launching the Deploy Software Updates wizard, which performs two tasks: configuring the deployment template (or selecting one that already exists) and configuring the deployment itself. I'll discuss this process shortly. The list of updates can get quite large and search folders, a new feature in SCCM 2007, can help greatly in finding updates of interest based on 28 update-related criteria, including Severity, Classification, Product, Date Released, and Supersendence.
Update lists allow admins to create a fixed set of updates to be managed. They can be used for compliance reporting, creating deployments, and administration delegation. Once built, these update lists are replicated down the hierarchy for reuse at child sites—just like packages or advertisements. Update lists are not tied to any particular deployment when replicated; they are just lists of updates. Once the list is complete and replicated, the updates within it may be deployed as a unit by selecting the list and choosing to deploy the updates.

Deployment Templates
Deployment templates contain common settings (such as schedule and target collection information) used during software update deployment and were developed to decrease the administrative time involved with deploying updates. Templates may be created to address multiple deployment conditions in an environment and they can be used in the deployment wizard by simply selecting the appropriate template. Settings prepopulated by the deployment template include the target collection, update deadline settings (as shown in Figure 6), restart settings, and maintenance windows. When large numbers of updates are in a deployment, the target system performance can be impacted as each updates deadline arrives simultaneously. Moreover, note that settings within the template are applied to the deployment when the deployment is created. Those settings remain in force on the deployment unless manually modified within the deployment. It's important to remember that simply modifying the template settings themselves at a later time will not cause the modified settings to be changed on existing deployments that use the modified template.
Figure 6 Setting deployment schedules in the wizard
Figure 6 Setting deployment schedules in the wizard
The options in Figure 6 may be configured while running either the deployment template wizard or the deploy software updates wizard (if a new template is being created). Notice that the duration setting in this screen is set to 0. The default value for this setting is two weeks. If this setting is set to a value other than 0, there will be a delay on the client of up to the length of that value before updates are deployed. In the time between when the deployment is made available and the configured deadline is enforced, the update will be available for manual install by end users for pre-deadline scheduled installations. At deadline time, the update will be enforced and installed. If you encounter delays in update deployment, ensure this setting is configured correctly per your needs.

Deployment Management
After a software update deployment is configured, it may be accessed in the Deployment Management node. Note that a configured software deployment is not the same as a deployment package. The settings related to the deployment, as well as the updates included in the deployment, may be viewed and manipulated on this node but not the package itself. The deployment settings found here reflect those settings that come from the software updates wizard as well as those that are inherited from the chosen template.

Deployment Packages
Deployment Packages are similar to standard SMS packages in that they define the specifics of the update—source directory, distribution point, and so forth. Deployment packages are independent of the actual deployment used to distribute the updates and are essentially a store location for updates. If more than one deployment package contains the updates needed for a deployment, SCCM agents will determine the best location from which to retrieve the update and may use any available deployment package that contains the update.
The Deploy Software Updates wizard is used to deploy updates in SCCM. This wizard may either be used with individual updates or update lists. Figure 7 shows how to start the wizard.
Figure 7 Starting the Deploy Software Updates Wizard
Figure 7 Starting the Deploy Software Updates Wizard
This wizard proceeds by asking for a name for the deployment, whether to use existing or new template settings (if a new template is chosen the wizard will prompt for the template settings described earlier), deployment package settings (either existing or new), distribution point settings, location options for download, update languages to download and deployment schedule information.
With configuration complete, policy will be updated to notify the clients of the targeted updates and the updates will be installed to the client systems. The client processes are quite different from previous releases and are beyond the scope of this article. A key difference, however, is that the update scans are no longer performed locally—the SUP handles all scanning—but the detail on update compliance is stored in the local Windows Management Instrumentation (WMI) repository, similar to previous versions.
Unlike previous versions, however, hardware inventory is no longer used to send update status to the site server. Instead, state messages (a new kind of streamlined status message) are sent to the site server via the management point. State messages supply data about each and every managed client for compliance scans and to track the progress of deployments. There are number of new reports in SCCM for Software Updates Management, including overall per-machine compliance for all deployed updates to each client, deployment enforcement state per client, and client error reports for scans and deployments that give a stack-ranked list of the biggest issues to resolve.
In addition to the traditional scheduled/mandatory enforcement of update deployments, SCCM lets you install approved updates prior to the installation deadline on a schedule very similar to what is in place today with Automatic Updates. This useful addition will do even more to increase the number of compliant systems prior to the deployment deadline.
If custom updates are of interest to you, SCCM allows for this as well. The custom updates publishing tool from SMS 2003 R2 has been modified for use with SCCM 2007 as the System Center Updates Publisher—so any custom updates may be deployed in the same manner as described.
The changes to update management are vast in SCCM, but from an administrative perspective they should be easier to use and are a welcome improvement over previous versions. Though some of the conceptual changes may be challenging at first for users accustomed to SMS 2003, the new approach is definitely worth the effort. Enjoy!

Steve Rachui is a Manageability Support Escalation Engineer in the Product Support Services group at Microsoft. He has supported SMS since version 1.2. Steve can be reached at steverac@microsoft.com.


 Communications: Exchange Edge Transport Servers at Microsoft
Communications Kay Unkroth

Exchange Edge Transport Servers at Microsoft
At a Glance:
  • The Exchange Edge Transport server role
  • Setting up a test lab
  • Transport agents and events
  • Agent internals
Download the code for this article: ExchangeEdge2007_10.exe (354KB)

Microsoft receives approximately 13 million message submission attempts from the Internet on an average business day—and blocks over 10.5 million of these as not legitimate. In critical
situations, such as during spam attacks or virus outbreaks on the Internet, the volume can swell to more than 90 million. Of course, this is a Microsoft-specific finding, yet scamming, spamming, phishing, e-mail-borne viruses, directory harvesting, distributed denial of service (DDoS) attacks, and similar concerns are not specific to Microsoft. Facing such issues, how do you ensure delivery of all legitimate messages to your users while keeping the flood of illegitimate, malicious content away from your messaging environment?
One way to achieve reliable messaging protection is to deploy Exchange Server 2007 Edge Transport servers and Forefront Security for Exchange Server in a perimeter network between the Internet and your production environment. This places an organized team of more than 10 Edge Transport agents at your perimeter to help protect your production systems and users. The value of blocking undesired content at the earliest point possible is obvious. Ideally, this occurs even before the content is delivered to your servers, considering the load your messaging environment might otherwise have to sustain in a critical situation. Yet there is a lot more to messaging protection than simply blocking messages.
This is the first of a two-part series that discusses the architecture and key features of anti-spam and antivirus agents available with Exchange Server 2007 and Forefront Security for Exchange Server. To keep the explanations realistic and practical, I will show you in this first article how to build a test lab that mirrors the messaging protection design Microsoft uses in its own corporate production environment. Then I'll cover the Edge Transport architecture of Exchange Server 2007 in more detail.
Throughout this article, I will make heavy use of scripts and batch files to automate the most important configuration tasks. These files include comments that explain the individual steps performed. You can find the scripts in the download available from the TechNet Magazine Web site at technetmagazine.com/code07.aspx.

Edge Transport Topology
Figure 1 illustrates the Edge Transport design that Microsoft IT implemented in the corporate production environment. There are some very interesting design features I'd like to point out before taking you through the Edge Transport architecture. If you are interested in the full implementation details, see the IT Showcase white paper referenced in the "Exchange Resources" sidebar.
Figure 1 Edge Transport topology at Microsoft
Figure 1 Edge Transport topology at Microsoft
As you analyze Figure 1, notice that the Edge Transport topology at Microsoft is fully redundant. There is no single point of failure in any location. For load balancing, Microsoft IT uses DNS round robin externally and messaging connectors with multiple bridgeheads internally. Also noteworthy is the fact that all transport servers (Hub Transport and Edge Transport) run Forefront Security for Exchange Server for virus scanning. This enables Microsoft IT to scan all inbound, outbound, and internal messages as soon as the messages reach a transport server in the message routing topology.
Another important security-related design feature is that the Edge Transport servers are not part of the corporate Active Directory® environment. In fact, it is not necessary to deploy Edge Transport servers in any Active Directory forest. However, to maintain a consistent management framework, apply a common set of policies, and support single sign-on, Microsoft IT deploys all Edge Transport servers in an extranet forest, separate from the production environment.
Finally, you can clearly see that all messages from the Internet reach Microsoft through Edge Transport servers that are stationed in North America. The Edge Transport servers based in Dublin and Singapore only handle outbound messages. The advantage of concentrating all inbound traffic onto the Edge Transport servers in North America is to centralize security and anti-spam controls while avoiding the transfer of outbound messages from large regional datacenters across the internal messaging backbone.
Microsoft IT Systems Engineer Omesh Desai designed the Edge Transport topology at Microsoft. When I asked Omesh about the most important design features, he said, "In our Edge Transport design, we capitalize on native Exchange anti-spam and antivirus capabilities for messaging protection at multiple layers in the messaging backbone. Messaging perimeter security is first priority but a simple administration and management model is also important for us. Edge Transport servers help us increase security through tighter firewall configurations while increasing the accuracy of spam filtering through IP reputation services, automatic content filter updates, safelist aggregation, and e-mail postmark validation. All internal server-to-server communication is encrypted by default and we also encrypt communication with external destinations if possible.
"We use two dual-core 64-bit processors and eight gigabytes of memory in our Edge Transport servers. With six of these servers, load-balanced across two data centers, we have sufficient capacities to sustain even large spikes in message submissions, such as during virus outbreaks on the Internet."

Edge Transport Test Lab
Test Lab Setup
Because it is a best practice to use non-existing domain names and private IP addresses, I use AdventureWorks.com and IP addresses from 192.168.xxx.0-24 ranges. There are no redundant systems or firewall arrays because I'm not testing load balancing or fault tolerance. (And, of course, the actual firewall systems that Microsoft IT uses can't be discussed for security reasons.) Such details are not important for this test environment anyway.
My test environment uses ISA Server 2006 because it is likely one of the most common firewall systems used with Exchange Server 2007. Running ISA Server 2006 on both the outer and the inner firewall helps to keep the complexity of the test environment at a moderate level, yet for production environments I recommend using diverse outer and inner firewall systems to increase security. I did not deploy IP Security (IPsec) policies or prepare the environment for Transport Layer Security (TLS) as these topics are outside the scope of this article.
However, I did use virtual machines and 32-bit evaluation software, which you can download from the Microsoft Web site. Microsoft does not support the 32-bit version of Exchange Server 2007 in production, but that's not an issue in a test environment.
My test lab relies on default settings wherever possible. Only the IP configuration, firewalls, and DNS zones require special attention prior to running Exchange Server 2007 setup and subscribing the Edge Transport server to the production environment. For details regarding the IP configuration, check out the Test Lab—IP Configuration.xls file you'll find in the companion download. If you use the same IP address assignments, you can quickly configure the outer firewall, called ISA01, by running the ISA01_Firewall_Policies.vbs script directly on the ISA01 computer, and use ISA02_Firewall_Policies.vbs for the inner firewall (ISA02). The companion download also includes batch files to configure the DNS servers (INTERNET01_DNS_Config.bat, AD01_DNS_Config.bat, and AD02_DNS_Config.bat). Because these batch files use the DNS command-line tool (dnscmd.exe), you need to have Windows Support Tools installed; otherwise you have to create the DNS records manually using the DNS console.
To avoid any interference from existing environments, my test lab is not connected to the Internet. This isolation is a good precaution. It causes all downloads of signature updates, IP reputation, and content filter updates to fail, but this is not critical for testing purposes. To avoid error messages, go to Scanner Updates in the Forefront Server Security Administrator console, then set the update frequency for all scanning engines to once and specify a date in the past.

To explore these features in action, it's good practice to build a test environment—common sense suggests never using a production system for testing purposes. At the minimum, it takes one server running Exchange Server 2007 for the Mailbox, Client Access, and Hub Transport server roles. You'll need a second Exchange server for the Edge Transport server role. You could skip the Edge Transport server installation if you deploy all transport agents on the multiple-role server by running the Install-AntispamAgents.ps1 script (you can find it on Hub Transport servers in the %ProgramFiles%\Microsoft\Exchange Server\Scripts folder). But this approach would hardly resemble the Microsoft IT deployment. For a realistic test lab, you'll need to include a few more servers. Figure 2 shows the test environment I used for researching this article. A more detailed illustration is available in the companion download. For more information about setting up the lab, see the "Test Lab Setup" sidebar.
Figure 2 Edge Transport test environment
Figure 2 Edge Transport test environment
During subscription of the Edge Transport server and configuration of associated connectors, Microsoft IT removes all default connectors and then proceeds to create four send connectors to communicate efficiently with different types of Simple Mail Transfer Protocol (SMTP) hosts and internal Hub Transport servers. The first send connector is a general Internet connector for all of the destinations that do not match specific address space definitions.
The second send connector is an Internet connector with detailed address space definitions for known destinations that do not support Extended SMTP (HELO domains). By setting the ForceHELO parameter for this connector to $true, Microsoft IT avoids an unnecessary sequence of EHLO, Failure Response 500, HELO when establishing SMTP connections.
The third send connector is an Internet connector with detailed address space definitions for partners and other remote domains that support TLS to communicate securely over encrypted connections (TLS domains). This connector has the RequireTLS parameter set to $true.
The fourth send connector is an inbound connector to transfer received Internet messages to Hub Transport servers in the corporate environment. Again, for more details regarding the Edge Transport server configuration, see the IT Showcase white paper referenced in the "Exchange Resources" sidebar at the end of this article.
To apply a Microsoft IT-style connector topology to the test lab, I relied on a procedure based on scripts that Omesh created for internal Microsoft IT use. For security reasons, I changed and shortened the individual commands drastically, but the resulting connector topology still corresponds to the Microsoft IT topology. Here are the steps:
  1. On the multiple-role Exchange server (HUB-MBX-01) and the Edge Transport server (EDGE01), remove the default connectors.
  2. On HUB-MBX-01, create a new receive connector by running the HUB-MBX-01_recv_connector.ps1 script that you can find in the download for this article.
  3. On EDGE01, create two new receive connectors for internal and external messaging connectivity by running the EDGE01_recv_connector.ps1 script.
  4. On EDGE01, create a subscription file by running this command:
    New-EdgeSubscription -FileName
    "c:\subscriptionfile.xml"
Then copy the resulting subscription file to the Hub Transport server's root folder (c:\subscriptionfile.xml).
5. On HUB-MBX-01, make sure the path to the subscription file is c:\subscriptionfile.xml and then run the HUB-MBX-01_complete_subscription.ps1 script. This script imports the subscription file for Edge Synchronization without automatic send connector creation, creates the send connectors for Internet and internal connectivity, and replicates the resulting configuration to the Edge Transport server via Edge Synchronization.
6. Verify the configuration by sending test messages as Contoso.User@contoso.com and Fabrikam.User@fabrikam.com from the Internet host to Administrator@adventureworks.com and replying to the received messages to ensure that inbound and outbound message transfer works.
I suggest you open the individual script files in Notepad and analyze the cmdlets and parameters these scripts use to perform the configuration. Detailed information about these cmdlets and parameters is available online in the product documentation for Exchange Server 2007.

Edge Transport Architecture
Now let's get in touch with the Edge Transport agents, which have been waiting for somebody to say HELO or EHLO since the moment I installed the Edge Transport server role. If you run the Get-TransportAgent cmdlet on the Edge Transport server, you should see the 11 entries that are listed in Figure 3. All agents are enabled by default to provide messaging protection with appropriate settings.
Figure 3 Transport agents (Close Figure 3)

SMTP Receive Agents
Connection Filter Agent
Address Rewriting Inbound Agent
Edge Rule Agent
Content Filter Agent
Sender ID Agent
Sender Filter Agent
Recipient Filter Agent
Protocol Analysis Agent
Attachment Filtering Agent
Routing Agents
Address Rewriting Outbound Agent
FSE Routing Agent
The Get-TransportAgent cmdlet and Figure 3 list the agents in the order of their priority, yet this is not the order in which the agents perform their work. The work order depends primarily on the sequence of SMTP receive events and routing events for which the agents are registered. To see how agents and events fit together, have a look at the diagram shown in Figure 4, which illustrates how transport agents integrate into the Edge Transport architecture.
Figure 4 Transport agents within the Edge Transport architecture
Figure 4 Transport agents within the Edge Transport architecture
Transport events occur at various stages during message processing to invoke additional code for spam filtering, virus scanning, and other tasks. In this loosely coupled and extensible design, the Edge Transport process (EdgeTransport.exe) assumes the role of the event source. The event handlers, in other words the transport agents, are managed delegate objects based on the Microsoft® .NET Framework 2.0, registered with the event source to receive callback notifications.
Figure 5 shows the event registrations for all agents installed on an Edge Transport server with Forefront Security. These registrations are perhaps somewhat difficult to sort out due to the large number of agent registrations, but don't despair. If you run the Get-TransportPipeline | Format-List command on an Edge Transport server, you can analyze the registrations for each individual transport event more conveniently. Just make sure that the Microsoft Exchange Transport service (MSExchangeTransport.exe) is running and that you have sent at least one message through the Edge Transport server since the last service restart. As the output reveals, multiple agents can register for the same event type and individual agents can register for multiple events. The event registrations just depend on the processing requirements of the corresponding agent.
Figure 5 Event registrations for transport agents
Figure 5 Event registrations for transport agents
One of the most important events is the OnSubmittedMessage routing event, triggered when a message reaches the submission queue. All messages must pass through this queue whether they come in through SMTP, the file system, or any other mechanism. The categorizer is a core component of the Exchange Server transport architecture, responsible for recipient resolution, message bifurcation and routing, and delivery status notification (DSN) generation. The OnSubmittedMessage event is therefore a perfect registration choice for agents that must process all received messages. The FSE Routing Agent is a Forefront Security component registered for the OnSubmittedMessage event in order to pass all received messages to the virus-scanning engines. Because the FSE Routing Agent is registered for the OnSubmittedMessage event, no message can bypass the antivirus solution.
So why don't all agents just register for the OnSubmittedMessage event and consider the job done? Because you want to block unwanted messages at the earliest point possible, before your server confirms a successful delivery. Otherwise, your servers might have to process 90 million unwanted messages during a spam or virus attack, possibly having to generate 90 million non-delivery reports (NDRs), which in turn could pose a serious threat to innocent bystanders. Spam and virus attacks almost always use falsified originator information. Sending millions of NDRs to recipients that did not create the original messages is not only a waste of your resources and the resources of the target organizations, it is also an opportunity for malicious users to launch mail-flooding and DDoS attacks. It is important to stop malicious senders in their tracks for your own protection and the protection of others.
To block a message efficiently, a transport agent must interrupt the SMTP conversation with the remote host before your server confirms the receipt of the data with a 250 OK status code. According to the SMTP store-and-forward principle, your server can safely discard any received data without generating NDRs if message delivery was not confirmed. SMTP receive agents can accomplish this. They interact with the SMTP session because the transport pipeline invokes these agents based on SMTP receive events as the remote host connects to the server, establishes an SMTP session, transmits SMTP verbs, submits messages, and terminates the connection. (The SMTP receive events related to each step are listed in Figure 6.) Because of the ability to reject messages prior to delivery and disconnect remote SMTP hosts, all Exchange Server 2007 anti-spam agents are implemented as SMTP receive agents.
Figure 6 SMTP Session events (Close Figure 6)

Action | Related Events
Connect to server | OnConnectEvent
Establish SMTP session | OnHeloCommand, OnEhloCommand, OnAuthCommand, OnEndOfAuthentication
Transmit SMTP verbs | OnMailCommand, OnRcptCommand, OnDataCommand, OnNoopCommand, OnHelpCommand
Submit messages | OnEndOfHeaders, OnEndOfData
Reject command or message | OnReject
Reset connection | OnRsetCommand
Terminate connection | OnDisconnect
It is important to recognize the difference between SMTP receive agents and routing agents with regard to their processing context. While routing agents have full access to the message properties, SMTP receive agents are more context-sensitive because these agents interact with the SMTP session. For example, it is not possible for a spam filter to act on message properties until the remote host has actually transferred the message. Thus it is important to register the agent for the correct SMTP receive event. See the "Transport Agent Development" sidebar for a deeper look.

Exchange Resources
  • Microsoft Exchange Server 2007 Edge Transport and Messaging Protection (Microsoft IT Showcase Technical White Paper)
  • Windows PowerShell Script Center
  • Exchange Server 2007 TechCenter
  • Microsoft Exchange Team Blog "You Had Me From EHLO"

Stay Tuned
Let's take a break before going deeper into the transport architecture and test scenarios. I covered a lot of ground, ranging from a Microsoft IT-style deployment of Edge Transport servers to the inner events triggered in the transport pipeline during message processing. In the next installment of this two-part series, I'll continue by analyzing the behavior of Edge Transport agents in a few interesting test scenarios.
In the meantime, I recommend downloading the 90-day trial version of Microsoft Visual Studio 2005 Professional Edition (go.microsoft.com/fwlink/?LinkId=98043) and follow Steve's explanations to compile and install sample agents in your test environment. Even if you are not a developer, you will find that accomplishing these tasks is very straightforward. It would not surprise me to see an increasing number of business applications rely on custom agents, given how convenient it is in Exchange Server 2007 to develop these components with Visual Studio 2005.

Kay Unkroth is an entrepreneur who has worked as a support engineer, system developer, consultant, trainer, and author with focus on Microsoft server technologies for more than 15 years. Kay is also co-founder and chairman of Biblioso Corporation, a company that specializes in managed documentation and localization services.


 SQL Server: Optimizing SQL Server CPU Performance
SQL Server Zach Nichter

Optimizing SQL Server CPU Performance
At a Glance:
  • Troubleshooting database performance issues
  • Reviewing hardware causes
  • Using PerfMon to track database bottlenecks
  • Evaluating query performance

Troubleshooting performance problems on a database system can be an overwhelming task. Knowing where to look for trouble is important, but more crucial is knowing why your system reacts the way that it does to a particular request. A number of factors can affect CPU utilization
on a database server: compilation and recompilation of SQL statements, missing indexes, multithreaded operations, disk bottlenecks, memory bottlenecks, routine maintenance, and extract, transform, and load (ETL) activity, among others. CPU utilization is not a bad thing in itself—performing work is what the CPU is there for. The key to healthy CPU utilization is making sure that the CPU is spending its time processing what you want it to process and not wasting cycles on poorly optimized code or sluggish hardware.

Two Paths Leading to the Same Place
When viewed from a high level, there are two paths to identifying CPU performance problems. The first is reviewing the system's hardware performance, an exercise that helps determine where to look when you head down the second path, reviewing the server's query efficiency. This second path is usually more effective in identifying SQL Server™ performance issues. Unless you know exactly where your query performance issues lie, however, you should always start with a system performance evaluation. In the end, you'll usually end up taking both paths. Let's lay some groundwork so that we can review both of these paths.

Laying the Ground Work
Hyper-Threading
Hyper-threading is a topic that is worth discussing a bit more because of the way that it affects SQL Server. Hyper-threading actually presents two logical processors to the OS for each physical processor. Hyper-threading essentially leases time on the physical processors so that each processor ends up more fully utilized. The Intel Web site (intel.com/technology/platform-technology/hyper-threading/index.htm) gives a far fuller description of how hyper-threading works.
On SQL Server systems, the DBMS actually handles its own extremely efficient queuing and threading to the OS, so hyper-threading only serves to overload the physical CPUs on systems with already high CPU utilization. When SQL Server queues multiple requests to perform work on multiple schedulers, the OS has to actually switch the context of the threads back and forth on the physical processors to satisfy the requests that are being made even if the two logical processors are sitting on top of the same physical processor. If you are seeing Context Switches/sec higher than 5000 per physical processor you should strongly consider turning off hyper-threading on your system and retesting performance.
In rare cases, applications that experience high CPU utilization on SQL Server can effectively use hyper-threading. Always test your applications against SQL Server with hyper-threading both turned on and off before implementing changes on your production systems.

A high-end dual-core processor will easily outperform the RAM in a machine, which will in turn be faster than an attached storage device. A good CPU can handle approximately six times the throughput of current top-end DDR2 memory and about two times that of top-end DDR3 memory. Typical memory throughput is more than 10 times that of the fastest fiber channel drives. In turn, hard disks can only perform a finite number of IOPS (input/output operations per second), a value which is entirely limited by the number of seeks per second a drive can perform. To be fair, it is not typical that only a single storage drive is used to handle all of the storage needs on enterprise database systems. Most setups today utilize Storage Area Networks (SANs) on enterprise database servers or larger RAID groups that can nullify or minimize the disk I/O Processor issue. The most important thing to remember is that no matter what your setup looks like, disk and memory bottlenecks can affect the performance of your processors.
Because of the I/O speed differences, retrieving data from disk is much more costly than retrieving data from memory. A data page in SQL Server is 8KB. An extent in SQL Server is made up of eight 8KB pages, making it equivalent to 64KB. This is important to understand because when SQL Server requests a particular data page from disk, it is not just the data page that is retrieved but the entire extent in which the data page resides. There are reasons that actually make this more cost-effective for SQL Server, but I won't go into details here. Pulling a data page that is already cached from the buffer pool, at peak performance, should take under half a millisecond; retrieving a single extent from disk should take between 2 and 4 milliseconds in an optimal environment. I typically expect a well-performing, healthy disk subsystem read to take between 4 and 10ms. Retrieving a data page from memory is generally between 4 and 20 times faster than pulling a data page from disk.
When SQL Server requests a data page, it checks the in-memory buffer cache before looking for the data page on the disk subsystem. If the data page is found in the buffer pool, the processor will retrieve the data and then perform the work required. This is called a soft page fault. Soft page faults are ideal for SQL Server because the data that is retrieved as part of a request must be in the buffer cache before it can be used. A data page that is not found in the buffer cache must be retrieved from the server's disk subsystem. When the OS has to retrieve the data page from disk, it's known as a hard page fault.
When correlating memory performance, disk performance, and CPU performance, a common denominator helps us put everything in perspective: throughput. In not-so-scientific terms, throughput is the measurement of how much data you can stuff down a finite pipe.

Path 1: System Performance
There are really only a few methods for determining if a server has a CPU bottleneck, and there aren't many potential causes of high CPU utilization. Some of these issues can be tracked using PerfMon or a similar system-monitoring tool while others are tracked using SQL Profiler or similar tools. Another method is to use SQL commands through Query Analyzer or SQL Server Management Studio (SSMS).
The philosophy I use when evaluating a systems performance is "Start broad, then focus deep." Obviously, you can't focus on problem areas until you've identified them. After you evaluate overall CPU utilization with a tool like PerfMon, you can use it to look at a couple of very simple and easy-to-understand performance counters.
One of the most familiar performance counters is % Processor Time; when you're in PerfMon, it's highlighted as soon as you open the Add Counter window. % Processor Time is the amount of time the processors stay busy executing work. Utilization on processors is generally considered high when this value is 80 percent or higher for most of your peak operating time. It's typical, and should be expected, that you will see spikes up to 100 percent at times even when the server is not operating with 80 percent utilization.
Another counter you should review is Processor Queue Length, which can be found under the System performance object in PerfMon. Processor Queue Length shows how many threads are waiting to perform work on the CPU. SQL Server manages its work through schedulers in the database engine, where it queues and processes its own requests. Because SQL Server manages its own work, it will only utilize a single CPU thread for each logical processor. This means that there should be minimal threads waiting in the processor queue to perform work on a system dedicated to SQL Server. Typically you shouldn't have anything higher than five times the number of physical processors on a dedicated SQL Server, but I consider more than two times problematic. On servers where the DBMS shares a system with other applications, you will want to review this along with the % Processor Time and Context Switches/sec performance counters (I'll discuss context switches shortly) to determine if your other applications or the DBMS needs to be moved to a different server.
When I see processor queuing along with high CPU utilization, I look at the Compilations/sec and Re-Compilations/sec counters under the SQL Server: SQL Statistics performance object (see Figure 1). Compiling and recompiling query plans adds to a system's CPU utilization. You should see values close to zero for the Re-Compilations, but watch trends within your systems to determine how your server typically behaves and how many compiles are normal. Recompiles can't always be avoided, but queries and stored procedures can be optimized to minimize recompiles and to reuse query plans. Compare these values to the actual SQL statements coming into the system through the Batch Requests/sec also found in the SQL Server: SQL Statistics performance object. If the compilations and recompilations per second comprise a high percentage of the batch requests that are coming into the system, then this is an area that should be reviewed. In some situations SQL developers may not understand how or why their code can contribute to these types of system resource problems. Later in this article I'll provide some references to help you minimize this type of activity.
Figure 1 Selecting the counters to monitor
Figure 1 Selecting the counters to monitor
While you're in PerfMon, check out the performance counter called Context Switches/sec (see Figure 2). This counter tells how many times threads have to be taken out of the OS schedulers (not SQL schedulers) to perform work for other waiting threads. Context switches are often much more frequent on database systems that are shared with other applications like IIS or other vendor application server components. The threshold that I use for Context Switches/sec is about 5000 times the number of processors in the server. This value can also be high on systems that have hyper-threading turned on and also have moderate to high CPU utilization. When CPU utilization and context switches both exceed their thresholds regularly, this indicates a CPU bottleneck. If this is a regular occurrence, you should start planning for the purchase of more or faster CPUs if your system is outdated. For further information, see the "Hyper-Threading" sidebar.
Figure 2 Performance counters to watch (Close Figure 2)

Performance Counter | Counter Object | Threshold | Notes
% Processor Time | Processor | > 80% | Potential causes include memory pressure, low query plan reuse, non-optimized queries.
Context Switches/sec | System | > 5000 x processors | Potential causes include other applications on the server, more than one instance of SQL Server running on the same server, hyper-threading turned on.
Processor Queue Length | System | > 5 x processors | Potential causes include other applications on the server, high compilations or recompilations, more than one instance of SQL Server running on the same server.
Compilations/sec | SQLServer:SQL Statistics | Trend | Compare to Batch Requests/sec.
Re-Compilations/sec | SQLServer:SQL Statistics | Trend | Compare to Batch Requests/sec.
Batch Request/sec | SQLServer:SQL Statistics | Trend | Compare with the Compilation and Re-Compilations per second.
Page Life Expectancy | SQLServer:Buffer Manager | < 300 | Potential for memory pressure.
Lazy Writes/sec | SQLServer:Buffer Manager | Trend | Potential for large data cache flushes or memory pressure.
Checkpoints/sec | SQLServer:Buffer Manager | Trend | Evaluate checkpoints against PLE and Lazy Writes/sec.
Cache Hit Ratio: SQL Plans | SQLServer:Plan Cache | < 70% | Indicates low plan reuse.
Buffer Cache Hit Ratio | SQLServer:Buffer Manager | < 97% | Potential for memory pressure.
The SQL Server Lazy Writer (as it's called in SQL Server 2000) or the Resource Monitor (as it's called in SQL Server 2005) is another area to monitor when CPU utilization is high. Flushing the buffer and procedure caches can add to CPU time via the resource thread called the Resource Monitor. The Resource Monitor is a SQL Server process that determines which pages to keep and which pages need to be flushed from the buffer pool to disk. Each page in the buffer and procedure caches are originally assigned a cost representing the resources that are consumed when that page is placed into the cache. This cost value is decremented each time the Resource Monitor scans it. When a request requires cache space, the pages are flushed from memory based upon the cost associated to each page; pages with the lowest values are the first to be flushed. The Resource Monitor's activity can be tracked through the Lazy Writes/sec performance counter under the SQL Server: Buffer Manager object within PerfMon. You should track how this value trends to determine what threshold is typical on your system. This counter is usually reviewed along with the Page Life Expectancy and Checkpoints/sec counters to determine whether there is memory pressure.
The Page Life Expectancy (PLE) counter helps determine memory pressure. The PLE counter shows how long a data page stays in the buffer cache. 300 seconds is the industry-accepted threshold for this counter. Anything less than a 300-second average over an extended period of time tells you that the data pages are being flushed from memory too frequently. When this happens it causes the Resource Monitor to work harder, which in turn forces more activity onto the processors. The PLE counter should be evaluated along with the Checkpoints Pages/sec counter. When a checkpoint occurs in the system, the dirty data pages in the buffer cache are flushed to disk, causing the PLE value to drop. The Resource Monitor process is the mechanism that actually flushes these pages to disk, so during these checkpoints you should also expect to see the Lazy Writes/sec value increase. If your PLE value goes up immediately after a checkpoint is completed, you can ignore this temporary symptom. On the other hand, if you find that you are regularly below the PLE threshold there is a very good chance that additional memory will alleviate your problems and at the same time release some resources back to the CPU. All of these counters are found in the SQL Server: Buffer Manager performance object.

Path 2: Query Performance
SP Tracing
When tracing your SQL Server app, it pays to become familiar with the stored procedures used for tracing. Using the GUI interface (SQL Server Profiler) for tracing can increase system load by 15 to 25 percent. If you can utilize stored procedures in your tracing, this can drop by about half.
When I know that my system is bottlenecked somewhere and I want to determine which current SQL statements are causing problems on my server, I run the query that you see below. This query helps me get a view of individual statements and the resources that they are currently utilizing, as well as statements that need to be reviewed for performance enhancements. For more information on SQL traces, see msdn2.microsoft.com/ms191006.aspx.
SELECT
    substring(text,qs.statement_start_offset/2
        ,(CASE
            WHEN qs.statement_end_offset = -1 THEN len(convert(nvarchar(max), text)) * 2
            ELSE qs.statement_end_offset
        END - qs.statement_start_offset)/2)
    ,qs.plan_generation_num as recompiles
    ,qs.execution_count as execution_count
    ,qs.total_elapsed_time - qs.total_worker_time as total_wait_time
    ,qs.total_worker_time as cpu_time
    ,qs.total_logical_reads as reads
    ,qs.total_logical_writes as writes
FROM sys.dm_exec_query_stats qs
    CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) st
    LEFT JOIN sys.dm_exec_requests r
        ON qs.sql_handle = r.sql_handle
ORDER BY 3 DESC

Query plans are evaluated, optimized, compiled, and placed in the procedure cache when a new query is submitted to SQL Server. Each time a query is submitted to the server, the procedure cache is reviewed to attempt to match a query plan with a request. If one is not found then SQL Server will create a new plan for it, which is a potentially costly operation.
Some considerations for T-SQL CPU optimization are:
  • Query plan reuse
  • Reducing compiles and recompiles
  • Sort operations
  • Improper joins
  • Missing indexes
  • Table/index scans
  • Function usage in SELECT and WHERE clauses
  • Multithreaded operations
So let's pull this back into perspective a bit. SQL Server typically pulls data from both memory and from disk, and it's not often that you are working with just a single data page. More often, you have multiple parts of an application working on a record, running multiple smaller queries or joining tables to provide a full view of relevant data. In OLAP environments, your applications may be pulling millions of rows from one or two tables so that you can consolidate, roll up, and summarize data for a regional sales report. In situations like these, returning data can be measured in milliseconds if the data is in memory, but those milliseconds can turn into minutes when retrieving the same data from disk instead of RAM.
The first example is a situation with a high volume of transactions and plan reuse depends on the application. Low plan reuse causes a large number of compilations of SQL statements which in turn causes a great deal of CPU processing. In the second example, the heavy system resource utilization can cause a system's CPU to be overly active, as existing data has to be constantly flushed from the buffer cache to make room for the large volume of new data pages.
Consider a highly transactional system, where a SQL statement like the one shown below is executed 2000 times over a 15-minute period in order to retrieve shipping carton information. Without query plan reuse you could hypothetically have an individual execution time of around 450ms per statement. If the same query plan is used after the initial execution, every subsequent query could probably run in about 2ms, bringing the total execution time down to about 5 seconds.
USE SHIPPING_DIST01;
SELECT
    Container_ID
    ,Carton_ID
    ,Product_ID
    ,ProductCount
    ,ModifiedDate
FROM Container.Carton
WHERE Carton_ID = 982350144;
Query plan reuse is critical for optimal performance on highly transactional systems and it is most often achieved by parameterizing your queries or stored procedures. Here are a few excellent resources for information on query plan reuse:
  • Batch Compilation, Recompilation, and Plan Caching Issues in SQL Server 2005 (microsoft.com/technet/prodtechnol/sql/2005/recomp.mspx)
  • Optimizing SQL Server Stored Procedures to Avoid Recompiles (sql-server-performance.com/rd_optimizing_sp_recompiles.asp)
  • Query Recompilation in SQL Server 2000 (msdn2.microsoft.com/aa902682.aspx)
One helpful place that has a wealth of information is the SQL Server 2005 dynamic management views (DMVs). When CPU utilization is high, there are a couple of DMVs that I use to help me determine if the CPU is being utilized appropriately or not.
One of the DMVs I review is sys.dm_os_wait_stats, which is used to supply DBAs with a means to determine each resource type or function that SQL Server uses and it measures the amount of time the system waits because of that resource. The counters in this DMV are accumulative. This means that in order to get a clean look at what resources could be affecting different areas of the system, you will first have to issue a DBCC SQLPERF('sys.dm_os_wait_stats', CLEAR) command to reset all of the counters after you review the data for any outstanding issues. The sys.dm_os_wait_stats DMV is the equivalent of the database consistency check DBCC SQLPERF(WAITSTATS) command in SQL Server 2000. You can find out more about the different wait types in SQL Server Books Online at msdn2.microsoft.com/ ms179984.aspx.
It's important to know that waits are typical in a system even when everything is running optimally. You just have to determine if the waits are being affected by a CPU bottleneck. Signal waits should be as low as possible in relation to the overall wait time. The amount of time a particular resource waits for a processor resource can be determined simply by subtracting the signal wait time from the total wait time; this value should not be greater than approximately 20 percent of the total wait time.
The sys.dm_exec_sessions DMV shows all of the open sessions on the SQL Server. This DMV provides a high-level view of the performance of each session and all of the work that each session has performed from its inception. This includes the total amount of time the session has spent waiting, total CPU usage, memory usage, and a count of reads and writes. The DMV will also provide you with the login, login time, host machine, and the last time the session made a request of SQL Server.
Using the sys.dm_exec_sessions DMV, you will be able to determine only the active sessions, so if you are seeing high CPU utilization this is one of the first places to look. Review the sessions that have a high CPU count first. Determine the application and the user that has been performing the work, then start to dig deeper. Pairing sys.dm_exec_sessions with the sys.dm_exec_requests DMV can provide much of the information that is available through the sp_who and sp_who2 stored procedures. If you join this data together along with the sys.exec_sql_text dynamic management function (DMF) through the sql_handle column, you can get the session's currently running query. The snippet in Figure 3 shows how to pull this data together to help determine what is currently happening on a server.
Figure 3 Determining server activity (Close Figure 3)
SELECT es.session_id
    ,es.program_name
    ,es.login_name
    ,es.nt_user_name
    ,es.login_time
    ,es.host_name
    ,es.cpu_time
    ,es.total_scheduled_time
    ,es.total_elapsed_time
    ,es.memory_usage
    ,es.logical_reads
    ,es.reads
    ,es.writes
    ,st.text
FROM sys.dm_exec_sessions es
    LEFT JOIN sys.dm_exec_connections ec
        ON es.session_id = ec.session_id
    LEFT JOIN sys.dm_exec_requests er
        ON es.session_id = er.session_id
    OUTER APPLY sys.dm_exec_sql_text (er.sql_handle) st
WHERE es.session_id > 50    -- < 50 system sessions
ORDER BY es.cpu_time DESC
I find that this statement helps determine which applications need to be focused on. When I compare CPU, memory, reads, writes, and logical reads for all of the sessions within an application and I determine that the CPU resource is much higher than other resources that are utilized, I start focusing on the on those SQL statements.
To track SQL statements historically for an application I use SQL Server traces. You can get to these either through the SQL Server Profiler tool or through the trace system stored procedures to help evaluate what is going on. (See the sidebar "SP Tracing" for more information on this topic.) Profiler should be reviewed for statements with high CPU utilization as well as hash and sort warnings, cache misses, and other red flags. This can help you narrow in on specific SQL statements or a specific time period that caused high resource utilization. Profiler is capable of tracking things like the SQL statement text, execution plans, CPU usage, memory usage, logical reads, writes, caching of query plans, recompiles, ejection of query plans from the cache, cache misses, table and index scans, missing statistics, and many other events.
After I have collected data from either sp_trace stored procedures or the SQL Server Profiler, I generally use a database, which is populated with trace data either after the fact or by setting the trace to write to the database. Populating the database after the fact can be done using the SQL Server system function called fn_trace_getinfo. The benefit of this approach is that I can query and sort the data in multiple ways to see what SQL statements used the most CPU or had the most reads, count how many recompiles occur, and many other things. Here's an example of how this function is used to load a table with a profiler trace file. The default specifies that all of the trace files for that trace will be loaded in the order that they were created:
SELECT * INTO trc_20070401
FROM fn_trace_gettable('S:\Mountpoints\TRC_20070401_1.trc', default);
GO

Wrapping It Up
As you've seen, high CPU utilization doesn't necessarily indictate that there is a CPU bottleneck. High CPU utilization can be masking a number of other application or hardware bottlenecks too. Once you've identified that you have high CPU utilization despite your other counters looking healthy, you can start looking for the cause within the system, and isolate a solution (whether it be purchasing more CPUs or optimizing your SQL code). And whatever you do, don't give up! With the tips provided in this article, along with a bit of practice and research, optimizing CPU utilization under SQL Server is an attainable execution plan.

Zach Nichter is a SQL Server professional with more than 10 years of experience. He has held a number of SQL Server support roles including DBA, team lead, manager, and consultant. Currently, Zach is employed by Levi Strauss & Co. as the DBA Architect, focusing on SQL Server performance, monitoring, and architecture as well as other strategic initiatives. In addition, Zach is the author of a video blog found on www.sqlcatch.com.


 From the Editor: The Best is Yet To Come
From the Editor
The Best is Yet To Come
Joshua Hoffman

Believe it or not, I'm going to begin my first Editor's Note with a flashback. Mind you, use of this sort of literary device so early in the game does not bode well for me, but, just the same, here we go.
In the summer of 2005, the two inaugural issues of TechNet Magazine having been very well received, it was time to ramp up the production schedule and start delivering great technical content to IT professionals on a regular basis. Joshua Trupin, our Executive Editor, and Stephen Toub (then Technical Editor of our sister publication, MSDN Magazine), had done an amazing job lining up experts and planning those first few issues, but they both had day jobs to get back to.
So I joined the team as the new Technical Editor, humbled by the confidence they placed in me and the job they had done so far. After all, producing a magazine is no small feat. I came from an IT pro background (pure geek), spending the previous four years in Microsoft Consulting Services. I had relatively little (no) knowledge of how publishing really worked. I had a reading list a mile long. I had three issues to start planning immediately. I had new teammates, authors, technical reviewers, business partners, and, of course, readers to get to know. And, above all, I had print deadlines that wouldn't budge (which, as you can imagine, is a difficult concept for a long-time Microsoft employee to grasp).
But the team embraced me, endured and humored me, and did more to help me get my job done than you can imagine. The authors, reviewers, partners, and you—our readers—have made this job fascinating, challenging, and deeply rewarding, and I'm privileged to have made so many new friends. I look back on the past two years proud of what this team has accomplished. But now, as the new Editor-in-Chief, I'm even more excited by what lies ahead.
This magazine exists with the sole mission of making IT professionals successful with Microsoft products and technologies (trust me; our paychecks depend on doing that, above all else). You've told us where we've done well and, more importantly, where we need to improve. We take that feedback very seriously. Over the coming months and years we will continue to improve, adding new authors and new columns, enhancing our online platform with new content and new media, and hopefully enabling you to learn in whatever way suits you best. None of that can be successful, however, without your continued input. Tell us what works for you and what doesn't. Where should we spend our time? You're in control, so you might as well take advantage of it. E-mail tnmag@microsoft.com and let your voice be heard. —Joshua Hoffman

Thank you to the following Microsoft technical experts: Omesh Desai, Raja Ganjikunta, Saleem Hakani, Justin Hall, Michael Murgolo, John Orefice, Jim Truher, Marc Umeno, Jeff Wettlaufer, Buck Woody, and Minxiao Zhou.


 Toolbox: New Products for IT Pros
Toolbox
New Products for IT Pros
Greg Steen

More Powerful Scripting
PowerShell Analyzer
www.pwershellanalyzer.com
There is a lot of buzz about Windows PowerShell™, and with good reason. If you haven't had a chance to play with this next generation scripting language, you really should check it out. And those of you who have upgraded to Microsoft® Exchange 2007 will quickly realize that the new Exchange Management Shell is built upon Windows PowerShell, allowing you to do everything the Exchange Management Console can but from the command line, which is great for automating repetitive tasks. Unfortunately, there can be a bit of a learning curve to get the syntax and semantics down. And here is where PowerShell Analyzer, from ShellTools, comes into play.
The application is basically an IDE for writing and debugging Windows PowerShell scripts. But unlike with a traditional write-compile-test cycle, the application keeps the spirit of the admin at the keyboard, giving you real-time interactivity with a Windows PowerShell runspace. PowerShell Analyzer allows you to either enter commands line-by-line or build, edit, and run Windows PowerShell scripts in an editor in the lower part of the UI. Both options give you basic code completion, which is a great help for guiding you on the many parameters of the cmdlets. (In case you're new to Windows PowerShell, cmdlets are abstract, task-oriented, parameterized commands.)
For even more help, PowerShell Analyzer provides a quick-access tab that shows you detailed descriptions, syntax, parameter explanations, and examples for each of the cmdlets. In addition, there are references for the Windows PowerShell Providers, the available built-in help files, and the cmdlet short aliases.
When you run commands from Windows PowerShell, you are actually returned Microsoft .NET Framework objects rather than just the apparent text results on the screen. PowerShell Analyzer allows you to capture the properties of the returned results and pass them to a number of "visualizers." These visualizers allow you to interact with the objects—such as offering a tree-like exploration of XML data or a sortable and groupable hierarchal table view of the data—as well as generate a number of different charts to visually represent the returned dataset. Of course, not all these options are available for every command or script you run, but, when applicable, they do give you a nice set of representations of your data that you can then save or export for reuse.
PowerShell Analyzer also allows you to work with multiple runspaces simultaneously, letting you easily switch between tasks. And the editor window gives you code-outlining and customizable code-highlighting styles to help make your scripts more readable.
All in all, if you are going to be working with Windows PowerShell, a tool like PowerShell Analyzer will make creating, editing, and debugging your scripts, and visualizing your data much easier.

Price: $129 (direct) for a single license.
Figure powershell

Defrag Disks
Auslogics Disk Defrag
www.auslogics.com/disk-defrag
The more you use your hard drive, the more fragmented the files on that disk will become. Creating and deleting files, installing and removing applications, and even generating temporary disk caches and files all contribute to fragmentation. And the rate of fragmentation can increase dramatically when the free space on the drive is limited, as new files will be spread across the available "holes" in a mostly filled space. All this fragmentation will lead to slower system performance.
Everyone knows, of course, that current versions of Windows® include built-in defragmentation tools. But if you are looking for a dedicated option or if you have recently upgraded to Windows Vista® and miss the control and UI presentation offered by earlier versions of the defragmentation tool, there's a very interesting alternative you might want to try: Disk Defrag, from Auslogics.
Disk Defrag works with Windows XP, Windows 2000, Windows Server® 2003, and Windows Vista. And, as you would likely expect, it can defragment FAT16, FAT32, and NTFS volumes. The application is lightweight and installation takes just a few quick clicks.
Running the application is just as easy. All you have to do is simply pick the drive you wish to defragment, click Next, and watch it go. Once it has completed running, Disk Defrag gives you a quick overview of the results, showing the total number of files, directories, fragmented files, defragmented files, and skipped files, as well as the percentage decrease in fragmentation. If you like, you can also click Disk Defrag's Display Report button to have the application generate an HTML version of the defragmentation report. This report gives you some additional useful information regarding your disk, such as a list of skipped files with their location, the number of fragments, the time elapsed for the run, and the cluster information for your disk.

Price: Free.
Disk Defrag is easy to install and use
Disk Defrag is easy to install and use

Transfer Files
FileZilla
filezilla.sourceforge.net
There's nothing like a good SFTP client to make your job easier—this is especially true when the client is solidly written and free, which is the case with the FileZilla client. FileZilla is an open-source project, hosted on SourceForge.net, that was started by Tim Kosse and a couple of his cohorts back in 2001 for a computer science class project. FileZilla runs on all flavors of Windows from Windows NT®4.0 to Windows Vista. The project has grown substantially over the past six years, but Kosse is still the project leader for the app.
In addition to standard FTP connections, the client supports connections via SFTP with Secure Shell version 2 (SSH2), FTP over TLS, and FTP over SSL with implicit and explicit encryption, so you'll be able to connect to a variety of servers. FileZilla can accommodate local firewalls by letting you limit the local ports used as well as set a specific IP binding for non-passive transfers for both IPv4 and IPv6. The client also supports a number of proxy configurations, including SOCKS4/5, HTTP1.1, and FTP proxies.
In terms of server authentication for your FTP connections, FileZilla supports anonymous, normal user name/password, and account-based configurations. Also, you can enable Kerberos Generic Security Services (GSS) support (which requires you to have Kerberos for Windows installed on your system) and create a list of servers that are GSS-enabled. Of course, you'll need a valid Kerberos v5 ticket before GSS will work for you. You can also enable support for your Ident server for connecting to servers that require it as a means of identifying your client.
FileZilla easily manages a large collection of FTP sites and allows you to organize your connections into a tree-like structure. For each connection that you set up, you can also set default local and remote directories, specify the port to which you should connect, choose to bypass your configured proxy, explicitly choose active or passive transfer modes, and even set a server time zone offset. In addition to the saved connections, FileZilla also has a Quickconnect feature for those one time transfers so you can just type the address, user name, password, and port, and then click a button to connect. Once you have all your settings and connections configured, you can export them to XML for backup or reuse on other systems.
The UI of the application, which is similar to the Windows Explorer interface, allows you to easily drag and drop files between the local and remote systems. For file transfers, you can specify a default file overwrite setting in addition to the option to limit your download and upload speeds based on a predefined set of rules or by a constant kBs speed limit. By default, FileZilla is configured to use MODE Z compressed file transfers on servers that support it, but you can adjust the compression level or turn the feature off. The client also has a transfer queue feature that lets you queue up a set of files to transfer and then import and export that transfer list for reuse. This is great for repetitive system administration tasks.

Price: Free.

Edit Executables
PE Explorer
www.heaventools.com
For those of you who really like to get into the nitty-gritty of your applications, you might want to check out PE Explorer from HeavenTools. This application lets you delve into the internals of your Portable Executable (PE) files, which are used for executable binaries for Windows applications. Simply put, PE Explorer provides a UI for exploring and editing the contents of these executables. PE Explorer can open a variety of file types ranging from the common, such as EXEs and DLLs, to the less familiar types, such as DPL and CPL files.
When you first choose to explore an executable, PE Explorer shows you information about the headers of the file, such as the number of code sections, the size of the image, the application subsystem, and the stack size information. Another view offers you an overview of the section headers in the executable—double-clicking a section header brings up a window gives you the ability to explore the contents of each section.
Definitely not for the faint of heart, the application lets you extract, recalculate, and delete sections of your loaded PE. A great tool for detecting viruses, malware, and other executable nasties, you can use the application's Digital Signature Viewer to review and validate the Microsoft Authenticode digital signature, if present, in the loaded executable file. This is a powerful way to verify the publisher and the integrity of the executable.
PE Explorer also contains a built-in quick disassemble, which lets you look at the assembly code of your executable. And it supports the common Intel x86 instruction sets along with extensions such as MMX, 3D Now!, and SSE/2/3. The disassembler also extracts ASCII text strings from the data portion of your PE.
Another feature is the Dependency Scanner, which scans all the modules that your PE file links to statically and those that are delay-loaded, and it then displays them in a hierarchal tree structure, showing where the PE reaches to.
One of the more fun parts of PE Explorer, in my opinion, is the Resource Editor. This feature allows you to view, extract, replace, edit, and delete the resources of a specified executable. The UI shows you a directory-like structure of the embedded resources, such as images, sounds, dialogs, menus, XML data, HTML data, and toolbars. Not all of those resources support direct editing, but for most you can replace and edit them. This would allow you, for instance, to add your own custom branding to an app, change dialog messages, customize toolbar actions, and so on without having access to the actual source code. PE Explorer is a very handy tool for those who want to dive into executables. But if Resource Editor is the only feature you really want, take a look at the company's much cheaper Resource Tuner application.

Price: $129 (direct) for a personal license; $199 (direct) for a business license.
Figure peexplorer

Book Review
Endpoint Security
www.awprofessional.com
Adding a rock-solid traditional network perimeter to your internal infrastructure is an important step in protecting your assets. But this isn't a silver bullet. With the increased mobility and connectivity of today's hardware and software, there is no real perimeter on your corporate infrastructure—instead, your infrastructure extends to those remote endpoints. In fact, while you guard yourself against attack from outside your environment, it is more likely that an attack will be launched accidentally by a user who brings his laptop in from home and hops on your uncontrolled corporate network.
If you're looking to get up to speed on network access control and secure the endpoints of your infrastructure, you might want to check out Mark Kadrich's book, Endpoint Security (Addison-Wesley Professional, 2007). The book starts by defining what exactly endpoints are and then gives you a taste of the many different variations that exist—Windows clients, Linux clients, embedded devices, mobile phones, and PDAs. Kadrich sees the network as a "control problem." Rejecting the typical representation of a complex network with some organic analogy, he instead describes it as something that can be delineated and visualized in a useful, manageable manner.
The book delves into the basics of Network Access Control (NAC) and establishing a base level of trust. This discussion includes how to put together a secure baseline for your endpoint systems by securing and controlling your source software and build environment. An overview of tools that can help keep endpoints secure and reliable covers the familiar, such as using a firewall and antivirus software, and points to the need for proactive patch management, and, finally, more advanced tools, such as intrusion detection and prevention systems, host integrity checkers, and encryption.
The second half of the book explores the details of securing the operating systems of various client types, including Microsoft Windows, Apple OS X, and Linux. Each of these chapters gives you an overview of how to perform an Initial Health Check for your endpoint system. And there are chapters dedicated to PDAs, Smartphones, and embedded devices that discuss how these devices have become a serious threat to your infrastructure and how to secure various types of communication on them.
Finally, Kadrich looks at four case studies and identifies in each how the endpoints were compromised and how failure could have been mitigated. Overall, Endpoint Security is loaded with useful information for the security professional, offering a vendor-agnostic view of securing your network from the vantage of an endpoint perimeter.

Price: $54.99 list.

Greg Steen is a technology professional, entrepreneur, and enthusiast on the hunt for new tools and references to help make operations and development easier for IT professionals. Have a suggestion? Let him know at tntools@microsoft.com


 SQL Q&amp;A: Login Triggers, Data File Defrags, and More
SQL Q&A
Login Triggers, Data File Defrags, and More
Edited by Nancy Michell

Setting the Service Account
Tip: More Secure Passwords
The SQL Server 2000 engine maintains two copies of each SQL Server login password. One version is the actual password supplied by the user, the second is the password in all uppercase letters.
This practice helps in case-insensitive validation of passwords, because a user can log in using either mixed-case or uppercase and be granted access to the server. However, this convenience comes with a catch. Saving passwords in all uppercase makes brute-force password-guessing attacks easier by reducing the number of possible passwords.
SQL Server 2005 stores only the original copy of the password. A password entered by a user must match the password stored on the server. If the password does not match, the login fails and the user is denied access. If the precise case of the password characters is forgotten, the password must be reset.
Assuming a user's login name is SQLCOMMUNITY, you can reset his SQL Server password using the following command:
Use Master;
ALTER LOGIN SQLCOMMUNITY WITH PASSWORD = 'k3t9h4s8wJF7t';
This command would reset the password for SQLCOMMUNITY SQL Server login to "k3t9h4s8wJF7t".

Q In SQL Server™ 2000, I used to set the service account for the SQL Server Engine and Agent using the Services applet in Administrative Tools. Now I hear that in SQL Server 2005 I'm supposed to use the Configuration Manager tool. Why can't I just keep using the Windows tools?
A SQL Server 2005 is built to be more secure than previous versions. In many shops, users would just set internal accounts, such as LocalSystem, to run SQL Server. But these accounts often either have more or fewer rights and permissions under Windows® than they need. You should create a Windows account with no elevated privileges to run the SQL Server 2005 Engine and Agent services. If you select these accounts with the Configuration Manager, they will automatically be granted the proper rights and permissions in both SQL Server and the operating system. If you use the Windows tools to manage the SQL Server services, you might not grant the proper rights, or you may grant too many.
For more details, see the tip Changing the Service Account.

Who's Logging onto My Server?
Q I want to know who is logging onto my server and when. I also want to restrict some specific users to certain time periods and would like to know how to fire a trace to track down user activity. Is any of this possible?
A Yes, you can do all of these things with SQL Server 2005 if you have Service Pack 2 installed.
SQL Server 2005 allows you to create login triggers that can fire a T-SQL or stored procedure in response to a LOGON event. You can use a login trigger to audit and control users by tracking login activity, restricting logins to SQL Server, or limiting the number of sessions for specific logins. Note that the event is fired only after a login is successfully authenticated, but just before the user session is actually established. Therefore, all messages originating from inside the trigger (such as messages or errors) from the PRINT statement are sent to the SQL Server error log. If the authentication happens to fail for a login, then the Logon triggers are not fired.
The following example shows how you can create a login trigger and send a message to the SQL Server error log as soon as any user logs in:
ALTER TRIGGER Ops_Login
ON ALL SERVER
AFTER LOGIN
AS
PRINT SUSER_SNAME() + ' has just logged in to ' + LTRIM(@@ServerName) + ' SQL Server at '+LTRIM(getdate())
GO
To view all the triggers set at the server level, use the following query:
SELECT * FROM sys.server_triggers;

Defragging Best Practices
Q What's the best way to fix data file fragmentation in SQL Server? If we use the defragmentation tools in Windows, they treat the SQL data file as a whole and will not defragment it granularly.
A You could back up the database and then restore it. If the space exists for a contiguous file, the database should then be written contiguously. That said, it is normally not worth the downtime to try to defrag the physical files. Typically there isn't much external fragmentation anyway. It's more helpful to regularly reindex your data to reduce the internal fragmentation as much as possible. This will maximize the effectiveness of the read-aheads and the amount of data that can be buffered.
The most important factors for efficient disk I/O are making sure the disk alignment and RAID configuration is correct, scaling your disk arrays to properly handle the I/O load, and maintaining proper layout of the Log, Data, TempDB, and backup files. If you avoid using auto-grow and auto-shrink as your primary method for sizing data files, you'll reduce the number of volume-level file fragments created. For instance, performing 10 auto-grows of 500MB each would probably add 10 new physical file fragments. In contrast, a single manual grow of 5GB will add only one.

Tip: Changing the Service Account
Did you know that when the SQL Server service login account is configured with a Windows NT® account, SQL Server sets Windows user rights and permissions on several files, folders, and registry keys? You can also set the SQL Server service account from the Services console within Administrative Tools. However, when you do this through Services, the rights and permissions are not set and you may run into serious issues due to lack of proper security settings on the aforementioned SQL Server and Windows items.
Therefore, it's strongly recommended that you use SQL Server Configuration Manager and not the Services console when changing SQL Server or the SQL Server Agent service account. However, if you've already made changes to the account using the Services console, you can still fix this problem.
Step 1: Apply full permissions on the following registry keys and its subkeys:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\90
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\<MSSQL.x>
Step 2: Set full control for the startup account for the MSSQLServer service and the SQLServerAgent service (either a local Windows NT account or a domain Windows NT account) on this NTFS folder:
Drive:\Program Files\Microsoft SQL Server\<MSSQL.1>\MSSQL
Instead of doing this manually, however, it's recommended that you use SQL Server Configuration Manager for making changes to SQL Server/Agent service accounts.

Thanks to the following Microsoft IT pros for their technical expertise: Cary Gottesman, Saleem Hakani, Trayce Jordan, Peter Kalbach, Al Noel, Uttam Parui, Amber Sitko, and Buck Woody.


 Utility Spotlight: Virtual Machine Remote Control Plus
Utility Spotlight
Virtual Machine Remote Control Plus
Matthijs ten Seldam
Download the code for this article: UtilitySpotlight2007_10.exe (10163KB)

Virtualization is a hot topic these days. Many IT professionals use hardware virtualization for learning and testing scenarios. Given this, Microsoft now offers Virtual PC 2007, a software virtualization solution that targets standalone installations with a limited number of virtual machines. Note that this new product does not replace Virtual Server
2005, which brought virtualization into the datacenter.
Microsoft® Virtual Server 2005 enables you to virtualize many server workloads, but, unlike Virtual PC, the Virtual Server interface is a Web application. Although Web-based management has its advantages, for day-to-day administration many people want a rich UI that behaves like a Windows® app.
Virtual Machine Remote Control Plus, VMRCplus for short, is an alternative UI for Virtual Server 2005. Because it is a Windows Forms application, VMRCplus offers easy management of Virtual Server and its virtual machines. Note that Virtual Server comes with a tool called VMRC that is used for remote control of virtual machines. But VMRCplus offers both configuration management of Virtual Server and remote control of virtual machines—all from within a single application.

The Virtual Machine Manager
The main window of VMRCplus, called the Virtual Machine Manager, is shown in Figure 1. In this example, the Virtual Machine Manager is connected to localhost, with five machines running, three in saved state and all others turned off. The sixth machine on the list is selected, displaying additional information including the guest operating system, CPU load, disk and network I/O statistics, and the installed additions version. Notice that the list is sorted by Status.
Figure 1 The VMRCplus Virtual Machine Manager
Figure 1 The VMRCplus Virtual Machine Manager
Through the Virtual Machine Manager, you can connect to Virtual Server hosts and display their virtual machines. You switch hosts by selecting a host's tab and you connect to a host from the dropdown list in the toolbar or by typing a host name or IP address. Upon successful connection, VMRCplus adds the name or IP address to the list.
Some Virtual Server host properties, like CPU and memory information, are displayed at the bottom of the window. A memory meter provides a quick view of the amount of available RAM left for virtual machines.

Virtual Server Management
The Virtual Machine Manager does more than simply display information; you can configure settings as well. From the View menu option, you can set the refresh interval VMRCplus uses to update the Virtual Machine list. The current refresh interval setting is displayed in the lower right-hand corner of the window, as Figure 1 shows.
The Virtual Server menu gives you access to Virtual Server configuration, including Virtual Machine Remote Control (VMRC) settings, the default location for new virtual machines, security settings, and service properties. On the VMRC settings page, you can set any of the properties, disable VMRC altogether, or revert back to the Virtual Server default settings.
The security settings page lets you specify Virtual Server permissions. By default, the local Administrators group has full control of Virtual Server. You can specify permissions for groups or users. If you are not a member of the local Administrators group, you must first add a group or your account and set appropriate permissions. You do not need to actually run VMRCplus as an administrator though; normal user privileges are sufficient.
The properties of the Virtual Server host, including the physical properties, are shown in Figure 2. Notice that this host supports hardware-assisted virtualization, as can be seen from the processor features.
Figure 2 Virtual Server properties
Figure 2 Virtual Server properties

Virtual Disks and Networks Management
VMRCplus includes other managers targeted at specific Virtual Server capabilities. The Virtual Disks Manager has a tabbed interface corresponding to the types of virtual disks you can create, including hard disks (dynamic or fixed), floppy disks, and differencing disks. You are also able to inspect your virtual disks.
The Virtual Networks Manager lets you manage virtual networks. You can add a virtual network from a virtual network configuration file (VNC) or create a new one. When creating the virtual network, you can attach a host network adapter or designate the network as guests only. From the same window you can enable a virtual DHCP server and open its settings.

Virtual Machine Management
The previous managers all focused on managing Virtual Server. What does VMRCplus offer for creating and configuring virtual machines?
Creating virtual machines is quite easy: simply select Virtual machine | Create from the menu. The dialog that results offers several defaults that are equivalent to those of Virtual Server, but there are differences beneath the interface. For example, VMRCplus always creates two virtual hard disks and attaches them to SCSI. VMRCplus also lets you create multiple virtual machines at once. Simply select "Create multiple virtual machines" and select the number you want. VMRCplus adds a hyphen and sequence number to the virtual machine name you provide.
A truly powerful feature is the ability to create multiple virtual machines from a parent virtual machine. Suppose you have created a virtual machine with the latest additions, service pack, hotfixes, and tools, and you'd like to use it as a base for other virtual machines. Assuming you've run sysprep as a final stage and shut down the parent, you can now create multiple virtual machines and they will inherit the properties of your parent.
Within seconds, you can create many virtual machines based on this parent, and you can quickly start these new virtual machines from the same reference. When you are done—with testing for example—simply delete them. And that's easy too: VMRCplus supports multi-select on the virtual machine list as well as sorting the columns. So you can easily multi-select and remove the virtual machines (which simply unregisters them), or you can delete the virtual machines (which deletes the files as well).
There are other multi-select commands you can use on virtual machines, such as multi-select and start, multi-select and open the console, or multi-select and save state. You can accomplish all of this either through the UI or through shortcut keys.
Configuring a virtual machine is also easy using the hardware settings page. You can set the amount of memory, add SCSI controllers, attach disks or images (yes, with drag and drop), and enable and configure network adapters. The advanced properties window enables you to set the more advanced configuration items. For example, you can run virtual machines under a specific user account so you can set a machine to start automatically at Virtual Server startup.

Remote Control
Finally, VMRCplus also offers the Console Manager, which contains the remote control sessions (VMRC sessions) with virtual machines. You open a console by selecting a running virtual machine (or several) and clicking the virtual machine console button on the properties toolbar.
The Console Manager window, shown in Figure 3, supports a tabbed ordering of remote sessions. You can switch between virtual machines by selecting the corresponding tab. There's also a toolbar containing the most frequently used functions.
Figure 3 Console Manager
Figure 3 Console Manager
As you can see, the Console Manager has context menus that let you attach a virtual floppy image or ISO image, mount host drives, and install current additions. There's also a status bar that shows the status of CD/DVD and floppy media, the current additions version in the virtual machine, the currently active host key and the dynamic screen resizing setting. VMRCplus lets you set the host key. You can also access the virtual machine hardware settings and advanced properties.
VMRCplus offers some other handy features. For example, you can take a screen shot of virtual machine settings and save it as a .jpg file, which can be quite useful in troubleshooting scenarios. You can also "Send Text to Virtual Machine," meaning you can send text to the virtual machine as keystrokes. This isn't designed as a clipboard function, but it's very handy in situations where, for example, you need to copy and paste a product key from the host to the virtual machine.
VMRCplus was initially authored by Paul Despe as a developer tool that handled some basic functions but did not cover all features of Virtual Server. It was used for testing purposes only and had a number of issues. Approximately a year ago, I picked up the code and started to rewrite, enhance, and extend its functionality. Since then it has become enormously popular with Virtual Server users within Microsoft. Note, however, that VMRCplus is not a supported Microsoft application. It is provided as-is and subject to the included end-user license agreement.
VMRCplus has been developed for Virtual Server 2005 Service Pack 1. Although previous versions should be compatible, it has only been tested with Service Pack 1. As a result, users should update their Virtual Server hosts to Service Pack 1 before using it. VMRCplus is available for download from the TechNet Magazine Web site at technetmagazine.com/code07.aspx.

Matthijs ten Seldam is a Principal Consultant for Microsoft Consulting Services. He is focused in the areas of security, deployment, and automation, and he is a subject-matter expert in the area of virtualization.


 The Business of IT: Calculating the Value of IT Infrastructure in Government
The Business of IT
Calculating the Value of IT Infrastructure in Government
Acer Maamoun

Demonstrating the benefits of an investment in IT infrastructure—both hardware and software—can be difficult in government organizations. As in other large organizations, managers must make a business case for any additions or improvements, which will be prioritized against all other requests for funding.
Since there are many different ways to frame business cases, IT managers must choose the one that is most suitable for their specific organizational culture. This is as true among government organizations as it is for businesses. Aligning IT with other business priorities is essential. In the government sector, IT organizations have found success in articulating business cases using the concept of public value in the approach to building portfolios of projects and initiatives.
Constructing a business case that demonstrates how infrastructure improvement leads to better productivity or improved service is a difficult process. Let's take a look at some solutions for defining and measuring the value behind IT infrastructure investments that are being adopted by real government IT organizations.

Justifying Investments
A common complaint within IT organizations is that they receive too many requests from too many different business areas—and each request, of course, has a high degree of urgency. New initiatives tend to map to individual departmental regulatory requirements or mission priorities. These initiatives may appear to be a random assortment, but they are often closely aligned to the public value sought by a given department.
On the other hand, operating costs that are apportioned by how much is used to support individual department operations can hide the cost (and value) of enterprise-wide functions or shared services. In this environment, the value of investments in new infrastructure or improvements to existing infrastructure that have an enterprise-wide impact or that indirectly map to the objectives of individual departments can be extremely difficult to evaluate properly.
Consequently, the overall IT organization often finds it difficult to support an enterprise view of IT as described by the GSA Federal Enterprise Architecture Framework (whitehouse.gov/omb/egov/a-1-fea.html). In this environment, most efforts to employ a shared view of enterprise architecture, a common business case development model, and a portfolio management process to place IT investment requests and spending on a level playing field are difficult to realize. A more effective model is to support a portfolio of IT activities that effectively maximize public value benefits. This requires that common public value criteria be selected and enforced throughout the processes of business case development, portfolio management, benefits realization, and assessment.

Measuring Value
There are several real issues with enterprise-wide IT investments. First, they have a more indirect and longer-lasting impact in terms of the public value benefits. These benefits are accrued, often unevenly, by different parts of the organization in ways that are not always easy to identify up front. In addition, the associated risks and costs are more readily perceived than the less readily visible and quantifiable public value.
A business case should be based on how an infrastructure investment enables future investments as well as current activities. There are different ways in which to calculate this value, each with its own advantages and drawbacks, and each with a different degree of fitness for a particular organization's culture and maturity.
Mathematical These methods are based on a mathematical modeling of potential benefits. One effective method, called real options, is based on financial theory. It embeds the consideration of future change and flexibility by explicitly considering the right to take an action (that is, the real option) at a given cost for a given period of time. Options include deferring, expanding, staging or abandoning investments; changing inputs or outputs; and developing pilots.
Real options are particularly appropriate when an investment decision is influenced by future events. In the case of IT infrastructure, such events may include the following: regulatory requirements, business demand, or even a critical event such as a major operational failure.
Simulation Uncertainty analysis is based on Monte Carlo simulation, which uses automated tools to calculate the results of alternative scenarios based on the selection of variable values and repetition of the process. The simulation requires the identification of a range of values (such as the number of affected workstations or electronic channel uptake) and the probability distribution for each variable. The outcome of a simulation is a range of possible results and permits the identification of the mean value of benefits as well as of costs and risks. Carefully choosing value and risk variables to reflect the possible impact of infrastructure investments enables placing the return of such an investment in perspective.
Scenarios When mathematical or simulation models are impractical or do not fit the organization culture and skills, scenario-based techniques can be used instead. Taking into consideration the agency business or mission, political priorities, and environmental trends (including technology), the organization can develop a limited number of plausible scenarios for the next three to five years. Each scenario should focus exclusively on demonstrating the benefits of the infrastructure investment.
Unlike longer-term planning exercises, scenarios do not need to be mutually exclusive and can be assigned probabilities. Scenarios can take into account future elections, outcomes of political debates, and possible IT market events such as acquisitions or major shifts in vendor strategies.
Worst-Case Scenario This variation of the scenario-based approach is applied when the IT organization believes an investment is urgently needed to meet forthcoming business or mission challenges, or to simply maintain current service levels. When infrastructure problems are unattended, they worsen and the effort needed to fix them increases. Whoever pushes for changes will likely face resistance and is unlikely to receive credit for their efforts. Such a situation can be partially remedied with a strong emphasis on metrics and measurement relevant to some of the affected performance—such as response time to requests and time to repair. This information must be made widely available so people are aware of possible changes in performance.
Worst-case scenarios intentionally stress the consequences of not making the investment by focusing on negative benefits and increased risks. This approach should be used with great care and only in truly exceptional circumstances. Using this approach routinely could create the impression that the IT organization is trying to overthrow established investment management processes, which would lead to conflicts and misalignment that those very processes have helped resolve.

Choosing a Planning Methodology
Figure 1 summarizes the pros and cons of each method and when to consider them. Once a method is selected, it should be used consistently throughout the organization to ensure comparability.
Figure 1 Business case methodologies (Close Figure 1)

Method Advantages Drawbacks Consider When
Real Options (Mathematical)
  • Rigorous
  • Solid theoretical basis
  • Complex
  • Not adequate when intangible values prevail
  • Value and future value are mostly quantifiable
  • There is a solid investment appraisal culture
  • Skills are readily available
Simulation
  • Allows exploration of alternative values
  • Easy way to build alternatives
  • Averages may make little sense with intangibles
  • Range selection in variable is critical
  • Tangible benefits are prevalent
  • Tool support is available
Scenario
  • Allows any combination of tangible and intangible benefits
  • Allows exploration of very different scenarios, combining different techniques
  • Requires strong discipline to converge toward comparable scenarios
  • Time horizon may vary
  • Intangible values prevail or actual nature of benefits is unclear
  • Organization culture is hostile to other methods
Worst-Case-Scenario
  • Effectively visualizes the consequences of not doing an investment
  • Simple to build
  • Does not help prioritize
  • Can be misused
  • Investment is absolutely critical and overdue
The effectiveness of any of these approaches depends on a variety of factors. Answering the following questions is a good starting point. How strongly is the approach integrated or compatible with mandatory processes (such as budget and capital planning)? Is the chosen business case method easy to use and effective in communicating value to stakeholders?
Organizations need to balance between flexibility in choosing the value indicators that fit the particular agency and guidance to ensure comparability across agencies. In addition, the ability to use a given valuation metric throughout the investment lifecycle and not only at business case time is vitally important. Likewise, the plan should have utility across different types of investments.
As a consequence, a value approach should be selected and developed in conjunction with the participants in the organization's governance structure. It is essential that the selected framework provide an effective way to communicate value to internal and external stakeholders. This is as true for enterprise executives as for elected officials or agency management.
Additional Resources
  • Webcast: IT as a Strategic Business Asset
  • IT Manager Connections Series

Acer Maamoun is an Enterprise Strategy Consultant working for Microsoft. He has more than 18 years of experience in IT strategy and business alignment.


 The Cable Guy: The Authenticated Internet Protocol
The Cable Guy
The Authenticated Internet Protocol
Joseph Davies
Portions of this column discuss a prerelease version of Windows Server 2008. Those details are subject to change.

Windows Vista and Windows Server 2008 (now in beta testing) both support Authenticated Internet Protocol (AuthIP), an enhanced version of the Internet Key Exchange (IKE) protocol. Both AuthIP and IKE are protocols used to determine keying material and negotiate security parameters for communications protected using
Internet Protocol security (IPsec). AuthIP provides simplified IPsec policy configuration and maintenance in many configurations and offers additional flexibility for IPsec peer authentication. This article describes the protocol details of AuthIP and coexistence behaviors between IPsec peers that support either both AuthIP and IKE or only IKE.

IKE Overview
Negotiation Examples
To demonstrate the coexistence behavior, let's examine what happens between Windows Vista and Windows XP-based IPsec peers when attempting to negotiate IPsec protection.

Example 1: Two Windows Vista-based IPsec peers in request mode
In this example, a Windows Vista-based IPsec peer (Peer 1) is the initiator, and the responder is running Windows Vista (Peer 2). Both peers are configured with request mode for both inbound and outbound communications. In request mode, an IPsec peer requests IPsec protection but does not require it. The messages exchanged are the following:
  1. Peer 1 sends a clear text TCP synchronize (SYN) segment, initiating a TCP connection with Peer 2.
  2. Peer 2 sends a TCP SYN-Acknowledgment (ACK) segment.
  3. Peer 1 sends a TCP ACK segment.
  4. Peer 1 sends an AuthIP-based ISAKMP message, initiating AuthIP main-mode negotiation.
  5. Peer 1 sends an IKE-based ISAKMP message, initiating IKE main-mode negotiation.
  6. Peer 2 responds with an AuthIP-based ISAKMP message, continuing AuthIP main-mode negotiation.
  7. Peers 1 and 2 complete AuthIP main-mode, quick-mode, and extended-mode (optional) negotiation.
  8. Subsequent segments sent over the TCP connection are protected with IPsec.
The exact order of Messages 1 through 5 depends on network latency. The examples described in this article are for a very low latency network, in which the TCP handshake (Messages 1 through 3) completes before Peer 1 can send the initial ISAKMP messages (Messages 4 and 5). On a higher-latency network, you would see Peer 1 send the clear text TCP SYN segment, the AuthIP-based ISAKMP message, and then the IKE-based ISAKMP message as the first three messages of the message exchange.

Example 2: Windows Vista-based IPsec peer in request mode and a Windows XP-based IPsec peer in request mode
In this example, a Windows Vista-based IPsec peer (Peer 1) is the initiator and the responder is running Windows XP (Peer 2). Both peers are configured with request mode for both inbound and outbound communications. The messages exchanged are the following:
  1. Peer 1 sends a clear text TCP SYN segment, initiating a TCP connection with Peer 2.
  2. Peer 2 sends a TCP SYN-ACK segment.
  3. Peer 1 sends a TCP ACK segment.
  4. Peer 1 sends an AuthIP-based ISAKMP message, initiating AuthIP main-mode negotiation.
  5. Peer 1 sends an IKE-based ISAKMP message, initiating IKE main-mode negotiation.
  6. Peer 2 responds with an IKE-based ISAKMP message, continuing IKE main-mode negotiation.
  7. Peers 1 and 2 complete IKE main-mode and quick-mode negotiation.
  8. Subsequent segments sent over the TCP connection are protected with IPsec.

Example 3: Windows Vista-based IPsec peer in request mode and a Windows XP-based IPsec peer in require mode
In this example, a Windows Vista-based IPsec peer (Peer 1) is the initiator and the responder is running Windows XP (Peer 2). Peer 1 is configured with request mode for outbound communications and Peer 2 is configured with require mode for inbound communications. In require mode, an IPsec peer requires IPsec protection and silently discards unprotected packets. The messages exchanged are the following:
  1. Peer 1 sends a clear-text TCP SYN segment initiating a TCP connection, which Peer 2 silently discards.
  2. Peer 1 sends an AuthIP-based ISAKMP message initiating AuthIP main-mode negotiation, which Peer 2 silently discards.
  3. Peer 1 sends an IKE-based ISAKMP message, initiating IKE main-mode negotiation.
  4. Peer 2 responds with an IKE-based ISAKMP message, continuing IKE main-mode negotiation.
  5. Peer 1 and Peer 2 complete IKE main-mode and quick-mode negotiation.
  6. Peer 1 retransmits the TCP SYN segment (protected with IPsec).
  7. Peer 2 sends the TCP SYN-ACK segment (protected with IPsec).
  8. Peer 1 sends the TCP ACK segment (protected with IPsec).

IKE is an Internet standard, defined in RFC 2409 (ietf.org/rfc/rfc2409.txt), that defines a mechanism to establish IPsec security associations (SAs). An SA is a combination of a mutually agreeable policy and keys that define the security services and mechanisms that help protect communication between IPsec peers. Specifically, IKE combines the Internet Security Association and Key Management Protocol (ISAKMP) in RFC 2408 and the Oakley Key Determination Protocol (RFC 2412). IKE uses the Oakley Key Determination protocol to generate secret key material for protected communications.
IKE uses ISAKMP to negotiate SAs. ISAKMP includes facilities to identify and authenticate peers, manage SAs, and exchange key material. ISAKMP is a framework for negotiating protected communications that is independent of specific key exchange protocols, encryption and integrity algorithms, and authentication methods.
IPsec peers use IKE to perform main-mode and quick-mode negotiation. Main-mode negotiation creates the ISAKMP SA, which protects security negotiations. During main-mode negotiation, the initiator and the responder exchange a series of ISAKMP messages in order to negotiate the encryption and hashing algorithms for the ISAKMP SA, exchange key determination material, and identify and authenticate each other.
In the quick-mode negotiation phase, the two IPsec SAs (one SA for inbound data and another SA for outbound data) are created, which protects the data sent between two IPsec peers. During quick-mode negotiation, the initiator and the responder exchange a series of encrypted ISAKMP messages to negotiate the encryption and hashing algorithms for both the inbound and outbound IPsec SAs.
For more information about IKE-based main-mode and quick-mode negotiation, see "IKE Negotiation for IPsec Security Associations" at microsoft.com/technet/community/columns/cableguy/cg0602.mspx.

AuthIP Overview
AuthIP is an enhanced version of IKE that offers additional flexibility with support for user-based authentication, authentication with multiple credentials, improved authentication method negotiation, and asymmetric authentication. Like IKE, AuthIP supports main-mode and quick-mode negotiation. AuthIP also supports extended mode, a part of IPsec peer negotiation during which a second round of authentication can be performed. Extended mode, which is optional, can be used for multiple authentications. For example, with extended mode you can perform separate computer-based and user-based authentications.
Multiple authentication support is part of IPsec enforcement for the Network Access Protection (NAP) platform, in which IPsec peers authenticate with computer credentials and also use a health certificate to prove that they are compliant with system health requirements. For more information about Network Access Protection, see microsoft.com/nap. And for further information about the features of AuthIP, see "AuthIP in Windows Vista" at microsoft.com/technet/community/columns/cableguy/cg0806.mspx.

AuthIP Messages
Both IKE and AuthIP use ISAKMP as their key exchange and SA negotiation protocol. ISAKMP messages are sent via the User Datagram Protocol (UDP) and consist of an ISAKMP header and one or more ISAKMP payloads. The ISAKMP header contains information about the message, including the message type. Figure 1 shows the format of the ISAKMP header.
Figure 1 Format of the ISAKMP header
Figure 1 Format of the ISAKMP header
In the ISAKMP header, the Initiator Cookie and the Responder Cookie are set to a nonzero random number chosen by the IPsec peers. The Next Payload field indicates the first payload in the ISAKMP message, just beyond the ISAKMP header. RFC 2408 lists defined values of the Next Payload field. Payload types 128-255 are reserved for private use. The Major Version and Minor Version fields indicate the versions of ISAKMP on the IPsec peer sending the ISAKMP message. For IKE and AuthIP, the major version is 1 and the minor version is 0.
The Exchange Type field indicates the type of ISAKMP exchange being performed. The type of exchange dictates the structure and order of ISAKMP payloads. RFC 2408 defines the values of the Exchange Type field. Exchange types 128-255 are reserved for private use.
The Flags field contains three flags defined in RFC 2408. The low-order bit (bit 0) is the Encryption bit, which indicates that the ISAKMP payloads are encrypted (when set to 1) or not encrypted (when set to 0). Encryption is done using the algorithm negotiated for the ISAKMP SA, which is identified by the combination of the Initiator Cookie and Responder Cookie fields. The next low-order bit (Bit 1) is the Commit bit, which indicates that the key exchange is synchronized (when set to 1) or not synchronized (when set to 0). The Commit bit is used to ensure that the SA completes its negotiation before encrypted data is sent. The next low-order bit (Bit 2) is the Authentication Only bit, which is used to indicate that the message either contains (when set to 1) or does not contain (when set to 0) the entire Notify payload of the informational exchange type and it has been authenticated but not encrypted.
The Message ID field contains a unique identifier for the message. The Message ID is used to prevent collisions when both IPsec peers attempt to simultaneously establish an IPsec SA. The Length field indicates the length of the entire ISAKMP message.
For IKE, the ISAKMP message contains a series of ISAKMP payloads. The first payload is indicated by the Next Payload field of the ISAKMP header. Each ISAKMP payload has its own Next Payload field to indicate the next payload. The Next Payload field in the last payload is set to 0. Figure 2 shows the format of IKE messages.
Figure 2 Format of IKE messages
Figure 2 Format of IKE messages
AuthIP uses ISAKMP messages with the exchange types 243 (Main Mode), 244 (Quick Mode), 245 (Extended Mode), and 246 (Notify) in the Exchange Type field in the ISAKMP header. An important difference in AuthIP-based ISAKMP messages is that they only contain one ISAKMP payload: either the Crypto payload or the Notify payload. The Crypto payload contains the embedded payloads used for the main mode, quick mode, or extended mode negotiation. The Crypto payload can contain a set of plain text or encrypted payloads, depending on the Encryption bit in the Flags field of the ISAKMP header. Figure 3 shows the format of AuthIP messages containing the Crypto payload.
Figure 3 Format of AuthIP messages containing the Crypto payload
Figure 3 Format of AuthIP messages containing the Crypto payload

AuthIP and IKE Coexistence
Windows Vista® and Windows Server® 2008 support both IKE and AuthIP. Windows® XP and Windows Server 2003, however, only support IKE. An initiating IPsec peer that supports both AuthIP and IKE and has a connection security policy that uses both AuthIP and IKE must determine whether the responding IPsec peer supports AuthIP or IKE. It must then use the most appropriate protocol for negotiating IPsec protection, preferring the use of AuthIP over IKE.
To determine the negotiation protocol of the responding IPsec peer, an initiating IPsec peer that uses both AuthIP and IKE sends the following messages at the same time:
  • Message 1—an AuthIP message initiating main-mode negotiation
  • Message 2—an IKE message initiating main-mode negotiation
If the responding node supports AuthIP, it must respond to Message 1 with an AuthIP message continuing main-mode negotiation and silently discard Message 2. A responding node that does not support AuthIP silently discards Message 1 because the Exchange Type field it contains a value it doesn't support, and responds to Message 2.
To prevent IKE-based negotiation between two IPsec peers running Windows Vista or Windows Server 2008 when Message 1 is dropped from the network or arrives after Message 2, IPsec peers running Windows Vista or Windows Server 2008 send Message 2 with an AuthIP Supported vendor ID ISAKMP payload. The AuthIP Supported vendor ID payload indicates that the IPsec peer supports AuthIP.
Therefore, if an IPsec peer running Windows Vista or Windows Server 2008 receives Message 2 with the AuthIP-supported vendor ID payload, it waits for the initiating IPsec peer to retransmit Messages 1 and 2, and it responds to Message 1.
The initiating IPsec peer keeps retransmitting both Messages 1 and 2 until it receives a response or times out. When the initiator receives a response, it determines the capability of the responder from the ISAKMP header of the response. If the Exchange Type is set to 243 (the exchange type for AuthIP-based main-mode negotiation), the responder is AuthIP-capable. If the Exchange Type is set to 2 (the exchange type for Identity Protection and IKE-based main mode negotiation), the responder is IKE-capable.
Based on the response, the initiator responds with either the next AuthIP message for AuthIP main mode negotiation or the next IKE message for IKE main mode negotiation. The IPsec peers must use the same protocol that was used to negotiate the ISAKMP SA for the lifetime of the SA.

Joseph Davies is a technical writer with Microsoft and is the author of the monthly online TechNet Cable Guy column at Microsoft.com/technet/community/columns/cableguy.


 Windows PowerShell: Working Without a Script
Windows PowerShell
Working Without a Script
Don Jones

Whenever I'm talking to folks about Windows PowerShell—whether at a conference, on the public Windows PowerShell newsgroup, or even on my site—I always meet administrators who tell me they're "going to wait" to learn Windows PowerShell. Why? The most common reason they give is that they
"just don't have time to learn how to script right now."
It's a common sentiment. We administrators are a busy group, and we really don't have much free time to learn a new technology. But we have to keep moving forward, learning new software such as Windows Vista®, Windows Server® 2008, and Exchange Server 2007, so we can stay on top of the technologies that are critical to our jobs. And Windows PowerShell™ is a big part of these technologies.
Another reason I often hear administrators cite for putting off learning to script in Windows PowerShell is that they have a "fear of becoming a programmer," as one fellow administrator put it. Well, that is something I can help you with. You may be surprised to learn that you can do some truly amazing things using Windows PowerShell, all without writing a lick of code.

Getting Ready to Pipeline
I've written before about the extremely flexible and powerful object-oriented pipeline that Windows PowerShell uses. (Even though the words "object-oriented" might make you suspect that I'm heading for programmer-land, stick with me.) One of my favorite things about the pipeline is how it can be used interactively. It can give you instant results and lets you easily refine those results so you can get exactly what you want. For example, suppose I want to get a free disk space inventory for several remote computers. Now say I have those computers' names listed in a text file called C:\Computers.txt. It's just a plain text file with one computer name per line—nothing as complicated as a database.
To start, I'll see if I can retrieve this information from my local computer. After all, if I can figure it out on one computer, performing the task on multiple computers should be pretty easy. And remember, there is to be no scripting involved here! I'm going to do everything interactively in the shell, getting results as soon as I press the Enter key.
Here's a hint about this sort of task. Any time you want to inventory management information from remote computers, you will probably use Windows® Management Instrumentation (WMI). Now assume that, for the purposes of this example, I know nothing more than that I need to use WMI. So I hop on Live Search and enter "windows wmi free disk space" as my search term. Several results come up that list the phrase "Win32_LogicalDisk" in their excerpts. That looks to me like a WMI class, and it may be the one I'm after. I don't even bother to click on any of those search results. Instead, I enter "Win32_LogicalDisk" as a new search term and the top result is for a Win32_LogicalDisk documentation page on the MSDN® site (msdn2.microsoft.com/aa394173.aspx). So I hop on over to that page and see that one of the properties for this class is FreeSpace, which sounds very promising.

Finding a Cmdlet
For the next step, I need to find a cmdlet that will retrieve a WMI class's properties for me. One of the notable aspects of Windows PowerShell is that it has fantastic built-in self-discovery features—that is, the shell can help you find out about its own capabilities. So I enter Help *wmi* to find out what the shell knows about working with WMI. The results, shown in Figure 1, are the Gwmi alias and the cmdlet it points to, Get-WMIObject. Basically, there are two ways to access the same functionality, which means I don't have a very tough decision about which one to use. Since they do the same thing and Gwmi is shorter to type, I'll use that:
Figure 1 What the shell knows about working with WMI
Figure 1 What the shell knows about working with WMI
PS C:\> gwmi win32_logicaldisk
Note that Windows PowerShell isn't case-sensitive—it knows we administrators don't have time for fussy details like pressing the Shift key.
The results of this command, shown in Figure 2, include all five of my local drives and their FreeSpace properties, which appear to be listed as bytes. Also listed is a DriveType property for each one of these drives, and I'm seeing values of 2, 3, and 5 on my computer.
Figure 2 All of my local drives and their FreeSpace properties
Figure 2 All of my local drives and their FreeSpace properties

Cmdlet of the Month
You may have used Get-Content to read the contents of a text file. It treats each line of the file as a [string] object, passing those objects into the pipeline for other cmdlets to work with. But Get-Content has a number of options that make it more flexible: the –readCount parameter, for example, allows you to specify how many [string] objects are passed into the pipeline at once (the default is to send them all), while the –totalCount parameter controls the total number of lines that are read from the file. Both of these parameters are useful for working with really large files where you might not want to process the entire file at once for performance reasons.
Here's another you'll find handy. The –encoding parameter allows a large variety of file encoding types to be read correctly, including Unicode, ASCII, UTF7, UTF8, and many others. To see a complete list of supported encoding types, run help gc (gc is an alias for Get-Content).

Refining the Data
I have two problems at this point. First, I don't care about all those other properties, I just want the FreeSpace. Second, I don't want to know about free space for optical drives, removable drives, or network drives; I am only concerned with local hard disks. Maybe that DriveType property can help me distinguish between them. I flip back to my Web browser and see that the documentation contains a table explaining what the different DriveType values represent. I only want the ones with a DriveType of 3, which indicates that the drive is a local hard disk. Having never done this before, I decide to see if the Gwmi command can do some kind of filtering for me. Running Help Gwmi –full pulls up details about how the command works, including several examples. I find a parameter that looks useful, –filter, and so decide to try running this:
gwmi win32_logicaldisk -filter "drivetype = 3"
It worked! My drive list now contains only local fixed disks. That's the second problem solved; now I need to solve the first one—getting rid of all the properties I don't care about.
I run Get-Command to list all the Windows PowerShell cmdlets and I eventually come upon Select-Object. The description says this will "Select specified properties of an object or set of objects." So I try this:
gwmi win32_logicaldisk -filter "drivetype = 3" | select freespace
By piping the results of Gwmi to Select (the alias for Select-Object), I can get just the property I want. Oops. These results aren't so good, as all I have is a list of numbers—I no longer know which drive goes with which free space listing. So I jump back to the documentation and see that the DeviceID property contains the drive letter. I quickly revise my command and try again:
gwmi win32_logicaldisk -filter "drivetype = 3" | select deviceid,freespace
Perfect! And since only two properties are being listed, the shell is able to put the information into a nice-looking table. (Next month I'll explain when Windows PowerShell chooses to use lists and tables.)

Calculating
I've retrieved information about free space, but it's in bytes and that isn't really as useful as megabytes or gigabytes would be. Perhaps I don't want the FreeSpace property itself, but rather a value calculated from the FreeSpace property. The ForEach-Object cmdlet (or one of its many aliases, such as %) can help with that. This cmdlet lets me use a special variable, $_, to refer to the current logical disk and it lets me access each disk's properties individually and do some math. Here you can see my revised command:
gwmi win32_logicaldisk -filter "drivetype = 3" | % { $_.deviceid; $_.freespace/1GB }
All I've done here is removed Select and replace it with the alias for ForEach-Object (%). That cmdlet just wants me to tell it what to do with each Win32_LogicalDisk it gets and I've told it to get the DeviceID property and to divide the FreeSpace property by a gigabyte—Windows PowerShell "knows" what a KB, MB, and GB are—so that my output is given in gigabytes.

Many Computers
Now that I have this working for one computer, it's time to get it working for several systems. I know how to get the content of a text file. This is done in the same way as back in the MS-DOS® days—using the Type command, which in Windows PowerShell, it turns out, is an alias for Get-Content:
Type c:\computers.txt
What I want to do is take each of these computers, and for each one, perform the WMI command I already worked out. The phrase "for each one" should remind you of a cmdlet—specifically, ForEach-Object. So let's just cut to the big reveal, as they say on those home-design television shows, and look at my final command:
type c:\computers.txt | % { $_;
gwmi –computername $_ win32_logicaldisk -filter "drivetype=3" | % { $_.deviceid; $_.freespace/1GB} }
Scary, isn't it? Thanks to my exclusive use of aliases rather than cmdlet names, it's tough to read. However, it's pretty easy to decipher if you walk through it one bit at a time:
  • I start by "typing" the contents of my text file.
  • I pipe those contents to ForEach-Object.
  • ForEach-Object outputs the current item, using the $_ variable. (That's the current computer name.)
  • ForEach-Object then executes my WMI command, which has another ForEach-Object call.
Expanding those alias names into cmdlet names may help. So here is the same command, but this time I've spelled out the cmdlets and broken the command apart into individual lines so that you are able to examine each section more easily:
Get-Content C:\Computers.txt |
ForEach-Object {
 $_; Get-WMIObject –computername $_
Win32_LogicalDisk -filter "DriveType=3" |
 ForEach-Object {
 $_.DeviceID; $_.FreeSpace/1GB
 }
}
The results—which aren't attractive, but are functional—are shown in Figure 3.
Figure 3 The final results
Figure 3 The final results

No Scripting Required
The point of this walk-through is to demonstrate that you can use Windows PowerShell without doing any scripting. And, with just a few examples like this one, plus a little exploring within Windows PowerShell itself, you can actually find some of the bits you need to accomplish whatever task you've set out to do.
The bottom line is that Windows PowerShell is here to stay. Why not start learning to use it now? You might even find yourself wondering how you ever lived without it—and you certainly shouldn't let the word scripting keep you from diving right in.

Don Jones is the Lead Scripting Guru for SAPIEN Technologies and an instructor for ScriptingTraining.com. Reach Don through his Web site, ScriptingAnswers.com.


 Hey, Scripting Guy!: Let There Be Silverlight
Hey, Scripting Guy!
Let There Be Silverlight
The Microsoft Scripting Guys
Download the code for this article: HeyScriptingGuy2007_10.exe (156KB)

Human history has long been marked by the clash of diametrically opposed forces: the Romans versus the barbarian hordes; the Hatfields versus the McCoys; the Scripting Guys versus those who believe a column about system administration scripting should actually talk about system administration scripting rather than, say, the historical clash of diametrically opposed forces. [Or, say, meat hooks ... —Ed.]
The computer world is not immune to these clashes between diametrically opposed forces. In particular, we're reminded of the clash between developers and system administrators. As Karl Marx said upon surveying the situation, "Either this man is dead or my watch has stopped."
Wait, sorry—that was actually Groucho Marx. Karl Marx said, "Art is always and everywhere the secret confession, and at the same time the immortal movement of its time." To tell you the truth, we have no idea what that means. Besides, Groucho Marx was quite a bit funnier than Karl Marx, even if Groucho's quips aren't particularly relevant to the clash between developers and system administrators.
Nevertheless, we're pretty sure that it was Karl Marx who used to make the Scripting Guys recite the automation pledge of allegiance every morning before work: "system administrators are not developers." And that was definitely true: system administrators were not developers. Developers were people who wrote computer programs, while system administrators were people who ... well, to tell you the truth, we're not sure anyone at Microsoft really knew what system administrators did. But whatever it was, it sure wasn't writing computer programs.
Of course, Karl Marx also wrote the song "The Times, They Are A-Changing." (Or at least he told us that he wrote it.)
Thanks in part to the TechNet Script Center (microsoft.com/technet/scriptcenter), system administrators gradually discovered VBScript and began writing their own utilities, utilities that, while not as complete and comprehensive as the typical computer program, still perform useful management tasks. In writing these utilities, system administrators have had to call upon skills very similar to those used by developers.
And that's not even the half of it. The advent of Windows PowerShell™ (microsoft.com/technet/scriptcenter/hubs/msh.mspx) means that the Microsoft® .NET Framework is no longer off limits to system administrators and script writers. Windows Vista® introduced the world to gadgets (microsoft.com/technet/scriptcenter/topics/vista/gadgets-pt1.mspx), putting system administrators—or at least system administrators who write scripts—on equal footing with developers. And now—drum roll, please—there's Microsoft Silverlight, the "cross-browser, cross-platform plug-in for delivering the next generation of .NET Framework–based media experiences and rich interactive applications for the Web."
Sounds impressive, doesn't it? But, you ask, what does that have to do with system administration scripters such as yourself? To be honest, we're not entirely sure. However, Silverlight does make it very easy to create very cool user interfaces and multimedia presentations. And while the marketing material invariably refers to Silverlight as a way to create Web applications, here's a little-known fact: Silverlight seems to work equally well in HTML Applications, or HTAs (microsoft.com/technet/scriptcenter/hubs/htas.mspx), a medium much-loved by script writers. So it just might be worth your while to give Silverlight a try.

Before We Go Any Further ...
We should point out that we aren't going to discuss Silverlight's many capabilities and potential uses today; our primary goal is to show you enough code snippets to get you playing around with the technology. If you'd like to install Silverlight (it's free, by the way), go to the Silverlight home page at microsoft.com/silverlight. You can also find the SDKs, sample projects, and lots of other useful stuff at silverlight.net/GetStarted.
For now, we'll assume that you have already installed Silverlight. In addition, we'll assume that you've gone to the Code Downloads section of the TechNet Magazine Web site (technetmagazine.com/code07.aspx) and downloaded the files described in Figure 1.
Figure 1 Silverlight project components (Close Figure 1)

File | Description
CreateSilverlight.js | This is a JScript script (in its initial release, Silverlight only supports JScript scripting) used to specify the initial Silverlight startup settings, including the XAML file used to configure user interface and graphic objects.
SampleProject.js | This is just a blank file where you can put JScript functions.
Silverlight.js | This file is used to initialize the Silverlight control.
SampleProject.html | This is where all the fun takes place. SampleProject.html is simply an HTML file that includes code for reading in the three .js files. It also includes code for instantiating the Silverlight control.
SampleProject.xaml | What's this for? To find out, you'll have to go back to the main text of this column.
You don't necessarily need SampleProject.js, at least not to follow along with this month's column. Silverlight allows you to attach scripts to objects and events in your project; these scripts can either be hardcoded into your HTML file or (as in this case) read in from an outside "include" file like this one. Also, we're going to turn SampleProject.html into an HTA: simply rename it SampleProject.hta. That was easy, right?
Make sure you save all these files into one folder. And then get ready to have some fun.

The XAML File
As you might have guessed, all the excitement in this month's Hey, Scripting Guy! revolves around the XAML file. (Incidentally, XAML is pronounced "zamel," rhyming with camel. Or mammal. Or even enamel.) XAML is short for Extensible Application Markup Language. Among other things, XAML provides a way to create user interfaces using an XML-like language. Does this make it easy for someone—particularly a system administrator—to create user interfaces? You're about to find out.
To begin this quest, start up Notepad and open the file SampleProject.xaml. If you see any existing code, replace it with the following:
<Canvas
 xmlns="http://schemas.microsoft.com/
    client/2007"
 xmlns:x="http://schemas.microsoft.com/
    winfx/2006/xaml">
</Canvas>
Note. SampleProject.hta is already preconfigured to load in information from SampleProject.xaml. To use a different XAML file, you need to edit the file CreateSilverlight.js.
After pasting in the code, save the file and then double-click SampleProject.hta. Guess what you'll see? That's right: a big, dumb, gray box.
But wait, don't go! That's exactly what you should see.
There's not much to see simply because we haven't done anything yet: all we've done so far is add a Canvas to our XAML file. But that's an important step. After all, if you want to try your hand at oil painting the first thing you need to do is find a canvas to paint on. The same thing is true in Silverlight: before we can start adding elements to our project we first need a canvas to "paint" on. And now we have one.
Incidentally, why did we get a gray box that is 300 pixels wide by 300 pixels high? That's easy: those are the specifications found, by default, in the file CreateSilverlight.js. If you don't like those default values, just open CreateSilverlight.js and tweak the settings as needed. Would you prefer a white canvas 800 pixels wide by 300 pixels high? (Which, for the examples we're about to use, is something you would prefer.) Then simply make sure you have these three lines in CreateSilverlight.js:
width:'800', // Width of rectangular region of
             // control in pixels.
height:'300', // Height of rectangular region
              // of control in pixels.
background:'white', // Background color of
                    // control.
Note. Good point: in the original .js file the background is set to #D6D6D6. Now we've set it to white. As it turns out, in a XAML file you can use RGB values to represent colors, or you can use color names to represent colors. (See msdn2.microsoft.com/bb188314.aspx for a complete list of color values and color names.) As much as the Scripting Guys like to refer to colors by their RGB values ("My house? It's an FFFFFACD with FFCD5C5C trim"), we decided to use color names here.
Bonus tip: you aren't limited to using solid colors as a canvas background. Paste the code from Figure 2 into your XAML file and see what you get.
Figure 2 Pretty colors (Close Figure 2)
<Canvas
 xmlns="http://schemas.microsoft.com/client/2007"
 xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
 Width="800"
 Height="300">

 <Canvas.Background>
 <LinearGradientBrush>
 <GradientStop Color="Blue" Offset="0.0" />
 <GradientStop Color="Black" Offset="1.0" />
 </LinearGradientBrush>
 </Canvas.Background>

</Canvas>
Did you try it? Pretty cool, huh?
Of course, a blank canvas by itself isn't all that interesting. Because of that, let's go ahead and add an object to that canvas. Although Silverlight allows you to add all sorts of different objects (including ellipses, rectangles, lines, and polygons), we'll add a good old-fashioned text block to our canvas. How? By inserting the following tags between the <Canvas> and </Canvas> tags in our XAML file:
<TextBlock
 Name="Test"
 FontSize="40"
 FontFamily="Georgia"
 FontWeight="Bold"
 Canvas.Top="20"
 Canvas.Left="20"
 Text="The TechNet Script Center">
</TextBlock>
As you can see, this XAML stuff really isn't all that hard. We start out by using the <TextBlock> tag, making sure that we include the various TextBlock parameters (such as Name and FontSize) inside the tag. We then indicate that we're through with our TextBlock by using the </TextBlock> tag. The cool thing about this? This is pretty much how you add any object to a XAML file. Want to add a rectangle 300 pixels wide by 100 pixels high? Okey doke:
<Rectangle
 Height="300"
 Width="100"
 Fill="Blue">
</Rectangle>
By the way, when adding a shape (such as a rectangle) to your canvas, make sure you specify a Fill color and/or a Stroke color. If you don't, Silverlight will add a transparent rectangle to the canvas. The rectangle will actually be there, but unless you have some sort of Superman-like x-ray vision you won't be able to see it.
Speaking of Fill colors and Stroke colors, that's the way you color some Silverlight objects. For other objects, you need to use a brush, such as the LinearGradientBrush we showed you (but didn't discuss) in our bonus tip. If you prefer to paint objects a solid color, you can use the aptly named SolidColorBrush. For example, take a look at the XAML tagging in Figure 3 that "paints" our text (technically, that paints the TextBlock's Foreground property) red.
Figure 3 Painting text (Close Figure 3)
<TextBlock
 Name="Test"
 FontSize="40"
 FontFamily="Georgia"
 FontWeight="Bold"
 Canvas.Top="20"
 Canvas.Left="20"
 Text="The TechNet Script Center">

 <TextBlock.Foreground>
 <SolidColorBrush Name="test_brush" Color="red"/>
 </TextBlock.Foreground>

</TextBlock>
For an even cooler effect, try the ImageBrush property, which fills text with a picture:
<TextBlock.Foreground>
 <ImageBrush Name="test_brush" ImageSource="Sunset.jpg"/>
</TextBlock.Foreground>
And here's yet another tip: if you try the ImageBrush property, you will most likely want to increase the value of FontSize because the bigger the text, the better the effect.
Give that a try and see what happens. You will be suitably impressed.

But Wait: You Ain't Seen Nothin' Yet
We have to level with you: up until now we've been keeping the good stuff to ourselves. It's cool that you can display text in Silverlight; it's even cooler that you can draw rectangles, ellipses, and polygons. But all that pales in comparison to the animations that can be added to a Silverlight project.
Again, about all we can do in this article is introduce you to some of the ways you can create Silverlight animation; we simply don't have room to go beyond that. (Don't blame us; we keep suggesting that TechNet Magazine become Hey, Scripting Guy! Magazine, but apparently our e-mails haven't been getting through.)
Without further ado, here's a cool effect that causes our text to slowly (but surely) fade in. Add the code in Figure 4 between the <TextBlock> and </TextBlock> tags in your XAML file.
Figure 4 TextBlock tricks (Close Figure 4)
<TextBlock.Triggers>
 <EventTrigger RoutedEvent=
     "TextBlock.Loaded">
 <BeginStoryboard>
 <Storyboard>
 <DoubleAnimation
 Storyboard.TargetName="Test"
 Storyboard.TargetProperty="Opacity"
 From="0.0" To="1.0"
 Duration="0:0:5" />
 </Storyboard>
 </BeginStoryboard>
 </EventTrigger>
</TextBlock.Triggers>
Admittedly, it looks like there's an awful lot going on here; fortunately, much of this is just boilerplate tagging. To begin with, you can't just add an animation to a XAML file; you need to indicate when that animation is supposed to play. That requires us to specify a set of Triggers for the TextBlock. In this case, we're going to create a trigger that will start the animation any time the TextBlock is loaded:
<EventTrigger RoutedEvent="TextBlock.Loaded">
In Silverlight, animations must take place on a storyboard (a term borrowed from the movie industry). That's why we have the <BeginStoryboard> and <Storyboard> tags.
In order to make our text fade in, we need to modify the Opacity property. We will gradually increase the value from 0.0 (transparent) to 1.0 (solid color). Because Opacity happens to be a Double property (a property based on a double-precision numeric value), we use a DoubleAnimation animation. And just how do we use a DoubleAnimation animation? Why, by indicating the object we want to animate (in this case Test, the name we gave the TextBlock); specifying the target property (Opacity); and then specifying both the animation range (from 0.0 opacity to 1.0 opacity) and the duration for the entire animation (5 seconds). In other words:
<DoubleAnimation
 Storyboard.TargetName="Test"
 Storyboard.TargetProperty="Opacity"
 From="0.0" To="1.0"
 Duration="0:0:5" />
Note. Make sure you use the syntax 0:0:5 (hours, minutes, seconds) to indicate 5 seconds. Using the value 5 by itself will not generate an error, but it will cause the animation to take 5 days to complete.
Did you try that? Now try this: within the DoubleAnimation tag, set the AutoReverse property to True:
AutoReverse="True"
Your animation will fade in and then fade back out.
Set the RepeatBehavior property to Forever and your text will continue to fade in and fade out, well, forever:
RepeatBehavior="Forever"
Is forever a bit long? Then set RepeatBehavior to, say, 3x, to make the animation play 3 times and then quit.
RepeatBehavior="3x"
That's all pretty cool, if we do say so ourselves. However, what if you don't like text that fades in and then fades out; what if you like text that gradually changes color? Well, then you should use a ColorAnimation animation:
<ColorAnimation
 Storyboard.TargetName="test_brush"
 Storyboard.TargetProperty="Color"
 From="Red" To="Blue" Duration="0:0:5" />
Note. Just make sure you start out with red text if you try this one. Remember, we already showed you how to paint your text red.
That was fun, wasn't it? And yet, you seem a little disappointed. Oh, so that's why: the ColorAnimation was nice, but you were really hoping to be able to make things fly around the screen. Hey, don't mope; what kind of software would Silverlight be if it didn't let you fly things around on screen? Try this:
<DoubleAnimation
 Storyboard.TargetName="Test"
 Storyboard.TargetProperty="(Canvas.Left)"
 To="150"
 Duration="0:0:5" />
As you can see, we're causing our TextBlock to slowly move from its beginning position to a spot 150 pixels from the left edge of the Canvas (Canvas.Left). If we wanted to move the TextBlock up and down, we just use the Canvas.Top property:
<DoubleAnimation
 Storyboard.TargetName="Test"
 Storyboard.TargetProperty="(Canvas.Top)"
 To="150"
 Duration="0:0:5" />
Could we—be still our beating hearts—combine these animations and make the TextBlock move both from side-to-side and up-and-down? Beats us. But here's a hint: paste both animations into your XAML file (one right after another) and see what happens.
Note. OK, we lied: we knew all along you could combine these animations. In fact, you could stuff a whole bunch of animations into a single Silverlight project. It's a little tricky—you might have to include additional tagging to ensure that one animation ends before the next begins. But this also enables you to do some pretty fancy animation simply by typing in a few XML-like tags.

Sorry, Folks; Show's Over
Yes, we know: we hate to go, too, especially now that we're just getting to the really good stuff. But don't fret; this doesn't have to be good-bye. If you'd like to know more about Silverlight, drop us a line at scripter@microsoft.com. If there's enough interest, well, we'll see what we can do in future columns. In addition, be sure to stop by the TechNet Script Center (microsoft.com/technet/scriptcenter/resources/tnmag/archive.mspx); we'll post links to complete XAML examples (eXAMLs?) for you. Until then, as Karl Marx and Frederick Engels said at the end of their manifesto Letter from the Brussels Communist Correspondence Committee to G. A. Kottgen, "Good night, and please drive home safely."

The Microsoft Scripting Guys work for—well, are employed by—Microsoft. When not playing/coaching/watching baseball (and various other activities) they run the TechNet Script Center. Check it out at www.scriptingguys.com.


 Security Watch: BitLocker and the Complexities of Trust
Security Watch
BitLocker and the Complexities of Trust
Justin Troutman

Windows Vista. I have not uttered those two words in an article before, which is why writing this column makes me feel like a five-year-old with a blank check walking into a toy store. There are so many features to talk about, so much real estate worth exploring, and so many possible avenues a discussion can take.
A while back, I was reading security guru Bruce Schneier's blog and I came across an entry about something that was new to me at the time—BitLocker™. I quickly gleaned that this was intended to provide cryptographic functionality in Windows Vista®—and immediately cringed, "Oh no. Windows Vista is doing cryptography? It can't be any good. And even if it does a decent job, there's gotta be a back door somewhere."
There are some historical arguments (valid or not) as to why one might hesitate to rely on security features built into Windows®. But that's not why I'm writing this column. Instead, I'd like to build a case detailing why I believe there are sound reasons for giving BitLocker a chance. I also want to toss in a few cents on the kind of philosophy that ensures the vitality of cryptographic software and hardware.
But make no mistake: I'm not proposing BitLocker as a panacea for security as a whole—BitLocker is just one of the many pieces of the security puzzle. Its goal is to provide resiliency under certain threat models.
In particular, BitLocker sets its sights on the lost (or stolen) laptop. Mobile workers are constantly toting all sorts of confidential information all over the place—trains and planes, restaurants and hotels, home offices and branch offices. And they're carrying this information on laptops and other mobile devices that typically lack any real security, leaving the data on them extremely vulnerable. So what happens to this data when one of your users leaves a laptop somewhere?
You can't eliminate the sloppiness of human nature. Luckily, you can limit the mess that could ensue. The cost to a company for replacing a laptop is minor compared to the cost of dealing with confidential data that's been compromised. This is the security that BitLocker aims to provide.

Traversing the Credits
I'm discussing BitLocker here from an interesting perspective. You see, I haven't tried it yet. I know what you're probably thinking: how can some guy who hasn't even used BitLocker have an opinion about it, let alone anything useful to say on the subject? Bear with me please. What I really want to discuss is cryptography from a higher level and the design philosophies that are critical to a successful solution.
This all started with what I read on Bruce Schneier's blog. Say there's a new movie you want to see. Based on the trailer and the lead actors in the cast, you've made some assumptions about the film. True, a good cast doesn't guarantee a good movie. However, the cast list can say a lot about the film and what you might expect from it. Does it star Chris Rock and Ben Stiller? Or is it a Kate Winslet and Johnny Depp film?
Similarly, when I saw that an internationally respected security guru was discussing BitLocker on his blog, I made some assumptions. First, I figured that this new technology must either be good enough to deserve an honorable mention or so ridiculously bad that it had to be called out publicly. To my surprise, Bruce Schneier's final verdict was positive.
One particular sentence stood out in his write-up: "There aren't any back doors for the police." This statement packs a lot of punch. It was stated with great confidence and included a link to the Microsoft® System Integrity team blog. I followed the link and found a post by Niels Ferguson, a security-focused developer at Microsoft, denouncing speculative rumors that Microsoft had intentionally included back doors in BitLocker for law enforcement purposes. Ferguson stated outright that back doors are unacceptable and that he'd have no part in a project that supported them. (He also explained that even if Microsoft was forced to include a back door by law, the company would either announce the back door publicly or withdraw the feature entirely.)
The name on the post—Niels Ferguson—explains Schneier's confidence, and it sent confidence my way as well. Ferguson is someone who knows what he's talking about, and he has a track record that ensures you can trust what he says. Bruce Schneier and Niels Ferguson are the coauthors of Practical Cryptography (a seminal book about applying good cryptography, simplistically, correctly, and securely) and the co-designers of the block cipher, Twofish—a 128-bit Feistel network that earned its cryptanalytical bones by surviving as a finalist in the Advanced Encryption Standard (AES) selection process.
Of course, having a seasoned cryptographer attached to a project is no guarantee that the final product will be secure. Even the masters of this art can sometimes miss a stroke or two. But regardless of how BitLocker pans out over time, you can at least be certain that sound design strategies played some part in its creation.
Speaking of missed strokes, there are the bona fide attempts by developers without a lick of cryptographic sense and the mala fide attempts by those who are more concerned with just getting a product out the door. Both endeavors, regardless of intent, are security failures in the making. Neither, however, appears to be the case with BitLocker. Having at least one competent cryptographer involved and ample resources supporting the effort bodes well.

A Word about Trust
Not long ago, Phil Zimmerman, creator of PGP encryption, shared some rules of thumb with me. Should there ever be a published "Security Creed for Developers," his advice would surely grace the top of the charter. While his points were not intended to be specific to BitLocker, the design philosophies they promote can be applied to most any cryptographic solution. You might find these obvious or overzealous. But given the state of security today, they are worth stating explicitly.
When designing a cryptographic infrastructure, the developer must be simple, correct, and secure. Although mistakes are inevitable, they should not be accepted as a natural occurrence and merely shrugged off. The developer must be a staunch advocate of nothing less than meticulous perfectionism. As Zimmerman put it, "design as if making a mistake will cost someone's life." Think this is an exaggeration? At the 2005 Annual Computer Security Applications Conference, NSA cryptographer Brian Snow (who has since retired) said, "I want functions and assurances in a security device. We do not beta test on the customer. If my product fails, someone might die." So remember—mistakes can have more than just a monetary cost.
Users deserve assurance. Gaining—and maintaining—the trust of users is pivotal. Zimmerman makes this very clear when he says: "Earn the trust of the users. Once you shoulder that burden, you can never put it down." By doing so, the company preserves the user's assurance and its own reputation as being secure. If lost, these can be irrecoverable.
These are perspectives I hope Microsoft kept in mind when designing BitLocker (or any of the company's other products, for that matter). That said, I want to explain why I'm optimistic that Microsoft has actually gotten this right and delivered a piece of cryptography technology that should be taken seriously.

A Poor Man and His Elephant
At its core, BitLocker is how Windows Vista (specifically, the Enterprise and Ultimate editions) encrypts all system volume data. This may sound like a straightforward application, but consider the constraints. Since BitLocker encrypts data at the per-sector level, and the ciphertext cannot exceed the length of the plaintext, there is no breathing room left for anything else—such as nonce (a number or string used only once for authentication), initialization vector (IV), or Message Authentication Code (MAC). I anticipated that such tight constraints and the conditions they impose were likely, but I also knew that message authentication would be addressed somehow.
BitLocker relies on what's often called poor-man's authentication. The compromise isn't as conservative as you usually want, since it assumes that manipulated ciphertext doesn't result in meaningful plaintext. In other words, if ciphertext is manipulated, it should, for example, cause the system to crash rather than allow an adversary to carry out some function.
The Microsoft team behind BitLocker knew that an entirely new block cipher would require more time to analyze than is acceptable, but existing designs either hadn't yet been analyzed well enough or weren't efficient enough. So, for encryption, Microsoft chose AES in CBC (Cipher Block Chaining) mode, which I'll refer to as AES-CBC. This is indistinguishable under the chosen plaintext attack model (abbreviated IND-CPA), but it does not preserve integrity. And since there's no room for a MAC, and CBC is a mode of confidentiality, there's no integrity preservation at all. This is Elephant's cue to enter the room.
The new Elephant component sports two diffusers, which are built to render much better poor-man's authentication than vanilla AES-CBC. (I should note, however, that there is an option to run AES-CBC without Elephant for those of you who must obey strict standards-compliance policies.) Although not always ideal, poor-man's authentication is the most suitable solution under the given constraints, and Elephant aims to make the best of the situation. How exactly does all this work?
Figure 1 illustrates the flow of understanding. When encryption takes place, the plaintext is combined, via XOR, with a sector key. The text then flows through two unkeyed diffusers. From there, the text is encrypted using AES-CBC. The sector key and AES-CBC both require key material and therefore they are keyed—independently. This method simplifies the formalization of a proof for reducing the security of the AES-CBC and Elephant construction to that of AES-CBC. Elephant is a new primitive, and new primitives may be met with reluctance until they've been rigorously analyzed. But this approach balances the scale since BitLocker can show that AES-CBC with Elephant is no easier to attack than AES-CBC alone.
Figure 1 Poor-man's authentication with Elephant
Figure 1 Poor-man's authentication with Elephant
The sector key and the AES-CBC components both receive 256 bits of key material, making the key length 512 bits. By default, however, these components use only 128 bits of key material each, meaning some of the key material isn't used. The reason for this is simply that it's easier to throw away bits you don't need than it is to change the key management infrastructure as key lengths vary.
The block length is allowed to be any power of two, within the range of 512 and 8192 bytes. To ensure that any alteration to the ciphertext results in all of a sector's plaintext being modified in a random way, the block cipher is designed to behave with such a variable block size. Furthermore, if it behaves like a tweakable block cipher (as described by Liskov, Rivest, and Wagner), with the algorithm changing slightly from sector to sector, an adversary can't successfully move one sector's ciphertext to another sector.

Only Time Will Tell
Speculating about the cryptographic security of BitLocker misses the much bigger picture. For BitLocker, being secure isn't necessarily sufficient for it to be able to do its job. This is because BitLocker is not a solution unto itself—it's one of many Windows Vista appendages. Microsoft says BitLocker is tightly integrated into Windows Vista. But if there is such tight integration with the OS, then isn't it plausible that another component's failure could in turn cause BitLocker to fail or be compromised?
Personally, I believe in modularity and isolated failure. Tight integration without modularity leads to complexity. Granted, this may not be the case with Windows Vista and BitLocker, but only time—and ample analysis—will tell if that is so.
So what do I think about BitLocker? I tip my hat to the Microsoft System Integrity team for relying on true, reliable cryptographers who did things as cryptographers should. It's obvious that this feature was designed for cryptography, not for a marketing campaign. So my response to that question is that BitLocker should be taken seriously. Notice that this sets aside the verdict of whether its security is sound. Again, only time and real-world evaluation will tell. Many cryptographic primitives and protocols have been broken. But at the very least, the Microsoft System Integrity team has given us understanding and platforms upon which to build even better solutions.

Justin Troutman is a maturing cryptographer and undergraduate student in mathematics. His primary focus is on symmetric cryptography. He is also the founder of Extorque, which specializes in cryptographic research and consulting.


 The Desktop Files: The Power User's Guide to WIM and ImageX
The Desktop Files
The Power User's Guide to WIM and ImageX
Wes Miller
This column is based on a prerelease version of the Windows Automated Installation Kit (WAIK) and Windows Server 2008. All information herein is subject to change.

In several of my previous columns I've discussed ImageX and the Windows Imaging (WIM) format. At Tech•Ed a few months ago, I heard some of the comments and questions from IT pros beginning to use Windows Deployment Services (WDS), ImageX, and the Windows Automated Installation Kit (WAIK) and realized there
was need for more user guidance. So in this month's column I'd like to focus on ImageX and the WIM format, and how you can get the most out of them. Together, ImageX and WIM allow you to easily deploy not only Windows Vista® and Windows Server® 2008, but also Windows® XP, Windows Server 2003, and Windows 2000 as well.

Changes to WAIK
The upcoming release of Windows Server 2008 may have you wondering what this means for the WAIK. Microsoft plans to release an updated version of the WAIK in the timeframe of the Windows Server 2008 release. There are no significant changes to the WIM format (the ImageX tool and the format are unchanged); WDS for Windows Server 2003 does not change either. Contrary to rumor, multicast will not be available for Windows Server 2003 WDS—only WDS servers running Windows Server 2008 will feature the new multicast deployment capability for WDS.
Windows PE 2.0 and Windows Setup will be revised in order to support the following:
  • Deployment of Windows Server 2008 and Windows Vista.
  • Multicast deployment via WDS from a Windows Server 2008 WDS server.
  • Deployment of x64 versions as well as x86 versions of Windows Vista and Windows Server 2008 from x86 Windows PE.
First, this means that Windows PE 2.0 will be able to deploy Windows Server 2008 just as readily as it can Windows Vista—the same tools and processes work as well for one as they do for the other. Second, you will be able to utilize multicast deployment via WDS once your WDS servers are running on Windows Server 2008. Finally, this means that you will have the ability to use an x86 Windows PE 2.0 disk and deploy either an x86 or x64 image of Windows from it.
I mentioned this latter point at Tech•Ed in my session on x64 Windows. Unfortunately, there has been some confusion around this in blogs and in the Windows technical community. This does not mean that Microsoft will be distributing x64 Windows and x86 Windows on every Windows Vista or Windows Server 2008 DVD. It also doesn't mean that you should combine Windows x64 and Windows x86 images into a single WIM file (although you can). Doing so causes no single-instance storage savings, though it does allow the WIM to travel as one file.
What this does mean is that enterprise customers and OEMs who have spent years developing x86-based deployment processes (and custom scripts and tools) will now be able to deploy either x86 or x64 Windows from within WIM using 32-bit setup—something that enterprise customers have been requesting since x64 Windows first arrived in 2005.

ImageX Day to Day
Using /mount, /mountrw, and /delete
Here are some important tips for using the /mount, /mountrw, and /delete switches with ImageX:
  • Mount a volume with /mountrw if you intend to edit it.
  • You should mount a WIM to an empty directory—just to eliminate confusion from any files you have stored there and to prevent ImageX from throwing an unneeded warning message.
  • Don't forget to /unmount /commit your read/write image if you want to save the changes you've made. All too often I've made changes, only to forget to commit them (and thus losing them).
  • Deleting files via /mountrw and deleting Volume Images via /delete doesn't recover any space—in fact it can actually use more space. See the section in this column on /export for more details.
  • You can only mount a single WIM file once for read/write access. Two users can't edit the same WIM at once, and you can't edit two Volume Images at once.
  • Even in read-only mode (/mount), you should limit the number of Volume Images you mount simultaneously, as the driver has a dependency on the amount of Windows non-paged pool memory available on your system for each image mounted. This results in a limit that may vary per-system (or per system architecture). I recommend not trying more than five images simultaneously.
  • Making significant edits via /mountrw will take almost no time when you make the changes, but may cause /commit to take a considerable amount of time, since it needs to compress and store all of the new file data.
  • Use /delete and /mountrw carefully—once the changes are committed, they cannot be undone.
  • You cannot run /delete if there is only one Volume Image in a WIM file (delete the entire WIM file or append another Volume Image instead).
  • To quickly get a directory tree of the contents of a Volume Image, use this command instead of /mount:
ImageX /dir <path_to_wim_file> <image index>
  • If you are editing two WIM files at the same time, try not to /unmount them at the same time; sequence your mount/unmount events so one won't block the other.

ImageX was specifically designed to mirror the way OEM and enterprise customers say they use the tools. Figure 1 shows the process.
Figure 1 Using ImageX
Figure 1 Using ImageX
It's worth emphasizing that Step 5 is critical—the reality is that an image is constantly changing. Thus, capturing and modifying the image needed to be as fast and easy as possible.
To that end, you can append an additional image (using the /append switch) to one you already created. Initially the development team thought you'd want to add in an additional SKU (version of Windows) to your earlier image, but the more I worked with enterprise customers early in Windows Vista development, the more I heard it being used to quickly deploy X, then recapture X1 where you had made one of the modifications in step 2. Doing so allowed you to quickly recapture the image since you had only modified a constrained number of files.
When the team understood this, they realized that it might not make sense in all situations to keep the original image. Later in the column I'll discuss the /export switch that was added specifically due to this scenario and the one that follows next.

Change Is Constant
Early on, the development team decided that WIM files should be editable. Since they had already decided that a sector-based imaging format would not work (see the December 2006 column at technetmagazine.com/issues/2006/12/DesktopFiles for an explanation), they had to deliver a solution that would easily mount the files natively as a part of the Windows file system and make changes to the mounted Volume Images in the WIM file. This meant a few more "moving parts," as I like to refer to them. Mounting a sector-based image is relatively easy, but mounting a proprietary archival format such as WIM required some very specific tooling.
The format needed to have access from the command prompt—ruling out the commonly used Windows Shell Extensions used to handle ZIP and CAB files natively. Unfortunately, these handlers only allow Windows Explorer abstract access to an object.
As a result, the WIM solution involves a driver (wimfltr.sys) and ImageX, which serves as the method to actually load the driver and handles the interface for mounting the WIM. The driver serves to add an abstraction to the Windows file system, as shown in Figure 2. When you run a command like one of these, you are telling ImageX to load the wimfltr.sys driver and to have the driver overlay the first Volume Image (the 1 in the commands used here) on the empty directory mount, as you can see here:
Figure 2 Mounting a WIM Volume Image
Figure 2 Mounting a WIM Volume Image
ImageX /mount iso\sources\boot.wim 1 mount
ImageX /mountrw iso\sources\boot.wim 1 mount
This is the default behavior in the WAIK. Thus, when you explore or generate a directory listing on the mount directory, you see the root of Volume Image 1 in boot.wim. Of course, whether you use /mountrw or /mount will determine whether this directory is read-only or read/write (see the sidebar "Using /mount, /mountrw, and /delete" for details).
The /mountrw capability was originally designed to easily add in drivers or other xcopy-able files or content, set up answer files, or to modify the offline registry of a mounted WIM file to change it as needed (see Figure 3). It was not designed to have entire applications added or removed (since MSI-installed applications cannot be installed offline).
Figure 3 Modifying a mounted WIM file
Figure 3 Modifying a mounted WIM file
It's critical to understand that while you're making changes to the WIM file when it is mounted, you are not changing the actual WIM file directly (at least not immediately); rather, you are changing a cache of the WIM file that is effectively a merging of the overlay and the changes you've made. As I note in the sidebar, I've been bitten by this before. Make sure that when you are finished you use the /unmount and /commit switches as only then will your changes be committed to the WIM file. Otherwise, they are completely lost.
When you save your changes by committing them, you are telling the driver to make the changes in the WIM file itself. The driver adds in the additional file data. (Note that if you've made a lot of changes, this means /commit can take a considerable amount of time, since you are only then adding the file data to the WIM.)
Also, unlike a sector-based imaging format, you may have changed one or more files in the Volume Image. As a result, they are no longer single-instanced (unless this same file has already been stored once) and will result in the original file data becoming lost. WIM, unlike an NTFS or FAT volume, doesn't mark a file as deleted—it no longer references it, but doesn't clean up. As a result, if you make frequent edits via /mountrw or you /append numerous images or use /delete to delete a Volume Image, it only makes the WIM file forget about the references to the file data—it does not drop the related file data altogether.
Thus the dev team created the /export switch, also known in a way as the "WIM defragmenter."

Exportation
Important Notes for /export
When /export is being used to create a new WIM file, you can specify what type of compression is used rather than accepting the default compression type currently used. If you are changing from the current type of compression, exporting may take considerable time.
When used effectively as /append (by adding a Volume Image to an existing WIM file), /export does not allow you to modify the /compression attribute of the WIM file.
You can specify the /boot switch if you are exporting an entirely new Windows PE 2.0 Volume Image.

The /export switch was created in response to both of the previously mentioned scenarios and a couple of other more esoteric scenarios the team discovered. In a nutshell, /export allows you to take an existing Volume Image (or two, or three) and export them. This means rapidly taking the X and X1 WIM file above and either creating a new WIM just with X, just with X1, or with X1 and Y1, by exporting from yet another WIM file.
Imagine the scenario where you make modifications to X and /append the changes as X1. You decide that you want to delete X since you have replaced it as your new image for this quarter. You can use the /delete switch and ImageX will forget about the X image. However, to save space by eliminating all information related to X, you would run this command:
ImageX /export source.wim 2 destination.wim "New image for Q4CY07"
This will export the second Volume Image to a new WIM file with the new name, completely eliminating the earlier information from Volume Image 1. You can also export from spanned WIM images if you want to reassemble them into one WIM file (as I'll discuss later in this column). Spanned images can't be edited via /mountrw—one reason why you would want to reassemble them. It's important to note that there is no /import command for ImageX since /export can either create new WIM files or append to existing ones. See the sidebar "Important Notes for /export" for additional information on this topic.
This is also critical if you run peimg /prep: though this command prepares your Windows PE image to be used, it does not clean up anything in the WIM. Ideally, as a part of your Windows PE release process, you should prepare the image and then export it to a new boot.wim file. Note that the boot process only knows to look in \sources\boot.wim for the WIM file, so make sure it is named correctly and marked as a boot Volume Image.
I've mentioned that you will probably want to optimize your images by running /export against them when you're ready to decommission old Volume Images or combine multiples together. You also will want to do this if you have made numerous edits via /mountrw or /delete. Otherwise, your WIM file will continue to grow over time (in mysterious ways that you can't diagnose). If you are using /mountrw or /delete on a regular basis, consider adding /export into your workflow to ensure your images are fully optimized.

Spanning
Early on, another problem that cropped up was the probable need for WIM files to fit across multiple pieces of shiny media. We anticipated that Windows would ship on more than one CD (and potentially on DVD, as it did), and as a result built spanning support into WIM and ImageX.
As I mentioned, a spanned image created via /split is basically read-only—you can't /capture directly to a new spanned image, it can't be mounted via ImageX, and it can't be appended to via /append. As a result, I would recommend splitting your WIM file only after you are finished with your standard workflow and ready to release. That is, make all of your edits on the normal WIM file and consider splitting as part of the release process for your deployment scenario.
When it comes to applying the image, spanned images can be challenging. I would recommend using ImageX rather than setup when using spanned images. I would also recommend either applying the images over the network (for optimal performance without imaging the disk you are applying from), or copying them directly to the drive and then applying (noting that you will be disk-bound, since the disk is reading and writing to itself at apply time).

Looking for Validation
When you're using ImageX, you're moving lots of data—lots of very important data. A frequent question I heard at Tech•Ed this year was, "How can I ensure my images are valid?" Well, it's not terribly easy, but it really isn't that hard, either.
The main thing you have to understand about WIM is that, like most archival formats, it is susceptible to corruption. Most often this corruption occurs when a WIM file is transmitted over the network. Early on in Windows Vista, we saw specific problems with specific network cards—if they dropped packets, the WIM file was corrupted, and not always in ways that were easy to diagnose. This situation was one that actually mirrored what had happened years before when the driver.cab file grew in size—similar random corruption had occurred. Generally, if you get reports of bad images or of setup failing, you want to check what type of NICs are in use on the systems generating the reports. This is the case whether you are using ImageX by itself or running WDS.
So how can you see if an image is good or not? First, you can use the /check switch. This saves additional integrity checks in the WIM file and will verify, before application, that the WIM has not been corrupted. Be sure to run /check at apply time as well, which will validate the image (though it takes more time).
In addition, if you are having problems—or just want to be meticulous—you can run a hashing utility against the files before capturing and after application. You can verify after capturing as well by mounting the image and rehashing—not a bad way to ensure an image has captured correctly.
You'll want to make sure that your images are being captured correctly from the beginning. The /check switch and the format itself are designed as checkpoints, not as recovery tools to repair an incorrectly captured image.

Booting and Flags
I've mentioned before that you can use /boot to capture a WIM Volume Image of Windows PE and boot it. You should think of this as just a boot sector on a particular Volume Image. It can only exist on one Volume Image per WIM, and is only valid for Windows PE 2.0. Note that you cannot WIM boot Windows PE 1.x—it is simply not possible.
You may see references to flags in Windows Volume Images. Flags are used only by Windows Setup. A flag on a Volume Image of 2 means a Volume Image contains Windows PE and setup. A flag of 9 means it contains just Windows PE. Any other flag or the lack of a flag means that the image will not and cannot be used by setup. You should note that you can set flags on an image after it has been captured by using ImageX to modify them. I recommend using the WAIK (and optionally the Business Desktop Deployment Solution Accelerator) to create your Windows PE WIM boot image, as the flags will be properly set to begin with.

A Note to HTA Users
Recently I've heard several reports of issues with running HTA files on Windows PE 2.0. Unfortunately, there appears to be an issue with HTA files that contain any significant scripting. In Figure 4, take a look at the screenshot of the sample file I created, and note the scripting error (with no information) returned. This problem was reported by several customers, and I experienced it myself.
Figure 4 HTA scripting error
Figure 4 HTA scripting error
Thankfully, though, the fix is relatively easy. When preparing your image, add XML support as well as HTA support. The missing dependency in the HTA component appears to be included in the XML add-in module, thereby solving this problem.
I would like to thank Bruce Green, ImageX Developer, and Minxiao Zhou and Raja Ganjikunta on the Windows Deployment Test team for their assistance with this column.

Wes Miller is a Development Manager at Pluck (www.pluck.com) in Austin, Texas. Previously, he worked at Winternals Software in Austin and at Microsoft as a Program Manager and Product Manager for Windows. Wes can be reached at technet@getwired.com.


 Field Notes: Recycle Your PC
Field Notes
Recycle Your PC
Jim Lynch

The Community Microsoft Authorized Refurbisher (MAR) program (microsoft.com/mar) is an extraordinary and relatively unknown project of the Microsoft Unlimited Potential initiative, in which Microsoft supports diverse projects to improve education, create businesses and jobs, and generate sustainable economic opportunities for many underserved communities. Started in the U.K. in 2000, the Community MAR program now consists of more than 800 PC refurbishers in 50 countries around the world.
One of the most amazing aspects of the Community MAR program is that it is able to supply licenses for Windows® 2000 and Windows XP for only $5 USD. MARs supply these licenses—over 200,000 a year—on refurbished PCs to nonprofit organizations, schools, libraries, colleges, and, in the U.S., technology-access programs for low-income or disabled individuals. In this way, Community MARs reach those on the wrong side of the digital divide, providing access to educational and employment opportunities.
In pursuit of reaching underserved children, Community MARs span the globe. For example, Operation Homelink supplies low-income U.S. military families with computers; Computer Aid International in the U.K. is the largest non-government organization supplying computers to Africa; and the Computers for Schools Canada network of refurbishers supplies a significant portion of the computers that go to Canadian schools.
The Value of Reuse
The U.S. Environmental Protection Agency (EPA) in cooperation with the University of Tennessee has recently demonstrated the value of electronics reuse in terms of energy, materials, CO2, and toxic emissions. According to their Electronics Environmental Benefits Calculator (eerc.ra.utk.edu/ccpct/eebc/eebc.html), reusing just one computer with a CRT monitor saves:
  • 30 lbs. of hazardous waste
  • 77 lbs. of solid waste
  • 77 lbs. of materials
  • 147 lbs. (17.5 gallons) of water from being polluted
  • 32 tons of air from being polluted
  • 1,333 lbs. of CO2 from being emitted
  • 7,719 kilowatts of energy
It is also the equivalent of taking ½ of a car off the road for a year.

In fact, the Community MAR program came about in 2000 to help increase the flow of working equipment to schools and charities in the U.K. that traditionally had limited access to technology. The program expanded to the U.S. and Australia in 2003 and became global in 2005. It is now available to all types of refurbishers (commercial, noncommercial, government, and school-based). Along the way, it has also created a vibrant computer reuse and refurbishing industry.
One of the less-obvious benefits of the Community MAR program is the environmental one. Extending the life of electronic equipment is the most efficient form of recycling because it retains the highest function of a device and generates the highest economic, environmental, and social rewards. End-of-life recycling is certainly good and necessary, but it loses 10-30 percent of the materials and energy in any given item.
Not surprisingly, manufacturing computers has a huge environmental cost. According to environmental scientist Eric D. Williams of Arizona State University, producing the average 53-pound desktop computer and monitor requires 530 pounds of fossil fuels, 50 pounds of chemicals, and 3,330 pounds of water. Extending its life by two or three years saves 5 to 20 times more energy than end-of-life recycling over the computer's lifecycle (see the "Value of Reuse" sidebar for more astonishing figures).
Overall, through this innovative program, Microsoft helps create opportunities that can transform communities by bringing the benefits of technology to people in need. A surprising added benefit is that the Community MAR program has raised public awareness of and support for computer reuse.

Jim Lynch is Program Director for computer recycling and reuse at TechSoup.org, a San Francisco-based nonprofit whose mission is to supply very low-cost hardware, software, and know-how to charities around the world.


 Windows Confidential: Calling Dr. Watson
Windows Confidential
Calling Dr. Watson
Raymond Chen

You may have noticed that if you ask Dr. Watson (drwtsn32.exe) to take a memory dump of the Winlogon process, Windows® shuts down as soon as it's finished. Some people wonder, "Why does connecting a debugger to Winlogon take down the entire system?"
But that's not what's happening. Running Dr. Watson on a process is not the same as attaching a debugger to a process. Attaching a debugger involves using something like ntsd—and this does work, assuming you have the necessary privileges. Indeed, this is how the Windows team debugs problems with Winlogon. So the literal response to this question is, "Attaching a debugger to Winlogon doesn't take down the system."
But Dr. Watson is a medical examiner, not a general practitioner, first taking a memory dump of a problematic process, then killing the process. Since Dr. Watson doesn't actually try to fix the problem, if it let the process continue running, that process would simply crash again. And the doctor would be called right back in, leading to an infinite loop. In a sense, Dr. Watson is a debugger that doesn't actually do any debugging. The good doctor just shows up on the scene, takes careful notes, and then kills the process. And it is just this—killing Winlogon after the memory dump has been taken—that causes the system to crash.
If you kill Winlogon, your system will reboot
If you kill Winlogon, your system will reboot
Winlogon is what is known as a "critical system process" and the death of a critical process forces the system to restart. Why does the system consider Winlogon critical to its functioning? For starters, Winlogon is responsible for handling the secure attention sequence—Ctrl+Alt+Del. If Winlogon dies, the secure attention sequence stops working. That in itself is kind of bad. What makes it worse is that with the official keeper of the secure attention sequence gone, another program can merely wait for the user to press Ctrl+Alt+Del and display a fake password dialog in response. It wouldn't be a very good secure attention sequence if it could be spoofed! As a protective measure, Windows shuts itself down before the bad situation becomes worse.
Now, if you run Task Manager, highlight a critical system process like Winlogon, click End Process, and confirm, then gosh darn it, you just killed Winlogon and the system will reboot (assuming, of course, you are running as an elevated administrator and therefore have sufficient privileges to kill the process).
In earlier versions of Windows, Task Manager tried to stop you from killing these critical system processes, but its algorithm for deciding which processes were critical wasn't very smart. If you were sufficiently devious, you could fake out Task Manager and create a program that it considered to be unkillable. Of course, this wasn't really the case. You could still kill the program—you just had to use a program other than Task Manager in order to do it. (For example, you might try the taskkill program.)
To avoid being faked out, Task Manager simply stopped trying to prevent you from killing critical system processes. After all, if you don't do anything, you can't do it wrong. This does mean that Task Manager is no longer watching over your shoulder when you kill a process. It won't slap your wrist when you try to kill something you'd probably be better off leaving alone. If you choose to kill a critical system process, you're just going to have to live with the consequences.

Raymond Chen's Web site, The Old New Thing, and identically titled book (Addison-Wesley, 2007) deal with Windows history and Win32 programming. He leaves the job of operationalizing mechanisms to others.