 TechNet Magazine December 2006

TechNet Magazine December 2006

Exchange Server 2007: Upgrading Your Infrastructure to Exchange 2007

Whether your organization is large or small, a smooth transition to Exchange Server 2007 will require up-front planning. With the right game plan, you can avoid any service interruptions. Here’s a look at the steps you’ll need to follow when performing the upgrade.

Kate Follis Exchange Server 2007: Take Command with the Exchange Management Shell

The Exchange Management Shell gives administrators a powerful interface for managing every aspect of Exchange Server 2007. Get an overview of how you can start writing scripts, integrating applications, and performing other powerful management tasks from the command line.

David Strome Exchange Server 2007: Exchange & Outlook: Better Together

Together, Exchange Server 2007 and Microsoft Office Outlook 2007 represent a giant leap forward in communications and collaboration. Find out how as we dive into simpler configuration, improved search capabilities, Unified Messaging, and more.

Michael Khalili Exchange Server 2007: Never Out of Touch with Exchange 2007

Communication has just gotten more powerful than ever. See how the Unified Messaging and Outlook Web Access capabilities built into Exchange Server 2007 allow you to access all your messages in one place…from anywhere.

Joshua Trupin Exchange Server 2007: More Powerful Journaling in Exchange 2007

Exchange Server 2007 provides much more flexible journaling capabilities, making it easier to monitor specific user accounts and meet compliance requirements within a large organization.

David Strome Exchange Server 2007: Fighting Spam and Phishing with Sender ID

E-mail authentication built on the Sender ID Framework (SIDF) is gaining wide adoption, providing a major advancement in the fight against spam and phishing messages. In fact, more than a third of the world’s e-mail volume is already authenticated and SIDF-compliant. Here’s how it works.

Craig Spiezle and Alexander Nikolayev From the Editor: Reaching Out

TECHNET MAGAZINE IS MORE than just this fabulous publication you hold in your hands today. Our team has been working furiously to create an innovative new Web site that will make our content accessible to more than a billion people around the world.

Joshua Trupin Toolbox: New Products for IT Pros

Greg Steen Windows PowerShell: Simple Commands. Powerful Administration

Windows PowerShell is about to launch, offering Windows administrators the easiest and most flexible way to automate a wide variety of administrative tasks. Discover what you need to know to start performing administrative tasks more effectively using this powerful command-shell tool.

Don Jones Post Mortem: Migrating from Exchange 5.5

The Kentucky Department of Education recently upgraded its 178 individual school districts from Microsoft Exchange Server 5.5 to Exchange Server 2003. Take an in-depth look at the hurdles they faced and how they ultimately planned, tested, and rolled out the new implementation while meeting stated business and security goals.

Whitney Roberts SQL Q&A: Preventing Reboots, Installing Multiple Updates, and More

Edited by Nancy Michell Hey, Scripting Guy!: Where Are the Cup Holders?

You've already got VBScript and Windows Script Host. Why should you bother with a new tool for scripting? We'll show you how Windows PowerShell is more powerful, allowing you to do things you haven’t been able to do in the past.

The Microsoft Scripting Guys Security Watch: Deploy a Globally Trusted PKI

Imagine running your own PKI that combines the benefits of an internal PKI and the key advantages of a globally trusted root. Louisiana State University (LSU) has just such a solution. Take a close look at the technical design of LSU’s implementation and find out best practices for deploying a similar system in your organization.

John Morello The Desktop Files: Inside the Windows Imaging Format

With the release of Windows Vista comes a change in how Windows is installed and deployed. These changes are built around the new Windows Imaging Format (WIM). Here’s an overview of the new format, how it works, and the decisions that went into its design.

Wes Miller Windows Confidential: Waiting for RunOnce

CONSIDER AN installation that needs to restart the system in the midst of the process. For example, the program might wish to upgrade a system DLL that is currently in use. You would need to trigger an intervening reboot to get the new DLL file installed, and then after the DLL has been upgraded, you could use it in the next stage of the installation.

Raymond Chen


 Exchange Server 2007: Upgrading Your Infrastructure to Exchange 2007
New information has been added to this article since publication.
Refer to the Editor's Update below.

Exchange Server 2007 Kate Follis

Upgrading Your Infrastructure to Exchange 2007
At a Glance:
  • What's new in Exchange 2007
  • Preparing for upgrade
  • Role-based deployment
  • Message routing topology

I used to teach a class about migrating from Exchange Server 5.5 to Exchange 2000 Server. The class would begin at 8:30 in the morning, and end around 6:00 P.M. I would leave the classroom
drained of energy. There was a lot of preparation work involved just to move the directory resources, plus lots of confusion about NTDSNoMatch, and the occasional Windows NT® 4.0 trust that would refuse to be trusted. Due to time constraints, we barely touched on Exchange 2000 Server administration, yet invariably there would be a couple of eager students who planned to leave the classroom Friday night and complete migration over the weekend. I would caution against this, advising more planning time, but I doubt they heeded the warning.
The transition from Exchange 2000 Server or Exchange Server 2003 to Exchange Server 2007 requires the same kind of caution-you need to spend time planning. The actual transition process will vary by organization and the complexity of the current deployment, but the transition process will still require several phases. A small organization may move through these phases as a single project, while a large organization may decide to gradually introduce server roles and support Exchange in a coexistence mode for a period of time. The end-to-end process is designed to maintain messaging functionality and stability through the entire transition period. In this article, I'll focus on the essential steps required to maintain message flow through your organization as you upgrade to Exchange Server 2007.

Considerations
The first requirement to note is that Exchange Server 2007 must be deployed on 64-bit hardware and does not support an in-place upgrade from any previous version of Exchange. You must use the swing upgrade method to move your existing messaging services to Exchange 2007. Further, the Exchange organization must be operating in native mode to support the addition of Exchange 2007 servers, so if your organization is currently using Exchange 5.5, you will be required to perform an interim upgrade to Exchange Server 2003 before moving to Exchange 2007.
There are some important changes in Exchange 2007 you must consider when planning your transition. For example, Exchange 2007 introduces role-based deployment, which lets you choose the messaging services you want to provide and deploy server roles specific to those services. You can deploy the server roles individually on dedicated hardware, or install multiple roles on the same physical server, administered as separate entities. See the sidebar "Exchange 2007 Server Roles" for an explanation of each.
Exchange Server 2007 Server Roles
Mailbox Server The Mailbox server role provides message storage for an organization. Exchange 2007 can support up to 50 stores per server. These stores can be deployed as 50 individual storage groups, or you can create up to 50 stores in a single storage group. The Mailbox server role is the only role that can be deployed as a cluster, so if you will be using clustering, you will need to install the Mailbox server on dedicated hardware.
Client Access Server The Client Access server role replaces the functionality provided by a front-end server. It provides mailbox access to clients accessing Exchange using POP3, IMAP4, Outlook® Web Access (OWA), RPC over HTTPS (now known as Outlook Anywhere), and Exchange ActiveSync®.
Hub Transport Server This role provides SMTP and MAPI message transport services for the Exchange organization. Every message that is sent or received by the users in your organization is processed by a Hub Transport server. This is great, because it ensures that no message can bypass the server-based rules or journaling policies that are provided by agents that fire at various points in the transport pipeline.
Unified Messaging Server This role provides voice access to your mailbox. It integrates with your IP/VoIP gateway or IP-PBX to provide telephone access to messages and calendar items and lets you transcribe a reply. This role is new to Exchange, and is not able to interoperate with any previous versions of Exchange.
Edge Transport Server The Edge Transport server is typically deployed in your perimeter network. It provides SMTP message transport between the Exchange organization and the Internet, and provides anti-spam and antivirus processing using transport agents. You can now standardize on a single technology for both your organizational and perimeter network servers. This seamless interaction model simplifies administration and allows for easy integration of perimeter servers.

Examine Your Topology
So, where do you begin? Well, it's difficult to figure out where you are going unless you know where you are. Take time to assess your current environment and to document the topology-not only Exchange but also your Active Directory® environment. You'll probably be pleased to know that Exchange 2007 has done away with link state routing and the topology based on routing groups and connectors. Instead, Exchange 2007 takes advantage of the investment your organization has already made in designing the Active Directory topology. As a result, all Exchange recipient and configuration data is stored in Active Directory, the routing topology is derived from the Active Directory site configuration, and all server roles will use site awareness to discover the services running on other server roles.
Make sure that the current Active Directory site topology truly leverages the underlying physical network and that all Active Directory sites have associated subnets, then document the existing Active Directory sites and IP site links. Also document where all of your Exchange 2003 servers are physically located, and the routing group and routing group connector structure.
This is the trickiest part of the transition process: moving from an Exchange routing infrastructure based on routing groups to a routing infrastructure based on Active Directory sites. For a small organization, it's generally a simple process. But a large organization with many routing groups will need to plan carefully for an interim phase when Exchange 2003 and Exchange 2007 are coexisting. The last thing you want is to end up having a message sent to your Exchange 2003 mailbox from the Exchange 2007 user across the hall routed through some remote routing group over a low bandwidth connection.
Let's start by making some assumptions about your current topology:
  • The Exchange 2003 organization is running in native mode.
  • You have more than one routing group.
  • You have more than one Active Directory site.
Figure 1 diagrams an Exchange and Active Directory topology. Remember, the more details you include in your preliminary plans, the better your design will be.
Figure 1 An Exchange and Active Directory Topology
Figure 1 An Exchange and Active Directory Topology

How Exchange 2007 Uses Active Directory
When an Exchange 2007 server starts, it is stamped with a site attribute that helps other Exchange 2007 servers locate the services provided by that server. Only the Hub Transport server can use SMTP to transport a message within the organization. Each Active Directory site that contains a Mailbox server must also contain a Hub Transport server and, if the mailbox users access their mailbox by using any non-MAPI method, each site must also contain a Client Access server. Any time that a message needs to be processed for delivery, it will pass through a Hub Transport server, which will make a decision about how the message should be routed. If the message is destined for a Mailbox server in the same Active Directory site as the Hub Transport server, the Hub Transport server will deliver the message to the mailbox. If the message is destined for a Mailbox server that's in a different site, the Hub Transport server will relay the message directly to a Hub Transport server in the remote site.
The Hub Transport server uses the Active Directory IP site link cost information to calculate the lowest-cost route to the Active Directory site where the recipient mailbox is located. The route selection algorithm is very similar to Exchange 2003, but it's based on IP site link costs instead of routing group connector costs. Moreover, the message does not stop at each Hub Transport server along the way. It goes directly from source to destination. So why does it bother to calculate the lowest-cost route if it's relying on the IP network to transport the message? There are a couple of reasons. One is to delay message bifurcation. A message that is being sent to more than one recipient may need to be delivered to Mailbox servers in more than one Active Directory site. Rather than bifurcate, or split, the message at the first Hub Transport server, Exchange 2007 will not split the message until it reaches a fork in the routing path. As a result, the message will be relayed directly to a Hub Transport server in the Active Directory site that represents the bifurcation point. This behavior is known as delayed fan-out.
The lowest cost route is also used to determine where to queue the message in case the destination can't be reached. If a Hub Transport server in the target Active Directory site can't be reached, the sending Hub Transport server will then attempt delivery to a Hub Transport server in the next closest Active Directory site according to the routing path. Message delivery will continue along the lowest cost route until it reaches an Active Directory site where a Hub Transport server is available. Finally, if no Hub Transport servers along the route to the recipient Active Directory site are available, the message is queued locally. This method queues the message as close to the delivery point as possible, helping to make diagnosis of network failures more deterministic. This behavior is known as queue-at-point-of-failure.
Exchange 2003 works in a completely different manner. It calculates the lowest-cost route from one routing group to another based on the costs assigned to the routing group connectors. A bridgehead server in each routing group along the routing path will receive and then relay the message. If the next connector in the path is not available, an attempt is made to calculate an alternative route. Link state update messages are also communicated throughout the Exchange organization to notify the other Exchange servers that the connection is down. The bridgehead servers will attempt to route around the down connector until a link state notification is received indicating that the connection is up.
The challenge when transitioning a large organization is to maintain mail flow during the coexistence period. To achieve this continuity when Exchange 2007 is introduced into the environment, all Exchange 2007 servers become members of a single routing group. This means that regardless of which Active Directory site the Exchange 2007 server is in, Exchange 2003 will see it as belonging to that single routing group. This allows you to establish a routing group connector between that routing group and the Exchange 2003 routing groups so that Exchange 2003 can figure out how to route messages to Exchange 2007. Exchange 2007 will also use the routing group connector to determine how to get messages to Exchange 2003. However, Exchange 2007 will always prefer to route a message through another Exchange 2007 server, and will never backbone across an Exchange 2003 routing group to reach another Exchange 2007 server.

Introducing the First Exchange 2007 Server
When you install the first Exchange 2007 server into the Exchange 2003 organization, you will need to select an Exchange 2003 bridgehead server with which to establish the first routing group connector.
Figure 2 shows how the introduction of the first Exchange 2007 server affects the organization topology.
Figure 2 Installing the First Exchange 2007 Server
Figure 2 Installing the First Exchange 2007 Server
So far, so good. Notice that the default cost of the routing group connector created by Exchange 2007 is only 1. Mail continues to flow as usual through the Exchange 2003 servers, and if a message is sent to a user on the Exchange 2007 server from any Exchange 2003 server, it will flow through the Acapulco routing group. When the routing group connector was created, the Exchange 2003 bridgehead server you selected became a member of the ExchangeLegacyInterop universal security group automatically and was granted the correct permissions to send and receive e-mail messages to and from Exchange 2007. You must always use the New-RoutingGroupConnector in the Exchange Management Shell to create a routing group connector whenever an Exchange 2007 server is specified as a source or target server for the connector. You can also use the Set-RoutingGroupConnector cmdlet to modify the configuration of a routing group connector. For example, you may want to add source and target servers to the initial routing group connector to provide redundancy. (See David Strome's article on the Exchange Management Shell in this issue of TechNet Magazine for more information.)

Prepare to Move Some Mailboxes
The next step is to move some mailboxes from the Exchange 2003 servers to the Exchange 2007 servers. Use the Move-Mailbox cmdlet to accomplish this. Before you can decommission the Exchange 2003 servers, however, you'll want to create Send and Receive connectors on the Exchange 2007 server to replace any SMTP connectors that were defined on the Exchange 2003 server to handle routing to external SMTP address spaces.
In the next diagram, the resources from the Acapulco routing group have been moved to Exchange 2007. The Acapulco routing group can be decommissioned, but this means you have to establish a routing group connector to a different routing group now. In Figure 3 I have configured the routing group connector to a bridgehead server in the Miami North routing group.
Figure 3 Configuring the Routing Group Connector
Figure 3 Configuring the Routing Group Connector
Let's move on to transitioning the LA East Routing Group to Exchange 2007. This is the kind of situation when maintaining routing gets a little tricky. In Figure 4, Exchange 2007 has been installed in the Los Angeles site, and a second routing group has been decommissioned, but no additional routing group connectors have been created.
Figure 4 Exchange 2007 Intsalled in the Los Angeles Site
Figure 4 Exchange 2007 Intsalled in the Los Angeles Site
Now, if a message is sent from the Exchange 2007 server in the Los Angeles site to the Exchange 2003 server in the LA West routing group, it will be relayed first to Acapulco, then to Miami North, to Miami South, and finally to LA West. If a message is sent from the Exchange 2003 server in the LA West Routing Group to the Exchange 2007 server in the Los Angeles site, it will be relayed first to Miami South, then to Miami North, to Acapulco and then finally arrive back in Los Angeles. Ouch.
In order to avoid this, we'll need to add a second routing group connector. Figure 5 shows the addition of a routing group connector from LA West to the Exchange 2007 server at the Los Angeles site. Note: If you want to avoid an abundance of routing group connectors, make sure that your first routing group connector is configured in a well-connected hub Active Directory site.
Figure 5 More Efficient Routing
Figure 5 More Efficient Routing
This introduces a problem that may not be immediately apparent. There is now more than one way in and out of the Exchange 2007 routing group. Since the Exchange 2007 routing group does not use link state, a potential routing loop now exists. The Exchange 2003 server may select an alternative route to go around a down connector, while Exchange 2007 will always select the lowest-cost route. The way around this problem is to suppress minor link state update messages, which will prevent link state messages that communicate a down connector state but will not prevent link state messages that communicate configuration changes. This is important, because we want the Exchange 2003 servers to know about the new routing group connectors, but we don't want them to try to calculate an alternative route. The end result is that the Exchange 2003 servers will exhibit the same queue-at-point-of-failure behavior as the Exchange 2007 servers.
In Figure 6 we have transitioned more Exchange 2003 resources to Exchange 2007. Link-state updates have been suppressed and we have been careful to establish routing group connectors that maintain a logical routing path.
Figure 6 More Exchange 2003 Resources Move to Exchange 2007
Figure 6 More Exchange 2003 Resources Move to Exchange 2007
There's another problem, though. We've created link-state islands in Redmond and Vancouver. Mail can flow just fine, albeit through Miami, but there's no way for Redmond and Vancouver to communicate link-state configuration changes.
[ Editor's Update - 12/19/2006: This sentence originally stated that Redmond and Miami had no way to communicate link-state configuration changes. This has been corrected to read that Redmond and Vancouver have no way to communicate link-state configuration changes.]
To maintain a current routing table, you'll need to configure another routing group connector between Redmond and Vancouver. You can adjust the cost of this routing group connector so that it is high enough to prevent Exchange 2003 from routing over the connection, while still allowing link-state configuration changes to be communicated.
With careful planning, you can circumvent the problems that may arise from maintaining two separate Exchange routing topologies. You'll achieve the most seamless results by transitioning one routing group at a time and then moving on to the next, until you are running only Exchange 2007 servers.
So far, I've focused on the routing changes in Exchange 2007 and how they will affect your transition. There are a couple of other considerations to bear in mind when planning the move to Exchange 2007.
Exchange 2003 front-end servers can't access Exchange 2007 mailboxes. However, an Exchange 2007 Client Access Server has no problem connecting you to your Exchange 2003 mailbox. When using the Exchange 2007 Client Access Server, you'll simply see the same OWA version as the mailbox server where your mailbox is stored. If you are deploying Exchange 2007 on separate hardware, the Client Access Server should be the first role you introduce to an Active Directory site. Any Exchange 2003 front-end servers that provide public Internet access to mailboxes should be replaced with Exchange 2007 Client Access Servers before any mailboxes are moved to Exchange 2007.
I mentioned before that each site that contains the Mailbox server role also requires the Client Access Server role (if you have non-MAPI clients-and who could live without OWA these days?) and a Hub Transport server. So, if you're going to have your server roles share hardware, be sure to include the Client Access Server in the role selection. A typical installation will include the Hub Transport Server role, the Client Access Server role, and the Mailbox server role-all that you need for a fully functional Exchange environment.
Another consideration for the transition process is when to upgrade your client application (Outlook). You will not be able to take full advantage of all Exchange 2007 server features until your clients have been upgraded to Outlook 2007. This includes enhanced out-of-office message settings and managed mailbox folder features.
You will also need to decide when to deploy the Edge Transport server role. Technically, you can supplant your current perimeter network SMTP-relay and smart host with the Edge Transport server before or after introducing any other Exchange 2007 server roles. If you deploy the Edge Transport server first, however, the overall functionality will be limited because Exchange 2003 has no way to replicate the recipient and configuration data that the Edge Transport server needs from Active Directory to ADAM (Active Directory Application Mode).
After you've installed at least one Hub Transport server, you can deploy the Edge Transport server in the perimeter network and then subscribe the Edge Transport server to the Active Directory site where the Hub Transport server resides. The EdgeSync service on the Hub Transport server will replicate a subset of the recipient data in Active Directory to ADAM.
This data allows the Edge Transport server to perform recipient-lookup and safelist aggregation so that you can take full advantage of the anti-spam agents running on the Edge Transport server. In addition, EdgeSync will create the Send connectors that are required for end-to-end mail flow to and from the Exchange organization and the Internet. This can simplify the process of replacing the Exchange 2003 SMTP connectors that provide routing to external address spaces.
So, take a little time to plan your transition to Exchange 2007 and you'll find the route straightforward and pain-free.

Kate Follis is a Senior Content Publisher on the Exchange 2007 technical writing team. Her group specializes in the Edge Transport and Hub Transport server roles. Prior to joining Microsoft, Kate worked as a Microsoft Certified Trainer for several years, teaching Exchange and Active Directory.
© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


 Exchange Server 2007: Take Command with the Exchange Management Shell
Exchange Server 2007 David Strome

Take Command with the Exchange Management Shell
At a Glance:
  • How the Exchange Management Shell works
  • How to use cmdlets
  • Cmdlets vs. commands

If you administer an Exchange environment, I’m sure by now you’ve heard that the upcoming Exchange Server 2007 includes a new command-line interface called the Exchange Management Shell. Built on Microsoft Windows PowerShell technology, the Exchange
Management Shell is a powerful management interface you can use to manage every aspect of Exchange Server 2007 from the command line. From there you can interact directly with the Exchange Management Shell, write scripts for automation, or integrate your applications with the underlying engine. Figure 1 shows the shell in action.
Figure 1 The Exchange Management Shell
Figure 1 The Exchange Management Shell
The Exchange Management Console doesn't just use the Exchange Management Shell to perform operations; the wizards show you what commands they're performing. Copy the commands the wizards create and paste them directly onto the command-line or into a script. After you've modified the parameters to your liking, test your scripts or commands by using the WhatIf parameter, which lets you see the results before you make any changes to your environment.
The Exchange Management Shell gives you a robust and flexible scripting platform to integrate with your current scripting solutions, and it lets you communicate natively with data sources like COM and Windows Management Instrumentation (WMI). Based on the .NET Framework, the Exchange Management Shell uses cmdlets—the smallest unit of funcionality in the Exchange Management Shell—to accept and return structured data instead of plain text. We'll look at cmdlets in more detail shortly.

Windows PowerShell Snap-In
The first step in understanding the Exchange Management Shell is to look at how it relates to Windows PowerShell™, a brand new, powerful, flexible command-line interface that's based on the .NET Framework. It combines the best features of several other shells, and adds many new features as well. Windows PowerShell is designed to be a host for other applications, much the same way that Microsoft Management Console (MMC) acts as a host for the various snap-ins that are provided with applications. The Exchange Management Shell is the first Windows PowerShell snap-in to be released.
Snap-ins are collections of cmdlets used to administer an application or major component. If you don't load them first, you can't use them. Luckily, it's extremely easy to start the Exchange Management Shell snap-in. But, before I get to that, I should explain some key concepts that you'll need to understand as you read this article and use the Exchange Management Shell.
Cmdlets Cmdlets resemble built-in commands in other shells, for example, the dir command found in cmd.exe. Like these familiar commands, cmdlets can be called directly from the command line in the Exchange Management Shell and run under the context of the shell, not as a separate process. Unlike commands in other shells, cmdlets have descriptive verb-noun names. The verb describes the action the cmdlet takes, and the noun describes the component or feature that is acted upon.
You don't have to guess what a cmdlet does. When you see the Move-Mailbox cmdlet, you know exactly what it's used for.
For a look at how cmdlets differ from ordinary commands, see the sidebar "Cmdlets vs. Commands", which is taken from the .NET Framework 3.0 section of the Windows® SDK and can be found at windowssdk.msdn.microsoft.com/en-us/library/ms714395.aspx.
Identity and Positional Parameter The Identity parameter can be used with most Exchange-related cmdlets. It gives you access to the unique identifiers that refer to a particular object in Exchange Server 2007 so you can perform actions on a specific Exchange object using the unique value that makes the most sense to you.
The Identity parameter is a collection of values from other parameters. These values are guaranteed to be unique across that set of objects. You can specify the values of these other parameters, such as Name and DistinguishedName, or they can be system-generated, such as a GUID. The additional parameters that are used, if any, and how they are populated, depend on the object you refer to.
The Identity parameter is also considered a positional parameter—a parameter that lets you specify its value without specifying its name. A parameter is positional if the Parameter Position attribute is an integer. This integer indicates the position on the command line where the cmdlet can find the parameter's value. Because Identity is a positional parameter that resides in position 0, which is the first position, any value entered in this position without a parameter name is considered to be an Identity parameter value. This reduces the number of keystrokes when you type commands. For example
Get-Mailbox –Identity "Kim Akers"
performs the same action as:
Get-Mailbox "Kim Akers"
Pipelining Pipelining in the Exchange Management Shell is the act of one cmdlet using the output of another cmdlet when it performs an operation. Pipelining is accomplished by using the pipe "|" symbol. All verbs in the same noun-cmdlet set can use piped information from another command. Some noun-cmdlet sets also let you pass data through the pipeline to another noun cmdlet set.
The use of pipelining to string together the actions of two or more cmdlets lets you take smaller components and convert them into something more powerful. For example, you can use one cmdlet to gather data, pass that data to a second cmdlet to filter the data to a subset, and then pass that data to a third cmdlet to act on the subset only.

Starting the Exchange Management Shell
The simplest way to load the shell is to click Start, click Programs, then Microsoft Exchange Server 2007, and then click Exchange Management Shell.
The second way you can load the Exchange Management Shell snap-in is to do it manually from a Windows PowerShell session. Click Start, then Programs, and then Windows PowerShell. After it has started, run the following command to load the Exchange Management Shell:
Add-PSSnapin Microsoft.Exchange.Management.PowerShell.Admin
There are some differences between loading the Exchange Management Shell snap-in using the Programs menu entry and starting it manually using the Add-PSSnapin cmdlet. When you use the Programs menu entry, it not only loads the Exchange Management Shell snap-in, but it also runs a Windows PowerShell script to set up a customized Exchange administration environment. If you want to set up a similar administrative environment after loading the Exchange Management Shell manually, change to your Exchange installation's bin directory—for example, C:\Program Files\Microsoft\Exchange Server\Bin. Then, run the following command:
.\Exchange.ps1
Take a look at the Exchange.ps1 script to see how to customize the Windows PowerShell environment to suit your needs. When you load the Exchange Management Shell, you'll see something similar to Figure 2.
Figure 2 Exchange Management Shell Welcome Banner
Figure 2 Exchange Management Shell Welcome Banner

Where Do I Start?
Because the shell has such an extensive array of cmdlets, the Get-Help cmdlet and Exchange Server 2007 Help file will be invaluable resources. The Operations section in the Exchange Server 2007 Help file provides you with dozens of how-to procedures for performing most administrative tasks.
Typing Get-Help by itself displays general help information about getting help. To get help on a specific cmdlet, type Get-Help followed by the cmdlet you want information on. You can control what information is presented by using the Detailed, Full, and Example parameters. Just add them to the end of the command. For instance, Get-Help cmdlet name –Full returns all of the help sections available for a cmdlet.
If you want to retrieve information about a specific parameter or parameters on a cmdlet, you can use the Parameters parameter with Get-Help. Thus, if you want to view all of the parameters and their descriptions on the Set-Mailbox cmdlet that contain the word "quota," use the command:
Get-Help Set-Mailbox –Parameter *quota*
You may want to view a list of cmdlets that manage a specific server role or component feature or that affect objects across a certain scope of functionality. The Get-Help cmdlet lets you do this with three parameters: Role, Component, and Functionality. When you use the Get-Help cmdlet with these parameters, you must enclose the values that you specify with these parameters in wildcard characters, "*". The following are examples of how to call Get-Help with each parameter:
Get-Help -Role *Mailbox*
Get-Help -Component *Recipient*
Get-Help -Functionality *Server*
For more information, the "Getting Help" topic in the Exchange Server 2007 Help file lists all of the role, component, and functionality values that can be used.

Formatting Command Output
By default, when output is displayed to the screen, the Exchange Management Shell displays only a subset of the available properties for each object. However, you can easily gain access to all properties by piping the output of a command to the following three formatting cmdlets: Format-List, Format-Table and Format-Wide. You'll probably use Format-List and Format-Table regularly so I'll talk about those specifically.
The Format-List cmdlet takes input from the pipeline and outputs a vertical columned list of all the specified properties of each object. You can specify which properties you want to display by using the Property parameter (although the Property parameter is a positional parameter, meaning you only need to specify the values and not the parameter name). If the Format-List cmdlet is called without any parameters specified, all properties are output. This cmdlet wraps lines instead of truncating them. One of the best uses for Format-List is to override the default output of a cmdlet so that you can retrieve additional or more focused information. For example,
Get-DistributionGroup | Format-List Name, *OnlyFrom,
PrimarySmtpAddress, *Size*
Figure 3 shows the output of this command.
Figure 3 Format-List Example Output
Figure 3 Format-List Example Output
The Format-Table cmdlet lets you display items in a table format with label headers and columns of property data. By default, many cmdlets, such as Get-Mailbox and Get-JournalRule, use the table format for output. Parameters for the Format-Table cmdlet include the Properties and GroupBy parameters. These work exactly as they do with the Format-List cmdlet. To have long lines of property information display completely instead of truncating at the end of a line, use the Wrap parameter, like so:
Get-Mailbox –Database Research | Format-Table Name,
ProhibitSendQuota, Database –Wrap
See Figure 4 to view the output.
Figure 4 Format-Table Example Output
Figure 4 Format-Table Example Output
If you include a wildcard character, you can match multiple properties without having to type each property name individually. For example, this returns all properties that begin with E-mail.
Get-Mailbox | Format-List Email*

Time to Practice
Now that you have some basic commands under your belt, why not run through the following examples. They'll illustrate the power and flexibility of the Exchange Management Shell and let you get your hands dirty.
The command you'll try first creates 10 users, named User1 through User10:
1..10 | ForEach { Net User "User$_" MyPassword=01 /ADD
/Domain; Enable-Mailbox "User$_" -Database "Mailbox
Database" }
Let's dissect the parts. The command 1.0 outputs the integers 1 through 10 and passes them through the pipeline to the ForEach cmdlet. The ForEach cmdlet acts on each object as it's received and runs the commands within the { } brackets for each object. The Net User command creates a new domain user, and the Enable-Mailbox cmdlet creates a new mailbox in the "Mailbox Database" on the local server. The semicolon separates the two commands so they can be placed on the same command line. The $_ variable is a special variable that contains the value of the current object in the pipeline. With this example, the $_ variable contains the integer that is currently in the pipeline.
This next example will allow you to view the mailboxes that reside on the server MBX and return the Name, mailbox database, and quota properties of each mailbox.
Get-Mailbox –Server MBX | Format-Table
Name,Database,*quota*
The Get-Mailbox cmdlet returns each mailbox specified with the Server parameter, and then pipes the output of each mailbox object to the Format-Table cmdlet. By default, the Format-Table cmdlet displays all of the properties returned by a cmdlet unless you specify which properties to display. Here only the Name and Database properties, as well as all properties that contain the string "quota" are requested. Figure 5 shows how this output might look.
Figure 5 Get-Mailbox Command Output
Figure 5 Get-Mailbox Command Output
In this example you'll set the send quota limit to 600MB for all mail-enabled users across the organization who have "Manager" in their title.
Get-User –Filter { Title –Like "*Manager*"}
–RecipientTypeDetails UserMailbox | Set-Mailbox
–ProhibitSendQuota 600MB
Run by itself, the Get-User cmdlet returns all user objects in Active Directory, including non-mailbox-enabled users. In order to retrieve a set of user objects that can be used by the Set-Mailbox cmdlet, you need to instruct the Get-User cmdlet to return only the Active Directory user objects that are mail-enabled users. The RecipientTypeDetails parameter lets you specify the Active Directory recipient type, in this case, UserMailbox. Next, you need to retrieve a set of mail-enabled users that are managers. In this example, the Title property of each Active Directory user object is populated. You can use this value to determine which users are managers using the Filter parameter, which instructs the server to return only the Active Directory objects that match your criteria. This is called server-side filtering. Once you have a set of objects, it gets passed on to the Set-Mailbox cmdlet, which then sets the ProhibitSendQuota property for each mailbox to 600MB.
Next we'll test the outcome of a command to remove all mailboxes that are members of a distribution group. Here's the command:
Get-DistributionGroupMember "Fourth Year Students" |
 Where { $_.RecipientType –Eq "UserMailbox" } | Remove-
 Mailbox -WhatIf
The first cmdlet retrieves the recipient objects that are members of the Fourth Year Students distribution group. Then, only the recipient objects that match the RecipientType of "UserMailbox" are passed on to the Remove-Mailbox cmdlet. Instead of removing the mailboxes, the Remove-Mailbox cmdlet will display the action it would have performed and the objects on which it would have taken the action. The Get-DistributionGroupMember cmdlet retrieves a list of all recipient objects that are members of the "Fourth Year Students" distribution group. However, because the Remove-Mailbox cmdlet can only work with mail-enabled users, you need to filter out all recipient types except "UserMailbox". Unlike some other cmdlets, the Get-DistributionGroupMember cmdlet does not have the Filter cmdlet, which means it can't perform server-side filtering. For cmdlets that can't perform server-side filtering (because they don't see any performance benefit), you can use the Where cmdlet to perform client-side filtering, which retrieves all objects and performs the filtering on the local client machine. The Where cmdlet only allows objects through that match the criteria you specify. The objects that have a RecipientType of "UserMailbox" are sent to the Remove-Mailbox cmdlet. This cmdlet removes the Active Directory user object and marks the mailbox for removal. When the WhatIf parameter is used, the Remove-Mailbox cmdlet displays text similar to the following:
What if: Removing the Mailbox "contoso.com/Users
/user10" will remove the Windows user object and mark
the mailbox in the database for removal.
If you're satisfied that the command will perform the actions you want, run the command again without WhatIf. For certain cmdlets, such as those with the Remove verb, the shell will automatically ask for confirmation before any changes are made. You have the option to step through each change, instruct the shell to not prompt for confirmation again, or cancel further processing. For cmdlets that don't implement confirmation automatically, you can force confirmation using the Confirm parameter.

Conclusion
Now that you've had a taste of cmdlet power, you're ready to experiment on your own and see just how helpful both the Exchange Management Shell and the Microsoft Windows PowerShell will be in your future systems management tasks. If you want to read more about both topics, take a look at the following resources. For the Management Shell see the Exchange Server 2007 Help file at go.microsoft.com/fwlink/?LinkId=69434, the Windows Powershell scripting center at go.microsoft.com/fwlink/?LinkId=71134, and the Exchange team blog at msexchangeteam.com.

David Strome has been a technical writer with the Exchange User Education group at Microsoft for just over one year. Prior to joining Microsoft in Redmond, WA, David spent about 10 years designing, implementing, and administering Exchange Server installations at various companies in British Columbia, Canada. He can be reached at dstrome@microsoft.com.


 Exchange Server 2007: Exchange &amp; Outlook: Better Together
Exchange Server 2007 Michael Khalili

Exchange & Outlook: Better Together
At a Glance:
  • Setup simplified
  • Integrating voice, fax, and e-mail
  • Managed Folders for compliance

For a number of years, Microsoft Exchange Server and Outlook provided reliable e-mail for millions, helped people collaborate and manage their calendars, and made organizing contacts a breeze. Some might wonder what this combination would do for an encore. Well, Exchange Server 2007
and Microsoft Office Outlook 2007 are soon to be released, and together they offer a quantum leap in the realm of communication and collaboration.
The improvements offered by Exchange Server 2007 and Office Outlook® 2007 are apparent from the moment you start up the Outlook client. In previous versions, Outlook expected you to know the name of your Exchange mailbox server, something that stymied a great many users. The result was frequent helpdesk calls or complicated initialization scripts, both of which increased the cost and complexity of running an effective messaging environment. Now, with the Exchange Autodiscover Web service, Outlook can automatically configure the account if the client is joined to the domain. If the client is not domain joined, the user will need to enter only a name, e-mail address and password. Outlook and Exchange will figure out the rest (see Figure 1).
Figure 1 Account Setup
Figure 1 Account Setup
Similarly, in the (almost) old days, users not only set up Outlook for the initial use, but were also responsible for reconfiguring Outlook any time they were moved to a different mail server. Even if an e-mail went out warning users of imminent mailbox migration, busy workers paid little heed to anything having to do with the impending migration—until they came in one morning and found themselves locked out of Outlook. At this point, yet another helpdesk call ensued. The Autodiscover Web services solves this as well. When a user is moved from one mailbox server to another, Outlook recognizes that the old configuration is no longer valid, queries the Autodiscover Web service again, then reconfigures the client for the new mailbox. This whole process is completely transparent to the end user.

Reinventing the Calendar
In the past, most calendar logic resided in Outlook. Each meeting update was treated as a separate meeting invitation, which meant the notifications could be accepted out of order. If two meeting updates were sent in quick succession, and a recipient accepted the second meeting update before the first, he might have out-of-date information on his calendar. Furthermore, accessing Outlook information from Smartphones or PDAs could lead to differing free/busy information according to the method by which users viewed the data.
The solution was to move all calendar logic from the Outlook client to Exchange Server 2007. When a meeting request is received by the server the meeting is automatically placed on the calendar as tentative until the user chooses to accept or decline the appointment. If a time or location change to the meeting is received, the meeting will be automatically updated and the old invitation marked out of date so users will not accidentally accept it. This logic is consistent among all clients—Office Outlook, Outlook Web Access, Outlook Mobile, or another third-party client—so only the correct, most up-to-date meetings are present.
With the calendar so reliably accurate, it becomes possible to offer a new set of tools that can make users more productive. In the past, it was not easy to set up meetings with multiple busy employees. Users had to find free times for meetings manually, scrolling through days of information to find a suitable time for all attendees. Now the Outlook 2007 Scheduling Assistant, shown in Figure 2, will find the best time for everyone. After entering in the required and optional attendees and any desired rooms or resources, the Scheduling Assistant presents a color-coded view of the best days for a meeting. Once a day has been chosen, the same scheme is used to find the best time during that day.
Figure 2 Scheduling Assistant
Figure 2 Scheduling Assistant
The Scheduling Assistant prioritizes first required attendees, then optional attendees, and finally rooms and resources to determine the most optimal time for a meeting. The Scheduling Assistant also makes dealing with issues like flexible working hours and different time zones easy. Users can set their own working hours, defining when they arrive each morning and when they leave at the end of the day. Therefore a late sleeper who doesn't arrive at work until 9 or 10 in the morning can set their working hours to begin at this time. When others need to schedule a meeting, an earlier time such as 8 A.M. will not be suggested. If two groups in different time zones—one in London and one in Los Angeles for example—need to meet, only times that overlap in their working days will be suggested. Londoners will get a late afternoon meeting and Angelinos one first thing in the morning as opposed to a meeting time that is 2 A.M. for one party or the other.

Finding the Needle in the Haystack
Over the years the number of e-mail messages that arrive in the average user's Inbox has been growing at an alarming pace. Whereas once a typical workplace mailbox might contain only a couple hundred messages, some people now get that many messages in a day. Buried in this avalanche of messages is a wealth of information critical to the operations of any business. Finding the right pieces of information when you need them can be a substantial challenge.
The search capabilities of Office Outlook 2007 and Exchange Server 2007 are vastly improved over previous versions. Whether a user is in cached mode or online mode, the same search algorithms are applied to the message. In online mode the search query is sent to the Exchange server, where the search is executed. The Exchange Content Indexer, which is up to 35 times faster than the indexer in Exchange Server 2003, is used to ensure messages can be quickly found and retrieved. In cached mode, Outlook uses Instant Search, based on Windows® Desktop Search 3.0, consolidating the indexing engines and reducing CPU time. Search can be performed for text found in the subject, body, to and from fields or even attachments, as shown in Figure 3.
Figure 3 New Ways to Search for that Important E-Mail Message
Figure 3 New Ways to Search for that Important E-Mail Message

Anywhere Access with Unified Messaging
Exchange Unified Messaging, a built-in component of Exchange Server 2007, brings all types of business communications into a single Inbox, including e-mail, voice mail and fax. In traditional systems, voice mail is available through the telephone, e-mail through the computer, and fax through a dedicated fax machine. Exchange Unified Messaging allows the user to choose the best client for their needs at a particular time and place. Within Outlook, voice and fax messages appear in the Inbox right beside e-mail messages. The voice mail message, when selected, contains an embedded player so that the message can be played directly from the reading pane (see Figure 4).
Figure 4 Playing Voicemail Messages through Outlook
Figure 4 Playing Voicemail Messages through Outlook
Outlook also allows messages to be handled in context. Typically, when you listen to a voice mail, you jot down some notes on a piece of paper. These notes are not associated with the message and can be lost—hardly the model for efficient communication. With Exchange Unified Messaging, notes can be taken directly in Outlook. They persist with the message and will be indexed as searchable text, ensuring the notes always appear in the context of that voice mail message. Thus, they can be quickly found using the improved search of Exchange Server 2007 and Office Outlook 2007.
Outlook can also manage numerous voice mail options (see Figure 5), including allowing users to change their voice mailbox Personal Identification Number (PIN) from within the client. (PIN resetting is a major source of helpdesk calls for Voice Mail systems.) From their options in Outlook, users can have a new PIN generated and e-mailed to them. This PIN gives them one-time access into the Exchange Unified Messaging phone interface, Outlook Voice Access, and then they are asked to change their PIN.
Figure 5 Unified Messaging Includes a Pin Reset Option
Figure 5 Unified Messaging Includes a Pin Reset Option

To Secure and Protect
Organizations today face many threats to their messaging environment. Outlook and Exchange work together to fight against these threats. Spam is just one of those threats, and Office Outlook 2007 adds support for Outlook E-Mail Postmarks, a new way of combating spam. Outlook performs a small computation puzzle on outgoing mail and appends the answer to this puzzle to the message. Exchange then checks the results. While these computational puzzles take fairly little CPU time when performed on a one-off basis, as is typical with legitimate mail, if a spammer's machine had to create these puzzles for a mass e-mail, the tax on their machine would be extreme. Therefore, a message with an Outlook E-Mail Postmark is far more likely to be legitimate than one without, and Exchange takes this into consideration when determining the validity of each e-mail message.
In addition to old threats like spam, new regulatory regimes are changing the requirements for messaging systems. Regulations such as Sarbanes-Oxley, HIPPA, and Basel II dictate strict rules for the retention or destruction of records. To meet this emerging need, Exchange and Outlook introduce Managed Folders. These folders are set up by an administrator for messages of a particular type. In Outlook the end user sees these folders with a description of what should be stored in the folder (see Figure 6). Messages can also be sent automatically into a particular folder using Exchange transport rules. Once in a given Managed Folder, the messages retain the archiving, retention and expiration policies of that folder, and will automatically be treated in a way consistent with the policy of the organization.
Figure 6 Managed Folders Enforce Compliance Rules for E-mail
Figure 6 Managed Folders Enforce Compliance Rules for E-mail
Despite best efforts, occasionally compliance violations do occur. In these instances, quick identification and reparation is imperative. The fast search offered by Exchange becomes a pivotal tool for compliance officers to find messages that might be clues to identifying compliance violations. Searches can be done cross-mailbox in order to track down and correct such violations.

A More Manageable Environment
Together, Exchange Server 2007 and Office Outlook 2007 offer a much improved messaging system than previous versions. They bring all your vital business communications into one place, make them easy to find, easy to set up, and more secure and compliant out of the box. Together, they take collaboration and communication to the next level.

Michael Khalili is the Product Manager for Exchange Server responsible for Client Access and Unified Messaging. He is responsible for, amongst other things, Outlook-Exchange integration.


 Exchange Server 2007: Never Out of Touch with Exchange 2007
Exchange Server 2007 Joshua Trupin

Never Out of Touch with Exchange 2007
At a Glance:
  • Improvements to Outlook Web Access
  • Unified Messaging in Exchange Server 2007
  • Security with ISA Server 2006

Years ago, online communication was basic. You sent e-mail through POP3 and SMTP. Simple enough. Of course, needs change. Today, organizations need to give employees the
flexibility to work when and how they choose, and they need to do it securely. Microsoft® Exchange Server 2007 has several built-in features that address these needs, including Microsoft Office Outlook® Web Access (OWA) and Unified Messaging. Let's look at these two key components of Exchange Server 2007 and how they outpace previous versions to create a more powerful communications platform for today's enterprise.
OWA has been part of Exchange since version 5.5. To be honest, the first versions were a bit clunky. They were an answer to the problem of accessing an Exchange mailbox with a Web browser, but they did it without much of the functionality provided by Outlook.
With the improved OWA that came along with Exchange Server 2003, things started to look up for end users. The browser-based experience felt a lot more natural, almost like you were working in a client program. OWA for Exchange Server 2007 goes even further, with a complete rewrite of previous versions, allowing it to build upon other recent technical innovations like RPC over HTTP and ASP.NET 2.0.
For instance, consider how OWA does its rendering. Previous versions of OWA were rendered by the mailbox server, during the store process. This added extra load to the mailbox server, potentially causing other slowdowns in the system. Outlook Web Access 2007, in contrast, renders data through ASP.NET, on Client Access Servers (CAS). This moves the page processing off the mailbox server, improving performance.
The work performed on the front end has also changed. In previous versions of Exchange, the OWA HTTP requests would be proxied to the appropriate mailbox server. Exchange Server 2007 reduces traffic associated with OWA by fetching data directly from the appropriate mailbox server through RPC. This is the same process, more or less, that allows you to use Outlook on a client that's not on a virtual private network (VPN) connection into your enterprise. The mailbox information is then rendered by OWA. Exchange Server 2007 can also proxy OWA requests, but instead of sending them directly to mailbox servers, it will forward them to other CAS servers.
Figure 1 shows the architecture of OWA on Exchange Server 2007.
Figure 1 How OWA Accesses Exchange Server Resource
Figure 1 How OWA Accesses Exchange Server Resource

Administration
The revamped architecture of Exchange Server 2007 includes expanded options for administering OWA. You can use the Exchange Management Console or use the Exchange Management Shell. Configuration settings are stored in several places.
Exchange Management Console lets you access most of the possible configuration settings, such as enabling or disabling features using segmentation, attachment blocking, authentication, and accessing features like the calendar, Task List, and spellchecking. Anything that can be achieved in the Exchange Management Console can also be done using Exchange Management Shell tasks.
In addition, the Exchange Management Shell allows you to create and delete new virtual directories for OWA. You can also get finer control over per-user segmentation settings and change the default language setting, among others.
The Windows registry is the default store for forms-based authentication time-outs. (These settings are not available through Exchange Management Console or the Exchange Management Shell.) Through Active Directory® you can access OWA-specific settings like attachment blocking and segmentation. The IIS Metabase contains any settings (authentication, GZIP) that influence the behavior of IIS. And web.config holds ASP.NET settings such as "maxuploadsize," which controls the uploading of OWA attachments.

Authentication
OWA provides several choices for authentication at the server. After setup, security defaults to forms-based authentication. This, however, can be boosted in several ways as needed to conform to your organization's requirements. IIS will perform authentication against Active Directory to ensure that the user who is coming through has matching domain rights. You can also use basic authentication, digest authentication, RSA SecurID, or even SmartCard sign-in to access e-mail.
The strongest level of security for OWA is Windows Integrated, combining Kerberos and NTLM. In fact, Windows Integrated is required for Outlook Web Access 2007 Web Part access, as well as for CAS proxying between Active Directory sites. It also provides an extra benefit for users accessing OWA on an intranet—it provides single-sign-on with Windows client authentication, so you don't have to log onto OWA if you're already a trusted user.
If you use forms-based authentication, the default, OWA will automatically time out after a period of inactivity. Depending upon the user's choice of public or private machine (when asked on login), the timeout might be 15 minutes (for public machines), or it could be as much as 8 hours (for a private computer). The choice of public versus private machine is important for your users, because other security settings can also be configured differently depending upon this setting. You can, for instance, block attachments on public machines, or change authentication time-outs. Forms-based authentication is, basically, a four-step process as shown in Figure 2.
Figure 2 Authentication Process
  1. Clients send request to the server, unencrypted.
  2. Servers send an encrypted cookie back to the client.
  3. Servers keep discarding and creating new encryption keys, keeping the three newest keys in memory.
  4. Clients that send requests with an out-of-date encryption key which is still in memory get a new cookie encrypted with the newest key.
    —or—
    Clients that send requests with an out-of-date encryption key which has been discarded and timed out must log on again.

msExchQueryBaseDN is the mechanism used to limit certain users to see only a subsection of the address book in OWA. msExchQueryBaseDN is stamped on Active Directory user objects and points to an Address List (AL) or an Organizational Unit (OU). This AL will be used as the Global Address List (GAL) for the user, and the user will see a GAL including only user accounts in this OU. You can set msExchQueryBaseDN using LDAP.

Backward Compatibility
The heart of Outlook Web Access in Exchange Server 2007 is the CAS, and the CAS model has been built up to provide the best possible interoperability with servers running earlier versions of Exchange. To ensure that this works, you should always deploy or upgrade your CAS servers before upgrading their associated mailbox servers.
An Exchange 2007 server with the CAS role can be used to connect to mailboxes on Exchange 2000 and Exchange 2003 servers. RPC over HTTP and Exchange ActiveSync® will both continue to work with a partial site upgrade. OWA and WebDAV will also work if you access them through the /exchange, /public, or /exchweb virtual directories. You can create custom OWA virtual directories for these earlier versions on Exchange Server 2007 as well.

ISA Server 2006 Is Your Friend
Outlook Web Access takes a number of steps forward in Exchange Server 2007, but it can be improved even further by pairing it with Internet Security and Acceleration (ISA) Server 2006.
ISA Server 2006 is an integrated edge security gateway. At the same time that it helps protect your site from Internet-originating threats, it also provides secure remote access to applications and data. Since that's also the goal of OWA, pairing the two gives you that most fabulous of all computing concepts, synergy. (For more background information on ISA Server, take a look at microsoft.com/isaserver/2006.)
ISA Server 2006 provides several benefits to the harried Exchange administrator. For instance, the Web Publishing Load Balancing (WPLB) feature lets you balance loads using HTTP cookies. You can ensure client-server affinity even when the client IP changes. ISA Server 2006 also provides GZIP compression and decompression, allowing you to analyze traffic or offload compression from Web servers. With ISA Server 2004, you had to make a choice: you could either use HTTP compression with OWA, or you could use forms-based authentication with ISA Server. Happily, ISA Server 2006 supports both simultaneously.
In addition, you can perform OWA link translation through ISA Server. If someone puts an intranet link in OWA e-mail, ISA Server can translate it to a proper Internet link. This can be quite useful, but you have to remember to publish the intranet links to the same ISA array that is publishing OWA.
Perhaps most importantly, ISA Server 2006 performs preauthentication to help protect your network at the perimeter. Preauthentication is intended to ensure that no rogue HTTP traffic reaches your servers. If the traffic that is being directed to your CAS is in fact valid, ISA Server will rush it along from the perimeter to the domain. (Perimeter servers sit outside the intranet domain, blocking traffic that doesn't belong in there. Your CAS shouldn't be deployed in the perimeter domain; it needs to be on the intranet domain so that it can access Active Directory user accounts.)

Troubleshooting Outlook Web Access
Outlook Web Access 2007 features new tools that improve monitoring and troubleshooting. A monitoring task runs automatically on the CAS server. The task tests connectivity for each mailbox server, as well as testing OWA logons. Exchange Server 2007 also provides far more detailed Event Log messages and Performance Counters.
The Exchange Server Best Practices Analyzer (see our coverage at microsoft.com/technet/technetmag/issues/2006/01/TuneUpExchange) has been improved to alert administrators to relevant warnings and errors if OWA is not configured correctly. You can also produce OWA usage reports from IIS logs through the LogParser tool (covered in the October 2006 issue at microsoft.com/technet/technetmag/issues/2006/10/LogParser).

Unified Messaging
Where OWA addresses the needs of the mobile business, Unified Messaging consolidates different forms of messaging in a unified place (hence the clever name). Right now, e-mail lives on Exchange Server. Voice (phone calls, voice mail) is available on a telephony server. You get your e-mail on your desktop, and you pick up voice-mail through your phone. And don't even get me started on fax messages!
Exchange Server 2007 Unified Messaging lets an organization deliver e-mail, voice and fax data into a single inbox, where users can access it all through Outlook and OWA. Unified Messaging goes the other direction as well: users can access voice-mail, e-mail, contacts, and calendars over the phone with Outlook Voice Access.
For the systems admin, Unified Messaging simplifies messaging by combining your Exchange and telephony infrastructures to give you one store solution, one directory, and one transport. The overall level of complexity is reduced when each person has a single mailbox.
The best part of all? This can be done with your existing servers, and if you know Exchange Server 2007 and telephony the learning curve is pretty flat. The features of Unified Messaging are integrated with Exchange—you use the same security model and the same inboxes. You can even access your voice-mail and faxes through OWA.
Figure 3 shows the architecture of unified messaging in Exchange Server 2007 with the Unified Messaging server role installed. Take a good look at it (there may be a quiz later), and then let's look at the Unified Messaging architecture in more detail.
Figure 3 Unified Messaging Pulls Together Voice, Fax, and E-Mail Communications
Figure 3 Unified Messaging Pulls Together Voice, Fax, and E-Mail Communications
When you're setting up Unified Messaging in an enterprise, there are a lot of elements to be aware of. Exchange Server contains objects that represent telephony hardware. These objects can then have their relationships defined within the Exchange Server environment. The Exchange Server Unified Messaging system objects include the UM Server, UM Dial Plan, UM IP Gateway, and UM Hunt Group, as well as the UM Mailbox Policy and UM Auto Attendant. Let's take a gander at how each of these objects justifies its existence.
The UM Server object is created in Active Directory where the UM server role is installed. It lets an admin control the setup and properties of an Exchange Server 2007 Unified Messaging deployment. UM Dial Plan is the basic administrative unit within Unified Messaging, representing how extensions are used internally. All users within a Dial Plan can reach all others simply by dialing their extension. If you are tying together other locations, each one will typically have its own Dial Plan.
The UM IP Gateway object represents a physical Voice over Internet Protocol (VoIP) gateway, or a Session Initiation Protocol (SIP)- enabled IP Private Brach Exchange (PBX). A unified messaging server can receive calls from an IP/VoIP gateway device. A UM Hunt Group links an IP Gateway with a UM Dial Plan. After a Hunt Group is configured at the PBX, a UM Hunt Group is created in Active Directory to represent this. You can configure the Hunt Group to provide load balancing across IP gateways. These gateways, in turn, can provide load balancing by associating a single UM Hunt Group with multiple gateways. UM Mailbox Policies are analogous to voice mail Classes of Service. They include a Dial Plan and properties that set operating policies for users and groups.
The UM Auto Attendant can be associated with a single UM Dial Plan. It can be customized to perform several tasks, such as custom prompt recording and defining tasks for different days and times. Auto Attendants can be scoped using address lists.
Figure 4 shows how these six Unified Messaging objects work together.
Figure 4 Interaction of Unified Messaging Telephony Objects
Figure 4 Interaction of Unified Messaging Telephony Objects

Setting It Up
Once a user has an Exchange Server 2007 mailbox, the admin must enable it for Unified Messaging. To do this, the admin must associate the mailbox with a UM Mailbox policy and an extension number, both within the Dial Plan. The PBX must also be configured to route the call to Unified Messaging on an RNA (ring, no answer). The PBX will first route the call to the user's extension. Then, if RNA occurs, the call is routed to the VoIP gateway, and then on to the Unified Messaging server.
Unified Messaging is scalable. You can add as many circuit-switched PBX channels and UM servers as you need to achieve capacity. When you use VoIP gateways, IP-based calls will be delivered to Exchange Unified Messaging. Since this is new functionality for many businesses, you should do a full round of network analysis to make sure your IP network has the bandwidth to handle both data and voice.
A typical Exchange Server 2007 Unified Messaging server can handle between 50 and 200 concurrent IP calls and up to 200 incoming voice and fax calls (these are set to 100 by default). You can build out additional Unified Messaging servers if you need more capacity than this, or if you want to increase your fault tolerance.

Wrap-Up
So there you have it! Exchange Server 2007 provides two features, Outlook Web Access and Unified Messaging, that together provide huge benefits to an organization. Outlook Web Access lets you securely access e-mail from anywhere, and Unified Messaging allows users to access their e-mail, voice mail, fax messages, and other mailbox data that can end up in their Exchange 2007 mailbox.

Joshua Trupin is the Executive Editor of MSDN Magazine and TechNet Magazine. He has written numerous articles for MSJ, MIND, MSDN Magazine, and TechNet Magazine, as well as a book, Hoop Stats: The Basketball Abstract.


 Exchange Server 2007: More Powerful Journaling in Exchange 2007
Exchange Server 2007 David Strome

More Powerful Journaling in Exchange 2007
At a Glance:
  • Exchange Server 2007 journal rules
  • The new Journal Wizard
  • What’s in a journal report?

Have you ever had to journal e-mail messages sent to or received from one specific user, then ended up with the messages you were looking for, along with messages from the other 300 mailboxes that
reside on that mailbox store? Microsoft Exchange Server 2007 solves this issue, giving you the granular control you need.
Per-recipient journaling, available with enterprise Client Access Licenses (CALs), lets you target who to journal. Through the use of journal rules, you can now target only the recipients and senders you want to journal. You can narrow your focus to a single mailbox, or expand it to include, for example, all of the personnel in a sales department. And their mailboxes don’t need to be on the same server, in the same Active Directory® site, or even in an Exchange organization. Through Active Directory replication, your changes are automatically applied to all the computers running the Hub Transport server role in your organization.

How Journaling Works
In Exchange Server 2003, journaling was implemented on individual mailbox stores on each physical server. If you wanted to journal all of the mailboxes in your organization, you needed to configure journaling on each mailbox store. And if you wanted to journal messages for a single recipient, you either had to journal everyone on that user’s mailbox store or create a new mailbox store specifically for that user.
Journaling in Exchange Server 2007 makes use of the new role-based topology in Exchange. As shown in Figure 1, all messages are processed by Hub Transport servers when going to or coming from Mailbox and Unified Messaging servers, other Exchange systems, third-party applications, and the Internet. All Hub Transport servers contain a transport agent called the journaling agent, which is responsible for applying journal rules to messages. Since the journaling agent is located on the Hub Transport servers, it encounters and evaluates every message before the message reaches its recipient. The Journaling agent acts on messages after categorization—this ensures access to all of the message’s recipient and sender attributes, and it allows the agent to determine if the message was sent directly to a recipient or if it was received via distribution group expansion. It can also tell whether the recipient was on the To, the Cc, or the Bcc lines of a message that originated from within the Exchange Server 2007 organization.
Figure 1 Hub Transport Server Mail Flow
Figure 1 Hub Transport Server Mail Flow
The journaling agent applies administrator-configured journal rules to messages as they move through the Hub Transport server. These rules determine whether the agent will capture information about a message, forwarding this info along with the original message to a journal mailbox. This data is sent in a message called a journal report.
With earlier versions of Exchange, you had to apply configurations to multiple servers. However, when you create a journal rule in Exchange Server 2007, that change is applied to all Hub Transport servers in your organization via Active Directory. All Hub Transport servers, and therefore all Journaling agents, read the same configuration from Active Directory. Therefore, this blanket coverage ensures that all journaling agents apply the same journaling configuration.
Keep Active Directory replication times in mind when you create or modify journal rules, as a configuration change needs to be replicated throughout your organization and read by the Hub Transport servers. This could take several hours. To help you identify when journaling configuration is updated, Exchange logs an event in the security event log on each server.
Exchange Server 2007 makes sure that a journal report is never lost due to an unavailable journal mailbox, be it full, misconfigured, or offline. (This is particularly handy for helping you comply with various legal and regulatory requirements, since lost messages can result in noncompliance.) If a journal report can’t be delivered to a journal mailbox, the report remains in the Hub Transport server’s queue until the journal mailbox becomes available. Since this can result in rapidly growing queues, you should monitor the availability of your journal mailbox to make sure it’s operating correctly. If a journal mailbox will remain unavailable for an extended period, you can configure an alternate to receive reports that are sitting in the queue.

Deciding What to Journal
Before you begin to create journal rules, you need a clear idea of who you want to target and what types of data you want to journal. By specifying a user on a journal rule, you can select specific users for journaling. These users may, for example, be subject to regulatory requirements, or they may be involved in legal proceedings that require e-mail messages and other communications to be collected as evidence.
In addition to specifying which recipients to journal, you can also determine the scope of messages to journal, with options for Internal, External, and Global. Internal targets only messages within your organization; External targets messages where the sender or one of the recipients is outside; and Global targets both internal and external messages. (Note that the Global option journals all messages that pass through a Hub Transport server—even those that may have already been processed by rules using the Internal and External scopes.)
Exchange Server 2007 supports Unified Messaging, so you can configure Exchange to collect a user’s e-mail, voicemail, and faxes all in his Inbox. This also means you can journal all of this data—or choose to exclude certain data.
If you’re running Exchange Server 2007 with standard CALs, you can still use per-mailbox store journaling. However, if you want to use the new per-recipient journaling features, you must upgrade to the Exchange Server 2007 enterprise CALs.

Where to Place the Journal Mailbox
Once you have decided what to journal, you need to decide where to send the journal reports. If you have multiple sites, you need to consider your current or planned network topology when deciding where to place your journal mailboxes. Depending on the size of your organization and the number of mailboxes, journaling can result in a significant number of repetitive reports.
You don’t need to stick with mailboxes on Exchange, though. You can send journal reports to any valid SMTP address. This address can point to Exchange Hosted Services, to a third-party archival solution, or to any combination of these using a distribution group. Beware, however, that there are certain security implications pertaining to where you send journal reports.
Regardless of where you decide to place your journal mailbox, you must create a recipient object for it in Active Directory. This can be an Exchange Server 2007 mailbox, a mail-enabled contact that redirects mail to Exchange Hosted Services or a third-party archival solution, or a distribution list that contains both mailboxes and contacts.

What’s in a Journal Report?
When the journaling agent journals a message, it tries to capture as much detail as possible about the original message in a journal report. This report is then sent to the journal mailbox. The information is very important in helping you determine the intent of the message, its recipients, and its senders. For example, where recipients are identified—whether directly addressed on the To field, included on the Cc field, or simply a part of a distribution list—can help determine how the recipient is involved in the message’s discussion. The original message is included as an attachment. The screenshots in Figure 2 show one journal report that indicates a message sent to david@contoso.com has been forwarded to christine@contoso.com and another journal report that indicates a message sent to the Sales_Group@contoso.com distribution group has been expanded and the recipient lukas@contoso.com, who is a member of the Sales Group distribution group, received the message. Both reports contain the original message as an attachment, show that the original sender was brian@contoso.com, and have the subject "Sales forecast".
Figure 2 Journals Reports Showing a Forwarded Recipient and an Expanded Distribution Grou Recipient
Figure 2 Journals Reports Showing a Forwarded Recipient and an Expanded Distribution Grou Recipient
Exchange only classifies information that it knows is correct. Information that cannot be determined automatically is put into the appropriate fields in the journal report. Figure 2 explains the fields that are included in the body of the journal report.

All About Security
By default, all communication between computers running Exchange Server 2007 in the same Exchange organization is encrypted. This encryption includes journal reports. Exchange Server does a number of things to help reduce the risk of journal reports being tampered with:
  • Secure links are used between Hub Transport servers and Mailbox servers in the Exchange 2007 organization.
  • Journal reports are sent as "Microsoft Exchange" on behalf of the sender of the original message.
  • Sessions between the Hub Transport server and Mailbox server are authenticated.
  • Only authenticated connections are accepted when journal reports are sent between the Hub Transport servers and the Mailbox servers in the same Exchange 2007 organization.
When you create a journal mailbox, you must secure it since the mailbox contains messages sent to and from recipients in your organization. Some messages may be part of legal proceedings or subject to regulatory requirements, and there are various laws that require messages to remain tamper-free before they are submitted to an investigatory authority. To increase journal mailbox security, you should configure the journal mailbox to accept only messages from the Microsoft Exchange sender, and you should require that all messages sent to the journal mailbox be sent by authenticated senders. Figure 3 shows the message delivery restrictions configured on a journal mailbox. You can also use the following Exchange Management Shell command to configure these message delivery restrictions:
Set-Mailbox <Journal Mailbox Name>
-AcceptMessagesOnlyFrom "Microsoft Exchange" -RequireSenderAuthenticationEnabled $True
Figure 3 Message Delivery Restrictions
Figure 3 Message Delivery Restrictions
If you send journal reports to a journal mailbox outside of your Exchange Server 2007 organization, you must manually encrypt and secure the connection between your Exchange Server 2007 computers and the receiving server. You can do this by requiring the use of Transport Layer Security (TLS) between the two systems, requiring authentication on the receiving system, accepting only messages on the receiving system from the SMTP address of the Microsoft Exchange recipient (this SMTP address will be similar to Exchange_UMUnique GUID@contoso.com), and configuring the Active Directory contact used to forward the messages to accept messages only from the Microsoft Exchange recipient.

Implementing Journal Rules
Enough talking already. Let’s implement some rules! Journal rules in the journaling agent can be configured either in the Exchange Management Shell or in the Exchange Management Console. Each can be used to configure the recipient, scope, and journal mailbox settings. By default, journal rules are enabled when created. The value you specify for the journal e-mail address must be an existing recipient object in the Exchange Server 2007 organization. This recipient object can be a mailbox, a distribution group, a dynamic distribution group, or a contact that sends mail to an SMTP address.
When using the Exchange Management Console, the New Journal Rule wizard guides you through the creation of new journal rules. In the Organization Configuration section, select the Hub Transport server role. Then in the Action pane, click New Journal Rule. Here, you must supply values for the rule name, journal e-mail address, and scope. You don’t need to specify a value for the Recipient field if you want the journal rule to journal messages for all recipients in your organization. You can also choose whether you want the journal rule to be disabled when it is created. Figure 4 shows a sample configuration that will journal all messages sent by or to brian@contoso.com. Journaled messages are sent to a journal mailbox named Compliance Mailbox.
Figure 4 New Journal Rule Wizard
Figure 4 New Journal Rule Wizard
In the Exchange Management Shell, you can use these six cmdlets to administer the journaling agent:
  • New-JournalRule
  • Set-JournalRule
  • Get-JournalRule
  • Remove-JournalRule
  • Enable-JournalRule
  • Disable-JournalRule
The New-JournalRule cmdlet lets you create a new journal rule. The basic criteria for creating a new journal rule in this manner are the same as when using the wizard. You must specify values for the Name, Scope, and JournalEmailAddress parameters. Again, you don’t have to specify a value for the Recipient parameter if you want the journal rule to journal messages for all the recipients in your organization. And if you want the journal rule to be disabled when created, you must specify a value of $False for the Enabled parameter. To create a journal rule that has the same configuration as the one shown in Figure 4, you would use the following command:
New-JournalRule
-Name "Brian Smith Journal Rule"
-Recipient brian@contoso.com
-JournalEmailAddress "Compliance Mailbox"
-Scope Global
You use the Get-JournalRule cmdlet to view a list of all the journal rules you’ve created. Running this cmdlet on its own displays a summary list of all the journal rules configured in your organization and their configuration settings. If you want to view all of the available information about a journal rule, you can pipe the Get-JournalRule cmdlet to the Format-List cmdlet. Piping sends the output of one cmdlet to the next cmdlet so the second cmdlet can do additional processing on the output. The Format-List cmdlet, for example, displays all of the output it receives. To view all of the information available for the journal rule created earlier, you can use this command:
Get-JournalRule -Identity "Brian Smith Journal Rule" | Format-List
To enable or disable a journal rule, you use the Enable-JournalRule and Disable-JournalRule cmdlets. When using these cmdlets, you need to specify the journal rule name in the Identity parameter. For instance, to disable the journal rule we just created, use the following command:
Disable-JournalRule -Identity "Brian Smith Journal Rule"
The Set-JournalRule cmdlet lets you modify an existing journal rule. The name of the journal rule is used with the Identity parameter to tell the journaling agent which rule you are modifying. You then specify the Recipient, JournalEmailAddress, Scope, or any combination of these parameters along with the new value. For example, to modify the value stored in the JournalEmailAddress parameter for the rule we created, use the following command:
Set-JournalRule -Identity "Brian Smith Journal Rule" -JournalEmailAddress "Seattle Users Compliance Mailbox"
Finally, you can use the Remove-JournalRule cmdlet to remove an existing journal rule. You specify the name of the journal rule using the Identity parameter. To delete our sample journal rule, use the command:
Remove-JournalRule "Brian Smith Journal Rule"
When you do this, the Exchange Management Shell will ask if you really want to delete the journal rule. And here’s one last tip: with nearly all cmdlets, you can omit the Identity parameter label and just specify the parameter value.

David Strome has been a Technical Writer with the Exchange User Education group at Microsoft for just over one year. Prior to joining Microsoft in Redmond, WA, David spent approximately 10 years designing, implementing and administering Exchange Server installations at various companies in British Columbia, Canada. He can be reached at dstrome@microsoft.com.


 Exchange Server 2007: Fighting Spam and Phishing with Sender ID
Exchange Server 2007 Craig Spiezle and Alexander Nikolayev

Fighting Spam and Phishing with Sender ID
At a Glance:
  • Sender ID basics
  • Configuring Sender ID in Exchange Server
  • Benefits of authentication

Many identification and filtering technologies have been developed in response to the growing threat of spam. To be effective, they rely on asking certain questions about each e-mail message, such
as who sent it. Unfortunately, the fundamental question of who sent the message is not always easy to answer. E-mail is typically sent over the Internet without any authentication of the sender or the computers acting on the sender's behalf. The fact is, sending an e-mail message while pretending to be someone else is simple, and there is no automated method of detecting spoofed messages.
Simple Mail Transfer Protocol (SMTP), which is used to send and receive e-mail, was never designed to verify the sender of an e-mail message. With this technological loophole, any name and address can be inserted as the sender. As a result, content-filtering or anti-spam measures cannot rely on header information alone to ensure that messages actually come from where they say they do.
E-mail authentication specifically addresses this loophole. With authentication, both sending and receiving e-mail systems validate that messages come from the domains they claim to come from. This makes it easier for organizations to filter junk mail effectively, yet also ensure that legitimate e-mail reaches its intended recipients.
Today, there are two royalty-free approaches to e-mail authentication: the Sender ID Framework (SIDF) and DomainKeys Identified Mail (DKIM). SIDF is an Internet Protocol (IP)-based solution created from the merger of the Sender Policy Framework (SPF) and Microsoft Caller ID for E-Mail. In April 2006, the Internet Engineering Task Force (IETF) published the Sender ID specifications as RFCs 4405-4408. A coalition of industry and business stakeholders, including Microsoft, recommends the deployment of SIDF based on factors including business and technical value, stability, maturity, ease of deployment, minimal impact on outbound or inbound performance, and interoperability with the e-mail ecosystem for ISPs and enterprise environments.
DKIM is the merger of Yahoo! DomainKeys and Cisco Systems Inc.'s Identified Internet Mail (IIM) specifications. In January 2006, the IETF approved the creation of a DKIM working group and the specification is currently under IETF review.
While there is no perfect solution to counter spam, SIDF represents a significant industry initiative to counter spoofed domains. As a result, it is a key component of reducing spam and phishing attacks and increasing trust and confidence online. Organizations around the world are adopting SIDF at a rapid rate. Today more than 5.5 million companies and domain holders have published SPF records and more than 600 million users are protected by SIDF. Currently, over a third of the world's daily e-mail volume is now authenticated and SIDF-compliant.
The development and worldwide adoption of SIDF would not be possible without the contribution and support of many organizations and companies including AOL, Authentication and Online Trust Alliance (AOTA), Bell Canada, E-Mail Senders Provider Coalition (ESPC), CipherTrust, Cisco Systems, IronPort Systems, MarkMonitor, Port25 Solutions Inc, Sendmail, Symantec, TRUSTe, VeriSign, and others.

Understanding Sender ID
Some industry surveys indicate that over 95 percent of phishing scams come from spoofed domains and have spoofed sender e-mail addresses. This is where SIDF makes a huge difference with regard to anti-spam processing. By itself, SIDF will not solve the problem of spam, but it greatly contributes to minimizing the consequences of spam and phishing attacks. SIDF does not prevent junk e-mail from being sent. However, it does make junk e-mail much easier to detect. The framework helps e-mail senders protect their domain names, their reputations, and their brands. It provides a solid basis for making filtering decisions based on the reputation and e-mail behavior of the sender.
Sender ID seeks to verify that every e-mail message originates from the Internet domain from which it claims to have been sent. This is accomplished by checking the address of the server sending the e-mail against a registered list of servers that the domain owner has authorized to send e-mail. Verification is performed automatically by the Internet service provider (ISP) or the recipient's mail server before the message is delivered to the user's Inbox.
Note that Sender ID, or any other authentication mechanism, does not replace content filtering systems. Neither SIDF nor DKIM scans actual message content. Instead, authentication notifies the inbound mail system of whether the message can be validated as coming from the claimed sender. Because most spam and phishing exploits do not actually come from the domain shown, this approach can help automatically identify these messages and separate them from the incoming mail stream.
Within SIDF, the SPF record provides a simple text record of all outbound mail servers associated with a domain, along with their respective IP addresses. An organization publishes the SPF record to its DNS server zone file, which is then checked by recipient mail servers. Setting up an SPF record is fast, easy, and free. The Sender ID Framework SPF Record Wizard provides a step-by-step process for surveying a domain's mail servers and creating customized records ready for posting. (Details on publishing an SPF record are available at microsoft.com/senderid. The wizard can be accessed directly at microsoft.com/senderid/wizard.)
The receiving SMTP mail server pings the domain's zone file in the DNS for the existence of an SPF record. Once found, the IP address of the sending server is checked against the IP addresses listed. If there's a match, the message is validated as authentic. If, on the other hand, the SPF record on the sender's domain does not match the IP address the message came from, it fails, resulting in a negative score and potential placement in the junk mail folder. Figure 1 shows this process.
Figure 1 Checking SPF Records for Incoming Messages
Figure 1 Checking SPF Records for Incoming Messages
After the message has been tagged, SIDF enables the recipient mail server to analyze the message based on past behaviors, the reputation of the sender, its content, and other criteria that can be defined as needed. This ability provides extra safeguards. For example, spammers can register look-alike domains and published SPF records to deceive a user and receiving network into thinking the mail is from a legitimate sender. And even though the e-mail message may pass the authentication check, the sender's reputation as a spammer is enough to block, junk, or delete the message.

Sender ID Deployment Options
Outbound e-mail authentication via SIDF does not require any infrastructure changes or software updates and is not specific to client or server software. However, organizations that want to incorporate inbound authentication to protect their company and employees will need to update their inbound systems and Message Transfer Agent (MTA), and deploy software or an appliance that supports SIDF. The majority of leading commercial and open-source MTAs and anti-spam vendors, including Microsoft® Exchange Server and Exchange Hosted Filtering, offer integrated solutions.
Microsoft has integrated SIDF into all of its messaging products, including Microsoft Exchange Server 2003 Service Pack 2 (SP2), Exchange Server 2007, Microsoft Exchange Hosted Filtering, Microsoft Windows Live™ Mail, MSN® Hotmail®, Outlook® Express, and the Office Outlook messaging and collaboration clients.
Even Microsoft is not exempt from receiving spam, however. The average daily incoming traffic at Microsoft is approximately 15 million messages, and during recent spam attacks we've seen as much as two to four times that volume. In such an environment, the route to successful spam protection lies in vigilant implementation of the technologies available.
Microsoft is running an internal Exchange Server anti-spam solution based on the Sender ID filter in Exchange Server 2003 and the Sender ID agent in Exchange Server 2007. In both versions of Exchange Server, the Sender ID status is based on the evaluation of the Sender ID record located on the corresponding DNS servers. The actual domain is determined by examining the RFC 2822 message headers in the following priority:
  1. Resent-Sender
  2. Resent-From
  3. Sender
  4. From
The sender's domain (or the entity most recently responsible for injecting a message into the messaging stream, because e-mail systems can legitimately forward mail on behalf of other mail servers) is determined by locating the first definition of the headers just mentioned in the order defined. If none of these headers are found, SIDF will use the SMTP RFC 2821 MAIL FROM value.

Configuring Sender ID
In Exchange Server 2007, the Sender ID agent can be enabled on servers that have the Edge Transport role installed. If the Sender ID agent is enabled, it will filter messages that are coming through the receive connectors-all incoming (from external sources) non-authenticated traffic will be subject to Sender ID processing. In Exchange Server 2003, the Sender ID status is persisted server to server in the EXCH50 blob; in Exchange Server 2007 it has also been added to the message metadata. At the final destination both are converted to a MAPI property for future client consumption.
The Sender ID Filtering configuration in Exchange Server 2003, available under the Message Delivery Properties, consists only of specifying how Exchange Server handles failed validation.
To enable Sender ID in Exchange Server 2007, just open the Exchange Management Console for a server with the Edge Transport role configured, select the Anti-spam tab, then Sender ID, and in the Actions pane click Enable or Disable to toggle the Sender ID agent as shown in Figure 2. Additionally, you can use the Exchange Management Shell to enable or disable Sender ID as shown in Figure 3.
Figure 2 Exchange Management Console Controls for the Sender ID Agent in Exchange Server 2007
Figure 2 Exchange Management Console Controls for the Sender ID Agent in Exchange Server 2007
Figure 3 Enabling Sender ID using the Exchange Management Shell
Figure 3 Enabling Sender ID using the Exchange Management Shell
The Sender ID Properties dialog box will reflect the status of the agent, either enabled or disabled. The Action tab provides very similar options to those available in Exchange Server 2003 (see Figure 4).
Figure 4 Sender ID Agent Action Properties in Exhange Server 2007
Figure 4 Sender ID Agent Action Properties in Exhange Server 2007
In terms of Sender ID implementation and functionality, there is no major difference between Exchange Server 2003 and Exchange Server 2007 and both Sender ID filter (in Exchange Server 2003) and Sender ID agent (in Exchange Server 2007) use the same underlying algorithm for extraction and verification. The range of possible actions is also the same: Delete Message (silent delete), Reject Message (500-level SMTP protocol response), and Stamp the message with the Sender ID result and continue processing. This latter option is the Default action in both versions of Exchange Server.
In Exchange Server 2007, both FAIL and TempError status codes can trigger a Reject Message action (in Exchange Server 2003 only a FAIL status code can trigger this action). The Sender ID agent needs to be explicitly configured to trigger the Reject Message action on messages with a TempError status code because, by default, those messages will be accepted into the system, stamped with Sender ID result, and then processed.
The configuration option for this is not exposed through the graphical user interface so you must use the Exchange Management Shell to configure Sender ID actions for TempError status code. For example, to force the Sender ID agent to reject messages with a TempError status code, run the following Sender ID agent cmdlet:
Set-SenderIDConfig -TempErrorAction Reject
The range of Sender ID status results and possible actions in Exchange Server 2007 is similar to that of Exchange Server 2003 Sender ID filter and is shown in Figure 5.
Figure 5 Sender ID Status and Actions in Exchange Server 2007 (Close Figure 5)

Sender ID Status | Description | Actions
Neutral | Published Sender ID data is explicitly inconclusive | StampStatus
Pass | IP address is in the permitted set | StampStatus
Fail | IP address is in the not permitted set | StampStatus, Delete, or Reject
SoftFail | IP address may be in the not permitted set | StampStatus
None | No published data | StampStatus
TempError | Transient error such as unavailable DNS server | StampStatus or Reject
PermError | Unrecoverable error such as an error in the record format | StampStatus
Exchange (if configured to do so) will delete only messages that failed Sender ID verification. No other results will trigger message deletion. Incoming messages that resulted in Neutral, None, SoftFail, TempError, or PermError status assignment will be appropriately stamped and passed on for further anti-spam verification. In some cases, when the message is hopelessly malformed and the FROM IP address is missing, the Sender ID status cannot be stamped on the message. In this situation the message is not discarded or rejected. Rather, it will be passed on for further processing without the Sender ID status set and an appropriate event will be logged to raise awareness.
In Exchange Server 2007 it is easy to define sender domains and Exchange Server recipients that should be excluded from Sender ID verification. Again, this configuration option is available through the Exchange Management Shell only. For example, to exclude domain contoso.com from Sender ID filtering, run the following command:
Set-SenderIDConfig -BypassedSenderDomains contoso.com
Similarly, to exclude messages sent to specified Exchange Server recipients from Sender ID filtering, run the following command:
Set-SenderIDConfig -BypassedRecipients user@contoso.com
In Exchange Server 2007, Sender ID configuration values set via Exchange Management Shell cmdlets but not available through the graphical user interface can be easily obtained by running the Get-SenderIDConfig command in the Exchange Management Shell as shown in Figure 6.
Figure 6 Sender ID Configuration cmdlets
Figure 6 Sender ID Configuration cmdlets
The Exchange Management Shell can be used for quick manual verification of the IP address and the corresponding domain name. To check the Sender ID status, run the following command:
Test-SenderID -IPAddress <IPAddress>-PurportedResponsibleDomain <SMTPDomain>
If the domain does not exist, you will see a result similar to Figure 7.
Figure 7 Verify Sender ID Status of an Address
Figure 7 Verify Sender ID Status of an Address

Benefits of Sender ID
Aside from its obvious benefits in authenticating e-mail messages and reducing the amount of spam users receive, SIDF as a protocol offers other advantages. During its development, Microsoft focused on several key goals, including performance, cost, deployment and scalability, and interoperability.
SIDF first authenticates an e-mail sender and allows for the application of the sender's reputation score. As a result, it has the potential to significantly decrease spam and spoofed e-mail messages, and improve deliverability of legitimate e-mail from known and trusted senders. Creating an SPF record is free, so any organization that wants to be SIDF-compliant can do so easily. Organizations wishing to comply with the provisions of a new standard should be able to do so. With SIDF, compliance is as easy as publishing an SPF record.
SIDF has been developed to operate within existing software wherever possible. It can be used with a broad range of e-mail architectures and client and server software. To be effective, any authentication service must be able to meet the needs of the largest ISPs as easily as the smallest home mail server. SIDF can support from one e-mail server to thousands, plus those who outsource their e-mail servers to another organization.
On April 18-19 2007, Microsoft and an industry consortium of over 30 organizations and partners are hosting the Authentication & Online Identity Summit in Boston, MA. This two-day program will review case studies and offer a technical review of Sender ID, detailing the business value of e-mail authentication. For more information, visit aotalliance.org/summit2007. In addition, for information including tools and resources, you can visit microsoft.com/senderid and microsoft.com/exchange.

Craig Spiezle is the Director of Technology Strategy and Planning for Windows Live Safety at Microsoft. As the Product Manager for e-mail authentication, Craig has been a driving force in bringing Sender ID to the industry. Starting at Microsoft in 1992, Craig has held several management positions including International marketing, product support strategies, OEM, and emerging market development.
Alexander Nikolayev is the Program Manager at Microsoft in charge of server-side protocols, Transport Core, and anti-spam components for Exchange Server and Windows Server. He holds an MBA from the University of Mary. Read Alexander's posts on the Exchange team blog at blogs.technet.com/exchange.


 From the Editor: Reaching Out
From the Editor
Reaching Out

TECHNET MAGAZINE IS MORE than just this fabulous publication you hold in your hands today. Our team has been working furiously to create an innovative new Web site that will make our content accessible to more than a billion people around the world. Beginning with the November issue, every article will be available online in seven languages: English, Spanish, French, German, Russian, Simplified Chinese, and Brazilian Portuguese. By adding six additional languages (we were already online in English, you know), we increase the number of people who can use our content in their native tongue by over 1.6 billion. That’s incredible!
As if this weren’t enough, we’ve added a passel of features to make the site more usable. Our new customizable RSS feeds let you select our content by date, column, author, or topic. You can even mix and match. Visit technetmagazine.com/rss to start creating your own feeds today.
We’re also introducing new topic pages. If you want to read all about what IT pros need to do to get ready for Windows Vista, just go to technetmagazine.com/vista. If you’re a postmaster, you should definitely check out technetmagazine.com/exchange. We’ve got 32 topics of interest at technetmagazine.com/topics, with more to come. (As of November, we had 43 articles on security waiting patiently for your attention.)
And that’s not all. Now you can get even further insight into the magazine and the world of the IT pro. Joshua Hoffman, TechNet Magazine’s technical and acquisitions editor, runs the magazine’s blog at blogs.technet.com/tnmag. And I host a monthly podcast on our site, at www.technetmagazine.com/podcasts. In each installment, I’ll be talking to the people who write the articles you read, going beyond the words to explain the why behind the piece. You’ll love hearing our top authors talk about their articles and the work they do.
We’re going all out to make TechNet Magazine the most useful resource on earth for IT pros. It’s been a good 2006, but we’d like to hear what else we could do for you. If you have any comments, drop us a line! Our editorial hotline is tnmag@microsoft.com. The suggestion box is always open.
—JOSHUA TRUPIN

Thank you to the following Microsoft technical experts: Kevin Allison, Jessica Arnold, Sebastian Bengochea, Naveen Chand, Li Chen, Alpa Dave, Jill Frank, Mihai Jalobeanu, Darcy Jayne, Harry Katz, Kerry Landen, Todd Luttinen, Brett Muzzey, Bruce Payette, Tony Smith, Shawn Thomas, Jim Truher, and Scot Vidican.


 Toolbox: New Products for IT Pros
Toolbox
New Products for IT Pros
Greg Steen

Stress Test Your Web Apps
WAPT Web Application Stress Tool
www.loadtestingtool.com
Knowing how the latest release of your Web application will perform is crucial to right-sizing your infrastructure and avoiding those high-pressure moments when the CEO asks you why she is getting a "Server Too Busy" response when checking out the site. And if you are on the hunt for a relatively low-cost, easy-to-use Web application stress and performance testing tool, you might want to check out WAPT 4.0 from SoftLogica LLC.
The application runs on most versions of Windows®, but you will want to use Windows 2000, Windows XP, or Windows Server® 2003 to gain the full potential because of the socket limitations on earlier versions. As with most performance testing applications that employ virtual users, the more memory, faster processor, and fatter network connection you have, the more virtual users you will be able to simulate with WAPT.
The documentation states a limit of around 2,000 virtual users for a PIV 2.4 with 512MB of RAM and a gig-e interface, but that is dependent on the type of application you are testing and how you set up your test profile. Another factor that will vary your testing is the delay between requests, which WAPT lets you set as user think time in a virtual user's profile.
Once you have finished setting up a virtual user profile (you can have many per-testing scenarios), you can create the test plan for that profile. WAPT divides the test sequence into three sections: Initial, Main, and Final Actions. These are more logical separations than functional, but they do suggest splitting your login or registration function, as an example, into the initial section and log out action in the final section. WAPT allows you to either enter the requests manually or use the integrated browser recorder to track how you would like users of this profile to surf through your site. Next, you can move onto the general settings to determine what the load should be for this test and whether you would like it to be fixed or ramp up to volume over a specified time period. During the test, WAPT can give you a real-time view of performance statistics for your test scenario. And after you have finished, you can save the results as HTML or CSV for future review.
Price: $350 per machine.
Knowing how the latest release of your Web application will perform is crucial to right-sizing your infrastructure and avoiding those high-pressure moments when the CEO asks you why she is getting a "Server Too Busy" response when checking out the site. And if you are on the hunt for a relatively low-cost, easy-to-use Web application stress and performance testing tool, you might want to check out WAPT 4.0 from SoftLogica LLC.
The application runs on most versions of Windows®, but you will want to use Windows 2000, Windows XP, or Windows Server® 2003 to gain the full potential because of the socket limitations on earlier versions. As with most performance testing applications that employ virtual users, the more memory, faster processor, and fatter network connection you have, the more virtual users you will be able to simulate with WAPT.
WAPT Web Application Stress Tool
WAPT Web Application Stress Tool
The documentation states a limit of around 2,000 virtual users for a PIV 2.4 with 512MB of RAM and a gig-e interface, but that is dependent on the type of application you are testing and how you set up your test profile. Another factor that will vary your testing is the delay between requests, which WAPT lets you set as user think time in a virtual user's profile.
Once you have finished setting up a virtual user profile (you can have many per-testing scenarios), you can create the test plan for that profile. WAPT divides the test sequence into three sections: Initial, Main, and Final Actions. These are more logical separations than functional, but they do suggest splitting your login or registration function, as an example, into the initial section and log out action in the final section. WAPT allows you to either enter the requests manually or use the integrated browser recorder to track how you would like users of this profile to surf through your site. Next, you can move onto the general settings to determine what the load should be for this test and whether you would like it to be fixed or ramp up to volume over a specified time period. During the test, WAPT can give you a real-time view of performance statistics for your test scenario. And after you have finished, you can save the results as HTML or CSV for future review.
Price: $350 per machine.

Book Review
Content Networking Fundamentals
www.ciscopress.com
Content is king and Content Networking Fundamentals (Cisco Press, 2006) by Silvano Da Ros helps you get on the fast-track of enhancing your content-delivery infrastructure. The book labels itself as "a comprehensive introduction to the theory and practical applications of content networking" and that is truly what it is. Though obviously biased towards Cisco's networking gear, the book provides a good sense of the different components of content networking along with practical guidelines for implementing the principles behind content networking that are applicable regardless of the types of content networking products in your infrastructure.
The book begins with a high-level overview of the different elements of content networking (which is really enhancing the performance and delivery of layer 4 through 7 communication of the OSI model). Then it delves into the nuts and bolts of major content presentation formats such as XML, HTML, WAP, and content transformation through XSLT at the network edge along with a section on delivery of streaming media. Finally, it gets into scalability and availability with discussions on server load balancing (both global and local), firewall load balancing, and use of content caching devices such as Cisco's content services switch.
If you use Cisco's content services switches in your infrastructure, you will find the chapter on switching secured content interesting. It will show you the details of configuring the content services module for secure sockets layer along with configuring URL and header rewriting. Generally, networking pros of any brand-loyalty will find the chapter on QoS useful for understanding different queuing mechanisms and congestion management.
In summary, though definitely heavy on the Cisco hardware implementations (after all, it is published by Cisco Press), this book will help you turn the buzz words of "content networking" into practical and applicable methodologies for your content delivery infrastructure.
Price: $55.00.

Monitor Your Services
WebWatchBot
www.exclamationsoft.com/webwatchbot/default.asp
One thing that can make or break an IT professional is his ability to detect (and hopefully fix) issues before end users in the organization detect them. Therefore, one of the most important tools an IT pro can have is an effective monitoring system that can delve into the different services you provide in order to verify that they are running as they should. ExclamationSoft's WebWatchBot can help you get there.
This application can monitor every common service you can think of. Instead of just verifying that a service is running, WebWatchBot takes the process one step further. It actually tests services to make sure they are functioning properly. For example, FTP verification can actually upload and download a file, DNS monitoring can execute any of 40 different query commands and search for a string in the response, and POP3 and SMTP tests actually log in and report on mail performance.
WebWatchBot
In addition, WebWatchBot lets you chain a sequence of steps in a transaction and will report any errors or unacceptable latency (according to the thresholds you set) for the transaction as whole. The application also will monitor HTTP and HTTPS URLs and can check for specified content. You can even set up database monitors that will execute queries and check the data against a templated response.
Price: $495.95 per single-user license.

Greg Steen is a technology professional, entrepreneur, and enthusiast. He is always on the hunt for new tools and methods to help make operations and development easier for IT professionals.


 Windows PowerShell: Simple Commands. Powerful Administration
Windows PowerShell
Simple Commands. Powerful Administration
Don Jones
This column is based on a prerelease version of Windows PowerShell. All information herein is subject to change.

It's been a long time coming, but Windows PowerShell is almost ready to launch. That means it's time for Windows administrators to start taking notice. Windows PowerShell offers what is perhaps the easiest and most flexible way to automate a wide variety of administrative tasks, making you more efficient and more effective in your work.
But even more importantly, Microsoft is building the graphical administrative consoles of products like Exchange Server 2007 and System Center 2007 on top of Windows PowerShell™. This means you'll be able to perform almost any administrative task from within Windows PowerShell. Microsoft plans to do the same with the administrative capabilities of more products over time. Thus, Windows PowerShell could eventually become the first all-purpose tool for administering nearly any Microsoft server product. To help you get started, I'll discuss Windows PowerShell regularly in this new column. Be sure to download a copy of the software.

Power and Simplicity
As the name implies, Windows PowerShell is a shell, not unlike the Command Prompt (Cmd.exe) that's been around since Windows NT® 3.1. Cmd.exe isn't going away, but with the availability of Windows PowerShell, there are few reasons to continue using Cmd.exe.
Using Windows PowerShell isn't all that different than using Cmd.exe—except that Windows PowerShell is, well, more powerful. Like Cmd.exe, Windows PowerShell has a built-in scripting language, although it is much more flexible than Cmd.exe's primitive batch language. How flexible? With Windows PowerShell, you can automate extremely complex tasks with a language that includes only about a half-dozen built-in keywords.
Now that I've mentioned scripting, I should probably touch upon security. Windows PowerShell benefits from what Microsoft has learned about security over the past decade plus. By default, Windows PowerShell won't run scripts; it can only be used interactively to run individual commands. If you do enable scripting, you can direct Windows PowerShell to run only digitally signed scripts. All of this is to help ensure that Windows PowerShell doesn't become the next VBScript—a great language that has been frequently misused to create malicious scripts. VBScript isn't going away either, but you'll likely find that Windows PowerShell is easier to use for many different tasks.
You can do most anything with Windows PowerShell that you would have done with Cmd.exe. For example, you can run ipconfig and you'll get the same familiar output. But Windows PowerShell introduces a whole new set of commands that aren't external executable files. These cmdlets (pronounced "command-lets") are built right into Windows PowerShell. (For a look at some of the most useful cmdlets to get you started using Windows PowerShell, see the sidebar "Top 10 Cmdlets to Start Using Immediately".)
Top 10 Cmdlets to Start Using Immediately
  • Get-Command retrieves a list of all available cmdlets.
  • Get-Help displays help information about cmdlets and concepts.
  • Get-WMIObject retrieves management information by using WMI.
  • Get-EventLog retrieves Windows event logs.
  • Get-Process retrieves a single or list of active processes.
  • Get-Service retrieves a Windows service.
  • Get-Content reads in text files, treating each line as a child object.
  • Add-Content appends content to a text file.
  • Copy-Item copies files, folders, and other objects.
  • Get-Acl retrieves access control lists (ACLs).
For a complete list of cmdlets that ship with Windows PowerShell, go to windowssdk.msdn.microsoft.com/en-us/library/ms714408.aspx.
All cmdlets are named with a standard verb-noun format, making them easy to learn and remember. For instance, running the Get-Command cmdlet will list all the available cmdlets. Perhaps the most useful cmdlet for an administrator is Get-WMIObject. Say you want to find out what service pack Server2 is running. Simply run:
Get-WMIObject Win32_OperatingSystem –Property ServicePackMajorVersion
 –Computer Server2
To discover this same information using VBScript, you would have to write several lines of code. Other cmdlets let you work with services (Start-Service, Stop-Service, and so forth), processes (Stop-Process and others), files (Rename-Item, Copy-Item, Remove-Item, Move-Item, for example), and much more. Many of these cmdlets even have shortcut names, called aliases. In the case of Get-WMIObject, you can just type gwmi. Running Get-Alias will provide you with a list of these shortcut names.

Why Object Oriented Matters
Built on the Microsoft® .NET Framework, Windows PowerShell is completely object-oriented. Usually, only a software developer would get excited about that, but in this case, the object orientation results in a huge time savings for administrators. This is because administrators can now work directly with rich objects right within a text-based shell. Consider this example:
Get-Process | Sort-Object pm –desc | Select-Object –first 10
This is just a single line with three different cmdlets separated by pipes (more on this in a moment). The first cmdlet retrieves all running processes and then passes, or pipes, those objects to Sort-Object. This second cmdlet sorts on each process object's pm (meaning Physical Memory) property, placing them into descending order. The sorted collection of process objects is then piped to Select-Object, which picks the first 10 and displays them. The result? This simple line displays the top 10 physical memory consumers on the machine, as shown in Figure 1. This is an extremely efficient way to take a quick look when doing some troubleshooting.
Figure 1 Using a Simple Cmdlet to Troubleshoot
Figure 1 Using a Simple Cmdlet to Troubleshoot
The use of pipes (the vertical line character typically located on the backslash key on U.S. keyboards) is an essential part of what makes Windows PowerShell so capable. Using this character, you can pass (or pipe) objects from one cmdlet to another, allowing each to further refine the results, format them for display, and so on. This mechanism works because every cmdlet returns one or more objects, rather than pure text, giving subsequent cmdlets the full object to work with.
The use of objects in Windows PowerShell is pervasive, all the way down to its variables. And you don't have to declare variables up front; you can just start using them by placing a dollar sign ($) before a variable name. While it's not required, you can also tell Windows PowerShell what type of data you want to put into the variable. That lets Windows PowerShell map the variable to one of the extremely powerful .NET Framework types, giving you a lot of additional built-in functionality. For example, suppose you want to prompt for a computer name and retrieve the service pack version from that computer, but you don't know if the person typing in the computer name will include two backslashes (such as \\Server2). Since you know that the Get-WMIObject cmdlet doesn't need the backslashes, you can save the computer name into a String variable and use the Replace method to replace backslashes with empty strings, as shown here:
[string]$c = Read-Host "Enter computer name"
$c = $c.Replace("\","")
    Get-WMIObject Win32_OperatingSystem
    –Property  ServicePackMajorVersion
    –Computer $c
The value for the –Computer parameter has been provided in the $c variable. That variable was initially created as a string, so it picked up all the functionality of the .NET Framework String type, including the Replace method. Of course, learning about all of these capabilities will take a while, but you should find them easy to pick up through examples. Windows PowerShell itself helps to simplify learning. For example, if you type $c = $c. (don't forget the period mark) and press Tab, Windows PowerShell will display Clone(), the first method of the String type. If you keep pressing Tab, Windows PowerShell will cycle through all the available methods. Essentially, when you do this, Windows PowerShell is showing you what it knows how to do with a String!
Here's a tougher task. Read a list of computer names from a file, with one name per line, and show each computer's service pack number. In VBScript, this task would require a dozen or more lines of code. In Cmd.exe, you would have to use a complicated batch file. In Windows PowerShell, this task takes just one line:
Get-Content "c:\computers.txt" | foreach
{ $_; gwmi Win32_OperatingSystem -prop
ServicePackMajorVersion -comp $_ }
The Get-Content cmdlet reads the contents of C:\Computers.txt. Each line of the file becomes an object in its own right. This collection of objects—computer names, that is—is piped to the foreach command, which is really just an alias for the ForEach-Object cmdlet. The commands inside the curly braces are repeated once for each object that is piped in—for this example, that means they run once for each computer name. The special $_ variable will contain the current object (that is, the current computer name).
Inside the curly braces are actually two commands: The first simply displays the current computer name by outputting the contents of $_. The second is the now-familiar gwmi. The result is a list of service pack versions for every computer listed in the file. All this was done with one relatively straightforward line of commands.
Notice that the –Property and –Computer parameter names have been abbreviated. Windows PowerShell only requires enough to uniquely distinguish the parameter names.

Readability and Reuse
Writing a single line of commands and parameters, however, doesn't help with readability. Windows PowerShell lets you break this into something more readable, which you can type directly into the shell without ever writing a script. Here's how it might look:
PS C:\> $names = get-content "c:\computers.txt"
PS C:\> foreach ($name in $names) {
>> $name
>> gwmi Win32_OperatingSystem -prop ServicePackMajorVersion -comp $name
>> }
>>
This time, the contents of the file are stored in the variable $names. This example still uses foreach, but it isn't being input through a pipeline, so you have to tell it which collection of objects to loop through and what variable to store each object in—the part that says ($name in $names). Everything else is pretty much the same, and as soon as you hit Enter, the code is executed and the results displayed.
If you want to use this same code repeatedly, you can simply make a function out of it. Once again, type this directly into the shell:
PS C:\> function Get-ServicePacks ($file) {
>> $names = get-content $file
>> foreach ($name in $names) {
>> $name
>> gwmi win32_operatingsystem -prop servicepackmajorversion -comp $name
>> }
>> }
>>
As you can see, not much has actually changed. This simply encloses the previous example in a function named Get-ServicePacks (in keeping with the Windows PowerShell verb-noun naming convention). The function now has an input parameter named $file, which has been substituted in the Get-Content cmdlet so that a different file can be specified when the function is run. Now that the function is defined, you can simply run it by calling its name, almost like a cmdlet, and passing the input parameter:
PS C:\> Get-ServicePacks c:\computers.txt
Figure 2 shows the results.
The downside here is that this function will only exist for as long as that instance of Windows PowerShell is running. Once you close the shell, the function vanishes. You can copy the function into your Windows PowerShell profile, which is a sort of auto-run script that executes each time Windows PowerShell starts. Doing so would make the function available in every Windows PowerShell window you open. Or, if you want, you can make the function into a standalone script, which you can then execute simply by typing the script's path and file name.
Figure 2 Results of Running the Get-ServicePacks Function
Figure 2 Results of Running the Get-ServicePacks Function

The World is a File (or Folder)
There's more to Windows PowerShell than just functions and cmdlets. Let's look at file management as a quick example of what else is in store. You're probably more than familiar with drive and folder navigation in Cmd.exe—type C: to switch to the C drive, and type cd \test to change into the C:\Test folder. Windows PowerShell works in exactly the same way, although cd is just an alias for the Set-Location cmdlet.
Try running Get-PSDrive, the cmdlet that lists all available drives. In addition to the usual C:, D:, and perhaps A: drives, you'll also find one named Cert, another named Env, and others named HKCU and HKLM. Windows PowerShell actually exposes many different types of storage resources as "drives," making things like the local certificate store, environment variables, and registry available through a familiar file-like navigational interface.
Change to the HKEY_LOCAL_MACHINE registry hive by typing Set-Location HKLM: (or cd hklm: if you prefer the shortcut) and hitting Enter. Then run cd software\microsoft to change into the SOFTWARE\Microsoft key. You can use dir—an alias for the Get-ChildItem cmdlet—to list the sub-keys in this portion of the registry. If you want to remove a key, use del to delete it as if the key were a file or folder. (Be very careful though—serious problems might occur if you remove required keys or modify the registry incorrectly.)
All of this flexibility comes from providers, which map resources (like the registry and certificate store) into a format that looks like a file system. Microsoft plans to extend Windows PowerShell through additional providers, giving you the ability to, for example, navigate an Exchange Server store as if it were a file system. This is a very important technique in that it takes the huge variety of repositories used by Windows and makes them all appear identically, plus it makes them all manageable through a system of commands and techniques that you're already familiar with.

Designed for Safety
I've already mentioned that Windows PowerShell was designed with security and safety in mind. The primary security feature in Windows PowerShell is its execution policy. By default, this policy is set to Restricted, which you can verify by running the Get-ExecutionPolicy cmdlet. In Restricted mode, scripts don't run. Period. Since that's the default mode, Windows PowerShell can't be used to run scripts out of the box.
You can specify other modes using the Set-ExecutionPolicy cmdlet. I personally prefer the RemoteSigned mode. This allows local scripts (but not remote scripts) to run without being digitally signed, providing the easiest way to develop and test scripts. The AllSigned mode doesn't run any scripts unless they've been digitally signed, using a certificate issued by a trusted publisher. Finally, the Unrestricted policy runs anything. I don't recommend this at all, as it opens up Windows PowerShell to run malicious scripts that may find their way onto your computer. Note that the execution policy can also be governed by Group Policy, which overrides any local settings. (Set-ExecutionPolicy will warn you if a Group Policy setting is overriding your local settings.)
In addition, Windows PowerShell won't run scripts from the current directory unless you specify the path. This is designed to prevent command hijacking. Say someone creates a script named IPConfig.ps1 (PS1 is the file-name extension for Windows PowerShell script files). If files could be run out of the current folder, there would be a risk that you might type ipconfig and run this user created script when, in fact, you were expecting to run the Windows program Ipconfig.exe. Since Windows PowerShell does not run scripts out of the current folder, this mistake can't happen. If you do want to run a script out of the current folder, just specify the path: .\myscript, for example. The explicit reference to the current folder ensures that you know you're running a script and not a shell command.
Windows PowerShell also has features that make experimentation safer. For example, consider (but please do not try) this frightening combination:
Get-Process | Stop-Process
The Get-Process cmdlet creates a collection of process objects and pipes them to the Stop-Process cmdlet—which will indeed stop them! The result is a blue screen STOP error after about five seconds as critical Windows processes are killed off. But you can see what will happen, without actually letting it happen, by adding the very convenient –Whatif parameter:
Get-process | Stop-Process -Whatif
Running this in Windows PowerShell results in a bunch of statements that tell you what the cmdlets would have done without actually letting them do it. The online help system in Windows PowerShell (accessible by using the help alias) doesn't yet document the –Whatif parameter, but keep it in mind. It's a great tool for testing scripts and cmdlets to verify their results without actually doing anything potentially harmful or disruptive.

Wrap-Up
Among the features that didn't make it into this version of Windows PowerShell, perhaps the most significant is support for Active Directory® Services Interface (ADSI). While Windows PowerShell can utilize the very robust .NET classes that work with Active Directory and other directory services, it doesn't yet have a convenient Get-ADSIObject cmdlet. The result is that it is a bit difficult to work with directory objects.
Also, Windows PowerShell often offers a number of different ways to perform the same task. That's good, but it can make learning Windows PowerShell more confusing since for any given task you might see a half-dozen different examples of how to do it.
All of that will settle over time and the Windows PowerShell team will continue to add features and capabilities to the product. To stay in the loop, visit the team's blog.

Don Jones is the founder of ScriptingAnswers.com, and the coauthor of Windows PowerShell: TFM (SAPIEN Press, 2006). Contact him at don@scriptinganswers.com.


 Post Mortem: Migrating from Exchange 5.5
Post Mortem
Migrating from Exchange 5.5
Whitney Roberts

THE PROJECT
The Kentucky Department of Education needed to migrate from its existing Microsoft® Exchange Server 5.5 implementation to Exchange Server 2003. The system supports approximately 600,000 faculty, staff, and students during any given school year.
THE PLAYERS
The project was tackled by the Kentucky Department of Education's Office of Education Technology (OET) assisted by KiZAN Technologies consultants. The OET staff provided management oversight of the existing Exchange 5.5 implementation. The day-to-day running of the Exchange servers was the responsibility of the local staff in the school districts.
THE CHALLENGES
The Department of Education provides technical direction for all 178 school districts in the Commonwealth of Kentucky. These school districts are connected to each other via a WAN but each hosts its own locally managed Exchange 5.5 server, which needed to be upgraded individually. This upgrade would bring support for Outlook® 2003 cached mode clients, an improved Outlook Web Access (OWA) experience, a standardized e-mail addressing scheme, and more.
In addition to rolling out a new e-mail server and client infrastructure, many of the school districts were adding e-mail support for students. Both state and federal policy required that student information be protected and not visible outside of the respective district.
THE PLAN
Since an Active Directory® Connector (ADC) could not be used, we created a new Exchange Organization and migrated into it. Standardizing e-mail addresses meant creating both new identifying subdomains for each school district and over 400 new recipient policies. Protection of student addresses required constructing a system of Global Address Lists (GALs) and custom address lists for each of the school districts.
The actual migration involved staging new hardware, creating and configuring mailboxes based on district membership and user type, and setting extension attributes based on district membership and user type so that the user would show up in the appropriate address lists.
We created a Visual Basic®-based application to detect and correct misconfigured account attributes or group memberships and to provision new mailboxes properly to make sure visibility requirements were maintained. The app was deployed to all 178 districts and runs on a timed basis from the school district servers.

Background
On the surface, this may not sound like a challenging or technically difficult job. However, as I have been humbly reminded, business requirements rule, and the business requirements of this project were eye-opening.
Over the years, decentralized management and a mish-mash of standards created numerous support "silos" that most often had to be managed by OET. In addition, OET also had to make sure the Exchange 5.5 implementation met the requirements of the school districts and the Department of Education at the same time. This meant that, for the districts that did utilize student e-mail, the student information was protected and not visible outside of the respective district as dictated by state and federal guidelines (see ed.gov/policy/gen/guid/fpco/ferpa/index.html).
In Exchange 5.5, these guidelines meant student information within the district could not replicate out to the other school districts or other state agencies that shared GAL data. Specifically, this necessitated going through a manual export/import process for each of the 178 Exchange 5.5 sites: hiding the students, replicating the list, un-hiding the students, and finally importing the address list back in. In this manner, other districts would see only the staff data, and the owning school district would see all of its own data but only staff data from the other school districts.
Given the strengths and weaknesses of the existing system, upgrading the messaging infrastructure had to satisfy some specific criteria. First, the Kentucky Department of Education wanted to standardize on a common, readable SMTP naming scheme for the entire enterprise. The scheme needed to provide separate SMTP address spaces for faculty and students.
The new system also needed to provide a centrally managed service that would remove the messaging management and responsibility for availability from the district. This meant centralizing management of disaster recovery, mail flow, and helpdesk call resolution. As such, the new system had to provide a centralized method for standardizing user configuration across all districts, regardless of geographic location or district affiliation of a given user.
Further, the new system had to provide secure Web-based and mobile-device access to e-mail across a common, unified namespace for all districts.
Given these requirements—many of which were not already implemented in the existing Exchange 5.5 system—the upgrade to Exchange 2003 looked quite daunting.

Solution Building Blocks
To satisfy the business requirements, several Exchange-specific options were decided on for the deployment.
Due to the lack of standardization and the sheer magnitude of the Exchange 5.5 environment, an ADC would not be used. Instead, we created a new Exchange Organization and migrated into it.
Cleaning up and standardizing e-mail addresses for staff and students in each district led to the creation of over 400 new recipient policies. The requirements mandated a unique identifying subdomain for each district, including an identifying district designation for students. For example, in the case of Shelby County, the primary subdomain would be shelby.kyschools.us. The students in each district received a stu. subdomain designation, so the students in Shelby county would have stu.shelby.kyschools.us addresses.
Another constraint was that student addresses should be visible only within the students' district, while staff addresses should be visible across districts. This meant the construction of an elaborate system of GALs and custom address lists, as shown in Figure 1 and Figure 2. One benefit of moving to Exchange 2003 was that the address lists could be secured with permissions, and access to each individual address list could be controlled by group membership. We made good use of this feature and created two district-level security groups (one for staff and one for students), then secured the corresponding district staff and student GALs and address lists so that only user accounts within a specific district had access to certain address lists. By removing the default GAL and address lists and then making sure that the Everyone and Authenticated Users groups were not listed on the access control list (ACL) for each address list, we could preselect the address list a specific user type in a specific district would see.
Figure 1 Custom Address Lists Based on Role
Figure 1 Custom Address Lists Based on Role
The permissions, however, only solved a portion of the visibility problem. We had to come up with a way to control who was visible in what address list. A base Lightweight Direct Access Protocol (LDAP) query was created that allowed us to query for staff and student visibility; the query was then customized slightly for each address list and GAL. For example, the LDAP query for the Shelby staff GAL would show only the appropriate staff and students in Shelby and all of the staff (but no students) from every other district in the state. The Shelby student GAL would show only the staff and students from Shelby and no other staff or students from any other district in the state. We repeated this query for all of the districts in the organization. Figure 3 shows examples of some LDAP queries used in this complex migration.
Figure 3 Address List LDAP Queries (Close Figure 3)

Staff GAL LDAP Query
(&(&(objectCategory=*)(mailnickname=*)
(!objectCategory=ExchangeAdminService)(!objectCategory=msExchPublicMDB)
(|(extensionattribute15=15)(&(extensionattribute14=<districtNumber>)
(extensionattribute15=20))(&(extensionattribute14=<districtNumber>)
(extensionattribute15=14)))))
District Address List LDAP Query
(&(&(&(objectCategory=*)(mailnickname=*)
(!objectCategory=ExchangeAdminService)(!objectCategory=msExchPublicMDB)
(&(extensionattribute14=<districtNumber>)(extensionattribute15=15)))))
District Students Address List LDAP Query
(&(&(&(objectCategory=*)(mailnickname=*)
(!objectCategory=ExchangeAdminService)(!objectCategory=msExchPublicMDB)
(&(extensionattribute14=<districtNumber>)(extensionattribute15=20)))))
District Staff Offline Address List LDAP Query
(&(&(objectCategory=*)(mailnickname=*)
(!objectCategory=ExchangeAdminService)(!objectCategory=msExchPublicMDB)
(|(extensionattribute15=15)(&(extensionattribute14=<districtNumber>)
(extensionattribute15=20))(&(extensionattribute14=<districtNumber>)
(extensionattribute15=14)))))
Figure 2 Global Address Lists Segregating Staff and Students
Figure 2 Global Address Lists Segregating Staff and Students
Custom values for the Exchange user properties extensionAttribute14 (Custom Attribute 14) and extensionAttribute15 (Custom Attribute 15) were the distinguishing mechanism behind the LDAP queries. All staff and students state-wide used a specific value for extensionAttribute14 to designate user type and a unique value in extensionAttribute15 to show district affiliation. By using unique values for each district and the same values for each user-type, the address lists were able to show exactly the information required for each user type in each district.
These address lists worked quite well for users accessing e-mail through Outlook in non-cached mode or through OWA. (We directed each user to the correct address list in OWA using the msExchQueryBaseDN attribute.). But users running Outlook in cached mode presented another problem. Because Outlook will not honor a customized GAL as the designated Offline Address Book (OAB), we had to create additional address lists that were duplicates of the GALs. Each district mailbox store was then configured with the appropriate OAB address list for its cached-mode users. All in all, this solution topped out at over 1,500 address lists to provide full functionality to the various types of users across all 178 districts.

Ensuring Compliance
To make sure the structure that was being put into place would work correctly, adherence to the newly established standards was paramount. There were several high-level tasks that had to occur for every Exchange-enabled user in the organization. The reason for this is that each user has to be properly located across all of the district Exchange servers, as well as be visible in the correct address lists.
The user needed to be a member of the proper security group within the district's domain because the security group grants permissions to open the appropriate address list. The user also needed the msExchQuerybaseDN attribute set correctly based on district and organizational unit (OU) membership. The user's OU determined the user type.
The next step was to create the user's mailbox on the correct store on the proper server based on district membership and user type. This affected service levels, mailbox size limits, and offline address book address lists so that Outlook cached-mode would function correctly. The extension attributes were set based on district membership and user type so the user would show up in the appropriate address lists and not be visible in others.
Finally, we needed to provide a means to automatically correct any account's attributes or group memberships if they were misconfigured (or changed) to inappropriate values. If a user moves from one OU to another within the district domain, this signifies a user classification change, and mailbox location, GAL selection, and address list visibility must be updated.
We needed a provisioning system that would accomplish these tasks and assure Active Directory consistency and accuracy. We created a console-based Visual Basic-based app that runs on a timed basis from the school district Exchange 2003 servers. The application, which checks policy and standards adherence and also handles new object creation and provisioning, was deployed to all 178 districts.
We felt that the tasks at hand were too complex for a series of scripts to execute quickly and reliably. As a result, the Visual Basic-based application was seen as the appropriate solution since it was efficient, had structured error and exception handling support, and its internal code could be easily adapted to Microsoft Identity Integration Server (MIIS) extension code in the future with minimal rewriting.
Because we were migrating to a new organization, some method of coexistence had to be found. There were more than 178 legacy SMTP domains to support in the new environment—some districts had preexisting student e-mail domains and others did not—but we had the luxury of migrating districts one at time and bringing their legacy messages and addresses over to the new system without affecting the other districts. However, we needed to avoid disrupting e-mail flow as a district was migrated from Exchange 5.5 to Exchange 2003. This meant implementing some type of directory synchronization between the Exchange 2003 Active Directory and the legacy Exchange 5.5 directory. Microsoft granted the Kentucky Department of Education limited usage of MIIS for the duration of the migration only, so customized extension code was developed and used to synchronize both Exchange organizations via MIIS before, during, and after the migration.
Moving to a new Exchange organization naturally affects MAPI client configuration. The Department of Education was faced with the fact that there were literally tens of thousands of MAPI profiles at the district level that would have to be reconfigured during the migration. Fortunately, Microsoft released the Exchange Profile Tool (ExProfRe.exe), which was used for profile migration at the districts. We created a packaged installer that ran ExProfRe.exe with preloaded values so that district-level profile migrations could be performed in an automated fashion. The program was invoked via Group Policy. We saw close to a 100 percent success rate on automated profile migrations between organizations.

Hardware Upgrades
Since the new Exchange 2003 servers would be taking the place of geographically dispersed Exchange 5.5 servers, the hardware aspect of this solution had to support the same number of users as the Exchange 5.5 deployment, plus the load of the additional student accounts. The servers would have to be hosted locally within the district because the WAN topology was designed in such a way that all inter-district traffic (Internet and internal) flowed over a private T1 line (in some districts, multiple T1 lines).
Due to budgetary, management, and equipment-hosting constraints, the Exchange servers would not have storage area network (SAN) attached storage, so all storage had to be either internal or direct-attached. Because most districts do not have datacenters with rack storage and centralized cooling, the servers had to be as self-contained as possible. Fortunately, most districts had fewer than 5,000 users so single-server deployments were possible. For the districts that had more users, more powerful servers (or multiple servers) and additional storage were used, sometimes with separate front-end servers to further segregate the student and staff traffic.
In most districts we were able to deploy well-equipped dual-processor servers that were configured with the maximum number of 15,000 RPM Ultra320 SCSI drives, multiple RAID controllers, and 4GB of RAM. Larger districts got one or more quad-processor servers, using smaller front-end servers where necessary. Benchmark and performance testing before deployment indicated no usage or load problems. Varying degrees of local usage patterns, though, would be the real determining factor on the server's overall performance.

Proof of Concept Testing
Although the production environment comprises a 178-domain Active Directory forest, the proof-of-concept (POC) lab was scaled down to 13 district domains and an empty root domain. All of the test domains were populated with exports from the production domains. Virtual Exchange 5.5 servers were built and configured with copies of the selected districts' databases so that we could test the migration process and build-out of the planned environment. The entire POC testing lab was constructed with virtual machines so that the environment could be baselined, frozen, and then redeployed when necessary.
A diverse combination of scripts (using VBScript), Systems Management Server (SMS) installer packages, and console applications were devised, coded, tested, and finalized. In the end, we had scripts that performed initial Exchange 2003 setup and configuration, including recipient policies, GALs and custom address lists, and permissions. Scripts also handled administrative group configuration, district domain configuration (security group creation and permissions delegation), and post-deployment configuration for migration.
Staging the Exchange 2003 servers presented some significant challenges. We needed to get all of the Exchange servers installed successfully in all 178 domains and do so in such a way that didn't jeopardize the rest of the migration. Exchange 2003 differs from Exchange 2000 in that the domain-specific Exchange Domain Servers group is not added to the ACL of the organization until the first Exchange 2003 server in the domain is installed. (In Exchange 2000, the group is added when running the DomainPrep process.) To make sure we had all of the necessary security groups listed on the organization object in Active Directory, we had to install in a given domain, manually replicate Active Directory from the domain to the Active Directory replication hub site, and then pull that replication down to the next district to be installed. Only after the ACL on the organization object was confirmed to have the Exchange Domain Servers group from the previous district could the installation of the next district begin safely. If this process was completed out of order in even one domain, that district would have to be reinstalled (at the minimum) or other districts would not be able to communicate between districts due to an organization object ACL that didn't list all of the other districts correctly.
We tested the entire deployment, configuration, server staging, and migration numerous times until we were satisfied that the process worked well within the POC environment.

Migration and Growing Pains
Once the POC procedures and results were approved by the project team, the migration was ready to begin. OET chose to migrate themselves first so that they could experience the process and after-effects firsthand. At the time of migration, all of the servers had already been staged and the initial environment had been verified. All of the front-end protocol servers and mailbox servers were in place and ready to go into production with users.
Execution of the migration plan went well and, after a lengthy 21 hours to transfer content from one server to another, OET was up and functional on Exchange 2003. Fortunately, there were no significant issues with migrating the Outlook profiles from Exchange 5.5 to Exchange 2003. We packaged a batch file that called ExProfRe.exe in different configurations based on some simple logic. We then distributed the batch file and ExProfRe.exe to the Netlogon share of a domain controller (DC) in each district and used Group Policy to run the batch for all clients at the conclusion of a migration. This worked well at OET and we were confident of the process going forward.
At this point, the migration plan was reviewed and reapproved based on our experience with the OET deployment. Moving forward with the migration, we chose six districts to be the first production sites. Before each district was migrated, MIIS would read the latest info about the district's mailboxes from Exchange 5.5 and update the corresponding users in the district domain. We used the Exchange Migration Wizard (Mailmig.exe) to actually move the mailbox data between organizations. As the wizard moved the mailbox data, all of the proxy addresses were brought over and updated from what MIIS originally exported to Active Directory. During the migration, mail would be queued at the front-end SMTP servers and then released to the district after the migration, post-migration server configuration, and user testing had completed; the district was prepared to perform wide-scale client migration using ExProfRe.exe.

Wrangling the RUS
For Outlook clients to work correctly, we needed to ensure that the provisioning solution was the ultimate authority on setting attribute values. We also need the Recipient Update Service (RUS) to finish the job, so to speak, on account setup, but the RUS should run only after provisioning is finished. To achieve this, the RUS instances for the districts (all 178 of them) were set to Never for their schedules and the provisioning solution used code that triggered the RUS to start after provisioning had completed.
Most districts had fewer than 5,000 users and far fewer updates on a daily basis, so the RUS finished quickly since the provisioning system kicked off an update instead of a rebuild at the end of its daily run. However, there were several districts with more than 10,000 users. Doing updates in those environments was more time-consuming, and if a full RUS rebuild was needed in these large domains, it could run for a week or more. Since the RUS evaluates all object types when it runs, the effect of doing a rebuild in such a large domain was debilitating.
Although we were not able to come up with a mitigation strategy (and we worked with members of the Exchange product group to try to come up with a solution), we enhanced the provisioning code to handle much of what the RUS normally does. Users get showInAddressBook and proxyAddresses attributes populated upon initial provisioning so that they can log in immediately instead of waiting for the RUS to complete its work. We also created a manual task that runs on one of the front-end Exchange 2003 protocol servers. It does a timed LDIFDE import and clears the RUS gatewayProxy address so that if a recipient policy is accidently applied in the environment, it is not put on the to-do list of every RUS in the enterprise.

Categorizer Issues
One interesting aspect of keeping student data from getting outside the school district was implementing controls to prevent students from e-mailing users outside their own district or Internet addresses outside the system. Instead of working through an elaborate system of connectors and related items, we decided to implement a lesser-known feature of Exchange 2003 and use connector restrictions on the existing Routing Group Connector from the district to the central hub where the front-end protocol servers sit. Two security groups were created in each domain—one for staff and one for students—and these groups were added to the connector restrictions.
Once we enabled connector restriction checking in the registry, we started to notice some very debilitating performance problems. Messages would be stuck in the categorizer and local delivery queues for hours—even for messages destined to recipients on the same server. These problems got worse over time as additional districts migrated. As a short-term solution we turned off connector restriction checking and the mail delivery performance improved. Further inspection revealed that the categorizer was checking not only the group memberships of the two groups within its own domain upon message submission, but also the group memberships of the same two groups in the other 177 domains as well. This was done for every message going into or out of the server.
We subsequently opened a support case with Microsoft. After evaluation, Microsoft is currently in the process of generating a hotfix to alleviate the issue by allowing the categorizer to work in a reduced topology complexity mode. Further details on the hotfix are not available as we go to press, but the goal is to provide a setting that allows the categorizer to make certain assumptions about the routing topology and not check group membership across all of the connectors.

Lessons Learned
If there's one thing I've learned all over again from this project, it's that business requirements drive everything we do as consultants. There were many times where the feasibility of the business requirement had to be weighed against the complexity of the solution. The decisions and choices I've described in this column are those we felt most appropriate given our environment (including elements I haven't had time to discuss here), our resources, and our business requirements. For you, the decisions may be different, but hopefully this column has given you some things to think about. Fortunately for us, Exchange 2003 and Active Directory allowed plenty of flexibility to meet all of the requirements in this complex migration. Once you get past the shocking scale of the Kentucky Department of Education deployment, the actual solution is rather simple and built on what I feel are underutilized features of Active Directory and Exchange 2003.
The second valuable lesson is to practice an even-keeled approach toward troubleshooting and risk/problem management. There are times when local troubleshooting is appropriate, and other times when calling on the advanced resources of Microsoft Product Support Services is required.

Looking Forward
The Kentucky Department of Education environment is certainly unique. In the course of this deployment, almost every feature in Exchange has been utilized or customized to satisfy the department's requirements. Now that the entire environment has been deployed and migrated successfully, the stage is set for implementing additional functionality. The Department of Education has already seen school district interest in the Windows Mobile® platform and expects the mobile device movement to continue.
There is significant value to be gained from server consolidation. The Kentucky Department of Education is currently looking at virtualizing some aspects of the Active Directory environment. And with Exchange 2007 on the horizon, the department has started looking at the requirements for an upgrade of their messaging systems.
The Kentucky Department of Education is also looking at additional collaboration offerings in Live Communication Server, Live Meeting Server, and so on. Microsoft Operations Manager 2005 is also currently under deployment for management and monitoring of the Exchange and Active Directory servers.

The Results
Every child in the Kentucky public school system will be affected by the quality and pervasiveness of this implementation—including my own children and the children of the people with whom I worked on this project. This implementation has poised the Kentucky Department of Education to go further faster with technology, and in a secure and standardized way, than ever before. I'm proud to have been a part of such an undertaking.

Whitney Roberts is a Principal Consultant with KiZAN Technologies (www.kizan.com) and has been working with Exchange since the release of Exchange 4.0.


 SQL Q&amp;A: Preventing Reboots, Installing Multiple Updates, and More
SQL Q+A
Preventing Reboots, Installing Multiple Updates, and More
Edited by Nancy Michell

Preventing Reboots

Q How can I limit the number of reboots when applying hotfixes and the like to SQL Server and even the server operating system in general?

A First, you may be interested to know that most post-installation reboots are not hardcoded into the installer. They are usually the result of locked files. This means the installer is trying to update files that are currently in use (locked) by some other application or by the operating system. There is a straightforward approach to eliminating these reboots, which is to make sure no processes are using the files in question. How do you do that?
The first step is to determine the specific files that were in use and were locked during the installation. The easiest way is to look them up in the registry at HKLM\system\currentcontrolset\control\sessionmanager\pendingfilerenameoperations. Checking this registry key serves two purposes: it confirms that the reboot request was indeed caused by locked files, and it tells you which files were locked. If you look these up after the installation, but before the reboot, you may be able to figure out what caused the reboot requirement, and take steps to eliminate it during future installs.
You may encounter two types of entries. If a file was read-locked, you'll see a one-line-per-file entry that means the updated file is already on the disk but you need to clean up the temporary copy that's still in use. If a file was write-locked, you'll see a two-line-per-file entry that means the update hasn't happened yet, the application is in an unknown state, and should not be used. It's literally like stopping in the middle of setup. In this case, one line indicates the real file and the other points to a temporary file (which will become the real file on reboot).
The next step depends on many factors, but the idea is to identify what software is using the locked files. Use dependency checkers if you can, or the DLL HELP database on MSDN® (see support.microsoft.com/kb/815065).
If the software you identify has a service, stop the service manually and then test again. If it's an application, close it and try again. Sometimes multiple applications are using a single file and you need to stop all of them.
The result of these tests (in your test environment) is a list of apps and services to stop in the production environment before installation. Thus the production environment will not require a reboot because no software will be holding locks on files that the installer is installing. Of course, remember to restart each of these once the installation is complete.
The nice thing about this is that locks generally don't change that much. So if you figure out this pattern for your systems once, it'll probably avoid a lot of reboots over the lifetime of your system.
There's one more trick worth mentioning. You may have multiple versions of SQL Server™ or another product installed side-by-side and these may even be at different version levels. You should always update the latest version first. Chances are good that the update will replace everything with the latest versions and all the other instances won't care about locking at all. For example, if you have three instances of SQL Server 2000 SP3, the first one you upgrade to SP4 will upgrade the MSXML3.dll file and need a reboot. Let's say you do that reboot now. You can then upgrade the next two instances to SP4 without needing to reboot, because the updated MSXML3.dll file is already present. The same is true for upgrading any SQL Server 2005 instances before SQL Server 2000 instances, and SQL Server 2000 before SQL Server 7.0 instances. This is a general reboot minimization strategy that really does work.
Looking at the big picture, Microsoft development teams have been doing a lot of work to stop or limit the number of reboots in many scenarios. Since this is the SQL Q&A column, let's take SQL Server as an example.
In SQL Server 2005, the SQL Server team split the Microsoft® Data Access Components (MDAC) code necessary for SQL Server out into SQLNCLI (new files) instead of updating MDAC itself, and moved MDAC back in-band with the OS. How does this help? Well, the Windows® operating system itself uses MDAC and keeps these files locked. That's why most SQL Server service packs and other packages always made you reboot. Now SQL Server can update its version of MDAC without touching the OS's in-use MDAC files, so you don't have to reboot.
Furthermore, there is now a dependency checker and pause built into the SQL Server 2005 SP1 setup. This shows you what's locked and what probably locked it, and gives you the chance right then and there to stop whatever's doing the locking and continue setup without a reboot. This means you can play around with this in real time without needing a test lab. Of course, you can always just continue and take the reboot if you choose or if you're running in unattended mode. Notice how easy this makes figuring out who's locking your files!
In addition, the Microsoft Installer technology also handles the existing PendingFileRenameOperations (PFRs) much better. You may recall that SQL Server 2000 setup would block if any files at all appeared on the registry key we discussed earlier. SQL Server 2005 setup is smart enough to block only if the files belong to SQL Server (meaning there could be a conflict), and even in those cases can often reach into the PFR key and directly update it without needing a reboot. This technology is included at some level in both MSI and update.exe, which are the current standard Microsoft installers that are used to perform all updates.
What are some of the known gotchas out there that cause reboots? MSXML3.dll is one of the biggest. This file is always locked by the Windows SVCHost service, so anything that includes this service will have to reboot. And it's in most MDAC stacks (mdac_typ.exe) as well. Fortunately, MDAC is "in-band" on Windows Server® 2003 and Windows XP SP2 and later. The SQL Server development team did some great work splitting off msxmlsql.dll so that the SQL Server updates won't cause a reboot, but you can expect to see this one occasionally for a while yet. There's nothing you can do here—you can't shut down SVCHost and still run updates.
Also, update.exe packages have a bit of a quirk around the uninstall process. If you uninstall one, it puts a lock (outside of the PFR keys, of course) on the installer's in-place files. That prevents other update.exe packages from being able to run until you reboot to clear that lock. All you can really do is be aware of it and plan to reboot after any hotfix uninstalls you may do.
Intermittency is another issue. Not all programs sit on a file 100 percent of the time. In fact, most of the usage of shared DLLs is point-in-time, meaning that other programs access them only as needed, depending on their own workload and code paths. That means you might test 10 times and see no locked files, then hit a locked file on the production deployment. There's no magic bullet, unfortunately. Just build your test environment as much like your production environment as you can, including load (a good goal anyway) and run enough tests to pop these permutations.
OS service packs and upgrades are going to reboot. And note that on downlevel versions (Windows 2000, Windows XP, and anything earlier), you haven't finished the installation until after the reboot—even if the pendingfilerenameoperations registry key is empty. There are exception package registration operations that may run to install code during that reboot. These aren't even part of the service pack setup, but are embedded commands in your current operating system that are triggered by the upgrade scenarios to allow certain pieces of code you are using to survive them.
Finally, the PFR key does not do file version checks, nor does it enforce order through serialization. This means that it kicks off the file replaces in the order they are listed. But the order they are applied to the hard disk has been proven to be the last to finish (not the first to be started). That makes sense, but it means that having two copies of the same file on the same path, each pointing to different versions, will produce a random result following the reboot. If you ever see this, you need to make sure it gets fixed. If you are unsure, simply run both installers again after rebooting. If you've got the right files, neither will do anything. If you got the wrong ones, then whichever package has the right one will now fix it. Frankly, that's both faster and safer than trying to work out what the desired result is.

Multiple Service Pack Installations

Q I have eight SQL Server 2000 clustered instances on a 4-node cluster and would like to install SP4 with only minimal reboots. Can I install SP4 on all 8 instances one after another and at the end reboot all the nodes?

A Once you've completely installed a single instance of SQL Server SP4—which may require a reboot—on a given Windows server (clustered or not), you should not experience additional reboots when installing SQL Server SP4 on any other SQL Server instances on that box. The reason for this is that all the shared files—tools, MDAC, MSXML, and so forth—are already the right version and locked or not do not need to be further updated. All the non-shared files are used only by the first instance, and it will be shut down by setup. It's worth noting that the most recent instance of SQL Server should be updated first if you have multiple instances of multiple versions.

Running as Network Service

Q Exactly what is the NT AUTHORITY\NETWORK SERVICE account and what is its main purpose? Also, what are the security implications of this account, especially if it is given sysadmin (sa) rights?

A The Network Service account has the same identity (machine identity) as System on the network, but it has reduced privileges on the local box. Thus if a service requires access to network resources, perhaps to resolve account names with the domain controller, it must run either as System, a domain account, or Network Service. The latter is generally the safest choice because it is scoped only to a local box and even there it has reduced privileges.
Making Network Service a sysadmin in SQL Server means that any other service running as Network Service on a local box has full control over the server. This happens, for example, when the SQL Server service account is configured to run as Network Service. Making a SQL Server service account a member of sysadmin is required for functionality of the server (for example, to allow loopback connections). While having Network Service run as admin on SQL Server is not a very good security practice, it may (or may not) be better than using a domain account for that purpose, and it is much more secure than having SQL Server run as LocalSystem.
If you don't want Network Service to have sysadmin privileges, the next choice to explore is using a domain account specifically dedicated to running SQL Server on one or more installations of Windows NT®.
The following articles offer some help: msdn.microsoft.com/library/en-us/dnpag2/html/paght000009.asp and microsoft.com/technet/security/topics/serversecurity/serviceaccount.

Creating Multiple Databases

Q I am thinking of partitioning data into individual databases, maybe up to 250 databases online at once. I estimate that only 20 percent will be used for active queries at any given time. Are more databases better than fewer?

A The number of databases within a database server instance should be driven by business needs and administrative factors. There is very little overhead in creating multiple databases within a single server instance, but more databases mean more administrative overhead for maintenance tasks like backup and restore, mirroring, user accounts, and user roles. The additional administrative complexities may outweigh the benefits of additional databases.
As a general rule, all data related to an application should be kept in one database, which makes recovery from a failure easier. If an application's data is spread over several databases, then all databases must be recovered during a failover. This can delay the recovery, or even prevent a recovery if one of the databases can't be recovered.
That said, there are reasons to segregate data into multiple databases:
  • A subset of data has different backup requirements from other data.
  • Certain data needs a separate security context, that is, a different database owner.
  • Sometimes data exists for historical purposes and needs to be kept in read-only mode.

Thanks to the following Microsoft IT pros for their technical expertise: Ramon Arjona, Frank De Waelle, Robert Djabarov, Herman Learmond-Criqui, Maxwell Myrick, Ruslan Ovechkin, Uttam Parui, Shashi Ramaka, Gary Roos, Gavin Sharpe, Vijay Sirohi, Jimmie Thompson, and Madhusudhanan Vadlamaani.

Edited by Nancy Michell


 Hey, Scripting Guy!: Where Are the Cup Holders?
Hey, Scripting Guy!
Where Are the Cup Holders?
The Microsoft Scripting Guys
This column is based on a prerelease version of Windows PowerShell. All information herein is subject to change.

Whenever a new technology is introduced it usually takes a while before people can really differentiate the new technology from whatever came before. Take the first automobiles, for example. There's a reason why those contraptions were known as horseless carriages: that's pretty much all they were. When you got right down to it, about the only difference between the first automobiles and horse-drawn carriages was the fact that one used horses and the other didn't. Other than that, it was hard to tell them apart.
Today, of course, few people would mistake an automobile for a horse-drawn carriage. That's because, over time, auto makers have managed to do things that clearly separate cars from carriages. For example, today's cars come straight from the factory replete with CD players, air-conditioning, and satellite navigation systems; few horse-drawn carriages can say the same. Likewise, suppose you were driving a horse-drawn carriage in the 1880s and needed to set down your Frappuccino for a moment; sorry, you're out of luck. By contrast, some SUVs come standard with as many as 17 cup holders, while there's at least one make of automobile that includes three cup holders for the driver alone. Some of the newer models even have cup holders that can heat or chill your beverage as needed.
Note Yes, it is hard to believe that people once had to survive without being able to drink three different beverages while driving. How are you supposed to drive with just one or two beverages at your disposal?
Before a new technology can really succeed people need to know why they should favor the new and unknown over the old and reliable. When DVDs first came out they allowed you to watch movies at home, just like video cassette recorders did. On the other hand, DVDs were way more expensive than video cassettes, and you couldn't use a DVD player to record your own shows. Needless to say, DVDs didn't exactly take the world by storm. In fact, it wasn't until DVD manufacturers started packing their discs with deleted scenes, commentary from an actor you didn't even know was in the movie, and other things you couldn't get on a cassette tape, that DVDs began to supplant their VHS counterparts. Being able to watch a movie at home wasn't enough; DVD makers had to provide capabilities above and beyond those capabilities people already had.
Note Whether or not anyone actually watches any of these bonus items is beside the point: human nature being what it is, if something is available then you simply have to have it. Coincidentally, the Scripting Guys are thinking about coming out with a DVD consisting primarily of deleted text from TechNet Magazine columns and alternate endings to system administration scripts.

Using Windows PowerShell
Windows PowerShell™, the new command-shell/scripting technology released by Microsoft, represents another example of this. When people first hear about Windows PowerShell they typically hear things like this:
  • "Windows PowerShell lets you manage processes and services."
  • "Windows PowerShell lets you retrieve events from your event logs."
  • "Windows PowerShell lets you use WMI to manage computers."
Now, there's nothing wrong with any of that; those are all useful and important tasks. The only problem, of course, is that you can already carry out all those tasks using VBScript and Windows Script Host. Backward compatibility is important, and Windows PowerShell wouldn't be much of an innovation if it couldn't even do the same things VBScript can do. (Imagine a DVD player that couldn't actually show movies. Actually, you don't need to imagine that: one of the Scripting Guys has one at home.)
At the same time, however, if Windows PowerShell can only do the same things that VBScript can, well, then why should we bother with this new technology in the first place? Admittedly, VBScript doesn't ship with any cup holders; then again, neither does Windows PowerShell. So what difference does it make which scripting technology we use?
To be honest, it might not make any difference whatsoever. On the other hand, lack of cup holders aside, Windows PowerShell does have some capabilities that VBScript does not. If all you need to do is manage processes and services then there might not be any compelling reason for you to look at Windows PowerShell; as the saying goes, if it ain't broke, don't fix it. (When it comes to the Scripting Guys this maxim tends to read "Don't you guys try to fix it because then it will be broke.") Where Windows PowerShell becomes truly attractive is when you can use it to do something that VBScript can't do. Something like, say, changing the last write time for a file.

Changing the Last Write Time
You heard right: changing the last write time for a file. For various reasons (such as a quick-and-easy form of version control) script writers have always dreamt of the day when they could change the last write time (or the creation time) for a file or set of files. Sadly, though, this capability does not exist in either Windows Management Instrumentation (WMI) or FileSystemObject; in both of those technologies, file dates and times are read-only.
With Windows PowerShell, however, changing the last write time of a file to the current date and time is as easy as running this command:
$a = Get-Item c:\scripts\test.txt; $a.LastWriteTime = Date
Before we explain how this command works, let's answer a question that's likely on everyone's mind: how can Windows PowerShell modify the last write time of a file when WMI can't? We thought we should address that question before doing anything else simply because it represents a common misconception regarding Windows PowerShell. People tend to assume that Windows PowerShell is just a new way to use WMI. Granted, you can definitely use Windows PowerShell to access WMI data; it's also true that, while we wait for additional cmdlets (native Windows PowerShell commands) to be constructed, most system administration scripts will continue to rely primarily on WMI. At heart, though, Windows PowerShell is based on the .NET Framework, and Windows PowerShell cmdlets typically deal with .NET objects rather than WMI objects. And that's important, because the properties available through the .NET Framework are not necessarily the same as the properties available through WMI.
For example, when we use the Get-Item cmdlet to bind to a file, we don't get back an instance of the WMI CIM_Datafile class; instead, we get back a .NET object. Is that really that big of a deal? You bet it is—the .NET object exposes a slightly different set of properties and methods for file management than WMI does. But, as we've just seen, those slight differences can be quite important, such as the difference between a property being read-only versus read/write.
A question in the back: how do we know that the Get-Item cmdlet returns a .NET object? In this case, and in most cases, the easiest way to find out what kind of object we're dealing with (as well as the easiest way to identify the properties and methods available to us) is to bind to the file and then pass that object to the Get-Member cmdlet:
Get-Item c:\scripts\test.txt | Get-Member
The results of this command are shown in Figure 1. As you can see, this really is a .NET object (as if the Scripting Guys would have been wrong about something like that!)
Figure 1 Members of the Object Returned by Get-Item (Close Figure 1)

TypeName: System.IO.FileInfo

Name                      MemberType     Definition
----                      ----------     ----------
AppendText                Method         System.IO.StreamWriter AppendText()
CopyTo                    Method         System.IO.FileInfo CopyTo(String destFileName),
                                         System.IO.FileInfo CopyTo(S...
Create                    Method         System.IO.FileStream Create()
CreateObjRef              Method         System.Runtime.Remoting.ObjRef CreateObjRef(
                                         Type requestedType)
CreateText                Method         System.IO.StreamWriter CreateText()
Incidentally, this simple little command illustrates two of the more interesting (and useful) capabilities of Windows PowerShell: pipelining and meta-information. Although we won't go into technical details today, the basic idea behind pipelining is that you can use a cmdlet to grab an object and then hand that object over to another cmdlet. In this command we've used the Get-Item cmdlet to get an object representing the file C:\Scripts\Test.txt, and then handed that object (as an object) through the pipeline to the Get-Member cmdlet. (The | symbol represents the pipeline in a Windows PowerShell command.)
Get-Member then does its thing, which happens to be retrieving the properties and methods for the retrieved object. This is one of the other nifty features of Windows PowerShell: it's very easy to access meta-information, information about information. By using cmdlets such as Get-Member and Get-Command you can interrogate Windows PowerShell all you want, much in the same way you can write a WMI script to retrieve information about the WMI classes, properties, and methods available to you.
Note Get-Member actually goes one step past what you can do with WMI. Although you can use WMI to get information about WMI objects, you can't use WMI to get information about other COM objects (like, say, FileSystemObject). That's not the case with Windows PowerShell. Get-Member works with any object you can bind to or create. Can't remember the methods available when you use the FileSystemObject? Then run the following command:
New-Object -comobject "Scripting.     FileSystemObject" | Get-Member
This is an incredibly useful—and thus far underutilized—capability.
Nice, huh? If we scroll through the list of properties and methods available to us when working with the file C:\Scripts\Test.txt we'll eventually see this line:
LastWriteTime             Property       System.DateTime LastWriteTime {get;set;}
Interesting. Apparently, LastWriteTime is a property that supports both get (read) and set (write). Does that mean we can use Windows PowerShell to change the value of the file's last write time? Hmmm...
Windows PowerShell Glossary
Alias A customizable name that represents an existing cmdlet. Windows PowerShell provides a number of default aliases that map common shell commands to their cmdlet equivalents.
Cmdlet The smallest unit of functionality in Windows PowerShell, analogous to the built-in commands in other shells.
Command A unit of execution in Windows PowerShell. Commands can include multiple piped operations and can span lines by using the ; character.
Frappuccino A blended iced coffee drink sold by Starbucks.
Monad/MSH/Microsoft Command Shell Former code names for Windows PowerShell.
Object Structured data retrieved from the system or a command. Windows PowerShell manipulates data as .NET objects, enabling you to manipulate object properties by name rather than having to parse text-based command output.
Parameter Options set on a cmdlet on the command line. Windows PowerShell uses an initial hyphen (-) character to designate a parameter.
Pipeline Enables sending the output of one command to the input to another command. Also known as a pipe; represented in commands by the | character.
Profile Settings, loaded when Windows PowerShell starts, that enable you to configure how the shell operates.
Script One or more commands saved as and executed from a file. Windows PowerShell uses a .ps1 extension for script files.
Shell A command-line environment in which you can run both commands and cmdlets.
Variable An object or parameter whose value can be set, changed, or retrieved during command excecution.

Let's go back now and talk about the command that sets the last write time of Test.txt to the current date and time. It looks like this:
$a = Get-Item c:\scripts\test.txt; $a.LastWriteTime = Date
As you can see, there really isn't much to this command. To begin with, we declare a variable named $a. Then we use the Get-Item cmdlet to connect to the file C:\Scripts\Test.txt. In turn, we store that object in the variable $a. After typing a semicolon (which enables us to carry out multiple commands on a single line), we set the value of the LastWriteTime property to the current date and time. You know, that almost makes up for the lack of cup holders, doesn't it?

Getting a Little Fancier
Of course, you aren't limited to setting the last write time to the current date and time, either. For example, consider this command:
$b = Get-Date "5/22/2006 8:15 PM";
$a = Get-Item c:\scripts\test.txt; $a.LastWriteTime = $b
This time we've strung together three commands. To begin with, we use the Get-Date cmdlet to create a date-time object equal to 5/22/2006, 8:15 PM. Why didn't we have to use Get-Date to create a date-time object in our first script, the script that sets the last write time to the current date and time? That's easy: the Date command automatically returns a date-time object. (To verify this for yourself, type the following command into the Windows PowerShell console: Date | Get-Member.)
After creating a date-time object equal to the desired date and time, we next use our familiar Get-Item command to get an object representing the file Test.txt (an object we store in the variable $a). Finally, we set the LastWriteTime property to the value represented by the variable $b:
$a.LastWriteTime = $b
Believe it or not, you haven't seen the half of it yet. Take a gander at this innocuous-looking command:
$b = Get-Date "5/22/2006 8:15 PM";
Get-ChildItem c:\scripts | ForEach-Object {$_.LastWriteTime = $b}
What's so interesting about this command? Well, let's take a look at what it does. The first part is easy: once again we're creating a date-time object equivalent to 8:15 PM on 5/22/2006. Notice, however, that in part two we don't use the Get-Item cmdlet; instead, we use this command:
Get-ChildItem c:\scripts
What's the point of that? Well, any time you apply the Get-ChildItem cmdlet to a folder you get back a collection of all the files in that folder. We're going to grab that collection, and then hand all the items in the collection over the pipeline (there's that | symbol again) to the ForEach-Object cmdlet.
And what's ForEach-Object going to do with all those files? Why, it's going to change the value of the LastWriteTime property for each and every one, of course:
ForEach-Object {$_.LastWriteTime = $b}
Note Although this month's column isn't meant to be a beginner's guide to Windows PowerShell, we should note that $_ is a special variable that represents the current item in a For Each loop. It's equivalent to objItem in the following VBScript loop:
For Each objItem in colItems
    Wscript.Echo objItem.Name
Next
When our Windows PowerShell command finishes running, all the files in the folder C:\Scripts will have the same last write time. We'd like to see one of those new-fangled horseless carriages do that!
We could run variations of this command all night. For example, maybe you want to change the last write time for only the .txt files in the folder C:\Scripts? OK; all you had to do was ask:
$b = Get-Date "5/22/2006 8:15 PM"; Get-ChildItem c:\scripts\*.txt | ForEach-Object {$_.LastWriteTime = $b}

Whetting Your Appetite
That's very cool. However, we should note that the point of this column is not to "sell" you Windows PowerShell, nor are we encouraging you to throw out all your VBScript scripts and convert them all to Windows PowerShell. If those scripts do what you want and need them to do then you should leave well enough alone; the same goes for batch files, command-line tools, magic spells, and any other tools you use to manage your computers. Remember, if it ain't broke...
The point is not that Windows PowerShell lets you do the things you're already able to do; the point is that, in part because it provides access to the .NET Framework, Windows PowerShell enables you to do things you haven't been able to do before. Modifying the last access time of a file (or a group of files) is one example of this. Whether or not there are other examples depends on what it is you need to do. It's those capabilities that make Windows PowerShell worth looking into.
For many of you this was probably an unusual introduction to Windows PowerShell: after all, instead of starting with a simple "Hello, world" kind of example we launched right into a multi-part Windows PowerShell command, all the while tossing out terms like pipeline and cmdlet as though everyone knew what we meant by all that. That was intentional. After all, we know that many of you have yet to try Windows PowerShell; we were afraid if we showed you how to retrieve a list of services on a computer you'd simply shrug and go back to whatever you were doing.
Note We hope what you were doing wasn't driving. For many of us it's hard enough to drink three beverages, watch a DVD, and talk on a cell phone while driving, without adding "reading TechNet Magazine" to the mix.
Hey, come on: we did say for many of us. Not for all of us.

The Microsoft Scripting Guys work for— well, are employed by—Microsoft. When not playing/coaching/watching baseball (and various other activities) they run the TechNet Script Center. Check it out at www.scriptingguys.com.


 Security Watch: Deploy a Globally Trusted PKI
Security Watch
Deploy a Globally Trusted PKI
John Morello
This column includes prerelease information which is subject to change.

A Public Key Infrastructure, or PKI, is a fundamental element in building trust among diverse applications, operating systems, and identity realms. It is built on a hierarchical trust model in which end entities trust a common key at the highest root level, and therefore any other keys signed by that root are implicitly trusted.
Because of this structure, a well-designed PKI can be easily scaled out by adding additional Certificate Authorities (CAs), without introducing changes to the end entities.
PKIs are typically implemented in one of two ways. The most common approach is to purchase certificates as needed from one of the implicitly trusted global root providers, such as Cybertrust or VeriSign. These certificates are often used for securing traffic on Web sites, but they can also be used for encrypting Virtual Private Networks (VPNs), signing e-mail, and similar tasks. Because these global roots are built into every modern Web browser and operating system, this method can easily extend a trust relationship across organizational boundaries. However, since these certificates are purchased on an as-needed basis, this approach can be costly to deploy widely within a large organization.
The internal PKI model has recently become more prevalent. As the name suggests, an internal PKI is only trusted within an organization's own network. This model provides a cheaper way for an organization to provision every user and computing device on its network using certificates. This approach is often used to facilitate technologies like smart card log on, Encrypting File System (EFS), and secure wireless networking. The downside is that since these certificates are not globally trusted, they are not well suited to outward-facing applications. Thus, most organizations that have deployed internal PKIs usually adopt a hybrid approach, purchasing certificates for public applications from one of the globally trusted roots.
But what if you could get the best of both worlds? What if your organization could run its own PKI—with the cost savings you get from issuing internal certificates—and at the same time provision publicly trusted certificates, all from your own local system? Such a solution would offer the benefits of an internal PKI (like Active Directory® integration, autoenrollment, and localized IT support), while delivering the key advantage of a globally trusted root. Louisiana State University (LSU) has just such a system. In this installment of Security Watch, I take a close look at the solution we implemented at LSU, discussing its technical design and highlighting best practices to use when deploying a similar system. It should be noted that the solution that I'm going to describe is one that met our particular needs given our environment, resources, and business requirements. You should evaluate all of these aspects of your business in advance in order to determine the type of solution that will be right for you.

How LSU Does PKI
LSU is a major research university with a student, faculty, and staff population that totals over 40,000. It has many publicly facing Web sites, and many of these require Transport Layer Security—the proper term for secure sockets layer (SSL) certificates. LSU was spending thousands of dollars annually on certificates from the globally trusted root providers.
Meanwhile, LSU had other IT security goals that could greatly benefit from a well-designed internal PKI. Given LSU's goals and requirements, you might expect that we would have chosen a hybrid model. But instead, we decided on what we felt was a more innovative approach.
We worked with one of the globally trusted roots, Cybertrust, to deploy a CA on LSU premises that is subordinate to a Cybertrust root CA (see Figure 1). Cybertrust offers this service in their OmniRoot program. In this scenario, LSU builds and operates a CA that runs locally on LSU-owned equipment. However, unlike the internal model discussed earlier, LSU's CA is not self-signed and its key (along with any of those it signs) is implicitly trusted across virtually all modern operating systems and browsers. Technically, this design is a logical extension of the openness of the x.509 standards. Because a PKI is built on these standards, it's possible to plug systems created by different vendors into the same overall hierarchy. That said, what's particularly interesting about the OmniRoot solution is the terms of use for the service.
Figure 1 Cybertrust Global Root Certificate
Figure 1 Cybertrust Global Root Certificate
The OmniRoot service is specifically designed for the sort of PKI deployment we implemented at LSU. In the OmniRoot program, organizations pay Cybertrust a set annual fee for participating in the program and then pay a per-certificate fee only for those certificates created for public usage. In other words, LSU can issue as many certificates as it requires for internal usage without worrying about per-certificate fees adding up. Meanwhile, the cost of the publicly facing certificates is now significantly cheaper than what we were paying when the university purchased certificates on an as-needed basis.
To be clear, the internal certificates and the public certificates are identical from a technical standpoint. They are issued from the same CA chain, they have the same cryptographic strength, and they are provisioned through the same registration authority system. The only difference between the two is the licensing terms.
Ultimately, we have been able to expand and enhance the LSU IT security platform at a lower cost. This accomplishment made a great business case for this sort of PKI solution at LSU. Now let's take a look at the more technical details of the deployment.

PKI Hierarchy
LSU's PKI hierarchy is built on a two-tier design, with the root tier managed by Cybertrust and an online issuing CA running at LSU (see Figure 2). The root tier forms the anchor for trust in the hierarchy. In other words, the trust relationship of all certificates issued by any CA in the hierarchy falls under the root CA (see Figure 3). Thus, the root CA forms the common bond between all CAs, certificates, and end entities in the hierarchy. Because the Cybertrust global root is already built into Windows® and virtually every other modern computing system, this common bond automatically extends beyond LSU's network.
Figure 2 Two-Tier PKI Hierarchy (Close Figure 2)

Root Tier (Cybertrust) | Issuing Tier (LSU)
Managed by Cybertrust | Managed by LSU
Located in Cybertrust’s secure datacenter | Located on LSU’s Baton Rouge campus
Included in the default root certificate of almost all modern computing systems | Issues certificates to end entities within LSU system organizations
 | Certificates issued by LSU fall under the Cybertrust root and therefore are globally trusted
Figure 3 Certificate Issued from LSU's Issuing CA
Figure 3 Certificate Issued from LSU's Issuing CA
Cybertrust manages the root CA in one of its secure hosting facilities. The hosting facility uses strong physical security controls, including man-traps, biometric locks, and a fully copper- jacketed storage vault. Cybertrust issues certificates only to the issuing CAs of organizations that have demonstrated compliance with its security and certificate policies.
LSU's issuing CA is an end-entity-facing system responsible for providing certificates to LSU's user and machine population. Its certificate is signed by the Cybertrust CA above it in the hierarchy and it issues certificates that chain through itself to the Cybertrust root CA (see Figure 3).

The Issuing OS and Hardware
LSU uses Windows Server® 2003 Enterprise Edition with Service Pack 1 for its issuing CA. The Windows Server 2003 Certification Authority service offers many new capabilities, including key archival, delta Certificate Revocation Lists (CRLs), and "version 2" templates. The instance of Windows used as the issuing CA is built according to the standard LSU Windows Server build policy. It's updated with all relevant security updates and is managed using the standard LSU change management practices and software inventory and update tools. In short, while there are some unique aspects to managing the CA service itself, the base server hardware and operating system are easily integrated into the organization's existing IT operations.
In addition to following the standard LSU best practices for server installation, we took advantage of the Security Configuration Wizard (SCW), which is available in Windows Server 2003 Service Pack 1. The SCW uses a database that contains dependency matrices of the various workloads that can be run on Windows Server and what security setting these workloads require. This database includes the two primary roles the CA will provide—the CA service itself and IIS (which provides the PKI self-service Web site).
Using the SCW, we easily created a security template that disabled all services other than those required to run these workloads, as well as hardened the network stack of the server. We then used scwcmd.exe to transform this template into a Group Policy Object (GPO) so that we could apply it to the server at the organizational unit level. This approach simplifies scaling out and disaster recovery since the role-based security configuration for an LSU CA is now stored within Active Directory, rather than being machine-specific.
LSU's issuing CA is built on an IBM xSeries 346 server, with dual 3.2GHz Intel Xeon processors, 4GB of RAM, and five 146GB SCSI hard disks. Given the performance demands of the CA service, and the fact that LSU utilizes a hardware security module, this platform should provide significant longevity and headroom for growth.
We followed standard best practices for separating databases and log files onto separate physical spindles by creating two RAID 1 arrays. The fifth hard disk is kept as a hot spare and is available to either of these arrays. We mounted the first array as our system volume, where we keep the CA database. The second array is used for log files and various other data files, such as the CRLs. The CA database is based on the same Jet technology found in other parts of Windows, like Active Directory, so the same maintenance and disk configuration best practices for other Jet-based storage solutions apply here.
A hardware security module (HSM) is a dedicated hardware device that is managed separately from the operating system. HSMs typically come as PCI adapters but are also available as network-based appliances. These modules provide a secure hardware store for CA keys, as well as a dedicated cryptographic processor to accelerate signing and encrypting operations. Windows utilizes the HSM through the CryptoAPI interfaces—the HSM is seen as a cryptographic service provider (CSP) device. The OmniRoot program requires that each CA use at least a FIPS 140-2 level 3 (level 4 devices are fairly rare) certified HSM for key generation and storage. As a unique and highly secure device, an HSM requires a notable financial investment; list prices for PCI-based HSMs are generally in the $10,000-15,000 range.
In the LSU design, an nCipher nShield 500 TPS (500 Transactions Per Second model) is used as a directly attached HSM. ("Directly attached" means that it is internal to the server and available for use only by that server, as opposed to being a network-based HSM that may be shared by multiple hosts.) The HSM is a FIPS 140-2 level 3 device and provides K of N protection of the keying material. The HSM provides significant protection against the private keys being compromised—an attacker would need to possess both the HSM storage as well as a defined number of access tokens and their PINs to access the key.
It is critical to note that HSMs are designed to prevent their contents from being tampered with by malicious parties. Thus, HSMs strictly enforce a limit on consecutive failed logon attempts. In LSU's design, the storage is irrevocably erased after 10 consecutive failed attempts.
We have a single CA in our setup, so the price premium for a network-based HSM is not justifiable. However, if an organization plans to implement two or more CAs, it can be significantly cheaper to purchase a single network-based HSM and share it among the CAs. This approach also makes token management easier, since all key storage can be protected by the same set of tokens.

Active Directory and Enrollment
One of the primary benefits of a Windows-based PKI solution is that certificate provisioning is accomplished without having to install additional software on entities that participate in the PKI. Rather, through a combination of autoenrollment and Group Policy, most certificate provisioning can be managed autonomously, invisible to the end user. LSU uses this technology to automatically distribute machine certificates to computers that are members of its Active Directory. And we use a customized Web site to provide self-service certificate request capabilities for non-Windows hosts.
As mentioned earlier, the OmniRoot program distinguishes between certificates used publicly and those used internally. Certificates are considered public if they are used by systems and users that are not part of the LSU organization (for instance, when a prospective student submits an admissions application through an SSL-secured Web site). Certificates used within the LSU organization (such as those used to encrypt data on LSU's servers) are considered internal.
Since the cost of the OmniRoot program is based on the number of publicly facing certificates, we need to be able to track how many public certificates are issued and to whom. Thus, for any certificate whose purpose is public, the distinguished name of the template begins with "PublicCertificate" and the display name begins with "Public Certificate". For example, the publicly facing LSU TLS certificate template is based on the default Windows Web Server template and is called Public Certificate LSU Web Server. For Web servers used internally, we have a second template called Internal Certificate LSU Web Server. Technically, the two templates are identical; the naming differences only serve to make accounting easier.
For bulk enrollment needs—such as enrolling entire departments for certificates to be used with IPSec—we rely on autoenrollment. This is a simple, highly efficient tool built into Windows. From a PKI administrator's standpoint, the only action required is to change the access control list on the template, granting any group of users or computers that need certificates the right to autoenroll. These entities will then automatically check Active Directory, enumerate what templates they have autoenroll access to, find the CAs with those templates, and automatically enroll for certificates. Also, the autoenrollment feature will ensure that certificates are replaced if a template is superseded and automatically renewed prior to expiration.
For non-bulk purposes, such as provisioning Web servers with certificates, we use a self-service Web site that lets users request certificates through a Web browser. This site is a customized version of the Web enrollment pages that ship with Windows Server 2003 (the /certsrv virtual directory); our implementation includes LSU's standard Web branding and precompletes the correct default entries on the request page. And we use the default exit module in the Windows CA service to automatically e-mail our PKI administration team when a request sent through this site requires approval.

Revocation Design
Each certificate has a validity lifetime. From time to time, LSU needs to invalidate certain certificates prior to the end of their validity period (for example, if an employee is given a certificate with a validity period that expires in May 2007, but that employee leaves LSU in January 2007). This type of revocation is accomplished with CRLs, which are files that contain a list of certificate serial numbers signed by a CA. The serial numbers contained on a CRL refer to certificates whose validity lifetime has not expired, but which LSU no longer considers trustworthy. Clients can then download this CRL and check against it to determine the validity of certificates.
Any x.509 v3 certificate (except the root CA certificate itself) should have a pointer to a valid CRL. This pointer is known as a CRL distribution point, or CDP. The CRL distribution point is included in the certificate itself (see Figure 4) and cannot be modified after a certificate is issued. The CRL is an essential part of providing assurance that certificates from a CA are valid. As such, we make sure the LSU CRLs are physically redundant, highly available, and accessible by external parties.
Figure 4 LSU's CRL Distribution Point
Figure 4 LSU's CRL Distribution Point
A Windows Server-based CA that is integrated with Active Directory (like LSU's) automatically publishes its CRL to Active Directory making it accessible through Lightweight Directory Access Protocol (LDAP). The default publishing location is the CDP container (see Figure 5) within the Public Key Services container in the forest. This publishing location is an excellent option, as it provides automatic replication, fault tolerance, and locality for CDP lookups.
Figure 5 The CDP Container
Figure 5 The CDP Container
In the LSU design, however, there are many non-Windows and non-Active Directory computers that utilize the PKI. And while LSU is a large organization, the great majority of its users are located on the same campus network, with a 10Gbps backbone, making the advantages of Active Directory-based CDP publishing minimal.
So we publish our CRL only to an HTTP path, making it easier to build a redundant hosting platform (our primary Web site is already mirrored). This also removes the potential complexities associated with enabling client LDAP lookups. We publish our CRL on a daily basis and publish delta CRLs on an hourly basis.

Going Forward
While our current PKI provides a strong security solution for the university community, we plan on enhancing its capabilities. Windows Vista™ and the next version of Windows Server (code-named "Longhorn") will provide numerous new key management capabilities that we plan to investigate. We are particularly interested in the planned Online Certificate Status Protocol (OCSP) client and responder capabilities, support for elliptic curve cryptography and SHA-256 algorithms, and a much improved autoenrollment interface. In addition, Windows XP Service Pack 3 is scheduled to include a feature known as credential roaming, which will provide secure roaming of key pairs without the overhead associated with roaming profiles. Finally, we're investigating the use of Certificate Lifecycle Manager (which is currently available in beta) in order to improve our reporting and management capabilities.
In summary, LSU's PKI provides the university with a foundational security service that is used for many critical IT functions. Using Cybertrust's OmniRoot service, we can run a PKI that is well integrated with LSU's other IT systems, but whose trust extends globally. And, by building the solution on the Windows CA service, we have efficient certificate provisioning and management capabilities and excellent integration with our existing directory and identity management system. In short, the PKI vision of strong security and seamless trust is already becoming a reality at LSU.

John Morello graduated Summa Cum Laude from Louisiana State University and spent six years with Microsoft as a Senior Consultant in Microsoft Consulting Services. With MCS, he architected PKIs for strategic clients such as Deloitte, GE, and various Federal organizations. He now serves as the Deputy CISO at LSU.


 The Desktop Files: Inside the Windows Imaging Format
The Desktop Files
Inside the Windows Imaging Format
Wes Miller
This column is based on prerelease versions of Windows deployment tools. All information herein is subject to change.

With the release of Windows Vista comes a significant change in how Windows is installed and deployed. The setup engine and the entire suite of deployment tools for Windows Vista (as well as the forthcoming version of Windows Server, code-named "Longhorn") have been rewritten. I discussed these changes and the work that
the Windows® deployment team put into making these changes in recent TechNet Magazine articles. But there is a lot to discuss on this topic, and so I will be covering Windows PE, Windows installation and deployment, and other Windows management and security topics in this new regular column.
In this first installment, I'll discuss the new Windows Imaging format (WIM). If you aren't already familiar with the details of imaging, be sure to read the sidebar "The Typical Imaging Process" for an overview. In my next column I'll discuss the Windows Automated Installation Kit—a new suite of tools that includes ImageX (the command-line tool used to create and manipulate WIM images) and Windows Deployment Services (WDS), which replaces Remote Installation Services (RIS)—looking at how these tools enable easier OS deployments.

The Turning Point
Year after year, the Windows deployment team has heard the same feedback from OEMs and enterprise customers. They want better support for imaging and they want Microsoft to provide unified imaging tools. These users don't want the task of selecting a third-party vendor and learning the intricacies of their product. Moreover, we had frequently heard the complaint that these OEMs and enterprise customers don't think they should have to pay for imaging tools used to deploy Windows; instead, they want Microsoft to provide these tools.
Early on in the development of Windows Vista™, the Windows deployment team made a very big gamble. We decided to completely discard the entire setup engine—many components of which had existed for nearly a decade. Setup had always been fragile and had far too many steps. We wanted to speed up the process, remove unnecessary steps, and improve the reliability of Windows setup overall. To do this, we would shift the entire setup process from the hand-stitching engine we had used for some time, and replace it with an entirely image-based setup process.

Designing a New Imaging Format
Early in the process of redesigning the setup, we set out to define what kind of image we should use. The classic argument has been sector versus file. That is, should we use an image format that stores a copy of a volume as a sector-by-sector duplicate or one that stores every single file on an individual basis. The benefit that most people expect is speed (with a sector approach) versus compression (with a file-based approach). In the end, we wound up with a format that I feel is a great compromise between the two. It delivers very good performance, but also provides for great compression.
To begin, we defined several key objectives we had to achieve to make an imaging format that meets the needs of OEM and enterprise users. Our list was heavily slanted towards a file-based architecture. I think we all saw the benefits of this approach, and therefore we first set off in that direction. And soon WIM files were born. (And in case you're wondering, the "M" in WIM doesn't stand for anything. We named it WIM based on Windows team events that recur near shipping milestones—and *.WI seemed like an odd abbreviation.) So let's take a look at the set of goals we came up with for the new image format.
Windows File System Format Neutral The image format had to function across both file system architectures that Windows supports. Though almost all systems use NTFS when running, many OEMs and enterprises use FAT file systems as a stopgap to allow them to use MS-DOS® to deploy Windows. They then convert to NTFS later on. We had to continue supporting this for Windows Vista.
Volume Geometry Neutral We had to get past the historical problem of capturing a large volume and applying it to a smaller one. Even when the imaging tool can get past this through partition manipulation tricks, it's unreliable. In a file-based architecture, the image only takes up as much space as the files that comprise it. Since the file system is restored independently, there is no reliance on volume size.
The Typical Imaging Process
When an OEM duplicates large numbers of systems or a large organization needs to deploy Windows to numerous machines, they usually use imaging technology. The premise is to take the host system, build it exactly as you want, and then duplicate it onto the target systems. In case you aren't familiar with the imaging process, here is a quick overview.
  1. The administrator installs Windows on a system and configures it as desired. This involves adding and removing components, installing commonly used apps, and tweaking the UI and other settings. Let's call this the host system.
  2. The administrator then runs Sysprep (provided by Microsoft in the form of the Windows deployment tools), which performs the heavy lifting of cleaning up the attributes that make a Windows installation unique (such as the computer name, the Windows Security Identifier (SID), and other unique identifiers across the system). They are reset when the second half of Sysprep runs.
  3. Next, the admin shuts down the computer and reboots into Windows PE, MS-DOS, or another OS that allows him to run the imaging app.
  4. Now the admin runs the imaging tool, creating an image of the reference system. This image is a file (or collection of files) that contains an exact replica of the reference disk or partition.
  5. The system is shut down and the admin archives the image, which can be updated (adding applications, hotfixes, and service packs) as needed.
  6. The image is replicated to one or more computers, and Sysprep completes its work, running through an abbreviated version of setup (called Mini-Setup or Windows Welcome, depending on what version of Windows you are running Sysprep on and how Sysprep is configured). This stage relies on a tiny native-mode executable called setupcl.exe.

System Architecture Neutral The image architecture had to work with any architecture that Windows might support. At the time, this meant supporting the very different x86 and IA64 (Intel Itanium) architectures. Today, this also includes the x64 (AMD64 and EM64T) architecture.
Must Be Fast to Capture and Apply The image had to be reasonably fast to capture, given compression. More critical, however, it had to be very fast at image application.
Support Non-Destructive Image Application Since we were replacing the upgrade option (which replaces Windows without harming user data or applications), as well as deriving a tool that could be used for system recovery, we wanted it to be able to image a system without destroying everything already on the volume. Sector-based formats, without considerable work, don't let a user do this—in general, you restore an entire volume or nothing at all.
Support Significant Compression When Windows Vista first started development we were still focused on shipping on CD. Considering an image of Windows XP is nearly 1GB, we decided it was important to compress the image as tightly as possible. There are tricks you can perform with a sector-based image to do this, but the moving parts in such an implementation concerned us. Many sector-based formats omit whitespace on disk to save space. (Why record a 0 to an image if it's always a 0?) However, by using a file-based approach, we handled this inherently, since we weren't picking up whitespace.
Support Single Instancing of Files This is the same as the previous issue. Windows is well-known for storing multiple copies of the same file in a few different locations. The file-based approach made it easy to store only one instance of each file and use the file as necessary.
Support Media (Disc) Spanning Even in the early days of WIM, when we could fit an image on a single CD, we knew that we had to support multiple pieces of media. Even if we shipped Windows on one CD, once users added applications, service packs, and so on, their images would likely grow beyond the size of a single CD or DVD. So we had to let them span across multiple pieces of media for their own installation and recovery solutions.
Support Multiple Volume Images in One File Blob One image rarely works for everything. When we engineered "single instancing," we also included the ability to pick up more than one volume—and the same compression and single instancing works across each. So if you pick up an image of Windows XP Professional, and then append a new image of Windows XP Tablet PC Edition (which is a superset of Windows XP Professional), the image will only increase by the files that are explicitly different between the two versions.
As a result, an OEM or enterprise user can append many derivative images to each other without an exponential increase in size. This ability would not have been easy to achieve with sector-based formats.

How WIM Works
The WIM format was designed to be as simple as possible, while meeting our guiding principles. The file-based approach allowed us to do this. But how exactly does it work?
In a nutshell, you can think of a WIM file as a CAB file (this is similar to the ZIP format, but Microsoft designed and owns the CAB format). In fact, I once heard WIM referred to as "son of CAB." The key difference between a WIM file and a CAB file is that, in addition to capturing and compressing the file itself, a WIM image stores metadata that applies to the files and directories that make up the volume captured in a given volume image. So it's sort of like an archive file with all the meta information needed to restore the volume to the way it was when the image was created—access control lists (ACLs), short/long file names, attributes, and so on.
Note that at no point during the capture process is partition information (size or type) gathered. Nor does the apply process partition the system. Unlike most imaging tools, ImageX, being partition neutral, requires that the partition be created and formatted prior to application. To automate the process before applying your image, you should use the diskpart and format command-line tools.
Nothing limits what can be stored in a volume image. An image can contain two volumes, the same volume captured on a Tuesday and a Thursday, an image with a service pack and one without, or anything else that meets your needs. The main thing to understand is that files that are identical will be captured only once. The less that two volume images have in common, the more space will be taken up.
I should also point out that there is nothing designed into the WIM format that precludes it from being used with earlier versions of Windows. It will work just as well with Windows 2000 as it will with Windows Vista. Now let's take a closer look at what exactly happens when images are captured and applied.

Capturing an Image
As an image of a volume is captured, the following steps take place:
Volume Metadata is Captured ImageX gathers data about file names, NTFS ACLs, and other file system attributes across the volume. (You can opt to exclude certain files with a script file.)
File Data is Captured File is loaded by ImageX. Each file is loaded on the volume and prepares to collect certain data about it.
File Hash is Generated A cryptographic hash of the file based on the file itself is generated. This hash becomes the unique identifier of the files in the image.
Duplicates are Checked for and Removed If another file with the same hash already exists in the image, it is assumed that this is the same file and the new file is referenced by pointing to the already existing file.
Unique File Data is Compressed Unlike in a CAB or many other archive formats, each file in a WIM file is compressed individually rather than as a collection of files grouped together as one data stream. You can also opt not to compress at all, in which case this step is skipped.
Volume Metadata is Compressed and Stored Once all of the files have been archived, a metadata entry for the volume is created. This lists each of the files that were added to the image for the volume.
Image XML Data is Generated and Stored This is a reference for each volume image that is captured. If you run ImageX /info, for example, this XML data is what you'll see.
Cached WIM Data Index is Written Finally, the master data index (along with the cached data) is written. This is the master file table for the entire WIM; it includes one entry per hash.
As you can see, a WIM is created on a file-by-file basis. Furthermore, you have one WIM file, regardless of the number of volumes that you append to the image.

Image Overload
There were a number of teams developing similar solutions when we started work on WIM. The Automated Deployment Services team was developing an imaging solution for a different customer segment. Their approach had some very interesting low-level implementation details, but their imaging engine, like most at the time, was sector-based. The Windows XP Embedded team also built a sector-based imaging engine to allow embedded independent hardware vendors (IHVs) to deploy their own images at system manufacturing time. And when the Virtual PC team was acquired by Microsoft in 2003, they brought along another file format for Virtual Hard Disk (VHD) files.
All of these formats were at the core sector-based. They met the needs of each of their customer segments, but these solutions wouldn't meet our guidelines for WIM. As a result, we continued building the new WIM format, and the ImageX command-line tool to capture images. Eventually, a number of other teams adopted our format well before it was seen publicly as part of the Windows Vista beta. The SMS team, for example, delivered an early implementation of WIM in their Operating System Deployment Feature Pack.

Applying an Image
Now let's look at the counter process: applying an image.
Cached WIM Data Index Loaded This correlates to the final step in the previous procedure. Since this is the master index, it is the first thing to be loaded from the WIM after you've specified which volume image to apply.
Image Metadata is Retrieved and Loaded ImageX loads the metadata specific to each volume image.
Directory Structure is Created To ensure that every file has a location to be restored to, the volume's entire directory tree is created.
File Data is Extracted The files are extracted from the WIM, in the exact same sequential order in which they were archived.
File Data is Applied In this step, files are loaded, decompressed (if the volume image was compressed in the WIM), and copied to each referenced location. Finally, all metadata for each file is applied.
All Directory Metadata is Applied It is important that this is the last step since there is a potential that an ACL on a directory could preclude the image from being applied correctly.

Flexible Compression
One of our key goals with the WIM imaging format was to provide flexible compression options. So we provided the ability to capture a volume image with compression when necessary, while also allowing for image capture without compression. The latter option is good for scenarios where the image doesn't need to be aggressively compressed or when capturing at high speed is critical.
There are a few key tenets to bear in mind when it comes to compression. Compression always takes longer than decompression. The more aggressive the compression algorithm, the longer it will take to compress a given set of data. And the more data you feed to a compression algorithm, the better it will compress.
WIM images can be compressed using XPress or LZX compression—and, of course, no compression is an option. XPress is a good compromise for when you need some compression, but you don't want to dedicate too much time to the capture process. LZX provides the optimal compression ratio—but, as you can guess, this method takes considerably longer.
Using no compression results in the fastest image capture, but it also results in the largest image. The compression type is a per-WIM option. Once you've captured one volume image into a WIM, all future volume images must be captured with the same compression setting.
Note that because WIM images compress on a file-by-file basis, the compression isn't as complete as it could be. This was a conscious decision—we did this so that the image would be more easily editable. The ability to edit images after capture is an important option. For a file to be edited after the image has been captured, the file data and its metadata must be replaced. If we had compressed the entire WIM all at once, support for editing files would have been incredibly complicated for us to engineer (if it were possible at all) and replacing a file in the image would take a considerable amount of time. With the compromise we made, we came up with a format that is well compressed and can still be edited using our tools.
I hope this gives you a good understanding of how the imaging capabilities we designed for Windows Vista work. In my next column, I'll discuss some of the tools you can use to manage and deploy WIM images, taking a closer look at ImageX and Windows Deployment Services.

Wes Miller is a Development Manager at Pluck (www.pluck.com) in Austin, Texas. Previously, Wes has worked at Winternals Software in Austin, and at Microsoft as a Program Manager and Product Manager for Windows. You can get in touch with Wes at technet@getwired.com. Wes would like to express his gratitude to John Macintyre, the current WIM/ImageX Program Manager at Microsoft, for his assistance with this column.


 Windows Confidential: Waiting for RunOnce
Windows Confidential
Waiting for RunOnce
Raymond Chen

CONSIDER AN installation that needs to restart the system in the midst of the process. For example, the program might wish to upgrade a system DLL that is currently in use. You would need to trigger an intervening reboot to get the new DLL file installed, and then after the DLL has been upgraded, you could use it in the next stage of the installation.
The RunOnce key was intended for this sort of scenario—to be used by setup programs to complete their installation after an intervening restart. (This was back in the days when programs were trusted not to screw up.)
Windows® Explorer waits for the RunOnce program to run to completion before it creates the desktop and taskbar. This is because RunOnce operates under the principle that it is for setup programs. The RunOnce program could be changing system configuration data and you wouldn't want the user using a system while it's being reconfigured.
Say, for instance, the RunOnce program is rebasing and binding the files it had just installed—and some of these are part of a shell extension. You certainly don't want the user trying to run the program or activate the extension while its executable is still in the process of being updated.
Another pitfall of the "it's for setup programs" principle is that the RunOnce key is processed only if a user with administrator privileges logs on. Thus, users who aren't administrators wouldn't be able to complete the installation. Besides, if the RunOnce key were executed for non-administrators, it would no longer "run once." In fact, it would create quite a dilemma. How would you say, "OK, I ran that one; don't run it again"? To change this system-global state, you would need administrative privileges. And, obviously, the non-administrative user doesn't have administrative privileges! The program wouldn't be "run once" anymore; instead it would be "run over and over again until an administrator logs on."
RunOnce, which is intended for finalizing program installation, is visible in Windows setup itself. When you try to install a service pack or upgrade to a newer version of Windows, the setup wizard checks the RunOnce key. If it's not empty, you will get an error message that says, "Hey, you still have some unfinished program installation in progress. Please finish that before you try to upgrade the operating system." This is fair since you wouldn't want to change the operating system right smack in the middle of a program's installation.
The whole design of RunOnce reveals its past. It was hatched in Windows 95, an operating system that did not make a distinction between administrators and non-administrators. Since everybody, in essence, was an administrator on Windows 95, a setup program that used RunOnce could rely on the RunOnce actions being performed upon the next restart.
On more recent Windows operating systems—this is especially true for Windows Vista™—it's possible that nobody with administrator privileges will log on for months at a time. As a result, a program that needs RunOnce to run may find itself in for a long, long wait.
As an aside, I've seen a perpetual RunOnce. It readded itself to the RunOnce key whenever it ran. This clearly goes against the intent of the RunOnce key, since this is no longer a part of the program installation but rather part of the program's normal operation. What's more, this behavior causes Windows setup always to report that there is an install in progress.
Thus, the user can never upgrade the operating system because this one pesky program is forever saying, "I just need to do a few more little things and I'll really be done." The program will not finish installing today, "but surely tomorrow." It's not unlike waiting for Godot—but hopefully without the existential crisis.

Raymond Chen's Web site, The Old New Thing, deals with Windows history and Win32 programming. He is currently working on a book, coincidentally titled The Old New Thing (Addison-Wesley, 2007).