 TechNet Magazine October 2006

TechNet Magazine October 2006

Communication & Collaboration: Get into the Groove: Solutions for Secure and Dynamic Collaboration

Office Groove 2007 lets users collaborate dynamically and securely with people from around the world. Find out how the Groove client can fit into your organization.

Yung Chou Communication & Collaboration: Building an Emergency Operations Center on Groove and SharePoint

Take an inside look at how Groove and SharePoint help the Louisiana State University Emergency Operations Center respond to disaster.

John Morello Communication & Collaboration: Reveal Deeper Meaning: Powerful Exchange Server Analysis with LogParser

There's a lot of useful data hidden in your Exchange Server logs. Learn to use LogParser to examine this data and improve your Exchange environment.

Michael Higashi Communication & Collaboration: Fight Spam on Your Terms with Custom Weight Lists

Explore the new features in Exchange Server 2003 SP2 that offer more powerful ways to stop spam from clogging up your network.

Cam Frenette and Alexander Nikolayev Communication & Collaboration: Achieve High Availability with Exchange Failover Clusters

Don’t let your Exchange Server system go offline. Here’s a real Exchange solution that allows an enterprise to recover quickly from an e-mail outage.

Mark Godfrey Deployment: Boot into RAM with Windows PE

With added support for variable boot configurations, Windows PE has become even more flexible. Learn three different ways to boot into RAM and how this can benefit your next deployment.

Wes Miller Deployment: Make Server Rollouts a Snap with Automated Deployment Services

Intrigued by standardized images, faster rebuilds, and consistent supportability? Automating the deployment of servers offers benefits for any organization—big or small. Here’s what you need to know to get started.

John Savill Deployment: Create More Flexible Office Deployments with the Local Install Source

LIS can greatly simplify Microsoft Office support. This overview covers everything from deployment to maintenance to configuration considerations.

Dean Yamada Small Business Server: Master Your Domain: Build a Corporate Network at Home

Want a home network that’s just as powerful as the one at work? Here’s an affordable solution to bring home secure messaging, shared resources, single sign-on, and everything else you’ve come to expect from your corporate network.

Matt Clapham and Jesper Johansson From the Editor: Downtime is a Four-Letter Word

WHAT'S MORE DISTURBING than snakes on a plane, exploding laptop batteries, and the badly deployed puns in this Editor's Note, all put together? It's the idea that the office you're responsible for might start to fall apart.

Joshua Trupin Toolbox: New Products for IT Pros

Greg Steen Utility Spotlight: File Server Migration Toolkit

Jeremy Moskowitz SQL Q&A: Running in RAM, Agent Jobs, and More

Edited by Nancy Michell Hey, Scripting Guy!: Setting Up Outlook E-Mail Signatures

The Microsoft Scripting Guys How IT Works: Windows Rights Management Services

Randy Muller Field Notes: Where Did the Net Go?

R'ykandar Korra'ti Windows Confidential: Compatibility Constraints

Raymond Chen


 Communication &amp; Collaboration: Get into the Groove: Solutions for Secure and Dynamic Collaboration
Communication & Collaboration Yung Chou

Get into the Groove: Solutions for Secure and Dynamic Collaboration
At a Glance:
  • What is Groove?
  • Teaming up across boundaries
  • Groove 2007 architecture

In today's global economy, your project teams may be spread across departments, offices, time zones—even continents. And thanks to rapid advances in computing technology and the
proliferation of broadband Internet and mobile devices, you face increasing expectations of service quality and availability. More than ever before you're likely to be working with people you've never met in person. At the same time, you need to work with team members in context and without worrying about the underlying complexities of secure communication and data synchronization.
Microsoft® Office Groove 2007 is a new collaboration tool in the 2007 Microsoft Office system that enables teams to collaborate dynamically, effectively, and with advanced security, even when team members work for different organizations, work remotely, or need to work offline. I'll introduce you to Office Groove 2007 and explain the architectural underpinnings that make it an ideal tool for dynamic team collaboration.
The Office Groove platform includes both decentralized client software (Office Groove 2007) for information worker productivity and centralized services (Office Groove Server 2007) for traversing network boundaries efficiently, enforcing IT management, and integrating data with other server-based systems. This article will cover only the Office Groove 2007 client software and the Office Groove Server Relay. Look for future coverage of the deployment capabilities of Office Groove Server Manager and the integration capabilities of Office Groove Server Data Bridge.
Office Groove 2007 is the client software that enables individuals to work as teams within collaborative virtual workspaces. Office Groove 2007 enables teams to accomplish collaborative tasks, even when team members work for different organizations, work remotely, or need to work offline.
Office Groove Server 2007 gives IT organizations enterprise-class server software and tools for deploying, managing, and integrating Office Groove 2007 clients across the enterprise. Among the components included in Office Groove Server 2007 is the Office Groove Server Relay. Office Groove Server Relay provides a store-and-forward service that automatically routes encrypted data changes between Office Groove 2007 clients when they can't connect directly. Also included are the Office Groove Server Manager and the Office Groove Server Data Bridge. I'll discuss those more in a future article.
Individual workers can use the Office Groove 2007 client software to form a team by creating workspaces, adding tools and data, and inviting other Groove users to join the workspace as team members.
As team members collaborate in a workspace, Office Groove 2007 works in concert with Office Groove Server Relay to keep each team member's copy of the workspace synchronized, even if no two users are online concurrently. The data within each Groove workspace is always protected by 192-bit Advanced Encryption System (AES) encryption, whether it's stored on a team member's local disk, being transferred between client workspaces on the network, or being temporarily stored on Office Groove Server Relay.
Collaboration Anytime, Anywhere
Collaboration Anytime, Anywhere

Using Office Groove 2007
From a user's perspective, Office Groove 2007 client software consists of five key elements: the launchbar, the workspace, tools, presence and communication, and alerts. The launchbar is the starting point for using Groove and provides easy access to a user's workspaces and contacts.
A Groove workspace (see Figure 1) is where you share information and work with others on a specific task or project. A workspace is intended for a small group of approximately 2 to 50 members. Common projects and tasks executed in a Groove workspace might include document reviews, project-specific meetings, threaded discussions, reviewing and sharing published content, tracking tasks or issues, and collecting or aggregating structured data.
Figure 1 The Groove Desktop
Figure 1 The Groove Desktop
Tools are applications that can be added to workspaces for sharing and manipulating structured and unstructured data. Structured data is captured with tools like the Groove InfoPath® Forms tool and the Groove Forms tool; unstructured data includes items such as documents, discussion threads, and images. Two new tools available in Office Groove 2007 include the SharePoint® Files tool for synchronizing files between a Groove workspace and a SharePoint document library and the InfoPath Forms tool for importing InfoPath solutions into Groove.
Presence and communication functionality includes built-in member presence awareness, workspace chat, messaging, and integration with Microsoft Office Communicator 2005 and 2007. Alerts are text and audio notifications of events and activities that direct user attention to the relevant events and activities in a workspace.

Teaming Up Across Boundaries
You don't need special network or domain privileges to create a new workspace or accept an invitation to join an existing workspace. Any user can create a workspace and send an invitation with Groove instant messaging or e-mail to prospective members. Furthermore, any Groove user can accept an invitation, join the workspace, become a team member, share information, and collaborate on a project without IT or network affiliations with other Groove members. This functionality is what enables dynamic team collaboration, even if team members are from different organizations or share no common infrastructure.
When you send a workspace invitation, you don't even have to worry whether the recipient is an existing Office Groove 2007 user. Recipients who don't already have Office Groove 2007 installed on their computer can be invited via an e-mail message and will be prompted to download a free trial version of Groove.
When you join a workspace, a copy of the workspace is transmitted over the network and stored on your computer. Office Groove 2007 can receive an initial copy of the workspace from any member who is online when the workspace invitation is created if the invitation is sent via Groove messaging. After the transmission is completed and a locally stored copy of a workspace is ready, you can enter the workspace and begin working with other members.

Staying in Sync
If a workspace member changes content in the workspace, a "delta" is created, representing a logical unit of workspace synchronization. The tool used to make changes determines what a delta contains. For example, a delta could be drawing a line in the Sketchpad tool (a shared whiteboard application that can be added to a Groove workspace), changing an entry on a calendar, adding a member to the workspace, or the binary differentials between a changed file and the previous version.
When a delta is created by a user, Groove first updates the user's local copy of the workspace, then sends the delta to the rest of the members for execution in their local copies of the workspace. When all members' copies of the workspace have introduced the delta, the workspace is synchronized. Though each member keeps a delta log locally with the associated workspace, Groove includes built-in logic to conclude that, after a delta has been executed by all members, the delta can then be deleted from the log.
If you are online, changes are synchronized in near real-time. If you're offline when deltas are executed by other workspace members, those deltas are sent to queues in your designated Office Groove Server Relay. The deltas are then consumed whenever you reconnect to a network and can establish a connection with the Office Groove Server Relay. In this way, all members' copies of the workspace can be updated or synchronized regardless of the network status of any single user at the time a delta is sent. Above all, once a member saves a change to a workspace, Office Groove 2007 and Office Groove Server Relay carry out the transmission of the delta and subsequent synchronization automatically and transparently to all members.
Factors such as how long a member has been offline, how much has changed in the workspace, how much network bandwidth is available, and how many members are online affect the time and computing resources Office Groove 2007 needs to automatically synchronize a workspace.
If a member has not entered a workspace for 21 days, Office Groove 2007 will no longer synchronize that workspace for the member. This design decision is based on the assumption that it is more efficient to simply re-invite the user and get a fresh copy of the workspace as opposed to synchronizing 21 days of data with unpredictable amounts and intricate data dependencies developed over time. To avoid this scenario, it's a good practice to leave Office Groove 2007 running while connected to the Internet whenever possible so Groove can synchronize soon after changes are introduced and keep the workspace content updated at all times.
If necessary, a workspace manager can uninvite a member from the workspace. To protect the privacy of the workspace, this action will consequently delete the workspace stored locally on the uninvited member's computer and generate a new workspace-specific encryption key for distribution to all remaining workspace members. If an uninvited member is online, deletion of the associated workspace occurs as soon as the uninvite message is received. If the user is offline, the uninvite message is queued at the designated Office Groove Server Relay and dequeued and executed the next time the user connects.

Data Transmission Details
The way Office Groove 2007 transmits data is in some ways similar to making a phone call. When Alice calls Bob, for instance, if Bob answers, a logical phone-to-phone connection is established. If the call is not answered, Alice might leave a voice message that Bob can hear the next time he dials into his voice message box.
Office Groove 2007 will always try first to initiate a direct client-to-client connection between users. The direct connection is carried out by a proprietary protocol, Simple Symmetric Transmission Protocol (SSTP), using TCP port 2492. This client-to-client connection is bidirectional and transmits data asynchronously with near real-time performance. If a connection on port 2492 is blocked or the endpoint client is not reachable on the network, the sender client automatically transmits messages to the recipient's designated Office Groove Server Relay, which queues up the messages to be consumed whenever the recipient reconnects (see Figure 2).
Figure 2 Client-to-Client and Client-to-Relay Communication Models
Figure 2 Client-to-Client and Client-to-Relay Communication Models
These two different connections—client-to-client and client-to-relay—do not necessarily imply a performance discrepancy that will be noticed by users. In most corporate network configurations, both connections offer a similar user experience.
A client-to-client connection is preferred, yet may not be allowed or available. A client-to-relay connection, using outbound connections from the client to the Office Groove Server Relay, is often the least common denominator. The client can initiate connections to the Office Groove Server Relay using one of three available ports and protocols: SSTP/:2492, SSTP/:443, and SSTP encapsulated in HTTP/:80. The client will automatically try each protocol/port combination in order. Office Groove 2007 can HTTP POST to the user's Office Groove Server Relay to publish the presence information and can HTTP GET to consume inbound messages and deltas. In addition, Groove clients can HTTP POST to an intended recipient's Office Groove Server Relay for outbound messages and deltas.
No matter which of the three connection types clients use, data is always encrypted and digitally signed to ensure confidentiality and integrity. At the same time, with HTTP encapsulations, Office Groove 2007 can communicate across firewalls and beyond any organizational boundaries.

Presence Awareness
Office Groove 2007 subscribes to presence information from all contacts in the contact store. The store is the superset of your contact list (visible from the launchbar) plus all workspace members. While it is true that when you have a contact in your contact list you subscribe to that person's presence, you also subscribe to his presence if he is not in your contact list, but is in a workspace with you. In other words, subscribing to presence occurs automatically when you learn of someone, either through workspace membership, or by adding to contacts.
Device presence is required first, though, before user awareness can be established. Device presence indicates that a user is logged into an Office Groove 2007 account on a particular computer, and the computer's IP address is transmitted to the user's designated Office Groove Server Relay. Device awareness within a subnet is achieved with LAN Device Presence Protocol (DPP), a proprietary protocol that provides discovery and online status of devices without the need of a server. LAN DPP works only within a subnet, however.
Device presence on a WAN requires a connection to an Office Groove Server Relay (see Figure 3). A user publishes device presence to his designated Office Groove Server Relay, and any contacts subscribing to the user's presence can connect to the relay and consume that device presence information automatically. WAN presence cannot occur without a relay being contacted. Once you receive another contact's device presence, you can go directly to that second contact to establish awareness.
Figure 3 Groove User Presence Awareness
Figure 3 Groove User Presence Awareness

Conflict Resolution
Because Office Groove 2007 employs a decentralized model for collaboration, data conflicts can occur from time to time and are handled differently based on the tool and data type. Whenever a user is online, Office Groove 2007 will synchronize the local copy of a workspace automatically with other members. But when a user is offline and making changes to a workspace, other online and offline members may be making changes as well. When reconnected to a network, the Office Groove 2007 client will identify the dependencies and serialize the changes, as applicable.
If there are conflicts, such as two people modifying the same data while offline, for example, Office Groove 2007 ensures that no data is lost, and that all users have the same view of the data. Many tools have discrete deltas, making conflicts in structured data very uncommon. For instance, Alice and Bob both respond to a discussion item at the same time. Regardless of network status, no conflict will occur since each entry is a self-contained transaction within the associated discussion thread.
For unstructured data such as documents, it is possible to create a conflict. One user may stay offline for an extended period after modifying a file. During this period, other members working online may have made changes and synchronized the file several times. In this case, Office Groove 2007 warns the user that there is a conflict on the document and automatically creates a copy of the file. Each file is titled with a name identifying the member whose changes are causing a conflict. At this point, user intervention is required to reconcile the conflict, and the points of conflict should be clear.

Roles and Permissions
Office Groove 2007 includes a built-in, roles-based permissions framework that affords content safeguards to workspace creators within the decentralized nature of a Groove workspace. In Office Groove 2007, a role is an access control mechanism for permitting users to perform tasks in a workspace and activities with individual tools. There are three built-in roles: manager, participant, and guest. By default, workspace creators are managers and those invited to workspaces are participants. Default permissions for each role can be changed by a manager. A manager can invite others as managers and a member's role can be changed by a manager.
The tool permissions associated with a role are customizable as well. The workspace rights and tool permissions associated with a role can therefore be configured at a workspace manager's discretion. This role-based model in Office Groove 2007 is an effective way of controlling access. Figure 4 illustrates some of the default permissions.
Figure 4 Default Permissions by Role in Office Groove 2007 (Close Figure 4)

Groove Feature | Manager Role | Participant Role | Guest Role
Workspace | Invite, Uninvite, Add tools, Delete tools, Delete workspaces | Invite, Add tools | None
Calendar | View, Add, Edit, Delete all entries | View all entries; Add, Edit, Delete own entries | View all entries
Discussion | View, Create, Edit, Delete all documents | View all documents; Create, Edit, Delete own documents | View all documents
Files | View, Add, Modify, Delete files and folders; Modify permissions and download settings | View, Add, Modify files and folders; Delete own files | View files and folders
Pictures | View, Add, Rename, Delete | View, Add, Rename | View
Notepad | View, Create, Edit, Delete | View, Create, Edit, Delete | View
Sketchpad | View, Create, Edit, Delete | View, Create, Edit, Delete | View

Conclusion
This article has given you a high-level overview of the decentralized, hybrid architecture employed by Office Groove 2007 and Office Groove Server 2007 to enable dynamic team collaboration. From an IT administrator's point of view, Office Groove 2007 and Office Groove Server Relay allow teams to work together efficiently both within and across organizational boundaries, both on and off the network, without compromising data security or significantly increasing server infrastructure costs.
To learn more, visit the Office Groove 2007 Web site at microsoft.com/office/preview/programs/groove and the Office Groove Server 2007 Web site at microsoft.com/office/preview/servers/grooveserver. And there's no better way to learn about Groove than getting your hands dirty and testing it out for yourself. Sign up for the beta program today at microsoft.com/office/preview/beta.

Yung Chou is a Technology Specialist for Microsoft Office Groove. He is both a Microsoft Certified Systems Engineer (MCSE) and a Certified Information Systems Security Professional (CISSP).
© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


 Communication &amp; Collaboration: Building an Emergency Operations Center on Groove and SharePoint
Communication & Collaboration John Morello

Building an Emergency Operations Center on Groove and SharePoint
At a Glance:
  • Emergency Operation Center design basics
  • Building on Groove and SharePoint
  • Lessons learned and best practices

Last year, as the rest of the world watched on television, Louisiana faced one of the worst disasters in U.S. history. Hurricane Katrina hurtled through the Louisiana and Mississippi Gulf Coast
on August 29th, destroying a good part of New Orleans and killing hundreds of people. In the aftermath, as thousands of evacuees fled the affected area, Louisiana State University (LSU) in Baton Rouge, 80 miles northwest of New Orleans, became a primary center for medical operations. Amid the chaos that reigned, many dedicated LSU staff worked tirelessly to help these evacuees. As part of that effort, another Microsoft employee, Matthew Lonergan, and I helped LSU use Microsoft® Office Groove to manage the vast numbers of people needing assistance.
This is not another article about Katrina, though, but one about the proactive steps LSU has taken to increase its preparedness for the next emergency, no matter what it is or when it occurs. In the months after the storm, realizing the great importance of having a centralized command and control system during emergencies, LSU instituted a formal Emergency Operations Center (EOC), which became the all-hazards C4I (command, control, communications, computing, and intelligence) center for LSU. Led by LSU's Chief of Police, the EOC includes a small number of key personnel from many LSU departments including public safety, facilities, public relations, procurement, and information technology. This article focuses on IT and explores the IT needs in an EOC and the types of technology that can help satisfy those needs.
Weather Map of the Site Hit by Katrina
Weather Map of the Site Hit by Katrina
When dealing with a crisis situation, there are two major classes of need technology can help satisfy. First, in any emergency there is a need to collect and store information. After Katrina, we tracked evacuees and medical volunteers. In future disasters, we may need to track specific supplies and equipment. Second, as a corollary to data collection, there is a need to share information. After data is gathered, you must be able to share it in ways that meet the needs of the mission at hand. For example, medical volunteer data must be made available to the police at the medical evacuation site so they can verify the identity of anyone claiming to be a volunteer.
IT systems operating in a crisis environment are faced with a number of constraints not commonly seen in typical day-to-day IT operations. This means that many common technologies used to satisfy the major needs identified earlier are not effective in an emergency environment. For example, information sharing within an organization is often done through file shares and access is controlled with access control lists (ACLs) based on Active Directory® users and groups. In an emergency, organizations will likely be collaborating with groups and individuals that don't have accounts in the organization's Active Directory and who may not have connectivity to file shares.
Based on our experiences with Katrina, we identified a number of constraints often faced by IT in an emergency. First, network connectivity and bandwidth are not guaranteed. While many organizations, including LSU, have large-scale wireless deployments and fast-wired connections, how many systems could sustain a ten-fold increase in demand? In an emergency situation, with mass mobilization of resources and spotty power, organizations can't rely on having fast, reliable connectivity.
Second, the Internet isn't always there. This may seem like a given in light of the previous constraint, but it's important to highlight the potential frailty of an Internet uplink. While you may have redundant hardware and fully meshed routing and your ISP may have the same, the local power company may not be able to restore power to them before backup generators run out of fuel.
Third, the definition of "users" expands greatly. Normally, an IT group considers its own organization's staff (and perhaps its customers and business partners) as its user base. Thus, identity provisioning and management systems, staffing levels, and general business processes are designed with a relatively well-established number and range of scenarios in mind. How many organizations are ready to bring potentially hundreds of volunteers, contractors, and various civilian and military governmental staff into their information systems in a rapid and secure way?
Finally, people need to access data using a variety of machines and connectivity scenarios. New users will often bring their own computers, configured in their own ways with their own sets of applications on them. You won't be able to use Systems Management Server to do a mass installation of Office or another application to provide a common baseline of software.
Understanding these needs and constraints led us to make some very specific choices about the types of systems we deployed in LSU's EOC. The system we designed needed to be able not only to easily collect data, but also to share it. We needed to be able to share data securely with people on our network, with those connected to random Internet access points, and with those connecting from other government agencies. We needed to be able to share this data effectively and with little user interaction, even when network connectivity was sporadic and bandwidth poor. We needed to be able to rapidly provision new users into the system without requiring them to authenticate into our Active Directory. And we needed to have a software platform that could be quickly and easily installed by the users themselves, without requiring any specific installation source.

EOC Design Basics
Given these needs, we built LSU's EOC around two Microsoft collaboration products—Groove and Windows® SharePoint® Services (WSS)—plus the infrastructure products that support them, mainly Windows Server® 2003 R2 and SQL Server™ 2005 (see Figure 1). Groove and WSS complement each other to satisfy the information collection and sharing needs, and they're ideally suited for operating in the constrained environment of an emergency situation. However, to provide a platform suitable for use in an EOC, we had to look at the solution from the bottom up, ensuring that everything from the physical plant to Active Directory was configured in a way that allowed these products to work optimally in an emergency.
Figure 1 Architecture of the EOC Computing Environment
Figure 1 Architecture of the EOC Computing Environment
There are both obvious and non-obvious parts of an EOC's physical plant that impact its IT operations. At LSU, the EOC (shown in Figure 2) is located in the university's Public Safety building, which also houses the LSU Police Department. So, in this case, physical security is already quite strong. There is redundant power to the building and directly attached generators with sufficient fuel for long-term operations without grid-provided current.
Figure 2 Interior of the University's EOC
Figure 2 Interior of the University's EOC
Less obvious is the fact that the machine room is not adjacent to any exterior wall and that it has a cooling system that can be powered separately from the rest of the building's air conditioning. When running on generator power, this allows us to save fuel by keeping the machines properly cooled even if the people aren't.
Another non-obvious aspect of the physical design is the server itself. Rather than a high-end quad-CPU server, we're using a relatively basic dual-CPU system to increase power efficiency. We're also using local storage on the WSS computer to eliminate the need to power a storage area network (SAN), network attached storage (NAS), or other external chassis. Finally, we've ensured that not only the servers but also the workstations, projectors, and networking equipment are on an uninterruptible power supply (UPS).
Because we need to ensure continuity of operations even in the event of a total power loss, the WSS environment is contained within an Active Directory forest completely separate from the primary LSU Active Directory. This forest, which is dedicated for use by the LSU Police Department, is wholly contained within the physical plant I've described. Thus, even a complete campus-wide outage of power or network connectivity will not affect operations within the EOC.
The EOC's Active Directory is based on Windows Server 2003 and uses a relatively simple single-forest, single-domain, single-site model. Two domain controllers (DCs) provide redundancy of directory services and each DC performs a system state backup on a daily basis. These backups are replicated to both onsite storage and (assuming the network is available) to an offsite storage location elsewhere in the state.
User objects have been provisioned for permanent EOC staff members. These accounts are used solely for access to the SharePoint site and are not used for logon at the workstation. Each EOC workstation is dedicated to a particular EOC role, such as Operations Commander, Logistics Specialist, and so forth. Because the EOC is designed to operate around the clock, the people filling these roles will change throughout the day. No data is stored locally on workstations, only on SharePoint and in Groove. Though normally this isn't a recommended best practice, in this case there is little value in having individuals log on to their workstations using specific accounts. Instead, we use a dedicated, heavily restricted user account across all EOC workstations that is configured to automatically log on after system restarts. This allows us to rapidly transfer anyone into any given role within the EOC without having to provision new accounts or manage passwords. It also has a significant advantage for Groove synchronization, as I'll detail later.
The DNS design within the EOC forest is also relatively simple. We utilize a dedicated, non-registered suffix with all zone authority held by the DCs. DNS forwarding is used and references LSU's primary public name servers for both LSU and Internet names.
Because the EOC is within the Public Safety building, it is segmented from the rest of campus and from the Internet at large. In many ways, it more closely resembles the classical corporate network design of a border firewall—with limited ingress ports and translated internal address spaces—than the more open designs often found in higher education. Access to the SharePoint site from the Internet is restricted to HTTPS, and no other connectivity is allowed from the public Internet to the EOC network.
Egress access from the EOC network to the Internet occurs through the local router's Network Address Translation (NAT) capability. Egress filtering is limited because of the variety of protocols used by agencies that may partner with the EOC in case of emergency. For example, assisting staff from local, state, and federal agencies may each require outbound connectivity to a different set of ports and destinations to facilitate access back to their organizations' networks. Because of the way Groove utilizes relay servers, workspace synchronization can also occur over this NAT connection. In the future, if we choose to implement more strict egress filtering, Groove would still function by wrapping its communications within HTTP frames for outbound access.

Building on Groove and SharePoint
Now that we've established the basic infrastructure components, let's look at the solution they support. SharePoint is the core of our EOC communication platform, providing an easy-to-use, full-featured Web site for storing documents, contacts, and task lists. Because it's a Web-based application, we can easily make its content available to anyone physically in the EOC building or anywhere on the Internet (assuming we have network connectivity). What we can't do with SharePoint is take the data offline, and that's where Groove comes in.
Groove provides key pieces of the overall solution. Through its Mobile Workspace for SharePoint, Groove allows us to take the entire SharePoint site and make it available offline. Groove users synchronize the contents of the SharePoint site to their local computers and can then view, edit, and add to the site even if they don't have a network connection. When they reconnect, the content is automatically synchronized back to SharePoint for the rest of the EOC to use.
Using Groove forms, we can quickly and easily collect large amounts of data in any emergency situation. During Katrina, we used Groove to track the location and health of thousands of evacuees and volunteers. Users could take a Groove-equipped laptop to a site with no power or network connectivity, collect as much information as battery power allowed, then return to a place with network connectivity. As soon as the Groove client reconnected back to the workspace, it began synchronizing the data collected offline with all its peers (see Figure 3). This is a tremendously powerful capability because it allows us to rapidly integrate dispersed information into a single distributed database.
Figure 3 Synchronizing Data through Groove and Sharepoint
Figure 3 Synchronizing Data through Groove and Sharepoint
Groove forms are easily created and modified, but their most important capability during a crisis is autonomous deployment. Once a user is provisioned into a Groove workspace, any forms created or modified in that space are added automatically to their workspace during the normal sync process. This allows us to react quickly to changing data collection needs on the fly without having to expend any effort updating the clients. For example, during Katrina, once we learned of the problems people were having locating friends and family, we modified our evacuee data collection form to include a "Transferred to" field that allowed us to provide much better intelligence on the location of anyone who had been through our medical centers.

Groove Design Elements
Because of the criticality of the EOC and the fact that Office Groove 2007 is still in beta, our current EOC software stack is built on an earlier Groove 3.1 release. Before the beginning of the next hurricane season, we plan on upgrading to Groove 2007 and SharePoint Server 2007, especially because of the advances in the integration between the two products. For now, though, our Groove deployment consists of the EOC workstations themselves, the laptops of some permanent EOC staff, and the machines that control our wall-mounted displays. The display systems allow us to project content in Groove onto the large displays in the EOC for easier analysis.
On the EOC workstations, Groove is configured with role-based accounts and automatic logon. As mentioned previously, EOC workstations are role-based so we have individual Groove accounts set up on a per-role basis. In other words, we have separate Groove accounts for the Incident Commander, Operations Commander, and Public Liaison roles, but not individual per-user accounts for the people who fill these roles. This approach allows us to easily transition work between shifts and to facilitate better collaboration, because whoever may be filling a role at a given time fully assumes its identity. It also makes tracking easier, since all events based on user names are represented by roles rather than individuals. For example, when viewing a discussion board or chat transcript, postings will be identified by role, making it easier for volunteers who don't know EOC staff members by name to understand. This approach also helps avoid creating replication storms during shift-change operations when all staff rotate out, which would require complete workspace synchronizations if we were using per-user accounts.
We utilize Groove Hosted Services to easily create and manage accounts on our own. The Hosted Services interface allows us to import account information from Active Directory or from Microsoft Excel® workbooks, making bulk account creation straightforward. Given our requirement to operate in the complete absence of any external network connectivity, we've already created all of the accounts we need and activated Groove on each workstation. This means we can continue using Groove even without any connectivity to the Hosted Services interface.
Two primary workspaces are used in the EOC. The first is the Mobile Workspace for SharePoint, which simply takes SharePoint offline and allows the synchronization of portal data changes between the Groove and SharePoint databases. The second is the more traditional type of workspace, in which we have forms for collecting data. We have pre-provisioned our workspaces with forms used for tasks we know will be important in any emergency, such as tracking evacuees or patients and tracking medical volunteers. As noted earlier, we can quickly update these forms automatically as new needs crop up. One of the important features of Groove 2007 is its integration with Microsoft Office InfoPath®, which will make form creation an even easier process.

SharePoint Design Elements
The EOC SharePoint design is quite simple. We use the basic Team Workspace template to provide the standard portal toolset. The primary tools we use are the Document Library, Links, and Contacts. Using Links and Contacts gives us an easy way to have shared favorites and address books; in combination with Groove, we have these data sources available offline as well.
One custom page we created was to drive the wall-mounted displays and show a list of major issues as they occur. To do this, we took a Custom List tool and optimized it for the wall-mounted displays, which included removing borders, headers, and tables to conserve screen space and adding a meta tag to force the page to refresh every 15 seconds. Because with SharePoint any user can update this list through a Web browser, it becomes simple for staff to post an alert to the wall-mounted displays.
SharePoint must be backed up using the Stsadm.exe backup and restore tool. Like the system state data from a DC, this backup is replicated both to local storage and to an offsite location. Because this backup can be easily restored to any Windows Server 2003 R2 system—even in a drastic case where the EOC itself is destroyed, we can set up our operations using the same toolset anywhere we can put a machine running Windows Server. With a few minutes of data restoration and reconfiguration of the Groove Mobile Workspace, we'd be able to continue using our existing set of tools and processes.

Best Practices
Here are a few of the lessons we learned in the process of setting up the EOC.
Groove Synchronization Limitations Groove workspace synchronization slows as the total number of records in the workspace approaches 10,000. In most EOC activation scenarios, 10,000 records are likely far more than is needed. However, in a large-scale disaster like Katrina, we did track more than 10,000 evacuees, which caused significant performance degradation of the Groove client. Thankfully, the solution to this problem is simple: as the total number of items in any given workspace approaches 10,000, create a new workspace and begin using it for any new items. Note, however, that this approach does make querying somewhat more complex since the data then exists in multiple workspaces, each of which would need to be queried individually. Luckily, however, a single instance of Groove on a single computer can be joined to many workspaces so all data is still available through the Groove client.
Another approach may be Groove Data Bridge, which allows an organization to develop a synchronization link between Groove and a traditional database system like SQL Server. The data bridge could be used to archive older records into SQL Server for more permanent storage, keeping the number of active records in Groove within the synchronization limit. At LSU, we chose not to use this approach primarily because of the complexity involved and the relatively infrequent need to work with such large datasets.
Groove Deployment and Activation Constraints Because of the hosted aspect of some Groove services (notably Groove Relay Server used by clients to find each other over the Internet), the Groove client must be activated prior to use. Since one of the primary objectives of the EOC's IT design is to facilitate disconnected operations, we obviously don't want to be dependent upon having an Internet connection available during an emergency. Furthermore, because each instance of Groove is uniquely dedicated to a particular user, it's not possible to deploy fully activated installations of Groove using traditional tools like Systems Management Server.
Because of these constraints, we deployed Groove as part of a standard image to all EOC workstations, but in a non-activated state. This meant that after imaging these systems, the Groove application binaries were available, but the product was not activated and no workspaces were joined. Thus, to complete our deployment, we created all of our role-based accounts (using the Hosted Services Web-based management tool) and manually activated each machine and joined it to the appropriate workspace. This provided a good balance between efficiency of deployment and the product's unique post-installation configuration needs.
Have Generic Role-Based Accounts Ready for Volunteers During a major emergency, there will most likely be many volunteers who can input data into the Groove workspace but who are not permanent EOC staff members and don't have assigned roles. Having pre-staged user accounts created for these data entry purposes allows LSU to more rapidly tap the services of such volunteers during an emergency. In our design, we have a number of generic accounts pre-built and ready for use by volunteers or any third party we may need to collaborate with.
Have the Groove Enterprise Installer Ready While Groove can be installed from www.groove.net, again we do not want to have dependencies on Internet connectivity during an emergency. Even if Internet connectivity is available, the installer on www.groove.net downloads files on demand, meaning it must download the entire application individually to each machine where it's used. In a bandwidth-constrained environment, this process may not complete quickly or successfully, or may consume bandwidth that could be used for higher purposes. Thus, rather than relying on the Web installer, use the Groove Enterprise Installer, which is an MSI-based installation package. Like other MSI-based packages, the installer can be fully contained on a local network share or USB disk and has no Internet dependencies during setup.
Groove Mobile Workspace for SharePoint Synchronization An important point to keep in mind about the Mobile Workspace for SharePoint is that only the computer that establishes the workspace can synchronize between the two environments. In other words, while all other Groove synchronization is peer-to-peer, the Mobile Workspace for SharePoint is one-to-one and only between the original creator of the workspace and SharePoint. Because of this, it's vital that the workspace be created on a machine that's always on and always connected to the network. In our case, the machine used to drive the wall-mounted displays is an ideal choice for this task.

Conclusion
An EOC is a unique environment for most IT professionals. Many of the services often taken for granted, such as high bandwidth and Internet connectivity, cannot be relied on during an emergency. Thus, the systems put in place to manage emergencies must be able to deal with scenarios where connectivity may be spotty and users widely dispersed, but where it's more crucial than ever to have secure, reliable access to data. By combining SharePoint and Groove, LSU has been able to build an EOC that's survivable and able to come to the needs of the university and the people of Louisiana even in the face of serious disasters.

John Morello spent six years with Microsoft as a Senior Consultant and is now the Deputy CISO at Louisiana State University.


 Communication &amp; Collaboration: Reveal Deeper Meaning: Powerful Exchange Server Analysis with LogParser
Communication & Collaboration Michael Higashi

Reveal Deeper Meaning: Powerful Exchange Server Analysis with LogParser
At a Glance:
  • Analyzing Exchange front end server log files
  • Using Log Parser commands
  • Generating reports via Office Web Components
Download the code for this article: HigashiExchangeLogs2006_10.exe (152KB)

As an IT professional, it's essential that you have access to information about how your users are impacting your servers. This is especially important when it comes to mission-critical
systems, such as e-mail servers. In this article, I will create a series of reports that help you see how users are impacting your Microsoft® Exchange 2003 front-end servers.
The Exchange front-end servers are built on top of IIS. If you look in IIS Manager on your Exchange Server 2003 front end machine, you will see a virtual directory hierarchy. When a client makes a request to one of the Exchange services that reside on an individual virtual directory, the service logs data about the command. These IIS logs contain some interesting information, but trying to read them can be frustrating. Fortunately, Log Parser lets you parse the IIS logs and dissect the info using a SQL-like query language.
The virtual directories that I will focus on in this article are Microsoft-Server-ActiveSync (for Mobile Devices), Exchange (for Outlook Web Access, or OWA), and RPC (for Outlook RPC over HTTP). I'll dive into each of these Exchange Server components and show how you can extract some interesting information to better understand what's going on across your entire Exchange Server 2003 front end. I will discuss five handy reports, showing you the SQL statement, the Log Parser command-line syntax, and the results you can produce for each. Jeffrey A. Johnson, a Senior Systems Engineer at Microsoft.com, helped me create some of the scripts I'll show.

Using Log Parser Commands
To help you make sense of the data that has been logged by your Exchange servers, I will show you how to create SQL queries and Log Parser commands that gather data and output it to a readable format. If you aren't already familiar with Log Parser, this is a powerful tool that offers query access to text-based data. It can be used with a number of data sources, including XML files, the Registry, and Active Directory®. You can download a copy at go.microsoft.com/fwlink/?LinkId=71149.
To format the data you are gathering, you pass in switches (or parameters) when running the Log Parser executable. Microsoft Office Web Components provides the mechanism for creating the actual charts. The Office Web Components are a collection of COM objects that allow you to create charts programmatically. Log Parser has built-in functionality to call the Office Web Component APIs so you do not need any special coding skills. All you need to know is what type of chart you want to create, specify it in your Log Parser command, and—BAM—you've got a pretty chart. Here's an overview of the switches I will use in my examples:
-o:<output-format> This is where you specify whether you want the output to be a CHART, CSV, or the default—NAT (which stands for native format and outputs to the console window).
-ChartType: You use this to specify what type of chart you want to display. To see a full list of available types, enter the following in the command line:
LogParser  -h  –o:Chart
-values: This lets you control whether the actual values are displayed on the chart.
-categories: Use this to specify whether you want the categories, usually the first column in the table, to be displayed on the chart.
-ChartTitle: This lets you give the chart a title.
-View: This setting lets you automatically see the chart once Log Parser completes processing.

Component Distribution Report
This report provides an overview of what Exchange services are being used the most. When tracked over time, this data can help with capacity planning or trend analysis. Based on these reports, you can put together an automated reporting solution that provides data you can use to examine usage patterns over a longer period of time.

SQL Statement and Log Parser Command
This SQL query deals with a single column in the IIS logs: cs-uri-stem. It uses Log Parser's Extract_Token function to parse the string and get the value after the first /. It also uses TO_UPPERCASE to format the data and force the values to be upper case. Since I know what virtual directories to look for—EXCHANGE, MICROSOFT-SERVER-ACTIVESYNC, and RPC—I can filter for those particular records. To do this, I use the CASE function and evaluate each incoming record to see if it matches any of these three Exchange services. If a record doesn't match, I drop it into the bucket called "other".
Next I need to gather the actual counts for each service. I use the COUNT(*) function, which simply counts all records that come into one of the three buckets.
Copy the SQL statement shown in Figure 1 into a text file and save it as TiSP2_Component_Breakdown.sql. You can use a different name for the file, but just be sure your Log Parser command calls that file name.
Figure 1 TiSP2_Component_Breakdown.sql (Close Figure 1)
SELECT
    CASE TO_UPPERCASE(extract_token(cs-uri-stem,1,'/'))
        when 'EXCHANGE' then 'Exchange'
        when 'MICROSOFT-SERVER-ACTIVESYNC' then 'Exchange ActiveSync'
        when 'RPC' then 'RPC'
        else 'Other'
    End as ExchangeComponent ,
    count(*) as Hits,
    mul(100.0, propcount(*)) as Percentage

INTO c:\LogParser\Output\TiSP2_Component_BreakDown_Pie3d.gif

FROM c:\Logs\SP2\ex06051423.log

GROUP BY ExchangeComponent

ORDER BY hits desc
For this example, I am outputting the results into a three-dimensional pie chart using the Microsoft Office 2003 Web Components. With all the switches included, here's what my command line looks like:
LogParser.exe -i:IISW3C -o:CHART -ChartType:Pie3D -values:OFF  -
categories:On -ChartTitle:"Component Breakdown Report" -view:ON file:
C:\Technet\TiSP2_Component_Breakdown.sql
And the resulting chart will look something like Figure 2.
Figure 2 Charting Exchange Service Usage
Figure 2 Charting Exchange Service Usage

OWA Activity Report
This report can help you drill further into OWA activity, letting you see which tasks users perform most frequently.

SQL Statement and Log Parser Command
This SQL statement has a similar construct to the one in the previous example, but this time we are looking for the value following the third "/". This will give us information about what data the user is trying to access from OWA, such as the Inbox, Calendar, Contracts, and so on. For this report, we use the WHERE clause to filter only OWA data. Figure 3 shows the complete SQL statement—place it in a new file named TiSP2_OWA_Activity.sql.
Figure 3 TiSP2_OWA_Activity.sql (Close Figure 3)
SELECT
    case extract_token(cs-uri-stem,3,'/')
        when 'Inbox' then 'Inbox'
        when 'Calendar' then 'Calendar'
        when 'Contacts' then 'Contacts'
        when 'Tasks' then 'Tasks'
        else 'Other' end as Activity ,
    count(*) as Hits

INTO c:\LogParser\Output\TiSP2_OWA_Activity_Column.gif

FROM c:\Logs\SP2\ex06051423.log

WHERE cs-uri-stem like '/exchange/%'

GROUP BY activity

ORDER BY hits desc
Figure 4 Breakdown of OWA Tasks
Figure 4 Breakdown of OWA Tasks
You can include a configuration file to further customize the chart. The Log Parser output format includes a switch called -config, which lets you point to a jscript or vbscript file, like so:
LogParser -i:IISW3C -o:CHART -values:ON -chartType:Column3D -view:ON
file:c:\Technet\TiSP2_OWA_Activity.sql -config:c :\Technet\MyScript.js
Place the following in a text file and name it MyScript.js:
// Customize title
chartSpace.HasChartSpaceTitle = true;
chartSpace.ChartSpaceTitle.Caption = "Generated by Log Parser 2.2";
chartSpace.ChartSpaceTitle.Font.Size = 6;
chartSpace.ChartSpaceTitle.Position = chartSpace.Constants.chTitlePositionBottom;

// Change the background color
chart.PlotArea.Interior.Color = chartSpace.Constants.chColorNone;
The resulting chart will look something like Figure 4.

OWA Hourly Report
This report breaks activity down into one-hour segments, showing you the OWA activity for each hour. This is handy for determining peak usage times. You can use this data, in turn, to plan for upgrades and server outages.

SQL Statement and Log Parser Command
This report uses the function QUANTIZE to round to the nearest unit—in this case, I round to the nearest 3600 seconds (or hour). I then sum up each activity, such as _Inbox, in the remaining columns of the chart. Copy Figure 5 into a text file named TiSP2_OWA_Hourly_Activity.sql.
Figure 5 TiSP2_OWA_Hourly_Activity.sql (Close Figure 5)
select
          quantize(to_localtime(to_timestamp(date,time)), 3600) as Hour,
          sum(_Inbox) as Inbox,
          sum(_Calendar) as Calendar,
          sum(_Tasks) as Tasks,
          sum(_Contacts) as Contacts,
          sum(_Other) as Other
using
          extract_token(cs-uri-stem,3,'/') as _activity,
          case _activity when 'Inbox' then 1 else 0 end as _Inbox,
          case _activity when 'Calendar' then 1 else 0 end as _Calendar,
          case _activity when 'Tasks' then 1 else 0 end as _Tasks,
          case _activity when 'Contacts' then 1 else 0 end as _Contacts,
          case add(add(add(_Inbox,_Calendar),_Tasks),_Contacts) when 0
              then 1 else 0 end as _Other

INTO
           C:\LogParser\Output\TiSP2_OWA_Hourly_Report.csv

FROM
           c:\Logs\SP2\ex06051423.log
where cs-uri-stem like '/exchange/%'
group by Hour
order by Hour
The Log Parser command for this example is quite simple, using the -o:csv switch to output the results into a CSV file, as shown here:
LogParser.exe -i:IISW3C –o:csv file:c:\TechNet\
    TiSP2_OWA_Hourly_Activity.sql
The result is a table similar to the one shown in Figure 6.
Figure 6 An Hourly Report of Usage (Close Figure 6)

Hour | Inbox | Calendar | Tasks | Contacts | Other
5/14/2006 23:00 | 4 | 0 | 0 | 0 | 1
5/15/2006 0:00 | 12349 | 2406 | 1777 | 19 | 6061
5/15/2006 1:00 | 10842 | 2002 | 1610 | 32 | 5257
...
5/15/2006 21:00 | 23570 | 4224 | 2856 | 48 | 11714
5/15/2006 22:00 | 21463 | 3904 | 2753 | 21 | 11770

Exchange RPC Hourly Activity
This report lets you analyze remote procedure call (RPC) activity by hour. For most Exchange customers, this tends to be the most commonly used component on the Exchange front-end servers. As in the previous example, this breakdown of RPC activity by hour is useful for determining peak usage times and scheduling planned outages.
The SQL statement here is very similar to the one used to break down OWA activity by hour. The only difference is that I have elected to report on the total hits between the client and Exchange front-end server. (Note that you could use this same SQL syntax and tweak it slightly to get reports for Exchange ActiveSync, OWA, or the entire Exchange front-end server.) This time around, I'll output the results into a chart to make the data more digestible and visually interesting.

SQL Statement and Log Parser Command
Again, start by copying the contents of the SQL block shown in Figure 7 into a text file and save the file as TiSP2_RPC_Hourly_Activity.sql.
Figure 7 TiSP2_RPC_Hourly_Activity.sql (Close Figure 7)
SELECT
         quantize(to_localtime(to_timestamp(date,time)), 3600) as Hour,
             Count(*) as [RPC Hourly Hits Report]

INTO
    C:\LogParser\Output\TiSP2_RPC_Hourly_Report.gif

FROM
    c:\Logs\SP2\ex06051423.log

WHERE cs-uri-stem like '/rpc%'
GROUP BY Hour
ORDER BY Hour
The Log Parser Command is as follows:
LogParser -i:IISW3C -o:CHART -values:OFF -chartType:Columnclustered -
view:ON file:c:\Technet\TiSP2_RPC_Hourly_Activity.sql -config:c:\Technet\
MyScript.js
And the resulting chart is shown in Figure 8. As you'll see, the data is much more readable when shown as a bar chart, as opposed to a table of numbers like in Figure 6.
Figure 8 Remote Procedure Calls by Hour
Figure 8 Remote Procedure Calls by Hour

Exchange ActiveSync User Activity Report
This report allows you to view the top 25 Exchange ActiveSync users based on number of hits. I'll use three columns to build a bar chart that represents the data. Of all of the Exchange services on the front-end machine, the Exchange ActiveSync service logs provide the most detail in the cs-uri-query field. For more info on Exchange ActiveSync logging, visit the Exchange team blog at msexchangeteam.com/archive/2005/03/28/403047.aspx.

SQL Statement and Log Parser Command
I first need to parse the cs-uri-query field to look for the sync command. To do this, I use Log Parser's INDEX_OF function and retrieve the value that sits between the Cmd= and the &. The parsed-out commands are then stored into the MyCmd variable where I can evaluate the value. I keep only the Sync command and discard the others. To create this report, copy the text in Figure 9 to a file and save it as TiSP2_EAS_User_Activity.sql. Then run the following Log Parser command:
LogParser -i:IISW3C -o:CHART -values:OFF -chartType:Barclustered -view:ON
file:c:\Technet\TiSP2_EAS_User_Activity.sql
The results will look like the table shown in Figure 10.
Figure 9 TiSP2_EAS_User_Activity.sql (Close Figure 9)
SELECT TOP 25
         TO_LOWERCASE (cs-username) AS User,
         COUNT(*)                   AS Hits,
         SUM (MySync)               AS Syncs
USING
         SUBSTR (cs-uri-query, ADD (INDEX_OF (cs-uri-query, 'Cmd='), 4),
         INDEX_OF (SUBSTR(cs-uri-query, ADD (INDEX_OF (cs-uri-query,
              'Cmd='), 4)), '&')) AS MyCmd,

         CASE MyCmd
                     WHEN 'Sync' THEN 1
                     ELSE 0
         END AS MySync

INTO
  C:\LogParser\Output\TiSP2_EAS_User_Activity.gif
FROM
  c:\Logs\SP2\ex06051423.log

WHERE cs-uri-stem = '/Microsoft-Server-ActiveSync'

GROUP BY User

ORDER BY Hits DESC
Figure 10 Top 25 Exchange ActiveSync Users
Figure 10 Top 25 Exchange ActiveSync Users

Michael Higashi is a Program Manager on the Exchange Server team at Microsoft and is currently working on Exchange Server 2007.


 Communication &amp; Collaboration: Fight Spam on Your Terms with Custom Weight Lists
Communication & Collaboration Cam Frenette and Alexander Nikolayev

Fight Spam on Your Terms with Custom Weight Lists
At a Glance:
  • Custom Weight Lists
  • The Intelligent Message Filter
  • How to filter messages to stop spam
  • How to search text appropriately

Spam, as we all know, is a huge problem. It clogs up your servers, aggravates your users, sucks up your bandwidth, and communicates unwanted and often inappropriate messages.
Can anything be done to stop it?
Well, if you're running Microsoft® Exchange Server 2003 you may have noticed that new features and functionality released last October in Service Pack 2 (SP2) significantly improved its ability to withstand different vectors of spam attacks. With multiple layers of anti-spam defense, Exchange Server 2003 can provide strong protection against unwanted messages. One of the most important elements in the Exchange anti-spam framework is the Intelligent Message Filter (IMF), which enables content filtering during the last stage of server anti-spam processing. Inside IMF, a little-known module called the Custom Weight List (CWL) turbocharges the anti-spam efforts of SP2. Let's take a look at what it does and how to use it.

What is a Custom Weight List?
The CWL is an XML file that contains a number of words or phrases that, if found in the body or subject of the message, can trigger modification of the final spam confidence level (SCL) assigned by the IMF. They are called Custom Weight entries and they look like this:
<?xml version="1.0" encoding="utf-8"?>
<CustomWeightEntries xmlns="http://schemas.microsoft.com/2005/CustomWeight">
<CustomWeightEntry Type=" " Change=" " Text=" " />
...
</CustomWeightEntries>
As you can see, there are three attributes related to examining an e-mail message for a word or phrase match: Type attributes, Change attributes, and Text attributes.
The Text attribute comes last for each entry, but it is the content the IMF will be looking for in the e-mail. This attribute can be a single word or a phrase, and it can contain escaped characters, upper ASCII characters, or double-byte characters, and any Unicode word or phrase up to 1000 characters long. There can be multiple Text attributes in a CWL file. One of the most common uses of Text attributes is to block adult-oriented advertising based on keywords in the message subject and body.
Where the IMF will look is determined by the Type attribute, which can be SUBJECT, BODY, or BOTH (indicating the part of the e-mail your Text should be matched against). Use one of the following Type attributes to specify that a search for a match will be performed on the subject of a message, the body, or both:
Type = SUBJECT
Type = BODY
Type = BOTH
The Change attribute can be any integer value (positive or negative). This attribute defines how the SCL value of the message will be changed if the Text attribute is matched. If a match is found, the value of the Change attribute will be added to the original SCL value (or essentially subtracted, if the Change attribute value is negative). If the combined (original and CWL) values exceed the SCL range, the final SCL assignment will be normalized between 0–9. Change can also use the MIN and MAX keywords. Any time a phrase with the MIN keyword is matched, the message is given an SCL of 0 regardless of any other weights. Any time a phrase with the MAX keyword is matched, the message is given an SCL of 9 regardless of any other weights. If there's a match for both MIN and MAX, the message is given an SCL of 0. To ensure that a mail item with matching characteristics will be blocked, use the MAX keyword.
SCL values for mail items fall between 0 and 9, with 0 indicating the least likelihood that the message is spam and 9 indicating the most. There is an additional value of "-1" (reserved by Exchange Server), which implies that the mail came from a trusted, authenticated source (another authenticated Exchange Server) and as such it is exempt from the anti-spam processing. Use of this value is enabled by default; to disable it, please read the Intelligent Message Filter Operations Guide at go.microsoft.com/fwlink/?LinkId=71143. Figure 1 shows how to interpret the SCL values.
Figure 1 SCL Values (Close Figure 1)

Value | Definition
-1 | Messages coming from a trusted source (authenticated Exchange Server)
0 | Messages categorized as not spam
1–5 | The likelihood of messages being spam is extremely low to low
6–9 | The likelihood of messages being spam is high to extremely high

Custom Weighting
As noted earlier, the IMF comes into play during the last stage of anti-spam processing. After a mail item passes through the layers of the Exchange Server 2003 anti-spam framework (such as the Connection and Protocol Filtering layers), it becomes a subject for IMF verification. IMF processes the mail item and assigns the appropriate SCL. The mail is then examined by the CWL to see if there is a match with any Custom Weight entries. If a match is found, the Change attribute is applied to the SCL value of the message. Hence, the Final SCL assignment looks like this:
Final SCL == original IMF SCL + CWL Change attrib.
The Custom Weight entries are loaded into memory when the IMF is starting up; any changes to the CWL file require a restart of corresponding Exchange and SMTP services to get the new entries loaded into memory. The entries are stored as hashtables to allow fast look-up for both single word and phrase matches while only parsing messages a single time.

Using the Custom Weight List
If you're under a spam attack and the offending mail items have a consistent phrase in their subject line, you can add that part of the line to your CWL entries, as shown here:
<CustomWeightEntry Type="SUBJECT" Change="MAX" Text="Bad Subject" />
This would ensure that any time a message with the subject line "Bad Subject" is encountered, the message's SCL would be forced to 9 and the message would be blocked.
On the other hand, if IMF is incorrectly classifying a message as spam, you can rescue that message. Suppose, for example, an auto-generated build status mail with the subject line "Build System Daily Mail" is getting blocked. You can add the following CWL entry:
<CustomWeightEntry Type="SUBJECT" Change="MIN" Text="Build System Daily Mail" />
This would ensure that such mail would get an SCL of 0 and would not be blocked.
If you have a list of key words or phrases that you want to use as modifiers to increase or decrease the SCL of a message, you can simply add them to the CWL entries. The following example illustrates that the word pear should be seen as a spam indicator in the subject; orange should be a spam indicator in the subject or body; banana should indicate good mail when in the body; and strawberry should indicate good mail when in either subject or body:
<CustomWeightEntry Type="SUBJECT" Change="3" Text="Pear" />
<CustomWeightEntry Type="BOTH" Change="5" Text="Orange" />
<CustomWeightEntry Type="BODY" Change="-2" Text="Banana" />
<CustomWeightEntry Type="BOTH" Change="-4" Text="Strawberry" />
After encountering the appropriate match, the IMF will adjust the final SCL value set on the message.

Order of Precedence
What happens when you have a mix of positive and negative weights, along with some MIN and MAX weights in your Custom Weight entries? If, by mistake, you assign the MIN and MAX keywords to the same Custom Weight entry, MIN takes precedence and the final SCL assignment is 0. This is done to prevent accidental message blocking due to CWL file misconfiguration. Precedence works as follows:
  • If an entry with MIN change is matched, the SCL of the message will be 0.
  • If an entry with MAX change is matched and there is no MIN token, the SCL of the message will be 9.
  • If no MIN or MAX entries are matched, then the SCL of the message will be the SCL determined by the IMF plus the change of any matched Custom Weight entries.
For example, suppose you have the following Custom Weight entries:
<CustomWeightEntry Type="BODY" Change="MIN" Text="hello" />
<CustomWeightEntry Type="BODY" Change="MAX" Text="world" />
<CustomWeightEntry Type="BODY" Change="1" Text="Internet" />
<CustomWeightEntry Type="BODY" Change="-3" Text="place" />
Let's see what happens if the incoming mail contains the following string in the message body: "Hello world and welcome to the Internet place". The IMF returns an SCL of 4 but the final SCL assignment would be 0. This is because the word "hello" has been matched and that entry has a MIN change keyword. None of the other entries are taken into consideration at this point.
Now let's see what happens if the message body contains the string "Welcome to the world and the Internet place." Regardless of the value originally produced by the IMF, the final SCL value would be set to 9 because the Text attribute value "world" was matched and the entry includes the MAX keyword. As there is no MIN entry to override, the message will be given an SCL 9 and blocked.
Here's another interesting example. Let's say the message body contains the "world Internet place" string and IMF returned an SCL of 6. What would be the final SCL assignment? It would be 4: the original SCL 6 from IMF plus 1 because of the "Internet" entry match and -3 because of the "place" entry match.

Substrings, Phrases, and URLs
CWLs will not match an entry to a substring of a word, but will match a shorter phrase to part of a longer phrase. Say you want to block mail with "Free Watches" in the subject line, so you put the following Custom Weight entry in your CWL file:
<CustomWeightEntry Type="SUBJECT" Change="MAX" Text="watch" />
This will not work as intended. You won't hit the CWL since "watch" is only a substring of "watches". You would need to explicitly include the word "watches" or the whole phrase "Free Watches".
Now suppose, based on this example, you've changed the Custom Weight entry from "watch" to "Free Watches" and your string looks like this:
<CustomWeightEntry Type="SUBJECT" Change="MAX" Text="Free Watches" />
If the subject of the e-mail is "Get your Free Watches Now!!!", you'd have a match. This is because the phrase in the CWL file will be matched against the part of the phrase found in the subject.
Now let's say you have the following URL in your custom weight entries:
<CustomWeightEntry Type="SUBJECT" Change="MIN" Text="microsoft.com" />
This token would match against the following subjects:
  • www.microsoft.com
  • http://microsoft.com
  • http://www.microsoft.com
  • http://schemas.microsoft.com/2005/CustomWeight
This is due to the way custom weighting tokenizes the phrases it sees. In essence, the phrases are split into words any time there's a break in the character type from alphanumeric to anything else: spaces, tabs, commas, periods, slashes, and so on. (Whitespace, though, is different. It is thrown away, while any of the punctuation characters become a word in the phrase.)
In this example, www.microsoft.com gets translated into a five-token phrase—www, period, Microsoft, period, com. The Custom Weight entry gets translated into a three-token phrase—Microsoft, period, com. The Custom Weight entry is now a match to part of the phrase of the subject.

Special Cases
Since the Custom Weight entries are stored in an XML file, there are certain characters that can't be used in a search entry without encoding. For example, if you wanted to block e-mails that contained the phrase "<Hello>", then your Custom Weight XML would contain the following entry:
<CustomWeightEntry Type="SUBJECT" Change="MIN" Text="&lt;Hello&gt;" />
CWL functionality supports Unicode characters in the CWL file. The actual custom weighting process is language agnostic—as long as the word or phrase in the entries matches a word or phrase in the e-mail being processed, then the Custom Weight will be used. So the Custom Weight entries in Figure 2 are perfectly legit.
Figure 2 Legitimate Custom Weight Entries (Close Figure 2)
<CustomWeightEntry Type="SUBJECT" Change="MIN" Text="こんにちは " />
<CustomWeightEntry Type="SUBJECT" Change="-7" Text=
    "Bienvenue à Redmond" />
<CustomWeightEntry Type="SUBJECT" Change="-4" Text="Первый" />
<CustomWeightEntry Type="BODY" Change="9" Text="Verlängertes Angebot" />
<CustomWeightEntry Type="SUBJECT" Change="MIN" Text="特別提供" />
<CustomWeightEntry Type="BOTH" Change="MAX" Text="Offre spéciale" />
There are, however, a few caveats to using high-ASCII and double-byte characters. First, you need to make sure you have the correct encoding in your Custom Weight XML header and that the file is saved as the correct encoding.
Second, the e-mail you are checking needs to be properly encoded. If the MIME type of the e-mail is incorrect or the wrong character-encoding is specified, the IMF might not be able to process the Custom Weight entries correctly.
Third, due to the way strings are tokenized in custom weighting, phrases with double-byte characters, especially East Asian languages, are tough to create for use with the CWL file. This is because the punctuation and whitespace characters used to tokenize many European languages don't match up well. If you are setting up your CWL to be used with double-byte characters, make sure to give it a round of testing to find a correctly matching Custom Weight entry.

Wrap Up
When you're ready to use the CWL file, remember that, by default, the Exchange Server 2003 SP2 installer does not create it for you. You'll need to do this manually (see the Intelligent Message Filter Operations Guide mentioned earlier for details). And don't forget that the CWL file, MSExchange.UceContentFilter.xml, should always be in the working path of IMF, as shown in Figure 3.
Figure 3 The Working Path of IMF
Figure 3 The Working Path of IMF
If the CWL file is not located in the IMF's working directory, it will not work. If you have custom weighting deployed and are using IMF Updates, make sure that every time new spam definitions are installed, the CWL file is moved to the new IMF location. You'll have to do this manually. The Intelligent Message Filter Operations Guide provides detailed information how to enable IMF Updates and what you need to do to ensure that the Custom Weight List continues to operate successfully.

Cam Frenette is a Software Design Engineer in Test at Microsoft who focuses mainly on anti-spam technologies for the Technology Care & Safety team. He holds a B.Math from the University of Waterloo.
Alexander Nikolayev is the Program Manager at Microsoft in charge of server-side protocols, Transport Core, and anti-spam components for Exchange and Windows Servers. Alexander holds an MBA degree from the University of Mary. Read his posts on the Exchange team blog blog.


 Communication &amp; Collaboration: Achieve High Availability with Exchange Failover Clusters
New information has been added to this article since publication.
Refer to the Editor's Update below.

Communication & Collaboration Mark Godfrey

Achieve High Availability with Exchange Failover Clusters
At a Glance:
  • Provisions for Exchange Server failure
  • Determining specific needs
  • Clustering Exchange Server
  • The backup process

Having your e-mail system go down is bad. Losing the data archived on your e-mail server is even worse. There's critical data in your users' e-mail and such a loss can have serious consequences. To prevent the repercussions of outages and the loss of historical information,
you can employ Exchange failover clusters, allowing you to recover your e-mail systems between datacenters quickly and efficiently.
A failover cluster is effectively a standby cluster of servers with as many or more nodes than exist on the production clusters. In the event of a disaster, the Exchange Virtual Servers from the failed datacenter are brought online at the backup datacenter, saving data that would otherwise be lost.
In this article I'll discuss a real-world case in which a Microsoft® Services team built an Exchange Virtual Server failover solution for a financial sector customer. But first let's look at some requirements for building an Exchange environment where data is safer.

General Requirements
First of all, you need to know that the key to having a high Recovery Time Objective (RTO) environment (an environment in which you can recover server functionality quickly after an outage) is to have a standby site with the same configuration as the production site. The key to having an environment with a high Recovery Point Objective (RPO) (an environment in which the data recovered is as up-to-date as possible) is data replication so that you have relatively current copies of the database backups and access to current log files. There are two common failover cluster scenarios that you could employ in building such an environment: the dedicated Exchange failover and the many-to-one failover cluster.
In the dedicated approach, each production cluster has a dedicated failover cluster. When cluster A goes offline and is not recoverable, the Exchange Virtual Servers are brought online using the A' (A prime) cluster.
In a non-dedicated, or many-to-one scenario (typically used when multiple datacenters have a backup datacenter), the standby cluster must have the same number of nodes as the production cluster with the most nodes. Note that only one production cluster can fail over to the standby cluster. If a second datacenter goes off line, it cannot be restarted on the standby cluster until the first one is failed back or otherwise removed.

The Banking Example
The case involves a major financial organization that runs both banking and securities operations. The company's IT department was looking for a solution that would allow them to give their executives and traders access to e-mail within hours of a major datacenter outage, without implementing geoclustering. First, the Microsoft team interviewed the major business units and gathered their requirements for system availability and access to existing mailbox data. The needs were significantly greater than the IT department had anticipated. We divided the groups in three tiers, which are represented in Figure 1.
Figure 1 Business Units and User Groups (Close Figure 1)

Tier | Group | Accounts | RTO | RPO | Data (TB )
Tier 1 | Traders and Executives | 4,000 | 2 Hours | 5 Minutes | 0.6
Tier 2 | Corporate Employees | 30,000 | 4 Hours | 5 Minutes | 6.0
Tier 3 | Retail Staff | 16,000 | 12 Hours | 1 Hour | 0.8
The customer has two datacenters connected over a high-speed network. This solution is used for a variety of data replication processes, and the customer needs it to be fully supported by Microsoft.
Initially, we considered using a dial-tone recovery model—giving users the ability to send and receive e-mail very quickly after a failure, but with no historical information available until it could be later restored using traditional methods. This idea was abandoned because most of the executives lived and breathed by their calendars and the traders were very nervous about not having access to their existing e-mail. The other big drawback to the dial-tone solution was the bandwidth required to resynchronize the Offline Storage (OST) files when Exchange Cached Mode is enabled—which is the default standard for the customer. Although there are a variety of workaround procedures, none of them would allow us to meet the specified requirements.
The Microsoft team determined that the customer was a good candidate for combining the ability to fail over Exchange Virtual Servers and restore data using Volume Shadow Copy Service backups. The key concerns were the amount of data that needed to be replicated between datacenters to meet the two-hour RTO and a five-minute RPO, and the relative complexity of the process to restore the service under pressure.
Log files presented another challenge. In completing Volume Shadow Copy backups every hour, how could we ensure that the RPO of five minutes was obtainable? We needed to ensure the logs were available at the failover site from the time of the last backup to five minutes prior to the disaster.
The first task was to split the accounts between datacenters and set up dedicated standby clusters. This ensured that only 50 percent of the users would be affected if either datacenter went offline and that the administrative effort to restore mail servers would be cut in half (see Figure 2).
Figure 2 Accounts and Servers Split between Datacenters
Figure 2 Accounts and Servers Split between Datacenters
The second decision we made was based on the requirements generated through the interview process. Given the high RTO and RPO of 68 percent of the users (Tiers 1 and 2), an aggressive data replication model was required. Frequent backup copies needed to be replicated to ensure that mailbox content was available, and synchronous copying of the log files was required to enable the high RPO. Figure 3 illustrates the configuration we designed.
Figure 3 Backup Configuration
Figure 3 Backup Configuration
To generate frequent database backup copies, we needed to be running the Volume Shadow Copy Service. This was especially important for the 4,000 executives and traders with the two-hour RTO service level agreement (SLA). Volume Shadow backups are made each hour at each site, staggered by half an hour. The backup data then needs to be copied over to the disaster recovery site within a half hour to ensure that there's enough time inside the two-hour window to bring the database back online and replay the logs generated since the backup.
Now that the conceptual design was understood and had been generally accepted, the team moved on to design the implementation. The first step was to identify the infrastructure required to support the solution—storage, network, servers, supporting services, and resources. The customer had identified a major standardization initiative and that, as always, cost control was important.

Storage and Network
Our top priority was to determine a storage solution that would allow us to create frequent backups required to ensure quick recovery times on the backup site. Network Appliance was selected because of its capability to make fast "incremental" backups, which would minimize the replication traffic between datacenters.
Next we identified the network requirements between datacenters. The customer had recently begun an Internet Small Computer System Interface (iSCSI) standardization initiative, and requested that the solution be compliant. Note that while this solution was implemented using an iSCSI storage solution, fiber-based Storage Area Network (SAN) solutions are also frequently used for standby cluster installations.
With this information in hand, we decided that the cluster pairs (production and standby) should be on the same IP network subnet. This reduced complexity in that the virtual server, when brought up on the standby cluster, could use the same IP address that it used on the production cluster. Thus, that client's DNS caches don't have to be forced to refresh or wait for the default DNS refresh. Note that each cluster pair must be on the same subnet, but when multiple cluster pairs exist, each pair can be on a different subnet. This is not a requirement for the virtual server failover scenario—but it sure saves some headaches for the administrators and help desk. (Although the network guys gave us some grief when we asked for spanned subnets!)
For the connection between the servers and the storage system, Microsoft Multipath I/O (MPIO) iSCSI was used with two connections per server. This is a dedicated network with no traffic other than Exchange-related I/O. For the interstorage connections, jumbo frame IP was enabled to allow for bulk data transmission. There is no requirement for the storage infrastructure to be on the same subnet; just make sure that each segment between the two has jumbo frames enabled. Figure 4 shows the network requirements.
Figure 4 IP Network Requirements
Figure 4 IP Network Requirements

Servers
Servers presented the usual sizing challenges, but the key problem in this environment was consistency. During disaster recovery, the last thing you need to worry about is that one server needs a different hardware or software patch than the others. So the rule for hardware was to make it the same and make the customer aware that a refresh must cover all servers in each cluster pair. In reality, especially when running a many-to-one solution, this is not always viable or practical—but when you can, stick as close to a standard model as possible.

Other Requirements
The biggest challenge with the Exchange configuration turned out to be the time required to enumerate the cluster volumes and bring them online. In our lab, it took approximately 10 minutes for each node to acquire its volumes, so a four-node cluster (in an Active/Active/Active/Passive configuration with three Exchange Storage Groups (SGs) per Exchange Virtual Server) with nine storage groups—one per logical unit number (LUN)—took up to 90 minutes before it was available. For the two-hour RTO group, we went with a three-node cluster (in an Active/Active/Passive configuration with two SGs per Exchange Virtual Server), with user accounts on one node, and calendar accounts and public folders (not widely used by this customer) on the second node. This allowed us to meet the requirement of full functionality for the two-hour RTO group.
[ Editor's Update - 10/19/2006: A major factor in the time it took for our clusters to acquire volumes and complete failover was the amount of time it took our storage solution to present the LUNs to the servers. However, with some configuration recommendations from Network Appliance we were able to manage the solution to meet the SLA requirements. Our lab environment consisted of four a node (AAAP) cluster on Windows Server 2003 Enterprise Edition SP1 and Exchange Server 2003 SP2 connected to Network Appliance FAS980 storage. The delays associated with presenting LUNs to the servers were specific to this storage configuration and should not be generalized to dissimilar configurations.]
Along the way we learned a few things. First, sufficient Active Directory®-related servers (global catalogs, DNS, and so on) must be available at each site to support the entire solution, the network infrastructure must be fully meshed (so clients can access both datacenters transparently), and there needed to be an SLA with the network group because any prolonged network outage between the datacenters would severely hamper the ability to meet the five-minute RPO.
The biggest sticking point in a disaster recovery solution is data replication. You've got to ensure that either you have sufficient capacity in place or you're willing to provision sufficient bandwidth. The following were some of the issues we encountered.
Data Replication after Online Defrag The size of the replication data required after the online defrag was surprisingly large. Some further investigation into how the online defrag works and what it actually does shed light on the dataset size; but beware, you're looking at 40 percent to 60 percent of the total database size.
Cluster Node Volume Enumeration The time that was required to do this surprised us, so make sure you build in time around this when determining the SLA. The general guidance is to use a single storage group and single data store. We found this simplified the process under a failover scenario, but it was just not realistic in terms of the number of users we wanted to support per server.
Another issue was where to validate the backup data. If you run the ESEUtil /K command against the database and back up on the production server, you can seriously impact performance. We got around this by copying the data to the standby site, then executing ESEUtil /K.
Log File Replication The number of log files must be taken into consideration early in the process. Synchronous replication of large volumes of log files requires significant bandwidth. We needed a sustained throughput of 11MB/sec to handle the log file traffic between datacenters at peak times in order to meet the five-minute RPO.

Final Design
At the end of the day, the solution looked like the one shown in Figure 5. Three different blocks are used to support the three recovery scenario requirements.
Figure 5 The Final Configuration
Figure 5 The Final Configuration
Block 1 and Block 2 consisted of three-node clusters of Exchange servers (Active/Active/Passive) for the Tier 1 users, and a four-node cluster of Exchange servers for the Tier 2 users.
Tier 1 consists of the 4,000 executives and traders and the calendar/resource accounts that require a two-hour RTO. Tier 2 has 30,000 employees with a four-hour RTO.
Both of the Block 1 and Block 2 three-node clusters hosts 2,000 mailboxes on one Exchange Virtual Server and 1,500 resource accounts on the other Exchange Virtual Server. The replication schedule for each of the LUNs associated with the Tier 1 environment is one copy per hour. Each Exchange Virtual Server of the four-node clusters hosts approximately 2,500 users, or 7,500 per cluster. To minimize the LUN enumeration time and avoid the need for using volume mount points, three storage groups per Exchange Virtual Server are used. The replication schedule for the Tier 2 LUNs is once every four hours (twice a day).
Block 3 consists of three-node clusters (two active nodes and one passive); each Exchange Virtual Server hosts approximately 4,000 users who are split into two storage groups of 2,000 users each. The replication of the backups happens once per day after the nightly on-line defrag. Synchronous log replication ensures that the high RPO can be met for all the users, regardless of the RTO.
A lot of other technology is tied into this solution, including archiving, journaling, and all the associated data storage and replication—but that's for another article. We achieved our goal of building failover clusters for Exchange to provide a disaster recovery solution that has a relatively high RTO and RPO, is fully supported by Microsoft, and avoids the complexity of full geoclustering.

Mark Godfrey is a messaging consultant with Microsoft Consulting Services in Toronto, Canada. His primary focus is on highly available messaging systems including Exchange, LCS, and Active Directory. Contact Mark at Mgodfrey@microsoft.com.


 Deployment: Boot into RAM with Windows PE
New information has been added to this article since publication.
Refer to the Editor's Update below.

Deployment Wes Miller

Boot into RAM with Windows PE
At a Glance:
  • Preparing a Windows PE ISO image
  • PXE-based RAMDisk booting
  • CD-based RAMDisk booting
  • Hard disk-based RAMDisk booting

Since its low-key introduction in August 2001, Microsoft Windows Preinstallation Environment has evolved into a much more powerful tool than Microsoft had originally intended. Last issue I
discussed the history of Windows® Preinstallation Environment, or Windows PE, and looked at how it has changed with subsequent versions. In this article, I'll explore a useful capability that has evolved over time: the ability to boot from a RAMDisk. (And the sidebar "Booting from a USB Flash Drive" looks at how boot options continue to become increasingly flexible.)
When first released, Windows PE could only boot from a CD, Windows Remote Installation Services (RIS) via Pre-Boot Execution Environment (PXE), and hard disk (using a cobbled-together method). With new versions came the ability to boot from an increasing number of devices.
When I was working on the Windows PE team at Microsoft and we were first developing the product, the Windows XP Embedded (XPE) team was busy building their own tools to make Windows XP work in new ways. Among their efforts was an initiative to define Windows XP through a series of components that could be custom-built using the embedded tools and then deployed to fixed-function Windows Embedded devices.
The XPE team outlined key scenarios that it wanted Windows XP Embedded to support. When Windows XP Service Pack 1 shipped, one of these scenarios was referred to as Remote Boot. Using a PXE bootstrap similar to RIS, Windows XP Embedded Remote Boot allowed an embedded device to boot over the network, using a RAMDisk as its boot drive. Let's take a closer look at this so you can see how it relates to Windows PE:
  1. The PXE client boots and a PXE boot is requested. (Depending on how your PC is configured, this may involve a BIOS boot selection at boot time or possibly pressing F12 once or twice.)
  2. The PXE client sends out a special UDP broadcast message—a modified Dynamic Host Configuration Protocol (DHCP) discover request.
  3. The DHCP server (which may or may not be the actual PXE server) sends back an offer, which includes valid network information and the address of the PXE server.
  4. The PXE client responds with a request for a Network Boot Program (NBP). This is a small executable image that can perform a defined task.
  5. The PXE server responds with an acknowledge response that includes the Trivial File Transfer Protocol (TFTP) download location of the NBP, which the client then begins to download.
  6. The NBP is executed. In the case of Windows (RIS, ADS, or Remote Boot), execution involves running a small 16-bit app called startrom.com (or startrom.n12, which is a version that doesn't require the user to press the F12 key). [ Editor's Update - 10/18/2006: In RIS, this file is called startnbs, not startrom.com] Startrom then goes back to the TFTP server and downloads the Windows NT Boot Loader (NTLDR) or a file named accordingly (setupldr.exe renamed as NTLDR in the case of Windows PE RAMDisk boot).
  7. In the case of Remote Boot, the NTLDR pulls down a boot.ini file (winnt.sif in the case of Windows PE RAMDisk boot), which specifies where to find the RAMDisk file to load.
  8. The NTLDR pulls this RAMDisk file down directly into RAM, and initializes just as it would if it were booting directly from a hard disk.
With Windows XP Embedded, this technique allowed for a group of devices (kiosks, point-of-sale terminals, and so on) to boot from one central PXE server without requiring dedicated hard disks on the devices. The RAMDisk image is placed completely in local RAM by the NTLDR. With this approach, a device must have enough local RAM to hold the "boot disk" while still leaving enough RAM free for the working set. This also means that if the device is reset, the boot image must be downloaded again.

Working PXE into Windows PE
From the time we first shipped Windows PE, customers liked the ability to PXE boot, but often didn't like that we required Remote Installation Services (RIS) to do so. Since RIS requires Active Directory®, and no manufacturer was likely to have Active Directory implemented on a fast-moving factory floor, OEMs had even more reason to complain. As we started planning for Interim Windows PE (iWinPE), we knew we had to support the ability to PXE boot Windows PE from any PXE server, not just RIS.
Our team (working on Windows PE) was integrated with the XPE team, and using the existing Remote Boot functionality as a bootstrap (forgive the pun) seemed the most logical place to start. We investigated what would have to change in order to allow Windows PE to remote boot and we found that the answer was "not much."
We did decide on one key change. The Windows XP Embedded implementation of Remote Boot worked by booting an SDI file. SDI files are sector-based disk images, not unlike those used by many deployment technologies. Since Windows PE discards all registry writes while running—effectively working read-only so it can run from a CD—the ISO file format (a virtual CD, if you will) was a more natural choice. We incorporated this change and shipped this with Windows Server® 2003 SP1. Note that this functionality is only available for use with x86 and x64 architectures of Windows. It does not work with the Intel Itanium (IA64) architecture.
The RAMDisk boot functionality allows the NTLDR to load any compatible ISO image of Windows PE into RAM and boot it. While our initial focus was PXE booting, the RAMDisk boot capabilities are also useful for several other scenarios that I'll discuss. This article will go over three key scenarios—PXE RAMDisk boot, Hard disk RAMDisk boot, and CD RAMDisk boot—and walk you through the steps to perform each.
Regardless of the method you use, RAMDisk booting uses up a significant (though not unreasonable) amount of RAM. For a default Windows PE image you'll need at least 145MB of RAM for the RAMDisk and at least 96MB for the working set. To use one of these RAMDisk methods, you also must build your Windows PE image from Windows Server 2003 SP1 media—either Standard Edition or Enterprise Edition. A Windows PE image created from the original release of Windows Server 2003 or any release of Windows XP will not work.

Windows PE Q&A
Q Why does Windows PE always use X: as its boot letter?

A Beginning with Windows PE 2004, the boot drive is assigned the drive letter "X". It changed both for consistency (so you always know which drive is your Windows PE drive) and because certain components added to Windows PE with Service Pack 1 required a consistent boot drive letter to work.

Q I added a driver to Windows PE—why isn't it working?

A This is usually due to a missing dependency. Check the .inf file for the driver and ensure you haven't missed any required files listed in the .inf. This may include files with extensions you aren't used to, not just *.sys files. The driver looks for them and silently fails to initialize if these dependencies aren't found.

Preparing a RAMDisk Boot Image
To create a RAMDisk boot image, you always begin by creating the actual ISO image that will be booted into RAM. This is the virtual CD that will be stored in RAM while Windows PE is running, regardless of what medium you boot it from. You should keep the ISO image's size to a minimum. This involves removing any unnecessary files (Windows PE documentation includes a list of removable files) and including only the files that are necessary for your image to work. For example, you should include any network interface card (NIC) and mass-storage controller (MSC) drivers, and any binaries or dependencies necessary for your application. Remember that only the files included in the ISO will be available in the RAMDisk image. In most scenarios, you will have two drives available: the drive or device that you booted from (unless you use PXE, which won't map the drive) and the RAMDisk-booted CD image, which will always appear as your X: drive. The X: drive will act as a CD-ROM drive, even if your system does not have a local CD or DVD drive.
The steps for preparing a RAMDisk boot image are nearly identical to those involved in creating a normal Windows PE image, with one exception: you do not need to specify a boot sector when creating the RAMDisk ISO image. Say you are creating a standard Windows PE image in a directory called C:\Staging\. You would create the Windows PE image using mkimg.cmd and then create the ISO image using OSCDIMG as follows:
OSCDIMG -n –betfsboot.com C:\Staging\ C:\Staging.ISO
To prepare a RAMDisk boot image, you could omit the command-line option "-betfsboot.com". And now the image itself is ready to be included in one of the following scenarios.

PXE RAMDisk Boot
The PXE RAMDisk boot scenario allows you to boot a system from any PXE server. PXE is invaluable for providing a centralized boot from a single server (for imaging a training lab or classroom, for example) or for rapidly booting systems without a CD-ROM drive (or without requiring a CD to be inserted). This can dramatically improve how quickly OEMs or businesses can install Windows onto new PCs.
Here's a general overview of the steps needed to perform a PXE RAMDisk boot, though some of the steps may differ slightly depending on your particular PXE server (consult your documentation for details):
  1. Insert your integrated Windows Server 2003 SP1 CD. (Alternatively, you can browse to a Universal Naming Convention, or UNC, share of Windows Server 2003 SP1.)
  2. Copy ntdetect.com and setupldr.exe from the I386 directory (do not select setupldr.bin).
  3. Paste these two files into the root of the PXE server. Rename the setupldr.exe file as ntldr (note that it has no file extension).
  4. Copy the ISO image you created earlier to the root of your PXE server.
  5. Create a new text file in the root of the PXE server and name it winnt.sif.
  6. Open winnt.sif in Notepad and enter the following text (but be sure to use the name you gave your ISO file):
    [SetupData]
BootDevice = "ramdisk(0)"
BootPath = "\I386\System32\"
OsLoadOptions = "/fastdetect /minint /rdexportascd /rdpath=Staging.iso"
Architecture = "I386"
    Note that case sensitivity may matter if you are using a non-Windows PXE server.
  7. Ensure that your clients are set to boot from the specified PXE server. This is typically done by specifying the client's MAC address or its SMBIOS GUID to the PXE server. (For information on retrieving the MAC and GUID, see support.microsoft.com/kb/302467.)

CD RAMDisk Boot
The CD RAMDisk boot is useful for what is referred to as CD swapping. In this scenario, the boot CD you use to initialize Windows PE can be removed once Windows PE has been loaded into an ISO RAMDisk. This CD swapping functionality was particularly critical to certain OEM and IHV recovery solutions, which traditionally work by reimaging an entire PC from disk images that are stored on more than one disk or from a hidden partition. Let's walk through the steps used to create the RAMDisk-booted CD:
  1. Insert your integrated Windows Server 2003 SP1 CD. (Or you can browse to a UNC share of Windows Server 2003 SP1.)
  2. Copy ntdetect.com and setupldr.bin from the I386 directory. (Note that this is different from the previous scenario where you copied setupldr.exe.)
  3. In the staging location where you are building your Windows PE image, create a new text file and name it winnt.sif. Create a directory named I386.
  4. Copy the ISO file, ntdetect.com, and setupldr.bin into the root of this new directory without renaming the files.
  5. Open winnt.sif in Notepad and enter the following text (use the name of your ISO file):
    [SetupData]
BootDevice = "ramdisk(0)"
BootPath = "\I386\System32\"
OsLoadOptions = "/fastdetect /minint /rdexportascd /rdpath=I386\Staging.iso"
Architecture = "I386"
  6. Now copy any content you need to the staging directory itself.
  7. Once you are ready to create your final CD image, run the following:
    OSCDIMG -n –betfsboot.com C:\Staging\ C:\Staging.ISO
Your image is now ready to be burned to CD. Anything you add to the CD outside of the RAMDisk image will be available when the disk is in the drive, but it will not be loaded into RAM. The RAMDisk will be drive letter X. The CD-ROM drive will be the first drive letter after the last partition enumerated on the system (often D). This is important if you are writing scripts that are looking for files on discs other than the RAMDisk.

Hard Drive RAMDisk Boot (Single Partition)
Though this method is not commonly used, hard drive RAMDisk booting is a useful option for certain scenarios. As the name implies, this approach boots from a hard disk into RAM. Once loaded, Windows PE no longer has any dependencies on the disk it just booted from. Thus, once Windows PE has been loaded, the partitions can be completely wiped clean. This can be done with a single partition (the easier, though potentially more dangerous, method) or you can use multiple partitions. Let's look at the steps involved in both approaches.
For a single partition, you must:
  1. Insert your integrated Windows Server 2003 SP1 CD. (Or browse to a UNC share of Windows Server 2003 SP1.)
  2. Copy ntdetect.com and setupldr.bin from the I386 directory.
  3. Create a new text file named winnt.sif and enter the following text into it (using the name you gave your ISO file):
    [SetupData]
BootDevice = "ramdisk(0)"
BootPath = "\I386\System32\"
OsLoadOptions = "/fastdetect /minint /rdexportascd /rdpath=Staging.iso"
Architecture = "I386"
  4. Use either scripting on the remote machine or the Windows administrative share (\\machinename\c$) to place all of the following on the remote machine's System drive (generally C:): ntdetect.com (save the old one); rename the setupldr.bin file as NTLDR and replace the existing NTLDR with it (again, save the old one); a copy of winnt.sif; and a copy of the RAMDisk ISO file you created earlier. The end result should be two new and two replaced files on the remote machine's System drive.
Booting from a USB Flash Drive
Windows PE 2005 introduced support for booting from USB flash drives. It's a handy option but, unfortunately, it requires a partitioning tool that is only available to Windows OEMs. There is a reason for this, however. In order to safely boot a PC from a USB flash drive, there are some essential requirements:
  • The BIOS must support boot from USB Mass Storage.
  • The BIOS must support USB. (Ideally it should also have built-in support for Hi-Speed USB 2.0, as this dramatically improves Windows PE boot performance.)
  • The PC must support power to the USB bus at boot time.
  • Ideally, the PC should also support Hi-Speed USB 2.0.
Some of these system capabilities can be easily tested. However, not all of them can—so there is no safe way for users to create a USB Flash Drive with Windows PE on it and ensure it will just work on all Windows systems. So the safest solution—until Windows Vista ships, that is—was to provide the necessary tools to OEMs. They can, in turn, develop and test USB flash drives, indicating which ones work with which systems. Windows Vista will offer a better solution. Several logo requirements were specifically designed to ensure that any PC carrying a Windows Vista logo supports USB Flash Drive boot.

At this point, any reboot will cause the system to boot into a Windows PE RAMDisk. This means you can script a reboot and the system can complete any necessary tasks, independent of the hard disk. The machine is actually stuck in a loop where only Windows PE will boot. If you want to reboot into Windows, you must put the original NTLDR and ntdetect.com files back in place.

Hard Drive RAMDisk Boot (Multiple Partitions)
For a setup with multiple partitions, the process is very similar to the one used in a single partition setup. The main difference is that you must prepare an additional partition for Windows PE. This requires enough space to store the Windows PE image and any other working files. Let's take a look at how this process differs from the process I described for a single partition setup.
  1. You must create a partition if it doesn't already exist.
  2. Copy ntdetect.com, setupldr.bin, winnt.sif, and the RAMDisk ISO file (see Step 4 of the single partition process described earlier) to the new partition.
  3. When you're ready to boot to this partition, mark it active, using DiskPart. You should also mark the existing partition as inactive. (For information on using DiskPart, see microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/diskpart.mspx.)
On the next reboot, your system will boot into Windows PE. Note that you're rebooting from the second partition, but into RAM. Therefore, you could theoretically wipe out the second partition, just as you can do when working with a single partition setup.
To resume booting into Windows, set the new partition as inactive and set the Windows partition as active. Remember to use both inactive and active. A PC will boot from the first active partition it finds with a Master Boot Record (MBR), even if all four potential primary MBR partitions are active.

Summary
Windows PE has evolved into a very flexible deployment and recovery tool. This article has explained how various methods for performing RAMDisk boots with Windows PE, and I've identified some specific scenarios where each method can be useful. In my next article, I'll discuss some of the new features and capabilities of Windows PE 2.0—the exciting new version that will ship with Windows Vista.

Wes Miller is the Product Technology Strategist at Winternals Software in Austin, Texas, where he focuses on all Winternals products, including Protection Manager, the Winternals Enterprise Security product. Previously, Wes worked at Microsoft as a Program Manager and Product Manager for Windows enterprise deployment.


 Deployment: Make Server Rollouts a Snap with Automated Deployment Services
Deployment John Savill

Make Server Rollouts a Snap with Automated Deployment Services
At a Glance:
  • Making machine images
  • Deploying servers
  • Making backups for DR
  • Other uses for ADS

It's easy in small environments to assume that you don't need to automate the deployment of servers. After all, the up-front effort to create the infrastructure and images for deployment without
intervention may not be easy to justify. But you don't need hundreds of servers for automated deployment to be worthwhile. The benefits of standardized builds, fast rebuild times, and consistent supportability can be compelling—even in small environments. Convinced? Great. Now let's look at how you can deploy servers automatically.
You might first think of the Operating System Deployment (OSD) Feature Pack for SMS, which is not, strictly speaking, a desktop deployment technology. Indeed, if you look at some of its dialog boxes, you'll notice provisions for licensing options, number of seats, and so forth. So, can you use OSD to deploy your servers and have a single deployment infrastructure? It depends on your server hardware. OSD deploys by booting to Windows® PE (Preinstallation Environment), which in the current version of OSD is based on the Windows XP operating system. This fact represents the first challenge. Windows XP lacks the drivers necessary for many servers to function, and while it sometimes may be possible to inject them into Windows XP, they may not function correctly if at all. A better option, Automated Deployment Services (ADS), has superior driver management compared to OSD, full server driver support, and support for deployment to partitions other than the C: drive (which is an OSD limitation).
There are other reasons to choose ADS. In most scenarios new servers require flashing (updating) the BIOS or configuring a RAID or SAN controller, which is often accomplished via MS-DOS® on a floppy disk. If you have to manually perform these floppy-style configurations, it's not exactly an automated deployment. OSD has no ability to virtualize an MS-DOS-type floppy disk, while ADS can capture the floppy content and required actions and send them over the network to enable a total hands-off deployment.
Furthermore, OSD doesn't really handle complicated multi-NIC configurations, while ADS has full support built in. In the end, OSD may be able to deploy a server operating system, but ADS is designed for it. When Windows Vista™ and Windows Deployment Services arrive we'll see a merging of these technologies, but for now ADS is the way to go.

Inside ADS
ADS is essentially a sector-based imaging technology, similar in many ways to other third-party imaging tools with which you may be familiar. Where ADS differs, though, is in its ability to understand the NTFS file system structure, remove duplicate and unnecessary files (such as page and hibernation files), and perform other optimizing techniques prior to image deployment. (Note, however, that this imaging technology is not the one used in Windows Vista; ADS uses a different technology.)
Another key feature of ADS is multicast support, which allows you to deploy up to 128 machines concurrently through one set of network traffic. In addition, highly configurable task sequences allow granular configuration and full system build-out. Plus ADS can mount captured images and modify content if required.
ADS can be used only for the deployment of 32-bit server operating systems, Windows 2000, or above. Deploying client operating systems such as Windows 2000 Professional or Windows XP is not supported. The ADS server component will run only on Windows Server® 2003 Enterprise edition or above. You can download it from go.microsoft.com/fwlink/?LinkId=71491. At press time, the current version is 1.1 (improvements from version 1.0 include support for booting to Windows PE via a new repository in ADS for the storage of PE environments).
ADS consists of three core services, though it also requires DHCP to be running to enable IP address allocation to servers.
The controller service is the core of ADS, performing all communication with the database. (The database type, either SQL Server™ or an MSDE instance, is selected at ADS installation; the controller provides the information to the other services.) It is through the controller service that administration inputs such as the Microsoft® Management Console (MMC) snap-in, Windows Management Instrumentation (WMI) interface, and command-line tools function. The controller maintains the records for each ADS device (a server, typically identified by its MAC address but possibly by its SMBIOS GUID). During the build process, the controller service instructs the pre-boot execution environment (PXE) component with the boot commands/images to provide to the device at each stage in the deployment process.
The controller service lets you group servers into sets, allowing them to be managed as a single entity (for deployment and post-deployment administrative functions). You can also have set references from other sets, forming a hierarchy.
The Network Boot Service (NBS) works with the DHCP service to enable the PXE client to find the NBS server. DHCP does not have to be running on the ADS server itself, but it can. NBS includes the PXE service and the ADS deployment agent builder. The PXE service can instruct the PXE client on target servers to download and boot an ADS deployment agent, boot to Windows PE (if configured), boot a virtual floppy disk, or just boot from the hard disk. Using instructions from the controller service, the NBS is mainly responsible for the building of the remote servers. Data is transferred using the built-in Trivial File Transfer Protocol (TFTP) service.
Image Distribution Service manages the storage of the operating system images.

ADS Agents
ADS supplies two agents for managing deployment. The deployment agent mentioned earlier is essentially a RAM-based mini-operating system that ADS uses to boot servers to when it needs to capture or deploy an operating system, partition disks, or perform certain other tasks. Additionally, an administration agent is installed on the reference server as a service before image capture. It allows the ADS server to manage deployed servers via various command and scripting engines, giving ADS capabilities not only to deploy the operating system but also to manage deployed servers. For example, servers can be placed into groups and then actions or commands can be performed on the entire group.

Task Sequences
At the heart of ADS is a powerful, XML-based format that allows sequences of tasks or steps to be defined and executed in order. These task sequences are stored in XML files, which gives ADS its ability to go beyond simple image deployment. Image deployment is one task in the sequence, but before you deploy an image you can send a virtual floppy to flash the BIOS, then another to configure a SAN controller, then wipe all partitions and create several in its place. You can also deploy images to separate partitions, perform changes to the file system, and once the deployed operating system is booted you can run any other number of other commands to customize the environment.

Creating a Reference Machine
The first task after downloading and installing ADS is to prepare the reference server, which will form the foundation of the image that will be deployed to the target servers. Since this machine will be the source for deployed servers, the size of its boot drive (C:) will become the minimum size of any deployed server. Thus, when you are preparing your reference machine, make its C: drive as small as possible; it only needs to be able to hold the operating system, the ADS Administration Agent, Sysprep, and any other applications or tools you decide need to be part of the image. Remember that any applications installed as part of the image must be fully Sysprep-compliant.
When the reference machine is ready, install the ADS Administration Agent, which allows the ADS server to perform the tasks defined in the task sequence and ultimately capture the reference machine. To install this agent, run the adssetup.exe you used to install the ADS services on the ADS server, choosing "Install ADS Administration Agent". During installation, you must pass the location of the certificate the ADS server created during installation. This certificate is used to secure communication between the device and the ADS server; you can copy it locally or specify a UNC path. You'll be asked for credentials for the administration agent, which can be the LocalSystem account or a domain account if various types of network resource access will be required. Note that manually installing the ADS Administration Agent on the reference machine is necessary to enable the ADS server to control it.
Finally, create a Sysprep folder on the root of the C: drive with an i386 subfolder underneath, then copy into it sysprep.exe and setupcl.exe from the deployment tools that match the operating system (and the service pack version) installed on the reference machine. You must also place a sysprep.inf (based on one of the four templates provided with ADS in the samples\Sysprep folder) in the Sysprep folder (not the i386 subfolder). Templates are provided for Windows 2000 Server and Windows Server 2003, one each for a workgroup member and a domain member. You can customize these templates to suit your business and technical requirements. Take note of the "^variable^" entries in the file. These variables will be replaced with values for the actual device during deployment, giving you customization of the target server beyond that defined in the captured image. For example, the default Windows Server 2003 domain Sysprep template includes entries for items such as computer name, product key, and domain. Figure 1 illustrates.
Figure 1 Sysprep Template Entries (Close Figure 1)
;SetupMgrTag
[Unattended]
    OemSkipEula=Yes
    InstallFilesPath=C:\sysprep\i386
    TargetPath=\WINDOWS

[GuiUnattended]
    AdminPassword=^ADMINPASSWORD^
    EncryptedAdminPassword=NO
    OEMSkipRegional=1
    OEMDuplicatorstring=sysprep-id
    TimeZone=4
    OemSkipWelcome=1

[UserData]
    FullName="ADS"
    OrgName="Microsoft"
    ComputerName=^ADS_COMPUTER_NAME^
    ProductKey=^ADS_WINDOWS_PRODUCT_KEY^

Capture the Image
After preparing a reference machine, the next step is to capture it. First, add the machine as a device. Without this step, the machine won't get past the stage of getting addresses from DHCP when it attempts to boot from the network and you'll have no way to send a task sequence to enable actions to be performed. You add the device via the ADS Management MMC snap-in that is part of the "Microsoft ADS" program group. Right-click Devices and select "Add Device...". After you enter a name for the server, a description, and its MAC address (or SMBIOS GUID), click OK, and close the dialog.
Right-clicking on a device brings up various context-sensitive options that will vary depending on the state of the device. Since this is a new device, the first action is to "Take control," which, if successful, will switch the status to Yes for Control. The device is now ready to accept commands that will allow us to perform the capture.
Now you need to create a task sequence that will run Sysprep, reboot the server to the deployment agent, and capture the C: drive to an image with the name of your choice. Under the samples\Sequences folder you'll find a number of templates. In this example I'll use capture-image.xml and tweak it slightly as needed. I'll leave the first two steps in the template alone—they just run Sysprep and boot to the deployment agent. In the third step I'll set a real name for the image and a description, as shown in Figure 2.
Figure 2 Giving the Image a Real Name (Close Figure 2)
<!-- start sequence -->

<sequence command="capture-image.xml" description="capture image"
     xmlns="http://schemas.microsoft.com/ads/2003/sequence" version="1">

    <!-- STEP 1 sysprep step -->
    <task doesReboot="true" description="sysprep target">
        <command>c:\sysprep\i386\sysprep.exe</command>
        <parameters>
            <parameter>-quiet</parameter>
            <parameter>-reseal</parameter>
            <parameter>-reboot</parameter>
        </parameters>
    </task>

    <!-- STEP 2 boot to deployment agent -->
    <task description="Boot to deployment agent">
        <command>/pxe/boot-da</command>
    </task>

    <!-- STEP 3 capture image -->
    <task description="Capture image">
        <command>/imaging/imgbmdeploy.exe</command>
        <parameters>
            <parameter>Win2003EntSP1</parameter>        <!-- name of the image  -->
            <parameter>\device\harddisk0\partition1</parameter>
            <parameter>"Windows 2003 Ent SP1"</parameter>    <!-- image description -->
            <parameter>-c</parameter>                   <!-- specifies capture mode  -->
            <parameter>-client</parameter>              <!-- required parameter  -->
        </parameters>
    </task>
</sequence>
You can now initiate the task sequence, but first make sure the server boot priority is configured to attempt to boot from the network first (prior to the hard drive). Otherwise get ready to manually force a network boot when the reference service shuts down after Sysprep execution. To initiate the capture task:
  1. Right-click on the device and select "Run job...", then click Next.
  2. From the job type options select "Create a one-time job" and click Next.
  3. Enter a description, for example "Capture reference server 2003", and click Next. A list of command types will be displayed, select "Task sequence", and click Next.
  4. Select the name of your customized task sequence (see Figure 3), and click Next. Then click Finish and the job will begin.
Figure 3 Always Use a Custom Task Sequence
Figure 3 Always Use a Custom Task Sequence
The device's state will change to Busy and a new entry will be added to the Running Jobs branch of the ADS Management snap-in, which, when selected, will give detail about the exact stage of the job process. You'll notice in Figure 4 that the Sysprep target and boot to deployment agent tasks have completed successfully and the actual image capture is in progress.
Figure 4 Detailed Status of the Selected Job
Figure 4 Detailed Status of the Selected Job
Once Sysprep has executed, the server will reboot, booting to the ADS deployment agent, and the capture process will execute, as shown in Figure 5.
Figure 5 Booting to the ADS Deployment Agent
Figure 5 Booting to the ADS Deployment Agent
The commands are executed as defined in the XML task sequence file, which in this case is running the imgbmdeploy.exe tool. Upon completion, the reference machine stays booted to the deployment agent, awaiting further action. At this point, you can instruct it to shut down from the ADS server or just manually cycle it, and then either deploy an image to it to test what you've captured (a good idea) or perhaps rebuild another flavor of server to form the basis of another image. The new image will show up under the Images branch of the ADS Management snap-in, and there will be a file created in the ADS Images folder (by default the root of the C:\ drive) named with the GUID of the capture. You can find it by looking at the image entry in the ADS Management snap-in.

Deploying the Image
Some of my clients use ADS to capture a server for basic disaster recovery and restoration. If you perform a typical capture without the Sysprep stage, you boot to the deployment agent and capture the disk. Doing this every couple of weeks is a very fast way to create an image for restoring the server in the event of a disaster. However, capturing machine images is not the goal of this process; the goal is to simplify deployment, so let's deploy a server.
Deployment is actually quite simple once the task sequence is created. Just as with a capture, you must create a device for the server to be deployed. The device will have unique variables defined by the administrator and they will be used to populate the placeholders in the sysprep.inf file. You configure the device to boot to the deployment agent and then run a deployment task sequence. It's easy.
Once you've created the device, open its properties and define the variables needed to complete its Sysprep population. The exact variables will depend on the Sysprep modifications made, the sample used, and the task sequence mapping. However, if you used the Windows Server 2003 domain sysprep.inf sample and your task sequence on the domain deployment (da-deploy-image-domain.xml), you'll need the variables as shown in Figure 6. Note that these names don't match those that were in the sysprep.inf. Don't worry; the task sequence for the deployment has a "Customize sysprep.inf" stage that shows which variable from the device it will place in the sysprep.inf placeholder.
Figure 6 The Device's User Variables
Figure 6 The Device's User Variables
The next step is to create the task sequence. I'll use da-deploy-image-domain.xml as the basis for the deployment. After you customize your file, save and name it based on its content, (like deploy-2003entserver.xml) for easier management in the future. Not much customization is required in this case, but let's walk through the main steps.
First, the bmpart.exe utility is called to delete all existing partitions and create a single 5GB partition that will become the C: drive. You can customize this to create additional partitions of varying sizes but remember, the first partition must be at least the size of the C: drive of the captured reference server. It does not matter if the content is only 3GB. If the reference machine had a 20GB C: drive, any server to be deployed must have at least 20GB available for its C: drive.
Next, the image is written to the first partition using the imgbmdeploy.exe utility. You'll need to change "imagename" to the actual name of the image to deploy (such as "Win2003EntSP1", the image I previously captured). Now the sysprep.inf placeholders and variable names I defined for the device will make sense, showing which variable for the device is read and which placeholder it replaces in the sysprep.inf file. Next, a number of ADS management functions are performed after the device is deployed, as long as the correct certificate is in place to allow communication with the ADS server.
The server is then rebooted from the local hard disk instead of the deployment agent and the device is set to boot from the local hard disk on subsequent reboots. You should leave all ADS managed servers to boot from the network by default and let ADS then instruct each server to boot from its local hard disk. Then you won't have to manually tell the server to boot to the network when an ADS-type capture or install is required.
The final step is to tell the device to boot to the deployment agent by selecting the properties of the device and on the General tab set its "Default job template:" to "boot-to-da" and click OK, as shown in Figure 7. Now turn on the server to be deployed (ensuring it's configured to boot from network as its first boot device). Note that after the server is deployed its default job template will change to boot-to-hd which is its local hard disk.
Figure 7 The Server's Default Job Template after Deployment
Figure 7 The Server's Default Job Template after Deployment
Once the server to be deployed is booted to the deployment agent (its status will be "Connected to DA" instead of "busy"), execute the deployment task sequence in the same way we executed a capture task sequence. That is, right-click on the device and select "Run job...", then follow the same steps except this time select the deployment task sequence created.
The restoration status will show on the server being deployed and once again on the ADS server the job status can be monitored to see exactly where it is in the process and which steps have been completed. The deployed server will reboot twice, once after the image has finished deploying and again after the mini-setup wizard execution that processes the Sysprep answer file. In Figure 8 you can see that the image has been deployed and the sysprep.inf file updated. The rest of the actions are performed quickly.
Figure 8 Deployment Task Status
Figure 8 Deployment Task Status
So there you have it—the entire process for a basic capture and deploy. Obviously, more steps will be added in many server deployments to take advantage of ADS functionality such as capturing a virtual floppy to perform a BIOS update or performing hardware RAID configuration and post-deployment actions, but getting the main process working is a vital first step. Once you understand the construction of task sequences, progress in more advanced areas will be rapid.

Extended Uses
Included with ADS are a number of tools that prove very useful for adding functionality to ADS. For example, the imgmount.exe tool lets you mount an ADS image to a drive letter and then view and modify the content. Use caution, however—modifying an image without careful control may render the image unusable.
Another great tool is imgdeploy.exe, which can be used in other environments such as Windows PE to perform image captures and restorations. It's via this method that you could, theoretically, capture desktop images if you so desired.
I edited the XML files in Notepad, but ADS includes the ADS Sequence Editor, which provides a graphical environment for manipulating the task sequence XML files. This may prove more intuitive to use for many administrators (see Figure 9).
Figure 9 Editing a Task Sequence File
Figure 9 Editing a Task Sequence File
You will also want to understand the role of ADS in server consolidation. Because ADS can capture entire disks and restore them, it is very useful if you want to take physical boxes and move them into a virtual environment. The Virtual Server Migration Toolkit, which can be downloaded from microsoft.com/windowsserversystem/virtualserver/evaluation/vsmt.mspx, fully leverages ADS capabilities while adding features to help you configure the Virtual Server 2005 environment to closely emulate the physical hardware you were running on.
If you have a large server farm and have ADS deployed, it's not too difficult to repurpose groups of servers as required by simply selecting them as a group and, for example, rebuilding them as IIS servers when needed, or quickly adding some additional domain controllers.
Ultimately the decision to use ADS will depend on a number of factors, including whether you have Windows Server 2003 Enterprise servers that can host the ADS service, whether you deploy enough servers to warrant the initial effort to set up and configure ADS (though it is possible to get ADS up and running in a minimal fashion with a very small amount of effort), and whether you need to back up servers or perform server consolidation. Once you determine your needs, however, you'll find that ADS has a lot to offer.

John Savill is Director of Technical Infrastructure for Geniant. He is a CISSP, a Security and Messaging MCSE on Windows Server 2003, a six-time MVP, and a Krav Maga instructor. He is the author of Windows Server 2003 Active Directory Design and Implementation (www.packtpub.com/book/active_directory).


 Deployment: Create More Flexible Office Deployments with the Local Install Source
Deployment Dean Yamada

Create More Flexible Office Deployments with the Local Install Source
At a Glance:
  • The inner workings of the Local Install Source (LIS)
  • Using system environment variables
  • Enabling fault-tolerant Office delivery path locations
  • Troubleshooting the LIS

Have you ever gotten calls from your traveling employees who need to repair their Microsoft Office 2003 applications, but can't connect to the company network? Wouldn't it be great if
the people you support could have their Office 2003 installation repaired without a VPN connection back to your corporate network? If they received their Office installation from an administrative installation point (AIP), they certainly can't do a repair unless they can access the network resource.
The Local Install Source (LIS), introduced with Office 2003, is an Office Source Engine service feature that eliminates the need to be connected to the corporate network in order to make a repair or change installation options. LIS accomplishes this magic by copying the compressed source files from the Office 2003 installation media (for example, the Office 2003 CD-ROM or a network-based AIP) to the Msocache folder, a hidden folder on your local hard disk that has security permissions applied to it.
In this article I'll cover some important things to know about the inner workings of the LIS and introduce new functionality in the Office Source Engine to ensure that enterprise deployments of Office 2003 with the LIS remain trouble free (especially as server names or mapped drive letters may change.) I will also cover LIS maintenance and finding a valid source. A thorough understanding of the topics in this article will prove highly beneficial for your new or current Office 2003 deployment and future maintenance.

Introducing the LIS
The LIS is different than the administrative installation points you may have used for Office 2000 and Office XP. (Note that you can still technically use, and get support for, an AIP for Office 2003; however, usage of AIPs is expected to be discontinued for the 2007 Microsoft Office system.) One main difference is that LIS source files remain compressed and are stored locally on your hard drive. An AIP, by contrast, contains uncompressed source files that are stored on a network shared resource. Under the AIP system, administrators would frequently run into patch synchronization issues with slipstreaming patches into the AIP, causing them to continually recache client machines to be current with the AIP's patch level.
Also, services such as Microsoft Update, Windows Server Update Services (WSUS), and the Systems Management Server (SMS) Inventory Tool for Microsoft Updates (ITMU) only work with Office XP AIPs that are at an RTM (original media version) level. For Office 2003, the AIP must be at an RTM level or Service Pack 2 (SP2) level. For a better explanation, two Knowledge Base articles describing these issues can help: support.microsoft.com/kb/903773 and support.microsoft.com/kb/902349. For Office XP, see support.microsoft.com/kb/922665.
The primary benefit of the LIS is that you are no longer tied to a network source location if your Office 2003 application needs to perform a self-heal or other resiliency operation. A traveling executive in a hotel room could have her installation of Office 2003 repaired without the need for a VPN connection back to her corporate network. If this executive had received her Office install from an AIP, she would not be able to repair back to a valid source unless she could see that network shared resource from her hotel room.
If you review the Windows® Installer source list location on an Office 2003 installation that uses LIS, you will see that the Installer references a local disk drive location, not a Universal Naming Convention (UNC) or shared network resource like you would see with an AIP. The added benefit of a LIS is that you can repair your Office 2003 applications from the LIS, and the LIS can repair itself as needed.
In the past, with an AIP not only would you have to be connected for a standard Windows Installer application to be repaired, but it could only repair if it was connected to the actual source specified in the installer information for that product. With Office 2003, if the LIS has lost that information, it references the following registry key where the path to the original source used to create the LIS can be found: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\Delivery.

What If the Source Is Unavailable?
Of course, all this only works if the original source is still available. Thankfully, the Custom Maintenance Wizard (CMW) utility in the Office 2003 Resource Kit can add extra Office delivery path locations and change the default Office delivery path as well. It's the only supported way to change the Office delivery key after an installation. To accomplish this, see the steps outlined in the Resolution section of the KB article at support.microsoft.com/kb/913420.
To change installer source list locations as needed, SMS provides such features as the Windows Installer (WI) Source Location Manager. In addition, the task of programmatically changing WI source list locations using the WI API is well documented in MSDN® online. However, changing the WI source list location for the LIS (<LocalDriveLetter>:\MSOCACHE) is not supported, so you've always needed another way to change the Office delivery path. Now there's a solution. Using system environment variables, combined with the CMW makes outdated source locations a worry of the past, and you can change the Office delivery locations just as easily as the Windows Installer source list locations. But for the Microsoft Office Source Engine (OSE) to be able to read system environment variables, you'll need the hotfix that is available with the following KB article: support.microsoft.com/kb/915433.
Here's a typical registry key for a static Office delivery path for Office 2003 Professional Enterprise Edition:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\
Delivery\SourceEngine\Downloads\90110409-6000-11D3-
8CFE-0150048383C9\Sources\90110409-6000-11D3-8CFE-0150048383C9.
"Path"="R:\\AppServer\\AppShares\\Office2003ProEnt\\"
Here's the same key with a system environment variable:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\
Delivery\SourceEngine\Downloads\90110409-6000-11D3-
8CFE-0150048383C9\Sources\90110409-6000-11D3-8CFE-0150048383C9.
"Path"="%SYSTEMVARIABLENAME%\\AppShares\\
Office2003ProEnt\\"
This dynamic variable will certainly make your enterprise deployment and Office software maintenance more flexible. Just remember these two things: this system environment variable must be predefined on all machines first, and at least one reboot is required the first time a system environment variable is created. Afterwards, if the value changes, you do not need to reboot. You should pre-stage your machines by declaring this variable well in advance of your Office deployment. An article explaining environment variables can be found at support.microsoft.com/kb/310519. If you want to automate the setting of a system environment variable after a single reboot, a quick way to do that is to deploy and merge a variable from the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment registry key.
For new deployments of Office 2003, you can specify Office delivery path locations in the original transform, but for existing deployments, you must use the CMW utility. Note that a transform is a Windows Installer Transform (MST) file, which holds various installation settings so you can customize your Office application deployment.
Using the Custom Installation Wizard from the Office 2003 Resource Kit (ORK) toolbox (office.microsoft.com/en-ca/assistance/CH011496861033.aspx), you can specify the locations that will be required when you reach Step 14 of the wizard, as you see in Figure 1. For even greater flexibility, you can specify more than one system variable for your source location.
Figure 1 Specify Office Delivery Path Locations
Figure 1 Specify Office Delivery Path Locations
Once Office has been deployed with a transform, you can only use the Custom Maintenance Wizard utility from the ORK Toolbox to revise settings and options that differ from the original transform. Office will only recognize the first and original transform settings unless a CMW file has been deployed.
If you need to add or modify the original Office Delivery source path locations, you can do so in Step 10 of the CMW. Please note that this step only appends locations unless you check the "Clear existing Server list" setting, which you will find in the bottom-left of the screen. In most cases, the sole reason to create this CMW file in the first place is that existing Office Delivery path locations were not valid, so you will almost certainly want to choose to clear the existing server list. As I said earlier, keep in mind that you can have multiple Office Delivery path locations with multiple system variables for extra flexibility.

LIS Maintenance
Now that you have your Office delivery path locations for the LIS properly set, you can focus on maintenance. One important administrative tool is the LISTool, available for download from microsoft.com/office/orkarchive/2003ddl.htm.
With the LISTool, you can move the LIS to a new root drive letter, delete, and enable it. Figure 2 shows LISTool running.
Figure 2 Command-Line Options
Figure 2 Command-Line Options
Figure 3 lists the command-line options you can use with LISTool.exe; you can also view these options by typing /? at the command line. The following Office Resource Kit online link describes the LISTool in more detail: office.microsoft.com/en-us/assistance/ha011402361033.aspx.
In certain troubleshooting scenarios, the best course of action is to delete and re-enable the LIS to ensure its integrity. For example, if you try to start a program such as Microsoft Word, Excel®, or PowerPoint® 2003 for the first time, you may receive the following error message: Installation Error: File not Found (as described here: support.microsoft.com/kb/896866). In this case, you'll need to delete and re-enable the LIS with the LISTool.
Setting Cache Properties
To handle the local installation source separately from the Office 2003 installation, you can set the CACHEONLY property in the [CACHE] section of the Setup.ini file or on the command line. When it is set, CACHEONLY does the following:
  • Sets the CDCACHE property to 2.
  • Sets the local installation source to NoCleanUp in the Windows registry.
  • Directs setup to download all local installation source files and then exit before starting the installation of the Office package.
A similar property—REMOVECACHEONLY—can be used by an administrator to delete the local installation source and all corresponding registry information from users' computers. (Note that several properties used to customize the behavior of local caching work only with an updated version of the Office Setup program.)
The final property is ENFORCECACHE, which must be set in the [CACHE] section of the Setup.ini file or on the command line. When it is set, ENFORCECACHE overrides several other properties. See office.microsoft.com/en-us/assistance/ha011402451033.aspx for details.

There are several other ways a LIS can become corrupt or lose files. During certain patching efforts prior to the release of Windows Installer 3.1 v2 there were instances where, during a failed or canceled patch installation, some of the CAB files in the LIS would be erroneously deleted during the rollback operation (see support.microsoft.com/kb/893803). Therefore, it is important that your enterprise also uses the latest version of Windows Installer.
Running the Disk Cleanup Wizard on a drive with more than one partition can be another source of problems. If Disk Cleanup Wizard is run on a partition that does not contain the LIS, the LIS entry for cleanup is still flagged and LIS registry information is erroneously deleted. A hotfix has been created for this issue; see the article at support.microsoft.com/kb/920355.

Final Thoughts
You can fine-tune some of the LIS initial deployment settings, including pre-staging the LIS by ensuring you are using the latest version of the Enhanced Setup.EXE for Office 2003. Always use the latest available setup.exe, which you can read about and download from office.microsoft.com/en-ca/assistance/HA011402451033.aspx.
The new setup can also deploy the local installation source separately from the Office 2003 installation in case you want the advantages of local caching but need to manage network resources carefully. The local installation source can be deployed first, and Office 2003 can be installed at any time after that. For the steps involved, see the sidebar "Setting Cache Properties." You should now be ready to enjoy more flexible, customizable Office installations.

Dean Yamada is a Product Escalation Lead for the Microsoft Office Setup and Deployment Team (microsoft.com/services/microsoftservices/srv_support.mspx). Dean and his team can answer your Office patching deployment questions or provide an Office deployment workshop onsite. For more information, e-mail him at owkshop@microsoft.com.


 Small Business Server: Master Your Domain: Build a Corporate Network at Home
Small Business Server Matt Clapham and Jesper Johansson

Master Your Domain: Build a Corporate Network at Home
At a Glance:
  • What's so cool about SBS?
  • Managing maintenance and updates
  • Networking, Active Directory, and security
  • New features in SBS 2003 R2

We're hardcore IT guys at heart, and we want well-managed, smooth-running computers—not only at work, but at home as well. We're willing to put some time into making that happen. We're
also privacy and security geeks. We like running things ourselves to retain the desired level of freedom and control.
We wanted an affordable solution that would provide easily shared resources (files, calendars, printing, and so on) to benefit all family members. We wanted an integrated solution that was familiar to us so we didn't have to cobble together a smattering of independent pieces. We wanted a flexible solution that could be expanded upon to do new things later. Enter Microsoft® Windows® Small Business Server 2003 R2 (SBS R2).
SBS R2 is based on Windows Server® 2003 SP1, but with a number of enhancements and additions that make it an ideal base platform for a home network server. From providing secure communications and seamless single sign-on, to storing pictures and serving up a Web page for family and friends, SBS R2 can handle it all. For a typical home network, the server doesn't even need to be terribly powerful. Any reasonably modern system with a fair bit of RAM (at least 512MB) and hard disk space (80GB or so) will do.
The whole premise of using a server in the home makes us sound pretty geeky, but there are actually some really good reasons to do so, apart from the innate coolness factor it will garner you at neighborhood parties! For instance, all the other parents may know the difference between a knuckleball, a free throw, and a field goal, but how many of them have a centralized backup system for their home network?
With SBS, and particularly with the R2 release, we have fundamentally changed our approach to home networking. SBS actually makes it possible to use at home many of the same techniques that we currently employ on much larger corporate networks. In addition, it gives us a safer, smaller, and more predictable environment to test products that we would otherwise not get to experiment with as IT professionals.

Maintenance and Updates
One prime area where SBS R2 can be useful is in maintenance and updates. Much of our maintenance routine is automated and centralized, just like a well-run corporate network. The daily performance and patch status reports help us easily keep an eye on the whole setup. SBS also includes a very comprehensive and useful backup feature that wraps a wizard around the built-in backup functionality of Windows Server 2003 and also turns on a couple of other niceties, such as the Shadow Copy feature. This feature is a versioning mechanism for file shares that keeps track of old versions of data files and allows users to recover older versions of documents, even accidentally deleted ones. It is highly useful as a reliability feature and saves time in having to restore files from backup. Figure 1 shows a screenshot of the dialog used to retrieve the old versions.
Figure 1 Shadow Copy Allows You to See Old Versions of Folder or Document
Figure 1 Shadow Copy Allows You to See Old Versions of Folder or Document
In addition, the wizards and checklists in SBS make typical configuration tasks, like adding user accounts, file shares, and backups, a snap. There are a few simple questions to answer and the task is done. Frankly, in some cases, the wizards add functionality that we sorely miss in the standard editions of Windows Server 2003. For example, SBS comes with a highly useful Change IP Address wizard. To fully appreciate this wizard you need to realize that a single SBS server runs Dynamic Host Configuration Protocol (DHCP), DNS, Active Directory®, and Windows Internet Name Service (WINS). To change the IP address of the server you have to update settings in all those services. The wizard does it all for you! Even for an industrial-strength server, with only one or two of those infrastructure services, the wizard would be handy!

Common Computing Environment
One feature that really makes SBS worthwhile is Active Directory. Using Active Directory we can allow users to have the same accounts and settings in multiple places. For example, we both have three or four computers at home (they really add up after a while). Being able to maintain a single user account across all of them is very useful for the whole family. As long as the applications are installed everywhere, the entire family can move across computers and have everything just work.
Using Active Directory, we also get Group Policy. By default, SBS comes with policies that make standardizing desktops a cinch. Both of us also have a couple of policies for custom settings though. We typically use one for settings that we want on the SBS server only and one for settings we want on the clients. This allows us to easily respond to various contingencies. For example, after the "Sony rootkit" fiasco in late 2005, Matt added a policy to turn off Autoplay on all of the computers.
SBS also includes an e-mail server (Microsoft Exchange Server 2003), so we have our own private e-mail addresses for friends and family. Because we operate the servers ourselves, we don't have to pay for the privilege or be subjected to on-screen advertisements. This can be a bit tricky to get working from your home environment though, and you'll likely need to purchase a domain name.
SharePoint® is all the rage in corporate networks these days. SBS includes a simple SharePoint site called http://companyweb. This can be easily adapted into a family info center where everyone can share upcoming events, plan trips, or report computer problems to the "helpdesk." The Geek-in-Chief (in our case, us) can even sign up to receive alerts from help desk submissions and solve user problems right away. SharePoint has a vibrant community with tips, tricks, and improvements that make it quite useful for knowledge management. Microsoft has even released a plethora of SharePoint templates to use in customization.

Networking
SBS also makes for a very nice file server, allowing you to quickly and easily share documents, pictures, music, and video within your home network. Additionally, you can connect a printer to the server and share it with everyone quite easily. One caveat is that many home-use printers do not come with Windows Server 2003 drivers. Their drivers would work, if they were not explicitly designed to not install on Windows Server 2003. If you have one of those you need to install it on one of the clients instead. SBS can still publish the printer for you though.
When we're on the road or at work, we use the SBS Remote Web Workplace (RWW) to check our private mail and calendars or access our home desktops. RWW is one of the amazingly cool features in SBS. Forget about the pay-for-play solutions that require you to install software on your computer and then tunnels your connection through some third-party you may not trust! RWW comes in the box and gives you remote access to e-mail and computers. It is one of our favorite features of SBS. Gone are the days of not having access to the files or e-mail from your home computer. Just connect to the secure sockets layer (SSL)-enabled RWW and access what you need.
Keep in mind here that for RWW, and actually other Web sites and e-mail as well, you need a hostname that you can find on the Internet. If you have a static IP address that is easy, but if you have a dynamic IP address, you need to use one of the dynamic DNS services that map your hostname to your IP address. This is not that difficult, but it is an extra step you need to consider. We use this approach ourselves, and once set up, we pretty much forgot about it except for the occasional IP address change. There are several services available including DynDNS and ZoneEdit.
How many home PCs are actually backed up regularly? Sadly, very few. SBS includes a wizard (of course) for the built-in Windows backup tool that configures and schedules jobs to run at regular intervals. Just connect (via USB2 or IEEE 1394) a large removable/external hard drive and point the backups there with the configuration wizard. Then all the data stored on the server—and by extension users' data via redirected My Documents, roaming profiles, SharePoint, and e-mail—will get backed up on the schedule you define. You can even use two drives and keep one off-site in a safe deposit box. Barring unforeseen problems, backup becomes a "turn it on and forget it" scenario, as long as your USB drives are reliable.

Security
Are you worried about protecting your children online? ISA Server 2004, included in SBS R2 Premium Edition, can be configured to disallow visits to certain Web sites and block unapproved programs from communicating over the Internet. There's even a huge list of third-party plug-ins for ISA Server to block various types of inappropriate content at the network edge. In fact, ISA Server is infinitely extensible to those who take the time. For instance, before the WMF update was available earlier this year, Jesper blocked all WMF files from getting into the network by using the firewall.
10 Tips for Deploying SBS
  1. Back up your important user data first. Purchase an external USB2 or IEEE 1394 (Firewire) hard drive and use it to back up important data from all your client systems. Afterward, you can use this drive for migrating data to the server and eventually it will become your backup drive for automatic server backups.
  2. Plan your network design. Take a look at the whitepaper "Understanding Your Network" on TechNet online (the link can be found in the "Additional Resources" sidebar) and choose the deployment scenario that best fits your home network design.
  3. Plan your external connectivity. If you want to set up your own Web site or receive external e-mail at your server, register a domain name. You might need to sign up with a DNS registration service like ZoneEdit or DynDNS if your Internet provider uses dynamic IP addressing.
  4. Plan your server. It doesn't take much to run a decent SBS server for a home environment. You'll need at least 512MB of RAM, two hard drives (small operating system drive and larger disk for user data), one or two NICs, UPS battery backup, and the removable hard drive mentioned earlier.
  5. Plan the domain details. Choose a domain name for your Active Directory—in other words, your Windows domain name. Figure out how many user accounts you need and what they'll be called. Decide what role each user will have in the domain. Choose a naming plan for servers, clients, and other resources.
  6. Use the wizards as much as possible. The wizards in SBS are your friends and should be the first course of action for anything using built-in features. They'll save you much time and hassle.
  7. Upgrade clients. If possible, upgrade your client operating systems to Windows XP Professional. SBS really shines when it's the heart of your domain. Windows XP SP2 is our recommended baseline until Windows Vista™ is available. Home editions and Windows 9x simply cannot join the domain.
  8. Enhance or expand your server as necessary. One big benefit of SBS is that it can be expanded in just about any way a Windows system can, but we suggest you try to keep it to processes that can be run as a service. Otherwise, you'll need to log onto the server and run applications after every reboot.
  9. Read the performance reports. We regularly (about once a week) scan through the performance reports and resolve any recurring problems or trends.
  10. Engage the SBS community. SBS has a vibrant community of MVPs, enthusiasts, and operating partners. See the links in the "Additional Resources" sidebar for a number of online sources of information and support. If you get stumped or have a problem, post it to the newsgroup. Chances are someone else has experienced the same issue and has a fix or workaround for your situation.

You don't want your children up late playing with the computer? Set appropriate logon time restrictions for them! SBS can put tech-savvy parents in control of their children's computer usage.
On this note, beyond keeping all systems up to date, nothing is more important to securing your computers than not running them as an administrator all the time. Jesper has a great blog entry detailing why that is so important. An interesting story appeared in The Wall Street Journal in May 2005. Intel's CEO, Paul Otellini, mentioned he spends about one hour per week removing spyware from his daughter's computer. But what he missed was that if users aren't running as administrators then malicious software can't install (it can still run, mind you, but won't be able to leave a lasting impression on the system). Maybe if Intel's CEO had the great information on how to run as a non-administrator that is available at Aaron Margosis' blog (see Aaron's article in the August 2006 issue of TechNet Magazine, as well as his blog), he could spend that extra hour on the weekend with his daughter instead of with her computer.
SBS can make running as a non-admin a bit easier by giving people separate accounts for separate things. For instance, Jesper's oldest son wants to play certain older games that only run when you are logged in as an administrator. To make this work while maintaining security, he now has two accounts (both of which have extremely strong pass phrases). When he runs his games he knows to log on with the admin account, and when he wants to surf the Web he uses the more restricted standard user account. Spyware, rootkits, and other general badness simply will not install if you do not run as an admin. Should you need to make tweaks to a computer to allow a particular program to work as a non-administrator, it is relatively easy to use Group Policy to ensure that those tweaks are made properly on the right set of computers.
Now for the really cool part: you can give your spouse two accounts, and you can tell him or her not to surf the Web with one of them, and even if he or she forgets, you can set up ISA server to block all connections to the Internet (except for Microsoft Update) when any administrator tries to surf the Web. Now it really does not matter whether your users listen to you or not; they can't get to the Internet to do anything more than update the PC! It may cause some temporary friction with your spouse, but it will help prevent an infection that will cost you much more time later on.

What's New in R2
In its original release, SBS omitted one significant feature in the eyes of many people: an ability to control updates. In R2 we finally get Windows Software Update Services (WSUS) integrated with SBS. WSUS can be installed on SBS by following the handy "Understanding Your Network" whitepaper published by Microsoft on the topic (see the "Additional Resources" sidebar). However, SBS R2 doesn't just have WSUS, it has an SBS-ized WSUS! This one is fully integrated into the management console for SBS and comes with predefined policies for managing updates on an SBS network (see Figure 2). It even adds client and server patch status to the daily report. Now if any systems patches slip through the cracks because someone hasn't restarted in a while, it can be noticed and corrected much more quickly.
Figure 2 Update Functionality of SBS R2
Figure 2 Update Functionality of SBS R2
Being built on Windows Server 2003, SBS R2 is an easily extendable platform for a home server. For example, Matt runs the TwonkyVision UPnP media server for his connected media devices and Kiwi Enterprises Syslog Daemon to collect network statistics from his router. Jesper uses Scorpion Software's Firewall Dashboard network usage statistics.
Additional Resources
  • Windows Small Business Server R2
  • Windows Server Update Services
  • SharePoint Templates and Applications
  • SBS Community Site
  • E-Bitz, Official Blog of the SBS "Diva"
  • Understanding Your Network

Even the base Windows platform features can be utilized on SBS. Matt built an external Web site for his family using FrontPage® and hosts it from his server with IIS 6.0. He even added some monitoring jobs to SBS performance reporting system to track link-up time on his cable modem. One of Matt's friends installed Community Server 2.0 on his SBS setup to enable blogging! Jesper uses his home server to try out new enterprise products.

What's Still Missing
In case you couldn't tell already, we really like SBS, especially now that it has WSUS in the box. However, there are a few things you still need to know before you dive in.
SBS goes to great lengths to simplify, but it's still not for those who aren't willing to tweak it on occasion. It is far simpler to manage than other Windows Server 2003 editions, which are already not too bad as servers go. However, even SBS still requires a fair bit of skill, which explains why the SBS community is so vibrant! In fact, some of the areas in the Server Management Console actually link to the online communities.
SBS has three instances of various versions of SQL Server™ by default, all of them running as Local System. This is a security faux pas to us, but not a tremendous risk as long as you follow the handy checklist in securing the server and run the Configure E-Mail and Internet Connection Wizard (CEICW). In general, SBS is actually very good security-wise, and includes a great walkthrough on how to secure your system. We just dislike the number of SQL Server instances and the fact that they are difficult to secure beyond what CEICW does.
SBS includes a built-in software distribution mechanism that relies on logon scripts. It works well, but it requires users to be administrators. In fact, if you assign users to computers when you set up the computers, they will be administrators. For this reason, neither of us use this particular feature. As we said earlier, beyond keeping systems up to date, nothing is more important for security than to ensure that all users run as normal users, not as administrators. This is one of the few areas where we dislike the SBS approach. The best approach here is probably to start out with a domain-wide local administrative user, install all the applications using that user, and then ensure that all other users are running with low-privileged accounts.
SBS uses the built-in scheduling tool heavily for various items, such as backup. However, all the tasks run as the built-in Administrator account, which impedes disabling it in favor of using individual (domain) administrator accounts. In fact, many tasks in SBS, including installing R2, seem to require use of the built-in Administrator account. This violates the principle of individual accountability in security and is the other main gripe we have with the current SBS design.
Finally, SBS works really well with Windows XP Professional SP2 and to a slightly lesser extent with Windows 2000 Professional, but not with other client operating systems. That is not the fault of SBS in most cases, but rather the limitations in those operating systems. However, it means that while you can manage your Windows XP Professional workstations adequately, you cannot do the same with a Windows XP Home Edition PC, or Windows Media® Center Edition unless it's modified with an unsupported hack.

Wrap Up
Running a well-managed home network based on SBS doesn't take much in either hardware or time. Matt spends less than one-half hour per week checking reports. Once it's up and running—see the "10 Tips for Deploying SBS" sidebar—an SBS R2-based home network pretty much takes care of itself, leaving plenty of opportunity to work on other things using computers, not spending a bunch of time fixing them. This is not without its drawbacks; your spouse may brag to your neighbors about how well your home network runs and next thing you know, the folks down the street want you to help them "make it work like yours!" Enjoy!

Matt Clapham, a Senior Security Technologist at Microsoft, is an active participant in the Seattle IT Security community and a member of the Risk Management team on an IT incubation project.
Jesper Johansson is a Principal Security Program Manager at a large online retailer and a contributing editor to TechNet Magazine. He holds a Ph.D in MIS and has over 20 years experience in computer security.


 From the Editor: Downtime is a Four-Letter Word
From the Editor
Downtime is a Four-Letter Word

WHAT'S MORE DISTURBING than snakes on a plane, exploding laptop batteries, and the badly deployed puns in this Editor's Note, all put together? It's the idea that the office you're responsible for might start to fall apart. Downtime is a four-letter word for IT pros (it would not be inappropriate to call it "d***time"), and it's one of the topics we address in this issue.
If you flip through this month's excitement-packed issue, you can't help noticing how great it looks. We've put a lot of effort into upgrading our design, introduced with this issue. Thanks to our art staff, led by Valerie Myers and Amy Filbin, together with designer Jackie Goldberg, we've got a brand new, more sophisticated look that's really easy on the eyes.
While the design is groovy, the articles in this issue can be described as Groove-y. As this strained wordplay indicates, we are taking a look at communication and collaboration, specifically with mail servers and Office Groove 2007. John Morello takes a look at how Louisiana State University used Groove and SharePoint together to modernize an emergency operations center in the wake of Hurricane Katrina.
If you're in charge of maintenance and deployment, Windows PE can save you from lots of d***time in your job. Years ago, installs consisted of a stack of 5.25" floppy disks that got carried from machine to machine. Today things are easier, but there are brand-new challenges when you're deploying software to several floors of cubicles—in several states, or even countries. Standardized desktops are now the administrator's best friend, but getting there is always a big challenge.
We've got a special section on deployment this month. We explore Windows PE, which lets you boot into RAM in order to ease desktop deployments. We also take a look at how to automate the deployment and reimaging of servers. We even examine the Local Install Source, a method you can use to deploy, configure, and maintain Office on desktops across your org.
Finally, we have an entertaining piece about the travails of running a small ISP service from frequent contributor R'ykandar Korra'ti. As we went to press, Dara was involved in a serious accident and is now on the mend. We'd like to wish her the best and hope for a speedy recovery.
—JOSHUA TRUPIN

Thank you to the following Microsoft technical experts: Leon Alexandrou, Richard Bond, Max Ciccotosto, Bo Downey, Glenda Farley, Ryan Hoppe, Jay Kremer, Abbott Lowell, Michael Murgolo, Bruce Payette, Diane Prescott, Dustin Tomlin, Patrick Tousignant, Jim Truher, and Jill Zoeller.


 Toolbox: New Products for IT Pros
Toolbox
New Products for IT Pros
Greg Steen

Inspect HTTP Traffic
Fiddler
www.fiddlertool.com/fiddler
I am always on the hunt for new tools to help me see what is really going on beneath the surface of the applications I run, and when it comes to debugging Web traffic and applications, the free HTTP debugging proxy, Fiddler, gives me the details I need. When you launch the application, Fiddler plugs itself in between Microsoft® Windows® Internet Services (WinInet process) and your external connections, registering itself as a system proxy. This allows it to filter all HTTP requests and responses traveling through the WinInet on your system.
As you start to capture traffic with Fiddler, each HTTP session is displayed in the interface, color-coded by traffic-type. This means you can easily see what is involved in each HTTP transaction, including the HTTP response code, the host and URL requested, size, caching status, and content-type (for example, "text/html", "image/jpeg", and so on).
Clicking on an individual request gives you even more detailed information regarding that request and this is where the Fiddler tool really gets interesting. On the Performance Statistics tab, you can see the number of bytes sent and received, the response bytes by content-type, and a pie chart showing the percentage of the total request size that was taken by each of the elements of the response. Then, on the Session Inspector tab, you can see a tree view of the request headers, a detailed view of any forms, a hex representation of the response, the real raw request, caching information, any privacy information, and even an image viewer for applicable content-types.
Once you have isolated a problem or would like to run a very specific test, you can use the Request Builder feature to engineer and request that item from your target server. There is a drag and drop clone feature as well, so you won't have to craft the request from scratch. If all of these capabilities aren't enough for you, the Fiddler tool can be extended programmatically as well. It is a .NET Framework-based application that allows you to extend the application in two ways: "Inspectors" and "Custom Rules."
Inspectors are .NET plug-ins that are split into two types: RequestInspectors and ResponseInspectors. If you have a developer on staff who can write these custom-derived objects you can really tailor the application to your specific needs. For example, since the tool runs as a proxy, you can write a read/write Response Inspector that could modify the HTTP response before it gets to the application that requested it, testing both the resilience of your requesting application and allowing you to create very precise test conditions. All of this for free makes it a definite addition to my IT toolbox.
Price: Free.
Getting Statistics on HTTP Traffic
Getting Statistics on HTTP Traffic

Defrag Your Disks
Diskeeper 10 Professional Premier
www.diskeeper.com/defrag.asp
Defragmentation is a necessary task for every heavy system user, and you have probably seen quite a few of those red-peppered disk analysis graphic representations on your clients' workstations. File fragmentation is a natural occurrence just because files are constantly being written, rewritten, resized, and deleted. And as those files get scattered, system performance degrades.
Diskeeper 10 Professional Premier Edition from Diskeeper Corporation could be the answer to fast, efficient disk defragmenting for your organization. The Premier Edition gives you features such as I/O SMART, which will monitor drive access from other processes on your system to ensure that the defragmentation process doesn't severely degrade the overall system performance. Diskeeper also includes both manual and automatic volume defragmentation scheduling.
In addition, Diskeeper allows you to defragment your Master File Table (MFT) with its Boot-Time Mode, in accordance with Microsoft recommended guidelines (since the OS utilizes the MFT extensively). The Frag Shield feature helps you configure system paging files and MFTs as recommended by Microsoft published guidelines with the goal of keeping those files contiguous to ensure maximum system performance. For those looking for remote defragmenting, a remote administration edition is also available.
Price: $99.95 Professional Premier Edition. $49.95 Professional Edition. $29.95 Home Edition.
Diskeeper Map Showing File Performance
Diskeeper Map Showing File Performance

Edit HEX Byte Data
FlexHEX
www.flexhex.com
Your PC stores everything as hex bytes and getting to that "real" data can be tough. If you are looking for a hex file editor that will let you explore the hidden side of your files and file systems, FlexHEX might be the tool you want to consider adding to your toolbox.
FlexHEX integrates into Windows Explorer, so editing a binary file is as easy as right-clicking the file and choosing "Edit with FlexHEX". The application supports editing logical drives, physical drives, and files. The interface splits the file into four views: the address, the array of hex bytes, an ANSI pane for character content, and even a Unicode pane for displaying two-byte characters. Selecting a set of 1, 2, 4, 8, or 16 bytes from the byte array pane gives you a nice tooltip of any valid representations of the data such as corresponding decimal or RGB values.
Editing is simple and the application has a great undo/redo feature that will help protect you from accidentally destroying a file by typing in the wrong byte values. You can also search, replace, search for multiple patterns simultaneously, scan for strings, scan for GUIDs, and compare the current edit stream with a file or disk. Another nice feature of FlexHEX is the Navigation pane, which allows you to quickly jump back and forth between areas you have edited in the stream.
Price: $39.95 for home and academic users. $59.95 for corporate users.

Check for Bad Links
SiteScan
www.byteshift.co.uk/shareware/Web_spider_link_checker.html
Bad links are bad for business. And let's face it, they happen. If you are looking for an extremely low-cost, easy-to-use, simple site link spider, ByteShift's SiteScan could be a nice addition to your toolbox. The interface is simple: you type in the start page URL or select one of your recent URLs from the dropdown list, set the maximum number of links to follow, and then click the Start button. The application reaches out (don't forget to give it outbound access on your host firewall) and spiders the site from that starting point, displaying near real-time results as each subsequent link is processed.
The interface summarizes the results showing you the target page, its URL, the response time for retrieving that page, the HTTP response code, and any server messages like "403.1 Access Denied". You can limit the scanner to spider only the file types you want to spider; for example, if you are only interested in spidering the static pages on your site, you can limit the scan to HTM and HTML files and ignore dynamically generated pages like ASP.
The interface has a built-in browser that displays the page response of any link you click. And you can click a button on the browser and instantly see the source code, which is a fast way to troubleshoot (rather than having to open another editor window to see the page source). When you click on a child link from the starting point, the application will start another page analysis from that point. In the results pane, the application lists the link title, which is a fast way to find out which links are not titled correctly on your site.
Price: $20 per user.
SiteScan LinkChecker

Book Review
SQL Server 2005 Distilled
www.awprofessional.com
If you are thinking of migrating your data stores to SQL Server™ 2005 and aren't sure what features have been improved or added, SQL Server 2005 Distilled by Eric L. Brown (Addison-Wesley Professional, 2006) will get you started. The book is not meant to be a huge reference tome about all the ins and outs of the new server system, but rather presents most of the new and extensively revised components in comparison to SQL Server 2000.
The security feature set of the SQL Server product family has been greatly enhanced and the book discusses each new feature, from data encryption to authentication. And SQL Server 2005 Distilled does a nice job of describing the SQL Server Integration Services (SSIS) architecture along with details of the control flow designer and error handling within SSIS. One section of the book that should set some IT pros' minds at ease is about Certificate Revocation List (CRL) integration. It explains CRL integration into the platform and how and when to use managed code versus T-SQL in your apps.
The XML features of SQL Server have also been greatly enhanced and the book does a good job detailing those features. For example, it explains when you should use a native XML storage and when you should use XML View technology. It also introduces the XML data type, a variant of the binary large object.
Price: $34.99.

Greg Steen is a technology professional, entrepreneur, and enthusiast. He is always on the hunt for new tools and methods to help make operations and development easier for IT professionals.


 Utility Spotlight: File Server Migration Toolkit
Utility Spotlight
File Server Migration Toolkit
Jeremy Moskowitz
Download the code for this article: Microsoft File Server Migration Toolkit 1.0 (1434 KB)

Admit it. You've got 'em. Those Windows NT and Windows 2000 file servers that you just can't seem to shake. You want to update your file servers to Windows Server 2003 SP1 or Windows Server 2003 R2, but there's a problem. Your users rely on Universal Naming Convention (UNC) paths to point to shares on your file servers,
and if you move the data from the original servers to the new servers, all those UNC paths will break. All of your users will call the help desk, and you'll have to go into hiding.
Or how about this problem. Say you started using Group Policy Software Installation and serving files using one file server. Now you have 50 Group Policy Objects (GPOs) deploying applications to your Windows XP and Windows 2000 machines. You're ready to turn off that original file server, but those 50 GPOs depend on it.
Just turning off these servers isn't an option. Simply copying the data to new shares on a new server won't work either. So what are you going to do? Microsoft has a cool tool to help you take control of your old file servers and seamlessly bring the data into the 21st century.

Introducing the File Server Migration Toolkit
The File Server Migration Toolkit (FSMT) is a free download available at microsoft.com/windowsserver2003/upgrading/nt4/tooldocs/msfsc.mspx. It consists of three parts: the DFS Consolidation Root Wizard, DFSconsolidate.exe, and the File Server Migration Wizard.
The DFS Consolidation Root Wizard is a GUI tool that works some serious magic. It lets you maintain the original UNC paths of the servers, even if you are planning to ultimately turn those servers off.
A command-line tool, DFSconsolidate.exe is called by the DFS Consolidation Root Wizard. While it's possible to use DFSconsolidate.exe on its own, I'm only going to discuss its use in conjunction with the DFS Consolidation Root Wizard.
The File Server Migration Wizard is a GUI tool that helps you plan your migration from the source servers to the target servers. It then performs the actual copying of original files to the target destination.

An Overview of Migrating Shares
The first thing you need to do is determine the source server, which is where files are currently stored, and the target server you want to migrate the files to. I'll work through an example to help you better understand what exactly is involved.
Consider the setup in Figure 1 as your starting point. You have a Windows NT® 4.0 file server (\\nt04) with three shares that contain user data. You also have a Windows® 2000 server with one share used to deploy software. The Windows XP machine needs access to the following:
  • \\nt04\ntshare1
  • \\nt04\ntshare3
  • \\w2k\software
Meanwhile, the Windows 2000 machine needs access to these servers and shares:
  • \\w2k\software
  • \\nt04\ntshare2
Figure 1 Original Setup with Windows NT and Windows 2000 Servers
Figure 1 Original Setup with Windows NT and Windows 2000 Servers
Now you introduce a target file server called \\fileserver6 (see Figure 2). This new server running Windows Server® 2003 SP1 will be responsible for receiving the shares and requests. The ultimate goal is to consolidate the existing shares onto \\fileserver6 and turn off the Windows 2000 and Windows NT servers. But you need to perform this consolidation in a way that preserves the original paths of each of your shares. Yes, you read that correctly. You want to be able to access the data that was on the original servers (the ones being turned off) as if the data is still stored on those original servers. Basically, requests that use the original paths will be directed to the new Windows Server 2003 file server after the old servers have been shut off. And, of course, you want to make sure security is preserved all along the way.
Figure 2 Moving the Data to Windows Server 2003
Figure 2 Moving the Data to Windows Server 2003
Figure 3 shows a Windows XP machine accessing a directory of files on both \\nt04\ntshare1 and \\nt04\ntshare3. The goal is for this Windows XP machine to continue to perform the same commands, using the same UNC paths after you move the files and turn the original file servers off.
Figure 3 Windows XP Viewing Files Using UNC Paths
Figure 3 Windows XP Viewing Files Using UNC Paths
To make this happen, you'll use a part of Windows that has been around for a while, but still isn't in widespread use. The Distributed File System (DFS) accepts incoming connections and routes them to existing shares—this is sometimes called referrals. It's sort of like a share of other shares in that it basically allows you to hang existing shares off a new DFS share or, more technically, the DFS root.
There are two kinds of DFS roots: standard and domain-based. Standard roots live on only one server. Domain-based roots can be hosted by multiple servers, which means they're more fault tolerant. If one of the servers that hosts the domain-based root goes down—no problem. The referrals just keep on truckin'. To learn more about DFS, check out the Distributed File System Technology Center.
Before we get into the details, here's a quick overview of the steps you will perform to achieve your goal of moving data to a new file share while maintaining the UNC paths:
  • Determine where you want to store your new files. In this example, you'll use fileserver6.contoso.com.
  • Rename any file servers that you have plans to permanently retire. Since you're retiring them, the new name doesn't really matter. For this example, \\nt04 will become \\nt04-ret and \\w2k will become \\w2k-ret.
  • Use the DFS Root Consolidation Wizard to reroute incoming requests for the retiring servers (\\nt04 and \\w2k) to the new location (\\fileserver6). Note that if you prefer, you could use two separate servers (one to hold the DFS roots and another to hold the files).
  • Move all of the files from the shares on your original servers to the new file share location.

Getting Started with the FSMT and DFS Consolidation Wizard
The FSMT comes as a single installation but, as noted, it consists of three separate components. The File Server Migration Wizard is meant to be run directly on the target file server. However, you can run the DFS Consolidation Root Wizard and Dfsconsolidate.exe anywhere.
Note that the FSMT documentation makes special mention of Knowledge Base article 829885 which talks about a DFS hotfix. The implication is that this hotfix must be loaded on the target DFS server. However, this hotfix is built into Windows Server 2003 SP1 and therefore is not needed when the target server is Windows Server 2003 SP1. Still, the FSMT documentation fails to mention one critical step: be sure the DFS service is started and set to Automatic for future restarts.
After the FSMT is loaded, you must change the names of the nt04 server (to nt04-ret) and the w2k server (to w2k-ret), where I'm using "ret" to signify retired. Feel free to use different names that are meaningful to you. This is necessary so that when clients try to connect to \\nt04 or \\w2k, neither actually exists anymore. This allows you to fool those incoming requests into shimmying over to the new shares on \\fileserver6.
In my tests, renaming a Windows NT4 server wasn't as easy as I would have liked. Simply renaming it doesn't magically change the name in Active Directory® (as would happen if I renamed a Windows Server 2003 or Windows XP machine). I had to drop the machine into a workgroup, rename the machine, and rejoin the domain (contoso.com). And, of course, along the way several reboots were required. Finally, I had to delete an orphaned computer account for Windows NT4 using Active Directory Users and Computers. In contrast, renaming the Windows 2000 server was a snap. I just renamed the server and rebooted. No muss, no fuss.
Once your Windows NT and Windows 2000 servers are renamed, you're ready to run the DFS Root Consolidation Wizard. This is pretty straightforward, asking for only a minimum of information.
The wizard will want to know your DFS root server. This is the location where the DFS root will be held. In DFS terms, this will be a type of standalone root known as a DFS consolidation root, which exists solely on the server you specify. Note that the root cannot be on a domain controller. In this example, you are putting the DFS consolidation root on the same server where the files will ultimately go (fileserver6). However, you can create the root on a server cluster if you want to increase the redundancy of the consolidation root.
You'll specify the local path of the top-level folder where the consolidation roots are stored. If you were migrating 10 servers, there would be 10 consolidation roots underneath this top-level directory. For this example, the local path of the folder is c:\dfsroots. The subfolders under c:\dfsroots are used only by DFS. You will not be storing migrated data in this folder.
And the wizard will ask you to specify which servers to consolidate (see Figure 4). This is where you map the original name (in this case, \\nt04) to the current name (\\nt04-ret).
Figure 4 Mapping Old Names to New Servers
Figure 4 Mapping Old Names to New Servers
If the Wizard finishes without errors, you've completed the first big step. Now, before you do anything else, take a moment to try something out. Go back to your Windows XP machine (see Figure 3) and run those same dir commands to access \\nt04\ntshare1 and \\nt04\ntshare3. Without restarting the Windows XP machine (or even logging off and back on), note that those same dir commands continue to work! This is because the DFS Root Consolidation Wizard has mapped \\nt04 to \\nt04-ret.
Let's take a look at what really happened in the c:\dfsroots folder on fileserver6. In Figure 5, you can see that the wizard created a shared folder (the DFS consolidation root) to represent each server you plan to migrate. Under each DFS consolidation root is a special type of folder, known as a link folder, that represents each of the shared folders you plan to migrate.
Figure 5 New Shares Representing the Old Servers
Figure 5 New Shares Representing the Old Servers
If you use Windows Explorer locally and drill down to one of the directories—say, ntshare1—you'll get an error. This is because DFS only provides a referral when the link folder is accessed via a UNC path. Also note that each server is now represented by a share called #servername (such as #NT04). This was created by the DFS Consolidation Wizard.
It is possible to use the DFS Consolidation Wizard if your shares are on domain controllers. However, the same rule applies for domain controllers as for regular file servers. That is, the server (domain controller) must also be renamed. Unfortunately, renaming domain controllers can be a real pain in the neck. Microsoft does have some domain controller renaming guidance in case you find this is something you need to do:
  • For Windows NT 4.0 domain controllers, see support.microsoft.com/kb/150298.
  • For Windows 2000 domain controllers, see support.microsoft.com/kb/296592.
  • And for Windows Server 2003, see support.microsoft.com/kb/325354.
Keep in mind that the DFS Consolidation Wizard cannot directly help if you have hardcoded, persistent mappings. This means that if someone has used the /persistent flag while using the net use command to map a drive letter (or the corresponding Windows Explorer commands), those mappings will now fail. But if you're using login scripts to map the drive letters each and every time a user logs in, you'll have no problem. This is because every time a request goes to the old server name, a new lookup to the DFS is generated and routed appropriately.

Moving the Actual Files
To actually migrate the files, you create a project contained within the File Server Migration Wizard, which appears as an icon on the start menu. The beginning steps are straightforward. Create a new project, point the File Server Migration Wizard toward the new DFS consolidation root that you created earlier (fileserver6), and wait for the wizard to recognize the servers, as shown in Figure 6.
Figure 6 The File Server Migration Wizard Will Recognize Your Servers
Figure 6 The File Server Migration Wizard Will Recognize Your Servers
At this point, you can specify which directory you want to place the new files in. For this example, they'll go in a new directory on fileserver6 called c:\migfiles.
A quick note on what is and is not copied. Obviously the files themselves are copied. In addition, the File Server Migration Wizard copies both NTFS and shared folder permissions. However, references to local groups are not copied. If local groups have permissions on the source's shared folders, you can use the command-line tool SubInACL.exe to adjust the permissions before or after the migration to replace the local groups. Alternatively, you can use a local group migration tool like the Active Directory Migration Tool.
After the project is formed, you can make any necessary minor adjustments (as shown in Figure 7). For instance, you might want to put the contents of ntshare1 in a directory named Sales Stuff instead of ntshare1. This might boggle the minds of your users, but you may have a valid reason.
Figure 7 Tweaking Settings for Each Share
Figure 7 Tweaking Settings for Each Share
Finally, you can step through the rest of the process by clicking the Continue button. The process is painless, but can take quite a while depending on how many servers you are consolidating. For lots and lots of servers, consider breaking up the effort into multiple projects—this will allow for some time for your servers to settle in and give you time to address any errors you may encounter.
Source and target servers are still available during the copying phase. In other words, there's no server downtime when copying files from the source server to the target server. And if there are errors during the copying process, you can try to fix the problem and recopy. A nice touch: since you can repeat this phase multiple times to fix errors, only failed copy attempts are retried. You don't need to copy the whole universe again if you've already copied 90 percent of it.
The last phase, which is called Finalize, should only be done when users won't be accessing the servers. In this phase, you disable any original access to the source shares and close any open connections. Additionally, all other project settings are locked.
After this, you're ready for the final test. Unplug the original servers from the network (but leave them running just in case). Then, as you did earlier on the Windows XP machine, use the dir commands to make sure you can access the copied servers using the original UNC path names.
Once you've confirmed everything is in order, you can turn off your old servers. Now they're free to be recycled, reformatted, and redeployed for another purpose. You can even make a fish tank out of them.

The Future
The FSMT is a great tool and it gets the job done. However, there are some small, nitpicky points I'd like to see addressed going forward.
As I mentioned, the FSMT uses what's called standalone DFS roots. The DFS root is put on one specific server. In this example, fileserver6 was used to store the standalone DFS root and acted as the storage point for the migrated files. However, you could have split the duty between file servers. That is, one server could have housed the DFS root while another was used to store the data. The problem with this approach is what would happen if the server holding the DFS root goes offline? That would be a major problem, as there would be no way to route to the new file server (or servers).
The ideal solution would be to use the more powerful domain-based roots which are fault tolerant. If the one server holding the DFS roots should fail, the fault tolerant domain-based DFS would kick in. Today, FSMT doesn't use domain-based, fault-tolerant roots, but I really wish it did. But as I mentioned earlier, you can try to work around this by putting the consolidation root on a set of clustered servers. If one node in the cluster goes down, referrals will continue. The FSMT product team tells me that all parts of FSMT are fully cluster aware and compatible. It will create cluster consolidation DFS roots and add cluster names instead of DNS aliases if roots are hosted on a cluster. That's a nice touch.
It's also unfortunate that you must rename the servers to perform any redirecting magic. I would love the ability to keep the server intact, name unchanged, and just redirect a specific share. This would allow me to keep using the server for whatever else it's doing, but still migrate specific shares. To do this, you would need symbolic links within the original share that would route requests to the new location. But right now Windows Server 2003 isn't quite there. Perhaps this will be possible in the next version of Windows Server, code-named "Longhorn".
The last note here is that you're not actually forced to turn off the servers you've migrated from. You can, if you want, keep the server online performing other roles. The problem is that this might make it difficult for other network services and clients to find the newly renamed machine. DFS is some pretty strong magic, but it's only for files. Other services won't be able to seamlessly find the renamed server.

Wrap Up
The FSMT is a cool solution that works as advertised. And the price—a free download—can't be beat. It should be noted that as good as the toolkit is, it's not meant to be a permanent solution.
The idea is that over time you should properly design your DFS, pointing users to the updated structure. Then you can phase out the roots you created. Essentially, the FSMT should bridge the gap as you bring your file servers into the 21st century.

Jeremy Moskowitz, MCSE and MVP in Group Policy, runs GPanswers.com, a community forum on Group Policy. His latest book is Group Policy, Profiles and IntelliMirror, 3rd edition (Sybex, 2005). Contact Jeremy at www.GPanswers.com.


 SQL Q&amp;A: Running in RAM, Agent Jobs, and More
SQL Q+A
Running in RAM, Agent Jobs, and More
Edited by Nancy Michell

Can Sql Server Run Entirely In Ram?
Q I'd like to know whether SQL Server can run entirely in RAM. If so, since RAM is volatile, how are changes committed so they are not lost in the event of a crash?

A The answer to your first question is yes and no. Database pages that have been accessed are stored in the buffer pool. (Pages are placed in RAM only when needed, and SQL Server™ doesn't reload pages.) These pages are kept in RAM as long as possible—that is, until memory becomes a problem. So let's say you've set SQL Server to the proper memory footprint and you've run some queries that touched every page in the database. The entire database would then be in RAM and it would stay there until you ran out of buffer pool space to hold new pages, or until other processes on the system cause physical memory pressure.
During each checkpoint, any modified pages are flushed to disk, so committed changes are taken care of. Log records are not cached per se. As soon as a commit is issued, the records are flushed to disk. So even if all dirty pages are still in RAM, a restart of SQL Server will replay the log records. Don't take this to mean you should rely on the log in this way. A huge log could mean a very long recovery period.
So pages stay in RAM until space is needed; a least recently used (LRU) algorithm is applied so that hot pages stay longer. Most of the time, you won't need the entire database in the cache anyway. So you shouldn't buy a ton of RAM to hold pages that probably aren't touched too often—you likely won't see any performance improvement.

Concurrent Sql Server Agent Jobs
Q I have a very large number of replication jobs set up on one of my servers (running SQL Server 2000). If I attempt to manually start one of the jobs that is not currently running, an error message sometimes appears that says something like the following: Error 22022: SQL Server Agent Error: Request to run job refused because the job already has a pending request for Schedule 618 (Replication agent schedule).
I've noticed that there are always 12 jobs running. Have I reached some kind of limit that prevents another job from launching?

A SQL Server Agent has a defined maximum number of threads it will run in total for all jobs. This is independent of the number of threads that can be run per subsystem. In SQL Server 2005, if this maximum number of threads is reached because so many jobs are running, then any jobs that are requested to start will be queued, and the agent will log a message that says:
"The job (%s) has been queued because the maximum number of working threads (%ld) are already running. This job will be executed as soon as one of the working thread finishes execution."
In SQL Server 2000, they are not queued and the agent logs this message:
"The job (%s) cannot be run because the maximum number of jobs (%ld) are already running. Please try this job again later."
Once a job is assigned to a thread, there is a limit on the number of threads used per agent subsystem. A Knowledge Base article for SQL Server 2000 that describes the thread count per subsystem in the registry can be found at support.microsoft.com/kb/306457. In SQL Server 2005, these values are stored in the syssubsystem table in the msdb database. SQL Server Agent queries these values by executing the sp_enum_sqlagent_subsystems stored procedure. Once a job is assigned to a thread, the subsystem being used by each job step is important because of the maximum number of threads allowed per subsystem. If the maximum number of threads per subsystem has been reached, the thread running the current job will pause for five seconds, then try again. This behavior should be the same for both SQL Server 2000 and SQL Server 2005.
You'll also want to consider job-start intervals. Suppose you have 100 distribution jobs attempting to start every minute, with SQL Server Agent configured to wait one second between starting jobs. The result would be that some of the jobs would start, run, and end. And this would probably work fine for up to about 60 jobs. However, some of the 100 distribution jobs would never even be initiated and SQL Server Agent would run the same jobs over and over. By changing the start interval from once every minute to once every five minutes, you'd actually improve the situation considerably. With this configuration, all of the distribution jobs are queued to start every five minutes, but only 60 actually start in the first minute. After about two minutes all of the jobs will have been initiated. The key point is that all of the jobs have a chance to be initiated within each five-minute interval.
To further improve the situation, you could stagger the start times of the jobs by a minute or so. You could set things up such that 20 jobs would start at minute zero, 20 more at minute one, 20 more at minute two, and so on. Then you would only be attempting to start 20 jobs every minute instead of 100 jobs simultaneously every five minutes. So if a five-minute interval turned out to be unacceptable, you could bring it back down to a minute or two and stagger start times as described.
To sum up, SQL Server Agent (2000) can start a maximum of 60 jobs a minute, as it explicitly imposes a one-second delay between starting jobs. By manually tweaking some registry settings, however, you can modify the number of maximum concurrent agent jobs, which is also covered in Knowledge Base article 306457.

Guidance On Forest Trusts
Q I'm planning to connect two separate Windows Server 2003 forests through a trust. Where can I find Microsoft guidance on SQL Server in trusted environments?
Forest Resources
The following documents provide valuable guidance on creating forests and implementing trust:
  • Planning and Implementing Federated Forests in Windows Server 2003
  • How to Raise Domain and Forest Functional Levels in Windows Server 2003
  • When to Create a Forest Trust
  • What's New in Windows Server 2003 Kerberos Authentication

A A good resource is Windows Server® 2003 Books Online. The section titled "Accessing Resources Across Forests" contains the best information around, though it's not SQL Server-specific. It applies to any resource in the forest. Other documents of interest are listed in the "Forest Resources" sidebar.

Renaming A Second Database
Q I have two databases managed by the same instance of SQL Server. Sprocs in db1 rely on data in db2. I'm concerned that if I rename the db2 database to something else, I'm going to have to modify all the db1 sprocs that refer to db2. I thought I'd create a "local linked server" against db2 and then reference the linked server name in all the db1 sprocs. If the name of db2 changes, I can just modify the linked server info. What are the performance implications of utilizing linked server functionality against a local database when I could otherwise access it normally? Should I reconsider this plan altogether?

A Using linked servers in this scenario is completely unnecessary. In SQL Server 2005, simply create a local synonym for each remote table and write a stored procedure that drops and recreates all the synonyms. In SQL Server 2000 do the same thing, except with views.
As long as all of your ad hoc and stored procedure code references synonyms or views, you can change the names of the underlying objects without having to change your application code. It sounds like this is your real goal after all.
Alternatively, as long as you perform all of your database access by stored procedure with no SQL code embedded within the application, it would be relatively easy to do a global replace of the database portion of an object reference throughout all of the procedure code if a database name changes.
To answer your specific question about performance, there are differences in the types of query optimization that may occur for all local tables vs. a mix of local/remote tables with the all local scenario generally providing the largest set of potential optimizations. If you do go down this road of using linked servers, please be aware of one limitation of using "loopback" linked servers. The section "Transaction Considerations With Linked SQL Servers" at msdn.microsoft.com/library/en-us/acdata/ac_8_qd_12_1prm.asp, states:
"The Microsoft OLE DB Provider for SQL Server does not support nested transactions. Therefore, XACT_ABORT should be set to ON for data modification operations inside implicit or explicit transactions and for data modification operations against distributed partitioned views.
"Loopback connections to the same instance of SQL Server are not supported when inside an implicit or explicit transaction or distributed partitioned view."

Storage Bottleneck
Q I perform a great deal of inserting and deleting, and I'm experiencing significant slowdown in insertion when deletion is happening at the same time. My configuration includes 14 logical unit numbers (LUNs) and 37 disks. The log, tempdb, and repldata devices are RAID 1/0. Data & Index are RAID 5. The perfmon log is shown in Figure 1. Does this indicate a storage area network (SAN) issue?
Figure 1 Perfmon Log (Close Figure 1)
The Disk Queue Length_Total reaching a maximum of 477 (average of 124)
% Disk Time_Total reaching a maximum of 2982 (average of 772)
Log Flushes/sec reaching a maximum of 486 (average of 39)
Locks: Average Wait Time (ms) reaching a maximum of 210,676 (average of 1237)
Latches: Total Latch Wait Time (ms) reaching a maximum of 11,298 (average of 6223)
Latches: Average Latch Wait Time (ms) reaching a maximum of 144 (average of 57)
Latches: Latch Waits/sec reaching a maximum of 1499 (average of 162)

A Your log certainly indicates that your bottleneck is in the storage area. There are a few things to consider when examining this data. The reason for RAID 1/0 on logs and tempdb is that they are update-intensive, and RAID 1/0 performs better there. The reason for RAID 5 on the data drives is that data disks are generally much less update-intensive, so you can tolerate the update overhead that RAID 5 imposes in exchange for using fewer physical disks to populate the same volume of storage. You may want to consider RAID 1/0 if your database is very active.
You also need to know that the number of LUNs doesn't tell the whole story in a SAN. The number of spindles backing those LUNs, and what the cache configuration is, are also important. You can have 30 LUNs configured on three physical disks. Whether this is what you're doing, it is another area to evaluate. Adding more physical spindles to the LUNs that have the greatest queue and wait times may help.

Add Identity
Q I have a table that looks like this:
CREATE TABLE [dbo].[TempSystem](
            [TempSystemID] [bigint] NOT NULL,
...
I need to create it to look like this:
CREATE TABLE [dbo].[TempSystem](
            [TempSystemID] [bigint] IDENTITY(1000000,1) NOT NULL,
...
What is the proper data definition language (DDL) to alter the table to add the identity? I can do it from the SQL Server console and it works beautifully, but I need to perform this action on many tables and would like to generate a script to take care of it.
Also, is there any easy way to see the SQL commands that are being submitted from the console? I'm using SQL Server Express.

A There is no slick way to add the identity. You'll need to do the following, which is crude, but effective:
Add a new column, with the IDENTITY property, to your table and set IDENTITY_INSERT ON for your table. Then, copy the values from TempSystemID to your new column. Next, set IDENTITY_INSERT OFF for your table, then drop the TempSystemID column. Rename the newly added column (the one with the IDENTITY property set) to TempSystemID.
To capture the statements issued by management tools, use SQL Server Profiler. Start it running, then execute your commands through Enterprise Manager or Management Studio. When you're finished, review the activity captured within Profiler to see exactly how the action was performed.

Thank you to the following Microsoft IT Pros for Their Technical Expertise: Dean Bethke, David Browne, Russell Christopher, Patrick Conlan, Alan Doby, Robert Dorr, Gops Dwarakanathan, Saravanan E, Keith Elmore, Michael Epprecht, Kevin Farlee, Christoph Felix, John Gose, Jerome Halmans, Chad Hower, John E. Huschka, Mandar Inamdar, Sudheer Maharana, Kirk Nason, Ward Pond, Chris Skorlinski, Brian Steck, John Suknaic, Gary Whitley, Tom Wisnowski, Roger Wolter, and Ning Zhu.

Edited by Nancy Michell


 Hey, Scripting Guy!: Setting Up Outlook E-Mail Signatures
Hey, Scripting Guy!
Setting Up Outlook E-Mail Signatures
The Microsoft Scripting Guys
Download the code for this article: ScriptingGuys2006_10.exe (151KB)

You know, some things are hard to resist, even when we know better. Take carnival games. Everyone knows that these games are often rigged (pac-c.org/Carnies.htm) and yet you just have to try them. After all, rigged or not, how hard can it be to break a balloon with a dart? Land a dime inside a goblet? Toss a basketball into a peach basket? Come on, it's easy! Well, maybe if we put a little more backspin on the ball...
In some ways, Microsoft® Outlook® is the ring-toss of the scripting world. Ring-toss looks ridiculously easy: you just throw a big ring over a relatively tiny little peg. As it turns out, though, some of the pegs are angled just right, making it nearly impossible for a ring to land on them. Likewise, there are lots of things you'd like to script with Outlook that look so easy. Create a new Outlook signature and then make that your default signature for all outgoing e-mail? Come on, how hard could that be? Well, maybe if we put a little more backspin into our script.
As many of you have discovered, this isn't as easy as it seems. Not that people don't try anyway: we Scripting Guys have received scores of sample scripts in which people have tried everything from manipulating the Outlook object model to modifying values in the registry to creating some weird sort of .INI file, all in a futile attempt to create and assign an Outlook signature. Apparently, Outlook signatures are rigged like the carnival games, and you just can't win.
Or can you? Although some carnival games are designed so that you can't win, others actually can be beaten; you just need to know the secret. For example, when throwing a basketball into a peach basket, just remember that the angle of incidence equals the angle of refraction. The same is true of Outlook: when trying to create and assign a new Outlook e-mail signature, just remember that the angle of incidence equals the angle of refraction. Oh, and use Microsoft Word to create and assign the signature.
That's right, Microsoft Word. Don't bother asking us why; you might as well ask us why the angle of incidence equals the angle of refraction. (No, we don't know the answer to that, either.) To tell you the truth, we aren't sure why the Microsoft Word object model includes methods for creating and assigning e-mail signatures while the Microsoft Outlook object model doesn't. But we Scripting Guys learned long ago that it's often better to just accept things as they are and not ask too many questions.
Here's the answer to why you couldn't knock a stuffed cat over by throwing a baseball at it: in some cases, at least, the shelves are too wide to cause the cat to fall off. (In the carnival game it's usually not enough to knock the cat over, you have to knock it clear off the shelf to win.) And there are times when the space between the back wall and the shelf might be too small to allow the cat to fall through. And, of course, you do have to hit the cat with your throw to begin with. For some of us, that's easier said than done (certainly not for us, mind you).
Let's forget stuffed cats and see what we can do with e-mail signatures. To begin with, let's try to create a new e-mail signature. As with the balloon dart toss (hint: surreptitiously heat the tip of the dart with a cigarette lighter before throwing) there's a trick involved here. To create a new e-mail signature you actually need to create a Word document, type in the signature (along with any fancy formatting you might want to add), select the text, and then create the signature. We know, that sounds complicated. But as you're about to see, it's really not all that bad. And much easier than heating the end of a dart with a cigarette lighter without the carny seeing you.
Figure 1 shows the code required for creating a new e-mail signature. Let's see if we can figure out what's going on here. We begin by creating an instance of the Word.Application object, and then setting the Visible property to True. (We don't really need to set the Visible property to True; we do that just so you can watch the excitement unfold onscreen.) We use the Add method to add a new, blank document to our instance of Word, then use this line of code to create an instance of the Word Selection object:
Set objSelection = objWord.Selection
Figure 1 Signature Builder (Close Figure 1)
Set objWord = CreateObject("Word.Application")
objWord.Visible = True
Set objDoc = objWord.Documents.Add()
Set objSelection = objWord.Selection

objSelection.TypeText "Ken Myer"
objSelection.TypeParagraph()
objSelection.TypeText "Fabrikam Corporation"

Set objSelection = objDoc.Range()

Set objEmailOptions = objWord.EmailOptions
Set objSignatureObject = objEmailOptions.EmailSignature
Set objSignatureEntries = objSignatureObject.EmailSignatureEntries

objSignatureEntries.Add "Scripted Entry", objSelection
In this case, the Selection object merely positions the cursor at the beginning of the document, giving us a place to start typing. We can then use these three lines of code to enter the new e-mail signature:
objSelection.TypeText "Ken Myer"
objSelection.TypeParagraph()
objSelection.TypeText "Fabrikam Corporation"
In other words, we want a signature that looks like this:
Ken Myer
Fabrikam Corporation
Granted, it's not very fancy, but it will do for now. And, needless to say, you can change fonts, put things in boldface, add images, add borders—basically do just about anything else you can do to a Word document. For educational purposes we just wanted to keep our example as simple as possible. (Yes, simple, just like the throw-a-ping-pong-ball-in-a-bowl-and-win-a-goldfish game, one of the few carnival games you really can win, mainly because you pay a dollar for the chance to win a goldfish that would cost you 50 cents at the pet store.)
Once the signature is complete, we use the following line of code to create a range encompassing all the text in the document:
Set objSelection = objDoc.Range()
Think of this as being equivalent to pressing Ctrl+A or choosing Select All from the Edit menu.
That brings us to the next few lines of code that create a series of object references:
Set objEmailOptions = objWord.EmailOptions
Set objSignatureObject = _
    objEmailOptions.EmailSignature
Set objSignatureEntries = _
    objSignatureObject.EmailSignatureEntries
Why all this code? Well, the EmailOptions object is a child object of the Word.Application object; in the first line we're simply creating an instance of the EmailOptions object. In turn, the EmailSignature object is a child of EmailOptions, and the EmailSignatureEntries collection is a child of EmailSignature. We actually need to add the new signature to the EmailSignatureEntries collection; it just takes us awhile to work our way down to that collection.
Once we get there, however, it takes only a single line of code in order to create a new e-mail signature named Scripted Entry:
objSignatureEntries.Add _
    "Scripted Entry", objSelection
As you can see, all we do is call the Add method and pass two parameters: the name of the new signature (Scripted Entry) and the object reference for the selected text (objSelection). Just like that you'll have a new e-mail signature within Word and Outlook. We tried this first with Word configured as the default e-mail editor and then without. The results were the same: you get the new signature regardless.
Of course, while that creates a new e-mail signature it doesn't actually assign the new signature to all outgoing messages. To do that you need to modify one (or both) of the following EmailSignature object properties: NewMessageSignature indicates the signature to be tacked on to the end of any new e-mail messages, and ReplyMessageSignature indicates the signature to be tacked on to the end of any e-mail replies.
Here's a sample script that assigns the new signature (Scripted Entry) to both the NewMessageSignature and ReplyMessageSignature properties:
Set objWord = CreateObject("Word.Application")
objWord.Visible = True

Set objEmailOptions = objWord.EmailOptions
Set objSignatureObject = _
    objEmailOptions.EmailSignature

objSignatureObject.NewMessageSignature = _
    "Scripted Entry"
objSignatureObject.ReplyMessageSignature = _
    "Scripted Entry"
It's that easy.
If Outlook is already open these changes are applied to your default profile; this will also be the case if you have only a single Outlook profile. If you have multiple Outlook profiles, however, you'll be prompted to select a profile when the script runs. To bypass that prompt, just have your script open Outlook with the desired profile. For more information go to—where else?—the TechNet Script Center: microsoft.com/technet/scriptcenter/resources/officetips/jun05/tips0614.mspx.
What if you don't want an e-mail signature? Is it possible to set the signature options to None? Of course, just set the value of the two properties to an empty string (""):
Set objWord = CreateObject("Word.Application")
objWord.Visible = True

Set objEmailOptions = objWord.EmailOptions
Set objSignatureObject = _
    objEmailOptions.EmailSignature

objSignatureObject.NewMessageSignature = ""
objSignatureObject.ReplyMessageSignature = ""
As nice as this is, it's still a little like paying a dollar to win a 50-cent goldfish: after all, having to edit a script each time you want to create a signature for a different user isn't that much better than creating that signature by hand. Now that we know how to create and assign an e-mail signature, let's see if we can come up with a script that's a bit more practical. For example, here's a script that:
  • Binds to the Active Directory® user account for the logged-on user, retrieving the user's name, job title, department, company, and phone number.
  • Creates a new instance of Word and a blank Word document. (Note that this time we don't set the Visible property to True—that way Word never appears onscreen while the script runs.)
  • Creates a new e-mail signature (named AD Signature) based on property values retrieved from Active Directory.
  • Assigns the new e-mail signature to both the NewMessageSignature and ReplyMessageSignature properties.
  • Exits Word.
Is that more like it? Figure 2 shows the script. Figure 3 shows what one of these Active Directory-derived signatures might look like. If you're looking for a way to create a corporate-wide e-mail signature, well, this might be the very thing you've been looking for.
Figure 2 Building a Signature from Active Directory (Close Figure 2)
On Error Resume Next

Set objSysInfo = CreateObject("ADSystemInfo")

strUser = objSysInfo.UserName
Set objUser = GetObject("LDAP://" & strUser)

strName = objUser.FullName
strTitle = objUser.Title
strDepartment = objUser.Department
strCompany = objUser.Company
strPhone = objUser.telephoneNumber

Set objWord = CreateObject("Word.Application")

Set objDoc = objWord.Documents.Add()
Set objSelection = objWord.Selection

Set objEmailOptions = objWord.EmailOptions
Set objSignatureObject = objEmailOptions.EmailSignature

Set objSignatureEntries = objSignatureObject.EmailSignatureEntries

objSelection.TypeText strName & ", " & strTitle
objSelection.TypeParagraph()
objSelection.TypeText strDepartment
objSelection.TypeParagraph()
objSelection.TypeText strCompany
objSelection.TypeParagraph()
objSelection.TypeText strPhone

Set objSelection = objDoc.Range()

objSignatureEntries.Add "AD Signature", objSelection
objSignatureObject.NewMessageSignature = "AD Signature"
objSignatureObject.ReplyMessageSignature = "AD Signature"

objDoc.Saved = True
objWord.Quit
Figure 3 Deriving the Signature from Active Directory
Figure 3 Deriving the Signature from Active Directory
That was easy, wasn't it? Now, you look pretty strong and healthy to us—we bet it would be a cinch for you to knock down these empty milk cans with a baseball. What do you say? Just one dollar a throw.

The Microsoft Scripting Guys work for—well, are employed by—Microsoft. When not playing/coaching/watching baseball (and various other activities) they run the TechNet Script Center. Check it out at www.scriptingguys.com.


 How IT Works: Windows Rights Management Services
How IT Works
Windows Rights Management Services
Randy Muller

In today's competitive business environment, information is a crucial commodity that requires proper security. With the numbers and types of external storage media available, it is even more critical for you to protect your data from accidental or intentional disclosure. There are an increasing number of
government initiatives to protect information that might be accessed or stored by companies. The Health Insurance Portability and Accountability (HIPAA) Act was passed to ensure the protection of personal health information. The Gramm-Leach-Bliley Act was passed to provide more security for consumers' financial privacy. In addition to these government actions, corporations are being held increasingly responsible for the safeguarding of information they hold on their own employees as well as on their customers. The loss or compromise of this data could result in expensive legal action and loss of confidence by consumers.
Windows® Rights Management Services (RMS) can help. RMS can provide the protection for digital information required by your organization or by government regulation. RMS is fully compliant with the Federal Information Processing Standard (FIPS) 140-1., and therefore transactions from private business can be conducted with the government.
Compromise of data can occur through malicious intent or unintentional actions. The ease with which data can be copied and moved to external storage media (CDs, flash drives, external hard drives, and so on) makes it difficult to protect. In many cases, accepted security practices are not able to adequately protect this information any longer. A person may have access to a document, but you may not want this person to be able to save it to the external storage devices. Controlling how a document can be handled is a key element of RMS, as is protecting it while crossing the network.
RMS Resources
  • For an overview of Windows RMS technologies for Windows Server 2003, see:
    microsoft.com/windowsserver2003/technologies/rightsmgmt
  • A Technical Overview of Windows Rights Management Services whitepaper is available from:
    microsoft.com/windowsserver2003/techinfo/overview/rmenterprisewp.mspx
  • You can try out RMS without the fuss of setting up your own lab by using the TechNet Rights Management Services Virtual Lab at:
    microsoft.com/technet/traincert/virtuallab/rms.mspx
  • Licensing and download information for Windows RMS is available from:
    microsoft.com/windowsserver2003/techinfo/overview/rmsoverview.mspx

Today it's common to view security as a means of providing protection for networks and the data contained within the network. The first line of defense for networks is the perimeter, which is protected through firewalls, allowing only traffic you approve to enter or leave the network. While a firewall will allow authorized traffic to exit, it doesn't know if what is contained in the authorized traffic is of a compromising nature and should be prevented from leaving the network.
The next layer in network security is the use of NTFS permissions and Access Control Lists (ACLs). Together these are used to limit who can access what resources and, to a certain extent, what they can do with these resources. But once a user has accessed a file, she can save it to another location, thereby bypassing the security measures enforced with permissions and ACLs.
RMS can control what a user can do with a document after it has been accessed. This control will last regardless of where the user might store the document and you can even have time limits associated with the document so that, after a specified period of time, the document is no longer accessible.
When trying to stem the flow of potentially compromising information, you have to take into account e-mail messages. While you may mark e-mail messages "confidential," there haven't been any built-in mechanisms to restrict the forwarding of these messages. Sensitive information that is contained within e-mail is just as susceptible to dissemination as it is within a document. RMS allows for the control of e-mail messages and will prevent the inadvertent forwarding of messages that may contain sensitive data.

RMS Components
There are three essential components that comprise RMS: the RMS server, the RMS client, and RMS-enabled applications and SDKs. The RMS server is responsible for the proper certification of trusted entities, provides licensing of content that is rights-protected, and enrolls any users and servers. It also serves as the administrative point for RMS (see Figure 1).
Figure 1 Rights Management Components
Figure 1 Rights Management Components
The RMS client must be installed to work with a RMS server. The client software allows applications that are RMS-enabled to communicate with a server. The client requests licenses to consume (access and use) content and allows for the publishing of content.
The final element is the actual application that can use RMS. Applications can be made RMS-aware by using the Rights Management Services SDK.
RMS applies persistent usage policies to ensure consistent and reliable control over content. A trusted entity (an application, computer, group, or user) has been granted permission to use RMS. Since it is trusted, that entity is allowed to make use of RMS. Trusted entities can apply usage rights and conditions to content. They can specify what rights, permissions and actions are applied to a document. A trusted entity can specify that a document cannot be printed or saved, that an e-mail message cannot be forwarded, and even when access to the message will expire, thus eliminating further maintenance of the resource.
RMS-enabled resources are protected with encryption. A user wanting to access a resource must first be validated. This validation determines if the user is allowed access to the resource and, if so, what kind of usage rights are permitted. This usage requirement is maintained for the document.

How RMS Works
RMS is involved in three areas to ensure proper utilization: the actual creation of rights-protected resources, licensing and distributing these rights-protected resources, and decryption and usage of rights-protected resources. A trusted entity (one that has been granted access to make use of RMS) can create resources that are protected. When a resource has been protected, an XrML certificate identifies who is allowed access and what usage requirements are imposed on the resource.
The RMS server will issue a publishing license that delineates who is allowed to access the resource. Once this is done, the protected resource can be sent. When a trusted entity, say a user, wants to access a resource, the user will be validated by the RMS server which holds the public key for the encrypted resource and will issue a use license to the user. This use license specifies how the resource can be used and actions that can be taken with it. So these licenses are employed as the actual control mechanisms. The publishing license is created when a document is RMS-enabled and has been encrypted. The use license is required when a document is consumed.

Encrypting and Securing Content
Exactly how does RMS direct the process whereby control is maintained over documents, e-mail messages, and applications? RMS employs Public Key Infrastructure (PKI) as the basis for controlling access to documents. PKI uses asymmetric encryption in which two keys are used for the encryption/decryption process: one public key and one private key. In a typical PKI environment, a user will encrypt a document that can only be unencrypted by the recipient. In an RMS environment, the document is encrypted by the user and is maintained by the server. Any requests to access the document are made to the server, which will validate the request and its purpose, to include printing, forwarding, and even the saving a document.
The keystone of RMS is in using a standardized rights expression language (REL) to provide a common framework for interoperability. The language that is used to provide this commonality is XrML version 1.2.1. The XrML language can be used to apply rights and security to digital information in the form of a license. This XrML license is attached to the resource and is used to specify the permissions and usage applied to it.
XrML provides a universal method for securely specifying and managing rights and conditions associated with all kinds of resources including digital content and services. It is fully compliant with XML namespaces using XML schema technology.
Now let's see how the process of using RMS on a document works. The IT hero in this scenario has been tasked with coming up with the raises and salary information for the next fiscal year. Let's assume that he has the appropriate RMS client software installed on his machine. He creates a spreadsheet containing the new financial figures (something any company would want to keep under tight control!) and wants to apply RMS security to this confidential document.
Our hero uses an RMS-enabled application—in this case Microsoft Office 2003—to contact the RMS server through the application to apply for a publishing license for this document. The RMS server issues the publishing license which is applied to the document. The document has been encrypted, has a publishing license attached, and has suitable usage restrictions applied due to this process. The document has now been made more secure than simply using NTFS permissions and ACLs.
The IT guy sends the document to his manager for final review and comments. Before the manager can access the document, she will need to contact an RMS server to receive a use license for the document. The request for a use license is performed through the RMS-enabled application and, once complete, allows the manager to access the document according to the usage restrictions that were applied by the author of the document (see Figure 2).
Figure 2 Using RMS to Protect Documents and E-Mail
Figure 2 Using RMS to Protect Documents and E-Mail

Requirements and Installation
The base requirements for RMS are Active Directory®, IIS 6.0, the .NET Framework and ASP.NET, and Microsoft® Message Queue Server (MSMQ). You'll also need RMS server software (RMS will only run on Windows Server 2003), RMS client software and RMS-aware applications, as well as a SQL Server™ 2000 SP3 (or greater) database.
Once you have the requisite software installed on the RMS server, you will need to provision the server. Provisioning simply means getting the server to communicate and respond as an RMS server on your network. The first RMS server you install will need to contact a Microsoft RMS enrollment server and request a Server Licensor Certificate (SLC). Once the SLC has been received from the enrollment server and installed, you will need to register the RMS Service Connection Point (SCP). RMS-enabled applications search SCP when trying to contact an RMS server. The actual SCP is stored as an entry in the Active Directory Configuration container.
The next step after configuring the RMS server is to configure the client machines for use with RMS. There are three distinct steps associated with configuring client machines for RMS, a process called Client Machine Activation. You need to install the RMS client software package on each machine; then you will go through the machine activation process that will create a lockbox (which holds the cryptographic keys) on each client machine. The final step involves getting a user certificate, which is used to generate an association between the user and a particular computer.
To take the RMS process a step further, you can create and deploy a Rights Policy Template to ensure that users who request a publish license receive a standard template that matches your requirements. This template will have the permissions listed that you can choose to apply.
RMS has a valuable place in a network where securing information from improper dissemination is vital. While there is a certain amount of effort required to install and provision an RMS solution, the end result more than justifies that effort.

Randy Muller, MCT, MCSE, MCSA, MCDST, teaches a variety of networking, security, and other computer classes. He is a former Army Signal Corp Officer and has been teaching since 2000. You can contact Randy at randy@randymuller.org.


 Field Notes: Where Did the Net Go?
Field Notes
Where Did the Net Go?
R'ykandar Korra'ti

We first noticed something was wrong when our network fell over and died. No, wait; that makes it sound way more catastrophic than it actually was. Let me back up and start again.
I'm one of the operators of a small, co-op ISP in the Seattle area. We have a heterogeneous workstation environment: mostly Windows® and Linux on the server side, a variety of Macintosh OS X boxes, some Windows clients, and even one ancient Amiga 4000/040 that doesn't get turned on very often. I'm sure you can imagine the kinds of joy this brings to our lives.
One day, while sitting quietly reading comic books studying some useful manuals, I heard the words I had come to dread: "Dara! The net's down!" OK, I thought to myself, what does that actually mean? For a change, our network actually was down—or, at least, our outbound connectivity was. The LAN was fine and we could talk to our router at the UDP and ICMP levels. From the router, we could talk to the rest of the world. But the router itself no longer passed TCP packets. Stranger still, the interfaces for its two NICs both reported normal status and a perfectly reasonable number of packet errors. I stopped and restarted the interface driver on the internal NIC, saw that everything came right back up, and decided to investigate further—but later.
Six hours passed and it happened again, just as we were leaving for the night. I crawled through the router logs, found nothing interesting at all, and reset the card again, not having time for anything else.
We didn't even make it through the night. There still wasn't a hint in the server logs—it didn't even notice that the card went down, which made me remember something. Aha! I thought, with words that might best be described as "famous" and "last," I've seen this behavior before. The onboard TCP/IP checksum hardware has gone pear shaped, and it's time for a new card! For this, I was prepared; the machine was down, re-NICed, and back up in 15 minutes. Back to bed I went.
Come the morning and guess what—we're down again.
Sitting in front of the primary server cluster and installing a different network monitor to see whether a new tool might help, I noticed our central switching hub light up. And by "light up," I don't mean "ah, someone's streaming the new Doctor Who," I mean Times Square at New Year's lighting up—but only for a moment, and then things returned to normal. I swapped to the backup switch and waited for it to happen again—and when it did, we fell off the net.
Our backup switch has poor indicators on the front panel, which is why it became our backup switch to start with—so I swapped in an older, slower unit with a good display set. When it lit up the next time, I spotted the culprit: one of our Web servers. I didn't find anything out of the ordinary until I noticed the brief appearance of an anomalous script in the task list, before it vanished and reappeared under another name.
After a little research and a lot of network sniffing—my goodness, those are a lot of SYN packets, and that's a very interesting login to a Japanese IRC server—I found we'd been hit by someone's exploit of a newly discovered PHP4 vulnerability for which there was not yet even a patch. Our Web server had become a bot in someone's round of Mixi wars—or had tried.
The funny part was that our router apparently had wanted no part of that game. Every time it got hit with the malformed SYN flood, the "experimental" (read: flaky) NIC driver on our router simply decided to go off on its own and sulk. This meant that the DDOS attack, at least on our end, was failing to DDOS its objective—but instead was successfully DDOSing itself. Like a poor marksman, it just. Kept. Missing. The Target.
A few days later—after a wipe and restore of the Web server, with PHP offline—we got a patch and were back to normal. But we kept the router just the same. Apparently, it's smarter than we are, and I'm definitely not going to mess with that.
Besides, if we tried anything—it might retaliate

R'ykandar Korra'ti, postmaster for a small co-op ISP, lives near Seattle with her partner Anna. Having previously shipped mail products at Microsoft, she is now looking at grad school in a CS-related field so esoteric it doesn't really have a name. Potential faculty advisors can reach her at darako@murkworks.net.


 Windows Confidential: Compatibility Constraints
Windows Confidential
Compatibility Constraints
Raymond Chen

WHEN YOU THINK of a new version of Windows® maintaining backwards compatibility with applications designed for previous versions of Windows, you tend to think of the explicit programming contract: things like the behavior of window messages, the semantics of the heap, and that GetTextMetrics will get the text metrics. But that's the easy part of maintaining compatibility, since those rules are actually written down. The hard part is maintaining compatibility with the things that merely happen. Some behaviors are not contractual, and often they aren't even designed—they just happen by circumstance.
The Windows Shell team recently came across an application that wanted to map a network drive when the user clicked the corresponding toolbar button. But to do this, the application's developers didn't call the WNetConnectionDialog1 function. Instead, they decided to create a context menu on the My Computer object and then invoke the Map Network Drive command from the menu, which in turn calls the WNetConnectionDialog1 function. In other words, the application's developers avoided the documented function and relied on a middle man to do this work for them. This middle man, I should point out, is under no contractual obligation to have a Map Network Drive command in the future.
But it gets better. The developers didn't write the program to invoke the Map Network Drive command. Rather, they observed that the command was always the third item on the menu and, therefore, simply had the application invoke item three.
But what about context menu extensions, such as Add to Playlist? Since all of these extensions are typically added near the top of the menu, the developers adjusted their algorithm: instead of executing the third item on the menu, they executed the fifth-from-the-bottom menu item.
But wait, it gets better still. They also noticed that the Map Network Drive menu option had a different position on Windows 2000 than on Windows XP. So before invoking the command, the application checks which operating system it is running on and adjusts the computations accordingly.
For Windows Vista™, we did some work to clean up the context menus, removing commands that are redundant and moving rarely used commands to the extended menu. (In case you're wondering, you reach the extended menu by holding the shift key when right-clicking the item.) After we cleaned up the menus, this program (not surprisingly) ended up executing some other command instead of the one it wanted, thanks to the reordering of menu items.
Some people may argue that Microsoft should leave the program broken as a form of punishment. But then who would really be getting punished? The customers who were using this program would be stuck until the vendor produces an upgrade that fixes the problem. They would lose the use of the program for a few months ("sorry, but you missed the contract bid deadline while you were out of commission") and then they may even have to shell out some cash for an upgrade that fixes the problem. The customer suffers lost productivity; the vendor gets upgrade revenue.
Looks like the vendor wins and the customer loses. (Not to mention that many users would probably blame the new version of Windows for breaking a program that ran perfectly fine in the first place.)
What's more, this program is a competitor to a Microsoft product. The conspiracy theorists would once again be apoplectic with glee, posting articles to blogs about how "Windows Vista intentionally cripples competing software in order to give Microsoft an unfair advantage." In the world of Windows, it sometimes feels like everything is a compatibility constraint

Raymond Chen's Web site, blogs.msdn.com/oldnewthing, deals with Windows history and Win32 programming. He also loves what you've done with your hair.