July 2008 July 2008
IIS: Going Live with IIS 7.0
The release of Internet Information Services 7.0 sets new standards, offers fundamental improvements, and brings new capabilities for consolidating Web environments. Explore the most important enhancements and get an in-depth guide to testing, staging, and migrating your Web apps to IIS 7.0. Fergus Strachan
Communications and Collaboration: Planning Your Migration to Unified Messaging
Moving from a legacy voicemail system to a unified messaging platform can be done fairly easily and with little disruption to your end users—as long as you have a solid plan in place. Here’s what you need to know to plan your migration, test your new systems, and implement unified messaging in your organization. Jeff Goodwin
Communications and Collaboration: How Voice Powers OCS 2007
Microsoft Office Communication Server brings important changes to enterprise telephony. Examine how voice calls are made layer by layer, learn how calls can be routed to various endpoints, and explore the importance of conversations in OCS communications. Rajesh Ramanathan
Communications and Collaboration: Create Powerful Connections with Groove and SharePoint
Microsoft Office Groove and SharePoint Server both provide powerful collaboration capabilities. Compare how they differ from and complement one another, and find out how you can integrate Groove and SharePoint to provide an easy and flexible collaboration solution that will support users at different locations. Alan Maddison
Windows Administration: Introducing Windows Server 2008 Failover Clustering
Users have complained for years that clustering in Windows Server is too complicated. With the release of Windows Server 2008, clustering received a complete facelift. Explore some of the most significant advancements found in the new Failover Clustering, and see how this new implementation makes clusters much easier to set up and maintain. Chuck Timon
Columns
From the Editor: Print is Dead: | Letters: Readers Speak Out: | Toolbox: New Products for IT Pros:
Sustainable Computing: Is It Time to Turn Off Your Servers?: When you're not using your PC, you shut it down. So why not do the same with your servers? Dave Ohara explores the strategy of powering down idle servers for significant reductions in power consumption.Dave Ohara | Exchange Queue & A: Load Balancing, Edge Transport, and More: Have questions about Exchange? In this installment, we answer questions about using Exchange with SharePoint, working with standby continuous replication, installing Exchange 2007 Mailbox servers on Windows Server 2008, and more.Henrik Walther | Utility Spotlight: Microsoft SharedView: Looking for a way to quickly collaborate with someone located in another city? Take a look at out how Microsoft SharedView, a free utility, lets you easily share documents and screens with members of a small group, even across physical boundaries.Greg Steen
Inside SharePoint: Integrating Office Applications: Find out how Microsoft Office applications integrate and communicate with SharePoint, and explore ways you can integrate non-Microsoft applications with SharePoint using the same principles.Pav Cherny | Windows PowerShell: The Drop Box: Don Jones discusses filtering functions, the pipeline, custom objects, and how ping-pong balls hold the secret to understanding Windows PowerShell.Don Jones | Hey, Scripting Guy!: The Adrenaline Rush: The Scripting Guys discuss adrenaline-filled scripting and show you how the new boot configuration data store in Windows Vista and Windows Server 2008 can be accessed using scripts.The Microsoft Scripting Guys
The Desktop Files: Network-Booting Windows: Explore the history of Remote Installation Services (RIS) and other PXE-related technologies used at Microsoft.Wes Miller | Security Watch: Passwords and Credit Cards, Part 1: The first installment in a three part series looking at how common security-related advice and common practices are outdated and actually hinder security efforts.Jesper M. Johansson | Field Notes: Ensure Election Accuracy: Examine vulnerabilities in today's U.S. election system and see how cryptography can be used to create fully verifiable election results.Josh Benaloh
Windows Confidential: History Taking Up Space: Microsoft Bob is a lot more prevalent than you might think. Raymond Chen explains how Microsoft Bob made its way into millions of homes.Raymond Chen


IIS
Going Live with IIS 7.0
Fergus Strachan
At a Glance:
  • Deploy IIS 7.0 in a Web farm environment
  • Security and performance enhancements
  • Migrate ASP.NET Web apps from IIS 6.0 to IIS 7.0
  • Migrate PHP Web apps to IIS 7.0

Code download available at: IIS72008_07.exe (861 KB)
Contents
Ready, Set, Test
My Test Setup
Important Enhancements for IT Admins
IIS Architecture
Integrated and Classic Modes
Modules and Functionality
Web Application Migration
Wrapping Up
The IIS team at Microsoft says Internet Information Services (IIS) 7.0 is the biggest release in the history of IIS. This version sets new standards, offers fundamental improvements, and
brings new capabilities for consolidating Web environments. IIS 7.0, which is included with Windows Server® 2008 and Windows Vista® SP1, already powers Microsoft.com, and several Web hosting companies have already started to provide IIS 7.0 hosting to Web designers and developers who want to port their existing Web apps to the new Web server platform.
In this article, I will explore key IIS 7.0 enhancements that are most important for IT professionals, and then I'll go into detail about migrating ASP.NET and PHP Web applications. But first I want to outline a test lab that resembles a basic production environment.
This is an important task. Before you deploy IIS 7.0 to your production servers, you need to spend time doing thorough testing in a lab environment to ensure your Web apps run smoothly on the new Web server.

Ready, Set, Test
My test lab includes systems running Windows Server 2003 and IIS 6.0, which host ASP.NET applications, as well as servers running the Ubuntu 7.10 Linux distribution and Apache HTTP Server 2.2, which host PHP-based Web applications. My ultimate goal is to deploy Windows Server 2008 on staging and production systems and then transition complex Web applications in order to replace IIS 6.0 and Apache instances with IIS 7.0.
I have two key Web apps: DotNetNuke 4.7 and osCommerce 3.0a4. DotNetNuke is a Web app framework based on ASP.NET 2.0 and Microsoft® SQL Server®. The other application, osCommerce, is the latest alpha version of a full-featured e-commerce solution built on PHP and MySQL. So let's put these advanced applications on IIS 7.0!
I want to be clear that it is not my intention to compare software versions, products, or platforms. There are, however, a number of advantages to standardizing on Windows Server 2008 and IIS 7.0 that I should point out. Namely, administrative complexity is reduced and you can minimize the overall number of Web servers you are running.
Figure 1 provides an overview of the test lab I am using for this article. If you want to follow my explanations in your own test environment, you can find links to required software components as well as detailed, step-by-step installation instructions in the companion material available from the Code Downloads section of the TechNet Magazine Web site at .com.
Figure 1 The test environment for rolling out IIS 7.0
Note that around the time I was wrapping up this article, Microsoft released a command-line tool (MSDeploy.exe) to support customers deploying, synchronizing, and migrating Web applications to IIS 6.0 and 7.0. I recommend that you check out this tool. Detailed information can be found on the Microsoft Web Deployment team blog at go.microsoft.com/fwlink/?LinkId=118272.

My Test Setup
At the time of this writing, Windows Server 2008 was still in a pre-release, so I therefore did not replace Windows Server 2003 on domain controller or database servers. With the official release of Windows Server 2008, you may want to consider migrating your Active Directory® infrastructure as well. Migrating SQL Server® 2005 and MySQL databases to SQL Server 2008 is also beyond the scope of this article.
I deployed SQL Server 2008 on my staging server mainly because it takes less effort than installing SQL Server 2005 with Service Pack 2. DotNetNuke had no problems running with a SQL Server 2008 database. Installing MySQL 5.0 on Windows Server 2008 was also straightforward with no complications.
While IIS 7.0 is available on Server Core, I did not use a Server Core installation due to certain requirements for my testing. My staging server required a full installation because this was my primary test system. In addition, the Microsoft .NET Framework is not available on Server Core.
PHP runs fine on Server Core, but my goal is to consolidate ASP.NET and PHP applications, so Server Core simply isn't an option. Until the .NET Framework is supported on Server Core, you'll need to perform a full installation to support ASP.NET apps. For detailed step-by-step guidelines to install the test environment, check out the 01_install_testlab.htm file in the companion material.
I opted to perform a clean install of Windows Server 2008 (as opposed to upgrading existing servers). Among other things, the clean installation makes it easier to implement a staged migration scenario like the one in Figure 2. The underlying strategy is to keep the existing Web farms running until all relevant Web app components have been successfully tested on the staging server and moved to the new IIS 7.0 Web farm.
Figure 2 Staging the switch to IIS 7.0
You can move all existing Web applications at once or gradually, depending on the complexity of your environment. For each Web application or site, after performing a final acceptance test on the new Web farm, you can make the switch by changing relevant DNS records to direct browsers to the new IIS 7.0 Web farm. This keeps downtime and interruptions to a minimum.

Important Enhancements for IT Admins
MSDN® Magazine has an excellent article by Mike Volodarsky titled "Explore the Web Server for Windows Vista and Beyond" (available at msdn2.microsoft.com/magazine/cc163453.aspx), which quickly brings you up to speed on improvements found in IIS 7.0.
Another good read is a blog post from the Microsoft.com team, titled "The Tasty Morsels Found in Dogfood" (available at go.microsoft.com/fwlink/?LinkId=117436). The post summarizes the team members' initial IIS 7.0 experiences. In a nutshell, they ranked the top improvements as:
  1. Simplified setup.
  2. Great compatibility.
  3. No more metabase.
  4. Centralized configuration is through applicationHost.config file placed on a UNC share.
  5. Delegated configuration by means of web.config files in application directories
  6. Improved management tools.
  7. Failed-request tracing.
  8. Request filtering without the need for URLScan.
  9. Simplified content synchronization availability through UNC shares.
10.Output caching of dynamic content.
Simplified setup is definitely on my list. In its blog post, the Microsoft.com team demonstrates how you can deploy all the features of IIS 7.0 (or, of course, just the features you want) with a single command line. I eagerly incorporated this approach into my test lab setup instructions, with the command line shown in Figure 3.
Admittedly, this command is rather long. (If you want to copy this command, I recommend you copy and paste it from the TechNet Magazine Web site rather than retype it by hand.) Although this command puts all available modules on the IIS 7.0 computer, it doesn't include PHP. IIS 7.0 doesn't include PHP, and the notion of downloading and installing Debian packages over the Internet is a foreign concept for the Windows® Package Manager (pkgmgr.exe). Enter the Windows Automated Installation Kit (AIK).
Figure 3 Deploying IIS 7.0 features with a single command line
start /w pkgmgr.exe /iu:IIS-WebServerRole;IIS-WebServer;IIS-CommonHttpFeatures;IIS-StaticContent;IIS-DefaultDocument;IIS-DirectoryBrowsing;IIS-HttpErrors;IIS-HttpRedirect;IIS-ApplicationDevelopment;IIS-ASPNET;IIS-NetFxExtensibility;IIS-ASP;IIS-CGI;IIS-ISAPIExtensions;IIS-ISAPIFilter;IIS-ServerSideIncludes;IIS-HealthAndDiagnostics;IIS-HttpLogging;IIS-LoggingLibraries;IIS-RequestMonitor;IIS-HttpTracing;IIS-CustomLogging;IIS-ODBCLogging;IIS-Security;IIS-BasicAuthentication;IIS-WindowsAuthentication;IIS-DigestAuthentication;IIS-ClientCertificateMappingAuthentication;IIS-IISCertificateMappingAuthentication;IIS-URLAuthorization;IIS-RequestFiltering;IIS-IPSecurity;IIS-Performance;IIS-HttpCompressionStatic;IIS-HttpCompressionDynamic;IIS-WebServerManagementTools;IIS-ManagementConsole;IIS-ManagementScriptingTools;IIS-ManagementService;IIS-IIS6ManagementCompatibility;IIS-Metabase;IIS-WMICompatibility;IIS-LegacyScripts;IIS-LegacySnapIn;IIS-FTPPublishingService;IIS-FTPServer;IIS-FTPManagement;WAS-WindowsActivationService;WAS-ProcessModel;WAS-NetFxEnvironment;WAS-ConfigurationAPI

# Command provided by the Microsoft.com team (blogs.technet.com/mscom).
With AIK, you can create a custom installation DVD for Windows Server 2008 that includes IIS 7.0 and PHP 5. You can also include MySQL, Web application files, and any other components that are needed in your deployment. All the components can be part of your Windows Server 2008 Setup, allowing you to apply customizations consistently on all your Web servers. There's no need for command lines or configuration file editing.
You can even create a custom DVD for fully unattended installations. The AIK includes documentation and tools to create an AutoUnattend.xml file, which you then place in the DVD's root folder. The Custom Image Deployment.htm file in the companion material offers step-by-step instructions.
Many administrators also agree that compatibility is a key improvement. Going in, I expected to encounter a few compatibility issues with DotNetNuke and osCommerce, but the transition to IIS 7.0 was rather painless, despite the fact that both of these Web applications include advanced features, such as forms authentication and search-engine-friendly URLs.
With DotNetNuke, I did not encounter any issues until I moved to a complex Web farm scenario with UNC content sharing on a 64-bit platform. The issue, however, was fairly minor. Ultimately, this improved compatibility means less time is required to get your Web applications running on IIS 7.0.
If you happen to encounter compatibility issues with your Web applications, the built-in diagnostics and tracing support will quickly become one of your favorite new features. The detailed diagnostics information is very meaningful, and the suggested solution paths are useful and actually work.
Figure 4 shows the diagnostics information you get when running DotNetNuke 4.7 on IIS 7.0 in integrated mode. In this example, there are three options (shown in the "Things you can try" section). Changing the application to support integrated mode is probably the best choice for DotNetNuke developers. Ignoring the error is probably a bad idea. I like the third choice, enabling classic mode by switching the Web app to the Classic .NET AppPool, because I want to run DotNetNuke 4.7 unmodified on IIS 7.0.
Figure 4 Diagnostics information for DotNetNuke running in integrated mode

IIS Architecture
You might be wondering what these integrated and classic modes are and why they affect the ASP.NET application (DotNetNuke) but not the PHP application (osCommerce). To better understand this, you must first take a look at the architecture of IIS 7.0. Previous versions integrate the ASP.NET runtime into the core Web server, primarily through an ISAPI extension (aspnet_isapi.dll). IIS 7.0, on the other hand, allows ASP.NET modules to plug directly into the core Web server through a global-level HTTP module called ManagedEngine, which loads the ASP.NET Support DLL (webengine.dll) into the IIS 7.0 request processing pipeline.
Native modules use the IIS Web Server Core API to register for specific events in the pipeline, such as BeginRequest, AuthenticateRequest, AuthorizeRequest, and ExecuteRequestHandler. ManagedEngine provides the necessary support to integrate ASP.NET modules into the request pipeline as well. With this new architecture, IIS 7.0 can invoke native and ASP.NET modules at any stage during request processing regardless of the requested content type.
As an example, let us consider ASP.NET user authentication. In IIS 6.0, it is possible for an ASP.NET-based HTTP module to register for OnAuthenticate events (such as FormsAuthentication.OnAuthenticate and WindowsAuthentication.OnAuthenticate) to set the identity of the user in the current HttpContext. This works great inside the ASP.NET runtime, but you cannot use this ASP.NET code to protect resources exposed through a PHP-based Web application.
According to its scriptmap configuration, IIS 6.0 forwards requests for ASP.NET content types to aspnet_isapi.dll, but it doesn't forward requests for PHP content types to the ISAPI extension for ASP.NET. After all, ASP.NET doesn't process PHP code, but it also means that the ASP.NET authentication code does not run if a PHP page is requested. Thus, with IIS 6.0, you need to duplicate the authentication logic because PHP applications must implement their own authentication mechanisms.
IIS 7.0 uses an event-driven processing model that supports mixing and matching individual modules in the request pipeline. Among other components, IIS 7.0 comes with managed WindowsAuthentication and FormsAuthentication modules that raise OnAuthenticate events when the request pipeline fires the AuthenticateRequest event. You can now have a native module, such as RequestFilteringModule or IpRestrictionModule, handle the BeginRequest event to reject critical requests as early as possible, then run custom ASP.NET authentication code at the AuthenticateRequest event and have the FastCgiModule run the PHP scripting engine (php-cgi.exe) at the ExecuteRequestHandler event to process PHP pages.
The result of this integrated architecture is that Web developers don't have to duplicate code to implement common business logic. You can use custom ASP.NET authentication code to protect all your IIS resources including PHP applications. And you can perform other pre- and post-processing tasks through ASP.NET modules, such as URL rewriting, custom tracing, and error logging.

Integrated and Classic Modes
The side effect caused by this new architecture is that existing ASP.NET applications might require code and configuration changes that should be performed by application developers to ensure that the changes don't lead to module conflicts in the request pipeline. But if you can't immediately port an existing Web application, you can enable classic mode in the app pool that the worker process uses to run the Web application. IIS 7.0 does not load ManagedEngine into classic-mode worker processes, which implies in this case that the request pipeline cannot resolve or invoke ASP.NET modules. Instead, IIS 7.0 activates a number of ISAPI handlers, which are based on IsapiModule (aspnet_isapi.dll), to process ASP.NET content types similar to IIS 6.0. You can see this if you analyze the handlers section under system.webServer in the applicationHost.config file, which you can find by default in the %WinDir%\System32\InetSrv\Config folder.
Search for the string aspx as an example and you will find one entry pointing to PageHandlerFactory-Integrated (loaded into integrated-mode application pool worker processes according to the setting preCondition="integratedMode") and other entries pointing to PageHandlerFactory-ISAPI-2.0 and PageHandlerFactory-ISAPI-2.0-64 (loaded into classic-mode application pool worker processes according to the setting preCondition="classicMode").
Because the pipeline mode is set at the application pool level, IIS 7.0 can run Web applications in integrated and classic mode simultaneously. And the worker processes only need to run in different application pools but can be hosted on the same server.
Keep in mind that integrated mode and classic mode only affect how IIS 7.0 integrates ASP.NET into the request pipeline. These pipeline modes do not affect PHP applications directly. The FastCgiModule and all other native modules load without pipeline-mode preconditions in both integrated mode and classic mode. FastCGI is the preferred interface to run the PHP scripting engine on IIS 7.0. Instead of using FastCGI, you can also create a handler mapping for the PHP scripting engine through CGI, or you can use the IsapiModule in conjunction with PHP-ISAPI (php4isapi.dll).
I, however, think these IIS 6.0-style configuration alternatives are outdated now that FastCGI offers substantially improved performance and stability. CGI is slower since IIS must initialize a new CGI process for every HTTP request while FastCGI reuses CGI processes for multiple requests. ISAPI requires a thread-safe PHP build, which involves more overhead than running non-thread-safe PHP.
Perhaps even more important, not all PHP extensions are available in a thread-safe version, and running non-thread-safe extensions with ISAPI is a bad idea because it can cause stability issues on the server. FastCGI, on the other hand, is a single-concurrency environment like CGI. Used to run non-thread-safe PHP, FastCGI is stable and fast, and it is clearly the way to go with PHP 5 on IIS 7.0. It is also available on IIS 6.0 if you are not yet ready for a transition to IIS 7.0.

Modules and Functionality
IIS 7.0 features a highly modularized architecture—or, as the IIS developers put it, a componentized feature set. This is good news for Web administrators who want to keep the memory footprint and attack surface of IIS 7.0 as small as possible. By enabling different modules or perhaps even replacing standard modules with custom modules, you can gain complete control over the IIS 7.0 features on your Web servers.
To run ASP.NET in integrated or classic mode, as well as PHP applications on the same server, you must install at a minimum the Web Server role and the core modules to support request processing, server configuration, ASP.NET, ISAPI, and CGI (which includes the FastCGI module). This can be done with the following command line:
start /w pkgmgr /iu:IIS-WebServerRole;
WAS-WindowsActivationService;WAS-ProcessModel;
WAS-ConfigurationAPI;IIS-ASPNET;
IIS-NetFxExtensibility;WAS-NetFxEnvironment;
IIS-ISAPIExtensions;IIS-ISAPIFilter;IIS-CGI
I generally prefer to install IIS 7.0 with all available options on my staging server to ensure that all components and administrative tools are available to get my Web applications up and running quickly. My strategy is to make sure things are working, and then I narrow down the configuration by uninstalling role services and disabling modules. When a Web app starts to act up, I add back role services or modules I just removed.
You also have the choice of using Package Manager (pkgmgr.exe) or the IIS 7.0 command line tool (appcmd.exe) or editing the globalModules section in the applicationHost.config file directly, but I find the graphical tools very convenient. Keep in mind that some modules, such as RequestFilteringModule and StaticCompressionModule, are quite useful even though they are—strictly speaking—not needed for your Web applications to run. If you uninstall a module that is required for an application pool, you will receive an HTTP error when accessing your Web application, as illustrated in Figure 5.
Figure 5 An ASP.NET application won't run in classic mode because IsapiModule is missing
Note: as a best practice, make sure the same set of modules is installed and enabled on all of your IIS 7.0 servers. To verify the installed components, go to HKEY_LOCAL_MACHINE\Software\Microsoft\InetStp\Components\ and check the registry keys. A REG_DWORD value of 0000 0001 for a registry key, such as for NetFxEnvironment or CGI, indicates that the corresponding component is installed.

Web Application Migration
Enough theory, now it's time to roll up my sleeves and move some Web applications to IIS 7.0. As mentioned earlier, my staging server runs IIS 7.0 with all available components and non-thread-safe PHP 5 registered with FastCGI. Before I begin, I need to ask a few basic questions:
  • Does the Web application require a database management system, such as SQL Server or MySQL?
  • Do I need to install or configure additional components, such as ODBC connections, COM objects, or SSL certificates?
  • Does the Web application require special service accounts and elevated local or remote access permissions to file shares and other resources?
  • Are there any dependencies on platform-specific features that are unavailable on IIS 7.0, such as dependencies on the Apache module for URL rewriting (mod_rewrite)?
Asking questions like these will help get your Web apps running on IIS 7.0 faster. For example, if you are trying to migrate a PHP application from Apache 2.2 that uses mod_rewrite (say, for search-engine friendliness or to prevent bandwidth hijacking through inline-linked images), you will have problems until you implement a compatible custom URL rewriting solution on IIS 7.0. Fortunately, osCommerce does not require mod_rewrite functionality on IIS 7.0, but some search engine optimization packages might.
Now that I have determined and installed the required additional components on the staging server, I am ready to get my Web applications up and running. Again, there are a few questions that I want to ask myself:
  • What is the simplest configuration I can use to support the Web application?
  • What configuration is currently used in the production environment?
  • Can I optimize the Web application configuration on the new platform?
  • What is the minimum set of role services and modules that the Web app needs to function in the desired configuration?
Getting the Web application running in the simplest configuration is a quick way to make sure that it works. If everything looks great, it is time to apply a configuration similar to the existing production environment and import production data into the test databases. Of course, some issues might only surface in advanced configurations.
You might also notice opportunities for improvements when mirroring your production configuration in a test lab. Placing the applicationHost.config file on a UNC share is a great way to centralize the configuration of multiple Web servers. To increase fault tolerance through redundancy and content synchronization, consider implementing Distributed File System (DFS) Replication, which is a state-based multimaster replication engine available with Windows Server 2003 R2 and Windows Server 2008. You can also minimize storage requirements on Web front ends by placing the content files of your Web applications on UNC shares.
Other potential optimizations may involve security or dynamic caching. I didn't address security and performance optimizations in my test lab because that is beyond the scope of this article. Nor did I configure DFS to avoid the installation of further servers. The step-by-step instructions in the companion material provide a simplified example of how to host applicationHost.config file and Web content on UNC shares. And Figure 6 illustrates how to implement a UNC-based Web farm with DFS.
Figure 6 Web farm deployment with applicationHost.config file and Web content on UNC shares

Wrapping Up
IIS 7.0 is an impressive Web server platform. It features a redesigned core architecture while maintaining nearly 100 percent backward compatibility with IIS 6.0. It should be able to run most existing ASP.NET Web applications without modification. But given the extent of the architectural changes, you cannot assume that you won't encounter compatibility issues.
Whether you are migrating ASP.NET or PHP Web apps to IIS 7.0, it's best to take a staged approach with emphasis on proper planning, preparation, testing, and documenting code and configuration changes.
IIS 7.0 is worth the work involved. There are many improvements in areas such as security, performance, configurability, and flexibility. It would take an entire book to discuss them all. Centralized configuration and content sharing based on UNC shares, delegated configuration by means of web.config files in application directories, improved management tools, detailed diagnostics information and failed-request tracing, and dynamic output caching are surefire ways to win the hearts of Web administrators.
The capability to mix native and managed modules in the request processing pipeline benefits ASP.NET developers. And the performance and stability improvements that come with FastCGI on IIS 7.0 are excellent news for PHP application developers.
And there is one more important thing that helps IT pros to be successful with IIS 7.0: the Microsoft IIS Community Portal (www.iis.net). It contains all the information you need, including detailed articles about the new architecture, source code for C++ and ASP.NET developers demonstrating how to create IIS 7.0 modules, and step-by-step instructions to get PHP applications up and running. You'll definitely want to check out this site. It's a good first stop for answers to your IIS-related questions.

Fergus Strachan is Founder and Director of Maestra Ltd London, a consultancy firm that specializes in IT infrastructure design and project management support, serving international banks and educational institutions in the UK. He writes articles about Microsoft server technology and is the author of Integrating ISA Server 2006 with Microsoft Exchange 2007. He is also coauthor of Microsoft Exchange Server 2003 Resource Kit.

© 2008 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


Communications and Collaboration
Planning Your Migration to Unified Messaging
Jeff Goodwin
At a Glance:
  • A brief history of voicemail
  • Things to consider when planning your migration
  • Migration strategies
  • Multi-site challenges

Contents
A Brief History of Voicemail
Planning Your Migration
Practical Migration Strategies
A Migration Walkthrough
The Importance of Following a Plan
Migrating from a legacy voicemail system to a unified messaging platform can be a challenging project. But with some proper planning, you can accomplish this migration quite easily while minimizing any disruption that end users may experience. In fact, minimizing the
service disruption should probably be your primary goal when running a migration project of this sort.
To know what you need to plan for during your migration, you must first understand why companies feel voicemail is a mission- critical application. This basic information will help you understand what features you need to implement when migrating to Unified Messaging and why those features will be important to administrators.

A Brief History of Voicemail
Since you can't really plan how to get to your destination without knowing where your journey will begin, I want to step back and give a quick overview of voicemail systems—both where they come from and how they are architected. There are arguments about who invented the first voicemail system. However, the first commercially available voicemail system was called VMX, which stood for Voice Message Exchange.
VMX was originally designed to provide a way to communicate by voice to other people within the company, just as e-mail is used today. The system had such commands as send a message, forward a message, and reply to a message. When a message was left, the user was notified. There were various notification methods, including out-calling (ringing the person on a specific phone number) and a message-waiting light (an indicator light on the person's phone).
As VMX continued to gain popularity, organizations began to network their voicemail systems together. This offered an enterprise- wide voicemail solution where employees could send broadcast messages to company- wide distribution lists.
Of course, to provide a way to interact with the voicemail system, a touch-tone user interface (TUI) had to be created. These TUIs have varied over the years, and, unfortunately, there is no standard interface for voicemail. If you have voicemail on your cell phone, you're probably used to hearing something like "press 1 for voicemail." But as speech-driven voicemail systems, which use voice user interfaces (VUIs), become more common, the TUI itself isn't as important as it used to be. However, you shouldn't underestimate the attachment that users may have to a specific interface.
Enterprise voicemail systems in use today rely on two distinct architectures: distributed and centralized. In Figure 1, the Seattle, New York, and Austin offices have a distributed voicemail architecture, where the voicemail systems are co-located at each of these sites. In contrast, the London, Paris, and Glasgow offices have a centralized model, where the PBX infrastructure is networked together and all calls are routed back to the central location in London. Both models have advantages and disadvantages. But note that even in a distributed model, the voicemail systems can be networked together for message delivery between sites.
Figure 1 The two common voicemail architecture models
There are three primary reasons for why an organization may choose to use a distributed model: a need for local availability, the use of disparate PBX systems that can't be networked together, and uniform dial plan restrictions.
Some organizations require the voicemail system so that in the event of a network outage, the local PBX and voicemail system will still operate normally, giving the site full system functionality. Interactive Voice Response (IVR) systems will still work, and outside callers can still leave messages. In effect, there is no impact on end users, and work continues as normal. This may sound familiar—this is the same reason many organizations choose to localize their servers running Microsoft® Exchange Server. Your organization may want to perform a risk assessment to determine whether this localized survivability is necessary.
When an organization grows by acquisitions, the remote sites will often have PBX platforms that cannot be networked with the rest of the infrastructure. In this case, organizations can either change out the PBX, which can be a costly project, or leave the site intact and network the voicemail platform to provide a seamless voice messaging exchange. Third-party servers are also available to help you network your disparate voicemail systems.
In order to network PBXs together, a uniform dial plan (or UDP) is required. A UDP says that phone and voicemail extensions cannot overlap. In Figure 1, you'll notice that Seattle, New York, and Austin have overlapping extension ranges. This is because there is no UDP for the sites located in the United States. A UDP could be created by extending the extension ranges to 7 or 10 digits, but this creates other issues.
You may have to change the dialing templates on each phone, change all user extensions on the PBX, and change any applications that integrate with the system. And this can cause user frustration for employees who now must remember and dial 7- or 10-digit numbers to call someone down the hall.
While these basic features and architectural challenges have been improved over the years, the basics of voicemail remain the same. Message exchange, networking, notification methods, user interface, and architecture all are important factors that you must keep in mind when you plan a migration.

Planning Your Migration
Unified Messaging brings changes to every part of the organization—from how users interact with voicemail to how architecture is designed to how administrators handle management. All these differences must be addressed when you migrate to a unified messaging system. What follows is a checklist of things you need to take into account when planning your migration.
Message Networking Any organization that transitions from a legacy voicemail platform to Exchange Unified Messaging must have a period of coexistence. If your organization has already deployed message networking, which is a system to interoperate different voicemail systems, simply add the Unified Messaging system to the list of existing systems. I strongly recommend message networking for any organization that has multiple locations with separate voicemail or PBX systems.
Infrastructure Designing the infrastructure for Unified Messaging will require you to make decisions on networking requirements, telephony integration, server placement, and the like. If you are not already familiar with basic deployment strategies, read my previous article "Deploying Unified Messaging with Exchange Server 2007" (see technet.microsoft.com/magazine/cc137737).
Automated Attendants Many systems contain multiple automated attendants—different ones for, say, daytime, nighttime, and various departments. You'll need to work with your organization's telecom team to mirror the automated attendants on the legacy voicemail systems.
Message Waiting Indication You might wish your solution could do without message waiting indicators, but your users will probably demand this feature. So don't underestimate how attached users are to that red light notification. Fortunately, there are several companies that have developed message indicator applications for Exchange Server.
Fax Support Many legacy voicemail platforms support inbound faxing. Exchange Unified Messaging also supports inbound faxing capabilities, so you may want to plan on supporting this in your new deployment.
Interactive Voice Response Some legacy voicemail platforms support Interactive Voice Response (IVR) applications that Exchange Unified Messaging can't duplicate. You may want to leave these systems in place until a suitable alternative can be found.
Administration Exchange Unified Messaging relies on Active Directory® and the Exchange Mailbox Server role. You will need to plan for how administration, system requirements, and the like will be affected by a Unified Messaging deployment.
User Interface When rolling out a new UI, you must provide training for your users. Exchange Unified Messaging employs an interface that is very similar to most cell phone providers, so it should be fairly intuitive for most users. It also uses speech recognition so that users don't need to use touch tones. In Exchange Server, this is referred to as Outlook® Voice Access.
Training Unified Messaging brings a paradigm shift in the way your users communicate and use voicemail. Beyond the UI, you must provide training to your users to teach best practices and present the new, more efficient ways they'll be able to communicate. For example, users may not know how to interact with voicemail in their e-mail inbox or how to configure some advanced unified messaging settings.

Practical Migration Strategies
There are challenges, no doubt, in migrating a large organization to a unified messaging solution. However, I have been working in unified communications for 10 years and have seen even the most complex of organizations with very specific requirements successfully migrated.
Whether you're dealing with a single- or a multi-site organization, it will help if you build your migration strategy around these five basic steps:
  1. Plan and design your solution.
  2. Install and configure the Unified Messaging Server role.
  3. Migrate a pilot group.
  4. Revise your design based on feedback from the pilot group.
  5. Migrate your users.
Of course, multi-site organizations do face certain challenges that are generally not found in single-site scenarios. I discuss those challenges separately in the "Multi-Site Challenges" sidebar.
Planning and designing your solution is the single most important step in the migration process. I recommend you assign a Unified Messaging team to the project. This team should include representatives from various parts of your organization with expertise in telecom, Active Directory, Exchange, networking, security, training, and project management. And get a detailed architectural drawing of your company's existing PBX, voicemail, and e-mail infrastructures.
Once you have a design, you can install and configure the Unified Messaging Server role. Don't be afraid to install a United Messaging server in parallel to your existing legacy voicemail system. Just be sure you follow the documented best practices.
At this point, you're ready to try out your Unified Messaging server. Designate and migrate a small pilot group of users to the new system. A group of between 25 and 50 users will do. Choose these users carefully, creating a pilot group that consists of a varied group of users, from IT and telecom staff to departmental managers and sales personnel. By involving such a wide range of people, you can better determine whether your design is on target, what specific requirements you need to take into account, what training may be necessary for users, and so on.
After you receive feedback from the pilot group, you'll need to revise your design to address any issues that have been uncovered in testing. More specifically, revisit the items I discussed in the "Things to Consider" section and make sure you've addressed those points appropriately for your organization. Note that the most common changes made here usually pertain to training.
The final step is to migrate your users. There are many opinions and strategies regarding how to migrate the system into full production. The two primary methodologies are a gradual migration and a flash-cut.
If you choose to gradually migrate your user community over a period of weeks or months, you will create voicemail islands between your legacy and Unified Messaging users. This prevents users from forwarding messages to one another via voicemail. And if your design doesn't include a message networking solution, you will disrupt some of your business communications. If you are going to do this, I recommend you be sure to include a message networking solution in your initial design.
For a flash-cut migration, you move all your users in a single phase. Organizations typically perform this sort of migration over a weekend so they have time to test the system, make configuration changes, and back out of the migration in case a serious blocker is encountered.
Whichever method you choose, be sure to train your help desk personnel on the common questions and answers you gathered from your pilot group. You should also make sure the help desk workers are proficient at performing key tasks, including enabling a user, validating that a user is enabled, and changing a password.

A Migration Walkthrough
As I go through my sample migration, keep in mind that the primary goal for a successful migration to Unified Messaging is to move the user's voicemail box from a legacy voicemail platform to Unified Messaging with as little disruption to users (both internal and external) as possible.
Figure 2 details an organization's architecture. It has no plans to change this mixed Exchange architecture. The U.S.-based offices have a primary datacenter located in Seattle. All IT applications for Austin are served from Seattle, and the New York office warrants localized Exchange services. Voicemail for the U.S. locations are distributed due to the disparate PBX infrastructure. Technically, the PBX infrastructure cannot be networked to provide centralized voicemail. The European offices, in contrast, have a centralized datacenter located in London. All the IT and telecom applications in Europe are served from that central datacenter.
Figure 2 Sample architecture before migration
The entire voicemail infrastructure is based on the Octel (now known as Avaya) platform. All voicemail systems have been networked together using Octel Analog Networking (OAN), which is a proprietary networking protocol that uses analog phone lines and long distance calling to send networked voicemail messages. This architecture allows for corporate-wide voice messaging between sites.
Multi-Site Challenges
Multi-site organizations are faced with any number of challenges when deploying Unified Messaging. Fortunately, you can learn from some of the challenges other organizations have already faced. I can't cover every challenge in this article, but I can tell you about the most common challenges I have encountered when working with various organizations.
"I have a centralized Exchange Server architecture and a mixed telephony environment."
A distributed environment is easy to design. You place a Unified Messaging server and integrate it with the local PBX at each site. But what if you have a centralized or even a mixed environment? The question you need to ask yourself is whether you require local Unified Messaging survivability at the site. With local Unified Messaging survivability, that location can lose its connection to the centralized Exchange server and still provide automated attendant functionality and allow outside callers (quite possibly customers) to leave messages. The answer will likely depend on how mission critical that location is and how manageable the solution will be.
If you require local survivability for your site, a remote Unified Messaging server at the site might be a good option. If it is not a requirement, you can install an SIP Gateway at the local site. Exchange Unified Messaging has the flexibility of not requiring you to network the PBX infrastructure together, and there is no requirement for a Uniform Dialing Plan.
"I am in the process of upgrading my Exchange architecture."
Many customers are hesitant to begin a Unified Messaging migration while they are in the process of upgrading their Exchange infrastructure. My advice is that if you've already made the decision to move to Unified Messaging, go ahead and migrate while the Exchange server is being upgraded. Simply follow your upgrade plans and there shouldn't be substantial complexity added to the project.
"I have a lot of small sites with fewer than 10 people and some of those sites don't have a PBX."
At smaller sites that have a PBX located on-site, all you have to do is simply install an SIP Gateway at the local site. For sites that do not have a PBX on-site, there are telephony techniques that can be used to forward the phones to the central PBX for call routing to the Unified Messaging system. In short, you can deploy Unified Messaging services to the smaller remote sites in your organization quite inexpensively.
It is important to note that these smaller sites actually provide a great test bed for your migration. One common migration strategy is to start at the edge of the network and then move to the core. You migrate the smaller sites first and then perform a postmortem with the Unified Messaging team in order to figure out what went well and what challenges were encountered. You can then implement those lessons learned when migrating the next site.
The Unified Messaging team has decided to migrate from the edge of the network to the core. For all locations, placement of the Unified Messaging server will follow the best practice of following the Exchange server architecture. The team builds its own planning documentation for each site, keeping in mind the items discussed in the "Things to Consider" section. The documentation is used during installation and configuration of the servers, during user training, and when planning migration strategies.
For each site, the team plans on flash-cutting users to the Unified Messaging system and leaving the legacy platform in place to support IVR applications that will remain in use until an alternative solution can be found. The team decided to wait for 30 days after migration before moving on to the next location. The wait period is meant to ensure that each site has adjusted to the new platform before moving more users and adding more help desk support calls.
The organization decides to migrate the Austin location first. Since the Austin e-mail is served from Seattle, the team installs the Unified Messaging Server role in Seattle and provides PBX integration by installing an SIP Gateway in Austin. And since the legacy voicemail platform is networked together, the team must address the need for networking the Unified Messaging platform by installing a Message Networking server. The users are trained on the new system and the migration goes smoothly, so the team decides to migrate New York next.
The New York location has its own Mailbox and Hub Transport servers. The best practices, therefore, say that a Unified Messaging server should be installed in New York. The team installs the server, following the migration planning documents. The server is added to the list of nodes in the Message Networking server to be included in the voicemail network. The users are trained and smoothly migrated to Unified Messaging.
The Seattle location already has a Unified Messaging server deployed from the Austin migration phase. The team determines that the server contains enough voicemail ports to support both the Seattle and Austin locations. The users in Seattle are trained and migrated to Unified Messaging.
The architecture in Europe is much easier to design. The PBX infrastructure is networked together and Exchange is centralized. A Unified Messaging server is deployed in London and each site is migrated, starting with the smallest site first. Figure 3 illustrates the final architecture.
Figure 3 Sample architecture after migration

The Importance of Following a Plan
I have been asked to resolve many failed Unified Messaging migrations. Many of these organizations had broken one or more of the key rules.
First, always remember that it's not just a voicemail system. This is a mission-critical application and should be treated like one. Users rely on voicemail technologies for exchanging information. Keep in mind that this is a business application and plan your migration to account for all the considerations I've discussed in this article.
You absolutely must involve the telecommunications team in any migration to Unified Messaging. The telecom team will have the best understanding of the telephony applications that are integrated into the corporate infrastructure.
When migrating to Unified Messaging, plan for the enterprise solution. While you may just be implementing a single site in the beginning, you should always have the big picture in mind. For example, implementing a single site without providing Message Networking to your other sites will impact communications between users. This, of course, can be a big task, so don't be afraid to consult a specialist to design an architecture that takes into account all your IT applications.
Finally, don't underestimate the importance of training. Unified Messaging brings a paradigm shift in the way users communicate and interact with voicemail. What seems intuitive to you, may not be as intuitive to others. Good training is key to creating the smoothest transition possible.

Jeff Goodwin is a Senior Technologist at The VIA Group, specializing in Exchange and Unified Messaging design and deployment. You can reach him at jgoodwin@theviagroup.com.


Communications and Collaboration
How Voice Powers OCS 2007
Rajesh Ramanathan
At a Glance:
  • Making calls in OCS 2007
  • How calls are secured
  • Multimodal conversations in OCS
  • Integrating with Exchange UM for voicemail

Contents
Basic Voice Calls
Using Phone Numbers
Secure Calls
Traversing NATs and Firewalls
Routing on Phone Numbers
Inbound Routing
Quality Reporting and Troubleshooting
Putting It All Together
Multimodal Behavior and Conversations
Voicemail
Wrapping Up
The first part in this series on Microsoft Office Communication Server (OCS) 2007 showed how OCS was built on the strengths of Live Communication Server (LCS) 2005 to deliver
enhanced enterprise-class instant messaging (IM) and presence capabilities, as well as advanced media and phone functionality (see the February 2008 issue of TechNet Magazine at technet.microsoft.com/magazine/cc194409). This article continues with a deeper dive into the Voice over IP (VoIP) aspect of the story. I will explain how simple voice calls are made in the OCS system and discuss the technology layer by layer, so each component is added along with the corresponding functionality.
As the previous article noted, OCS can be set up to provide telephony to users in a number of different configurations:
Enterprise Voice This is the full Unified Communications solution that uses OCS along with the Microsoft® Mediation server without requiring a PBX. Exchange Unified Messaging (UM) provides voicemail features in this system. I'll refer to users who have this service as "UC users" in the remainder of this article. I'll also use the term "UC endpoint" to refer to a client in this configuration.
Enterprise Voice with PBX Integration This configuration lets users reap the benefits of unified communications while retaining the existing PBX phones on their desktops. It allows OCS and PBX to be set up in parallel so incoming calls can ring Office Communicator and PBX endpoints simultaneously. The PBX still owns the call routing and still provides the voicemail services.
Remote Call Control This feature uses the PBX phone as the primary phone and allows the phone to be controlled via the Office Communicator client.

Basic Voice Calls
I'll focus here on the workings of Enterprise Voice users. Simple voice calls are set up in the OCS system using the SIP INVITE mechanism as described in RFC 3261. The OCS server plays the role of a proxy similar to a postmaster that relays messages between clients (or endpoints). New SIP INVITEs are originated by clients whenever a real-time session, such as a call or instant messaging session, is created. If the INVITE is acknowledged with an answer (that is, the remote endpoint sends a 200 OK response), the call is established (see Figure 1).
Figure 1 INVITE format and 200 OK
INVITE sip:bob@example.com SIP/2.0
To: <sip:bob@example.com>
From: <sip:alice@example.com>;tag=5c5ffe5428;epid=d793aff63a
Call-ID: 3522acd5acd349b4855871e3100a5f4f
CSeq: 2 INVITE
Contact: sip:alice@xyz.example.com
Content-Type: application/sdp
Content-Length: 156

**Note: Alice Audio SDP payload not shown**

SIP/2.0 200 OK
To: <sip:bob@example.com>;tag=f5c728454a;epid=e73443245
From: <sip:alice@example.com>;tag=5c5ffe5428;epid=d793aff63a
Call-ID: 3522acd5acd349b4855871e3100a5f4f
CSeq: 2 INVITE
Contact: sip:bob@xyz.example.com
Content-Type: application/sdp
Content-Length: 160

**Note: Bob Audio SDP payload not shown**
In Figure 2, Alice calls Bob by choosing Communicator Call from Office Communicator (OC) 2007, and OC 2007 originates an INVITE to Bob's SIP Uniform Resource Identifier or URI (sip:bob@example.com). The INVITE carries the audio session descriptor (called SDP, for Session Description Protocol) of the media endpoints where Alice can receive audio. OCS forks the INVITE to Bob's registered SIP endpoints (for example, Communicator Phone Edition and Communicator desktop). The INVITE contains a From header of sip:alice@example.com; this is used by Bob's client endpoints in a Reverse Name Lookup (RNL) to find the name (Alice) to show in the incoming call notification.
Figure 2 Routing a call to multiple endpoints
The Globally Routable User Agent URI (GRUU) is shown for each endpoint. The GRUU uniquely identifies a SIP endpoint and is obtained during the registration process from the OCS server. The GRUU address helps in routing SIP messages, such that once the call is answered by an endpoint, subsequent SIP signaling for other mid-call operations can be carried between the endpoints directly using the GRUU address.
Figure 3 continues the process, in which Bob answers from the Communicator desktop. Bob's OC sends a 200 OK message telling where he can receive audio. As soon as the OCS proxy detects that one endpoint has answered, it cancels the call to the other Communicator Phone Edition endpoints. Once the 200 OK response reaches Alice's Communicator endpoint, both Communicator endpoints have sufficient information (IP ports, encryption parameters, and so forth) to start the media.
Figure 3 Answering from one endpoint

Using Phone Numbers
So far we have seen the user clicking on Communicator Call to create the invitation. The process is a little different when the user selects or types in a phone number:
  • The client first normalizes the phone number and represents it as a TEL URI, which describes resources identified by telephone numbers as indicated in RFC 3966.
  • Since the OCS server only recognizes SIP URIs, clients convert the TEL URI to a corresponding SIP URI by adding the domain suffix and a user=phone tag.
  • If the number corresponds to an internal user, OCS routes the invitation directly. If the number is external, the invitation is routed to the nearest SIP-PSTN gateway.
Some calls to a public switched telephone network (PSTN) number require the media path be set up before the call is answered so that remote announcements can be played or additional digits collected to complete the call. In such scenarios, the PSTN gateway sends a 183 Session Progress indicator with an audio SDP. Communicator uses this information to set up a two-way media path with one destination endpoint before the call is answered by the remote user.
Once the early media path is all set up, Communicator enables the Dual-Tone Multi-Frequency or DTMF (the telephone signaling tones) keypad so that the user can enter any additional digits required by the remote system. Any DTMF digits entered are sent in the media path as RFC 2833 packets. The PSTN gateway takes care of generating the appropriate DTMF tone signals on the PSTN side.
Note that if Bob has enabled Simultaneous Ring to his cellular phone, the OCS proxy would route the call to the gateway and ring Bob's other Communicator endpoints at the same time. In such scenarios, the OCS proxy indicates that forking is active in the 183 Session Progress response, and this makes Communicator establish a receive-only media channel with the PSTN gateway.

Secure Calls
One of the key features of voice calls set up by OC is that media is encrypted by default using the Secure Real-Time Transport Protocol (SRTP), as defined in RFC3711. SRTP provides confidentiality, message authentication, and replay protection to RTP traffic. During the call setup, clients negotiate security capabilities between themselves and exchange cryptographic keys as part of the INVITE mechanism.
The default encryption setting for OC is "optional," and this allows two OC endpoints to set up an encrypted media channel. This setting can be adjusted by the administrator to suit the organization's compliance needs. For example, it can be made tighter to force encryption on all calls, or it can be turned off altogether.

Traversing NATs and Firewalls
Clients in the OCS system use the Interactive Connectivity Establishment (ICE) technology to provide media connectivity to users behind Network Address Translation (NAT) devices and firewalls, without requiring any changes to the existing NAT components. ICE technology is currently being standardized by the Internet Engineering Task Force (IETF). Each client is aware of the audio/video (A/V) Edge server that serves it through an inband provisioning mechanism and maintains an authenticated link to the A/V Edge during sign-on.
Before making a call, the client allocates resources on the possible connectivity locations (the addresses and ports, which are also known as candidates) on the A/V Edge server (for media relay), on the NAT, or on the host client itself. When the Session Initiation Protocol (SIP) INVITE is sent out, it carries this connectivity information as part of the INVITE. The 200 OK answer carries similar candidate information about the peer. Once each endpoint has a list of peer candidates, an elaborate ranking and checking mechanism then selects the most optimal path between the two peers where media is guaranteed to succeed.

Routing on Phone Numbers
Routing on phone numbers introduces some complexities to the basic call mechanism, due to the following factors:
  • Organizations may have different dial plans that are deployed internally for short dialing.
  • Numbers may be stored in non-standard formats (for example, users may have saved seven-digit strings in Microsoft Office Outlook®).
  • There may be different policies mapped to different outbound numbers. For example, international number dialing can be blocked for certain users.
The OCS system requires phone numbers to be in the RFC 3966 TEL URI format before they can be routed correctly. Numbers that are not in this format are converted before the client issues an INVITE. The diagram in Figure 4 show how this happens within the OCS system.
Figure 4 Routing telephone numbers
Numbers available to the client may be from various sources. Pre-normalized numbers are from the Address Book Service (ABS), which can have admin-defined normalization rules for converting numbers to a normalized E.164 format. Once a client issues a SIP INVITE to the normalized number, OCS applies a translation process to map the number to any internal user.
  • Step 1 shows that numbers entering the system are either unique (normalized, following the E.164 numbering plan) or with a phone-context that identifies the location. These numbers are sent to the server as part of the SIP INVITE. OCS routes the INVITE to a translation process.
  • The translation process (Step 2) attempts to map the phone number to a UC endpoint using server-side RNL. The translation can identify a route for that number to an outbound routing, or to a UC user, or it can fail the call with a 4xx code if the number cannot be translated correctly.
  • If the number is translated to a non-UC or external number, it is sent to the outbound routing component, which redirects the INVITE to the appropriate SIP-PSTN gateway for further processing, after applying policies for the number related to the caller (Step 3A). Outbound routing takes care of balancing the call load across gateways or failing over to alternate gateways if necessary. The outbound routing process can fail or reject the call and return a SIP 403 response code if access is forbidden for a certain number. Note that outbound routing only applies when the caller is a UC user. When that is not the case, OCS tries to use the static routes to gateways set up for that URI.
  • If the number is translated to a UC user's number, the INVITE is routed to the SIP URI of that user (Step 3B). Inbound routing is a function of OCS that applies only to UC users and to all calls targeted to the SIP URI. As you'll see, inbound routing applies rules for ringing timeouts, call forwarding, and voicemail forwarding for the UC user.
Note that the number that results from the normalization process can vary depending on client location. Administrators can configure location profiles and assign number conversion rules specific to a certain location (such as how four-digit dialing will work in that location). Each UC user is assigned a location profile, and all clients in the system download the rules specific to their own location profiles using inband provisioning. I'll take a look at the details of Inbound routing next.

Inbound Routing
Inbound routing rules specify how calls to a user should be routed in the presence or absence of registered clients in the system. The inbound routing component also takes care of applying presence-based rules to the incoming call; for example, it can send incoming calls to voicemail if the user has set the presence state to Do Not Disturb. Inbound routing is aware of the presence container levels and automatically rejects calls from users in blocked containers. Figure 5 shows a summary of options supported by the OCS inbound routing component.
Figure 5 Inbound routing capabilities
Capability | Comment
Ring Duration | Default is 20 seconds. User can change this to 60 seconds max. Calls are diverted to unanswered calling destination after this timeout duration.
Route Unanswered Calls to Voicemail | Default if the user is enabled for voicemail. Call is routed according to Inbound Routing rules.
Generate Missed Call Notifications when caller hangs up before call reaches voicemail | Notifies Exchange UM of such missed calls.
Call Blocking | Rejects calls from blocked callers (only calls from users with SIP identities can be blocked).
Do Not Disturb | Routes calls to voicemail. Simultaneous ring destinations are not rung if the user is in this mode.
Allow Interrupt List | Allows calls if the caller is in the Team container, even if the user is set to Do Not Disturb.
Simultaneous Ring | Configures incoming calls to ring a PSTN phone number as well as Communicator and Communicator Phone Edition clients.
Call Forwarding Immediate | Forwards an incoming call immediately to another user, PSTN phone number, or voicemail.
Call Forwarding Unanswered | Forwards an unanswered call to another user, PSTN phone number, or voicemail.
Working Hours | Uses the working hours configured on the Outlook Calendar for activating call forwarding settings for a user.
Inbound routing rules are uploaded to the server as an XML schema as part of a user's self-provisioning information. Figure 6 shows how inbound routing works. An incoming call for a user in the OCS system rings the user by default (shown as "Me"). If the call is not answered within the ring duration, the unanswered call is sent to voicemail by default. The user can choose to modify the default configuration by choosing immediate forwarding to a number, to another person, or to voicemail directly.
Figure 6 Routing calls (during working hours)
Any calls that are immediately forwarded to another person or number will cause the inbound routing rules for that person or number to take effect. The user can also set unanswered call forwarding to another number or person.
If the user checks the Apply only during my Outlook Working Hours option in the forwarding rules, then the rules shown in Figure 6 are applied during the working hours, and the default behavior of ringing the registered endpoints is applied outside those hours. Note that this behavior requires that the organization has deployed Microsoft Exchange Server 2007 and Outlook 2007 clients because Communicator uses working hours information from the Exchange 2007 Availability Web service and leverages Outlook 2007 auto-discovery support to get the server location.

Quality Reporting and Troubleshooting
Although clients in the OCS 2007 system use RTAudio, a newer-generation audio codec that can tolerate imperfect network conditions such as jitter, packet loss, and so forth, monitoring is still essential so administrators can locate and fix potential hotspots. Clients in the OCS 2007 system report the quality of each call and provide detailed statistics including bandwidth, loss, jitter, mean opinion score (a measure of user perception), devices used, and more to a central Quality of Experience (QoE) server at the end of each call. This is done by sending the payload in a SIP SERVICE request to the QoE server, whose address is programmed using inband provisioning mechanism.
Note that all client endpoints report call quality to the QoE server independently. Should a call fail due to an error in the signaling, clients report the same to OCS as well, which provides a repository of all the errors created by clients.﻿

Putting It All Together
Now that we have walked through several aspects of setting up a call, including entering numbers, inbound and outbound routing, and connectivity checks, let's take a look at the process as a whole. Figure 7 summarizes what happens from the time a user makes a call. Clients in the OCS system play a critical role in setting up a call, and they manage the entire call setup process. OCS is involved in initial routing of the call, and edge servers help the clients find the media route that is most optimal. Now let's look at what happens next—the conversation.
Figure 7 End to-end call flow

Multimodal Behavior and Conversations
The concept of conversations is central in the OCS system. A conversation is a multimodal session between two or more people and can include voice, video, and IM at the same time, as well as artifacts such as file transfers during the session, e-mail in Outlook or notes stored in Microsoft Office OneNote®.
There are several aspects of conversations that can have an impact on the design of OCS client systems:
  • All modalities involved in a conversation use the same window. If an instant message is added to an audio conversation, it is ﻿associated with the same window.
  • Conversations can occur across devices. Users can choose to have audio on Communicator Phone Edition but have IMs on Communicator desktop.
  • All modes escalate to a conference together: if escalation occurs, all modalities of a conversation must be escalated together, no matter which device the modality is currently on. For example, if an audio mode is available on Communicator Phone Edition, and an IM mode for the same conversation is available on Communicator desktop, then escalation to conference ensures that both audio and the IM have the same set of participants for that conversation.
  • Conversations, not individual sessions, are logged. Call logs contain full information about the conversation, including details about notes taken during the conversation, instant message exchanges, how long the audio lasted, and so on. This gives users a handy consolidated view of the entire conversation when they open the call log in Outlook.
  • Conversations can continue from the conversation logs. OCS clients tie conversations together using a conversation ID (a number that uniquely identifies the conversation across devices and apps. There is an elaborate mechanism to compute deltas of conversations when new conversations are started from a history log of an old conversation. The same conversation ID is also stored as the e-mail's ID in Outlook.
The conversation ID travels as part of the SIP INVITE as a custom property, Ms-Conversation-Id. Figure 8 shows such a transaction where voice is added first, followed by an IM. Note that at the end of the multimodal conversation, the history log that is stored in Outlook has the same conversation ID.
Figure 8 Multimodal conversations and conversation ID

Exchange UM is the voicemail system for OCS 2007. OCS will only route calls for users that are UC-enabled to Exchange UM. OCS provides several features for integrating with Exchange UM:
  • The Call Voice Mail option from Communicator's Phone UI (see Figure 9) allows the user to manage the UM mailbox, change greetings, and so forth without having to enter the PIN because he is already authenticated.
  • The message waiting indicator in the Communicator UI (see Figure 10) allows the user to open the voicemail folder in Outlook while the one in Communicator Phone Edition lets the user play the voice messages directly from the phone.
  • Calls can be forwarded to voicemail from Communicator's UI.
  • Declined calls are routed automatically to voicemail.
  • Play on Phone from the voicemail item in Outlook rings Office Communicator clients directly.
Figure 9 Call Voice Mail option in Communicator
Figure 10 Message waiting indicator in Communicator
The mechanism with which OC receives the voicemail notification is unlike the Communicator Phone Edition. OC registers for new mail notifications on the Voicemail search folder in Outlook and reports new messages in this folder. Communicator also uses Missed Conversations and Call Logs.

Wrapping Up
I've shown here how voice calls work in OCS 2007, a system built upon SIP that leverages several RFCs. Client endpoints play a central role in managing calls in OCS. All voice calls are securely encrypted by default. OCS provides flexible components to allow manipulating numbers and managing their flow throughout the system.
Conversations are a central concept in OCS, involving voice, IM, and video. OCS integrates with Exchange UM and allows Communicator endpoints to be notified of voicemail, and to access voicemail through Outlook, or directly from the server.

Rajesh Ramanathan has worked in communications for 14 years and has designed voice protocols, user experiences, and voice and conferencing for Office Communicator 2007. He currently works as Lead Program Manager on the Office Communicator team at Microsoft. Rajesh can be reached at rajeshra@microsoft.com.


Communications and Collaboration
Create Powerful Connections with Groove and SharePoint
Alan Maddison
At a Glance:
  • Creating a SharePoint document library
  • Creating a Groove workspace
  • Synchronizing Groove and SharePoint
  • Allowing access to the workspace

Contents
Groove and SharePoint Together
Getting Started
Setting up Approval Control
Creating the Groove Workspace
Synchronizing Groove and SharePoint
Allowing Access to the Workspace
Last Words
IT professionals often wonder about the apparent similarity between Microsoft Office Groove 2007 (Groove) and Microsoft Office SharePoint Server 2007 (SharePoint), and they're unsure under which
circumstances each is appropriate. At first glance, the two products do seem very similar, since they both provide collaboration capabilities. However, once you take a deeper look at the architecture, you begin to see how these products differ.
SharePoint® offers a robust and scalable content management framework and collaboration toolset with support for structured workflows. Groove™, in contrast, is a collaboration toolset that allows rapid workspace deployment with little or no IT overhead. With support for document collaboration, threaded discussions, and calendaring among its features, Groove provides the ideal framework for dynamic, secure, ad hoc collaboration. In short, most users will naturally end up using these products in different ways.
Moreover, the release of Groove 2007 adds built-in support for SharePoint integration using the SharePoint Files tool. This tool allows Groove workspaces to be synchronized with SharePoint document libraries and, as we will see, the built-in integration means you can easily make the most of both products. In fact, combining the flexibility and dynamic nature of Groove with the sophistication and power of SharePoint will yield significant benefits for many.
Groove and SharePoint Together
Today's IT world demands ever-increasing collaboration between organizations and their partners and vendors. However, the powerful collaboration solutions that are being implemented using products such as SharePoint tend to focus heavily on internal teams and the need to make information broadly available throughout an organization. In such cases, the challenges, both from an administrative and security perspective, of providing access to users external to an organization are often significant. In the face of such challenges, the goal becomes enabling users to reach out to external partners on an ad hoc basis.
The combination of Groove and SharePoint offers exactly that solution. SharePoint provides intra-organization collaboration, content search, and a workflow framework; adding Groove lets companies extend those capabilities to include easy inter-organization collaboration. With Groove, users can quickly set up collaboration workspaces that cross company boundaries. Furthermore, support for 192-bit Advanced Encryption Standard (AES) symmetric keys ensures that the collaboration is secure. In fact, Groove security is built-in and always on, so users don't have to worry about configuring it.
The security architecture of Groove provides a high degree of protection for these workspaces and their contents, both while stored locally and during network transmission. The SharePoint Files tool allows this information to be bi-directionally synchronized with a SharePoint document library to ensure that the information is available centrally for easy internal access. Let's use a concrete example to see how this process works.
Getting Started
Organizations often collaborate with many companies during day-to-day business activities. This typically involves not only external organizations but also individuals from departments that don't necessarily work together regularly, such as finance, corporate communications, and legal. Thus these organizations need both to create an ad hoc internal team that will collaborate and to make the product of this collaboration available to those who aren't involved in the collaboration process but who must have access to the final deliverable. Using SharePoint and Groove, it is easy to achieve these goals.
If you don't already have a SharePoint document library that you'd like to synchronize with your Groove workspace, the first step is to create one. To do so, connect to SharePoint and then select the View All Content link. After this page has loaded, select Create and then, as Figure 1 shows, choose Document Library from the Libraries column.
Figure 1 Creating a SharePoint document library
There are a number of settings to configure on the new document library page—the name and description of the library, whether to include the library in the Quick Links section, whether to enable versioning, and the default format for new documents created directly in the library. The Document Version History selection is disabled by default, but you should enable this feature if versioning is appropriate to your environment. You should also consider some of the advanced functionality that is available with SharePoint and can be utilized as part of your Groove and SharePoint implementation. This enhanced functionality includes:
  • Version history
  • Approval control
  • Checkout procedures
  • Workflow control
  • Document retention policies
  • Auditing
  • Really Simple Syndication (RSS)
These additional functions available through SharePoint can offer tremendous value when working with Groove. In my example, this additional functionality—in particular Approval control—will prove very useful. For example, suppose team members collaborate within Groove to draft a document. When the final draft is ready for review, it could be synchronized with SharePoint and then it could become subject to the approval process.

Setting up Approval Control
To configure this functionality, you select the Settings menu from the document library and then choose the settings item. This lets you view all of the configuration and customization options. To configure Approval, select the Workflow settings link in the Permissions and Management section to bring up the Add a workflow page. Now select the Workflow template, which in our example will be the Approval template, then enter a name for the workflow. Next you must decide whether to use an existing task list or create a new one. A new task list offers better confidentiality, but since this is not an issue in our example, just select the default.
After choosing the task list, you must decide how the Approval process should begin. Let's make it a manual process by allowing any authenticated user with the correct permissions to start the process. Other options allow you to automate this process based on versioning, if it is enabled on the document library, or whenever a new document is added or an existing document is changed.
After pressing Next to accept these configuration options, you'll need to enter some final configuration information. The first option to configure is how tasks should be assigned. Since it is possible to have multiple people in the approval process, you can assign the approvers serially, where one person approves the item and then it moves to the next person, or in parallel, where anyone can approve it at any time. You must then decide whether to allow approvers to assign tasks to other people and whether they can request a change before the task is complete.
The next step is to select the actual approvers, and then decide when the task should be completed by. Finally, you decide when the workflow is completed. The default option is that the workflow is completed when all the tasks are done. In the final configuration option, you specify whether the Approval status is updated once the workflow is complete. After you press the OK button, the Approval process is created and you can begin the process of approving documents.

Creating the Groove Workspace
Once you have created the SharePoint document library, the next step is to create a Groove workspace. To do so, first ensure you have a Groove account and at least one identity. Identities are the basis for presence and communication within Groove. Each Groove account can be associated with multiple identities, and each identity is associated with a unique contact file so you can customize your contact information for each identity you use. You can also export your Groove account and associated identities in order to use them on multiple computers.
Once you have created an account and an identity, open the Groove Launchbar by double-clicking on the Groove icon in the System Tray. Click on the New Workspace option, enter an appropriate name for the workspace, and choose a Standard type of workspace. The Groove standard workspace facilitates presence, alerts, and role assignment for workspace members. When you click OK, Groove will create the workspace and open it. Figure 2 shows a new workspace with some files added to the Files tool. At this point, the workspace is functional and you can complete its setup by configuring alerts or by adding any tools you wish.
Figure 2 A new Groove workspace
Alerts can be configured at the tool, workspace, or individual file level. The four levels of alerts are none, medium, high, and auto. A setting of medium highlights new content, while high or auto will play an audio file. You can also configure alerts to notify you when a user enters a particular workspace. To configure alerts, you select Set Alerts from the Options menu. Alerts on individual files are set on the properties of the file itself; right-click on a file in the workspace and select Properties.
In addition to the Files tool for document collaboration and the Discussion tool for threaded discussions that are part of the default Standard workspace, there are a number of other tools including Calendar, Forms, Issue Tracking, Meetings, Sketchpad, and, of course, the SharePoint Files tool. Each provides valuable additional collaboration functionality. To add a tool, select the Add a tool to this workspace button on the bottom right of the workspace, then choose the one you want to add, as shown in Figure 3. It's also easy to delete a tool at any time.
Figure 3 Adding a tool to the Groove workspace

Synchronizing Groove and SharePoint
After adding the SharePoint Files tool, you need to set the URL of the SharePoint site that contains the document library you want to synchronize with. To do so, you just press the Setup button in the Groove workspace, shown in Figure 4, after selecting the SharePoint Files tab at the bottom of the workspace screen.
Figure 4 Setting up the SharePoint Files tool
Once you enter the URL, you will see all the libraries available on that SharePoint site. As Figure 5 shows, you then select the library you wish to use. Groove will connect and begin automatic synchronization. This means that any documents currently in the SharePoint library will be downloaded to and made available through the Groove workspace. Figure 6 shows an example.
Figure 5 After you set the URL, your SharePoint libraries will be available in Groove
Figure 6 Synchronizing Groove and SharePoint
The documents are displayed as unread in the Groove workspace because they have just been downloaded. Note that all files that are synchronized with SharePoint can take advantage of the SharePoint Check in/check out functionality. Furthermore, when you check out a file, you are ensuring that no other users can make changes while you are editing the document.
It is also important to remember that the SharePoint files tool provides for two-way synchronization, so if you add files to the SharePoint files tool via Groove, they will be synchronized to the SharePoint document library. While the default synchronization is set to happen automatically, you can initiate synchronizations manually by pressing the Synchronize Now button. If you do this, you will see a summary of changes for any files that require synchronization and an associated action, get (from SharePoint) or send (to SharePoint), as shown in Figure 7.
Figure 7 After manual synchronization, you’ll see a summary of changes
One important point to remember is that the Files tool that is deployed as part of the Groove Standard workspace is different from the SharePoint Files tool. The Groove Files tool does not synchronize with SharePoint and, depending upon your needs, may be removed. However, because it does not synchronize with SharePoint, the tool can actually be used as a storage location for temporary files that the team needs access to during the course of a project. The files stored there will still be synchronized among workspace members.

Allowing Access to the Workspace
With the basic collaboration infrastructure now deployed, you can begin to allow users access to the workspace. First, however, you need to invite them, which you can do through e-mail or Groove Instant Messaging. Alternatively, Groove allows you to export the invitation to a file. This file, with a .grv extension, which by default is associated with the Groove application, can then be delivered in some other fashion, such as placing it on a Web site and allowing the recipient to download the invitation.
Groove instant messaging includes security checks to ensure that the invitation is readable only by the recipient. When inviting by e-mail or using an invitation file, make sure that Require acceptance confirmation is checked, so you get to double-check that the person joining your workspace is the invitee you intended. This provides an additional layer of security for the workspace. You should also note that you can cancel pending invitations.
When inviting users to the workspace, you need to select a role for each recipient. Roles include Manager, Participant, and Guest, and the default assignment is Participant. Each role has specific rights within the workspace, and, as a general rule, you should restrict the Manager role to those who really need all rights in the workspace, such as the ability to delete tools. Workspace creators obtain the Manager role by default, and assigned roles can be changed at any time.
Given the ad hoc nature of the project in this article, one last task would be to delete the Groove workspace once everything was complete. After making sure you had synchronized the workspace with the SharePoint library, you would open the workspace, go to the File menu, and choose Delete workspace. You would then choose either the option to delete From This Computer or For All Members. (Each user gets a copy of the workspace on his machine, and it lives on indefinitely unless you delete for all members.)
The culmination of the project would be another opportunity to leverage the complementary nature of Groove and SharePoint. Once the final synchronization of documents has occurred, you would be in an ideal position to take advantage of the Information Management Policy settings in SharePoint. In particular, data expiration could be enabled to provide a method for automatically managing the archived data.
Enabling expiration allows you to specify a period of time for which to retain data. At the end of this retention period, SharePoint can automatically delete the data or start a workflow process. This functionality provides a great way to incorporate Groove data into your company's information lifecycle management policy. You can configure the data retention policy at any time. In fact, if you did not do this when you initially configured the library, you can do it now.
In order to enable expiration, select the Settings menu from within the document library, then choose the document library settings menu item. You can then view the Information Management Policy Settings menu option in the Permissions and Management section of the page. Once you have clicked on the Information Management Policy Settings link, you can either specify a new custom policy that you define or you can choose to enable the document library to use the policy defined at the site level.
Note that if you define a custom policy, you will need to enter some configuration information. Therefore, in my example, you would enable expiration, then select a retention period and an action to perform once this retention period had expired.
Groove is a rich collaboration client designed to provide a dynamic, intuitive, and secure framework for ad hoc collaboration. With its rich toolset, Groove provides a lightweight and adaptable collaborative platform that can be easily used within an organization or among different organizations.
If SharePoint already provides a robust and powerful content management framework and collaboration toolset for your company, I think you'll find that combining it with Groove is a compelling solution. The two products are highly complementary and when used together can add significant productivity gains for collaborative teams.﻿

Alan Maddison is a Senior Consultant with Strategic Business Systems, a division of Brocade.


Windows Administration
Introducing Windows Server 2008 Failover Clustering
Chuck Timon
At a Glance:
  • Failover Cluster Management snap-in
  • New features and enhancements
  • Backup and Restore Functionality
  • Migrating from Windows Server 2003

Contents
New Management Interface
Improved Configuration Processes
Embedded Validation Procedure
New Quorum Model
Enhanced Security Features
Expanded Networking Functionality
Increased Reliability When Interacting with Storage
Built-In Recovery Processes
New Backup and Restore Functionality
Migrating from Windows Server 2003 Server Clusters
Ever since clustering was first introduced in Windows NT 4.0 Enterprise Edition, users have complained that it is too hard to set up and even harder to maintain. Administering a cluster has
required the administrator to have more than just an understanding of the clustering piece itself—the administrator has also needed intimate knowledge of storage technologies and how the cluster service would interact with a variety of storage solutions. The comprehensive skill set required to get a high availability solution up and running, and then maintain it, was hard to come by in many organizations.
Clustering has improved over the years, but it still left much to be desired when Microsoft began working on Windows Server® 2008. With this in mind, the team set out to redesign clustering with the primary goal of simplicity. In Windows Server 2008, Microsoft® Cluster Services (MSCS) has received a complete facelift and is now called Failover Clustering.
That's not to say simplicity is the only thing the new Failover Clustering implementation brings to the table. Over the years, Microsoft has learned many lessons, as organizations have provided valuable feedback about what they would like to see in a listened clustering solution. The new Failover Clustering functionality addresses many of the top issues users have reported and also tosses in some exciting new features that make it a very attractive proposition. So in this article, I'd like to introduce you to some of the cool new features you'll find in Windows Server 2008 Failover Clustering.

New Management Interface
After you install Failover Clustering, the Failover Cluster Management interface can be accessed in Administrative Tools or by running Cluadmin.msc. The Failover Cluster Management snap-in, like other management interfaces in Windows Server 2008, is a Microsoft Management Console (MMC) 3.0. For cluster veterans, opening the Failover Cluster Management snap-in for the first time will feel a bit like being dropped in a foreign country with no map.
The new interface is divided into three distinct panes, as shown in Figure 1. The left-hand pane lists all the Windows Server 2008 Failover Clusters in your organization. The center pane provides details about whichever part of a cluster configuration you've selected in the left-hand pane, while the right-hand pane shows you the actions that can be executed.
Figure 1 Failover Cluster Management snap-in
Say, for example, you select Storage in the left-hand pane. The center pane then details what storage has been provisioned in the cluster, and what storage, if any, is currently available. As you can see in Figure 1, the cluster contains a piece of storage that supports a witness disk, storage that has been provisioned for a File Server, and some available storage. The right-hand pane lists relevant actions, such as adding more storage. Note that the Failover Cluster Management snap-in cannot be used to administer previous versions of Microsoft Clustering Services.

Improved Configuration Processes
Configuring a Failover Cluster is quite easy. Many of the actions for configuring, reconfiguring, and maintaining a cluster have wizards. Thanks to these wizards, the administrator no longer has to be concerned about whether resources are configured correctly or whether they will come online in the correct order.
Figure 2 shows the High Availability Wizard. In this particular example, a File Server has been configured. On the left, you can see the list of steps the wizard has guided the administrator through. Once the process is complete, a summary page is displayed and a report can be viewed.
Figure 2 High Availability Wizard

Embedded Validation Procedure
In previous versions of Windows Server, to be considered a supported cluster solution, hardware configurations had to be listed as a Cluster Solution in the Windows Server Catalog. This included multi-site clusters, which were listed separately under the Geographically Dispersed category. To be listed in the catalog, the hardware vendors had to run a set of Windows Hardware Quality Lab (WHQL) tests and submit the results to Microsoft. This was a costly proposition for the vendor, and the Windows Server Catalog database was difficult to maintain.
In Windows Server 2008, a built-in validation process is included in Failover Clustering. This process consists of a series of tests that are grouped into four main categories, as shown in Figure 3.
Figure 3 Failover Cluster validation test categories
You can see that the Network category is expanded to show the tests that are run; each category contains a series of tests. The Storage category, which is perhaps the most critical of the four categories, includes tests that ensure storage solutions comply with the new requirements put in place for Windows Server 2008 Failover Clusters.
Specifically, hardware vendors now must use drivers based on the Microsoft Storport driver, and they must support SCSI-3 Persistent Reservations. Additionally, multi-path software Device Specific Modules, when used, must be based on the Microsoft Multi-Path Input\Output standard.
With the incorporation of the validation process, the support model has changed. All hardware must have a Windows Server 2008 logo, and all validation tests must pass. The only exceptions are multi-site clusters that have two separate and distinct storage enclosures—one at each site—and the Exchange Server 2007 cluster continuous replication implementation, which doesn't use any shared storage.

New Quorum Model
The Quorum Model has also changed in Windows Server 2008 Failover Clustering. In older systems, when an administrator heard the word quorum, he thought of a shared disk where the cluster configuration and some replicated files resided. This was a single point of failure in the cluster. If the quorum disk failed, the cluster service terminated and high availability was lost.
Windows Server 2003 Server Clusters offered a second quorum type called the Majority Node Set quorum. This type of quorum was typically implemented in multi-site clusters and required no shared storage. The Majority Node Set quorum consisted of a file share that resided on the system drive on each cluster node. Connections to this quorum type were by way of Server Message Block (SMB) connections. Once again, in order for the cluster to function, a majority of nodes had to be participating.
Then, with the introduction of Exchange Server 2007 ﻿cluster continuous replication (CCR), File Share Witness (FSW) capability was added to Windows Server 2003 Server Clusters. This allowed for a single Exchange 2007 CCR cluster node (or any multi-site cluster) to continue to provide services as long as a connection to the FSW resulted in a majority being achieved.
In Windows Server 2008 Failover Clustering, the concept of quorum now truly means consensus. Quorum, or consensus, is now achieved by having enough votes to bring the cluster into service. Enough votes can be obtained in several ways, depending on the quorum configuration. There are four quorum modes available in a Windows Server 2008 Failover Cluster, and they are shown in Figure 4. Of the four modes listed, only the first two (Node Majority, and Node and Disk Majority) can be automatically selected during the create cluster process. The following logic should be used:
  • If an odd number of nodes are being configured in the cluster, select Node Majority mode.
  • If an even number of nodes are being configured in the cluster and shared storage is connected and accessible, then select Node and Disk Majority.
Figure 4 Quorum Modes in the Configure Cluster Quorum Wizard
To select a witness disk from available storage, select the first disk that is at least 500 megabytes in size and has an NTFS partition configured. The remaining quorum modes can only be selected manually by running the Configure Cluster Quorum Wizard. The Node and File Share Majority option is typically used in a multi-site cluster configuration or in an Exchange 2007 CCR cluster. The last option, No Majority: Disk Only mode, is equivalent to the shared quorum model in legacy clusters. It is a single point of failure and generally should not be used.
There are only two types of witness resources, a physical disk and a file share, that can be configured in the cluster to help achieve consensus.
A witness disk is a piece of storage that the cluster service can bring on line. This disk is located in the Cluster Core Resource Group along with the cluster Network Name and associated IP address resources. When the witness disk is configured, a Cluster folder is placed on the disk and a full copy of the cluster configuration (a cluster ﻿hive or replica) is placed on the disk.
An FSW is a network share that is located, in an ideal situation, on a server on the network that is not part of the cluster. An SMB connection is made to the FSW, and the FSW maintains a copy of the witness log file, which contains versioning information for the cluster configuration.
There can only be one witness resource configured in a cluster. This resource provides an extra vote should the cluster need it to achieve quorum. In other words, if the cluster is one vote (and therefore one node) short of achieving a consensus, the witness resource is brought online so quorum can be achieved. If the cluster should be more than one vote short of achieving quorum, the witness resource is left alone and the cluster remains in a dormant state, waiting for another cluster node to join.

Enhanced Security Features
Failover Clusters feature several new security enhancements. Perhaps the most significant involves the removal of the requirement for a Cluster Service Account (CSA). In previous versions of the Microsoft Cluster Service, a domain user account was required during the configuration process. This account, which was used to start the cluster service, was added to the local administrators group on each node of the cluster and given the required local user rights to allow the cluster service to function properly. As a domain user account, the CSA was subject to a number of domain-level policies that could be applied to cluster nodes. These policies could adversely impact high availability by causing the cluster service to fail.
The cluster service now runs under a local system account with a specific set of rights on the local cluster node that allows it to function properly. The security context for the cluster has transitioned to the Cluster Name Object (CNO), which is the computer object that is created, by default, in the Computers container in Active Directory® when the cluster is first created. Once a cluster is successfully created and the CNO exists in Active Directory, the user account that was used to install and configure the cluster is no longer needed.
Additional computer objects created in Active Directory in the Computers container are associated with a Failover Cluster. These objects, called Virtual Computer Objects (VCOs), equate to cluster Network Name resources that are created as part of Client Access Points (CAPs) in the cluster. The CNO, which is responsible for creating all VCOs in a cluster, is added to the System Access Control List (SACL) for the object in Active Directory (see Figure 5).
Figure 5 Security on a VCO in Active Directory
The CNO also takes on the responsibility for synchronizing the domain passwords for all the VCOs it has created. This process is accomplished in accordance with the configured domain policy for password rotation. Additionally, since the CNO is responsible for creating all the computer objects associated with the VCOs in a cluster, the CNO (Computer Account) must have the domain-level right to create computer objects on the container where the VCOs are created (by default, that is the Computers container).
Another change, Kerberos is now used as the default authentication method. The existence of computer accounts in Active Directory allow for this enhanced security feature. But the cluster has the ability to use NT LAN Manager (NTLM) authentication should an application that is not able to use Kerberos for authentication ever need to access cluster resources.
Communication between cluster nodes that deal directly with the cluster process are also more secure. All intra-cluster communications are signed by default. Using the cluster.exe common language interface (CLI), this cluster property can be changed so all communications between nodes are encrypted to provide an additional level of security.

Expanded Networking Functionality
New networking capabilities in Failover Clusters enable more flexibility when designing high availability and disaster recovery solutions. At the same time, these networking enhancements provide more reliable connectivity among the nodes in the cluster.
Probably the most requested customer feature has been the ability to locate cluster nodes on separate networks. This is now possible. The cluster network driver has been completely rewritten so that it provides highly reliable and fault-tolerant communication among nodes in a cluster, provided each node is connected to at least two separate and distinctly routed networks.
The cluster network driver constructs its own internal routing table based on the connectivity information provided during the cluster startup process. This includes local connectivity information as well as information that is provided in the cluster configuration database (cluster registry hive).
Part of the cluster validation process includes a network connectivity discovery process. Being able to locate cluster nodes on different, routed networks has eased the networking requirements for multi-site clusters. This will make it easier and less expensive for organizations to deploy them. It also makes the use of iSCSI storage a more attractive storage solution for use in Failover Clusters.
Cluster nodes can also obtain IP Address information via DHCP (Dynamic Host Configuration Protocol). This can ease the burden on network administrators, if they can accept using dynamic addressing for the servers in their environment.
The configuration of a cluster node's network interfaces determines which networks will use static or dynamic IP addresses. Even if an IP address resource in a cluster is obtained from a DHCP server, it can be changed to a static IP address in the Failover Cluster Management snap-in.
In the past, all cluster communication used User Datagram Protocol (UDP) broadcast, and sometimes, multicast. Multicast functionality has been discontinued and cluster communications now use UDP unicast. (Port 3343 is still the common port used by Microsoft clusters.) Many network administrators will be very happy to find that broadcast is no longer used. The real payoff in a cluster, however, has to do with the new messaging processes that are internal to the cluster service itself. (This, however, is beyond the scope of this article.) Intra-cluster communications now have characteristics of more reliable TCP communications even though UDP is used as the transport mechanism.

Increased Reliability When Interacting with Storage
The way in which Failover Clusters interact with storage is drastically different. The cluster disk driver (clusdisk.sys) was completely rewritten and is now a true Plug and Play (PnP) driver. And how it interacts with storage has changed.
In Windows Server 2003, the cluster disk driver was in a direct path to the storage. But in Windows Server 2008, the cluster disk driver communicates with the partition manager (partmgr.sys) driver in order to interact with storage. These two approaches are illustrated in Figure 6.
Figure 6 How the storage stack has changed in Windows Server 2008
Partition manager has the primary responsibility of protecting cluster disk resources. All disks on a shared storage bus are automatically placed in an offline state when first mapped to a cluster node. This allows storage to be simultaneously mapped to all nodes in a cluster even before the cluster is created. No longer do nodes have to be booted one at a time, disks prepared on one and then the node shut down, another node booted, the disk configuration verified, and so on.
There are still, however, the storage tests, which are run as part of the cluster validation process and require the disks to be initialized. This can be accomplished on one of the nodes of the cluster before the validation process is executed. Once storage is added to a cluster, the disks show a status of Reserved in the Disk Management interface and are never left in an unprotected state.
Another change has to do with the SCSI commands. In Windows Server 2003, SCSI-2 Reserve\Release commands were used with the cluster disk driver writing to sectors on the disk itself. In Windows Server 2008, SCSI-3 PR (Persistent Reservation) commands are required. Cluster nodes must register before they are allowed to place a reservation on the storage, and cluster nodes periodically defend their reservations using the Registration Defense Protocol.
One of the storage tests in the validation process verifies this functionality. If a storage solution does not support SCSI-3 (PR) commands, it is therefore not supported in a Failover Cluster.
Many organizations use multi-path software for redundancy when connecting to storage. This is supported and even promoted as a best practice. However, third-party multi-path software solutions, or device specific modules, must be rewritten using the Microsoft Multi-Path Input\Output standard to be supported in a Failover Cluster. This ensures that all SCSI-3 PR commands are simultaneously sent down all paths to the storage whether the path is active or not. This functionality is also verified as part of the validation process.
Additional storage improvements include an improved check disk (chkdsk.exe) process, built-in disk repair functionality that was previously part of the Cluster Server Recovery Utility, and self-healing disks. In Failover Clusters, the disk signature and the LUN ID are both used when identifying a cluster disk resource. If either of these has changed, the cluster configuration is updated. This translates into reduced errors simply due to an attribute change on a physical disk resource resulting in greater high availability.

Built-In Recovery Processes
The previously mentioned disk repair is obviously one of the built-in recovery capabilities. Another is an Active Directory Repair capability. If the computer object that represents the CNO is deleted, you will no longer be able to create the computer objects associated with cluster CAPs. However, the first issue you'll experience will likely be when highly available apps or users can't gain access to resources outside of the cluster because a security token cannot be obtained.
Recovering from a deleted CNO is a two-step process. First, you must engage a Domain Administrator to recover the deleted computer object from the DeletedObjects container in Active Directory. Then, after the object has been restored and re-enabled, you execute the Repair Active Directory Object process in the Failover Cluster Management snap-in.
In Windows Server 2003 Server Clusters, there was a possibility that the cluster configuration file located in the %systemroot%\cluster subdirectory would become corrupted and then have to be replaced. In Failover Clusters, the self-healing capability can help. If the cluster service starts on a node and the configuration database is corrupted, a minimal configuration template will be loaded using information contained in the HKLM\System\CCS\Services\ClusSvc\Parameters registry key. The node will attempt to join an already formed cluster and, if this attempt is successful, a fresh copy of the cluster registry hive will be pushed to the node. If a node cannot join a cluster, the cluster service will be terminated.

New Backup and Restore Functionality
Failover Clustering comes with its own Volume Shadow Copy Service writer. This plays a key role in backing up and restoring a cluster database and the data that resides on the physical disk resources. Backing up the cluster configuration is pretty straightforward. As long as the system state is part of a backup, the cluster configuration can be restored. But note that a cluster should only be backed up if the cluster has quorum. This guarantees the most up-to-date cluster configuration is backed up.
There are two distinct types of cluster restores: authoritative and non-authoritative. A non-authoritative restore uses Windows Server Backup or a third-party backup application to execute a restore from a selected backup. An authoritative restore of a cluster node, on the other hand, can only be accomplished using the Windows Server Backup CLI (wbadmin.exe).
An authoritative restore essentially takes the cluster configuration "back in time" to when the backup was performed. To accomplish an authoritative restore, the cluster service is stopped on all nodes except the node on which the restore is being executed. Once the restore is complete and the cluster service has been started on the restored node, the restored configuration of the cluster becomes the definitive new cluster configuration. Then, when the cluster service is restarted on the remaining nodes in the cluster, the restored configuration is pushed out during the join process.
This can save significant time and money in some certain scenarios. Suppose you have a print cluster that hosts multiple print spooler resources, each supporting 1,500 printers, and you accidentally delete one of the print spooler resources. Now a large number of users cannot print. Rather than manually adding all those printers back into the cluster configuration, it would be much faster to conduct an authoritative restore of the cluster configuration. This, of course, depends on you having a sound backup and restore strategy in place.

Migrating from Windows Server 2003 Server Clusters
Due to all these architectural changes in Windows Server 2008 Failover Clustering, in-place or rolling upgrades from Windows Server 2003 are not supported. When upgrading from Windows Server 2000 clusters to Windows Server 2003, many organizations would systematically evict each node in the cluster, do a clean installation of the operating system, and then add the node back into the cluster. This approach cannot be used for migrating to Windows Server 2008, as Windows Server 2003 and Windows Server 2008 cluster nodes cannot be part of the same cluster.
Fortunately, to help with the migration, a wizard-based migration process has been included. But migrating to a Windows Server 2008 Failover Cluster will, nonetheless, require some planning. There are three basic migration scenarios:
  • Using the same servers and storage.
  • Using the same servers, but using them with new storage.
  • Using new servers and new storage.
Any of these scenarios will involve making sure the hardware has been certified under the Windows Server 2008 logo program and that the Failover Cluster validation process has been run and all tests passed. Once those steps are complete, the migration process can move forward.
Not all resources in a Windows Server 2003 Server Cluster can be migrated. You can migrate network names, IP addresses, physical disks, file shares, distributed file share (DFS) roots, DHCP, and WINS. You can also migrate, to a limited extent, generic services, generic apps, and generic script resources.
Meanwhile, such applications as Microsoft Exchange and SQL Server® have their own procedures for migrating to a Failover Cluster. Printers can be migrated to Windows Server 2008 using the Print Management snap-in (which is installed with the Print Server Role) to first export and then import printers to a newly configured, highly available print server. No third-party resource types are eligible for migration.
The migration process does not migrate any data. It involves migrating cluster configuration settings from Windows Server 2003 to Windows Server 2008.
All migrated resources are initially placed in an offline state when the migration process completes. This is done because there may be additional necessary steps. Therefore, it is important to review the post-migration report to see what additional steps (exclusive of the data migration if migrating to new storage) are required prior to bringing the cluster into service. As an example, if a DHCP server is being migrated, the DHCP Server Role must be installed on all nodes in the cluster. If migrating a WINS Server, the WINS Server feature must be installed on all nodes in the cluster.

Chuck Timon is a Support Escalation Engineer at Microsoft supporting Cluster and Setup technologies He wrote the Windows Server 2008 Failover Cluster training and is now working on the Hyper-V training.


From the Editor Print is Dead
Joshua Hoffman

Print is dead. Or so many would have you think. In the era of unlimited information, available in every form, at any time, from anywhere, on myriad fantastical devices, what place does a collection of pieces of paper bound together with glue really hold? Has it's time passed? Should the magazine and the newspaper be relegated to the halls of the Smithsonian, so that one day we can wax nostalgic about what it was like to have to carry around compressed bits of tree and ink?
A bit melodramatic? Yes. But I'm sure you've guessed that this is a premise with which I completely disagree (and not only for the sake of my job security; there are plenty of things I could be doing at Microsoft). I believe firmly in the notion that the printed word provides an incredibly deep and rich experience, despite (or perhaps because of) its simplicity. When I sit down to read a long article, I would always rather do so with the print in my hand instead of on my computer screen. To me, it's simply more engaging. Is that an antiquated viewpoint? Perhaps, but according to our latest reader survey, 90 percent of you feel exactly the same way.
Now, I'm not suggesting we stick our head in the sand. I'm as geeky as they come, and I love technology. But a publication such as ours can—and should—be a holistic experience, one that takes many forms—online, offline, and in person. Sure, if I'm going to read a long, deeply technical article by Mark Russinovich, I do want it to be in print. But I also want to benefit from hearing Mark lecture and watching him demonstrate the technique he's discussing.
This notion has become a guiding principle for TechNet Magazine. While we remain proud of the publication we ship to the printer ever month, all of the content we've ever published in print is also available on .com. And in the near future, you'll be seeing more content that takes advantage of the unique platform available to us in the Internet. For example, in our last issue we kicked off our Screencast series with Don Jones and his Windows PowerShell column. You can now watch video of Don as he walks you through the topic of that month's column. There's also a new Sustainable Computing column on our Web site that is devoted entirely to the environmental impact of IT and what we as IT professionals can do to be responsible global citizens.
As always, your feedback is critical as we continue to build TechNet Magazine into a rich media experience. Drop us a line at tnmag@microsoft.com.
—Joshua Hoffman

Thanks to the following Microsoft technical experts: Ed Banti, Avronil Bhattacharjee, Nino Bilic, Mai-lan Tomsen Bukovec, Eric Charran, Elden Christensen, Trina Gorman, Scott Landry, Michael Murgolo, Sanjeev Nair, Hugh Pyle, Amritansh Raghav, Jez Sadler, and Jim Truher.


Letters Readers Speak Out

Active Directory Domain Services
I have definitely enjoyed the March 2008 issue of TechNet Magazine. I think I've read it about 12 times by now! In particular, I really enjoyed the article "What's New in Active Directory Domain Services," by Gil Kirkpatrick. In the article, Gil mentions a product called Microsoft® Audit Collection Services. Is this product available today, and does it work as a standalone application?
—Erasmo M.

Microsoft Audit Collection Services (ACS) is part of System Center Operations Manager (SCOM), the management software formerly known as Microsoft Operations Manager (MOM). It's not available as a standalone product, only as part of SCOM.
—Gil Kirkpatrick, NetPro CTO and TechNet Magazine Author

Signatures
In the second paragraph of the Windows PowerShell column from the April 2008 issue of TechNet Magazine ("Sign Here, Please," technet.microsoft.com/magazine/cc434702), author Don Jones states, "The word signature is the correct technical term for what I'm talking about, though the word itself isn't really a good description of what is actually happening here. Signing a script doesn't mean you're approving or authorizing it, as you might do with a contract or a credit card slip."
I would argue that signing a contract or credit card slip doesn't necessarily approve or authorize those items either. The action performed is only meaningful in the context of a larger view of the environment. For instance, when I was younger, I played with credit card slips at my father's service station—never did anyone attempt to collect on debts that I incurred in those transactions.
In the context of a "properly configured" Active Directory® network implementation, the signature is exactly what authorizes the script to operate. The presence of the signature indicates that the person signing is accepting responsibility for the thing signed, and the presence of the signature may then trigger further activity—billing by the credit card company or acceptance of code as authentic by the network.
Please keep up the good work folks— TechNet Magazine is very beneficial.
—Jeremy P.

Thanks for the feedback, Jeremy. It's nice to know that our readers pay such close, critical attention to what TechNet Magazine has to say.

SharePoint Services
When running Windows® SharePoint® Services (WSS)—not Microsoft Office SharePoint Server—are additional CAL licenses required for users accessing it through a Web site interface/URL?
—Douglas M.

There are a few documents available regarding this topic. Here are some helpful links: www.microsoft.com/windowsserver2003/howtobuy/licensing/caloverview.mspx; microsoft.com/windowsserver2003/howtobuy/licensing/extconnector.mspx; and technet.microsoft.com/windowsserver/sharepoint/bb684457.
—Pav Cherny, Columnist, Inside SharePoint, TechNet Magazine

Inside SharePoint Idea
I recently read the SharePoint column in the May 2008 issue of TechNet Magazine (technet.microsoft.com/magazine/cc462809). You asked readers for suggestions on topics. A big article on integrating SharePoint with Project/Project Server would be wonderful.
—Phil H.

Project/Project Server is a great idea! We will definitely be considering this topic for a future column. Thanks for responding.

Other Inside SharePoint Ideas
The Web as a whole has a wealth of documents regarding site customization using either the standard SharePoint UI or SharePoint Designer, but there is very little regarding Site Definitions (and the articles that are out there are so basic that they are of little use when faced with a real-world problem). As this is an area many people are beginning to explore, a wealth of how-to's can only help the adoption of SharePoint. In particular, at my company we have struggled with the following topics:
  1. Setting custom permissions to lists within a Site Definition.
  2. Custom list item forms within a Site Definition.
  3. Web part connections from within a Site Definition.
  4. How a Site Definition could be delivered incrementally.
—Adam C.

Thank you for your suggestions. This is exactly the kind of reader feedback we'll be taking into account as we continue to develop this content.


Toolbox New Products for IT Pros
Greg Steen

Automate Admin Tasks
Automise 2
automise.com/automise.aspx
Repetitive tasks that require no thought are boring, laborious, and prone to error. Automating these types of tasks is a key to successful administration and can improve job satisfaction. Aside from using dreary batch files and custom VBScripts to automate your tasks, there are a number of macro-like utilities for Windows® and a few utilities that even force you to learn some sort of custom scripting language. But there is another option. Automise 2, from VSoft Technologies, lets you generate automation routines through a drag, drop, and customize style GUI.
The application's interface is much like a simple programming IDE, offering familiar options. You get a toolbox from which you drag items onto your designer surface, debugging controls that allow you to step through your code, a view of your different projects, output run logs, history, and property windows. But with Automise, your "code" is really a visual representation of your automated task's workflow.
Automise's toolbox consists of actions, each of which performs a single step in the automation process. There are 380 actions included with the application, covering the gamut of possible operations from creating .zip files to restarting IIS.
When you design your task, you can hierarchically structure these actions to cover many of the administrative processes you perform regularly. In addition, you can connect your actions by flow control actions, such as try/catch blocks, exception raising, if-then-else statements, loops, switches, delays, and asynchronous action groups.
Each of your automation projects can also include other projects so you can separate your master flow of control from its subtasks. There are also actions for manipulating strings and XML data, setting variables, waiting on processes and commands, and much more. With all these flow of control options, even complex operations are in reach of being automated.
Automise lets you create, run, edit, and delete Scheduled Tasks. This means you can use Automise to manage your existing processes as well. I don't have enough space to list all the actions, but some other notables are: actions to register a DLL in the Global Assembly Cache (GAC), rebuild SQL Server® indexes, run chkdsk, edit the registry, back up the event log, edit Active Directory® users, mount volumes, create restore points, and create a virtual directory. And, if the built-in actions can't do what you want, Automise also has custom script, command, and ad hoc SQL execution actions.
All in all, this is a handy tool to have at your disposal. And if you are really just looking to automate your own Windows workstation, as opposed to automating the network infrastructure, take a look at Automise Lite, which trims out a number of features but will handle most of your power-user needs.
Price: Single-user license starts at $195 (direct).
Automate tasks visually with Automise 2

Monitor Logs in Real time
BareTailPro
baremetalsoft.com/baretailpro
Log files, log files, everywhere! But using them, that's the real bear. It's great that you have your application spewing out a log of transactions. But in addition to using your log-aggregation tools, how can you really watch what is transpiring in real time? One solution is Bare Metal Software's tried and true BareTailPro. Though not updated since 2006, this simple utility still remains an enhanced Windows analog to the old console *nix command 'tail -f'.
With its straightforward GUI, you can watch multiple log files of any size simultaneously, transparently watching what is written to them in real time. To switch among files, you just click on the tab. And, a very nice feature, each tab displays an icon to signal of the state of each log file, indicating whether the file is being written to, or the "tail" has been stopped.
Though most applications log, how they log is usually quite different. Fortunately, BareTailPro can read Unicode, UTF-8, ANSI, and ASCII files, and can handle when their lines end in CR/LF pairs, single LFs, and nulls. You can easily configure both line wrapping and tab expansion to give yourself the best view for the particular type of log file. In addition, you can enhance the readability of the log file by configuring the font size, type, spacing, and line height.
Another great feature is the application's configurable highlighting that you can apply to a log view. You can set any number of foreground/background coloring coding rules based on a string of text. For example, you can highlight any error in red and warning in yellow, or you can choose to highlight all lines that have a specific IP address or host reference. You can spec the lines as bold or italic and you can choose to ignore the casing of your string of text.
Even with text highlighting, an aggressive log file can overwhelm your tail view, making it difficult to spot the events you are looking for. BareTailPro offers another feature that can help. Regular expression and text search functionality let you quickly trim down your log view, isolating the row instances of concern into a manageable subset. The regular expression syntax used by BareTailPro is made up of a subset of the standard notation used by Java, Perl, and PHP. In case you're not familiar with that usage, the product Web site provides a quick reference.
The utility shows your filtered results in a tabular format, adding a few useful columns such as a timestamp, the line number, and the results of any capturing groups you may have defined along with the original line source. In the filtered view, these columns are also sortable.
I can never remember the syntax of the "magic" regular expression that got me the results I needed. That's why I really appreciate the tool's pattern save feature. This lets you name, edit, and save patterns for reuse. Once you do have your important rows in your filtered view, BareTailPro also lets you export those results to a file or to the clipboard with a tailored format, allowing you to set things like the width, alignment, prefix, and suffix to each of the results columns.
Impressively, this tiny utility is contained within a single small executable, making it portable and being low-impact on your environment. Note that if you just need to watch your log files and don't require the regular expression filtering, saving of search patterns, and exporting of search and filter results, you may get by with the free version, named BareTail.
Price: Starts at $35 for an individual license.
Monitor your network with BareTailPro

Inventory your systems
Total Network Inventory
softinventive.com
Having quick access to detailed information about the machines throughout your environment is a key part of successful systems management. Spreadsheets and vendor listings of this information are easy to lose track of, hard to keep complete, and quickly fall out of date.
It can be very helpful to use a dynamic tool that presents the info "as-you-need-it," gathering information about who has what installed and what kind of hardware they are running. A tool like this also offers a great way to keep tabs on what extras users may have put on their machines. (This, of course, is if your group policy doesn't already limit installation rights or provide quick tallies on how many licenses you should have based on the installation base for a particular software package.) One tool that fits this bill is Total Network Inventory from Softinventive Lab.
After installing Total Network Inventory, you can start the inventory process using the Scan wizard, which guides you through node discovery in your network. You can choose to scan immediately, via a specified IP address range or Network Places computer browsing, or you can use a logon script scan, which lets you collect information as a machine logs onto the domain.
Total Network Inventory relies on Windows Management Instrumentation (WMI)—either IPC$\ADMIN$ or RPC/DCOM—and classic sharing for local accounts to perform its scanning. So you need to run it under the context of an Administrator account.
Once your scan is complete, you can then browse through the machines or search for specific computers to inventory their assets. Total Network Inventory groups the collected inventory information into three areas: hardware, software, and other information. The hardware portion includes details about the processor, motherboard and chipset, memory, video subsystem, storage (including physical and logical devices), network adapter types and configurations, peripherals, and any other devices installed in the system.
The software section provides OS information, including product keys, serial numbers, system directories, and installed services packs. There are details about hotfixes (with their respective Knowledge Base article numbers) and any antivirus and database apps that may be installed. And you get a list of programs that are installed (along with a view into the Program Files directory, so you can see applications that may have been hidden from the standard Add/Remove Programs listing), as well as a listing of any autorun applications.
Finally, the "other information" section wraps up details on the system environment, such as user and system variables, shared resources, running processes, services installed, and local user accounts.
Total Network Inventory really gives you the information you need for performing asset audits, licensing checks, upgrade calculations, and performance evaluations. One of the best parts of this tool is the Report Builder component. This feature gives you multiple, well-organized, detailed reports for everything from a quick reference to an audit paper trail. And the reports can be aggregate or for a single machine. A feature I find particularly useful is the Software and Licenses Accounting tab view. It displays the aggregate count of each type of software installed across your environment to ensure you are in compliance with your vendor agreements.
Price: Starts at $95 for a 25-node license.
The Scan wizard guides users through network node discovery

Book Review
Mastering Microsoft Exchange Server 2007
sybex.com
Whether you like it or not, e-mail is a core component of a smoothly operating organization. (Although, I'll admit that I do sometimes wish users would be required to take a class on effective written communication before they are allowed to use e-mail.) And over the years, the desire to integrate voice communications, calendaring, resource scheduling, uniform client access, and mail-filtering (for spam, viruses, and the like) has become all too apparent. This is where Microsoft® Exchange Server 2007 enters the picture.
The latest version of Exchange smoothly integrates all these components in a scalable, robust solution. But with all the features and functionality, the administrator, architect, and support staff need a few good reference books to help them plan for, deploy, manage, and maintain such a dynamic electronic messaging solution.
In the last installment of the Toolbox column, I discussed Microsoft Exchange Server 2007 Administrator's Companion. Another good resource that is worth considering is Mastering Microsoft Exchange Server 2007 by Barry Gerber and Jim McBee (Sybex, 2007). This is a clearly written guide that leads you through the Exchange Server lifecycle.
The first three chapters of the book, a section called "Understanding and Planning," go over the basic components and implementation requirements of Exchange Server 2007. Those of you familiar with previous versions of Exchange will appreciate the section on functions that are no longer emphasized or implemented. And everyone will appreciate the delineation of the different server roles and the components that comprise an Exchange Server 2007 deployment. A deployment of such a complex server definitely requires some forethought, and you will want to pay close attention to the sections on planning for growth, disk space requirements, and Active Directory integration.
The next section goes into the actual installation, covering both a clean install and an upgrade from either Exchange Server 2000 or Exchange Server 2003 (you can't directly upgrade from Exchange Server 5.5). Here you also get more detail on the different roles each Exchange server can play, including Edge Transport, Client Access, Hub Transport, Mailbox, Unified Messaging, and Management Tools.
The book then moves onto scalability, showing you how to monitor performance, add more mailbox storage, and so on. Here, you explore the very cool new Local Continuous Replication feature, which allows you to keep an up-to-date local backup of your mail store. If something happens to the main database, you can quickly switch to the backup without having to restore from some other less available and less up-to-date sort of backup.
The next section of the book provides detailed overviews of the different management tasks and tools you get with an Exchange Server 2007 deployment. This includes such tools as the Exchange Management Shell (based on Windows PowerShell™) and the Exchange Management Console GUI (which also relies on the Windows PowerShell engine). Throughout this section, you learn how to manage mailboxes, address lists, and user accounts. You also cover messaging records management, as well as managing message classification, journaling, and transport rules.
The fourth section gives more in-depth information on availability and reliability. Now that businesses have become dependent on electronic messaging, this section is a great place to pay close attention.
Here you'll learn the basics of load-balancing, server redundancy, storage redundancy, network redundancy, and clustering. You'll learn briefly about best practices for scheduled maintenance, running backups, and disaster recovery scenarios–basically, the things you need to know to keep your company's electronic messaging systems functioning efficiently even when problems arise.
The next two sections of the book cover client access including Microsoft Office Outlook® 2007, Outlook Web Access, IMAP, and POP3. And finally, the book ends with a section, all too important to the Exchange administrator, regarding security and auditing, showing you basic security practices and precautions along with the different auditing tools at your disposal. All in all, this in-depth reference will get you up to speed and get your Exchange deployment on the right track.
Price: $49.99 (direct).

Greg Steen is a technology professional, entrepreneur, and enthusiast. He is always on the hunt for new tools to help make operations, QA, and development easier for the IT professional.


Sustainable Computing Is It Time to Turn Off Your Servers?
Dave Ohara

Contents
Weill Medical College of Cornell University High-Performance Computer Cluster
Citrix Presentation Servers
Microsoft Research
Commercial Solutions
Tips to Get Started
Wrapping Up
One of the simplest ways to achieve energy savings on desktops and laptops is to turn them off (or even use the sleep or hibernation feature). Users have accepted the idea that turning off their devices is a small step that can have significant results. But the idea of turning off your servers—that's a pretty new topic that people
haven't entirely embraced yet. Currently, only small numbers of IT departments shut down servers, but this practice is starting to change. More organizations are taking advantage of the energy savings of server shutdown.
Why would you turn off a server? In general, an idle server consumes about half the amount of power consumed by a server running under high load. Server utilization for most data centers runs in the range of 10 to 20 percent, which means there are a lot of machines idling (systems using energy, but not being used).
Of course, the idea of turning off servers raises some important questions. I'm not suggesting that you shut down Web servers and other critical servers that should be running 24 hours a day, 7 days a week. So what type of machines can you turn off? To help give you an idea, I'd like to share some real-world examples of what some organizations have implemented to cut down on system idle time and reduce energy consumption.

Weill Medical College of Cornell University High-Performance Computer Cluster
For more than a year, the Institute of Computational Biomedicine at Weill Medical College of Cornell University has been turning off servers in its high-performance computing (HPC) cluster. The HPC cluster resources are used for simulation of biological systems, analysis of large datasets, and imaging/visualization.
The compute nodes use 250 watts when in use and 125 watts when idle. The system administration team set out to create a cluster power management solution that would shut down nodes that were not in use while minimizing impact on performance. The group minimized the impact by maintaining a target number of stand-by nodes ready to run new jobs immediately, reducing latency perceived by end users.
One of the biggest challenges in turning servers off is the method used to turn the servers back on. Fortunately for the Cornell team, all of its compute nodes are from one vendor, thus the team could use baseboard management controller (BMC) features and a remote access card to control power on events. On top of BMC, the team used Intelligent Power Management Interface (IPMI) to build a complete cluster power management solution, managing the server states of In Use, Standing By, Shutting Down, Power Save, and Powering Up.
With the clusters' high power density in mind, the team also added the capability to monitor temperature per server, measuring temperature fluctuations as machines run under load, idle, and powered off. Over a six-month period, 16 percent of the managed nodes were shut down, resulting in a power savings of 8 percent.
In the process, the team discovered its lowest energy consumption is in the late morning, and its peak load is in the early evening. This, the team realized, was due to researchers running their biggest jobs at the end of the day before leaving the office and their smaller jobs in the morning when a quick turnaround is priority.

Citrix Presentation Servers
Citrix PowerSmart Utility for Presentation Server is a solution that helps customers reduce power consumption by powering off idle Presentation Servers during off business hours. The tool, which is currently in beta, offers an innovative method for administrators to proactively reduce Presentation Server power consumption, and it is suited to IT professionals who are interested in building "green" IT solutions.
(Note that at the time of this publication, this tool in not yet part of the Citrix product offering, and Citrix provides forum support only.)
The utility is initially designed to work with HP servers with iLO2 (integrated lights out) support. However, support for other types of servers can be added with scripting and simple configuration changes.
I find this solution worth discussing because it provides a good explanation of how turning off servers can be done. Here are the key elements that make this solution work:
  • One presentation server is designated the controller. Few files are added, no registry changes are needed, and uninstall is supported.
  • A configuration file on the controller identifies the group of servers to be power managed.
  • Off business hours are defined to specify when the servers can be turned off.
  • The frequency for checking server status and time is defined.
  • Load balancing rules are configured to prevent connections to turned-off servers.
  • User activity is monitored during off business hours, and servers with no active sessions are turned off.
  • Servers are turned on automatically during operational business hours.
  • All power events are logged to the controller event log to provide a history.

Microsoft Research
Microsoft Research has investigated methods for turning off idle servers. The group published an interesting paper called "Power-Aware Server Provisioning and Load Dispatching for Connection-Intensive Internet Services" (available at research.microsoft.com/research/pubs/view.aspx?0rc=p&type=technical+report&id=1373&0sr=p).
What I find particularly interesting about this paper is that it focuses on how servers can be turned off in a connection-intensive Internet Service environment—Windows Live™ Messenger. In this paper, the authors offer up server provisioning and load dispatching algorithms, studying the subtle interactions between values. The end result is Microsoft Research was able to show energy savings in a range of 20 to 30 percent, depending on the number of disconnections initiated on the servers.
Another paper published by Microsoft Research, "Write Off-Loading: Practical Power Management for Enterprise Storage" (available at research.microsoft.com/camsys/paper-final.pdf), discusses the idea of saving power in storage by spinning down hard drives. To perform this research, the team observed 36 volumes in an enterprise data center to identify the idle periods. The general perception is that idle periods are too short to allow for spinning down the disks. The researchers, however, discovered significant idle periods, and they could further increase the idle time by modifying the read/write patterns using write-off loading. Write off-loading allows write requests on spun-down disks to be temporarily redirected to persistent storage located elsewhere in the data center. The results showed a 28 to 36 percent energy savings resulting from spinning disks down. And the use of write-off loading increased the energy savings to between 45 and 60 percent.

Commercial Solutions
Companies such as Verdiem and 1E provide software solutions for managing desktop power consumption. And at least one company, Cassatt, has a solution for servers. Cassatt Active Response provides power management for physical and virtual server resources. It uses a policy-driven control to intelligently and systemically turn off servers. Its policies can be based on load, time, specific events (such as cooling unit failure), and power capacity changes (such as switching to a back-up generator).

Tips to Get Started
If you are interested in investigating server shutdown as a power-saving strategy, be sure to plan wisely and test before jumping in. Here are some tips to get you on your way:
  1. Start with a small step and turn off the servers in your development and test labs. Use this opportunity to get comfortable with how to do this in a non-critical environment.
  2. Avoid Wake on Lan. This is a 15-year-old feature that has been found to be unreliable in server environments.
  3. Sleep and hibernate are new capabilities for servers. You may have used these features successfully on a desktop or laptop running Windows Server,® but hardware support may be different on your server class machines. Check for support on your specific hardware before implementing these features on your servers.
  4. Windows Server 2003 and Windows Server 2008 will not hibernate on machines with more than 4GB of memory due to poor performance. You can find more details on this in the Knowledge Base article available at: support.microsoft.com/kb/888575.
  5. Turning off servers with scripts is the easy part; turning them on is the difficult part. One method is to use BMC support. Another approach is to use the remote power management solution offered on power strips from such companies as Server Technology. These solutions allow you to reset power over an IP connection, forcing a server to turn on.
  6. Be sure to design a solution that is automated and reliable. You don't want this to be a manual task, nor do you want it prone to human error.

Wrapping Up
Just as hybrid cars and compact fluorescent lightbulbs were once thought to be eccentric practices but became mainstream solutions, so will the concept of powering down servers.
As of today, turning servers off and on automatically is difficult to do, but with growing interest in environmental sustainability and a need to reduce rising energy costs, more people and businesses are motivated to find and develop solutions that work. After all, there really is no justifiable reason to leave all our servers running all the time, especially when they are not being used.
Be brave and take a first step yourself. Look analytically at the machines throughout your environment to see which ones can be turned off safely. And then institute a means to do it.

Dave Ohara has 26 years of experience in technology. He is now working with multiple companies implementing green initiatives.


Utility Spotlight Microsoft SharedView
Greg Steen

Code download available at: UtilitySpotlight2008_07.exe (150 KB)

I frequently work with teammates on the other side of the planet, and I constantly encounter situations where, if everyone could look at and interact with the same desktop, then we could all illustrate and communicate our opinions so much more effectively.
Video
Watch Greg Steen demonstrate how you can quickly share a desktop and collaborate with remote users using Microsoft SharedView.

Suppose your sales team is out on the road and needs to quickly get together to share information, verify quotes, and reduce overlapping effort. Think of how much more productive it would be if everyone on the team could look at and edit the same Microsoft® Office Excel® spreadsheet or Word document almost simultaneously, rather than e-mailing around a "living document" that can quickly get out of sync. Now there's a lightweight collaboration tool that allows just that capability—Microsoft SharedView 1.0.
Microsoft SharedView lets you get together with up to 15 people and share the desktop of the meeting host simultaneously, just as if you were looking at the same desktop. Your first thought may be that if 15 people had simultaneous control over the same desktop, absolutely nothing but chaos would ensue, but what really happens is that control of the desktop can be passed from one member to another.
While you're the one who's in control, you can see your collaborators' mouse movements (everyone gets a personalized pointer), so they can point out items on a document or application. Unfortunately, SharedView doesn't have integrated audio capabilities, but you can use Windows Live™ Messenger's audio client or your favorite Voice over IP (VoIP) client to conference while you share the desktop.
Microsoft SharedView lets you share the desktop of a meeting host simultaneously
Installation of SharedView is simple, and the client is intuitive enough that even less technically savvy users can get up and running quickly. After signing in with your Windows Live account, you can start a SharedView session with a couple of clicks. Joining a session is just as easy; you simply type in the session name (usually the Live ID of the host) you wish to connect to along with a password. Invitations are sent via e-mail or by pasting the generated invite text into your IM client.
Other options within SharedView include enabling or disabling remote copy and paste (so session users can add text from their local workstation to the host's machine); directly enabling track changes in Word so you can see how people are changing the shared document; and an explicit block list to exclude certain Windows Live IDs from participating. The application also has a cool Handout feature that lets you make supplemental files available to session participants; you can have up to 25 files or a total of 200MB per session.
In addition, you can type broadcast messages to participants that will appear at the top of their connected session. You can choose to share everything if you want or, if you'd rather keep some things private as the meeting goes on, you can quickly and easily start and stop sharing windows from your host machine. All in all, if you are looking for a quick and easy collaboration tool, Microsoft SharedView will probably fit the bill.
To get started, you can just go ahead and download Microsoft SharedView from the Code Downloads section of the TechNet Magazine Web site at technetmagazine.com.

Greg Steen is a technology professional, entrepreneur, and enthusiast. He is always on the hunt for new tools to help make operations, QA, and development easier for the IT professional.


Inside SharePoint Integrating Office Applications
Pav Cherny

Code download available at: SharePoint2008_07.exe (8,622 KB)

Contents
Office Application Integration
Extending the SharePoint UI
Using the Office OpenDocuments Control
Communicating with SharePoint
Implementing a Custom OpenControl Solution
Summing Up
Microsoft Windows SharePoint Services (WSS) 3.0 and Microsoft Office SharePoint Server (MOSS) 2007 integrate seamlessly with desktop applications in the 2007 Microsoft Office system, facilitating collaboration on documents, spreadsheets, calendars, contact information, and more. In fact, the integration is so seamless you
can say that the 2007 Office system is a unified office solutions platform.
This is great for organizations that focus on Microsoft® Office technology to boost information worker productivity. However, organizations with a diverse portfolio of Office applications cannot enjoy the same level of interoperability out of the box. The 2007 Office system provides the necessary integration capabilities, yet the interfaces and components are minimally documented and don't work under all circumstances. This makes it hard for third-party vendors to integrate SharePoint® technology into their applications, and even harder for their customers to provide information workers with a unified experience.
In this column, I'll show you how Office applications integrate and communicate with SharePoint, and how you can integrate non-Microsoft applications with SharePoint based on the same principles. But first, I'll briefly discuss the server configuration settings, client-side components, and communication protocols used to achieve seamless integration.
Once these details are clear, I can approach the integration of apps that come without built-in SharePoint support. In this effort, I'll go beyond the usual level that typically focuses on implementing IFilter components to extend search capabilities and add custom icons on SharePoint servers. Extending search and returning results with proper icons does not achieve full app integration; users might also want to open these documents directly from within the SharePoint user interface.
This is where it gets complicated. You need a client-side component that isn't readily available if you haven't deployed Office on your workstations. But even if you have deployed Office, you might find that this component does not work reliably, depending on your SharePoint site topology and configuration.
To solve this problem, I created a custom solution that lets you integrate any application, such as Notepad, Adobe Reader, or Autodesk AutoCAD, without Office deployment requirements. This custom solution also illustrates why application integration based on standard Office components fails in some situations. You can find this solution, as well as step-by-step deployment and configuration instructions, in the companion material for this column, available in the Code Download section of .com.

Office Application Integration
From the user perspective, SharePoint menus appear to work like the Windows® Start menu. Creating a new document in a document library is simple: click on New, then on New Document, and Microsoft Office Word 2007 starts. Editing an existing document is similarly straightforward. Hover over the document, open then Edit Control Box (ECB) dropdown menu, and then click Edit in Word. It seldom comes to mind that you started Word 2007 from a Web page through JavaScript, that you are running the application locally while the document resides far away in a SQL Server® database, and that a Web server exists within the data access path, as illustrated in Figure 1.
Figure 1 Working with a Word document in SharePoint
Despite this complexity, the user experience on a Windows workstation with the 2007 Office system installed is seamless. Working with documents in a SharePoint library is pretty much like working with local files or files on a network share.
However, on a Windows workstation without Office 2003 or the 2007 Office system, the UI is different. Clicking on New Document or Edit in Word only brings up a dialog box informing you that a Windows SharePoint Services-compatible application is not available.
This isn't really a surprise. There are several elements that must work together to provide the unified experience that users of Office enjoy. SharePoint must be aware of the content types maintained in the document library to render the desired commands in the New and ECB menus. In response to mouse clicks on these commands, JavaScript code must run to start the associated application, passing along the path to the document. This part is not independent of the workstation configuration because the JavaScript code runs locally. Furthermore, the associated application must communicate with SharePoint to read and possibly write files. These are the elements that you must bring together if you want to integrate your applications with SharePoint.
On the other hand, communication between SharePoint and the database server is transparent to the application, and so are the indexing processes on the Web server to facilitate searches. For this reason, I will not discuss these aspects further in this article, but I recommend that you check out the Microsoft Filter Pack documented in the Microsoft Knowledge Base article "How to Register Microsoft Filter Pack with Windows SharePoint Services 3.0," available at support.microsoft.com/kb/946338. Now, let's focus our attention on adding commands, implementing necessary client-side components, and facilitating application communication.

Extending the SharePoint UI
An abundance of options are available to extend the SharePoint user interface and functionality. You can change the design of your sites, customize ASP.NET pages, develop Web Parts, or change the JavaScript code included in WSS and MOSS to start your applications directly.
There's nothing to stop you from opening the Ows.js file within a text editor (the file can be found in the COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\12\Template\Layouts\1033 folder on your SharePoint front-end servers) and changing the way the createNewDocumentWithProgIDCore, editDocumentWithProgIDNoUI, and DispDocItem functions work. But there's a better method that doesn't require you to change the WSS code base in unsupported ways: using content types and document type mappings.
In my article "Standardize Data Management with Custom Content Types" (see TechNet Magazine, February 2008, technet.microsoft.com/magazine/cc194408.aspx), I explained how to create global and site-specific content types for managing documents and other content in SharePoint document libraries and lists. You can also check out the worksheet Text Content Type.pdf in this article's companion material, which illustrates how to create a new content type for .text files and associate this content type with a document library. The result is that you can select the Text Content command from the New menu just as you can select the Document command for Word documents (see Figure 2).
Figure 2 A custom content type in the SharePoint UI
The ECB menu is likewise extensible. You might have expected the ECB menu to be content-type aware, but that's not how the current WSS version implements this menu. Instead, you must register your document type in Docicon.xml, which you can find in the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\12\Template\Xml folder on every SharePoint front-end server.
By adding the following document type mapping to the <ByExtension> section of Docicon.xml, for example, you will cause SharePoint to display the Edit in Notepad command (see the Text Content Type.pdf file for more detailed instructions):
<Mapping Key="text" Value="ictxt.gif"
  EditText="Notepad"
  OpenControl="SharePoint.OpenDocuments"/>
The Key parameter identifies the file name extension, the Value parameter defines the document icon to display in the user interface, the EditText parameter defines the string that SharePoint appends to the "Edit in" command, and the OpenControl parameter specifies the ProgID of a client-side COM component. This is the ProgID that the SharePoint JavaScript functions pass to an ActiveXObject call (see Ows.js for details) to instantiate the COM object, which can be the app itself or a helper control that starts the app based on the associated file type.
SharePoint Products and Technologies Web Site

  • SharePoint Products and Technologies Web Site
  • Windows SharePoint Services TechCenter
  • Windows SharePoint Services Developer Center
  • Microsoft SharePoint Products and Technologies Team Blog
You should note that the OpenControl that's named SharePoint.OpenDocuments is referring to an ActiveX® control that ships with recent versions of Office (%PROGRAMFILES%\Microsoft Office\Office12\Owssupp.dll). If this file exists on your workstations and if the specified document icon (Value parameter) is in the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\12\Template\Images folder on your SharePoint servers, you might have already accomplished most of the necessary integration work.
The Windows SharePoint Services 3.0 SDK includes some information about the client-side APIs that ship with the 2007 Office system, including the OpenDocuments control. See the section "Client-Side API Reference" at msdn2.microsoft.com/ms440037.

Using the Office OpenDocuments Control
The OpenDocuments control addresses the most vital application integration needs, but it requires Office 2003 or the 2007 Office system, and its capabilities are somewhat limited. The commands on the New menu may not always work, and they sometimes display what might be misleading user notifications.
As illustrated in Figure 3, the OpenDocuments control informs the user that the required application is not correctly installed or that the template file cannot be opened, but neither is the case. The Edit in functionality also has issues. The second error message shown in the foreground of Figure 3 is frequently seen. It is my all-time favorite because it misleads even SharePoint experts, but I'll tell you more about it in a moment.
Figure 3 Misleading error messages of the OpenDocuments control
Nevertheless, the OpenDocuments control is useful in environments with recent versions of Office installed on all workstations. Among other things, you can hide your content types from the New menu (display the Document Library Settings, click on Change New Button Order And Default Content Type, and then uncheck the Visible checkbox for all content types in question), so that users won't encounter the first error message. You can also keep your site topology simple in order to avoid Web-Based Distributed Authoring and Versioning (WebDAV) communication problems; this eliminates that second error message.

Communicating with SharePoint
So far, I have managed to extend the SharePoint UI and perhaps start an application by means of the OpenDocuments control, but my application still needs a way to communicate with SharePoint to access the data. As always, SharePoint supports multiple methods to do this, such as Microsoft Office FrontPage® Server Extensions, WSS remote procedure calls (RPCs), WebDAV, and Web services. In fact, Office apps, such as Word 2007, might use any or all of these communication methods, depending on how you access a document, such as through Web folders, mapped network drives, or the SharePoint UI.
The SharePoint client/server communication methods each rely on HTTP as the underlying protocol. For example, FrontPage and WSS RPCs all use HTTP POST and HTTP GET requests directed at ISAPI extensions that reside on SharePoint servers within the %COMMONPROGRAMFILES%\Microsoft Shared\Web Server Extensions\12\ISAPI folder and all its subfolders.
One of the most important ISAPI extensions is Owssvr.dll, which implements, among other things, functionality to work with lists and document libraries. Figure 4 shows the SharePoint Save dialog box in Word 2007 (left side) and in a browser (right side) opened directly through the URL request http://sharepoint/HR/Administration/_vti_bin/owssvr.dll?dialogview=FileSave&location=Shared%20Documents&FileDialogFilterValue=*.docx. The similarities between the two screenshots are obvious.
Figure 4 The SharePoint Save dialog box in Word 2007 and in Internet Explorer
Other important ISAPI extensions are Author.dll, which implements FrontPage and WSS RPCs for client-side editing operations, such as uploading, renaming, and deleting documents; Admin.dll to manage sites and perform a number of other administrative tasks; and Shtml.dll to support HTML form submissions.
It is generally impossible to add FrontPage and WSS RPC support to existing applications, such as Notepad or Adobe Reader, without having access to source code. But you could provide the necessary communication facility by means of the Web Client feature included in Windows.
SharePoint also supports WebDAV through Httpext.dll, which resides in the %WINDIR%\System32\Inetsrv\ folder, but let's stay on the client side. Computers running Windows Server® 2008 or Windows Vista® have the Web Client feature installed by default. You can find a corresponding WebClient service in the Services applet in Administrative Tools in the Control Panel. On Windows XP and Windows Server 2003, you must install the Web Client explicitly. In any case, make sure the WebClient service is started and the Startup type is set to Automatic.
Figure 5 is a depiction of the architecture of the Web Client. The WebClient service is implemented in a user-mode DLL (Webclnt.dll) the Service Control Manager loads in a Svchost.exe host process. Webclnt.dll provides the Network Provider interface for non-I/O operations (such as authenticating the user for WebDAV access; mounting SharePoint sites as network drives; enumerating SharePoint sites, lists, and document libraries as network resources; and disconnecting mounted drives).
Figure 5 The WebDAV Client Redirector architecture
To perform this work, Webclnt.dll communicates with a kernel-mode file system driver that provides the actual redirector functionality. The WebDAV Client Redirector Driver (Mrxdav.sys) is based on the Redirected Drive Buffering Subsystem (RDBSS), which integrates with I/O Manager and other kernel components in order to provide the services of a remote file system. Mrxdav.sys implements the WebDAV communication facility to support file system-level access to SharePoint sites and document libraries.
Having access to SharePoint sites and document libraries through a network redirector eliminates the need for FrontPage and WSS RPC support in user applications. You can map network drives to document libraries (such as net use x: http://wss/doclib/Shared%20Documents), and you can access SharePoint resources through UNC paths as well.
The URL http://wss/doclib/Shared%20Documents maps to \\wss\doclib\Shared%20Documents. So you have choices to open a document in an application. For example, you can open a document in Notepad with the HTTP path http://wss/doclib/Shared%20Documents/New%20Text%20Document.txt or the UNC path \\wss\doclib\Shared%20Documents\New%20Text%20Document.txt.
Unfortunately, the Web Client feature has several limitations. You cannot access custom properties or Web applications that use custom TCP ports. The Web Client that ships with Windows Vista also fails if the user does not have access to a parent site in the hierarchy, as illustrated in WebDAV Access.pdf (see the companion material).
The path http://sharepoint/HR/Administration/Shared%20Documents/ includes a nonexisting root site (that is http://sharepoint), yet without access to the root, the Web Client cannot determine the capabilities of the Web server. The Web server rejects the Web Client's OPTIONS request with Status Code 401, which states that access is unauthorized and consequently the Web Client continues to prompt for user credentials, as shown in Figure 6, even though the user has administrative access to the site collection sharepoint/HR and all sites in it.
Figure 6 Unsuccessful WebDAV access
When using the OpenDocuments control, you receive the error message displayed in Figure 3 if the Web Client fails to open a document. The application is available and the document type mapping is correct. It is the document that cannot be accessed through the WebDAV redirector.

Implementing a Custom OpenControl Solution
Generally, you have two options to address the Web Client shortcomings. You can wait until Microsoft provides an updated Web Client version, or you can implement a custom OpenControl solution that can deal with the current situation. Implementing a custom OpenControl is not a trivial task, but it eliminates the need for Office on your workstations, it enables you to handle the New command in addition to the Edit in command meaningfully, and it allows you to address situations when the Web Client fails.
If any of these issues are compelling to you, take a look at the AppStart source code included in the companion material. It shows you how to expose the OpenControl COM interfaces in a Microsoft .NET Framework assembly, which the SharePoint JavaScript code can call. The AppStart source code also demonstrates one possible way to check file accessibility and download a file to the local computer via HTTP if direct access through WebDAV is not possible. Finally, the AppStart source code responds to the New command by downloading the template associated with the content type to the local computer so that the user can start working on the document. The worksheets Text Content Type.pdf and Adobe Reader Support.pdf outline how to deploy this OpenControl solution.
The diagram shown in Figure 7 shows the AppStart architecture. My custom OpenControl component (called Biblioso.dll) exposes two identical COM interfaces that SharePoint JavaScript calls to create new documents or to open existing documents for editing (Biblioso.AppStart.2 and Biblioso.AppStart.3).
Figure 7 The AppStart architecture
If a document is opened for editing, Biblioso.dll verifies that the file exists and starts the associated application along with the path to the document if the file is directly accessible through WebDAV. If the file is not accessible, Biblioso.dll starts an out-of-process COM server that in turn loads OpenDocsUtility.dll to download the file through HTTP and start the application along with the path to the downloaded document.
The out-of-process COM server enables the solution to break out of the Internet Explorer® process, which restricts downloads to the Temporary Internet folder in Protected Mode. Users must be able to download files without the Protected Mode restrictions, and an out-of-process COM server working as an application broker provides this option.
Developing out-of-process COM servers in .NET is not supported, so I switched to C/C++ for this executable. I only implemented the bare minimum Save As dialog box in C++. To keep the solution as straightforward as possible and to keep development overhead low, I placed the actual download code in a .NET assembly (OpenDocsUtility.dll), which I then call through another COM interface.
In order to facilitate deployment, I added a Setup project to the solution. Among other things, the setup routine registers all COM components and writes application-specific settings to HKEY_LOCAL_MACHINE\SOFTWARE\Biblioso\AppStart. The most important parameters are both AllowedApps and AllowedFileTypes. The AppStart solution will only work with those applications and file types that you must explicitly specify in these parameters.
The setup routine also creates an elevation policy for the out-of-process COM server so that Biblioso.dll in the Internet Explorer process can start AppBrokerEngine.exe without triggering security warnings. If you are interested in learning more about Internet Explorer Protected Mode and how to deal with it in application development, I highly recommend Marc Silbey and Peter Brundrett's article "Understanding and Working in Protected Mode Internet Explorer," available at msdn2.microsoft.com/bb250462.
When examining the AppStart components, keep in mind that this solution was developed simply to show what can be done; it is not yet ready for production environments. There wasn't enough time to optimize the code and test the solution thoroughly, or to document the features, apart from source code comments.
You should use this solution at your own risk. If you are interested in studying the source code to create your own solution, start with AppStart.cs in the Biblioso code project. This file implements the OpenControl COM interface and the entry points for the JavaScript calls from Ows.js.
Summing Up
WSS 3.0 and MOSS 2007 provide extensive application integration capabilities for a seamless user experience when working with documents and other items in SharePoint document libraries and lists. The desktop applications in the 2007 Office system demonstrate this very vividly, and it is possible to achieve the same level of integration and usability with non- Office applications as well.
At the core of the application integration architecture are COM components, which the SharePoint JavaScript functions use, along with the document path, to start the application. The 2007 Office system provides several of these COM components, optimized for specific Office application needs, though it is also possible to reuse the Office OpenDocuments control to integrate non-Microsoft applications. The OpenDocuments control meets the most basic needs. For more advanced application integration requirements, you can implement a custom control.
Fully integrating an application with SharePoint entails not only installing IFilter components and document icons to extend search and display capabilities but also involves creating custom content types and document type mappings on the SharePoint servers to provide appropriate New and Edit in commands in the SharePoint user interface. These commands call JavaScript functions that invoke methods exposed through an OpenControl component. The OpenControl component must also be available on the workstation.
Another important piece of the puzzle is the Web Client, included and installed by default on Windows Vista. The Web Client implements a WebDAV redirector and remote file system driver so that any application can access resources in SharePoint lists and document libraries similar to file shares on the network. Although the Web Client that ships with Windows Vista has shortcomings, it is a vital piece of the application integration story.
WebDAV support also bridges the gap between applications running on non-Windows workstations and SharePoint. COM and ActiveX control technology is generally not available on these operating systems, so you can't use OpenControl components to start applications automatically, but most operating systems include WebDAV redirectors so that users can at least access documents directly in document libraries without having to download the files to the local workstations.
WSS 3.0 and MOSS 2007 are cornerstones of the 2007 Office system success story, and users of third-party Office-based applications can benefit from SharePoint in the same way. Integration capabilities reach far beyond Office. By taking full advantage of SharePoint, you can build a unified office solutions platform that provides the same user experience for Office and third-party software and, as a result, boosts productivity for information workers.

Pav Cherny is an IT expert and author specializing in Microsoft technologies for collaboration and unified communication. His publications include white papers, product manuals, and books with a focus on IT operations and system administration. Pav is President of Biblioso Corporation, a company that specializes in managed documentation and localization services.

© 2008 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited


Windows PowerShell The Drop Box
Don Jones

Contents
Be the Cmdlet
The Filtering Function
Think Physical
Practical Applications
Make Time for Play
At one of the Windows PowerShell classes I was recently teaching, some of the students were having a tough time visualizing what the shell's pipeline was all about. I'll admit that the pipeline isn't completely intuitive conceptually, and thus it can be quite difficult for visual learners to picture what exactly is going on. When I got to the
concept of filtering functions, which work directly in the pipeline, things got really sticky, and I could tell from looking at their faces that I was definitely losing a few folks.
To try and help them out, I showed up the next day with a box, some name tag stickers, and some ping-pong balls. (Hey, nobody said Windows PowerShell® can't be fun.) I decided to do a demonstration using the following command line:
Get-Process | Where { $_.VM –gt 1000 } | Sort VM
–descending | Export-CSV C:\Procs.csv

Be the Cmdlet
Using the name tag stickers—you know, the "Hello, My Name Is …" stickers you're always made to wear at school reunions and similar dreaded events—I assigned a cmdlet name to each student. I explained that the ping-pong balls represented Windows® process objects (more specifically, objects of the System.Diagnostics.Process type), and I then asked the students to tell me which cmdlet name sounded like it was most likely to generate process objects.
Video
Watch Don Jones demonstrate the examples in this column and explain how filtering functions, the pipeline, and custom objects work.

Looking around at each others' tags, they settled for the obvious choice of Get-Process. That demonstrates one of the main reasons I really like the cmdlet names used in Windows PowerShell: they are usually very obvious.
So the student portraying Get-Process grabbed all the ping-pong balls and dumped them into the cardboard box. The box represents the shell's pipeline, and what the student did is pretty much what a cmdlet does. A cmdlet generates one or more objects and then dumps them in the pipeline.
The next cmdlet in the pipeline then takes over. In this example, the student representing the Where-Object cmdlet picked up all of the ping-pong balls and examined each, one at a time, checking to see if a given ball's VM property was greater than 1,000. In the shell, "VM" stands for virtual memory, and for this exercise I'd written in marker the various amounts of virtual memory on each ping-pong ball.
Every ball (otherwise known as the process) that had a VM of 1,000 or more went back into the box (sometimes called the pipeline), while anything of a lesser value was dropped in a trash can, never to be seen again. (Actually, that's not true, I rescued them to be used in future classes.)
Next up, the student pretending to be Sort-Object went through the box of ping-pong balls and put them in order, based on their VM property. I have to admit that this was one part of the exercise that wasn't so well thought-out—keeping the balls from rolling all over the place was a bit tricky! It seems I'll have to find ping-pong cubes for my next class, or else keep some egg cartons handy.
Finally, the student playing Export-CSV picked up the balls and wrote out the info to a CSV file. In the physical realm, this translates into writing the process's properties onto a flip chart that we pretended was a CSV file.

The Filtering Function
With a simple pipeline out of the way, we decided to look at filtering functions, which are a bit more complicated. I proposed the filtering function and command line shown in Figure 1.
Figure 1 Sample filtering function and command line
Function Do-Something {
 BEGIN { }
 PROCESS {
  $obj = New-Object PSObject
  $obj | Add-Member NoteProperty "TimeStamp" (Get-Date)
  $obj | Add-Member NoteProperty "ProcessName" ($_.Name)
  Write-Output $obj
 }
 END {}
}
Get-Process | Do-Something | Format-Table *
First, I want to give a quick review of what filtering functions are supposed to do. The idea is that the function contains three script blocks, named BEGIN, PROCESS, and END.
When the function is used in the pipeline, the BEGIN script block is executed first. This script might do any set-up tasks, such as opening a database connection.
Then, the PROCESS script block executes once for each object that was piped into the function. Within the PROCESS script block, the $_ variable is automatically populated with the current pipeline object. So, if 10 objects are piped in, the PROCESS script block executes 10 times.
Finally, after all of the piped-in objects have been executed, the END script block executes and does any necessary clean-up tasks, such as closing a database connection.
Within any of these script blocks, anything written using Write-Output winds up in the pipeline for the next cmdlet. Anything not written using Write-Output is discarded.
Since our command started with Get-Process, our erstwhile student rounded up all the ping-pong balls (we'd taken a brief break and somehow the balls had been playfully tossed all over the room—go figure) and dropped them in the pipeline box. With her job complete, the student portraying the "Do-Something" filtering function then took over.
He first executed his BEGIN script block. That being empty, it didn't take much effort. Next, he grabbed all the pipeline objects (the ping-pong balls) and started looking at them one at a time. This was kind of fun, as there were about a dozen balls and he was trying to balance all of them on his lap—well, maybe you had to be there.
Anyway, he picked up one process object at a time and executed his PROCESS script block. That script block had him create a brand-new custom object. (I used special yellow ping-pong balls for this, and don't think those weren't tough to find at the local sporting goods store.) On these new ping-pong balls, he inscribed the current date and time along with the name of the process object he was currently looking at.
This new custom object was then written to the pipeline, meaning he placed the yellow ping-pong ball into the box, and the original process object ping-pong ball was discarded. He did this for each process object ping-pong ball that was in the box, about 12 in total. When he was finished, each white ping-pong ball had been replaced with a yellow custom object ping-pong ball. Finally, he executed his END script block, which was empty and didn't take any time at all.
To wrap things up, our Format-Table student took over. She picked up everything in the box—which was all yellow "custom" objects at this point—and started constructing a table, using both of the properties written on each ball. The result was a table, written on our flip chart, with two columns—TimeStamp and ProcessName—and about a dozen rows.

Think Physical
This exercise made the pipeline and filtering functions really hit home for everyone in the class. The ping-pong balls did a great job of representing objects, something that tends to get a bit abstract when you're just talking about the shell.
Everyone could see how the cmdlets manipulated those objects, how the results were placed in the pipeline, and how the next cmdlet picked up those results and manipulated the objects further. The sequence of events in a filtering function were more obvious, as was the PROCESS script block's technique of working with one input object at a time.
This also demonstrated the means by which a new custom object can be created and placed into the pipeline. Furthermore, it helps to demonstrate the advantages of producing custom objects as opposed to simple text—the new custom objects were able to be consumed by other cmdlets, such as Format-Table, offering a lot of flexibility in how the data could then be used and represented.

Practical Applications
The obvious question from the students was how can the pipeline and these filtering functions we just demonstrated be used in practical applications. The answer was an easy one for me to give, as I'd produced several examples for a recent conference session (you can download the examples for yourselves at scriptinganswers.com/essentials/index.php/2008/03/25/techmentor-2008-san-francisco-auditing-examples).
One example has a goal of listing several properties from the Win32_UserAccount class in Windows Management Instrumentation (WMI) from several computers. Assuming the computer names are listed in C:\Computers.txt, this simple command will do the job:
Gwmi win32_useraccount –comp (gc c:\computers.txt)
The problem is that the Win32_UserAccount class doesn't include a property that tells you which computer each instance came from, so the resulting list will be a useless hodge-podge of accounts from many computers. I solve the problem by creating a new custom object that includes the originating computer name, as well as select class properties that I am interested in. The code for this custom object is shown in Figure 2.
Figure 2 Using a custom object to gather computer names and select properties
function Get-UserInventory {
  PROCESS {
    # $_ is a computer name
    $users = gwmi win32_useraccount -ComputerName $_
    foreach ($user in $users) {
      $obj = New-Object
      $obj | Add-Member NoteProperty Computer $_
      $obj | Add-Member NotePropertyPasswordExpires ($user.PasswordExpires)
      $obj | Add-Member NoteProperty Disabled ($user.Disabled)
      $obj | Add-Member NotePropertyFullName ($user.FullName)
      $obj | Add-Member NoteProperty Lockout ($user.Lockout)
      $obj | Add-Member NoteProperty Name ($user.Name)
      $obj | Add-Member NotePropertyPasswordRequired ($user.PasswordRequired)
      $obj | Add-Member NotePropertyPasswordChangeable ($user.PasswordChangeable)
    }
  }
}

Get-Content c:\computers.txt |
  Get-UserInventory |
  where { $_.PasswordRequired -eq $False } |
  selectComputer,Name |
  Export-Csv c:\BasUsersBad.csv
The final command line passes all the computer names to the filtering function, which produces custom objects. Next, I filtered out all the users except those whose PasswordRequired property is False (the idea is to produce an audit report of problem accounts). Then I simply keep the Computer and Name properties, so the final report is a list of computer names and account names that need attention due to having non-expiring passwords. The filtering function makes this multi-computer report possible because it adds the originating computer name to the output while trimming down the properties to just those that I am interesting in seeing.
While there are other similar ways to accomplish this task, this approach is perhaps the most straightforward. It also serves to illustrate important concepts and techniques.

Make Time for Play
Even if you are comfortable with the pipeline, there is a lesson to be learned here. Thinking in terms of physical objects can help you figure out what it is you're trying to do.
So the next time you're having difficulty with a Windows PowerShell concept, try stepping away from the computer and replicating the task using everyday objects as an illustration. I humbly recommend that you keep a small bag of ping-pong balls (or cubes, if you can find them somewhere) handy for just this purpose.
Cmdlet of the Month: Out-File
How many times have you directed a cmdlet's output to a file using the > symbol? That is something like Get-Process > Processes.txt. Did you know that you're really just using the Out-File cmdlet in disguise? Here's a command that performs the same exact function: Get-Process | Out-File Processes.txt.
Of course, that requires a bit more typing, so why bother typing out the full Out-File cmdlet? One reason is that Out-File is much clearer when read. Someone who comes along in, say, six months might look at one of your scripts and wonder what the > stands for (it is, after all, kind of a legacy thing).
By contrast, Out-File makes it pretty obvious that a file is going to be created and written to. Furthermore, Out-File gives you the -append parameter (pretty much the same as >>), along with -force and -noClobber parameters, letting you control whether existing files are overwritten. Finally, typing out Out-File also gives you access to the -whatif parameter. This handy parameter will show you what Out-File will do, without actually doing it! That's a great way to test a complicated command line with little or no danger.

Don Jones is the coauthor of Windows PowerShell: TFM and teaches Windows PowerShell classes (www.ScriptingTraining.com).


Hey, Scripting Guy! The Adrenaline Rush
The Microsoft Scripting Guys

Code download available at: HeyScriptingGuy2008_07.exe (151 KB)

There's no doubt in our minds that writing for TechNet Magazine is the most prestigious—and rewarding—job anyone could have. (Of course, it would be even more rewarding if we actually got paid, but that's another story.) Just how prestigious is this job? Let's put it this way: right now, all over the world, children who are being tucked into bed are looking up at their mothers and saying, "Mommy, when I grow up, I want to write a monthly scripting column for TechNet Magazine."
Note Of course, there are people all over the world who believe that this column is already written by children. But that, too, is another story.
It's well-known that the Scripting Guys enjoy basking in the adulation that comes with writing for TechNet Magazine; we've even learned to put up with the autograph seekers and the paparazzi. But the truth is, we don't do this job for the honor and the glory; we do this job for the pure adrenaline rush that comes with writing system administration scripts.
Admittedly, it's not often you see the terms pure adrenaline rush and system administration scripting in the same sentence. (Unless, of course, pure adrenaline rush is preceded by the complete and total lack thereof.)
The average person considers system administration scripting to be extremely useful but probably not very exciting; in fact, he or she likely finds scripting a bit humdrum. And here's the reason: each of these people haven't tried writing boot configuration data scripts in either Windows Vista® or Windows Server® 2008.
Ah, yes, that raised your heart rate a beat or two, didn't it? As most of you reading this probably know, in Windows Vista and Windows Server 2008, the old boot.ini file has been discarded in favor of a new boot configuration data store that provides increased flexibility (and capabilities) when it comes to managing the boot process.
That's pretty cool. But what's really cool is this: thanks to a new Windows® Management Instrumentation (WMI) provider, the boot configuration data (BCD) store now can be easily accessed—and completely managed—using scripts.
Don't worry about that strange sensation that just ran up and down your spine; that's what a pure adrenaline rush feels like. After you start working with BCD scripts, you'll begin to get used to it.
Before we go much further, we should point out that we can't possibly even begin to cover everything that can be done using the BCD provider, at least not in this one column. (We sent e-mail to the magazine's editors asking about the possibility of doing a special double issue devoted entirely to writing boot configuration data scripts, but we haven't heard back from them yet.)
If you want more complete information, check out the BCD WMI provider documentation at go.microsoft.com/fwlink/?LinkId=116953. In the meantime, we'll show you some sample code designed for multi-boot computers—computers with more than one operating system installed.
Oh, these scripts work only on Windows Vista and Windows Server 2008. That's because, as we mentioned earlier, these are the only two operating systems that currently support BCD.
Let's start by taking a look at a script that tells us which OS is currently in use on a computer. Granted, that's not the most exciting use for the BCD provider; after all, we can already use the WMI class Win32_OperatingSystem to determine which OS is currently in use on a computer. However, this is a reasonably simple script (as far as BCD scripts go, at least), and it provides a good way to illustrate the basic techniques required when working with BCD. (Techniques that are, to say the least, a tad bit … unusual.)
Besides, we can't include too much excitement in our columns: TechNet Magazine has already told us that we will be liable for any damages caused by overexcited and overzealous fans of Hey, Scripting Guy! Considering the general mayhem that erupted around the world after our column on regular expressions, well, we can't afford to do anything that exciting ever again.
Note Remember all those videos of the Beatles being mobbed by their fans, and all those pictures of girls fainting and crying as John, Paul, George, and Ringo came into view? That's pretty much what things are like for us each time we publish a new column.
Well, OK, that might be something of an exaggeration. But there's no doubt that a lot of people—men as well as women—sob after reading one of our columns.
At any rate, take a look at the script in Figure 1, which uses the BCD provider to determine which OS is currently in use on a computer. Before you try to run this script, you should know that you must be running as an administrator or else the script will fail.
Figure 1 Getting the current operating system
Const BcdLibraryString_Description = &h12000004
Const Current = "{fa926493-6f1c-4193-a414-58f0b2456d1e}"

strComputer = "."

Set objStoreClass = GetObject("winmgmts:{(Backup,Restore)}\\" & _
 strComputer & "\root\wmi:BcdStore")

objStoreClass.OpenStore "", objStore
objStore.OpenObject Current, objDefault

objDefault.GetElement BcdLibraryString_Description, objElement
Wscript.Echo "Current operating system: " & objElement.String
And no, that doesn't mean you simply need to be logged on to an account with administrator privileges. It means that you need to open a command prompt window by right-clicking Command Prompt in the Start menu and selecting Run as administrator. Then you can run this script from the command prompt.
So how does this script work? You know, we were afraid you were going to ask. But that's all right; we'll try to explain the best we can (which, if nothing else, should keep the excitement to a minimum, at least for a while).
We start out by defining two constants: BcdLibraryString_Description and Current. BcdLibraryString_Description refers to the object we want to retrieve; in this case, that object contains the Description (name) of the current operating system.
This, by the way, is one of the endearing quirks of working with the BCD provider. Typically, you don't simply retrieve the value of a property; instead, you use one of these constant values (and the GetElement method) to retrieve another object, then echo back the value of a property belonging to that object. Sure, that's a little goofy, but you know what they say: getting there is half the fun.
Note Which kind of makes you wonder where they were going, doesn't it?
As for the constant Current, well, that's simply a GUID that represents the operating system currently in use. The value
{fa926493-6f1c-4193-a414-58f0b-2456d1e}
is considered a "well-known" GUID (which, if nothing else, proves that people at Microsoft actually do have a sense of humor).
Despite the fact that this is a well-known GUID, we went ahead and included the value anyway, just in case some of you reading this have trouble remembering which of the following two values is correct:
{fa926493-6f1c-4193-a414-58f0b2456d1e}
{fa926493-6f1c-4193-a414-58f0b2456d1f}
Note In case you're wondering, GUID can be pronounced either like "gwid" or like "goo-id." We could tell you more, like the fact that the preceding GUID must be a V1 GUID, seeing as how the third group of digits begins with a 4. But, we did promise to try and keep the excitement to a minimum. And we keep our promises.
As you might have guessed, you'll need to use constants, hexadecimal values, and GUIDs pretty much everywhere you turn when working with the BCD provider. We can't cover all of these items in this month's column, but you can visit the BCD page we mentioned earlier on MSDN® (go.microsoft.com/fwlink/?LinkId=116953) for more information. If you can stand all the excitement, that is.
After defining our two constants, we next connect to the WMI service on the local computer. Can you use this script to retrieve boot configuration data for a remote machine? Of course you can.
To be honest, if you couldn't, this BCD stuff would be of only minimal value. In order to retrieve boot configuration data from a remote machine (again, the remote computer must be running either Windows Vista or Windows Server 2008), you simply assign the name of that machine to the variable strComputer:
strComputer = "atl-fs-001"
We should also point out a couple of key items in our WMI connection string. To begin with, you might have noticed that we included both the Backup and Restore privileges in our connection string; that's what the {(Backup,Restore)} construction does. Is that important? Well, if you want your script to work, then it's very important: so if you don't explicitly include those two privileges, the script will fail.
Second, note that we don't connect to the root\cimv2 namespace, the namespace most commonly used in system administration scripts. Instead, we connect to the root\WMI namespace and bind directly to the BCDStore class. That's what this bit of code is for:
"\root\wmi:BcdStore"
And no, that wasn't the exciting part. The exciting part is yet to come.
Although there might be an exception or two somewhere, for the most part any BCD script you write is going to kick off with the same three steps: define the constants, connect to the WMI service, and then open the BCD store. We've already completed Steps 1 and 2; the following line of code takes care of Step 3:
objStoreClass.OpenStore "", objStore
Like we said, here we're opening the BCD store, the operating system entity that stores all your boot configuration information. To open the store we simply call the OpenStore method, passing the method two parameters:
  • An empty string (""). This tells the script that we want to open the default store.
  • objStore. This in an "out" parameter that we supply to the script. We give the method the name of a variable and, in a gesture of gratitude, the method returns an object (in this case, an object representing the BCD store) that uses that variable name as an object reference.
As soon as the store is open, we can use the OpenObject method to retrieve yet another object (stored in the out parameter objDefault):
objStore.OpenObject Current, objDefault
And what is this new object? That's right: it's the operating system currently in use. This is something we know because we passed OpenObject the constant Current, the well-known GUID that represents the current operating system.
OK, so now we know which operating system is currently in use on the computer, right? Well, almost. To get that information, we still need to use GetElement to retrieve an object representing the Description of that operating system:
objDefault.GetElement _
  BcdLibraryString_Description, objElement
Now, do you promise you won't get overly excited and engage in any sort of general mayhem? Good. In that case, we can tell you that now we can echo back the value of the String property and—finally!—determine the operating system currently in use on the computer:
Wscript.Echo "Current operating system: " _
  & objElement.String
Please remain calm. Remember your promise: no general mayhem. Yes, we know it's hard. Just do the best you can. Try taking deep breaths; that always works for us when we're writing BCD scripts.
Like we said, that's a lot of trouble to go through just to determine which operating system is in use on a computer; there are easier ways to accomplish that task. On the bright side, however, you now understand how a BCD script works, which means you can do some things that script writers have never been able to do. (Stay calm, stay calm.) For example, on a multi-boot computer, one operating system is always tagged as the "default"; if a computer reboots and no one is around to tell it different, the computer will automatically load the default operating system. Prior to Windows Vista (and the BCD provider), there was no way for a scripter to determine the default operating system on a computer. Now, however, that's as easy as running a script like the one in Figure 2.
Figure 2 Determining the default operating system
Const BcdLibraryString_Description = &h12000004
Const BootMgrId = "{9dea862c-5cdd-4e70-acc1-f32b344d4795}"
Const DefaultType = &h23000003

strComputer = "."

Set objStoreClass = GetObject("winmgmts:{(Backup,Restore)}\\" & _
    strComputer & "\root\wmi:BcdStore")
objStoreClass.OpenStore "", objStore

objStore.OpenObject BootMgrId, objBootMgr

objBootMgr.GetElement DefaultType, objDefaultOSIdentifier
objStore.OpenObject objDefaultOSIdentifier.Id, objDefault

objDefault.GetElement BcdLibraryString_Description, objElement
WScript.Echo "Default operating system: " & objElement.String
You're right: this does call for a little general mayhem, doesn't it? But just a little, OK? We aren't going to explain this script in any great detail; assuming you didn't just skip over the preceding 10 or 12 paragraphs, you should be able to follow the logic for yourself. You might note, however, that in order to get at the default operating system, we have to take an intermediate step: we have to use the OpenObject method to open an instance of the Boot Manager object. After we open the Boot Manager, we can use the constant DefaultType to retrieve an object representing the default operating system.
Now, admittedly, that's pretty exciting: we can determine the current operating system, and we can determine the default operating system. But you know what would be really cool? That would be to retrieve a list of all the operating systems installed on a computer. Now that would be exciting! Fasten your seat belts and look at Figure 3.
What are we doing here? Well, to begin with, we're defining two new constants: WindowsImages, which lets us retrieve instances of all OSs that support BCD (that is, Windows Vista and Windows Server 2008); and LegacyImages, which lets us retrieve instances of all the "legacy" operating systems on the computer. After we connect to the BCD store, we then use the EnumerateObjects method to retrieve all the instances of the BCD-capable operating systems installed on the computer:
objStore.EnumerateObjects _
  WindowsImages, colObjects
Figure 3 Finding all operating systems on a computer
Const BcdLibraryString_Description = &h12000004
Const WindowsImages = &h10200003
Const LegacyImages = &h10300006

strComputer = "."

Set objStoreClass = GetObject("winmgmts:{(Backup,Restore)}\\" & _
 strComputer & "\root\wmi:BcdStore")

objStoreClass.OpenStore "", objStore

objStore.EnumerateObjects WindowsImages, colObjects

For Each objObject in colObjects
 objObject.GetElement BcdLibraryString_Description, objElement
 Wscript.Echo objElement.String
Next
Wscript.Echo

objStore.EnumerateObjects LegacyImages, colObjects

For Each objObject in colObjects
 objObject.GetElement BcdLibraryString_Description, objElement
 Wscript.Echo objElement.String
Next
After EnumerateObjects does its thing, we will then set up a For Each loop to loop through all the operating systems in that collection. Inside this loop, we use these two lines of code to retrieve and then display the operating system Description:
objObject.GetElement _
  BcdLibraryString_Description, objElement
Wscript.Echo objElement.String
And then we repeat the process with any legacy operating systems installed on the machine:
objStore.EnumerateObjects _
  LegacyImages, colObjects

For Each objObject in colObjects
 objObject.GetElement _
  BcdLibraryString_Description, objElement
 Wscript.Echo objElement.String
Next
Note We understand the thrill involved, but please run your BCD scripts responsibly; there is probably a limit to the number of adrenaline rushes the human body can withstand. Women who are pregnant, or may become pregnant, or may never become pregnant, or who were once pregnant, or who aren't even women after all but are actually men should not run BCD scripts without first checking with their doctor.
Well, OK, you don't need to check with your doctor first. But it would be interesting to hear what a doctor would say if you did ask him about running BCD scripts, wouldn't it?
Now let's try something really crazy; let's see if we can change the default operating system. For example, suppose we have a dual-boot computer running both Windows Vista and Windows Server 2008. And suppose we want to set Windows Vista to be the default operating system. How are we going to do that? Well, take a look at Figure 4. It shows one way to accomplish this task.
Figure 4 Changing the default operating system
Const BootMgrId = "{9dea862c-5cdd-4e70-accl-f32b344d4795}"
Const BcdLibraryString_Description = &h12000004
Const DefaultType = &h23000003
Const WindowsImages = &h10200003

strComputer = "."

Set objStoreClass = GetObject("winmgmts:{(Backup,Restore)}\\" & _
 strComputer & "\root\wmi:BcdStore")

objStoreClass.OpenStore "", objStore
objStore.EnumerateObjects WindowsImages, colObjects

For Each objObject in colObjects
 objObject.GetElement BcdLibraryString_Description, objElement
 If Instr(objElement.String, "Vista") Then
  objStore.OpenObject BootMgrId, objBootMgr
  objBootMgr.SetObjectElement DefaultType, objObject.ID
 End If
Next
With this script, we're once again opening the BCD store, then we'll be using EnumerateObjects to retrieve a collection of all the BCD-aware operating systems installed on the computer. From there, we set up a For Each loop to loop through all the items in the collection, using the following (and by now familiar) line of code in order to retrieve the Description for each operating system:
objObject.GetElement _
  BcdLibraryString_Description, objElement
As soon as we have the Description for a given operating system, we use the InStr function to see if the word Vista appears anywhere in that value:
If Instr(objElement.String, "Vista") Then
And you're right, that does seem a little clunky, doesn't it? The cool way would be to use the GUID for Windows Vista and then directly open that operating system, without having to enumerate and walk through all the operating systems on the computer.
Unfortunately, the cool way would require us to know the GUID for Windows Vista. Instead, this way we don't have to know anything (always a bonus for the Scripting Guys); all we have to do is keep searching until we find an operating system with the word Vista in the title.
Note What if you have multiple instances of Windows Vista installed on the computer? Well, in that case you might have to look for a string like Vista Ultimate or Vista Enterprise.
As soon as we find Windows Vista, we use these two lines of code to open the Windows Boot Manager and then set the default operating system to, well, Windows Vista:
objStore.OpenObject BootMgrId, objBootMgr
objBootMgr.SetObjectElement _
  DefaultType, objObject.ID
That's all we have time for this month; after all, even the Scripting Guys can't handle non-stop excitement 24 hours a day, 7 days a week. But don't worry; we'll be back next month with another exciting installment of Hey, Scripting Guy! With any luck, you'll have recovered by then.
Dr. Scripto's Scripting Perplexer
The monthly challenge that tests not only your puzzle-solving ability but also your scripting skills.

July 2008: VBScript Squares
OK, so they're probably rectangles, not squares, but that's beside the point. To solve this puzzle, fit each square on the right into an empty square on the left to create the names of VBScript functions. Each square will be used once. Here's an example:
To correctly solve this example, you need to move the OU square (rectangle) to the blank square in the first word, and the MS and the OX to the blank squares in the second word. This results in the VBScript functions UBound and MsgBox, as shown here:
Now you try it:

ANSWER:
Dr. Scripto's Scripting Perplexer
Answer: VBScript Squares, July 2008

The Scripting Guys work for—well, are employed by—Microsoft. When not playing/coaching/watching baseball (and various other activities), they run the TechNet Script Center. Check it out at www.scriptingguys.com.


The Desktop Files Network-Booting Windows
Wes Miller

Contents
How PXE Works
Diving into RIS
WDS—The Beginning
Other Players
Wrapping Up
Over the next few months I'm planning to cover Windows Deployment Services (WDS), which is available for Windows Server 2003 and is built into Windows Server 2008. WDS can be a very important component in your deployment infra-structure, so I want to make sure you have a good foundation for the discussion. This
first column will drill into the architecture of the Pre-boot eXecution Environment (PXE; pronounced pixie), the history of Remote Installation Services (RIS), as well as other PXE-related technologies used at Microsoft.
When I moved to the Windows core OS group in 2001, RIS was one of the technologies I inherited (and was a bit terrified to own, due to its complexity and dependency on BIOS implementations and hardware). But it was, in addition to Windows® PE, one of the technologies I enjoyed most in my role as a Program Manager.
I remember when I first installed Windows 3.0—using a set of 3.5" diskettes. Over time, installation became easier by virtue of bootable CDs (included in some versions of Windows 98). But the thing was, installation always required local media of some type, as well as a local hard disk.
Through the years, the ability to network-boot Windows—that is, to boot it completely over the network without requiring a hard disk—has been a frequent request of customers. Though some early versions of Windows had that ability, Windows NT® never did. And while current versions of Windows Server® 2003 and Windows Server 2008 can be booted via an iSCSI initiator, the process is quite different in that it isn't truly local—it entails an ongoing dependency on a remote drive as the boot drive.
Prestaging Clients
  • Evaluating the RIS Client Prestaging Process
  • Planning for Network Security Enhancement Using Prestaged Clients
  • Prestaging Client Computers
  • Remote Operating System Installation
Beginning with Windows 2000, Microsoft began developing the technology that would eventually come to be called RIS and that would allow for network-based installation. The goal of RIS was a relatively simple one—to put an operating system image onto the local disk of the target computer using PXE.

How PXE Works
Figure 1 shows the PXE boot sequence. PXE is a relatively simple protocol that was developed by Intel and other vendors as part of the Wired for Management initiative. PXE is derived from Dynamic Host Configuration Protocol (DHCP), itself derived from BootP, and is typically implemented in your Network Interface Card (NIC). Simply put, here's what happens:
Figure 1 PXE boot sequence
Step 1 The system BIOS boots up and determines boot order.
Step 2 If the boot order puts PXE ahead of hard disks, or flash drives, or CD-ROMs, or if none of those devices are present, the Universal Network Driver Interface (UNDI) is loaded from the NIC. The NIC features an extremely small network device driver and a Trivial File Transfer Protocol (TFTP) implementation. (Some BIOS implementations require end users to press the F12 key in order to PXE boot. That isn't standard, and I appreciate the ability to disable it.)
Step 3 The system begins making a simple User Datagram Protocol (UDP) broadcast, looking for a DHCP server. This is actually the first step of the PXE boot sequence and is referred to as Discover. Notice that the protocol is UDP—meaning that if you haven't yet, you will need to spend some quality time with your routers and switches to ensure that the PXE communications can all make it across.
Step 4 If a DHCP server hears the broadcast, it responds accordingly with an IP address. This step is referred to as Offer. The important point to remember here is that PXE is stateless and the amount of system-unique state information the client has to offer at this point is pretty limited (the MAC address and, if available, the System Management BIOS GUID, also known as SMBIOS GUID).
Step 5 The client, after receiving the packet with the IP address, then states that it actually needs further information—namely the PXE server's address. Another broadcast occurs, which includes the information from the DHCP server that originally responded. Here the client is telling the DHCP server, "I need more information—specifically, I need the location of a Network Boot Program (NBP)." This step is called Request.
Step 6 The PXE server responds with the address of the PXE server and the location of the NBP, an extremely small boot executable that is required to be smaller than 32KB. This step is called Acknowledge. If you're playing along, you've probably noticed the acronym DORA (Discover, Offer, Request, Acknowledge), a good way to remember the sequence.
Note that if you have installed Microsoft DHCP and WDS (or used some other vendor's technologies), the request step doesn't occur and, in fact, the original Offer packet from the DHCP server already includes the location of the PXE server and the NBP program (thus removing two steps, as well as some time).
Step 7 The client, which has the small TFTP protocol stack mentioned earlier, downloads the NBP from the network location specified by the PXE server. TFTP is a dated, exceedingly small, stateless protocol. It wasn't chosen for its security or its performance—and many router administrators disable it by default as a result. It must be enabled for PXE to function.
Many PXE implementations (including RIS) include the ability to ask the user to press F12 to continue at this point, but typically this can be disabled by the administrator of the PXE server. Next month when I look further at WDS, I'll take a look at some of the enhancements Microsoft has put into the TFTPD (TFTP Daemon) in WDS for Windows Server 2008 to improve performance.
Step 8 The NBP is initialized. In the case of RIS, this starts a Windows boot loader that begins the process of taking deployment forward. PXE (at least the actual boot-level protocol) is no longer a component in the process.
It's important to remember that PXE (whether RIS, WDS, or any other infrastructure) doesn't work well over slow links (it can be transferring considerable amounts of data) or high-latency links such as satellites (the communication simply doesn't perform well and may not even survive).
You may notice in the PXE boot process that when the client sends the request, there is nothing that specifically asks, "Are you my mother?" There simply isn't a lot of state information for the PXE server to know one way or another. What usually occurs is a race condition—where the first server to respond to your client request will win. A couple of things can help reduce this problem:
  • Adjust the response speed of one PXE server or the other. Network latency and server horsepower will impact how fast the servers respond. In fact, at Microsoft it used to be that the servers used by Microsoft IT were so good that even if the PXE server was in your office, the corporate servers would sometimes win. In this case, you just set your local PXE server to have no timeout at all—for your prestaged clients.
  • Prestage the clients. This is very important if you are manipulating your PXE server to respond ahead of other corporate IT servers. By prestaging your clients, you allow Active Directory® to tell WDS or RIS that yes, in fact, "I am your mother." Note that the use of the SMBIOS GUID is far preferred as the unique identifier for your systems in Active Directory—but if an SMBIOS GUID is not implemented in the systems (more likely in relatively older hardware), you can (and will have to) use a GUID based on the MAC address. For more information, see the "Prestaging Clients" sidebar.
  • Don't allow PXE communications to cross switches or routers; put a PXE server on each side. This has the downside of being both expensive to implement and expensive to maintain (each server must have its own image maintained).
RIS (and now WDS) servers, like Microsoft DHCP servers, must be authenticated against the Active Directory implementation they are associated with. The goal is to reduce the issues that rogue PXE servers may cause (such as PXE broadcast storms) by letting the Active Directory know about all of those servers.
Note that this protects only against servers Active Directory knows about. If you set up your own domain, or a non-Microsoft PXE server, that won't be the case.
At Microsoft, an overzealous employee once configured a "zero-touch" deployment non-RIS PXE server. This implementation worked by completely erasing the hard disk and putting down a new image. That would have been fine if the deployment occurred in an isolated (off-network) lab, but unfortunately it didn't—and it wound up erasing the disk of a Microsoft executive who had PXE early in his boot order before the hard disk.
That hadn't been a problem, as Microsoft IT always required pressing F12 to PXE boot, but this PXE server did not have a delay, an F12 prompt, or any type of notification. This meant that the executive effectively lost his computer and any data not protected by his Roaming User Profile.
Let this story serve to highlight to you the necessity of isolating your PXE servers if you're going "zero-touch," or at the very least to require pressing F12.

Diving into RIS
I inherited RIS after Windows 2000 had shipped. Time hadn't been kind to Windows 2000 as far as RIS was concerned—and testing, performance, and other constraints led to RIS for Windows Server 2000 being used solely for Windows 2000 Professional deployment. The server products, unfortunately, couldn't be deployed via RIS. Windows 2000 was only available for x86 machines, so it proved to be a good test bed for RIS since it involved one product on one architecture. RIS included (and required) complete integration with Active Directory, integrated well into the Microsoft DHCP server, and included its own TFTPD.
RIS uses the NBP to continue the TFTP download—and bring down enough of the Windows kernel to begin the setup process. (At a certain point, once Windows has switched from TFTPD to a Server Message Block, or SMB,-based connection to the server, the literal codepath is actually shared with a traditional floppy-disk-initiated install of Windows 2000 or Windows XP.) Once native-mode Windows has been initialized, Windows setup begins the RIS OS Chooser (OSC) Wizard.
OSC screens are somewhat configurable, HTML 2.0-like pages. They are severely constrained and cannot contain images or the like and, in fact, can't contain non-ANSI characters (which made deploying certain locales of Windows complicated).
The end product of RIS is a txtsetup.sif file that sits on the RIS server. When the OSChooser Wizard completes, the client is "soft-rebooted," but the location of the RIS server and of the txtsetup.sif file are retained and reloaded after the soft reboot. This txtsetup.sif file is essentially the same as an unattend.txt file, with several additional fields included to help RIS complete the setup process.
RIS could also perform a setup that looked much like traditional unattended setup (RISetup), and it had a cloning-based infrastructure similar to Sysprep (RIPrep) and, in fact, shared code with it. But RIPrep could also upload an image of itself to a RIS server.
RIS had some fundamental limitations that became apparent, however. The first was the lack of support for server deployment. Certain exploits, such as Code Red and Sasser, combined with the IT complexities that several key customers experienced recovering directly from the September 11th tragedy in 2001, led us to actually fast-track a solution for existing Windows 2000 RIS servers to allow for Windows Server deployment. This was something we had been working on for Windows Server 2003 but had not formally released.
More on PXE-Related Technologies
  • Description of PXE Interaction among PXE Client, DHCP, and RIS Server
  • Planning for PXE-Initiated Operating System Deployments
  • PXE Architecture
  • Remote Installation Services
  • Remote Operating System Installation
  • Step-by-Step Guide to Remote OS Installation
  • WDS PXE Provider
  • What Is Remote Installation Services?
  • Windows Server 2003 Automated Deployment Services
Second, RIS lacked the ability to completely automate the OSChooser Wizard, which was later enabled with the <META ACTION="AUTOENTER"> element in Windows Server 2003. Finally, the OSChooser was unable to function properly with non-ANSI characters—a key weakness pointed out by several customers outside of the United States.
As a result, you could not complete a RIS installation with a French keyboard, for example. Getting non-ANSI characters working safely at the BIOS level on PCs from around the world was extremely complex and it just wasn't easy to accomplish.
With the release of Windows Server 2003, we formally added support for the Intel Itanium architecture and all server variants of Windows 2000 and Windows Server 2003. Windows Server 2003 took that one step further with support for the x64 architecture.
RIS also had a significantly rewritten TFTPD to increase performance. Windows Server 2003 supported booting up to 75 clients at the same time; bear in mind that the upper bound here is reached as the SMB pipe fills up with network traffic to the clients.

WDS—The Beginning
When we started working on RIS for "Longhorn" (the code name for what became Windows Server 2008), it became apparent that we needed to take a step back. As I've mentioned in my column before, we had already made big bets on image-based (WIM) setup from Windows PE. As a result, the key tenet underlying WDS was image-based deployment from a PXE-booted instance of Windows PE.
We also knew that Windows Server 2003 would be the common platform for Windows Vista® deployment, and that we would need an "out of band" solution for WDS downlevel. As a result, WDS was able to run on Windows Server 2003 SP1 and was built into Windows Server 2003 SP2.
Since it can operate as a RIS server (Legacy mode), a hybrid server (Mixed mode), or a WDS-only server (Native mode), WDS allows you to migrate to formal WDS-style deployments as warranted. I have heard customers asking if there is a way to install RIS on a Windows Server 2003 SP2 system. Yes, there is—you install WDS and run in Legacy mode. Figure 2 shows the supported operating systems.
Figure 2 Platforms supported for deployment
Operating System | RIS (Windows 2000) | RIS (Windows Server 2003)** | WDS (Windows Server 2003)**** | WDS (Windows Server 2008)
Windows 2000 Pro | X | X | X | X
Windows 2000 Server | * | X | X | X
Windows XP Pro |  | X (x86 and IA64)*** | X | X
Windows Server 2003 |  | X (x86 and IA64)*** | X | X
Windows Vista |  |  | X | X
Windows Server 2008 |  |  | X | X
* support.microsoft.com/kb/308508 and support.microsoft.com/kb/313069 added support for Windows 2000 Server via RIS. ** WDS Legacy and Mixed mode support this same matrix for legacy installs. *** Windows Server 2003 SP1 added support for x64-based systems. IA64 systems only supported RISetup-based installation. **** Native Mode support.
WDS in Windows Server 2003 SP1 and SP2 sought to begin the migration process to WDS. As I've mentioned before, the key features WDS added in Windows Server 2008 were a revised TFTPD, Extensible Firmware Interface (EFI) boot support, and of course, multicast-based deployment.

Other Players
Automated Deployment Services (ADS) was built by another team within Microsoft, primarily with the goal of rapid server provisioning. ADS featured formal sector-based imaging, its own boot agent (smaller than Windows PE but not as full-functioned), its own TFTPD, and very advanced multicast. The functionality built into ADS became available to a degree in System Center Configuration Manager (SCCM), though there is not 100 percent feature parity.﻿
Windows XP Embedded featured full PXE-boot via its own TFTPD into a RAMDisk, but could not remote deploy that way. The technology was designed to support booting numerous systems from the same disk image at the same time via PXE.

Wrapping Up
So that's the history, in brief. To find out more, see the "More on PXE-Related Technologies" sidebar. Next month, I'll dive into WDS fundamentals, to be followed by columns on WDS advanced functionality (multicast and more) and, finally, using WDS without using WDS, by which I mean going beyond the existing WDS/setup experience to roll your own deployment techniques.

Wes Miller is a Senior Technical Product Manager at CoreTrace (www.CoreTrace.com) in Austin, Texas. Previously, he worked at Winternals Software and as a Program Manager at Microsoft. Wes can be reached at technet@getwired.com.


Security Watch Passwords and Credit Cards, Part 1
Jesper M. Johansson

Contents
Inactionable Security Advice
Inactionable and Incorrect
Incomprehensible Advice
The Pitfalls of Image-Based Site Authentication
I was recently contacted by the University of Minnesota in order to be interviewed for its magazine. Apparently, they wanted to run a feature on some successful alumnus and—not being able to find any—they settled for me instead. The interviewer asked what I work on and I went on for a few minutes trying to describe security
infrastructure software. She then exclaimed, "That sounds so complicated! To me, security is about passwords and credit cards."
I pondered this reaction for a few minutes and then realized she had a really good point. Security really is about passwords and credit cards. At least, that is what it's about to the end user. Those of us in the business think security is about cryptographic algorithms, whether Kerberos is a better choice than TLS or NTLMv2, the merits of WS*, whether password hashes should be salted, and all the other esoteric topics we love to discuss. Sure, we have a very different and deeper perspective than end users, but we have, by and large, lost sight of one fundamental thing: security, to those whom we ought to be serving, is about passwords and credit cards.
Granted, all that esoterica we so love to argue about and the new technologies we so love to invent are all to protect the end user's data. Still, I think we have lost our way. We, the security subculture of the IT world, exist to serve a particular need of our constituents—the need to keep data safe. That, of course, includes ensuring that IT assets can be used safely. That's really it.
In previous columns, I have made the point that no one goes out and purchases a computer so he can run antivirus software. The user purchases a computer to do online banking, play computer games, write e-mail, do homework, or some other primary function. Likewise, no business has funded an IT security group simply so they could implement NTLMv2. Businesses fund IT security groups so these groups can protect the organization's assets, enabling the business at large to safely use its IT resources and achieve its business objectives.
We do not exist but to serve.
And so I must ask whether we are really doing a good job "serving" these days. Or are we, the security subculture, actually getting in the way more than helping? And are the legislators and regulators helping us to get in the way? I am not convinced that all this new technology we are putting in place really helps the end users. Therefore, I would like to explore a few areas where we, the IT providers of the world, are actually causing more harm than good.
Some days it feels like most of the security advice and many of the security technologies we inflict upon our users is inactionable, incorrect, incomprehensible, or (in many cases) some combination of the three. In this three-part series, I am going to look at some of the ways we confuse users by giving advice and deploying technologies that are guilty of one or more of these three I's.

Inactionable Security Advice
One of the best ways to confuse people is to give inactionable security advice. For bonus points, you can make it incorrect, as well. Figure 1 illustrates a popular piece of time-tested, theoretically sound, and utterly useless advice.
Figure 1 Inactionable security advice
See where it says to use a different password for each online account that you have? Thirty years ago this recommendation made sense. The number of people on what eventually became the Internet numbered in the low hundreds, and they were all very smart people who were not picking particularly good passwords. Unfortunately, this advice has persisted and keeps being repeated over and over again, and there has been no apparent effort to reconcile this advice with how computers are used today.
How many online accounts do you have? Personally, I have 115, give or take a few that I don't keep track of. Not only does the advice in Figure 1 suggest that I should have 115 different passwords, but that I should also change these 115 passwords every 30 to 60 days. In other words, I should change 2 to 4 of my passwords every day. (Do the math: that also means I would have between 690 and 1380 passwords in one year.)
While the technical personnel at some sites offering this advice may be able to come up with 4 good passwords per day and keep 115 current passwords in short-term memory, you can be certain that 99.99 percent of the general Web-surfing public will be unable to do this.
The advice to use different passwords everywhere is actually correct and sound, from a purely theoretical perspective, as is the advice to change all your passwords every 30 to 60 days. But this advice is also inactionable. With the number of passwords users have these days, they simply cannot follow this advice without an aid of some sort, such as paper or software. Enter the next example.

Inactionable and Incorrect
The piece of advice shown in Figure 2, which comes from the Web site of one of the world's largest banks, is both inactionable and incorrect. The information provided under the heading of "Read our password advice" parrots the "different passwords everywhere" line. It also recommends that you should "never write them down."
Figure 2 Inactionable and incorrect advice
So now I have 115 different passwords, and I have to create 4 new passwords each day, and I'm not allowed to write them down. For a while I thought I was just stupid since I couldn't remember all my passwords. Then I discovered that everyone else is exactly the same way. Human beings simply can't remember 115 passwords. We can remember and process about seven chunks of information—that's just how we're programmed. Following most of the password advice you will find on the Internet, that's actually not even enough for one password (see "The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information" by George A. Miller, which is available at musanim.com/miller1956).
The unfortunate fact is that when it comes to password advice, our industry, on a very regular basis, misleads users. If the security subculture is going to tell users that they should use different passwords everywhere, then we need to also tell them how—and that is to record and store those passwords somewhere secure. Write them on a piece of paper, use a secured document, or use a specialized tool, such as PasswordSafe (sourceforge.net/projects/passwordsafe). Face it, we all either record our passwords, or we use the same password everywhere. In fact, a recent survey found that 88 percent of users have the same password on every system they need to authenticate to (see msnbc.msn.com/id/24162478).
Advice against writing down your password is surely a contributing factor to this trend. What we need to do is teach users how to manage passwords (and other sensitive data) effectively, rather than teaching them not to manage this information at all. Then they will be a lot less likely to compromise their credentials.

Incomprehensible Advice
The first two figures repeat age-old advice that worked well back in the 1960s, 70s, and 80s, in a time before mainstream users were going online and using Web-based services. Nobody has, as of yet, made loud complaints about this type of advice because it is generally accepted wisdom.
This type of advice confuses users and can make them feel guilty about having to store their passwords somewhere. By giving this type of advice, rather than helping users do what they have to do, security professionals cause harm. After all, it is not the user's responsibility to figure out how to manage their security. That is up to the security professionals. We need to figure out acceptable ways for our users to manage all their online accounts. But as most guidance just repeats the same, outdated advice, users resort to sticky notes and spreadsheets to track their many passwords.
As an authentication mechanism, passwords have a lot to offer. The primary problem with passwords, though, is that human beings are terrible when it comes to remembering them. Instead of trying to solve a problem of human nature, the IT world keeps inventing all kinds of new mechanisms to replace passwords—or, worse still, to augment them. This just confuses users even more.
Imagine my surprise the last time I accessed a certain financial services site and was presented with a logon screen that contained only a username textbox (see Figure 3).
Figure 3 Where do I type my password?
At first I thought I had landed on some malicious Web site. Then I quickly validated the site—that step was easy since the site presented a certificate—and realized that I was, in fact, in the right place. The problem is that people are used to seeing two text fields, username and password, presented together when logging into a site. This is from years of encountering the same common workflow. So when you are suddenly presented with a site that asks only for your username, things come to a halt. It turns out that in this case the provider had implemented a technology that uses pictures to identify the site to users in an attempt to prevent phishing attacks. As it goes, when you fill in your username, the site pops up a screen with an identifiable picture, as shown in Figure 4.
Figure 4 Some sites now use images to authenticate the site to a user
The theory is that you know which image goes with which site. If the correct image isn't shown, you can identify the site as being fake. This, in and of itself, is a sound concept. Under the assumption that the user knows which image goes with which site, this strategy makes some amount of sense.
Of course, the astute reader will have noticed the green address bar in Figure 4. This means that this site uses SSL and Extended Validation (EV) certificates, which is why the address bar is green. It also means that the entire premise of using the image to identify the site provides no additional value. Instead, the image does little more than add confusion for at least some end users. The site has already identified itself to the user—it has provided a certificate that contains the company name, the Web site address, and the name of the trusted issuer. And the fact that the address bar is green tells me that the company has even gone the extra step of paying three times as much for an EV certificate.
Then, of course, the pictures can also be faked. If the user can submit her own picture, there are ways in which the attacker can probably figure out what image is being used. There is also a good chance the user will use the same picture on every site, so all the attacker will have to do is create a site with content the user will value (I will let you come up with your own examples) and ask the user for a picture to use for site verification. If the user does use the same picture for all sites, this new site will now have access to the image that the user uses on, for example, her banking site.
While some sites let you pick your own image, there are others that use a library of stock photos. The site in Figure 4, for instance, has 318 stock photos to choose from. The previous trick doesn't work for sites that do not permit the user to submit her own photo However, if the user cannot pick her own image, the likelihood that the user will remember which site goes with which picture will be unlikely for sites she does not visit frequently. I honestly have no idea what picture I use on the site shown in Figure 4, though I can assure you that it's not the one shown in the screenshot.
The problem with this image-based approach is that an attacker can therefore show just about any one of the 318 pictures or just pick a random shot off of Flickr, and many users will just assume the image is the correct one. If the majority of people could remember things like which picture goes with which site, we wouldn't have all the phishing- and security-related problems we have today.
So why use a picture to authenticate the site to the user when the site has already authenticated itself using the certificate? Why not just use that certificate and help users learn how to validate them? Certificates already prove the site's identity to the user.
The process for getting a certificate is certainly much more secure than the process of obtaining a user's site authentication image. If the certificate is an EV certificates and you are using Internet Explorer® 7 or Firefox 3, the browser will even highlight the relevant certificate information in the address bar. Unfortunately, the highlighting only works with the very expensive EV certificates.

The Pitfalls of Image-Based Site Authentication
The picture authentication technology has a number of problems. First, it becomes very easy to harvest usernames from a site. In fact, the site shown in Figure 4 will present the dialog shown in Figure 5 if you enter the wrong username. Once you type a correct username for a user who has chosen a secret image, you get to see that person's secret image. Obviously, that knowledge could be very valuable to an attacker trying to harvest information about a user.
Figure 5 Image-authenticated sites allow for easy username harvesting
The type of implementation shown here has virtually no security value. The attacker can simply duplicate the logon workflow on a fake site. The fake site asks for a username and passes it to the real site. You can even use AJAX on the client to update the form in real time for an extra slick look. Furthermore, if the legitimate site has not mitigated cross-site request forgery attacks on the logon form, the AJAX code can even submit the request to the real site directly unless the browser has mitigations for cross-site XML-HTTP requests.
Now, once the result comes back, the attacker can parse the data, pull out the picture, and display it to the end user. In other words, any attacker that can present a fake login site to the user can also display the user's secret picture. The net result is that there is absolutely no added value from using image-based site authentication. The picture is displayed prior to the user authenticating to the site and, therefore, the picture is available to an attacker that has, or can get, the user's username.
Assuming the user was never taught to look for a certificate before submitting forms—and this is a safe assumption considering that the use of image-based site authentication is itself based on that same assumption—getting a username from the user is trivial. In addition, since many image-based site authentication schemes respond differently to a valid username versus an invalid username, username harvesting is trivial. The attacker can even do it out-of-band, before an attack is even started.
In the end, we have users who believe they are more secure or who are just more confused. We have spent significant amounts of shareholder equity implementing image-based site authentication and we have not made it any more difficult at all for malicious users to convince end users to submit their credentials to fake Web sites.
That's all I have space for in this installment of Security Watch. Please check back next month for part 2 of this series when I will be discussing more examples of misguided security practices and bad authentication implementations.

Jesper M. Johansson is a Software Architect working on security software and is a contributing editor to TechNet Magazine. He holds a Ph.D. in MIS, has more than 20 years experience in security, and is a Microsoft MVP in Enterprise Security. His latest book is Windows Server 2008 Security Resource Kit.


Field Notes Ensure Election Accuracy
Josh Benaloh

In this U.S. presidential election year, public attention is focused upon voting technologies. As we struggle with options, the pendulum swings between hand-counted paper, which is cumbersome and subject to mishandling; lever machines, which can be manipulated by knowledgeable insiders; punch cards with their now-famous hanging chad; optically scanned ballots, which can easily be misread by poorly calibrated scanners; touch-screen systems, which can be compromised by sloppy or malicious software; and various hybrids, which often create more problems than they solve.
Indeed, this is just a small sampling of the vulnerabilities in these systems. Further, the techniques in common use throughout the world depend upon honest and skilled election administrators to maintain election integrity.
But there's a better way, one in which the integrity of the election does not depend on either administrators or equipment. Elections can be run so that their accuracy can be verified end-to-end by officials, individual voters, and even interested observers.
To see this, imagine that eligible voters are provided with codes that allow them to post their votes next to their names on a publicly readable Web site. Voters can check the Web site to see that their own votes are accurately recorded, and anyone can perform an independent audit to check that all voters are legitimate and that the count matches the announced tallies.
But hold on—the one missing element from this "open-faced" election is privacy. Yes, and cryptography can be used to fill this gap—thereby enabling fully verifiable secret-ballot elections. Instead of posting raw votes, the posted votes are encrypted with public keys whose encryption keys are shared by election officials so as to require cooperation to decrypt. Once voting has been completed, election officials engage in a cooperative process to produce both an election tally and a cryptographic proof that the tally corresponds accurately to the posted set of encrypted ballots.
Individual voters can check that their encrypted ballots correspond to their intentions, and anyone can examine the cryptographic proof to check that the encrypted ballots have been correctly tallied. In this way, the election tally can be verified end-to-end.
Is it really this simple? No. This is a faithful overview, but there are numerous details that must be carefully managed to get it all right. For instance, we must still depend upon traditional methods to ensure that voting devices maintain adequate voter privacy. And just how are voters supposed to translate their intentions into encrypted ballots? Voters must feel confident that the inscrutable encrypted values posted beside their names really do correspond to their intentions without needing to show third parties the correspondence (lest they become susceptible to coercion or vote-buying). And we must do all this without requiring voters to do anything more complex than they do today in traditional voting systems.
End-to-end verifiable election processes have been studied in cryptographic literature for more than a quarter century, and there are now solid and practical ways to provide the cryptographic proof of a tally's accuracy. Recent attention within the research community has focused upon the front end that is presented to the voter, and much progress has been made there as well.
The commercial demand for election technology is large, but it's not enormous. Vast conglomerates with widespread interests may be viewed with suspicion in this market since some of their interests may benefit from certain election outcomes. Thus, this market may be best filled by smaller niche players whose entire business depends upon getting elections right. For more information and links to end-to-end verifiable election methods, go to research.microsoft.com/crypto/voting.

Josh Benaloh is Senior Cryptographer in Microsoft Research. He served eight years on the Board of Directors of the International Association for Cryptologic Research, and his extensive publications include his 1987 doctoral dissertation, "Verifiable Secret-Ballot Elections."


Windows Confidential History Taking Up Space
Raymond Chen

Remember Microsoft Bob? It was intended to be a friendly user interface on top of Windows® 3.1, but instead it flopped infamously (earning the dubious distinction of being named one of the worst technology products of all time by PC World). In the years that have passed since Bob's demise, the product has been relegated to the status of a running joke with the mere mention of its name drawing snickers.
Incidentally, Microsoft® Bob's internal code name was "Utopia"—an ambitious name to be sure, but at least one that had some snap and pizzazz. When we learned that the marketing folks had decided to name the product Bob, we all shook our heads in disbelief.
But there's more to Bob's legacy than you might know. It turns out Bob was actually more useful dead than alive.
When you intend to distribute your software on a CD, one thing you have to worry about is making sure your product actually fits on a single CD. Fortunately, it so happened that even after taking into account the disk space required for translations, support tools, and the other stuff that has to go onto the Windows XP CD, there was still about 30 megabytes of storage capacity remaining. The people who worry about these sorts of things figured, well, we already paid for all that storage capacity on the CD so we might as well use it, right?
Microsoft Bob's internal code name was "Utopia"—an ambitious name to be sure, but at least one that had some snap and pizzazz.
The result was a rather feeble attempt to slow down the people who like to make illegal copies of Windows. Somebody decided to fill that extra capacity on the CD with dummy data and to have the Windows Setup program verify that the dummy data was still there. This, the logic went, would force people downloading a copy of the CD image to download an additional thirty or so megabytes of data. Remember, this was back in the day when "broadband" hadn't yet become a household word and mainstream users were using dial-up connections. Having to transfer an additional thirty megabytes of data over a 56Kb modem was a bit of an obstacle to slow users down—not that it would slow them down much by today's standards.
The person who was asked to implement this check needed a source for the dummy data. Now, he could have just called the CryptGenRandom function to generate 30 megabytes of cryptographically random bytes, but where's the fun in that? Instead, he dug through the archives and found a copy of Microsoft Bob. He took all the floppy disk images and combined them into one big file. The contents of the Microsoft Bob floppy disk images are not particularly random, so he decided to scramble up the data by encrypting it. When it came time to enter the encryption key, he just smashed his hand haphazardly across the keyboard and out came an encrypted copy of Microsoft Bob. That's what went into the unused space as ballast data on the Windows XP CD.
In the end, Windows XP became the most effective Microsoft Bob deployment tool ever developed. And if you go way back into your closet, dig out your copy of Windows XP, and can somehow channel the right spirits to mash your hands on the keyboard in exactly the right way, then out of your encryption program will come a copy of Microsoft Bob.

Raymond Chen's Web site, The Old New Thing, and identically titled book deal with Windows history and Win32 programming. He wonders whether nuclear fusion will become viable in his lifetime.

© 2008 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited