 TechNet Magazine February 2008

TechNet Magazine February 2008

SharePoint: A Smart Approach to Gathering Data in the Enterprise

Information is one of the most important assets in an enterprise. But it can be a challenge to manage the mountain of data being gathered through e-mail, Web-based forms, surveys, and other mechanisms. Here's a look at how the Microsoft Office system can help you integrate systems and make data more accessible.

Keith Deshaies SharePoint: Building a Powerful Survey Infrastructure

Gathering and analyzing data is an essential part of business, but you need tools that will allow independent teams to collect the data they need, when they need it. Find out how you can use WSS 3.0, MOSS 2007, and InfoPath 2007 to build a complete and flexible survey solution that can be used throughout the enterprise.

Jim Bradley SharePoint: Standardize Data Management with Custom Content Types

The vast number of disparate documents and other content items stored throughout a large organization can make it difficult to manage documents, their associated metadata, and their behaviors in a centralized and reusable way. Take a look at how you can use content types in SharePoint 2007 to simplify content management across the enterprise.

Pav Cherny Office Communications Server: How Presence Powers OCS 2007

Office Communications Server 2007 uses information about user availability to route communications to the most appropriate place—whether it's sending a voice call, a video conference, an e-mail message, or an IM. See how the OCS 2007 system uses presence to connect endpoints.

Rajesh Ramanathan Office Communications Server: Plan Your OCS 2007 Voice Deployment

Integrating Office Communications Server 2007 with an existing PBX system can be a challenging task. You need to select an integration scenario, understand call routing, know how to configure user settings, deal with number plans, and more. But don't worry. This overview will show you what you need to know.

Jochen Kunert and Rui Maximo High Performance Computing: An Introduction to Windows Compute Cluster Server

Rather than relying on large specialized systems, high performance computing uses clusters of smaller, standard systems. Spreading the workload in this fashion offers an economical solution for solving complex problems quickly. Explore the compute cluster functionality and support provided by Windows Compute Cluster Server 2003.

John Kelbley and Doug Lindsey Online Extra: Deploy the 2007 Office System with Terminal Services

The 2007 Microsoft Office system includes design changes you must be aware of before deploying it in your Terminal Services server farm. This article gives you an overview of the changes.

James D. Silliman From the Editor: Dealing with Information Overload

Joshua Hoffman Letters: Readers Speak Out

Toolbox: New Products for IT Pros

Greg Steen SQL Q&A: Non-Clustered Indexes and Retaining Permissions

Saleem Hakani and Dan Carollo Utility Spotlight: ADMX Migrator

With Windows Vista, the format for Group Policy templates has changed to an XML-based structure. The ADMX Migrator utility can convert your legacy ADM files into the new ADMX format, as well as help you easily create your own custom ADMX files.

Lance Whitney The Cable Guy: IEEE 802.1X Wired Authentication

IEEE 802.1X authentication provides an additional security barrier for access to your intranet. See how Windows Vista and Windows Server 2008 make it a snap to implement IEEE 802.1X authentication for your wired network.

Joseph Davies Windows PowerShell: Shell Permissions

Permissions in Windows are, unfortunately, innately complicated. But Windows PowerShell can help. Get a quick overview of how permissions in Windows work and find out how you can simplify the task of working with them.

Don Jones Hey, Scripting Guy!: The Games Are Afoot! Oh, and Some XML, Too

The Microsoft Scripting Guys go on and on about the 2008 Scripting Games, and they also show you how to create, append, and modify XML documents. By the way, have you heard about the 2008 Scripting Games?

The Microsoft Scripting Guys Security Watch: Island Hopping: Mitigating Undesirable Dependencies

Unwanted security dependencies can expose your entire network to attacks. Find out how security dependencies work so you can analyze dependencies in your environment and resolve potential problems.

Jesper M. Johansson The Desktop Files: Dual Booting with Windows XP and Windows PE 2.0

There are a number of advantages to booting Windows PE 2.0 from the same partition as Windows. Learn how to dual-boot Windows XP and Windows PE 2.0 from the same partition and explore the problems this can solve.

Wes Miller Field Notes: Donate Your PC

When you donate and reuse old hardware, you keep old hardware out of landfills-and that helps to preserve our environment. These tips can help you effectively delete personal data from your old computer equipment so it will be ready for reuse.

Jim Lynch Windows Confidential: Scrapping the Scraps

Here's some history about how a couple of not so incredibly popular features disappeared.

Raymond Chen


 SharePoint: A Smart Approach to Gathering Data in the Enterprise
Sharepoint Keith Deshaies

A Smart Approach to Gathering Data in the Enterprise
At a Glance:
  • Gathering and processing data
  • Separating presentation logic from information management logic
  • Using Microsoft Office 2007 technologies to build a data-gathering solution
Download the code for this article: DataGathering2008_02.exe (567KB)

Enterprises gather vast amounts of information using a variety of methods. The data arrives via e-mail, surveys, Web forms and other data collection mechanisms. Data, usually, is a good
thing. However, managing the array of data collection tools and all the disparate information is difficult. Reliable integration and secure sharing of data are constant challenges for IT organizations. Standards and service-oriented architectures are evolving, making it easier for IT professionals to make data more accessible, more securely. But while you have the tools and technologies you need to build an efficient enterprise architecture, it is far too common to get caught in a net of proprietary interfaces—and this leads to isolated solutions.
Take the technologies available in the Microsoft® Office system as an example. You can quickly create a departmental survey based on Windows® SharePoint® Services 3.0, but whether this is a standard solution depends on your organization. If your company uses ASP.NET and SharePoint as the platform for Web-based collaboration and data integration, then this survey does provide a standard solution. But if your environment is like the one I work in, SharePoint is just one platform among many.
Granted, SharePoint provides many options for integrating with systems from IBM, HP, Siebel, and so on. That's good news for power users who want to create ad hoc solutions and still have the resulting data flow into a variety of back-end systems. However, if you're a solution architect, there is an even better way to go—InfoPath® 2007.
With InfoPath 2007, which is part of the 2007 Office system, you can decouple the presentation logic of your solutions from the information management logic hosted on your servers. With XML-based InfoPath technology, you can build enterprise-ready data-gathering solutions. And, for the most part, InfoPath form designers do not need detailed knowledge of XML, Web services, solution architectures, ASP.NET, or the SharePoint object model.
In this article, I will discuss how you can build flexible data-gathering solutions using InfoPath 2007, Office SharePoint Server 2007, and Forms Services. And I'll show you how XML enables you to separate the presentation logic from the business logic in a multitier, enterprise architecture.
Note that when I refer to "data gathering" I am referring to the process of collecting information from human sources. There are, of course, other ways to gather data, such as crawling data sources, but those automated methods are beyond the scope of this article.

Acquiring and Handling Data
Data aquisition requirements can be complicated, but these processes have some things in common. By addressing these similarities in centralized modules while handling unique requirements in decentralized components, you can limit redundant efforts, maintenance overhead, and the total cost of ownership.
For example, compliance regulations for public companies result in business requirements that in turn translate into company-wide information management policies. These policies affect data gathering solutions across departmental boundaries and often lead to duplicated efforts within individual departments—for instance, rules around the collection of personally identifiable information gathered by an HR department (handling employee info) and a customer- service department (handling customer info). Even business processes between individual departments that are similar but unrelated provide opportunities for unifying information management solutions.
Figure 1 shows an example of a typical business process. An employee who wants to trade an assignment with a colleague must first obtain an agreement from the colleague, then approval from a manager or coordinator of the assignment schedule, and finally from the supervisor. This could involve employees trading work shifts, for example. Though these exchanges occur in different departments and may rely on different forms, the workflows and information management logic may be shared among the various department solutions.
Figure 1 Sample data-gathering process that may be shared among departments
Figure 1 Sample data-gathering process that may be shared among departments
Of course, consolidating redundant components is a huge task. Driving organizational change across a company is not easy, but with the Office system technologies you can build a solid foundation to facilitate these changes. InfoPath 2007 enables individual departments to create forms applications that integrate with centralized, standardized, information management systems. SharePoint 2007, meanwhile, enables IT departments to delegate administrative control over site collections, sites, and document libraries to individual departments and teams. As a result, teams can build and deploy their own solutions with minimal involvement from IT, while the IT department remains in control of all the shared services and components, such as workflows, information-management policies, and backup procedures.

Centralizing Your Data-Gathering Efforts
Enterprises often give teams departmental application servers to accommodate individual information management needs. The IT department is merely responsible for keeping the hardware and operating system running, while the individual departments take care of all aspects of their solutions. There is little coordination between departments, and information sharing is difficult.
Technical challenges to centralizing data-gathering efforts revolve mainly around the security, performance, maintenance, and support of custom components hosted in a shared environment. For instance, the effects of a malfunctioning component are isolated if individual solutions are hosted on departmental application servers. In a shared environment, however, a malfunctioning component can affect business processes on a much larger scale. It follows that the IT department must establish strict policies regarding the deployment and maintenance of custom components on centralized systems.
Hosting departmental SharePoint solutions on a central server farm requires that you deploy and maintain all the custom components of these departmental solutions on the central application servers. One solution might rely on custom field types to extend the solution's UI with dropdown lists populated from back-end Web services. Another solution might rely on Web Parts for the same purpose while yet another uses custom workflows—all of which are written in managed code and deployed as Microsoft .NET Framework assemblies.
Moving even a relatively small number of SharePoint solutions to a central application server farm can lead to difficult configuration and support issues. If assemblies must be deployed in the Global Assembly Cache (GAC), security becomes an issue because these assemblies run with full trust. Poorly programmed components might open the system to SQL injection, cross-site scripting, or denial-of-service attacks. You need to ensure that the components can sustain the typical workload as well as peak demand and long-running operations. You need to ensure that the components don't block other processes, handling events synchronously, and that the components perform reliable input validation—so users cannot insert SQL statements or scripts into columns used to update a database or remote Web system.
In short, the goal is to emphasize secure and scalable server configuration based on standard product features. By relying on reusable, thoroughly tested solutions, you can avoid the trap of creating numerous custom components. It makes sense to keep the front end decentralized and the back end centralized. The key is to integrate the components in a loosely coupled way that promotes reuse of existing solutions

Splitting the Business Logic
So how do you build flexible data-gathering solutions that can be configured on your servers? The best strategy is to separate the solution architecture into individual tiers as shown in Figure 2: data storage, business logic, and presentation or UI. These days, the UI is typically browser-based while the business logic resides on Web application servers. These, in turn, access databases and non-relational data repositories.
Figure 2 A typical enterprise solution built on a three-tier architecture
Figure 2 A typical enterprise solution built on a three-tier architecture
Business logic often includes transaction logic to ensure that transactions are applied atomically across database management systems. Business logic may also integrate multiple middle-tier services through HTTP, message queuing, RPCs, and so on. The overall solution architecture, however, remains essentially a three-tier model.
What Figure 2 does not illustrate is the complexity of the business logic in an enterprise environment. It looks as if the application server in this figure is merely focusing on rendering a browser-based form and handling the submitted data;, but that representation does not take into account workflows, compliance, or information management requirements. To address these requirements, you need to split the business logic in two—the presentation logic and the information management logic. This allows you to mix and match the middle-tier components as needed without rebuilding components from scratch for each solution.
Figure 3 shows this architecture. At the core is the content or data, surrounded by the information management logic, which governs the content throughout its lifecycle. The presentation logic interfaces with information management logic to provide access to the data via the user interface.
Figure 3 Separating presentation logic from information management logic
Figure 3 Separating presentation logic from information management logic

Gathering and Processing XML
In service-oriented application (SOA) environments, XML is the dominant standard used to define and share data and data structures among components. And XML, therefore, is a good choice for interfacing between presentation and information management components.
Communication must move in two ways: you'll need to translate the XML into a browser-readable document, as well as an XML document generated by the form. Until recently, building XML-based forms applications required extensive programming skills. This was especially true when the resulting XML data had to adhere to an industry schema to facilitate interorganizational information exchange.
InfoPath 2007 makes XML-based forms development much easier. A strong grasp of XML details is certainly helpful, but forms designers and power users need not be XML wizards to build XML-based forms apps. The forms designer simply imports an XML document or XML schema into InfoPath and then maps the individual attribute and element nodes from that data source to the fields in the form. A forms designer can also start with a Web service or a SQL Server® database or from a blank template and build a form from scratch while InfoPath automatically creates the underlying schema and data bindings in the background.
Standardizing forms using InfoPath and XML schemas has several advantages. If you already have a well-defined XML schema, forms designers and developers of workflows and information management components can create solutions against the same data structures. If a forms designer starts from scratch, developers must wait for the form to be finished in order to see how it affects the underlying data structures. And once the data structures are defined, future solutions, such as new form templates, can reuse existing workflows and information management components if they rely on the same data structures. And future workflows and information management components can work together with existing forms and data. If you build your data-gathering solutions based on established industry schemas, the results become even more flexible. In fact, these solutions will be compatible with solutions built by other companies that use the same schemas.
I created a simple DirectReports schema that associates employees with managers. Managers can use the resulting form to evaluate their direct reports. You can find the schema, the form, and a readme.htm with instructions to recreate the form in the Direct Reports folder in the download that accompanies this article, available at technetmagazine.com/code08.aspx. The form is basic, but it illustrates the general concept.
A very important point here: I did not create any validation logic in InfoPath, yet InfoPath still requires that user ID and e-mail addresses be entered in a specific format (domain\account and recipient@domain.tld). Otherwise, the resulting XML document is not valid. This is because the XML schema defines these formats. You can save the form with invalid data but you can't submit it, as shown in Figures 4a and 4b. (I've added a dummy submit rule to the form so you can test this without actually submitting data to any location.) InfoPath 2007 validation automatically ensures that the form is filled out completely and without these sorts of errors.
Figure 4a Validation errors prevent the user from submitting a form
Figure 4a Validation errors prevent the user from submitting a form
Figure 4b The error message that is generated
Figure 4b The error message that is generated
The XML schema serves as the binding contract between the presentation logic and the information management logic. InfoPath locks the schema so the forms designer can't intentionally change the data structures. This is important because changing an established XML schema can potentially break existing enterprise solutions, such as workflow modules that you intend to use in combination with the new forms template.
InfoPath provides an abundance of features for building advanced presentation logic into forms applications. You can consume data from XML files, Web services, SharePoint libraries and lists, databases, and so on to pull in meaningful default values. You can change field values based on user selection through rules, include validation logic, add managed code for the most advanced customization requirements, and use Forms Service to make the form template accessible over the Web. In any case, the data from the form eventually reaches the information management logic as an XML document that conforms to a schema definition.

Working with XML or Metadata
You might wonder whether you should apply information management logic directly to the submitted XML document or instead use a parser that extracts the required information into metadata. SharePoint supports both approaches. Developers are accustomed to working directly with XML documents, but metadata offers more flexibility.
To demonstrate this, I created a simple Web service that parses an XML document passed in from the Direct Reports form shown in Figure 4a. (The source code, setup files, and a readme.htm with step-by-step instructions can be found in the XMLParsingWebService folder in the accompanying download.) The Web service simply reads the manager's user ID from the XML document, splits the user ID into domain and user name parts, creates a message based on these parts, and then raises a generic exception to return and display the processed information in the form of a pseudo-error message in the InfoPath form. This is an easy way to pop up a dialog box in InfoPath after data has been submitted. The Web service works great, but if you change the underlying data source (for example, if you rename the OrgPerson element as Manager in DirectReports.xsd and update the InfoPath form with the new schema as outlined in the readme.htm file) the Web service fails. But this shouldn't be a surprise. The XML document is now different and the old XPath expression to access the user-id element is invalid. The OrgPerson and Manager schemas are almost identical, the InfoPath forms are identical, and the desired processing results are the same, but despite the differences being minimal, you still need to deploy and maintain duplicate Web services.
This is not a good approach if you want to minimize the footprint of custom code on your application servers. In contrast, mapping the XML nodes to metadata fields and performing the processing based on the metadata allows you to use the same workflows and information management logic for similar data structures even though the XML schemas are different. You just need to make sure that the child element maps to the correct metadata field and that the data format meets the processing requirements—then you can reuse the existing code.
Mapping XML nodes to metadata fields is similar to binding XML nodes to UI controls in an InfoPath form, as shown in Figure 5. In SharePoint, metadata fields correspond to columns defined at the site or list level and referenced in content type definitions. Content types define the characteristics of content items, such as metadata fields, workflows, and forms. To keep the metadata fields of a content type synchronized with the corresponding nodes in the associated XML document, SharePoint relies on a built-in XML parser that performs property promotion and demotion. During property promotion, the XML parser extracts node values from an XML document into corresponding columns on the document library. Property demotion refers to the reverse process in which column values are taken from the document library and written back into the document. The most important point is that the XML parser keeps the metadata fields and mapped XML nodes synchronized.
Figure 5 XML Schema mappings between InfoPath and SharePoint
Figure 5 XML Schema mappings between InfoPath and SharePoint
When you use the SharePoint object model, your Web Parts, workflows, and information management logic can work with metadata fields as well as the underlying XML documents. Changing the value of a mapped metadata field changes the node value in the XML document, and vice versa. Yet, working directly with the XML document tightly couples the business logic with the XML schema. Mapped metadata fields, on the other hand, increase the reusability of code. Obviously, you need to decide which approach is right for your environment, but for the most part SharePoint solutions that rely on metadata fields provide more flexibility and more opportunity for code reuse.
To illustrate how SharePoint associates XML nodes with metadata fields, I included a SharePoint feature in the companion material to provision a custom document library (see the OrgPersonContentType.xml file located in the OrgPersonLib folder in the accompanying download). The OrgPerson content type references four fields: UserID, FullName, EMail, and Direct_x0020_Reports. FullName and EMail are built-in fields. UserID and Direct_x0020_Reports are custom fields defined in OrgPersonSiteColumns.xml.
The field definitions are fairly straightforward. It is possible to bind fields to XML nodes directly in the field definitions, yet it is also possible to overwrite this information in the content types. I decided to use the latter technique because this let me use the custom fields in content types not related to XML documents as well as in content types that rely on different XML structures. The OrgPerson content type binds the metadata fields to XML nodes that match in their arrangement the OrgPerson schema I discussed earlier. There is also an AdditionalContentTypes.xml file that defines more content types that bind the same metadata fields to different XML nodes. You can see the differences if you look at the XPath expressions in the Node attributes.
Document libraries of the OrgPersonLib type can store different types of XML documents while the node values are exposed through the same library columns. This simple mapping technique also lets you reuse workflows and information management logic since the four content types (OrgPerson, Manager, Supervisor, and User) reference a common set of metadata fields.
Figure 6 shows you the FieldRef element from the OrgPerson content type for Direct_x0020_Reports, which maps the field to the user-id nodes of direct reports according to the XPath expression, /OrgPerson/direct-report/user-id. Since the XML document can include multiple direct report entries, it is important that you specify an Aggregation attribute. This defines how the XML parser will handle the returned collection of values. If you omit this attribute, the XML parser extracts only the first node value. Supported aggregation values are sum, count, average, min, max, merge, plain text, first, and last.
Figure 6 Metadata field mapped to an XPath expression
Figure 6 Metadata field mapped to an XPath expression
All of the sample content types use the standard upload.aspx page as the DocumentTemplate so that you can upload XML files into the document library when you click on the New button in the SharePoint UI. As long as you upload files with an .xml file-name extension, SharePoint will automatically invoke the built-in XML parser (one exception is WordProcessingML files, for which SharePoint invokes a WordProcessingML parser). The XML parser examines the uploaded .xml file to determine the associated content type. This is so it can extract the node values from the locations specified in the field definitions and perform property promotion. (You can verify this process when you upload the OrgPerson.xml file included in the OrgPersonLib\XMLFiles folder.) The structure of this XML document matches the XPath expressions specified in the OrgPerson content type definition. Accordingly, SharePoint extracts the data from the .xml file, writes the data into the corresponding library columns, and displays the data in the EditForm.aspx page so you can verify and update the document properties that are not marked as read-only. Figure 7 shows the EditForm.aspx form with the data extracted from OrgPerson.xml.
Figure 7 EditForm.aspx form with extracted data
Figure 7 EditForm.aspx form with extracted data
If you change the User ID, Full Name, or E-Mail value in EditForm.aspx, SharePoint performs property demotion to change the node values in the underlying XML document. Note, however, that SharePoint does not enforce XML Schema restrictions unless you implement the required logic into the form yourself.
SharePoint also does not run the presentation logic of a forms application. For example, when you change the User ID, SharePoint does not validate that the new value conforms to NetBIOS conventions and does not automatically update the Full Name and E-Mail fields to match the new selection. Thus, you should mark the corresponding column in the content type definition as read-only if changing an individual field may cause inconsistencies. This forces the user to use the forms application, such as InfoPath, to update the data. And the XML parser will promote any changes from the XML document to the corresponding metadata fields in SharePoint.
In the OrgPersonLib sample, you can upload any of the .xml files from the OrgPersonLib\XMLFiles folder. The .xml files use very different data structures, but SharePoint recognizes the content types and promotes the correct node values into the site columns. This is because I used a processing instruction in the XML files to associate the XML documents with their corresponding content types. The OrgPerson.xml file, however, doesn't include this information, but this is not a problem. If SharePoint cannot associate an XML doc with a content type through a processing instruction or the document template, SharePoint uses the default content type. In the OrgPersonLib case, this happens to be the OrgPerson content type and therefore the XML doc is parsed correctly.
Figure 8 shows how you can associate an XML doc explicitly with a content type. The MicrosoftWindowsSharePointServices processing instruction defines the ContentTypeID as 0x010100668E393E4F0EFF4294DBD202D5D8321D. This happens to be the ID of the User content type, as defined in AdditionalContentTypes.xml.
Figure 8 Processing instructions and XML data of the User.xml sample file
Figure 8 Processing instructions and XML data of the User.xml sample file
The XML parser processes the MicrosoftWindowsSharePointServices processing instruction and writes the ContentTypeID value into the ContentType metadata field. All SharePoint content types share this metadata field because it is defined at the root level in the System content type. If you open the fieldswss.xml file on a SharePoint server (located in %CommonProgramFiles%\Microsoft Shared\Web Server Extensions\12\Template\Features\Fields folder) and search for MicrosoftWindowsSharePointServices, you can see how SharePoint associates the processing instruction with the ContentType field. The PITarget attribute points to MicrosoftWindowsSharePointServices (this is the processing instruction) and the PIAttribute points to ContentTypeID (which contains the ID of the User content type).

Content Type Associations in InfoPath
The technicalities of XML parsing and content type associations are intimidating for many forms designers, but InfoPath 2007 takes care of all the nitty-gritty details. The readme.htm file that accompanies the OrgPersonLib sample includes instructions to publish the Direct Reports form template in SharePoint and create a content type that binds yet again to the same metadata fields (UserID, FullName, EMail, and Direct_x0020_Reports). You can easily add the new content type to the OrgPersonLib document library in the SharePoint UI. But the new content type also points to the InfoPath form template as the document template to invoke the forms application when updating existing XML documents. Figure 9 illustrates how the InfoPath Publishing Wizard simplifies property mapping between XML node values and SharePoint site columns. And, again, if you associate the node values with existing site columns, you can reuse existing SharePoint components.
Figure 9 Property mapping in InfoPath 2007
Figure 9 Property mapping in InfoPath 2007

Wrapping Up
Additional Online Resources
  • InfoPath 2007 Team Blog
  • W3C XML Schema Specifications and Development
  • Introduction to XML Schemas
  • Workflows in Office SharePoint Server 2007
  • InfoPath 2007 Forms Services

With technologies available in Office, enterprise architects can build data-gathering solutions that readily promote code reuse and information exchange. InfoPath 2007 allows departments to create solutions that can pull information from various sources, and the data can then be submitted to various systems, such as SharePoint. SharePoint also provides developers with a comprehensive set of Web services and interfaces to build workflows and information management components. By hosting these components on centralized SharePoint servers, departments then have the infrastructure they need to build their individual applications.
Meanwhile, individual departments can create their data-gathering solutions faster. Compliance regulations and other global business requirements can be addressed at a cross-departmental level, and the maintainability and reliability of the IT environment increases with the use of fewer custom components on application servers.

Keith Deshaies is a freelance technical writer and an IT analyst for a large telecommunications company. He specializes in Microsoft Office and SharePoint technologies and is a member of the Society for Technical Communications.
© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


 SharePoint: Building a Powerful Survey Infrastructure
Sharepoint Jim Bradley

Building a Powerful Survey Infrastructure
At a Glance:
  • Planning and implementing a survey solution
  • Gathering and processing survey results in SharePoint libraries
  • Managing performance and security issues

You can't make good decisions without information. Hunches just don't cut it—not when your decisions will impact your business. This is true whether you're deciding where to host a team
party, or if you're contemplating the reallocation of 50 percent of your company's resources to launch a new product. But how do you get this information?
Surveys provide an effective—and cost-effective—way to collect feedback on everything from how satisfied customers are with your product offerings to whether or not employees liked the sandwiches at your latest team meeting. Surveys yield the substance needed for future product, system, or process improvements and development. The question is, how do you, the IT professional, implement a survey system that can collect data and store it so the information can be used in meaningful ways?
You probably already have one or more of the tools needed to gather, structure, and analyze this sort of data. The 2007 Microsoft® Office system provides these tools and makes them quite easy to use. In fact, there are numerous ways you can go about performing a survey. The key is to match the approach to your particular needs. For instance, e-mail-based surveys are a good choice for impromptu, real-time data gathering when dealing with a small number of respondents and non-critical data. For more information, see the sidebar below "Using E-Mail for Quick Surveys". Database-driven surveys, on the other hand, are a better choice for more complex initiatives that include a larger group of respondents and pertain to mission-critical data. See the sidebar below "Build a Database-Driven Survey Solution" for more information on this topic .
In this article, I'll focus on another, more comprehensive and flexible solution: SharePoint®-based online surveys. This approach is suitable for both mission-critical and non-critical surveys of any level of complexity. Because SharePoint surveys are Web-based, they can be completed by anyone who has access to a Web browser—even mobile devices are supported. With a SharePoint survey, responses can be named or anonymous, real-time results are available, and you even have access to analysis tools.
This survey type is implemented using either Windows® SharePoint Services 3.0 (WSS) with Forms Server and InfoPath® or Microsoft Office SharePoint Server 2007 (MOSS) with Forms Services and InfoPath® (see Figure 1 for a description of each of these technologies). Because both scenarios offer essentially the same functionality, I will focus this discussion on the latter combination. For a comprehensive comparison of SharePoint products and their functionality, see Microsoft Office SharePoint Server 2007 Products Comparison Download at office.microsoft.com/en-us/sharepointserver/HA101978031033.aspx.
Figure 1 Microsoft technologies that enable SharePoint surveys (Close Figure 1)

Technology | Description
Microsoft Office InfoPath 2007 | Microsoft Office InfoPath 2007 is a forms creation and information gathering tool. For additional information, see office.microsoft.com/infopath.
Windows SharePoint Services 3.0 | Formerly SharePoint Team Services, Windows SharePoint Services 3.0 acts as a foundation for the creation of extended SharePoint applications. For additional information, see microsoft.com/technet/windowsserver/sharepoint/techinfo/overview.mspx.
Microsoft Office SharePoint Server 2007 | Microsoft Office SharePoint Server 2007 (formerly SharePoint Portal Server 2003) provides a server-side infrastructure that turns Office 2007 clients into generators and consumers of content for SharePoint applications. For additional information, see microsoft.com/sharepoint.
InfoPath Forms Services, Microsoft Office Forms Server 2007 | InfoPath Forms Services allows users to complete InfoPath forms in a Web browser, without having InfoPath installed, making surveys cross-platform, cross-browser compatible. Forms Services requires Windows SharePoint Services 3.0, though the same functionality is also available as a standalone product called Microsoft Office Forms Server 2007. For additional information on Forms Services, see microsoft.com/ms540731. For more information about Forms Server, see office.microsoft.com/en-us/formsserver/FX100490391033.aspx.

Plan and Implement
When designing a survey, you need to consider a number of factors up front. This takes some advance analysis, so be sure to take the time to develop a well-thought-out survey plan. In the initial planning stage, you'll want to define the problem space and the sort of data you are looking to obtain, determine the technologies you need to use, and establish the budgetary and administrative requirements. Then you can move on to the implementation. The general workflow is to design a survey form, publish the form, collect and validate responses, aggregate and analyze the data, and then report the results.
A survey implementation workflow typically includes at least a survey designer, respondents, and an analyst (as shown in Figure 2). Of course, survey workflows can vary greatly in complexity. A company-wide survey, for instance, would likely go through several rounds of design edits and approvals. It would probably utilize automated reminders to respondents, offer various forms of support and troubleshooting, provide real-time views into the results for managers, and have a clear process for determining and delivering the final analysis. A complex workflow of this sort really requires a Web-based system that supports workflow and reporting capabilities.
Figure 2 A survey workflow from form design to data analysis
Figure 2 A survey workflow from form design to data analysis

What to Look for in a Comprehensive Solution
There is a long list of key requirements you should expect a comprehensive survey solution to satisfy. The solution should allow individual teams or business units to create, disseminate, and collect survey results with minimal involvement from the IT department—and it should not require any programming skills. The forms creation tool should offer an easy-to-use WYSIWYG interface, as well as a rich set of features that facilitate piping, branching, and conditional logic. And, it goes without saying, the survey-form creation tool should accommodate all surveys, regardless of size and complexity.
Moreover, the survey should run on a Web server and integrate with a centrally maintained SQL Server® database, eliminating the need for individual departments to maintain dedicated SQL databases. Any respondent with Internet access should be able to complete the survey from any standards-compliant browser.
The survey process should be easily linked to workflows and it should meet business needs without compromising security. And, for many organizations, the survey solution should provide multi-language support.
When used together, MOSS 2007, WSS 3.0, and InfoPath 2007 create an integrated solution that addresses all of these requirements. Figure 3 shows how the various components fit together in the SharePoint stack. But before I discuss this complete survey solution, I want to take a quick look at what you can achieve with WSS alone. Then I will show you the added benefits you get when adding MOSS and InfoPath to the mix.
Figure 3 Components of the SharePoint stack
Figure 3 Components of the SharePoint stack

Working with WSS Alone
Even without the other components, you can use WSS to create and implement a survey. In fact, WSS includes a built-in survey template that makes the process easy. To create a survey, click Start | All Programs | Administrative Tools and select SharePoint 3.0 Central Administration. From the dropdown list, select Site Actions and then click Create. The Create page is displayed, showing you headings for Libraries, Communications, Tracking, Custom Lists, and Web Pages. Under each heading, there are template options. Under Tracking, click Survey.
At this point, WSS leads you step-by-step through the survey creation process. You can create open or closed questions and specify whether a question can be left unanswered. You can even create branching logic surveys, which will lead respondents down different paths, depending on their answers. Surveys can be anonymous or, if necessary, identify the respondent. You also have the option of assigning a workflow to the survey.
The survey is browser-based, so no special software is required to create or respond to the survey. To complete the survey, respondents simply go to the SharePoint site and complete the form. Access permissions are inherited from the parent site but can be edited directly from the SharePoint Actions menu. Survey responses are saved to the SharePoint survey site, and results can be viewed in list form or as graphical summaries or exported to Excel®.
To reiterate, this solution (with the process shown in Figure 4) can be achieved with WSS and no additional software. Assuming you already have a well-designed SharePoint site in place, an individual department should be able to create and implement a survey with little to no assistance from IT.
Figure 4 A survey based on Windows SharePoint Services
Figure 4 A survey based on Windows SharePoint Services
One limitation of the WSS-only process becomes apparent when a survey needs customization. WSS creates survey lists using various default ASPX pages (AllItems.aspx, DispForm.aspx, EditForm.aspx, NewForm.aspx, overview.aspx, and summary.aspx). While page customization beyond the parameters inherent in WSS is possible, most features come from Web Parts that are not readily customizable. Moreover, the WSS-only solution is best suited to surveys that do not need to integrate external data sources, such as your company's ERP system.
When you do need to expand the survey solution to include a custom UI or integrate with additional data sources, you should seriously consider the solution that combines MOSS, WSS, and InfoPath.

The Complete Survey Solution
MOSS includes many features that extend WSS, but for now I want to focus on a single component—InfoPath Forms Services. With InfoPath Forms Services, only the survey designer needs InfoPath installed on her computer. Others can access the survey through a Web browser.
In InfoPath, the designer selects the option that enables the form to be filled out using a browser. InfoPath then creates a form that can be viewed on any standards-compliant Web browser. This Web-based form performs in the same way as an InfoPath form, with the exception of some advanced InfoPath features (such as user roles, vertical text, dialog box actions, and advanced controls). The form is then published to a SharePoint library or list. Meanwhile, another InfoPath feature, called Design Checker, ensures that the survey form is compatible with InfoPath Forms Services.
There is no difference between an InfoPath forms library and a browser-enabled forms library. In each case, a form template is an .xsn file. Either InfoPath Forms Services displays the form in a browser or the file is downloaded to the client and displayed directly in InfoPath. If a survey designer has at least Contributor permissions for a SharePoint site, he can use SharePoint document libraries to publish form templates.
A respondent with a standards-compliant browser can complete the InfoPath Forms Services survey, and his responses are then seamlessly returned to the SharePoint server. Within SharePoint, the data can be securely stored, parsed, shared, and analyzed (using Excel, SQL, or other tools available on the network). All this can be accomplished with minimal IT administration assistance.

Postbacks and Performance Issues
There are potential performance issues, however. For example, browser-enabled forms run in the context of a system account on the SharePoint server, which means that if a form includes code or data connections (as shown in Figure 5), the code or data connections are executed on the server, not the client. And complex forms may need to frequently post data back to the server, which increases the workload on the server.
Figure 5 InfoPath Forms Services-based survey solution with additional data connections
Figure 5 InfoPath Forms Services-based survey solution with additional data connections
The more that a survey uses postback, the higher the load on the front-end Web servers. However, there are times when postback is unavoidable, such as when implementing response branching. To minimize potential adverse effects on the system, survey designers must be aware of postback issues and understand how to best implement surveys when postback can't be avoided. For instance, designers should use wizard-like surveys that post data back to the server when the user clicks the Next button. This helps reduce the number of postbacks.
InfoPath Forms Services must maintain the state of each currently active form on the server. The default session timeout is 60 minutes, which means that if a complex survey takes longer than 60 minutes to fill out and there are no postbacks during that time, the session will be closed on the server. Data that is entered but not submitted is lost, and the respondent must start again.
Such issues are magnified when a large number of respondents are working concurrently on a survey form and when surveys have large data sources or include file attachments. For more information, see "Improving the Performance of InfoPath 2007 Forms" at msdn2.microsoft.com/bb380251.

Security Considerations
While an in-depth security discussion is beyond the scope of this article, it is important to highlight a few security considerations. For starters, SharePoint and InfoPath 2007 adhere to the Trustworthy Computing Initiative that Microsoft adopted in early 2002. InfoPath forms have three possible security levels: Restricted, Domain, and Full Trust. And, by default, InfoPath automatically determines and applies a recommended security level to the form.
Digital signature functionality can help insure that a form is created or filled out by a specific user and that it is not altered. And Information Rights Management (IRM) can restrict access to templates and completed forms. To prevent malicious users from uploading forms that contain harmful code or using the SharePoint platform to launch attacks against other systems through data connections, InfoPath distinguishes between user-deployed and administrator-deployed forms. Users can upload forms as long as the forms do not contain any custom code and use only the Domain security level, which limits cross-domain data connections. These parameters are sufficient for the majority of survey solutions.
When a survey design requires the Full Trust security level, in order to permit unrestricted access to resources, form publishing requires IT administration approval. By default, IT administration approval is required when a form contains managed code, when cross-domain data connections are defined in the form template, when the form uses data connections defined in the centrally managed data connection library, and when the option to support rendering on mobile devices has been enabled.
For the Full Trust security level, the survey designer creates an InfoPath form and saves the form template. The template or .xsn file is sent to the SharePoint administrator as an e-mail attachment or passed along through a network share. The IT administrator then checks the survey features and any code it contains before making the survey available on the production system. The IT administrator completes deployment by uploading the survey template to a site collection and activating the survey template. Both procedures are completed on the Application Management page of the WSS Central Administration console. The end result is that an IT administrator can delegate forms publishing to individual departments and still maintain oversight of the publishing process if forms go beyond basic data-gathering activities.
IT administrator oversight is necessary in certain common scenarios: when forms must be populated with default data and when survey results are submitted to multiple data sources. By default, Domain Trust form templates cannot establish cross-domain data connections, but there are a few actions you can take to resolve this limitation including:
  • Granting the form Full Trust permissions.
  • Creating Domain Trust using data connections from the data connection library.
  • Using Domain Trust with the centrally managed connection library.
Full Trust and the centrally managed connection library options require administrator approval during forms publishing. However, the data connection library (DCL) allows user-published forms to cross the domain boundaries using the DCL maintained at the site-collection level. This DCL can be under the control of the individual department. It's important to understand, though, that allowing individual departments to define their own server-based data connections can be a security issue.
The most secure option is to define data connections in the centrally managed connection library and then have an IT administrator deploy advanced survey forms that use these data connections. The centrally managed connection library is advantageous because it is available across all site collections and the entire server farm. This allows the administrator to define central authentication settings for accessing data sources that are not on the local SharePoint server. For additional information, read the article titled "About Data Connections, Authentication, and Alternate Access Mapping," online at msdn2.microsoft.com/ms771995.

Wrapping Up
Surveys provide a powerful way to gather mission-critical (and not so mission-critical) information for every business, at every level. Creating an effective survey, however, requires the right tools, a sound plan, and a balanced mix of art and science. Thoughtful planning upfront will help you build a solution that allows each department within your organization to create, send, receive, store, and analyze survey data.
The most effective surveys are those that yield timely, relevant information. In order to gather information when it is needed, your solution must make it easy to create and manage surveys at the department level—without compromising security. InfoPath empowers information workers to create high-quality surveys by providing an intuitive interface that can be used with little training. And InfoPath lets you gather high-quality survey data without requiring a lot of IT administration.
InfoPath 2007 seamlessly works with WSS and MOSS to facilitate rich, complex survey solutions that are cross-platform and cross-browser compatible. And this allows your business to harness one of its most important assets: information.
Using E-Mail for Quick Surveys
Impromptu surveys are an essential tool for quickly collecting data from a small group of respondents. E-mailbased surveys are perfect for this sort of real-time research—they can be created quickly and don't require dedicated servers or IT assistance. One of the simplest approaches, voting buttons embedded inside a Microsoft® Outlook® message, lets you send, receive, and tabulate a one-dimensional survey.
After launching a voting buttons survey, your recipients receive an e-mail message that contains a voting band, which displays a dropdown list of options. The recipient responds by clicking one of the options. A popup window then displays a message, "You have chosen to respond: <your choice>", and offers the choice to Send or Edit the response.
The survey designer, or an assigned delegate, receives the individual responses in e-mail and manually tabulates the results. If the survey includes a large number of recipients, you should create a rule in Outlook (using the Rules Wizard) that will automatically route the responses to a dedicated folder, as demonstrated in Figure A.
Figure A Survey responses automatically routed to a dedicated folder
Figure A Survey responses automatically routed to a dedicated folder
Voting buttons, however, are quite limited. Survey forms inserted into e-mail messages can collect much more data, but manually retrieving and tabulating this data can be very unmanageable. On the more complex side of e-mail-based surveys is the use of VBScript to customize Outlook surveys. Customized forms can make it easier to gather and analyze larger quantities of more complex data. To access developer functionality in Outlook 2007, you must first display the Developer tab, shown in Figure B.
Figure B Developer tab in Outlook 2007
Figure B Developer tab in Outlook 2007
Creating custom Outlook forms is beyond the knowledge of the typical Outlook user. But InfoPath® 2007 provides new features that make it easy to implement customized forms. With no special expertise, survey designers can create InfoPath forms and templates in three ways:
  • By importing existing Microsoft Word documents or Excel® spreadsheets.
  • By downloading predesigned InfoPath templates and modifying them to meet specific needs.
  • By designing templates from scratch using the form design functionality of InfoPath.
In all three cases, templates can be published to Outlook and distributed via e-mail. With InfoPath 2007 and Outlook 2007, a few hours, and little to no assistance from IT, an information worker can create a twenty-question survey, e-mail it to a group, receive the results in Outlook, and analyze the data in an Excel spreadsheet.
The only real requirement in this scenario is that all participants have InfoPath and Outlook installed on their computers. And since InfoPath 2007 actually extends the functionality of Outlook 2007, the recipients can respond to the survey right from within the Outlook interface.
But InfoPath supports much more than just e-mail-based surveys. You can create templates that query and submit data to Web services and SQL Server® databases, or you can use an existing XML document or XML Schema as the data source. In other words, with minimal training and little or no added code, a user can tap into the power and flexibility of XML.
E-mail as a survey solution has some notable disadvantages. For example, these surveys are not anonymous, as the respondents e-mail address is available. This can reduce candor and skew the results. There are also security concerns, such as phishing attacks, so there is a limit to the type and depth of information you can gather using e-mail-based surveys.

Build a Database-Driven Survey Solution
E-mail-based surveys aren't suited to more complex studies that involve large numbers of respondents—using a folder in Outlook® just isn't sufficient for managing and analyzing all the data. A more robust solution is to direct survey responses to a database. The advanced storage, indexing, processing, and reporting capabilities you get with a database can be very helpful. Microsoft® SQL Server® 2005, for example, includes SQL Server Reporting Services, which can be used to process survey results automatically.
To create an InfoPath form template that will submit responses to a database, you must first create an Access® or SQL Server database within your network. Then, in InfoPath®, you can start the template creation process using the Design a Form Template wizard. This opens the Data Connection Wizard, which steps you through the process of linking your form to the database. InfoPath uses the information in your database to build the data source query and data fields.
Most workers will have enough knowledge to complete and submit the survey with little training or support. However, in this scenario (where data is passed directly to the database), respondents must have InfoPath and Outlook installed, and they must be behind the corporate firewall or have VPN access to the corporate network.
The InfoPath Data Connection Wizard interfaces only with Microsoft SQL Server 2000 or later and Access databases that are natively using ADOXML. Direct submission to the database is not supported for other databases. Other limitations are that InfoPath does not allow rich text controls to be bound to database fields, and large binary data types are not supported.
As the network topology becomes larger and more complex, it becomes increasingly advantageous to submit data to an intermediate Web service and then pass that data to a database. Decoupling the survey form front end from the database back end, by means of a Web service, also facilitates implementation of business logic on the Web server. And since the Web service is performed over HTTP or HTTPS connections, participants can submit data through firewalls, as shown in Figure C. Note, however, that respondents still must have InfoPath 2003 or 2007 installed.
The downside to this scenario is that the implementation requires IT participation. Describing this process in detail is beyond the scope of this article, but I will warn you that building a database solution from scratch is a significant undertaking. It requires advance planning, good survey design, programming of the Web service, and design of the database back end. InfoPath 2007 simplifies the forms design aspect of the process, but the end-to-end solution requires solid programming and database design skill. And adding advanced functionality, such as workflow or multi-language support, quickly increases the cost of implementation.
Figure C InfoPath data flow through a Web service
Figure C InfoPath data flow through a Web service

Jim Bradley is the owner of CoyoteTech LLC, a technical communications company that specializes in user assistance documentation for Microsoft server products. Documentation projects include "Microsoft Exchange Server 2007 Edge Transport and Messaging Protection", "Going 64-Bit with Microsoft Exchange Server 2007", and "Microsoft Exchange 2003 Technical Reference Guide".


 SharePoint: Standardize Data Management with Custom Content Types
Sharepoint Pav Cherny

Standardize Data Management with Custom Content Types
At a Glance:
  • Content lifecycle management with SharePoint
  • Creating document information panels
  • Editing Office and SharePoint documents via InfoPath
Download the code for this article: ContentTypes2008_02.exe (779KB)

The large number of documents and other content items typically found in an enterprise environment pose business and technical challenges for managing the documents, their
metadata, and their behaviors in a centralized and reusable way. Microsoft® Office SharePoint® Server 2007 promotes companywide collaboration by allowing diverse teams within an organization to share workspaces on Web sites, document libraries, and lists.
SharePoint 2007 makes it possible to standardize many aspects of content and lifecycle characteristics through content types. Site content types are metadata definitions that can be established independently of any specific site collection, site, list, or document library. This enables you to establish companywide properties, workflows, information management policies, and other elements consistently while also allowing individual departments or site owners to customize content types for specific purposes.
In this article, I will show you how to use the new SharePoint content model introduced with Windows® SharePoint Services (WSS) 3.0 and Microsoft Office SharePoint Server (MOSS) 2007 to build hierarchies for enterprise content based on general characteristics. These content hierarchies enable uniform application of metadata, workflows, and information management policies at a global level, while also providing the flexibility to accommodate unique content management needs at the level of site collections, sites, document libraries, and lists.
To illustrate some of the low-level aspects of SharePoint content types, I have included a number of custom tools in the companion material, and I have also included the source code in case you want to extend these tools according to your specific needs. Just keep in mind that these tools have not been tested thoroughly and should not be used on a production system. You can download the code from the TechNet Magazine Web site at technetmagazine.com/code08.aspx.

Content Lifecycle and Content Type Definitions
SharePoint Resources
  • InfoPath Team Blog
  • Workflows in Office SharePoint Server 2007
  • TechNet Webcast: Installing and Configuring Search in SharePoint Server 2007
  • InfoPath Forms Services

There are many details to consider when managing documents and other content in an organization. Among other things, it is necessary to define which person or process needs to perform what action as content is created, published, archived, and disposed. It is often also necessary for an organization to develop specific templates for content creation, define roles to assign responsibilities and access privileges to users, provide version control, monitor compliance, store and archive, define metadata, and so forth.
However complex a particular content lifecycle may be, the SharePoint content model recognizes that there are some general characteristics and typical phases that determine how the individual content items should be handled. For example, you can structure content creation through templates and input forms, and content display and searches through metadata. You can also use editing, approval, or other workflow requirements, archiving requirements, expiration timeframes, and applicable information management policies to distinguish individual content pieces. Some content might not require any special templates or might never be archived, but even these are lifecycle characteristics that proceed to distinguish these items from other items.
The SharePoint content model allows you to define individual content types and establish hierarchical relationships. In a hierarchical relationship, children inherit general characteristics from parent content types, adding specific characteristics as needed.
The best way to illustrate this is to examine the built-in content types of SharePoint. WSS 3.0 and MOSS 2007 include a number of predefined content types for typical items that can be stored in a document library or list, including documents and tasks. You can find the definitions of these standard content types on a SharePoint server in the %CommonProgramFiles%\Microsoft Shared\Web Server Extensions\12\Template\Features\Ctypes folder. There you'll find a manifest file called feature.xml. Looking at this file, you can see that SharePoint implements the standard content type definitions as a hidden feature (Hidden="TRUE") with a site collection scope (Scope="Site") and that the Element manifest file, which contains the actual content type elements, is ctypeswss.xml (<ElementManifest Location="ctypeswss.xml" />).
If you are interested in learning more about SharePoint features, I recommend you read Ted Pattison's Office Space column in MSDN® Magazine titled "Features for SharePoint," which is available at msdnmagazine.com/issues/07/05/OfficeSpace.
Now, let's open the ctypeswss.xml file in Notepad to examine the standard content types regardless of their visibility in the SharePoint user interface. You should not modify ctypeswss.xml. If you're thinking of editing ctypeswss.xml to add new fields to standard content types or to make hidden content types visible so that SharePoint users can use them to derive new content types, note that this is usually not necessary. Plus, it leads to unsupported configurations, and later installation of service packs may overwrite your customizations and break your content management solutions.
A much better approach is to copy what you need into a new element manifest file, add your customizations as necessary, and then deploy your custom content types as a new feature with a site collection scope so that they are available to all sites within the site collection.
The Collaborative Application Markup Language (CAML)-based definition of the System content type as specified in ctypeswss.xml is revealed here:
<ContentType ID="0x"
    Name=$Resources:System
    Group="Hidden"
    Sealed="True"
    Version="0">
   <FieldRefs>
      <FieldRef ID="{c042a256-787d-4a6f-8a8a-cf6ab767f12d}" Name="ContentType"/>
   </FieldRefs>
</ContentType>
The Group and Sealed attributes show that the System content type is hidden and sealed so that it cannot be changed in the SharePoint user interface. The System content type has only one FieldRef element that references a built-in site column called ContentType. This is the highest level of abstraction. All other SharePoint content types inherit this property from System content type. In this way, SharePoint ensures that all content items stored in any document libraries or lists have this property in common.
Figure 1 shows you a ContentTypeHierarchy Web Part, included in the companion material for this article, which illustrates the hierarchies more intuitively. System is at the root of all other content types, followed by Item, and so forth. The Item content type, for example, establishes the next level of detail. If you check ctypeswss.xml, you can see that Item defines a single metadata field that references a site column called Title. In that way, all built-in content types at lower levels have a Title property.
Figure 1 WSS 3.0 built-in content type hierarchy
Figure 1 WSS 3.0 built-in content type hierarchy
It is also possible to remove an inherited field, as the Document content type definition in ctypeswss.xml demonstrates. The Document content type includes several corresponding RemoveFieldRef elements, yet you might want to keep the Title field in place in your custom content types because the Title column provides access to the Edit Control Box (ECB) dropdown menu in SharePoint document library and list views.
A good example that illustrates how to rename inherited fields is the Far East Contact content type in ctypeswss.xml. Search for 0x0116, which is the corresponding content type ID, and then check the FieldRef element with the attribute Name="Title" to see how you can use the DisplayName attribute to rename a field in the user interface. In this particular example, the DisplayName attribute changes the name of the Title field in the user interface to a localized data value referenced by "$Resources:core,Last_Name;".
If you take a closer look at Figure 1, you can see the ID attribute of the ContentType element uniquely identifies the content type and establishes the hierarchical relationship. All IDs begin with the ID of the parent content type appended with additional hexadecimal values. Standard content types typically have two additional hexadecimal values appended to create a new unique ID for the child content type. Another technique is to append a "00" and a hexadecimal GUID. That's how the Office Data Connection File and Universal Data Connection File content types derive from the Document content type, for example.
An important point for users to remember is that the ID attribute of the ContentType element cannot be longer than 1,024 characters. This can turn out to be a problem in a large hierarchical relationship if all content types use the hexadecimal GUID addressing technique. However, it is not a good idea to start with the shorter technique because these IDs may not be unique.
To ensure uniqueness, use the GUID technique to establish a global namespace for your enterprise content types and switch to the shorter technique within that namespace. For detailed information about the ID attribute and other elements of content type definitions, see the topic "Content Type Definition Schema" in the WSS 3.0 SDK.

Content Type Dependencies
Now let's turn to the question of how to use SharePoint content types to structure the management of content items. You need to consider several dependencies, such as the interdependencies among document libraries and lists, content types, site columns, and field types. Libraries and lists reference content types, which reference site columns, which in turn reference field types (such as the standard WSS field types text, note, choice, number, currency, and so forth), which in turn reside in Microsoft .NET Framework assemblies, such as the WSS core assembly Microsoft.SharePoint.dll.
For an example, let's return to the System content type as illustrated earlier in the CAML definition of the System content type manifest file entry. As you saw, this content type contains a FieldRef element that refers to a site column called ContentType. Note that the content type definition does not define the actual ContentType site column. ContentType is a Text field, defined in the Elements manifest fieldswss.xml, located in the %CommonProgramFiles%\Microsoft Shared\Web Server Extensions\12\Template\Features\Fields folder. The Text field type, again, is defined in fldtypes.xml, which you can find under %CommonProgramFiles%\Microsoft Shared\Web Server Extensions\12\Template\XML. An important point to take away from this dependency tree is that you must ensure that all of the resources are available on your SharePoint servers.
The content types you want to use must be created at the site level of your document libraries and lists or higher up in the site hierarchy. Similarly, the metadata fields of your content types must refer to existing site columns. You can add standard or custom site columns based on standard or custom field types to your content types, but you must ensure that the site columns exist on the local site. Furthermore, if you want to use custom field types, you also need to ensure that the underlying .NET assemblies are deployed on your SharePoint servers.
Similar dependencies exist for content types that reference document templates, custom event receivers, workflows, and other components. For example, the content type definition can contain a DocumentTemplate element that points to the document template associated with the content type. The content type definition can also contain an XmlDocuments element with one or multiple XmlDocument child elements that define additional characteristics of the content type, such as namespace definitions, document information panel definitions, or any custom information.

Establishing Global Content Types
Users with authoring permissions can create new content types and site columns directly in the SharePoint user interface, but the content types are then only available in the local site and below in the site hierarchy. Custom site columns are only available in the local site. This is not sufficient if you want to establish global content types. You need to ensure that global content types are available across all site collections in your environment. This is where SharePoint features come in handy. It is straightforward to install and activate SharePoint features across multiple site collections to apply the corresponding site column and content type definitions consistently across all locations.
The companion material for this article includes a sample feature called AdventureWorks that demonstrates how to establish a global content type. The feature defines a general content type called Customer Documentation that can be used to create any type of customer materials such as proposals and sales letters. It is reasonable to assume that all these types of documents should be associated with customer information such as company name, contact details, and address. I specifically created a custom field called Customer Name for your content type and added some built-in fields such as Department and Primary Phone. You can change the content type and fields by editing the ContentTypes.xml file and the field references by editing the SiteColumns.xml file in the companion sample.
If you deploy the feature as described in the readme.htm file, you can activate it consistently across multiple site collections. The Customer Documentation content type is then available globally. With that, the individual departments can create specific customer documentation through derived content types associated with targeted document templates. The companion material includes sample document templates that could be used in sales and consulting. Figure 2 shows the resulting content types.
Figure 2 Parent and child content types for customer documentation
Figure 2 Parent and child content types for customer documentation
You have several options to choose from to create features for custom site columns and content types in WSS 3.0 and MOSS 2007. You can study the WSS 3.0 SDK and write XML files from scratch. You can also choose to use SharePoint Designer to export a list into a personal Web package, rename the resulting .fwp file into a .cab file, extract the manifest.xml file from the .cab file, and then search for ContentType definitions in the manifest.xml file. Unfortunately, both approaches are inconvenient and error prone.
In contrast, it is remarkably easy to create custom site columns and content types in the SharePoint user interface. Yet, the interface does not provide an option to edit the XML files of a feature. Features are an efficient way to provision SharePoint resources, but once provisioned the resources only exist in the content database. Perhaps a future version of WSS will include the ability to export site columns and content types into XML files through the SharePoint user interface, similar to exporting Web Parts. For now, you need to work with what's here.
The companion material includes a Web Part called ContentTypeSchema that can serve as a starting point. It uses the SharePoint object model to extract the SchemaXML property from the selected content type. Through eXtensible Stylesheet Language Transformation (XSLT)-based transformations, the Web Part derives Field definitions or ContentType definitions depending on the option you select in the user interface before you click on the Display button.
Figure 3 shows this Web Part in action. It displays the Active Directory® Documentation content type, which I based on the Customer Documentation content type. The Active Directory Documentation content type is associated with a custom Microsoft Word template (Active Directory Template.dot). Note that this content type has no additional fields. All fields are inherited from the parent content type.
If you are planning to use the ContentTypeSchema Web Part in your own environment, keep in mind that this Web Part has not been tested thoroughly. My Web Part uses relatively complex XSL transformations to build a delta between the SchemaXML property of the currently selected content type and its parent content type. Yet, XSLT 1.0 is not really designed for advanced XML document conversions. There is no built-in feature to compare XML nodes. Also, XML namespaces, as well as CDATA sections, represent difficulties that XSLT 1.0 cannot handle efficiently.
SharePoint stores the definition of site columns and site content types created by using the SharePoint user interface or SharePoint object model in the content database in a table called ContentTypes. In Figure 3, take a look at the ID of my custom content type. Accordingly, the following T-SQL statement would deliver exact results: SELECT Definition FROM ContentTypes WHERE ContentTypeId = <content type ID>.
Figure 3 Custom content type definition in the ContentTypeSchema Web Part
Figure 3 Custom content type definition in the ContentTypeSchema Web Part
Whichever approach you decide to use, creating features for enterprise-wide content types is very straightforward once you've obtained the site column and content type definitions. I recommend that you use a single feature for all global site columns and content types of your department or company. That way, it is clear where to add new site column and content type definitions.

Enterprise-Wide Searches on Custom Metadata
An immediate advantage of content type hierarchies is that all child content types inherit the parent content type's metadata fields. Because all content types have metadata fields in common, it is quite easy for users to create custom views and filters in the document libraries.
The AdventureWorks sample feature demonstrates this quite intuitively. Regardless of the content that a consultant or salesperson creates, saving the resulting Word 2007 document in the document library requires specifying a Customer Name because the Customer Documentation parent content type defines this metadata field as required. By grouping or filtering the items in a document library view according to the Customer Name, consultants and sales team members can locate all of the existing documentation for a customer quickly and conveniently, as shown in Figure 4.
Figure 4 Grouping various documents in a library based on common metadata
Figure 4 Grouping various documents in a library based on common metadata
Common metadata also makes it easy to locate content across multiple document libraries and sites by using the search capabilities of WSS 3.0 and MOSS 2007. WSS supports searches across libraries, lists, and sites within a single site collection. MOSS 2007 goes beyond these basic capabilities by allowing you to implement an enterprise-wide Search Center and to manage custom metadata in the SharePoint 3.0 Central Administration user interface. For this reason, the following explanations focus on MOSS 2007.
When you configure a Shared Service Provider (SSP) in MOSS 2007 and crawl your SharePoint sites, MOSS 2007 can automatically discover the custom metadata fields used in the content sources. Accordingly, you can find the custom metadata fields in the list of crawled properties.
Metadata fields defined in SharePoint features usually fall under the SharePoint category. To find its location in SharePoint Central Administration, on the Quick Launch menu under Shared Services Administration, click on the link to your SSP, click on Search settings, click on Metadata property mappings, then click Crawled Properties on the Quick Launch menu, and then proceed to open the SharePoint category.
I'll give you an example of this in action. The metadata field named Customer Name ends up as a crawled property called ows_Customer_x0020_Name. SharePoint prefixes crawled properties with "ows_", and "_x0020_" is the encoded version of a single space. If you display the details of this crawled property, you can find that it is included in the search index by default so that users can employ the values of this crawled property to perform content searches. However, to increase the search precision, you might not want to map the crawled property to a managed property so that users can explicitly search for content by customer name.
You have two options when mapping crawled properties to managed properties. You can automatically generate a new managed property for each crawled property or you can map crawled properties manually to managed properties. The first option is available when you display the settings of the desired crawled property category (when displaying the crawled properties in a category, such as the SharePoint category, click on the Edit Category option on the Quick Launch link). Under Bulk Crawled Property Settings, you would have to select the checkbox "Automatically generate a new managed property for each crawled property discovered in this category." However, SharePoint prefixes automatically created managed properties with "ows" and escapes spaces with "x0020". The managed property for the crawled property ows_Customer_x0020_Name would be owsCustomerx0020Name. This, however, is not a very user-friendly property name.
Perhaps a better strategy is to map crawled properties to managed properties manually after you deploy your enterprise-wide content types. You can map a crawled property to one or multiple managed properties. To create new managed properties, in SharePoint Central Administration, on the Quick Launch menu under Shared Services Administration, click on the link to your SSP, click on Search settings, then Metadata property mappings, and then click on New Managed Property to specify the required information and associate the new managed property with the desired crawled property.
Users can then locate relevant content items by using the managed properties either in property: syntax or by using advanced search. For example, if you map the crawled property ows_Customer_x0020_Name to a managed property named Customer, users can then search for all documents of a customer simply by specifying Customer: <Customer Name> in the standard search box, such as Customer: Contoso.
It is also a good idea to include managed properties for the most important metadata fields of your content types in the properties list on the Advanced Search page. To accomplish this, display the Advanced Search page, then click on-site Actions, and select the Edit Page command. Now you can click Edit on the Advanced Search Box to select the Modify Shared Web Part option. When you expand the Properties category and place the cursor into the corresponding text field, you can click on the button to edit the Properties XML document. You need to add a property definition to the <PropertyDefs> element, such as <PropertyDef Name="Customer" DataType="text" DisplayName="Customer Name"/>, and you also need to add a reference to this property definition under a ResultType element (for example, the element <ResultType DisplayName="All Results" Name="default">), such as <PropertyRef Name="Customer" />. Figure 5a displays the resulting Advanced Search user interface. Figure 5b displays the results of the search.
Figure 5a Searching for IT infrastructure documentation based on customer name
Figure 5a Searching for IT infrastructure documentation based on customer name
Figure 5b The results of the customer name search
Figure 5b The results of the customer name search

Ensuring Information Consistency
At this point, I have successfully standardized the most important metadata fields and content types and extended search capabilities across site collections in my enterprise based on MOSS 2007. Now it's important to ensure that users enter accurate information into the metadata fields.
There are two ways to do this. First, you can replace the standard document information panel in your document templates with a custom InfoPath® form that provides your users with a predefined selection of input options, such as you have in a listbox. Second, you can bind an event receiver to the content type and then check the accuracy of metadata and other information whenever users create, modify, or delete corresponding content items.
Both options complement each other. An InfoPath form primarily provides a convenient way to edit the properties of a SharePoint content type, whereas an event receiver can ensure valid metadata even if users edit the content type properties outside the InfoPath form. You can bind event handlers to a specific content type, which enables you to specify events and their responses for all documents associated with an individual content type in all site collections, regardless of the document library. You can find more information on how to develop and deploy event handlers at msdn2.microsoft.com/ms453149.
Perhaps the easiest method for associating a content type with a custom document information panel is to display the Document Information Panel settings of the content type in the SharePoint user interface on a computer with InfoPath 2007 installed. You can then click on the Create a New Custom Template link under Document Information Panel Template to start InfoPath 2007 in design mode. However, if your site content type includes site columns without an explicit SourceID attribute, you might encounter a situation in which InfoPath cannot create a valid XSD schema for the Document Information Panel form. For example, the Customer Documentation content type introduced earlier includes several contact-specific columns such as Department, Office, and e-mail that can cause this problem, as illustrated in Figure 6.
Figure 6 XSD schema incompatibilities in InfoPath 2007
Figure 6 XSD schema incompatibilities in InfoPath 2007
At this point, you have two options should you encounter this problem. You can either remove references to site columns without an explicit SourceID attribute from the content type definition, or you can replace the built-in site columns that cause the problem with custom site columns that are compatible with InfoPath 2007. Keep in mind that you cannot change the field references of a CAML-based content type after it was provisioned into the content database, especially if you already created child content types. You cannot simply update the CAML-based content type definition file anymore, nor can you push down changes to child content types because Windows SharePoint Services does not track changes made to the parent content type definition.
To push down changes, you must change the parent content type through the SharePoint user interface or object model, or you must define a new content type that is derived from the existing one. The latter technique enables you to remove the critical fields and add new columns. Your users can then derive future content types from the new content type. To prevent users from choosing the wrong content type, add the previous content type to the _Hidden content type group.
As I said before, you cannot update CAML-based content types after deploying and activating them. Thus, it is very important to test global content types before deployment. For more information, see the MSDN article "Updating Content Types" at msdn2.microsoft.com/aa543504.
Once you have created a content type with proper field references, you can create a custom document information panel in InfoPath 2007. The best strategy is to let site owners create custom document information panels for their child content types. InfoPath 2007 provides the option to publish the custom document information panel directly to the selected content type, which facilitates the deployment scenario. Yet, it is also possible to publish the InfoPath form at a central location, such as a document library, and include a reference to the document information panel in the content type. This is the best option if you plan to provide a custom document information panel along with your CAML content types. Figure 7 illustrates the implementation architecture.
Figure 7 XSN file implementation in a central location in MOSS 2007
Figure 7 XSN file implementation in a central location in MOSS 2007
The companion download for this article includes a feature called AdventureWorks_Update that extends the previous AdventureWorks feature by adding additional site columns that work with InfoPath 2007. The AdventureWorks_Update feature marks the original Customer Documentation content type as hidden and derives a new content type called Customer Docs, which replaces the inherited built-in fields with the new site columns and associates the new content type with a custom document information panel.
The new Customer Docs content type definition includes an XmlDocument element that provides information about the document information panel. Specifically, the xsnLocation element points to the InfoPath form http://companyresources/DIPs/customerDIP.xsn, which implements the document information panel. For detailed instructions on how to apply the AdventureWorks_Update feature, see the readme.htm file in the AdventureWorks_Update folder.

Wrap-Up
You can use SharePoint content types to create metadata policies and use them consistently across the enterprise for content management. The hierarchy of content types enables you to standardize characteristics relevant to the entire enterprise environment and apply them uniformly to all sites through inheritance.
Among other things, it is possible to extend the built-in search capabilities of MOSS 2007 so that users can locate specific content faster and more conveniently. It is also possible to enforce information consistency related to metadata and establish centralized information management policies The best strategy is to standardize global content types on the most abstract metadata characteristics to minimize the need for later changes. Based on a carefully designed content model, SharePoint content types can provide new capabilities to standardize content lifecycle management across the enterprise.

Pav Cherny is the president of Biblioso Corporation and an IT expert and author specializing in Microsoft technologies for collaboration and unified communication. His published works focus on IT operations and system administration.


 Office Communications Server: How Presence Powers OCS 2007
Office Communications Server Rajesh Ramanathan

How Presence Powers OCS 2007
At a Glance:
  • Authorization and directory services
  • Messaging options in OCS 2007
  • Routing communications with presence

Microsoft Office Communication Server (OCS) 2007 builds upon the strengths of the Live Communication Server (LCS) 2005 release. LCS 2005 delivered enterprise-class instant messaging
(IM) and presence. It also introduced telephony integration with existing PBX installations via Remote Call Control (RCC).
OCS 2007 further enhances the presence and IM capabilities in LCS 2005 and adds advanced media and phone capabilities with Office Communicator 2007, making it a full-fledged "softphone." Microsoft has also developed conferencing servers and edge servers for media that enable you to fulfill all the communication needs of your organization along with an existing PBX or independently as a complete solution in itself.
Office Communicator 2007 provides "softphone" capabilities that allow your users to choose the USB devices they like and provides a rich end-user experience to make common call control features such as hold and transfer discoverable and easy to use. Users no longer need to remember telephone numbers or use a dial pad. Instead, they can easily click to make a call from a desktop application or create a conference call by dragging and dropping people or distribution lists into a conversation.
For those who need actual desk phones, Office Communicator Phone Edition (or Tanjay phone) acts as just another endpoint for the user, using touch-screen-enabled context menus and the familiar presence experience of Office Communicator.
These OCS system capabilities are built around the concept of user presence. OCS 2007 uses user availability, communication endpoints, and user relationships to connect people using the most appropriate medium at any point in time. And since OCS ties together voice, e-mail, IM, and other communication paths, it can help you route messages in the most productive way possible.
In this article, I will give an overview of the OCS 2007 solution and explain how various pieces fit together. I will also cover how presence is used as an important ingredient in the unified communication recipe and how it can be utilized for routing voice calls more effectively.

Authorization and Directory Services
An enterprise voice system must be able to verify the identities of the people who are allowed to make calls. It also should be able to apply restrictions or policies on a call-by-call basis. In a Voice over IP (VoIP) system, clients need to authenticate against the server to initiate calls. This is similar to a traditional PBX system where access to the physical line was sufficient to enforce identity.
The OCS 2007 system, shown in Figure 1, uses Active Directory® for the authentication and storage of access policies, and OCS authorizes calls by validating them against the policies for a user in Active Directory. Because OCS uses the same Active Directory information used by other Microsoft applications such as Exchange Server or Microsoft® Outlook®, you can simply add voice capabilities to existing users in Active Directory by extending a user's properties and policies (via Active Directory schema extensions) and providing a unified directory that can be used for real-time apps such as voice or IM.
Figure 1 OCS unified communication components
Figure 1 OCS unified communication components
In Active Directory, the most important property related to unified communication is the Session Initiation Protocol (SIP) address (also known as SIP URI), which is very similar in nature to the user's e-mail address. For example, a user in Active Directory with the e-mail address someone@contoso.com would be assigned a SIP address sip:someone@contoso.com. Since the SIP address is tied to the user object, you can provide a single identity where the same credentials that the user uses for logging onto their desktop computer or Exchange server can be used to sign into their OCS server.
Another important property is the user's phone number. The OCS server internally retargets calls sent to phone numbers to a user's SIP URI to be able to route the call.
OCS 2007 provides an Address Book Service (ABS) interface that is available for directory look-up within an organization. As an offline version of the address book, it allows client endpoints to avoid hitting Active Directory every time a search operation is performed at the client end.
In addition to ABS, OCS 2007 makes distribution lists more useful by providing a Distribution List Expansion (DLX) service that allows users to view the same distribution list in Office Communicator that they see in Outlook and to expand the distribution list in Communicator as well. Users can set up conference calls with members of these distribution lists from Office Communicator directly or start a group IM chat.

Voice and Messaging Services
Additional OCS Resources
For more information, see the following resources:
  • Unified Communications Events and Webcasts
  • Office Communications Server TechCenter
  • TechNet Virtual Lab: Enabling Users and Installing Office Communicator 2007

An enterprise voice solution must provide the call control features that users are most familiar with. With that in mind, OCS 2007 provides popular features such as call hold, call transfer and consultative transfer, forwarding, and simultaneous ring, plus non-voice modes such as IM.
OCS 2007 is at the core of the voice call routing, providing both outbound and inbound call routing functions. Outbound call routing involves translating numbers, applying policies (such as international call restriction) related to a user, and routing the call to the appropriate endpoints or out to the public switched telephone network (PSTN). Inbound call routing functions take care of a user's selected call forwarding or time of day/presence preferences and routes the incoming call appropriately.
OCS does not terminate incoming calls itself. In SIP parlance, it acts like a SIP registrar and a SIP proxy. Signaling and media are provided point-to-point entirely by the client endpoints in the system. Endpoints in the system such as Office Communicator 2007 provide a wideband audio codec (called RTAudio) that is adaptive and resilient to network conditions.
OCS 2007 provides on-premise Web conferencing using the Conferencing Server roles for all modes of communication: audio/video, IM, and data. OCS-based Conferencing Servers allow client endpoints to provide seamless transition from peer-to-peer to multi-participant, multi-modal communication with capabilities such as mute, eject participant, and lock. It can scale to hundreds of participants for scheduled conferences and up to a hundred for ad hoc discussions. You can schedule conferences through Outlook add-ins or make in-call escalation to a conference. The conferencing solution also provides tools such as file sharing, whiteboarding, and recording.
Of course, you must be able to provide connectivity to outside PSTN networks and phone numbers and to federated enterprises or existing PBX installations within the organization. OCS 2007 integrates with SIP-to-PSTN gateways offered by common gateway providers that allow connection to PSTN or to an existing PBX. A Mediation server can be configured for codec and signaling translation. Mediation servers are optional and are configured for gateways that do not support the Microsoft codecs.
Exchange Server 2007 Unified Messaging is the voicemail solution for OCS 2007. Exchange Unified Messaging provides call answering for inbound voice and fax calls and will place the received message in the user's Exchange mailbox. Additionally, Exchange Unified Messaging provides corporate auto-attendants for outside callers looking for a specific individual or department.
OCS provides real-time IM as an alternative to using voice, and it also allows users to engage in multimodal conversations that involve voice, video, and IM at the same time. Furthermore, Office Communicator 2007 allows rich text and formatting in IM. Plus, audio and video can be added any time to the IM session, seamlessly escalating the IM conversation to an audio conversation.
Accounting, logging, and troubleshooting are additional components. Accounting is provided using the OCS 2007 Call Detail Recording server functionality. For every call that is made in the system, a record is created of the time that the call came in, the destination that answered the call, and various other parameters such as whether the call was transferred. OCS also provides archiving servers that log IM conversations to meet compliance requirements. The Quality of Experience (QoE) Monitoring server stores information about the quality of calls that were made and can be used for troubleshooting voice quality issues in the network.

Registration and Initialization
Every client endpoint in the OCS system needs to register or sign on to OCS as the first step in initialization of the client. Registration is the process of connecting to the OCS server, and this step advertizes the existence of the client. The process of registration involves authentication of the user identity by the server; this also creates a security association between the client instance and the server. This security association is used for subsequent calls that the client makes through the server and is refreshed periodically by the client by re-registering. The re-registration duration varies depending on the server topology. For instance, client endpoints registered from outside the enterprise through a Microsoft AccessTM edge server refresh their registration more frequently than clients inside the enterprise firewall.
Note that only those client endpoints that need a persistent authenticated channel with OCS to receive incoming notifications such as voice calls or presence change notifications need register. Live Meeting is a client endpoint that needs a connection to OCS only for joining a meeting and therefore it does not need to register with OCS.
To better understand the details of client interaction within OCS 2007, let's take a closer look at Office Communicator as a client endpoint. (Most of the signaling flows that relate to Office Communicator are identical to those used for other client endpoints such as Office Communicator Phone Edition or Office Communicator Mobile.) Once registration completes, Office Communicator retrieves configuration information that is critical to the operation of the client endpoint. This information includes:
  • Self information such as contact card, e-mail address, SIP URIs, display name, and telephone number
  • Capabilities that are enabled on the client and policies
  • Server addresses such as AV edge server SIP address and voicemail server address
  • Buddy list of contacts stored on the server
  • Number formatting rules for the client's location
This mechanism for retrieving all provisioning information using the existing SIP channel to the server is called inband provisioning. This lets the client endpoints auto configure and refresh their configuration whenever the clients sign on and also from outside the network or firewall.
The registration sequence proceeds from registering with the server to initialization, getting and setting presence, and finally to entering a ready state (see Figure 2). Note that OCS allows multiple client endpoints to be registered for the same user or SIP URI. During registration, the server returns a unique SIP address to the client so that any incoming signal can be targeted to the specific client. This address is also known as the Globally Routable User URI (GRUU). Each of the client endpoints that register with OCS get assigned a separate GRUU address by the OCS server, uniquely identifying the client endpoint.
Figure 2 Registration procedure
Figure 2 Registration procedure
To keep registration under control, there is a limit of eight active registrations allowed by the server for any user. When multiple clients are registered for the same user, the configuration is known as multiple points of presence (MPOP). When only a single client is registered, it is referred to as single point of presence (SPOP).
The concept of MPOP is core to the OCS system. It lets the user receive incoming calls or IM notifications from any endpoint, essentially allowing the user to have multiple phone or IM endpoints. It introduces interesting dynamics for answering IM or even for depicting presence accurately for a user.
For example, a remote user might be signed on to a Communicator Phone Edition IP Phone (an SPOP endpoint) that cannot receive IM. This capability of the endpoint is published by the IP Phone along with the presence information. The user's presence status on the phone is online. Another user can view the online presence status and attempt to send an IM using Communicator. However, since presence has already published the device capability, a notification is shown that the remote user is not on an IM-capable device and therefore may not receive the message.
Another example is for auto-accepting IM. Whenever an IM conversation is started, the message is auto-accepted immediately if the remote user is on a single Communicator instance (again, an SPOP endpoint). But if the remote user is signed on to two Communicator instances (say a laptop and a desktop), then a 10-second delay is provided to allow the user to accept the instant message from one device. When there is no answer, the instant message is auto-accepted on the most active Communicator instance.

Presence
Presence has an important role to play in unified communications. It allows a user to know beforehand about the availability of the remote user to communicate via a given mode. Thus, the OCS states, such as "busy—in a call," are associated with a user's busy state rather than a device or a line being busy. Additionally, OCS 2007 provides an enhanced presence infrastructure that allows the user to share information such as location, working hours, and meeting times.
In addition, OCS allows the user to specify permissions so that different groups of people can have access to different sets of presence information based on their access level. OCS also employs a user's presence state, such as "Do Not Disturb," as well as the calendar working hours and the permission level provided to the caller to make routing decisions on whether the call should ring the user or should be sent to voicemail.
The phone numbers that are displayed in click to call in Office Communicator are sourced from three locations—two are directory-based (Global Address List/Address Book Server and Outlook contacts), and the third is through presence. Using presence, users have the flexibility to publish the phone numbers that they want other people to see.
Presence plays a key part in a lot of voice routing scenarios as well. For example, if the presence for a remote user indicates Busy and shows the text In a Meeting, then the likelihood the user will pick up a voice call is low. In that case, the caller may prefer to use IM or e-mail. Similarly, when the presence for a remote user is Do Not Disturb, then it is an indication that alternate means of communication may be preferred.
Presence also carries information about a user's current meeting title and calendar location, and this details is published to users in the Team container. Presence carries information about the preferred endpoint for a certain capability, and this is used to elect an endpoint that will take a default action. For example, calendar data publication (which is essentially the same at all endpoints) will be done only from a preferred endpoint for calendar capability. OCS has the logic to elect preferred endpoint.
Figure 3 highlights the various capabilities that presence brings to the OCS system. Presence, as stated earlier, is a combination of availability and willingness to communicate. The Online state represents the state where the user is most willing to communicate, and Do Not Disturb is the presence state where the user is least willing to communicate. Except for Do Not Disturb, which has to be manually set, all the other states are automatically captured by Communicator based on various conditions such as whether the user is in a meeting (busy) or is away from the computer (inactive, then away), as shown in Figure 4.
Figure 3 How presence information is set (Close Figure 3)

Capability | Source
Availability and Willingness (Online, Offline, Away, Busy, Do Not Disturb) | Automatic. Based on meeting status, phone status, and so on. Can be set manually as well.
Calendar State (In a Meeting, Meeting location) | Automatic. Based on user preference.
Device capability (IM allowed) | Automatic. Based on MPOP device.
Phone numbers | Automatic. Based on user preference.
Most Active Endpoint | Automatic. Based on user activity.
Note | Manual. Overridden by Out Of Office Note.
Out of Office Note | Automatic. Based on calendar.
Working Hours | Automatic. Based on free/busy data.
Location | Manual. Based on user selection.
Figure 4 Presence as a mix of availability and willingness
Figure 4 Presence as a mix of availability and willingness
Note that availability can be driven by various different devices that the user is currently signed on to. Because there are several devices for the same user and each device publishes presence information and user data from the device, there needs to be an aggregation logic that presents one view to watchers. OCS has such an aggregation logic that computes the presence based on various parameters, devices, and inputs that are received by the presence system.
Before delving into aggregation, it is worth taking a look at how the aggregated information is published to users. OCS 2007 introduced the concept of containers, which are somewhat analogous to access levels associated with social circles. Each container provides a different amount of information, and people who are added to a specific container only have access to information that is made available to that container. Everyone in the user's buddy list has to be present in one of the available containers. Office Communicator allows the user to assign people to containers or move them to different containers based on the level of the relationship. Office Communicator uses smart heuristics to select the default container (listed in Figure 5) automatically.
Figure 5 Presence containers (Close Figure 5)

Container | Purpose
Blocked | Blocks presence information. Calls are not allowed from blocked users. However, blocked users can see name and e-mail address information.
Public | Provides name, title, company, e-mail address, and limited availability.
Company | Provides work contact information, basic schedule, and availability.
Team | Provides work and mobile numbers, schedule, availability, and can interrupt the user when in Do Not Disturb state.
Personal | Can see all published contact information including home and mobile numbers.
Presence aggregation in OCS determines the correct presence for the user based on various presence states that devices report to the server. Figure 6 illustrates how presence is aggregated from a variety of sources. Note that some information, such as presence state and user activity, is sent through the OCS presence aggregation mechanism before the information is made available in the containers, and other information, such as contact details, is added by the client endpoints to the respective containers. The blocked container is missing because no information other than the membership information is published to that container.
Figure 6 Presence aggregation and publishing
Figure 6 Presence aggregation and publishing
In the OCS system, a presence relationship is established when one user adds another user to the contact list (buddy list). But the OCS system also allows you to see the presence of other users in the corporate directory without having to add these users individually to your contacts.
The simplest way to get presence for someone is by searching for someone in Office Communicator. This generates a single query on the server and does not subscribe to notifications from the server about remote users' presence status changes. However, when a user adds another user to his buddy list, he subscribes for notifications on the presence status changes so OCS automatically sends alerts whenever there is a change.

IM and Audio
IM takes advantage of both registration and presence. In the OCS system, messages travel peer to peer and the OCS server performs as a proxy that shuttles the messages between Office Communicator clients.
The first message that a user initiates to another user is an important one. It establishes the session, in SIP parlance. Part of establishing the session is to find the right MPOP client from the registered clients, and this is where OCS plays an important role as a proxy. Once the user accepts the incoming IM session, or if one of the user's MPOP endpoints auto-accepts, then subsequent messages travel smoothly end to end without requiring OCS to find an appropriate client.
Users can leave their IM conversation windows open on their desktops, and Office Communicator 2007 will end the SIP session if there is no activity within a 10-minute interval (though the IM window will stay open). Either side can also terminate the IM session. Whenever the IM session is terminated, Office Communicator 2007 will automatically create a history item for the conversation that records the entire conversation in the Outlook Conversation History folder. This is a special folder that is created in Outlook by Office Communicator. This client-side logging capability is optional and is turned off by default. Conversation History is used by Office Communicator in restarting a conversation with the same user; thus, users that turn on this feature will have a better experience for continuing the IM conversation after the IM window is closed.
Note that the IM constructs presented here for forking and MPOP pretty much apply to the voice scenario as well. Instead of the invitation containing an IM session description, it would contain a voice session description indicating audio capability. OCS would apply forking to the voice call to all endpoints, similar to what it applies for IM.
Voice signaling derives a lot from the concepts of IM, but it introduces the need to handle media. I plan to cover this more in a future article.

Rajesh Ramanathan has worked in the communications space for 14 years and currently works as Lead Program Manager on the Office Communicator team. He can be reached at rajeshra@microsoft.com.


 Office Communications Server: Plan Your OCS 2007 Voice Deployment
Office Communications Server Jochen Kunert and Rui Maximo

Plan Your OCS 2007 Voice Deployment
At a Glance:
  • Telephony integration scenarios
  • Configuring Office Communications Server
  • Users, numbering plans, and routes

Microsoft Office Communications Server (OCS) 2007 is an enterprise product that integrates real-time communication media with user presence awareness, including Web conferencing
(sharing data, audio, and video), instant messaging, and audio/video conversations. Integrating OCS 2007 with an existing PBX system can be challenging. Office Communications Server 2007 is not designed to be a PBX because today's Information Workers use a variety of media communications—e-mail, instant messaging, telephone, and voicemail—in their daily workflow, while PBX systems deliver telephone calls only.
To provide the flexibility that administrators need to create the optimal individual or company telephony integration strategy, Office Communications Server 2007 provides three configuration scenarios for integration with existing telephony environments. These scenarios are: Remote Call Control, Enterprise Voice, and Enterprise Voice with PBX integration.
This article focuses on the planning considerations for a telephony deployment of OCS 2007, which may need to integrate with an existing telephony system. This information will be of particular interest to system architects and IT and Telephony administrators. In addition we will provide an overview of the other scenarios, but our focus will be on Office Communications Server 2007 Enterprise Voice.

Remote Call Control
Additional OCS Resources
  • Pro LCS: Live Communications Server Administration
  • Integrating Telephony with Office Communications Server 2007
  • Enterprise Voice Planning and Deployment Guide

In a Remote Call Control (RCC) scenario, users have an existing PBX, IP-PBX, or Time-Division Multiplexing (TDM) desk phone. From their Office Communicator 2007 client, they can place and answer calls on their desk phone without dialing the phone's dialpad or picking up the phone's headset. While users can control their desk phone using Office Communicator 2007, the audio is still handled by the PBX system and will not be converted to a Voice-over-IP (VoIP) call. Even though Office Communicator 2007 can control the desk phone, calls can still be placed and answered from the PBX phone, regardless of whether Office Communicator 2007 is running.
To enable RCC with Office Communicator 2007 and OCS 2007, you must deploy a Session Initiation Protocol (SIP)/Computer Supported Telephony Application (CSTA) gateway, in addition to enabling your Office Communications Server 2007 environment for RCC. The SIP/CSTA gateway must interoperate with the type of PBX system you use and must be compatible with OCS 2007 because it bridges the two systems. To control the PBX phone, Office Communicator 2007 uses the CSTA protocol. This protocol defines an XML Schema for interfacing with telephone switches (PBX), as described in the ECMA (European Computer Manufacturers Association) 269 standard, which you can find at www.ecma-international.org/publications/standards/Ecma-269.htm.
Office Communicator 2007 sends CSTA commands to control the PBX phone as SIP payloads that are routed by Office Communications Server 2007 to the SIP/CSTA gateway. Office Communications Server 2007 does not need to understand CSTA. It just needs to know how to route it. The SIP/CSTA gateway then converts the commands into the native protocol used by the PBX. This is illustrated in Figure 1.
Figure 1 RCC integration scenario
Figure 1 RCC integration scenario
Enabling RCC requires five steps. The first step is to configure the SIP/CSTA gateway to communicate with your PBX. Next, configure OCS 2007 to route CSTA requests to the SIP/CSTA gateway. The third step is to configure users for RCC. The fourth step is to configure Office Communicator 2007. The fifth step is to define the OCS 2007 normalization rules, which are also used for Address Book phone number normalization and different from the phone number normalization rules that are being used in combination with Enterprise Voice.

Enterprise Voice
When a user is enabled for Enterprise Voice, Office Communicator 2007 or Office Communicator Phone Edition becomes his primary phone, instead of his PBX phone. Office Communicator 2007 will have a unique phone number assigned to it. (For testing, the existing PBX phone can be forwarded to a new extension given to Office Communicator 2007.) Office Communicator 2007, in combination with a recommended USB audio device, will handle both the call control (or signaling) and the media (audio and video).
In this scenario, Office Communications Server 2007 routes calls among Office Communicator 2007 endpoints on the IP network (IP-IP calls) and routes calls to the public switched telephone network (PSTN)/PBX (IP-PSTN calls). OCS 2007 enforces dialing restrictions on calls routed to the PSTN.
To connect your Office Communications Server 2007 environment to a PBX or PSTN, you must deploy a media gateway, also referred to as a SIP/PSTN or IP/PSTN gateway. The media gateway must be paired with a Mediation server (a server role of OCS 2007) to connect to your Office Communications Server 2007 environment, as shown in Figure 2. The Mediation server and associated media gateway should be installed within the same physical location.
Figure 2 Enterprise Voice integration scenario with PSTN or an existing PBX
Figure 2 Enterprise Voice integration scenario with PSTN or an existing PBX
The Mediation server makes it possible to connect media gateways that support standard, even unsecured SIP implementations with Office Communications Server 2007, as OCS 2007 allows only authenticated and encrypted communication. Another task of the Mediation server is to handle codec translations between the standard codecs (G.711, G.723, and so on) and the Microsoft real-time audio codec. For a list of media gateways that Office Communications Server 2007 supports, see Strategic Hardware Partners at technet.microsoft.com/bb735838.
You can connect the media gateways directly to the PSTN, or to a Primary Rate Interface (PRI) trunk connection of an existing PBX. Figure 2 shows possible integration scenarios for the Enterprise Voice scenario with the PSTN and an existing PBX. A Director (usually an Office Communications Server Standard Edition Server) is not required for an Enterprise Voice deployment. However, it is recommended.
Enabling Enterprise Voice requires four steps. The first step is to deploy and configure the media gateways. Because this is a Microsoft partner product, you should refer to the vendor's documentation, as each media gateway has different deployment steps.
Next, you deploy a Mediation server for every media gateway. In this step, you must configure one network card of the Mediation server to send and receive traffic from the media gateway and configure the other network card of the Mediation server to send and receive traffic from an Office Communications Server 2007 (typically a Director in an Enterprise Edition deployment). Unless your media gateway is an advanced gateway, it will need the assistance of an OCS 2007 Mediation server role to interoperate with your OCS 2007 Enterprise Voice infrastructure.
The third step is to define a numbering plan. This involves defining voice policies, phone routes, and location profiles.
Following that, you configure users for Enterprise Voice. In addition, users must be enabled for Enterprise Voice to be able to make and receive calls from the PBX or PSTN. This requires assigning unique phone numbers to each user.

Enterprise Voice with PBX Integration
This configuration is an extension of the Enterprise Voice configuration. The difference is that the user's PBX phone is synchronized to the user's Office Communicator client. The configuration is similar to the Enterprise Voice scenario connecting to the PSTN via a PBX, as shown in Figure 2. However, in this case, instead of replacing the user's PBX phone with an Office Communicator 2007 client, Office Communicator 2007 and the existing PBX phone will use the same phone number.
If a caller dials the user's extension from another PBX phone or from a PSTN phone, the PBX system will fork the call to the user's PBX phone and to the user's Office Communicator 2007 client through Office Communications Server 2007. The user's Office Communicator 2007 client and PBX phone will ring simultaneously when there is an incoming call. Conversely, Office Communications Server will fork calls initiated from OCS 2007 to the PBX through the media gateway. This scenario is not possible when using Office Communicator Phone Edition. This configuration requires that voicemail be handled by the PBX's voicemail system. Exchange Server 2007 Unified Messaging cannot be used as a voicemail solution for such users.
Enabling Enterprise Voice with PBX integration involves the same steps as deploying Enterprise Voice but with three additional steps. In the first additional step, the media gateway must be connected directly to the PBX. Then the PBX must be configured to support this scenario. (This may require a software upgrade of the PBX depending on your vendor.) Finally, users must be enabled for PBX integration.
Microsoft is working to provide support to connect the Mediation server directly to the IP PBX without the intermediary of a media gateway. Please visit technet.microsoft.com/bb735838 for the latest information regarding this topic.

Configuring Office Communications Server
Enterprise Voice functionality is handled by services running on Office Communications Server 2007 Enterprise Edition and Office Communications Server 2007 Standard Edition. Figure 3 describes the services that provide the Enterprise Voice functionality to route inbound and outbound calls
Figure 3 Enterprise Voice services (Close Figure 3)

Service | Description
Translation Service | Applies normalization rules based on location profile or phone context.
User Services | Performs reverse number lookup on target phone number.
Inbound Routing Component | Applies target user's calling preferences.
Exchange Unified Messaging | Forwards request to voicemail as needed.
Outbound Routing Component | Determines optimal route based on URI and user policy.
Figure 4 illustrates how each of these components operates on incoming calls as they are routed by OCS 2007 for a PC-to-PC call. Incoming calls can originate from a connected endpoint such as Office Communicator 2007 or Office Communicator Phone Edition, or from a Mediation server coming from the PSTN/PBX (PSTN-SIP call). Outgoing calls originating from one of the caller's Office Communications Server 2007 client endpoints, such as Office Communicator 2007, are sent to the Standard Edition Server or Enterprise pool where the user is homed. This is referred to as the user's home pool.
Figure 4 Interaction of Enterprise Voice related services on OCS front-end servers
Figure 4 Interaction of Enterprise Voice related services on OCS front-end servers
At this point, Office Communications Server 2007 processes the call to determine where to route it. This next hop can be to either the called party's home pool, a Mediation server, or Microsoft® Exchange Server 2007 Unified Messaging (see Figure 5). If the call is routed to another home pool, OCS 2007 sends the call to all active endpoints of the called party. The called party's Office Communicator 2007 or Office Communicator Phone Edition will ring and a pop-up window notification will appear on the device.
Figure 5 Call routing
Figure 5 Call routing
If the call is routed to a Mediation server, the server is configured to route to a specific media gateway. If the media gateway is connected directly to a PBX, the call is then routed by the PBX to a desk phone or to the PSTN. However, if the media gateway is connected directly to the PSTN, the call goes out to the PSTN.
If the next hop is to Exchange Server 2007 Unified Messaging, the caller can leave a message. Voicemail is available regardless of whether the called party is signed in to Office Communications Server 2007.
When the called party answers, the audio is sent directly from the calling party's Office Communicator 2007 or Office Communicator Phone Edition to the called party's Office Communicator 2007 or Office Communicator Phone Edition in an IP-to-IP call. If the calling party is using a PBX or PSTN phone, the Mediation server sends the audio traffic directly to the called party's Office Communicator 2007 or Office Communicator Phone Edition in an IP-to-PSTN call. OCS 2007 handles call signaling only, which is why the audio and video goes directly between the endpoints and does not get routed by OCS 2007.

Configuring Users
Users enabled for Enterprise Voice must be assigned phone numbers that are unique within the organization. This configuration is accomplished in Active Directory®. This is different from most telephony systems where the phone is assigned the phone number. The benefit of this design is that users can turn any computer running Office Communicator 2007 into their work phone by signing on to OCS 2007 from the client. This portability feature can be appealing for telecommuters or field employees.
The user's phone number is specified in E.164 format in Active Directory. This format defines the international public telecommunication numbering plan, a numbering scheme to uniquely represent phone numbers unambiguously. The E.164 format specifies the phone number in this format:
+<country code><area code><phone number>
A user enabled for Enterprise Voice has two unique representations: a SIP URI and an E.164 phone number. When a phone number is dialed, Office Communications Server resolves the E.164 representation of the phone number (+14255551212) to the user's SIP URI (joe@contoso.com). Office Communications Server 2007 then routes the call based on the called party's SIP URI. If the phone number cannot be resolved to a SIP URI, OCS 2007 routes the call out to the PBX/PSTN through a Mediation server.
When you plan an Enterprise Voice deployment where every Office Communications Server user must be assigned a unique phone number, you need to take into consideration your existing telephony system. If you connect to the PSTN directly through a media gateway, you will likely have an entire number block 2- or 3-digit extensions, Direct-Inward-Dialing (DID) numbers that can be distributed to OCS 2007 users. If you connect to the PSTN via a PBX, you will likely have a block of DID numbers provided by the PBX to assign to OCS 2007 users.

Numbering Plans
Even though E.164-formatted phone numbers provide a unique representation, this is not a convenient way to dial numbers. It is also not possible to enter a number in this format in Office Communicator Phone Edition or standard PBX/PSTN phones. How do you specify a + character using the 10-digit dialpad? Users are accustomed to dialing shorter extensions, which may be unique in a specific location but perhaps not company-wide. Therefore, it is necessary to specify translation rules to convert them to E.164 format.
The administrator must define location profiles that specify normalization rules to perform these number translations. After the phone number is extended by the client (Office Communicator 2007 or Communicator Phone Edition) or the Translation Service of Office Communications Server 2007 into an E.164 format, the User Services can perform a reverse number lookup and match the E.164 number to a corresponding SIP URI. The normalization rules defined in a location profile use regular expressions to perform the translation if a match is found. For example, if 1234 is dialed, a normalization that matches any four-digit pattern to translate into +1425555{4-digit} would translate 1234 into +1425551234. Location profiles thus allow users to keep the same dialing habits they are accustomed to.
Administrators can assign location profiles to users in the following three ways:
  • Define the location profile to users using Office Communicator 2007 through Group Policy.
  • Allow users to select a location profile from Office Communicator Phone Edition.
  • Assign a location profile to the home pool. Users homed on that pool will automatically be assigned that location profile.
Defining a numbering plan involves creating Voice policies, phone routes, and location profiles with normalization rules. The Voice policies define the type of call access users are allowed. Office Communications Server 2007 determines how to route calls to the PBX/PSTN by means of phone routes. Because phone numbers can be dialed in a multitude of ways, the normalization rules of a location profile define how to convert the dialed phone number into E.164 format, which can then be routed.
Every user enabled for Enterprise Voice is associated by the administrator to a Voice policy. A Voice policy defines the call privileges assigned to a user. The call privileges determine which routes the user is allowed to use and therefore which numbers the user is allowed to dial. Each user must be associated to a single Voice policy. The number of policies available is created by the administrator.
Each Voice policy contains a setting to allow or disallow users to enable simultaneous ringing and an ordered list of phone usages. The Voice policy, similar to the Meeting policy, is a logical container of settings defined as an XML document stored in Active Directory. The value of this design is that it can be extended in future versions of OCS 2007 to support additional policy settings, without requiring an Active Directory schema extension. For example, if a Voice policy is not associated with any phone usages, this has the effect of preventing users assigned to that policy from placing any outbound calls to the PBX/PSTN. Such users would only be able to place IP-IP calls.

Phone Usage and Routes
A phone usage defines the phone routing privileges users are allowed. A phone usage is a collection of phone routes. It is a string that must be a unique keyword, meaning no other phone usage can have the same name. The administrators can create as many usages as they wish.
Although a Voice policy could be directly associated with a route or a set of routes, instead of phone usages, the phone usage keyword is an abstraction to maintain the association between policies and routes. If policies were directly associated to routes, effectively removing the concept of phone usages, and the administrator modified the name of a route, then every policy associated with that route would need to be updated. By using a phone usage, which is an attribute of both a policy and a route to create the association between policy and route, the relationship between policy and route is preserved even if the route's name is changed.
A route contains a phone number pattern and a list of Mediation servers connected to media gateways. A route must be assigned to at least one phone usage. This association between the pattern and gateways specifies how to route phone numbers that match that particular pattern.
The pattern specifies a range of phone numbers that it can match. It is defined as a regular expression that can include and exclude phone numbers. The Enterprise Voice Route Helper tool is included to help build these regular expressions more easily. As a result, if the phone number that is dialed matches the route's regular expression pattern, then the call is routed to one of the gateways defined in the route.
More than one Mediation server (and therefore media gateway) can be listed in a single route. Office Communications Server 2007 routes call to the Mediation servers in a round-robin fashion as a way to balance the traffic across the gateways. If a gateway fails or is taken out of service for maintenance, the corresponding Mediation server will send back a response to Office Communications Server 2007, which will immediately attempt to route the call to another Mediation server (and therefore a different media gateway) in the route's list.
After 10 attempts to route calls to a failed media gateway, Office Communications Server 2007 will subsequently scale down traffic to that media gateway until it becomes responsive again. If the media gateway continues to fail to respond after an additional 10 attempts, Office Communications Server 2007 will stop routing calls to that media gateway until it becomes responsive.
The list of media gateways defined in the phone route is described by the gateway's fully qualified domain name and port number. If the phone number does not match the regular expression pattern, then the next route associated with the phone usage assigned to the user is checked until a match is found. If no match is found, the call cannot be routed and fails to reach its destination. The user receives a notification that the call could not be completed.

Next Steps
Office Communications Server 2007 provides three levels of telephony integration. When planning your telephony deployment, you need to determine which level best meets your current and future telephony business needs. After this is determined, you can plan your deployment. An important aspect of your planning is defining a numbering plan for your Office Communications Server Enterprise Voice deployment. This should be carefully considered.
See the "Additional OCS 2007 Resources" sidebar for more about Office Communications Server 2007 Enterprise Voice and related technologies.

Jochen Kunert has 10 years of experience in the telecommunications industry and is now working as a Senior Program Manager in the Office Communications Group at Microsoft. He is a coauthor of the soon-to-be-released book, Microsoft Office Communications Server 2007 Resource Kit. Jochen can be reached at jkunert@microsoft.com.
Rui Maximo is a 14-year veteran of the IT industry and coauthor of two books, Pro LCS: Live Communications Server Administration and Microsoft Office Communications Server 2007 Resource Kit. Rui can be reached at ruim@ruimaximo.com.


 High Performance Computing: An Introduction to Windows Compute Cluster Server
High Performance Computing John Kelbley and Doug Lindsey

An Introduction to Windows Compute Cluster Server
At a Glance:
  • Using clusters for solving complex problems
  • General requirements for computational clusters
  • Setting up a Windows-based compute cluster
  • Running commands remotely

High Performance Computing (HPC) refers to a branch of applied computing that focuses for the most part on solving computationally intensive problems. Years ago, HPC (then more commonly referred to as "supercomputing") was dominated by
specialized large (expensive) systems found primarily in research centers. As the computing power of small systems increased, however, the cost/performance ratio has changed, and computational workloads have moved to PC-class systems.
Many computationally intensive problems can be solved by completing calculations in parallel; that is to say, a particular calculation or process may not be dependent on the output of another to complete. In cases like this, large problems can take advantage of multiple smaller systems (nodes) grouped into computational clusters. Here are just a few examples of the kinds of applications that take advantage of computation clusters:
  • Financial models—an algorithm or formula is run thousands of times, each time with different inputs.
  • Engineering—simulating effects in individual parts, applying textures to models.
  • Computer animation—applying texture and lighting effects to each frame of a movie.
Computational clusters provide an economical way to solve complex problems in a short amount of time. As you'll see here, Microsoft provides key compute cluster functionality and support with Windows® Compute Cluster Server 2003.
It is important to understand that a computational cluster is different from an availability (failover) cluster. You might already be familiar with the availability clusters for Exchange, SQL Server®, or other applications in Windows Server® 2003. They typically leverage shared storage to maximize application uptime. The intention of computational clusters is not to resume the work of another system in the event of an outage but to allow all nodes to work in a coordinated fashion.

General Requirements for a Computational Cluster
HCP Resources
  • Windows HPC Community
  • High Performance Computing with WCCS 2003
  • Microsoft 64-Bit Computing Overview
  • Microsoft Servers Home
  • Compute Cluster Server TechCenter
  • Step-by-Step Guide to Installing, Configuring, and Tuning a High Performance Compute Cluster Deployment

Modern HPC systems share some key elements that facilitate the processing of workloads. For starters, you need more than one system. You also need a scheduler to coordinate work assignments across the nodes. The scheduler runs on the node in charge (the head node) and identifies available resources, assigning and distributing tasks and tracking the overall status of jobs. It is the coordinator of resources within the cluster of systems as well as the point where users and administrators submit jobs for processing.
Clusters require a means of communicating among nodes. Depending on the type of work, nodes may need high-speed, low-latency interconnects to pass messages among one another to coordinate processing. At a minimum, each compute node and the head node must be connected to a common network.
Large numbers of interconnected systems working together to solve complex problems also need a fair amount of care and feeding. Processes and tools geared toward provisioning, monitoring, managing, and maintaining dozens to hundreds or thousands of systems are required to ensure a consistent and stable environment.
Lastly, and most importantly, applications that can exploit the parallel-processing capabilities of a computational cluster must be available. Without applications that are able to break up jobs so they can be processed on multiple computers, or development tools that include "parallel debugger" support, a cluster would do little more than heat up your computer room.

Windows Server-Based Clustering
In 2006, Microsoft introduced Windows Server 2003 Compute Cluster Edition (CCE) and Windows Compute Cluster Server 2003 (WCCS) to meet the requirements for a broad range of HPC applications. CCE and WCCS are based on the same Windows Server 2003 you are already familiar with. CCE is a version of Windows Server 2003 licensed for use with HPC applications. WCCS is identical to CCE with the addition of the Microsoft® Compute Cluster Pack (CCP). This means that you can deploy, manage, monitor, and maintain a Windows Server-based cluster using the same tools you already use to manage existing Windows Server 2003 systems.
One key point to note is that CCE and WCCS are x64 only—there is no 32-bit (x86) version of these products. The hardware requirements for CCE with WCCS are identical to those for Windows Server 2003 Standard x64 Edition. In addition to support for high-performance hardware (64-bit architecture), the products include Remote Direct Memory Access (RDMA) support for high-performance interconnects (Gigabit Ethernet, InfiniBand, Myrinet, and others).

Compute Cluster Pack
As mentioned earlier, computational clusters have some basic requirements that can be met through the installation of the Compute Cluster Pack. CCP is a standalone installation package that includes:
  • Integrated Job Scheduler
  • Messaging Passing Interface (MPI) support for industry standard MPICH2
  • Cluster Resource Management and User Tools
The CCP is the key differentiator between CCE and WCCS. You might be wondering why, if computational clusters need these components, they are included only in WCCS and not in CCE. The answer is that some HPC solutions leverage different job schedulers or specialized MPIs (or do not require MPI support) and simply require a great platform on which to run—namely, Windows Server 2003. Because the CCP is a standalone package, you can install it on other x64 versions of Windows Server 2003 (CCE, Standard, Standard R2, Enterprise, and Enterprise R2).

Nuts and Bolts
WCCS takes advantage of key Windows fundamentals to simplify the management and operation of Windows-based clusters, including Active Directory® and Remote Installation Services (RIS). Compute Cluster Server uses Active Directory to manage security transparently. With Active Directory, a user can submit a job to the head node—using a single set of credentials—that can run across hundreds of server nodes. When jobs are executed on one or more compute nodes, they run in the context of the user credentials provided at job submission, and they are securely cached thereafter. WCCS relies on Active Directory in order to provide this "single point of logon" functionality. An additional benefit of having Active Directory present in the environment is that server and configuration policy can be centrally administered via Group Policy.
If your IT organization already has Active Directory deployed, you can save significant time and administrative effort by building your cluster in the existing domain. This is the recommended scenario.
If you need to deploy a compute cluster into an environment where Active Directory does not exist, a recommended practice is to deploy one or more dedicated domain controllers to host Active Directory for the cluster. It is not uncommon to make the head node an Active Directory domain controller, since all nodes already have network connectivity to the system, regardless of network topology. However, making the head node a domain controller is not recommended for larger-scale clusters due to increased load on the head node. Common practices for Active Directory deployment and management should be followed (installation of redundant domain controllers, adequate back-ups, good security practices, and so forth).

Remote Installation Service
WCCS provides an integrated front end for the Microsoft Remote Installation Service image deployment platform. RIS is used to deploy operating system images from the head node to each of the compute nodes, meaning that you can use RIS to install new cluster nodes rapidly. You don't have to do it this way—RIS is integrated into WCCS as a convenience. Other common Windows Server deployment technologies may also be used, such as Windows Server 2003 Automated Deployment Services (ADS) or a manual installation of Windows Server 2003.
If you want to use RIS, before installing CCP on your head node, make sure you have at least two logical disks defined. RIS requires a disk that is separate from the operating system for the storage of server images. Your RIS partition should have enough free disk space to house one or more full copies of a Windows Server image.
Newer server hardware, particularly networking and storage gear, may require Plug and Play drivers that are not contained in the default Window Server 2003 images. In this case, you will need to add those drivers to the image manually. The procedure for doing this is documented online at support.microsoft.com/kb/254078.

Installing Your Cluster
The first step is to configure the brains of your cluster—the head node. Begin by installing one of the x64 versions of Windows Server 2003 noted earlier. During the operating system installation, you choose whether or not to join the server to an existing domain (recommended) or to install Active Directory on the server (not recommended for large-scale clusters).
After installing the operating system, make sure you have also downloaded and installed all recommended fixes from Microsoft Update. If you plan to use RIS for compute-node imaging, you then need to use the Computer Management | Disk Management administrative tool to ensure that you have at least two logical disks defined. In the interest of space, we won't cover RIS-based installation in this article.
Once you have successfully logged into and updated your server, you will run the CCP setup program. CCP setup will install or guide you through the downloading and installation of the following prerequisite files and updates:
  • RIS hotfix (see go.microsoft.com/fwlink/?linkid=55167)
  • Microsoft Management Console (MMC) 3.0 (see go.microsoft.com/fwlink/?linkid=62400)
  • SQL Server Desktop Edition, installed as part of the Compute Cluster Server 2003 head node installation process
  • Microsoft .NET Framework 2.0
The CCP setup program does a great job of assessing your system's readiness for installation. It will let you know what components are required and installed as part of the process, and install them, as shown in Figure 1.
Figure 1 Components required for CCP installation
Figure 1 Components required for CCP installation

The To Do List
When the CCP setup is complete, it will launch the administrator console, with the focus on the To Do List. As Figure 2 shows, the To Do List contains several panes, each addressing a different area and listing key tasks to complete the cluster configuration.
Figure 2 To Do List shows what you need to do to configure your cluster
Figure 2 To Do List shows what you need to do to configure your cluster
The To Do List lets you easily implement the cluster architecture you've laid out, including selecting and configuring the network topology, node installation process, and user management model. Tasks for each of the panes are listed on the right, and each task launches a wizard. Complete each task in sequence for a fully configured head node that is ready to deploy RIS images to a known set of compute nodes and that can be administered and accessed by a defined set of users.
Networking Topology WCCS supports the five most common network topologies used in HPC. At a minimum, all compute nodes and the head node must share a common network. The topologies include support for multiple types of interconnects among the nodes and have separate benefits and costs.
Your network topology will depend on the performance, security, and deployment requirements for your cluster. For instance, your application might require a high-speed interconnect for message passing that you would not like exposed to your corporate network. Maybe you would like to take advantage of the integrated, RIS-based, automated deployment capabilities. Perhaps the hardware you have chosen can only accommodate a single NIC or, possibly, you want the head node to run Internet Connection Sharing (ICS) to manage name resolution and addressing for the compute nodes. The "Configure Cluster Network Topology" wizard invoked from the To Do List will present you with the five topologies typically supported by common HPC implementations as well as help you configure your network connections, including Windows Firewall configurations on the head node and compute nodes. Figure 3 shows one of the supported network topologies.
Figure 3 One of the supported WCCS network topologies
Figure 3 One of the supported WCCS network topologies
Remote Installation Service As noted previously, RIS enables the automated deployment of cluster nodes.
Node Management You have the ability to specify the machine names of the servers that will be compute nodes in this cluster. When the CCP is executed on a node, the head node of the cluster that the node will join is specified at this time.
User Management Here you can specify which Active Directory user accounts or groups will be designated as system administrators and which will be designated as users of the cluster.

Node Installation
Deployment of compute nodes can be completed automatically via the RIS administration utility or using other supported deployment methods. As with the head node, the CCP must be installed on each compute node. As Figure 4 shows, CCP setup configuration options are much simpler for a compute node and consist of the following:
Figure 4 Compute Cluster Pack setup
Figure 4 Compute Cluster Pack setup
  • Specifying that the server should be a compute node (and not a head node).
  • Specifying the name of the head node of the cluster that the server should join.
  • Specifying whether or not to install the administrative and user tools on the compute node.

Administrator Console
Once your nodes are in communication with your head node, most management and administration tasks for all systems can be accomplished via the administrator console, shown in Figure 5. The console provides a central view of the entire cluster including a list of all associated nodes, access to key administrative actions, and other details. The left-most pane provides high-level navigation through the console, including access to the "Cluster Administrator," which, when highlighted, provides a summary screen showing cluster status and job statistics. Access to all remote desktop and system monitor sessions can be accessed here as well, but most of your time administering the cluster will be spent using "Node Management."
Figure 5 Administrator console provides a view of the entire cluster
Figure 5 Administrator console provides a view of the entire cluster
You should explore the administrator console on your own, as it provides access to key tools and functions required to centrally administer Windows and cluster nodes. Clicking once on a cluster node in the top-center pane will populate several of the other windows, and you can begin exploring node-specific functions.
The functions supported by the right-click menus allow you to easily administer compute nodes centrally from Node Management. You can use Node Management to pause and resume nodes; approve or remove nodes to and from the cluster; launch remote desktops, system monitor, or event viewer; or cause the CD tray to eject (which can be quite helpful when you're trying to physically identify a single machine in a large cluster).

Cool ClusRun
The single coolest feature in this list is "Run Command"—that is, the ability to run any arbitrary command remotely, just as if you were sitting at a command prompt on the target machine. WCCS includes a command-line version of the Run Command feature called Clusrun.exe, and "ClusRun" is used to describe both the GUI and command-line versions.
The benefits of the utility of ClusRun cannot be overemphasized—particularly on a large cluster. It is a huge timesaver for running repetitive command-line tasks and can often eliminate the need to write more complex administrative scripts.
To use this feature, highlight a list of the machines against which you want to run a command, then right-click, and select Run Command. The dialogue box in Figure 6 is displayed. Now just type in the command you want, click on Run, and wait for the output to be displayed in the Result window. You will be prompted for credentials the first time you use ClusRun, but you can opt to have those credentials cached for subsequent reuse.
Figure 6 You can run a command on multiple nodes
Figure 6 You can run a command on multiple nodes
A common way to patch clusters or install programs that include automated setup routines is to place them on a file share and then use ClusRun to force all of the compute nodes to invoke the command. Rebooting all compute nodes in the entire cluster can be done through a single command:
CLUSRUN SHUTDOWN –R –F –T 0

Tool Pack
In May 2007, the HPC team released the Compute Cluster Pack Tool Pack (you can get it online at windowshpc.net/resources/Pages/default.aspx). The tool pack includes support for Windows PowerShellTM, an MPIPingPong tool for diagnosing connectivity health and a very simple but useful graphical cluster monitor.
As shown in Figure 7, Simple Cluster Monitor can show you on one screen each of the nodes in the cluster, how many cores each node has, the utilization of each CPU (in bright green), memory utilization (yellow), percent disk time (red), and network bandwidth utilization (orange). It's a great at-a-glance tool admins can run to see how the cluster is doing.
Figure 7 Monitoring several nodes with Simple Cluster Monitor
Figure 7 Monitoring several nodes with Simple Cluster Monitor
The cluster monitor also includes "remoteable" functionality. You must run the first instance of cluster monitor on the head node. However, if you run an instance of the cluster monitor in a console session on the head node, you then have the opportunity to launch instances of cluster monitor from a workstation, point them at the head node, and get the same display.

What Are You Waiting For?
You are now armed with an understanding of HPC and Windows Server 2003 Compute Cluster Server. You know how to install the CCP, you have seen the great all-in-one console, and you now have cool new commands and graphical tools with which to dazzle your peers and users. You'll find sources for further information on High Performance Computing in the "HPC Resources" sidebar.

John Kelbley is a Technical Product Manager for Microsoft on the Global Solutions Technology Team based in the North Eastern U.S. John can be reached at Johnkel@microsoft.com
Doug Lindsey is a Program Manager on the Microsoft HPC Team. He is also an administrator of production compute clusters (including #116 on the November '07 list at www.top500.org). Doug can be reached at Dougli@microsoft.com.


 Online Extra: Deploy the 2007 Office System with Terminal Services
Online Extra James D. Silliman

Deploy the 2007 Office System with Terminal Services
At a Glance:
  • Deployment for the 2007 Office system
  • Managing user settings
  • The new Quick Access Toolbar
  • Customization Tool for the 2007 Office system

The release of the 2007 Microsoft Office system brought important design changes you should know about before you deploy or upgrade from previous versions in your Terminal Services server farm. You'll need to be familiar with the global template, the Quick Access Toolbar,
the .adm file snap-ins for Group Policies, and the Office Customization Tool (OCT). If you understand all of these tools in advance, your deployment will go more smoothly. Let's begin our tour of salient changes by looking at Microsoft® Word.

Deploying Word 2007
Word has always utilized a global template file, and in the 2007 Office system this file is called normal.dotm. In a Terminal Services environment, there are three different approaches you can take to deploy Word and the normal.dotm file. You can leave the file alone on all Terminal Services servers where Office is installed so users receive the standard Word default view. You can open the file on each Terminal server and modify it yourself, adding customizations, so that users don't get a default view in Word. Or you can rename the file altogether so that users get their own file (modified when they change a setting in Word and click Default).
Whatever choice you make, it's best made up front, before you deploy. If you make no changes out of the box, then users cannot save their global template changes because the global template file that sits on your Terminal Services servers overwrites their file in their individual profiles. If you do not open the template file and tweak it, then Word 2007 will only present its default settings to users.
You can, however, tweak the file on each server in your Terminal Services farm, if you so desire. To take this route and maintain the same Word settings for every user (such as default font and size, paragraph spacing, and layout), you'll need to keep this global template file in its default path location. Then to modify any settings that you want to deploy globally to your users, simply open this file on each Terminal Services machine, make your desired changes in the Word 2007 layout (click Default), and then save them, writing your changes out to the default normal.dotm file.
Be sure to back up the default file first before you make any changes or you might find yourself reinstalling Word 2007 to retrieve the original file. Note that if you make any changes to the Quick Access Toolbar while you're modifying the normal.dotm file, these changes will not be stored in this template file. I'll address the Quick Access Toolbar and its behavior shortly.
What if you decide that users should be able to manage their own global settings within Word? They can, but you need to rename the normal.dotm file to something like normal.dotm.old on all servers in your farm. I recommend this route as the best choice since users can get template errors when they close Word if other applications hook into it.
If you rename this file, Word 2007 recognizes that you want to allow users to manage their own preferences rather than having global preferences and, as a result, will allow them to modify and save their settings. Word 2007 now places the customized normal.dotm file in each user's profile path. If you're using roaming profiles (depending on how you set up your paths), the template file is stored in this folder location for each user: \\domain\sharename\%username%\Application Data\Microsoft\Templates.
One user preference setting not stored in the normal.dotm file is any change made to the Quick Access Toolbar. This toolbar is a new feature of the 2007 Office system, separate from the main Ribbon Toolbar (which is also new). The Quick Access Toolbar is the tiny toolbar located by default in the top-left corner of your screen in the 2007 Office system. This toolbar exists for the applications within the 2007 Office system, as shown in Figure 1. See support.microsoft.com/kb/926805 for more information.
Figure 1 Quick Access Toolbar files (Close Figure 1)

Office Application | Feature Area | File Name
Access 2007 | All | Access.qat
Excel 2007 | All | Excel.qat
Outlook 2007 | Contacts Calendar Distribution list Journal entry Mail Post in folder Tasks | Olkaddritem.qat Olkapptitem.qat Olkdistitem.qat Olklogitem.qat Olkmailitem.qat Olkpostitem.qat Olktaskitem.qat
PowerPoint 2007 | All | PowerPoint.qat
Word 2007 | All | Word.qat
As you can see in Figure 1, the Quick Access Toolbar uses files with the .qat extension to store individualized settings. The files are kept in this default file path location on each Terminal Services server in your Farm, and they do not roam: C:\Documents and Settings\username\Local Setting\Application Data\Microsoft\Office (or C:\Users\username\AppData\Local\Microsoft\Office on a machine running Windows Vista®).
If you look closely at the path, you'll notice the .qat files are located under the Local Settings subfolder of the Terminal Services user's locally cached profile. This subfolder is not synchronized with a user's roaming profile on logoff because Local Settings does not exist in the standard roaming profile path structure. Thus, unfortunately, if you blow away the locally cached profiles on your Terminal Services servers by enabling this Group Policy Object (GPO) setting, you'll lose the user's customized .qat files (see the Knowledge Base article at support.microsoft.com/kb/274152 for more information).
It's a fact that deleting cached profiles is widely practiced among Terminal Services administrators for house cleaning local profiles when a user logs off, so this change in the 2007 Office system is significant as it pertains to cached profiles and the Quick Access Toolbar. One obvious solution is to make sure that the GPO setting discussed in the Knowledge Base article I just mentioned is not set. If you do not enable deleting cached profiles on logoff and you have many users, you will be consuming valuable hard drive space on the local %systemroot% drive of each Terminal Services server and will have to set up some other process to keep an eye on it because your server could run out of hard drive space. This is not uncommon as local profiles can become quite large.
Whatever approach you take, you should at least add a routine to your login script that performs a backup of user .qat files periodically so they can be recovered if they disappear or are accidentally deleted. By keeping a backup of these customized files, you can avoid frantic calls to your local help desk to recover them.
Too few Terminal Services administrators back up these locally cached profiles; they back up the network roaming profiles instead, which, as I've mentioned, do not hold copies of the .qat files. Another alternative is to consider a third-party product that allows you to deploy a mandatory profile for each user, and then map each user's Local Settings changes into a separate folder on your network.

Group Policy Snap-Ins
For some time now, there have been Group Policy .adm files available for Office 2003 that allow for greater administrative control over Office policies and the user experience. Similarly, in the 2007 Office system there are new .adm files that you can use in order to make a number of customizations, including disabling the First Run dialog box and modifying Autosave settings and other preferences.
If you've already deployed Office 2003 group policies and decide to move to the 2007 Office system, you will first need to remove your existing Office 2003 .adm files from your GPOs before adding the 2007 Office system .adm files. Remember to back up your GPOs using the Group Policy Management Console (GPMC) interface so you can revert if you encounter errors after you add the new snap-ins. I know of at least one Terminal Services administrator who has experienced ghost policy entries created by removing Office 2003 snap-ins and then adding the 2007 Office system .adm files into his GPOs.
I mentioned in my May 2007 article, "A Guide to Deploying Terminal Services" (technetmagazine.com/issues/2007/05/TerminalServices), that Terminal Services users should not be prompted to perform some actions, such as Outlook® AutoArchiving, or accept First Run dialog screens. Be aware that by default on the first launch of Outlook 2007, users are presented with two First Run screens: one for Privacy Options and the second for Desktop Search. The first screen, displayed in Figure 2, can be disabled in several ways: by modifying GPO settings (using the new Outlook 2007 .adm files), using the Office Customization Tool, or tweaking the registry.
Figure 2 Privacy options dialog in the 2007 Office system
Figure 2 Privacy options dialog in the 2007 Office system
The screen shown in Figure 3 appears immediately after Privacy Options (if you don't disable it) and in all other 2007 Office system applications.
Figure 3 Instant Search feature dialog in the 2007 Office system
Figure 3 Instant Search feature dialog in the 2007 Office system
To turn off the Windows® Search prompt for Office applications, download the 2007 Office system ADM templates from go.microsoft.com/fwlink/?LinkId=78161 and load them to your hard drive on your domain controller. In Group Policy, you must load the Office Outlook 2007 template (Outlk12.adm) under Computer Configuration or User Configuration, Add Snap-Ins. Next, go to User Configuration\Administrative Templates\Microsoft Office Outlook 2007\Tools | Options\Preferences\Search Options and enable the "Prevent Installation prompts when Windows Desktop Search component is not present" setting.
There are many Group Policy settings available in the 2007 Office system to control the user experience, but you'll need to add the respective .adm files, such as Word (Word12.adm) and Excel® (Excel12.adm) for each, to your GPOs. Figure 4 shows an example of what the Outlook 2007 ADM template (Outlook12.adm) looks like once it's been added to your Group Policy.
Figure 4 Outlook 2007 ADM template in Group Policy
Figure 4 Outlook 2007 ADM template in Group Policy

Additional Changes
Note that in Office 2003 you can also download and use the User Customization Wizard to create customized Transforms. The new OCT in the 2007 Office system does not require a separate download. It is launched by using "Setup /admin" from the Run command, directed toward your 2007 Office system media or install source. Executing this command against your install source files and then changing individual settings to your liking creates an .msp file (versus the .mst file in Office 2003), and you do not need to specify any command-line options to include it in your later deployment. Simply placing the customized .msp file in the Updates folder at the root of the network installation point automatically deploys it along with your preferred settings.
Outlook 2007 has another advantage over its previous versions; it automatically locates your Exchange 2007 servers, sending out a Service Connection Point (SCP) query directly to Active Directory®, if launched inside your LAN. This query makes it easy to connect to a deployed Exchange 2007 infrastructure. For further information, see go.microsoft.com/fwlink/?linkid=76177.
Microsoft has made some important design changes in the way the 2007 Office system works in a Terminal Services environment by simplifying its deployment tools, adding new .adm file snap-ins, and streamlining the interface. By understanding some of the important concepts I've discussed here before you deploy or upgrade your Office applications, you'll have a much better understanding of the upgrade, allowing you to make important design decisions in advance. If you don't plan to upgrade at this time, you might want to consider just loading the 2007 Office system Compatibility Pack so your users can open attachments sent to them from users who have already upgraded. You can find the Compatibility Pack at go.microsoft.com/fwlink?LinkID=77512.
With this information in hand, you will be ready for a successful deployment of the 2007 Office system. Remember to back up your policies and test all recommended settings in a lab environment before deploying them to your user community.

James D. Silliman, MCSE, is a Microsoft-focused technologist who specializes in Terminal Services deployments. He currently works for an ASP, where he architects IT solutions for small businesses. Contact him at jsilliman@gmail.com.


 From the Editor: Dealing with Information Overload
From the Editor
Dealing with Information Overload
Joshua Hoffman

As I think about our two primary topics this month—SharePoint and Office Communications Server—it's actually not all that surprising to me that we often have trouble connecting with one another. You see, as I write this note (likely about a month or so in advance of your reading it), New York City is in full-tilt holiday mode. And as it happens, TechNet Magazine headquarters is located at the epicenter of the holidays in New York: Rockefeller Center. It's really a wonderful area of the city in many ways, but for about a month every year it becomes ... let's call it a challenge. The population swells (even further) with visitors from around the country and around the world who come to shop, or see a show, or go ice skating against the backdrop of majestic skyscrapers. Many New Yorkers, however, beat a hasty retreat.
Some argue that this occurs out of frustration with the tourists who stop in the middle of the sidewalk to take a picture and thus force those of us behind them to either stop short or run into them. But that's not it (really, we're glad you're here—snap away). I think it's just overload. Amidst an already crowded and busy city, we hit a tipping point, and it's just too much.
With this in mind, I turn to our industry and see much the same trend. Our organizations continue to grow (which is probably what we're hoping for), and, as a result, we generate more and more information every day. Our file shares are full. Our databases are full. And let's not even discuss e-mail. It wasn't that long ago that a gigabyte was a lot of storage. Today, terabytes are common—even in home PCs. Many datacenters are now being planned with petabytes in mind. I think we can agree; there's a lot of information out there. But is it useful, or is it just overwhelming? Is it structured? Easy to analyze? Easy to share?
So again, with the overwhelming amount of information, I'm not surprised that it's hard for us to connect. But it could be a lot easier, which is what we discuss this month. Inside this issue, learn how to harness the power of SharePoint to make data gathering easier and more organized in your enterprise. Learn how to rapidly build surveys to collect that data from your users (or your customers). Learn how to define custom content types so the information you do collect is far more meaningful. And finally, learn how to use Office Communications Server to enable you to make those critical connections. After all, what good is having all that data if you don't have someone to share it with? —Joshua Hoffman

Thank you to the following Microsoft technical experts: Ed Banti, Susan Bradley, Byron Hynes, Michael Murgolo, Ken Oien, Jim Truher, and Kay Unkroth.


 Letters: Readers Speak Out
Letters
Readers Speak Out

ActiveX Installer Service Availability
I just read your July 2007 article on "The ActiveX Installer Service in Windows Vista" (online at technetmagazine.com/issues/2007/07/AxIS). Could you please let me know if this service is available now?
—Eugene Y.

Yes, the ActiveX® Installer Service is available today. It can quite easily be enabled through the "Turn Windows features on or off" option in the Programs and Feature control panel in Windows Vista®.
The ActiveX Installer Service is an optional component only on the Ultimate, Enterprise, and Business versions of Windows Vista. The list of sites allowing ActiveX installation with the ActiveX Installer Service is controlled via Group Policy. The list is stored in the Computer Configuration | Administrative Templates | Windows Components | ActiveX Installer Service section of the Group Policy Object Editor.

Active Directory Tools
Thank you very much for the article "11 Essential Tools for Managing Active Directory" (September 2007). I have been trying to delete users from Active Directory® with the dsrm utility outlined in the article, using the following command line:
          dsquery user -samid <samid> | dsrm
I get an error message about "dsrm" input being in the improper form. However, I am able to perform the command separately without any issues. Any ideas?
—Keith E.

We passed your question along to the author of that article, Laura Hunter. Laura points out that when piping dsquery results into dsrm, dsrm requires the -noprompt switch. As such, you would need to use the following command line:
dsquery user -samid <samid> | dsrm -noprompt
We apologize that this information was omitted from the original article and we will be posting a correction to the online version.

Alternative Puzzle Possibilities
I presume you do realize, and have probably already been reminded by many others, that Dr. Scripto's Scripting Perplexer puzzle (November 2007) actually has two different solutions that both work. The puzzle will also unscramble correctly if the 4-character variable that appears in the first and second lines is nnde and if the 5-character variable that appears in the third and fifth lines is iamex. (I grant that the difference is trivial, and the names in your solution make the script much more readable.)
Thanks for the diversion of the puzzle; I enjoyed it.
—Stan A.

Thanks for sharing your alternative solution, Stan! We're glad to hear that you're enjoying the new Scripting Guys puzzle, and we're very excited to be adding it as a regular feature of TechNet Magazine.

Windows Vista Migration Persuasion
The article "Changes to Offline Files in Windows Vista" from the November 2007 issue of TechNet Magazine (go to technetmagazine.com/issues/2007/11/Offline) shows the advantages of Windows Vista in the area of offline files very well. The article has helped us in the decision to migrate to Windows Vista because we had many problems with offline files when using Windows® XP Professional.
—Daniel Z.

Thanks, Daniel! We're very glad to hear that you found the article useful in your organization.

Missing Pages
Your magazine is enjoyable and informative. However, my copy of the December 2007 issue was missing pages 49-64. I was interested in the computer forensics article ("A Guide to Basic Computer Forensics") by Tom Cloward and Frank Simorjay that was supposed to be on page 60. Perhaps your print forensics team can find my missing pages/data? If this article is available online I will be more than happy to read it there, if you can send me the URL.
—Dave S.

Unfortunately, we did experience a problem during the printing of the December 2007 issue, and as a result, a small number of copies were mailed to readers without pages 49-64. On behalf of our printer and the TechNet Magazine team, we sincerely apologize for this inconvenience.
In the meantime, the article you've referenced, along with the rest of the December issue, is now available online at technetmagazine.com/issues/2007/12.


 Toolbox: New Products for IT Pros
Toolbox
New Products for IT Pros
Greg Steen

Unlock Files
Unlocker
ccollomb.free.fr/unlocker
We've all had it happen to us before—some automated process hangs midway through its operation and a file gets stuck with a writelock, causing another operation to fail because it can't copy, move, write to, or delete the file. And sometimes you just don't know who was the culprit maintaining the file lock, but you know which file was locked. So what to do?
There are many ways to deal with this, but one tool that provides a quick-and-easy Windows® Explorer-integrated solution to file-locking issues is the free Unlocker tool, written by Cedrick Collomb. Thanks to its integration with Windows Explorer, you can find the offender by simply right-clicking on the locked file (or even a set of files) and then selecting Unlocker from the context menu. Then, if a lock is discovered, you can choose from a number of options, such as killing the offending process, unlocking the file (closing the handle), deleting the file, moving the file, or renaming the file. You can also copy the locked file in its current state to another location.
Unlocker has a full application installer, so you won't have to register or unregister any DLLs manually, nor do you have to reboot your system to start the tool. It also supports automatic update notification (if you don't mind having it "phone home" every now and again) to make sure you have the latest version.
Unlocker even supports command-line execution, which is great for automation. This means, for example, you can easily configure a log shipping process to automatically close the handle and kill the compression process if you encounter a locked .zip file while trying to move the logs, perhaps saving you from having to get up in the wee hours of the morning to try to fix the problem manually.
The utility also has a system tray tool called the Unlocker Assistant. This can automatically launch the Unlocker interface when you try to move, rename, or delete a locked file, saving you the extra clicks to locate the file and choose Unlocker from the right-click context menu. Unlocker is compatible with most versions of Windows, including Windows Vista®.

Price: Free
With Unlocker, you can quickly find and unlock locked files
With Unlocker, you can quickly find and unlock locked files

Analyze Network Traffic
Wireshark
wireshark.org
A good network protocol analyzer allows you to get to the fundamentals of the network traffic on your infrastructure to see what is really happening under all those layers of interconnected applications. Perhaps the most well-known open source analyzer is Wireshark, which utilizes the open source WinPcap packet capture library for Windows and is derived from Ethereal, with Gerald Combs still on board.
With Wireshark, you can easily capture the traffic passing across a network interface and examine the details of each packet in an easy-to-use, graphical interface. Starting a new packet-capture session is as easy as selecting your interface and clicking start. The GUI displays the captured packets in a color-coded frame, which details the time, source, destination, protocol, and a predetermined description of the event in near real time. Clicking on a transmission highlights the detail and interprets the packet data into an easy-to-follow detail pane. Below that, you can see the raw packet data in hex and ASCII.
Once you have the dataset you need for your analysis, you can, of course, also save or export the session for later analysis. Wireshark can decode many different protocols, and should cover most of your troubleshooting needs.
Networks are noisy, and to help you filter through all that data, the application has a great expression-based filter feature, which lets you narrow your search by each parameter of a protocol. For example, you could filter out all traffic except that which is related to BitTorrent from and to specific IPs in your network. Or, the filters could allow you to easily sniff out which machine is causing network havoc by throwing out tons of bogus broadcast messages.
The interface also lets you easily flag packets for review and set a specific transmission as your time reference in the capture. One of my favorite features of Wireshark is the ability to follow a TCP stream by choosing that option on the context menu off the right-click of a TCP packet. This brings up a window with a color-coded and formatted text view of the TCP stream content, which is great for detecting Web application errors, bogus HTTP headers, or other TCP communication issues you may encounter. And from here, you can quickly print or save the conversation for later examination or demonstration.
I've only scratched the surface of the app's functionality. Wireshark is released under the GPL license; in addition to being available for Windows, it also has compiled binaries for most versions of Linux and FreeBSD.

Price: Free
Wireshark lets you analyze your complete network protocol
Wireshark lets you analyze your complete network protocol

Beef Up Windows PowerShell
PowerShell Community Extensions
codeplex.com/PowerShellCX
If you've had a chance to really work with Windows PowerShellTM, you've probably noticed a few recurring patterns of execution for which it would be great to have a cmdlet or alias. To that end, a group of Windows PowerShell users and advocates from the open source community have created the PowerShell Community Extensions project, which wraps up commonly requested functionality into a distributable set of cmdlets, aliases, providers, functions, filters, and scripts
Some of the cmdlets included in the package are Get-Clipboard and Write-Clipboard, which let you add and retrieve the data on the clipboard buffer. Meanwhile, Get-DomainController retrieves a list of available domain controllers in the current domain, and Write-GZip and Write-Zip let you write GZIP or ZIP files.
You can create a new symbolic link or junction in Windows Vista or Windows Server® 2008 with New-Symlink and New-Junction, respectively. There's a useful DirectoryServices provider that gives you access to LDAP servers, such as Active Directory® and AssemblyCache, allowing you to access the Microsoft .NET Framework Global Assembly Cache and the assemblies within it.
There are also a number of great scripts. The Set-Writable and Set-ReadOnly scripts let you quickly change a file's read-only status. Get-Url makes a WebRequest and returns the WebResponse as a string, and Export-History dumps your command history to a .clixml file for later reuse. You can snag disk usage information with the Get-DiskUsage script or even make your computer speak with the fun Out-Speech command. And the package includes a good number of other useful functions such as Quote-List, which shortens your command-line typing, and Add-PathVariable, which can add a set of paths to a specified environment.
This project adds a very useful set of extensions to the standard Windows PowerShell environment. And if you happen to be a Windows PowerShell ninja, I am sure the project would be open to your contributions.

Price: Free
PowerShell Community Extension provides commonly requested functions
PowerShell Community Extension provides commonly requested functions

Greg Steen is a technology professional, entrepreneur, and enthusiast. He is always on the hunt for new tools to help make operations, QA, and development easier for the IT professional.


 SQL Q&amp;A: Non-Clustered Indexes and Retaining Permissions
SQL Q&A
Non-Clustered Indexes and Retaining Permissions
Saleem Hakani and Dan Carollo

Non-Clustered Indexes and Their Uses
Q What are non-clustered indexes, and what are the pros and cons of using them?
A A non-clustered index is similar to the type of index typically found at the back of a book. All the information contained within the book is represented in the index by topic, and there are pointers in the form of page numbers that direct you where to find the information perhaps in several areas of the book. Furthermore, the data in a book index is not listed in the same order as it appears in the text of the book. It is the same with non-clustered indexes. If you have a clustered index on a table, you can specify what the order of items will be. Otherwise, there is no way to be certain what the order will be.
In addition, non-clustered indexes have two limitations: only 16 columns can be included in the index and the maximum size of the index key cannot exceed 900 bytes. So what does that mean? Let's see what happens if you want to index the following columns in the Movie table from the sample MovieList database: MovieTitle NVarchar(50), DirectorName NVarchar(50), ShortStory NVarchar(400).
Assume you issue the following statement to create the table:
Use MovieList;
CREATE INDEX Movie_IDX ON Movie(MovieTitle, DirectorName, ShortStory);
This will spawn the following error message: "Warning! The maximum key length is 900 bytes. The index 'Movie_IDX' has maximum length of 1000 bytes." For some combination of large values, the insert/update operation will fail. You will receive the above message because "nvarchar" datatype consumes 2 bytes for each character; an index that contains the previous three columns would exceed the 900 byte size limit.
With the release of SQL Server® 2005, you now have the opportunity to overcome this problem by adding the columns to the INCLUDE clause. This is a very useful feature if you want to overcome both size and column limitations. You can do this by executing the following statement:
CREATE INDEX Movie_IDX ON Movie(MovieTitle, DirectorName) INCLUDE (ShortStory);
It's important to note that when you use the INCLUDE clause during index creation, the database engine does not consider non-key columns when calculating the number of index key columns or index key size.
To find more helpful information on non-clustered indexes, visit "Non-clustered Indexes" at msdn2.microsoft.com/aa174537 and "Using Non-clustered Indexes" at msdn2.microsoft.com/aa933130. For tips on optimizing indexes, see sql-server-performance.com/optimizing_ indexes.asp.

Preserving Permissions Data
Q How can I avoid losing permissions when a subscription is reinitialized? I have experienced the problem several times that when a snapshot is reinitialized, it loses all the granted permissions.
A By default, all the objects in the subscription database are dropped and recreated whenever a subscription is reinitialized. But there are two ways you can handle this scenario.
First, you can reapply all the permissions after the reinitialization. When you set up the permissions manually, you should script out all the object/statement-level permissions and store them separately so you can use them immediately after you reinitialize the subscription.
Second, you can configure your subscription so that it does not drop any objects whenever you reinitialize the subscription. To do so you can use SP_CHANGEARTICLE system stored procedure to configure the value of PRE_CREATION_CMD for the parameter @PROPERTY and a value of NONE, DELETE or TRUNCATE for the parameter @Value.
Also, in the Article Properties dialog box in the destination object section, select a value of "Keep existing object unchanged, delete data. If article has a row filter, delete only what matches the filter. Truncate all data in the existing object." Make sure to try this in your test environment, and if you need further help, check the latest version of SQL Server Books Online for up-to-date information.

Saleem Hakani is a Senior Database Engineer, Senior Problem Engineer, and Worldwide Microsoft SQL Server Community Lead with 14 years of experience in database systems. He leads the external SQL Server Community Web site sqlcommunity.com and can be reached at Saleem@sqlcommunity.com.
Dan Carollo is an Operations Engineer and SQL Server DBA who works with the Windows Anti-Malware Research and Response team at Microsoft. He has an MCT in SQL Server.


 Utility Spotlight: ADMX Migrator
Utility Spotlight
ADMX Migrator
Lance Whitney
Download the code for this article: ADMX Migrator (2765KB)

Those of you who employ Group Policy are, of course, familiar with using administrative templates to configure operating system and application settings. The ADM files included with Windows® and key software, such as Microsoft® Office, cover most common desktop configurations. However, sometimes you have to create your own ADM templates to incorporate custom settings. Designing ADMs from scratch can be a chore, as they have their own unique syntax and require that you build them using a text editor. But given time and experience, many Group Policy pros have become quite adept at fashioning their own ADM files.
With Windows Vista®, Microsoft has changed the format for Group Policy templates to an XML-based structure. This new format, called ADMX, offers several advantages over legacy ADM files, such as multi-language support and a central storage point where you can manage all your templates.
That's good news, but what if you want to create your own ADMX files and don't know XML? More importantly, what becomes of all those customized ADMs you spent years creating? Luckily, the ADMX Migrator utility comes to the rescue. This free tool, developed by FullArmor Corporation and licensed to Microsoft, offers two key benefits: it lets you create your own custom ADMX files, and it can convert your legacy ADM files into ADMX.
ADMX Migrator graphical interface
ADMX Migrator graphical interface
To create templates in the new format, the ADMX Migrator provides the ADMX Editor, which sports a graphical user interface—no more text editor required. You build your templates using pulldown menus and dropdown lists within a clean and simple interface (see the figure). To start, choose the New Template command, name your template file, and enter a category by which to classify your file. Next you create the actual policy setting by entering the Registry key to which the policy will apply, the name of the Registry value, and the class that it will affect (User, Machine, or both). You can also describe what the policy does.
When you're done, save your new template in the ADMX format. It is stored as both an ADMX file and an ADML file. The ADMX file contains the actual policy settings, and the ADML file provides language-specific information for the policy. You can now copy both files to the appropriate directories on your domain and incorporate them into Group Policy.
You'll probably want to convert your ADM files into ADMX to take advantage of the new format. The ADMX Migrator provides two conversion methods—through the editor or through a command-line program. From the ADMX Editor, choose the option to Generate ADMX from ADM. Browse to your ADM file, and the tool quickly and automatically converts it. You then can open the converted file in the editor to examine its values and properties and modify it if you wish. The ADMX Migrator Command Window is a little more complicated; it requires you to type a lengthy command string at a prompt to perform the conversions. However, it includes some options and flexibility not available in the graphical editor.
The ADMX Migrator is a Microsoft Management Console (MMC) snap-in, so it requires MMC 3.0 as well as the Microsoft .NET Framework 2.0. ADMX Migrator will run under Windows XP SP2, Windows Server® 2003 SP1, and Windows Vista. You can download the software at go.microsoft.com/fwlink/?LinkId=77409.

Lance Whitney is an IT consultant, trainer, and technical writer. He has spent countless hours tweaking Windows workstations and servers. Originally a journalist, Lance took a blind leap into the IT world 15 years ago.


 The Cable Guy: IEEE 802.1X Wired Authentication
The Cable Guy
IEEE 802.1X Wired Authentication
Joseph Davies
This article is based on a prerelease version of Windows Server 2008. Details herein is subject to change.

With the rise in popularity of IEEE 802.1X authentication for IEEE 802.11 wireless networks, network administrators want to use this standard for their wired network connections as well. Just as a wireless client must submit a set of credentials to be validated prior to allowing wireless frames to be forwarded to the intranet, an
IEEE 802.1X wired client must perform an authentication prior to being able to use its switch port. IEEE 802.1X authentication provides an additional security barrier for your intranet—you can prevent guest, rogue, or unmanaged computers that cannot perform a successful authentication from connecting to your intranet.
IEEE 802.1X authentication is probably already supported by your wired switches and just needs to be enabled and configured. For authentication and authorization of a wired connection, 802.1X-capable switches typically use the Remote Authentication Dial-In User Service (RADIUS) protocol to send connection request information to a RADIUS server, such as a Windows Server® 2008-based Network Policy Server (NPS) or a Windows Server 2003 Internet Authentication Service (IAS) server.
After you configure your switches for RADIUS but before you require 802.1X authentication, you will have to enable and configure 802.1X authentication on your wired computers. IEEE 802.1X authentication for wired network connections has been supported in Microsoft® Windows® since Windows XP. However, you must manually configure 802.1X authentication settings in Windows XP and Windows Server 2003 on each individual wired client (from the Authentication tab for the properties of a network connection in the Network Connections folder). Unfortunately, there is no way to centrally configure or script wired 802.1X settings for these earlier operating systems.
Fortunately, support for wired settings in Group Policy and scripting support with the Netsh tool in Windows Vista® and Windows Server 2008 mean that deployment of 802.1X wired settings is much simpler than before.

Wired Network Settings in Group Policy
Using the Wired AutoConfig Service
In Windows XP and Windows Server 2003, the 802.1X behavior on wired connections is controlled by the Wireless Zero Configuration service. On those operating systems, this service was enabled by default and wired network connections were placed in a passive listening mode, waiting for the switch to initiate authentication.
In contrast, in Windows Vista and Windows Server 2008, the Wired AutoConfig service controls 802.1X behavior on wired connections, but it is disabled by default. Therefore, the Authentication tab for the properties of network connections does not appear until after the Wired AutoConfig service is started.
For an individual wired client running Windows Vista or Windows Server 2008, you can use the Services snap-in to start the Wired AutoConfig service and configure it for automatic startup. When the Wired AutoConfig service is started, wired network connections operate in an active listening mode in which the network connection attempts to initiate authentication with the switch.
For an Active Directory domain, you can use Group Policy to configure the Wired AutoConfig service for automatic startup. Using the Group Policy Management Editor snap-in, configure the Computer Configuration | Windows Settings | Security Settings | System Services | Wired AutoConfig setting for Automatic startup mode.

To centralize and automate the configuration of wired network settings, Windows Server 2008 and Windows Server 2003 Active Directory® domain services support wired policy settings in Group Policy. These settings allow you to configure wired network settings as part of Computer Configuration Group Policy for a domain-based Group Policy Object.
With these wired policy settings, you can specify the authentication method and other 802.1X settings for wired clients running Windows Server 2008 or Windows Vista. When joining the domain, starting up, or periodically after starting, these operating systems automatically download the wired Group Policy settings and apply them. Just note that a Windows Server 2003 Active Directory domain must be extended to support these new policies. For information about how to extend a Windows Server 2003 Active Directory domain, take a look at technet.microsoft.com/bb727029.
You can configure wired policies from the Computer Configuration | Windows Settings | Security Settings | Wired Network (IEEE 802.3) Policies node in the Group Policy Management Editor snap-in. By default, there are no Wired Network (IEEE 802.3) policies. To create a new policy, right-click Wired Network (IEEE 802.3) Policies in the console tree and click Create a New Windows Vista Policy.
The properties dialog box of a Windows Vista wired policy consists of a General tab and a Security tab. Figure 1 shows the default General tab. On the General tab, you can configure a name and description for the policy and specify whether to use the Wired AutoConfig service, which controls 802.1X behavior on wired connections. For more information, see the "Using the Wired AutoConfig Service" sidebar in this column.
Figure 1 The default General tab of a Windows Vista wired policy
Figure 1 The default General tab of a Windows Vista wired policy
Figure 2 shows the default Security tab for a Windows Vista wired policy. On the Security tab, you can enable or disable 802.1X authentication, select and configure the Extensible Authentication Protocol (EAP) authentication method, select the authentication mode (user re-authentication, computer only, user authentication, or guest authentication), configure the number of times authentication attempts can fail before authentication is abandoned, and configure whether to cache user information for subsequent connections. When caching is disabled, Windows removes the user credential data from the registry when the user logs off. The result is that the next user will be prompted for his or her credentials (such as user name and password) at logon.
Figure 2 The default Security tab of a Windows Vista wired policy
Figure 2 The default Security tab of a Windows Vista wired policy
When you click the Advanced button on the Security tab, you can configure advanced settings for 802.1X and Single Sign-On. Figure 3 shows the default Advanced security settings dialog box for a Windows Vista wired policy. From the Advanced security settings dialog box, you can configure 802.1X settings shown in Figure 4.
Figure 4 802.1X settings in the Advanced security settings box (Close Figure 4)

Setting | Description
Max EAPOL-Start Messages | The number of successive EAP over LAN (EAPOL)-Start messages that are sent out when no response to the initial EAPOL-Start messages is received.
Held Period | The time interval between the retransmission of EAPOL-Start messages when no response to the previously sent EAPOL-Start message is received.
Start Period | The period during which the authenticating client will not perform any 802.1X authentication activity after it has received an authentication failure indication from the authenticator.
Auth Period | The period of time the authenticating client will wait before retransmitting any 802.1X requests after end-to-end 802.1X authentication has been initiated.
EAPOL-Start Message | When the wired client sends the EAPOL-Start message.
Figure 3 The default Advanced security settings dialog box for a Windows Vista wired policy
Figure 3 The default Advanced security settings dialog box for a Windows Vista wired policy
Wired clients running Windows Server 2008 support Single Sign-On for wired connections. This feature is also planned for the forthcoming release of Windows Vista Service Pack 1. More information is available online at technetmagazine.com/issues/2007/11/CableGuy.
There are Single Sign-On settings available to perform user-level 802.1X authentication prior to the user logon process or after the user logon process, and to wait the configured number of seconds for user-level 802.1X authentication to complete before starting the user logon process. You can determine whether or not to display dialog boxes for user-level authentication beyond the consolidation of input fields on the Windows logon screen. For example, if an EAP type wants the user to confirm the certificate sent from the RADIUS server during authentication, the EAP type can display the dialog box.
Additionally, you can specify that after performing user-level authentication, the system initiates a Dynamic Host Configuration Protocol (DHCP) renewal of the TCP/IP configuration of the wired adapter. Select this option if there are separate virtual LANs (VLANs) for computer-level and user-level authenticated wired clients and if those VLANs are different IPv4 or IPv6 subnets.

Scripting Support with the Netsh Tool
Windows Server 2008 and Windows Vista support commands in the netsh lan context of the Netsh tool to configure wired settings or export or import a wired profile, which is a named set of wired settings in XML format. With command-line configuration of wired settings, you can more easily deploy wired networks by creating automated scripts for wired settings without using Group Policy. The Wired Network (IEEE 802.3) Policies Group Policy settings apply only in an Active Directory domain. For an environment without a Group Policy infrastructure, a script that automates the configuration of wired connections with a wired profile can be run manually or automatically, including as part of the logon script.
To perform command-line configuration of wired clients running Windows Vista or Windows Server 2008, run netsh lan commands with the appropriate parameters. For example, the following command enables Single Sign-On for the network connection named "Local Area Connection" and configures Single Sign-On to perform user authentication before user logon:
netsh lan set profileparameter interface="Local Area Connection" ssomode=prelogon
For more information about netsh lan command syntax, see technet.microsoft.com/aa905084.
Wired XML profiles can be exported from a Windows Server 2008 or Windows Vista wired client and then imported to a Windows Server 2008 or Windows Vista wired client using the Netsh tool. To export a wired profile, use the netsh lan export profile command. If you want to import a wired profile, use the netsh lan add profile command. For some useful examples of wired profiles, see msdn2.microsoft.com/aa816372.
With command-line and XML profile support, you can bootstrap a wired client onto the organization's 802.1X-authenticated wired network. A wired client computer that is not a member of the domain cannot connect to the wired network using computer credentials. Additionally, a computer cannot join the domain until it has successfully connected to the wired network. However, command-line and XML profile support allow a wired computer to connect to the organization's wired network using user credentials and then join the computer to the domain. For more information, see technet.microsoft.com/bb727031.

Joseph Davies is a technical writer with Microsoft and has been teaching and writing about Windows networking topics since 1992. He has written five books for Microsoft Press and is the author of the monthly online TechNet Cable Guy column.


 Windows PowerShell: Shell Permissions
Windows PowerShell
Shell Permissions
Don Jones
Download the code for this article: PowerShell2008_02.exe (151KB)

When discussing Windows PowerShell, I am often asked about permissions, particularly if and how the shell lets you automate changes to permissions. File permissions are perhaps the most common request, although directory and registry permissions also come up quite frequently. When it comes to permissions and nearly
any command-line or scripting interface (including Windows PowerShellTM and VBScript), the news I have to give is both good and bad.
The bad news is that permissions in Windows® are just innately complicated, and there's not much Windows PowerShell can do to fix this. The problem stems from the fact that on a Windows operating system, there are several objects when it comes to permissions, regardless of whether you're looking at the file system, Active Directory®, the registry, or nearly anywhere else.
First there is the resource itself—for instance, a directory object, a file, or a folder. Next is the Access Control List (ACL) object, which is attached to the resource. The ACL consists of one or more Access Control Entries, or ACE objects. An ACE essentially ties together a security principal (a user or group, which is yet another object) with a permission (such as Read or Full Control) and a Grant or Deny status. That's no less than five objects: the resource, the ACL, an ACE, a principal, and a permission.
Permissions are further complicated by the fact that every type of resource has different permission types. Directory objects, for example, have complex permissions governing access to individual object attributes. File permissions are relatively uncomplicated in comparison.
And, technically, all of this doubles up when it comes to auditing. Each resource really has two ACLs. First there is the Discretionary ACL (DACL), which governs access to the resource. Then there is also a Security ACL (SACL), which controls auditing on the resource. Trying to work with these permissions means wading through a pile of fairly complicated-looking programming objects, and there's only so much a shell like Windows PowerShell can do to simplify that for you.

Limited by the .NET Framework
Can't Get Enough?
For more Windows PowerShell resources, including a searchable library of cmdlets, and to connect with Don Jones and other Windows PowerShell experts, visit www.PowerShellCommunity.org.

The nature of Windows PowerShell makes things a bit more complicated. Being built on the Microsoft® .NET Framework, Windows PowerShell uses the underlying classes of the .NET Framework to represent and work with permissions. Those underlying .NET Framework classes are designed to provide software developers with granular and precise control over permissions. Of course, "granular and precise" means lots of information and often translates to "very complicated."
Additionally, the .NET Framework doesn't provide classes that represent the permissions in every type of Windows resource. For example, while the .NET Framework provides classes that let you manipulate file security, it doesn't provide classes that let you work with the security on shared folders. The result is that Windows PowerShell can't manipulate permissions on every type of Windows resource. Simply put, Windows PowerShell is limited to the capabilities exposed by the .NET Framework.

Permissions in the Shell
Permissions management in Windows PowerShell is derived from two cmdlets: Get-ACL and Set-ACL. As you might expect, Get-ACL retrieves the ACL from a resource. You can then modify the ACL to suit your needs and use Set-ACL to write it back to the resource. Both of these cmdlets are generic and rely on the Windows PowerShell system of PSDrive providers. Hence, these two ACL cmdlets could in theory work with any type of resource, provided there is a PSDrive provider capable of understanding a given type of resource and capable of modifying the permissions on that type of resource. The FileSystem and Registry PSDrive providers, which are included with Windows PowerShell, support permissions management via the two ACL cmdlets. (Other PSDrive providers may not support this.)
The sample script shown in Figure 1 allows you to specify a starting directory, a security principal, and a permission. The script then applies the permission you specified for that principal to all files and folders within the specified directory. The script recurses subdirectories, too.
Figure 1 Apply a permission for a principal to files and folders in a directory (Close Figure 1)
#ChangeACL.ps1
$Right="FullControl"

#The possible values for Rights are
# ListDirectory, ReadData, WriteData
# CreateFiles, CreateDirectories, AppendData
# ReadExtendedAttributes, WriteExtendedAttributes, Traverse
# ExecuteFile, DeleteSubdirectoriesAndFiles, ReadAttributes
# WriteAttributes, Write, Delete
# ReadPermissions, Read, ReadAndExecute
# Modify, ChangePermissions, TakeOwnership
# Synchronize, FullControl

$StartingDir=Read-Host "What directory do you want to start at?"
$Principal=Read-Host "What security principal do you want to grant" `
"$Right to? `n Use format domain\username or domain\group"

#define a new access rule.
#note that the $rule line has been artificially broken for print purposes.
#it needs to be one line. the online version of the script is properly
#formatted.
$rule=new-object System.Security.AccessControl.FileSystemAccessRule
($Principal,$Right,"Allow")

foreach ($file in $(Get-ChildItem $StartingDir -recurse)) {
  $acl=get-acl $file.FullName

  #Add this access rule to the ACL
  $acl.SetAccessRule($rule)

  #Write the changes to the object
  set-acl $File.Fullname $acl
  }
The meat of this script is where it defines a new access rule in the variable $rule. To do this, I am using a "raw" .NET Framework class, which is perhaps the most complicated part of permissions management under Windows PowerShell. The script then retrieves the ACL from each file and folder in turn, using Get-ACL, applies the new rule using the ACL's SetAccessRule method, and writes the modified ACL back to the resource using Set-ACL.
In reality, this is not that complicated, although more complex operations like removing an ACE from an ACL are more involved. In my opinion, what makes it appear complicated is the fact that it involves so many steps: get the ACL, define a rule, modify the ACL, and write the ACL. It's not a single, clean command that does everything all at once. And since I wanted to modify the ACLs on multiple files, I had to wrap everything in a foreach loop, using Get-ChildItem to actually access all of the files and folders.

I'll Take the Easiest Way
Cmdlet of the Month: Get-QADUser
This cmdlet is a must-have if you're managing Active Directory. It isn't built into Windows PowerShell, but it is available as a free download as part of Quest Software's ActiveRoles Management Shell for Active Directory (www.quest.com/activeroles-server/arms.aspx).
Once you've got the snap-in installed, you can use Get-QADUser to retrieve user objects from the directory. Of course, simply running the cmdlet returns all of your users, which isn't exactly helpful for most administrative tasks. But the cmdlet allows you to specify filtering criteria, which are applied at the domain controller so that only the users you are concerned with are actually returned to Windows PowerShell. This is a much faster technique than first getting all the users and then filtering them with the Where-Object cmdlet. For instance, Get-QADUser -l Redmond will just return those users whose "l" attribute contains "Redmond." (The "l" attribute, by the way, stands for locality and is listed as city in the GUI.) You can then pipe those users to other cmdlets in order to generate an HTML report, place them into a group, or even delete them.

I don't find my example to be complex, but I know that many administrators would prefer a more simplified way of working with permissions. Perhaps they want to remove a particular user group from a set of folders all at once but don't want to write a multi-step script like the one in Figure 1.
Fortunately, Windows PowerShell does offer a simplified means of working with permissions, and you're probably already familiar with it.
Windows PowerShell doesn't make you give up on the tried-and-true ways of doing things. You can still use command-line tools—such as Dsacls.exe, Cacls.exe, and Xcacls.exe—that were specifically designed to provide a simpler way to work with various types of permissions. While their syntax is completely different from the standardized syntax used by Windows PowerShell cmdlets, these command-line utilities no longer have to be used in an ad hoc way at the command-line as they were used under the old cmd.exe shell. And you can use Windows PowerShell to automate these utilities.
I have run across some folks—I'll call them "purists"—who insist that using an old-style command-line utility within Windows PowerShell is somehow wrong. I don't quite follow their logic. I'm all about getting the job done, and if the easiest way to do that is to use a 10-year-old command-line utility, then sign me up. I am not going to spend added hours trying to do something a certain way just because it seems "more pure." And the Microsoft Windows PowerShell team holds a similar attitude. If an existing tool does the job, then Windows PowerShell lets you use that tool within the shell. Why reinvent the wheel?
Figure 2 shows a short script—taken from my book Windows PowerShell: TFM (SAPIEN Press, 2006)—that uses Windows PowerShell to automate Cacls.exe. This script is just a demonstration, meaning you'll have to tweak it if you want to make it into a real automation tool. But it does show how a Windows PowerShell script can collect information and use it to launch Cacls.exe, just as if Cacls.exe were a "native" Windows PowerShell command. (Although this script collects information by prompting the user for the information, the parameters could also be read from a file or database.)
Figure 2 Automate Cacls.exe (Close Figure 2)
#SetPermsWithCACLS.ps1
# CACLS rights are usually
# F = FullControl
# C = Change
# R = Readonly
# W = Write

$StartingDir=Read-Host "What directory do you want to start at?"
$Right=Read-Host "What CACLS right do you want to grant? Valid choices are F, C, R or W"
Switch ($Right) {
  "F" {$Null}
  "C" {$Null}
  "R" {$Null}
  "W" {$Null}
  default {
    Write-Host -foregroundcolor "Red" `
    `n $Right.ToUpper() " is an invalid choice. Please Try again."`n
    exit
  }
}

$Principal=Read-Host "What security principal do you want to grant?" `
"CACLS right"$Right.ToUpper()"to?" `n `
"Use format domain\username or domain\group"

$Verify=Read-Host `n "You are about to change permissions on all" `
"files starting at"$StartingDir.ToUpper() `n "for security"`
"principal"$Principal.ToUpper() `
"with new right of"$Right.ToUpper()"."`n `
"Do you want to continue? [Y,N]"

if ($Verify -eq "Y") {

 foreach ($file in $(Get-ChildItem $StartingDir -recurse)) {
  #display filename and old permissions
  write-Host -foregroundcolor Yellow $file.FullName
  #uncomment if you want to see old permissions
  #CACLS $file.FullName

  #ADD new permission with CACLS
  CACLS $file.FullName /E /P "${Principal}:${Right}" >$NULL

  #display new permissions
  Write-Host -foregroundcolor Green "New Permissions"
  CACLS $file.FullName
 }
}

Success with Permissions?
I've heard it argued that Windows PowerShell falls short when it comes to permissions because it fails to provide a native cmdlet that offers simplified, universal permissions management. I don't agree. First, this argument is often made by administrators with a strong UNIX background, and the fact is that the Windows permissions system is vastly more complicated than what UNIX uses. When exposing that system, Windows PowerShell will also be complicated. This is not a problem of Windows PowerShell, though.
In addition, Windows PowerShell does allow you to use simplified tools in the form of good old command-line utilities, as I've demonstrated here. True, the .NET Framework needs some expansion to more fully cover permissions management across the entire Windows platform. But overall, Windows PowerShell gives you both worlds: simplified permissions management as well as access to granular and precise management tools.

Don Jones is a contributing editor for TechNet Magazine and is the coauthor of Windows PowerShell: TFM (SAPIEN Press, 2007). He teaches Windows PowerShell (see www.ScriptingTraining.com) and can be reached through the Web site www.ScriptingAnswers.com.


 Hey, Scripting Guy!: The Games Are Afoot! Oh, and Some XML, Too
Hey, Scripting Guy!
The Games Are Afoot! Oh, and Some XML, Too
The Microsoft Scripting Guys
Download the code for this article: HeyScriptingGuy2008_02.exe (151KB)

You know, any time people talk about the legends of the sporting world, the same names inevitably come up: Babe Ruth. Pele. Muhammad Ali. Walter Payton. ScooterK and MrRat. Johnny—
Ha, that's funny, you ... oh, you're serious? You actually don't know who ScooterK and MrRat are? Well, for those of you who obviously don't keep up with the Winter Scripting Games (February 15th through March 3rd at the TechNet Script Center; go to microsoft.com/technet/scriptcenter/funzone/games), ScooterK and MrRat are true legends, competitors who earned perfect scores in at least one division in the 2007 Winter Scripting Games.
And here's the really cool part: can you realistically hope to become a soccer legend like Pele? Probably not. Can you ever hope to become the heavyweight boxing champion of the world? Well, some of the Scripting Guys seem to have the heavyweight part down, but that boxing champion bit is a much tougher nut to crack. But you—yes, you—can very easily become the next ScooterK or the next MrRat.
Note: In theory, you might be able to become the next Babe Ruth, too. All you would have to do is be able to eat 24 hot dogs between games of a baseball doubleheader.
Take it easy; we'll tell you how you can become the next ScooterK or MrRat (and maybe even the next Bizzy or H2Data). All you have to do is show up at the Script Center and take part in the Scripting Games. On February 15th we're going to post 10 different events (10 scripting challenges) and dare you to complete any or all of them. Write a script that solves the problem we pose and e-mail it to the Scripting Guys. (You'll find full instructions at the Scripting Games home page.) We'll test your script and, if it works, award you points. Successfully complete all 10 events and you too will become a legend of the sporting world. Or at least of the scripting sporting world, which is pretty much the same thing.
The Scripting Games (February 15th through March 3rd) are fun and challenging. Best of all, the Scripting Games are for everyone. Are you still somewhat new to system administration scripting? Then enter the Beginners Division; we'll have separate competitions for beginners in VBScript, Windows PowerShellTM, and (new this year) Perl. Do you find the Beginners Division a bit easy? Then you should try the Advanced Division, which also has competitions in VBScript, Windows PowerShell, and Perl.
The Games (did we mention they run from February 15th through March 3rd?) are quite simply the event of the scripting season, and you don't want to miss them. Visit the Script Center home page now for tips and tricks on training for the competition, then come back on February 15th when we officially let the Games begin.
February 15th through March 3rd. Just in case you missed it.
What's that? Actually, we agree with you: coming out with a new Hey, Scripting Guy! column on top of the announcement about the Scripting Games probably is too much excitement for a single month. However, in order to keep the good folks at TechNet Magazine happy (our primary goal in life, needless to say), we decided to risk it and put out a new column anyway.
Exactly one year ago (wow, has it really been that long?), we published a column that explained how you could use a script to read an XML file. What that column didn't tell you was how you could use a script to create, write, and modify an XML file. This month, we're here to right that wrong. You say you'd like to know how to write a script that can create an XML file? All you had to do was ask. Well, ask and then wait a year for us to get around to it. Figure 1 shows the script. Admittedly, this might look complicated, but we'll explain how it all works.
Figure 1 Creating an XML file (Close Figure 1)
Set xmlDoc = _
  CreateObject("Microsoft.XMLDOM")

Set objRoot = _
  xmlDoc.createElement("ITChecklist")
xmlDoc.appendChild objRoot

Set objRecord = _
  xmlDoc.createElement("ComputerAudit")
objRoot.appendChild objRecord

Set objName = _
  xmlDoc.createElement("ComputerName")
objName.Text = "atl-ws-001"
objRecord.appendChild objName

Set objDate = _
  xmlDoc.createElement("AuditDate")
objDate.Text = Date
objRecord.appendChild objDate

Set objIntro = _
  xmlDoc.createProcessingInstruction _
  ("xml","version='1.0'")
xmlDoc.insertBefore _
  objIntro,xmlDoc.childNodes(0)

xmlDoc.Save "C:\Scripts\Audits.xml"
To begin with, we create an instance of the Microsoft.XMLDOM object. As you might have guessed, this is the object that enables us to work with XML files. Our goal is to produce a simple XML file that looks like Figure 2.
Figure 2 Our target: a simple XML file
Figure 2 Our target: a simple XML file
To create this XML file, the first thing we need to do is create the root node (ITChecklist). How do we do that? Like this:
Set objRoot = _
  xmlDoc.createElement("ITChecklist")
xmlDoc.appendChild objRoot
That was pretty easy, wasn't it? All we had to do was call the createElement method, passing createElement the name we want to give to the root node. We then simply call the appendChild method, passing the object reference to our new element (objRoot) as the sole method parameter. At that point, we have ourselves a root node.
But wait; there's more. Next we create the ComputerAudit node, a child node of the ITChecklist node that represents the information for a single computer. As you can see, the code to create this node is similar to the code used to create the root node:
Set objRecord = _
  xmlDoc.createElement("ComputerAudit")
objRoot.appendChild objRecord
The only difference is this: when we created the root node, we called appendChild on our XML document itself. (Note the object reference xmlDoc.) To add a new child node to the root, we call appendChild on the root node (objRoot) rather than the XML document. It's that easy.
It's equally easy to add the ComputerName and AuditDate nodes as child nodes of ComputerAudit. To do that, we're going to go through a similar process: we're going to call createElement to create a new node, and we're going to call appendChild to append this new node to the file.
(Quick quiz: Where do we call appendChild from this time around? Yes, that's right: from objRecord, the parent (ComputerAudit) node we just took the time to create.)
Oh, and because the ComputerName and AuditDate nodes need to contain values, it's important that we also specify a value for the Text property of each of these nodes right before we add them to the document.
That should all become clear once we take a peek at the code that actually creates the ComputerName node:
Set objName = _
  xmlDoc.createElement("ComputerName")
objName.Text = "atl-ws-001"
objRecord.appendChild objName
As you might expect, the code for creating AuditDate is almost identical. We just have to specify a different node name when calling createElement and assign a different value to the Text property.
Man, we're making this too easy on everyone, don't you think?
After we finish creating the nodes for our first (and in this case, only) record, we execute this sweet little block of code:
Set objIntro = _
 xmlDoc.createProcessingInstruction _
  ("xml","version='1.0'")
xmlDoc.insertBefore _
  objIntro,xmlDoc.childNodes(0)
That simply inserts the <?xml version="1.0" ?> tag at the beginning of the file, ensuring that we have a well-formed XML document.
At that point, all we have left to do is call the Save method and save our new file as C:\Scripts\Audits.xml:
xmlDoc.Save "C:\Scripts\Audits.xml"
Just like that, we have a brand-new XML document.
Now, that's actually pretty useful: you now know how to create a brand-new XML file using a script. Of course, the odds are pretty good that most of the time you don't really need to create a brand-new XML file; instead, you simply need to append new data to an existing file. So will the Scripting Guys show everyone how to do that?
You know, originally we decided that the answer to that question would be no, we wouldn't show everyone how to append data to an existing XML file. And then, being kind-hearted and generous people, we decided to try and strike a deal with the readers of TechNet Magazine: if everyone reading this magazine agrees to enter the 2008 Winter Scripting Games, then, in return, we'll show you all how to use a script to append data to an XML file. So, does everyone agree to enter the Scripting Games?
Um, we're waiting on you. Yes, you—the guy in Rochester, MN.
There, that's more like it. And trust us, you'll have a good time during the Scripting Games; everyone always does. We promise.
And now, seeing as how a deal's a deal, we'll show you a script that can append data to an existing XML file. Just take a look at Figure 3.
Figure 3 Appending to an XML file (Close Figure 3)
Set xmlDoc = _
  CreateObject("Microsoft.XMLDOM")

xmlDoc.Async = "False"
xmlDoc.Load("C:\Scripts\Audits.xml")

Set objRoot = xmlDoc.documentElement

Set objRecord = _
  xmlDoc.createElement("ComputerAudit")
objRoot.appendChild objRecord

Set objFieldValue = _
  xmlDoc.createElement("ComputerName")
objFieldValue.Text = "atl-ws-100"
objRecord.appendChild objFieldValue

Set objFieldValue = _
  xmlDoc.createElement("AuditDate")
objFieldValue.Text = Date
objRecord.appendChild objFieldValue

xmlDoc.Save "C:\Scripts\Audits.xml"
As you can see, this isn't too terribly different from the script that created the XML file. We start out by creating an instance of the Microsoft.XMLDOM object, then setting the Async property to False, which tells the script that we want to load the document synchronously rather than asynchronously. What difference does that make? Well, if we loaded the document asynchronously, the script would be free to continue even though the document was not fully loaded. Needless to say, trouble can arise in a hurry if you try to perform a task on a document that doesn't exist yet. By ensuring that our XML file loads synchronously, we also ensure that the file will be fully loaded before our script proceeds.
Speaking of fully loaded—well, never mind; we'll just let that one go. Instead, we call the Load method to open the file C:\Scripts\Audits.xml. As soon as the file is open, we use this line of code to create an instance of the documentElement class, which has the net effect of binding us to the document root. In this case, of course, that's the ITChecklist node:
Set objRoot = xmlDoc.documentElement
It's all downhill from that point. We create a new instance of the ComputerAudit node, using the appendChild method to add that new node to the file. We then create new instances of the ComputerName and AuditDate nodes, specifying the appropriate values (atl-ws-100 and the current date, respectively) for each new node. We tuck these two items inside the new ComputerAudit node we just created, call the Save method to save the file, and then happily go back to working on our scripts for the upcoming Scripting Games.
Which, in case we didn't mention it, are scheduled for February 15th through March 3rd at the TechNet Script Center.
So far, so good. We can create a new XML file, and we can add new records to that file. That's all pretty cool, but we still have a ways to go; for example, how do we modify existing records in the file? Well, here's at least one way to do that, as shown in Figure 4.
Figure 4 Modifying the XML (Close Figure 4)
Set xmlDoc = _
  CreateObject("Microsoft.XMLDOM")

xmlDoc.Async = "False"
xmlDoc.Load("C:\Scripts\Audits.xml")

Set colNodes=xmlDoc.selectNodes _
  ("/ITChecklist/ComputerAudit " & _
   "[ComputerName = 'atl-ws-100']/AuditDate")

For Each objNode in colNodes
   objNode.Text = Date
Next

xmlDoc.Save "C:\Scripts\Audits.xml"
The important part of this script occurs when we call the selectNodes method, a method that determines which records our query will return. As you can see, when we called selectNodes we were careful to specify two criteria: we only want records where the ComputerName attribute is equal to atl-ws-100, and we only want to get back the AuditDate attribute.
Note: You say you can't see how that works? Then take a look at our first article on working with XML files (technetmagazine.com/issues/2007/02/HeyScriptingGuy); the selectNodes query syntax is explained in more detail there.
As it is wont to do, selectNodes returns a collection of all the XML records that meet the specified criteria. In turn, that means we can update the value of the AuditDate attribute (the only attribute we asked for) simply by setting up a For Each loop to walk through all the items in the collection and then, within that loop, assigning a new value to AuditDate:
For Each objNode in colNodes
   objNode.Text = Date
Next
What's that? You're wondering if we could modify more than one attribute at a time? Yes, we could. But sadly, not today; that's something we'll have to address in a future column.
You say you have another question: How would you update the audit date for all the computers in the file? That's easy; you'd use the script in Figure 5.
Figure 5 Changing the audit date (Close Figure 5)
Set xmlDoc = _
  CreateObject("Microsoft.XMLDOM")

xmlDoc.Async = "False"
xmlDoc.Load("C:\Scripts\Audits.xml")

Set colNodes=xmlDoc.selectNodes _
  ("/ITChecklist/ComputerAudit/AuditDate")

For Each objNode in colNodes
   objNode.Text = Date
Next

xmlDoc.Save "C:\Scripts\Audits.xml"
The only difference between this script and our previous XML-modifying script? With this one we didn't specify that we only wanted to view those records where the ComputerName was equal to atl-ws-100. By leaving out that criteria, we get back all the records by default.
Let's take a look at one last script before we go. Suppose our old pal atl-ws-100 has played its last game of solitaire and has shuffled off to that great computer graveyard in the sky.
Theological note: So where do computers go when they die? As it turns out, they're usually given to one of the Scripting Guys. For example, the Scripting Guy who writes this column was once given a laptop computer because, "You do a lot of good things for Microsoft, and you really should have a laptop computer." The only downside to this generous offering? The computer was broken and couldn't actually be turned on. Which was probably just as well. As it turned out, it didn't have a hard disk either.
In order to complete the great circle of silicon life, we need to delete atl-ws-100 from our XML file. How are we going to do that? Check out Figure 6.
Figure 6 Deleting from the XML file (Close Figure 6)
Set xmlDoc = _
  CreateObject("Microsoft.XMLDOM")

xmlDoc.Async = "False"
xmlDoc.Load("C:\Scripts\Audits.xml")

Set colNodes=xmlDoc.selectNodes _
  ("/ITChecklist/ComputerAudit " & _
   "[ComputerName = 'atl-ws-100']")

For Each objNode in colNodes
  xmlDoc.documentElement.removeChild _
    (objNode)
Next

xmlDoc.Save "C:\Scripts\Audits.xml"
As you can see, this resembles our previous script, the one that modified the AuditDate property. There're really only two differences here. For one, notice that we didn't bother to specify any property values in our selectNodes query; when we do that, we get back the entire XML record for atl-ws-100:
Set colNodes=xmlDoc.selectNodes _
  ("/ITChecklist/ComputerAudit " & _
   "[ComputerName = 'atl-ws-100']")
And then, within our For Each loop, we simply call the removeChild method to delete all the records where the ComputerName is equal to atl-ws-100:
xmlDoc.documentElement.removeChild _
  (objNode)
And that's it. Farewell, atl-ws-100; we hardly knew ye.
OK, any TechNet Magazine editors still reading this month's column? You say they've all gone off to try the monthly scripting puzzle? Excellent. Now that no one's reading over our shoulders, we can tell you this: drop whatever you're doing (like, say, reading this magazine) and get on over to the Script Center and start preparing for the Scripting Games. After all, TechNet Magazine will be here for a long time to come. But the Scripting Games are only here once a year, and for a limited time at that (specifically, from February 15th through March 3rd, just so you know). Don't miss out!
Note: You know, now that you mention it, we did wait until you'd read our entire column before we told you to stop reading TechNet Magazine, didn't we? Wonder how that happened ...

Dr. Scripto's Scripting Perplexer
The monthly challenge that tests not only your puzzle-solving skills, but also your scripting skills.

February 2008: Mystery Symbols
The computer keyboard includes all sorts of crazy-looking characters; even crazier is the fact that most of these characters have a specific use in the scripting languages VBScript or Windows PowerShell. See if you can match each symbol with its scripting use. The first one has been done for you.

Dr. Scripto's Scripting Perplexer
Answer: Mystery Symbols, February 2008

The Microsoft Scripting Guys work for—well, are employed by—Microsoft. When not playing/coaching/watching baseball (and various other activities), they run the TechNet Script Center. Check it out at www.scriptingguys.com.


 Security Watch: Island Hopping: Mitigating Undesirable Dependencies
Security Watch
Island Hopping: Mitigating Undesirable Dependencies
Jesper M. Johansson

In last month's installment of Security Watch, I discussed the beginning steps of using a USB flash drive to attack a network. The attack started with an infected USB flash drive being plugged into a computer. The malicious code then executed, either automatically or with minimal help from the user (making use of some simple social
engineering). This attack remained local to the workstation, but it is clear that the malware could have spread to the rest of the network. I discuss this form of attack on a network in detail in my upcoming book, Windows Server 2008 Security Resource Kit (Microsoft Press®, 2008), and I am adapting parts of that chapter for this column.
Obviously, one option for protection is to ban removable drives. While this may seem prudent, your users are likely to ambush you on the way to your car if you try to do this; and it can be hard to fault them for doing so. The better option, in all but the most sensitive environments, is to attempt to manage the risk involved and contain the exposure.
Furthermore, removable drives are by no means the only method of compromising a client computer. Remember the "10 Immutable Laws of Security" (available at microsoft.com/technet/archive/community/columns/security/essays/10imlaws.mspx)? Well, the general sentiment of Law 3 is still true: "If a bad guy has unrestricted physical access to your computer, it's not your computer anymore." In the context of this discussion, if an attacker has access to your computer, that computer must be considered compromised. This can even be perpetrated remotely if the attacker can get you to run his code on your computer. You may recognize this as Law 1 of the Immutable Laws: "If a bad guy can persuade you to run his program on your computer, it's not your computer anymore."
It's safe to assume that the immutable laws still hold—they have proven to be remarkably resilient, and they are unlikely to change significantly until we fundamentally change the way computers work. Therefore, considering how these laws apply to the scenario outlined, it's critical that you defang removable drives. And you can do this with a few registry tweaks.
Of course, you must use additional layers of protection as well. You can reasonably assume that many of your client computers have either already been compromised or are operated by users who do not always have your organization's best security interests in mind. This means you need to mitigate their effects on the remainder of the network, and it increases the importance of understanding, analyzing, and mitigating security dependencies.

Definition of a Security Dependency
A security dependency occurs when the security of one computer is dependent on the security of another. You may have heard people say that if your domain controller (DC) has been hacked, your entire network has been hacked. This is a simplistic way to state that all domain members are dependent on the DCs for their security. If the DC is not kept secure, the member computers cannot be kept secure. If an attacker can change the security configuration of the domain, he can take over any computer in the domain by, for example, adding new accounts to the Administrators group on a member computer. Any vulnerability that permits compromise by an administrator just isn't interesting. An administrator is supposed to have complete access to whatever he is administering.
Dependencies in computer systems are unavoidable. In fact, they are common and often desirable; however, that doesn't mean all dependencies are acceptable. In this column, I discuss which types of dependencies are acceptable and which are not, and then I will analyze the types of dependencies and how to mitigate them. In Windows Server 2008 Security Resource Kit, I go into more detail on the specific dependencies and discuss how to manage them.

Acceptable Dependencies
In general, a dependency is acceptable when a less sensitive system depends on a more sensitive system for security. Computers and systems in general can be divided into classes based on how sensitive they are. (The specific set of classes in any particular environment is irrelevant to the general discussion—all that matters is that there are inherent classifications.) Just for the sake of discussion, assume I have two classifications: workstations and DCs. In that scenario, it is acceptable for the workstations to depend on the DCs for their security. The DCs class is far more sensitive than the workstations so, obviously, it must be better protected.
The same argument can be made for user accounts. It is acceptable that an administrator can compromise data owned by a user. This is because administrators take on more responsibility and have unfettered (although not always direct or obvious) access to the computer and everything on it. If you understand this point and manage the computers appropriately, there is nothing wrong with this dependency.
Even software can be analyzed in the same way. It is certainly acceptable for a less sensitive piece of software, such as a Web browser, to use and depend on a more sensitive piece of software, such as the operating system, for its security. If the OS has a bug, the fact that the Web browser is now vulnerable to some new problem is not surprising and is probably low on the list of immediate concerns—the OS and other critical apps and data will be the primary focus of attention. This pertains to how bugs are fixed or where hotfixes go—a bug should be fixed as close as possible to the problem. In doing so, the protective impact of the fix is maximized. Thus, rather than working around the problem in the Web browser, the OS itself must be fixed.
Alternatively, the dependency can be removed by changing the design. For example, the Web browser can be rewritten to reduce its dependencies on the OS. This latter approach is appropriate in cases where the functionality of the more secure components (the OS, in this case) was never actually intended to be used in the way that the less sensitive component (the Web browser) is using it.

Unacceptable Dependencies
Given my explanation of acceptable dependencies, the definition of an unacceptable dependency should now be obvious. Basically, a more sensitive system must never depend on a less sensitive system for its security.
If the compromise of a workstation means that the DC's security has been breached, you have a serious security problem on your hands. It is impossible to protect a network if its aggregate security is dependent on the security of every single computer on that network.
Consider this statistically. If every computer on the network is "secure" 99.999 percent of the time, you might think you have an extremely secure network. In fact, that percentage is probably far beyond what is realistic in all but the smallest networks today. But now assume that your network has 40,000 computers, any one of which is vulnerable .001 percent of the time. Anecdotally speaking, your overall network would be unsecure up to a potential of 40 percent of the time.
And, of course, with unmanaged dependencies it just takes one computer at that .001 percent of the time to compromise the entire network. In these terms, how secure is the network? Clearly, it is absolutely paramount that more sensitive systems are protected from less sensitive ones.
This argument can easily be extended to user accounts and software. For example, the new Terminal Services client for Windows® allows you to store user names and passwords for virtually transparent Terminal Services logon. Those credentials are stored using the Credential Manager API, protected by the credentials used for the primary logon session.
This can create a security dependency. Consider the case of a network administrator logging on to her personal workstation. She uses this workstation for e-mail, Web browsing, and other typical information worker tasks. Naturally, she uses a low-privileged domain account for this purpose.
At some point during the day, this administrator connects to one of the DCs to perform some management tasks. She uses the Terminal Services client to do this and chooses to store her password to make future connections easier. This results in at least one, and possibly two, unacceptable security dependencies. The first is that her domain administrative account credentials are now protected by her low-privileged information worker credentials. If this low-privileged user account is compromised, the domain administrative user account is also compromised and, therefore, the entire domain is compromised.
The second dependency results from the fact that she typed a domain administrative credential on a non-domain controller. Unless her personal workstation is protected at least as well as the DCs—and it probably is not—there is a dependency where the security of the DCs depend on the security of this user's personal workstation. If a disgruntled employee in the same office, for example, has installed a keystroke logger on the network administrator's workstation, the domain administrative credentials have now been captured. Any time you type a domain administrative credential on a non-domain controller you expose the entire domain to any security flaws on the non-domain controller.
Now suppose an attacker inserts a removable drive into a computer where a domain administrator is currently logged on, or has ever logged on, or will ever log on. That domain administrator becomes compromised, and, by extension, the entire domain is compromised. It is imperative that you understand this situation so you can avoid it. And, of course, the same situation can happen with software when a secure application relies on the functionality of a less secure application to perform a certain task.

Analysis of an Attack
Earlier I described what can happen if a malicious removable drive is inserted into a computer. However, it may only be obvious what would happen to the computer in which the drive is initially inserted. So assume that the computer in question is domain joined, as shown in Figure 1.
Figure 1 Ideal domain dependencies
Figure 1 Ideal domain dependencies
The scenario illustrated here depicts an ideal dependency. The arrows are directional and point to where the dependency is inherited from. For instance, the security of the workstation depends on the security of the DC, and the security of the user is dependent on the security of the workstation. The attacker may be able to compromise the workstation, which would compromise any information the user has placed on that workstation, but the compromise would be isolated there.
But suppose the user logging on to the workstation is a member of the local administrators group on the server. And say the domain administrator frequently logs on to the server. Now you have the dependencies shown in Figure 2.
Figure 2 Compromised domain dependencies
Figure 2 Compromised domain dependencies
By simply changing the assumption as to who has logged on to the computers in question, the security of the entire network has been compromised. Since a domain admin logs on to the server, the security of the DC and, hence, the entire domain is dependent upon the security of that server.
This would be acceptable if the server were managed as securely as the DC. However, in this case, a user who logs on to the workstation is a member of the Administrators group on the server. Thus, the security of the server is dependent on the security of the workstation. That means the security of the entire domain is dependent on the security of the workstation. And, guess what: the user on that workstation just unwittingly ran the attacker's tool.
There are probably few concepts in today's world of information security that are more important to grasp than security dependencies. If you start analyzing your network and try to understand what the dependencies are, you will almost certainly encounter unacceptable dependencies. In the worst-case scenario, which is far more common than you might think, the security of the entire network is dependent on the entire network—in other words, the security of every single computer is, in some way, dependent on the security of every other computer. It is therefore impossible to create any kind of reasonable and realistic risk management strategy in that type of environment since the relationships are impossible to control and the complexity is incomprehensible. The solution is to analyze and manage your dependencies.
In this column, I've presented only a short overview of dependencies and how you can analyze and mitigate them. You can find more information in my book, Windows Server 2008 Security Resource Kit. It contains an entire chapter devoted to the topic of securing the network by analyzing dependencies and managing them using both sophisticated techniques, such as Server and Domain Isolation, as well as more mundane techniques, such as administrative account management.
I would like to thank David LeBlanc for helping shape the embryonic ideas that led to this column.

Jesper M. Johansson is a Software Architect working on software security issues and is a contributing editor to TechNet Magazine. He holds a PhD in MIS and has more than 20 years of experience in security.


 The Desktop Files: Dual Booting with Windows XP and Windows PE 2.0
The Desktop Files
Dual Booting with Windows XP and Windows PE 2.0
Wes Miller
Download the code for this article: Desktop2008_02.exe (151KB)

In previous installments of The Desktop Files, I've mentioned that Windows PE has become a tool for recovery, and I've explained how you can use it from a second partition. An inquiry I received recently has made me think that it might be worth discussing a new spin on this—using Windows PE 2.0 and Windows XP together.
After all, many of you may be using Windows® XP for some time, and Windows PE 2.0 is the version Microsoft is supporting going forward (note that there will be no further updates to Windows PE 1.x).
One important difference between Windows PE 1.x and 2.x is that Windows PE 2.x can boot via a standard boot interface (versus the specialized method that Windows PE 1.x uses). While you can boot Windows PE 1.x or 2.x from a second partition, the convenience of booting it on the same boot partition as your operating system is installed on has lots of potential, which I'll discuss shortly.

Two Boots for Windows
Windows NT® has always been relatively tolerant of dual booting. But as Extensible Firmware Interface (EFI)-based computers have begun to replace the old IBM PC AT BIOS, Microsoft has changed the Windows boot mechanism to work more optimally with EFI and to have one boot mechanism across all architectures. The new boot mechanism for Windows, beginning with Windows Vista®, is referred to as the Boot Configuration Database, or BCD (see microsoft.com/whdc/system/platform/firmware/bcd.mspx). But whether you are talking about the old boot.ini or the new BCD mechanism, Windows can generally boot other operating systems, even if it means handing off the boot process to another boot loader.
Before we start, it is critical that you remember that Windows PE 2.0 only works on systems that have both full Advanced Configuration and Power Interface (ACPI) compatibility and 384MB (ideally 512MB) of RAM. As I noted in my May column (technetmagazine.com/issues/2007/05/DesktopFiles), non-ACPI systems will fail to boot Windows PE 2.0, and, because Windows PE 2.0 must boot to a RAMDisk, it needs to have enough RAM to hold the RAMDisk, any scratch space, and all RAM working set. If your system has less than 384MB of RAM, don't even consider this route; use Windows PE 1.6 instead. Even a hand skilled at reducing Windows PE size cannot make it run well on 256MB.

Oh, the Problems You'll Solve
Now let's take a look at the problems you might tackle by booting Windows PE 2.0 from the same partition as Windows XP.
Full-System Recovery This is just one task made possible by booting Windows PE right from the Windows XP partition. Since Windows PE 2.0 (as Windows PE 1.6 before it) can boot entirely into a RAMDisk, you can reimage the entire disk—or use ImageX to restore the Windows XP files by themselves, since it's not necessary to reimage the entire partition.
By allowing users to boot to Windows PE at startup, they can recover their own systems even while working remotely. The image could come from a secondary partition, a network share (if the users won't be mobile), a recovery DVD or USB hard disk, or, if you are using ImageX, you could even store it on the partition being recovered. This sort of on-disk recovery is a great solution for many problems (with the exception of physical disk corruption). It's obviously not great at restoring user data unless that data is protected in the first place and then restored as a part of the recovery. Remember, you should be using roaming profiles and backing up data from the PC to prevent that kind of loss. You should also ensure that any recovery mechanism you use can mitigate any potential data loss on behalf of the user (including loss due to EFS encryption keys being reset).
File Recovery Windows PE also provides the mechanism to get a system up and running even if the primary installation of Windows will not boot. In that case, you are able to boot and recover user data either to another system (provided you have warm backup systems available for users) or to another location so that the user may begin working again soon.
System Repair Finally, booting Windows PE on Windows XP lets you repair a system by running either a set of troubleshooting scripts or a simple user-driven repair or recovery application. The process is similar to full system recovery, but it will be narrower in scope (and won't risk user data).
Now that you have some idea of what you can do if you boot Windows PE on the Windows XP partition, let's see what is going on behind the scenes so you can apply the process to any solution you like.

Before Windows Vista
Before the advent of Windows Vista, here's how BIOS/Master Boot Record (MBR)-based systems booted. The BIOS pointed to the boot device (in the case of the hard disk, it points to the MBR on the disk). The MBR executed code that specified which partition to boot. Note that this is the first partition that has the active flag set for it. In the earliest versions of diskpart, you couldn't make a partition inactive (that is, you couldn't remove the active flag). The switch was added to diskpart to fix this—otherwise it would be the first partition in partition table order that had its active flag set. The partition contains the boot sector—which tells it which code to execute next. From Windows NT to Windows Server® 2003, the code to run is in a file named ntldr, with no file extension. If you've ever formatted a disk, didn't install Windows, and then attempted to boot, you've seen the "NTLDR not found" error message.
You'll be interested to know that ntldr is fully backward-compatible—but of course it's not forward-compatible. This means that Windows XPs ntldr can boot Windows 2000, but not vice versa. This is why it's always important to install the oldest operating system first if you plan to dual boot. Since ntldr doesn't have version numbering attached to it, it's even more important to keep this straight.
Ntldr is a real-mode (16-bit) app that launches NTDetect.com, which performs basic device detection and then launches the kernel, ntoskrnl.exe (this may be one of several files—always renamed to be ntoskrnl.exe). The kernel is the critical component in the Windows boot process.
This summary of the boot process is somewhat simplistic, and I've omitted the entire initialization of Win32®, but that's not pertinent to this discussion. If you want more information about it, I recommend you read Windows Internals, Fourth Edition by Mark Russinovich and David Solomon (Microsoft Press®, 2004).
Another thing to note is that when you boot from CD, Windows Setup and Windows PE run setupldr.bin instead of ntldr. This file actually contains slightly different boot code so that it can boot from CD (which ntldr cannot). This is why in order to boot Windows PE 1.x, you renamed setupldr.bin to ntldr and put it on disk. It's also why that file couldn't then load a boot.ini file (because setupldr.bin was designed just for CD booting, and, as a result, boot.ini wasn't expected to be necessary).

Windows Vista and Beyond
Windows Vista changed many pieces of this process. Windows Server 2008 changes it further by enabling IA32 (32-bit support) for booting from EFI—something that hasn't been possible to date outside of Itanium-based systems. For BIOS-based systems, the initial steps of the boot process are largely unchanged. The key difference is that BOOTMGR is the boot loader that is initialized and then performs the following steps.
It looks in the boot directory for the BCD file, which is effectively its own registry hive. The BCD contains the boot entries (including the default) to boot. See Figure 1 for image types (executable file types), and see Figure 2 for the specific application types to launch. One (or more) of these entries can be a reference to ntldr. Only one of them can actually reference the well-known GUID used to load ntldr (see the fifth row of Figure 3). The bootmgr then decides which entry to load next, depending on whether the system is being booted, has been hibernating, or is booting to ntldr or another boot manager (in which case it would point to another boot sector and launch that loader). This would be most common if you were dual booting with a non-Windows operating system. You can find more information at microsoft.com/whdc/system/platform/firmware/bcd.mspx.
Figure 3 Standard application objects (Close Figure 3)

Description | BCDEdit ID | GUID
Windows Boot Manager | {bootmgr} | 9dea862c-5cdd-4e70-acc1-f32b344d4795
Firmware Boot Manager | {fwbootmgr} | a5a30fa2-3d06-4e9f-b5f4-a01df9d1fcba
Windows Memory Tester | {memdiag} | b2721d73-1db4-4c62-bf78-c548a880142d
Windows Resume Application | No alias | 147aa509-0358-4473-b83b-d950dda00615
Legacy Windows Loader | {ntldr} | 466f5a88-0af2-4f76-9038-095b170dc21c
Current Boot Entry | {current} | fa926493-6f1c-4193-a414-58f0b2456d1e
Default Boot Entry | {default} | no predefined GUID
Figure 2 Application types (Close Figure 2)

Description | Value | Description
Firmware Boot Manager | 0x1 | Applies only to EFI systems.
Windows Boot Manager | 0x2 | Controls boot flow. In a dual boot system, displays a boot selection menu to the user.
Windows Boot Loader | 0x3 | Loads a particular version or configuration of Windows.
Windows Resume Application | 0x4 | Restores Windows to its running state when a computer resumes from hibernation.
Windows Memory Tester | 0x5 | A memory diagnostics application.
Ntldr | 0x6 | Applies only to PC/AT BIOS systems. Loads versions of Windows earlier than Windows Vista.
Boot Sector | 0x8 | A 16-bit real-mode application. Applies only to PC/AT BIOS systems. Can be used to restart the boot process and load a non-Windows operating system.
Figure 1 Image types (Close Figure 1)

Description | Value
Firmware Application | 0x1
Boot Application | 0x2
Ntldr-Based Loader Object | 0x3
Real-Mode Application | 0x4
You'll notice that the Windows Vista revamped boot infrastructure is much more organized than previous versions—outside of the bootmgr, all of the boot code exists within the root-level \boot directory and is handed off to Windows-specific binaries once BCD has specified to do so.

Booting Windows XP and Windows PE 2.0
You can probably start to see now how Windows PE, using the Windows Vista boot manager, can act as an intermediary that allows you to continue to boot Windows XP or earlier versions of Windows while adding Windows PE 2.0 to the mix for an integrated recovery tool.
Of course, you are going to be adding another step to the boot process, but it is a step that Microsoft designed. Knowing that the new bootmgr infrastructure needed to be backward-compatible with the existing ntldr and other third-party boot managers, bootmgr was intentionally built to support such solutions.
So now that we've examined how Windows boots and how the new BCD works, how do we tie the two together and get Windows PE 2.0 down on our existing Windows XP installation? There are two ways to put down the Windows Vista boot sector—the piece of boot code that tells the system to start bootmgr. One of them is to use diskpart and create and format a new partition with Windows PE 2.0. But that's not going to be an option here, since you've already got an existing Windows XP installation that you don't want to damage or move. Instead, follow this process:
  1. Ensure you have a copy of bootsect.exe and bcdedit.exe on your Windows PE 2.0 boot CD (they can both be found on your Windows Vista installation DVD).
  2. Boot your Windows XP system to the Windows PE 2.0 CD.
  3. Once booted, run this command:
    bootsect.exe /NT60 ALL
This will set the boot sector to be the Windows Vista-compatible boot sector that knows how to launch bootmgr.
4. Copy the boot manager to the hard disk:
    xcopy BOOTMGR C:\BOOTMGR /I
5. Copy the boot.SDI file to the \BOOT directory on the hard disk. You'll need to make a copy of this before you boot, as it will be locked and unable to be copied while running Windows PE:
    xcopy BOOT.SDI C:\BOOT\
Correct the source path accordingly.
6. Copy the boot.WIM to the hard disk. Like the SDI, you should have a secondary copy available; you will not be able to copy the file you are running from:
    xcopy BOOT.WIM C:\BOOT\BOOT.WIM
Correct the source path accordingly.
7. Now, create a new BCD store:
    bcdedit /createstore
8. Next, create the BCD entries to load Windows PE:
    bcdedit /create {ramdiskoptions} /d
    "Boot Windows PE"
    bcdedit /set {ramdiskoptions}
    ramdisksdidevice partition=c:
    bcdedit /set {ramdiskoptions}
    ramdisksdipath \boot\boot.sdi
    bcdedit /create /d "Windows PE" /
    application
    OSLOADER
9. The last line will return a GUID. Replace {YOURGUID} with the actual GUID that was returned. These are the entries to RAMDisk that boot Windows PE from your hard disk, shown in Figure 4.
Figure 4 Entries to RAMDisk that boot Windows PE from your hard disk (Close Figure 4)
bcdedit /set {YOURGUID} device ramdisk=[c:]\boot\boot.wim,{ramdiskoptions}
bcdedit /set{YOURGUID} path \windows\system32\winload.exe
bcdedit /{YOURGUID} osdeviceramdisk=[c:]\boot\boot.wim,{ramdiskoptions}
bcdedit /set{YOURGUID} systemroot \windows
bcdedit /set{YOURGUID} winpe yes
bcdedit /set {YOURGUID} detecthal yes
10. Now, create bootmgr entries so Boot Manager will start:
    bcdedit /create {bootmgr} /d
    "Boot Manager"
    bcdedit /set {bootmgr} device boot
11. Finally, set up a BCD entry for your existing Windows installation (change the display text from Windows XP to whatever you would like to be displayed in the boot manager boot menu. The last command also sets it to be the first (default) entry in the boot menu:
    bcdedit /create {ntldr} /d "Windows XP"
    bcdedit /set {ntldr} device partition=C:
    bcdedit /set {ntldr} path \ntldr
    bcdedit /displayorder {ntldr} /addfirst
12. You should now be able to restart your system and have it launch either the previous installation of Windows or Windows PE 2.0 by selecting the appropriate boot menu entry. Figure 5 shows the sample output from bcdedit /enum all. This command lists all entries in the BCD after the entries have been added (this is from within Windows PE 1.6).
Sample input
I have been working on a script that can perform this task automatically. Currently it can only copy the boot files to the hard disk and set up your BCD entries. It cannot set the boot sector to be the new Windows Server 2008 value while running Windows —that has to be done offline, under Windows PE. You can find the script in the code downloads section of the TechNet Magazine Web site at technetmagazine.com. To run the script, you will want to have the following items in the same directory as the VBScript you are running:
  1. BCDEdit
  2. BOOTMGR
  3. A \BOOT\ directory (from the Windows Vista or Windows PE 2.0 media)
  4. A \SOURCES\ directory with a valid, bootable Windows PE 2.0 BOOT.WIM file
It will create the BCD and copy the appropriate boot files. After booting to Windows PE and running bootsect, you should be able to dual boot.

Undoing the Dual Boot
Undoing the dual boot is even easier than putting it in place. You just need to restore the boot sector to the pre-Windows Vista boot sector by running the following command:
bootsect.exe /NT52 ALL
Once it is restored, you can delete bootmgr and the \boot directory. On the next reboot, the boot sector will force ntldr to start instead and completely remove Windows PE 2.0 and all Windows Vista boot binaries from the boot flow.

Wes Miller is a Technical Product Manager at Initiate Systems (InitiateSystems.com) in Austin, Texas. Previously, he worked at Winternals Software and as a Program Manager for Microsoft. Wes can be reached at technet@getwired.com.


 Field Notes: Donate Your PC
Field Notes
Donate Your PC
Jim Lynch

The conventional wisdom for many years about the most cost-effective way to dispose of computers was best captured in a research paper, published in 2003 by Frances O'Brien of The Gartner Group, titled "PC Disposal: TCO's Last Surprise." This comparative analysis of PC disposal found that donating equipment was the most expensive option (out of six disposal options) due to the amount of time it took to locate the right charitable organization willing to accept donations.
Tips for Donating
TechSoup.org offers additional tips for donating and also obtaining donations at techsoup.org/recycle. Some things to consider before donating are:
  • Don't hang on to IT equipment after you've upgraded to new hardware. Computer equipment loses its reuse value after five years. Likewise, cell phones typically reach end-of-life after three years.
  • Donate IT equipment that is less than five years old to refurbishers rather than directly to nonprofits or schools.
  • Package the accessories with the equipment, including the keyboard, mouse, printer, modem, software disks, and manuals that came with the equipment.
  • Keep a list of what you donated for your records and get a receipt. You may well be eligible for a tax deduction if you donate to a nonprofit refurbisher. In the U.S., business donors can deduct the un-depreciated value of the computer, and individuals can deduct the current market value of a computer. To determine the fair market value of a computer, go to Canada's free Used Computer Evaluator (servlets.edeal.com/servlets/PcEval).

Today it's a whole new ballgame. In fact, the U.S. EPA recently embarked on a campaign called "Do the PC Thing" that educates users on how to prepare their old hardware for reuse and offers a list of resources. This campaign is a part of the EPA's electronics recycling and reuse public awareness site, Plug-In To eCycling (epa.gov/plugin/). Click on the "Reuse It!" link for articles aimed at consumers or businesses.
On this site you'll learn that the most important issue with respect to donating or recycling hardware is protecting yourself. The "Do the PC Thing" literature offers basic pointers on hard-drive wiping and the legal requirements for data security and confidentiality. Whether you are disposing of your computer either for end-of-life recycling or for reuse, you must remove your sensitive data, such as passwords, documents, credit card information, e-mail messages, and Web logs.
Most electronics recyclers, refurbishers, and asset management companies provide data security services either by destroying hard drives or by using a disk cleaning software utility that overwrites data so that it is unrecoverable. Note that if your computers are earmarked for reuse, it is preferable to keep the computer hardware intact.
Reputable disk cleaning software uses U.S. Department of Defense 5220.22-M specifications. This type of software-based data cleansing systematically overwrites all addressable locations on your hard drive with a character, verifies the overwrite, and then overwrites all addressable locations again with random characters. In 2004, the U.S. National Security Agency (NSA Advisory LAA-006-2004) found that a single overwrite using Department of Defense standard software is sufficient to render electronic files unrecoverable.
It is now reasonably easy to do your own data sanitation. There are reputable freeware apps such as Active@ Kill Disk Hard Drive Eraser (killdisk.com/eraser.htm) or Darik's Boot and Nuke (dban.sourceforge.net). If you have several computers to data clean, consider more robust commercial applications such as Blancco Data Cleaner (blancco.com) or WipeDrive (whitecanyon.com).
If your IT equipment is more than five years old or badly broken, one of the six disposal options that Gartner studied is end-of-life electronics recycling. There are now more than 900 companies that do this work in the U.S. To properly recycle older IT equipment, see the Electronic TakeBack Coalition's list of responsible recyclers (www.computertakeback.com/the_solutions/recyclers_map.cfm).

Jim Lynch is the Program Director for computer recycling and reuse at TechSoup.org, a San Francisco-based high-tech nonprofit whose mission is to supply low-cost hardware, software, and know-how to charities around the world.


 Windows Confidential: Scrapping the Scraps
Windows Confidential
Scrapping the Scraps
Raymond Chen

It doesn't happen often , but occasionally an old feature gets deleted—with much celebration from the product team. In Windows® XP, for instance, the interface for setting a desktop pattern was removed. The underlying functionality is still there, if you can manage to write a program that calls SystemParametersInfo(SPI_SETDESKPATTERN), but there is no longer a built-in interface to it. Why was the desktop pattern removed from the desktop control panel?
Our research indicated that practically nobody used desktop patterns anymore. They were more popular when memory was more expensive and users didn't want to waste a quarter of the machine's memory on a 1MB wallpaper bitmap. A tiny little 8×8 monochrome pattern repeated over the screen gave some degree of personalization on a 640×480 display at a much lower cost.
However, changes over the years made desktop patterns less and less attractive. First, of course, is that computers now have much more memory than they did back in 1983. Second (also of course) is that monitors run at much higher resolutions than they did back then. A cute 8×8 houndstooth pattern is basically a gray smear when displayed on a 1600×1200 monitor.
That's not the only old feature on the scrap heap. Another example is scraps. Introduced in Windows 95, scraps appear as fragments of documents captured into a file. That file fragment is really an OLE object. The idea behind scraps was that you could, for example, highlight some text in a word processor and drag it to the desktop, where a scrap was then created. Later on, you could take that scrap and drag it back into the word processor or some other program, and the text you used to create the scrap would be inserted into the new document. It's sort of like copy and paste, except that instead of storing the data temporarily on the clipboard, it was stored in a file.
Now, once you created such a scrap file, the only thing you really could do with it was to drop it into another document, in the same way that the only thing you can really do with something on the clipboard is to paste it somewhere. At some point, somebody added the feature where you could double-click the scrap file, and it would open in the application that originally created it. (The contents could be anything, so the only way to display the scrap is to ask the program that originally created it.)
I don't know whether this feature was added as a debugging tool or so users could see what was in a scrap without having to create a dummy document to drop it into. Either way, virus writers loved this "open the application that originally created the scrap" functionality because it allowed them to create a scrap file and fill it with data that said, "The original application is cmd.exe, and the document fragment is this batch file." When somebody double-clicked that particular scrap file to see what was in it, the scrap viewer handed a batch file to the command processor and said, "Please open this and show it to the user."
Given the history of scraps, the shell team set out to remove the feature from Windows Vista®. In order to do this, the team had to research how popular scraps were in the real world. So the shell team asked the product support group to call up the group's logs to see how many people called with questions about scraps. The theory was that a popular feature gets support calls and unpopular features don't (because nobody is using them). Scraps in particular don't exactly fall in the "blatantly intuitive" category of features, so a small number of calls couldn't be explained away as, "Well, the feature is so obvious that nobody needs to call to get help with it!"
The answer came back from the product support group. In the past year, the group received a total of four calls. And all of them were of the form, "I created this strange file. What is it and how do I get rid of it?"

Raymond Chen's Web site, The Old New Thing, and identically titled book (Addison-Wesley, 2007) deal with Windows history and Win32 programming. It's been ages since Raymond has seen you, and he wonders how the family is doing.