April 2009
Internet Explorer: What’s New in Internet Explorer 8
Everyone expects something different from their browser. Some users are most concerned with security and privacy while others are more focused on customizability. Some want full standards compatibility, and others just desire ease of use. Explore the numerous new features and enhancements in Internet Explorer 8 that allow it to meet every user’s needs. Matt Hester
Active Directory: Export, Compare, and Synchronize Active Directory Schemas
If your organization has multiple Active Directory forests, you need to manage multiple Active Directory schemas and ensure consistency between schemas. Check out our step-by-step guide to comparing and synchronizing Active Directory schemas in multi-forest environments. John Policelli
Office Communications Server: Stay Connected with Office Communicator Web Access
Office Communicator Web Access is a messaging client that requires nothing more than a Web browser. Discover how it can extend the capabilities of Office Communicator—instant messaging, presence, desktop sharing, and so on—to anyone with an Internet connection and a supported browser. Greg Stemp and Jean Ross
Office Communications Server: How Voice Conferencing Powers OCS 2007 R2
Office Communications Server 2007 provides very powerful and flexible conferencing capabilities. Learn how OCS 2007 R2 allows users to set up real time conferences with other users inside and outside the corporate firewall, supporting both ad-hoc escalation of calls to a conference and pre-scheduled conferences and meetings. Rajesh Ramanathan
Windows HPC Server 2008: High Performance Computing in the Real World
Windows HPC Server 2008 was designed specifically for use in compute-intensive environments. See how it is used, along with other Microsoft technologies, in a life science laboratory to provide a solution that enables automated processing of data generated from an imaging device. Joshua M. Kunken
Columns
From the Editor: A Moment’s Pause: | Toolbox: New Products for IT Pros: Greg Steen discusses NetCmdlets V2 for Windows PowerShell, KeePass Password Safe V2 for managing passwords, and Regular Expression Designer. He also looks at the book "Windows Administration Resource Kit".Greg Steen | Sustainable Computing: How Digital Healthcare can Help the Environment, Reduce Costs, and Improve Patient Service: Explore how advances in technology can help reduce material waste in the medical field, reduce a variety costs, and help to keep patients safer and healthier.Nihar Ganju and Michael Walsh
SQL Q & A: Database Consistency, Temporary Tables, and More: Why does DBCC CHECKDB find corruption errors that disappear? Does tempdb really cause performance issues? Are there any drawbacks to using the FILESTREAM datatype? Paul Randal answers these reader questions and more.Paul S. Randal | Utility Spotlight: RichCopy: RichCopy is a powerful, multi-threaded alternative to the popular RoboCopy file copy utility. Take a look at how this free tool can significantly improve file copy performance.Joshua Hoffman | Inside SharePoint: Security and Compliance with AD RMS: SharePoint integration with AD RMS can be useful for extending security and regulatory compliance solutions beyond the SharePoint environment. But for real security, the SharePoint environment must be secure within itself.Pav Cherny
Windows PowerShell: Automating User Provisioning, Part 2: Don Jones continues his look at how you can use Windows PowerShell to automate user provisioning. In this installment, Don details the function that will create a mail-enabled user in Active Directory.Don Jones | Geek of All Trades: Simple Clustering with Hyper-V: Struggling to keep computers running with a minimum of resources and a shrinking budget is a challenge. One thing you can do is increase the availability of your servers with Hyper-V and failover clustering. Find out how.Greg Shields | Hey, Scripting Guy!: Creating CAB Files with Windows PowerShell: File compression is even more useful when you can automate the process via scripting. Here are some Windows PowerShell scripts you can use to create and expand cab files.The Microsoft Scripting Guys
Security Watch: The Challenge of Information Security Management, Part 1: The first in an ongoing series, Jesper Johansson discusses the broad and varied challenges faced in the field of Information Security Management and the keys to planning a sound InfoSec strategy.Jesper M. Johansson | Field Notes: Customer Centricity: A look at why IT organizations need to really care about the customers they serve, act on that commitment consistently, and collect feedback on their performance.Romi Mahajan | Windows Confidential: Forcing Handles Closed: Raymond Chen explores how forcing file handles closed can lead to a series of cascading errors.Raymond Chen


Internet Explorer
What’s New in Internet Explorer 8
Matt Hester
At a Glance:
  • Ease-of use features in Internet Explorer 8
  • A variety of enhancements
  • Standards-based browsing and compatibility
  • Security and reliability

Contents
Ease of Use
Better Together
More New Features
Compatibility
Security and Reliability
More Reliable
Enterprise Support
Developer Tools
Wrapping Up
What makes the World Wide Web so great? It’s the world, of course, a world full of people with wildly diverse needs. Some, like my dad and my kids, want safe and easy Web browsing. IT and support professionals want a secure and reliable browser that protects their businesses while supporting important internal Web applications. And Web developers want to realize the fantastic visions they have for the Web.
Internet Explorer 8 is a next generation Web browser that can meet all of these needs. In this article, I'll discuss the many improvements in Internet Explorer 8 that will enhance browsing for everyone.

Ease of Use
In Internet Explorer 8, a tremendous amount of work has been done to make the Web browsing experience easier and more intuitive. Two great innovations that typify the browser's new capabilities are Accelerators and Web Slices.
Accelerators are quick and easy ways to help you tie into the Web services and providers you use every day. To use an accelerator, all you need to do is highlight some text on a Web site and select the appropriate service to accomplish what you want, such as search, mapping, shopping, and even language translation.
Suppose you are looking for directions to a restaurant. You typically highlight the address, copy it, navigate to your favorite map service, paste the address, and then click go. With Internet Explorer 8 accelerators, it's much easier: when you select the address, the accelerator icon pops up. Then all you need to do is select the mapping service to generate a preview of the location (see Figure 1). If you need more, click on the preview and you'll be taken directly to that Web service.
Figure 1 Internet Explorer accelerators
Web Slices are another great enhancement in Internet Explorer 8. You're probably familiar with RSS, which lets you get frequently changing content automatically by subscribing to the service. Web Slices use very similar plumbing and are, in fact, are based on RSS technology. However, unlike an RSS feed where you subscribe to an entire category like weather, finance, or movies, Web Slices let you subscribe to only a portion, or slice, of information.
Suppose, for example, you want to be notified and updated about your city's local weather. With Web Slices, you can simply subscribe to your city's weather report by doing a Live search for, say, Dallas weather and then subscribe to the Web Slice (see Figure 2).
Figure 2 A Web Slice found through Live search
Alternatively, when you are browsing around and you land on a Web page with Web Slices, the Web Slice icon will appear when you hover over content you can subscribe to. If you click on the icon, the subscription will show up on your Favorites bar. Then, no matter where you surf, you will be able check in on that Web Slice. Better yet, the Web Slice will actually let you know when something has changed: it gets highlighted on the Favorites bar when there has been an update.

Better Together
Individually, accelerators and Web Slices are extremely powerful. When used together, however, they can really help a business bring users to their service and keep them engaged. For example, eBay provides both an accelerator and Web Slice for its online auctions. Customers of eBay can use the eBay accelerator to quickly go to auctions they are interested in and then use the Web Slices to keep track of those auctions. That sure beats having to constantly click on the refresh button in your browser to keep up-to-date with your auction!
There are many accelerators and Web Slices available at the Internet Explorer 8 gallery, where you'll also find some great articles that discuss how to create Web Slices and accelerators for your Web site.

More New Features
There are many other exciting new features in Internet Explorer 8 that enhance browsing. Here are a few:
InPrivate Filtering lets you browse where you want using Internet Explorer—and no information from your browsing session will be retained locally on your computer. You simply open a new tab and choose Start InPrivate Browsing. When you're done, you close the tab and that's it—nothing is recorded. This is great when you use a kiosk or shared computer where you do not want your history left behind. It's also particularly useful when you are shopping for gifts for a spouse or researching health information or investment strategies. InPrivate Filtering puts you in control of your browsing history.
Delete Browsing History is not new, but the feature has been enhanced in Internet Explorer 8 to allow you to retain your favorite Web site data. Now you can choose to preserve cookies and temporary Internet files for any Web site in your Favorites folder.
Enhanced Instant Search Box makes it much easier to find content of interest, and it also increases the odds that search results will be relevant. As you type a search term, you can see real-time search suggestions, including images, from your chosen search provider (see Figure 3). In addition, the enhanced Instant Search Box presents results from your Favorites and browsing history. The search will also include a quick pick menu at the bottom of the Search Suggestions, enabling you to toggle back and forth between search providers' Search Suggestions with a simple mouse.
Figure 3 Visual cues can help you zero in on what you’re looking for on the page
Enhanced Find on page is one of my favorite new features. To see it, just press Crtl+F and start typing into the box. You'll see letters and words highlighted as you type, homing in on what you're looking for.
Tab Grouping automatically collects associated tabs into color-coded groups, making it quicker and easier to keep track of different sets of content or information.

Compatibility
One of the major changes to Internet Explorer 8 is the new rendering engine based on World Wide Web Consortium (W3C) standards. This means that Internet Explorer is now designed to render Web pages built to those standards, like Firefox, Safari, and others.
The idea was not only to embrace community standards, but also to ease the burden on Web site administrators and developers. Previously, most administrators and developers had to develop and support two pages for each site—one for those following Internet Explorer standards and another for those designed according to other Web standards. Internet Explorer 8 provides support for emerging future standards such as the W3C's HTML 5 draft DOM Storage standard, and the Web Applications Working Group's Selectors API and ECMAScript 3.1 endorsed syntax. With the new engine, they need only be concerned with a single Web page, one that is built to W3C standards. It should be noted, however, that this significant change does create one obvious challenge—what about all the Web sites designed for previous versions of Internet Explorer? Clearly, such sites may not work as expected, but here's something to keep in mind when discussing this situation with customers: If pages don't work with Internet Explorer 8, they probably would not render correctly in other browsers either, unless the Web site had developed pages for those other browsers.
Happily, Internet Explorer 8 provides a solution in the form of a compatibility view toolbar button, which appears when users encounter pages that expect Internet Explorer 7-style rendering. Clicking the button reverts to IE7-style rendering, fixing any display problems.
Internet Explorer 8 also includes a feature called the Compatibility View List, which lets you choose to receive a list of major sites that are best viewed in Compatibility View. When navigating to a site on the list, Internet Explorer 8 will automatically display the site in Compatibility View. Even though controlling the rendering behavior of Internet Explorer 8 with the compatibility view button is an excellent solution for users, it is not the recommended method for compatibility. As Web site administrators and developers, you will want something that provides more control.
One way administrators can manage compatibility issues is by using some simple Group Policies to control these settings. You can also have your Web server administrator build in the compatibility view by making a simple modification to the HTTP Response Headers of your Web server by adding an x-ua compatible tag:
"X-UA-Compatible value="IE=EmulateIE7"
This metatag would affect all of the Web sites on the server.
Ultimately, developers control how these Web sites and pages render inside of Internet Explorer 8. Developers can still leverage the same browser detection logic they used in the past—user agent strings—that ensured a particular version of the page rendered as envisioned.
However, there is an easier way to control this setting: the Web developer can add a similar metatag to the site or page. The x-ua-compatible tag works on a per-site basis and on a per-page basis. So, to use it, add the following after the <head> tag:
<meta http-equiv="X-UA-Compatible" content="IE=EmulateIE7" />
The fun question is, which metatag wins if there is a conflict between the server, site, page, or user setting? In fact, when the metatag has been set for the site, the compatibility view button will not be visible. The page setting will always win in the case of a conflict. The metatag is applied in following order, and the last setting applied wins:
  1. User Compatibility View Button
  2. Group Policy Setting
  3. Server
  4. Site
  5. Page
You'll find more information on controlling compatibility at the Internet Explorer Compatibility Center.
By default, Internet Explorer 8 is in standards-based mode when browsing Internet sites and is in compatibility view mode for intranet sites. The main reason for this was to ease any compatibility issues that may arise from internal applications with Internet Explorer 8.

Security and Reliability
We are constantly at risk from the moment we plug our computers into the Web. We have many layers for defense: firewalls and antivirus tools, as well as other Internet security mechanisms. Internet Explorer 8 provides additional features that help protect users in ways that other tools do not always catch.
Attacks today are not necessarily the brute-force technical variety; more and more, they are social engineering ploys. Hackers and online criminals often succeed using phishing schemes, in which users are notified that they just won a free trip or that they need to visit their bank's site to verify account information. Instead of targeted exploits, the threat landscape is changing to blended attacks that combine Web server/Web exploits with clever social engineering schemes—and traditional tools can do little to thwart them.
Fortunately, Internet Explorer 8 has a number of enhancements that will help protect you from these kinds of attacks. Moreover, developers can use the built-in support for cross-document messaging (xdm) and cross-document requests (xdr), both of which provide for more security for Web applications communicating with other applications.
The phishing filter, first introduced in Internet Explorer 7, is now called the SmartScreen filter. It has been greatly improved with a new user interface, faster performance, new heuristics, integrated anti-malware blocking, integrated ClickJacking Prevention, and improved Group Policy support.
Perhaps the best part of the SmartScreen filter is the "teachable moment" it provides when users try to go to sites that trigger the filter. In Internet Explorer 7, when you hit a site flagged by the phishing filter, the error message was subtle and easily mistaken for a certificate error. In Internet Explorer 8, in contrast, the error message clearly indicates that the site is bad, as Figure 4 shows.
Figure 4 Internet Explorer 8 blocks an unsafe site
Another increasingly common style of attack involves cross-site scripting. In such scenarios, users are encouraged via e-mail to visit a site that is legitimate—but has been compromised. Hackers take advantage of a poorly written remote server application on the site and inject their own malicious code, which then attacks visitors. What makes these attacks particularly scary is that the hacker has not actually taken control of the Web site or hardware. As a result, the remote site may be completely unaware of what is going on.
Internet Explorer 8 counters such threats with the cross-site scripting filter, which enables the browser to detect the compromised portion of a script and prevent that portion from running. This is a solution that protects against server-side vulnerabilities and goes well beyond the traditional boundaries of browser security.
The InPrivate Filtering feature mentioned earlier also aids security. No longer do you have to remember to delete your browsing history to remove the traces of your session when using a public or shared computer. With InPrivate Browsing, your cookies, history, temporary Internet files, form data, passwords, queries, and so forth will all work during the session, but they are not left behind on the machine.
It may seem that InPrivate Filtering could allow your users to bypass your other security mechanisms like the firewall and any parental controls. However, InPrivate Filtering does not bypass those mechanisms; in fact, it will work in conjunction with them.
Although most people know that the Web sites they visit can track their activities, consumers are often not fully aware of the degree to which their Web browsing activities can be tracked by sites or content providers other than those they have consciously chosen to visit. InPrivate Filtering in Internet Explorer 8 helps alleviate concerns by providing users with greater choices and control over which third-parties can potentially track and aggregate their Web browsing activities.
It works by noting the Web sites a user visits and building a table of the unique calls for third-party content or objects that are observed for each unique domain. When this filtering is active, third-party content is blocked based on the default threshold of 10 such unique observances. (  Repeated visits to the same first-party domain do not increment this count.)
Finally, by default, Internet Explorer 8 in Windows Vista Service Pack 1 will have data execution prevention (  DEP  ) to help provide a secure and reliable browsing experience. DEP helps protect your computer from damage from viruses and other security threats by preventing certain types of code from writing to executable memory space.

More Reliable
One of the common issues in previous versions of Internet Explorer is that if you had a browsing session opened with multiple tabs and you visited a site that caused a crash, you would lose all of your tabs. That's not the case any longer. In Internet Explorer 8, tabs reside in separate processes in a feature called Loosely-Coupled Internet Explorer (  LCIE, pronounced "Lucy"    ). LCIE is an internal architecture change that separates the user interface frame from the tabs. When you first start Internet Explorer, you will typically have two processes, one for the user interface and one for the first tab. This gives Internet Explorer a method to isolate failures in the tab processes, providing much better stability.
Crash recovery is another new reliability feature in Internet Explorer 8. When a tab crashes, Internet Explorer 8 will automatically try to reload and restore that tab. If that doesn't work, the browser will stop trying to restore the Web site and provide a choice either to be redirected to the home page or to try the problematic Web site again.
What happens if the entire browser crashes? In that case, Internet Explorer 8 gives you the choice to reload your last browsing session or go to your home page. Under the hood, the browser keeps track of several aspects about the tabs, including current URI, the back/forward history, tab order, active tab, form data and session cookies. When the process does crash, a new process can be started and that information can be recovered back into the new tab.

Enterprise Support
Whenever you roll out a new application into your organization, you want to make sure the application provides robust support, security, deployment, and manageability. Internet Explorer 8 is enterprise-ready and provides several key enhancements.
Deployment Internet Explorer 8 is a Windows update package that is available through Windows Update. This provides a robust delivery mechanism for a majority of users around the world. Still, business deployments generally require a little more control, which you get with several tools.
When you deploy Windows updates today, most likely you deliver them to your users via Windows Server Update Services (   WSUS   ), Systems Management Server (  SMS  ), or System Center. Those methods still work very nicely with Internet Explorer 8 and allow you to control the timing of your deployment. However, other methods let you deploy Internet Explorer 8 preconfigured with the settings that meet your business needs.
One such method involves the Internet Explorer Administration Kit (   IEAK   ), which has been updated to support the new functionality in Internet Explorer 8. IEAK is an incredibly easy-to-use collection of tools that let you customize, deploy, and manage Internet Explorer 8. To run the IEAK, Internet Explorer 8 must be installed. You can import the settings from your local browser, so customizing your browser with the settings you want your users to have will save you time.
When you first run the IEAK, you must synchronize the version of the browser with the most current version on the Web. You will need to synchronize once per platform and language combination. The synchronization can take some time, but you can save time by importing your previous settings files (.ins). Once the synchronization is complete and you import your settings, you will not need to go through the entire wizard again.
You can create three different output packages using the IEAK customization wizard:
  • The Full option creates an MSI installer package containing the source files for Internet Explorer 8 as well as your customization settings.
  • The Configuration-only choice creates a Microsoft Installer (  MSI  ) installer package with only the customization settings. This is very useful for users who have already deployed Internet Explorer 8 as well as for maintaining the browser in your environment.
  • The CD selection creates a CD package that contains the source files for Internet Explorer 8 as well as your customization settings.
MSI installer package deployment can be achieved via a variety of vehicles including Group Policy, SMS, and System Center. You could even copy the package to a removable drive (like a USB thumb drive) and take it to the PCs you want to configure. The CD package provides a nice alternative if the other choices are not available.
Another way to deploy Internet Explorer 8 is to slipstream the Internet Explorer 8 update package into the Windows Imaging Format (WIM) file for Windows Vista or Windows Server 2008 images. WIM is a file-based disk image format that allows the customization of Windows deployments, and slipstreaming is a way to add updates into operating system deployments.
Group Policy After deployment, you can use Group Policy settings to control Internet Explorer 8 on your desktops. You can use IEAK configuration packages to set the baseline settings for the users but, in most cases, these settings can be changed by the user.
If you want more control in your organization, there are over 1,300 Group Policy settings that can give you very granular control over the browser. The compatibility view discussed earlier can be set for intranet and Internet sites via Group Policy. By default, Internet Explorer 8 renders Internet Sites in Internet Explorer 8 standards mode and intranet sites in compatibility view (   IE7 mode). You can also control which Web sites will render in compatibility view.
Security can also be set via Group Policy. For example, you can disable InPrivate mode and you can disable the option that lets users disregard a SmartScreen Filter message that a Web site is unsafe and visit the site anyway.
You can also use Group Policy to control if accelerators are available to your users and control which ones they are allowed to use. Furthermore, you can control whether users can delete their history and if they retain their favorite settings when they delete the history.

Developer Tools
The final improvements to note are those for developers. Internet Explorer 8 has a commitment to CSS 2.1; with a nod to the future, it also implements some of the most requested portions of CSS 3. Moreover, there has been enhanced AJAX development in Internet Explorer 8. Specifically, it now supports Connection events, which allow a developer to control how the application acts when the connection is lost.
There are more great tools that help the developer. One is the ability to view source by going to the page menu or right-clicking on the page and selecting view source. Now you may be thinking, who would consider a Notepad view of the HTML useful? But Internet Explorer 8 doesn't use Notepad. Instead it uses an HTML rendering engine that provides a rich and colorful UI, as you can see in Figure 5.
Figure 5 Built-in developer tools in Internet Explorer 8
Want to see another great tool? Go ahead and press the F12 key during your browsing session. Doing so brings up the built-in developer tools in Internet Explorer 8, including tools to work with the HTML, CSS, and Scripts of the Web sites. These tools allow you to make real-time changes to your Web site; the changes are temporary but they allow your developers to test modifications before committing them to production Web sites. You'll also find a built-in profiler, which allows you to watch the speed of the scripts that are running on a Web site.

Wrapping Up
Throughout this article you have learned about the enhancements in Internet Explorer 8 that provide a faster and safer browsing experience. So take some time and download your free copy on Internet Explorer 8 today.
Thanks very much to Craig Spiezle and Jane Maliouta for their contributions to this article.

Matt Hester is an IT Pro Evangelist for Microsoft in the United States. Visit his blog at blogs.techhnet.com/matthewms.


Active Directory
Export, Compare, and Synchronize Active Directory Schemas
John Policelli
At a Glance:
  • Using LDIFDE to export the schema from the source forest
  • Comparing schemas with the Active Directory DS/LDS Schema Analyzer
  • Import your schema into the target forest

Contents
Exporting the Schema from the Source Forest
Compare the Active Directory Schemas
Create an LDIF File with the Missing Elements
Import the Schema into the Target Forest
Wrapping Up
Each Active Directory forest has its own schema, which defines the objects and attributes that the directory service uses to store data. When organizations have multiple Active Directory forests, IT administrators have to manage multiple Active Directory schemas; ensuring consistency between schemas is vital when managing multiple forests. In this article, I will walk you through a streamlined process to manage multiple Active Directory schemas.
The Flexibility of Schema Synchronization and Comparison
One of the biggest advantages of the process I've detailed in this article is that it is not exclusive to Active Directory schema management. This process can also be used to synchronize schemas between various combinations of Active Directory, ADAM (Active Directory Application Mode), and AD LDS directories. The schema synchronization can be performed to synchronize schemas between the following:
  • Active Directory and Active Directory
  • ADAM and ADAM
  • AD LDS and AD LDS
  • Active Directory and ADAM
  • Active Directory and AD LDS
  • AD LDS and ADAM
This process can also be used to compare schemas bet-ween various combinations of Active Directory, ADAM, and AD LDS directories. The schema comparison can be performed to compare schemas between the following:
  • Active Directory and Active Directory
  • ADAM and ADAM
  • AD LDS and AD LDS
  • Active Directory and ADAM
  • Active Directory and AD LDS
  • AD LDS and ADAM
  • Active Directory and an LDIF file
  • ADAM and an LDIF file
  • AD LDS and an LDIF file
Organizations may deploy more than one production Active Directory forests for a variety of business or technical reasons. Often, additional Active Directory forests are deployed well after the production forest has been deployed. In some cases, this occurs years later.
Active Directory was released nearly a decade ago. Over the years, organizations have undoubtedly made numerous schema modifications to their production forest. Identifying these schema modifications to the forest is a challenging task. It's even more difficult to ensure that the schema modifications that were previously made to the production forest are made in new testing forests in a consistent manner.
In this article, I focus on a scenario in which you are deploying a new user acceptance testing ( UAT ) Active Directory forest that will be used by end users to test applications that leverage Active Directory for authentication and authorization. Five custom attributes exist in your production Active Directory schema, and you need to ensure that the schema from the source production forest is consistent with the new target UAT forest.
There are a number of scenarios, however, in which you can use the process I discuss in this article to streamline the management of multiple schemas. For more on this, see the sidebar "The Flexibility of Schema Synchronization and Comparison."

Exporting the Schema from the Source Forest
The first step is to export the schema from the source forest. This is required so that you can later compare the source forest's schema with the target forest's schema to decide which attributes and classes to synchronize.
The LDIFDE command-line tool, which ships with Windows Server 2003 and Windows Server 2008, can be used to export the schema from the source forest. This tool creates a file that is formatted with the LDAP Data Interchange Format (LDIF). No special permissions are required to export the schema from the source forest, and any domain user can perform this task.
To export the schema from the source forest, do the following:
  1. Log on to a member server or a domain controller.
  2. Open a Command Prompt window.
  3. Type the following into the Command Prompt window:
    ldifde -f PRODSchema.ldif -d CN=Schema,CN=Configuration,DC=WS08DOMAIN01,DC=local
  4. Press Enter.
Figure 1 shows the output you'll see from this command.
Figure 1 Exporting the Schema from the Source Forest
In this command, the -f PRODSchema.ldif parameter tells LDIFDE to write the output to a file called PRODSchema.ldif. The -d CN=Schema,CN=Configuration,DC=WS03DOMAIN01,DC=local parameter tells LDIFDE to use the schema partition as the root of the LDAP search. The DC=WS08DOMAIN01,DC=local part of the command must be replaced with the distinguished name of the forest root domain in your source forest.

Figure 2 Load target schema window
Compare the Active Directory Schemas
Now that you have exported the schema from the source forest, you are ready to compare this schema with that in the target forest. This step allows you to identify any attributes and classes that exist in the source forest but do not exist in the target forest.
Windows Server 2008 includes the AD DS/LDS Schema Analyzer tool when the Active Directory Lightweight Directory Services server role is installed. It can be used to compare schemas in a number of different ways. Note that this tool was previously called the AD Schema Analyzer in Windows Server 2003. In this article, I refer to it as the AD DS/LDS Schema Analyzer since my examples are using Windows Server 2008. The steps in the comparison and export, however, can also be performed using the Windows Server 2003 version of this tool.
To compare the Active Directory schemas of the source and target forests, do the following:
  1. Log on to a member server or a domain controller that has AD LDS installed and belongs to a domain in the target forest.
  2. Find the PRODSchema.ldif file that was created in the previous section and copy it to the server you log on to.
  3. Go to Start, click Run, and type the following: C:\WINDOWS\ADAM\ADSchemaAnalyzer.exe
  4. Hit Enter and the AD DS/LDS Schema Analyzer will open.
  5. On the File menu of the AD DS/LDS Schema Analyzer window, click Load target schema.
  6. In the Load target schema window, shown in Figure 2, click the Load LDIF button.
  7. Browse to the location of the LDIF file and click Open.
  8. The LDIF file will be imported into the AD DS/LDS Schema Analyzer.
  9. On the File menu, click Load base schema.
  10. In the Load base schema window, enter a domain controller to connect to in the Server[:port] field, a username, a password, and a domain, as shown in Figure 3.
  11. Click Ok.
  12. To filter for the non-present elements, select Hide present elements from the Schema menu. The missing elements will be listed under the Attributes node, as shown in Figure 5.
  13. Expand the Attributes node and the present and non-present elements (attributes and classes) will be listed, by default. The attributes that are consistent between forests appear with a checkmark in the box beside the element name, as shown in Figure 4. The elements that exist in the source forest, but are missing from the target forest appear with an empty box.
Figure 3 Load base schema window
Figure 4 Present and missing attributes
Figure 5 Viewing only the missing elements
Figure 6 Mark non-present elements you want to include

Dealing with Domain-Relative SIDs in Schema Class Objects
If the schema in the source forest was prepared for Read-Only Domain Cont-rollers (RODCs), by running the adprep /rodcprep command, you might need to perform an additional task after you complete the import the schema into the target forest. When the adprep /rodcprep command is run, the following three schema classes may have a domain-relative SID in their default security descriptor:
  • Domain-DNS
  • DMD
  • SAM-Domain
The default security descriptor for these three schema classes includes the Enterprise Read-Only Domain Controllers security group, which may contain the domain-relative SID. The SID for this security group is in the form of <domain SID>-498. For example, if the domain SID is S-1-5-21-886666173-4210462831-1041481479, the SID for the Enterprise Read-Only Domain Controllers security group will be S-1-5-21-886666173-4210462831-1041481479-498. As you would expect, a domain-relative SID from the source forest will not be usable in a security descriptor in the target forest.
The steps required to address this issue vary depending on your forest functional level (FFL) of your source and target forests:
  • If the target forest has an FFL of Windows Server 2008 or later, you can replace the offending SID with "RO".
  • If the source forest has an FFL of Windows Server 2008 and the target forest has an FFL earlier than Windows Server 2008 (for example, Windows Server 2003), you need to replace the "RO" in the SDDL with the SID of the target forest's Enterprise Read-Only Domain Controllers security group.
Only Windows Server 2008 domain controllers understand the meaning of "RO"; Windows Server 2003 and Windows 2000 Server domain controllers do not. Therefore, if all domain contro-llers in the target forest have Windows Server 2008 installed, and you do not plan to introduce any domain controllers that have Windows Server 2003 or Windows 2000 Server installed in the future, you should aim to use "RO" in the default security descriptor of a schema object. If the target forest includes domain controllers that have Windows Server 2003 or Windows 2000 Server installed, or such domain controllers may be introduced in the future, you must use the SID of the Enterprise Read-Only Domain Controllers security group, which will include the domain-relative SID.
Create an LDIF File with the Missing Elements
Now you have completed a comparison of the Active Directory schemas and identified the elements (classes and attributes) that exist in the source forest but do not exist in the target forest. You now have to create another LDIF file that will contain these missing elements. This new LDIF file will be used to import the missing elements into the target schema.
You can use the AD DS/LDS Schema Analyzer to create an LDIF that contains the missing elements, by doing the following:
  1. To include all missing elements in the LDIF file, on the Schema menu in the AD DS/LDS Schema Analyzer window, click Mark all non-present elements as included, and then click OK on the confirmation. To control which missing elements are included in the LDIF file, click the box beside each element you want to include. A plus (+) sign will be added beside the element, as shown in Figure 6.
  2. On the File menu in the AD DS/LDS Schema Analyzer, click Create LDIF file.
  3. In the Select LDIF file window, enter a location and filename for the LDIF file and click Save.

Import the Schema into the Target Forest
The final step in this process is to import the Active Directory schema into the target forest. The LDIFDE tool can be used to import the missing elements from the source forest into the target forest. As previously mentioned, the missing elements are contained in the LDIF file that was just created by the AD DS/LDS Schema Analyzer.
To import the Active Directory schema into the target forest, use an account that is a member of the Enterprise Admins and Schema Admins groups to perform the following tasks:
  1. Log on to the domain controller that holds the schema Master Operations Master role.
  2. Open a Command Prompt window.
  3. In the Command Prompt window, type the following:
    ldifde -i -f MissingElements.ldf -c dc=X
DC=WS08DOMAIN02,DC=net
  4. Hit Enter.
In this command, the -i parameter tells LDIFDE to perform an import. The -f MissingElements.ldf parameter tells LDIFDE to import from a file called MissingElements.ldf. The -c dc=X DC=WS08DOMAIN02,DC=net parameter tells LDIFDE to replace all instances of dc=X with DC=WS08DOMAIN02, DC=net. The DC=WS08DOMAIN02,DC=net part of the command must be replaced with the distinguished name of the forest root domain in your target forest.

Wrapping Up
At this point, the schema in the target forest has been extended to include the missing elements.
Active Directory schema management is a complex task. And it becomes even more complex when multiple Active Directory forests are deployed in your environment. However, by using the process explained in this article, you can streamline the management of multiple Active Directory schemas and ensure you have a consistent schema across all your forests.

John Policelli (Microsoft MVP for Directory Services, MCTS, MCSA, ITSM, iNet+, Network+, and A+) is a solutions-focused IT consultant with over a decade of experience in architecture, security, strategic planning, and disaster recovery planning. John has spent the past 9 years focused on Identity and Access Management and providing thought leadership for some of the largest installations of Active Directory in Canada. You can reach him via his blog at policelli.com/blog.


Office Communications Server
Stay Connected with Office Communicator Web Access
Greg Stemp and Jean Ross
At a Glance:
  • How Communicator Web Access extends Office Communicator
  • Communicator Web Access capabilities
  • Overview of Communicator Web Access setup and requirements
  • Adding the Appear Offline status to Communicator Web Access

Be forewarned: the story we are about to tell you is among the saddest ever written. But it’s one that needs to be told. When one of the authors turned 7, his parents decided to throw him a birthday party. Before we go any further, we need to point out that this particular author never gets sick; in all honesty, he hasn't had so much as an upset stomach since 1999.
Headaches? OK, sure, he gets those on a daily basis. But, then, he does work for Microsoft.
On the day of his birthday party, at age 7, however, our poor little author woke up deathly ill. Because the birthday party was scheduled for the morning, there was no time to cancel; all the kids came over anyway. Even worse, they played games, they ate hot dogs, and they ate birthday cake. Meanwhile, the author lay upstairs in his bed, too sick to come down even to watch the festivities. By the time he felt well enough to get out of bed, the party was over, the cake had been eaten, and some of the presents had even been opened (so that the party goers would have something to play with).
There, there; it's OK. Dry your eyes, blow your nose, and then we'll continue.
If you are familiar with Office Communications Server, you might know exactly how this author felt on his birthday all those many years ago.
How many years ago? Well, he turned 7 back then so … 16 (or thereabouts).
As you probably know, Office Communications Server is cool, in large part because of Office Communicator, Microsoft's offering for instant messaging and "presence." ( Presence is the ability to let people know whether you're online and, if so, whether you're available for conversation. ) Now, we're not saying that Office Communicator is the greatest invention in the history of the world, but—wait, scratch that. We are saying that Office Communicator is the greatest invention in the history of the world. It's that cool.
But there's a catch here.
What kind of catch? Well, suppose you're a sales person, and you're on the road, staying in a hotel room in Des Moines, Iowa. Can you use Office Communicator from your room? Sure you can … as long as you can establish a virtual private network ( VPN ) connection from that room. Or suppose you're doing a site visit in Outer Mickmacklestan and you'd like to log on from the Internet café down the street. Could you use Office Communicator from there? Sure you can … as long as that Internet café has Office Communicator installed, and as long as that Internet café is connected to your internal corporate network. We're not sure what the odds are of that, but we're guessing they're not great.
Or consider this scenario: back on your seventh birthday your parents gave you a copy of Linux. (Gee … thanks, Mom and Dad.) Can you use Office Communicator if you're running Linux? Do you even need to ask?
The fact of the matter is that, as cool a product as Office Communicator might be, it's simply not for everyone. If you don't have access to your corporate network, you can't use Office Communicator. If you're running on Linux, you can't use Office Communicator. If your company has decided to "lock down" computers and limit you to a Web browser and a few choice in-house applications, well, you can't use Office Communicator. Instead, you're forced to lie upstairs in bed, listening to all the other kids exchanging instant messages, checking each other's presence, and eating all the birthday cake that ships free with every copy of Office Communicator. (Well, maybe not every copy.) And there's absolutely nothing anyone can do about it.
Now, now, don't cry; you'll rust yourself again. (In case you're wondering, that's a clever reference to "The Wizard of Oz.") Tell you what, we can let you in on a little secret: maybe something can be done about this. In fact, it's time we tell you about Microsoft Office Communicator Web Access 2007 R2.
So what is Microsoft Office Communicator Web Access 2007 R2, other than a product that, from now on, we'll refer to simply as Communicator Web Access? (Yes, that's still a mouthful, but our first choice—Microsoft Bob—has already been used.) In a nutshell, Communicator Web Access is a nifty little piece of software that provides you with many of the capabilities found in Office Communicator, including these:
  • The ability to send and receive instant messages. Even better, this capability isn't limited to exchanging text messages with people in your organization; you can also exchange instant messages with anyone who has an account on a public instant messaging service such as Yahoo!, AOL, or Windows Live. As a matter of fact, you can even—using a little sleight-of-hand that we'll discuss in a minute—exchange instant messages with people who have no instant messaging account or instant message software at all. Is this all magic?
  • The ability to set your presence (Available; Busy; Do Not Disturb; Be Right Back; Away) as well as the ability to check the presence of others. Is there anything worse than sending an instant message to someone and then not getting a reply because that person is offline or in an all-day training session? ( War, pestilence, famine, and death? Well, sure, those are bad, but we're not convinced that they're worse.) You won't have this problem with Communicator Web Access: Communicator Web Access can tell you, at a glance, whether people are online and whether they are available to communicate with you. ( This could be by instant message, or it could be by phone: Communicator Web Access can even set up a phone conference for you.)
  • The ability to "share your desktop." What does that mean? It means you can configure things so that everyone involved in an instant messaging session (and as many as 32 people can be involved in an instant messaging conversation) can view everything that takes place on your computer. Want to show everyone a PowerPoint slide show? That's fine: share your desktop, start the slide show, and everyone will be able to view that show on your screen. And it doesn't matter if these people are from Outer Mickmacklestan and don't have Microsoft Office installed; after all, the slide show is running on your computer. Everyone else is just sitting there watching.
  • Well, unless you want to them to actually take control of your computer. With Communicator Web Access, you can allow others to use their mouse and their keyboard to run programs and enter data on your computer.
So is there a catch here, too? Of course there is; there's always a catch. As it turns out, you get these capabilities only if you 1) have a supported Web browser and, 2) have an Internet connection.
Believe it or not, that's it—that pair of requirements is the only catch. Are you running on Microsoft Windows? That's fine—as long as you have Internet Explorer 6.0 Service Pack 2 or later you can use Communicator Web Access. Are you running on Macintosh OS 10? No problem; just make sure you're using Safari 1.3 (or later) or Firefox 3.0 or later. Red Hat Linux? It's Firefox 3.0 to the rescue again. The list goes on.
But what about that hotel room in Des Moines, Iowa or that Internet café in Outer Mickmacklestan? No problem: like we said, all you need is an Internet connection to use Communicator Web Access. Connect to the Internet, fire up your Web browser, and you can use Communicator Web Access. You'll be able to send instant messages, check the presence of your contacts, share your desktop, and so on.
Figure 1 Contacts visible in Communicator
Oh, and here's a bonus. As we implied, both Office Communicator and Communicator Web Access let you set up contacts. Contacts are simply people you tend to communicate with regularly. When you start Office Communicator or Communicator Web Access, your contacts—and their presence—are immediately available, as shown in Figure 1.
You see just two contacts, one being his coauthor and the other being himself? It appears that after that birthday party, life simply has not been kind to one of our authors.
As you can see, contacts and your contact list enable you to tell at a glance who's online and who isn't. They also make it easy to send instant messages to people: just right-click the contact name and then click Send an Instant Message.
Now that's cool, but that isn't the bonus. The bonus is this: Office Communicator and Communicator Web Access share the same contact list. Back at the home office, did you dutifully use Office Communicator to create a list of contacts? Good for you. Now that you're on the road, that contact list will be available to you when you run Communicator Web Access. And vice versa.
See? We told you this was cool.
Of course, you might be thinking, "Gee, I don't know, I mean, Outer Mickmacklestan? Wouldn't we run into security issues logging on from Outer Mickmacklestan?"
Believe it or not, the answer is no. After all, you can configure Communicator Web Access to run under HTTPS and SSL; do that, and all your instant messaging sessions will be encrypted and secure.
Then again, based on the acronyms people often use, we'd argue that many instant messages are cryptic enough as it is. IBRB? I'll be right back. 2G2BT? Too good to be true. And, of course, the immortal BJJDI: Billy Joel just drove in. As near as we can tell, nobody in the world has any idea what that last one means. Probably some sort of Outer Mickmacklestanian expression.
It makes sense that you can use Communicator Web Access to exchange instant messages with people in your own organization; it even makes sense that you can use Communicator Web Access to exchange instant messages with people who have accounts with a public instant messaging service such as Windows Live. But how in the world can you exchange instant messages with people who don't have an instant message account of any kind?
As we said earlier, there's a little bit of trickery involved here. You can't randomly send an instant message to just anyone. You can't normally address an instant message to someone unless that person has a Session Initiation Protocol (SIP) account somewhere; addressing instant messages to someone without a SIP account would be like trying to mail a letter to a person without a mailing address. Even so, you can still have an instant messaging conversation with someone who does not have a SIP account as long as you initiate the session, and do the following:
  1. Log on to Communicator Web Access and then click the Meet Now button. This action opens a Conversation window that features you as the only participant. (Those of you who prefer to talk only to yourselves can quit at this point. )
  2. In the Conversation window, click the arrow next to the Invite button and then click Invite Using E-mail. When you do that, Communicator Web Access generates a message similar to this one:
    Greg Stemp has invited you to a conversation using Microsoft Office Communicator Web Access. To join the conversation, click the following link:
    imdf.microsoft.com/Join?uri=sip%3Agstemp%40microsoft.com%3Bgruu%3Bopaque%3Dapp%3Aconf%3Afocus%3Aid%3AKK75ASPD6VCW6VA92NCMWB70641P7PGX&key=29FUUQUS9TED
  3. Copy that URL, paste it into an e-mail message, and send the message to anyone who can run Communicator Web Access. That person can paste the URL into her Web browser and be taken to a Communicator Web Access "conference join" page. From there, she can join your instant messaging session, even though she doesn't actually have an instant messaging or other kind of SIP account.
Of course, you might be thinking, "Yes, that is nice. But there has to be a real catch here. For example, why haven't they told us how difficult it is to set up and configure this Communicator Web Access thing?"
Actually, there's a good reason: it's not the least bit difficult to set up and configure this Communicator Web Access thing. To begin, you need to have Office Communications Server 2007 R2 running somewhere in your Active Directory forest. (But that should come as no surprise: if you want anything good in life you need to have Office Communications Server 2007 R2 running somewhere in your Active Directory forest.) You'll also need a dedicated computer running the 64-bit version of either Windows Server 2003 or Windows Server 2008.
In case you're wondering, a single Communicator Web Access server can handle several thousand simultaneous connections. ( The exact number will vary depending on a host of factors that we won't go into today.) What if you have more users than that? No problem: just get yourself a hardware load balancer and set up an entire array of Communicator Web Access servers, as many as you think you might need to handle all your users.
From there, you simply need to install and activate Communicator Web Access, a process that will—literally—take only a few minutes. ( Definitely less than half an hour.) At that point, it's time to create a virtual server or two.
We won't get too technical here: virtual servers in Internet Information Services (the kind of virtual server we're talking about here) are simply the Web sites people use to log on to Communicator Web Access. Creating a virtual server isn't hard; a wizard walks you through the process and creates all the pages for you. ( No, it's not a real wizard, just one of those software things.) However, you will have to put a tiny bit of thought into the matter.
And, yes, we know: no one told you there would be thinking involved, did they? At least not until now.
So what must you think about? For one thing, you'll need to decide which connectivity protocol you want the site to use: HTTPS or the less-secure HTTP. (Hint: this shouldn't be a real hard one to decide.) You'll also need to pick an authentication mechanism. Communicator Web Access supports built-in authentication methods such as integrated Windows password authentication and forms-based authentication. (In the former, you can log on with the same credentials you used when you logged on to your computer. In the latter, you need to supply a user name and password to log on.) You can also use custom authentication methods. (For example, you can require a smart card and PIN number to log on.) If you're using Outlook Web Access and Microsoft Internet Security and Acceleration Server (ISA) 2006, you can even use single sign-on authentication: log on to Outlook Web Access, and you'll automatically be logged on to Communicator Web Access as well.
You'll also need to determine whether each virtual server you create is intended for internal users (users logging on from behind the organization's firewall) or for external users (users logging on from outside the organization's firewall). When it comes to authentication, Communicator Web Access treats people logging on from inside the firewall a little differently from the way it treats people logging on from outside. (After the users are logged on, however, everything is the same.) For example, internal users can log on using the same credentials they used to log on to their computer, which means they can be automatically logged on to Communicator Web Access. This isn't true for external users, however; external users must always supply a user name and password.
In case you're wondering, the answer is: yes, these conditions mean that, if you have both internal and external users, you need at least two computers running Communicator Web Access: one to handle internal users, the other to take care of external users. If you want to get technical, it's possible to host both an internal virtual server and an external virtual server on a single computer. For security reasons, however, this is highly discouraged.
Figure 2 You can set your status to appear offline, even if you’re really online.
Admittedly, this is a somewhat cursory overview of setting up Communicator Web Access. As you might expect, you'll also have to concern yourself with matters like DNS records and certificates and Active Directory permissions. But none of those should prove too daunting, at least not to anyone with even a modest amount of experience administering a Windows domain. And, besides, all of those tasks are covered in considerable detail in the document called Deploying Communicator Web Access 2007 R2.
Before we call it a day, we need to level with you: we held back a little when we told you the story of the birthday party. As it turns out, the story was even sadder than we let on. ( We thought it would be too traumatic to tell you the whole tale all at once.) That year, our little author wanted only one thing for his birthday: he wanted the ability to set his presence in Communicator Web Access to Offline. "That would be so cool," he told his parents. "I could actually be online and working in Communicator Web Access; I could even send and receive messages, although anyone who checked my status would think I was offline. That would be the greatest present any kid could ever receive."
Sadly, our poor little tyke never got his wish. (His parents claimed it was because there was no such thing as Communicator Web Access way back then.) From that day forward, he vowed to never let that disappointment befall another child. With that in mind, here's how you—the system administrator—can help by adding the Appear Offline status to Communicator Web Access (as shown in Figure 2):
  • On the Communicator Web Access server start Regedit and locate the key HKEY_LOCAL_MACHINE\Software\Policies\Microsoft\Communicator.
  • Create a new DWORD value named EnableAppearOffline and set the value to 1.
That's it. Any users logging on to this Communicator Web Access server will now have an additional status option:
Mom, Dad? If you're reading this, would that really have been all that difficult to take care of?
As long as we're on the subject, Communicator Web Access offers a number of ways that administrators can customize the software. For example, without needing to write a single line of code or know any programming language, an administrator can:
  • Create a custom Communicator Web Access sign-on page
  • Add a set of custom tabs to the Communicator Web Access client window
  • Add a custom menu to the Communicator Web Access client menu
  • Display a user notice in the instant messaging conversation window
Pretty cool, huh? Just for the heck of it, here's a simple little VBScript script that will cause a notice ( "Instant messaging provided by Fabrikam, Inc." ) to appear in the conversation window each time one of your users starts a new instant messaging session:
strComputer = "."

Set objWMIService = GetObject("winmgmts:\\" & _
    strComputer & _
    "\root\default\rtccwa_repository")

Set colWMISettings = objWMIService.ExecQuery _
    ("Select * from MSFT_CWASiteSetting")

For Each objWMISetting in colWMISettings
    objWMISetting.UserNotice = & _
        "Instant messaging provided by Fabrikam, Inc."
    objWMISetting.Put_
Next
That's all we have time for today. As you can imagine, articles need to be written well in advance of publication. As this particular one is being written, it's December, which means, believe it or not, it's almost time for our author to have yet another birthday (December 18th, for those of you planning to send him a present next time around). Will he finally overcome his years of trauma and have a birthday celebration this year? Let's put it this way: it'll be a cold day in Outer Mickmacklestan before this author has another birthday party.

Against all odds, Greg Stemp and Jean Ross work for Microsoft.


Office Communications Server
How Voice Conferencing Powers OCS 2007 R2
Rajesh Ramanathan
At a Glance:
  • Ad hoc conferencing
  • Centralized Conference Control Protocol
  • Multimodal conferences

Contents
Types of Meetings
Basic Conferencing Architecture
The C3P Protocol
How Conferences Are Created
Inviting Others to the Conference
Escalating a Two-Party Call to a Conference
In a previous article in this series, "How Voice Powers OCS 2007," I discussed how voice calls work in the Office Communications Server (OCS) system. I also explained how a basic SIP INVITE is routed in the system to set up a peer to peer voice session and how clients traverse NAT and Firewalls for call establishment.
In this article, I will explore the conferencing part of the story. OCS allows real-time conferences to be set up with users inside and outside a corporate firewall, supporting both ad hoc escalation of calls to a conference and pre-scheduled conferences or meetings.
Conferencing in OCS 2007 is built on its Session Initiation Protocol (SIP) support and leverages the NAT/Firewall traversal capabilities introduced for peer to peer calls. Like peer to peer calls, conferences can be joined from anywhere outside the corporate firewall. Conferencing in OCS relies on the OCS 2007 Conferencing server roles. With dedicated servers that provide the conferencing features, conferences can scale from a few members to a hundred or even more.
Figure 1 Invite options in Office Communicator 2007 R2
Office Communicator clients provide the ability to escalate an audio or audio/video call, an instant message session, or a multimodal call and instant message session to a conference, by seamlessly escalating all modes together. Users can add additional attendees to two-party audio calls by dragging and dropping other users from the Contact list in Office Communicator or by using the Invite menu (see Figure 1) in a conversation window, which brings up a contact picker where the user can enter a phone number into the invitation.
The Invite by E-mail option, which is also available from the Invite menu, creates an e-mail message using Microsoft Office Outlook and sends the conference URI to the remote user (more on the conference URI in a moment). Invite by E-mail also sends the dial-in number in the message if the conference is scheduled using the Conferencing Add-In for Microsoft Office Outlook. When Invite by E-mail is used, the remote participant can join the conference using Communicator Web Access, authenticate (or join anonymously), and select to dial out to the local phone number. This provides a flexible solution.
Office Communicator 2007 R2 also provides the Meet Now option (shown in Figure 2) for an easy way to quickly create ad hoc conferences.
Figure 2 Meet Now in Office Communicator 2007 R2
Types of Meetings
Before I delve into the details of how conferencing works, I should discuss the types of conferences OCS supports. Conferences can be created in Office Communications Server 2007 R2 with the following types:
Open Authenticated Users authenticated against OCS 2007 can join the conference and invite other authenticated users. The word "open" signifies that membership is open. Ad hoc conferences created by Office Communicator 2007 R2 are open authenticated by default.
Closed Authenticated Authenticated users can join the conference, but the membership to the conference is restricted and controlled by the organizer.
Anonymous These conferences have the loosest restrictions on who is allowed to join. Users are able to join from Communicator Web Access without authenticating against OCS, or they can use a dial-in number to join a conference from a phone.
In this article, I am focusing primarily on open authenticated conferences.

Basic Conferencing Architecture
Conferencing in the OCS architecture is based on a star topology where all the clients connect to a central conferencing server pool. The conferencing solution in OCS typically has two major server types that participate in the conference.
First off, there is a focus factory, which acts as a conference manager. This manages the participant list in the conference and the modalities that various participants are currently using. The focus factory also terminates the signaling control connection with the conference leader and ensures that commands (such as mute, eject, and so on) are properly channeled to the correct media servers. The focus factory also maintains the connection to the Conferencing database, which is used for looking up scheduled conferences and dial-in numbers.
Secondly, there are one or more multipoint control units (MCUs). The MCU provides media multiplexing capabilities for a conference. In the OCS system, there are the following MCU roles:
  • IM MCU provides Instant Messaging between multiple parties.
  • Audio/Video MCU provides audio mixing and video switching between multiple parties.
  • Data MCU provides Live Meeting features, such as desktop sharing, whiteboarding, and so on.
Since the focus factory is the conference manager, all the clients are given an address for the focus factory when they are provisioned. This address is sent through the inband provisioning mechanism I mentioned in the article "How Presence Powers OCS 2007."
Figure 3 shows the logical architecture for a conference that has three Office Communicator clients for three users. The dotted lines represent the SIP-based signaling channel that is established between the clients and the focus, which in turn establishes the channel with the various MCUs. The solid lines show the audio/video stream that is terminated on the Audio/Video MCU. Other media streams, such as IM and data, are intentionally omitted from this diagram for the purpose of simplicity.
Figure 3 Logical architecture of a three-party conference

The C3P Protocol
The OCS conferencing solution is based off of Centralized Conference Control Protocol (commonly referred to as CCCP or C3P for short). This is an XML-based client-server protocol that piggy backs on a SIP and provides the following mechanisms:
  • A conference document (or roster) that lists the participants in the conference and the various modes in which the various participants are currently in.
  • A command/response mechanism that allows clients to issue commands to the conferencing server (focus factory) so they can create the conference or control other aspects of it.
For instance, AddConference is a C3P command that is used to add a conference to the focus factory. The focus factory responds with a unique conference SIP URI, which is based off the user's own SIP URI. For example, say my SIP URI is sip:rajesh@contoso.com. When the client issues an AddConference command, the focus factory would return a unique key to the conference that looks something like this: sip:rajesh@contoso.com;gruu;opaque=app:conf:focus:id:A0DB798E3EDA984FACAD30D1A8DCD35A. This SIP URI key uniquely identifies the OCS conference. It can be shared with other participants to give them access to the conference. This is the same URI that is sent in the message generated by using the Invite by E-mail option.
Since the conference's SIP URI is created using the conference creator's own SIP URI, this ensures that the policies applied to the conference can be derived from the creator's policies. It also means that policies related to dial out to PSTN and the like can be applied based on what is allowed for the specific conference leader.
Another C3P command, AddUser, is a command that adds a participant into the conference. It also specifies the role of the participant, such as attendee or presenter. The leader/presenter has to add himself to the conference using AddUser as soon as the AddConference command is issued. The leader/presenter can use AddUser to invite endpoints or clients with a SIP URI into a conference, as well as PSTN phone numbers. To initiate dialing out to phone numbers from the MCUs, a <dialout> XML node has to be set in the AddUser command. I will refer to this combined command as AddUserDialOut.
A third C3P command, GetConference, is used to retrieve all conference capabilities. Once a client connects to the focus, it needs to retrieve the SIP URI information about the various MCUs in the system so it can talk directly to the MCUs. This information about the MCUs is retrieved using GetConference. An Audio/Video MCU SIP URI that is retrieved using GetConference looks something like this: sip:rajeshra@microsoft.com;gruu;opaque=app:conf:audio-video:id:A0DB798E3EDA984FACAD30D1A8DCD35A. Note that each of the SIP URIs— whether a conference focus factory or a specific conferencing server—is actually a Globally Routable User URI (GRUU). I briefly talked about GRUUs in the "How Presence Powers OCS 2007" article.
As I mentioned before, C3P rides on top of the SIP, and the SIP allows sessions to be created between any two user agents (or, to be syntactically correct, between a user agent client and a user agent server). The payload of a SIP session need not always be an audio or video SDP (Session Description Protocol); it can be a way to establish a pure signaling channel. This concept is used by clients to establish a SIP-based signaling channel with the focus factory to a particular conference session.

Figure 4 Creating and joining a conference
How Conferences Are Created
The first step is to create a conference and establish a SIP signaling session with the focus factory. The first command a client uses is to create a conference session in the focus factory so that the client can begin interacting with the conference focus. This is done using a special SIP request called SERVICE. The SERVICE request/200 OK pair carry the AddConference command and the AddConference response, respectively. Once the SERVICE/200 OK step is completed, the client has obtained a unique conference ID that it can use to talk to the focus to obtain information related to the MCUs, add other users, and manage the conference membership.
The client then adds itself to the conference by sending an INVITE to the conference URI that contains a C3P AddUser command specifying the client's own session. This completes the initial step of creating the conference and joining the focus.
The next step is for the client to join the media on the various MCUs. For example, if the conference will be an audio call, then the Office Communicator client issues a regular SIP INVITE with an audio or audio/video SDP body.
The last step is for the client to invite other clients to participate in the conference. Figure 4 illustrates this sequence of events used to initialize a conference in the focus factory.
Note that once the SERVICE request is made, the first <addUser> command is carried as an INVITE payload instead of an SDP. Once this dialog is created, other commands, such as adding new users, are sent as SIP INFO messages over the same INVITE dialog with the focus. The first C3P command that is sent to the focus factory is the GetConference command, which returns the MCU SIP URIs I've already mentioned. The client now sets up a second media session (this time with the audio SDP) to the SIP URI of the A/V MCU indicated in the getConference response.

Links to Previous Parts of this Series
"How Presence Powers OCS 2007"
technet.microsoft.com/en-us/magazine/2008.02.ocsim.aspx
"How Voice Powers OCS 2007"
technet.microsoft.com/en-us/magazine/2008.07.ocs.aspx
"How Remote Call Control Powers OCS 2007 R2"
technet.microsoft.com/en-us/magazine/dd440963.aspx
Inviting Others to the Conference
Now that you are familiar with how a client creates a conference from the focus factory and joins the A/V MCU, I can detail how the client can invite other clients into the conference. There are three basic methods that can be used here: an ad hoc application invite sent to the remote client using SIP, a dial-out request sent to the focus factory to create a VoIP INVITE from the A/V MCU, or joining from the conference URI (which is part of a scheduled meeting).
When a contact is selected, Office Communicator attempts to send an Application-INVITE (or App-INVITE) to the remote client to invite it into a conference instead of directly initiating a dial-out request from the A/V MCU. An App-INVITE is a special SIP INVITE that contains an XML data payload containing the conference's SIP URI. A client that receives the App-INVITE is able to join the conference based on the focus-URI in the App-INVITE. The App-INVITE is always the preferred way to invite another client because it contains information about other potential modalities, such as Instant Messaging in the conference.
When a user selects a specific phone number for a contact, or when the user selects Invite by Phone, Office Communicator instead issues an AddUserDialout command to the A/V MCU. A dial-out INVITE is a simple VoIP INVITE originating from the A/V MCU that allows downlevel clients and PSTN endpoints to join an Audio/Video conference.
Creating a 3 Person Conference
Based on the information I've presented so far, I'll detail how Alice is able to create a conference with Bob and Carol directly. Alice chooses to right-click on Bob and Carol in Office Communicator's contact list and start a conference from Communicator. Figure 5 shows that Alice's Office Communicator client first creates the conference and joins it using the constructs I have discussed.
Figure 5 How a conference is created
Media is now flowing from Alice's instance of Office Communicator to the AV MCUs. Next, Alice's Office Communicator fires separate App-INVITEs to Bob's SIP URI and Carol's SIP URI. Bob happens to be at work and his Office Communicator phone rings. When Bob accepts the conference, his Office Communicator client sends 200 OK to the App-INVITE and sends a BYE to the session immediately since it has all the information from the App-INVITE body and does not need to keep the virtual session with Alice's Office Communicator client alive. Bob's Office Communicator then joins the focus factory and the A/V MCU in the same way Alice's Office Communicator first joined the conference.
Carol also receives the App-INVITE and Office Communicator rings for her, as well. Carol decides to divert the conference call to her mobile phone from the incoming call toast. When Carol selects this option, the Office Communicator client joins the focus based on the focus-URI and then issues an AddUserDialout C3P command to dial out to Carol's mobile phone rather than joining directly.

Escalating a Two-Party Call to a Conference
Escalating a two-party conference is a bit more involved than simply starting a conference. This is because the two-party call must be maintained until both parties switch over to the conference. Office Communicator clients conduct a synchronized escalation process where each client joins the conference before terminating the peer to peer call.
Say, Alice is talking to Bob and decides to drag and drop Carol into the conference. When escalating the two parties into a conference, the following steps occur:
  1. Alice's Office Communicator client creates a conference session based on the focus factory.
  2. Then Alice's Office Communicator joins the A/V MCU with the call on hold (RTP stream inactive).
  3. Once the conference join is successful, Alice's Office Communicator sends an App-INVITE to Bob's client specifying the conference URI.
  4. Bob's Office Communicator begins the escalation step and joins the focus and the A/V MCU with the RTP stream on hold.
  5. Once Bob's Office Communicator joins the A/V MCU successfully, it sends a BYE to the peer session .
  6. Both Alice's client and Bob's client activate the RTP stream to the A/V MCU at the same time to maintain continuity of the call.
  7. Now Alice's client sends an App-INVITE to Carol's Office Communicator client to invite the third party into the conference.
Note that this sequence of steps is very carefully executed. If Bob's Office Communicator client is unable to join the conferencing server, then the conference escalation fails and the peer to peer calls continue.
Escalating PSTN calls to a conference is also supported, but instead of sending the App-INVITE in step 3, Office Communicator would issue an AddUserDialOut to the A/V MCU and add a Replaces header so that the call from the A/V MCU can replace the peer to peer call between the two Office Communicator endpoints. This is illustrated in Figure 6 and Figure 7.
Figure 6 Escalating PSTN Calls to a Conference—In Process
Figure 7 Escalating PSTN Calls to a Conference—Final State

Office Communicator supports interoperating with other clients that do not support the App-INVITE mechanism. For this, it uses a fallback mechanism to using AddUserDialout from the A/V MCU. In the scenario above, if Carol was on a client that didn't support App-INVITE, a 415 response code to the App-INVITE would trigger Office Communicator to fall back to the A/V MCU dial out.
I've described an audio call between two people being escalated to a conference. But what would happen if there was instant messaging in the session? When there is more than one modality in a conference, Office Communicator clients ensure that both the modalities are successfully escalated before the conference is committed and the third party invited to the conference. This ensures that the multimodal experience between the two participants is maintained.

Rajesh Ramanathan has worked in the communications space for 15 years, and has designed voice protocols, user experiences, and more recently the Communicator voice and Conferencing experience for Office Communicator 2007 and R2. He currently works as Lead Program Manager on the Office Communicator team. He can be reached for comments at rajeshra@microsoft.com


Windows HPC Server 2008
High Performance Computing in the Real World
Joshua M. Kunken
At a Glance:
  • Setting up and managing the compute cluster
  • Submitting jobs to the HPC scheduler using Windows PowerShell
  • Processing spreadsheet data on the server
  • Making processed data available to end users

Contents
Why HPC 2008?
Hit the Floor Running
Accessing the Compute Cluster
Administration
Taking Advantage of Windows PowerShell
Application Development
Server/cluster-side Processing of Spreadsheets
Client-side Access to Processed Data
A Very Large Dataset
Windows HPC Server 2008 (HPC 2008), the latest Microsoft parallel computing package, was specifically designed for use in compute-intensive environments, which exactly describes our research laboratory. This article describes the deployment of a wide variety of Microsoft technologies, in particular HPC 2008, in the development of a custom system that enables the automated processing of data generated from an imaging device in a life science laboratory.
The system is capable of performing unattended and automated Excel spreadsheet data analysis, including nonlinear regression. Where such analysis would typically require a research scientist to spend 10 to 15 minutes per experiment, it now takes less than half an hour to process 96 experiments at a time (without human intervention—and error). It takes even less time if a computing cluster is used.
Our solution also uses a small set of Windows PowerShell scripts, deployed on the HPC 2008 compute cluster, that process en masse hundreds of gigabytes worth of XML files within a very short time frame. The aim is to illustrate how you can leverage Microsoft High Performance Compute Cluster 2008 along with other Microsoft technologies to readily develop applications capable of performing high-level analysis and processing of file types routinely used in a variety of fields ranging from finance to medicine.

Why HPC 2008?
Our decision to use HPC 2008 was based on several factors. First, HPC 2008 integrates very well with our existing Active Directory infrastructure. This enables us to allow existing Active Directory users, distributed across different geographic locations, to submit jobs as well as extend their access rights across the cluster using their existing username/password pairs. There is no need for them to have multiple credentials to submit jobs across the cluster as well as log on to their workstations.
Second, lab scientists deal with Excel spreadsheets constantly to keep track of experimental results and to perform further analyses. We needed to process Excel spreadsheets computationally and it would be difficult to do so on a non-Windows platform. Third, we could execute applications developed on the Microsoft .NET Framework using C#, which made it easier to perform calculations on data extracted from Excel spreadsheets on a Windows compute cluster. Finally, we could also use such applications to easily transmit information to SQL Server database tables.

Hit the Floor Running
HPC 2008 makes it fairly easy to take off-the-shelf hardware and set up a compute cluster. With a relatively small cluster, it takes only hours to get all of the compute nodes up and running and communicating with the head (or master) node. And due to the nature of parallel computing, it takes the same amount of time to deploy HPC 2008 and Windows Server 2008 to a small set of perhaps eight compute nodes as to a set ranging in the hundreds.
After setting up our head node with Windows Server 2008 and HPC 2008, we were able to quickly join it to our Active Directory domain (running on Windows Server 2003 R2). We then created a hard disk image (containing a copy of Windows Server 2008 along with HPC 2008) based on the head node, and deployed the disk image across our many compute nodes. Using the HPC 2008 Network Configuration wizard (see Figure 1), we selected a network topology in which our head node was accessible via public IP address while the compute nodes received non-public IP addresses from the head node via the DHCP role deployed on Windows Server 2008. You can also easily deploy the many other roles such as DNS, WINS, and RAS that are relevant to setting up a compute cluster either in standalone mode or joined to a Windows domain.
Figure 1 Selecting a network topology

Accessing the Compute Cluster
Windows Server 2008 comes with a brand-new version of the Remote Desktop Connection ( RDC) application, mstsc.exe. With RDC, system administrators can connect to and control other Windows hosts ( Windows Vista clients as well as Windows Server clients) through a graphical client directly over a TCP/IP connection. Windows Vista also comes with a new version of RDC that makes it easy to access the HPC 2008 compute cluster's head node and compute nodes. When logged on to the head node, cluster administrators can add new nodes, remove nodes, deploy or cancel jobs across nodes, and perform many other administrative functions.
The HPC Cluster Manager console gives the cluster administrator, as well as those with appropriate privileges, a user-friendly interface for submitting jobs and tasks manually (see Figure 2). This is where you specify the working directory, path to executables, output folders, and advanced parameters, as well as the minimum and maximum number of CPU cores, compute nodes, node groups, and node types.
Figure 2 Specifying a job in the console

Administration
With the built-in HPC Cluster Manager, you can easily configure and administer cluster compute nodes, monitor which jobs are running on which nodes, determine which nodes are in use at capacity at any given moment, and control which job types are running on any particular node. Further, the cluscfg utility allows users to obtain at-a-glance statistics, including how many processors are in-use, how many are idle, and which compute nodes require maintenance.
The clusrun command lets you explore the number of jobs running at any given time, as well as quickly see how many compute nodes and CPUs are currently in use. You can also open the HPC Cluster Manager console to view the list of jobs running on the cluster as a whole, as well as jobs still in queue, jobs that have completed, and those that are stalling. If jobs are stalling, you can drill down on a single job or task to investigate the cause.
In our experience, the typical causes of failed jobs on the compute cluster include poorly mounted file shares or trust not set on shares across multiple compute nodes; poorly set access control list permissions; and inadequate permission settings for execution of Windows PowerShell scripts across compute nodes. Resolving these issues is relatively straightforward.
The HPC Cluster Manager console allows you to see the health of all compute nodes in a single snapshot as well as in the more visually intuitive heat map view. Here, you can quickly get a glimpse of how resources are being consumed on each node by all of the jobs submitted. Diagnostics tests can also be performed across all compute nodes at once (see Figure 3) and the results from these tests provide the cluster administrator with relevant information for resolving issues across the whole set of nodes or for individual groups. Further, HPC 2008 also provides very useful graphical displays showing compute cluster usage across a variety of time periods (Figure 4). These pie charts and key performance indicators let relevant decision makers determine whether server and IT resources are properly allocated.
Figure 3 Tests can be performed across all compute nodes at once.

Figure 4 Useful information is readily available.

Taking Advantage of Windows PowerShell
With HPC 2008, you can not only submit jobs interactively to the scheduler via the HPC Cluster Manager console, you can also do this via the command line or through Windows PowerShell, enabling users to automate jobs via the Task Scheduler, C# system calls, or Windows PowerShell calls.
Our project makes use of a primary Windows PowerShell script that queries a networked file share. The share serves as an input directory of files waiting to be processed that were deposited by the imaging system subsequent to experiments being performed. The script proceeds by checking an output directory for post-process output files. If input files have not yet been processed (that is, if output files are not found within the output directory), the script generates individual child scripts specifically for submitting individual jobs to the HPC scheduler via the command line. The main Windows PowerShell script then iteratively submits all such child scripts to the scheduler by piping them into job.exe, the HPC command-line job submission executable. The job scheduler lets you submit any job or task to the HPC compute cluster using a variety of arguments, giving you a flexible set of options for running an application on the cluster.

Application Development
Our core software development efforts relied on Visual Studio Professional 2008, which provides the ability to develop console, Windows, and Web applications. Visual Studio enables developers to design a visually appealing graphical user interface while simultaneously working on the underlying business logic within the same project.
For our project, we developed several applications based on the same core business logic to suit different needs. We started with a Windows presentation Foundation ( WPF) application for processing our input Excel spreadsheets, thus enabling our researchers to manually test different initial fitting parameters to determine whether a larger set of experiments ( based on similar fitting parameters) could be processed automatically on the HPC compute cluster.
Next, we built a console application to actually run on the HPC compute cluster; it accepts a number of arguments, including a file path to an input spreadsheet, as well as seed fitting parameters. Windows PowerShell provides the arguments to the application while simultaneously submitting the task to the HPC compute cluster via the command line. The console application, running on every individual compute node, processes the input spreadsheets while also generating output spreadsheets containing the solutions and simultaneously performing SQL Server INSERT queries to keep track of results from all experiments. The .Net Framework 3.0 System.IO.Packaging API is leveraged on every node to process the spreadsheets for their content. Once values are obtained from the input sheets, C# is used to handle the rest.
We also developed a more graphical WPF application that lets researchers view results sent to SQL Server from the HPC compute cluster. A subset of the experiments undertaken in our laboratory is performed within standard 96-well plates routinely used in structural biology, and the user interface of the WPF application reflects the physical plate configuration (see Figure 5). Users need only click on one well to retrieve results for the single experiment trial. Clicking on a well initiates SQL Server queries against several database tables to retrieve results previously processed on the HPC compute cluster and submitted to the tables. We also used a ListView control holding a GridView control to provide a high-level overview of all 96 trials for a single experiment. Researchers can obtain a sortable table displaying all fitting parameters and statistical analyses for their processed experiments. This table becomes available as soon as the HPC compute cluster finishes processing the respective spreadsheets. Finally, we made use of SSH tunnels within the application, enabling end users to securely track their experimental results from outside the lab.
Figure 5 The application’s graphical interface

HPC Resources
Windows HPC Server 2008
microsoft.com/hpc/
Windows HPC TechCenter
technet.microsoft.com/en-us/hpc/
Windows HPC Community Portal
windowshpc.net/Pages/
The Windows HPC Team Blog
blogs.technet.com/WindowsHPC/
Evaluate Windows HPC Server 2008
technet.microsoft.com/en-us/cc835241.aspx
An Introduction to Windows Compute Cluster Server
technet.microsoft.com/en-us/magazine/2008.02.ccs.aspx
Scripting for Windows HPC
microsoft.com/technet/scriptcenter/hubs/ccs.mspx
Windows HPC Server Deployment, Management, and Administration
social.microsoft.com/Forums/en-US/windowshpcitpros/threads/
Server/cluster-side Processing of Spreadsheets
Researchers in our laboratory are keen on observing changes in conditions over very brief timescales within each of the 96 wells used in the experiments. Conditional changes are recorded using digital cameras attached to microscopes. Software then converts digital imagery information to numerical intensity levels, on a temporal basis, which are then stored within Excel spreadsheets. These spreadsheets can be processed manually and the resulting data is fitted, via nonlinear regression, with carefully crafted mathematical functions.
The point of fitting the experimental data with regression curves is to determine fitting parameters, which can shed light on the experimental changes in conditions. However, the painstaking process behind calculations and curve fitting requires, at minimum, 10 minutes per well and carries the potential for human error as well as the tedium associated with cut-and-paste operations.
Within Task Scheduler (the snap-in replacing Scheduled Tasks in Windows Server 2008), we enable the server-side application to run at predefined intervals. If the main Windows PowerShell script finds unprocessed spreadsheets on the file server, it generates a job submission script that is then deployed to the job scheduler, which then calls on the server-side processing application (residing on every compute node through a shared folder) to process the unprocessed data.
SQL Server, running on a remote server, is used to organize results obtained after processing Excel spreadsheet data per each experiment (that is, per plate well ). Our scientists are interested in viewing both the raw spreadsheet output and an intuitive visual representation of experimental results in a user-friendly format, which we provide. The server-side application not only generates results spreadsheets containing the nonlinear regression parameters, associated statistics (goodness of fit and so forth), and curve plots, it also executes SQL queries storing the same information into an associated database table containing data for all experiments. This lets end users query information more efficiently (via GUI ), rather than laboriously searching through many files worth of spreadsheets.

Client-side Access to Processed Data
The graphical application we developed to facilitate end users' viewing of processed data is used to execute remote queries against the SQL Server containing processed results and to present query results to end users in a friendly format. Moreover, end users can export results from SQL Server to Excel spreadsheets directly from the GUI, allowing them to incorporate their results into publications and presentations. The ability to export results to a new Excel spreadsheet depends on results from SQL Server being bound through a DataSet data structure to a ListView control in the application. Researchers can now view experiment results from a consolidated perspective that they were not able to leverage before. Whereas it would normally take a lab member 16 hours to process a single 96-well plate (at 10 minutes per well), it now takes a fraction of that time.

A Very Large Dataset
In addition to leveraging HPC 2008 in automating the processing of Excel spreadsheets generated from our imaging system, we also use the Windows compute cluster to process queries against datasets containing tens of thousands of content-rich XML files. Our XML library consists of 300GB worth of XML files, each containing more than one hundred nodes or fields against which we can perform queries. Clearly, with such a large dataset, it becomes unfeasible to import our XML library into a single SQL Server database instance while also keeping it current.
Database performance suffers as soon as database files exceed a certain size threshold (more than 100GB). Consequently, sharing our XML library across a large number of compute nodes and distributing the sum of our queries across all compute cluster nodes results in minimal query turnaround time for such a large search space. At the same time, by storing the contents of the library on a single file server share, we keep dataset maintenance overhead to a minimum.

Joshua M. Kunken serves as system developer and administrator at The Scripps Research Institute in La Jolla, California. He earned his Bachelor's degree at the University of California, Berkeley. You can reach Josh at jkunken@scripps.edu. The work described in this article was supported by the Accelerated Technologies Center for Gene to 3D Structure.


From the Editor A Moment’s Pause
Joshua Hoffman

We certainly live in interesting times. It's impossible to glance at the news and not hear about another brutal day on the stock market or another age-old institution on the verge of collapse. Surely every generation has seen its ups and downs, its share of economic prosperity and its equal share of distress. Sadly, today, we find ourselves experiencing the latter—a financial downturn on a massive scale, rippling through our now global economy.
Each of us experiences these peaks and valleys in different ways, but it's probably relatively safe to say that in this particular course of events, few have remained unscathed. Unfortunately, TechNet Magazine is no exception, so I'd like to take a moment to acknowledge and thank those members of our staff who have been most directly impacted by these events. Hopefully they know how much their hard work and dedication has been valued by our entire community over the years. They leave behind a lasting legacy of skill, commitment, and quality that is permanently engrained in these pages. For that, and for having been able to call each one of them a friend, I am truly grateful. I think I can speak for everyone here, as well as our reader community, in wishing each one of them every bit of luck and success as they forge ahead to conquer their next great challenge.
But, in age-old tradition, the show must go on. So we too forge ahead and bring you another edition of TechNet Magazine packed with information aimed at making your lives easier. We focus first this month on the imminent release of Internet Explorer 8. In our time, and especially in our line of work, it's pretty difficult to imagine a day going by without some sort of Internet experience. Virtually every aspect of this latest release of Internet Explorer strives to improve that experience by leaps and bounds. From Web Slices and Accelerators to InPrivate Filtering and Tab Grouping, to standards compliance, developer tools, and more, there's something here for everyone. Matt Hester digs in to show you everything that's new, and I think you'll be pretty impressed.
Of course, it doesn't stop there. John Policelli discusses Active Directory schemas, Rajesh Ramanathan talks about Voice Conferencing in OCS, Greg Stemp and Jean Ross walk us through Communicator Web Access, and Joshua Kunken highlights his organization's recent success with High Performance Computing. And many of our regular columnists join us this month as well, including Don Jones, Pav Cherny, Paul Randal, Greg Shields, Jesper Johansson, Raymond Chen and more. Enjoy!
—Joshua Hoffman
Thanks to the following Microsoft technical experts: Chris Bellee, Tim Carroll, Elden Christensen, Steve Ekren, Justin Hall, Jim Harper, Rick Kingslan, Sanjeev Nair, Dan Reger, Dhigha Sekaran, Omar Sinno, Ken Tamaru, Kimberly L. Tripp, and Jim Truher.


Toolbox New Products for IT Pros
Greg Steen

The opinions expressed in this column are solely those of the author and do not necessarily reflect the opinions of Microsoft. All prices were confirmed at the time of writing, and are subject to change.

More Cmdlets for Windows PowerShell
NetCmdlets V2
www.nsoftware.com/powershell/netcmdlets/v2.aspx
Windows PowerShell is a command-line shell for Windows desktop clients and servers. One of the best parts of the .NET integrated shell is that it is easily extensible and those extensions can easily be packaged and redistributed. Developers extend the base functionality of Windows PowerShell by creating special .NET classes that expose functionality to the shell as compiled cmdlets. Because of this, numerous third-party vendors have developed extension toolkits for Windows PowerShell to help you manage your systems more effectively through new features and packaged automation.
One of the vendors that that does this is /n Software, which has compiled a very useful set of cmdlets for systems administrators in a package called NetCmdlets version 2. As its name suggests, the NetCmdlets package focuses on network protocols, remote device management, and messaging.
After installation, your Start menu will include a Windows PowerShell link with NetCmdlets enabled. Or, as with any extension, you can add it via the standard Add-PSSnapin shell command from any Windows PowerShell prompt. With a new feature in V2, you can now pipe input to each of the cmdlets as well as use parameter sets to form a more logical command structure with the extensions. Each of the cmdlets also supports the Get-Help usage displays (if you are new to Windows PowerShell, these are kind of like *nix "man" pages).
At the time of this writing, NetCmdlets includes 30 base cmdlets for your shell. For network protocol and service access, you get cmdlets for using NNTP, RSS, FTP, TFTP, LDAP, RAS, WebDav, HTTP, and even the Amazon Simple Storage Service (S3). With these you could easily set up monitoring checks for Web sites or Web folders, automate your log archiving to your FTP server securely, and script user or group creation via LDAP. You could even synchronize your clocks with the additional network tool Get-Time, which utilizes the Network Time Protocol (NTP).
As for network device management and monitoring, you can monitor and report your Syslog servers, monitor and send traps, and manage devices through SNMP. You can perform remote command execution via ssh, rexec, or rshell. At a lower level, you can use Get-Packet to monitor traffic across a network interface, Get-Trace to see routes to devices, and Get/Send-UDP to send Wake On LAN requests or other UDP datagrams.
For messaging, you are able to send HTML and text e-mails via SMTP, check e-mail via POP or IMAP, send IMs via XMPP (Jabber), and send SMS messages. And for "utility" type functions, you get Convert-Data, which can encode or decode data between formats (such as Base64 and MD5).
As you can see, NetCmdlets provides a useful set of tools for the system or network administrator with the potential for making complex management tasks much easier to automate via Windows PowerShell scripts. If you would prefer SSH-style command line access for your Windows machines, you might want to check out the company's paired PowerShell Server product, which allows you to connect via SSH to a Windows PowerShell environment on a remote machine, even from a PDA or BlackBerry device.
Price: Starts at $79.00 (direct) for a single workstation license and $299 (direct) for a single CPU server
Streamline administration with NetCmdlets

Manage your Passwords
KeePass Password Safe V2
keepass.info
It's an old complaint, but a real problem that gets worse every day: keeping track of your list of access codes, usernames, passwords, and management URLs is nearly impossible if you're not some sort of savant. For the rest of us, a password management utility can be a real time saver and prevent headaches, especially when time is of the essence. One password management utility I like is KeePass Password Safe.
The application is free, open source, and OSI certified. With V2 (which is in beta at the time of this writing), there are a number of new features and enhancements to existing features, which improve integration, speed up tasks, and strengthen security. KeePass is also a "stand-alone" .NET-wrapped application that requires no modifications to your system outside of the program directory, making it very portable.
To protect sensitive data, KeePass uses AES/Rijndael to encrypt its entire database. It encrypts in-memory data for all field data stored within the database and helps protect against dictionary attack by transforming the master-key rapidly. When you create a new repository, you can specify a master-key that is a composite of one or more of three key sources: a master password, a key file (which you can create within the application), and your Windows user account. Once your repository is created, you can then enter your passwords and organize them.
For those with dozens or even hundreds of passwords (which is not uncommon), KeePass includes a built-in search function that lets you find entries by title, password, username, or a variety of other reference fields. With each entry, you can also add a set of extra key/value pairs of string data as well as file attachments, allowing you to store as much information as you need (such as IP lists, DR instructions, infrastructure diagrams, contact lists, and so on), and it will all be encrypted within the repository when you save the entry. Each entry also maintains a history so you can track changes.
The application integrates with your desktop, allowing you to easily copy, drag and drop, and auto-type passwords into other applications (with in-memory timeouts and protections on each). KeePass has a built-in password generation helper to generate single or lists of passwords based upon the requirements you specify, such as length and acceptable characters.
There is also built-in support for Transaction Authentication Numbers (TANs), or single-use passwords. KeePass will keep track of these, expiring each in sequence after you use it.
KeePass V2 includes an import function that supports more than 25 file formats, from CSV to other password management programs like CodeWallet and KDB3. If it natively doesn't support your file format, the product Web site also has an additional set of 10 or so importer plug-ins you can drop into KeePass. You can also export your KeePass data for backups and transport. Or, if you prefer, you can synchronize your password database to a central or backup version via URL (HTTP) or file path, making it easy to share copies of the repository among your fellow administrators.
If you don't like the idea of shared copies of a repository, KeePass V2 even supports multi-user editing, so your team can change and share the same repository without having to worry about locked data. There are numerous plug-ins available to help you back up, synchronize, and integrate with Web browsers, as well as additional encryption plug-ins if the native AES/Rjindael doesn't suit your needs.
If you are dissatisfied with your current password management utility, if you like the security of being able to see the source of the application that protects your passwords, or if you are still using Notepad (gasp!) to keep track of your secure information, consider taking a look at the latest version of KeePass as a possible addition to your toolbox.
Price: Free
Keep track of your passwords with KeePass Password Safe

Build Powerful Regular Expressions
Regular Expression Designer
www.radsoftware.com.au/regexdesigner/
Information overload is a common problem for the systems administrator. Too frequently you find yourself chasing through thousands, if not millions, of lines of log files for clues to help troubleshoot or garner usage information for your environment. Fortunately, there are many tools out there, such as Windows PowerShell, to help you filter out irrelevant data through the use of incredibly powerful regular expressions.
Well, perhaps I should say "potentially" powerful regular expressions, since getting the hang of them can be tough. But because of this there are also numerous tools out there to help you design and test those regular expressions to ensure they are catching all the lines in those logs that you need to catch.
The free Regular Expression Designer, by the Australian company Rad Software, is a simple utility with no frills that can help you quickly build and test your regular expressions before applying them in applications. The UI is divided into logical sections: Input Text, Regular Expression, Match Results, Replace Expression, Replacement Results, Options, and Language Elements.
Most of the fields are self-explanatory. For Options, you can check off the usual regular expression options you want to apply to your data, such as ignoring case, spanning multiple lines, ignoring patterned whitespace, and explicit capture. In addition, you can also choose to evaluate your text from right to left, which is great for applicable languages and UNICODE. And for the Language Elements section, you can browse the different regular expression components, double-clicking on any to have them added to your current Regular Expression structure.
With so many expression options, I find it helpful to have this right in front of me so I don't have to remember all the grouping constructs and substation particulars. Regular Expression Designer also lets you save your expression workspace for later reuse. While it may not have all the bells and whistles of some other regular expression tools out there, Regular Expression Designer is easy to use, effective, and free.
Price: Free
Build powerful regular expressions with Regular Expression Designer

Book Review
Windows Administration Resource Kit
www.microsoft.com/learning/en/us/books/11297.aspx
One of the best things about Windows Server (and Windows in general) is that it supports many different levels of end-user expertise, allowing new administrators as well as veterans admins to get up and running quickly. But as your experience grows with systems administration in a Windows environment, you'll definitely find yourself wanting to get under the covers and forego the wizards to tune and tweak the systems to meet the requirements of your organization.
One of the first leaps for systems administrators in this vein is to use the incredibly useful set of "real-world" tools found in the Windows Server resource kits that have been updated and extended alongside releases of the Windows Server OS since the beginning. The release of Windows Server 2008 is no exception. One resource kit companion book that not only shows you how to use the tools but also educates and expands your systems administration horizons is Windows Administration Resource Kit by Dan Holme (Microsoft Press, 2008).
The book is organized into 10 solution collections, each of which helps address a group of the various administrative tasks you undoubtedly encounter as a systems administrator. Throughout the book there are references to the scripts and HTAs on the companion CD. As the author states, these are meant to be "templates" that you can tweak, customizing the VBScript to manage the particulars of your own infrastructure. With the book as your guide, this is very easy to do.
The first solution collection covers role-based management and explains how to script user group membership changes; it also details the basics of HTML Applications (HTAs), and it covers implementing role-based access control. The next group of solutions offers extensive details on how to manage files, folders, and shares (covering management of ACLs, quotas, distributed file systems, and the like). Then it covers user data and settings (UDS), showing you how, for example, to redirect user folders to SMB and DFS namespaces, configure offline files, and implement roaming profiles. One nice thing about this solution collection is that it also shows you what types of data you should only manage and access on the local system.
The next solution collection delves into SharePoint. This details the basics of SharePoint Services document repository management, covering document libraries, content types, templating, user uploads, alerts, document versioning, and more.
Active Directory is a hefty part of most Windows systems deployment, and fortunately, the book devotes the next four solution collections to its various components to help ease Active Directory management. Throughout, you see reference to and learn how to use a number of helpful management scripts and utility references to ensure the security and organization of your domains. For instance, you learn how to use the Delegation of Control Wizard to, well, delegate control of various administrative tasks to different groups within your organization. You explore practices for computer management as it pertains to Active Directory, including naming schemes, provisioning, reassignment, and computer-to-user association.
And then you get into Active Directory as it pertains to user objects, again covering best practices for creating, moving, naming, and provisioning users, as well as user attribute extensions and LDAP queries. And, finally for this group of solution collections, you cover administration of Groups, while also learning how you can automate and delegate administrative tasks for efficiency.
The next solution collection then gets into software distribution and patch management across your infrastructure, detailing how to package, remotely deploy, and automate those functions. Finally, the last solution collection covers Group Policy. This section covers best practices for creating, scoping, and deploying Group Policy Objects and explores how you can use Group Policy to implement auditing. In the Group Policy discussion, it even discusses the very important—though often ignored—art of testing, piloting, and rolling back policies.
One of my favorite things about this book is that it is driven by the real world, and while it references the scripts and utilities on the companion CD, it more importantly educates you in the methods of management, allowing you to apply and tailor the knowledge to the specifics of your own infrastructure. In addition to those included scripts, the companion CD also provides a complete electronic version of the book, so you can quickly reference it from wherever you are, rather than having to carry the hefty tome from desk to desk in case you need it. So if you are looking for inspiration on how to automate those redundant tasks, tweak your servers, or manage your end-users en-masse, you might want to take a gander at Windows Administration Resource Kit.
Price: $59.99 (list)

If you have a favorite tool or utility you would like to see featured here, please write to the author at tntools@microsoft.com.

Greg Steen is a technology professional, entrepreneur, and enthusiast. He is always on the hunt for new tools to help make operations, QA, and development easier for the IT professional.


Sustainable Computing How Digital Healthcare can Help the Environment, Reduce Costs, and Improve Patient Service
Nihar Ganju and Michael Walsh

Contents
Paper "Less" Healthcare
E-fficient Healthcare Management
The Healthcare industry is on the verge of great transformation. There has been an increased investment in Healthcare Information Technology (HIT) that will affect all aspects of healthcare services, from costs to patient safety to the security and privacy of patient records. An often overlooked benefit of this transformation is the potential reduction of environmental impact from the healthcare industry. This comes through reduced material consumption and increased healthcare management efficiency.
Considering that the cost burden of American healthcare has reached over $2 trillion, the future of the US healthcare industry will depend on identifying significant areas of wasted resources and streamlining health care processes. This installment of the Sustainable Computing column examines key environmental opportunities that can be addressed directly through existing software solutions and future software innovation.

Paper "Less" Healthcare
It is imperative that the healthcare industry address the amount of resource waste it generates. Reports suggest that in the United States alone, hospitals generate some 6,600 tons of waste per day. Most of that waste—perhaps as much as 80-85%—represents materials that typically end up in ordinary landfills: paper files, cardboard packing materials, glass, plastics, and other types of commercial trash. (See the Practical GreenHealth site at cms.h2e-online.org/ee/waste-reduction.)
The healthcare industry must look to dematerialize many of the activities that produce this waste in order to drive down costs, increase productivity, and minimize the impact on the environment. Healthcare information transformation affords the greatest opportunity to significantly minimize material consumption across the healthcare industry.
Scheduling and Notification. Physicians and hospitals often require patients to schedule appointments by phone and they typically notify patients of standard medical results by mail, on the phone, or in follow-up appointments. Online scheduling systems, e-mail appointment confirmations, and video teleconferencing sessions between patients and physicians could minimize material waste, mail distribution, and travel-related carbon emissions.
Prescreening Materials. Prescreening materials are often delivered to patients on paper when they arrive for an appointment. Even when a physician's office provides the patient with a digital file, the file is almost always printed by the patient prior to the appointment and then delivered to the office upon check-in. This reliance on paper forms is inefficient and results in unnecessary material use, decreased productivity, and unnecessary costs.
Electronic Medical Records. Electronic medical records (EMRs) provide one of the surest means of eliminating material waste in the healthcare industry. Unfortunately, only 15% to 20% of U.S. physician offices and 20% to 25% of hospitals have adopted EMR systems. Barriers to adoption include procurement costs, lack of industry or government certification and standardization, concerns about data security and privacy, and a disconnect between who pays for EMR systems and who profits from them. Yet, cumulative cost savings over the next 15 years from gained efficiency and safety of EMR implementation has been calculated at $371 billion for hospital systems and $142 billion for physician systems.
Physician Patient Records. Physicians maintain ongoing records for each patient. A single patient record file may contain a range of data, such as physician notes, lab results, the patient's allergic reaction warnings, and a history of health problems. Currently, patient charts are largely paper-based, and considerable resources are spent in creating, maintaining, and filing these patient records. The paper based nature makes the charts slow and cumbersome when physicians need fast access to patient data. A potential loss of $5 per chart pull can be attributed to paper-based charts as opposed to when using electronic records.
Computerized Physician Order Entry. Care instructions, laboratory orders, and prescriptions have typically been provided in handwriting on paper. Computerized Physician Order Entry (CPOE) is designed to change the way in which physician instructions are communicated will also transform the process by which a physician arrives at her instructions—this is especially true for scenarios where the CPOE is integrated with a clinical or diagnostic decision support system. Considerable evidence indicates that CPOEs help to reduce medical errors and adverse drug events. Limiting adverse events and medical errors and, in turn, limiting related legal costs can prove to be a significant source of savings. CPOEs can also help reduce material waste, especially in hospitals, and improve patient safety.
Paperless Billing and Claims Management. Over the past decade, there has been significant growth in paperless billing utilization by hospitals and insurers in a number of North American and European countries. But the full potential of paperless billing and claims management has not been realized (and in some cases, technology is not the limiting factor):
  • Hospital discharge papers and insurance billing releases are often signed on multi-copy paper rather than digitally.
  • Insurance providers often mail patients an explanation of benefits for processed claims as paper statements delivered by mail rather than through secure electronic delivery to a patient healthcare system.
  • Unlike most insurance medical and prescription policies, reimbursement claims for programs such as U.S. Treasury Health Savings Accounts (HSAs) often require patients to mail a paper copy of the receipt itemizing the covered expense to the program administrator for claims processing.
  • Pharmacy receipts for prescription and non-prescription medicines are physically provided at checkout on paper rather than through secure electronic delivery to a patient healthcare system.
Physician Collaboration. Delivering specialized healthcare services requires collaboration between healthcare providers. When providers are geographically separated, they need an efficient means of communicating. In the past, most communication between providers occurred through written files, faxed notes, and telephone calls. Unified communications provides a better option—it streamlines communications, increases efficiency, reduces costs, and increases collaboration.

E-fficient Healthcare Management
For obvious reasons, it's impossible to drive out all material consumption by the healthcare industry. It therefore is important to consider how the healthcare industry can better manage the materials required to render health services. Improving efficiency eliminates waste and minimizes the environmental impact of the industry. If properly implemented, it can also cut costs, increase productivity, and improve quality of life for patients.
Chronic Care Management. The key to reducing the financial and environmental impact of chronic care is to enable healthcare providers to prevent chronic conditions from becoming acute ones. Patients with chronic conditions such as diabetes, hypertension, and asthma currently account for up to 75% of all healthcare expenses, with much of that money spent when people lose control of their health problems and end up in a hospital. If people can be kept out of a hospital or if treatment at a healthcare facility can be managed to avoid emergency situations through the use of remote health monitoring systems, the healthcare industry can avoid costly emergency room visits and minimize the environmental impact of healthcare itself. In the end, it is estimated that the potential financial benefit of such prevention could double the estimated net savings that can be realized through EMR systems.
Priority Care Management. One of the greatest challenges to a physician is keeping up to-date on the needs of his patients. In the past, physicians conducted regular rounds of their patients to receive updates and relied on phone calls when they were off-duty. But mobile solutions just coming to market can constantly monitor patients and provide real-time updates to physicians. These services can help physicians make more efficient use of their time and provide them with the ability to remotely issue urgent care instructions if a change in a patient's status demands immediate action.
Rural Health Management. Unified communications and mobile devices allow rural physicians to remotely monitor and interact with patients. These technologies assist physicians when planning travel requirements and allow them to eliminate unnecessary trips. This in turn frees up physician time and resources, letting physicians deliver a higher level of service by freeing-up critical time and resources. In the end, rural physicians can expand coverage to additional patients and devote more of their time and resources to higher priority care cases. This is especially valuable to physicians in underserviced communities.

Nihar Ganju is an experienced systems integration consultant and a computer sciences graduate of JHU. He currently attends FSU College of Medicine where he is exploring how technology can play a more signifi cant role in medicine.

Michael Walsh is the Senior Environmental Communications Manager at Microsoft. He can be reached at micwalsh@microsoft.com or walshtechnet@live.com.


SQL Q & A Database Consistency, Temporary Tables, and More
Paul S. Randal

QI've noticed some very strange behavior on one of our SQL Server 2005 databases. We run a DBCC CHECKDB on the database as part of our nightly database maintenance and some nights the DBCC returns corruption errors. The weird thing is that we don't see page checksums failures during the previous day and if I manually run DBCC CHECKDB the morning after the maintenance job failure, the corruptions have disappeared. Can you explain what's going on? This has been happening for about a month and I'm concerned that I can't trust DBCC CHECKDB.
AIt's good to see that you have DBCC CHECKDB as part of your regular maintenance and that page checksums are enabled. The behavior you describe can be very disconcerting when it occurs—it almost looks like DBCC CHECKDB is giving incorrect results. But this scenario does occur every so often—there are no run-time indications of corruption, DBCC CHECKDB finds corruption, and then many hours later the corruptions are gone when DBCC CHECKDB is run again. Let's take this step by step.
First, it's very common for corruptions to be reported by DBCC CHECKDB that have not been reported during regular database operations. Although page checksums are a great method of detecting corruptions caused by the I/O subsystem, they are effective only when the data pages are read by SQL Server after a corruption has occurred.
Imagine that a data page has a page checksum applied to it and it is then corrupted at some later point by the I/O subsystem. Even though the page checksum will detect the corruption, it is only when the page is read into memory by SQL Server that the page checksum is validated and the corruption is discovered. If that page is never read by SQL Server, the corruption will never be discovered. This is why it is vital to enable page checksums and regularly perform consistency checks—consistency checks will read all pages in the database, validating their page checksums and discovering corruption as early as possible.
In your case, it looks as if the corruptions have occurred in data pages that were not read as part of regular database operations and so the corruptions were not discovered until DBCC CHECKDB read the corrupt pages. Although it seems as if the page checksums didn't detect the corruption as they should, that's not the case.
Second, "disappearing" corruptions can occur quite easily between successive executions of DBCC CHECKDB, but only on databases where changes occur between the two DBCC runs. Say a page is indeed corrupt and DBCC CHECKDB reports it as being corrupt. Now say the page is then deallocated from a table (for example, because it becomes empty). A subsequent DBCC CHECKDB will not read it this time and thus will not report it as being corrupt. DBCC CHECKDB reads only allocated pages (meaning pages that are currently in use). In your case, I'm guessing that the rest of the nightly maintenance job includes index rebuilds or reorganizing—both of which can drastically alter the set of pages that are allocated to a particular object or index. This would account for the behavior you're seeing. An index rebuild deallocates the corrupt pages as a side effect and then the next DBCC CHECKDB comes back clean.
To catch the corrupt pages, alter your maintenance job so that it stops if the DBCC CHECKDB step fails. This way, you'll be able to manually verify the corruptions and take further action.
QWe're planning to upgrade from SQL Server 2000 directly to SQL Server 2008, skipping SQL server 2005. But I'm worried about tempdb. This already caused problems for us in SQL Server 2000, as we have lots of short-lived temp tables. From what I understand, tempdb is used a lot more heavily now (in both SQL Server 2005 and SQL Server 2008) and so special provisioning has to be done to prevent performance dropping after an upgrade. Can you explain why this is the case and what we should do?
AFrom your question, I'm guessing that you've had to implement the one-file-per-processor tempdb architecture that is commonly required when many short-lived temporary tables are created by multiple database connections. And you might have even had to enable the 1118 trace-flag (see "Concurrency enhancements for the tempdb database" for more information on this).
Although there is the potential for heavier tempdb use in SQL Server 2005 onward, you might not see this unless you specifically use features that rely on tempdb. Note that if you take a solution that experienced tempdb performance problems with SQL Server 2000 and upgrade it to SQL Server 2008 on the same hardware, you should in many cases see the performance problems alleviated somewhat because of some tempdb-specific changes to the SQL Server Storage Engine in SQL Server 2005.
The features that do make heavy use of tempdb on SQL Server 2005 and SQL Server 2008 are:
  • Online index operations
  • DML triggers
  • Multiple-active resultsets ( MARS)
  • Snapshot isolation (both transaction-level and statement-level)
All four of these features use an underlying technology called versioning to store various point-in-time versions of data records. Simply put, these record versions are stored in the version store in tempdb, with all user databases sharing the same version store in the single tempdb. The more you use these features, the heavier the use of the version store and hence tempdb will be and the more potential there will be for a performance impact.
The key to any successful upgrade is to implement a representative production load on a test system with the new schema and measure the performance to avoid surprises you might encounter if you were to go straight into production.
Unfortunately, this discussion is way beyond the scope of this column to cover in-depth, but there are some excellent resources available that I recommend you check-out:
  • The white paper "Working with tempdb in SQL Server 2005"
  • In-depth blog posts on the Storage Engine team blog
  • Various SQL Server 2008 Books Online topics, starting with "Capacity Planning for tempdb"
  • The general SQL Server upgrade page
QWe're implementing a nightly database maintenance plan that includes improving index performance. I've heard that setting the "fill factor" option for indexes can completely remove the need for maintaining indexes. Is this true? It seems that some indexes in our database suffer from fragmentation and some don't. Should we set a default fill factor for the database that will apply to all indexes and if so, what value should we use?
AThe fill factor setting can indeed be used to partially mitigate the need for index maintenance, but rarely can it be used to completely remove the need. In a nutshell, the fill factor setting instructs the Storage Engine to leave a certain percentage of free space in pages of clustered and nonclustered indexes when they are created or rebuilt. ( Note that the fill factor setting is not maintained during regular insert/update/delete operations.) A fill factor of 90, for instance, leaves 10% free space. Fill factors of 0 or 100 both leave no free space (this has been the source of much confusion).
The idea is that space is left in the pages, which allows records on the page to expand or new records to be inserted on the page without causing an expensive, fragmentation-causing operation called a page split. You specify a percentage of free space so the pages can become more steadily full until the next index maintenance operation occurs, which resets the fill factor again. The trick is to choose a percentage that minimizes page splits between index maintenance operations.
For an OLTP (online transaction processing) database, there's no easy answer except to choose a fill factor for each index based on trial and error. For data warehouses, where the indexes don't change, the fill factor should be 100% (meaning no free space is left on the pages). It is pretty uncommon that the default fill factor for a database is changed from the default of 100%, as the best fill factors for various indexes are usually different. The SQL Server 2008 Books Online topic "Fill Factor" has a lot more information on this.
One other option is to change the index so that page splits do not occur. This might involve changing the index key so that inserts are not random (for instance, by not using a random GUID primary key) or disallowing operations that change the size of variable-length columns.
QWe're going to be moving to SQL Server 2008 as soon as SP1 comes out and one of the features we're looking forward to using is FILESTREAM, as it removes the 2GB size limit for column values. Before we start designing the next version of our schema to use the FILESTREAM datatype, are there any drawbacks or problems we should be aware of that could lead to issues in production?
AIt's always a smart idea to determine all the characteristics of a new feature before designing it into a new schema or application, especially one that utilizes technologies outside of SQL Server itself, as is the case with FILESTREAM. Most of the information you need is contained in a white paper I wrote for the SQL Server team called "FILESTREAM Storage in SQL Server 2008." I highly recommend you read that paper for a complete discussion, but I will summarize the major areas of concern here.
First, the FILESTREAM data is stored in the NTFS file-system, rather than inside SQL Server data files. There are various configuration steps that must be taken to ensure that NTFS performs well with very large numbers of files in a single directory, such as turning off 8.3 name generation, setting the NTFS cluster size appropriately, and possibly separating the FILESTREAM data onto separate physical disks from other data.
Second, you should ensure that the average size of data stored using FILESTREAM will be 1MB or higher. Research has shown that for data sizes of less than 256KB, and sometimes between 256KB and 1MB, better performance can be gained by storing the data directly inside SQL Server rather than using a mechanism such as FILESTREAM.
Third, you should consider the operations that will be performed on the FILESTREAM data. Partial updates are not supported for FILESTREAM data, so even updating a single byte of a 200MB data value will result in an entirely new 200MB value being created. Apart from being an expensive operation, this can lead to NTFS-level fragmentation, which can further reduce performance. If partial updates are commonplace in the application, some kind of batching mechanism may be required to avoid repeated updates to a single FILESTREAM value.
Last, you should consider the cross-feature compatibility of FILESTREAM with high-availability technologies. FILESTREAM fully supports backup and restore operations (including point-in-time recovery), log shipping, and replication. It is not, however, compatible with database mirroring in any way in SQL Server 2008. ( I've been told this will be fixed in the next version of SQL Server.)
This is just a taste of the things to consider. For the full picture, you should read the white paper. As with any new feature, though, before designing an application around it, be sure to do some extensive testing to see whether its capabilities match your requirements. Given that FILESTREAM also encompasses NTFS storage, I'd also do preproduction performance and disaster-recovery testing to ensure that nothing trips you up when you go live.

Paul S. Randal is the Managing Director of SQLskills.com and a SQL Server MVP. He worked on the SQL Server Storage Engine team at Microsoft from 1999 to 2007. Paul wrote DBCC CHECKDB/repair for SQL Server 2005 and was responsible for the Core Storage Engine during SQL Server 2008 development. Paul is an expert on disaster recovery, high availability, and database maintenance, and is a regular presenter at conferences around the world. He blogs at SQLskills.com/blogs/paul.


Utility Spotlight RichCopy
Joshua Hoffman

Code download available at: HoffmanUtilitySpotlight2009_04.exe (5,896 KB)

Back in the November 2006 issue of TechNet Magazine, I wrote about a handy little utility called Robocopy GUI (see "Utility Spotlight Robocopy GUI"). This simple tool was written by a Microsoft engineer named Derk Benisch, and all it really did was create a graphical interface for the very popular Robocopy command-line utility, which provided file copying capabilities far beyond what was built into Windows. Believe it or not, more than two years later, the Robocopy GUI article remains the single most popular piece of content we've ever published, having been viewed well over 220,000 times.
Needless to say, this speaks far more to the value of the tool than to the quality of my writing. Nevertheless, it also makes perfectly clear just how crucial certain simple tasks are in our day-to-day lives, and how staggering an impact even a small improvement in performing those critical tasks can make.
In this case, we're discussing the simplest of tasks: copying files. Except copying files is not always that simple. What if you're copying thousands of files across a slow connection? What happens if your network hiccups and interrupts the copy? What if you want to make sure that you preserve particular file attributes, such as a Last Modified date, but not other attributes, like security descriptors? What if you want to filter the files you're copying from source to destination based on filename or extension?
If any of these apply to you, the simple Ctrl+C/Ctrl+V combination probably won't cut it. And though Robocopy GUI certainly improves on that basic functionality, it still has some limitations in terms of granular control and usability. What you need is RichCopy.
RichCopy is a free utility that comes to us from Ken Tamaru of Microsoft. The tool was first developed in 2001 and has been updated regularly to keep pace with evolving needs. Trust me when I tell you, this is the answer to all your file copying needs. What you'll find most striking the first time you take RichCopy out for a spin is that it's a multithreaded copying tool. That means that rather than copying one file at a time in serial order, RichCopy can open multiple threads simultaneously, allowing many files to be copied in parallel and cutting the total time required to complete the operation several times over. You can also pause and resume file copy operations, so if you lose network connectivity at any point, you can just pick up where you left off.
Of course, these are really just the simplest features of RichCopy. As Figure 1 shows, you also get a vast array of granular controls that allow you to customize all of those fancy aspects of your file copying that I talked about earlier—filtering files, saving attributes, adjusting cache size, and so on. If you regularly copy lots of files over the network or between various storage devices, these features will significantly ease your daily life.
Figure 1 RichCopy offers a number of granular controls that allow you to tailor file copying to your needs
Remember, as with all of our free utilities, RichCopy is provided as-is, and it's not supported by Microsoft.

Joshua Hoffman is the Editor-in-Chief of TechNet Magazine.


Inside SharePoint Security and Compliance with AD RMS
Pav Cherny

Code download available at: ChernySharePoint2009_04.exe (846 KB)

Contents
Information Security versus Usage Policy
SharePoint Integration with AD RMS
AD RMS Policy Settings Override
AD RMS Protection versus AD RMS Protection
Document Protection Outside of the SharePoint Environment
Enforcing Read-Only Permissions
Conclusion
Organizations today maintain enormous quantities of sensitive and confidential information in SharePoint, including all sorts of financial data and personally identifiable information, and protecting that information can be a challenge. With fines up to $15 million per offense, the cost of noncompliance with content storage and protection regulations, such as SOX, HIPAA, GLBA, and ISO 27000+, is high. It is therefore vital to review and use SharePoint compliance features appropriately, as well as to enforce usage policies outside of the SharePoint environment and provide compliance documentation.
The good news is that Windows SharePoint Services (WSS) 3.0 enables you to achieve compliance via an Information Rights Management (IRM) framework. Microsoft Office SharePoint Server (MOSS) 2007 goes even further by including Microsoft Office document protectors that take advantage of the IRM framework and its integration with Active Directory Rights Management Services (AD RMS).
However, it is easy to stumble over the IRM framework implementation details. Not even Microsoft gets it right all the time: The webcast titled "Best Practices for Managing and Securing SharePoint 2007" claims that you can automate and force AD RMS encryption by moving files into a document library; that content protection in an AD RMS-enabled document library is the same as when using Microsoft Office Word; and that AD RMS policy follows a document wherever this document goes.
But the reality is more complicated. Misunderstanding the IRM framework can lead to ineffective security solutions and noncompliance at customer organizations, so it's important to get it straight.
In this column, I review the integration of SharePoint with AD RMS, with an emphasis on issues you must consider when designing and implementing SharePoint-based security and compliance solutions. First, I discuss the difference between AD RMS itself and SharePoint integration with AD RMS, and I show you first hand in a test environment that SharePoint integration with AD RMS requires a secure SharePoint environment to begin with.
Once this is clarified, I focus on certain usability issues you might want to address in your security and compliance solutions and documentation. I'm assuming you have already deployed AD RMS and MOSS. The deployment in production environments is covered very well in the "AD RMS Deployment with Microsoft Office SharePoint Server 2007 Step-by-Step Guide." There is nothing to add, apart from some worksheets for this test environment.
In the next column, I'll cover the deployment of Microsoft Office 2007 and MOSS 2007 in an AD RMS pre-production hierarchy because this isn't covered in the WSS SDK or the AD RMS SDK. It's an interesting exercise and a great opportunity for a deep drilldown into the IRM framework architecture.
For now, however, let's focus on general security, compliance, and usability aspects that are not so well covered in the SharePoint documentation. The companion material for the April issue of TechNet Magazine includes worksheets, compiled tools, and source code. As always, the companion material is for illustration and test purposes only and is not to be used in a production environment.

Information Security versus Usage Policy
No introductory AD RMS presentation is ever complete without a demonstration of the technology's amazing document protection capabilities, including disabled copy and print options, document expiration, and so forth. It seldom occurs to people, however, that these aren't actual security features.
When you see a disabled print command, isn't it logical to conclude that the user cannot carry a printout to the competition? When you hear that a user can't forward a document and that unauthorized persons can't even open the document, doesn't it follow that the information in that document is protected from unauthorized disclosure?
No, unfortunately, it doesn't. You would have to outlaw all virtualization and remote desktop technology, third-party screen capture solutions, cell-phone cameras, and other such tools that enable the user to take an image of the document for purposes of printing, forwarding, and viewing beyond the document expiration date. (    Just take a glance ahead at Figure 2, which includes a screenshot of a protected document.) Perhaps you should also outlaw all Windows-based portable computers and Windows Mobile devices, which enable the user to open protected documents and show them to unauthorized viewers at any location.
Obviously, these document protection features don't help to enforce information security, though they can help to implement usage policy, such as to mitigate the risk that employees conduct business based on outdated information from expired documents. Digital Rights Management (DRM) may have been originally about preventing users from creating illegal copies so that the rightful owner can monetize intellectual property, but today this technology is clearly moving towards regulatory compliance enforcement.
Of course, AD RMS does go beyond merely enforcing usage policy by encrypting the content using Advanced Encryption Standard (AES) and Electronic Codebook (  ECB) ciphers. To encrypt the content, AD RMS generates a symmetric 128-bit AES content key, which is then encrypted using a public key obtained from the Server Licensor Certificate (SLC) of your AD RMS server.
You generate the SLC and the server's encryption keys when you install AD RMS in your Active Directory forest. AD RMS then embeds the encrypted content key together with the specified document protection settings in a publishing license (also called an issuance license), which AD RMS then signs using the owner's private key from his or her Client Licensor Certificate (CLC). The CLC—created as part of the user activation process when the user first creates an AD RMS-protected document—defines the user's right to publish content. The AD RMS-enabled application then attaches the publishing license to the encrypted content and then the content is protected. The arrangement of content pieces in the AD RMS-protected file depends on the application. Figure 1 illustrates the container format for the AD RMS Add-on for Internet Explorer, as documented in the AD RMS SDK.
Figure 1 The structure of an AD RMS-protected document in compound file binary format (CFBF)
Any user who wants to access the content must open the protected file, extract the signed publishing license, and upload this license together with the user's Rights Account Certificate (   RAC) to the AD RMS server in order to download a use license (   UL) for the document, also called an end-user license (   EUL). The RAC identifies the user by e-mail address or security identifier and it includes the user's encryption key, encrypted using the local computer's machine key as specified in the Security Process Certificate (SPC). The UL returned from the AD RMS server contains the content key, which the server encrypted using the user's public key obtained from the user's RAC. The user can now decrypt the content key with his or her private RAC key and then finally decrypt the protected file content.
As you can see, there are a lot of keys, licenses, and certificates involved in this process. The application can also add the UL to the protected file so that the user does not have to retrieve the UL again from the AD RMS server, but you can change this behavior in the AD RMS policy settings. You can specify that the user must obtain a new UL whenever he opens the document, or you can define a validity period for the UL so that the user must acquire a new UL when the current one expires. For all the details, check out the AD RMS SDK. It's an excellent information source even if you are not a developer.
For SharePoint security architects and administrators, there are at least three important facts to take away from the AD RMS implementation details. First, AD RMS protection is end-to-end, meaning protection remains with the document wherever the document goes (note that I am talking about AD RMS here, not about SharePoint integration with AD RMS). So you can place an AD RMS protected file on a Windows BitLocker drive or on an unencrypted drive, you can place it on a file share with Everyone read access, you can upload it to a SharePoint document library, you can send it to an unauthorized recipient outside your organization, but the content remains protected end-to-end, from the document owner on one end to the consuming user on the other end.
Second, decrypting an AD RMS-protected document requires that the end user obtains a UL from your AD RMS server at least once. By logging all licensing activities on your AD RMS server, you can reliably track who opened the document and, perhaps even more importantly, who attempted to gain access unsuccessfully.
Logging is a new feature of AD RMS in Windows Server 2008 and it's really cool because you can create tools for security analysts based on .NET and SQL Server reporting technology to process this information automatically for forensic purposes. Of course, you can also see all the requests and certificate information directly in the AD RMS management console.
Finally, and this may seem obvious yet is nevertheless very important, AD RMS-based document protection relies on encryption. If you don't encrypt the document, you cannot enforce information security. If you create a tool that decrypts a protected document and stores it in unencrypted format, you break AD RMS end-to-end security.
Who is to tell that an unauthorized user didn't get an unencrypted copy? Microsoft mitigates against this by requiring developers to sign a Production License Agreement with Microsoft in order to obtain a production certificate. A production certificate signs an AD RMS application into the AD RMS production certificate hierarchy. It specifies among other things the modules that the system can load into the process space of the application to protect the unencrypted data in memory. It works. Microsoft says that AD RMS security has never been compromised.

SharePoint Integration with AD RMS
With these facts in mind, let's approach SharePoint integration with AD RMS. First and foremost, Microsoft states in all relevant product documentation pieces that AD RMS-enabled document libraries store content items unencrypted. So, there is no bulk encryption when you move items into an AD RMS-enabled document library. More importantly, because the items are unencrypted, there is no AD RMS protection and no security gain in the SharePoint environment.
SharePoint administrators and users might believe that AD RMS end-to-end security exists, but SharePoint integration with AD RMS fully depends on SharePoint security. According to "Information Rights Management in Windows SharePoint Services Overview" in the WSS 3.0 SDK, Microsoft opted not to store the items in encrypted, rights-managed formats due to customer demand.
Take a look at Figure 2. It illustrates the IRM framework architecture in an unsecure SharePoint environment that permits unauthorized users direct access to the content databases. The key point is that integrated document protectors apply AD RMS protection dynamically when you download a document through SharePoint. This means that you perceive an unprotected content item as an AD RMS-protected document. When you upload changes, however, SharePoint removes the AD RMS protection again.
Figure 2 An AD RMS-protected document that is, in fact, a non-AD RMS-protected content item
It's important that you make sure that this decryption behavior is consistent with your security and compliance requirements. Just imagine a scenario where you host sensitive HR documents in an AD RMS-enabled document library and fail to inform the HR department that all SQL Server and SharePoint farm administrators as well as any SharePoint developers outside of the HR department have unrestricted access to the content items in unencrypted form. Don't neglect this issue in your solution design and in your compliance documentation. For example, you might have to deploy separate SQL Server instances and SharePoint farms that are maintained by HR-internal administrators only.
When designing security and compliance solutions, keep in mind that SharePoint integration with AD RMS does not eliminate the need to protect your SharePoint environment as described in the Office SharePoint Server Security guide and the Windows SharePoint Services Security Account Requirements worksheet. If you don't protect your security accounts and passwords, if you deploy unverified code from questionable sources, or even enable server-side scripts on your SharePoint servers, then an IRM framework and AD RMS integration won't save you when an unauthorized person gets access to the content items, as explained in the January column titled "SharePoint Security Accounts."
Check out the April companion material, which includes two Web Parts and a worksheet that demonstrates how a questionable Web Part can render all your security and compliance solutions ineffective. One Web Part downloads documents the proper way by using the SharePoint object model, and the other Web Part takes a shortcut by accessing the SharePoint content database directly within the security context of the application pool account.
As you can verify, the second Web Part provides the user with full access to all document libraries in the site collection regardless of SharePoint permissions and AD RMS settings. So, make sure your SharePoint developers don't break the rules and take shortcuts; don't program directly against content databases, and keep questionable components that do so away from your production servers.

AD RMS Policy Settings Override
Another important issue you must address in your compliance solutions and documentation is that content protection in an AD RMS-enabled document library is not the same as when working in Microsoft Office Word outside of document libraries. Check out Figure 3 and the worksheet titled "Specifying AD RMS Permissions in Microsoft Office Word inside and outside of Document Libraries" in the companion material.
Figure 3 Document library’s policy overrides owner’s policy.
The worksheet demonstrates that AD RMS-enabled document libraries do not retain AD RMS policy settings and user authorizations specified by the document owner. This is a result of the fact that SharePoint decrypts the document on upload, so the owner's policy definition does not follow the document into the SharePoint environment. The next time a user downloads the document, SharePoint applies the AD RMS policy settings defined for the document library and the user's permissions in SharePoint, as indicated in Figure 3.
Note that this issue primarily affects site administrators because SharePoint grants only users with Manage Permissions privileges the right to modify the AD RMS permissions of the document. Site members do not fall into this category. Nevertheless, it can be confusing for site administrators and it can lead to unintentional disclosure of confidential information to the wrong SharePoint users. When using AD RMS-enabled document libraries, make sure your site administrators and managers understand that the AD RMS document settings they change in their Office applications are not retained. For details how the IRM framework translates SharePoint permissions into AD RMS document permissions, see the topic "Information Rights Management in Windows SharePoint Services Overview" in the WSS SDK.

AD RMS Protection versus AD RMS Protection
The policy override issue is hard to explain to site administrators and managers because the root cause is a miniscule difference between AD RMS-protected documents created outside of the SharePoint environment and those created within the SharePoint environment. The former retain their policies even when you upload them into an AD RMS-enabled document library, but the latter do not. AD RMS end-to-end security prevents SharePoint from decrypting documents created outside of the SharePoint environment.
Figure 4 Verifying that the application pool account has Full Control
For decryption to work on upload, the site's application pool account must have Full Control permissions as a document owner. SharePoint adds this application pool account to the document template that you download when you click the New button in an AD RMS-enabled document library. As a document owner, you can verify that the application pool account has Full Control. Just display the advanced AD RMS options by clicking on the More Options button in the Permission dialog box of the document in Word 2007, as illustrated in Figure 4.
Also, do not forget to mention in your compliance documentation that the list of users with permissions to the document does not reflect the list of users with permissions to the content item in SharePoint. SharePoint grants only the current user and the site's application pool account AD RMS document permissions on download.
The advantage is that the SharePoint user cannot share this document with anybody else outside of the SharePoint environment. The disadvantage is that document owners might incorrectly assume that AD RMS protection is more restrictive than it actually is. In the example of Figure 4, SPAdmin and ITUser have access to the document but the document permissions do not reveal this fact. Fortunately, only site administrators and managers can view document permissions. Site members don't and can't view the list of users with permissions to the document.

Document Protection Outside of the SharePoint Environment
It might take some time to get used to the way SharePoint maps site permissions to AD RMS document permission, but one aspect you might appreciate immediately is that you can't open a document downloaded from an AD RMS-enabled library under the identity of the application pool account. It's quite amazing.
In Figure 4, this is the WssDefaultSite account. The account has Full Control permissions and still it cannot open the document. So, an unauthorized person might guess the login name and password of the application pool account, but as long as you prevent direct connectivity to the content database and force all document access through SharePoint so that SharePoint can apply AD RMS protection, the content remains inaccessible, as illustrated in Figure 5.
Figure 5 The application pool account cannot open the AD RMS-protected document.
Check out the companion worksheet "Examining Content Item Security in an AD RMS Enabled Document Library." It explains how to use the GetDocument tool, which is also included in the companion material. GetDocument.exe uses FrontPage RPCs via Author.dll to download documents to the user's Desktop in a way that is similar to Microsoft Office Word. The tool demonstrates that SharePoint applies AD RMS protection no matter which way you download the document (and it makes it easier to specify different account information for each document download).
The application pool account can't open the document, even though the account has the Full Control permissions, because the application pool account doesn't have an e-mail address. If you use Microsoft Exchange Server, your user accounts most likely have valid e-mail addresses, and if you don't use Exchange Server, you must set the mail attribute of your users manually or by using tools such as Ldifde.exe; the application pool account doesn't need to be e-mail-enabled, however.
SharePoint uses the application pool account's security identifier when applying AD RMS protection on the server. AD RMS client applications, on the other hand, such as Word 2007, use the mail attribute to associate users with AD RMS permissions. If there is no mail attribute, then there is no match that would grant AD RMS permissions and the account cannot open the document.

Enforcing Read-Only Permissions
The inability to share documents outside of the SharePoint environment provides workflow designers with greater control over the collaboration process. Even more compelling, however, is the ability to enforce read-only document permissions outside of the SharePoint environment.
Out of the box, SharePoint prevents site visitors and other users with only View List Item permissions from uploading documents, but you can't prevent these users from modifying downloaded documents and distributing them by other means, such as e-mail messages. SharePoint integration with AD RMS closes this gap. Now, users with only View List Item permissions get AD RMS read permissions, but this doesn't include copy or edit permissions for the document content, as illustrated in Figure 6. What's meant to be read-only remains read-only within and outside of SharePoint!
Figure 6 The AD RMS-protected document remains read-only after download.
This seems to be one of the biggest benefits of SharePoint integration with AD RMS because it enables you to implement managed repositories for boilerplate documents. Talk about lowering business risks! Give the legal department Edit List Items permissions and the rest of the company View List Item permissions in SharePoint and you can rest assured that no bogus, manipulated, or outdated boilerplates fly around in your company. Just don't forget to allow document printing in the AD RMS-related library settings so that employees can print out the contract documents.

Conclusion
SharePoint integration with AD RMS is a useful feature that enables you to extend the reach of your security and regulatory compliance solutions far beyond the perimeter of your SharePoint environments. By using this technology, you can enforce usage policy and information security of documents after download. However, be sure to keep in mind that the IRM framework and AD RMS integration do not increase security within your SharePoint environment. It remains imperative to apply correct SharePoint access controls; protect security accounts; properly configure application pools, content databases, SQL Server instances, and SharePoint servers; and keep questionable components and unverified code away from your production servers.
The IRM framework is definitely worth evaluating. Without a doubt, you will notice rough edges and certain limitations of the current implementation, but the issues do not speak against the technology; instead, they highlight that this technology addresses a huge field of requirements and opportunities. Microsoft says that future AD RMS versions will provide more granularity and control, and this will certainly affect integration with SharePoint as well.
It would be great to have more control over permission mappings. It would be useful to have expiration dates relative to the download date. It would be nice to have the ability to configure document protectors for each individual document library separately. It would be helpful to combine document protectors with document parsers so that you can disable decryption of documents while retaining indexing and property promotion capabilities. The wish list is long, but the current implementation is definitely a step in the right direction and compelling for organizations that use SharePoint and must comply with regulations covering content storage and protection.
The only concern is that WSS 3.0 organizations cannot benefit from AD RMS integration out of the box because the Microsoft Office document protectors are included only in MOSS 2007. But don't despair! In the next column, I show you how to extend WSS 3.0 to achieve Office document protection based on the IRM framework and AD RMS, and it doesn't require developer skills. I will show you the nuts and bolts of this technology by deploying a pre-production environment, which is useful if you want to develop your own AD RMS-based Office and SharePoint solutions or simply compile solutions that other developers provide you free of charge in order to address security and compliance requirements beyond the standard capabilities built into WSS 3.0 and MOSS 2007. Stay tuned!

Pav Cherny is an IT expert and author specializing in Microsoft technologies for collaboration and unified communication. His publications include white papers, product manuals, and books with a focus on IT operations and system administration. Pav is President of Biblioso Corporation, a company that specializes in managed documentation and localization services.


Windows PowerShell Automating User Provisioning, Part 2
Don Jones

Contents
Creating a User
Missing Exchange?
Options, Options, Options
In the previous installment of this column, I created the basis for an automated user provisioning script. I created a function called ProvisionInputCSV, which is designed to read user information from a CSV file and return that information in the form of a hashtable. I took that approach because it allowed me to then create multiple other "import" functions—import from a database, import from a spreadsheet, or whatever—and have each one return an identical-looking hashtable. My actual provisioning function simply needs to accept that hashtable, which might contain elements for a user's first name, last name, city, department, and so forth.
Video
Don Jones continues his four-part series with a look at how you can use Windows PowerShell to create a real-world user provisioning script. Watch Don demonstrate these techniques in his video screencast.

In this month's column, I can begin using that hashtable to create a new, mailbox-enabled user in Active Directory (or a non-mailbox user, if you're not using Exchange Server 2007).
The basic framework of the provisioning function looks like this:
Function Provision {
  PROCESS {
  }
}
I've already established that the function will receive a hashtable on the pipeline, which it will access via the special $_ variable within the PROCESS scriptblock. The hashtable will have keys that correspond to user attributes, and the values of those keys will contain each user's information. Some examples include:
  • $_['sn'] = "Jones"
  • $_['givenName'] = "Don"
  • $_['samAccountName'] = "donj"
The exact attributes available will depend on what you've put into your ProvisionInputCSV function or other import function.
Keep in mind that user provisioning typically involves several steps, such as creating an account, creating a folder, and so forth. It would be easy to stick all those tasks into my Provision function, but doing so would make the function pretty complicated—and more complicated means harder to debug and troubleshoot. Instead, I'll use the Provision function as a sort of master task list, breaking each task out into its own subsidiary function. In the end, the Provision function will look something like this:
Function Provision {
  PROCESS {
    CreateUser $_
    CreateHomeFolder $_
    AddToGroups $_
    UpdateAttributes $_
  }
}
Windows PowerShell Q&A
Q I don't see a –computerName parameter on very many cmdlets. Can Windows PowerShell be used to manage remote computers?
A Windows PowerShell version 1 has limited remoting capabilities. The Get-WmiObject cmdlet offers, as you might have discovered, a -computerName parameter so that you can retrieve information from one or more remote computers, like so:
Get-WmiObject Win32_Service –computerName
  "localhost","server2","server3"
But that's about the only built-in remote management capability you'll find in the first version of Windows PowerShell.
However, other technologies, such as Exchange Server and Active Directory, are inherently "remoted"—that is, your client computer knows it isn't a mail server or a domain controller and will contact one as appropriate. So you don't actually need a -computerName parameter for commands that work with those technologies.
Each of the four subsidiary functions handles one of the major tasks that I outlined in the first part of this series, and each is being passed the full set of user information in the $_ variable. So there—my provisioning script is done!

Creating a User
Just kidding. Obviously I still need to write the real functionality, which lives in those four subsidiary functions. In this installment, I focus on the CreateUser function. The first iteration of this code assumes that you have Exchange Server 2007 in your environment. (Sorry, this won't work for older versions of Exchange Server.) Whatever computer you're using to run this script must have the Exchange Server management tools installed, and you'll need to make sure the Exchange Server snap-in for Windows PowerShell is loaded. To check, run Get-PSSnapin and see if it's listed. If it isn't, run Get-PSSnapin -registered to make sure it's installed. Then, to load the snap-in into the shell, run this:
Add-PSSnapin Microsoft.Exchange.Management.PowerShell.Admin.
If you get tired of typing that command, you can add it to the script that contains all the provisioning functions or you can add it to your Windows PowerShell profile to have it load when you open the shell. (I explained how you can create custom profiles in the October 2008 installment of this column, "The Power of Profiles."
Here's the basic framework for my function:
Function CreateUser {
  Param($userinfo)
}
Notice that I've included a param block, which tells the shell what parameters I'm expecting. When I call this function, the hashtable I pass in will be automatically stored in the $userinfo variable.
Here's another way to write this:
Function CreateUser($userinfo) {
}
This syntax has exactly the same effect. However, the shell uses the first version internally so I tend to use that version myself. I also think using the param block makes the function easier to read down the road.
Inside the function, I need to run only a single command: New-Mailbox. This actually does more than just create a new mailbox; it also creates a new user in Active Directory. I need to know a couple of pieces of information up front, including the mailbox database where I want the mailbox to be stored. The basic syntax looks like this (which is really all a long, single command line):
New-Mailbox –UserPrincipalName don@concentratedtech.com
  -alias DonJ
  –database "Storage Group 1\Mailbox Database 1"
  –name Don Jones
  –organizationalUnit Users
  –password $password
  –FirstName Don
  –LastName Jones
  –DisplayName "Don Jones"
  –ResetPasswordOnNextLogon $true
You can see that I need to come up with a password for the new user and decide what OU (organizational unit) to put the user into. (In this example, "Users" is technically a container, not an OU, but the command will accept either.) Assuming my ProvisionInputCSV function has been modified to produce everything I need, I can expect the $userinfo variable to contain the following minimum information:
  • UPN (user principal name)
  • OU (destination OU)
  • MailDatabase
  • givenName (first name)
  • sn (last name)
  • samAccountName (which I'll use as the 'alias')
That gives me the following function:
Function CreateUser {
  Param($userinfo)

New-Mailbox –UserPrincipalName $userinfo['upn']
  -alias $userinfo['samAccountName']
  –database $userinfo['mailboxDatabase']
  –name ($userinfo['givenName'] + ' ' + $userinfo['sn'])
  –organizationalUnit $userinfo['ou']
  –password 'P@ssw0rd!' –FirstName ($userinfo['givenName']
  –LastName $userinfo['sn'])
  –DisplayName ($userinfo['givenName'] + ' ' + $userinfo['sn'])
  –ResetPasswordOnNextLogon $true
}
(Again, the command should be a single line, but I've broken it up here to make it easier to digest.) Combined with my Provision and ProvisionInputCSV functions, this function will create the new users and make them mailbox-enabled.
A few interesting things are worth noting here:
  • Nothing in the command is case sensitive, although Active Directory preserves whatever casing you use in things like names.
  • I'm giving the -name and -displayName parameters an expression as a value. I enclose the expression in (parentheses) to make sure the shell evaluates the expression and passes the results to the parameter. This technique lets me use the givenName and sn attributes to construct a full name.
  • I've hard-coded a temporary password rather than making one up. Making up passwords requires a way to communicate the passwords to users, and that's beyond the scope of this discussion.
  • The variable $true is built into the shell and represents the Boolean value True (the opposite of False).

Missing Exchange?
Windows PowerShell won't help in creating a mailbox-enabled user if you're not using Exchange Server 2007. However, you can still create the basic user account in Active Directory. To do so, you need the free Active Directory cmdlets for Windows PowerShell, which are available at www.quest.com/powershell. Install those on the computer where you'll be running your script and add the snap-in to the shell by running this:
Add-PSSnapin Quest.ActiveRoles.ADManagement
Note that the snap-in name includes the name of a commercial Quest product (ActiveRoles Server), but you don't need that product in order to use the cmdlets in the snap-in.
You'll use the New-QADUser command, running something like this in the CreateUser function:
New-QADUser –samAccountName $userinfo['samAccountName']
  –parentContainer $userinfo['OU']
  –FirstName $userinfo['givenName']
  –LastName $userinfo['sn']
  –Name ($userinfo['givenName'] + ' ' + $userinfo['sn'])
  –UserPrincipalName $userinfo['UPN']
  –displayName ($userinfo['givenName'] + ' ' + $userinfo['sn'])
  –userPassword "P@ssw0rd!" | Enable-QADUser
Note that I've piped the output of this command—which consists of the newly-created user—to a second command that enables the user account. You can decide whether that design is appropriate for your environment.
The New-QADUser command has dozens of parameters you can use to set additional Active Directory attributes, but I'm leaving this command as-is to make it more parallel with the Exchange Server version. I'll populate additional attributes in the last part of this script.

Options, Options, Options
Even if you run Exchange Server 2007, you might still prefer using New-QADUser to create user accounts. Doing so certainly gives you more flexibility, as this command can handle any directory attribute. And New-QADUser will make your script compatible with any environment, whether or not Exchange Server 2007 is in use.
If you have Exchange Server 2007, you could use an alternate syntax of New-Mailbox to create a mailbox and link it to the just-created Active Directory account. You'll find that Windows PowerShell typically offers several different ways to do something—the right approach is typically the one that works best for you with the least amount of learning involved.
Next month, I'll take this provisioning function a bit further by having it create home folders for users and apply the right Access Control Lists (ACLs) to those folders. ACLs are especially tricky to work with, and you might be surprised by the Windows PowerShell technique that can help you get the job done most easily.

Don Jones is a cofounder of Concentrated Technology, where he blogs weekly about Windows PowerShell, SQL Server, App-V, and other topics. Contact him through his Web site.


Geek of All Trades Simple Clustering with Hyper-V
Greg Shields

Contents
Clustering Hyper-V
Working with Clustered Hyper-V
Increasing Your Availability
In the early days of my IT career, I used to answer the oft-asked “What do you do for a living?” with “Oh, I fi x computers.” But for years, something bothered me about that statement. It felt too negative, as if computers were always breaking and the security of my living relied on that fact. So, after answering that question over and over, it finally hit me. My real priority as IT technician and systems administrator wasn't necessarily to fix computers—it was to keep them running.
That priority hasn't changed. In fact, as computing becomes more and more critical to the daily operations of most businesses, being able to fix computers has grown less important. What's become much more critical is that the computers keep on working. Nowhere is this more important than in your job as a jack-of-all-trades IT professional. Struggling to keep computers running with a minimum of resources and a shrinking budget often requires a healthy measure of duct-tape and bailing-twine.
With the right combinations of technology, however, you can inexpensively increase the availability of your servers. By converting your physical servers to virtual ones, you immediately get extra capabilities that make them less likely to go down and easier to bring back up when they do:
  • Snapshots enable you to go back in time when a software update or configuration change blows up an otherwise healthy server.
  • Virtual hard disks consolidate the thousands of files that comprise a Windows server into a single file for backups, which significantly improves the reliability of those backups.
  • Volume Shadow Copy Service (VSS) support, which is natively available in Hyper-V, means that applications return from a restore with zero loss of data and immediately ready for operation.
  • Migration capabilities improve planned downtime activities by providing a mechanism for relocating the processing of virtual machines to new hosts with little to no disruption in service.
  • Failover clustering means that the loss of a virtual host automatically moves virtual machines to new locations where they can continue doing their job.
The hard part in all of this is figuring out exactly what technologies you need to deploy—and which you can afford. There is a direct relationship between the levels of redundancy you can deploy in an IT environment and the cost. With that in mind, adding a technology like Windows Server 2008 Failover Clustering to Hyper-V might get overlooked because of the perceived complexity. Yet you can implement not necessarily highly reliable, but "reasonably reliable" virtualization for less than you'd think.
To that end, let's explore one very simple solution that integrates these two technologies. Be aware that this implementation doesn't immediately bestow the highest levels of availability, but it is the simplest solution you can deploy today. Once you understand the basics, I'll suggest some ways you can add even more, with added cost and complexity, of course. Let your budget be your guide.

Clustering Hyper-V
To get started, take a look at the article called "Hyper-V Step-by-Step Guide: Hyper-V and Failover Clustering." I'll describe the high-level steps necessary to implement a clustered configuration. For the detailed click-by-click instructions, refer to this document.
Implementing redundancy with Hyper-V involves clustering its services across two or more computers. Each computer must have installed a copy of the x64 version of Windows Server 2008 Enterprise Edition, and each should include a minimum of two network cards per server plus one or more network cards for iSCSI. In this very simple example, you will use the network to connect these two servers to a third server or device that hosts a set of shared iSCSI disks. iSCSI is used here because it is easy to set up, requires little to no additional equipment or specialized training, and runs across your existing network.
On those disks you will store the files for your virtual machines so they can be directly accessed by both hosts. As shown in Figure 1, two network cards connect each server to two different networks (one for the cluster's private network, the other for your production network), while the third connects to the iSCSI disks. In this example, your iSCSI connections are on different networks from your production traffic. Segregating iSCSI from regular network traffic is important protect the traffic of each type from the network congestion of the other.
Figure 1 At minimum, six network connections are needed to start a cluster.
The first step in preparing this environment is validating that your servers can in fact run the Hyper-V role. Hyper-V has three hardware requirements:
  • The server must be capable of running the x64 version of the Windows OS.
  • The server's processor must be equipped with hardware Data Execution Prevention (   DEP    ).
  • The server's processor capabilities must include virtualization extensions.
These three requirements are all determined by the processor capabilities available on the server, and there are two ways to determine whether your servers meet these needs. For server-class equipment purchased from a server hardware vendor, you can verify your server's make and model against the Windows Server Catalog. There, click on the Hyper-V link in the left column to narrow the results to just those servers that have been tested and found to successfully support Hyper-V.
If your server is not in this list or is your own design, you can download the Securable test tool. This tiny tool tests your processor's capabilities against these three requirements. If your maximum bit length is 64 and the tool reports Yes for both hardware DEP and hardware virtualization, you've got what you need.
The next step in the process is to create two disks that are exposed to your network via iSCSI. The first disk will be 512MB in size for your witness disk, while the second disk will be of a size that is needed for the full installation of one virtual machine. The exact steps to do this will be different depending on what product you use to host the disks. If your environment has iSCSI hardware in place, simply create and expose two new disks through its interface. If you don't have iSCSI hardware, iSCSI target software exists on the Internet that you can install to a third server in your environment for this purpose.
If you need to use software-based iSCSI, look carefully at the features available. Microsoft clustering requires iSCSI to support SCSI Primary Commands-3, specifically the support of Persistent Reservations. Most for-cost iSCSI software currently supports this capability, but there is very little support for it in most open source software packages.
One inexpensive and easy-to-use software package is the StarWind iSCSI Target from Rocket Division. You need to use one of the paid versions for clustering because the no-cost Personal Edition does not support connecting multiple servers to the same disk. Other options exist, but at present there are no freeware downloads from Microsoft that can help.
Once you have created and exposed these two disks to the network via iSCSI, follow these steps on each computer that will be a part of the cluster to connect to the disks:
  • Navigate to Administrative Tools | iSCSI Initiator. On your first attempt to connect to the tool, you will be prompted twice to enable the Microsoft iSCSI Service and unblock its firewall connection. Choose Yes for both to start the console.
  • To find your disks, go to the Discovery tab and click Add Portal. In the resulting screen, enter the IP address for the iSCSI disk and click the Advanced button. Next, change the local adapter to Microsoft iSCSI Adapter. Next, change the source IP to the IP address for the network card you have selected for your disk connection. Firewalls can prevent the initiator from accessing the target if an error in connecting occurs at this point.
  • If you have successfully connected to your disks via the network, you will see the name of each disk under the Targets tab. The next step is to log on to each disk. Do this by clicking the disk and selecting Log On. On the resulting screen, check the box for Automatically restore this connection when the computer starts and click the Advanced button. Again, set the local adapter to Microsoft iSCSI Adapter, the source IP to the IP address for the correct network card, and the target portal to the IP address and port for the iSCSI device. Repeat this step for each disk. A successful logon will look similar to Figure 2.
  • Under the Favorite Targets tab, click the Refresh button. The disk names should appear in the list of favorite targets.
  • Under the Volumes and Devices tab, click the Autoconfigure button. If everything has been done correctly to this point, you will see references to each disk appear in the box.
Figure 2 A successful connection to iSCSI disks
After connecting to your disks, you need to initialize and format them to make them ready for use. You do this in Server Manager under the Storage | Disk Management node. This process must be run on each computer, but must be done only when the other computer is powered down. Do this now and launch Disk Management. Now right-click each disk and bring the disk online. Then right-click again and initialize the disk. Finally, right-click a third time to create a new Simple Volume, formatting the disk as NTFS and assigning it a drive letter. For this example, we will assign Q: to the quorum drive and F: to the drive for our virtual machine.
Power down the first computer and start the second. On the second computer, ensure that the disk is assigned the same drive letter as was assigned on the first computer. When this is complete, both computers can be turned on. Your disks are now ready for clustering.
The next step in this process is to install the Hyper-V role to your computer as well as the Failover Clustering feature. You do both from Server Manager. Prior to installing the Hyper-V role, upgrade your server's Hyper-V code to the RTM version by installing update 950050 first. You will find it in the Windows Update control panel as a Recommended update. Installing these components requires a reboot.
Two steps remain. First, validate your cluster by navigating to Administrative Tools | Failover Cluster Manager. There, click the link to Validate a configuration. This step ensures that everything you've configured is correct to prepare for a successful cluster creation. If your configuration passes the validation tests, click the link to Create a cluster. The wizard will ask for the names of the cluster members as well as an IP address to be used for cluster communication. This address will be an unused IP address on your network and should not be any of the IP addresses you've used to this point in this procedure. Complete the wizard to create the cluster.

Working with Clustered Hyper-V
When you use Hyper-V alone and without System Center Virtual Machine Manager, the process of managing your clustered virtual hosts happens in two places. You create and work with virtual machines from within the Hyper-V Manager. The configuration of both clustering itself and the necessary cluster resources is done from within the Failover Cluster Manager.
To create a new virtual machine, open the Hyper-V Manager and select New | Virtual Machine. Answer the questions in the wizard and ensure that you store that virtual machine's disk files on the F: drive. All files associated with a clustered virtual machine must always be stored on shared disks. Once you've created your virtual machine, power it down and launch the Failover Cluster Manager. This next step configures the resources necessary to make the virtual machine cluster aware. Right-click Services and Applications, and select Configure a Service or Application. Select the Virtual Machine item in the menu, and click next. Your screen will look similar to Figure 3. There, select the machine you want to be made cluster aware. You will be unable to select machines that are powered on or whose disk files are not stored on shared drives. Complete the wizard to finish this process.
Figure 3 Selecting a machine to be made cluster aware
You now have a virtual machine that is cluster aware, and this machine now benefits from the high availability provided by Windows Server 2008 Failover Clustering. This means that if the host of the virtual machine goes down or if the machine experiences any problems, it will automatically restart itself on the other cluster node. Figure 4 shows an example of how your virtual machine appears in the Failover Cluster Manager.
Figure 4 A virtual machine hosted in Failover Cluster Manager
If you need to perform maintenance on the cluster host, you can now also quick migrate that virtual machine to the other host by right-clicking its instance in Services and Applications and selecting Move this service or application to another node. In the same location, the properties of this virtual machine group provide a place to instruct the cluster which nodes are preferred for hosting the machine. In Windows failover clusters, a preferred owner corresponds to the cluster nodes on which you prefer the machine to be hosted in the event of a failover. This choice has greater impact on clusters with more than two nodes but can be a way of manually load-balancing virtual machines across multiple nodes.

Increasing Your Availability
These steps are presented to show how easy adding extra availability to your servers can be. If you have two copies of Windows Server 2008 Enterprise Edition lying around and either an iSCSI hardware device or software installation, these few steps will build a basic Windows cluster for supporting high availability.
Yet, as I explained in the beginning of this column, this is only the first step. There are other layers of technology you will likely want to add to this architecture to gain even more availability. Each layer adds a little more resiliency to your environment—but also adds a little more cost and complexity. Depending on your needs and your budget, consider the following as potential add-ons:
Server-class equipment. The use of equipment that is not listed in the Windows catalog can impact supportability and may not best meet the needs of your virtual machines. Moving to tested and supported server-class equipment will ensure full support in the case of a problem.
Network card redundancy. With the minimum three network cards connecting each cluster node to its disks and networks, the loss of any can mean the loss of that server as a cluster node. Thus, adding more network cards for multipath iSCSI or network redundancy should be your first choice for more hardware. And, remember, NICs are cheap Many servers these days can inexpensively support four, six, or even more network cards, so this simple addition will net a lot of resiliency bang for your budgetary buck.
Network path redundancy. The use of iSCSI across your production network can have a significant impact on its performance. The reverse holds true as well because traditional network traffic can have an impact on a cluster node's connection to its disk. Isolating iSCSI traffic to its own network path isolates that traffic to its own network segment, ensuring its full availability as network conditions change.
Using hardware-based iSCSI. Although iSCSI target software works, it can pose a problem. The loss of the server hosting your iSCSI disks means the loss of the entire cluster. Because servers require reboots from time to time, hardware-based iSCSI tends to assure higher levels of availability.
iSCSI redundancy. Whether you use hardware or software for your iSCSI connection, either is still a single point of failure if it is not made redundant. Look for solutions that provide redundancy for your iSCSI disks.
System Center Virtual Machine Manager. Using Hyper-V alone across multiple systems can be challenging. There is a lot to be configured and many steps involve manual intervention. Adding the extra management capabilities gained through System Center Virtual Machine Manager makes the process of managing your virtualization environment much easier, which reduces the potential of mistakes and costly downtime.
Additional cluster nodes. Windows Server 2008 Enterprise Edition x64 can support up to 16 nodes in a failover cluster. Adding additional nodes to a cluster requires running through the steps described here on an additional server and adding that server to the cluster. Once you have that extra server, you have another location where virtual machines can be failed over when problems occur. This is especially handy when you consider how the loss of a single node can affect virtual machine performance. For example, if you store four virtual machines on each cluster node of a two-node cluster, the loss of one node moves all eight virtual machines onto a single node. Collocating this many machines onto one server affects the performance of all virtual machines.
Multi-site Clusters. Traditional failover clusters are designed to protect against the loss of a host, but they don't do well in the case of a disaster. To protect against the loss of every cluster node at once, consider building a geocluster. A much-more complex undertaking, geoclusters span subnets and leverage replication technology for keeping far-removed data stores in sync. With a geocluster, if you lose one entire site, your virtual machines will automatically migrate to another site with only a small outage.
All these options add more cost to your environment and many add significant levels of complexity. But all of them take a highly-available architecture and make it even more so.
If you've got thoughts about clustering and Hyper-V, or you've built an inexpensive virtual cluster you'd like to share, drop me a line at gshields@concentratedtech.com.

Greg Shields, MVP, is a partner at Concentrated Technology. Get more of Greg's Jack-of-all-Trades tips and tricks at www.ConcentratedTech.com.


Hey, Scripting Guy! Creating CAB Files with Windows PowerShell
The Microsoft Scripting Guys

Contents
Expanding Cab Files
Using the Makecab.exe Utility
One of my favorite movies ("Johnny Mnemonic") was about a guy who was a data smuggler in the future. He would travel to various countries and meet with clients. He had a jack surgically implanted into the base of his head, and the customers would plug their Zune-looking device into the jack and transfer massive amounts of data into his brain. Because this data smuggler had his brain partitioned and was evidently using NTFS file permissions, he did not have rights to the data-smuggling portion of his brain. The data was therefore safe—even from the smuggler's own wandering thoughts. In one scene, the client wanted him to courier a massive amount of data, but evidently our smuggler did not have a large enough capacity to carry so much data. So what did he do? He compressed his brain—I love it!
Data compression is not only good fodder for movies, it is equally useful for network administrators. We all know, and most likely use, various file compression utilities. I use them every week in sending chapters of my new book by e-mail to my editor at Microsoft Press. I do this not so much because I am bandwidth constrained, but because we have e-mail quotas that restrict the size of the e-messages we can store, send, and receive. I have also used file compression utilities when I needed to archive files from my laptop to one of the several portable disks floating around my office.
When I used to travel around the world teaching scripting workshops, I would be asked routinely, "How can I compress files through scripting?" And I'd answer, "You need to buy a third-party utility that supports command-line switches." One day, I was reading through the registry, and I ran across a COM object named (interestingly enough) makecab.makecab. Hmm, what do you suppose that object does? Yep, you're right. It lets you make cabinet (.cab) files—highly compressed files used by various applications for packaging and deployment. But there is nothing to stop an enterprising network administrator or scripting guru from appropriating these tools for themselves. And that's just what we'll do. Let's start with the script in Figure 1.
Figure 1 CreateCab.ps1
Param(
  $filepath = "C:\fso",
  $path = "C:\fso\aCab.cab",
  [switch]$debug
  )
Function New-Cab($path,$files)
{
 $makecab = "makecab.makecab"
 Write-Debug "Creating Cab path is: $path"
 $cab = New-Object -ComObject $makecab
 if(!$?) { $(Throw "unable to create $makecab object")}
 $cab.CreateCab($path,$false,$false,$false)
 ForEach ($file in $files)
 {
 $file = $file.fullname.tostring()
 $fileName = Split-Path -path $file -leaf
 Write-Debug "Adding from $file"
 Write-Debug "File name is $fileName"
 $cab.AddFile($file,$filename)
 }
 Write-Debug "Closing cab $path"
 $cab.CloseCab()
} #end New-Cab

# *** entry point to script ***
if($debug) {$DebugPreference = "continue"}
$files = Get-ChildItem -path $filePath | Where-Object { !$_.psiscontainer }
New-Cab -path $path -files $files

CreateCab.ps1 –filepath C:\fso1
The first thing we need to do is to create some command-line parameters using the Param statement. The Param statement must be the first noncommented line in the script. When the script is run from within the Windows PowerShell console or from within a script editor, the command-line parameters are used to control the way the script executes. In this way, you don't have to edit the script each time you want to create a .cab file from a different directory. You need only supply a new value for the –filepath parameter, as shown here:
The good thing about command-line parameters is that they use partial parameter completion, which means you need to supply only enough of the parameter for it to be unique. Therefore, you could use a command-line syntax such as this:
CreateCab.ps1 –f c:\fso1 –p c:\fso2\bcab.cab –d
This syntax would search the c:\fso1 directory and obtain all the files. It would then create a cabinet file named bcab.cab in the fso2 folder off the c:\ drive. It would also produce debugging information while it was running. Note that the –debug parameter is a switched parameter, which means that it affects the script only when it is present. Here's the relevant section of the CreateCab.ps1 script:
Param(
  $filepath = "C:\fso",
  $path = "C:\fso\aCab.cab",
  [switch]$debug
  )
Now we'll create the New-Cab function, which accepts two input parameters, –path and –files:
Function New-Cab($path,$files)
You can assign the program ID, makecab.makecab to a variable named $makecab, which will make the script a bit easier to read. This is also a good place to put the first Write-Debug statement:
{
 $makecab = "makecab.makecab"
 Write-Debug "Creating Cab path is: $path"
Next we will create the COM object:
$cab = New-Object -ComObject $makecab
A bit of error checking is in order, which can be accomplished using the $? automatic variable:
if(!$?) { $(Throw "unable to create $makecab object")}
If no errors occurred during the attempt to create the makecab.makecab object, you can use the object contained in the $cab variable and call the CreateCab method:
$cab.CreateCab($path,$false,$false,$false)
Once the .cab file has been created, you can add files to it by using the ForEach statement:
ForEach ($file in $files)
 {
 $file = $file.fullname.tostring()
 $fileName = Split-Path -path $file -leaf
After you have turned the full filename into a string and removed the directory information using the Split-Path cmdlet, you include another Write-Debug statement to let the user of the script know what's going on, this way:
Write-Debug "Adding from $file"
 Write-Debug "File name is $fileName"
Next you add the file to the cabinet file:
$cab.AddFile($file,$filename)
 }
 Write-Debug "Closing cab $path"
To close the cabinet file, you use the CloseCab method:
$cab.CloseCab()
} #end New-Cab
Now we'll go to the entry point of the script. First we check to see whether the script is being run in debug mode by looking for the $debug variable. If the $debug variable is not present, you don't need to do anything. If it is present, you need to set the value of the $DebugPreference variable to continue, which allows the Write-Debug statements to be printed on the screen. By default, the $DebugPreference is set to silentlycontinue, which means skip past the command without doing anything. Here's the code:
if($debug) {$DebugPreference = "continue"}
Now you need to obtain a collection of files. To do this, you can use the Get-ChildItem cmdlet:
$files = Get-ChildItem -path $filePath |
 Where-Object { !$_.psiscontainer }
Then you pass the collection to the New-Cab function as shown here:
New-Cab -path $path -files $files
When you run the CreateCab.ps1 script in debug mode, you'll see output as shown in Figure 2.
Figure 2 Running CreateCab.ps1 in debug mode

Expanding Cab Files
You can't use the makecab.makecab object to expand the cabinet file because it doesn't have an expand method. Nor can you use the makecab.expandcab object because it doesn't exist. But the ability to expand a cabinet file is inherent in the Windows shell, so you can use the shell object. To access the shell, you can use the Shell.Application COM object, as shown in the ExpandCab.ps1 script in Figure 3.
Figure 3 ExpandCab.ps1
Param(
  $cab = "C:\fso\acab.cab",
  $destination = "C:\fso1",
  [switch]$debug
  )
Function ConvertFrom-Cab($cab,$destination)
{
 $comObject = "Shell.Application"
 Write-Debug "Creating $comObject"
 $shell = New-Object -Comobject $comObject
 if(!$?) { $(Throw "unable to create $comObject object")}
 Write-Debug "Creating source cab object for $cab"
 $sourceCab = $shell.Namespace($cab).items()
 Write-Debug "Creating destination folder object for $destination"
 $DestinationFolder = $shell.Namespace($destination)
 Write-Debug "Expanding $cab to $destination"
 $DestinationFolder.CopyHere($sourceCab)
}

# *** entry point ***
if($debug) { $debugPreference = "continue" }
ConvertFrom-Cab -cab $cab -destination $destination
First the script creates the command-line parameters, much as CreateCab.ps1 did:
Param(
  $cab = "C:\fso\acab.cab",
  $destination = "C:\fso1",
  [switch]$debug
  )
Next it creates the ConvertFrom-Cab function, which accepts two command-line parameters, one that contains the .cab file, and one that contains the destination to expand the files:
Function ConvertFrom-Cab($cab,$destination)
Now you create an instance of the Shell.Application object, a very powerful object with a number of useful methods. Figure 4 shows the members of the Shell.Application object.
Figure 4 Members of the Shell.Application object
Name | MemberType | Definition
AddToRecent | Method | void AddToRecent (Variant, string)
BrowseForFolder | Method | Folder BrowseForFolder (int, string, int, Variant)
CanStartStopService | Method | Variant CanStartStopService (string)
CascadeWindows | Method | void CascadeWindows ()
ControlPanelItem | Method | void ControlPanelItem (string)
EjectPC | Method | void EjectPC ()
Explore | Method | void Explore (Variant)
ExplorerPolicy | Method | Variant ExplorerPolicy (string)
FileRun | Method | void FileRun ()
FindComputer | Method | void FindComputer ()
FindFiles | Method | void FindFiles ()
FindPrinter | Method | void FindPrinter (string, string, string)
GetSetting | Method | bool GetSetting (int)
GetSystemInformation | Method | Variant GetSystemInformation (string)
Help | Method | void Help ()
IsRestricted | Method | int IsRestricted (string, string)
IsServiceRunning | Method | Variant IsServiceRunning (string)
MinimizeAll | Method | void MinimizeAll ()
NameSpace | Method | Folder NameSpace (Variant)
Open | Method | void Open (Variant)
RefreshMenu | Method | void RefreshMenu ()
ServiceStart | Method | Variant ServiceStart (string, Variant)
ServiceStop | Method | Variant ServiceStop (string, Variant)
SetTime | Method | void SetTime ()
ShellExecute | Method | void ShellExecute (string, Variant, Variant, Variant, Variant)
ShowBrowserBar | Method | Variant ShowBrowserBar (string, Variant)
ShutdownWindows | Method | void ShutdownWindows ()
Suspend | Method | void Suspend ()
TileHorizontally | Method | void TileHorizontally ()
TileVertically | Method | void TileVertically ()
ToggleDesktop | Method | void ToggleDesktop ()
TrayProperties | Method | void TrayProperties ()
UndoMinimizeALL | Method | void UndoMinimizeALL ()
Windows | Method | IDispatch Windows ()
WindowsSecurity | Method | void WindowsSecurity ()
Application | Property | IDispatch Application () {get}
Parent | Property | IDispatch Parent () {get}
Because you will want to use the name of the COM object more than once, it is a good practice to assign the program ID of the COM object to a variable. You will be able to use the string with the New-Object cmdlet, as well as when providing feedback to the user. Here's the line of code that assigns the Shell.Application program ID to a string.
{
 $comObject = "Shell.Application"
To provide feedback, you can use the Write-Debug cmdlet with a message that you are attempting to create the Shell.Application object:
Write-Debug "Creating $comObject"
Next we'll actually create the object:
$shell = New-Object -Comobject $comObject
Then we test for errors. To do this, you can use the automatic variable $?, which tells you if the last command completed successfully. It's a Boolean true/false. You can use this fact to simplify the coding. You use the not operator, !, in conjunction with an if statement. If the variable is not true, you use the Throw statement to raise an error and halt execution of the script, this way:
if(!$?) { $(Throw "unable to create $comObject object")}
If the script successfully creates the Shell.Application object, we provide some feedback:
Write-Debug "Creating source cab object for $cab"
The next step in the operation is to connect to the .cab file. To do this, you can use the Namespace method from the Shell.Application object. This is another important step, so it makes sense to use another Write-Debug statement as a progress indicator for the user:
$sourceCab = $shell.Namespace($cab).items()
 Write-Debug "Creating destination folder object for $destination"
Now we connect to the destination folder. To do this, you use the Namespace method, then another Write-Debug statement to let the user know which folder was actually connected to:
$DestinationFolder = $shell.Namespace($destination)
 Write-Debug "Expanding $cab to $destination"
With all that preparation out of the way, the actual command for expanding the cabinet file is somewhat anticlimactic. You use the CopyHere method from the folder object that is stored in the $DestinationFolder variable. You give the reference to the .cab file that is stored in the $sourceCab variable as the input parameter this way:
$DestinationFolder.CopyHere($sourceCab)
}
The starting point to the script does two things. First, it checks for the presence of the $debug variable. If $debug is present, it sets the $debugPreference to continue to force the Write-Debug cmdlet to print out messages to the console window. Second, it calls the ConvertFrom-Cab function and passes the path to the cab file from the –cab command-line parameter and the destination for the expanded files from the –destination parameter:
if($debug) { $debugPreference = "continue" }
ConvertFrom-Cab -cab $cab -destination $destination
When you run the ExpandCab.ps1 script in debug mode, you will see output similar to that in Figure 5.
Figure 5 Running ExpandCab.ps1 in debug mode

Using the Makecab.exe Utility
If you run these two scripts on Windows Server 2003 or Windows XP, you won't have any problems, but the makecab.makecab COM object doesn't exist on Windows Vista or later. This misfortune doesn't stop the determined scripter, however, because you can always use the makecab.exe utility from the command line. To do so, you can use the CreateCab2.ps1 script, shown in Figure 6.
Figure 6 CreateCab2.ps1
Param(
  $filepath = "C:\fso",
  $path = "C:\fso1\cabfiles",
  [switch]$debug
  )
Function New-DDF($path,$filePath)
{
 $ddfFile = Join-Path -path $filePath -childpath temp.ddf
 Write-Debug "DDF file path is $ddfFile"
 $ddfHeader =@"
;*** MakeCAB Directive file
;
.OPTION EXPLICIT
.Set CabinetNameTemplate=Cab.*.cab
.set DiskDirectory1=C:\fso1\Cabfiles
.Set MaxDiskSize=CDROM
.Set Cabinet=on
.Set Compress=on
"@
 Write-Debug "Writing ddf file header to $ddfFile"
 $ddfHeader | Out-File -filepath $ddfFile -force -encoding ASCII
 Write-Debug "Generating collection of files from $filePath"
 Get-ChildItem -path $filePath |
 Where-Object { !$_.psiscontainer } |
 ForEach-Object `
 {
 '"' + $_.fullname.tostring() + '"' |
 Out-File -filepath $ddfFile -encoding ASCII -append
 }
 Write-Debug "ddf file is created. Calling New-Cab function"
 New-Cab($ddfFile)
} #end New-DDF

Function New-Cab($ddfFile)
{
 Write-Debug "Entering the New-Cab function. The DDF File is $ddfFile"
 if($debug)
 { makecab /f $ddfFile /V3 }
 Else
 { makecab /f $ddfFile }
} #end New-Cab

# *** entry point to script ***
if($debug) {$DebugPreference = "continue"}
New-DDF -path $path -filepath $filepath
As with the other scripts, CreateCab2.ps1 first creates a couple of command-line parameters:
Param(
  $filepath = "C:\fso",
  $path = "C:\fso1\cabfiles",
  [switch]$debug
  )
When you run the script with the –debug switch, the output will be similar to what is shown in Figure 7.
Figure 7 Running CreateCab2.ps1 in debug mode
Next the script creates the New-DDF function, which creates a basic .ddf file that is used by the MakeCab.exe program to create the .cab file. The syntax for these types of files is documented in the Microsoft Cabinet Software Development Kit. After you use the function keyword to create the New-DDF function, you use the Join-Path cmdlet to create the file path to the temporary .ddf file. You could concatenate the drive, the folder, and the filename together, but this could be a cumbersome and error-prone operation. As a best practice, you should always use the Join-Path cmdlet to build your file paths, this way:
Function New-DDF($path,$filePath)
{
 $ddfFile = Join-Path -path $filePath -childpath temp.ddf
To provide a bit of feedback to the user if the script is run with the—debug switch, use the Write-Debug cmdlet as shown here:
Write-Debug "DDF file path is $ddfFile"
Now you need to create the first portion of the .ddf file. To do this, you can use an expanding here-string, which means you won't have to concern yourself with escaping special characters. As an example, comments in a .ddf file are prefaced with a semicolon, which is a reserved character in Windows PowerShell. If you were to try to create this text without a here-string, you would need to escape each of the semicolons to avoid compile-time errors. By using an expanding here-string, you can take advantage of the expansion of variables. A here-string begins with the ampersand and a quotation mark, and ends with a quotation mark and an ampersand:
$ddfHeader =@"
;*** MakeCAB Directive file
;
.OPTION EXPLICIT
.Set CabinetNameTemplate=Cab.*.cab
.set DiskDirectory1=C:\fso1\Cabfiles
.Set MaxDiskSize=CDROM
.Set Cabinet=on
.Set Compress=on
"@
Next you may want to add feedback via the Write-Debug cmdlet:
Write-Debug "Writing ddf file header to $ddfFile"
Now we come to the part that could cause some problems. The .ddf file must be pure ASCII. By default, Windows PowerShell uses Unicode. To ensure that you have an ASCII file, you must use the Out-File cmdlet. Most of the time, you can avoid using Out-File by using the file redirection arrows, but this is not one of those occasions. Here's the syntax:
$ddfHeader | Out-File -filepath $ddfFile -force
-encoding ASCII
You probably want to provide some more debug information with Write-Debug before you gather your collection of files via the Get-ChildItem cmdlet:
Write-Debug "Generating collection of files from $filePath"
 Get-ChildItem -path $filePath |
It is important to filter out folders from the collection because MakeCab.exe is not able to compress folders. To do this, use the Where-Object with a not operator that says the object is not a container:
Where-Object { !$_.psiscontainer } |
Next, you need to work with each file as it comes across the pipeline. To do this, use the ForEach-Object cmdlet. Because ForEach-Object is a cmdlet, as opposed to a language statement, the curly brackets must be on the same line as the ForEach-Object cmdlet name. The problem with this is that it has a tendency to bury the curly brackets in the code. As a best practice, I like to line up the curly brackets unless the command is very short like the previous Where-Object command. To do this, however, requires the use of the line continuation character (the backtick). I know some developers who avoid line continuation like the plague, but I think lining up curly brackets is more important because it makes the code easier to read. Here is the beginning of the ForEach-Object cmdlet:
ForEach-Object `
Because the .ddf file used by MakeCab.exe is ASCII text, you will need to convert the fullname property of the System.IO.Fileinfo object returned by the Get-ChildItem cmdlet to a string. In addition, because you may have files with spaces in their names, it makes sense to enclose the file fullname value in a set of quotation marks:
{
 '"' + $_.fullname.tostring() + '"' |
You then pipeline the filenames to the Out-File cmdlet, making sure to specify the ASCII encoding and to use the –append switch to avoid overwriting everything else in the text file:
Out-File -filepath $ddfFile -encoding ASCII -append
 }
Now you can provide another update to the debug users, and then call the New-Cab function:
Write-Debug "ddf file is created. Calling New-Cab function"
 New-Cab($ddfFile)
} #end New-DDF
When you enter the New-Cab function, you can also supply that information to the user:
Function New-Cab($ddfFile)
{
 Write-Debug "Entering the New-Cab function. The DDF File is $ddfFile"
Next, if the script is run with the –debug switch, you can use MakeCab.exe's /V parameter to provide detailed debugging information (3 is full verbosity; 0 is none). If the script isn't run with the –debug switch, you don't want to clutter the screen with too much information so you should stick with the utility's default:
if($debug)
 { makecab /f $ddfFile /V3 }
 Else
 { makecab /f $ddfFile }
} #end New-Cab
The entry point to the script checks to see if the $debug variable is present. If it is, the $debugPreference automatic variable is set to continue, and debugging information will be displayed via the Write-Debug cmdlet. After that check has been performed, the New-DDF cmdlet is called with the two values supplied to the command line: the path and the filepath:
if($debug) {$DebugPreference = "continue"}
New-DDF -path $path -filepath $filepath
Well, that is about it for this month's article on compressing and uncompressing files. If you feel like your head is about to explode, you can try to plug your Zune into your ear and see if you can run one of these scripts to compress part of your brain. But…I'm telling you right now…it works only in the movies. Sorry. If you didn't lock up, you should stop by the TechNet Script Center and check out our daily "Hey, Scripting Guy!" article. See you there.

Ed Wilson, a well-known scripting expert, is the author of eight books, including Windows PowerShell Scripting Guide (2008) and Microsoft Windows PowerShell Step by Step (2007). Ed holds more than 20 industry certifications, including Microsoft Certified Systems Engineer (MCSE) and Certified Information Systems Security Professional (CISSP). In his spare time, he enjoys woodworking, underwater photography, and scuba diving. And tea.

Craig Liebendorfer is a wordsmith and longtime Microsoft Web editor. Craig still can't believe there's a job that pays him to work with words every day. One of his favorite things is irreverent humor, so he should fit right in here. He considers his greatest accomplishment in life to be his magnificent daughter.


Security Watch The Challenge of Information Security Management, Part 1
Jesper M. Johansson

Contents
Can We Get Rid of the Technology?
What We Really Protect
The Defender's Dilemma
Major Conflict
Keeping the Lights Off
When You Neglect Information Security
Accountability at the Government Level
Inappropriate Acceptance of Risk
A Blueprint
Wrapping Up
For a while now, I have been thinking about the broader challenge of Information Security ( InfoSec) management in an organization. It seems the older I get, the farther away I get from technology—or, rather, the more I realize technology is fundamentally not the solution to our problems. Technology, in so many ways, is the problem. In fact, InfoSec Management is almost exclusively about preventing the kinds of problems we get into with technology.
I originally thought it would be interesting to write a book on this topic, but I eventually decided that a series of essays would be more appropriate. And here's where these essays will live. Over the next year or so, I will use the Security Watch column to occasionally visit the topic of Information Security Management. In this first installment of the series, I look at the fundamental principles of InfoSec management.

Can We Get Rid of the Technology?
Technology, for good and for bad, is inevitable today. Technology empowers organizations to do more, faster and more efficiently. It also enables criminals to do the same thing. And as much as many of us in the technology fields would like to believe that technology exists for technology's own sake, it does not. Technology exists to store, process, and transmit data—data that can be turned into information. Data and the information we derive from it are our truly valuable resources.
Shouldn't we, as security professionals, then call ourselves "data security" professionals? Are we not more accurately engaged in protecting data? In many ways, this is true, but data is just a raw material. Certainly it has value on its own, but it is putting the data together and creating information from it that really provides value. Therefore, we work at a layer of abstraction slightly higher than data. I will use the term "information" (except when I really wish to speak about raw materials) because that is what we eventually turn the data into.
Many technology professionals work at layers of abstraction below data—the bits, bytes, and electron layers. We like technology for technology's own sake. It is very often why we went into the technology field in the first place. Technology is more comfortable for us than, say, people. However, as Information Security professionals, this is dangerous because we must have a far broader scope than just technology. While Information Security professionals must be experts in one or more technology domains, we must look at the whole ecosystem of technology, information, people, and processes—the four pillars of information security. We must, by necessity, focus on all four pillars. The truth is, however, that many people ignore one or more.
Several years ago, Microsoft embarked on a Trustworthy Computing campaign, proclaiming that security is people, process, and technology. Many were skeptical, saying that Microsoft was trying to shift focus from poor security technologies to factors it could not control. That was an unfair analysis, though, considering the security of Microsoft solutions, particularly in Windows XP SP2 and more recent operating systems, is actually quite good. Windows Vista and Windows Server 2008, and particularly Windows 7 and Windows Server 2008 R2, are exemplary in most respects when it comes to security and are head and shoulders above any competing solution.
Another, kinder, take was that the People, Process, and Technology mantra was simply stating that InfoSec professionals need to consider people and process, as well.
However, the people-process-technology triad failed to take into account the basic purpose of all we do. The process is how people use technology to turn data into information so that they can make informed decisions.

What We Really Protect
Then, why is it that I say that information, and the data it is derived from, is our only valuable resource? Clearly, a more technologically advanced company with better processes is more capable of achieving its objectives than one without. I think we can take that as a given. But from a security perspective, process and technology are problematic. Technology, by definition, adds complexity, and complexity breeds insecurity. Components of a system must interact, and the number of interactions is exponentially related to the number of components. Each of those interactions creates new avenues to transmit data, new mechanisms to process it, and new storage locations. All of those elements represent possible areas of weakness, as well as weak points that reduce our ability to visualize the technology and the problems it may pose. The less complex a technology, the easier it is to understand and secure. In fact, inability to easily visualize and understand the scope of technologies in use in an organization is one of the key causes of information security problems. As InfoSec professionals, we should consider reducing the amount of technology being used, not increasing it.
Processes are also complex, nebulous, and, in most cases, not sensitive in and of themselves. A process may provide competitive advantage, but a process cannot be copied per se. It is the information documenting the process that can be copied. A process is ephemeral—it disappears. Only by turning it into data can one copy it. This is similar to music. Music exists only temporally as we listen to it. To enable people to listen to music over and over again, Thomas Edison invented the phonograph (an ingenious piece of technology that permitted us to record the process of creating music as data so that people could listen to it again and again).
Now, more than one hundred years later, people of elastic morals and questionable virtue steal the data representation to feed the process of playing music. The music industry, in a failed attempt at thwarting this trend, turned to security technologies to obfuscate the process of re-creating music. The result is primarily to make it more difficult for its legitimate customers to enjoy their music. This technology, known as Digital Rights Management ( DRM ), is practically useless in combating theft, although it is very successful at combating customer satisfaction. Looking at this rationally, you realize that the problem is really about securing information, not processes. In some cases, as in music, the data essentially cannot be protected, necessitating its owners to simply accept that fact and move on.
This leads to a number of interesting observations. For example, can't you just obfuscate the process? Can't you make the process of creating value from data a secret? Can't you create a secret algorithm, as it were? Well, you can, but it would usually do no good. Claude Shannon put this very bluntly as "the enemy knows the system," a pronouncement commonly known as Shannon's Maxim.
What does this mean? The algorithm itself typically cannot remain secret. The process will become known and the competitive edge comes from the difficulty of implementing the process. Information Security must, once again, focus on protecting information as the core valuable asset. Securing processes is only interesting insofar as it provides protection for the information.

The Defender's Dilemma
This leads me to the "defender's dilemma." Your job is to protect all your data, including where it is exposed in all interactions between your components, under the assumption that the bad guys know the system. The bad guys, on the other hand, need to find only one way to compromise your system. They do not need multiple copies of your data. One copy will suffice, regardless of how they get it. Just as one copy of an MP3 file is enough to fuel the entire criminal ecosystem, so is a single copy of your data enough for your adversaries. Furthermore, theft using any one approach cannot be undone. There is no undo switch in cyberspace.
The defender, therefore, must protect all possible points. You must provide adequate protection for data at rest and in flight, on all devices, no matter who owns the devices. The more places information is stored, processed, and transmitted, the more difficult your job becomes. Complexity is the enemy of security!
Note the use of the term "possible." In many cases, such as with DRM technology, it is impossible to provide protection for technical reasons. In such cases, the defenders must find a way to live with a "lossy" system or prevent all access to the data. In general, data that is distributed to would-be adversaries can never be protected. If data needs to be protected, the only way to do so is to never distribute it.

Major Conflict
Because of our natural aversion to complexity, the InfoSec function is often perceived as a road-block. Those of us in the InfoSec field often, and largely correctly, view our jobs as preventing access to things.
The InfoSec group is where you go if you want your project to be killed. The evil security guys will always try to stop you from doing what you want. That is often true, and it is unfortunate, but not because the evil security guys shouldn't try to minimize complexity and stop bad ideas. It is unfortunate because it highlights a fundamental disconnect between the business and the security group.

Keeping the Lights Off
If you have ever studied for an information security exam, you've undoubtedly learned about the CIA triad—Confidentiality, Integrity, and Availability. It provides a way to describe the objectives in protecting information. We must provide confidentiality of the information against those who should not have access to it, integrity to ensure that the information is accurate, and availability for those who need access to the information. Consequently, some security groups consider much of their job to be about ensuring that the lights stay on, that availability is assured. Others assume the exact opposite approach and consider their jobs to be keeping the lights off—preventing anyone, including those with legitimate needs, from accessing the information.
I would argue that, if you have an appropriate partnership with the business, the security group can ignore the availability part of the triad. The business has other people who are more expert in availability and service level agreements. If security simply partners with them and makes sure that an adequate trade-off between confidentiality/integrity and availability is reached, the security function can consider availability to be of secondary importance. Security then becomes to some extent about ensuring the lights are off, and stay off; but while taking business needs into account. The business, left to its own devices, will ensure that the lights stay on. Security just needs to help the business ensure that only the correct lights stay on and others stay off. Security, in a sense, is then a reasonable white listing function.
This is one of the things I find most fascinating when working with other security people. We walk into a meeting with the business executives. The executives say "we need you to help secure our product." The security guys say "OK, tell me about the product." The business folks say "It's a widget"; at which point the security guys immediately start telling them how to secure widgets.
What is missing here? Sure, the security guys tried to learn what the product was. But what is the business objective? What is the business trying to achieve with this widget? How valuable is it to the business? How strategic is it? How important is it? How much risk is the business willing to accept to get it done? Do the business folks even want to build it?
The security group so rarely knows the business. Yet the security folks pretend to understand so they can tell the business how to do things. It is not our job as InfoSec professionals to tell the rest of the organization how to run a business. It is merely our job to inform the business as to the correct set of lights to turn on, and which ones must stay off, in accordance with the business' tolerance for risk and its needs. We support and advise the business on how to achieve its objectives with an acceptable level of risk—but the objectives are still owned by the business, not by the InfoSec group.

When You Neglect Information Security
I have finally arrived at risk. InfoSec is really information risk management. We manage the risk to our information assets. Or, at least we are supposed to. But in many cases, we simply are not allowed to. Security is a really hard sell because the upside is so unclear. What, after all, is the benefit of security? What is it that constitutes success? Is it merely "we did not get hacked this year?" You can never make that statement and be assured it is correct—you may have been hacked, but failed to notice. In fact, failing to invest enough money and time on InfoSec is a really good way to ensure you won't notice when you are hacked.
There are many potential consequences of neglecting InfoSec and, in some cases, an appalling lack of consequences. In some areas, neglect can cause you to lose your job or result in criminal charges. Failing to protect the privacy of children, for instance, is criminal in the United States under the Children's Online Privacy Protection Act (COPPA), and in other countries, such as Canada and Australia, under national statutes.
For a business that is based on customer trust, and where customers find it obvious that there is a risk, security is a cost of doing business. In a world where people are already nervous, and switching costs are negligible, a single breach can doom a business. A single major and very public breach in the world of online banking, for example, would probably be enough to set online banking back years.
This, however, does not seem to hold in other industries. Take the TJX Companies or Heartland, the credit card processor, for example. Even after appalling levels of neglect for security led to the theft of the credit cards of virtually every American who has one of these cards, the companies are still in business. The TJX breach was made public in early 2007. That same year, the company's CEO, Bernard Cammarata, enjoyed a salary of $911,539 plus about $1.6 million in stock and other compensation. The president of the company, Carol Meyrowitz, earned a posh $7.5 million while overseeing the organization that enabled the largest credit card theft in history and then failed to notice when it happened—a figure that pales in comparison to the cost incurred by her customers and their credit card companies. The exact cost of the Heartland breach is unclear as of this writing. However, I have no doubt its executive management will be handsomely rewarded for the valiant way it dealt with a crisis that was entirely avoidable had it simply implemented the most basic information security measures. Neglect and rationalization are still virtues in far too many companies. Clearly, we have a long way to go when it comes to accountability for InfoSec.

Accountability at the Government Level
On the topic of accountability, I recommend that you read the report from the Center for Strategic & International Studies (CSIS) entitled "Securing Cyberspace for the 44th Presidency." The report, which was published in December 2008, was written by U.S. Representatives James R. Langevin and Michael T. McCaul, along with Microsoft VP of Trustworthy Computing, Scott Charney, and Lieutenant General Harry Raduege, USAF ( Ret. ). The objective was to lay out a strategy for the incoming Obama administration around cybersecurity. More interesting, however, is the critical assessment of how cybersecurity was neglected under the previous administration; the report states that "cybersecurity is now a major national security problem for the United States."
Interestingly, the report advocates a position long opposed by the software industry, with Microsoft at the forefront of the opposition: the use of procurement rules to drive a desired direction in information technology products—specifically software. The report does not mince words when detailing how important this is. It specifically points out that cybersecurity is a "battle we are losing." Furthermore, it says "weak cybersecurity dilutes our investment in innovation while subsidizing the research and development efforts of foreign competitors." It is no stretch to take that same statement and apply it to almost any organization's InfoSec efforts.

Figure 1 Do you really have to sign your credit cards?
Inappropriate Acceptance of Risk
Many of the failures come from an inappropriate acceptance of risk. Human beings tend to underestimate risks and overestimate the benefits. We particularly underestimate risks in areas we find difficult to understand, such as cyberspace. We can very easily visualize physical risks. Most people lock their cars, even though the potential downside of a car theft is a $500 insurance deductible and being inconvenienced for a few days. We lock the doors to our houses, even in places where burglaries are very rare. However, we throw bank statements in the garbage, literally handing criminals all the information they need to steal everything we own. We sign the backs of our credit cards and store them alongside our checkbooks. Put these ingredients together and a criminal has everything he needs to empty your checking account. It is for that very reason that my credit cards look like the card shown in Figure 1.
One of the most critical aspects of InfoSec management is to have an accurate perception of risk. This is where the InfoSec group comes in. It is up to this core advisory group to help the business understand the risks it is accepting, ensuring the business understands the risks and values them properly. We have traditionally been far too simplistic about how we value risk. In my May 2008 installment of the Security Watch column, entitled "Principles of Quantum Security," I introduced a revised version of the Annual Loss Expectancy (ALE) equation used to asses risk (see Figure 2).
Figure 2 The revised Annual Loss Expectancy (ALE) equation
However, this is not just about valuing risk. Being a trusted advisor goes beyond that. In a future piece in this series on InfoSec Management, I will discuss risk in far more depth.

A Blueprint
At this point, we are getting closer to the true purpose of InfoSec. I argue that there are four central aspects, all rooted in the overriding principle that should guide InfoSec:
Keep Risk at an Acceptable Level The job of the InfoSec professionals, first and foremost, is to keep risk at an acceptable level. This includes the things I've discussed here—making sure the lights stay on, turning off the right lights, and generally ensuring that information is available to those who need it and not to those who do not. The core problem with keeping risk at an acceptable level is establishing what is meant by "acceptable." It turns out that acceptable is influenced by many factors, which I will discuss in a later article.
Enable the Business First and foremost, the InfoSec group is a member of the business. Your job is to enable the business, not to stop it. InfoSec professionals who view their role as being to protect the business from itself rarely experience much long-term success. They also will not be very busy as much of the business will simply avoid them and proceed without input from InfoSec, often making even more inappropriate risk management trade-offs. Once again, we are here to protect the business and help it achieve its objectives, while managing risk.
Advise on Risk InfoSec has an operational role, managing network devices, security software, and processes (such as patch management and incident response). This operational side is where InfoSec is often most visible. The part that can have the broadest impact, however, is the advisory capacity. As an advisor, InfoSec should act as an internal consulting resource, lending a security perspective to projects far and wide. This can be a very fulfilling role for InfoSec professionals who enjoy creating direct value for the company.
Establish Risk Management Policies and Processes Finally, InfoSec manages the overall information risk posture through policies and processes. That means that the InfoSec group must assess the risk to the business, establish policies, and define processes. The assessment of risk consists largely of an analysis of how the organization is managing risk and what its preferred posture should be. Based on that philosophy, InfoSec establishes a set of security policies to codify the organization's risk posture and the principles of managing them. These policies are then implemented in a set of processes to assist the organization with compliance.

Wrapping Up
In this article, I have looked at a basic blueprint for InfoSec management. The key here is to understand that InfoSec is a core part of the business and must be a trusted advisor to the rest of the business. Rather than being in conflict with the business, the InfoSec group must establish a relationship with other parts of the business to help the whole business achieve its objectives with an acceptable level of risk. Only then can InfoSec be effective.
But this is just the start of the conversation. Watch for future installments of the Security Watch column when I continue this series on Information Security Management.

Jesper Johansson is Principal Security Architect for a well-known Fortune 200 company, working on risk-based security vision and security strategy. He is also a contributing editor to TechNet Magazine. His work consists of ensuring security in some of the largest, most distributed systems in the world. He holds a Ph.D. in Management Information Systems, has more than 20 years experience in security, and is an MVP in Enterprise Security. His latest book is the Windows Server 2008 Security Resource Kit (Microsoft Press, 2008).


Field Notes Customer Centricity
Romi Mahajan

In the Nicomachean Ethics, Aristotle wrote, "We are not studying in order to know what virtue is, but to become good, for otherwise there would be no profit in it." Virtue in his conception could not be realized theoretically but had to be the product of experiences and direct actions that led to "good" happening. Similarly, theoretical conceptions of "customer-centricity" and "caring for the customer" don't make organizations more focused on great outcomes for their customers; only action born of experience and true learning do—actions that customers themselves recognize as having helped them .
So far, I assume I have agreement (whether or not you are Aristotelian). But what in Heaven's name you might ask does this have to do with IT organizations? This is TechNet Magazine after all not Philosophical Inquiry. The simple answer: everything.
So how indeed do we show that IT organizations are "virtuous?" In order to do so we need to establish three things, the first theoretical and the second and third practical:
  • We care about the customers we serve
  • We act on that caring every day
  • These actions are attested to by our customers as having helped them do something better
Let's go about this in an analytical way. First of all, who indeed are our customers? I argue that our customers are not only the internal people who avail themselves of our services but also the external partners and customers who do business with our companies. The net is, we have customers like anyone else does. Do we care about them? I have never seen an IT organization whose charter did NOT include clear mention of their role of, inter alia, as a services org. Serving is caring as long as you mean it. So far, we are covered on the theoretical side pretty well but as Aristotle admonishes us, the practical side is what matters more .
Do we act on that caring every day? I would argue that the answer is no. Too much of the "services" part of our collective gets intermediated by bureaucracy, abstraction, and fatigue. Again, it is not a question of bad intention—just of lack of clear application of our fundamental premise every day. Do we do good every day? Yes. Do we do enough good every day? No .
Finally, do our customers recognize our actions and are they positive about the help we've rendered? This is the simplest question of all. The answer is a stentorian (and ironic) NO. In my experience it is very rare indeed to find people touting the greatness of IT.
Well, what do we want to do about it? In previous months, I've written that justifying whether "IT matters" is unnecessary and counterproductive because it gives credence to the premise that indeed we don't matter. However, I suggest something very different here: let's make it easier to show that we are virtuous not by defending ourselves but by acting like ourselves. Let's ask our customers what they want and then see how we stack up against what they ask of us.
If I were you I'd go to ideascale.com (or any other crowd-sourcing tool) and set up an instance to get your customers input on your organization and what they want of you. Solicit input from internal and external customers. And through hearing them and interacting with them, I can bet my bottom dollar you find you are already covering 90% of what they want. And that's a pretty good ratio, certainly befitting virtue.
My own experience dealing with IT organizations has been generally very good. While at Microsoft, I found IT to be very helpful and open to new ideas on how to improve. The same is true in my current company, Ascentium. Am I lucky? Maybe. Am I a customer of IT? Yes. Am I touting IT? Yes.
Wow, we just completed a virtuous cycle. Now let's go do a lot more.

Romi Mahajan is Chief Marketing Officer of Ascentium Corporation. Before joining Ascentium, he spent 7+ years at Microsoft where his last role was as Director of Technical Audience and Platform Marketing. Romi is widely published in the areas of technology, politics, economics, and sociology.


Windows Confidential Forcing Handles Closed
Raymond Chen

Have you ever wanted to delete a file but couldn't because some program had the file open? The correct thing to do in this situation would be to convince the program that has the file open to close it—you could, for instance, ask the person editing the document to close it. But sometimes you get impatient and use some utility that can force file handles closed. And then you've traded temporary relief for long-term data corruption.
Forcing a handle closed is equivalent to reaching into a program and freeing some memory. The program thinks the handle (or memory) is still valid and will continue to use it. But since the handle is really free, it will be reused for something else.
Say you are in high school and you see a book in locker number 305 that you want. You force the locker open and take the book. Now the locker is open and somebody else comes by and puts his own book bag into it. Meanwhile, the original owner of locker 305 returns from lunch, grabs the book bag, and takes it to class.
Now you have two problems: The original user of locker 305 has the wrong book bag and is in danger of taking notes in somebody else's notebook. The second person, meanwhile, just had his book bag stolen.
If you're extremely lucky, the next thing the program will do is close the handle you forced to close earlier. The program gets an invalid handle error, but since there's nothing it can do, the program just proceeds as if the handle closed successfully. But this is rarely the case.
More likely, the program will eventually come back to the handle and try to use it, getting invalid handle errors. Even more likely is that the handle you forced closed will be used for something else. And that's when things start to go completely berserk. Suppose the handle were reused as another file. Now the program thinks it's reading and writing from the first file, but in fact it's operating on the second file. The result is data corruption in the first file (data that should have been written wasn't) and in the second file (data intended for the first file was written to the second file).
When the program closes the first file, it also closes the recycled handle. You've created a cascade error: You forced the first file closed, and the result is that some second handle was forced closed.
This data corruption can continue indefinitely, as each component that requests a handle has that handle inadvertently corrupted by the previous component. When the previous component finally closes what it thinks is a valid handle, it actually closes the handle used by the current component, which now proceeds to corrupt the next component that requests a handle.
Suppose a search index service has a file open for indexing but has gotten stuck temporarily and you want to delete the file, so you (unwisely) force the handle closed. The search index service opens its log file in order to record some information, and the handle to the deleted file is recycled as the handle to the log file. The stuck operation finally completes, and the search index service finally gets around to closing that handle it had open, but it ends up unwittingly closing the log file handle.
The search index service opens another file, say a configuration file for writing so it can update some persistent state. The handle for the log file gets recycled as the handle for the configuration file. The search index service wants to log some information, so it writes to its log file. Unfortunately, the log file handle was closed and the handle reused for its configuration file. The logged information goes into the configuration file, corrupting it.
Meanwhile, another handle you forced closed was reused as a mutex handle, which is used to help prevent data from being corrupted. When the original file handle is closed, the mutex handle is closed and the protections against data corruption are lost. The longer the service runs, the more corrupted its indexes become. Eventually, somebody notices the index is returning incorrect results. And when you try to restart the service, it fails because its configuration files have been corrupted.
You report the problem to the company that makes the search index service and they determine that the index has been corrupted, the log file has mysteriously stopped logging, and the configuration file was overwritten with garbage. Some poor technician is assigned the hopeless task of figuring out why the service corrupts its indexes and configuration files, unaware that the source of the corruption is that you forced a handle closed.

Raymond Chen's Web site, "The Old New Thing," and identically titled book (Addison-Wesley, 2007) deal with Windows history, Win32 programming, and misreading newspaper headlines.