 MSDN Magazine January 2004

MSDN Magazine January 2004

Code Name Longhorn: A First Look at Writing and Deploying Apps in the Next Generation of Windows

The next version of the Microsoft Windows operating system, code-named "Longhorn," marks a significant change not only in terms of how the operating system works, but also in the way in which applications are built. The Longhorn version of Windows includes a new storage system, natural search technology, and an increased emphasis on security and trustworthy computing. Here the author provides an overview of Longhorn, focusing on the build-once, deploy n-times application model. In addition, he discusses the new language, code-named "XAML," that's used to create UI elements, then presents some working samples.

Dino Esposito Code Name Indigo: A Guide to Developing and Running Connected Systems with Indigo

This article describes a collection of new programming frameworks that are part of "Longhorn," the upcoming version of Windows. "Indigo," the code name for this framework, provides rich support for service-oriented design that is complementary to traditional object-oriented approaches. Indigo marries the best features of .NET Remoting, ASMX, and .NET Enterprise Services into a unified programming and administration model. Indigo's deep support for standard protocols, including HTTP, XML, and SOAP, makes it easier to integrate applications and services without sacrificing security or reliability.

Don Box Code Name Avalon: Create Real Apps Using New Code and Markup Model

The presentation subsystem in the next version of Windows, code-named "Longhorn," offers powerful new capabilities to developers. This subsystem, code-named "Avalon," allows developers to take advantage of its capabilities through a new markup language code-named "XAML." In addition, modern object-oriented programming languages such as C# and Visual Basic .NET can be used to tie everything together. Because most applications written to Avalon will probably be a mix of XAML and programming code, this article discusses XAML tags used to control page layout along with the procedural code written to respond to events.

Charles Petzold Code Name WinFS: Revolutionary File Storage System Lets Users Search and Manage Files Based on Content

One of the monumental problems organizations face today is aggregating information that's stored in disparate formats. Knowledge workers have long wanted to be able to search for content independent of format. The next version of the Windows operating system, code-named "Longhorn," boasts a new storage subsystem that makes that task easier. That subsystem, code-named "WinFS," allows the user to perform searches based on the metadata of the stored item, regardless of what type of file it is or which application created it. This article covers the basic architecture of WinFS and explains how to use the WinFS managed API.

Richard Grimes Windows Shell: Create Namespace Extensions for Windows Explorer with the .NET Framework

Extending the Windows shell with namespace extensions allows you to create some custom functionality for Windows Explorer. One common use is to enable Explorer to present a list of items that do not exist in one real folder, but actually reside in a number of places. The view on the folder makes it look like these items are in one place, so managing them becomes easier. This article illustrates the process of creating custom shell namespace extensions using C# and the .NET Framework. The author dispels some myths about the difficulty of writing such extensions, and shows that it is easier than it was before .NET. Along the way he outlines undocumented interfaces and describes advanced techniques for consuming them in .NET.

Dave Rensin Editor's Note: PDC 2003 Trip Report

New Stuff: Resources for Your Developer Toolbox

Theresa W. Carey Web Q&A: DTS Follow-up, Web Services, Access Over the Network, and More

Edited by Nancy Michell Data Points: Exploring SQL Server Triggers: Part 2

John Papa Cutting Edge: Extend the ASP.NET DataGrid with Client-side Behaviors

Dino Esposito The XML Files: IntelliSense for XSLT, Namespace Collisions, and More

Aaron Skonnard Advanced Basics: Windows Forms Controls: Z-order and Copying Collections

Ken Spencer Basic Instincts: Asynchronous Method Execution Using Delegates

Ted Pattison .NET Column: Practical Multithreading for Client Apps

Jason Clark C++ Q&A: Get Logical Drives with VolInfo, Modifying the System Menu

Paul DiLascia Talking To…: Carl Franklin Talks About Being a New MVP and His Passion for Visual Basic .NET


Code Name Longhorn: A First Look at Writing and Deploying Apps in the Next Generation of Windows

[ Editor's Update - 8/27/2004: The WinFS content in this article is out-of-date. Microsoft will deliver a Windows storage subsystem, code-named "WinFS," after the "Longhorn" release. The new storage system provides advanced data organization and management capabilities and will be in beta testing when the "Longhorn" client becomes available. For more information, please refer to Microsoft Announces 2006 Target Date for Broad Availability Of Windows "Longhorn" Client Operating System.]
Code Name Longhorn

A First Look at Writing and Deploying Apps in the Next Generation of Windows

Dino Esposito
This article was based on a pre-PDC build of Microsoft Windows Code Name "Longhorn" and all information contained herein is subject to change.
Code download available at: Longhorn.exe (113KB) Note: This document was developed prior to the product's release to manufacturing and, as such, we cannot guarantee that any details included herein will be exactly the same as those found in the shipping product. The information represents the product at the time this document was printed and should be used for planning purposes only. Information subject to change at any time without prior notice.
SUMMARY

The next version of the Microsoft Windows operating system, code-named "Longhorn," marks a significant change not only in terms of how the operating system works, but also in the way in which applications are built. The Longhorn version of Windows includes a new storage system, natural search technology, and an increased emphasis on security and trustworthy computing. Here the author provides an overview of Longhorn, focusing on the build-once, deploy n-times application model. In addition, he discusses the new language, code-named "XAML," that's used to create UI elements, then presents some working samples.

Contents
The Longhorn Application Model
Understanding Longhorn Applications
The XAML Language
Building a Sample Application
Conclusion

T
he next release of the Microsoft® Windows® operating system, code-named "Longhorn," is an essential milestone for many reasons. It is the first operating system built with managed code and the first to host a new storage subsystem (code-named "WinFS") that revolutionizes the concept of a file system. It is also the first operating system to support a natural search technology (Natural UI) that automatically resolves many of the ambiguities inherent in query text. In addition, Longhorn is the first operating system designed from the ground up with security and trustworthy computing at the core. These and other features suggest that Longhorn will change the way applications are built—not something that happens every day. Since the advent of Windows, I remember two similar milestones—the move to 32-bit Windows and the dawn of the managed environment of the Microsoft .NET Framework.

One of the most important changes in Longhorn is that this operating system makes it possible to write an application one time and use it in multiple deployment scenarios. To achieve this ambitious goal, Longhorn-based applications are completely object-oriented and based on a central Application object that provides all key services that are needed for running the application. In this article, I'll examine the Longhorn application model in some depth and apply it to a few basic scenarios, including the classic Hello World application.

The Longhorn Application Model

The Application object is the heart of the Longhorn application model. Through its set of properties, methods, and events, the object enables you to arrange a collection of markup pages—a sort of enhanced version of HTML—into a coherent and classic Windows-based application. The Application object is the root application object available in Longhorn. It provides basic application support and will typically be used by those applications that need low overhead and don't use page navigation and state management. More complex Longhorn applications will use the closely related NavigationApplication object, which inherits from Application but adds support for navigation.

A typical Longhorn application can be thought of as a set of pages scripted with some procedural code. The Application object controls the execution of the program and raises events to the user code. Pages are written in a new declarative markup language, code-named "XAML" (Extensible Application Markup Language). With XAML elements you control the layout of each page, including the display of text and images and the insertion of interactive components such as buttons and textboxes. In summary, XAML is the language used to declaratively render the user interface of the pages that make up the application. In addition to using XAML, you can also write applications for Longhorn entirely with procedural code. In general, successful Longhorn-based applications have two ingredients—XAML pages and managed procedural code. How you combine them is up to you, but any combination of the two is acceptable.

By using a combination of XAML and C# (or Visual Basic® .NET) code, you can build various types of output files including traditional Windows desktop executables, DLL libraries, and console applications. In addition, if your application is simple enough, it can be rendered using freestanding XAML markup, thus producing another type of application to add to the list. Standalone XAML files are runnable within the Longhorn shell and the browser as long as they don't reference some codebehind class. Finally, it is worth noting that Windows executables can be hosted in a window (by default) as well as in the browser. In both cases, the code remains the same and only needs to be compiled again with a different project property.

For Longhorn, desktop executables are the next version of today's Windows Forms client-side apps. On the other hand, XAML and browser-hosted applications represent an evolution of today's client-side programming model to work over the Web. Right now, existing client-side applications can rarely be deployed over the Web. If you want to embed a Windows Forms form into a browser page, you'll get a reduced feature set and have to tweak bits and pieces of your code. With Longhorn, the common application model will let you write one application and deploy it over the Web. However, the final application is Longhorn-specific—very different from a traditional Web application like ASP.NET.

When you compile an application, the next version of Visual Studio® and the .NET Framework, code-named "Whidbey" (or the underlying MSBuild.exe tool) will produce an .exe file, an application manifest (.manifest), and a deployment manifest (.deploy). If you click on an .exe file, the application runs as expected. If configured to run within a browser, an instance of Internet Explorer starts up and hosts the application. As an alternative, you can also deploy an application from a remote server. The steps to accomplish this are as follows. First, you copy the deployment manifest to the appropriate location on your server. It can be an FTP or HTTP path. Next, you copy the compiled application files and manifests to the appropriate locations on your server. The server-side location of the deployment manifest and the application's files and manifests are not necessarily the same. If they're different, you edit the deployment manifest manually so that it points to the location of the application manifest. Manifest files are plain XML files. When users point their browser to the specified deployment location, Longhorn automatically downloads and installs the application and its manifests on the client computer and creates a shortcut to the .deploy file. At the end of the process, users run the application by clicking on the .deploy file.

All Longhorn applications have a common structure—XAML pages with procedural code either inline or codebehind—and the root object derives from Application. The Application object acts as a controller; its lifetime coincides with the lifetime of the application. The Application object allows you to handle top-level events and sometimes share code and state among pages. It is also responsible for letting the user navigate among pages according to the logic of the application. In a typical Longhorn program, the user performs tasks and progresses through the app by navigating from one page to the next. Navigation is usually accomplished by replacing the old page with the new one. However, you also have the option of opening a new pop-up window to display the new page. Navigation is not necessary in all Longhorn applications; simple apps that consist of a single page aren't going to need it.

As mentioned, XAML pages can contain procedural code in addition to markup elements. Procedural code is needed, for example, to handle an event that is raised by one of the XAML elements on the page. Procedural code can either be placed inline in the body of the XAML file or in a separate codebehind file.

Programming in Longhorn is based on managed code. However, only a few .NET-compatible languages can be used to write XAML-based applications. Today the list includes C#, Visual Basic .NET, and JScript® .NET. By the time Longhorn is released, other .NET-compatible languages are expected to be added to the list. The current three-language limit is due to the fact that the source code of a XAML file must be parsed and compiled on the fly, hence a compiler and a related code document object model must be available beforehand. Note, though, that if you write an application entirely in procedural code, you can use any .NET-compatible language for which you can get a valid compiler. Note that only XAML-based apps are lmited to those three languages. If procedural code is embedded in a XAML page, then you must compile the application before you can run it; if there is no procedural code in the XAML page, it can be displayed by double-clicking just as HTML pages are. Longhorn doesn't process uncompiled code, nor does it have the ability to compile code on the fly. Can't wait to see what a Longhorn "Hello World" application looks like? Here's a sample of the simplest XAML code you can write:

<Canvas
    xmlns="http://schemas.microsoft.com/2003/xaml"
    Background="LightCyan"
    Width="100%" Height="100%">

  <Image Source="lh.bmp" Canvas.Left="5" Canvas.Top="5" />
  <Text Canvas.Left="90" Canvas.Top="20" FontSize="36">Hello, Longhorn!
  </Text>
</Canvas>

Figure 1 Simple XAML Page
Figure 1 Simple XAML Page

Save this snippet to a text file with the .xaml extension and point your Longhorn browser to it or just double-click it from within the shell. Figure 1 shows the results. The <Canvas> node defines the area of the application's user interface—basically, the drawing surface. The Background attribute indicates the background color of the area, whereas Width and Height indicate the size of the surface. The <Image> and <Text> elements define the contents of the page. Both elements define their absolute position using the Left and Top attributes of the parent Canvas object.

Understanding Longhorn Applications

Longhorn provides a set of framework classes that extend and enhance the .NET Framework 1.1. Extensions available in Longhorn range from XAML support to the storage system, and from the application model to trustworthy computing and advanced Web Services. Longhorn and Whidbey represent two distinct milestones in the Microsoft roadmap. Whidbey is expected to ship several months before Longhorn, so that when Longhorn is released, an updated version of Whidbey will be included that provides core services in Longhorn.

Let's take a closer look at the common substrate of all Longhorn applications—the XAML language. XAML is an XML-based language specifically designed to describe your application's user interface. Programmers familiar with Win32® and the .NET Framework will recognize a distinct similarity between XAML tags and traditional Windows controls. However, the suite of XAML markup tags is more abstract and broader than the set of Win32 common controls or Windows Forms controls. To get an immediate grasp of what XAML represents and how you should look at it, think of ASP.NET pages. Or, more specifically, think of a particular ASP.NET page in which you only find server-side controls (runat="server") and no literal text.

Each and every XAML tag corresponds to a .NET Framework class and comes with a collection of methods, properties, and events. You can set properties and wire events declaratively in the XAML script, as well as use procedural code packed into a codebehind class. The control behind each tag is instantiated at run time and given an area of the screen on which to render its output. At the highest level of abstraction, the model looks a lot like ASP.NET, only abstracted to work for a generic, richer Windows platform.

Every element in XAML uses an underlying class, but there are many classes that do not have corresponding XAML elements. These are often abstract classes that are mostly used for inheritance. Anything you can create in XAML can be created in procedural code. For example, to create a button element in XAML, you can use the following code:

<Canvas xmlns="http://schemas.microsoft.com/2003/xaml">
   <Button Canvas.Left="10"
           Canvas.Top="10"
           Width="90px"
           Height="32px">Click Me</Button>
</Canvas>

In procedural C# code, you create the same button as follows:

Button btn = new Button();
btn.Width = new Length(90);
btn.Height = new Length(32);
Canvas.SetTop(btn, new Length(10));
Canvas.SetLeft(btn, new Length(10));
btn.Content = "Click Me";

In XAML files you cannot use methods, but you can set attributes. Attributes usually correspond to properties, but a few properties of a class don't have a XAML attribute. You can set class properties individually or rely on styles. Styles are named collections of style attributes that the compiler automatically applies to a control that uses the style. All XAML pages have at least one panel element that acts as a container window and controls the positioning of child content and global properties such as the background color and the font. The elements of an XAML page are organized in a hierarchical manner, with a single root element. Typically, this is a Panel-derived class such as DockPanel or Canvas, or a Decorator-derived class such as Border. The Canvas element is used to position content according to absolute coordinates. Elements are normally drawn in a unique location; if elements occupy the same coordinates, the order in which they appear in markup determines the order in which elements are drawn.

In addition to properties and methods, XAML elements may supply a number of events. You can make the page respond dynamically to those notifications by creating event handlers. The way in which Longhorn event handlers are created and wired to the element is nearly identical to the way you do it today in .NET-based applications. The event handler is declared through an attribute, and the handler is then incorporated in the application through the codebehind class:

<Button Width="90px" Height="25px" Click="OnHandleClick">
Click Me
</Button>

The elements in the page form the application tree, which is an object model that encompasses all runtime components of the application and is programmatically accessible. Each control exposes its own way of accessing children. The major patterns are: panel.Children.Add(element), listbox.Items.Add(object), button.Content = object, and textbox.TextRange.Text = string.

Controls that support children also support arbitrary objects as children—strings, elements, or some completely random object (in which case you call ToString). Each control has a Parent property to provide access to its parent in the tree. Notice that only direct children, not descendants, are returned by the various properties accessing the tree. The object model allows you to manipulate every aspect of the elements on a page. It also provides some additional capabilities that cannot be implemented through XAML. For example, you can create elements dynamically, but you can also create them on the fly based on runtime conditions.

To make some of this clearer, let's take a look at a second sample application. As I mentioned earlier, any serious Longhorn application will include procedural code as well as an XAML script. Let's see how to go beyond the initial Hello World application and handle some events.

Figure 2 shows the full source of a new application in which the XAML is bound to some procedural code stored in a codebehind class. If you compare the XAML solution with an ASP.NET Web Forms page, you won't notice any significant difference aside from the syntax. The def:CodeBehind and def:Class attributes wrap the C# code in much the same way that a Visual Studio codebehind class encloses ASP.NET code. Let's name the XAML file in Figure 2 events.xaml and name the codebehind class also shown in Figure 2 events.xaml.cs. If you point your browser to it, a run-time error is raised, as shown in Figure 3. The error message isn't very clear in the build I used, but the reason for the error is quite clear. It has to do with the fact that XAML code can't be dynamically compiled. Unlike ASP.NET, Longhorn requires that you explicitly compile any procedural code that's embedded, or simply referenced, by the XAML script. This feature may significantly change in future builds, but for now it forces you to arrange a Longhorn project file in Visual Studio or, as an alternative, become familiar with MSBuild, whose executable is named msbuild.exe. Longhorn solution and project files have the same extensions and structure as in Visual Studio 2003. A Longhorn project file may look like the code that's listed in Figure 4.

Figure 4 Longhorn Project File (Close Figure 4)
<Project DefaultTargets="Build">
    <PropertyGroup>
        <Property Language="C#" />
        <Property DefaultClrNameSpace="LonghornApp" />
        <Property TargetName="Events" />
    </PropertyGroup>

    <!--Imports the target which contains all the common
        targets-->
    <Import Project="$(LAPI)\WindowsApplication.target" />

    <ItemGroup>
        <!-- Application markup -->
        <Item Type="ApplicationDefinition"
            Include="eventsApplication.xaml" />

        <!-- Compiled Xaml Files list -->
        <Item Type="Pages" Include="events.xaml"/>
    </ItemGroup>

</Project>
Figure 2 XAML Plus Procedural Code (Close Figure 2)
<!-- XAML file -->

<Window
    xmlns="http://schemas.microsoft.com/2003/xaml"
    xmlns:def="Definition"
    def:Class="Application1.Window1"
    def:CodeBehind="Window1.xaml.cs"
    Text="Application1" Visible="True"
    >

    <DockPanel xmlns="http://schemas.microsoft.com/2003/xaml"
        xmlns:def="Definition" def:Language="C#" Background="White">
        <FlowPanel ID="sourceFlowPanel"
            MouseLeftButtonDown="OnLeftMouseDown"
            Margin="10,10,0,0" Background="cyan" Width="150px"
            Height="150px">
            <Text ID="caption" Margin="10,10,20,20" FontSize="16">Click on
                Me</Text>
        </FlowPanel>
    </DockPanel>
</Window>

// C# Codebehind class

using System;
using MSAvalon.Windows;
using MSAvalon.Windows.Input;
using MSAvalon.Windows.Controls;
using MSAvalon.Windows.Documents;
using MSAvalon.Windows.Navigation;
using MSAvalon.Windows.Shapes;
using MSAvalon.Windows.Data;

namespace Application1
{
    public partial class Window1 : Window
    {
        private bool expanded = false;

        private void OnLeftMouseDown(object sender, MouseButtonEventArgs e)
        {
            if (expanded)
            {
                sourceFlowPanel.Width = new
                    Length(sourceFlowPanel.Width.Value/2);
                sourceFlowPanel.Height = new
                    Length(sourceFlowPanel.Height.Value/2);
                caption.TextRange.Text = "Click on Me";
                expanded = false;
            }
            else
            {
                sourceFlowPanel.Width = new Length(2 *
                    sourceFlowPanel.Width.Value);
                sourceFlowPanel.Height = new Length(2 *
                    sourceFlowPanel.Height.Value);
                caption.TextRange.Text = "Thanks for clicking";
                expanded = true;
            }
        }
    }
}
Figure 3 XAML Error
Figure 3 XAML Error

The project file contains important information about the project. The <Project> tag indicates the type of the package that the compiler must generate. Project properties and items are grouped in nested tags—<ProjectGroup> and <ItemGroup>. Each element is named to make it human-readable. A project file is passed to the msbuild.exe utility on the command line. MSBuild resides in the Framework folder under Windows\ Microsoft .NET. The following simple command generates an executable based on the Events.proj project file:

msbuild.exe Events.proj

Unless you want to practice with the MSBuild tool, you shouldn't care that much about it. As mentioned, the Whidbey version of Visual Studio integrates very well with the Longhorn SDK and lets you build Longhorn applications by clicking the usual toolbar buttons or hitting F5. Figure 5 shows the code of Figure 2 in action. In the application tree, only a blue panel is visible and marked with text. The application tree is slightly more complex than you would expect just looking at Figure 5. The root node is a window which contains a DockPanel object. The DockPanel is the outermost container of all user interface elements and contains one child, a FlowPanel, which in turn contains a Text object. The text is represented by an instance of the Text class. You should notice that XAML tags can have several attributes to control their appearance. More importantly, they support the ID attribute, which provides runtime identification. The MouseLeftButtonDown event associated with the FlowPanel class is raised whenever you click within the area assigned to the control. When this happens, the OnLeftMouseDown event handler code is called. The width and height of the flow panel are doubled and the text is then changed. By clicking again, the original size is restored.

Figure 5 Running the App
Figure 5 Running the App

Although nothing prevents you from writing purely procedural applications, the XAML file makes it easier to describe and templatize the user interfaces of your applications.

The XAML Language

There are four main categories of elements that form the XAML vocabulary—panels, interactive controls, document-related elements, and graphic shapes. The Border element is also important, although it doesn't really fall into any of those categories. Technically it's a "decorator" that provides trivial layout, hosting only one child, but adding rendering effects. In XAML, panels handle the page layout and act as containers for other elements. Panels are components that govern the rendering of contained elements, their size and position, and the arrangement of their content. There are six built-in panel classes provided in Longhorn, but developers can create their own panels with custom rendering behaviors. The predefined panels are Canvas, DockPanel, FlowPanel, GridPanel, Table, and TextPanel. These panel types are described in Figure 6.

Figure 6 Panel Classes (Close Figure 6)

Panel Class | Description
Canvas | Defines an area in which you can position child elements using a system of coordinates relative to the Canvas area.
DockPanel | Defines an area in which you can arrange child elements relative to each other (docking).
FlowPanel | In this area child elements are rendered one after the next, proceeding in the specified direction. When the boundary is reached, the next element wraps accordingly.
GridPanel | Defines a grid area consisting of columns and rows. It has a limited set of features.
Table | Defines an area that displays complex data in a tabular format. It supports advanced capabilities such as header, footer, column, and row grouping.
TextPanel | Defines an area optimized to display text. It provides support for multiple lines of text and more complicated text formatting.
Text | Like TextPanel, but more lightweight. For simpler unformatted text.

The Canvas panel is the only one in the group to support explicit positioning. All of its child elements begin rendering at a fixed location and extend for the specified dimensions. If two elements overlap, the last to render wins and overwrites the underlying area. Notice that the coordinates are relative to the canvas, meaning that (0,0) indicates the top-left pixel of the area. Here's an example of a canvas object:

 <Canvas xmlns="http://schemas.microsoft.com/2003/xaml"
    Height="600" Width="800">
    <Border Background="red"
       Canvas.Top="0px" Canvas.Left="0px"
       Height="100px" Width="100px" />
    <Border Background="green "
       Canvas.Top="100px" Canvas.Left="100px"
       Height="100px" Width="100px" />
    <Border Background="blue"
       Canvas.Top="50px" Canvas.Left="50px"
       Height="100px" Width="100px" />
 </Canvas>

The same concept of containment is developed differently by the DockPanel component. All children of this panel are docked to each other either horizontally or vertically. In this case, positioning is relative to the previous control and no coordinates are ever needed. The DockPanel is ideal for creating a strip of buttons like a toolbar would display. Here's an example:

<Border xmlns="http://schemas.microsoft.com/2003/xaml"
    Background="black" >
    <DockPanel>
        <Button DockPanel.Dock="Left" Width="50px" Height="32px">
            Open
        </Button>
        <Button DockPanel.Dock="Left" Width="50px" Height="32px">
            Save
        </Button>
        <Button DockPanel.Dock="Left" Width="50px" Height="32px">
            Print
        </Button>
    </DockPanel>
</Border>

The first button is positioned on the left side of the panel and each additional button is aligned next to the previous one in a horizontal line (see Figure 7).

Figure 7 Button Layout
Figure 7 Button Layout

The FlowPanel element makes child components flow in the available area in various directions. The panel also contains the logic to handle situations in which the content exceeds the width of the panel. In this case, according to the configuration of the object, the exceeding content can wrap to the next line or be clipped. The following XAML example demonstrates how the FlowPanel breaks and wraps content logically designed to fit on a single line. Four square elements are contained by one larger element. The aggregate width of the smaller squares is greater than the width of the container. Because all of the smaller squares do not fit on a single line, the line breaks and the last square wraps to a second line. Notice that the default directional flow of the panel is from left to right and top to bottom:

<FlowPanel xmlns="http://schemas.microsoft.com/2003/xaml"
                 Width="250px" Height="250px">
    <Border Background="red" Width="75px" Height="75px" />
    <Border Background="green" Width="75px" Height="75px" />
    <Border Background="blue" Width="75px" Height="75px" />
    <Border Background="orange" Width="75px" Height="75px" />
</FlowPanel>

GridPanel is a lightweight element for laying out objects in a grid-like fashion and subsequently building relatively simple tables. The GridPanel allows you to populate a matrix of rows and columns with any combination of controls, but has a limited feature set overall. More complex tables, characterized by advanced functions, can be built using the Table panel. The structure of the Table panel consists of groups of rows. You can define a group of rows for the header or the footer and insert groups of rows having different settings. The layout capabilities of the panel let you create quite sophisticated tabular representations of data.

Finally, the TextPanel is ideal when complex text layout support is required. However, the Text element is a better choice for basic text support because it has less overhead (and fewer capabilities).

The MSAvalon.Windows.Controls namespace groups all the user controls that are responsible for user interaction. Examples of classes belonging to this namespace are old acquaintances like Button, ComboBox, ListBox, and CheckBox, but also new entries like ContextMenu, PageViewer, and RadioButtonList. The base class for user-interactive elements is Control, which supplies a common set of properties and methods.

Figure 8 Page Viewer
Figure 8 Page Viewer

In particular, the PageViewer control provides the user interface for viewing online documents and encompasses pagination and page navigation functionalities. The following example generates a page viewer (shown in Figure 8) that takes up the entire client area. The page viewer paginates and displays text from the file specified in the PageViewer tag's Source attribute:

<DockPanel ID="root" xmlns="http://
  schemas.microsoft.com/2003/xaml"
  xmlns:def="Definition">
  <Text DockPanel.Dock="Top">See a
  PageViewer control in action below.
  </Text>
  <PageViewer DockPanel.Dock="Top"
    Source="Sample.xaml" Height="80%" Width="80%"/>
  <Button DockPanel.Dock="Top" Width="50%">OK</Button>
</DockPanel>

It is interesting to note that the source file viewed through a PageViewer control can't be pure text, RTF, or HTML. At least at this time, it has to be an XAML file that contains plain ASCII text wrapped by an Avalon control that supports pagination. Avalon is the code name for a new presentation sybsystem in Longhorn (see the article by Charles Petzold in this issue). For example, take a look at the TextPanel class:

<TextPanel ID="root" xmlns="http://schemas.microsoft.com/2003/xaml">
text to show goes here
</TextPanel>

Save this code snippet to sample.xaml in order to run the previous example. Bear in mind that the xmlns namespace is necessary.

The classes in the MSAvalon.Windows.Documents namespace handle document presentation. Taken altogether, these classes may look to you like a superset of advanced HTML tags. You will find classes such as Block, Column, Heading, and Footer, as well as ColumnGroup, RowGroup, HyperLink, List, and more.

The Text class that you saw earlier belongs to the System.Windows.Controls namespace as of this prerelease version of Longhorn. The Text class supports multiple lines and a variety of text formatting specs, including bold and italic, font size, as well as nested child elements. In cases where relatively simple text is required, Text is the preferred class because of its lightweight nature and range of features.

The Longhorn presentation model renders graphics content through the use of XAML panel elements. Panels provide a number of size and alignment properties, as well as control over borders, background color, and fill.

Graphic contents in Longhorn are rendered using Windows Vector Graphics, which provides a number of benefits over GDI and GDI+. Windows Vector Graphics is an XML-based graphical markup system that is easy to use and reuse. If you're a fan of Scalable Vector Graphics, you'll love this, too. Windows Vector Graphics provides predefined shapes including Ellipse, Line, Path, Polygon, Polyline, and Rectangle, which inherit from the Shape class. These elements inherit a number of common attributes from Shape, including Stroke and StrokeThickness, along with Fill, plus some attributes to specify coordinates and vertices. Shapes can be skewed, rotated, translated, and scaled by applying transformations. Just as you can specify solid color fills and backgrounds for shapes, you can specify gradients. The following example sets a horizontal gradient as the Fill property of a Rectangle, with Red as the start color and Blue as the end color:

<Rectangle xmlns="http://schemas.microsoft.com/2003/xaml"
    Canvas.Top="10" Canvas.Left="10"
    Fill="HorizontalGradient Red Blue"
    RectangleLeft="0" RectangleTop="0"
    Width="50" Height="50"/>

Building a Sample Application

Using the XAML language you can prototype a user interface quickly and efficiently. I'll bet that by the time Longhorn ships, a wealth of development tools will be updated (or built from scratch) to fully support XAML with WYSIWYG features. The Tech Preview edition of Longhorn already includes some extensions to the Whidbey version of Visual Studio .NET. Let's take a look at a more serious, interactive application. You'll quickly discover that it is not very different from writing a Windows Forms app, and you'll be pleasantly surprised to discover how applicable your existing .NET skills are to Longhorn. In Figure 9, you can see the UI for a small sample application involving a textbox and a context menu. The textbox is defined in the XAML; the context menu is created on the fly. The full source code for this app is shown in Figure 10.

Figure 10 The Context Menu App (Close Figure 10)
<Window
    xmlns="http://schemas.microsoft.com/2003/xaml"
    xmlns:def="Definition"
    def:Class="Application2.Window1"
    def:CodeBehind="Window1.xaml.cs"
    Text="Application2" Visible="True">

    <FlowPanel DockPanel.Dock="Fill">
        <TextBox ID="input" Margin="10,10"
                 Background="LightCyan"
                 BorderBrush="black" FontFamily="verdana"
                 FontSize="16"
                 BorderThickness="1"
                 ContextMenuEvent="OnContextMenu"
                 Height="40" Width="360">Right-click to see a
                     context menu</TextBox>
    </FlowPanel>
</Window>

// C# source file for the application

namespace MsdnLHSample
{
    public partial class MyApp : Application
    {
        void AppStartingUp(object sender, StartingUpCancelEventArgs e)
        {
            Window mainWindow = new Window1();
            mainWindow.Show();
        }
    }
}

// C# source file for the window

using System;
using MSAvalon.Windows;
using MSAvalon.Windows.Media;
using MSAvalon.Windows.Controls;
using MSAvalon.Windows.Documents;
using MSAvalon.Windows.Navigation;
using MSAvalon.Windows.Shapes;
using MSAvalon.Windows.Data;

namespace MsdnLHSample
{

    // input is the textbox declared in the XAML

    public partial class Window1 : Window
    {
        private void OnContextMenu(object sender, ContextMenuEventArgs e)
        {
            CreateMenu();
        }

        private void CreateMenu()
        {
            ContextMenu cm = new ContextMenu();
            cm.FontFamily = "verdana";
            cm.Background = Brushes.LightYellow;

            MenuItem mi0 = new MenuItem();
            mi0.Header = "Lower case";
            mi0.FontSize = new FontSize(16);
            mi0.Click += new ClickEventHandler(LowerCase);

            cm.Items.Add(mi0);

            MenuItem mi1 = new MenuItem();
            mi1.FontSize = new FontSize(16);
            mi1.Header = "Upper case";
            mi1.Click += new ClickEventHandler(UpperCase);
            cm.Items.Add(mi1);

            MenuItem mi2 = new MenuItem();
            mi2.Header = "Select All";
            mi2.FontSize = new FontSize(16);
            mi2.Click += new ClickEventHandler(SelectAll);
            cm.Items.Add(mi2);

            input.ContextMenu = cm;
        }

        public void LowerCase(Object sender, ClickEventArgs args)
        {
            MenuItem mi = (MenuItem) args.Source;
            ContextMenu menu = (ContextMenu) mi.Parent;
            TextBox thisTextBox = (TextBox) menu.PlacementTarget;

            thisTextBox.Text = thisTextBox.Text.ToLower();
        }

        public void UpperCase(Object sender, ClickEventArgs args)
        {
            MenuItem mi = (MenuItem)args.Source;
            ContextMenu menu = (ContextMenu)mi.Parent;
            TextBox thisTextBox = (TextBox)menu.PlacementTarget;

            thisTextBox.Text = thisTextBox.Text.ToUpper();
        }

        public void SelectAll(Object sender, ClickEventArgs args)
        {
            MenuItem mi = (MenuItem)args.Source;
            ContextMenu menu = (ContextMenu)mi.Parent;
            TextBox thisTextBox = (TextBox)menu.PlacementTarget;

            thisTextBox.SelectAll();
        }

    }
}
Figure 9 Context Menu
Figure 9 Context Menu

In Longhorn, all applications are represented by an instance of the Application class. This object is at the core of the Longhorn application model. However, the Application object provides only basic application support and will typically used only by applications that need low overhead and do not use navigation. In the real world, most Longhorn applications will use the closely related NavigationApplication object, which inherits from Application and adds support for navigation.

The NavigationApplication object supports a variety of methods, properties, and events that enable you to make a collection of XAML pages that fit into a single application. In a certain sense, Longhorn applications based on the simpler Application class are comparable to dialog-based Win32 applications. Likewise, you can compare navigation-enabled applications to full-fledged Win32 applications in which a variety of windows and forms contribute to the overall functionality. The following code shows an example of such navigation:

myApp = NavigationApplication.Current;
win = (Navigation.NavigationWindow) myApp.Windows[0];
•••
private void Button_Back(Object sender, ClickEventArgs e)
{
    // If possible, go to the previous window
    if(win.CanGoBack())
       win.GoBack();
}

You get the reference to the application object by using the static Current property on the Application (or NavigationApplication) navigation object. The previous example demonstrates how to obtain a reference to the first window in the stack. The Button_Back event handler checks to see if a previous window object exists and, if so, jumps back to it in a browser-like manner.

The next example describes how to customize the start-up behavior of a navigation application by deriving a new class and overriding the OnStartingUp method. The following code fragment overrides OnStartingUp to create and display a navigation window when the application starts up:

public class MyApp : NavigationApplication
{
    NavigationWindow win;
    •••
    protected override void OnStartingUp(StartingUpCancelEventArgs e)
    {
        win = new NavigationWindow();

        // Add elements to the window
        •••
        navWin.Show();
    }
    •••
}

Let's switch back to the sample application shown in Figure 9. In light of the considerations I just discussed, you could either use Application or NavigationApplication as the base class because the application is going to use a single window, dialog-based application. The working class—named Sample1—overrides OnStartingUp and defines quite a few event handlers. Overriding the OnStartingUp method is a necessity because it represents the initialization step of the application startup and therefore it is the ideal location in which to perform your own manipulations before the window opens.

The operation accomplished in OnStartingUp is the creation of the window. The contents of the window are described in the XAML file and its codebehind class. The XAML page itself is constructed of controls and other components organized in a hierarchical tree. It is the relationship between these various components, called elements, that accounts for much of how a page renders and behaves. The sample page in Figure 9 has a TextBox control with a given size and font. The textbox is bound to the ContextMenuEvent event. The event is raised whenever the user right-clicks the client area of the control.

The event handler creates a ContextMenu object (see Figure 10). The context menu is populated and graphically configured, then bound to its parent object, which is a TextBox. It is worth noting that Longhorn allows you to easily customize the appearance of the menu (as well as that of all controls) by choosing background and foreground colors, borders, and fonts. Don't be fooled by the fact that this was also possible in previous versions of Windows and, of course, in the .NET Framework. In Win32, this sort of customization required a lot of programming work and was anything but simple. In the .NET Framework, wrapper classes buried only the necessary code in the folds of the Framework, exposing only a few properties to the user. In Longhorn, on the other hand, the user interface of a control is as simple as it looks.

The elements of a context menu are created from the MenuItem class. The way to wire them to event handlers is nearly identical to that of a Framework-based application:

mia = new MenuItem[3];
for (int i=0; i<3; i++)
{
    mia[i] = new MenuItem();
    cm.Items.Add(mia[i]);
    mia[i].Foreground = Brushes.Black;
}
mia[0].Header = "Lower Text";
mia[1].Header = "Upper case";
mia[2].Header = "Select all";
mia[0].Click += new ClickEventHandler(LowerCase);
mia[1].Click += new ClickEventHandler(UpperCase);
mia[2].Click += new ClickEventHandler(SelectAll);

When a particular menu item is clicked, the event handler executes and retrieves a reference to its source object using the first argument that gets passed:

public void LowerCase(Object sender, ClickEventArgs args)
{
    MenuItem mi = (MenuItem) args.Source;
    ContextMenu menu = (ContextMenu) mi.Parent;
    TextBox thisTextBox = (TextBox) menu.PlacementTarget;
    thisTextBox.Text = thisTextBox.Text.ToLower();
}

To retrieve the TextBox, you have to walk your way up to the tree. First, get to the MenuItem, then to the ContextMenu, and finally you can access its owner, the TextBox. At this point, modifying the contents of the TextBox is straightforward.

Conclusion

Longhorn represents a landmark in the history of the Windows operating system. It will be the first version of Windows designed specifically around managed code. Further establishing the significance of Longhorn is its new application model; its design and implementation is the culmination of previous experience in application modeling for .NET Framework-based applications and ASP.NET applications in particular.

Longhorn applications are built around the Application object and enable you to make a collection of markup pages into a coherent application. The presence of the XAML language behind Longhorn apps is also a sign of the abstraction level around which the Longhorn application model has been designed. The ultimate goal of Longhorn is to let you write applications once and deploy them in a variety of scenarios, both Web- and client-based.

Introducing the application model of a new platform that very few people have experimented with is not an easy task. It is even more difficult if you consider the likelihood that some aspects will change significantly in the future. In light of this, think of this article as snapshot of where Longhorn is now and a glimpse at where it is going.

I've included a sample project as a code download to help you get up and running with Longhorn. You can get this project from the link at the top of this article.

Dino Esposito is an instructor and consultant based in Rome, Italy. Author of Programming ASP.NET, for Microsoft Press, he spends most of his time teaching classes on ADO.NET and ASP.NET and speaking at conferences. Get in touch with Dino at cutting@microsoft.com.

© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


Code Name Indigo: A Guide to Developing and Running Connected Systems with Indigo

Code Name Indigo

A Guide to Developing and Running Connected Systems with Indigo

Don Box
This article was based on a pre-PDC build of Microsoft Windows Code Name "Longhorn" and all information contained herein is subject to change.
Note: This document was developed prior to the product's release to manufacturing and, as such, we cannot guarantee that any details included herein will be exactly the same as those found in the shipping product. The information represents the product at the time this document was printed and should be used for planning purposes only. Information subject to change at any time without prior notice.
SUMMARY

This article describes a collection of new programming frameworks that are part of "Longhorn," the upcoming version of Windows. "Indigo," the code name for this framework, provides rich support for service-oriented design that is complementary to traditional object-oriented approaches. Indigo marries the best features of .NET Remoting, ASMX, and .NET Enterprise Services into a unified programming and administration model. Indigo's deep support for standard protocols, including HTTP, XML, and SOAP, makes it easier to integrate applications and services without sacrificing security or reliability.

Contents
Service-orientation Fundamentals
What is Indigo?
Programming Indigo
Where Are We?

T
he upcoming version of Microsoft® Windows® code-named "Longhorn" will include a unified programming model and communications infrastructure for developing connected systems. This infrastructure, code-named "Indigo," simplifies development through a service-oriented programming model in which autonomous programs are composed using asynchronous message passing. To enable this programming model, Indigo provides a rich set of technologies for creating, consuming, processing, and transmitting messages.

One of the key benefits of Indigo is that it provides a unified programming model and protocol stack for all common language runtime (CLR)-based remoting technologies. Indigo marries the best of .NET Remoting, ASMX, System.Messaging, and .NET Enterprise Services into a single framework, allowing developers to freely combine features such as declarative transactions, pluggable interceptors and transports, and XML Schema-based serialization.

Moreover, Indigo unifies a wide range of transports (HTTP, TCP, UDP, IPC), security mechanisms (public and symmetric keys, certificates), topologies (point-to-point, end-to-end through intermediaries, peer-to-peer, publish-and-subscribe), and assurances (transacted, reliable, durable) enabling Indigo to provide rich connectivity to many existing systems.

Indigo makes service-oriented programming viable in a broad spectrum of mainstream applications. By taking advantage of various facilities of both the CLR and Windows, Indigo can be used in performance-sensitive situations such as single-host and even single-process integration. This scale-invariance makes Indigo-based services accessible to a broader range of deployment options than current technologies.

Because interoperability and reach are important, Indigo has deep support for both XML and SOAP, as well as for the emerging advanced Web Service protocols that add security, reliability, and transactions to SOAP-based applications.

As part of Windows Longhorn, Indigo is available to every Longhorn application. Indigo will also be available as a separate download for Windows XP and Windows Server™ 2003. A small number of advanced features or optimizations of Indigo may only be available on Longhorn due to intrinsic differences between the various operating systems. However, Indigo provides the developer with a consistent service-oriented programming model no matter which operating system it is deployed on.

Service-orientation Fundamentals

Indigo is based on the principles of service-oriented architecture and programming. Service-orientation is an important complement to object-orientation that applies the lessons learned from component software, message-oriented middleware and distributed object computing. Service-orientation differs from object-orientation primarily in how it defines the term "application." Object-oriented development focuses on applications that are built from interdependent class libraries. Service-oriented development focuses on systems that are built from a set of autonomous services. This difference has a profound impact on the assumptions one makes about the development experience.

In Indigo, a service is simply a program that one interacts with via message exchanges. A set of deployed services is a system. Individual services are built to last—the availability and stability of a given service is critical. The aggregate system of services is built to allow for change—the system must adapt to the presence of new services that appear a long time after the original services and clients have been deployed, and these must not break functionality.

Service-oriented development is based on the four fundamental tenets that follow:

Boundaries are explicit A service-oriented application often consists of services that are spread over large geographical distances, multiple trust authorities, and distinct execution environments. The cost of traversing these various boundaries is nontrivial in terms of complexity and performance. Service-oriented designs acknowledge these costs by putting a premium on boundary crossings. Because each cross-boundary communication is potentially costly, service-orientation is based on a model of explicit message passing rather than implicit method invocation. Compared to distributed objects, the service-oriented model views cross-service method invocation as a private implementation technique, not as a primitive construct—the fact that a given interaction may be implemented as a method call is a private implementation detail that is not visible outside the service boundary.

Though service-orientation does not impose the RPC-style notion of a network-wide call stack, it can support a strong notion of causality. It is common for messages to explicitly indicate which chain(s) of messages a particular message belongs to. This indication is useful for message correlation and for implementing several common concurrency models.

The notion that boundaries are explicit applies not only to inter-service communication but also to inter-developer communication. Even in scenarios in which all services are deployed in a single location, it is commonplace for the developers of each service to be spread across geographical, cultural, and/or organizational boundaries. Each of these boundaries increases the cost of communication between developers. Service orientation adapts to this model by reducing the number and complexity of abstractions that must be shared across service boundaries. By keeping the surface area of a service as small as possible, the interaction and communication between development organizations is reduced. One theme that is consistent in service-oriented designs is that simplicity and generality aren't a luxury but rather a critical survival skill.

Services are autonomous Service-orientation mirrors the real world in that it does not assume the presence of an omniscient or omnipotent oracle that has awareness and control over all parts of a running system. This notion of service autonomy appears in several facets of development, the most obvious place being the area of deployment and versioning.

Object-oriented programs tend to be deployed as a unit. Despite the Herculean efforts made in the 1990s to enable classes to be independently deployed, the discipline required to enable object-oriented interaction with a component proved to be impractical for most development organizations. When coupled with the complexities of versioning object-oriented interfaces, many organizations have become extremely conservative in how they roll out object-oriented code. The popularity of the XCOPY deployment and private assemblies capabilities of the .NET Framework is indicative of this trend.

Service-oriented development departs from object-orientation by assuming that atomic deployment of an application is the exception, not the rule. While individual services are almost always deployed atomically, the aggregate deployment state of the overall system/application rarely stands still. It is common for an individual service to be deployed long before any consuming applications are even developed, let alone deployed into the wild. Amazon.com is one example of this build-it-and-they-will-come philosophy. There was no way the developers at Amazon could have known the multitude of ways their service would be used to build interesting and novel applications.

It is common for the topology of a service-oriented application to evolve over time, sometimes without direct intervention from an administrator or developer. The degree to which new services may be introduced into a service-oriented system depends on both the complexity of the service interaction and the ubiquity of services that interact in a common way. Service-orientation encourages a model that increases ubiquity by reducing the complexity of service interactions. As service-specific assumptions leak into the public facade of a service, fewer services can reasonably mimic that facade and stand in as a reasonable substitute.

The notion of autonomous services also impacts the way failures are handled. Objects are deployed to run in the same execution context as the consuming application. Service-oriented designs assume that this situation is the exception, not the rule. For that reason, services expect that the consuming application can fail without notice and often without any notification. To maintain system integrity, service-oriented designs use a variety of techniques to deal with partial failure modes. Techniques such as transactions, durable queues, and redundant deployment and failover are quite common in a service-oriented system.

Because many services are deployed to function over public networks (such as the Internet), service-oriented development assumes not only that incoming message data may be malformed but also that it may have been transmitted for malicious purposes. Service-oriented architectures protect themselves by placing the burden of proof on all message senders by requiring applications to prove that all required rights and privileges have been granted. Consistent with the notion of service autonomy, service-oriented architectures invariably rely on administratively managed trust relationships in order to avoid per-service authentication mechanisms common in classic Web applications.

Services share schema and contract, not class Object-oriented programming encourages developers to create new abstractions in the form of classes. Most modern development environments not only make it trivial to define new classes, modern IDEs do a better job guiding you through the development process as the number of classes increases (as features like IntelliSense® provide a more specific list of options for a given scenario).

Classes are convenient abstractions as they share both structure and behavior in a single named unit. Service-oriented development has no such construct. Rather, services interact based solely on schemas (for structures) and contracts (for behaviors). Every service advertises a contract that describes the structure of messages it can send and/or receive as well as some degree of ordering constraints over those messages. This strict separation between structure and behavior vastly simplifies deployment, as distributed object concepts such as marshal-by-value require a common execution and security environment which is in direct conflict with the goals of autonomous computing.

Services do not deal in types or classes per se; rather, only with machine readable and verifiable descriptions of the legal "ins and outs" the service supports. The emphasis on machine verifiability and validation is important given the inherently distributed nature of how a service-oriented application is developed and deployed. Unlike a traditional class library, a service must be exceedingly careful about validating the input data that arrives in each message. Basing the architecture on machine-validatible schema and contract gives both developers and infrastructure the hints they need to protect the integrity of an individual service as well as the overall application as a whole.

Because the contract and schema for a given service are visible over broad ranges of both space and time, service-orientation requires that contracts and schema remain stable over time. In the general case, it is impossible to propagate changes in schema and/or contract to all parties who have ever encountered a service. For that reason, the contract and schema used in service-oriented designs tend to have more flexibility than traditional object-oriented interfaces. It is common for services to use features such as XML element wildcards (like xsd:any) and optional SOAP header blocks to evolve a service in ways that do not break already deployed code.

Service compatibility is determined based on policy Object-oriented designs often confuse structural compatibility with semantic compatibility. Service-orientation deals with these two axes separately. Structural compatibility is based on contract and schema and can be validated (if not enforced) by machine-based techniques (such as packet-sniffing, validating firewalls). Semantic compatibility is based on explicit statements of capabilities and requirements in the form of policy.

Every service advertises its capabilities and requirements in the form of a machine-readable policy expression. Policy expressions indicate which conditions and guarantees (called assertions) must hold true to enable the normal operation of the service. Policy assertions are identified by a stable and globally unique name whose meaning is consistent in time and space no matter which service the assertion is applied to. Policy assertions may also have parameters that qualify the exact interpretation of the assertion. Individual policy assertions are opaque to the system at large, which enables implementations to apply simple propositional logic to determine service compatibility.

What is Indigo?

Indigo is a set of .NET Framework-based technologies for building and running connected systems. Indigo is the next step in the evolutionary path that started with COM, COM+, and MSMQ, and continues through .NET Remoting, ASMX, System.Messaging, and .NET Enterprise Services. Indigo subsumes these technologies and offers a single unified programming experience for developing services using any CLR-compliant language.

Figure 1 Indigo Architecture
Figure 1 Indigo Architecture

As shown in Figure 1, the major subsystems of Indigo are the Indigo service model, the Indigo connector, hosting environments, and system and messaging services.

Indigo service model The Indigo service model makes service-oriented development explicit and simple from any CLR-targeted language. Developers use declarative attributes to mark up which aspects of their type should form the external contract of a service. The Indigo service model supports two kinds of contracts: data contracts and service contracts. Data contracts are structural in nature and describe the external format of a CLR type. A data contract is analogous to an XML Schema definition. A service contract is behavioral in nature and describes the pattern of message exchanges that are used to interact with a service. Service contracts are analogous to WSDL portType definitions. The Indigo service model supports both code-first and contract-first development models, including support for Web Service Description Language (WSDL) and XML Schema import/export.

The primary function of the Indigo service model is to associate user-defined code with incoming messages. This is accomplished using the [ServiceMethod] attribute. Methods marked with this attribute support either one-way messaging or correlated request/reply messaging, depending on the value of the OneWay property. Methods marked as service methods can be declared to work in terms of the entire message or (if desired) in terms of typed parameters. The messages themselves can be viewed as either strongly-typed objects or as XML information sets (infosets).

The Indigo service model provides instance and context management for a service. Indigo routes incoming messages to instances of user-defined service types. Developers use declarative attributes to control how instances are associated with incoming messages as well as how the session management feature of Indigo routes multiple messages to a common session object. Additionally, the Indigo service model provides developers with explicit control over the propagation of execution context (causality, call context, transactions) between cooperative services.

Finally, the Indigo service model provides declarative behaviors that automate security and reliability. These attributes influence the behavior of Indigo as methods enter and leave a service, enforcing the policies that are initially declared by the developer and then made concrete through configuration by the administrator.

Like ASMX, Indigo enables services to be consumed without sharing assemblies or source code. Additionally, Indigo provides support for ASMX compatibility that will allow many ASMX-based Web Services to become Indigo-based services with relatively little modification.

The Indigo connector The Indigo connector is a managed framework that provides secure and reliable message-based connectivity. The Indigo connector is a layered I/O framework that is based on the SOAP data and processing model. The Indigo connector allows you to build message-based applications independent of transport or target platform using a small number of concepts (port, message, channel) all of which are described later in this article. The Indigo connector is designed to have a small footprint and excellent throughput and latency characteristics, which enables it to be used in performance-sensitive systems.

The Indigo connector is based on ports and channels. A port represents a location in network space and provides a base Uniform Resource Identifier (URI) for all services that it hosts. A channel is an I/O device that services use to send and receive messages through their port. Some channels are transport channels that map underlying communication mechanisms to Indigo. Other channels are messaging channels that provide a particular messaging discipline or quality of service on top of one or more transport channels. The connector also includes a message encoder that allows encodings to be added to the system to translate between the abstract data model used internally by Indigo and byte sequences used by a specific transport. The serialization layer takes the world of CLR objects and classes and distills them down to a simple data model that is suitable for sharing across the service boundaries.

The Indigo connector has explicit support for intermediaries that include firewalls, proxies, and gateways. The intermediary model of the Indigo connector is consistent with the SOAP processing model. While direct communication with a service is supported, Indigo assumes that a direct connection is the exceptional circumstance, not the rule. Instead, Indigo adapts to configuration, metadata, and service policy to determine the best route for a message and will take advantage of gateways or proxies as needed. Moreover, the Indigo connector vastly simplifies the creation of application-level gateways that can offload work from back-end services by providing message validation, security enforcement, and content-based load balancing.

The presence of intermediaries often means that there is no single end-to-end transport connection between the sender and ultimate receiver. To ensure that message delivery is reliable in these situations, Indigo implements end-to-end reliable messaging over SOAP using the WS-ReliableMessaging protocol. The Indigo connector provides two modes of reliable messaging: express and durable. Express messaging uses in-memory buffers to hold messages in transit as they travel from the source to the destination. There are no guarantees that an express message will be sent if the sender crashes immediately after sending the message. In contrast, durable messages use on-disk buffers and provide delivery guarantees even in the face of system crashes.

The Indigo connector provides intrinsic support for secure messaging that provides end-to-end protection independent of message transport. Both message-based and session-based modes are supported. Session-based security is the preferred model as both signing and encryption can use a more lightweight session key that is negotiated as part of session establishment. Message-based security is needed for disconnected or datagram scenarios in which the receiver of the message is not available at the time of transmission. Both modes support the WS-Security family of protocols (WS-Trust and WS-Federation), ensuring interoperation with non-Indigo Web Services even in the face of intermediaries. Indigo also takes advantage of transport-level security when possible (for example, when a direct TCP/Secure Sockets Layer (SSL) connection between sender and receiver is used).

Hosting environments The Indigo connector and service model can be explicitly loaded and used in any AppDomain on the system. This makes Indigo well suited for exposing a service-oriented facade to any piece of managed code. Initializing the Indigo connector in your AppDomain provides you with basic "service dialtone" independent of how your AppDomain is created. In fact, how your AppDomain is started is completely outside the scope of the role of the Indigo connector.

To make Indigo accessible to the widest possible range of developers, various hosting systems currently used on the Windows platform (such as svchost.exe and dllhost.exe) have been adapted to specifically support Indigo-based services. The most interesting of these hosting sytems is ASP.NET and IIS.

With the release of Indigo, ASP.NET now provides a system-wide facility for managing AppDomains and OS processes. This allows Indigo-based services to be demand-started based on messages that arrive on any properly configured transport. Indigo has built-in support for TCP, HTTP, and Interprocess Communication (IPC) transports. Indigo hosting allows applications to be associated with URI suffixes much like the IIS model for mapping URI suffixes to VRoots. Unlike IIS, however, Indigo is not tied to a particular transport protocol or programming model.

Indigo-based services that are hosted using ASP.NET gain an additional set of management services , including configurable recycling and passivation both at the process/AppDomain level as well as at the individual service level. Indigo also supports basic health monitoring for processes, and AppDomains. A notable new feature to ASP.NET's hosting facilities is the ability for service code to participate in service management (start, stop) requests. This helps services that have intermediate work avoid process recycling at inopportune moments. Most new server-based Indigo services are expected to use ASP.NET hosting, which results in a common configuration and management experience for all managed middle-tier code.

System and messaging services An Indigo-based system has several system services that provide essential functionality to all services. One of these services is identity federation. An upcoming version of the security system in Windows supports a federation service that allows identities from foreign trust boundaries to be managed and validated. The Windows federation service uses WS-Federation to securely broker the authentication between the service and the corresponding trust authority.

Indigo also provides significant support for transactional programming. Indigo-enabled versions of Windows support a service-based transaction manager that may be accessed via the System.Transactions framework or the WS-AtomicTransactions protocol. The new System.Transactions framework makes transactional programming simple and efficient throughout the platform (it supports SQL Server™, ADO.NET, MSMQ, distributed transaction coordinator (DTC), etc). System.Transactions supports both an explicit programming model based on the ITransaction interface as well as an implicit programming model in which transactions are automatically managed by Indigo. Both models are available to Indigo-based apps.

System.Transactions provides a new in-memory transaction manager that allows volatile transactions (that is, transactions that do not involve durable resources) to commit or roll back efficiently without incurring any disk I/O. To support durability, the in-memory transaction manager will transparently promote volatile transactions to durable transactions by coordinating through a disk-based transaction manager like the DTC the moment a durable resource manager enlists itself with a transaction.

System.Transactions defines a simple managed interface for writing both volatile and durable resource managers. System.Transactions also supports any resource manager that can speak either OLE transactions or the broadly adopted WS-AtomicTransaction protocol. To enable efficient transacted access to the file system, the Longhorn version of System.Transactions directly supports both the Kernel Transaction Manager (KTM) and the Transactional NTFS (TxNTFS).

Indigo also provides implementations of many of the common abstractions used in messaging applications. Specifically, Indigo provides simple yet extensible versions of queues, eventing, and content-based routing that can be used in any Indigo-based app.

Programming Indigo

The Indigo model is based on four simple concepts. First, a message is a transmissible piece of data that adheres to the SOAP data model. Messages have zero or more headers and a body and in Indigo are represented by the concrete Message class. Though Indigo allows objects with arbitrary methods to be stored in a message, all message parts may be interacted with uniformly according to the SOAP and XML data models.

Second, messages flow between ports. Indigo ports are endpoints for communications between services. A port can send and receive messages independent of transport. Ports come in two flavors: anonymous and named. Named ports can be explicitly addressed independent of any session or connection. Anonymous ports can only be addressed implicitly as part of an established communication session. The only messages an anonymous port will receive are responses to messages sent from that port.

Third, a service is an opaque piece of code behind a port. Indigo refers to the executable code that resides behind a port as a service. Services have both policy and contract; policy indicates the capabilities and requirements of the service and contract indicates the acceptable message exchanges that the service can participate in.

Fourth, services communicate with their ports via channels. Services can create and use a range of channels to build an application; the properties, policies, and behaviors of a channel vary based on the characteristics of the communication services the channel provides. For example, a dialog channel can be created that connects another service (identified by a URL). The dialog channel provides the ability to set a transfer assurance policy of "exactly-once, in-order" delivery; such a channel can be used to ensure messages are delivered reliably.

The diagram in Figure 2 shows the relationship between messages, ports, channels, and services.

Figure 2 Relationships
Figure 2 Relationships

The easiest way to make these concepts concrete is to look at some code. The code shown in this section is written against the PDC build of Indigo—naturally the details are subject to change between now and the final release. Here is the smallest possible Indigo service written in C# using only the Indigo connector:

using System;
using System.MessageBus;

class app {
  static void Main() {
// create and open a named port
    Port port = new Port(new
                 Uri("soap://localhost/simple"));
    port.Open();
// wait forever
    Console.ReadLine();
  }
}

This service creates a new named port whose relative URI is "/simple" and is accessible over all installed transports. Indigo treats the "soap:" URI scheme as the transport-neutral address of the service. The Indigo connector will project this address onto both TCP and HTTP transports automatically, enabling the service to be addressed via any of these URIs:

soap://localhost/simple
http://localhost/simple
soap.tcp://localhost/simple
Unless an application needs to explicitly control transport selection, soap: is the preferred URI scheme and therefore is used throughout the remainder of this article.

The service just shown is extremely flexible—you can send it any sort of message over any transport and it will silently drop the message onto the floor. Writing code that touches the message requires a little more work. Figure 3 shows a service that when sent a message whose content is a simple string, prints that string to the console.This service assumes that the body of the message is a simple string and will consistently and reliably fail if the body of the message is anything else.

Figure 3 Print Single Message (Close Figure 3)
using System;
using System.MessageBus;

class MsgPrint : SyncMessageHandler {
  public override bool ProcessMessage(Message msg)
  {
    Console.WriteLine(
        msg.Content.GetObject(typeof(string))
    );
    msg.Close();
    return false;
  }
}

class app {
  static void Main() {
    Port port = new Port(new
                 Uri("soap://localhost/simple"));
    port.ReceiveChannel.Handler = new MsgPrint();
    port.Open();
// wait forever
    Console.ReadLine();
  }
}

To allow services to cope with the body in a more data-oriented form, Indigo also gives you access to message content as raw XML-structured data using System.Xml. The service in Figure 4 dumps the content of every message into an XML text file.

Figure 4 Dump Message to File (Close Figure 4)
using System;
using System.Xml;
using System.MessageBus;

class MsgDump : SyncMessageHandler {
  public override bool ProcessMessage(Message msg)
  {
    string fn = String.Format("output{0}.xml",
                              DateTime.Now.Ticks);
    XmlWriter xw = new XmlTextWriter(fn, null);
    xw.WriteNode(msg.Content.Reader, false);
    xw.Close();
    msg.Close();
    return false;
  }
}

class app {
  static void Main() {
    Port port = new Port(new
                 Uri("soap://localhost/simple"));
    port.ReceiveChannel.Handler = new MsgDump();
    port.Open();
// wait forever
    Console.ReadLine();
  }
}

This program takes advantage of the fact that the body of the message is always accessible via a streaming XmlReader interface. This capability is important when dealing with large messages, as the forward-only traversal model of XmlReader allows incremental processing to take place along the way as message data is streamed into memory.

The Indigo connector defines a small kernel of managed types for processing messages. The most central of these types is the concrete Message class, which is shown in Figure 5. The Message class maintains an ordered list of headers and a single body which represents the content of the message. For convenience, the Action header (as defined by WS-Addressing as well as the HTTP bindings for SOAP) is called out as a distinguished property. Each message header implements the abstract MessageHeader interface (shown in Figure 6) that exposes core SOAP-isms such as mustUnderstand and actor/role. The body of the message must implement the abstract MessageContent interface which supports both object-based and XML-based access.

Figure 6 Message Headers and Body (Close Figure 6)
public abstract class MessageHeader {
// protected constructors
  protected MessageHeader(bool mustUnderstand, Uri role);
  protected MessageHeader(MessageHeader source);

// SOAP processing/data model properties
  public abstract string Name { get; }
  public abstract string Namespace { get; }
  public bool MustUnderstand { get; }
  public Uri Role { get; }
  public bool DidUnderstand { get; set; }

// header serialization methods and properties
  public abstract MessageHeader Clone();
  public virtual XmlElement Element { get; }
  public abstract void WriteTo(XmlWriter writer);
  public abstract bool CanWrite { get; }
  public virtual bool CanTransmit { get; }
}

public abstract class MessageContent {
// object I/O
  public abstract object GetObject(Type type);

// XML I/O
  public virtual XmlReader Reader { get; }
  public abstract void WriteTo(XmlWriter writer);

// async XML I/O support
  public virtual IAsyncResult BeginWriteTo(XmlWriter writer,
                                           AsyncCallback callback,
                                           object state);
  public virtual void EndWriteTo(IAsyncResult result);

// public properties and methods
  public virtual bool IsEmpty { get; }
  public virtual bool IsException { get; }

  public abstract MessageContent Clone();
  public virtual void Close();

}
Figure 5 The Message Class (Close Figure 5)
public class Message  {

// public constructors
  public Message();
  public Message(Uri action);
  public Message(Uri action, object obj);
  public Message(Uri action, MessageContent content);
  public Message(MessageException exception);

// create message targeted at sender
  public Message CreateReverse();
  public Message CreateReverse(Uri action);
  public Message CreateReverse(Uri action, object obj);
  public Message CreateReverse(Uri action, MessageContent content);
  public Message CreateReverse(MessageException exception);

// create correlated reply targeted at sender
  public Message CreateReply();
  public Message CreateReply(Uri action);
  public Message CreateReply(Uri action, object obj);
  public Message CreateReply(Uri action, MessageContent content);
  public Message CreateReply(MessageException exception);

// public operations
  public Message Clone();
  public void Close();

// public properties
  public Uri Action { get; set; }
  public MessageHeaderCollection Headers { get; }
  public MessageContent Content { get; set; }
}

Next to the Message class, the most important message processing type is easily IMessageHandler. The IMessageHandler interface represents a piece of message processing code and has a single method that that is used to invoke the message processor: ProcessMessage.

public interface IMessageHandler {
  bool ProcessMessage(Message msg);
// async support elided for clarity
}

In addition to the primary ProcessMessage method, the IMessageHandler interface also provides several methods for interacting with the IAsyncResult idiom used by asynchronous method execution. Implementations that want the default asynchronous behavior (such as the examples shown at the beginning of this section) often derive from the SyncMessageHandler abstract base class and provide only a ProcessMessage method body.

Recall that messages support streaming I/O over the body of the message. This means that message handlers that traverse the body effectively render the message useless for all subsequent use. For this reason, the ProcessMessage method returns a boolean indicating whether or not the message has been "consumed." If the body has not been consumed, handlers return true from ProcessMessage indicating that the message is still viable for further processing. If a handler consumes the message body, it is required to return false from ProcessMessage and is required to call Close on the message object as well, freeing up any of the underlying resources held by the message.

The following example demonstrates how to consume a message using a series of message handlers:

static void
ConsumeMessage(IList<IMessageHandler> code, Message data) {
  bool live = true;
  for (int i=0; i < code.Count && live; i++)
    live = code[i].ProcessMessage(data);

  if (live)
    data.Close();
}

The third and final core message processing type is IMessageProducer. IMessageProducer is a simple interface that models message sources and, as shown here, has only one public member, the Handler property:

public interface IMessageProducer {
  IMessageHandler Handler { get; set; }
}

You've already seen an implementation of this interface. The Port.ReceiveChannel property used in the previous service examples implements the IMessageProducer interface. Code that wants to process messages that arrive on a port simply attach themselves through the Handler property of the receive channel.

Up to this point, you have only seen how messages are received. Indigo obviously allows messages to be sent as well—this capability is exposed via a port's send channels. Every port has a default send channel (exposed via the Port.SendChannel property) that can be used to send a message that is explicitly addressed using a To header (per the WS-Addressing specification). This channel is typically only used to send reply messages created by the Message.CreateReply method:

static bool Process(Port port, Message request)
{
    Message data = request.CreateReply(null,
                                       "Blue");
    IMessageHandler channel = port.SendChannel;
    if (channel.ProcessMessage(data))
      data.Close();
}

The more common usage is to create a per-address send channel by calling the Port.CreateSendChannel method as follows:

static void SendAMessage(Port source,
                         Uri target) {
{
    Message data = new Message(null, "Cyan");
    IMessageHandler channel =
            source.CreateSendChannel(target);
    if (channel.ProcessMessage(data))
      data.Close();
}
In this case, the send channel will add the required addressing headers to the message prior to handing it off to the underlying transport manager.

Where Are We?

This article has provided just a glimpse into the capabilities of and the motivations behind Indigo. Given the broad scope of the Indigo project, many aspects of the Indigo programming experience have been glossed over by this introductory article. As the beta release of Indigo grows nearer, watch the MSDN® Web Services Developer Center for more complete coverage of Indigo.

Don Box is an architect on the Indigo project at Microsoft, working on next generation Web Service protocols and plumbing. His interests include type systems for XML and Web Services, metadata, discovery, and service-oriented software integration. Don's work with Web Services began in 1998 as one of the original authors of the SOAP specification. Don's latest book is Essential .NET Volume 1: The Common Language Runtime (Addison-Wesley, 2002).


Code Name Avalon: Create Real Apps Using New Code and Markup Model

Code Name Avalon

Create Real Apps Using New Code and Markup Model

Charles Petzold
This article was based on the pre-PDC build of Microsoft Windows Code Name "Longhorn" and all information contained herein is subject to change.
Code download available at: Avalon.exe (141KB) Note: This document was developed prior to the product's release to manufacturing and, as such, we cannot guarantee that any details included herein will be exactly the same as those found in the shipping product. The information represents the product at the time this document was printed and should be used for planning purposes only. Information subject to change at any time without prior notice.
SUMMARY

The presentation subsystem in the next version of Windows, code-named "Longhorn," offers powerful new capabilities to developers. This subsystem, code-named "Avalon," allows developers to take advantage of its capabilities through a new markup language code-named "XAML." In addition, modern object-oriented programming languages such as C# and Visual Basic .NET can be used to tie everything together. Because most applications written to Avalon will probably be a mix of XAML and programming code, this article discusses XAML tags used to control page layout along with the procedural code written to respond to events.

Contents
Dueling Interfaces
Layout Options
Event Handling
Elements and Objects
Animation
Styles
An Avalon Color Scroll
Conclusion

F
or thousands of years, philosophers and scientists have attempted to explain the cosmos in terms of dual opposing but coexistent principles: good and evil, yin and yang, matter and energy, mind and body, waves and particles, and, of course, programming code and markup languages.

Programming and markup currently coexist in an uneasy truce. In theory, programming languages can do anything the computer is capable of, but they're often clunky for the job of laying out text, images, and controls in a simple visual interface. Markup is great for defining highly textured pages of text and images that adapt to different screen sizes and environments, but is hopelessly inept when it comes time to interact with the user in any nontrivial way.

In creating a new programming interface for building Windows®-based client applications, the developers at Microsoft have decided not to deny this dualism, but to embrace and celebrate it. They have created an environment in which programming and markup boldly and intricately mesh in mutually supporting roles. The result—the presentation subsystem code-named "Avalon"—may well be the greatest experiment in synergistic duality since Adam and Eve. Vive la différence!

Dueling Interfaces

Avalon, part of the next version of Windows, code-named "Longhorn," consists mostly of a new collection of classes added to the .NET Framework. The most important new namespaces for Avalon programming currently have names like MSAvalon.Windows, MSAvalon.Windows.Controls, and MSAvalon.Windows.Media. (Expect these names to change before Longhorn reaches final release.) With Avalon you can write applications in C#, Visual Basic® .NET, or any other language that supports the .NET Common Language Specification (CLS). These programs are quite similar to Windows Forms applications that you can write today. That's the normal part of Avalon.

In addition, Avalon also defines a new markup language you can use in Longhorn that's code-named "XAML"—Extensible Application Markup Language (pronounced "zammel"). You use XAML much like HTML to define a layout of text, images, and controls. Being based on XML, XAML has stricter and much less ambiguous syntax than HTML. It is expected that most XAML will be machine-generated by visual design programs, but it may be a good learning experience to hand-write your own XAML (initially).

Most applications written to Avalon will probably contain both program code and XAML. You'll use the XAML for defining the initial visual interface of your application, and write code for doing everything else. You can embed the program code directly in XAML or keep it in a separate file. Everything you can do in XAML you can also do in program code, so it's possible to write a program without using any XAML at all. The reverse is not true, however; there are many tasks that can only be done in program code, so only the simplest applications will consist entirely of XAML. Here's a little snippet of some XAML:

<Button Background="LightSeaGreen" FontSize="24pt">
    Calculate
</Button>

This snippet is a single XML element, which consists of a start tag, an end tag, and content between the two tags. The element type is Button. The start tag also includes two attribute specifications; the attribute names are Background and FontSize. They are assigned attribute values, which, under the requirements of XML, must be enclosed in single or double quotation marks. Between the start tag and end tag is the element content, which in this case is the text that appears on the face of the button.

What this Button element does not include is a width or height. Normally, buttons and other controls under Avalon are automatically sized to fit their content. (You can override that behavior, of course.) In this particular case, the Button is made large enough to accommodate the text string "Calculate" in a 24-point font. This Button element also does not include a coordinate location. Generally controls are dynamically positioned in a window at run time based on the size of the window and the controls.

XAML has very intimate ties with the Avalon class library: every element type you can use in XAML is actually a class and, specifically, a descendent of the UIElement or ContentElement classes declared in the MSAvalon.Windows namespace. Among the descendents of UIElement is Control, from which are descended all the common user-interface controls such as buttons, scroll bars, list boxes, edit fields, and so forth. Classes derived from ContentElement include Bold and Italic.

The attribute names you specify in the XAML start tags are actually properties of these classes. Properties have always played an important role in the .NET Framework and here are elevated to an even higher stature. If you haven't yet done any .NET Framework programming, you should know that properties are members of classes just like methods and fields. In use, they look like fields but they are implemented more like methods. Unlike fields, properties contain code. Properties can be readable, writable, or both. A read/write property named Background is basically equivalent to a pair of symmetric methods named set_Background and get_Background.

Of course, the modern programmer wants to know: can I use my own classes as XAML elements? And the answer is: of course you can. XAML is called the Extensible Application Markup Language for a reason. Any class that has a public parameterless constructor and settable properties can be used in XAML.

The Button element I just showed is equivalent to code that creates an object of type Button and then assigns the Background, FontSize, and Content properties. The equivalent C# code is:

Button btn = new Button();
btn.Background = Brushes.LightSeaGreen;
btn.FontSize = new FontSize(24, FontSizeType.Point);
btn.Content = "Calculate";

As you can see, defining the object in XAML considerably simplifies the assignment of these three properties. The Brushes class consists of static properties for all the predefined colors (the same colors commonly supported in HTML). FontSize is a structure encapsulating a unit-independent measurement. FontSizeType is an enumeration. Regardless of whether a property is defined as a string, an integer, a floating-point value, a Boolean, an enumeration, or some other type, the equivalent XAML attribute is simply assigned a text string.

The Content property indicates what appears on the surface of the button and corresponds to the content of the Button element in XAML. This means that the Button element could also be written in this way:

<Button Background="LightSeaGreen" FontSize="24pt" Content="Calculate">
</Button>

Because this Button element sets the Content attribute and no longer includes any content between its opening and closing tags, it could be written as an empty element tag:

<Button Background="LightSeaGreen" FontSize="24pt" Content="Calculate" />
Notice the closing slash at the end of the element.

In this example Content seems to be a property of type string, but that's not the whole story. The Content property is declared to be of type object, strongly suggesting that the button content doesn't necessarily have to be a simple text string. It could, for example, be an Image element:

<Button> <Image Source="Calc.jpg"/> </Button>

If you want your button to display both text and an image, you can include both:

<Button>
    Calculate
    <Image Source="Calc.jpg"/>
</Button>

The text and image elements will appear side by side. (Such elements are often referred to as children of the parent Button element.) The Button element doesn't support more sophisticated layout qualities, but if you want it, it's available. The trick is to specify that the content of the Button is an XAML element such as Text that supports flexible layout options. Here's a button whose content is a single Text element, which in turn contains an image, a line break, and a text string:

<Button FontSize="24pt">
    <Text>
        <Image Source="Calc.jpg"/>
        <LineBreak/>
        Calculate
    </Text>
</Button>

The LineBreak element is equivalent to <br> in HTML. A button that displays an image with text underneath might be appropriate for a tool bar. You could make it a bit more dramatic with some italic text formatting:

<Button FontSize="24pt">
    <Text>
      <Image Source="Calc.jpg"/>
      <LineBreak/>
      Calculate <Italic>this</Italic>
    </Text>
</Button>
This is a good example of something that in markup looks very natural, but which is rather awkward when expressed in code. You can, in fact, specify all of these elements in code, but why would you want to?

Now that you've had a little taste of XAML, let's look at some complete programs. Figure 1 shows a simple XAML implementation of the traditional Hello World program. Only one root element is allowed in XML, and in XAML that root element is generally an output surface. TextPanel is an output surface that provides automatic layout similar to HTML. Normally you'll have one XAML file for each page of your application (if, indeed, your application has pages), and also a XAML file for each dialog box. A feature called Styles (that I'll discuss a little later on) can help you maintain a consistent visual style throughout your program.

Figure 1 Hello World (Close Figure 1)
<TextPanel xmlns="http://schemas.microsoft.com/2003/xaml"
        Background="BlanchedAlmond"
        FontFamily="Comic sans MS"
        FontSize="36pt"
        HorizontalAlignment="Center">
    Hello, world!
</TextPanel>

The xmlns (XML namespace) attribute in the TextPanel element serves the same function as the using directive in C#. Because all the elements in a XAML file map to equivalent classes in the Microsoft .NET Framework, the xlmns attribute provides a URI pointing to a file that lists the names of the .NET namespaces where the classes are declared.

Just to make the Hello World program interesting, I added a few attributes to the TextPanel element to set colors and a font, and to specify that the element's children are centered horizontally within the panel. The content of the TextPanel element is the character data "Hello, world!" The content could also include images, controls, more text, and so forth.

Since it has no code in it, you can load the HelloWorld.xaml file directly into the Longhorn version of Microsoft Internet Explorer, and you'll see something that looks like a Web page. You can also compile HelloWorld.xaml using a program currently called MSBuild. You'll need a couple other short files (not shown here) for this compilation. A file with the extension PROJ or MSPROJ provides some information about the program and lists all the required source files (XAML and otherwise). Another short XAML file is also required to indicate the XAML page that is first displayed when the program is executed. Run the Hello World executable and you see something that looks like a Windows program. Figure 2 shows both versions.

Figure 2 Hello World as a Page and a Program
Figure 2 Hello World as a Page and a Program

The roughly equivalent C# version of this program is shown in Figure 3. (I say roughly equivalent because MSBuild is capable of emitting C# code, and this is not exactly what it emits. My version is considerably less abstruse.) You can compile this program on the command line like this:

csc /r:WindowsBase.dll;PresentationCore.dll;PresentationFramework.dll
Anyone who has experience with Windows Forms will find the structure and look of this program vaguely familiar. The program is responsible for explicitly creating an application object and a window object before creating the TextPanel object. These preliminary jobs are implied in the XAML version of the program.

Figure 3 Hello World App in C# (Close Figure 3)
using System;
using MSAvalon.Windows;
using MSAvalon.Windows.Controls;
using MSAvalon.Windows.Media;

class HelloWorldApp: Application
{
    [STAThread]
    static void Main()
    {
        HelloWorldApp app = new HelloWorldApp();
        app.Run();
    }
    protected override void OnStartingUp(StartingUpCancelEventArgs args)
    {
        Window win = new Window();
        TextPanel tp = new TextPanel();

        tp.Background = Brushes.BlanchedAlmond;
        tp.FontFamily = "Comic sans MS";
        tp.FontSize = new FontSize(72f, FontSizeType.Point);
        tp.HorizontalAlignment =
                        MSAvalon.Windows.HorizontalAlignment.Center;
        tp.TextRange.Text = "Hello, world";

        win.Children.Add(tp);
        win.Show();
    }
}

The odd part of this C# code involves the text string displayed in the window. Normally in a Windows program you'd specify a coordinate position where the text is to appear, but TextPanel incorporates automatic layout that arranges the panel's content based on the size of the window and the content. This is one way in which HTML has now influenced client application programming: with Avalon you can take advantage of automatic layout in your programming code.

Layout Options

The root element of an XAML file is generally a surface on which other elements appear. For this purpose, Avalon includes a Panel class and several descendents of Panel, including the TextPanel element used in HelloWorld.

FlowPanel is similar to TextPanel. Like HTML, both provide automatic layout based on the size of the elements and the size of the window. TextPanel has more sophisticated text-formatting abilities, and can even arrange text into multiple columns. FlowPanel is the more efficient choice if extensive text formatting is not required.

If you want to use a FlowPanel with the HelloWorld program, you'll have to change the way you specify text that appears on the panel. Unlike TextPanel, FlowPanel does not allow text content, but it does allow content consisting of a Text element. You'd need to use a Text element and move the FontFamily and FontSize attributes into it:

<Text FontFamily="Comic Sans MS" FontSize=36pt>Hello, world!</Text>

Another panel option is DockPanel, which arranges its child elements in a tiled fashion, fastened to the edges of the panel. Windows Explorer is a typical example of an application that uses docking; there's a tool bar at the top, a status bar at the bottom, a directory tree at the left, and directory contents at the right.

GridPanel arranges child elements in rows and columns. Avalon also provides a Table control that provides additional features similar to HTML tables.

The final layout option is called Canvas, to which fans of Descartes' coordinate system have been banished. You need to explicitly position elements on the Canvas panel using properties provided for that purpose.

You're not limited to using just one panel on a page. Everything in XAML is hierarchical. For example, you can divide a page into three areas using DockPanel, and then use FlowPanel, Canvas, and GridPanel for these three areas:

<DockPanel>
    <FlowPanel DockPanel.Dock="Left">
        •••
    </FlowPanel>
    <Canvas DockPanel.Dock="Top">
        •••
    </Canvas>
    <GridPanel DockPanel.Dock="Fill">
        •••
    </GridPanel>
</DockPanel>

The hierarchical nature of XAML is one reason why a markup language makes more sense than programming code for defining a visual interface. The markup can mimic the hierarchy with nesting and indentation. In fact, when Windows 1.0 was first introduced in 1985, programmers used a text resource script to define the hierarchical structure of their menus and dialog boxes. At the time, the hierarchy only went one level deep, but it was a start. A XAML file is just the resource script.

The content of Button elements can be text, an image, or a combination of text and images in a nested Text element. The content of a Button can also be a panel onto which other elements are positioned. Figure 4 shows a program named SmileyButton that does precisely that. The content of this button is a Canvas. The program then puts several elements from the MSAvalon.Windows.Shapes namespace on the Canvas. Figure 5 shows the program running.

Figure 4 SmileyButton (Close Figure 4)
<TextPanel xmlns="http://schemas.microsoft.com/2003/xaml"
        Background="Cyan" HorizontalAlignment="Center">
    <Button Width="150" Height="140">
        <Canvas Width="120" Height="120">
            <Ellipse CenterX="60" CenterY="60" RadiusX="50"
                     RadiusY="50" Fill="Gray" Stroke="Black"/>
            <Ellipse CenterX="40" CenterY="40" RadiusX="10"
                     RadiusY="10" Fill="Blue"/>
            <Ellipse CenterX="80" CenterY="40" RadiusX="10"
                     RadiusY="10" Fill="Blue"/>
            <Polyline Points="30, 70, 60, 80, 90, 70"
                      StrokeThickness="10" Stroke="Red"/>
        </Canvas>
    </Button>
</TextPanel>
Figure 5 SmileyButton in Action
Figure 5 SmileyButton in Action

The Shapes namespace consists of an abstract class named Shape and seven descendents: Line, Polyline, Rectangle, Ellipse, Polygon, Glyphs, and Path. Unfortunately there isn't an Arc or Bezier class in the Shapes namespace, so I drew the mouth as a two-segment polyline. I could have mimicked an arc using a Bezier curve in the Path element, but that would be showing off. You can do more extensive graphics in programming code attached to a XAML page.

Event Handling

Windows-based applications are more than just Web page wannabes. Programs need to respond to user interface events in very specialized ways. Here is where XAML must be supplemented by real programming code. You can put the code in a separate file or embed it directly in the XAML. In this article I'll be showing the latter approach for the sake of simplicity.

The .NET Framework and C# have a generalized mechanism called the event that classes use to signal each other. Events are members of a class just like methods, fields, and properties. One class declares an event. Another class can then choose to install an event handler for that event, which is simply a method with a specific signature and return type. The first class can then signal the second class by calling that method. Events are always associated with delegates, which are basically method prototypes. An event handler must be declared in accordance with this delegate.

The archetypal user interface event is the button click. The Button class has an event named Click that is associated with the ClickEventHandler delegate, which requires that an event handler for Click be declared something like this:

void ButtonClick(object el, ClickEventArgs args)
{
    MessageBox.Show("The button has been clicked");
}

I've given the first parameter the name of el to indicate an element. In your XAML you specify the name of this event handler as an attribute in the Button element:

<Button Click="ButtonClick" ...>

Embedding that programming code into the XAML file requires some special syntax. Programming code often includes symbols such as < and & that have special meanings in XML. The XML specification includes a facility called the CDATA section precisely for including arbitrary character data in an XML file. Any programming code in an XAML file must be enclosed in a CDATA section. The CDATA section always begins with the string "<![CDATA[" and always ends with the string "]]>". Within the CDATA section, the characters "]]>" must not appear under any circumstances. That may actually be a problem if you write C# code that looks like this:

if (arr1[arr2[I]]>5)
Simply insert a little white space somewhere within that inadvertent CDATA delimiter ("]]>")and all will be well.

In XAML the CDATA section must also be enclosed in a def:Code element, and additional elements must indicate what programming language is being used. (At this time you can use C#, Visual Basic .NET, and JScript® .NET.) Figure 6 shows a complete XAML file with a button with a Click event handled by embedded C# code. This file must be compiled because the Longhorn version of Internet Explorer does not interpret C# code. Click the button and a message box pops up.

Figure 6 XAML Plus C# Event Handler (Close Figure 6)
<TextPanel xmlns="http://schemas.microsoft.com/2003/xaml"
           xmlns:def="Definition" def:Language="C#"
        HorizontalAlignment="Center"
        Background="LightCyan">
    <Button Click="ButtonClick" FontSize="72">
        Push Me!
    </Button>
    <def:Code>
        <![CDATA[
            void ButtonClick(object el, ClickEventArgs args)
            {
                MessageBox.Show("The button has been clicked.");
            }
        ]]>
    </def:Code>
</TextPanel>

Elements and Objects

If your page has multiple buttons, you can have different event handlers for each button. Just give them different names. Or a single event handler can service more than one button. If so, the handler usually must determine precisely which button is being pressed. There are a couple solutions to this problem.

First, notice that the Click event handler has two parameters. The first parameter is the object generating the event. In this case it's the specific Button object that the user clicks. Within the event handler, you can cast that parameter to an object of type Button:

Button btn = el as Button;

You can then proceed to access all the members of the Button class. You can change the button color, change the text, perhaps disable the button, or even set a different event handler.

The FrameworkElement class (from which all controls and panels derive) includes a very special property named ID, which is of type string. You can set the property to a text string identifying the button. MSBuild requires that all the ID values on a page be unique. Here are two Button elements that have the same event handler but also include the following ID attributes:

<Button ID="OkButton" Click="ButtonClick">OK</Button>
<Button ID="CancelButton" Click="ButtonClick">Cancel</Button>

Within the event handler, you can use the ID property to distinguish between the buttons, perhaps in a switch statement:

switch(el.ID)
{
case "OkButton":
    •••
    break;
case "CancelButton":
    •••
    break;
}
That's one way to use the ID property. But I said the ID property was very special and it truly is.

Let's look at these two XAML elements again:

<Button ID="OkButton" Click="ButtonClick">OK</Button>
<Button ID="CancelButton" Click="ButtonClick">Cancel</Button>
When MSBuild converts this XAML to C# code in preparation for compilation, it also creates two fields of type Button named OkButton and CancelButton. These fields are set equal to the created Button objects. Within the event handler you can use these ID names just as if they were Button objects. For example:
if (el == OkButton)
{
    •••
}
else if (el == CancelButton)
{
    •••
}
I'm using an if statement here rather than a switch because the switch allows only integer and string expressions.

Not only can you differentiate between objects using these ID names, but you also have the ability to explicitly access all the other elements on your page—or at least the ones to which you've assigned ID values. For example, some other control may want to enable or disable the OK button based on a Boolean value that it determines. It's as simple as this:

OkButton.IsEnabled = bEnable;

Actually, an event handler can access any element on the page—even those that don't have ID attributes—because the elements are all linked together in an element tree. Earlier I discussed how elements support children. The ILogicalTreeNode interface (which is implemented by the FrameworkElement class and thus by the many classes that serve as elements) defines two properties named Parent and Children. With these two properties you have access to the entire element tree, and therefore you have the ability to dynamically add elements to it, remove elements from it, or change it.

Figure 7 shows a program named MoveButtons that has three buttons labeled "Tom," "Dick," and "Harry." Whichever button you click moves itself between the other two simply by shifting elements in the panel's Children property. Can you imagine what the equivalent code would look like in a plain Win32® or MFC program, or even in a Windows Forms application?

Figure 7 MoveButtons (Close Figure 7)
<FlowPanel xmlns="http://schemas.microsoft.com/2003/xaml"
           xmlns:def="Definition" def:Language="C#"
       HorizontalAlignment="Center" VerticalAlignment="Center">
  <Button Click="ButtonClick" FontSize="72">Tom</Button>
  <Button Click="ButtonClick" FontSize="72">Dick</Button>
  <Button Click="ButtonClick" FontSize="72">Harry</Button>
  <def:Code>
    <![CDATA[
      void ButtonClick(object el, ClickEventArgs cea)
      {
        Button btn = (Button) el;
        FlowPanel parent = (FlowPanel) btn.Parent;
        parent.Children.Remove(btn);
        parent.Children.Insert(1, btn);
      }
    ]]>
  </def:Code>
</FlowPanel>

Animation

With the notable exception of animated GIF files, graphical animation almost always involves code. So one of the most surprising aspects of XAML is a whole animation facility you use entirely with markup. For better or for worse, it's fairly easy to slip a few animation effects into your XAML pages.

To understand how to use this facility, you must first know that there is an alternative to setting element attributes in a start tag. The element

<Button Background="Red">
    Cancel
</Button>
can also be written like this:
<Button>
    Cancel
    <Button.Background>
        Red
    </Button.Background>
</Button>

The Background property is now a separate tag enclosed in the Button element, and the value of that property is now specified as the content of the Button.Background tag. That's the property I'm going to animate. Replace the content with a ColorAnimation element, like so:

<Button>
    Cancel
    <Button.Background>
        <ColorAnimation From="Red" To="Blue"
            Duration="10" Fill="Hold"/>
    </Button.Background>
</Button>
Now when the button is first displayed, the color changes from red to blue over the course of 10 seconds. Actually, it's not quite as simple as this: because the Background property is defined not as a Color but as a Brush, you must include a SolidColorBrush element as a child of Button.Background, and also include a SolidColorBrush.ColorAnimations element as a child of SolidColorBrush. The ColorAnimation element is then a child of SolidColorBrush.ColorAnimations, as shown here:
<Button>
    Cancel
    <Button.Background>
        <SolidColorBrush>
            <SolidColorBrush.ColorAnimations>
                <ColorAnimation From="Red" To="Blue"
                    Duration="10" Fill="Hold"/>
            </SolidColorBrush.ColorAnimations>
        </SolidColorBrush>
    </Button.Background>
</Button>
This approach is bulkier but allows more flexibility in animating gradients and other complex brushes. With a Brush background, you can even introduce rich backgrounds like images and video.

There are many animation options, of course. You can specify that the animation repeat (either in terms of a number of repetitions or a time) and indicate that the animation reverses itself after each repetition. You can specify an acceleration and a deceleration. For much more control over the animation, you can even provide a series of key values and key times, and an interpolation method.

You can also animate floating-point values, coordinate points, sizes, and Length values. The AnimationButton program in Figure 8 is similar to the SmileyButton except that it animates one axis of the left eye to make it look like it's winking.

Figure 8 Smiley Winking (Close Figure 8)
<?Mapping XmlNamespace="anim"
        ClrNamespace="MSAvalon.Windows.Media.Animation"
        Assembly="PresentationFramework" ?>
<TextPanel xmlns="http://schemas.microsoft.com/2003/xaml"
        Background="Cyan" HorizontalAlignment="Center">
    <Button Width="150" Height="140">
        <Canvas Width="120" Height="120">
            <Ellipse CenterX="60" CenterY="60" RadiusX="50"
                     RadiusY="50" Fill="Gray" Stroke="Black"/>
            <Ellipse CenterX="40" CenterY="40" RadiusX="10"
                     RadiusY="10" Fill="Blue"/>
            <Ellipse CenterX="80" CenterY="40" RadiusX="10"
                     RadiusY="10" Fill="Blue">
                <Ellipse.RadiusY>
                    <LengthAnimationCollection xmlns="anim">
                        <LengthAnimation From="10" To="1"
                            Duration="0.5" AutoReverse="True"
                            RepeatDuration="Indefinite"/>
                    </LengthAnimationCollection>
                </Ellipse.RadiusY>
            </Ellipse>
            <Polyline Points="30, 70, 60, 80, 90, 70"
                      StrokeThickness="10" Stroke="Red"/>
        </Canvas>
    </Button>
</TextPanel>

Styles

Sometimes you find yourself setting the same properties for a related group of controls. If you were working in HTML, you might use a Cascading Style Sheet (CSS) to help out. In Avalon, you do something similar using a Style element. If, for example, you wanted every button on your FlowPanel to have a cyan background and a 14-point font, you could include a Style tag like this:

    <FlowPanel.Resources>
        <Style>
            <Button Background="Cyan" FontSize="14pt"/>
        </Style>
    </FlowPanel.Resources>

You can specify multiple element types in the same Resources section using one Style element for each element type. You can also specify that you want styles to apply to all classes that derive from a particular element, or which have a certain ID value, or which don't have a certain ID value.

You can also give a name to a particular style and apply that style only to certain elements. For example, suppose a dialog box has a bunch of radio buttons. You want some of these radio buttons to have a different size, or color, or whatever. For those, you can assign a named Style:

<RadioButton Style="{Special}" •••>

In the following Style declarations, the first style applies to those radio buttons that specify Style equal to "{Special}" and the second to those radio buttons that don't set the Style property:

<FlowPanel.Resources>
    <Style def:Name="Special">
        <RadioButton •••/>
    </Style>
    <Style>
        <RadioButton •••/>
    </Style>
</FlowPanel.Resources>
There's even a way for the property sheet to specify properties that apply only to elements that already have other properties set on them.

An Avalon Color Scroll

Figure 9 shows a program named ColorScroll.xaml that incorporates many of the Avalon features discussed in this article. ColorScroll is an Avalon version of a simple Windows program that was originally published in the May 1987 issue of Microsoft Systems Journal, the predecessor to this magazine, and which has also appeared in every edition of my book Programming Windows. More recently, I've converted the program to Windows Forms for my books Programming Microsoft Windows with C# (Microsoft Press®, 2001) and Programming Microsoft Windows with Microsoft Visual Basic .NET (Microsoft Press, 2002).

Figure 9 ColorScroll.xaml (Close Figure 9)
<DockPanel xmlns="http://schemas.microsoft.com/2003/xaml"
           xmlns:def="Definition" def:Language="C#">
    <FlowPanel Width="50%" DockPanel.Dock="Left" >
        <FlowPanel.Resources>
            <Style>
                <Text Height="36pt" FontSize="12pt"
                      HorizontalAlignment="Center"
                      VerticalAlignment="Center"/>
            </Style>
            <Style>
                <VerticalSlider Width="50%" Margin="25%,0,25%,0"
                                Minimum="0" Maximum="255"
                                SmallChange="1" LargeChange="16"
                                DockPanel.Dock="Fill"/>
            </Style>
        </FlowPanel.Resources>

        <DockPanel Width="33.3%">
            <Text DockPanel.Dock="Top" Foreground="Red">Red</Text>
            <Text DockPanel.Dock="Bottom" Foreground="Red">0</Text>
            <VerticalSlider ID="RedScroll" ValueChanged="OnScrolled"/>
        </DockPanel>

        <DockPanel Width="33.3%">
            <Text DockPanel.Dock="Top" Foreground="Green">Green</Text>
            <Text DockPanel.Dock="Bottom" Foreground="Green">0</Text>
            <VerticalSlider ID="GreenScroll" ValueChanged="OnScrolled"/>
        </DockPanel>

        <DockPanel Width="33.3%">
            <Text DockPanel.Dock="Top" Foreground="Blue">Blue</Text>
            <Text DockPanel.Dock="Bottom" Foreground="Blue">0</Text>
            <VerticalSlider ID="BlueScroll" ValueChanged="OnScrolled"/>
        </DockPanel>

        <def:Code>
            <![CDATA[
                void OnScrolled(object el, ValueChangedEventArgs args)
                {
                    // Set the text element to the new value
                    Control slider = (Control) el;
                    DockPanel parent = (DockPanel)
                                ((ILogicalTreeNode) slider).Parent;
                    Text elemText = (Text) parent.Children[1];
                    elemText.TextRange.Text =
                                ((int) args.NewValue).ToString();

                    // Set the background color of the panel
                    ColorPanel.Background=
                        new SolidColorBrush(
                                Color.FromRGB((byte) (RedScroll.Value),
                                              (byte) GreenScroll.Value,
                                              (byte) BlueScroll.Value));
                }
            ]]>
        </def:Code>
    </FlowPanel>
    <TextPanel ID="ColorPanel" DockPanel.Dock="Right"
               Width="50%" Background="Black"/>
</DockPanel>

Figure 10 shows the program in action. You manipulate the trio of sliders at the left of the program to select a color based on red, green, and blue primaries. The values of the primaries are shown below the sliders and the resulting color is shown in the right half of the window. As the window is resized the sliders adapt themselves to the total width and height of the window.

Part of the challenge of writing such a program using either the Windows API or Windows Forms involves moving and resizing all the components of the window. This is an area where Avalon really shines.

Figure 10 ColorScroll in Action
Figure 10 ColorScroll in Action

The main panel is a DockPanel element. Two child elements each occupy half the width of the parent. On the left is a FlowPanel and on the right is a TextPanel, which is responsible for displaying the color. The TextPanel is given an ID value of ColorPanel.

The FlowPanel on the left has three DockPanel children, each of which occupies one-third of the width of the parent. Each of these DockPanel elements has three children: two Text elements at the top and bottom, and a Slider element in the middle.

The dimensions of the Text and Slider elements are provided in Style elements towards the beginning of the file. The size of the Text font is set at 12 points, and the height of the Text element is 36 points. The text is vertically centered in the Text element, so the additional space provides a visual comfort zone around the text. Because the two Text elements are docked at the top and bottom of the DockPanel, they occupy the full width of the parent, each of which is a sixth of the total width of the window.

The three Slider elements fill up the middle of each DockPanel. However, the property sheet gives each slider a width of 50 percent of its parent, and also provides a margin of 25 percent of the width of the parent on the left and right sides. These margins provide another area of visual breathing room at the sides of the slider.

Except for the font size and the height of the Text boxes, all dimensions are given in percentages of the width of the window. It took a little thought, of course, but it was much easier than determining the dimensions in pixels.

Each Slider element contains several attributes that must be initialized identically: Each slider has a range from 0 through 255. The arrow keys scroll by 1; paging up or down or clicking the central area of the slider scrolls the slider by 16. These properties are all included in the styles specification. (Perhaps an even better approach would have been to derive a new class from DockPanel that would have included the two Text elements and the Slider element, and then to create three objects of that class for the red, green, and blue scroll bars.)

The three scroll bars have ID values of RedScroll, GreenScroll, and BlueScroll; the TextPanel at the right side of the window has an ID value of ColorPanel. All these ID values are referred to in the OnScrolled event handler for the slider ValueChanged event.

The event handler first does something clever: it uses the Parent property and the panel's Children property to obtain the previous sibling of the particular slider raising the event. That's the Text element underneath the slider that displays the numeric color value. The method then sets the text of that element to the new value of the slider. The method obtains this value from the Value property of the ValueChangedEventArgs object passed to the event handler. It could also have cast the object parameter to a Slider and accessed the Value property of that.

The method then calculates a new color based on the Value properties of all three sliders, and uses that color to set a new Background property of the TextPanel at the right, which has the ID of ColorPanel.

The most recent Win32 API version of this program (called COLORS1 in the fifth edition of Programming Windows) is 250 lines long. The Windows Forms version in Programming Microsoft Windows with C# is about 100 lines. This new version is only about 60 lines long, and I'm pretty sure that it can be pruned even more by using inheritance.

From 250 lines to 100 and now 60. And that, my friends, is what is commonly called progress.

Conclusion

Avalon and XAML represent a departure from Windows-based application programming of the past. In many ways, designing your application's UI will be easier than it used to be and deploying it will be a snap. With a lightweight XAML markup for UI definition, Longhorn-based applications are the obvious next step in the convergence of the Web and desktop programming models, combining the best of both approaches.

Charles Petzold, a long-time contributing editor to MSDN Magazine (and its predecessor, MSJ), is the author of an upcoming book about Avalon from Microsoft Press tentatively titled Programming the Windows Client Platform. His Web site is http://www.charlespetzold.com.


Code Name WinFS: Revolutionary File Storage System Lets Users Search and Manage Files Based on Content

[ Editor's Update - 8/27/2004: This content is out-of-date. Microsoft will deliver a Windows storage subsystem, code-named "WinFS," after the "Longhorn" release. The new storage system provides advanced data organization and management capabilities and will be in beta testing when the "Longhorn" client becomes available. For more information, please refer to Microsoft Announces 2006 Target Date for Broad Availability Of Windows "Longhorn" Client Operating System.]
Code Name WinFS

Revolutionary File Storage System Lets Users Search and Manage Files Based on Content

Richard Grimes
This article was based on a pre-PDC build of Microsoft Windows Code Name "Longhorn" and all information contained herein is subject to change.
Note: This document was developed prior to the product's release to manufacturing and, as such, we cannot guarantee that any details included herein will be exactly the same as those found in the shipping product. The information represents the product at the time this document was printed and should be used for planning purposes only. Information subject to change at any time without prior notice.
SUMMARY

One of the monumental problems organizations face today is aggregating information that's stored in disparate formats. Knowledge workers have long wanted to be able to search for content independent of format. The next version of the Windows operating system, code-named "Longhorn," boasts a new storage subsystem that makes that task easier. That subsystem, code-named "WinFS," allows the user to perform searches based on the metadata of the stored item, regardless of what type of file it is or which application created it. This article covers the basic architecture of WinFS and explains how to use the WinFS managed API.

Contents
Basic Architecture
WinFS Storage
WinFS Types and .NET Types
The WinFS Managed API
Containing Items in Folders
Searching for Items
Notifications
Synchronization
WinFS and Win32
Conclusion

E
very year, as new hard disks get bigger and faster, applications catch up by producing more data. Hard disks are commonly used to store personal information: correspondence, personal contacts, and work documents. These items are currently treated as separate entities, yet they are interrelated on some level; and it's no surprise that e-mail comes from your personal contacts list and influences the work that you should be doing and hence determines the documents that you'll create. When you have a large number of items, it is important to have a flexible and efficient mechanism to search for particular items based on their properties and content. Up until now, storage mechanisms like Outlook®, the Windows® Contact List, or Windows Media® Player media library used unrelated technologies for the storage of items and the mechanism to search for them. Well, those days are over, thanks to Microsoft® Windows code-named "Longhorn."

The Longhorn generic storage subsystem, code-named "WinFS," can store data of all types and use one mechanism to access it. This is a huge change in an OS that has relied on multiple, incompatible storage for data including the registry, event log messages, contact information, and e-mail, or simply used multiple flat files for data such as images and audio. Since WinFS is implemented using database technologies, you get an efficient and flexible mechanism to query for data items, the ability to replicate data to stores on other machines, and facilities to handle backups and restores.

In this article I will outline the basic architecture of WinFS and describe how data is stored, how it is retrieved, and the services it provides for the operating system. Like much of Longhorn, WinFS has its own managed API, which will be my focus.

Basic Architecture

The perennial problem with data is that you always have too much of it and it is always too difficult to find what you want. This problem will get far worse in the near future when typical machines will have hard disks measured in terabytes. The bigger the hard disk, the more data you will want to store, and the more difficult it will be to find a particular item. Finding a piece of data depends on identifying the specific item, which means that the item has to be categorized.

For example, imagine that you have several thousand digital photographs. Typically, your camera and desktop software will catalog this data by naming the photos in sequence and maybe place these in a folder according to the date that the photos were taken. Searching for a particular photo in this scheme involves remembering the date and photo number. If you do not know either of these things, you will have to resort to looking through thumbnails of the photos. Of course, if you have the patience, you could name each photograph as you download it to your desktop. A coarse-grained system, like the file and folder system in Win32®, is not flexible enough to allow you to store all the data you need.

Desktop software may fill this gap, allowing you to build a catalog that stores other descriptive properties like camera settings, light condition, and a general description of each picture. Such software, of course, is an implementation of a database, but the important point is that it acts as a supplement to the file system. You could argue that these descriptions of your photos—metadata—are actually part of the photograph's object. Digital photographs hold other information within the image file's EXIF (Exchangeable Image File Format) header. This header contains information obtained when the photo was taken. To include this metadata in a search would require a query engine to extract this information from each candidate photo and compare it with the query condition. A sophisticated photograph organizer could extract the EXIF information when the photograph is transferred to the desktop and include it as additional metadata in the database.

This is where WinFS fits in. Everything that is stored in the WinFS store is an item and each item has metadata properties that are described by a schema. Items that follow the schema are stored in the WinFS store as serialized .NET objects and are accessed through T-SQL views that give access to the items' properties. Finding a photo stored on your disk involves querying the store against the metadata properties for an Image item, the information obtained from the EXIF header, as well as file information like the creation date and the actual bits that make up the image.

Figure 1 Files in WinFS
Figure 1 Files in WinFS

Figure 1 shows the conceptual architecture of items in WinFS. Each item exists in a containment folder; these are used to control the lifetime of items. Folders are items themselves and each folder must be contained within another folder. The only exception to this containment rule is the root containment folder. Items are stored in a volume, which is the largest autonomous container of items. On each machine, the WinFS service acts upon these volumes to store, delete, modify, and locate items.

WinFS Storage

WinFS provides T-SQL views of the Item tables, with one view per item type. To enforce the security of the store and the consistency of the data, the data can only be read through the views and can only be modified using stored procedures. For example, the following line will return the requested information for all tracks in the Media Library on your machine:

SELECT Artist, Title, Album FROM Audio.[Track!V1]
Some objects contain references to other objects. For example, a Track object has a member called Authors, which is a collection of Author objects, each of which has a DisplayName property, which gives the name of an artist who authored the track. It is possible to construct a query for all of the tracks that are attributed to a particular artist, which you can cross-reference to your personal address book to see if you have the cell phone number of any artist for which you have a track. (This assumes, of course, that you have a celebrity-filled address book!)

Not everyone is a database developer and able to construct complicated select queries, so WinFS provides several higher-level APIs to make inserting, deleting, and querying for items simpler.

WinFS Types and .NET Types

When you access WinFS items through the managed API, you do so through .NET objects. Each WinFS item type has a corresponding .NET Framework class which has System.Storage.Item in its hierarchy. The Item type has an ItemID_Key property and a GUID, which provides the identity for the WinFS item. When you obtain a WinFS object, the API returns the corresponding .NET type initialized with the metadata of the WinFS object. If you obtain two Item instances that have the same ItemID_Key values, then they refer to the same item in the store, even though you will have two different .NET references. When comparing two WinFS objects, it is important that you compare each of the values of the ItemID_Key of the items that are in question rather than using ReferenceEquals.

In WinFS parlance, a WinFS type is an item that is made up of elements that are either items, nested elements, or scalar types—or a collection of any of these. An item is the unit of consistency and coherence, so when an item is copied or deleted, all the elements within the item are also copied or deleted. A nested element can only be part of a single item. This behavior is different from the way it works in .NET. For example, a System.Storage.Contacts.Person object has a property called PrimaryName that is a nested element of type FullName. If you create two Person objects with the same PrimaryName (for example, a father and son who have the same name) then you should create two FullName objects, one for each Person object. If you use the same FullName object for both Person objects you will get an exception when you try to save the Person objects to the store because although they are valid .NET objects, they are not valid WinFS objects.

The WinFS Managed API

Figure 2 shows the general architecture of WinFS. The WinFS store can be accessed through several APIs, and in this release an app can use ADO.NET with the SQL Server™ Provider, OLE DB with native code, a COM-based API, and a managed API. In general, developers will use the APIs rather than T-SQL, but will still have access through T-SQL because there are a number of things that you can do in T-SQL that you cannot do with the other APIs.

Developers who are concentrating on .NET will use the WinFS API found in the System.Storage assembly, which contains the general WinFS API. Developers will also use the System.Storage.Schema assembly, which holds the API for the standard Longhorn types (Track, Image, Folder, Message, and so on). In addition to the common language runtime (CLR) types defined for the WinFS Items, the System.Storage.Schema assembly contains the data model mapping for the items. This mapping indicates to WinFS the data source used to access the item and the mapping from the properties of the managed type to the fields in the data store table. Deep in the bowels of the WinFS API, this mapping information is used to construct the T-SQL queries to access the item in the store.

Figure 2 WinFS Architecture
Figure 2 WinFS Architecture

All WinFS API code has to run under an ItemContext. An ItemContext means many things. First, the WinFS API must create a connection to the store; this connection remains active while the ItemContext is active. Second, since the ItemContext represents a data connection, it provides transactional facilities. If you like, you can make several changes to the store as an atomic unit. Finally, an ItemContext describes the scope of the work that you will do by defining which items your searches will act upon.

The first action that you need to take is to open an ItemContext with the static Open method. The parameterless version of Open will open the context for all items, but you can limit the scope of the context with the other overloads. WinFS uses a naming scheme that allows you to identify both the volume and a backslash-delimited path to a folder.

The ItemContext provides access to the data connection through BeginTransaction and EndTransaction. BeginTransaction creates a new transaction and returns a Transaction object. Once you have done this, all the changes you make to the store will be performed under the influence of the transaction until you call EndTransaction and treat all of your changes to the store as an atomic action.

It is also worth pointing out that when you have finished accessing the store you should call Close on the ItemContext to release the data connection. If you have made changes to the store, you should also call Update to ensure that the changes have been persisted to the store. For this prerelease version of WinFS, once you have created a Folder object, you should save it before attempting to insert any items into the folder. To do this you can call Folder.Save, which is merely a wrapper around ItemContext.Save.

Containing Items in Folders

The broadest level of object categorization can be achieved through folders. WinFS folders are used to contain items; however, unlike the concept of a file system folder in Win32, a single item can exist in more than one WinFS folder. If you want to create your own folder hierarchy, start by adding your folder to an existing one. To do this, you create a new Folder object by passing the container folder object and the new folder name as constructor parameters. The containing folder can be any existing folder, and you can traverse the entire hierarchy by obtaining the root folder (just call Folder.GetRootFolder) and then iterating all the folder objects it contains. Typically, you will not store items in the top-level folder, but instead use a folder further down the hierarchy.

The root folder of each volume has four folders by default: System Data, Volume, Schema, and Store Information. The folders you will be creating should exist under SystemData. To do this, you could search for and then open the SystemData folder and add your folder under it. It is easier, though, to use the WinFS API to provide access to the folder that is allocated specifically for your items. The static method UserDataFolder.FindMyUserDataFolder will return a folder under SystemData specifically for the current user and some item types will have their own standard folder. For example, the UserDataFolder.FindMyPersonalContactsFolder method will return a folder called Personal Contacts.

Once you have access to the appropriate folder, you can populate it with items (including other folders) using the Folder.AddMember method. Removing items from a folder is just as simple—you call the Folder.RemoveMember method and pass either the item or the item's name. When you place an item in a folder, you are using what is known as a holding link. The folder object is just another item that has a holding link for each item it contains. A holding link is special because it determines the lifetime of the item it refers to, and each holding link on an item acts as a reference count. A holding link can only be applied to an item, and this highlights another difference between items and nested elements. When all the holding links on an item are deleted, so is the item. WinFS folders are not constrained to the type of items they contain. This means that if you have a folder called "Summer Vacation 2003," it can hold all the photos you took on your vacation, the e-mail messages confirming your booking, and even a WAV file of "Holiday" by Madonna.

Accessing items in folders is straightforward: the folder item has a property called Members, which is actually a collection of Link items. Each Link is the holding link and its SourceItem property provides access to the appropriate item. In addition, the Folder class has members that let you retrieve members that have a specific name or are of a particular type.

Searching for Items

Once you have a context and have determined the folder where the items are located, you next need to access the items. The simplest way to do this is to iterate through all the items in the folder and check each one individually, as shown in Figure 3. The FindResult object contains the results of the query performed by GetAllMembers, which should include all of the items in the folder. WinFS then creates a CLR object of type Item for each of these items and puts them in the FindResult object. Of course, GetAllMembers is a rather blunt tool, but you can refine this process by using GetAllMembersOfType to return only the Person items in the folder:

using (FindResult result
   = myDataFolder.GetAllMembersOfType(typeof(Person)))
{
foreach(Person person in result)
{
   if (person.DisplayName == "Richard Grimes")
   {
      // use person here
      break;
   }
}
}

Figure 3 Searching for the Richard Grimes Person (Close Figure 3)
using(ItemContext ctx = ItemContext.Open())
{
   Folder myDataFolder =
       UserDataFolder.FindMyPersonalContactsFolder(ctx);
   using (FindResult result = myDataFolder.GetAllMembers())
   {
      foreach (object o in result)
      {
         Person person = o as Person;
         if (person == null) continue;
         if (person.DisplayName == "Richard Grimes")
        {
            // use person here
            break;
         }
      }
   }
}

The search can be refined further through a filter passed to one of the Find methods:

Person person = (Person)myDataFolder.GetOneMemberOfType(
   typeof(Person), "DisplayName='Richard Grimes'");

This method does all of the work in the previous code. It searches a specific folder for objects of a specific type that also have the property values specified in the filter string. The filter string is written in a new query language called WinFS OPath. You can provide a filter to check the properties of the type passed to the method, so in this case I am performing a check on the Person.DisplayName property. During the search, WinFS will extract the DisplayName property of each Person object it finds and compare the value with the literal string I provide.

WinFS OPath is very flexible, allowing you to provide multiple criteria appended with the && or || operators. In addition, you can even use the like operator:

FindResult result = myDataFolder.GetAllMembersOfType(
   typeof(Person), "DisplayName like 'R% Grimes'");
This will return all the Person Items that have a DisplayName made up of a first name that starts with "R" and a surname "Grimes".

Some properties are collections, so WinFS OPath provides a syntax that allows you to check the properties on the items in the collection. For example, the Person class has a property called PersonalNames, which is a collection of FullName objects (allowing for the fact that some people may use more than one name). If I want to search for all people that have the last name Grimes as one or more of their personal names, I can use:

FindResult result = Person.FindAll(
   ctx, "PersonalNames[Surname='Grimes']");

Here, the square brackets indicate that PersonalNames is a collection and that for each item in the collection the Surname property should be compared with the literal string. In this example I have used the static method, Person.FindAll, which will find all the Person items in the given context. All item classes will have an overloaded FindAll inherited from the Item class. In addition, all item classes will have a FindOne method that will return a single typed item from the context that satisfies the filter.

Item classes can perform searches for items based on the item's ID, or can search for items that have a specific category. Categories are essentially known names attached to an item. The Item class has a property called Categories, which is a collection of all the categories attached to the item, and each category is a CategoryRef object. Categories have a Key property, which is the name of the category, and also a Type property, which is a GUID identifying the category. You can create your own categories, or you can use the predefined categories provided as static members of the GeneralCategories type. To add a category to an item, you have to create a new CategoryRef object and add this to the Categories property. You cannot simply add one of the members of the GeneralCategories to an item. The code in Figure 4 will fail at run time because in the store, members of Categories are nested elements, which are owned by a single item and cannot be shared between items. A nested element has an ItemKey property which has a unique value for each nested element. However, in Figure 4, the WinFS code will attempt to add a nested element with the same ItemKey to two items. The solution in Figure 5 is straightforward: a new CategoryRef object is added to the Categories property and this is initialized with a value from the GeneralCategories class. Since a new CategoryRef object is created, it will have a unique ItemKey.

Figure 5 Correct Way to Add a Category (Close Figure 5)
Person person = Person.CreatePersonalContact(ctx);
person.DisplayName = "Richard Edwards";
SmtpEmailAddress email;
email = new SmtpEmailAddress("dai@parkfarm.com");
email.Categories.Add(new CategoryRef(GeneralCategories.Home));
person.PersonalEmailAddresses.Add(email);
person.Save();
person = Person.CreatePersonalContact(ctx);
person.DisplayName = "Roberto Marchini";
email = new SmtpEmailAddress("rob@robshouse.com");
email.Categories.Add(new CategoryRef(GeneralCategories.Home));
person.PersonalEmailAddresses.Add(email);
person.Save();
Figure 4 Incorrect Way to Add a Category (Close Figure 4)
Person person = Person.CreatePersonalContact(ctx);
person.DisplayName = "Richard Edwards";
SmtpEmailAddress email;
email = new SmtpEmailAddress("dai@parkfarm.com");
email.Categories.Add(GeneralCategories.Home);
person.PersonalEmailAddresses.Add(email);
person.Save();
person = Person.CreatePersonalContact(ctx);
person.DisplayName = "Roberto Marchini";
email = new SmtpEmailAddress("rob@robshouse.com");
email.Categories.Add(GeneralCategories.Home);
person.PersonalEmailAddresses.Add(email);
person.Save();

If you know that your items have categories, you can use FindByAllCategories, FindByAnyCategory, or FindByCategory to find the items that have all of the specified categories, any one of the specified categories, or one specific category.

Notifications

When you use WinFS, it is often too easy to forget that the items you are accessing are actually items in a data store. WinFS notifications make this apparent. As the name suggests, WinFS notifications are a mechanism to allow your code to be informed when an item changes. WinFS uses the .NET events model to subscribe to and handle notifications. Subscriptions will survive until the application instance quits or until the application unsubscribes. The ItemWatcher object can be created to monitor a particular item, and this object has an event called ItemChanged:

public event ItemChangedEventHandler ItemChanged;

This delegate has an ItemChangedEventArgs parameter. The ItemChangedEventArgs.Details is a collection of ItemChangedDetail objects that indicates the type of change that has occurred on the item. So, one application can access an item from the store and add an ItemChanged handler for that item. Then, when another application makes a change to the item, the event in the first application is triggered.

The listener application only needs to add a delegate to the ItemWatcher.ItemChanged event, and the application will be notified whenever that item has changed. That change may be from code in the same process, in another process, or on another machine. If the change occurs from another machine, then it manifests itself on your machine due to synchronization.

Synchronization

As I mentioned, WinFS is a data storage service, so it seems natural for it to support replication to other stores on other machines: to WinFS this is called synchronization. To implement synchronization, WinFS first has to determine what data has changed and then determine what data needs to be replicated. In WinFS, elements are the smallest parts of an item's schema that are tracked. If one element in an item changes, WinFS knows that the item has changed. Items are the smallest unit of consistency in WinFS and so they are the smallest unit of data that can be replicated to another machine—an item is replicated in its entirety or not at all.

The user has to identify the machines that will take part in the synchronization. To do this, the user creates a community folder that is used by all the machines and associates it with the actual folder of items on the local machine. Applications can initiate synchronization through the WinFS API or it may occur as a scheduled task. In both cases, the application that initiated the action provides a synchronization profile to WinFS. This profile has information about the folder that is being synchronized, the direction (send-only, receive-only, or send and receive) and information about what items should be synchronized. In addition, the profile contains information about how to handle conflicts. Conflict policy can invoke an action as simple as rejecting the replication, which can be handled automatically by a piece of code called a conflict resolver, or it can solicit user intervention.

WinFS and Win32

WinFS is an exciting and feature-rich facility, and developers writing new applications for Longhorn should take full advantage of it. Of course, most applications that are moved over to Longhorn will have some legacy Win32 code and so this raises the question of how the two will cooperate.

The first point to make is that WinFS will not supplant NTFS—machines running Longhorn will still have NTFS-based drives. WinFS currently only applies to folders within the Documents and Settings folders. All other folders still reside under the control of NTFS. In particular, %systemroot% and Program Files are NTFS folders, which means that your application executable files will not actually live in the WinFS store.

Of course, a Win32-based application can still read and write files in the folders under Documents and Settings. The Win32 file APIs have been implemented in Longhorn to take advantage of the items in the WinFS store. All of the items in Documents and Settings that were migrated from Win32 are file-backed, which means that there will be a file representation of the item as well as a WinFS item. Known file types will each have an associated metadata handler that has the responsibility to read the file and extract the metadata to store as properties in WinFS. For example, a metadata handler associated with Microsoft Word will load a Word file and then read the SummaryInformation OLE Property set and use the information in order to initialize a Document WinFS type in the WinFS store.

This duality of file-backed items and metadata properties in the store means that Win32 can continue to access items in Documents and Settings and WinFS can use properties as part of a query. These properties are treated as read-only to WinFS. When a Win32-based application changes a file, the metadata handler updates the store with the new property values. What this means in practice is that when you copy a Word document to your My Documents folder on a machine running WinFS, information about the file will be put into the WinFS store so that applications can perform searches based on the file's properties. However, the file will still be accessible by Win32, so Word will be able to load and edit the file. When Win32 makes changes to the file, WinFS will read the updated properties and write them into the store. So when you add a page to your Word document and save the file, WinFS will update the Document.DocPageCount property so that searches based on this property will work as expected.

Conclusion

WinFS, the new Longhorn file storage system, is the basic building block of all applications on Longhorn, and I have only just scratched the surface of what you can do with it in this article. WinFS provides you with all of the features you need to have persistent items that can be categorized and searched for based on properties of the item type. Your job is to use these facilities to create the next generation of Windows-based applications.

Richard Grimes speaks and writes about the .NET Framework. He is the author of Programming with Managed Extensions for Microsoft Visual C++ .NET (Microsoft Press, 2002) and Developing Applications with Visual Studio .NET (Addison-Wesley, 2002). His book on developing applications for Longhorn will be available nearer the release of Longhorn. He can be contacted at longhorn.dev@grimes.demon.co.uk.


Windows Shell: Create Namespace Extensions for Windows Explorer with the .NET Framework

New information has been added to this article since publication.
Refer to the Editor's Update below.

Windows Shell

Create Namespace Extensions for Windows Explorer with the .NET Framework

Dave Rensin
This article assumes you're familiar with C#
Code download available at: WindowsShell.exe (224KB)
SUMMARY

Extending the Windows shell with namespace extensions allows you to create some custom functionality for Windows Explorer. One common use is to enable Explorer to present a list of items that do not exist in one real folder, but actually reside in a number of places. The view on the folder makes it look like these items are in one place, so managing them becomes easier. This article illustrates the process of creating custom shell namespace extensions using C# and the .NET Framework. The author dispels some myths about the difficulty of writing such extensions, and shows that it is easier than it was before .NET. Along the way he outlines undocumented interfaces and describes advanced techniques for consuming them in .NET.

Contents
Introducing Shell Namespace Extensions
The Basics
First Secret—Use the Source, Luke...
Second Secret—Beware the Inheritance Tax
Third Secret—Triple-check Your Parameters
More Basics
ShellFolder
Fourth Secret—Lies, Damn Lies, and PIDLs
The ShellFolder Class
Properties
Methods
The ShellView Class
Variables
Methods
Common Dialogs with Extended Messaging
Common Dialogs with No Extended Messaging
Microsoft Office Dialogs
A Parting Thought
Your Very Own Shell Extension
Conclusion

[ Editor's Update - 6/23/2006: Because shell extensions are loaded into arbitrary processes and because managed code built against one version of the runtime may not run in a process running an earlier version of the runtime, Microsoft recommends against writing managed shell extensions and does not consider them a supported scenario.]

S
uppose you're a programmer working on a new software product that's going to change the world, a worthy goal for all new software products. For the sake of efficiency and performance, you decide to store the objects your product manipulates in a database on a far-away server. You're using C#, IIS, and Web Services and things are going great. Then one day the head of product marketing comes by your office to have a chat.

It seems, he says, that potential customers love the product concept, but hate the idea of using a special application to file and retrieve the objects. Instead, they want to be able to browse the objects from any Windows®-based application using the standard File Open and File Save dialogs.

Introducing Shell Namespace Extensions

For the uninitiated, shell namespace extensions are special COM objects that allow you to add onto, and generally manipulate, the Windows shell. Their uses are many and varied, but if you believe what you read on various message boards and FAQs, there's no such thing as an easy or quick-and-dirty namespace extension.

In this article, I'll show you how this just isn't so. Namespace extensions are eminently buildable, as long as you know the secrets. I will acquaint you with those secrets so that you can join the action-packed world of custom extension writing. If, at the end of this article, you still don't feel comfortable building your own extensions, take heart. I will point you to a base class I have written that will make your life much easier. All you will need to do is inherit from it and add less than 10 lines of code, and you'll have a fully working namespace extension.

The Basics

Not well documented, and often misunderstood, the core interfaces for a shell namespace extension (shown in Figure 1) aren't as hard to use as you may think. In order to create your own extension from scratch you will need to build two classes. The first class needs to implement the interfaces IPersistFolder and IShellFolder. The second class will have to implement IShellView. Along the way, you will also need to stub out interfaces for ICommDlgBrowser, IOleWindow, and IShellBrowser.

Figure 1 Windows Shell
Figure 1 Windows Shell

A word to the wise: if you've never written a class that implemented an interface not native to the Microsoft® .NET Framework, then stop reading this and review interop now. The online help is a good reference, but the information is a bit scattered. I usually recommend the book C# and the .NET Platform by Andrew Troelsen (APress, 2003). In any case, you should pay particular attention to the .NET ComImport attribute.

For those of you with at least a passing familiarity with the subject, I'll move on. When I began to compile the list of interfaces, structures, and enumerations that I would have to define in .NET, I started to feel a bit queasy. Like any good programmer, I don't want to do any more work than I have to do to get the job done.

The big problem with stubbing out these interfaces is that they are all inherited from IUnknown instead of IDispatch. The main implication is that the functions have to be defined in a particular order known as vtable order. That means you need to make sure that you define the methods of the interface in exactly the same order they are defined in their respective .h files. Since the online help doesn't always tell you what the vtable order is for a given interface, you will be stuck flipping back and forth between several different C++ .h files—but perhaps there's another way.

First Secret—Use the Source, Luke...

Any old C++ programmer will tell you that the first thing a compiler does when building a project is preprocess all of the source files. This means that it takes your various #include statements and resolves them all into one large file. It also substitutes all special literals, like macros and string constants, into their actual values as set forth in various #defines. The bottom line is that the preprocessor will put all the little bits of information you will need into one handy spot. Getting it to work was remarkably easy.

There are three main .h files concerned with namespace extensions. (They, of course, make references to many other .h files, but that's the preprocessor's problem.) They are shlobj.h, shobjidl.h, and shlguid.h. To get the complete set of secrets contained within them, and all the other files they reference, I created a simple file named test.cpp that had the following three lines of code:

#include <shlobj.h>
#include <shobjidl.h>
#include <shlguid.h>
Next, I opened a command prompt and typed the following line:
cl /E test.cpp > test.txt
CL.exe is the .NET compiler. The /E option tells it to just preprocess the file without trying to compile it. The output is then redirected to the file test.txt.

Note that CL.exe is not normally in the path, so you will need to open the command prompt shortcut installed in the Microsoft Visual Studio® .NET Start Menu program group in order to run it. After this operation, the resulting test.txt file contains a greatly resolved set of interface definitions. For example, the definition for the interface IOleWindow now looks like this:

struct __declspec(uuid("00000114-0000-0000-C000-000000000046"))
IOleWindow : public IUnknown
{
public:
    virtual  HRESULT __stdcall GetWindow(
         HWND *phwnd) = 0;

    virtual HRESULT __stdcall ContextSensitiveHelp(
         BOOL fEnterMode) = 0;

};
This is just a short step from its required C# form of:
[ComImport()]
[Guid("00000114-0000-0000-C000-000000000046")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface IOleWindow
{
    void GetWindow( out IntPtr phwnd);
    void ContextSensitiveHelp( bool fEnterMode);
}
This trick saves a lot of unnecessary typing and guarantees that your definitions are in the correct vtable order, but you're not out of the woods yet.

Second Secret—Beware the Inheritance Tax

So now you have all your interfaces in one convenient place and you're ready to define them in C#. Along the way, you notice that some of the interfaces are inherited from each other.

For example, IPersistFolder is based on IPersist which, in turn, is inherited from IUnknown. Given this, you might think that you could save some typing by doing the following in your C# code:

public interface IPersist {
    // stuff specific only to IPersist
}

public interface IPersistFolder : IPersist {
    // stuff specific only to IPersistFolder
}
This makes sense, right? After all, IPersistFolder should inherit all of IPersist's methods and the resulting vtable should come out right. Unfortunately, I discovered that the wrong functions were getting called. At first I thought I had mixed up the order, but a quick look at the original C++ definitions showed that this was not the case. I then defined IPersistFolder thusly:
public interface IPersistFolder {
    // stuff specific to IPersist only
    // stuff specific to IPersistFolder only
}

Lo and behold, it worked! Notice that I've abandoned any idea that IPersistFolder is inherited from anything at all and just included the stubs from IPersist right in its definition. In all candor, I can't tell you why this is but it definitely works just fine and shouldn't give you any problems.

So now you have all your interfaces in place and you know how to handle their inheritances in your definitions. It's time to start coding, right? Not quite yet.

Third Secret—Triple-check Your Parameters

Technically this isn't so much a secret as it is an admonishment, but so many people are stung by it that it might as well have been classified information. Let's say that the C++ definition for some method called Foo looks like this:

void Foo (long i, int j)
Let's also suppose you stub it out in C# with the same header. Chances are the code in that function will never get called. The reason is that unmanaged code treats a long as 32 bits but managed code (generated by C#) treats it as 64 bits. This means the call stack for your function will be off by 4 bytes and the Windows shell will most probably get an exception when trying to call it. Instead, your method should be defined as follows:
void Foo(Int32 i, Int32 j)

I know this seems like a small detail, but if you're not careful this will come back to bite you. For example, you may someday want to implement context menus, property pages, or some other thing for which I have not already provided an interface. Should that situation ever come to pass, you will certainly be better off having read the last few pages.

Now that all the housekeeping is out of the way, it's time to dig seriously into the heart of the extension.

More Basics

The included source code is actually split between two projects. The first is the base class assembly I made reference to in the beginning. Its name is MSDNMagNSEBase (short for MSDN® Magazine Namespace Extension Base Class). The files in this project are mmc.cs, Shell.cs, ShellFolder.cs, and ShellView.cs.

The second project is a sample extension I created that shows how to use the base class. It's called MSDNMagazineSampleNamespaceExtension and its files are Class1.cs and Form1.cs. For the time being, I'm going to concentrate on the first project as that represents the heart of the code.

The average extension has two parts—a folder and a view. Typically, these parts are represented by two different classes, although this is not a strict requirement. The folder class must implement the interfaces IPersistFolder and IShellFolder and the view class must implement IShellView. In my case, these classes are located in the files ShellFolder.cs and ShellView.cs, respectively.

The folder class is responsible for managing the data associated with your extension, while the view class is responsible for managing the user interface of the extension. For those readers who are or were MFC programmers, this is conceptually similar to the document/view architecture.

ShellFolder

The most common use for namespace extensions is to present a virtual directory of items to the Windows shell. These items may be actual files on disk or not; it really doesn't matter. For example, when you open My Network Places you are looking at a shell extension that presents a virtual folder of visible remote computers. Of course, these items don't actually represent files, but the Windows shell neither knows nor cares about that. All it cares about is that it has been presented with a list of unique items. The details of how those items are managed are completely up to the extension.

The items managed by the folder are represented by a PIDL—a pointer to a list of unique identifiers. Each virtual item managed by the folder has its own PIDL.

The first thing you should know about PIDLs is that they are defined only as a high-level concept. This is intentional. What you store in the PIDL is entirely up to you and Windows sets almost no restrictions. The downside to the opaque nature of these odd little creatures is that documentation surrounding them is rather vague. In fact, it has been my experience that PIDL management is the most confusing and daunting part of writing a shell namespace extension. A quick scan of message boards dealing with shell extensions makes this fact manifestly evident.

Like everyone else who has ever written a namespace extension, I spent a lot of time writing my own PIDL manager class to handle these nettlesome beasts. It wasn't until I finally completed my first extension that I noticed something truly surprising about them.

Fourth Secret—Lies, Damn Lies, and PIDLs

PIDL management is a largely useless activity that can be avoided when writing your own shell extension. Forget about writing a PIDL manager, implementing IEnumIDList, or any of the many other things usually required for managing objects. It's all completely unnecessary.

After my third or fourth extension, I settled upon a technique that makes life a lot easier. To understand how it works, though, you first need to understand a little bit about how the shell, the folder, and the view all communicate with each other.

The Windows shell (explorer.exe) is natively equipped to manage and display physical items like files and directories. In order to display virtual items (such as remote computers or printers), it relies on shell namespace extensions.

Consider the case of the My Network Places. When you open a Windows Explorer window and double-click My Network Places (or any shell namespace extension), a certain sequence of events occurs. First, the extension's object that implements IShellFolder is loaded into memory. Then Windows asks the object for its unique ID—its PIDL. Armed with the PIDL, Windows asks the extension for its display name. Next, Windows asks the extension for a list of any items that it is currently managing—things like remote computers or virtual files. The list is returned using the interface IEnumIDList. Windows iterates through each item in the list and asks the extension for the display name of the item. Windows next asks the extension for a reference to the object that implements IShellView—in this case, my view class. Finally, Windows asks the view object to load its UI by calling CreateViewWindow.

This is the shorthand version of what happens. A few other calls are also made to the extension to ask it about other kinds of things it can do, but that's not particularly important for the moment. From my own experience, and from the experiences of people who have talked about writing shell extensions, the hardest part of the process is managing PIDLs. There seems to be universal agreement that the thing to do is to write a PIDL management class that implements the IEnumIDList interface. Inside that class, you would allocate and free memory and generally manage the identifiers for your items.

This is hard in C++, but in C# the structures and types required for this aren't native and don't have good analogs. This means that a PIDL manager class in C# will be doing a lot of reading, writing, allocating, and freeing with IntPtrs. It's complicated, difficult to debug, and very prone to memory leaks. As it turns out, though, it's also not necessary. If you return a NULL when Windows asks for your IEnumIDList, it will just assume that you have no items in your view and drop the matter. If you never tell Windows that you have any items to manage, then there is no need for complicated PIDL management. This will save you time, energy, and code.

Of course, just because Windows thinks there are no items being managed by your extension doesn't mean that there really aren't any items. The UI displayed in your view is controlled by a window you create. This means that the UI for your extension can show any number of items while Windows thinks you have none. The two are completely separate.

Besides ease of implementation, there are performance reasons why you may want to tell Windows there are no items under management. In certain circumstances, Windows will ask for more than just the display name of your items. It may also ask for things like size, creation time, and other attributes. If, like me, you are writing an extension to manage items on another server, that could mean a lot of unnecessary round-trips.

I prefer to use a technique that's much simpler than traditional PIDL management. My extensions only worry about two PIDL values—0 and 255. A PIDL of 0 represents the extension itself or the directory on disk where the extension is rooted. A PIDL of 255 represents the currently selected item on my view.

For example, in my extension I might show a list of 100 or more virtual items. As the users click each item, I send Windows a special notification that the selected item has changed. Windows then asks me for the PIDL of the newly selected item; I tell it 255. Next, Windows asks me for the display name of the item with PIDL 255. Since I obviously know the display name of the item just selected, I return it. The salient code can be found in the GetItemObject method of the ShellView class and will be covered in much greater depth a little later in this article.

The ShellFolder Class

I store a number of variables internally in my ShellFolder class to make life a bit easier. These are shown here:

m_view This is a reference to the view object.

m_PIDLData This variable is a byte array that holds the PIDL passed initially into the folder object from Windows.

m_mdo When Windows asks for a specific PIDL it will sometimes ask for it in the guise of an IDataObject. The MyDataObject class is a simple implementation of that interface. The m_mdo variable is just a reference to the current data object under management.

m_workingDir My extension is managing a list of virtual items on a remote server. Since my users want to access those items through applications like Notepad or Microsoft Word, I need a place on disk to reconstitute them before the application tries to open them. This is that place. In general, it's a good idea to have a working directory associated with your extension if you plan to access your extension through the File Open or File Save dialogs of a Windows-based application.

m_currentItem This holds the file name of the reconstituted item. It follows the same rationale as m_workingDir.

m_form This is the System.Windows.Forms.Form that will hold your view UI. This is a form that you design.

m_parentWindow This represents the window housing my extension. It's particularly important if the extension is being loaded inside of a common dialog.

m_parentWindowText This variable holds a string representing the title of the parent window.

Properties

In addition to the variables I have just outlined, I have added a number of properties to the Shellfolder class. The following sections describe them briefly.

ParentWindow and ParentWindowText These are exposed as properties so that people inheriting from this base class can only retrieve them, not set them.

WorkingDirectory This is exposed as a property because some checking needs to happen when the value is set. Specifically, I want to make sure that the value always has its trailing backslash (\).

SelectedItem This is exposed as a property because when it is changed I need to notify the Windows shell. This is only relevant if the extension is sitting inside of a File Open or File Save dialog. In the set code, I query to see if I can find a pointer to a ICommDlgBrowser interface. If I can, then I'm in such a dialog and I notify it that the selected item has changed by calling OnStateChange.

Methods

In this section, I'll flesh out some of the details of the various methods in the ShellFolder class. While not exactly a line-by-line description, it should give you a good idea of what's going on.

byte[] PidlToBytes At a couple of places in the code it was easier for me to debug if I converted the contents of a PIDL into a byte array like this:

length0data0length1data1 ... lengthNdataN
This function computes the total length of the PIDL by iterating through it, then reads the whole thing into a byte array. Item lengths are 16-bit unsigned integers (ushort in C#).

void GetClassID At various points in the life of your extension, the Windows shell will want to know the GUID of the folder object. This method gets it.

long Initialize Namespace extensions can be anchored in any number of places in the Windows shell. They can be on the desktop, inside My Computer, My Network Places, Control Panel, a specific directory on disk, or many other places. This is called rooting. When Windows first initializes an extension, it calls the Initialize method and passes in a PIDL. This PIDL represents the unique path to the place in the Windows shell where your extension is rooted. You won't often need to make reference to this PIDL, but it's handy to keep around just in case. In my case, I just store it in the m_PIDLData variable and construct a new IDataObject with it.

void ParseDisplayName On occasion, Windows will ask your extension for the PIDL and attributes of a particular item that it thinks you are managing. It does this by calling ParseDisplayname. Since all my objects have the same PIDL value (255), I just return it. In addition, I tell Windows that are no attributes to report and that I have consumed 100 percent of the string that was passed in. (Some extensions only consume part of the string when they have subfolders, but that's not an issue here.)

void EnumObjects When Windows wants the complete list of items your extension is managing, it calls EnumObjects and expects to get an IEnumIDList interface back. Since I'm taking shortcuts with my PIDL management and want Windows to think I have no items, I just set the value to NULL.

void BindToObject Sometimes Windows wants a particular interface that it thinks your folder object might know about. When that happens it calls BindToObject, passing in a PIDL to the item it cares about and a CLSID for the interface it wants for that specific item. In my case Windows doesn't think I have any items, so when it calls this method it's really asking if the folder object implements the given interface. All I do is query the IUnknown interface of the folder to see if it implements the requested interface. If so, that's great. Otherwise, Windows gets a NULL in return.

At this point you might be wondering why the shell doesn't just call the QueryInterface method of my interface. After all, that's how one normally acquires a specific interface for an object. Windows doesn't necessarily think that the folder object is the item that supports the interface it is interested in. Instead, the folder might be holding a reference to an item that implements the desired interface. In that case, a call to QueryInterface won't help.

void CompareIDs When Windows thinks an extension is managing some objects, it may try to sort them. Since it has no idea what your PIDLs mean, it has to ask you about the relative value of two PIDLs. It does so by calling this method. As I have no items, this method will never be called so its implementation is blank.

long CreateViewObject Once the basic housekeeping is out of the way, Windows is going to ask your extension for a reference to its view object—the class that implements IShellView. That happens here. All I do is query the view class for its IShellView interface and return it. In addition, Windows gives me a handle to the window which is presently hosting the extension. I store that handle for future use and get the title bar text of the window as well. This will be important later when you need to know if you are in a File Open or File Save dialog.

long GetAttributesOf This method is called when Windows wants to know something about an item it thinks you are managing. In my case, I make sure the SFGAO_FILESYSTEM (0x40000000) bit is set. This makes Windows think that the item is part of the file system. This little white lie makes things work properly when the extension is loaded in a common dialog box. There is, however, an important side effect to this. Windows will absolutely think that the item is part of the file system. That means the extension will have to be notified just before Windows tries to actually open the item on disk so it can materialize it in the right spot. I'll dig much deeper into this problem later on when I discuss the implementation of the view class.

void GetUIObjectOf GetUIObjectOf is called when Windows wants a specific interface for an item it thinks you are managing. These can include context menus, drop targets, and other things. Since I don't bother implementing these interfaces, I just return NULL.

void GetDisplayNameOf In the simplified PIDL management system I use, there are only ever two possible PIDL values—0 and 255. GetDisplayNameOf is called when Windows wants the display name of a given item. If the value I get is 0, then I return the display name of the extension. In my case, that's the working directory. If the value of the PIDL is 255, then I return the display name of the currently selected item. In my case, that's a virtual file name. The reason I always return directory and file names is because it makes operating the extension in an open/save dialog much easier. For example, if in response to a PIDL of 0, I might return "c:\temp\" and in response to a PIDL of 255 I might return "test.txt," then Windows will think that the currently selected item is c:\temp\test.txt. When the user clicks the OK button in the open/save dialog, I will get the event and know which file the user thinks they're opening. Then, I'll reconstitute that file in the place the user is looking before the application actually tries to open it. I'll show you how that works shortly. The thing to remember here is that this function is your only opportunity to influence where Windows thinks your file actually is.

void SetNameOf Windows calls this method if the user is changing the name of an item managed by the extension. This is only important if you have written an extension that allows Windows to manage the UI instead of you. Since that's not the case here, I just set the return value to NULL and move on.

Many first-time extension writers think that they can take a shortcut by telling Windows to manage the UI of the extension. Their logic is that by doing so they can avoid having to create a class that implements IShellView. They are correct, but they are badly misled. While they will save some effort in UI development, they will absolutely have to implement a thorough PIDL management system. In my experience, that's a lot harder than creating a custom user interface.

void DoDefaultAction I wrote this method so I could easily trick the Windows shell into thinking that the OK button of an open/save dialog was pressed. This is important in order to allow users to double-click an item in your extension and have it automatically open. I will cover the criteria for deciding when to use this method a little later. For now, it's important to note that this function is only germane if the extension is hosted inside an open/save dialog, and all the checks for that condition are hidden from you in the method body.

That said, let's look at how this method operates. It's vital to check that I can successfully query for an ICommDlgBrowser interface. If I can, then I know the extension is in an open/save dialog and I call OnDefaultCommand to signal to the dialog that the OK button has been pushed. If not, then I know that my extension is being loaded someplace other than in a common dialog control and I needn't worry about telling Windows that the OK button was pushed.

void RegisterFunction/void UnregisterFunction These functions are conveniences that take care of the setup and cleanup of all the necessary registry keys associated with this extension. For example, in order for an extension to load, it has to be registered in a list of approved extensions under the following key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Shell
Extensions\Approved
For maximum compatibility with different versions of Windows, two special keys, WantsFORPARSING and Attributes, need to be added to the extension's normal COM registration keys. Since the RegisterFunction and UnregisterFunction methods are marked with the [ComRegisterFunction] and [ComUnregisterFunction] attributes respectively, they will be called when the assembly is registered and unregistered with COM. Alternately, you could build a setup project in Visual Studio and have it make the needed registry changes.

void Setup This is another special method I created that makes things a little easier for you. In the constructor of your custom extension that inherits from this base class, you call the Setup method to make sure that the base class can properly manage your working directory, event notifications, and the form to display for your UI. Once again, I'll discuss this a bit later.

The ShellView Class

Now let's look seriously at the view part of the folder/view pair. The view object is where all the UI for your extension lives. The principal responsibility of the view object is to manage a window you design that holds the visual contents of the extension.

Although there are a couple of pitfalls to avoid, the view object isn't nearly as difficult to implement as the folder object. In this project, the view object is in the class named ShellView and is contained in the file ShellView.cs.

Variables

My base view class needs to keep track of some important items, such as the following:

m_mc This is a reference to the ShellFolder class that created this view. I use this in a few places when I need to ask the folder for some information. In general, it's good practice to have your ShellFolder and ShellView classes keep references to each other.

m_form This is the System.Windows.Forms.Form-derived window that you supply. You can put anything you want on the window—the extension doesn't care.

m_shell This holds a reference to an IShellBrowser. The IShellBrowser interface is important for querying information about the window that is hosting the extension, as well as for signaling state changes in the extension.

m_hhook In order to trap when the users clicks the OK button in an open/save dialog, I have to install a Windows hook. A Windows hook tells the operating system to notify you when certain events occur. (The specific events depend on the kind of hook you set.) This particular hook notifies me whenever a window in the current thread has finished processing a message in its WndProc function. When the hook is installed, it returns a handle. The reason I keep this handle around is so I can uninstall the hook before the extension closes. It's very important to uninstall your hooks when you no longer need them because each hook slows down the operating system somewhat.

m_folderSettings When the extension loads, it will be informed of the current view settings in the Windows shell. These include the kind of view the folder had last, such as list, detail, and icon. It's important to restore these settings before the extension exits, so I keep them here.

h When a Windows hook is installed it requires a pointer to a callback function. That function will receive all the hooked notifications. The common language runtime (CLR) has no specific idea of a function pointer, but it will marshal a delegate as a function pointer when the delegate is passed as a parameter to an external function. This variable is a reference to that delegate.

m_fired Certain events that I look for in my hook may fire more than once on certain occasions. Since I only want to notify my view once that a certain event has occurred, I keep a Boolean around that lets me know if the event has already fired.

Methods

Most of the method implementations in this class are quite sparse; only two of them have much meat. Nonetheless, in this section I'm going to walk you through all of them and point out why some are stubs while others are well implemented.

ShellView This is the constructor for the class. There's not a lot going on in here. All I do is store a reference to the folder that created this view and create the delegate that will be used as the callback for the window hook. In fact, it's important to keep to a minimum the number of things you do in this constructor. This is because this call must complete before the UI can draw. If you load it up with lots of queries on remote machines or other time-consuming tasks, there will be a noticeable delay in the drawing of your window.

void GetWindow On occasion, Windows will ask the view for the HWND of its window, which this method returns.

void ContextSensitiveHelp If this extension supported context-sensitive help, then I would do something useful here. It doesn't, so the implementation is empty. If you did want to implement help, this is how it would work: when the user presses either F1 or Shift-F1 while your extension is active, this method will be called with a value of true. If you want to display help, this is where you do it. If you want the parent window hosting your extension not to bother showing its own help (since you'll be handling it), set the parameter of this method to false. If you don't, your user will see two help screens, which would be no help at all.

long TranslateAcceleratorA My extension doesn't worry about special accelerator keys, so I return S_FALSE (1) to indicate to Windows that it should handle the key. But just because I don't handle special keys doesn't mean you won't. For example, you might want to handle the Backspace key as a convenient way to go to a previous screen. If so, you should tell Windows that you have already handled this key and not to bother with it. In that case, you should return S_TRUE (0). Otherwise, both you and your parent window will try to handle the key.

void EnableModeless This method is used to enable or disable modeless dialogs. However, it's not currently used in Windows so the implementation is empty.

void UIActivate UIActivate is called whenever the UI state of the extension changes. When the extension is being closed (UI state 1), I have to close the underlying form.

void Refresh This is called whenever Windows thinks the extension needs to refresh the contents of the view. I don't care about this event in my extension, but you might. Specifically, this method is called when the user presses F5 while your extension has focus. If you are displaying a list of virtual items from a server, this is a good place to update your UI. If you're interested in making a small improvement to this base class, let the base class handle this event; you can still use this base class and get notified of refresh events.

void DestroyViewWindow This method is called when the extension is about to be unloaded. It's important here to close the underlying form and to uninstall the Windows hook. It's particularly important to perform the latter. If you don't, you will slow down your computer at the very least. In the worst-case scenario, you will leak RAM or crash the shell.

void GetCurrentInfo Windows calls the GetCurrentInfo method when it wants to know the view state of the extension. This is the same view state that was stored earlier in m_folderSettings. In this case, I just return that value.

void AddPropertySheetPages Although I have chosen not to add property pages to the Options property in the View menu item, if you are interested in doing this, refer to the IShellView::AddPropertySheetPages method in the Visual Studio on-line help.

void SaveViewState Before Windows unloads the extension, it provides it with an opportunity to save its current view state. Since this extension draws its own UI and doesn't really care about the view state, I do nothing. If, however, you wanted to alter your UI based on the current view state, you would need to save the last known state here.

void SelectItem Windows calls this method when it thinks a new item in the view has been selected. The manner in which this extension is constructed makes this method meaningless since you already know which item is currently selected.

If I were managing a list of items using a PIDL management system, then this method would be pretty important. Of course, that's more work than I really need to do.

long GetItemObject Windows calls this method when it wants extension-specific data about the items in the view. It tells the extension what specifically it wants through a UInt16 named uItem. In my case, I only care about two cases for uItem. First, there is SVGIO_SELECTION. Here, Windows is asking for the IDataObject for the currently selected item. In this case, I return an IDataObject interface that contains the PIDL value of 255. Second, there is SVGIO_ALLVIEW. With this case, Windows wants the same thing as SVGIO_SELECTION but for all the items. In this case, I return an IDataObject with a PIDL of 0. It's also possible that Windows will ask for some other kind of interface (other than IDataObject) in this call. For maximum flexibility, I query the view object for the requested interface. If it exists, return it. If the view doesn't have the interface, I query the folder for it. If I find it, I return it. If neither the view nor the folder have the interface, I return an error.

As you have most likely noticed, all the methods of the view class thus far have been remarkably short on code. I've intentionally saved the best for last. By far, the most interesting code is found in the next two methods.

void CreateViewWindow This method is what Windows calls when it wants you to render your extension's UI for the first time. In the case of this extension, that means something quite specific. All the UI for this extension is on a separate form that you maintain. This method will modify that form's properties so that it becomes a child of the window that contains the extension, not the desktop. It will also make sure that certain style bits are set on the form and that the form is properly positioned in the window.

Once the current folder settings for the Windows shell are passed into this method, you should save them for later use. Windows will pass in a reference to an IShellBrowser interface. This interface is important because it's the only way to get an HWND for the parent window. Also, the IShellBrowser is needed in order to signal the shell that the selected item has changed. (You can see this in use by examining the set handler for the SelectedItem property of the ShellFolder class.)

Now, call the GetWindow method of IShellBrowser to get the handle to the window hosting the extension and change the parent of the custom .NET Framework form to the newly acquired window. This is done with the following code:

ShellFolder.SetParent( m_form.Handle, hwnd );
ShellFolder.SetWindowLong( m_form.Handle, -16, 0x40000000 );
ShellFolder.SetWindowPos( m_form.Handle, 0, 0, 0, w, h, 0x0017 );
The first line of this code changes the parent. The next line changes the window style of the form to WS_CHILD. This is critical as any window that is the child of another window must have this style. The third line calls SetWindowPos to save the style change for the form. The SDK documentation is pretty clear on this point. You must call SetWindowPos directly after a call to SetWindowLong in order for the style changes to stick.

Resize the new form to fit within the parent window. One of the other parameters passed into this method is the RECT for the newly created area for your form. This is represented in the code by the variable prcView.

Finally, install the Windows hook that will post-process the events for all the Windows in the current thread. You must also make sure that the extension is properly notified when the user pushes the OK button in an open/save dialog.

long OnDialogMsg The extension I'm creating here manages virtual items that reside on a remote server. Windows, however, thinks that the items reside somewhere in the local file system. This means that I have to insert code between the time when the user selects the item he wants to open and when the application calling into the extension actually tries to read it from disk. The right time to do this is when the user selects the OK button in an open/save dialog. The interfaces Windows provides for shell namespace interaction do not provide an easy mechanism to capture this event. In fact, there are three distinct cases that need to be handled: common dialogs with extended messaging, common dialogs with no extended messaging, and Office applications.

Common Dialogs with Extended Messaging

When a Windows-based application uses the common open/save dialog boxes, it can either use a standard dialog or an extended dialog. The extended dialog will actually send a message when either the OK or Cancel buttons are selected. Unfortunately, most applications don't use the extended dialog, but for those that do, the code is fairly straightforward.

First, you have to look for a WM_NOTIFY event (0x004E). Then, you have to look for a special notify code of 0xFFFFFDA2. The dialog sends this code when the OK button is selected.

If you have triggered both of those conditions and a delegate has been registered to receive a notification of the event, then you fire the custom OnAction event.

Common Dialogs with No Extended Messaging

For those applications that use the nonextended common dialogs for open and save, the situation gets a bit trickier. In this case, I can only infer that the OK button has been pushed. This is because whenever a button on a dialog is selected, a BN_SETSTATE event is generated. It doesn't matter why the button was pushed. It could be because the user tabbed to it and pushed SPACEBAR, because the user used the mouse to click it, or because the user used a keyboard accelerator. In all cases, this event is triggered. Furthermore, the event is triggered with a nonzero WPARAM parameter.

Of course, there could be many buttons on the dialog, so you want to make sure that you're catching the correct one.

Each control on a dialog has a unique identifier known as a control ID. In the case of common dialogs, the control ID of the OK button is always 1. So, after you catch the BN_SETSTATE event with a nonzero WPARAM, you find out which control ID generated the event by calling GetDlgCtrlID. If the ID is equal to 1 and there are delegates registered to handle the event, then you should fire it off.

The preceding two scenarios will suffice for most Windows-based applications. However, one very important class of Windows-based applications does not use the common dialog boxes for opening and saving items. Unfortunately, this also happens to be the most popular set of applications currently in use—the Microsoft Office suite.

Microsoft Office Dialogs

The various Microsoft Office applications use a different set of dialogs for their open/save operations. Thankfully, these dialogs are common across all the applications in the suite. Nonetheless, in order to correctly support these applications you have to develop a different set of code.

The internal operation of the Office open/save dialogs isn't documented so, as in the case of nonextended common dialogs, you have to infer how they work. In my case, I spent a lot of time in the Spy++ tool that comes with Visual Studio.

Spy++ is a standalone application you can use to examine all the windows that are currently open, as well as all the messages they generate. I did a lot of spying on the message flow to and from the open/save dialogs in the various Office applications.

Apparently, when an Office application starts, it registers a special message named WM_OBJECTSEL with Windows. Then during the course of various open/save operations, it sends these messages. The trick, it seems, is to figure out which of those messages means that the Open.Save button was pushed.

Upon closer examination of the messages I noticed that when either the Open or Save buttons were pushed, the various Office apps would send a WM_OBJECTSEL with an LPARAM of 37 (0x25). Knowing this, the process became easy. I needed to find out what the message ID corresponding to WM_OBJECTSEL is by calling RegisterWindowMessage. If that message has been received, check the LPARAM. If the LPARAM is 0x25 and there are delegates to be triggered, fire the OnAction event.

A Parting Thought

If you look closely at the OnDialogMsg code you will notice that I maintain a Boolean value named m_fired. Although I've never seen it happen, it is possible that one of these events may fire more than once during an open or save process. I use this Boolean value to make doubly sure that I only trigger the OnAction event once.

As a convenience I thought it would be nice to include some information about the kind of event that was fired when I trigger OnAction. This method checks the text of the window hosting the extension to see if either the words "open" or "save" appear. As a result, if you register a delegate to handle this event, you will receive one of three values:

  • ShellActionType.Open—the result of an open request
  • ShellActionType.Save—the result of a save event
  • ShellActionType.Unknown—from an unknown dialog type
At this point you know everything you need to know to create your very own shell namespace extension from scratch. Beware: it's still a lot of work.

Your Very Own Shell Extension

I hate to rewrite what I can repurpose. For that reason I've wrapped up all of my accumulated understanding of extensions into a simple base class named ShellFolder. Creating a custom extension from this class couldn't be easier. You can start by compiling the files comprising the base class into an assembly and creating a new project of type Class Library. Add a reference to the base class assembly and create a new Form in your project, setting its window border style to none. Then create a method in your form with the signature of public void OnAction(MSDNMagazine.Shell.ShellActionType t). Make your new main class inherit from MSDNMagazine.Shell.ShellFolder. Figure 2 shows a complete sample implementation for your new class.

Figure 2 Custom Class Extension (Close Figure 2)
[ProgId("MSDN Magazine Sample Shell Namespace Extension")]
[Guid("6B49E580-186E-4f8c-AB6A-E55D6F0F171D")]
public class Class1 : MSDNMagazine.Shell.ShellFolder
{
    public Class1()
    {
        //
        // TODO: Add constructor logic here
        //

        Form1 m = new Form1();

        m.m_folder = this;

        this.Setup( "c:\\temp\\test", new
            MSDNMagazine.Shell.OnDefaultAction(m.OnAction), m );

    }

There are two things to notice here. First, I've given my new object a very descriptive progId. This is a shortcut. The base class will take this progId and make it the text label you see when you browse to your extension. For example, if I root my extension on the desktop its label will be "MSDN Magazine Sample Shell Namespace Extension." Of course, you can change this to whatever you like. Second, the constructor has to do three important things. It must create a new instance of my custom form (Form1), hand the form a reference to the ShellFolder-derived object, and call Setup in the base class. This function takes three parameters. First, it takes a working folder (c:\temp\). Second, it takes a delegate to be called when the OK/Open/Save button is pressed. My custom form (Form1) has a method named OnAction that I would like called in this case, so I pass it as the second parameter. Finally, it takes the form that holds the display contents of my extension. In this case, the form that contains my custom UI is in variable m.

That's the whole implementation for the main class—all that's left is for you to design the layout of your custom form. Just to round things off, though, Figure 3 shows a sample of a basic implementation of your OnAction event handler.

Figure 3 OnAction Event Handler (Close Figure 3)
public void OnAction(MSDNMagazine.Shell.ShellActionType t)
{

if ( t == MSDNMagazine.Shell.ShellActionType.Open )
    {
        // do special Open stuff
    }

    if ( t == MSDNMagazine.Shell.ShellActionType.Save )
    {
        // do special Save stuff
    }

    if ( t == MSDNMagazine.Shell.ShellActionType.Unknown )
    {
        // probably do nothing...
    }

}

There's one other thing I should mention. Let's say you want to support double-clicking as a method to open an item in your extension. To do this, you need to override the relevant double-click event in your UI and use code similar to the following:

// get whatever remote data you need Windows to open and
// store it in a definite place on disk — like c:\temp\test\foo.txt

this.m_folder.SelectedItem = "c:\\temp\\test\\foo.txt";
this.m_folder.DoDefaultAction()
This tells Windows that the currently selected item in your view is c:\temp\test\foo.txt and simulates to Windows that the user has clicked the OK/Open/Save button. All the code for this extension is available for download at the link at the top of this article.

Conclusion

After I had finished and tested my extension base class, one of my developers asked what the hardest part was. For me, it was setting aside some of the things I thought I knew about writing shell namespace extensions. You may never have to go beyond the base class, but if you do, at least you'll know how. Mark Twain was right. It's not what we don't know that hurts us—it's that we know so much that isn't true.

For related articles see:
Creating a Shell Namespace Extension

For background information see:
C# and the .NET Platform by Andrew Troelson (APress, 2003)

Dave Rensin recently left Omnisky Corporation to become Chief Technology Officer of QKnow Corporation, located in Washington, D.C. He is the author of four books and numerous articles on distributed computing systems. Contact him at Dave@rensin.com.


Editor's Note: PDC 2003 Trip Report

Editor's Note
PDC 2003 Trip Report

We almost didn't make it to Los Angeles in time for the Microsoft PDC in October. During our flight from New York, the wildfires that raged throughout Southern California took dead aim at the San Diego air traffic control center. As the center was being evacuated, we cruised over the Rockies, looking for a place to land. Well, it wasn't a problem in the end—we arrived almost half an hour early, which sort of obviates the whimpering tone of this whole paragraph. Sorry about that.

If you've never been to a PDC, it's the one must-go event if you're an early adopter. Sometimes the focus is a single advance—like the introduction of 32-bit Windows (1993) or the .NET Framework (2000). What made this event unique was that it provided the first glimpse of not one, but three upcoming products—Microsoft Windows "Longhorn," SQL Server "Yukon," and Visual Studio "Whidbey." You'll be hearing a lot more about those three products in the years to come, so it pays to know about them now so you can sound like an expert when they come up at cocktail parties. We also discovered a fourth product, Virtual PC, which makes it a lot easier to test out new operating systems while allowing you to keep your current configuration. More about that later.

Longhorn is the code name for the next version of the Microsoft Windows operating system. It sports a revamped user interface (code-named "Aero") and a new graphical subsystem (code-named "Avalon"). Avalon features a new procedural graphics language (code-named "XAML") that lets you create real programs using XML-based markup. Longhorn is also the first operating system from Microsoft that will be built with managed code.

A big question for developers is how to prepare for an upcoming OS that's a year or two away from final release. If you're working with the .NET Framework today, you've got a great head start. Since Longhorn will be based on the managed code model, you'll get a clear upgrade path from your existing .NET Framework-based apps. If you're still writing unmanaged code (with Visual Basic 6.0, for instance), this might be a good time to plan your migration path. The API for accessing Longhorn features, code-named "WinFX," is going to take some learning, but you'll understand it better if you have a solid grounding in the .NET Framework.

Yukon, the code name for the next version of SQL Server, was the second product unveiled at the PDC. Yukon has a number of remarkable features planned, the most noteworthy of which is that the CLR is built in. With Yukon, you'll be able to write stored procedures and server-side routines in managed code, which means you'll get all the advanced functions of .NET and you'll consolidate your work around a single model. Yukon also has other modern features, such as enhanced XML support and an XML datatype.

Whidbey, the code name for the next version of Visual Studio .NET and the .NET Framework, has similar advances that will make life better for developers. You'll get a radically improved IDE for development, pre-built code templates for common tasks, a whole bunch of language improvements (in C#, C++, and Visual Basic), and the introduction of improved development features for Windows Forms apps. And of course, everyone's favorite, the Edit and Continue feature, is scheduled to make a triumphant return. If you want more information about Visual Studio, see the Whidbey center on MSDN Online.

The fourth product we mentioned, Virtual PC, wasn't a focus of the PDC, but we've found it so useful that we had to give it a mention. Apple users may already be familiar with the product, but there is also a Windows-based version available from Microsoft (Microsoft Virtual PC 2004). Suppose you want to show off Longhorn, or even just run a copy of Windows 98 on your machine for support purposes. Using Virtual PC, you can cordon off a virtual disk drive and some memory, which together behave like a second machine running inside your main one. When you're looking at an early beta product like Longhorn, it's recommended that you not install it anywhere you have data you don't want to lose. What's great about Virtual PC is that you avoid dual booting. Your boot OS doesn't know it's sharing the machine with another system, and your test OS thinks that it's king of the virtual castle.

In this issue of MSDN Magazine, we're featuring information about Longhorn. Even though it's early, we wanted to make sure that our readers got the first looks at the future of Windows computing. In next month's issue, we'll follow up with coverage of Yukon. In the months to come, we'll be taking in-depth looks at all the technologies that were unveiled at the PDC. And yes, we'll still have room for articles that will help your work today.

—J.T.

FrontPage, IntelliSense, JScript, Microsoft, Microsoft Press, MSDN, Outlook, Visual Basic, Visual C++, Visual Studio, Windows, Windows Media, Windows NT, and Win32 are registered trademarks of Microsoft Corporation. Windows Server is a trademark of Microsoft Corporation. Other trademarks or tradenames mentioned herein are the property of their respective owners.

MSDN Magazine does not make any representation or warranty, express or implied with respect to any code or other information herein. MSDN Magazine disclaims any liability whatsoever for any use of such code or other information.


New Stuff: Resources for Your Developer Toolbox

New Stuff
Resources for Your Developer Toolbox
Theresa W. Carey

Contents
Assemble .NET-based Apps
Create and Edit Manuals and Help Files
R:BASE Gets an Upgrade
Mobile Apps with SMS Messaging
Where Do Your Online Customers Come From?
Shorten Build Time
What Time Is It?
The Bookshelf

Assemble .NET-based Apps

SembleWare has released Visual Welder 2.0, which lets you assemble Microsoft® .NET-based applications from components called Parts, which can be downloaded directly into an application from an online Parts catalog or corporate Parts repository. By snapping Parts together in Visual Welder's unique 3D Spatial Editor, you can assemble and modify ready-to-run .NET-based applications, view entities and relationships, preview screens, and modify individual Parts.

Visual Welder is an add-in that manages the assembly process and generates code in Microsoft Visual Basic® .NET. The Parts are the building blocks of a business application, spanning multiple tiers of the system including user screen database structures and business rules. Examples of typical Parts include an invoice, customer, or country. Parts and Composite Parts, such as a billing system or contact management system, are self-contained, functional units that can be readily assembled using Visual Welder to create ready-to-run applications.

http://www.sembleware.com

Create and Edit Manuals and Help Files

EC Software recently released Help & Manual 3.3, a help authoring tool that lets you create and edit your online help project and export it to a variety of file formats. Using a WYSIWYG word processor-style interface, it can output your initial help file to Adobe PDF, HTML Help, HTML, Winhelp, and other file formats. Tables of contents, keyword indices, and context-sensitive help are integrated; you create hyperlinks to other topics with a simple drag and drop and test it in design mode.

Help & Manual comes with a built-in screen capture function, an image editor optimized for screen shots, and a manual designer to customize the layout of print and PDF manuals.

http://www.ec-software.com

R:BASE Gets an Upgrade

R:BASE Technologies Inc. has released R:BASE 7.0, an upgrade of the company's database management system for Microsoft Windows®. Initially introduced in 1981, the product gets an update with integrated report writer options that allow users to print to most formats without third-party software. There are more than 60 new form controls, including Internet and animated AVI controls, and the ability to create customized programs in self-contained binary application files.

Internet controls, scrolling LED labels, and animated AVI controls have been added to the Form Designer, while the Report Designer also includes built-in barcode and crosstab controls for quick translation of users' data. R:BASE 7.0 includes expanded reporting options that allow output to a wide variety of formats.

The built-in HTML Writer allows reports to be viewed in HTML, and has a built-in PDF Writer. R:BASE also outputs to BMP, EMF, GIF, JPG, RTF, TIFF, WMF, and XLS formats. A variety of import and export formats, including XML, are supported by R:BASE 7.0. Data can also be shared with Microsoft Excel and other database programs. A built-in ODBC driver supports data exchange with any ODBC-compliant database. Self-contained binary application files can be password protected from design changes.

R:BASE also includes a suite of wizards (forms, reports, labels, queries, and applications).

For developers who want to fine tune their code directly, the software includes R:BASE 4GL, a full-featured programming language. R:BASE also retains the familiar R> prompt for quick searches of data sets. R:BASE 7.0 can be integrated and synchronized with any Pocket PC or Windows CE-based handheld device.

http://www.rbase.com

Mobile Apps with SMS Messaging

SYWARE Inc. recently announced that Visual CE 8 will add support for Short Message Service (SMS) messaging. This means mobile users can use Pocket PC or Pocket PC Phone Edition devices to send SMS messages directly from a database app. Users will not need to install additional hardware to use SMS messaging on Pocket PC Phone Edition devices. Now Pocket PC devices can use a General Packet Radio Service (GPRS) card, or a Bluetooth or tethered connection to a regular cell phone.

Visual CE is a relational database program with a drag and drop interface for designing forms and databases. Database forms can be designed, maintained, and run in color either on the handheld device or the desktop computer. Once the application is developed, records can be synchronized between handheld devices and desktop machines, or beamed from one handheld device to another. Users can also share data with ODBC-enabled software like Microsoft SQL Server™, Sybase, Oracle, Microsoft Access, and Visual Basic.

An SMS message can be sent from within a Visual CE database record by simply pushing a button. A set of predefined messages can also be stored in the database. Messages can include information such as time of day, location of user, and device ID. Visual CE runs on any Windows-based mobile device, including the Pocket PC and Windows CE.

http://www.syware.com

Where Do Your Online Customers Come From?

Melissa Data has announced IP2Location, which allows companies to see where customers are located based on their IP addresses. IP2Location matches an incoming IP address to the country, region, state, city, and ISP of the Web visitor.

By learning the location of visitors in real time, Web managers can display localized content, improve click-throughs and sales, prevent fraud, and implement many other solutions. For e-commerce Web sites, the IP2Location data tool can be used to identify suspicious Internet orders and their origins to reduce charge-backs, display native currency on order forms, and calculate local tax.

IP2Location can be used to improve network security as well. Web developers can use the tool to identify the origin of network intrusion, apply law enforcement, and track IP addresses to ensure compliance with national or international laws. The IP2Location database contains more than 1.5 million IP addresses from 235 countries. It has over 95 percent matching accuracy at the country and ISP levels.

http://www.MelissaData.com

Shorten Build Time

Electric Cloud has announced the availability of its software build accelerator, Electric Cloud 2.0, designed to reduce the length of the build process. The program is intended for medium to large software development teams that are frustrated by ever-increasing build times.

At the core of Electric Cloud 2.0 is Electric Make, which is plug-compatible with existing Make-based build infrastructures. It accelerates the build process by distributing the software build in parallel across clusters of inexpensive servers. It scales to accommodate additional servers to enhance both performance and throughput. In order to ensure that parallel builds will not break, Electric Cloud includes a dependency management system that captures dependency information by monitoring all files accessed during a build.

Electric Cloud 2.0 records and reuses build dependency information, minimizing the Makefile maintenance required to achieve and maintain build performance.

http://www.electric-cloud.com

What Time Is It?

Absolute Time Corrector 3.0, published by FlexibleSoft Co., runs on Windows 9x, Windows NT®, Windows 2000, and Windows XP and is designed to provide the exact time to local and network computers based on the time standard set forth by the National Institute of Standards and Technology (NIST).

Absolute Time Corrector queries NIST timeservers using the Internet and then compares NIST time to the time setting of the host computer. If there is a disparity, Absolute Time Corrector automatically adjusts the computer's time to correspond to the correct NIST time.

The new version of the software features an adjust hours-only option and smart time server selection, improved time zone support, and a LAN setup wizard. Once set up, the host computer can serve as a timeserver to any size network. The host can be set to check NIST time as often as desired, and each computer in the network can be set to check time on the host computer as often as necessary. This program comes with a list of about 250 tested time servers.

http://www.flexiblesoft.com

The Bookshelf

O'Reilly has published the ADO.NET Cookbook, written by Bill Hamilton. A solution is provided for each problem addressed in the book—a code sample that programmers can insert directly into their applications—as well as a discussion of how and why it works along with alternatives, limitations, and other relevant considerations. The book is intended as a companion to O'Reilly's ADO.NET in a Nutshell, which is coauthored by Hamilton.

Topics covered range from basic to highly advanced, and include data source connectivity, writing database-independent code, working with parameterized queries, retrieving and managing various types of data, and handling hierarchical data.

The book also addresses issues such as using identity and auto-increment columns, working with XML and Web Services, using strongly typed datasets, performing asynchronous operations, and maintaining database integrity. Keep up to date on new releases from O'Reilly at their Web site.

http://www.oreilly.com

Send your New Stuff to Theresa at newstuff@microsoft.com.

Theresa W. Carey is a freelance writer who lives in California. Her byline has appeared in Barron's and PC World.


Web Q&amp;A: DTS Follow-up, Web Services, Access Over the Network, and More

Web Q&A
DTS Follow-up, Web Services, Access Over the Network, and More
Edited by Nancy Michell

Q In the October 2003 issue of MSDN® Magazine, you suggested programmatically executing a Data Transformation Services (DTS) package. You failed to mention the "two lines of code" required to execute the command. Can you help?

A The two lines required to programmatically execute a DTS package really represent two steps. The first step is to load the package (from any of the three places a package can be saved: SQL Server®, Meta Data Services, or a file). The next step is to execute it. A complete program actually requires a few additional lines, of course, to declare variables and so forth.

Complete documentation for the DTS programmability object model can be found in SQL Server Books Online.

After setting a COM reference to the Microsoft DTSPackage Object Library and adding a using statement at the top of the C# module, the C# code would look like Figure 1.

Figure 1 DTS Package (Close Figure 1)
{
    Package2Class myPackage;
    myPackage = new DTS.Package2Class();
    object pVarPersistStgOfHost = null;
    myPackage.LoadFromSQLServer(
        "(local)",   //server name
        null,        //server user name
        null,        //server user password
        DTSSQLServerStorageFlags.DTSSQLStgFlag_UseTrustedConnection,
        null,        //package password
        null,        //package GUID
        null,        //package version GUID
        "MyPackage",                 //package name
        ref pVarPersistStgOfHost);   //internal use only
    myPackage.Execute();
}

There are a number of Visual Basic® and Visual Basic .NET articles on doing DTS programmatically, but not many on C#. A good article for C# that shows code like this, and also shows how to take things to the next step, is Knowledge Base article 319985 "Handle Data Transformation Services Package Events in Visual C# .NET".

A good troubleshooting article on problems that occur when programmatically executing DTS packages is Knowledge Base article 323685 "Troubleshoot DTS Packages That You Run from Visual Basic Applications".

Q I have an ASP.NET application talking to a Web Service. I am getting this intermittent error: "Underlying connection closed error. An unexpected error occurred on send." The architecture includes a third-party hardware load balancer which routes the calls to the Web Service.

After some troubleshooting, I found that the application works fine if the load balancer is not used. It turns out that the load balancer redirects clients to an available server if there isn't any activity from this client for one minute. How can I fix this problem?

A If the ASP.NET Web app and the Web Service are both on the load balancer, you should know that it is not a good idea to be load balancing the ASPX-ASMX call. The client is already load balanced to a particular node when they hit the ASPX page; going back to the load balancer to get routed to another machine to call the ASMX page will not be good for performance. Furthermore, the idea of invoking a Web Service over the local machine will hurt performance. Instead you should simply add a standard reference to the DLL and invoke the code directly. It serves as a workaround.

Q I built a Web Service and added a public property to the class. When I generate the proxy class in my consumer ASP.NET app, this property doesn't come along with it. Do I have to add an attribute to the property the same way as you add [WebMethod] to Web Service methods?

A It doesn't make much sense to expose public properties on a stateless, distributed request. It has been a discouraged practice since Microsoft® Transaction Server 1.0, although the platform still supported it to an extent. That said, there are some situations in which you may want to use "properties" in a Web Service, such as when you're passing context or data that's consistent across all Web methods. You can mimic properties in a Web Service by using a SOAP header. It would look like the code shown in Figure 2. The Web client would look like this:

ws3.BillingService3 service = new ws3.BillingService3();
ws3.Properties props = new ws3.Properties();
props.FirstName = "John";
props.LastName = "Doe";
service.PropertiesValue = props;
service.UpdateOrder("hello", "world", 1, true);

Figure 2 Web Service (Close Figure 2)
public class BillingService3 : System.Web.Services.WebService
 {
  public class Properties : SoapHeader
  {
   public string FirstName;
   public string LastName;
  }

  public Properties PropertiesHeader;

  public BillingService3()
  {
   //CODEGEN: This call is required by the ASP.NET Web Services Designer
   InitializeComponent();
  }
  [SoapHeader("PropertiesHeader", Direction=SoapHeaderDirection.In)]
  [WebMethod]
  public bool UpdateOrder(string foo, string bar, long i, bool flag)
  {
   return false;
  }
}

Q I am authoring a Web page in FrontPage 2000 that connects to a Microsoft Access database that resides on another machine. The IUSR and IWAM accounts are local on both machines with the same name and password. This machine is running Windows NT® 4.0 with IIS with SP 6a; the other machine running IIS has FrontPage® 2000 installed on it.

For some reason this does not work and I get the following error: "The Microsoft Jet Database Engine cannot open the file '(unknown).' It is already opened exclusively by another user or you need permission to view."

By switching the roles of each of these machines (making the IIS the file server and the file server the IIS), everything works fine. What's wrong here?

A Did you give the user's (IUSR/IWAM) permissions to create files in the folder that the Access database lives in? When Access databases are opened, they create a file to mark who is accessing the database. If you cannot create the file, you will get this error and sometimes you even have to give your current user profile temp catalog relaxed permissions, depending on where the database lives. Note that if you open the file as read-only/exclusive, then no file needs to be created. In addition, you can get this error with Access if your memory is fragmented on the server running IIS. When you open an Access database via Jet, Jet attempts to allocate 64MB of virtual memory in one continuous block; if this fails, you get the Unknown error as well.

Q Is there a limit to the size or number of JScript® files in Microsoft Internet Explorer? Here's why I ask: I have many JScript files included in an HTML page. In Internet Explorer 5.5 SP2 with a slow network connection, I sometimes get "object expected" errors (refering to a function in a .js file). I confirmed that the file is in the cache, but when it loads in the debugger, it is empty.

I considered a workaround to add the following exception handling code, but I don't know if this is a valid approach:

Try
{
   // call a test
   }
   catch
   {
    //insert <SCRIPT language=JavaScript src="test.js"></SCRIPT> again.
}

A This isn't a good workaround because if Internet Explorer does in fact reload the script this way you could wind up with an endless loop of loading scripts. Scripts that load themselves would be a potential problem due to unchecked resource allocation. In addition, the Document Object Model isn't guaranteed to be ready for use before the body OnLoad event has fired.

Since scripts are executed in Internet Explorer sequentially, to see if your script has loaded you could set a sentinel variable:

<script language="JavaScript">
function myfunction {}
•••
// script is finished *loading*
myScriptHasLoaded = true;
</script>

To answer your first question, there is one known app out there that has 100,000 lines of JScript code in an HTA application. Of course this is very bad, but it does work, and they haven't hit a limit in Internet Explorer yet.

Q I have a single domain and need to implement a certain password complexity criteria for only one group of users. What are the best practices around this?

A You need to implement a password filter, which are standard DLLs invoked by the security provider each time a user changes his password. Passwords may be accepted or refused, based on custom policy.

Now you probably are wondering how a password filter can facilitate multiple password policies within a single domain. Policy is stored within the account database that stores the accounts, so there can only be a single policy for each domain or a single policy for the local account database. This may change in the future, but right now that's how it works.

However, it sounds like you just need different complexity requirements for your passwords (such as minimum length, maximum length, strong password, and so on). This can be accomplished with a password filter, which will let you apply your custom policy (even more complex or flexible than the one provided by Windows) to your specific groups.

When a password filter is invoked, it is provided with the user name of the user who wants to change the password, along with the new password. So the filter can check which groups it belongs to and, according to the specific settings with which your password filter has been configured, decide which type of policy to apply and whether to accept or refuse the new password.

Just remember that there can only be a single password policy for each account database; that means a single policy for each domain. Individual computers within the domain can have unique password policies, but those will only affect local accounts on that computer. There is currently no way around this, but that may change in the future.

Q I have a question regarding the use of DataAdapter classes in the Microsoft .NET Framework. One of my engineers has suggested that, based on his research, my team should be using data adapters in our Web Services instead of having stored procedures return data into our datasets. Others have looked at this issue and come to the conclusion that data adapters are primarily intended for traditional 32-bit client/server applications, not Web-based applications. Which approach is better?

A Basically, you are asking if data adapters are best used for Visual Basic .NET rich client apps where you're maintaining your database connection rather than in a Web Service environment where you create the connection, use it, and then drop it.

One way to look at it is that since ADO.NET favors a disconnected model, there is no reason not to use them in Web-based apps. Additionally, the adapter can leverage existing stored procedures, manage the connection, and cut out a bunch of ADO.NET code that would otherwise have to be written and maintained. You seem somewhat concerned that you could have concurrency issues with the data adapters.

Here is some real-world experience to help answer your concerns about the role of data adapters in a large scale, enterprise Web Services application considering performance, concurrency, and code maintenance. The .NET Data Access Architecture Guide on the MSDN Patterns and Practices Developer Center offers good information on performance. MSDN also has an article which contains some actual measurements from simulated scenarios (see Performance Comparison: Data Access Techniques).

A data reader can offer better scalability and performance if used appropriately. The longer you hold a connection, the more scalability suffers. Whether it makes a difference in a specific situation depends on a lot of factors.

Stored procedures are recommended over dynamic SQL unless you want to turn your database server into a database query compilation engine. Again, the .NET data access architecture guide discusses this.

An additional factor is how the resulting data is to be transported from one tier to the other. If you need to remote your data tier and transfer the resulting data to a client, then a data reader is not the way to go. However, be aware of the serious performance issues you will encounter if the resulting dataset is large. If you use data adapters, then you should make sure the connections are closed when the request is completed.

The real issue here is that the resulting dataset performance is more a function of dataset complexity than of size. While more data obviously increases the payload, the complexity of datasets can be even more of an issue.

Further, consider your methods of transferring data between client and server. When you have highly complicated data structures (such as complex datasets), the Web Service means of serialization offers much better performance than remoting serialization.

In response to the initial question, you may be confusing their concepts. Datasets always use data adapters if they are getting data from another dataset. Data adapters do not create some kind of connected state to the data. The reasons not to use data readers are really scalability and performance, as noted.

Q What is the best way to store an XML file (as BLOB data) in SQL Server? Should I transform and traverse the XML, and after parsing it store it in a meaningful format in database tables? Or should I store RAW XML BLOB data in a field (text field) in a table, or use something else completely?

A The answer depends on the structure of the data and the types of queries. For example, if it is representing mainly relational data encoded into XML and you want to query it and return parts, then shred it into relational tables using annotated schema, bulkload, or OpenXML. Otherwise, if you always want the same XML document in full without complex queries (XML structure is markup-centric and does not easily map into relations), then use a TEXT/NTEXT field.

Q Is there any support for an XML data type in SQL Server 2000? Specifically, I want to push a small collection of elements into a field as XML and then issue queries that test for the presence of some elements. I would like to be able to do an XPath query on each row that contained an XML field.

A While there is no XML datatype in SQL Server 2000, you can store XML in nvarchar columns and use OpenXML to test for the presence or absence of paths if you can live with the following limitations. First, all of the XML documents must fit into an nvarchar/varchar column (4000 Unicode characters, 8000 single-byte characters maximum length). Second, you have to use a cursor inside a stored procedure to iterate over every column so that you can use sp_xml_preparedocument to parse the XML (and use sp_xml_removedocument to clean the memory). This limitation could unnecessarily shift complexity from managed code to procedural T-SQL—a trade-off that you have to decide upon.

Of course, you can do the parse/XPath execution using the System.XML component inside the managed code on the middle tier. The OpenXML part is only required if you need to process your data in SQL. On the middle tier, you have more flexibility, but the cost is that you have to transfer all your XML data to the middle tier for processing. Also note that in the upcoming version of SQL Server, code-named "Yukon," there will be native XML support. See Introducing SQL Server 2005 for more information.

Got a question? Send questions and comments to webqa@microsoft.com.

Thanks to the following Microsoft developers for their technical expertise: Earl Beaman, Steve Beaugé, Efi Bregman, Guillaume Cadet, Mark Cleary, Denny Dayton, Kurt Dillard, Naphtaly Friedman, Miguel Isidoro, Anders Janson, Michael Kaplan, Bhalinder Kehal,Andreas Klein, Rogvi Knudsen, Douglas Laudenschlager, Chun Liang, Dan Liebling, Rick Lievano, Maurizio Macagno, Bill Maurer, Matt Neerincx, Mauro Ottaviani, Malini Puttaiah, David Qiu, Pavel Rozalski, Michael Rys, K. Saravana, Chad Sheffield, Louis Weinstein


Data Points: Exploring SQL Server Triggers: Part 2

New information has been added to this article since publication.
Refer to the Editor's Update below.

Data Points
Exploring SQL Server Triggers: Part 2
John Papa

Code download available at: DataPoints0401.exe (111KB)
Contents
INSTEAD OF Triggers
Updating Derived Columns
Checking for Changes
Wrapping Up

This month I will expand on last month's column—the foundation of SQL Server™ triggers and the features that they expose. I have targeted this column in response to many of the questions I have received regarding triggers. I'll begin with INSTEAD OF triggers and the differences between them and AFTER triggers. I'll then discuss some situations in which to use INSTEAD OF triggers including allowing updates to multiple tables through views. Finally, I'll discuss how to handle transactions in triggers, writing triggers to handle changes to multiple rows in a table, and determining which columns were modified within triggers.

INSTEAD OF Triggers

AFTER triggers (also known as FOR triggers) execute following the triggering action, such as an insert, update, or delete. For example, an AFTER trigger on an Employees table will fire after an UPDATE statement has been executed against the Employees table. Thus, the trigger does not fire until the row or rows have been inserted and constraints have been checked and passed. INSTEAD OF triggers are inherently different from AFTER triggers because INSTEAD OF triggers fire in place of the triggering action. So using the same example, if an INSTEAD OF UPDATE trigger exists on the Employees table and an UPDATE statement is executed against the Employees table, then the UPDATE statement will not change a row in the Employees table. Instead, the UPDATE statement is used to kick off the INSTEAD OF UPDATE trigger, which may or may not modify data in the Employees table.

So how do you determine the right time and place to use an INSTEAD OF trigger? Several key factors are worth considering when making this decision. AFTER triggers are more commonly used in situations where actions must be taken following data modifications on tables. For example, an AFTER trigger could be used to log any data updates to a separate auditing table. INSTEAD OF triggers could do the same job, but they are less efficient in this particular scenario since the update will be allowed exactly as it occurred after writing to the audit table.

In general, in any situation where the data modification will not be affected, an AFTER trigger is more efficient. An AFTER trigger is also a great choice when the data modification is evaluated and is either allowed to commit as a whole or denied entirely. For example, a rule could exist that any change to a product's price of more than 30 percent in the Products table must be undone. An AFTER trigger could do the job here nicely, using the inserted and deleted tables to compare the price of the product and then roll back the transaction if need be. These are ideal situations for AFTER triggers, but sometimes INSTEAD OF triggers are better.

INSTEAD OF triggers are a great feature that allow you to perform complex action queries in place of a single action query on a table or a view. Unlike AFTER triggers, which can only be created against tables, INSTEAD OF triggers can be created against both tables and views. I have often been asked how to resolve the situation in which there is a view that represents a join of multiple tables and you want to allow an update to the view. If the view exposes the primary key and required fields of a base table, it is often simple to update the view's base table. However, when there are multiple tables represented in a join, the update logic is more complicated than a single UPDATE statement. So how do you resolve this situation using alternative tools? One solution is to place an INSTEAD OF UPDATE trigger on the view. INSTEAD OF triggers can be defined on views with one or more tables. The INSTEAD OF trigger can then extend the type of modifications that will take place on the multiple base tables.

For example, if a view joins the Customers, Products, Orders, and Order Details tables together to show all of the data on a screen through an application, updates could be allowed to take place through this view. Assuming a view exists that joins these four tables in the Northwind database and is named vwCustomersOrdersOrderDetailsProducts, it could look like Figure 1.

Figure 1 View Joining Customers and Their Order Details (Close Figure 1)
CREATE VIEW vwCustomersOrdersOrderDetailsProducts
AS
    SELECT    c.CustomerID,
        c.CompanyName,
        o.OrderID,
        o.OrderDate,
        od.UnitPrice,
        od.Quantity,
        od.Discount,
        p.ProductID,
        p.ProductName
    FROM Customers c
        INNER JOIN Orders o ON c.CustomerID = o.CustomerID
        INNER JOIN [Order Details] od ON o.OrderID = od.OrderID
        INNER JOIN Products p ON od.ProductID = p.ProductID
GO

The vwCustomersOrdersOrderDetailsProducts view joins the four tables and exposes a sampling of fields from each of the tables. One key to remember when designing views that will have INSTEAD OF UPDATE triggers is that it is helpful to include the primary key fields from each table in the SELECT statement. Even if these fields are not used in the application, they can be used in the INSTEAD OF trigger to locate the row(s) that were intended to be modified and then make the appropriate changes in the base tables. Let's assume that you want to allow updates to this view to funnel down to the base tables on the non-key fields. Code would then have to be written in the INSTEAD OF UPDATE trigger to update the CompanyName in the Customers table, the OrderDate in the Orders table, the UnitPrice and Quantity in the Order Details table, and the ProductName in the Products table. At this point using an AFTER trigger won't cut it, but an INSTEAD OF trigger is a good option (see Figure 2).

Figure 2 INSTEAD OF Trigger Updating a View (Close Figure 2)
CREATE TRIGGER tr_vwCustomersOrdersOrderDetailsProducts_IO_U
    ON vwCustomersOrdersOrderDetailsProducts
    INSTEAD OF UPDATE
AS
    — Update the Customers
    UPDATE     Customers
    SET    CompanyName = i.CompanyName
    FROM    inserted i
        INNER JOIN Customers c ON i.CustomerID = c.CustomerID

    — Update the Orders
    UPDATE     Orders
    SET    OrderDate = i.OrderDate
    FROM    inserted i
        INNER JOIN Orders o ON i.OrderID = o.OrderID

    — Update the Order Details
    UPDATE     [Order Details]
    SET    UnitPrice = i.UnitPrice,
        Quantity = i.Quantity
    FROM    inserted i
        INNER JOIN [Order Details] od ON i.OrderID = od.OrderID AND
            i.ProductID = od.ProductID

    — Update the Products
    UPDATE     Products
    SET    ProductName = i.ProductName
    FROM    inserted i
        INNER JOIN Products p ON i.ProductID = p.ProductID
GO

Notice that the code in the INSTEAD OF UPDATE trigger in Figure 2 (tr_vwCustomersOrdersOrderDetailsProducts_IO_U) contains four UPDATE statements. Each targets one of the base tables with the purpose of modifying the exposed non-key fields. The key fields for each base table are joined in the UPDATE statements to the corresponding fields that the view exposes. This allows the UPDATE statements to locate the appropriate rows in the underlying tables and update only those rows. The following UPDATE statement puts this INSTEAD OF trigger to the test:

UPDATE   vwCustomersOrdersOrderDetailsProducts
SET    Quantity = 100,
       UnitPrice = 20,
       CompanyName = 'Fake Name',
       OrderDate = '11/23/2001',
       ProductName = 'Widget'
WHERE    OrderID = 10265
AND      ProductID = 17

If you examine the values in the underlying tables (either through the view or in the tables themselves) it is evident that the values have been updated. Of course, several modifications could be made to this INSTEAD OF UDPATE trigger to yield different results. For example, it is not a requirement to write the INSTEAD OF trigger to modify all four base tables. Therefore, one or more of the UDPATE statements contained within the trigger could be removed. Assuming that the trigger was intended only to update the Order Details values, it could be modified to update only those fields in the Order Details table—ignoring any changes that were attempted on the other base tables. In this situation, no error is raised nor are the changes made to Customers, Products, or Orders tables. Of course, an error could be raised if one of these fields was updated. The UPDATE and COLUMNS_UPDATED functions are ideal for checking what fields have been modified, as I'll demonstrate later in this column.

Figure 2 also demonstrates how to prepare a trigger to deal with changes to multiple rows. Note how the UPDATE statements join the inserted table to each base table on the key field. This ensures that the updates will occur on all rows that were modified by the original UPDATE statement on the view. This could be done by looping through the rows in the inserted table, too. However, it is a good idea to avoid using cursors in general, and especially when using triggers. SQL Server is built to handle data in sets while cursors handle rows one at a time. Using cursors in a trigger can slow down your application, so it's better to use a more efficient alternative like the one in Figure 2 or a subquery.

Another way that this INSTEAD OF UPDATE trigger could be modified is to make it also fire on INSERT and DELETE statements on the view. This would mean the trigger would likely perform an INSERT or a DELETE where appropriate. Keep in mind, however, that a DELETE could remove several records depending on how the trigger is written. Therefore, it is critical that the requirements of the trigger be reviewed and thorough testing be performed before implementation. An INSTEAD OF INSERT trigger could be written on this view so that it inserts a new customer, order, order detail, and product. It could also be written to determine if the customer was new before inserting a new one (and so on with the other records). The opportunities are numerous when deploying INSTEAD OF triggers, but of course it is essential that the trigger is designed to resolve the appropriate requirement.

Usually when an UPDATE statement that references a table attempts to set the value of a computed, identity, or timestamp column, an error is generated because the values for these columns must be determined by SQL Server. These columns must be included in the UPDATE statement to meet the NOT NULL requirement of the column. However, if the UPDATE statement references a view with an INSTEAD OF UPDATE trigger, the logic defined in the trigger can bypass these columns and avoid the error. To do so, the trigger must not try to update the values for the corresponding columns in the base table (leave them out of the SET clause of the UPDATE statement). When a record is processed from the inserted table, the computed, identity, or timestamp column can contain a dummy value to meet the NOT NULL column requirement, but the INSTEAD OF UPDATE trigger ignores those values and the correct values are set by SQL Server.

Updating Derived Columns

INSTEAD OF triggers are also commonly used to UPDATE the base columns in calculated columns. For example, assume that a view exists called vwOrdersOrderDetailsProducts as shown here:

CREATE VIEW vwOrdersOrderDetailsProducts
AS
    SELECT    o.OrderID,
        o.OrderDate,
        od.UnitPrice * od.Quantity AS ExtendedPrice,
        p.ProductID,
        p.ProductName
    FROM Orders o
        INNER JOIN [Order Details] od ON o.OrderID = od.OrderID
        INNER JOIN Products p ON od.ProductID = p.ProductID
GO
This view exposes a calculated column called ExtendedPrice that cannot be updated directly because it does not resolve itself to a single column in a table. However, you could implement a business rule that says that if the ExtendedPrice field is updated through this view, then Quantity should not change but the UnitPrice is modified. (I know this rule is a bit odd, but bear with me on this.) An INSTEAD OF UPDATE trigger could be written to enforce this business rule using the code shown here:
CREATE TRIGGER tr_vwOrdersOrderDetailsProducts_IO_U
    ON vwOrdersOrderDetailsProducts
    INSTEAD OF UPDATE
AS
    UPDATE     [Order Details]
    SET        UnitPrice = i.ExtendedPrice / Quantity
    FROM       inserted i
        INNER JOIN [Order Details] od ON i.OrderID = od.OrderID AND
               i.ProductID = od.ProductID

GO
This code shows how to replace an update to a calculated column with logic in INSTEAD OF triggers. Assuming that the Quantity for a product on a particular order is 100 and the ExtendedPrice is updated to 200, the new UnitPrice value will be 2. In this scenario, when the UPDATE statement is executed that modifies the ExtendedPrice column, the net effect is that the UnitPrice is set to the ExtendedPrice value divided by the Quantity. The following code can be used to test this situation:
UPDATE   vwOrdersOrderDetailsProducts
SET      ExtendedPrice = 200
WHERE    OrderID = 10265
AND      ProductID = 17

Checking for Changes

The UPDATE and COLUMNS_UPDATED functions are available within both types of triggers to allow the trigger to determine which columns were modified by the triggering action statement. For example, the following trigger prevents any modifications to the lastname column in the Employees table. Here, the UPDATE function is used to determine if a modification was made to the column. If so, then an error is raised using the RAISERROR function and the transaction is rolled back, which undoes any changes that were made. The UPDATE function works in both the AFTER and INSTEAD OF triggers, but not outside of the triggers:

CREATE TRIGGER tr_Employees_U on Employees AFTER UPDATE AS
    IF UPDATE(lastname)
    BEGIN
        RAISERROR ('cannot change lastname', 16, 1)
        ROLLBACK TRAN
        RETURN
    END
GO

The UPDATE function is designed to determine if a single column was modified by an INSERT OR UPDATE statement. UPADATE (column) is the standard method used to check for updates. However, it becomes less efficient when the need arises to check if multiple columns were affected by an INSERT or UPDATE. This is where the COLUMNS_UPDATED function steps into the spotlight. The COLUMNS_UPDATED function returns a bitmask to evaluate if specific columns were modified. The bitmask contains a bit for every column in the table that was modified, in the order that the columns are defined in the table's schema. If the column was modified, the bit's value is 1; otherwise it is 0. Unlike the conventional way to read bytes going right to left, the bitmask reads from left to right. For example, the following code shows a trigger on the Order Details table that checks to see if both the Quantity and UnitPrice fields are modified:

CREATE TRIGGER tr_OrderDetails ON [Order Details] AFTER UPDATE
AS
    IF (COLUMNS_UPDATED() = 12)
    BEGIN
        RAISERROR ('Cannot change both UnitPrice and Quantity at the
                    same time', 16, 1)
        ROLLBACK TRAN
    END
GO

If both fields are modified, an error is raised and the transaction is rolled back. For the Order Details table, the COLUMNS_UPDATED function returns a single byte with the first five bits representing the columns in the Order Details table. Since the situation called for determining if only the third and fourth columns were modified, it looked to see if only those bits were set to 1. When the third and fourth bits are turned on, it looks like this: 00110. Since this bitmask represents the powers of 2, the first bit represents 1, the second represents 2, the third represents 4, the fourth represents 8, and the fifth represents 16 (yes, this is the reverse order of bits in a normal binary number). Therefore, the bitmask value that means only the UnitPrice and Quantity were changed is 00110, which equates to the integer value of 12 (4 + 8). Note that this trigger only rolls back the transaction if the UnitPrice and Quantity fields were modified. If any other fields were modified, the bitmask would be different and thus not equal to the integer value of 12. If the trigger were modified to prohibit changes to these two fields even if other fields were modified, it could be rewritten like so:

ALTER TRIGGER tr_OrderDetails ON [Order Details] AFTER UPDATE
AS
    IF (COLUMNS_UPDATED() & 12 >= 12)
    BEGIN
        RAISERROR ('Cannot change both UnitPrice and Quantity
                   at the same time', 16, 1)
        ROLLBACK TRAN
    END
GO
Notice how the COLUMNS_UPDATED function is now checked to see if the value is greater than or equal to 12. [ Editor's Update - 12/6/2004: This trigger incorrectly compared the sum of the bit mask to the sum of the 3rd and 4th columns using the condition "COLUMNS_UPDATED() >= 12". Any time a column after the 4th column was updated, this trigger would incorrectly raise an error. The code for the trigger has been corrected to take this into consideration by using the binary & operator.] If you modified the UnitPrice, Quantity, and Discount columns, the bitmask would be 00111, resulting in an integer value of 28 (4 + 8 + 16). When there are more than eight columns in a table, this function returns the bitmasks in chunks of multiple bytes with the first eight columns in the first byte, columns 9 through 16 in the second byte, and so on. This function is more useful in determining which columns were updated than the UPDATE function for each column.

As demonstrated earlier, triggers can roll back transactions if a specific business rule is satisfied. When a trigger that contains a rollback is executed from a SQL batch the entire batch is canceled. Thus, all data that was modified by the triggering action is rolled back by the trigger's ROLLBACK TRANSACTION statement. However, a rollback does not cause the trigger to stop executing statements. Any statements following a ROLLBACK TRANSACTION statement are executed, which is fine, but there are consequences to putting statements after a ROLLBACK TRANSACTION. Specifically, as a trigger continues to execute any remaining statements after the rollback, any modifications that happen after the rollback are not rolled back. This occurs because once a ROLLBACK TRANSACTION has been issued within the trigger, the transaction is closed. Thus, when a new action query statement is executed, a new transaction begins separate from the original. Therefore, it is generally not recommended that you put any statements after a ROLLBACK TRANSACTION.

Just as a rollback does not automatically exit the trigger, it also does not raise an error automatically. If the rollback must occur and an error must be raised, the RAISERROR statement should follow the rollback immediately before exiting the trigger code.

Wrapping Up

There are other noteworthy characteristics of INSTEAD OF triggers. If an INSTEAD OF trigger on a table fires an action query that causes a data modification on the same table that would normally fire the same INSTEAD OF trigger, the trigger is not called recursively. So, if an INSTEAD OF UPDATE trigger existed on the Employees table and this trigger had code inside of it that updated the Employees table, it would not continue to recursively call the same INSTEAD OF trigger. It would get ugly fast if this kind of recursion were allowed. Another difference between INSTEAD OF triggers and AFTER triggers is that Text, Ntext, and Image columns can appear in the inserted and delete tables in INSTEAD OF triggers. These binary fields will show up as VARCHAR values, however, in the INSTEAD OF trigger's inserted and deleted tables, where applicable, and not as their original datatypes.

There is a useful stored procedure for examining triggers—the sp_helptrigger system stored proc. It returns the type of triggers defined on a table that you pass to the stored proc. Using this, you can see what triggers are associated with a table, what actions cause them to fire, and if they are AFTER or INSTEAD OF triggers.

In the past two columns I've explored many facets of AFTER and INSTEAD OF triggers. Of course, there are other scenarios in which they are useful and times they are not recommended. One situation in which triggers are not efficient is when the trigger must perform queries against other tables. In these cases, the performance of the trigger and the triggering action query may suffer. Triggers are great tools when used appropriately, but be sure to test your application thoroughly when using them.

Send your questions and comments for John to mmdata@microsoft.com.

John Papa is a baseball fanatic who spends most of his summer nights rooting for the Yankees with his two little girls, wife, and faithful dog, Kadi. He has authored several books on ADO, XML, and SQL Server and can often be found speaking at industry conferences such as VSLive. You can reach him at mmdata@microsoft.com.


Cutting Edge: Extend the ASP.NET DataGrid with Client-side Behaviors

Cutting Edge
Extend the ASP.NET DataGrid with Client-side Behaviors
Dino Esposito

Code download available at: CuttingEdge0401.exe (128KB)
Contents
A Quick Tour of DHTML Behaviors
The Draggable DataGrid
Enhancing the Dragdrop Behavior
Persisting the New Column Order
The DataGrid Whisperer
What About Paging?
What About Sorting?
Summing It Up

To the majority of ASP.NET developers, the DataGrid control is a basic and essential tool, much like the rolling pin for serious pizza makers. The DataGrid control in ASP.NET 1.x is an extremely powerful and versatile control and a great server tool. But you can make it even more powerful with the addition of just a little bit of client-side script. Until now, I hadn't played with JavaScript and DataGrid controls together. Recently I ran into a wonderful piece that Dave Massy wrote a couple of years ago for his "DHTML Dude" column on MSDN® Online. It discusses creative ways to revive the HTML <table> element. Among other things, Dave shows how to sort the content of a table by column and how to drag columns around.

He also demonstrates the use of DHTML behaviors with a <table> element. I realized that when rendered to HTML and served to the browser, the DataGrid control is a plain <table> element. Sure, it may contain a lot of style attributes, but its skeleton is a classic HTML table. This made me realize that I could create a DataGrid control with draggable columns and client-side sorting. This month's column chronicles my experiments. You can download the source code to see that I'm not cheating.

A Quick Tour of DHTML Behaviors

DHTML behaviors play an essential role in the implementation of a rich DataGrid control. As you'll see in a moment, I couldn't use Dave's behaviors in their original form. A few changes proved necessary to make those behaviors work in the context of an ASP.NET control. While no JavaScript skills are needed to use the modified components, a quick refresher of the DHTML behavior technology is in order for a better understanding of the mechanics that make client-side and server-side code work together.

A DHTML behavior is a script component that is bound to an HTML tag using a cascading style sheet (CSS) style—the "behavior" style. If your code is executed in an older browser that doesn't support CSS, or doesn't recognize the behavior style, the unknown style is simply ignored. For an in-depth look at DHTML, see Scripting Evolves to a More Powerful Technology: HTML Behaviors in Depth.

A DHTML behavior is a collection of JavaScript functions plus some public members defined using special syntax. Typically, public members include properties and events; sometimes they also include methods. Behaviors work on top of existing HTML elements and let you override and extend the behavior of HTML elements. To do so, a behavior attaches its own code to one or more DHTML standard events. For example, the behavior that provides column dragging handles the onmousedown and onmouseup events. Furthermore, all nontrivial DHTML behaviors can handle the oncontentready event, which fires when the HTML subtree (all the HTML in the particular element) has been completely parsed. The oncontentready event is a good time to initialize a behavior.

At their core, behaviors are COM objects that expose a few interfaces to Microsoft® Internet Explorer (version 5.0 and higher). However, you can write them as C++ binary components or as HTML Component (HTC) text files. HTC files can be deployed on the server alongside the files that use them (HTML, ASP, and ASP.NET), without any installation on the client.

The following code shows how to add column dragging capabilities to a <table> tag using the dragdrop.htc behavior:

<TABLE style="behavior:url(dragdrop.htc);" >
Here the file must be deployed on the server in the same directory as the file that uses it.

The Draggable DataGrid

After reading Dave Massy's column, I downloaded the sample dragdrop.htc component and tried to bind it to a DataGrid component in a sample page, like so:

theGrid.Style["behavior"] = "url(dragdrop.htc)";
Surprisingly, it didn't work. Armed with the certainty that on the client a DataGrid is nothing more than a table, I decided to compare the source code of Dave's sample table and the HTML source code of the ASP.NET DataGrid. I noticed that in ASP.NET 1.x, DataGrid-generated tables don't contain THEAD and TBODY elements. But these elements are essential for the sample behavior to work. THEAD and TBODY tags aren't strictly necessary to implement column drag and drop behavior, but they make it significantly easier to locate the table header and the table body.

You can either rewrite the behavior without THEAD and TBODY or write a custom DataGrid control that outputs a table with THEAD and TBODY tags. For an ASP.NET developer like me, I believed that it would be easier to write a custom control than edit a behavior. I thought that I'd at least have an effective debugger to step through the code. So I started a new Visual Studio® .NET solution and created a sample ASP.NET app and a Web control library project. The new DataGrid class had this prototype:

[ToolboxData("<{0}:DataGrid runat=\"server\" />")]
public class DataGrid : System.Web.UI.WebControls.DataGrid
{
   public DataGrid() : base()
   {
      EnableColumnDrag = true;
      DragColor = Color.Empty;
      HitColor = Color.Empty;
   }
   •••
}

The constructor initializes three public custom properties: EnableColumnDrag, DragColor, and HitColor. EnableColumnDrag is a Boolean property that enables column dragging. With this property set to False, the custom DataGrid doesn't add the drag behavior. The other two properties define the background color of the dragged column and the color of the column that's the target of the drop.

Note that these two color properties don't affect the logic of the DataGrid server control. They are simply server-side properties that are used to output values to HTML that will be consumed by the client-side behavior. The values of the two properties are rendered as custom attributes to the <table> tag generated for the grid. The markup code of the DataGrid is built in the Render method of the control listed in Figure 1.

Figure 1 Render Method (Close Figure 1)
protected override void Render(HtmlTextWriter output)
{
    // Sets attributes for the DragDrop behavior
    if (EnableColumnDrag)
    {
        if (DragColor != Color.Empty)
            Attributes["DragColor"] = DragColor.Name.ToLower();
        if (HitColor != Color.Empty)
            Attributes["HitColor"] = HitColor.Name.ToLower();

        // Capture the default output of the DataGrid
        StringWriter writer = new StringWriter();
        HtmlTextWriter buffer = new HtmlTextWriter(writer);
        base.Render(buffer);
        string gridMarkup = writer.ToString();

        // Parse the markup to insert missing tags

        // Find the first occurrence of > and insert <thead>
        int insertPoint;
        insertPoint = gridMarkup.IndexOf(">") + 1; // len of >
        gridMarkup = gridMarkup.Insert(insertPoint, "<thead>");
        insertPoint = gridMarkup.IndexOf("</tr>") + 5; // len of </tr>
        gridMarkup = gridMarkup.Insert(insertPoint, "</thead><tbody>");

        // Replace </table> with </tbody></table>
        gridMarkup = gridMarkup.Replace("</table>", "</tbody></table>");

        // Output the markup
        output.Write(gridMarkup);
        return;
    }

    base.Render(output);
    return;
}

The two behavior properties are added to the Attributes collection of the DataGrid to be rendered as in the following snippet:

<table dragcolor="..." hitcolor="..." ...>
The Render method generates a <table> element with THEAD and TBODY tags. There's just one way in which you can force this—capture the default HTML markup and parse it. You can capture the HTML code generated for a given control using the following code:
StringWriter writer = new StringWriter();
HtmlTextWriter buffer = new HtmlTextWriter(writer);
base.Render(buffer);
string gridMarkup = writer.ToString();

You create a new HtmlTextWriter object and bind it to the writer object providing the writing surface. In this case, the writing surface is represented by a block of memory, a StringWriter object. Everything that is sent to the HTML writer is actually accumulated in the string writer. The base Render method generates the standard markup of the DataGrid and you capture that to a string using the ToString method of the StringWriter. Simple and effective. At this point, parsing the string and adding THEAD and TBODY is child's play. The THEAD tag wraps the first table row, which typically contains the header of each column. The TBODY identifies the body of the table:

<table>
<thead>
   <tr><td>Column 1</td><td>Column 2</td></tr>
</thead>
<tbody>
   <tr><td>Some</td><td>Data</td></tr>
   <tr><td>More</td><td>Info</td></tr>
</tbody>
</table>

Finally, you write the modified markup to the response stream. Then you're ready for testing. Compile the source code to an assembly and register the new control with the Visual Studio .NET toolbox. Next, add the control to the sample page. This operation inserts the following code in the .aspx file:

<%@ Register TagPrefix="msdn" Namespace="Expoware.Controls"
   Assembly="MyCtl" %>
Using the custom DataGrid is identical to using the base DataGrid. The only thing that changes is the namespace prefix and, if need be, custom properties are added:
<msdn:datagrid runat="server" id="grid1" ...>
•••
</msdn:datagrid>

Figure 2 Dragging Columns
Figure 2 Dragging Columns

Figure 2 shows the sample page in action. If you grab the header of a column and drag it, a semi-transparent panel that represents the column is moved with the mouse. To improve the user's experience, any column underneath the mouse changes the background color of its header to reflect that it is a potential target for the drop. When you release the mouse button, the transported column is inserted before (to the left of) the column that's under the cursor. Finally, the table is restructured to reflect the new column order. All these operations exploit the client-side DHTML object model and require no round-trip to the server. The mouse events and the table rebuilding are governed by the DHTML behavior.

Enhancing the Dragdrop Behavior

Figure 3 provides a high-level view of the behavior's code, based on a few event handlers. The oncontentready event represents the entry point. The handler for this event initializes the state of the component and accesses public properties as defined in the attached element. In particular, the dragdrop.htc behavior builds an array of cell coordinates to be used later during the dragging and dropping to detect the underlying columns.

Figure 3 Dragdrop Pseudocode (Close Figure 3)
<public:property name='DragColor' />
<public:property name='HitColor' />

<public:attach event=oncontentready onevent="init();" />

<script>

// Declare some global variables

// Handles the OnContentReady event for the attached element (<table>)
function init()
{
    // Store references to THEAD and TBODY elements
    // Fill out global variables with information available at this time
    // Build an array of cell coordinates for hit testing
    // For each cell, attach an handler to the OnMouseDown event
    // Attach event handlers for the OnMouseMove and OnMouseUp events
}

function onMouseDown(e)
{
    // Retrieve the outermost TD tag so that drag starts wherever you
    // click in the cell
    // Create the panel to use for feedback during the drag and drop
    // operation
    // Configure the panel to be transparent, have the same width and
    // height of the real column,
    // support absolute positioning, and have a dashed border. The panel
    // is hidden at first
    // Insert the panel before the cell clicked
}

function onMouseMove(e)
{
    // Save the current mouse position
    // Turn visibility on for the panel
    // Figure out the underlying cell and refresh its visual settings
    // Stop bubbling of the event
}

function onMouseUp(e)
{
    // Figure out the cell target of the drop
    // Remove the dynamically created panel from the DHTML tree
    // Update the array of cell coordinates for hit testing
    // Rebuild header and body of the table
}

// Some more helper functions

</script>

The width of each cell is tracked using the DHTML ClientWidth property. This is one of the changes that I made to the original source code from Dave's column. The ClientWidth property retrieves the width of the DHTML object including padding, but not including margin, border, or scroll bar. The use of this property is a key enhancement as it allows you to support columns with no explicit width set at the HTML source level. Since I intend to apply the behavior to a custom DataGrid control, the use of ClientWidth is essential to supporting the AutoGenerateColumns mode of ASP.NET DataGrids.

The dragdrop.htc file in this month's code download also contains other minor modifications, mostly due to personal preferences. One example is the style of the feedback control displayed during the drag and drop operation.

The custom DataGrid control sets the behavior style when the EnableColumnDrag property is set. The property stores its value in the view state. If the property is set to True, the behavior attribute is added to the Style object of the DataGrid control. Note that multiple behaviors can be bound to the same element.

The EnableColumnDrag property is tightly coupled to ShowHeader—the standard Boolean property that turns the header of the grid on and off. If the DataGrid doesn't display a header, the column dragging is silently disabled. Figure 4 shows the get and set accessors of both properties. Note that you need to override the set accessor of the ShowHeader property to ensure that EnableColumnDrag is set to False if the ShowHeader property is False.

Figure 4 Get and Set (Close Figure 4)
public bool EnableColumnDrag
{
    get {return Convert.ToBoolean(ViewState["EnableColumnDrag"]);}
    set {
        if (!ShowHeader)
        {
            ViewState["EnableColumnDrag"] = false;
            return;
        }
        ViewState["EnableColumnDrag"] = value;
        string behavior = Style["behavior"];
        if (behavior == null)
            behavior = "";
        behavior = behavior.Replace("url(dragdrop.htc)", "");
        if (value)
            behavior += " url(dragdrop.htc)";
        Style["behavior"] = behavior;
    }
}

public override bool ShowHeader
{
    get {return base.ShowHeader;}
    set {
        base.ShowHeader = value;
        if (!ShowHeader)
            EnableColumnDrag = false;
    }
}

So far so good. You can now drag and drop columns when the DataGrid is displayed on the client. However, the original column order is restored as soon as the page or the DataGrid posts back. Is there something you can do to persist the new order?

Persisting the New Column Order

The new column order on the client is a piece of information that must be passed to the server when the page posts back. Exchange of information between the client- and server-side environments is nothing special in the ASP.NET programming model. Communicating the new column order is like passing to the server the text typed in a textbox or the selected item in a dropdown list. You pass the desired sequence of columns to the server and force the DataGrid control to render the columns accordingly. Let's tackle information exchange first.

Using HTTP and HTML today, there's only one way to carry client-side information to the server—using a hidden field. The DataGrid control adds a hidden field to the page; the DHTML behavior figures out the new sequence of columns and writes it out to the field. When the page posts back, the DataGrid retrieves the desired sequence from the hidden field and renders itself accordingly. You can choose any name for the hidden field as long as it's unique. You are also responsible for retrieving the posted value. The desired order of columns can be expressed with a comma-separated string that concatenates the header of each column. Note that this approach is arbitrary but, in any case, you must return a string. You retrieve the client value using Page.Request:

string desiredOrder = Page.Request[HiddenFieldName].ToString();

This solution solves the problem and allows you to have a persistent column drag feature. Although functionally correct, this is not the best approach in ASP.NET. Have you ever run across the IPostBackDataHandler interface? Well, that interface comes in handy when you have a control that imports data from the client. Figure 5 shows the implementation of the IPostBackDataHandler interface in the custom DataGrid control. Basically, the methods of the interface allow you to automatically bind the data coming through a hidden field with one or more properties of the control. You don't have to call Page.Request on your own and you don't have to worry much about the hidden field. ASP.NET takes care of most of the plumbing.

Figure 5 Implementing IPostBackDataHandler (Close Figure 5)
public class DataGrid : System.Web.UI.WebControls.DataGrid, IPostBackDataHandler
{
    public DataGrid() : base()
    {
        EnableColumnDrag = true;
        DragColor = Color.Empty;
        HitColor = Color.Empty;
        ColumnOrder = "";
    }

    // *************************************************************
    // IPostBackDataHandler::LoadPostData
    // Automatically updates the ColumnOrder property based on the
    // content of the hidden field named as this control
    public virtual bool LoadPostData(string key, NameValueCollection
                                     values)
    {
        string currentColumnOrder = ColumnOrder;
        string postedColumnOrder = Convert.ToString(values[key]);

        // Discard new values if empty
        if (postedColumnOrder == "")
            return false;

        // New value posted?
        if (!currentColumnOrder.Equals(postedColumnOrder))
        {
            ColumnOrder = postedColumnOrder;
            return true;
        }

        return false;
    }
    // *************************************************************

    // *************************************************************
    // IPostBackDataHandler::RaisePostDataChangedEvent
    public virtual void RaisePostDataChangedEvent()
    {
        // Do nothing here
        // No need of firing server-side events
    }
    // *************************************************************

    // *************************************************************
    // Return the name of the hidden field
    private string HiddenFieldForDragging
    {
        get {return ID;}
    }
    // *************************************************************

    // *************************************************************
    // Stores the comma-separated string indicating the column order
    private string ColumnOrder
    {
        get {return Convert.ToString(ViewState["ColumnOrder"]);}
        set {ViewState["ColumnOrder"] = value;}
    }
    // *************************************************************

    •••
}

The interface lists two methods, only one of which is used here: LoadPostData, which is passed a key and the collection of posted values. The key is simply the ID of the control; the collection is a subset of Page.Request. The collection contains all the posted input fields that match existing controls in the page. There's just one case in which the collection doesn't match Page.Request—when the page contains dynamically created controls.

The LoadPostData method is invoked after the OnInit event but before the OnLoad event. ASP.NET calls the LoadPostData method of a control that implements the interface only if the ID of the control matches one of the input fields. For this reason, you should give the input field the same ID as the grid. The DataGrid only works on posted data, and has no need to send server-side information to the client. For this reason, you can create the hidden field with an empty string, like this:

Page.RegisterHiddenField(ID, "");

The LoadPostData method refreshes the value of a new property named ColumnOrder. This property will be used to determine the order of the columns in the grid. It is important to note that if no column dragging occurs between two consecutive postbacks, the ColumnOrder property is empty. To persist the previously set order you must persist the value of ColumnOrder—for example, in the view state. In addition, you should never overwrite the property with an empty posted value.

The second method of the IPostBackDataHandler interface gives you a chance to raise a server-side event when posted values affect the state of the control. Getting the string with the column order set on the client is only half the task. Now you have to tell the Data-Grid so that the new page presents the columns in the right order.

The DataGrid Whisperer

Have you ever tried to whisper to a DataGrid about rendering? A friend once called me "the DataGrid whisperer." He said it was for my ability to force the darned control to behave any way I want it to behave. However, I must confess that forcing a DataGrid to change the fixed order of columns proved quite difficult even for someone with lots of experience.

I spent a few hours trying to reorder the Columns collection of the DataGrid from within the OnLoad handler. Then I realized that the collection is empty when columns are autogenerated. Plus, any changes entered are lost when the control renders its HTML. Just before I gave up, I noticed the CreateColumnSet protected virtual method:

protected virtual ArrayList CreateColumnSet(
   PagedDataSource dataSource,
   bool useDataSource
);

The method is considered part of the Microsoft .NET Framework and left undocumented because it is not intended to be used directly from user code. Personally, I believe that it's hard to keep it from being used this way when you're talking about a protected virtual (overridable) method. So I went on and wrote an override for that method. I didn't know exactly what the method was doing, but what else could a method named CreateColumnSet do but create all the columns needed in a DataGrid? My first approach was a cautious one:

protected override ArrayList CreateColumnSet(
   PagedDataSource dataSource, bool useDataSource)
{
   ArrayList a;
   a = base.CreateColumnSet(dataSource, useDataSource);
   return a;
}

I put in a breakpoint and ran the code. The ArrayList returned by the base method contains DataGridColumn objects and any change entered at this point was reflected in the HTML generated by the grid. The returned array is correct and up to date whatever setting AutoGenerateColumns has. Figure 6 shows the final version of the method that overrides CreateColumnSet. First, you retrieve the column set and then order it based on the ColumnOrder property. Sorting an array of objects requires a custom comparer class like the one in Figure 6. The comparer class compares two DataGridColumn objects on the value of their HeaderText property.

Figure 6 Overriding CreateColumnSet (Close Figure 6)
protected override ArrayList CreateColumnSet(PagedDataSource dataSource,
    bool useDataSource)
{
    // Let the grid generate the base column set (take the
    // AutoGenerateColumns property into account)
    ArrayList a = base.CreateColumnSet(dataSource, useDataSource);

    // If column dragging is disabled or reordering unnecessary
    if (ColumnOrder == "" || !EnableColumnDrag)
        return a;

        // Apply the desired sequence of columns
    if (ColumnOrder != "")
      {
        // Reorder the column set to reflect the client-side changes
        IComparer myComparer = new ColumnComparer(ColumnOrder);
        a.Sort(myComparer);
    }

    return a;
}

public class ColumnComparer : IComparer
{
    private string[] _columnsOrder;

    public ColumnComparer(string order)
    {
        _columnsOrder = order.Split(',');
    }

    int IComparer.Compare(object x, object y)
    {
        DataGridColumn dgc1 = (DataGridColumn) x;
        int indexOf1 = Array.IndexOf(_columnsOrder, dgc1.HeaderText);

        DataGridColumn dgc2 = (DataGridColumn) y;
        int indexOf2 = Array.IndexOf(_columnsOrder, dgc2.HeaderText);

        if (indexOf1 < indexOf2)
            return -1;
        if (indexOf1 == indexOf2)
            return 0;
        else
            return 1;
    }
}

The DHTML behavior is essential to the persistence mechanism. It is responsible for storing the new order in the hidden field when a drag completes (see Figure 7). The JavaScript does the work.

Figure 7 Write to Hidden Field (Close Figure 7)
var buf = "";
for(var i=0; i<headRow.children.length; i++) {
   buf = headRow.children[i].innerText;
   if (i < headRow.children.length-1)
      buf += ",";
}

// Move up the hierarchy until the FORM is found
var obj = element.parentElement;
while (obj.tagName != "FORM") {
   obj = obj.parentElement;
   if (obj == null) return;
}

// Write to the hidden field
var hiddenField = element.id;
if (hiddenField != null)
   obj[hiddenField].value = buf;

What About Paging?

Proud of my creation, I passed it on to colleagues for feedback. It didn't take long to realize that something bad happens if the DataGrid contains a pager or footer row. Why? Because the pager has a different layout and some null values creep into the JavaScript code, making it fail. Wrapping pager and footer in a TFOOT block does the trick. The DHTML behavior moves only the header row and the rows in the body. Anything in the footer is left unchanged.

Unfortunately, in ASP.NET you can place the pager on top of the DataGrid as well. This setting clashes with the THEAD block because the pager is displayed as the first row of the table. If you don't place the topmost pager in the THEAD, it will be displayed as the second row in the table and treated as a body row. As a result, you'll receive the same runtime errors, just fixed with the TFOOT tag. On the other hand, if you place the topmost pager in the THEAD you have to edit the DHTML behavior so that it operates on the real header row and not just the first one found. In other words, you have to force the Render method of the DataGrid to generate the markup according to the following schema:

<table>
<thead>
   <tr>pager</tr>
   <tr><td>Column 1</td><td>Column 2</td></tr>
</thead>
<tbody>
   <tr><td>Some</td><td>Data</td></tr>
   <tr><td>More</td><td>Info</td></tr>
</tbody>
<tfoot><tr>footer</tr><tr>pager</tr></tfoot>
</table>

The behavior will need to check how many children the THEAD property has on the client and pick the second row if the original grid contains a topmost pager. This information is passed on to the client using a new attribute—HasTopMostPager.

<public:property name='HasTopMostPager' />
if (HasTopMostPager == "true")
   headRow = element.thead.children[1];
else
   headRow = element.thead.children[0];

The final version of the Render method is too long to show here. You can check it out by downloading the source code.

As a final note about DataGrid drag and drop, remember that you can use any standard column, including templated columns.

What About Sorting?

If the DataGrid supports sorting, the header of a sortable column isn't plain text but is made of HTML elements, typically an anchor tag. The DHTML behavior you get with this column knows how to get the text inside. It takes the inner text of the clicked cell, automatically removing any HTML formatting.

Now let's see how you can build some client-side sorting capabilities. Once again, Dave Massy's column proves useful and provides another powerful component—the sort.htc behavior. The behavior intercepts any clicking on the header of a table and sorts the table by the values found in the column. The behavior tracks the last clicked column, and if you click the column again, the order is reversed. In addition, a glyph is added alongside the header of the column to indicate the sorting column and its direction. Figure 8 shows a DataGrid that utilizes this behavior. The glyph is a "3" in Wingdings font inserted through a dynamically created SPAN tag.

Figure 8 Sortable DataGrid
Figure 8 Sortable DataGrid

The sort.htc behavior captures the OnContentReady event, does some initialization work, and attaches a handler to the OnClick event for each cell in the table's header. When the grid is set up on the client, all cells have an empty glyph and the order is the default one. When the user clicks to sort by a particular column, the textual content of the column is sorted. Note that the client-side sorting is purely text based. If you want column-based sorting, use the default server-side sorting mechanism that the DataGrid control provides. Only the currently displayed set of records are sorted.

The sort.htc behavior is controlled on the server by a new property named EnableClientSort, which is nearly identical to EnableColumnDrag and enables sorting by adding the sort.htc URL to the behavior style. The two behaviors work together easily.

If you're content with client-only sorting, you're pretty much finished. The behavior sorts the contents of the table and the sort is lost with the next postback. Client- and server-side sorting can be supported in the same grid. When the user clicks on a server-sortable column (see the ID column in Figure 8), the page posts back and fires a server event to the DataGrid. When the user clicks on any other columns, the sorting takes place on the client. This version of the sort.htc component supports sorting on all columns. You can improve the component to make it accept sorting on only a subset of columns.

Is there a way to make client-side sorting persistent? You bet. You create a second hidden field and have the sort.htc component put the expression to use on the server to sort data into that hidden field. At first, this looks like a good idea, but it turns out not to be.

The problem is that on the client you have no information at all about the data columns that generated the data on the screen. The only information the behavior can store to the hidden field is the index or the header text of the clicked column and a flag to indicate the direction (ascending or descending) of the sorting.

On the server, this information is difficult to make sense of. The first thing I tried to do was sort the data source in a text-based way. The data source bound to the DataGrid can be any object that implements the IEnumerable interface. Unless you want to limit yourself to a few common ADO.NET objects, this is the object to work with. However, the big challenge lies in the way you perform the sort. Consider that to offer users the same order of rows they produced on the client, you must sort all data objects in the data source as text. But the data source is a collection of strongly typed objects—dates, strings, numbers with possible formatting options. To be sure you get exactly the same order of the client, you should first transform any object into its markup counterpart.

An approach that is worth exploring is sorting in the Render method immediately after having generated the HTML rows. I haven't tried this but I bet that with a little help from regular expressions and a custom comparer class, you can sort an HTML string that represents a table.

To solve the problem, I've chosen another approach. I use the hidden field to track the sort expression in use on the client the last time the user worked with the page. This information is not used on the server. On the client, the behavior reads the content of the hidden field and reinitializes the table accordingly. The following lines in the Render method make this possible:

if (EnableClientSort)
{
   string buf = "";
   buf = Page.Request[HiddenFieldForSorting].ToString();
   Page.RegisterHiddenField(HiddenFieldForSorting, buf);
}

When the behavior is initialized, the table may be partially rendered on the client but with the default sorting. In the initialization step, the behavior changes the structure of the table and refreshes. But this sequence of operations produces a nasty flickering effect. To remedy this, wrap the table created for the DataGrid in a <div> tag and set its visibility to hidden. So when the page is first displayed, the browser saves the space for the table but doesn't display it. When the behavior has completed its sorting, it retrieves the <div> in the DHTML object model and turns the visibility on. The behavior identifies the <div> through a well-known ID.

Summing It Up

A symbiotic relationship between the DataGrid and DHTML can be attained and has lots of advantages for users. But as you can see, it takes a bit of work to coerce the DataGrid into doing everything you want it to do. If you think this little exercise will be useful to you, download the source code and let me know the outcome.

Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is an instructor and consultant based in Rome, Italy. Author of Programming Microsoft ASP.NET (Microsoft Press, 2003) he spends most of his time teaching classes on ADO.NET and ASP.NET and speaking at conferences. Get in touch with Dino at cutting@microsoft.com.


The XML Files: IntelliSense for XSLT, Namespace Collisions, and More

The XML Files
IntelliSense for XSLT, Namespace Collisions, and More
Aaron Skonnard

Q Is there some way that I can enable IntelliSense® for my XSLT files in Visual Studio® .NET?

A There are at least two ways that you can get IntelliSense for XSLT in Visual Studio .NET. One way is to buy a third-party product like the ActiveState Visual XSLT plug-in, and the other is to take advantage of the generic XML IntelliSense in Visual Studio .NET, which is driven by XML Schema definitions. The former option requires spending a little cash while the latter choice simply requires finding an XML Schema definition for XSLT.

Unfortunately, the authors of the XSLT specification didn't include an official XML Schema definition when they first authored the language, so don't bother looking around the W3C site. There are, however, a handful of different XSLT schemas that have been written over the years for personal or company use. The best one I've found is at http://www.fesersoft.com/dotNet.

Visual Studio .NET provides XML IntelliSense for all of the XML Schemas that it knows about. Visual Studio .NET automatically knows about the XML Schema definitions found within a given project as well as a set of global XML Schema definitions found in the C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Packages\schemas\xml directory. You'll notice that this directory already contains several schema files including the schema for XML Schema.

Once you've obtained the XSLT schema you want to use for IntelliSense, simply add it to your project or drop it in the C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Packages\schemas\xml directory and you're ready to go. To test IntelliSense, create a new XSLT file by selecting New | File | XSLT File from the File menu. You should see a template XSLT file that contains an empty stylesheet element from the http://www.w3.org/1999/XSL/Transform namespace. Put your cursor between the stylesheet start and end tags, type "<", and you should see IntelliSense kick in, listing information about the allowed XSLT elements.

Visual Studio .NET uses the element's XML namespace to determine which schema to use. For example, if you create a new XSLT file and use xsl:transform as the root element (instead of xsl:stylesheet), IntelliSense will work identically as long as you specify the same namespace again.

Visual Studio .NET can even assist you in choosing the correct namespace by providing IntelliSense for all of the namespace names it knows about within a given project or the global schema directory. Notice that Visual Studio .NET now includes the XSLT namespace in the list since I dropped it in the global schema directory (see Figure 1).

Figure 1 XSLT IntelliSense
Figure 1 XSLT IntelliSense

These techniques work with any XML Schema definition. If you have some custom schemas that you commonly author against, then you might want to take advantage of this somewhat underutilized feature in Visual Studio .NET.

Q Can you customize the XSLT template used when creating new XSLT files in Visual Studio .NET?

A Yes, there are several New File templates found in the Visual Studio .NET installation directory. There are some global templates for creating new files from the File menu found in C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\IDE\NewFileItems and there are also project-specific templates found in some of the different language subdirectories (see C:\Program Files\Microsoft Visual Studio .NET2003\VC#\CSharpProjectItems for C#-specific templates).

The template for new XSLT files is called XSLTFile.xslt. Simply modify this file to fit your preferences. If you're like me and prefer to use transform (instead of stylesheet) and the "xsl" prefix (instead of a default namespace declaration), you can change the default XSLT templates to look like the following document:

<?xml version="1.0" encoding="UTF-8" ?>
<xsl:transform version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
>

</xsl:transform>
With this in place, every new XSLT file you create in Visual Studio .NET will look like this template.

Q I need to write an XSLT transformation that generates another XSLT document as output and I seem to be running into namespace problems. What's the best way to handle this?

A Trying to output XSLT elements from within an XSLT transformation is problematic because you run into a namespace conflict that confuses the processor. For example, consider the following XSLT fragment that's attempting to output an xsl:transform element from within the root template:

<xsl:transform version="1.0"
     xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:template match="/">
        <!-- the transform element I'm trying to output -->
        <!-- THIS DOESN'T WORK -->
        <xsl:transform version="1.0">
            <xsl:template match="/">
               •••
            </xsl:template>
        </xsl:transform>
    </xsl:template>
</xsl:transform>

The problem is that the transform element that I'm trying to output is in the same namespace as the "real" transform element that the XSLT processor is designed to understand as representing the start of the program. So when the processor gets to the nested xsl:transform element (meant for output), it thinks it has found another program construct.

There are two ways to get around this problem. The first is to use the xsl:element and xsl:attribute elements to dynamically generate the XSLT elements, as illustrated in Figure 2.

Figure 2 Dynamically Generate XSLT (Close Figure 2)
<xsl:transform version="1.0"
     xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:template match="/">
        <!-- the transform element I'm trying to output -->
        <xsl:element name="transform" namespace="http://www.w3.org/1999/
          XSL/Transform">
           <xsl:attribute name="version">1.0</xsl:attribute>
           <xsl:element name="template namespace=" http://www.w3.org/
             1999/XSL/Transform">
              <xsl:attribute name="match">/</xsl:attribute>
              •••
           </xsl:element>
        </xsl:element>
    </xsl:template>
</xsl:transform>

Now I don't have a namespace conflict because I'm dynamically generating the XSLT elements (as opposed to writing them into the template). But as you can see, this approach is tedious, incredibly hard to read, and even harder to test and validate.

The other approach XSLT provides for dealing with this problem is called namespace-aliasing. XSLT has an element named xsl:namespace-alias that essentially tricks the processor into believing that the XSLT elements I want to output are in a different namespace than the real XSLT constructs that make up the program. The syntax for xsl:namespace-alias is as follows:

<xsl:namespace-alias
    stylesheet-prefix ="alias namespace prefix"
    result-prefix ="real namespace prefix"/>

You use this element to specify which namespace prefix is mapped to the "real" namespace (using the result-prefix attribute) and which one is the temporary alias (using the stylesheet-prefix attribute). For example, the following transformation indicates to the processor that the "alias" prefix is the alias used in the transformation. Notice that it's mapped to a different namespace than that of XSLT (I just tacked "alias" onto the end of the XSLT namespace). Now I can simply write the XSLT transformation I'm trying to produce using the "alias" prefix, which makes the transformation much easier to author and read, as you can see in Figure 3.

Figure 3 Using the Alias Prefix (Close Figure 3)
<xsl:transform version="1.0"
     xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
     xmlns:alias="http://www.w3.org/1999/XSL/TransformAlias">
    <xsl:namespace-alias
         stylesheet-prefix="alias" result-prefix="xsl"/>
    <xsl:param name="column">name</xsl:param>
    <xsl:template match="/">
        <alias:transform version="1.0">
            •••
            <alias:template match="/">
                <results>
                <alias:for-each select="/rowset/row">
                    <alias:sort select="{$column}"/>
                    <alias:apply-templates/>
                </alias:for-each>
                </results>
            </alias:template>
        </alias:transform>
    </xsl:template>
</xsl:transform>

Then, when the XSLT processor executes the transformation and produces the result document, it replaces the namespace name for the "alias" prefix with the one mapped to the result-prefix (in this case "xsl"). Hence, the output document for Figure 3 will contain the correct XSLT namespace, as you can see here:

<alias:transform version="1.0"
      xmlns:alias="http://www.w3.org/1999/XSL/Transform">
    •••
    <alias:template match="/">
        <results>
        <alias:for-each select="/rowset/row">
            <alias:sort select="name"/>
            <alias:apply-templates/>
        </alias:for-each>
        </results>
    </alias:template>
</alias:transform>
This technique is especially useful if you have large XSLT libraries that have to be maintained and upgraded to newer versions every once in a while.

Q Can you explain how built-in numbering works in XSLT?

A The built-in numbering functionality is one of XSLT's best-kept secrets, which is unfortunate because it can simplify complex numbering tasks used by many transformations. XSLT provides an element named xsl:number that makes it possible to insert a formatted number into the result document. You can influence how the number is generated and formatted in the result document using the element's various attributes shown here:

<xsl:number
  level = "single" | "multiple" | "any"
  count = pattern
  from = pattern
  value = number-expression
  format = { string }
  lang = { nmtoken }
  letter-value = { "alphabetic" | "traditional" }
  grouping-separator = { char }
  grouping-size = { number } />
The xsl:number element first evaluates the expression provided in the value attribute and the result object is then converted to a number as if by calling the XPath number function. The number is then rounded to an integer and formatted according to the last five attributes shown (format, lang, letter-value, grouping-separator, and grouping-size). See Figure 4 for descriptions of all xsl:number attributes.

Figure 4 xsl:number Attributes (Close Figure 4)

Name | Default | Description
level | "Single" | Specifies which levels of the source tree should be considered during the counting process.
count | The pattern that matches the current node's type and name (if any) | A pattern that specifies which nodes should be counted at the specified levels (see the level attribute).
from | / | A pattern that specifies where counting starts.
value | Position (based on other attributes) | An XPath expression whose result is coerced to a number. If not specified, the value is evaluated as the current node's position with respect to the level, count, and from attributes.
format | 1 | Specifies the format to use for numbering.
lang | Determined from system environment | Specifies which language's alphabet to use when numbering with an alphabetic sequence (same value space as xml:lang).
letter-value | (None) | Disambiguates between different numbering schemes for a given language. In English, the "a" format token identifies an alphabetical sequence (such as a, b, c, ..., aa, ab, ac, ...) while the "i" format token identifies a numeric-alphabetic sequence (for example, i, ii, iii, iv,...). In some languages both numbering schemes start with the same letter. In these cases, a letter-value of "alphabetic" identifies the alphabetical sequence while a letter-value of "traditional" identifies the other form.
grouping-separator | (None) | Specifies the separator used as a grouping separator in decimal numbering sequences (such as 1,000,000).
grouping-size | (None) | Specifies the size of the grouping (such as 3 in the grouping-separator example).

For example, consider the following XSLT statement:

<xsl:number value="5000000 div 3"
   grouping-separator="," grouping-size="3" />
This expression displays the following value in the result tree:
1,666,667
It's more common, however, to omit the value attribute and use xsl:number for automatic counting. When you don't specify the value attribute, xsl:number calculates and uses the position of the current node as the value. The level, count, and from attributes influence how the position of the current node is calculated. The from attribute takes a pattern that identifies where the counting starts in the tree while the count attribute takes a pattern that identifies exactly which nodes should be counted. The level attribute specifies which levels of the source tree should be considered during the counting process.

There are three possible level values: "single," "any," and "multiple." A value of "single" counts the siblings that precede the target node (same as XPath preceding-sibling axis). If the current node matches the count pattern or count is not specified, it is the target node. Otherwise, the target node is the first ancestor to match the count pattern. The first ancestor node (of the current node) that matches the from pattern identifies the start node where counting begins. Only those nodes that come after the start node up to the current node are considered in the count.

A value of "any" counts the nodes, identified by the count pattern, that come before the current node in document order (same as the union of XPath's preceding and ancestor-or-self axes). The first node (that comes before the current node) that matches the from pattern identifies the start node where counting begins. Only those nodes that come after the start node up to the current node are considered in the count.

A value of "multiple" produces a sequence of numbers, each of which is produced in the same way as a single level count (such as 1.1.1, 1.1.2, 1.2.1, and so on). The count of the outermost ancestor that matches the count pattern becomes the first number in the sequence. The count of the next outermost ancestor that matches the count pattern becomes the second number in the sequence, and so on. The count of the innermost ancestor that matches the count pattern becomes the last number in the sequence. The count of each ancestor is determined like a single level count.

Let's take a look at some examples. Consider the following XML document that contains a sample book structure:

<books>
  <book title="Sample Book">
    <chapter title="Sample Chapter">
      <section title="Sample Section">
        <p>This is some sample text.</p>
          •••
In this particular schema, the books element may contain several book elements, each book element may contain multiple chapter elements, and each chapter element may contain multiple section elements. The following transformation uses the default xsl:number functionality (level="single") to number each of the books, chapters, and sections:
<xsl:transform version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method="text"/>

  <xsl:template match="book">
    <xsl:number/>. <xsl:value-of select="@title"/>
    <xsl:apply-templates/>
  </xsl:template>

  •••
</xsl:transform>

This transformation generates the text shown in Figure 5 (with white space added to improve readability). Notice that the book, chapter, and section counts are kept independently. Now consider the transformation in Figure 6, which uses level="any."

Figure 6 Transform Using Any (Close Figure 6)
<xsl:transform version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method="text"/>

  <xsl:template match="book">
    <xsl:number level="any"/>. <xsl:value-of select="@title"/>
    <xsl:apply-templates/>
  </xsl:template>

  <xsl:template match="chapter">
    <xsl:number level="any"/>. <xsl:value-of select="@title"/>
    <xsl:apply-templates/>
•••
Figure 5 Independent Numbering (Close Figure 5)
1. Sample Book
   1. Sample Chapter
      1. Sample Section
      2. Sample Section
      3. Sample Section
   2. Sample Chapter
      1. Sample Section
      •••
2. Sample Book
   1. Sample Chapter
      1. Sample Section
      2. Sample Section
      3. Sample Section
   2. Sample Chapter
      1. Sample Section
•••

With "any," the chapter numbering is no longer scoped by the containing book, and the section numbering is no longer scoped by the containing chapter. Now the chapters and sections are numbered from start to finish in a continuous fashion throughout the entire document as shown in Figure 7.

Figure 7 Continuous Numbering (Close Figure 7)
1. Sample Book
   1. Sample Chapter
      1. Sample Section
      2. Sample Section
      3. Sample Section
   2. Sample Chapter
      4. Sample Section
      5. Sample Section
      6. Sample Section
   3. Sample Chapter
      7. Sample Section
      8. Sample Section
      9. Sample Section
2. Sample Book
   4. Sample Chapter
      10. Sample Section
      11. Sample Section
      12. Sample Section
   5. Sample Chapter
      13. Sample Section
•••

If you want each numbered item to include ancestor-level numbers (such as 1.2.1), you use the "multiple" level value. With "multiple," you specify a pattern that indicates which nodes you want to count from the ancestors, as shown here:

<xsl:template match="book">
    <xsl:number level="multiple"/>. <xsl:value-of select="@title"/>
    <xsl:apply-templates/>
  </xsl:template>

  <xsl:template match="chapter">
    <xsl:number level="multiple" count="chapter|book"/>.
    <xsl:value-of select="@title"/>
    <xsl:apply-templates/>
  </xsl:template>
Notice that you don't have to do anything special for the "book" element because it's the outermost element as far as the counting goes. With the "chapter" element, however, you have to specify that you want to count both "book" and "chapter" elements, and with the "section" element you have to specify that you want to count the "section," "chapter," and "book" elements. This example produces the numbered list shown in Figure 8.

Figure 8 Multiple Levels (Close Figure 8)
1. Sample Book
   1.1. Sample Chapter
      1.1.1. Sample Section
      1.1.2. Sample Section
      1.1.3. Sample Section
   1.2. Sample Chapter
      1.2.1. Sample Section
      1.2.2. Sample Section
      1.2.3. Sample Section
   1.3. Sample Chapter
      1.3.1. Sample Section
      1.3.2. Sample Section
      1.3.3. Sample Section
2. Sample Book
   2.1. Sample Chapter
      2.1.1. Sample Section
      2.1.2. Sample Section
      2.1.3. Sample Section
   2.2. Sample Chapter
      2.2.1. Sample Section
•••

You can also use the format attribute to control how the numbers are formatted when using these techniques. I hardcoded the period to follow the generated number in the previous examples, but I could have also accomplished this by using the format attribute as follows:

   <xsl:template match="book">
     <xsl:number level="multiple" format="1. "/>
              <xsl:value-of select="title"/>
              <xsl:apply-templates/>
            </xsl:template>

You may also want to use Roman numerals (such as I, II, III, IV, and so on) or alpha characters (such as A, B, C, D) in your numbering scheme. This is made possible by specifying a special format token in the format attribute. Figure 9 provides some examples of format tokens and what the numbering sequence would be like for each one. For example, the transformation in Figure 10 uses Roman numerals for book numbering, alpha characters for chapter numbering, and digits for the section numbering.

Figure 10 Scheme with Several Numbering Systems (Close Figure 10)
<xsl:transform version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output method="text"/>

  <xsl:template match="book">
    <xsl:number level="multiple" format="I. "/>
    <xsl:value-of select="@title"/>
    <xsl:apply-templates/>
  </xsl:template>

  <xsl:template match="chapter">
    <xsl:number level="multiple" count="chapter|book" format="I.A. "/>
    <xsl:value-of select="@title"/>
    <xsl:apply-templates/>
  </xsl:template>

  <xsl:template match="section">
    <xsl:number level="multiple" count="section|chapter|book" format="I.A.1. "/><xsl:value-of select="@title"/>
  </xsl:template>
  •••
</xsl:transform>
Figure 9 xsl:number Format Token Examples (Close Figure 9)

Token | Sequence
1 | 1, 2, 3, 4, ..., 10, 11, 12, ..., 20, 21, ...
01 | 01, 02, 03, 04, ..., 10, 11, 12, ..., 100, 101, ...
A | A, B, C, D, ..., Z, AA, AB, AC, ...
a | a, b, c, d, ..., z, aa, ab, ac, ...
I | I, II, III, IV, V, VI, ..., X, XI, ...
i | i, ii, iii, iv, v, vi, ..., x, xi, ...

This transformation produces the same numbered list as before but with different level formatting:

I. Sample Book
   I.A. Sample Chapter
      I.A.1.Sample Section
      I.A.2.Sample Section
      I.A.3.Sample Section
   I.B. Sample Chapter
      I.B.1.Sample Section
      •••
II. Sample Book
   II.A. Sample Chapter
      II.A.1.Sample Section
      II.A.2.Sample Section
•••

As you can see, the numbering functionality provided by XSLT can be a powerful and flexible addition to your transformations as long as you understand how it works.

Q Can you explain the rules for white space in XSLT? It's not clear to me when white space is significant and when it's not.

A Before an XSLT processor executes a transformation against a given source document, it first strips all white space-only text nodes from both documents. A white space-only text node is a text node that contains only space, tab, and carriage return characters.

Developers often trip over this behavior when they need to explicitly output white space in the result tree because they usually first try typing the white space directly into their transformations. However, unless the text node also contains non-white space characters, it's going to be stripped out before the transformation runs. For example, consider the following transformation that attempts to output a space character between the two xsl:value-of statements (so the first and last names are spaced in the output tree):

<h1><xsl:value-of select="//first"/> <xsl:value-of select="//last"/>
</h1>

The text node containing the space is an example of a white space-only text node that will be stripped by the XSLT processor before executing and, as a result, it won't appear in the rendered HTML document. Now consider the text node between the xsl:value-of expressions for the middle-initial and last elements shown here:

<h1><xsl:value-of select="//first"/> <xsl:value-of select="//middle-
  initial"/>. <xsl:value-of select="//last"/> </h1>
Since this text node contains a non-white space character (the period), it will be preserved by the XSLT processor.

Luckily XSLT provides a special element, xsl:text, that's able to get around this issue. The xsl:text element explicitly preserves white space in text nodes. The following example preserves the space between the two xsl:value-of elements:

<h1><xsl:value-of select="//first"/><xsl:text> </xsl:text>
<xsl:value-of select="//last"/></h1>

Although XSLT also strips white-space-only text nodes from the source document by default, it does provide a way to influence this behavior. You can use xsl:preserve-space to specify which elements in the source document should not be subject to white space stripping. For example, the following transformation specifies that white space should not be stripped from the story, poem, and song elements in the source document:

<xsl:transform version="1.0"
   Xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
   <xsl:preserve-space elements="story poem song"/>
   •••
</xsl:transform>

The elements attribute takes a white space-separated list of name tests that can include QNames and "*". XSLT also provides xsl:strip-space that can be used with xsl:preserve-space when you primarily want to preserve white space in the source document with a few exceptions. Consider the following example:

<xsl:transform version="1.0"
   Xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

   <!-- preserves white space for all elements in source
        document -->
   <xsl:preserve-space elements="*"/>
   <xsl:strip-space elements="author title"/>
   •••
</xsl:transform>
In this case, the transformation only strips white space-only text nodes within the author and title elements and preserves white space everywhere else.

In addition to these XSLT techniques, XML 1.0 also provides a way for authors to request white space preservation in the context of a particular element. This is accomplished through the global xml:space attribute that's used like this:

<!-- some source document -->
<foo xml:space="preserve"> ... </foo>
XSLT processors are required to respect this attribute in source documents, allowing the author of the source document (as opposed to the author of the XSLT document) to indicate whether white space preservation is important.

Send your questions and comments for Aaron to xmlfiles@microsoft.com.

Aaron Skonnard teaches at Northface University in Salt Lake City. Aaron coauthored Essential XML Quick Reference (Addison-Wesley, 2001) and Essential XML (Addison-Wesley, 2000), and frequently speaks at conferences. Reach him at http://www.skonnard.com.


Advanced Basics: Windows Forms Controls: Z-order and Copying Collections

Advanced Basics
Windows Forms Controls: Z-order and Copying Collections
Ken Spencer

Code download available at: AdvancedBasics0401.exe (131KB)

Q Are the SendToBack and BringToFront methods the only options for modifying the z-order of Windows® Forms controls?

A It took a suggestion from one of my buddies at Microsoft to figure this stumper out. Of course, it's pretty simple once you know how to do it.

There is no property on a Windows Forms control to manipulate z-order as there was in previous versions of Visual Basic® (ZOrder). It turns out the method you need is in the Controls collection. The SetChildIndex method allows you to set the z-index of a particular control. The following line, from the click event in Figure 1, changes the z-index of Label2:

Me.Controls.SetChildIndex(Me.Label2, Label2NewIndex)
To retrieve the current index of a control call the GetChildIndex method, as shown in the ShowIndex method in Figure 1.

Figure 1 frmMain Move Controls (Close Figure 1)
Private Sub cmdMove_Click(ByVal sender As System.Object, _
  ByVal e As System.EventArgs) Handles cmdMove.Click
        Dim Label2NewIndex As Integer
        Try
            Label2NewIndex = numIndex.Value
            Me.Controls.SetChildIndex(Me.Label2, Label2NewIndex)
            ShowIndex()
        Catch ex As Exception
            Throw New Exception(ex.Message)
        End Try
End Sub

Sub ShowIndex()
        txtLabel1Index.Text = Me.Controls.GetChildIndex(Me.Label1).ToString
        txtLabel2Index.Text = Me.Controls.GetChildIndex(Me.Label2).ToString
End Sub

Q How can I use the CopyTo method of the Windows Forms controls collection to copy controls into an array?

A The CopyTo method will take the current controls collection and copy all the controls into an array. In order to use it, you must specify the array and the starting point. For instance, the following code copies the controls to the MyArrayOfControls array starting at the first element:

Me.Controls.CopyTo(MyArrayOfControls, 0)

Follow along with me while I create a sample to illustrate this. First, I created a form named frmControlsToArrayMaster that contains a set of six label/textbox pairs and three buttons. Next, I created a second form, named frmArrayClient, which has no controls but is the same size as the first form. Then I was ready to write some code.

The first code I wrote wired up the cmdCopy_Click event (see Figure 2). This event redims the MyArrayOfControls array to the number of controls in the collection. I then used CopyTo to copy the controls to the array starting at the first position. I put this code inside a button click event to make it easier to see when you run it.

Figure 2 frmControlsToArrayMaster (Close Figure 2)
Private Sub cmdCopy_Click(ByVal sender As System.Object, _
  ByVal e As System.EventArgs) Handles cmdCopy.Click
        ReDim MyArrayOfControls(Me.Controls.Count)
        Me.Controls.CopyTo(MyArrayOfControls, 0)
End Sub

Private Sub cmdNew_Click(ByVal sender As System.Object, _
  ByVal e As System.EventArgs) Handles cmdNew.Click
        Dim frm As New frmArrayClient
        Dim ct As Control

        frm.ShowDialog()

        Me.Controls.AddRange(MyArrayOfControls)
        For Each ct In Me.Controls
            If ct.GetType.ToString = "System.Windows.Forms.Button" Then
                ct.Visible = True
            End If
        Next
End Sub

Private Sub cmdNewForm2_Click(ByVal sender As System.Object, _
  ByVal e As System.EventArgs) Handles cmdNewForm2.Click
        Dim frm As New frmArrayClient
        frm.ShowDialog()
End Sub

Next, I wired up the cmdNew_Click event. The first version of the code looked like this:

Private Sub cmdNew_Click(ByVal sender As System.Object, _
   ByVal e As System.EventArgs) Handles cmdNew.Click
   Dim frm As New frmArrayClient
   frm.ShowDialog()
End Sub

Then, I added the following code to frmArrayClient:

Dim ct As Control
If MyArrayOfControls.GetUpperBound(0) > 0 Then
   Me.Controls.AddRange(MyArrayOfControls)
For Each ct In Me.Controls
   If ct.GetType.ToString = "System.Windows.Forms.Button" Then
   ct.Visible = False
   End If
Next
End If
This code adds the controls from the first form by calling Controls.AddRange. Then it sets the Visible property of any buttons to False because I don't want the same command buttons on this form. Pretty cool? Let's see. Take a look at Figure 3.

Figure 3 Reparented Controls
Figure 3 Reparented Controls

The top form is frmControlsToArrayMaster, which has the controls on it. But where are these controls? When AddRange was called in frmArrayClient, it actually loaded the controls from my array on the new form. Since a control can only have one parent, the controls were reassigned as children of the new form (frmArrayClient). This means the controls are no longer on the first form (frmControlsToArrayMaster).

The cmdNew_Click code shown in Figure 2 takes a stab at fixing this. It puts the controls back on frmControlsToArrayMaster. You can prove this to yourself by running the sample application, putting data in two controls, then clicking the New button. Now add data to two more controls and close the new form. All the data you entered is now on the original form. In most cases this is pretty cool, but what happens when you want to actually copy the controls onto the new form instead of moving them?

I played with this idea a bit, then called my buddy Bill Martschenko at NetEdge Software who sent me some code that elegantly solves the problem of copying (cloning) controls.

The goal is to take the controls in the array generated by CopyTo and create a new set of controls that have the same properties (or a subset of them) as the original control. Then you can change the new controls without impacting the original controls.

The form frmNewArrayClient2 takes the approach Bill suggested. Figure 4 shows this new form. As you can see, the original form (top) still has its controls but the new form has the same controls minus the command buttons (bottom).

Figure 4 Copied Controls
Figure 4 Copied Controls

The form load event of frmNewArrayClient2, shows only this code:

Dim ctNew As Control
Dim i As Integer

If MyArrayOfControls.GetUpperBound(0) > 0 Then
    For i = 0 To MyArrayOfControls.GetUpperBound(0)
        ctNew = CloneControl(MyArrayOfControls(i))
        If ctNew.GetType.ToString <> "System.Windows.Forms.Button" Then
            Me.Controls.Add(ctNew)
        End If
    Next
End If

This code loops through the controls in MyArrayOfControls, creates a new control for each, and adds this new control to the form if it's not a button. The new controls are created by a call to the CloneControl function. This code is quite simple and performs the task of copying the controls. Now let's dig into the code that does the work behind CloneControl (see Figure 5). First, the CloneControl function executes this line of code to create a new control of the same type as the current control:

Dim newControl As Control = CType(NewAs(c), Control)

Figure 5 modControls Module (Close Figure 5)
Module modControls
    Friend Function CloneControl(ByVal c As Control) As Control
        ' instantiate another instance of the given control and
        ' clone the common properties
        Dim newControl As Control = CType(NewAs(c), Control)
        CloneProperties(newControl, c, "Visible", "Size", "Font", _
            "Text", "Location", "BackColor", "ForeColor", "Enabled", _
            "BackgroundImage")

        ' clone properties unique to specific controls
        If TypeOf newControl Is ButtonBase Then
            CloneProperties(newControl, c, "DialogResult", _
                "BackgroundImage", "FlatStyle", "TextAlign", "Image", _
                "ImageAlign", "ImageIndex", "ImageList")
        ElseIf TypeOf newControl Is LinkLabel Then
            CloneProperties(newControl, c, "VisitedLinkColor", _
                "LinkVisited", "LinkColor", "LinkBehavior", "LinkArea", _
                "FlatStyle", "BorderStyle", "DisabledLinkColor", _
                "ActiveLinkColor", "Image", "ImageAlign", "ImageIndex", _
                "ImageList")
        End If
        Return newControl
    End Function
End Module

Public Module ObjectFactory
    Public Function NewAs(ByVal t As Type) As Object
        Return t.Assembly.CreateInstance(t.FullName)
    End Function

    Public Function NewAs(ByVal x As Object) As Object
        Return ObjectFactory.NewAs(x.GetType())
    End Function

    Public Sub CloneProperties( ByVal target As Object, ByVal source As _
        Object, ByVal ParamArray propertyNames() As String)

        Dim sourceProperties As New PropertyAccessor(source)
        Dim targetProperties As New PropertyAccessor(target)
        Dim p As String
        For Each p In propertyNames
            targetProperties(p) = sourceProperties(p)
        Next
    End Sub
End Module

A close examination of the CType call shows a call to the NewAs method. The parameter of this function is the control you want to copy. The NewAs function is also shown in Figure 5.

If you examine the NewAs function definitions (there are two in Figure 5), you will see that they make what looks like recursive calls to themselves, but they're merely overloaded. This handy trick allows you to pass something as an object, making the first function call clean. Then that function can use some type of operator (such as GetType) on the parameter and then call the second function. This makes it easier for developers to use the function since they don't need to know the details of how to call the private function. The second NewAs function makes a call to the CreateInstance method of the assembly class. This method creates a new instance of the control by looking up the control's type. This allows you to create an instance of the control you want to clone.

The second line of the CloneProperties function actually copies the properties of the control (c) and maps those properties to the new control (newControl), as shown here:

CloneProperties(newControl, c, "Visible", "Size", "Font", _
    "Text", "Location", "BackColor", "ForeColor", "Enabled", _
    "BackgroundImage")

The CloneControl function finishes with an If block that allows you to set up special handling for particular types of controls. For instance, the sample has an If block that will copy particular properties when the control is a button or linklabel. It also makes a second call to CloneProperties to grab the additional properties.

The CloneProperties function is also shown in Figure 5. This function takes an array of the properties you want to set. Then it creates an instance of the PropertyAccessor class for each control. Finally, the code loops through the parameter names array and for each sets the target control's property equal to the source control's property. This copies the properties from the source control to the new control one by one.

Let's walk through the PropertyAccessor class (see Figure 6). The constructor for the class takes one parameter—a reference to either the source or target control. This reference is then set to the target_ variable. The Target property allows you to retrieve a reference to the control.

Figure 6 PropertyAccessor Class (Close Figure 6)
Imports System.Reflection
Public Class PropertyAccessor
    Public Sub New(ByVal target As Object)
        Me.target_ = target
    End Sub
    Public ReadOnly Property Target() As Object
        Get
            Return Me.target_
        End Get
    End Property

    Default Public Property Item(ByVal propertyName As String) As Object
        Get
            Dim prop As PropertyInfo = _
                Me.Target.GetType().GetProperty(propertyName)
            Return prop.GetValue(Me.Target, Nothing)
        End Get
        Set(ByVal value As Object)
            Dim prop As PropertyInfo = _
                Me.Target.GetType().GetProperty(propertyName)
            prop.SetValue(Me.Target, value, Nothing)
        End Set
    End Property

#Region " Private State "
    Private target_ As Object
#End Region
End Class

The interesting code in this class defines the Item property. This property is declared as the Default property for the class. You can see that this property takes one parameter:

Default Public Property Item(ByVal propertyName As String) As Object

The Get method of this property returns the property you want to access. Then it creates a new instance of PropertyInfo and sets this to the property you requested:

Dim prop As PropertyInfo = Me.Target.GetType().GetProperty(propertyName)
Once you have the property, you can return its value by calling the GetValue method. The Set method is also pretty straightforward. The new value is passed, so all you need to do is create an instance of the PropertyInfo class as in the Get method:
Dim prop As PropertyInfo = Me.Target.GetType().GetProperty(propertyName)
Then you can call SetValue to actually set the value of the property:
prop.SetValue(Me.Target, value, Nothing)
That's it. Now you have the code to easily copy a set of controls from one form to another.

Both of this month's questions looked difficult to accomplish at first. But in both cases the Microsoft® .NET Framework provided a straightforward way to accomplish the task. Sometimes you just have to be persistent to find the solution you're looking for.

Send your questions and comments for Ken to basics@microsoft.com.

Ken Spencer works for 32X Tech (http://www.32X.com), where he provides training, software development, and consulting services on Microsoft technologies.


Basic Instincts: Asynchronous Method Execution Using Delegates

Basic Instincts
Asynchronous Method Execution Using Delegates
Ted Pattison

Contents
Invoking a Method Asynchronously
IAsyncResult
Calling EndInvoke
Conclusion

This month I'll discuss the use of delegates to execute a method in an asynchronous fashion. I'll assume that you already know the fundamentals of programming with delegates, but if not, you should read the December 2002 and January 2003 Basic Instincts columns.

There are a number of scenarios in which asynchronous execution can be a valuable design technique. For example, you should use asynchronous execution in a Windows® Forms application when you want to execute a long-running command without blocking the responsiveness of the user interface. As you will see, programming with delegates makes it relatively easy to run a command asynchronously on a secondary thread. That means you can build a Windows Forms application that executes long-running calls across the network without freezing the user interface.

Executing methods asynchronously can also serve as a useful technique for a server-side application. Imagine you are writing the code behind an ASP.NET page to service a client request. What should you do if your code needs to execute two or more time-consuming calls across a network? For example, imagine you are required to write an ASP.NET page that must run a query against a remote database and also must execute a SOAP method on a remote Web Service to complete its work. If you don't use asynchronous method execution, you can only make one call across the network at a time. This means you can't start the second network call until the first network call returns. Asynchronous method execution using delegates lets you execute two or more network calls at the same time. This can significantly reduce the time it takes to process a server-side ASP.NET request, increasing the responsiveness of your application from the user's perspective.

Invoking a Method Asynchronously

Since you already know the fundamentals of programming delegates, I am going to present an example using a delegate object to execute a method in a synchronous fashion. Suppose you have written a shared method named GetCustomerList that executes a time-consuming call across the network to complete its work:

Class DataAccessCode
  Shared Function GetCustomerList(State As String) As String()
    '*** call across network to DBMS or Web Services to retrieve data
    '*** pass data back to caller using string array return value
  End Function
End Class

In this example the GetCustomerList method is defined with a single string parameter that accepts the name of a U.S. state. The method's return value is defined in terms of a string array that is used to pass a list of customers back to the method's caller.

To execute the GetCustomerList method using a delegate object, you must first define a delegate type with a matching signature. For example, you can define a delegate type named GetCustomerListHandler that takes a string parameter and returns a string array, like this:

'*** a delegate for executing handler methods
Delegate Function GetCustomerListHandler(State As String) As String()

Now that you have defined a delegate type with the correct calling signature, you can create a delegate object and bind it to the GetCustomerList method. Once you have created and bound a delegate object, you can execute GetCustomerList synchronously by calling the delegate object's Invoke method:

'*** create delegate object and bind to target method
Dim handler1 As GetCustomerListHandler
handler1 = AddressOf DataAccessCode.GetCustomerList

'*** execute method synchronously
Dim retval As String()
retval = handler1.Invoke("CA")

When you call Invoke on the delegate object, it forwards the call by executing the handler method for you. It's important to point out that a call to Invoke is synchronous. In other words, it's a blocking call because it doesn't return until the delegate object has finished executing GetCustomerList. When GetCustomerList completes its execution and returns, the delegate object takes its return value and passes it back to the caller of the Invoke method.

When the Visual Basic® .NET compiler generates a delegate type definition, it adds the Invoke method to support synchronous execution. The compiler also adds two other methods—BeginInvoke and EndInvoke—to support asynchronous execution.

You can rewrite the previous example using a delegate object to execute the GetCustomerList method in an asynchronous fashion. Instead of calling Invoke on the delegate object, call BeginInvoke, which requires you to pass any input parameters that are defined in the calling signature of the delegate. In my example, there is a single string parameter for passing a state name.

'*** create delegate object and bind to target method
Dim handler1 As GetCustomerListHandler
handler1 = AddressOf DataAccessCode.GetCustomerList

'*** execute method asynchronously
handler1.BeginInvoke("CA", Nothing, Nothing)
BeginInvoke also takes two other parameters that are not part of the calling signature of Invoke. I'll explain these two parameters later in this column.

When you make a call to BeginInvoke on a delegate object, you are essentially asking the common language runtime (CLR) to begin executing the handler method asynchronously on a secondary thread. While executing a method asynchronously with a call to BeginInvoke is powerful, it's also fairly easy because you don't have to be concerned with creating and managing a secondary thread. The CLR does this for you automatically.

Figure 1 shows how the CLR deals with a call to BeginInvoke. When you call the BeginInvoke method, the delegate object places a request in a special internal queue. The CLR maintains a pool of worker threads that are responsible for servicing the request in this queue. Asynchronous execution is achieved because the thread that calls BeginInvoke is not the same as the thread that executes the handler method.

Figure 1 Asynchronous Method Execution
Figure 1 Asynchronous Method Execution

The CLR dynamically determines how many worker threads to add to the pool by examining a number of factors. In a typical desktop application, the CLR limits the pool to a handful of threads. In a busy server-side application where codebehind ASP.NET pages are making lots of concurrent calls to BeginInvoke, the CLR will grow the pool size up to a maximum of 25 threads for each processor on the host machine. That means the number of worker threads in the pool grows proportionally with the number of processors. For example, the maximum pool size on a host server computer with four processors is 100 threads.

Now consider the differences between a call to Invoke and a call to BeginInvoke. Imagine you have a month's worth of dirty laundry and you would like to have it cleaned. You have the choice of doing it yourself at the laundromat or dropping it off at the cleaners and having someone else do the work for you.

Going to the laundromat and doing the laundry yourself is like calling Invoke; it is a synchronous undertaking. You must stay at the laundromat until the work is completed, and when you leave you take the completed work (the clean laundry) with you.

Dropping your dirty clothes off at the laundry is like a call to BeginInvoke; it is an asynchronous undertaking. After you drop off your laundry, you can immediately move on to other business. However, you don't leave with clean clothes. You leave with a laundry ticket and you must return at some later time to pick up your clean clothes. Furthermore, you must keep track of your ticket because the guy behind the counter might refuse to give your clothes back if you lose it.

When you call BeginInvoke, you are essentially dropping off a request for a method execution at the CLR laundry. The call to BeginInvoke returns right away and the calling thread can then move on to other business without having to wait for the CLR to execute the target method. After calling BeginInvoke you know that the CLR plans to execute that target method on the secondary thread, but you don't know exactly when it's going to happen. At a later point in time, you can ask the CLR whether it has finished executing that method asynchronously. You can also retrieve the method's return value. However, monitoring the status of an asynchronous call or harvesting an asynchronous method's return value requires you to provide the CLR with information about which asynchronous request you are investigating.

IAsyncResult

A call to BeginInvoke returns something that is analogous to a laundry ticket. More specifically, the BeginInvoke method has been designed to return an object that implements an interface named IAsyncResult. Here's an example of calling BeginInvoke and capturing the IAsyncResult object (the laundry ticket) when dispatching an asynchronous method call:

'*** create delegate object and bind to target method
Dim handler1 As GetCustomerListHandler
handler1 = AddressOf DataAccessCode.GetCustomerList

'*** execute method asynchronously and capture IAsyncResult object
Dim ar As System.IAsyncResult
ar = handler1.BeginInvoke("CA", Nothing, Nothing)
The IAsyncResult object allows you to monitor an asynchronous call in progress. It also allows you to retrieve the return value and any output parameters whenever an asynchronous method has finished executing.

It is important for you to understand that there isn't necessarily a one-to-one relationship between a delegate object and an IAsyncResult object. For example, one delegate object can be used to start two or more asynchronous calls that will execute at the same time, as shown here:

'*** create delegate object and bind to target method
Dim handler1 As GetCustomerListHandler
handler1 = AddressOf DataAccessCode.GetCustomerList

'*** execute multiple methods asynchronously
Dim ar1, ar2 As System.IAsyncResult
ar1 = handler1.BeginInvoke("CA", Nothing, Nothing)
ar2 = handler1.BeginInvoke("WA", Nothing, Nothing)
In this example, two different asynchronous calls are started at once. Each of these asynchronous calls is going to be run on a separate thread from the worker thread pool, as shown in Figure 2. This can be valuable because you can make two or more calls across the network in parallel. However, it's important for you to remember that each one of these asynchronous calls has its own associated IAsyncResult object.

Figure 2 Two Asynchronous Calls
Figure 2 Two Asynchronous Calls

The IAsyncResult interface exposes a property named IsComplete that allows you to monitor the status of an asynchronous call and determine whether it has completed. While this type of polling technique isn't efficient in most designs, it can be useful in a handful of situations. The following is an example of using the IsComplete property to determine whether the CLR has finished executing an asynchronous call:

Dim ar As IAsyncResult
ar = handler1.BeginInvoke("CA", Nothing, Nothing)

'*** allow some time to pass

'*** check to see if the async call has completed
If (ar.IsCompleted) Then
  '*** retrieve method return value
End If

Calling EndInvoke

You retrieve the return values and any output parameters by calling the EndInvoke method on the delegate. EndInvoke is defined with the same type of return value as the calling signature of the delegate type itself:

Dim ar As IAsyncResult
ar = handler1.BeginInvoke("CA", Nothing, Nothing)

'*** allow some time to pass

Dim retval As String()
retval = handler1.EndInvoke(ar)

As you can see, calling EndInvoke allows you to retrieve the return value from an asynchronous method call. Also note that a call to EndInvoke requires you to pass the IAsyncResult object associated with a particular asynchronous call. This is just like giving the cleaners your ticket when you return to pick up your clean clothes. The IAsyncResult object allows the CLR to determine which asynchronous call you are interested in.

Also note that the parameter list for EndInvoke will include any ByRef parameters defined within the delegate type so you can also retrieve any output parameters. However, the example I used here doesn't involve any output parameters, so the call to EndInvoke only requires that you pass a single parameter holding a reference to the IAsyncResult object.

You have seen that a call to EndInvoke is important because it allows you to retrieve the return value and any output parameters. Calling EndInvoke is also vital because it allows you to see if the asynchronous method executed successfully. If the asynchronous method experiences an unhandled exception, that exception object is tracked by the CLR and then thrown when you call EndInvoke, as shown in the following code:

Dim retval As String()
Try
  retval = handler1.EndInvoke(ar)
Catch ex As Exception
  '*** deal with exception here
End Try

It's important to complement every call to BeginInvoke with a call to EndInvoke. If you never call EndInvoke, you can never really be sure whether an asynchronous call executed successfully. Forgetting to call EndInvoke can allow runtime exceptions to go unnoticed, which can introduce bugs to your code that are hard to track down and fix.

You have already seen a few good reasons why a call to BeginInvoke requires a call to EndInvoke. There is one more very important reason why you should always call EndInvoke—it's the law. Forgetting to call EndInvoke will prevent the CLR from cleaning up some of the resources required in dispatching asynchronous calls. Therefore, you should assume your application will leak if you make calls to BeginInvoke without also making associated calls to EndInvoke.

Now let's focus our attention on how the timing works when you call EndInvoke. A call to EndInvoke returns right away if the worker thread has already completed the execution of the handler method. However, a call to EndInvoke will block if the asynchronous call hasn't yet started or is still in progress. You must consider the implications of this fact when you design your applications.

Think about that situation in which you were writing the code behind an ASP.NET page and you wanted to execute two calls across the network at the same time. The fact that EndInvoke is a blocking call makes it pretty easy to coordinate multiple paths of execution. The code in Figure 3 simultaneously executes two methods asynchronously in order to make two network calls in parallel. However, it's essential that both network calls complete successfully before the request as a whole can finish. The first call to EndInvoke blocks until the first asynchronous method has completed. The second call to the EndInvoke method blocks until the second call has completed.

Figure 3 Executing Multiple Methods Asynchronously (Close Figure 3)
'*** server-side request in code behind an ASP.NET page

'*** execute multiple methods asynchronously
Dim ar1, ar2 As System.IAsyncResult
ar1 = handler1.BeginInvoke("CA", Nothing, Nothing)
ar2 = handler1.BeginInvoke("WA", Nothing, Nothing)

'*** capture all return values
Dim retval1, retval2 As String()
retval1 = handler1.EndInvoke(ar1)
retval2 = handler1.EndInvoke(ar2)

'*** finish processing request

What happens if the second asynchronous call finishes before the first asynchronous call? It really doesn't matter. In this design it is irrelevant which of the two network calls finishes first. Both calls to GetCustomerList have to complete before the server-side request can finish its work. Therefore, the blocking behavior of EndInvoke plays an important role. It takes multiple paths of execution and joins them to create one logical path of execution.

All the while, the CLR is doing all the complicated work behind the scenes. It is managing threads, dispatching asynchronous calls, and blocking certain threads until other threads have completed their work. All you have to do is call BeginInvoke and then call EndInvoke. You can clearly see that delegates provide an easy, powerful abstraction for asynchronous method execution.

Now let's turn our attention to asynchronous method execution in a Windows Forms application running on a desktop application. This is a much different scenario than a server-side application because here all the code usually runs on a single thread known as the primary UI thread. The primary UI thread is important in a Windows Forms UI application because it is in charge of maintaining the responsiveness of the user interface. If you freeze the primary user interface thread with a long-running call across the network, the hosting application will be unresponsive to the user until the call returns.

In this case it's critical to use asynchronous execution so you don't lock down the user interface. You've seen that it's fairly easy to dispatch an asynchronous call using the BeginInvoke method, but a problem arises. How does your application respond when an asynchronous method call has finished executing? How do you update the user interface when an asynchronous call is finished? You can use a polling technique and continually test for completion, but that isn't very efficient. Instead, you should take advantage of a handy feature of delegates that lets you set up a callback method that's automatically executed by the CLR when an asynchronous method call is completed.

The code in Figure 4 shows an example of a Windows Forms application that executes the GetCustomerList method asynchronously. All the code has been structured within a class named Form1. Within this class definition, you can see one possible approach for setting up a callback method when dispatching an asynchronous method call using BeginInvoke.

Figure 4 Callback Method (Close Figure 4)
Public Class Form1 : Inherits System.Windows.Forms.Form

  Private TargetHandler As GetCustomerListHandler _
                           = AddressOf DataAccessCode.GetCustomerList

  Private CallbackHandler As AsyncCallback _
                             = AddressOf MyCallbackMethod

  Sub cmdExecuteTask_Click(sender As Object, e As EventArgs) _
      Handles cmdExecuteTask.Click
    '*** execute method asynchronously with callback
    TargetHandler.BeginInvoke("CA", CallbackHandler, Nothing)
  End Sub

  '*** callback method runs on worker thread and not the UI thread
  Sub MyCallbackMethod(ByVal ar As IAsyncResult)
    '*** this code fires at completion of each asynchronous method call
    Dim retval As String()
    retval = TargetHandler.EndInvoke(ar)
  End Sub
End Class

Let's walk through the code. When you want to set up a callback method, you must work with a delegate type named AsyncCallback that is defined by the Framework Class Library in the System namespace. When you want to create a callback method, you must define this method with a calling signature that conforms to the calling signature of the AsyncCallback delegate type. MyCallbackMethod in Figure 4 is an example of a method defined with the proper calling signature. As you can see when looking at the code, the callback method must be defined as a Sub procedure with a single parameter of type IAsyncResult.

You should observe that the Form1 class contains two fields, each of which holds a reference to a delegate object. The first field, TargetHandler, holds a reference to a delegate object that's used to dispatch the asynchronous call to the GetCustomerList method. The second field, CallbackHandler, holds a reference to a delegate object that's bound to MyCallbackMethod. These delegate objects are both used in the call to BeginInvoke.

Now examine the call to BeginInvoke inside the event handler method cmdExecuteTask. The TargetHandler field is used to execute BeginInvoke and dispatch the asynchronous call. The second parameter passed to BeginInvoke is the reference to the delegate object that's bound to MyCallbackMethod. In essence, you pass a reference to a delegate to point the CLR to whichever callback method you'd like to use. This is all that's required to set up a callback method after an asynchronous method call.

Now let's see how things work in my example when an asynchronous call is dispatched. When the application calls BeginInvoke, the CLR executes the handler method using a worker thread from the CLR thread pool. Next, that same worker thread executes the callback method. When this work is complete, the CLR returns that worker thread to the pool so it is available to service another asynchronous request.

It's very important to understand that the callback method does not run on the primary UI thread that made the call to BeginInvoke. Once again, the callback method is executed on the same secondary thread that executed the asynchronous method.

The last parameter passed in a call to BeginInvoke is the AsyncState parameter. AsyncState is a very open-ended parameter that you can use to pass anything you want. It is defined in terms of the Object class so it can be used to pass any value or object. In the example I presented in Figure 4, there was no need to utilize the AsyncState parameter, so a value of Nothing was passed.

In other designs, it can be convenient to use the AsyncState parameter to pass a value or an object from the code that calls BeginInvoke to the implementation of the callback method. For example, imagine you need to pass an Integer so that it's available in the implementation of the callback method. When you call BeginInvoke, you can pass the Integer value in the AsyncState parameter like this:

Dim MyValue As Integer = 100
TargetHandler.BeginInvoke("CA", CallbackHandler, MyValue)

The value or object that is passed in the AsyncState object is tracked by the IAsyncResult object and can be retrieved using a property named AsyncState. Here's an example of retrieving the Integer value in a callback method:

Sub MyCallbackMethod(ByVal ar As IAsyncResult)
  Dim y As Integer = CInt(ar.AsyncState)
  '*** other code omitted for brevity
End Sub

Remember that the AsyncState property is defined in terms of the Object class, so you typically need to convert it explicitly to a more specific type before you can program against the value or object that it contains.

You've seen that it's relatively easy to call BeginInvoke from the code behind a Windows Form to dispatch an asynchronous method call. You have also learned how to set up a callback method that fires automatically when the asynchronous method call has completed. The last thing you need to learn is how to update the UI to tell the user that the work of the asynchronous call has been finished. This can be tricky. It's hard to write the code correctly and you can get into serious trouble if you don't do it right.

There is an important threading rule to follow when programming with Windows Forms. The primary UI thread is the only thread that's allowed to touch a form object and all of its child controls. That means it's illegal for code running on a secondary thread to access the methods and properties of a form or a control. However, you must remember that a callback method such as MyCallbackMethod executes on a secondary thread and not the primary UI thread. That means you should never attempt to update the user interface directly from this kind of callback method.

To be more concise, the callback method that's running on the secondary thread must force the primary UI thread to execute the code that updates the user interface. The code to do this is fairly involved and I am going to wait until next month's column to explain exactly how to do it. For now just understand that it's not an acceptable programming technique to update the user interface directly from a callback method such as MyCallbackMethod due to the threading constraints imposed by the framework.

Conclusion

When executing an asynchronous method call using a delegate, each delegate type provides a BeginInvoke method to start a method call asynchronously on a secondary thread from a thread pool maintained by the CLR. Each delegate type provides the complementary EndInvoke method that can be used to harvest an asynchronous method return value and to determine whether the method has executed without any unhandled exceptions.

One of the most valuable aspects of asynchronous execution using delegates is that you don't have to worry about creating and managing secondary threads. The CLR automatically maintains a pool of worker threads for you, which means you can benefit from techniques involving asynchronous and parallel method execution without ever having to program directly against threads. All that remains to be learned is how to call BeginInvoke and EndInvoke at the right times.

Send your questions and comments for Ted to instinct@microsoft.com.

Ted Pattison is a cofounder of Barracuda.NET (http://barracuda.net), an education company that assists companies building networked applications using Microsoft technologies. Ted is the author of several books including Building Applications and Components with Visual Basic .NET (Addison-Wesley, October 2003).


.NET Column: Practical Multithreading for Client Apps

.NET
Practical Multithreading for Client Apps
Jason Clark

Code download available at: NET0401.exe (118KB)
Contents
Threading Background
Why Use More Than One Thread?
Threading and the Windows UI
The Managed Thread Pool
Thread Synchronization at its Simplest
Details—Implementing a Cancel Button
More Details—a Word on Timers
A Peek into the Future

Writing applications that use multiple threads is often considered an advanced programming task, prone to errors. In this month's column, I'll focus on a practical application of multithreading in Windows® Forms applications with some real benefits, while attempting to keep things simple. My goal is to present multithreading in an approachable way that addresses a very common need: writing applications with a user interface that remain responsive to the user.

Multithreading is often associated with server-side software, scalability, and performance technology. However, with the Microsoft® .NET Framework, many server-side applications are hosted in the ASP.NET architecture. As such, these applications are logically single threaded because IIS and ASP.NET perform much or all of the multithreading in an ASP.NET Web Form or Web Service application. You can usually ignore threading altogether for ASP.NET applications. This is one reason why the threading features of the .NET Framework are oriented towards client-side uses such as assuring a responsive user interface while performing lengthy operations.

Threading Background

Threads execute code. They are implemented by the OS and are an abstraction of the CPU itself. While many systems have only a single CPU, threads are a way of dividing a fast CPU's processing ability among a number of operations to make them seem simultaneous. Even a system that has multiple CPUs will typically have more threads running than it has processors.

In Windows-based applications, each process has at least one thread which is capable of executing machine-language instructions. Once all of the threads in a process have terminated, the process itself and all of its resources are cleaned up by Windows.

Many applications are designed as single-threaded applications, meaning that the process that implements the application never has more than one thread executing even though other processes may be active in the system at the same time. Normally, a process does not have to concern itself with the threads executing in other processes in the system.

All of the threads in a single process, however, share not only the virtual address space of the process but also many other process-level resources, such as file and window handles. Because of the shared nature of the resources in a single process, threads typically have to be concerned with what other threads in the same process are doing. Thread synchronization is the art of keeping multiple threads in a single process from stepping on each other's toes. This is what makes multithreading difficult.

The best approach is to use multiple threads only as needed, keeping things as simple as possible. Also, try to avoid situations that will require thread synchronization. In this column, I will show you how to do these things for common client applications.

Why Use More Than One Thread?

Many single-threaded client applications already exist, and many more are being written daily. In many cases, single-threaded behavior is sufficient. However, for certain applications, adding a little asynchronous behavior can improve the user's experience. A typical database front end makes a good example.

Database queries can take varying amounts of time to complete. In a single-threaded application, these queries can block the application's ability to process window messages, causing the application's UI to freeze. The solution, which I will cover in more detail, is to continue processing messages from the OS in one thread while doing lengthy work in another. Keeping your application's UI responsive, even while it's busy working behind the scenes, is one great reason to make use of a second thread in your code.

Let's look first at a single-threaded GUI application that performs a lengthy operation. Then we will spruce up the application with additional threads.

The code in Figure 1 is the complete source for an application written in C#. It creates a form on which there is a textbox and a button. If you enter a number into the textbox and then press the button, the application will process for that number of seconds, beeping once per second to represent background processing. In addition to the code in Figure 1, the complete download for this column can be found at the link at the top of this article. Download or type in the code in Figure 1, compile it, and run it before reading on. (Before compiling, right-click your project in Visual Studio® .NET and add a reference to the Microsoft Visual Basic® .NET runtime.) When you try to run the SingleThreadedForm.cs application in Figure 1, you'll immediately see some problems.

Figure 1 SingleThreadedForm.cs (Close Figure 1)
using System;
using System.Drawing;
using System.Threading;
using System.Windows.Forms;
using Microsoft.VisualBasic;

class App {

   // Application entry point
   public static void Main() {

      // Run a Windows Forms message loop
      Application.Run(new SingleThreadedForm());
   }
}

// A Form-derived type
class SingleThreadedForm : Form {

   // Constructor method
   public SingleThreadedForm() {

      // Create a text box
      text.Location = new Point(10, 10);
      text.Size = new Size(50, 20);
      Controls.Add(text);

      // Create a button
      button.Text = "Beep";
      button.Size = new Size(50, 20);
      button.Location = new Point(80, 10);

      // Register Click event handler
      button.Click += new EventHandler(OnClick);

      Controls.Add(button);
   }

   // Method called by the button's Click event
   void OnClick(Object sender, EventArgs args) {
      // Get an int from a string
      Int32 count = 0;
      try { count = Int32.Parse(text.Text); } catch (FormatException) {}

      // Count to that number
      Count(count);
   }

   // Method beeps once per second
   void Count(Int32 seconds) {

      for (Int32 index = 0; index < seconds; index++) {
         Interaction.Beep();
         Thread.Sleep(1000);
      }
   }

   // Some private fields by which to reference controls
   Button  button = new Button();
   TextBox text = new TextBox();
}

On your first test run, enter 20 into the textbox, and press the button. You will see that the application's UI becomes entirely unresponsive. You can't click the button or edit the textbox, the application cannot be closed gracefully, and if you obscure and subsequently expose a portion of the window, it will not repaint itself (see Figure 2). The application has effectively locked up for 20 seconds, and yet it continues to beep away, proving that it has not really crashed. This simple application illustrates the problem with single-threaded GUI applications.

Figure 2 No Repaint
Figure 2 No Repaint

This unresponsive UI is the first problem I'll solve with multithreading, but first I'll explain what causes this phenomenon.

Threading and the Windows UI

The Windows Forms class library is built around a Win32® API known as User32. User32 implements GUI elements such as windows, menus, buttons, and the like. All windows and controls implemented by User32 use an event-driven messaging infrastructure.

In brief, here is how it works. Things that happen to windows, such as mouse clicks, coordinate changes, size changes, and paint requests, are called events. Events in the User32 API model are represented by window messages. Each window has a function, called a window procedure or WndProc, which is implemented in the application. WndProc functions are responsible for processing the window messages for the window.

But the WndProc is not magically called by the system. Instead, the application must actively get window messages from the system by calling an API function called GetMessage. These same messages are dispatched to their destination window's WndProc method by an application call to the DispatchMethod API function. Applications do the receiving and dispatching of window messages in a loop commonly called a message pump or message loop. The thread owns all of the windows for which it pumps messages, and the WndProc functions are called by this same thread.

Now back to the Windows Forms classes. Windows Forms probably abstracts 95 percent of the messaging infrastructure of User32 away from the application. Instead of WndProc functions, Windows Forms applications define event handlers and virtual function overrides to handle the various system events that relate to a form (window) or control. However, the message pump must still be run, and it is implemented in the Application.Run method in the Windows Forms API.

The code in Figure 1 seems only to call Application.Run and then exit. However, it's less apparent that the application's main thread lives its entire life in a single call to Application.Run pumping messages, which results in calls into the various event handlers for the rest of the application. When the button on the form is clicked, the OnClick method in Figure 1 is called by the main thread; this is the same thread that is responsible for pumping messages in the Application.Run method.

This explains why the UI becomes unresponsive when a lengthy operation is occurring. If a lengthy operation (such as a database query) occurs in an event handler the thread is tied up, which keeps it from pumping messages. The inability to pump messages translates to the window or form's inability to respond to resizes, repaint itself, handle clicks, or do anything in response to user interaction.

In the following section I will modify the sample code shown in Figure 1 in order to use the common language runtime's (CLR) thread pool for lengthy operations so that the main thread will remain available for message pumping.

The Managed Thread Pool

The CLR maintains a thread pool for every managed process, meaning that if your application needs something to happen asynchronously of your main thread, you can easily borrow a thread from the thread pool to do the work. Once your work is finished the thread is returned to the pool for later use. Let's look at the form example, modified to use the thread pool.

Note the lines of FlawedMultiThreadedForm.cs in Figure 3 that are shown in red; these are the only code changes necessary to turn the single-threaded code in Figure 1 into a multithreaded application. If you build the code found in Figure 3 and set it running for 20 seconds, you will see that the UI remains responsive while processing the 20-beep request. This responsive UI is the compelling reason to multithread in client applications.

Figure 3 FlawedMultiThreadedForm.cs (Close Figure 3)
using System;
using System.Drawing;
using System.Threading;
using System.Windows.Forms;
using Microsoft.VisualBasic;

class App {

   // Application entry point
   public static void Main() {

      // Run a Windows Forms message loop
      Application.Run(new FlawedMultiThreadedForm());
   }
}

// A Form-derived type
class FlawedMultiThreadedForm : Form {

   // Constructor method
   public FlawedMultiThreadedForm() {

      // Create a text box
      text.Location = new Point(10, 10);
      text.Size = new Size(50, 20);
      Controls.Add(text);

      // Create a button
      button.Text = "Beep";
      button.Size = new Size(50, 20);
      button.Location = new Point(80, 10);

      // Register Click event handler
      button.Click += new EventHandler(OnClick);

      Controls.Add(button);
   }

   // Method called by the button's Click event
   void OnClick(Object sender, EventArgs args) {
      // Get an int from a string
      Int32 count = 0;
      try { count = Int32.Parse(text.Text); } catch (FormatException) {}

      // Count to that number
      WaitCallback async = new WaitCallback(Count);
      ThreadPool.QueueUserWorkItem(async, count);
   }

   // Async method beeps once per second
   void Count(Object param) {

      Int32 seconds = (Int32) param;
      for (Int32 index = 0; index < seconds; index++) {
         Interaction.Beep();
         Thread.Sleep(1000);
      }
   }

   // Some private fields by which to reference controls
   Button  button = new Button();
   TextBox text = new TextBox();
}

However, by making the changes shown in Figure 3, a new problem is introduced (as indicated by the figure's name); now the user can fire off multiple lengthy simultaneous beep operations. In many real apps this would qualify as one thread stepping on another's toes. To fix this requires thread synchronization. I will address this, but first let's take a closer look at the CLR's thread pool.

The System.Threading.ThreadPool class in the class library provides an API to the CLR's thread pool. The ThreadPool type cannot be instantiated and consists of static members. The most important methods on the ThreadPool type are the two overloads of ThreadPool.QueueUserWorkItem. Both of these methods let you specify a method that you would like to be called back by a thread from the thread pool. You specify the method using an instance of the WaitCallback delegate type defined in the class library. One of the overloads allows you to specify a parameter to the asynchronous method; this is the version used in Figure 3.

The following two lines of code are used to create a delegate instance representing a method named Count followed by a call to queue up a callback to the method from the thread pool:

WaitCallback async = new WaitCallback(Count);
ThreadPool.QueueUserWorkItem(async, count);

Both of the ThreadPool.QueueUserWorkItem methods add your specified asynchronous callback method to a queue and then return immediately. Meanwhile, the thread pool monitors the queue and begins dequeuing the methods and calling them using a thread or threads from the pool. This is the primary use of the CLR's thread pool.

The CLR's thread pool is also used by other APIs in the system. For example, the System.Threading.Timer object queues up callbacks on the thread pool at timed intervals. The ThreadPool.RegisterWaitForSingleObject method queues up callbacks from the thread pool in response to a kernel synchronization object being signaled. And finally, callbacks performed by the various asynchronous methods in the class library are performed using the CLR's thread pool.

In general, an application that needs to use multiple threads only for simple asynchronous operations should use the thread pool exclusively. This is recommended over the manual creation of thread objects. Calls to ThreadPool.QueueUserWorkItem are simple to perform and make better use of system resources than the repeated creation of manual threads.

Thread Synchronization at its Simplest

Earlier in this column I defined thread synchronization as the art of keeping multiple threads from stepping on each other's toes. The problem with the FlawedMultiThreadedForm.cs application shown in Figure 3 is that after the user initiates a lengthy beep operation by clicking the button, they can continue to click the button and initiate more beep operations. If, instead of a beep, the lengthy operation is a database request or the manipulation of a data structure in the process' memory, you certainly would not want two or more threads doing the same work at exactly the same time. At best this is a waste of system resources, and at worst it can cause data corruption.

The simplest solution to this problem is to disable the portion of the user interface that initiates any operation that is incompatible with the lengthy operation currently running on a thread from the pool. Then, when the operation has finished, the thread pool thread can communicate back to the main thread, letting it know that it is finished and that it is safe to reenable the disabled portions of the user interface.

The easy part of this task is the disabling of UI elements like buttons; communication between the two threads is slightly more difficult. I will show you how to do both of these things in a moment. But first, let me point out that all thread synchronization is performed using some form of inter-thread communication—a means of communication from one thread to another. Later I will discuss an object type known as an AutoResetEvent that is used solely for inter-thread communication.

Now let's look at some code that adds thread synchronization to the FlawedMultiThreadedForm.cs application shown in Figure 3. Again, the CorrectMultiThreadedForm.cs shown in Figure 4 is only a minor modification of its predecessor, with the changes shown in red. If you run the application you will see that the UI is disabled (but doesn't hang) while a beep operation is running, then reenabled when finished. This time there are enough code changes that I will run through each of them.

Figure 4 CorrectMultiThreadedForm.cs (Close Figure 4)
using System;
using System.Drawing;
using System.Threading;
using System.Windows.Forms;
using Microsoft.VisualBasic;

class App {

   // Application entry point
   public static void Main() {

      // Run a Windows Forms message loop
      Application.Run(new CorrectMultiThreadedForm());
   }
}

// A Form-derived type
class CorrectMultiThreadedForm : Form{

   // Constructor method
   public CorrectMultiThreadedForm() {

      // Create a textbox
      text.Location = new Point(10, 10);
      text.Size = new Size(50, 20);
      Controls.Add(text);

      // Create a button
      button.Text = "Beep";
      button.Size = new Size(50, 20);
      button.Location = new Point(80, 10);

      // Register Click event handler
      button.Click += new EventHandler(OnClick);

      Controls.Add(button);

      // Cache a delegate for repeated reuse
      enableControls = new BooleanCallback(EnableControls);
   }

   // Method called by the button's Click event
   void OnClick(Object sender, EventArgs args) {
      // Get an int from a string
      Int32 count = 0;
      try { count = Int32.Parse(text.Text); } catch (FormatException) {}

      // Count to that number
      EnableControls(false);
      WaitCallback async = new WaitCallback(Count);
      ThreadPool.QueueUserWorkItem(async, count);
   }

   // Async method beeps once per second
   void Count(Object param) {

      Int32 seconds = (Int32) param;
      for (Int32 index = 0; index < seconds; index++) {
         Interaction.Beep();
         Thread.Sleep(1000);
      }

      Invoke(enableControls, new Object[]{true});
   }

   void EnableControls(Boolean enable) {
      button.Enabled = enable;
      text.Enabled = enable;
   }

   // A delegate type and matching field
   delegate void BooleanCallback(Boolean enable);
   BooleanCallback enableControls;

   // Some private fields by which to reference controls
   Button  button = new Button();
   TextBox text = new TextBox();
}

Near the end of Figure 4 is a new method named EnableControls which enables or disables the textbox and button controls on the form. Earlier in Figure 4 I added a call to EnableControls to disable the controls on the form immediately before queuing up a work item to perform the background beeping. At this point, half of the thread synchronization job is done because the user is unable to initiate a conflicting background operation due to the disabled UI. Near the end of Figure 4 you'll find the definition of a delegate type named BooleanCallback whose signature is compatible with the EnableControls method. Just before that definition, a delegate field named enableControls is defined (note the case), which refers to the EnableControls method for this form. This delegate field is assigned earlier in the code.

You'll also see a callback from the main thread that owns and pumps messages for the form and its controls. This call is made to the EnableControls method with a parameter of true to reenable the controls. This is done by calling the Invoke method on the Form object from the background thread once it has finished its lengthy beep operation. The code passes the delegate which refers to EnableControls to Invoke, along with an Object array for the method parameters. The Invoke method is a very flexible form of inter-thread communication specific to forms and controls in the Windows Forms classes. In this example, Invoke is used to tell the main GUI thread to reenable the controls on the form by calling the EnableControls method.

The changes made to CorrectMultiThreadedForm.cs in Figure 4 implement the solution I suggested earlier—disabling the portion of the UI that initiates the operation that you don't want running while the beep operation is executing. Then tell the main thread to reenable the disabled portions when the operation has finished. The call to Invoke is unique, and it deserves special attention.

The Invoke method is defined on the System.Windows.Forms. Control type, making the method available to all derived controls in the class library including the Form type. The purpose of the Invoke method is to marshal a call from any thread to the thread that implements the message pump for the control or form.

When accessing Control-derived classes, including the Form class, you must do so from the thread which pumps the control's messages. This happens naturally in single-threaded applications. But if you begin to use threads from the thread pool, it is important that you avoid calling methods and properties on your UI objects from the background thread. Instead, you must access the Control-derived type indirectly using its Invoke method. Invoke is one of the very few methods on controls that is safe to call from any thread because it's implemented using the Win32 PostMessage API.

Inter-thread communication using Control.Invoke is a bit complicated. But once you are comfortable with this process, you have all of the tools necessary to achieve your goal in using multiple threads in client applications. The remainder of this column covers some extra details, but the CorrectMultiThreadedForm.cs application shown in Figure 4 is a complete solution for remaining responsive to the user while performing arbitrarily lengthy operations. Although most of the user interface is disabled, the user can still reposition and resize the window, as well as close the application. However, the user can't abuse the asynchronous behavior of the application. These small details go a long way in terms of a user's confidence in your application.

It is worth noting that my first thread-synchronized solution is synchronized without making use of any traditional threading constructs such as mutexes or semaphores. Instead, I used the more common UI notion of disabled controls.

Details—Implementing a Cancel Button

Sometimes you want to provide your users with a way to cancel your lengthy operations. All you need is some way for your main thread to communicate to a background thread that its operation is no longer needed and it can halt. The System.Threading namespace provides a class for this, AutoResetEvent.

AutoResetEvent is a very simple mechanism for inter-thread communication. An AutoResetEvent object can be in one of two states: signaled or non-signaled. When you create an instance of AutoResetEvent, you decide its initial state using a constructor parameter. Then threads that are aware of the object can communicate with each other by checking or modifying the AutoResetEvent's state via its Set and Reset methods.

In a way the AutoResetEvent acts like a Boolean, but it offers features that make it more appropriate for inter-thread communication. One such example is the ability for a thread to efficiently wait until an AutoResetEvent's state changes from non-signaled to signaled. This is accomplished by calling the WaitOne method on the object. Any thread that calls WaitOne on a non-signaled AutoResetEvent object will block efficiently until some other thread signals the object. Using a Boolean variable, the waiting thread would have to inefficiently poll the variable in a loop. Typically there is no need to un-signal an AutoResetEvent object using Reset because the object is automatically reset to the non-signaled state as soon as any other thread notices that the object is signaled.

Now you need a way for the background thread to test the state of an AutoResetEvent object without blocking, and you'll have all the tools necessary to implement thread cancellation. To do this, call the overloaded form of WaitOne that takes a timeout value and indicate a timeout of zero milliseconds. This way WaitOne returns immediately, whether the AutoResetEvent object is signaled or not. If the return value is true, then the object is signaled; otherwise it is returned because of the timeout.

Let's bring it all together to implement a cancel feature. If you want to implement a cancel button on a form that performs lengthy operations on background threads, follow these steps:

  1. Add a field of type AutoResetEvent to your form.
  2. Instantiate an AutoResetEvent object in the non-signaled state by passing false as the object's constructor parameter. Then store the object's reference in the field on your form. You will use this object for the life of the form in order to communicate cancellation of background operations to background threads.
  3. Add a cancel button to your form.
  4. In the cancel button's Click event handler, signal the AutoResetEvent object by calling its Set method.
  5. Meanwhile, in your background thread's logic periodically call WaitOne on the AutoResetEvent object to check if the user has canceled:
    if(cancelEvent.WaitOne(0, false)){
   // cancel operation
}
    You should remember to use a timeout parameter of zero milliseconds in order to avoid an unnecessary pause in the background thread's operation.
  6. If the user has canceled the operation, the AutoResetEvent will have been signaled by the main thread. Your background thread is alerted to this when it gets the true return value of WaitOne and ceases its operations. Meanwhile the event is automatically reset to the non-signaled state by the background thread's call to WaitOne.

To see an example of a form that can cancel its lengthy operations look for the CancelableForm.cs file that's available for download. The code is a complete app that differs only slightly from the CorrectMultiThreadedForm.cs app shown in Figure 4.

Note that the CancelableForm.cs sample file also employs a more advanced use of Control.Invoke, where the EnableControls method is designed to recall itself if it is being called from the wrong thread. It does this check before touching any methods or properties of the GUI objects on the form. This makes EnableControls safe to call directly from any thread, and effectively hides the complexity of the Invoke call in the method's implementation. Doing this can make for a more maintainable application overall. Note also that in this example EnableControls uses Control.BeginInvoke, which is the asynchronous version of Control.Invoke.

You might have noticed that the cancellation logic relies on the background thread's ability to periodically check for cancellation by calling WaitOne. But what if the lengthy background operation in question can't be canceled? What if the background operation is a single method call, like DataAdapter.Fill, that takes a long time? Sometimes there is a solution, though not always.

If your lengthy operation is fundamentally uncancelable, you may be able to get away with a pseudo-cancel where you let the operation finish, but do not act on the results of the operation in your application. This is not technically canceling the operation and does tie up a thread pool thread for the duration of the operation, but is an acceptable compromise for certain cases. If you implement a solution like this, you should reenable your disabled UI elements directly from your cancel button's event handler, rather than rely on the still-tied-up background thread to Invoke a call to enable your controls. It is also important that your background operation be designed to test to see if it has been canceled when it does return, so that it does not continue to act on the results of the now-canceled operation.

This is a more advanced approach to cancellation of lengthy operations that only works in certain cases. For example, a pseudo-cancellation of a database query is one thing, but pseudo-canceling a database update, deletion, or insert is a stickier operation. Operations that have persistent results or feedback-related operations, such as sounds and graphics, are not as easily pseudo-canceled using this approach because the effects of the operation remain obvious to the user after cancellation.

More Details—a Word on Timers

It is not uncommon for an application to need a timer to fire every so often to do some recurring task. For example, if your application showed the current time in a form's status bar, you might update the time every five seconds. The System.Threading namespace includes a multithreaded timer class called Timer.

When you create an instance of the Timer class, you indicate a period for the timer callback in milliseconds, and you also pass the object a delegate that it uses to call you back each time the timer period elapses. The callback occurs on a thread pool thread. In fact, what really happens each time the timer period is up is that a work item is queued to the thread pool; typically, the call is made right away, but if the thread pool is busy the callback may occur at a later point in time.

If you are considering making your application multithreaded, then you might consider using the Timer class for your timing needs. However, if your application uses Windows Forms, and you are not necessarily looking for multithreaded behavior, there is another timer class defined in the System.Windows.Forms namespace, also called Timer.

The System.Windows.Forms.Timer has a clear benefit over its multithreaded counterpart: since it is not multithreaded, it does not call you back on another thread, but rather uses your main thread which is pumping window messages for your application. In fact, the System.Windows.Forms.Timer class is implemented using a window message in the system called WM_TIMER. This way you do not have to worry about thread synchronization or inter-thread communication of any kind in your event handler for a System.Windows.Forms.Timer event.

For Windows Forms applications, a good rule of thumb is to use the System.Windows.Forms.Timer class unless you specifically need the callback to occur on a thread pool thread. Since this requirement is rare, keep things simple and use the System.Windows.Forms.Timer as the rule, even when other parts of your application are multithreaded.

A Peek into the Future

Microsoft recently demonstrated a forthcoming GUI API, code-named "Avalon," featured in Charles Petzold's article in this issue of MSDN Magazine (see page 70). User interface elements in the Avalon framework are not tied to a particular thread; instead each user interface element is associated with a single logical thread context implemented in the UIContext class. You might not be surprised to find out, though, that the UIContext class includes an Invoke method, as well as its sister BeginInvoke method, which serve logically identical purposes to the methods with the same name on the Control class in Windows Forms.

Send your questions and comments for Jason to dot-net@microsoft.com.

Jason Clark provides training and consulting for Microsoft and Wintellect (http://www.wintellect.com) and is a former developer on the Windows NT and Windows 2000 Server team. He is the coauthor of Programming Server-side Applications for Microsoft Windows 2000 (Microsoft Press, 2000). You can get in touch with Jason at JClark@Wintellect.com.


C++ Q&amp;A: Get Logical Drives with VolInfo, Modifying the System Menu

C++ Q&A
Get Logical Drives with VolInfo, Modifying the System Menu
Paul DiLascia

Code download available at: CQA0401.exe (127KB)

Q I'm trying to write an MFC program that lists the disk drives on the system (C:, D:, and so on). I also need to know if the drive is a hard disk or a CD. Is there a class to get this information?

Zachary Oscarmot

A There's no MFC class for that information, but Microsoft® Windows® has a group of volume management functions for just this purpose. They include functions for getting information about logical drives as well as volume mount points, an advanced NTFS feature that's not relevant here. For your purposes, you can just deal with logical drives. Figure 1 shows the relevant functions.

Figure 1 Windows Volume Management Functions (abridged) (Close Figure 1)

Function | Description
GetLogicalDrives | Returns a bitmask representing the currently available disk drives. Bit 0=drive A, bit 1=drive B, and so on. If a bit is ON, there's a logical drive for the corresponding drive letter.
GetLogicalDriveStrings | Fills a buffer with strings that specify valid drives in the system. For example, "C:\D:\F:\" and so on. Each string ends with a NULL character, with an extra NULL (empty string) at the end.
GetDriveType | Determines whether a disk drive is a removable, fixed, CD-ROM, RAM disk, or network drive. Returns one of the following codes: DRIVE_UNKNOWN DRIVE_NO_ROOT_DIR DRIVE_REMOVABLE DRIVE_FIXED DRIVE_REMOTE DRIVE_CDROM DRIVE_RAMDISK
GetVolumeInformation | Retrieves information about a file system and volume.
SetVolumeLabel | Sets the label of a file system volume.

There are four basic functions: GetLogicalDrives, GetLogicalDriveStrings, GetDriveType, and GetVolumeInformation. A fifth, SetVolumeLabel, sets the volume label in case you want to do that. The functions are all fairly straightforward, but just to make your life as easy as possible, I've encapsulated them in an MFC-friendly class, CVolumeMaster (see Figure 2), which lets you deal with CStrings instead of TCHAR arrays. I also wrote a program, VolInfo.exe, that shows how to use it. The source is available in the code download at the link at the top of this article. Figure 3 shows VolInfo displaying detailed information about my own computer.

Figure 2 VolumeMaster.h (Close Figure 2)
////////////////////////////////////////////////////////////////
// MSDN Magazine — January 2004
// If this code works, it was written by Paul DiLascia.
// If not, I don't know who wrote it.
// Compiles with Visual Studio .NET on Windows XP. Tab size=3.
//

// Handy class to encapsulate logical disk drive functions in Windows.
class CVolumeMaster {
public:
   // Get bitmask of logical drives—same as Windows function.
   DWORD GetLogicalDrives() {
      return ::GetLogicalDrives();
   }
   // Get drive type—same as Windows function.
   UINT GetDriveType(LPCTSTR lpPath) {
      return ::GetDriveType(lpPath);
   }
   // Get logical drive strings as array of CStrings
   int GetLogicalDriveStrings(CStringArray& ar);

   // Get volume information using CStrings instead of LPTSTR
   BOOL GetVolumeInformation(LPCTSTR drive,   // for example,
                                              // "\\Server\Share" or
                                              // "C:\"
      CString& volname,                       // volume name (label)
      DWORD& VolumeSerialNumber,              // volume serial number
      DWORD& MaximumComponentLength,          // maximum component len
                                              // (name between \'s)
      DWORD& FileSystemFlags,                 // flags
      CString& filesys);                      // file system, for
                                              // example, FAT or NTFS

   // Handy helpers to get human-readable names of things
   static CString FormatDriveType(UINT type);
   static CString FormatBitMask(DWORD bitmap, int group=5);
   static CString FormatFileSystemFlags(DWORD flags, LPCTSTR sep);
};
Figure 3 Detailed Drive Info
Figure 3 Detailed Drive Info

The first function, GetLogicalDrives, returns a DWORD that's a bitmask telling which drive letters are assigned. Bit 0 is drive A, bit 1 is drive B, and so on. Since there are 26 letters in the English alphabet and 32 bits in a DWORD, the mathematicians among you will quickly discern that one DWORD provides enough room to describe all possible combinations of drive letters, and then some. It's a good thing Redmond isn't located in Siberia! (Cyrillic has 33 letters.) CVolumeMaster has a static method, FormatBitMask, to format the bits as ASCII—VolInfo uses it to display a message like

10110 10001 11000 00000 00000 00000 00
which says that on my computer I have drives at A,C,D,F,J,K, and L. Whew! If programming in binary is too much for your cerebral cortex, there's always GetLogicalDriveStrings, which returns all the drive letters concatenated into one big string. Each drive letter has the form D:\ (with a trailing \), where D is the drive letter and each string is terminated with a null character and two nulls at the end. Since I know how stressful it is dealing with TCHARs, I wrote a handy CVolumeMaster wrapper that gets the drive letters in a CStringArray. This is a C++ column, after all. To get the drive letters, all you have to write is:
   CVolumeMaster vm;
   CStringArray arDrives;
   int n = vm.GetLogicalDriveStrings(arDrives);
Now arDrives holds the drive letter strings and n is the total number of logical drives. Got it?

Once you know which letters have drives, how do you know what kind of drive each one is? That's what GetDriveType is for. GetDriveType returns a code like DRIVE_FIXED for hard disks or DRIVE_CDROM for CD-ROM drives. CVolumeMaster has a static function to format the type as a human-readable string; VolInfo uses it for output. See the code download for details.

Finally, if you want even more information about a logical drive, such as its volume label, what file system it's using, or whether the drive supports named streams and encryption, GetVolumeInformation is the function to call. This Swiss-army-knife function gets the volume label, file system name (for example, NTFS or FAT), volume serial number, file system flags, and maximum component length.

"What the heck is the component length?" you ask. That's file-system-speak for the length of the part of a path name that can come between backslashes. In other words, if the file name is c:\mumble\bletch\oops, then mumble, bletch, and oops are the components, and there's a limit to how long each component can be. You can use VolInfo to discover that NTFS supports a maximum component length of 255, while CD-ROM drives usually only allow 127. This explains why, when saving your entire MP3 collection to CD, you often get a message stating that some file name or other is too long and asking if it's OK to truncate it.

CVolumeMaster has its own version of GetVolumeInformation—one that uses CString instead of LPTSTR:

CString volname,filesys;
DWORD serno, maxcomplen, flags;
vm.GetVolumeInformation("C:\",
  volname, serno, maxcomplen, flags, filesys);

Incidentally, the reason I keep stressing the use of CString is not only because it's easier, it's also more secure. With all the focus on security and bad viruses floating around these days, even Arnold Schwarzenegger knows what a buffer overflow is. Using CString is a good way to avoid one.

As for the flags, they're defined in Figure 4. WinBase.h and WinNT.h show the flags GetVolumeInformation can return. Once again, CVolumeMaster has a function to format them in a human-readable string—just the ticket for VolInfo and debugging your own application.

Figure 4 GetVolumeInformation Flags (Close Figure 4)

Value | Meaning
FILE_NAMED_STREAMS | The file system supports named streams
FILE_READ_ONLY_VOLUME | The specified volume is read-only (not supported on Windows 2000, Windows NT and Windows Me, Windows 98, or Windows 95)
FILE_SUPPORTS_OBJECT_IDS | The file system supports object identifiers
FILE_SUPPORTS_REPARSE_POINTS | The file system supports reparse points
FILE_SUPPORTS_SPARSE_FILES | The file system supports sparse files
FILE_VOLUME_QUOTAS | The file system supports disk quotas
FS_CASE_IS_PRESERVED | The file system preserves the case of file names when it places a name on disk
FS_CASE_SENSITIVE | The file system supports case-sensitive file names
FS_FILE_COMPRESSION | The file system supports file-based compression
FS_FILE_ENCRYPTION | The file system supports the Encrypted File System (EFS)
FS_PERSISTENT_ACLS | The file system preserves and enforces ACLs; for example, NTFS preserves and enforces ACLs and FAT does not
FS_UNICODE_STORED_ON_DISK | The file system supports Unicode in file names as they appear on disk
FS_VOL_IS_COMPRESSED | The specified volume is a compressed volume; for example, a DoubleSpace volume

Q Now that you're a C# guru (as well as a C++ guru), I have a question. How can I modify the system menu? In C++, I used the GetSystemMenu function, but I can't figure out how to do it in C#.

Philippe Morvan

A Hey, Philippe—I don't get to call myself a C# guru until I've had at least 10 years experience, and C# hasn't been around that long! However, I do know the answer to your question: use GetSystemMenu. That's right—just do it the same old way you would in C++. How's that? By using interop, of course.

Sometimes I feel like a bit of a broken record because so many of the C# questions I get have the same answer: use interop. That's because I mostly get GUI questions and Windows Forms currently exposes only a basic subset of Windows. As soon as you want to do something sophisticated, you have to fall back on Win32®. Fortunately, the Microsoft .NET Framework interop services make it easy.

If there was a way to get the system menu using Windows Forms, the place to look would be in the Form class for a property called something like SystemMenu. Alas, there's no such property. Control has Control.ContextMenu for the context menu, and Form has Form.Menu for the main Menu, but there's no SystemMenu or any other property to access the system menu directly using Menu. That's why you need interop. I wrote a little program, SysMenu, that shows how. Figure 5 shows the code. Figure 6 shows it running with the modified system menu displayed.

Figure 5 SysMenu (Close Figure 5)
////////////////////////////////////////////////////////////////
// MSDN Magazine — January 2004
// If this code works, it was written by Paul DiLascia.
// If not, I don't know who wrote it.
// Compiles with Visual Studio .NET on Windows XP. Tab size=3.
//
using System;
using System.Diagnostics;
using System.Windows.Forms;
using System.Runtime.InteropServices;
using TraceWin;

//////////////////
// This class shows how to add items to the system menu in C#/.NET
//
public class SysMenu : Form
{
   // menu flags (from WinUser.h)
   public enum MenuFlags {
      MF_INSERT = 0x00000000,
      MF_CHANGE = 0x00000080,
      MF_APPEND = 0x00000100,
      MF_DELETE = 0x00000200,
      MF_REMOVE = 0x00001000,
      MF_BYCOMMAND = 0x00000000,
      MF_BYPOSITION = 0x00000400,
      MF_SEPARATOR = 0x00000800,
      MF_ENABLED = 0x00000000,
      MF_GRAYED = 0x00000001,
      MF_DISABLED = 0x00000002,
      MF_UNCHECKED = 0x00000000,
      MF_CHECKED = 0x00000008,
      MF_USECHECKBITMAPS = 0x00000200,
      MF_STRING = 0x00000000,
      MF_BITMAP = 0x00000004,
      MF_OWNERDRAW = 0x00000100,
      MF_POPUP = 0x00000010,
      MF_MENUBARBREAK = 0x00000020,
      MF_MENUBREAK = 0x00000040,
      MF_UNHILITE = 0x00000000,
      MF_HILITE = 0x00000080,
   }

   // WM_SYSCOMAND value from WinUser.h
   const int WM_SYSCOMMAND = 0x0112;

   // Windows API fns imported
   [DllImport("user32.dll")]
   private static extern IntPtr GetSystemMenu(IntPtr hwnd, int bRevert);

   [DllImport("user32.dll")]
   private static extern bool AppendMenu(IntPtr hMenu,
      MenuFlags uFlags, uint uIDNewItem, String lpNewItem);

   // My new command ID.
   const int IDC_MYCOMMAND = 100;

   public SysMenu()
   {
      this.Text = "Check out the system menu, dude.";
      Debug.Listeners.Add(new TraceWinListener());

      IntPtr hSysMenu = GetSystemMenu(this.Handle, 0);  // get handle to
                                                        // system menu
      AppendMenu(hSysMenu,MenuFlags.MF_SEPARATOR,0,null);
      AppendMenu(hSysMenu,
         MenuFlags.MF_BYCOMMAND|MenuFlags.MF_STRING|MenuFlags.MF_CHECKED,
         IDC_MYCOMMAND,
         "Do you like interop?");
   }

   [STAThread]
      static void Main()
   {
      Application.Run(new SysMenu());
   }

   protected override void WndProc(ref Message msg)
   {
      if (msg.Msg==WM_SYSCOMMAND) {
         if(msg.WParam.ToInt32() == IDC_MYCOMMAND) {
            Trace.WriteLine("Got IDC_MYCOMMAND");
            MessageBox.Show("Yeah, baby!");
            msg.Result = IntPtr.Zero; // (not really necessary)
            return;
         }
      }
      base.WndProc(ref msg);
   }
}
Figure 6 Modified System Menu
Figure 6 Modified System Menu

To use GetSystemMenu, first declare it the interop way, using DllImport. For SysMenu, you actually need two functions: GetSystemMenu and AppendMenu.

using System.Runtime.InteropServices;
public class Form1 : Form
{
  [DllImport("user32.dll")]
  private static extern IntPtr GetSystemMenu(IntPtr hwnd, int bRevert);
  [DllImport("user32.dll")]
  private static extern bool AppendMenu(IntPtr hMenu,
    MenuFlags uFlags, uint uIDNewItem, String lpNewItem);
}

You should always use IntPtr for HWNDs, HMENUs, and any other kind of Windows handle. For LPCTSTRs, declare the argument as String. The interop services will automatically marshal your System::String to LPCTSTR before passing it to Windows. As for MenuFlags, that's an enum you must define yourself:

public enum MenuFlags {
  MF_INSERT = 0x00000000,
  MF_CHANGE = 0x00000080,
    ••• // etc
}

You don't have to use an enum, but it's more type safe. The MF_XXX values come from WinUser.h. Finally, you need an ID for your new command. In SysMenu, IDC_MYCOMMAND has the value 100. If you use values below 0xF000, you're guaranteed not to conflict with SC_MINIMIZE, SC_MAXIMIZE, or any other built-in system commands. Make sure you don't conflict with your own main menu commands, either.

With all these definitions in place, you're ready to add your menu item. All it takes is a few lines in your Form's constructor. First, get the system menu

// Get system menu
IntPtr hSysMenu = GetSystemMenu(this.Handle, 0);
then add your command:
// Add separator and new command
AppendMenu(hSysMenu,MenuFlags.MF_SEPARATOR,0,null);
AppendMenu(hSysMenu,
  MenuFlags.MF_BYCOMMAND, IDC_MYCOMMAND,
  "Do you like interop?");

Now when the user clicks the system menu in the window's title bar, your new menu item appears, as in Figure 6. Just for fun, I gave it a checkmark. But what happens if the user invokes your command? Right now, nothing. To handle the command, you have to override your Form's virtual WndProc method.

const int WM_SYSCOMMAND = 0x0112;
protected override void WndProc(ref Message msg)
{
  if (msg.Msg==WM_SYSCOMMAND) {
    if (msg.WParam.ToInt32() == IDC_MYCOMMAND) {
      // handle it!
      return;
    }
  }
  base.WndProc(ref msg);
}

Whatever you do, don't forget to call base.WndProc if the message isn't yours. Otherwise your app will go home in a box.

Well, that's it for today. As usual, you can download the source for all programs described here from the MSDN® Magazine Web site. Happy Programming!

Send your questions and comments for Paul to cppqa@microsoft.com.

Paul DiLascia is a freelance writer, consultant, and Web/UI designer-at-large. He is the author of Windows++: Writing Reusable Windows Code in C++ (Addison-Wesley, 1992). Paul can be reached at http://www.dilascia.com.


Talking To...: Carl Franklin Talks About Being a New MVP and His Passion for Visual Basic .NET

Talking To...
Carl Franklin Talks About Being a New MVP and His Passion for Visual Basic .NET

.NET Rocks!, an online audio show created and co-hosted by Carl Franklin, was just picked up by MSDN as its first third-party on-air program. Besides being a great proponent of Visual Basic .NET, Carl is a new member of the Microsoft Most Valuable Professionals (MVP) program. MSDN Magazine recently spoke to Carl about being an MVP and his thoughts on the future of software development.

MSDN How did you get started in the computer industry? Did you study computer science in college?

Franklin I began by working a summer tech support job at Crescent Software, one of the first companies to offer third-party custom controls for Visual Basic 1.0. I had time between calls to learn Windows programming from Charles Petzold's book Programming Windows and from the API reference. I started writing tools for Crescent. At the same time, I began writing a Q&A column in BasicPro Magazine (later renamed Visual Basic Programmer's Journal) and started writing feature columns and speaking at VBITS.

MSDN What were you working on that led to the title of MVP?

Franklin Apparently I was nominated because of my .NET Rocks! MP3 talk show (http://msdn.microsoft.com/dotnetrocks or http://www.franklins.net/dotnetrocks.asp). I had co-created Carl & Gary's Visual Basic Home Page, one of the first sites created outside of Microsoft for Visual Basic, which lasted several years. I guess Microsoft measures your overall commitment to the community you serve.

MSDN What do you do as an MVP?

Franklin The MVP program is designed to recognize people who give of their time to the community in support of a Microsoft product. So, as long as I keep doing my shows, speaking at user groups and conferences, and teaching Visual Basic .NET, I have a shot at staying an MVP.

MSDN What has MVP status done for you professionally?

Franklin I've only recently become an MVP, so it's definitely the result of my professional experience. I must say, though, I'm very impressed with how Microsoft runs the MVP program. They go out of their way to give MVPs access to people inside Microsoft. That's a wonderful thing for everyone involved.

MSDN How do you keep up with software developments?

Franklin Fortunately, I am really connected to Microsoft. I'm an MSDN Regional Director, of which there are fewer than 200 worldwide. We have a great mailing list where we share knowledge and new discoveries. Also, I read the blogs. I recently joined the blogging community and I'm hooked (see my blog at http://weblogs.asp.net/CFranklin). Blogs are a great way to keep up with things as they happen in the whole community.

MSDN What are your favorite problems to solve?

Franklin I really like communications in general and I've always liked programs that communicate across phone lines or networks. I enjoy thinking up new ways to plug all these technologies together to make something spectacular. Recently I've been writing Web Services to serve up Web statistics for my show to an auto-deployed Windows Forms app. I just love .NET! As a musician, I'm also interested in MIDI. Right now I'm working with a local artist on a project that incorporates the Internet, MIDI, instruments, lights, and various controllers.

MSDN There are many new products coming out in 2004. What was the most unexpected for you?

Franklin I was blown away when developers on the Visual Basic .NET team told me about the new features of Visual Basic .NET, some of which I did not expect to see, namely, XML comments (the ability to generate documentation and IntelliSense from comments above methods and properties), support for unsigned types, generics (which can't be explained in one sentence), and Edit and Continue. I think people will take a second look at the new Visual Basic .NET. It is essentially version 3.0 of the product.

MSDN Where do you see the industry in the next five or ten years?

Franklin I see more and more companies deploying the .NET Framework across their enterprise through a broad variety of means. I believe that Web-based applications will continue to become less attractive in the enterprise, giving way to a wave of development of auto-deployed Windows-based applications that use Web Services and remoting. Of course, Web Services will become the dominant technology as they get better, faster, and more robust. Longhorn applications written with XAML will be like a hybrid of Windows and Web-based applications, which should satisfy many Web developer's concerns about the rigidity of the Windows Forms interface.

I also believe that Whidbey will be seen as the definitive reason for programmers to move to the .NET Framework. With the new language and debugging features (like Edit and Continue), this is the Visual Basic .NET we wished we had a few years ago. It really feels like Visual Basic 6.0 with a much richer environment and true object-oriented features.