 MSDN Magazine July 2006

MSDN Magazine July 2006

Atlas At Last: ASP.NET Atlas Powers the AJAX-Style Sites You’ve Been Waiting For

Matt Gibbs Code & Seek: Bring Windows Desktop Search Into Visual Studio With Our Cool Add-In

Sergey Mishkovskiy Beyond WinFX: Transactions, Aero Wizards, And Task Dialogs In Windows Vista

Kenny Kerr Web App Follies: Keep Sites Running Smoothly By Avoiding These 10 Common ASP.NET Pitfalls

Jeff Prosise WSS 3.0 Preview: Discover Significant Developer Improvements In SharePoint Services

Ted Pattison Patterns & Practices: Speed Development With Custom Application Blocks For Enterprise Library

Mark Seemann Editor's Note: Get Ready for AJAX

If you’re fortunate enough to attend a developer conference this year, look around at the variety of people. There are all kinds out there, a diverse ecosystem as the marketing folks might say. Even within a relatively narrow group like programmers, you’ll find a wide range of goals and interests.

Toolbox: Networking, Rich UIs, and More

End users today expect a level of interconnectivity between applications that was unheard of a mere five years ago. Many applications periodically determine if a new version is available and, if so, offer to download and install it.

Scott Mitchell Advanced Basics: Monitor Your Apps with System.Diagnostics

It never fails. The application you just deployed ran great on your development machine—but stumbles in production. The problem might show up right away or maybe it creeps up over time. Now what?

Brad McCabe CLR Inside Out: Using Strong Name Signatures

Strong name signatures (and signing in general) are a key facet of Microsoft® . NET Framework security. But regardless of how well designed . NET signatures may be, they won’t offer the maximum benefit if you don’t know how to use them properly.

Mike Downen Data Points: Report Controls in SQL Server 2005 Reporting Services

Reporting has always been one of the dark arts of development. The tools typically seem to do just enough to get you to a certain point, then leave you to find workarounds to solve more complex issues.

John Papa Cutting Edge: Context-Sensitive PictureBox Controls

Great ideas are timeless. A long time ago in Microsoft Systems Journal Paul DiLascia demonstrated a neat trick to display context-sensitive tooltips floating over pictures. As the user moved the mouse over the picture, the tooltip control updated its text to reflect the name of the pointed figure.

Dino Esposito Inside MSDN: Building the MSDN Aggregation System

Have you visited msdn2. microsoft. com? It’s the new online face of the MSDN® Developer Tools and Enterprise Server documentation. The infrastructure behind it includes a system developed by my team at Microsoft for aggregating information related to our content.

John Mollman Extreme ASP.NET: Asynchronous Web Parts

Building a customizable Web site complete with a collection of pluggable Web Parts is fairly easy with the portal infrastructure of ASP. NET 2. 0. This model is very flexible, allowing users to easily place your Web Parts anywhere on the Web page so they are free to customize your site.

Fritz Onion C++ at Work: Controlling Balloon Tips, and More

Paul DiLascia End Bracket: Refactor for Life

There are certain fundamental laws of the universe that we just can’t escape. One of these is the Second Law of Thermodynamics as it applies to the general branches of science. It says that everything moves to a state of equilibrium and an increase in disorder, or entropy.

Eric N. Bush


Atlas At Last: ASP.NET Atlas Powers the AJAX-Style Sites You’ve Been Waiting For

Atlas At Last

ASP.NET Atlas Powers the AJAX-Style Sites You’ve Been Waiting For

Matt Gibbs
This article is based on a prerelease version of ASP.NET "Atlas." All information contained herein is subject to change.
This article discusses:
  • Introduction to ASP.NET “Atlas”
  • Atlas architecture
  • Client and server controls
  • Atlas and Web services
This article uses the following technologies:
ASP.NET, Visual Studio 2005, Internet Explorer
Contents
What is Atlas?
The Atlas Architecture
The Client Script Core Library
Client Script Controls and Components
Server Controls
Web Services
Conclusion

I
n September 2005, the ASP.NET team released the first Community Technology Preview (CTP) of the new features in ASP.NET code-named "Atlas." This extension to the Microsoft® .NET Framework 2.0 enables developers to more easily create rich, interactive Web sites that take advantage of both browser and server features.

The type of rich development that Atlas enables is broadly referred to as AJAX (Asynchronous JavaScript and XML), which is a relatively new acronym for a combination of technologies that have been around for quite some time. Modern browsers include the XMLHttpRequest object that can be used from JavaScript for making calls back to the server. This allows the page to react to user input and perform out-of-band operations without requiring that the whole page be refreshed. This concept is generally simple, but AJAX libraries can greatly alleviate the arduous task of writing client-side JavaScript to communicate with the server and deal with the XML that is returned from a Web service.

The general problem that AJAX attempts to solve is rooted in the HTTP protocol itself. HTTP is the standard used by browsers to communicate with Web servers to retrieve pages and post data back from them. The protocol is stateless, which means that preserving user input between page refreshes is up to code on the server. The typical user experience has been that the entire page gets refreshed to carry the state information back to the server. The user input on the page is then restored in the HTML sent back to the browser.

ASP.NET manages this process for you by carrying a hidden view state form field. Even though only part of the page may actually be getting updated, the entire page’s HTML is transmitted and the screen blanks out momentarily. While this refresh occurs, the user is not able to interact with the page until the new view is received and rendered by the browser. AJAX improves this experience by using the XMLHttpRequest object to make calls back to the server to invoke Web services without refreshing the entire page. The portion of the page being updated is then modified directly in JavaScript based on the XML that is received. The user may not even perceive that a page update is occurring and can continue reading or interacting with the page while the out-of-band work is happening asynchronously in the background.

What is Atlas?

The Atlas features of ASP.NET are not just another AJAX script library for writing client-centric Web applications. Atlas builds on the .NET Framework 2.0 and adds support for better utilizing the capabilities of client-side JavaScript and the XMLHttpRequest object. It includes server-based features that make it easy to enrich existing ASP.NET applications, as well as a client script library that is used by the Atlas controls and services. The architecture diagram in Figure 1 shows that the Atlas architecture extends across client and server and should be viewed as a broad set of development technologies for creating richer, more responsive cross-browser Web applications.

Figure 1 ASP.NET Atlas Architecture
Figure 1 ASP.NET Atlas Architecture

Scenarios enabled by Atlas are not limited to updating regions of the page with an asynchronous JavaScript call. You also get richer client experiences that would otherwise be impractical. For example, consider a Web application built around movie data. The application might want to allow users to search for a specific actor. Obviously it wouldn’t be practical to provide a comprehensive dropdown list of all actor names to choose from, so the application would probably break the problem down. Users might be asked to select the first letter of the actor’s name. A request to the server could then provide a list that would be somewhat more manageable, but the user experience is not great. The application could present the user with a textbox to enter in part of the actor’s name. The server would then at least have some data to use in narrowing the search. This is better, but still leaves room for improvement. With Atlas you could provide a textbox that reacts dynamically as the user types to help narrow the search without waiting for the browser to refresh the whole page. Figure 2 shows how you could use Atlas to add autocomplete behavior that provides feedback as the user types.

Figure 2 Filtering Combobox
Figure 2 Filtering Combobox

The Atlas CTP is available as a download from atlas.asp.net. When installed, it adds an additional Web Site template for C# and Visual Basic® .NET to Microsoft Visual Web Developer™. When you create a new Web site project in Visual Web Developer (click File, then New, then Web Site), you will see a dialog like the one shown in Figure 3. The Atlas Web site includes Microsoft.Web.Atlas.dll and an updated web.config file that configures the Web application to be able to use Atlas-based ASP.NET features. In the current release, Microsoft.Web.Atlas.dll is placed in the bin directory of the application as a local assembly available throughout the application.

Figure 3 Creating an Atlas Web Site
Figure 3 Creating an Atlas Web Site

Atlas-based applications can be easily deployed by copying the files from a development machine to a server with ASP.NET 2.0 without worrying about separately installing Atlas. The installation is at the application level instead of machine level. As subsequent CTP releases become available, they can be used on a machine with older versions of Atlas even as the features evolve and change. This provides more flexibility in migrating to newer versions than if the installation were systemwide.

The Atlas Architecture

The first thing to notice about the Atlas architecture diagram shown in Figure 1 is that it spans both client and server. ASP.NET 2.0 added some additional client features but not to the extent that Atlas does. On the right side of the architecture diagram, notice that Atlas server features are built on top of ASP.NET 2.0 and extend its capabilities. Atlas includes a new set of server controls as well as new features aimed at accessing server-based data and services from the browser.

On the left you’ll see a comprehensive client script library, which can be used independently of the server features in writing client-centric JavaScript. It is the foundation on the client that is used heavily by the new Atlas features to enable richer application development with improved client-server interactions.

In Figure 4 you can see the typical client-server interaction in a Web application. First, a page is requested by the browser and the user interacts with it. When some action from the user requires data from the server, a complete page refresh occurs to update part of the page based on their input. Unfortunately, this behavior does not allow the user to continue interacting with the page. Instead, the user must pause continuously while working with the Web application.

Figure 4 Typical Client-Server Interaction
Figure 4 Typical Client-Server Interaction

Figure 5 shows that client-server interaction with Atlas shifts away from complete page refreshes. Instead, the initial HTML is retrieved and subsequent calls to the server get updated data in XML, JavaScript Object Notation (JSON), or snippets of HTML to update the page incrementally. The background asynchronous calls can invoke Web services or retrieve page changes without leaving users feeling that they must pause. These asynchronous calls manage updated view state information for subsequent server postbacks so that when a complete page refresh is necessary, the accurate state of the page is carried to the server.

Figure 5 Atlas Client-Server Interaction
Figure 5 Atlas Client-Server Interaction

The Client Script Core Library

The Atlas client script library is delivered to the browser as several distinct pieces. The script core comprises the bottom layers on which the rest of the library is built. At the base is the browser compatibility layer. A key feature of Atlas is that it will run on the modern browsers that support the key elements of AJAX. In the CTP builds, this support has included Mozilla Firefox, Apple Safari, and Microsoft Internet Explorer®. The browser compatibility layer is an abstraction that allows you to write script with less worry. It hides the details of where the browser implementations differ. It also allows for Atlas support to be easily augmented as the browsers evolve and as new versions are released. The type of browser making a request automatically determines what browser-specific parts of the compatibility layer are used. The higher-level code has been written to the abstraction layer, so you don’t need to code around variations in browser implementations.

On top of the compatibility layer is the core type system. The type system allows for an object-oriented approach to JavaScript development. It gives the developer working in JavaScript the ability to create namespaces and add classes to them. It also simulates object inheritance. There is support for interfaces, delegates, and enumerations, which makes it easier to switch back and forth between developing code on the server in an object-oriented programming language like C# and writing JavaScript code on the client.

The base class library layer built on top of the type system completes the core of the client script library. The concept is borrowed from the .NET Framework and exposes some familiar types. There is an Event object that supports multicasting of events in a natural way in JavaScript. There is a StringBuilder object, too. There is also support for object serialization, including support for JSON and XML data. The base class library also includes WebRequest and WebResponse classes that provide an abstraction over the browser’s XMLHttpRequest object similar to those found in the System.Net namespace in the .NET Framework. The code in Figure 6 demonstrates creating two simple types in JavaScript using the Atlas script core. The Movie type that is created first exposes properties for the movie title and its genre as well as a toString method. The Drama type then extends the Movie type and overrides the toString method.

Figure 6 Using the Atlas Type System (Close Figure 6)
Type.registerNamespace(‘AtlasSample’);

AtlasSample.Movie = function(title, genre) {
    var _title = title;
    var _genre = genre;

    this.get_title = function() {
        return _title;
    }
    this.get_genre = function() {
        return _genre;
    }
    this.toString = function() {
        return this.get_title() + " : " + this.get_genre();
    }
    AtlasSample.Movie.registerBaseMethod(this, ‘toString’);
}
AtlasSample.Movie.registerClass(‘AtlasSample.Movie’);

AtlasSample.Drama = function(title, year) {
    AtlasSample.Drama.initializeBase(this, [title, ‘drama’]);

    var _year = year;

    this.get_year = function() {
        return _year;
    }
    this.toString = function() {
        return AtlasSample.Drama.callBaseMethod(this, ‘toString’) +
            ‘ : ‘ + this.get_year();
    }
    AtlasSample.Drama.registerBaseMethod(this, ‘toString’);
}
AtlasSample.Drama.registerClass(‘AtlasSample.Drama’, AtlasSample.Movie);

The page that uses the Movie and Drama types is shown in Figure 7. It first references the .js file where the types are defined in an Atlas ScriptManager control. Then, in the Click handlers, the page creates instances of the Movie and Drama types and calls their toString methods. The inheritance behavior is what you would expect to see from any object-oriented programming language, but the code that is running happens to be dynamic JavaScript. Another nice aspect of the current Atlas installation is that debug versions of the client script library are included to make debugging and troubleshooting easier. Debugging JavaScript has always been cumbersome, so this may ease the pain.

Figure 7 Inheritance Demonstration (Close Figure 7)
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" >
<head id="Head1" runat="server">
    <title>Atlas Type System</title>
</head>
<body>
    <form id="form1" runat="server">
        <atlas:ScriptManager runat="server" ID="scriptManager">
            <Scripts>
                <atlas:ScriptReference Path="drama.js"
                     ScriptName="Custom" />
            </Scripts>
        </atlas:ScriptManager>
        <div>
            <input id="btn1" value="Show Movie Data" type="button"
             onclick="return Click_ShowMovie()" />
            <input id="btn2" value="Show Drama Data" type="button"
             onclick="return Click_ShowDrama()" />
        </div>
    </form>
    <script type="text/JavaScript" language="JavaScript">
        function Click_ShowMovie() {
            var aMovie = new AtlasSample.Movie(‘Ray’, ‘drama’);
            alert(aMovie.toString());
        }
        function Click_ShowDrama() {
            var aDrama = new AtlasSample.Drama(‘Ray’, ‘2005’);
            alert(aDrama.toString());
        }
    </script>
</body>
</html>

Client Script Controls and Components

Above the layers that make up the client script core of the Atlas architecture are the component model and controls layers. This part of the script library builds on top of the script core underneath it, but is rendered to the client separately. You could choose not to include the component layer and instead just use the JavaScript type system and base class library directly when writing script, but you would not have access to the client components provided by Atlas and would not be able to use xml-script, which is a new set of declarative elements contained in the page markup that is sent to the browser. The xml-script elements are contained in a script tag that uses the new type value:

<script type="text/xml-script">

The key to enabling an additional set of elements in the markup is the use of the script tag. The browser recognizes the script element but does not have a way to handle the text/xml-script type. The Atlas script library is then able to deal with the elements contained in the script tag itself. The markup is handled by the component layer of the client script library. The xml-script is parsed on the client to create instances of components and controls. It can contain property settings for the components and controls it defines and can declare bindings between them and HTML elements elsewhere on the page. The xml-script elements can also declare Web service resources and then reference them as data sources elsewhere in the markup. The example page in Figure 8 shows the use of xml-script to declaratively establish that the movie name associated with a year is displayed as a popup element when the mouse pointer is hovered over the year.

Figure 8 Page Showcasing Pop-Up Behavior (Close Figure 8)
<%@ Page Language="C#" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" >
<head id="Head1" runat="server">
  <title>Hover Example</title>
  <atlas:ScriptManager ID="ScriptManager1" runat="server" />
</head>
<body>
  <form id="form1" runat="server">
  <div>
  <h2>Movie of the Year</h2>
  <div id="popup2004" width="30">2004
    <span id="movieName2004" style="visibility: hidden;
     display: none; border: solid 1px black;
     background: cyan;">Million Dollar Baby</span>
  </div>
  </form>
  <script type="text/xml-script">
    <page xmlns:script="http://schemas.microsoft.com/xml-script/2005"
     xmlns:samples="samples">
      <components>
        <control id="movieName2004">
          <behaviors>
            <popupBehavior id="popupBehavior2"
             parentElement="popup2004"
             positioningMode="BottomLeft" />
          </behaviors>
        </control>
        <control id="popup2004">
          <behaviors>
            <hoverBehavior hoverElement="movieName2004">
              <hover>
                <invokeMethod target="popupBehavior2"
                 method="show" />
              </hover>
              <unhover>
                <invokeMethod target="popupBehavior2"
                 method="hide" />
              </unhover>
            </hoverBehavior>
          </behaviors>
        </control>
      </components>
    </page>
  </script>
</body>
</html>

The page in Figure 8 contains a DIV element to display a year, and a SPAN element to display the movie name, but declares it as being hidden. In the xml-script, a popupBehavior is associated with the name and invoked by a hoverBehavior associated with the year. The code for the popupBehavior is contained in the component and control layer of the Atlas script library. The xml-script is easy to examine compared with poring over the JavaScript that would typically be included in a page, particularly when you start factoring in code to deal with multiple browser implementations. The declarative syntax like the xml-script in Figure 8 can be produced and consumed easily by development tools. And the Atlas server controls produce xml-script to enable a richer user experience when the page executes. Much of the xml-script used in an Atlas application will never exist directly in the .aspx file, nor will most developers have to code it by hand.

The variety of behaviors that are part of the Atlas CTP serve to improve the user experience. The progress behavior can provide info that an operation is pending in the background. The click, hover, and popup behaviors allow for rich user interaction. The behaviors can easily be attached to HTML elements on the page in a declarative way using xml-script. Because the behaviors themselves are implemented in JavaScript, more complex behaviors are possible, but their use across pages can be through xml-script.

Server Controls

The server controls included in the Atlas CTP make it easy to avoid the pause that accompanies a page postback. Users can continue interacting with the page while controls update their rendering in the background. Two server controls work together to make this possible. They can be added to existing pages to achieve dramatically improved results. The ScriptManager control works to modify the client postback behavior while the UpdatePanel control manages the page lifecycle on the server to achieve the change.

The ScriptManager control must be included on all pages that will use Atlas features. It coordinates what JavaScript is sent to the client. Server controls can provide JavaScript for the client and utilize the ScriptManager control for handling it. The ScriptManager control will utilize the new IScriptComponent interface that the control implements. The ScriptManager control also provides the support for the component script libraries that are associated with xml-script elements.

Setting the EnablePartialRendering property of the ScriptManager control to true results in new behavior for postbacks from the client:

<atlas:ScriptManager EnablePartialRendering="true" runat="server" />
The postback behavior is modified to allow a request to occur without interrupting the user’s experience. The view state information required to persist control information between requests is maintained for the partial rendering requests. And the HTML for the region being refreshed or modified is updated by JavaScript interacting with the browser Document Object Model (DOM). The regions on the page that should allow for partial updates are specified in the ASP.NET page using the UpdatePanel control.

The UpdatePanel control indicates to the ScriptManager control what regions on the page should be updated independently of the rest of the page. When an action in the browser causes a postback for that region of the page, the form data is posted and the page lifecycle begins to execute on the server. Because the script is initiating the postback asynchronously behind the scenes, the page is still displayed to the user. On the server, the control state is restored from view state data posted by the client. When the render phase occurs, the ScriptManager control isolates the rendering for the UpdatePanel region whose rendering is being refreshed to send back to the browser. The view state data for the page is also gathered and accompanies the HTML as part of the response. The script in the browser then replaces the HTML corresponding to the previous rendering of the UpdatePanel contents with the new HTML.

The UpdatePanel control can contain elements for Triggers and for a ContentTemplate:

<atlas:UpdatePanel ID="UpdatePanel1" runat="server">
<Triggers>
        ...
    </Triggers>
<ContentTemplate>
        ...
</ContentTemplate>
</atlas:UpdatePanel>
The area inside the ContentTemplate is refreshed when the ScriptManager control manages an asynchronous postback. The Triggers element can contain ControlEventTrigger and ControlValueTrigger elements. The trigger elements allow a page developer to specify specific changes that should cause the region to update. This allows controls outside of the UpdatePanel control to cause the change, without having to be contained within the UpdatePanel control directly. It also allows for using simple declarations to control the behavior of the page and the UpdatePanel control and to dictate when new data is retrieved.

Multiple UpdatePanel controls can be placed on a single page with different triggers causing them to update independently. The contents of the UpdatePanel control can be scoped to the minimum necessary to respond to specific user input. Using the UpdatePanel control, an existing ASP.NET page can quickly be made to feel more responsive without significant change.

Web Services

Web applications are built around service-oriented architectures. Being able to access services from the browser is central to enabling interactive applications. Two distinct types of services are enabled by Atlas. The ScriptManager control will use automatically generated proxies for Web service references:

<atlas:ScriptManager EnablePartialRendering="true" runat="server">
    <Services>
        <atlas:ServiceReference GenerateProxy="true"
         Path="~/nominees.aspx" Type="Custom"/>
    </Services>
</atlas:ScriptManager>

Client-side components can then call the Web services directly from script. Web services are also bound to controls to enable richer behaviors. For example, AutoCompleteBehavior can be defined in xml-script to use a Web service to retrieve the relevant possibilities (see Figure 9).

Figure 9 AutoCompleteBehavior in xml-script (Close Figure 9)
<script type="text/xml-script">
  <page xmlns:script="http://schemas.microsoft.com/xml-script/2005">
    <components>
      <textBox id="suggestTextBox">
        <behaviors>
          <autoComplete
           completionList="completionList" serviceURL="suggest.asmx"
           serviceMethod="GetSuggestions" minimumPrefixLength="1"
           completionInterval="500" completionSetCount="15" />
        </behaviors>
      </textBox>
    </components>
  </page>
</script>

The behavior is attached to an element on the page and augments its behavior. When established in the .aspx markup, it is referred to as an extender. AutoCompleteBehavior can be associated with an element by using an AutoCompleteExtender control. Instead of writing the xml-script directly, the extender is associated with a control on the server. The control behavior is then extended by rendering the appropriate xml-script to get the client-side behavior. When calling Web services, the call and the return results are typically carried in XML. Atlas also supports serializing data from Web services in JSON, which eliminates some of the overhead inherent in XML. The JSON data can be deserialized directly into JavaScript objects in the browser. Atlas supports serializing more complex .NET managed types from the server into representative JavaScript objects on the browser. This simplifies the task of accessing Web services from the browser.

The idea of using Web services from the browser extends from custom Web services that are part of the application to ASP.NET application services. Atlas exposes the ability to use the Forms Authentication service directly from JavaScript:

Sys.Services.AuthenticationService.login(
    username, password, completionFunction);
The user doesn’t have to be redirected to a login page and then back to the original page; the HTML can then be dynamically altered when the user supplies login credentials. The Profile data available in .aspx pages is also available through a Web service call. It supports storing and retrieving profile data that is stored on the server through a JavaScript object.

The Web services that an application might use are not always located on the same host server. In fact, they are not all necessarily even in the same domain. The browser will block calls using the XmlHttpRequest to a domain other than where the page originated. There are some clever ways to circumvent this limitation using hidden IFrame objects to initiate child requests, but they are cumbersome at best. Atlas provides Web services bridging to enable this scenario. The client can initiate a Web service call destined for another domain. The call is sent to the originating Atlas application which then proxies the request to the target server for the user and serializes the results back to the client. Atlas is also capable of using the IFrame techniques to communicate directly with other domains.

Conclusion

Atlas provides a range of features for creating richer Web applications. The client script library simplifies the task of writing JavaScript and provides constructs for using an object-oriented approach to writing JavaScript. The Web services features allow easy access to remote and local services. Serialization of complex types makes it easy to utilize rich types from both client and server. Server controls take advantage of the client script library and allow existing and new applications to greatly reduce the busy-wait pauses common in Web applications today.

A new CTP build has been released about every other month with fixes, changes, and new features. Ultimately, Atlas will be integrated into the next release of the .NET Framework with design-time support in Visual Studio. In the near term, Microsoft has announced a limited license that allows you to deploy Atlas on live sites and start taking advantage of it in your Web applications. See atlas.asp.net for more information and to download the latest Atlas CTP.

Matt Gibbs is the Development Manager of the ASP.NET team at Microsoft. He has coauthored several books on ASP, Mobile Controls, and ASP.NET. He can be reached at mattgi@microsoft.com.

© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


Code &amp; Seek: Bring Windows Desktop Search Into Visual Studio With Our Cool Add-In

Code & Seek

Bring Windows Desktop Search Into Visual Studio With Our Cool Add-In

Sergey Mishkovskiy
This article discusses:
  • Creating a Visual Studio add-in
  • Creating an add-in tool window
  • Developing with Windows Desktop Search
This article uses the following technologies:
Visual Studio 2005, Windows Desktop Search SDK
Code download available at: WDS2006_07.exe (255KB)
Contents
Creating the Add-in
Creating the Tool Window
Options Dialog Page
Windows Desktop Search SDK
Background Queries and UI Updates
Putting It All Together
Installation and Deployment
Conclusion

A
s computer hard drives get larger, so does the amount of information you hold onto. You end up with thousands of files and e-mail messages to plow through to find just the information you need. Thankfully, Windows® Desktop Search can help.

Windows Desktop Search is very straightforward. It indexes all of your documents, files, and e-mail messages, and any other data provided to it through extensibility mechanisms. As new items are added and changes are made to existing items, they're reindexed. The UI lets you search for strings and keywords in indexed items and open any found item with the associated app.

Windows Desktop Search provides an SDK that allows for its indexes and searching functionality to be consumed by other applications. As a developer, I spend a lot of time in Microsoft® Visual Studio®, and I spend a lot of time hunting around in my source code for material relevant to my development task at hand. For example, I'll search for other code snippets I previously authored using a particular class or e-mail conversations I had with a colleague concerning a certain collection's usage. By writing an add-in for Visual Studio that communicates with Windows Desktop Search, it's possible to construct a Dynamic Help-like tool window that uses Desktop Search to show relevant search results for the currently active text in Visual Studio. In this article, I'll show you how to build such an add-in. Download the code for this issue so you can follow along.

The project demonstrates two technologies: building an add-in for Visual Studio 2005 using extensibility interfaces, and integrating the add-in with Windows Desktop Search.

Visual Studio add-ins that present data in a non-modal fashion to the user normally consist of a tool window, at least one menu command, an Options dialog box to provide configuration alternatives, and an About dialog box. I want my add-in tool window to include Windows Desktop Search location filters, such as Everything, Files, and E-mail. I want the add-in to query automatically based on the selected text in the Visual Studio editor (see Figure 1), but also to include an option that turns this auto-refresh feature off. I want the query results to be displayed in a list view and to open with their associated applications when users double-click. Finally, I want to provide configuration capabilities through the Options dialog box. Let's first build the add-in and then move on to Windows Desktop Search integration.

Figure 1 Search Based on Select Text
Figure 1 Search Based on Select Text

Creating the Add-in

Creating the add-in is a very straightforward process. Select the File | New | Project menu item. In the New Project dialog box, choose Other Project Types, then click Extensibility. Under Templates, select Visual Studio Add-in, give the new add-in a name, and click OK. The Visual Studio Add-in Wizard walks you through the rest. Choose C# as the programming language and Visual Studio 2005 as the host, add a name and description, then check the option to create a Tools menu item and add information for an automatically created About box.

After clicking the Finish button, you'll end up with an add-in skeleton to work with. Note that the project generator adds IDTExtensibility2 and IDTCommandTarget interface implementations for you, as well as a design time environment (DTE) extensibility application object reference.

Most of the Visual Studio tool window-related items appear under the View menu. To match that, change the Tools menu reference in the add-in to View, update its text to Windows Desktop Search, and move the generated menu item and command code into a method called CreateCommand.

The next step is to change how the code calls the AddNamedCommand2 command that creates the View item. Visual Studio provides a default image for the new menu item, but you may prefer to use your own image. For that to work, the image must reside in a satellite assembly. With previous versions of Visual Studio you had to create a separate project for the satellite assembly. With Visual Studio 2005 all it takes is adding a resource file to the project.

Add a resource item to the add-in with the name WDSAddin.en, then insert an icon resource named 1. Compiling the add-in project with the new resource automatically generates a resource assembly under the en folder. That resource ID 1 is what you use with the AddNamedCommand2 call to assign your own menu item image. Don't forget to change the AddNamedCommand2's MSOButton parameter value to false to indicate you want to use your own image.

After adding the command, you get a Command object reference back and can assign a keyboard shortcut to it. The Command interface has a Bindings property for just that purpose. It returns a collection of shortcuts assigned for all scopes. Scope would be something like Global, Text Editor, Windows Forms Designer, and so on. Be aware that scope names are localized.

The BindCommandKey method is responsible for assigning the keyboard shortcut. It includes a try/catch block around the code for setting shortcuts. Setting it up that way suppresses an exception raised when trying to change read-only keyboard scheme settings. Ctrl+Alt+Shift+M makes a good shortcut for the tool window as it mimics the Windows Desktop Search default Ctrl+Alt+M shortcut.

Notice that the generated add-in class code implements an IDTCommandTarget interface, which tells Visual Studio whether the command is available in the given context. That's done via the QueryStatus method. If the command is determined to be available to the user, then after it's selected, the Exec method gets called. With both methods, the command name gets checked to make sure it is in fact that command that needs to be processed. You use the Exec method to show the Windows Desktop Search tool window by calling into the private Show-WDSToolWindow method. This method checks to see if the tool window reference is set. If it is, the method shows the tool window by setting its Visible property to true. Otherwise the ShowWDSToolWindow method creates and shows a new tool window.

Creating the Tool Window

You used to have to rely on C++ shim controls to host managed user controls in the tool window. And there were known tool window state persistence issues, such as saving tool window position, size, and docked state. Microsoft received a lot of questions and feedback on that and the nice folks on the Visual Studio extensibility team listened. Visual Studio 2005 introduced a new Window2 interface with a CreateToolWindow2 method that lets you host managed user controls in the tool window. The last parameter of CreateToolWindow2 returns a reference to a managed user control.

To set up a tool window, first add a user control called ToolWindowUserControl to your project, then insert a top-aligned panel to serve as a toolbar. Add a client-aligned List view below the header panel for search results. Figure 2 shows the code that creates the tool window.

Figure 2 Creating the Tool Window (Close Figure 2)
string assembly = this.GetType().Assembly.Location;
string userControlClass = typeof(ToolWindowUserControl).FullName;

Window2 appWin = (Window2)application.Windows;
string toolWindowGuid = "{AB66652C-89AD-4229-B80C-C5B87545297F}";
object control = null;

_toolWindow = appWin.CreateToolWindow2(_addin, assembly,
    userControlClass, "Windows Desktop Search",
    toolWindowGuid, ref control);
_toolWindowControl = control as ToolWindowUserControl;

Notice that the CreateToolWindow2 method expects you to provide the user control assembly location and full class name. It also needs a GUID to associate the tool window with a Window class reference. This parameter can be any GUID value, but it needs to be generated once and kept the same afterwards for the tool window state persistence to work correctly. If the CreateToolWindow2 call succeeds, you get an object reference that can be safely cast back to the tool window user control class type.

With a tool window reference on hand, you can set the picture via a SetTabPicture call and show the tool window picture by setting the Enabled property to true. The Window.SetTabPicture method expects a GDI bitmap so add another resource file named Resources, and then add an icon resource named WDSIcon to that file. Since Visual Studio generates a resource wrapper class, retrieving the resource is very simple (the tool window image must be set before making the tool window visible):

_toolWindow.SetTabPicture(Resources.WDSIcon.ToBitmap().GetHbitmap());
_toolWindow.Visible = true;

Why not reuse the View command icon for the tool window? The transparency requirements for the command and tool window icons are different. Visual Studio command icons use a lime transparency color (0,255,0 RGB), whereas tool window icons use a magenta transparency color (255,0,255 RGB).

Common tool window creation practice is to create the tool window either at add-in load time or in response to command execution. After that you should keep the tool window reference around until the add-in gets unloaded, instead of creating a new tool window instance. You should also provide a View menu item to show the tool window in case the user closes it. When that happens, the tool window reference should still be live and the tool window can still be made visible.

Options Dialog Page

With the tool window in place, you can give the add-in some configuration options, as shown in Figure 3. First create a new class library called WDSToolsOptions, then add a new user control called OptionsPage to this class library.

Figure 3 Tools Options Dialog Page
Figure 3 Tools Options Dialog Page

To add Options dialog box support, you need to implement an IDTToolsOptionsPage interface on the new user control—this is a fairly straightforward process. There are only two interface methods of interest here. The OnAfterCreated method reads previously saved configuration options from the registry and updates the UI accordingly. And the OnOK method saves UI options to the registry. You'll want to include a static Settings class to hold all configuration options to avoid frequent registry reads. You can see these in the code download for this issue.

The final step is to set the add-in to use your new options page assembly. There's no better place for that than the .addin XML configuration file, the new Visual Studio 2005 add-in deployment feature, which simplifies add-in installation and eliminates the need for registry-based registration. Here is all it takes to configure the Options page with your add-in:

<ToolsOptionsPage>
   <Category Name="Windows Desktop Search">
      <SubCategory Name="General">
         <Assembly>[Path-To-Addin]\WDSToolsOptions.dll</Assembly>
         <FullClassName>WDSToolsOptions.OptionsPage</FullClassName>
      </SubCategory>
   </Category>
</ToolsOptionsPage>

As you can see, you simply tell Visual Studio what assembly to load and what class name within that assembly to use for the Options page. Note that it's a good practice to call the first sub-page General to follow the standard Microsoft page-naming convention.

The .addin file also contains all the About dialog information for the add-in, including an icon. The default icon created for the add-in is a binary encoded string, which makes it difficult to modify, so it's a good idea to use a resource file icon instead.

Remember the resource file you added when you first created the add-in? For the About dialog, you'll create a new 32×32 icon called AboutIcon and add it under the WDSAddin.en resource file. The .addin configuration file lets the icon be loaded from the satellite assembly and have its resource ID used instead of a binary encoded string:

<AboutIconData>@AboutIcon</AboutIconData>

Windows Desktop Search SDK

The information that Windows Desktop Search has indexed can be queried via a simple COM API. There are only two public methods on the ISearchDesktop interface: ExecuteSQLQuery and ExecuteQuery. ExecuteSQLQuery is used to execute well-formed SQL queries against the underlying index store. ExecuteQuery, on the other hand, is more user-friendly and its query string syntactically matches what's offered through the Windows Desktop Search query input box. The following shows how the ExecuteQuery method is defined:

HRESULT ExecuteQuery(LPCWSTR lpcwstrQuery, LPCWSTR lpcwstrColumn,
    LPCWSTR lpcwstrSort, LPCWSTR lpcwstrRestriction, Recordset **ppiRs);

The ExecuteQuery parameters are explained in Figure 4. As you can see, the first four parameters are various string input values that control the query and resultset, while the last parameter is the output resultset.

Figure 4 ExecuteQuery Parameters (Close Figure 4)

Parameter | Description
lpcwstrQuery | The query as you would type it in the query input box.
lpcwstrColumn | Columns to include in resultset.
lpcwstrSort | Columns to sort the resultset by.
lpcwstrRestriction | Restrictions to append via WHERE clauses in the Windows Desktop Search SELECT.
ppi Rs | The resulting recordset.

Luckily, you don't have to worry about translating that into a managed call. The Windows Desktop Search SDK is available for download and includes two managed assemblies, one of which is found it WDSQuery.dll. This COM interop assembly wraps the relevant Windows Desktop Search COM classes and interfaces. Most importantly, it exposes a class called SearchDesktopClass that implements ISearchDesktop and is used to execute WDS queries. Rather than creating one instance of SearchDesktopClass and keeping it for all searches, the recommended usage pattern for this class is to create a new instance for each query, retrieving the results and allowing the garbage collector to clean up after it. The following shows how to use SearchDesktopClass in your add-in:

SearchDesktopClass wdsSearch = new SearchDesktopClass();
_Recordset result = wdsSearch.ExecuteQuery(
  "Secondary", "Rank, FileName, DisplayFolder, Url, PerceivedType",
  "Rank DESC, DocTitle", "Contains(PerceivedType,'document')");

The query specifies four elements. It is executed for the string "Secondary". The comma-separated columns start with Rank and end with PerceivedType. The columns to sort on are Rank and DocTitle. And finally the filter is set to retrieve all items of the 'document' type. This call essentially translates to a request for all documents that have the word "Secondary" in them.

Windows Desktop Search supports a number of built-in columns to query and sort on. Some of the common ones you can use are shown in Figure 5. Note that the results are intentionally sorted by Rank in descending order in the sample code. This ensures that the most probable results are at the top of the resultset.

Figure 5 Common Query and Sort Columns (Close Figure 5)

Column | Description
Rank | Represents how well items match or score on query request. The higher the number, the more relevant the match is.
File name | Name of the file without path.
DisplayFolder | The file’s path.
Url | Item path that can be used to run a shell execute command on the file.
PerceivedType | The item type.

Background Queries and UI Updates

One of the early design goals was to build a tool that would have little impact on Visual Studio responsiveness. Windows Desktop Search runs index queries, and thus the queries execute very quickly. Still, depending on what you search for, a query may take anywhere from a few milliseconds to a second or more to execute. If queries are executed on the main Visual Studio thread, the UI will appear temporarily locked while the queries run. The obvious solution is to execute queries on a secondary thread.

You might consider using the new .NET Framework 2.0 BackgroundWorker class. Unfortunately, you can't. It turns out that this class is incompatible with SearchDesktopClass and its Windows Desktop Search COM object. Trying to execute a query using BackgroundWorker yields an InvalidCastException exception. In order to use SearchDesktopClass on a secondary thread, you must use a single-threaded apartment (STA) thread. BackgroundWorker uses the .NET ThreadPool, which by default contains multithreaded apartment (MTA) threads.

So add a new class to your add-in project called WDSQueryWorker that creates an STA thread using the Run method as a thread start delegate parameter. Run uses the WaitHandle class to wait on a single event handle, as shown here:

private void Run() {
  try {
    while (true) {
      WaitHandle.WaitAny(_runHandles);
      if (_stopped) break;
      ProcessEvent();
      _runEvent.Reset();
    }
  }
  finally { _stopped = true; }
}

The WDSQueryWorker class also exposes a public method called DoWork that is used to set the event the Run method waits on, which in turn triggers query execution in the ProcessEvent method. Fields to query on should be set prior to calling DoWork.

The WDSQueryWorker class constructor takes three parameters: the UI control to call when your query completes, the query completion delegate, and the query error delegate. Both of the delegates are invoked via Control.Invoke. The Invoke method ensures that the delegate is executed on the control's UI thread. It's worth mentioning that arguments passed to the query completion delegate include the query _Resultset. Query error delegate arguments, on the other hand, include an Exception reference.

While moving query execution onto a separate thread takes care of some of the Visual Studio responsiveness issues associated with running large queries, there is one problem it doesn't address. As noted, the control's Invoke method executes a delegate on the control's UI thread, which happens to be the Visual Studio UI thread. If Windows Desktop Search returns a large resultset, it still takes time to populate the add-in's results List view. The obvious solution is to limit the displayed results. As a matter of fact, if you've enabled the Search-as-I-type option, Windows Desktop Search simply displays the first six results for each category, adding a "more..." item to show the rest. Using the Options dialog, you can set up the add-in to limit display results to, say, one hundred items per category by default. You can suppress this by setting the Display first results value to 0 if you prefer to see the entire resultset.

You'll want to make sure that as the user types code in the editor, even with limited query results, there is no delay caused by query execution. Luckily, Visual Studio extensibility provides editor events just for that. These events are accessible via the Events interface. Figure 6 shows how to set it up.

Figure 6 Hooking Editor Events (Close Figure 6)
_editorEvents = _application.Events.get_TextEditorEvents(null);
if (_editorEvents == null) throw new NullReferenceException(
    "Document events are not available.");

_editorKeyPressEvents =
    ((Events2)_application.Events).get_TextDocumentKeyPressEvents(null);

if (_editorKeyPressEvents == null) throw new NullReferenceException(
    "Document key press events are not available.");

_editorEvents.LineChanged +=
    new _dispTextEditorEvents_LineChangedEventHandler(
        _editorEvents_LineChanged);

_editorKeyPressEvents.AfterKeyPress +=
    new _dispTextDocumentKeyPressEvents_AfterKeyPressEventHandler(
        editorEvents_AfterKeyPress);

The idea is to delay queries if the user is entering code, scrolling through the code, or moving from one window to another. You can add your own delegates and set up all delegates for all these events to call into a private EditorUpdateInProgress method to delay the query execution.

Putting It All Together

When I set out to create this add-in, my goal was to have the query execute when the user selects some text in the Visual Studio editor, and originally I hoped to find some Visual Studio extensibility text selection event to work with. Though there is a TextSelection extensibility interface that returns the currently selected text, no text selection change events are exposed by the extensibility. This was an unfortunate setback. The only solution seems to be polling, which leads back to the issue of Visual Studio responsiveness. Polling is an inherently expensive operation because you end up polling when you don't necessarily need to. Thus, careful safeguards need to be put in place to prevent excessive polling.

One approach is to add a timer to your user control and set it to kick in every two seconds. You can also add several more less-aggressive timer-refresh settings to the Options dialog page. The timer's Tick event handler simply calls the private ExecuteQuery method, which is where the query is set up and secondary worker thread processing is initiated, as shown in Figure 7. Naming this method to match the Windows Desktop Search SDK's method name—SearchDesktopClass.ExecuteQuery—is a good idea, in order to emphasize its purpose.

Figure 7 Creating the Tool Window (Close Figure 7)
private void ExecuteQuery() {
    _refreshing = true;
    bool clearResults = true, enabled = timerRefresh.Enabled;
    string currentDocument = string.Empty;

    timerRefresh.Stop();
    try {
        // Check if auto-refresh has been disabled
        if (enabled && !Settings.AutoRefresh) return;

        // Sync up update frequency
        timerRefresh.Interval = GetOptionsUpdateFrequency();

        // Get document’s selection
        Document activeDocument = _application.ActiveDocument;
        if (activeDocument != null)
            currentDocument = activeDocument.FullName;

        string selectedText = GetQueryText(activeDocument);
        if (selectedText == null) return;
        selectedText = selectedText.Trim();

        // Selected text minimum length check
        if (selectedText.Length < Settings.MinimumLength) {
            _lastQuery = string.Empty;
            return;
        }

        // Unchanged search check
        if ((string.Compare(_lastQuery, selectedText, true) == 0) &&
            (string.Compare(_lastDocument, currentDocument, true) == 0)) {
            clearResults = false;
            return;
        }

        // Save last query info
        _lastQuery = selectedText;
        _searching = true;

        _queryWorker.Query = selectedText;
        _queryWorker.DoWork();
    }
    finally {
        // Update UI, save the last query, and restart the timer
        if (clearResults) ClearResults();
        _lastDocument = currentDocument;
        if (Settings.AutoRefresh) timerRefresh.Start();
        _refreshing = false;
    }
}

When ExecuteQuery runs, the first thing it does is set a private processing field to true and disable the polling timer. It then retrieves the Visual Studio active document, if there is one, and retrieves its TextSelection, which has a Text property. This property returns the currently selected text in the editor.

Keep the last executed selection in a private field. After retrieving TextSelection.Text, check whether the currently selected text differs from the last text query. If they are the same, there's no point in re-executing and you can simply exit the method.

You should also check whether the selected text meets the minimum selected text requirement, as configured in the Options dialog page. You'll want the text selection to include at least three characters for the query to execute, a performance safeguard against a user selecting, for example, just the letter A and Windows Desktop Search returning thousands of entries.

If everything checks out, the query worker (the WDSQueryWorker class described earlier) is set up with the text to search for and its DoWork method is called. On the way out of the ExecuteQuery method, the timer is restarted (if it was previously running) and the processing field is set back to false. The results List view is cleared and the update status changed so "Searching..." is displayed.

The query worker's DoWork method sets an event, which causes it to execute a query. That's all done on a secondary thread, of course. When the query completes, it calls back into the main thread. The callback delegates happen to be on ToolWindowUserControl's side. One of them, the QueryCompleted method, simply calls DisplayResults to update the results List view. If the query fails, it calls into ToolWindowUserControl's QueryError method. Depending on the configuration in the Options dialog page, QueryError either displays an error message or silently ignores the error.

The DisplayResults method receives a single _Recordset parameter. If you recall, this is what the Windows Desktop Search SDK ExecuteQuery method returns. DisplayResults loops through the results fetching one row at a time. It retrieves column values based on a predefined set of fields to query on and updates the corresponding List view columns. It also groups items based on the PerceivedType column, which is used by Windows Desktop Search to identify item type. List view grouping is a new feature for the .NET Framework 2.0 and won't work on systems prior to Windows XP.

In addition to having a safeguard on minimum query string length, you should also restrict the number of results to display, perhaps to just the first 100 items per category. You can change this option to 0 to see the entire resultset. With a reasonable cap for the List view result, UI update processing occurs almost instantaneously, dramatically improving Visual Studio responsiveness. Play with the results option to see what works best for you.

One area for potential improvement is to see whether Windows Desktop Search supports limiting the resultset at the query level. That is, you might be able to speed the process if you moved implementation of the display-first-results option from trimming the display results to trimming the scope of the actual query.

Installation and Deployment

Visual Studio 2005 introduced a new registration-free deployment scheme based on .addin configuration files. This makes add-in installation as simple as copying assemblies to a designated folder and restarting Visual Studio. Here's the manual Windows Desktop Search add-in installation procedure:

  • Copy the contents of the WDSAddinBinaries folder to the \Program Files\WDSAddin folder. If you use a different folder, manually update the file path references in WDSAddin.addin.
  • Copy the WDSAddin.addin file to your \My Documents\Visual Studio 2005\Addins folder.
  • Make sure that a copy of Msvcp71.dll, also located in theWindows Desktop Search installation folder, exists somewhere on your Windows path. If that's not the case then simply copy that file from the Windows Desktop Search installation folder to \Windows\System32.

If you plan on debugging the Windows Desktop Search add-in, shut down Visual Studio and run the ToolWindow.reg file included in the code download at least once. It contains a workaround for Visual Studio 2005 add-in tool window state persistence issues. Otherwise, just skip this step.

Conclusion

The Windows Desktop Search SDK is rather simplistic at what it does and how it can be used, and the add-in I've described is not too complicated either. But the combination of the two is remarkably useful. Think of this add-in as analogous to Visual Studio Dynamic Help, but with the ability to scan your entire computer instead of just a predefined set of help files.

Sergey MishkovskiySergey Mishkovskiy is a Senior Software Developer at OpenSpan Inc. In his spare time he plays Xbox, reads, and works on a popular free Visual Studio add-in called DPack. To learn more about DPack, visit www.usysware.com/dpack. You can contact Sergey at mailto:sergey@usysware.com.


Beyond WinFX: Transactions, Aero Wizards, And Task Dialogs In Windows Vista

Beyond WinFX

Transactions, Aero Wizards, And Task Dialogs In Windows Vista

Kenny Kerr
This article is based on a prerelease version of Windows Vista. All information herein is subject to change.
This article discusses:
  • Kernel Transaction Manager
  • Using command links for customizable buttons
  • Building more intuitive interfaces with Aero wizards
  • Replacing message boxes with task dialogs
This article uses the following technologies:
Windows Vista, C++, C#, Visual Studio 2005
Code download available at: Vista2006_07.exe (178KB)
Contents
Kernel Transaction Manager
Command Links: Getting to the Point
Streamlined UI: Aero Wizards
From Message Box to Task Dialog
Sophisticated Task Dialogs
Conclusion

W
hat comes to mind when you think about programming for Windows Vista™? Odds are you’ll think of WinFX®. WinFX encompasses a number of technologies that have been getting a lot of attention, including the Microsoft® .NET Framework 2.0, Windows® Presentation Foundation, Windows Communication Foundation, and Windows Workflow Foundation. But WinFX is only one part of the new OS. Windows Vista is a dramatic upgrade to the Windows operating system, providing a wealth of new and improved libraries and services that reach beyond WinFX, allowing you to build more powerful applications with less code.

This article introduces a number of new features that you can expect from Windows Vista. These features don’t normally get the attention they deserve because they’re either aimed at native C++ developers or just don’t fall into the category of WinFX programming. Nonetheless, they’re worth getting to know.

Kernel Transaction Manager

One of the most exciting new services that developers have at their disposal is the Kernel Transaction Manager (KTM). This acts as both a transaction manager for local transactions and a resource manager for distributed local transactions. KTM offers a new concept of multiple transaction managers on a single machine, as it manages the multiple instances and their associated resource managers. Before digging into the KTM’s functionality in more detail, let’s quickly recap the purpose of transaction and resource managers.

A resource manager is responsible for some local resource, such as a database engine or a message queue server. Resource managers work in conjunction with transaction managers to coordinate changes to the resources being managed within the context of a transaction. Any number of resource managers can participate in a transaction by coordinating with a transaction manager. In practice, most resource managers also act as transaction managers when the transaction is confined to a single resource, such as a database engine. This is referred to as a local transaction.

If a transaction enlists multiple resource managers it is said to be a distributed transaction and an independent transaction manager steps in to coordinate with the various resource managers. This coordination ensures that all the resource managers either commit or roll back any changes to their resources. Of course, this enlistment and coordination of distributed resource managers incurs a noticeable performance overhead and thus local transactions are preferred whenever possible.

Microsoft SQL Server™ is a good example of a resource manager. Explicit or implicit transactions involving a single SQL Server instance use SQL Server’s own transaction manager to transact any commands executed within the context of a transaction. The Microsoft Distributed Transaction Coordinator (DTC), a service provided by Windows, is used as a transaction manager when a transaction spans more than one resource manager. The DTC coordinates transactions with SQL Server and other user-mode resource managers, including KTM. Furthermore, you can start a transaction in managed code and use the transaction to transact file operations.

Just as SQL Server acts as a resource manager for local database transactions and as a transaction manager for distributed database transactions, the KTM acts as a transaction manager for local file system and registry operations and as a resource manager for distributed local transactions that involve file system and registry operations. The KTM allows you to call file system and registry functions in the context of a transaction, and will ensure that those operations are transacted as a whole.

The beauty of the KTM is that it doesn’t require you to change the way you interact with files and registry keys. Whatever API you use today for interacting with files and registry keys will continue to work as long as those abstractions end up interacting with kernel objects via file and registry handles.

Let’s take a look at a simple example using the file and registry management functions from the Windows SDK. Suppose you are writing a program that needs to keep track of a file system folder, perhaps for storing a history of instant messenger conversations. And say you want to allow the user to change the location of this history folder. You could achieve this with the simplistic function shown in Figure 1.

Figure 1 Basic Folder-Moving Function (Close Figure 1)
HRESULT MoveHistoryFolder(PCWSTR oldPath, PCWSTR newPath)
{
    HRESULT result = E_POINTER;

    if (0 != oldPath && 0 != newPath)
    {
        if (!::MoveFile(oldPath, newPath))
        {
            result = HRESULT_FROM_WIN32(::GetLastError());
        }
        else
        {
            CRegKey key;
            result = HRESULT_FROM_WIN32(key.Create(
                HKEY_CURRENT_USER, L"Software\\Kenny Kerr\\Sample"));

            if (SUCCEEDED(result))
            {
                result = HRESULT_FROM_WIN32(
                    key.SetStringValue(L"HistoryFolder", newPath));
            }

            // Move the folder back if we failed to update the registry.
            if (FAILED(result)) VERIFY(::MoveFile(newPath, oldPath));
        }
    }

    return result;

The MoveHistoryFolder function is intended to move the history folder and update a registry setting that stores its location for the application’s use. But this implementation is actually far too simple. There are a number of scenarios it doesn’t take into account.

MoveHistoryFolder starts off by moving the history folder. If the move is performed successfully the function will then attempt to update the registry. If the registry operations fail, MoveHistoryFolder carefully attempts to move the folder back to its original location so that the function can fail without side effects, providing a strong guarantee.

So what’s wrong with this function? Even in this simple example, it is possible for the application’s state to become corrupt. A power or hardware failure, for example, might result in the history folder being moved without the corresponding registry update occurring. The problem is that the operating system sees the various file and registry management function calls as independent operations. (Your application, on the other hand, considers these functions to be a single logical operation that should have no potential side effects.)

Thanks to the KTM, solving this problem is quite simple. Since all the resources involved are local resources controlled with kernel objects (assuming that you don’t allow users to move the history folder to a remote computer), you can use the KTM both as the transaction manager for a local transaction and also as the resource manager.

The CreateTransaction function creates a new transaction object controlled by the KTM. Transaction objects are regular kernel objects in the same way that other OS constructs like events and files are represented by kernel objects and accessed through secured handles. CreateTransaction returns a handle to the transaction object. A GUID uniquely identifies the transaction on the computer (if this isn’t specified, the KTM will create a GUID by default). As with other kernel objects, you can specify a SECURITY_ATTRIBUTES structure to control the security attributes of the handle. Additionally, you can pass a user-readable description string as the last parameter to CreateTransaction—this is recommended for management purposes.

When you no longer need the transaction, you simply close the transaction handle with the CloseHandle function. Closing the last handle to the transaction rolls it back if the transaction hasn’t been committed. You can request that a transaction be committed using the CommitTransaction function. A transaction can be rolled back by using the RollbackTransaction function. The use of these functions is greatly simplified with a little help from C++, as shown in Figure 2.

Figure 2 KtmTransaction Class (Close Figure 2)
class KtmTransaction
{
public:
    HRESULT Create(PSECURITY_ATTRIBUTES securityAttributes = 0)
    {
        HRESULT result = S_OK;
        Handle.Attach(::CreateTransaction(
            securityAttributes, NULL, 0, 0, 0, 0, 0));
        if (INVALID_HANDLE_VALUE == Handle)
        {
            result = HRESULT_FROM_WIN32(::GetLastError());
        }
        return result;
    }

    HRESULT Commit()
    {
        HRESULT result = S_OK;
        if (!::CommitTransaction(Handle))
        {
            result = HRESULT_FROM_WIN32(::GetLastError());
        }
        return result;
    }

    HRESULT Rollback()
    {
        HRESULT result = S_OK;
        if (!::RollbackTransaction(Handle))
        {
            result = HRESULT_FROM_WIN32(::GetLastError());
        }
        return result;
    }

    CHandle Handle;
};

The code illustrates the use of the KtmTransaction class. However, before you can actually use this class, you need a way to bind the transaction to the thread that is making the calls to the file and registry management functions so those functions, in turn, will be aware that they’re being called from within the context of a transaction. This is the role of the SetCurrentTransaction function: it binds the calling thread to the given transaction. The thread is bound until SetCurrentTransaction is called again with an invalid handle. Figure 3 shows the KtmTransactionBinding class, which you can use to simplify the process of binding a transaction to the calling thread.

Figure 3 Ktm TransactionBindingClass (Close Figure 3)
class KtmTransactionBinding
{
public:
    KtmTransactionBinding(KtmTransaction&amp; transaction) :
        m_transaction(transaction) { /* Do nothing */ }

    ~KtmTransactionBinding() { COM_VERIFY(Unbind()); }

    HRESULT Bind() { return Bind(m_transaction.Handle); }

    HRESULT Unbind() { return Bind(INVALID_HANDLE_VALUE); }

private:
    static HRESULT Bind(HANDLE transaction)
    {
        HRESULT result = S_OK;
        if (!::SetCurrentTransaction(transaction))
        {
            result = HRESULT_FROM_WIN32(::GetLastError());
        }
        return result;
    }

    KtmTransaction&amp; m_transaction;
};

With the KtmTransaction and KtmTransactionBinding classes defined, you can call the sample application’s MoveHistoryFolder function within the context of a transaction, as shown in Figure 4. Unless MoveHistoryFolder returns a success HRESULT, the transaction object’s Commit function will not be called and any file and registry operations will be rolled back when the transaction object is destroyed.

Figure 4 Moving a Folder Within a Transaction (Close Figure 4)
KtmTransaction transaction;
HRESULT result = transaction.Create();

if (SUCCEEDED(result))
{
    KtmTransactionBinding binding(transaction);
    result = binding.Bind();

    if (SUCCEEDED(result))
    {
        result = MoveHistoryFolder(L"C:\\Users\\Kenny\\My History",
                                   L"C:\\Users\\Kenny\\History");
        if (SUCCEEDED(result)) result = transaction.Commit();
    }
}

I can’t stress enough how beneficial transactions can be for file and registry operations. Since all rollback logic is performed automatically, you can focus on the code needed to perform the necessary operations, accounting for any and all scenarios, without worrying about writing all the complex rollback logic. As an example, the MoveHistoryFolder function shown in Figure 1 does not take into account the fact that the new folder location may be nested while the MoveFile function requires the immediate parent directory to exist in order for the operation to succeed. Adding the logic to ensure that the parent directory exists is not hard, but all the additional rollback logic can be extremely tricky.

Command Links: Getting to the Point

Command links are a new style of button that simplifies the user experience by focusing the user’s attention on one or more "commands." Although designed primarily for use in wizards, command links are little more than old-fashioned button controls with special style attributes.

The simplest way to create a command link is by using the Visual C++® 2005 Resource Editor. Open the dialog editor’s Toolbox view and drop a custom control onto the dialog. (Although command links are just button controls, the dialog editor has no knowledge of command links and so does not provide any attributes for styling button controls as command links.) Be sure to make the custom control reasonably large as command links are intended to be displayed quite a bit larger than the average button. Now set the control’s Class property to BUTTON and its Style property to 0x5001000e. You may also want to update the Caption property with the text you want to display on the command link. You can see these settings in the right-hand panel of Figure 5. At this point you can compile the project and display the dialog box, which is also shown in Figure 5.

Figure 5a Creating Command Links
Figure 5a Creating Command Links

Figure 5b

Command links also let you add some additional text to the button to provide further explanations. You can do this by sending the button a BCM_SETNOTE message or by using the simpler Button_SetNote macro. You can also amplify a given command link by replacing the arrow with a shield. To do this, you either send the BCM_SETSHIELD message or use the Button_SetElevationRequiredState macro. The following shows an example of sending these messages in the context of an Active Template Library (ATL) dialog box class:

CWindow button1 = GetDlgItem(IDC_BUTTON1);
    Button_SetNote(button1,
    L"You know you want to click it.");
Button_SetElevationRequiredState(button1,
    TRUE);

Streamlined UI: Aero Wizards

Although wizards have been around for a while, the Wizard 97 specification is full of useless and redundant displays of information, such as exterior pages and headers. Aero™ wizards, which replace the aging Wizard 97 specification, are designed to be both clear and concise, offering a fresh new experience for guiding users through some potentially complex operations.

As with command links, Aero wizards are simply exposed as additional styles on existing controls, in this case the property sheet. Figure 6 shows a minimal Wizard 97-style wizard. Let’s take a look at what it takes to transform this into a shiny new Aero wizard. To keep this as clean and short as possible, I’ll use ATL and Windows Template Library (WTL) to illustrate. You can, of course, use any framework you like.

Figure 6 Aging Wizard 97 Conversion
Figure 6 Aging Wizard 97 Conversion

The window in Figure 6 is created by the Wizard class. The CPropertySheetImpl base class wraps the functions, messages, and structures necessary to implement a property sheet. Wizards, after all, are just property sheets with some additional flags to change their appearance and behavior. The PROPSHEETHEADER structure is used to control virtually all aspects of a property sheet. The CPropertySheetImpl class automatically populates this structure on your behalf. If you want to change the behavior in some way, you can freely modify the PROPSHEETHEADER structure that is exposed through the inherited m_psh member variable. This is exactly what I’ve done in Figure 6 to change the property sheet into a wizard.

Changing the wizard into an Aero style wizard is as simple as replacing the PSH_WIZARD97 flag with PSH_AEROWIZARD. But as you’ll see when you look at the result (see Figure 7), there is a bit more work to be done.

Figure 7 Aero Wizard Conversion
Figure 7 Aero Wizard Conversion

The most striking difference is the size of the window. The Wizard 97 specification dictates a minimum size for wizard pages. The dialog template used for these samples is smaller than this minimum. With the PSH_WIZARD97 flag, the window is padded to ensure the minimum size is honored. The Aero wizard has no such restriction and it wraps the dialog template more cleanly.

The other thing you should notice is that the Back button at the bottom of the wizard has been replaced with a Web browser-style back button at top the top of the client area. Other than the appearance, the button is controlled programmatically in the same way as before.

Aero wizards are designed with command links in mind so that you can reduce the clutter of the traditional set of buttons that adorn the bottom of most wizards today. The first button to go was the Back button. The Next button can also be removed in many cases. In the past, the PSM_SETWIZBUTTONS message could be used to disable the Next button. But prior to Windows Vista you could not hide both the Next and Finish buttons. Windows Vista makes this possible, allowing the Next button to be replaced by any number of various command links.

Aero wizards introduce a new message that lets you control the visibility of the wizard buttons. This is where you start to see the usefulness of command links. In the Wizard 97 style you can create a wizard page that includes a number of radio buttons. In this wizard, the user would make a selection and then click the Next button. Using Aero wizards and command links you can replace the radio buttons with command links and hide the Next button altogether. Now the user makes a selection and he’s finished, all with a single click.

You can hide the Next button by sending the property sheet PSM_SHOWWIZBUTTONS message or by using the PropSheet_ShowWizButtons macro:

SetWizardButtons(0);
PropSheet_ShowWizButtons(m_hWnd, PSWIZB_CANCEL,
                    PSWIZB_CANCEL | PSWIZB_NEXT);

The SetWizardButtons function is provided by the CPropertySheetWindow class and posts the PSM_SETWIZBUTTONS message to the property sheet. Passing 0 indicates that both the Back and Next buttons should be disabled. If you need to keep the Back button enabled, simply specify the PSWIZB_BACK flag. The PropSheet_ShowWizButtons macro is then used to set the visibility of the wizard buttons. Flags that appear in both the second and third parameters indicate buttons that will be displayed. Flags that appear only in the third parameter indicate buttons that will be hidden. Finally, you can include command links that provide the user with options for how to proceed, as shown in Figure 8.

Figure 8 Command Links
Figure 8 Command Links

From Message Box to Task Dialog

The task dialog is a powerful new tool for displaying user prompts with a small amount of code. The MessageBox function has been widely used for many years to prompt users for an acknowledgment, to confirm an operation, or to make a yes-or-no decision. Message boxes are popular because the MessageBox function is convenient, but the user experience often suffers. The MessageBox function is rather limited so developers don’t have much opportunity to improve the user experience short of rolling their own message box implementation.

Message boxes have finally been replaced with task dialogs, which retain the simplicity of message boxes but optionally offer considerable customization options for improving and controlling the user experience. Task dialogs are exposed through one of two functions. This section deals with the simpler TaskDialog function, while the next section will dig into the more powerful TaskDialogIndirect function.

The TaskDialog function is very similar to the MessageBox function and in most cases you can simply replace one with the other and change the parameters to get the desired behavior. Consider the following MessageBox call:

  int result = ::MessageBox(0, // no owner window
      L"Do you want to save changes?",
      L"Application",
      MB_YESNOCANCEL | MB_ICONINFORMATION);
This code can be replaced using the TaskDialog function. The code and resulting dialog are shown in Figure 9.

Figure 9 Using TaskDialog to Control Message Display (Close Figure 9)
int result = 0;
COM_VERIFY(::TaskDialog(0 /* no owner window */,
    0 /* no module resources */, L"Application",
    L"Do you want to save changes?", L"",
    TDCBF_YES_BUTTON | TDCBF_NO_BUTTON | TDCBF_CANCEL_BUTTON,
    TD_INFORMATION_ICON, &result));

Although not dramatically different, the TaskDialog function has some handy features that make it worthwhile. Perhaps the best new capability is the ability to specify a module handle as the second parameter, which allows you to specify resource identifiers instead of literal strings. This will save you a lot of boilerplate code calling the LoadString function. Specifying a module handle also allows you to display the icon of your choice on the dialog box. In addition, the task dialog is capable of displaying separate content with a smaller font to provide additional information if needed. Finally, there is a separate flag for each button so you can create arbitrary combinations of buttons, as shown in Figure 10.

Figure 10 Traditional Nondescript Buttons (Close Figure 10)
int button = 0;
COM_VERIFY(::TaskDialog(0, instance,
    MAKEINTRESOURCE(IDS_APPLICATION_TITLE),
    MAKEINTRESOURCE(IDS_SAVE_CHANGES),
    MAKEINTRESOURCE(IDS_SAVE_CHANGES_INFO),
    TDCBF_YES_BUTTON | TDCBF_NO_BUTTON | TDCBF_CANCEL_
    BUTTON,
    MAKEINTRESOURCE(IDI_APPLICATION), &button));

Sophisticated Task Dialogs

One of the glaring problems with the simple TaskDialog function is that you cannot change the text that appears on the buttons. In Figure 10, it’s obvious that the wordy descriptions of the buttons would not be needed if the buttons had more specific wording, such as "Save", "Don’t Save", and "Cancel". This is the kind of problem that TaskDialogIndirect is meant to solve. But TaskDialogIndirect does more than that: it can turn the simple dialog box into a rich, interactive interface for communicating more information without the need to write lots of UI code.

The TaskDialogIndirect function is prototyped as follows:

HRESULT WINAPI TaskDialogIndirect(
    const TASKDIALOGCONFIG* config, int* button,
    int* radioButton, BOOL* verificationChecked);
As you can imagine, most of the work to use this function goes into preparing the config parameter. The caller-provided TASKDIALOGCONFIG structure controls all of the behavioral and user interface aspects of the task dialog.

To start, let’s recreate the "Do you want to save changes?" dialog with more descriptive buttons. Figure 11 shows the result.

Figure 11 Adding More Instructive Button Text (Close Figure 11)
TASKDIALOG_BUTTON buttons[] =
{
    IDYES, L"&Save",
    IDNO, L"&Don’t Save"
};

TASKDIALOGCONFIG config = { sizeof (TASKDIALOGCONFIG) };
config.pszWindowTitle = L"Application";
config.pszMainInstruction = L"Do you want to save changes?";
config.pszContent = L"Click Cancel to return to the application.";
config.dwCommonButtons = TDCBF_CANCEL_BUTTON;
config.pszMainIcon = MAKEINTRESOURCE(TD_INFORMATION_ICON);
config.cButtons = _countof(buttons);
config.pButtons = buttons;

int button = 0;
COM_VERIFY(::TaskDialogIndirect(&config, &button, 0, 0));

An array of TASKDIALOG_BUTTON structures is provided. Each structure specifies a button ID as well as the text caption for the button. The various text fields of the TASKDIALOGCONFIG structure are then populated. Many more fields are available for your customization. The TaskDialogIndirect function provides a tremendous amount of control over the appearance of the dialog—so much so that it would not be practical to examine each of its options in detail in this article. To allow you to easily explore the TaskDialogIndirect function’s options, I wrote the TaskDialog Designer shown in Figure 12.

Figure 6 Aging Wizard 97 Conversion
Figure 6 Aging Wizard 97 Conversion

The TaskDialog Designer is a small Windows Forms application written in C# that exposes most of the TaskDialog options in a set of property pages so you can experiment with various options. The TaskDialog Designer makes use of a .NET assembly written in C++/CLI that provides a managed interface to the TaskDialogIndirect function. You can rewrite the previous native C++ code with this C# code to create the same task dialog:

using (TaskDialog dialog = new TaskDialog())
{
    dialog.Title = "Application";
    dialog.MainInstruction = "Do you want to save changes?";
    dialog.Content = "Click Cancel to return to the application.";
    dialog.Buttons = TaskDialogButtons.Cancel;
    dialog.CustomButtons.Add(new TaskDialogButton("&Save", 6));
    dialog.CustomButtons.Add(new TaskDialogButton("&Don’t Save", 7));
    dialog.MainIcon = TaskDialogIcon.Information;

    int button = dialog.ShowDialog();
}

One of the many uses of task dialogs is to create an error reporting dialog box. Using the task dialog’s expandable surface area, you can create an unobtrusive dialog box for reporting an error to the user, allowing him to see more information if he so desires.

In Figure 13, the TaskDialog is prepared with information from the Exception object. Initially, the error message is displayed while the complete contents of the exception are only displayed if the user chooses to view them.

Figure 13 Reporting Errors Through TaskDialog (Close Figure 13)
try
{
    // Attempt to open file...
}
catch (Exception e)
{
    using (TaskDialog dialog = new TaskDialog())
    {
        dialog.Title = "Application";
        dialog.MainInstruction = "The file could not be opened.";
        dialog.Content = e.Message;
        dialog.ExpandedInformation = e.ToString();
        dialog.MainIcon = TaskDialogIcon.Error;
        dialog.PositionRelativeToWindow = true;
        dialog.AllowDialogCancellation = true;
        dialog.Buttons = TaskDialogButtons.Close;
        dialog.ExpandFooterArea = true;

        TaskDialogButton button = new TaskDialogButton(
           "&Send Report", 0);
        dialog.CustomButtons.Add(button);

        if (button.Id == dialog.ShowDialog(this))
        {
            // Send error report...
        }
    }
}

Conclusion

Windows Vista is a major upgrade to the Windows OS and includes a considerable upgrade to the Windows SDK. Of the many new features, a few like the Kernel Transaction Manager, Aero wizards, and task dialogs are worthy of a closer look. The KTM allows you to focus on application logic while the operating system takes care of all the complex rollback logic required to ensure that your applications have no unwanted side effects. Aero wizards combined with command links provide a fresh and simple new interface for interacting with users. And the task dialog provides a powerful new API for simple communication and represents a very real replacement for the aging message box.

Kenny Kerr designs and builds distributed apps for Windows. He also has a particular passion for C++ and security programming. Reach Kenny at weblogs.asp.net/kennykerr or visit his Web site.


Web App Follies: Keep Sites Running Smoothly By Avoiding These 10 Common ASP.NET Pitfalls

Web App Follies

Keep Sites Running Smoothly By Avoiding These 10 Common ASP.NET Pitfalls

Jeff Prosise
This article discusses:
  • Caching and forms authentication
  • View state and session state
  • Profile property serialization
  • Thread pool saturation
  • Impersonation and profiling
This article uses the following technologies:
.NET Framework, ASP.NET, Windows Server 2003
Contents
LoadControl and Output Caching
Sessions and Output Caching
Forms Authentication Ticket Lifetime
View State: The Silent Perf Killer
SQL Server Session State: Another Perf Killer
Uncached Roles
Profile Property Serialization
Thread Pool Saturation
Impersonation and ACL Authorization
Don’t Just Trust It—Profile Your Database!
Conclusion

O
ne of the reasons ASP.NET is successful is that it lowers the bar for Web developers. You don’t need a Ph.D. in computer science to write ASP.NET code. Many of the ASP.NET people I encounter in my work are self-taught developers who wrote Microsoft® Excel® spreadsheets before they wrote C# or Visual Basic®.Now they’re writing Web applications and, in general, they’re doing a commendable job.

But with power comes responsibility, and even veteran ASP.NET developers aren’t immune to mistakes. Years of consulting on ASP.NET projects has shown me that certain mistakes have an uncanny predisposition to keep pitfalls occurring. Some of these mistakes affect performance. Others inhibit scalability. Still others cost development teams precious time tracking down bugs and unexpected behavior.

Here are 10 of the pitfalls that litter the path to releasing your production ASP.NET applications, and what you can do to avoid them. All of the examples draw from my experiences with real companies building real Web applications, and in some cases I provide background by describing some of the problems that the ASP.NET development team encountered along the way.

LoadControl and Output Caching

Rare is the ASP.NET application that doesn’t employ user controls. Before the advent of Master Pages, developers employed user controls to factor out common content, such as headers and footers. Even in ASP.NET 2.0, user controls provide an effective means for encapsulating content and behavior, and for dividing pages into regions whose ability to be cached can be controlled independently of the page as a whole—a special form of output caching known as fragment caching.

User controls can be loaded declaratively or imperatively. Imperative loading relies on Page.LoadControl, which instantiates a user control and returns a Control reference. If the user control contains custom type members (for example, public properties), then you can cast that reference and access the custom members from your code. The user control in Figure 1 implements a property named BackColor. The following code loads the user control and assigns a value to BackColor:

protected void Page_Load(object sender, EventArgs e)
{
    // Load the user control and add it to the page
    Control control = LoadControl("~/MyUserControl.ascx");
    PlaceHolder1.Controls.Add(control);

    // Set its background color
    ((MyUserControl)control).BackColor = Color.Yellow;
}
This code, simple as it is, is a trap waiting to ensnare the unwary developer. Can you identify the flaw?

Figure 1 MyUserControl (Close Figure 1)

MyUserControl.ascx
<%@ Control Language="C#" AutoEventWireup="true"
  CodeFile="MyUserControl.ascx.cs" Inherits="MyUserControl"
%>
<h1><asp:Label ID="Label1" runat="server" Text="Label" /></h1>

MyUserControl.ascx.cs
using System;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Drawing;

public partial class MyUserControl : System.Web.UI.UserControl
{
    public Color BackColor
    {
        get { return Label1.BackColor; }
        set { Label1.BackColor = value; }
    }

    protected void Page_Load(object sender, EventArgs e)
    {
        Label1.Text = DateTime.Now.ToLongTimeString();
    }
}

If you guessed that the problem is related to output caching, you are correct. These code samples compile and run fine as shown, but try adding the following (perfectly legal) statement to MyUserControl.ascx:

<%@ OutputCache Duration="5" VaryByParam="None" %>
Next time you run the page, you’ll be greeted by an InvalidCastException (oh joy!) accompanied by the following error message:
"Unable to cast object of type ‘System.Web.UI.PartialCachingControl’ to type ‘MyUserControl’."

So, here’s code that works great without an OutputCache directive, but bombs when an OutputCache directive is added. ASP.NET isn’t supposed to work this way. Pages (and controls) are supposed to be agnostic towards output caching. So what gives?

The problem is that when output caching is enabled for a user control, LoadControl no longer returns a reference to an instance of the control; instead, it returns a reference to an instance of PartialCachingControl that might or might not wrap a control instance, depending on whether the control’s output is cached. Consequently, developers who call LoadControl to load a user control dynamically and who also cast the control reference in order to access control-specific methods and properties must take care in how they do it in order for the code to work with or without an OutputCache directive.

Figure 2 demonstrates the proper way to load user controls dynamically and cast the returned control references. Here’s a synopsis of how it works:

  • If the ASCX file lacks an OutputCache directive, LoadControl returns a MyUserControl reference. Page_Load casts the reference to MyUserControl and sets the control’s BackColor property.
  • If the ASCX file includes an OutputCache directive and the control’s output isn’t cached, LoadControl returns a reference to a PartialCachingControl whose CachedControl property contains a reference to the underlying MyUserControl. Page_Load casts PartialCachingControl.CachedControl to MyUserControl and sets the control’s BackColor property.
  • If the ASCX file includes an OutputCache directive and the control’s output is cached, LoadControl returns a reference to a PartialCachingControl whose CachedControl property is null. Seeing this, Page_Load does nothing more. It’s powerless to set the control’s BackColor property because the control’s output is delivered from the output cache. In other words, there is no MyUserControl on which to set a property.

Figure 2 The Proper Way to Load MyUserControl.ascx (Close Figure 2)
protected void Page_Load(object sender, EventArgs e)
{
    // Load the user control
    Control control = LoadControl("~/MyUserControl.ascx");
    PlaceHolder1.Controls.Add(control);

    // Set its background color (if possible)
    MyUserControl uc = control as MyUserControl;
    if (uc == null)
    {
        PartialCachingControl pcc = control as PartialCachingControl;
        if (pcc != null) uc = pcc.CachedControl as MyUserControl;
    }
    if (uc != null) uc.BackColor = Color.Yellow;
}

The code in Figure 2 will work with or without an OutputCache directive in the .ascx file. It’s not pretty, but it averts nasty surprises. Simpler doesn’t always equate to more maintainable.

Sessions and Output Caching

Speaking of output caching, there’s a potential issue with ASP.NET 1.1 and ASP.NET 2.0 that affects output-cached pages on servers running on Windows Server™ 2003 and IIS 6.0. I’ve personally seen this manifest itself on production ASP.NET servers twice, and both times it was resolved by turning off output caching. I later learned there’s a better solution that doesn’t require output caching to be disabled. Here’s how I first encountered the problem.

The saga began when a dot-com—let’s call it Contoso.com—that runs a public e-commerce app on a small ASP.NET Web farm contacted my team and complained that they were experiencing "cross-threading" errors. Every now and then, a customer using the Contoso.com Web site would suddenly lose the data they had entered and instead would see data corresponding to another user. A bit of sleuthing revealed that cross-threading wasn’t an accurate description; "cross-session" errors was more like it. It seems that Contoso.com was storing data in session state, and for some reason users were occasionally—and randomly—being connected to other users’ sessions.

One of my team members wrote a diagnostic tool to log key elements of each HTTP request and response, including cookie headers. Then he installed it on Contoso.com’s Web servers and let it run for a few days. The results were remarkable. Roughly once in every 100,000 requests, ASP.NET was correctly assigning a session ID to a brand new session and returning the session ID in a Set-Cookie header. It would then return the same session ID (that is, the same Set-Cookie header) in the very next request, even if that request was already associated with a valid session and was correctly submitting the session ID in a cookie. In effect, ASP.NET was randomly switching users away from their own sessions and connecting them to other sessions.

Astonished, we began to look for causes. We first examined Contoso.com’s source code and satisfied ourselves that the problem lay elsewhere. Next, just to be sure the problem wasn’t related to the fact that the application was hosted on a Web farm, we turned off all the servers but one. The problem persisted, which wasn’t surprising since our logs showed that the matching Set-Cookie headers never came from two different servers. It wasn’t credible that ASP.NET accidentally generated duplicate session IDs because it uses the .NET Framework RNGCryptoServiceProvider class to generate those IDs, and session IDs are of sufficient length to ensure that the same one will never be generated twice (not in the next trillion years, anyway). Besides, even if RNGCryptoServiceProvider was erroneously generating duplicate random numbers, that wouldn’t explain why ASP.NET mysteriously replaced valid session IDs with new (and non-unique) ones.

On a hunch, we decided to look at output caching. When OutputCacheModule caches HTTP responses, it must be careful not to cache Set-Cookie headers; otherwise, a cached response containing a new session ID would connect all recipients of the cached response (as well as the user whose request generated the cached response) to the same session. We checked the source code; Contoso.com had output caching enabled in two pages. We turned it off. Lo and behold, the application ran for days without a single cross-session incident. It has run without error for more than two years since. And we saw the exact same scenario play out at a different company with a different application and a different set of Web servers. As at Contoso.com, eliminating output caching made the problem go away.

Microsoft has since confirmed that this behavior stems from a problem in OutputCacheModule. (There may be an update available by the time you read this.) When ASP.NET is paired with IIS 6.0 and kernel-mode caching is enabled, OutputCacheModule sometimes fails to strip Set-Cookie headers from the cached responses it passes to Http.sys. Here’s the specific sequence of events that causes the bug to manifest itself:

  1. A user who hasn’t visited the site recently (and therefore doesn’t have a corresponding session) requests a page for which output caching is enabled, but whose output isn’t currently available in the cache.
  2. The request executes code that accesses the user’s newly created session, causing a session ID cookie to be returned in a Set-Cookie header in the response.
  3. OutputCacheModule provides the output to Http.sys, but fails to strip the Set-Cookie header from the response.
  4. Http.sys returns the cached response in subsequent requests, inadvertently connecting other users to the session.

The moral of the story? Session state and kernel-mode output caching don’t mix. If you use session state in a page that has output caching enabled, and if the application runs on IIS 6.0, then you need to turn off kernel-mode output caching. You’ll still get the benefit of output caching, but because kernel-mode output caching is substantially faster than ordinary output caching, the caching won’t be as effective. For more information on this issue, see support.microsoft.com/kb/917072.

You can turn off kernel-mode output caching for individual pages by including VaryByParam="*" attributes in the page’s OutputCache directives, although doing so can cause memory requirements to explode. The safer alternative is to turn off kernel-mode caching for the entire application by including the following element in web.config:

<httpRuntime enableKernelOutputCache="false" />
You can also disable kernel-mode output caching globally—that is, for entire servers—with a registry setting. For details, see support.microsoft.com/kb/820129.

Whenever I hear about inexplicable things happening with sessions, I ask the customer if they’re using output caching in any of their pages. If the answer is yes, and if the host OS is Windows Server 2003, then I advise them to disable kernel-mode output caching. The problem usually goes away. If it doesn’t, then the bug is in their code. Be warned!

Forms Authentication Ticket Lifetime

Can you spot the problem with this code:

FormsAuthentication.RedirectFromLoginPage(username, true);
As innocuous as it seems, this code should never be used in an ASP.NET 1.x app unless there is mitigating code elsewhere in the application to counteract this statement’s debilitating effects. If you’re not sure why, then read on.

FormsAuthentication.RedirectFromLoginPage performs two tasks. First, it redirects a user to the page they originally requested when they were redirected by FormsAuthenticationModule to the login page. Second, it issues an authentication ticket—typically carried in a cookie, and always carried in a cookie in ASP.NET 1.x—that allows the user to remain authenticated for a predetermined period of time.

The problem is the period of time. In ASP.NET 1.x, passing RedirectFromLoginPage a second parameter equal to false issues a temporary authentication ticket that expires, by default, after 30 minutes. (You can change the time-out period using a timeout attribute in web.config’s <forms> element.) Passing a second parameter equal to true, however, issues a persistent authentication ticket that’s valid for—get this—50 years! That’s an accident waiting to happen, because if someone steals that authentication ticket, they can access the Web site using the victim’s identity for the life of the ticket. There’s no shortage of ways to swipe authentication tickets—sniffing unencrypted traffic at public wireless access points, cross-site scripting, gaining physical access to a victim’s PC, and so on—so passing true to RedirectFromLoginPage is little better than disabling security on your Web site. Fortunately, this problem was fixed in ASP.NET 2.0. Today’s RedirectFromLoginPage honors the timeout specified in web.config for temporary and persistent authentication tickets alike.

One solution is to never pass true in RedirectFromLoginPage’s second parameter in ASP.NET 1.x apps. But that’s not very practical because login pages typically feature a "Keep me signed in" box that users can check to receive persistent rather than temporary authentication cookies. An alternate solution is a snippet of code in Global.asax (or, if you prefer, an HTTP module) that modifies cookies containing persistent authentication tickets before they go back to the browser.

Figure 3 contains one such snippet. If present in Global.asax, this code modifies the Expires property of outgoing persistent forms authentication cookies so that the cookies expire after 24 hours. By modifying the line commented "New expiration date," you can set the timeout to whatever you like.

Figure 3 Limiting Persistent Authentication Cookie Lifetime (Close Figure 3)
void Application_EndRequest(Object sender, EventArgs e)
{
    // Change the expiration date on outgoing persistent forms
    // authentication tickets to 24 hours hence.
    HttpCookie cookie1 = GetCookieFromResponse(
        FormsAuthentication.FormsCookieName);

    if (cookie1 != null && !String.IsNullOrEmpty (cookie1.Value))
    {
        FormsAuthenticationTicket ticket1 = FormsAuthentication.Decrypt(
            Response.Cookies[FormsAuthentication.FormsCookieName].Value);
        if (ticket1.IsPersistent)
        {
            FormsAuthenticationTicket ticket2 =
                new FormsAuthenticationTicket (
                ticket1.Version, ticket1.Name, ticket1.IssueDate,
                DateTime.Now.AddHours (24), // New expiration date
                ticket1.IsPersistent, ticket1.UserData,
                ticket1.CookiePath
            );

            Response.Cookies.Remove(FormsAuthentication.FormsCookieName);
            HttpCookie cookie2 = new HttpCookie(
                FormsAuthentication.FormsCookieName,
                FormsAuthentication.Encrypt(ticket2));
            cookie2.Expires = ticket2.Expiration;
            Response.Cookies.Add(cookie2);
        }
    }
}

HttpCookie GetCookieFromResponse (string name)
{
    HttpCookieCollection cookies =
        HttpContext.Current.Response.Cookies;
    int count = cookies.Count;
    for (int i=0; i<count; i++) {
        if (String.Compare (cookies[i].Name, name, true) == 0)
            return cookies[i];
    }
    return null;
}

You may find it curious that the Application_EndRequest method calls a local helper method (GetCookieFromResponse) to check outgoing responses for authentication cookies. The helper method is a work-around for another bug in ASP.NET 1.1 that causes a bogus cookie to be added to the response if you check for a nonexistent cookie using HttpCookieCollection’s string indexer. Using the integer indexer as GetCookieFromResponse circumvents the problem.

View State: The Silent Perf Killer

In some ways, view state is the greatest thing since sliced bread. After all, it’s view state that allows pages and controls to persist state across postbacks. That’s why you don’t have to write code to keep the text in a TextBox from disappearing when a button is clicked as you did in classic ASP, or requery a database and rebind a DataGrid following a postback.

But view state has a dark side, too: when it grows too large, it’s a silent performance killer. Some controls, such as TextBoxes, are judicious with view state. Others, notably DataGrids and GridViews, emit view state in proportion to the amount of information displayed. I cringe when I see a GridView displaying 200 or 300 rows of data. Even though ASP.NET 2.0 view state is roughly half the size of ASP.NET 1.x view state, one lousy GridView can easily cut the effective bandwidth of a connection between a browser and a Web server by 50 percent or more.

You can turn off view state for individual controls by setting EnableViewState to false, but some controls, particularly DataGrids, lose some of their functionality when denied the freedom to use view state. A much better solution to taming view state is keeping it on the server. In ASP.NET 1.x, you can override a page’s LoadPageStateFromPersistenceMedium and SavePageStateToPersistenceMedium methods and handle view state however you like. The overrides shown in the code in Figure 4 prevent view state from being persisted in a hidden field and persist it in session state instead. Storing view state in session state is particularly effective when paired with the default session state process model—that is, when session state is stored in memory in the ASP.NET worker process. If session state is stored in a database instead, then only testing will show whether keeping view state in session state improves or degrades performance.

Figure 4 Storing View State in Session State in ASP.NET 1.x (Close Figure 4)
protected override object LoadPageStateFromPersistenceMedium ()
{
    string key = Request.RawUrl + &quot;_VIEWSTATE&quot;;
    object state = Session[key];
    return (state == null) ?
        base.LoadPageStateFromPersistenceMedium () : state;
}

protected override void
    SavePageStateToPersistenceMedium (object viewState)
{
    string key = Request.RawUrl + &quot;_VIEWSTATE&quot;;
    Session[key] = viewState;
}

The same technique works in ASP.NET 2.0, but ASP.NET 2.0 offers a simpler means for persisting view state in session state. You begin by defining a custom page adapter whose GetStatePersister method returns an instance of the .NET Framework SessionPageStatePersister class:

public class SessionPageStateAdapter :
    System.Web.UI.Adapters.PageAdapter
{
    public override PageStatePersister GetStatePersister ()
    {
        return new SessionPageStatePersister(this.Page);
    }
}
Then you register the custom page adapter as the default page adapter by dropping an App.browsers file like the following into the application’s App_Browsers folder:
<browsers>
  <browser refID="Default">
    <controlAdapters>
      <adapter controlType="System.Web.UI.Page"
        adapterType="SessionPageStateAdapter" />
    </controlAdapters>
  </browser>
</browsers>
(You can name the file anything you like as long as it has a .browsers extension.) Thereafter, ASP.NET will load the page adapter and use the returned SessionPageStatePersister to persist all page state, including view state.

One downside to using a custom page adapter is that it acts globally for every page in the application. If you’d prefer to persist view state in session state for some pages but not for others, use the technique shown in Figure 4. Additionally, you can run into issues with this technique if a user creates multiple browser windows within the same session.

SQL Server Session State: Another Perf Killer

ASP.NET makes it easy to store session state in databases: just flip a switch in web.config and session state magically moves to a back-end database. This is a critical feature for applications that run on Web farms because it allows every server in the farm to share a common repository for session state. The added database activity slows the performance of individual requests, but the loss in performance is offset by an increase in scalability.

This is all fine and good until you take a moment to ponder the following points:

  • Even in an application that uses session state, most pages do not use session state.
  • By default, the ASP.NET session state manager performs two accesses—one read access and one write access—to the session data store in every request, regardless of whether the page requested uses session state.

In other words, when you use the SQL Server™ session state option, you pay the price (two database accesses) in every request—even in requests for pages that do nothing with session state. This directly (and adversely) affects the throughput of the entire site.

Figure 5 Eliminating Unnecessary Session State Database Accesses
Figure 5 Eliminating Unnecessary Session State Database Accesses

So what do you do about it? Simple: you disable session state in pages that don’t use it. It’s always a good idea to do so, but it’s especially important when session state is stored in a database. Figure 5 shows how to disable it. If a page doesn’t use session state at all, include an EnableSessionState="false" in its Page directive, like so:

<%@ Page EnableSessionState="false" ... %>
This directive prevents the session state manager from reading and writing the session state database in each request. If a page reads data from session state but doesn’t write it (that is, doesn’t modify the contents of the user’s session), then set EnableSessionState to ReadOnly, as shown here:
<%@ Page EnableSessionState="ReadOnly" ... %>
Finally, if a page requires read/write access to session state, then either omit the EnableSessionState attribute or set it to true:
<%@ Page EnableSessionState="true" ... %>
By taming session state in this way, you’ll ensure that ASP.NET only accesses the session state database when it really needs to. Eliminating unnecessary database accesses is the first step toward building high-performance applications.

Incidentally, the EnableSessionState attribute is no secret. It’s been documented since ASP.NET 1.0, yet I rarely see developers take advantage of it. Perhaps this is because it’s not terribly important with the default in-memory session state model. But it’s critical with the SQL Server model.

Uncached Roles

The following statement is frequently found in the web.config files of ASP.NET 2.0 applications and in samples introducing the ASP.NET 2.0 role manager:

<roleManager enabled="true" />
As presented, however, this statement can have a demonstrably negative impact on performance. Do you know why?

By default, the ASP.NET 2.0 role manager doesn’t cache roles data. Instead, it consults the roles data store each time it needs to determine which roles, if any, a user belongs to. This means that once a user is authenticated, any page that utilizes role data—for example, pages that use site maps with security trimming enabled, and pages to which access is restricted using role-based URL directives in web.config—causes the role manager to query the roles data store. If roles are stored in a database, that’s one more database access per request that you can easily do without. The solution is to configure the role manager to cache roles data in cookies:

<roleManager enabled="true" cacheRolesInCookie="true" />
You can use other <roleManager> attributes to control the characteristics of role cookies—for example, how long the cookies should remain valid (and consequently how frequently the role manager will go back to the roles database). Role cookies are signed and encrypted by default, so the security risk, while not zero, is mitigated.

Profile Property Serialization

The ASP.NET 2.0 profile service provides a ready-made solution to the problem of persisting per-user state, such as personalization preferences and language preferences. To use the profile service, you define an XML profile containing the properties you want to persist on behalf of individual users. ASP.NET then compiles a class containing the same properties and provides strongly typed access to class instances via the profile property added to the page.

Profiles are flexible enough to allow even custom data types to be used as profile properties. But therein lies the problem—one that I’ve personally seen trip developers up. Figure 6 contains a simple class named Posts, as well as a profile definition that uses Posts as a profile property. However, this class and this profile produce unexpected behavior at run time. Can you determine why?

Figure 6 Custom Data Type Used Incorrectly (Close Figure 6)

Class Definition
public class Posts
{
    private int _count = 0;

    public void IncrementPostCount()
    {
        _count++;
    }

    public int GetPostCount()
    {
        return _count;
    }
}

Profile Definition
<profile>
  <properties>
    <add name="Posts" type="Posts" />
  </properties>
</profile>

The problem is that Posts contains a private field named _count that must be serialized and deserialized to fully hydrate and rehydrate class instances. But _count doesn’t get serialized and deserialized because it’s private and, by default, the ASP.NET profile manager uses XML serialization to serialize and deserialize custom types. The XML serializer ignores nonpublic members. Therefore, instances of Posts will get serialized and deserialized, but each time a class instance is deserialized, _count is reset to 0.

One solution is to make _count public rather than private. Another is to wrap _count with a public read/write property. The best solution, and one that preserves the design of the class itself, is to mark Posts as serializable (using the SerializableAttribute) and to configure the profile manager to use the .NET Framework binary serializer to serialize and deserialize class instances. The binary serializer, unlike the XML serializer, serializes fields, regardless of accessibility. Figure 7 shows a fixed version of the Posts class and the accompanying profile definition with changes highlighted.

Figure 7 Custom Data Type Used Properly (Close Figure 7)

Class Definition
[Serializable]
public class Posts
{
    private int _count = 0;

    public void IncrementPostCount()
    {
        _count++;
    }

    public int GetPostCount()
    {
        return _count;
    }
}

Profile Definition
<profile>
  <properties>
    <add name="Posts" type="Posts" serializeAs="Binary" />
  </properties>
</profile>

The thing you should remember is that if you use a custom data type as a profile property, and if that data type has nonpublic data members that must be serialized in order to fully serialize type instances, then use a serializeAs="Binary" attribute in the property declaration and make sure the type itself is serializable. Otherwise, complete serialization will not occur and you’ll waste time trying to determine why the profile isn’t working.

Thread Pool Saturation

I am often amazed at the number of real-life ASP.NET pages I see that perform database queries and wait 15 seconds or more for the queries to return. (I’ve also seen queries that take 15 minutes!) Sometimes the delay is an unavoidable by-product of the volume of data returned; other times it’s the result of poor database design. But whatever the reason, long database queries or lengthy I/O operations of any kind are throughput killers in an ASP.NET app.

I’ve written about this problem at length before, so I won’t belabor the point here. Suffice it to say that ASP.NET relies on a finite thread pool to process requests, and if all the threads are occupied waiting for database queries, Web service calls, or other I/O operations to complete, additional requests must be queued up until an operation completes and a thread is freed. Performance drops off precipitously when requests are queued. And if the queue fills up, ASP.NET fails subsequent requests with HTTP 503 errors. This is not a situation a production app on a production Web server ever wants to find itself in.

The solution, of course, is asynchronous pages—one of the best but lesser-known features of ASP.NET 2.0. A request for an asynchronous page begins its life on one thread, but returns that thread and an IAsyncResult interface to ASP.NET when it begins an I/O operation. When the operation completes, the request signals ASP.NET through IAsyncResult and ASP.NET pulls another thread from the pool and finishes processing the request. Significantly, no thread pool threads are consumed while the I/O operation takes place. This can dramatically improve throughput by preventing requests for other pages—pages that don’t perform lengthy I/O operations—from waiting in a queue.

You can read all about asynchronous pages in the October 2005 issue of MSDN®Magazine. Any page that is I/O-bound rather than compute-bound and that takes more than a few seconds to execute is a candidate to be an asynchronous page.

When I tell developers about asynchronous pages, they often reply, "That’s cool, but I don’t really need them in my app." To which I reply, "Do any of your pages query a database? Do any of them call a Web service? Have you checked the ASP.NET performance counters for stats regarding queued requests and average wait times? Even if your app runs fine today, is it possible that the load on it will increase as your customer base grows?"

The reality is that few real-world ASP.NET apps have no need whatsoever for asynchronous pages. Write it down!

Impersonation and ACL Authorization

Here’s a simple configuration directive, but one that raises my eyebrows whenever I see it in web.config:

<identity impersonate="true" />

This directive enables client impersonation in an ASP.NET app. It attaches access tokens representing clients to the threads that process requests so that any security checks performed by the operating system act against the client’s identity rather than the worker process’s identity. Impersonation is rarely necessary in an ASP.NET app; my experience is that when developers enable it, they usually do so for the wrong reasons. Here’s why.

Too often, developers enable impersonation in ASP.NET apps so they can use file system permissions to restrict access to pages. If Bob doesn’t have permission to view Salaries.aspx, then the devs enable impersonation so they can prevent Bob from viewing Salaries.aspx by setting the access control list (ACL) to deny Bob read permission. But here’s the kicker: impersonation isn’t necessary for ACL authorization. When Windows authentication is enabled in an ASP.NET app, ASP.NET automatically checks the ACL for each .aspx page requested and denies the request if the caller lacks permission to read the file. It does so even if impersonation is disabled.

There are times when impersonation is justified. But you can usually avoid it with good design. For example, suppose Salaries.aspx queries a database for salary info that should only be available to managers. By impersonating, you can use database permissions to deny non-managers the ability to query for salary data. Or you can forget about impersonation and restrict access to salary data by setting the ACL for Salaries.aspx so that non-managers lack read permission. The latter approach provides better performance because it avoids impersonation altogether. It eliminates unnecessary database accesses, too. Why query a database only to be denied for security reasons?

Incidentally, I once helped troubleshoot a classic ASP app that periodically rebooted due to unconstrained memory consumption. A junior developer had turned a targeted SELECT statement into a SELECT *, not considering the fact that the table being queried contained images—large images, and lots of them. The problem was exacerbated by an undetected memory leak. (My kingdom for managed code!) Suddenly, an app that had run fine for years began grinding to a halt because SELECT statements that previously returned a kilobyte or two of data now returned several megabytes. Coupled with inadequate version control, it made life pretty exciting for the development team—exciting, that is, if you consider sleeping at night and seeing your kids’ soccer games boring.

In theory, classic memory leaks can’t happen in ASP.NET applications composed entirely of managed code. But inefficient memory usage impacts performance by forcing garbage collections to occur more frequently. Even in ASP.NET apps, beware of SELECT *!

Don’t Just Trust It—Profile Your Database!

As a consultant, I’m often called in when an app isn’t performing the way it should. Recently my team was asked to determine why an ASP.NET app was only achieving about 1/100th of the throughput (requests per second) that the requirements document called for. What we discovered was typical of what we find in underperforming Web apps—and a lesson that all of us can take to heart.

We ran SQL Server Profiler and watched the interactions between the app and the database on the back end. In one of the more extreme cases, a simple button click produced more than 1,500 trips to the database. You can’t build a high-performance application that way. Good architecture always begins with good database design. No matter how efficient your code is, it’s hamstrung if saddled with a poorly written database.

Bad data access architectures typically stem from one or more of the following:

  • Poor database design (usually designed by developers, not database administrators).
  • Use of DataSets and DataAdapters—in particular, DataAdapter.Update, which is great for Windows Forms apps and other thick clients, but usually not ideal for Web apps.
  • Poorly designed data access layers (DALs) with poor factorization and too many CPU cycles expended performing relatively simple operations.

Problems must be identified before they can be treated. And the way to identify data access problems is to run SQL Server Profiler or an equivalent tool to see what’s going on behind the scenes. Performance tuning isn’t complete until you’ve examined the traffic between the app and the database. Try it—you may be surprised by what you find.

Conclusion

Now you’ve seen some of the problems and solutions you’re likely to encounter in the process of building production ASP.NET apps. The next step is to take a close look at your own code and try to avoid some of the issues I’ve outlined here. ASP.NET may lower the bar for Web developers, but there’s no reason your apps can’t be slick, solid, and speedy. Put on your thinking cap and avoid making those rookie mistakes.

Figure 8 provides a short checklist you can use to avoid the pitfalls described in this article. You can build a similar checklist for security pitfalls. For example:

  • Have you encrypted configuration sections containing sensitive data?
  • Are you checking and validating input used in database operations, and are you HTML-encoding input used as output?
  • Do any of your virtual directories contain files with unprotected extensions?

Figure 8 ASP.NET Pitfalls Checklist (Close Figure 8)

Does your app use LoadControl to load user controls dynamically? If so, make sure this is done in such a way that LoadControl will work both with and without output caching.
Are you using kernel-mode output caching and session state in the same page? If so, remove the OutputCache directive to avoid caching that page or turn kernel-mode output caching off for the entire app.
Are you calling FormsAuthentication.RedirectFromLoginPage in your ASP.NET 1.x-based app? If so, either change the second parameter to false or modify the Expires property of outgoing persistent forms authentication cookies to a more appropriate timeout.
Do any of your pages generate inordinate amounts of view state? If so, turn off view state for individual controls or keep it on the server.
Does your app use SQL Server session state? If so, make sure you disable sessions in pages that don’t use sessions.
Does your app use the ASP.NET 2.0 role manager? If so, enable role caching to increase performance.
Does your app use custom data types as profile properties? If so, you'll avoid problems by creating custom data types that are compatible with XML serialization or by attributing the types as serializable and using the binary serializer to serialize and deserialize types.
Does your app do database queries that take a long time to return? If so, consider whether the pages are candidates for becoming asynchronous.
Does your ASP.NET app use client impersonation? If so, simple authentication in ASP.NET might enforce ACL-based permissions with less hassle.
Have you profiled your app’s database activity? Examining communication between the app and database is a good first performance-tuning step.

These are important questions to ask if you value the integrity of your Web site, the servers that host your site, and the back-end resources upon which these rely.

Jeff Prosise is a contributing editor to MSDN Magazine and the author of several books, including Programming Microsoft .NET (Microsoft Press, 2002). He is also a cofounder of Wintellect, a software consulting and education firm.


WSS 3.0 Preview: Discover Significant Developer Improvements In SharePoint Services

WSS 3.0 Preview

Discover Significant Developer Improvements In SharePoint Services

Ted Pattison
This article is based on a prerelease version of Windows SharePoint Services. All information herein is subject to change.
This article discusses:
  • Integration with ASP.NET 2.0 Web Parts
  • Enhancements in content storage and site configuration
  • Workflows in WSS
  • Internet-style security features
This article uses the following technologies:
WSS, Windows Server 2003
Contents
Integration with ASP.NET 2.0
Page Templates
Working with Master Pages
Web Parts in WSS
Developing Custom Web Parts
Enhancements in Content Storage
Event Handlers
Workflows in WSS
Site Definitions, Features, and Solutions
Internet-Style Security
Conclusion

T
he next major release of Microsoft Office SharePoint Server (MOSS), including Windows® SharePoint® Services (WSS) version 3.O, is fast approaching release. As with previous versions, WSS supplies out-of-the-box collaboration features that make it simple for users to create and design Web sites with shared elements such as calendars, contacts lists, and document libraries. But WSS is far more than a collaboration tool for end users. It is a full-fledged development platform that adds tremendous value to ASP.NET.

WSS 3.0 is an add-on for Windows Server™ 2003. At its core, WSS acts as a scalable site-provisioning engine, easing the process of creating and managing hundreds or thousands of Web sites and making them accessible to tens of thousands of users. WSS scalability is achieved by using an architecture designed with a Web-farm environment in mind. This architecture is based on stateless front-end Web servers that rely on SQL Server™ for storing content and other site-related data.

The added value WSS brings to the ASP.NET 2.0 development platform results from its extensibility model, which facilitates provisioning and storage for pages, lists, and document libraries. The provisioning can be driven through either custom code or user actions in the browser-based UI. Behind the scenes, WSS automatically works out how and where to store the content, and it also supplies the UI elements that let users add, view, and modify content.

I’m going to concentrate on the most significant developer enhancements to version 3, and on the accompanying changes in terminology. For more information, see "SharePoint: Use Windows SharePoint Services as a Platform for Building Collaborative Applications" in the July 2004 issue of MSDN®Magazine.

Keep in mind that research and code samples for this article are based on the Beta 1 release of WSS. It’s possible that some of the terms and code might change in the released version.

Integration with ASP.NET 2.0

WSS 3.0 provisioning starts with an IIS Web site. Before you can create your first WSS site, you need to run an administrative procedure that extends the WSS functionality to one or more IIS Web sites. With WSS 2.0, the term "virtual server" was used to describe an IIS Web site extended with SharePoint functionality. To avoid confusion with another Microsoft product of the same name, the WSS 3.0 documentation now refers to an IIS Web site extended with WSS functionality as a Web application.

WSS 2.0 was integrated with IIS 6.0 and ASP.NET 1.1 using an ISAPI filter DLL, which resulted in IIS routing requests to WSS before ASP.NET. That routing could be problematic in certain situations because WSS takes control of an incoming HTTP request before it has had a chance to be properly initialized with ASP.NET context.

But WSS integration with ASP.NET has been completely redesigned in version 3. First, WSS 3.0 is built on ASP.NET 2.0, which provides significant enhancements over ASP.NET 1.1. Furthermore, the routing infrastructure was improved by removing the ISAPI filter and by adding an HttpModule and an HttpHandler that are registered with ASP.NET using standard web.config entries. This means that incoming HTTP requests always enter the ASP.NET runtime environment and are fully initialized with ASP.NET context before they are forwarded to the code that carries out WSS-specific processing.

When you extend an IIS Web site to become a Web application, WSS 3.0 adds a wildcard application map to the IIS metabase. This map routes all incoming HTTP requests to the ASP.NET runtime regardless of file type (.pdf, .doc, .docx, and so on). ASP.NET then forwards the request to WSS for processing.

The new architecture also deals with shortcomings related to how .aspx pages are parsed and compiled. The .aspx page parser used by ASP.NET 1.1 works only with pages that reside on the local file system. But WSS architecture relies on storing .aspx pages in a SQL Server database so it couldn’t take advantage of the ASP.NET 1.1 page parser. And, unfortunately, the SharePoint parser developed in its place does not support many of the cooler features offered by the ASP.NET page parser.

But ASP.NET 2.0 introduced a new pluggable component type known as a virtual path provider. The idea is that a developer can write a custom component that retrieves .aspx pages (or any other content that ASP.NET processes) from any location, including a database such as SQL Server. Once a custom virtual path provider retrieves an .aspx page, it can then hand the page off to ASP.NET to conduct the required parsing and compilation. A separate component, PageParserFilter, lets SharePoint control how pages are parsed and compiled and what executes on those pages.

WSS 3.0 includes its own virtual path provider named SPVirtualPathProvider, as illustrated in Figure 1. As you can see, SPVirtualPathProvider is able to retrieve .aspx pages from SQL Server and then hand them off to the .aspx parser that is supplied by ASP.NET 2.0. As a result, WSS 3.0 does not suffer from a reduced feature set with respect to page parsing as did the previous version of WSS.

Figure 1 Custom Virtual Path Provider and ASP.NET 2.0
Figure 1 Custom Virtual Path Provider and ASP.NET 2.0

Page Templates

The terms ghosting and unghosting are often used in conjunction with the .aspx pages of a WSS 2.0 site. Page ghosting lets a front-end Web server store an .aspx page template on its local file system and share that template across many different sites. Page ghosting offers performance benefits because WSS can serve up pages for thousands of sites using a page template stored on the local file system and loaded into memory just once.

WSS 2.0 supported user modifications to the page template using tools such as Microsoft® Office FrontPage® 2003. When a user modifies a page template and saves the changes, a customized version of the page is stored in SQL Server for that particular site. This is often referred to as unghosting a page.

WSS 3.0 still supports page templates that live on the Web server as well as customized versions that are stored in SQL Server. However, the documentation no longer uses the terms ghosting and unghosting. Now the term "uncustomized page" describes a page template used directly from the local file system of the Web server, while "customized page" is used when talking about a modified version of the page template that has been written to the content database for a particular site.

Microsoft Office FrontPage 2003 has been renamed Office SharePoint Designer 2007, but its target audience remains users rather than developers. Still, it’s a handy tool for WSS developers. SharePoint Designer provides both a code editor and a WYSIWYG designer for customizing .aspx and .master pages within WSS sites, and you can create new pages within a site that has no corresponding page template on the Web server. You can also create lists and document libraries and there’s even a new wizard for creating custom workflows in a WSS site. (I’ll discuss this in more depth later.)

Working with Master Pages

One of the most tedious aspects of customizing and branding sites in WSS 2.0 involved creating a consistent appearance across pages, due to the fact that ASP.NET 1.1 didn’t provide any suitable page templating techniques. As a result, many developers and designers resorted to copying HTML layouts and pasting them from page to page. As you can imagine, this makes it very hard to customize and maintain a site whose layout requirements differ from the standard WSS site.

A remedy appeared in the form of a powerful page templating feature known as Master Pages, introduced with ASP.NET 2.0. A Master Page is a template that allows you to define a standard page layout for an entire site with elements such as a banner, navigation controls, and other menus. The pages that link to a Master Page are known as content pages.

The key concept is that each content page links to the Master Page to acquire the shared layout and then extends the Master Page by inserting customized content in named placeholders. For more information about how Master Pages work in ASP.NET, see Fritz Onion’s article "Master Pages: Master Your Site Design with Visual Inheritance and Page Templates" in the June 2004 issue of MSDN Magazine.

WSS 3.0 was designed from the ground up to embrace the Master Page infrastructure of ASP.NET 2.0. Every version 3 site is provisioned with a special catalog known as the Master Page gallery that contains a page named default.master. This page defines a common layout for every site’s home page (default.aspx) as well as all the standard WSS form pages associated with lists and document libraries (AllItems.aspx and NewItem.aspx, for example). The Master Page layout includes standard menus and navigation controls. Figure 2 shows an example page based on the standard layout defined by default.master.

Figure 2 Standard Layout of Pages Defined by default.master
Figure 2 Standard Layout of Pages Defined by default.master

The default.master definition includes a number of named placeholders such as PlaceHolderPageTitle, PlaceHolderMain, and PlaceHolderLeftNavigation, making it relatively simple to create a new custom content page with the same layout as other pages in a site. Take a look at the simplicity of the content page definition:

  <%@ Page language="VB" MasterPageFile="~masterurl/
    default.master" %>
  <asp:Content ContentPlaceHolderId="PlaceHolderMain"
    runat="server">
    <h3>Welcome to customization with Windows      SharePoint Services 3</h3>
    This is so much easier than it was in version 2!
                     </asp:Content>

The content page definition does nothing beyond placing a minimal fragment of HTML content inside PlaceHolderMain, but it produces the typical page layout with site icons, menus, and navigation bars as shown in Figure 3. Once you learn how to use the standard set of placeholders defined in default.master, you can easily replace WSS elements like menus and navigation bars with your own ASP.NET controls. This can be done for pages at the scope of a site or a site collection.

Figure 3 Custom Content Page from a Master Page
Figure 3 Custom Content Page from a Master Page

Master Pages and content pages are stored and loaded like the page templates and page customization discussed earlier. There are page templates defined for the Master Page as well as for content pages that reside on the local file system of the front-end Web server. Each site initially uses uncustomized (ghosted) versions of the Master Page template and content page templates. When a user customizes and saves one of these pages for a particular site using a tool such as SharePoint Designer, a customized (unghosted) version is saved into the SQL Server database.

You can customize the Master Page for a site if you like, leaving the content pages uncustomized. Or you can customize one or more content pages while leaving the Master Page uncustomized. And you can undo any changes; both WSS and SharePoint Designer provide simple menu commands to discard customization changes from the SQL Server database and revert back to the original page template.

As you might have noticed, the content page that was shown previously links to the Master Page using a special syntax in the form of ~masterurl/default.master. This is a tokenized reference to a Master Page that can be changed programmatically on a site-wide basis. To do this, you acquire an SPWeb reference to the current site and then update the MasterUrl property (see Figure 4).

Figure 4 Acquiring an SPWeb Reference (Close Figure 4)
Imports System.Windows.Forms
Imports Microsoft.SharePoint

Public Partial Class frmMain : Inherits Form
    Sub cmdSetMasterUrl_Click( _
            ByVal sender As Object, ByVal e As EventArgs)

        Using SiteCollection As New SPSite("http://localhost")
            Using TopLevelSite As SPWeb = SiteCollection.OpenWeb()
                TopLevelSite.MasterUrl = txtMasterUrl.Text
                TopLevelSite.Update()
            End Using
        End Using

    End Sub
End Class

Note that each site has its own Master Page gallery with a default.master and its own MasterUrl property. That means all the sites inside a site collection do not automatically use the same Master Page. However, with the use of recursion, it’s pretty easy to write some code against the WSS 3.0 object model to synchronize a top-level site and all the child sites in a site collection to use the same Master Page, as shown in Figure 5.

Figure 5 Synchronizing Master Pages (Close Figure 5)
Sub cmdSynchronizeChildren_Click( _
        ByVal sender As Object, ByVal e As EventArgs)

    Using SiteCollection As New SPSite("http://localhost")
        Using TopLevelSite As SPWeb = SiteCollection.OpenWeb()
            UpdateMasterUrlRecursive(TopLevelSite, txtMasterUrl.Text)
        End Using
    End Using

End Sub

Sub UpdateMasterUrlRecursive(ByVal site As SPWeb, ByVal Url As String)
    site.MasterUrl = Url
    site.Update()
    For Each child As SPWeb In site.Webs
        Using child
            UpdateMasterUrlRecursive(child, Url)
        End Using
    Next
End Sub

Another useful dynamic token for Master Pages takes the form of ~masterurl/custom.master. This dynamic token works in conjunction with the CustomMasterUrl property of a site and provides a secondary target Master Page that can be redirected programmatically. There are also two static Master Page tokens which start with either ~site or ~sitecollection. These static tokens allow you to hardcode a relative path to a Master Page from either the root of the current site or the root of current site collection.

Web Parts in WSS

One of the most popular ways for developers to extend WSS 2.0 sites was to create custom Web Parts. Web Parts are great because they add the extra dimensions of user customization and personalization. Largely due to the popularity of Web Parts in WSS 2.0, Microsoft decided to add support for a new Web Part infrastructure for ASP.NET 2.0 that is similar yet distinct from the Web Part infrastructure in WSS.

Consequently, there are now two different Web Part styles. The older WSS-style Web Parts have a dependency on Microsoft.SharePoint.dll and must inherit from the WebPart base class in the Microsoft.SharePoint.WebPartPages namespace. The newer ASP.NET-style Web Parts have a dependency on System.Web.dll and must inherit from a different base class named WebPart in the System.Web.UI.WebControls.WebParts namespace.

It was an important design goal for WSS 3.0 to run both the WSS-style Web Parts and the ASP.NET-style Web Parts. This goal was accomplished by building WSS 3.0 support for Web Parts on top of the ASP.NET Web Part infrastructure and then changing the Microsoft.SharePoint.dll so that WSS-style Web Parts written for the WSS 2.0 environment would be compatible with the version 3 runtime environment.

To run Web Parts in an ASP.NET 2.0 application, you must create an .aspx page that contains exactly one instance of the WebPartManager control and one or more WebPartZone controls. The WebPartManager is responsible for serializing Web Part-related data as well as storing and retrieving it from the tables in the ASP.NET services database.

The .aspx page serving as a Web Part Page can contain editor parts that allow users to customize and personalize persistent Web Part properties. Web Part Pages can also contain catalog parts that let users add new Web Parts into zones. WSS 3.0 does the work to add the catalog and editor parts for you, so a Web page designer does not need to do so explicitly. (For more background information about how the ASP.NET 2.0 Web Part infrastructure works, read the article "ASP.NET 2.0: Personalize Your Portal with User Controls and Custom Web Parts".

The WSS 3.0 Web Part infrastructure is built on top of a control named SPWebPartManager that is derived from the ASP.NET 2.0 WebPartManager control. The SPWebPartManager control overrides the standard behavior of the WebPartManager control to persist Web Part data inside the WSS content database instead of in the ASP.NET services database. In most cases, you don’t have to worry about dealing directly with the SPWebPartManager control because the one and only required instance is already defined in default.master. When you create a content page that links to default.master, the SPWebPartManager control is already there.

The other controls that appear on a typical WSS 3.0 Web Part Page are shown in Figure 6. Note that Web Part zones for a Web Part Page in WSS 3.0 should be created using the WebPartZone control defined inside Microsoft.SharePoint.WebPartPages namespace, not using the standard WebPartZone control from ASP.NET 2.0.

Figure 6 Controls in a Web Part Page
Figure 6 Controls in a Web Part Page

Instances of the WebPartZone control are usually defined in content pages. Figure 7 shows a simple example of creating a content page designed to act as Web Part Page in a WSS 3.0 site. As you can see, this .aspx file links to default.master just like the earlier example. However, it also explicitly inherits from the WebPartPage base class and adds two WebPartZone controls into PlaceHolderMain.

Figure 7 Creating a Custom Web Part Page (Close Figure 7)
<%@ Assembly Name="[Fully-qualified name for
    Microsoft.SharePoint.dll]" %>
<%@ Page language="VB" MasterPageFile="~masterurl/default.master"
    Inherits="Microsoft.SharePoint.WebPartPages.WebPartPage" %>
<%@ Register Tagprefix="WebPartPages"
    Namespace="Microsoft.SharePoint.WebPartPages"
    Assembly="[Fully-qualified name for Microsoft.SharePoint]"  %>

<asp:Content ContentPlaceHolderId="PlaceHolderMain" runat="server">
  <h3>My Custom Web Part Page</h3>
  <table border="5" cellpadding="5" cellspacing="0">
    <tr>
      <td valign="top">
        <WebPartPages:WebPartZone runat="server" ID="Left"
                                  Title="Left Zone" />
      </td>
      <td valign="top">
        <WebPartPages:WebPartZone runat="server" ID="Right"
                                  Title="Right Zone" />
      </td>
    </tr>
  </table>
</asp:Content>

When you create a Web Part Page for a standard ASP.NET 2.0 application, you need to add logic that interacts with the WebPartManager control to manage the Web Part display mode, and generally you also need to explicitly add editor parts and catalog parts to the page along with the HTML layout to accommodate them. Fortunately, you don’t have to do these things when creating content pages for a WSS 3.0 site. Instead, you inherit from the WebPartPage class that’s defined inside the Microsoft.SharePoint.WebPartPages namespace and let it do all this work for you behind the scenes.

Developing Custom Web Parts

The preferred approach to creating Web Parts for WSS 3.0 sites is to create ASP.NET-style Web Parts. At a minimum this involves creating a class that inherits from the WebPart base class defined in the System.Web.UI.WebControls.WebParts namespace and overriding the RenderContents method. If you want to add persistent Web Part properties, you use the same technique that is used in ASP.NET.

The Web Part definition shown in Figure 8 has no dependencies on Microsoft.SharePoint.dll so it can be used in either a standard ASP.NET application or a WSS 3.0 site. In many cases, though, you’ll want to add a reference to Microsoft.SharePoint.dll so that the code behind your Web Parts can program against the WSS 3.0 object model as well.

Figure 8 Creating ASP.NET-Style Web Parts (Close Figure 8)
Imports System
Imports System.Web.UI
Imports System.Web.UI.WebControls.WebParts

Public Class SimpleWebPart : Inherits WebPart

    Protected _ZipCode As String

    <Personalizable(), _
     WebBrowsable(true), _
     WebDisplayName("Zip Code"), _
     WebDescription("used to track user zip code")> _
    Public Property ZipCode() As String
        Get
            Return _ZipCode
        End Get
        Set(ByVal value As String)
            _ZipCode = value
        End Set
    End Property

    Protected Overrides Sub RenderContents( _
            ByVal writer As HtmlTextWriter)
        writer.Write("You live at " & _ZipCode)
    End Sub
End Class

WSS 3.0 was also designed to support Web Parts created for the version 2 environment. These older-style WSS Web Parts inherit from the WebPart base class in the Microsoft.SharePoint.dll that is defined inside Microsoft.SharePoint.WebPartPages namespace.

The WebPart class in the older version of Microsoft.SharePoint.dll was designed to inherit from the ASP.NET Control class, as shown in Figure 9. However, you can also see that the WebPart class in the new version of Microsoft.SharePoint.dll has been modified to inherit from the ASP.NET Web Part class instead. This technique of changing a base class in a later version of an assembly is known as rebasing. The rebasing of the WebPart base class in Microsoft.SharePoint.dll is one of the keys to supporting older-style Web parts in a WSS 3.0 environment.

Figure 9 The Rebased WebPart Class
Figure 9 The Rebased WebPart Class

If you look at the standard web.config file for a WSS 3.0 Web application, you’ll see that it contains configuration elements to redirect references from the older version of Microsoft.SharePoint.dll to the new one. This redirection, in combination with the rebasing of the Web Part base class, allows Web Part DLLs written for the version 2 environment to run inside the version 3 environment without any recompilation.

If you decide to convert a version 2 Web Part project over to Visual Studio® 2005, you can continue to evolve your code using the same style you’ve used in the past and things will still work. However, you have another option as well once you’ve moved to Visual Studio 2005 and switched the old project reference to Microsoft.SharePoint.dll over to the new WSS 3.0 version. Since the ASP.NET WebPart base class is now a part of the inheritance hierarchy, you can change some aspects of your WSS-style Web Part class as if it were an ASP.NET-style Web Part. This is known as a hybrid Web Part.

Enhancements in Content Storage

One problem developers had with WSS 2.0 was that several valuable features supported with document libraries were not supported with lists. For example, document libraries supported versioning and events, but lists did not. With WSS 3.0, lists support many of the same features as document libraries including versioning, events, and folders. There are also some new features supported by both lists and document libraries, such as exposing data through automatic RSS feeds.

WSS 3.0 introduces a new column-indexing feature to alleviate some performance problems with document libraries and lists that existed in WSS 2.0. From a list settings page or a document library settings page you can add an index to any column. This does not actually create a physical index in SQL Server. Instead, it creates a table with the integer ID of the list item or document and the value of the indexed column. WSS then uses this table to improve the performance of return data from views, especially when you use a view with a filter based on the indexed column.

Many developers wanted to be able to work with WSS fields at a lower level to obtain more control over field rendering and validation. In response, WSS 3.0 adds new extensible field types. You can create an extensible field type by writing a class in C# or Visual Basic® that inherits from one of the built-in SharePoint field types such as SPFieldText and SPFieldNumber. An extensible field type can also utilize an ASP.NET user control that contains your favorite Web controls, letting you use the same techniques for control initialization and validation you’ve used in ASP.NET applications.

Another nice innovation in WSS 3.0 is custom site columns. A site column is a reusable definition that can be applied across multiple lists. It defines the name for a column, its underlying field type, and other characteristics such as the default value, formatting, and validation. Once you’ve defined a site column, you can then use it as you define the structure of your user-defined lists. An obvious advantage is that you can update the site column in a single place and have that update affect all the lists where the site column has been used. A site column is defined within the scope of a single site, yet it is visible to all child sites below. You can create a site column that is usable across an entire site collection by defining it inside the top-level site.

Site columns give you the ability to perform field lookups across sites, something you couldn’t accomplish in previous versions of SharePoint without writing custom code. For example, you can create a site column in a top-level site that performs a look up on a list in the same site. Then you can create other lists within child sites that use this site column to perform lookups on the list in the top-level site. Imagine you want to store several different types of documents in the same document library. For example, suppose you need to store customer presentations, proposals, and reports and you want each of these document types to have its own unique set of custom columns and its own unique event handlers. In WSS 2.0, you could only add extra columns and event handlers to the document library itself, which always affects every document in that document library. Furthermore, a document library could have only one associated document template.

WSS 3.0 introduces a powerful new storage mechanism called content types to solve this problem. A content type is a flexible and reusable WSS type that defines the shape and behavior for an item in a list or a document in a document library. For example, you can create a content type for a customer presentation document with a unique set of columns, an event handler, and its own document template. You can create a second content type for a customer proposal document with a different set of columns, a workflow, and a different document template. Then you can create a new document library and configure it to support both of these content types. This is a significant enhancement because it lets you deal with heterogeneous types of content in lists and document libraries.

Event Handlers

Events in WSS 2.0 supported document libraries but not lists. Moreover, version 2 supported only asynchronous events that fired after a user action had been committed to the SQL Server database. There was no way for the developer to cancel a user’s action inside an event handler.

WSS 3.0 improves event handling by retaining support for the asynchronous events that existed in version 2 and adding support for synchronous events that let you cancel user actions. For example, you can stop a user from deleting a document once it has been approved or from creating an order with an order date in the future. And events are supported on list items as well as on documents in a document library.

You create an event handler by writing a custom class that inherits from one of the WSS receiver classes and overriding methods to handle events. For example, to handle update events for items in a list, you’d create a class that inherits from the SPItemEventReceiver class and override the ItemUpdating method as shown in Figure 10.

Figure 10 Event Handler Created from Receiver Class (Close Figure 10)
Imports System
Imports Microsoft.SharePoint

Public Class MyReceiver : Inherits SPItemEventReceiver
    Public Overrides Sub ItemUpdating( _
            ByVal properties As SPItemEventProperties)

        Using web As SPWeb = properties.OpenWeb()
            Dim ListId As Guid = properties.ListId
            Dim ListItemId As Integer = properties.ListItemId
            Dim Orders As SPListItem = _
                web.Lists(ListId).GetItemById(ListItemId)
            Dim OrderDate As DateTime = _
                Convert.ToDateTime(Orders(&quot;OrderDate&quot;))
            If OrderDate.CompareTo(DateTime.Today) &gt; 0 Then
                properties.ErrorMessage = _
                    &quot;You cannot enter orders for future days.&quot;
                properties.Cancel = true
            End If
        End Using

    End Sub
End Class

You can bind the event handler to a particular list or document library by either writing code against the object model or defining an event receiver in a WSS feature. For example, you can bind this event receiver to a list using code similar to this:

Dim list As SPList = web.Lists("Orders")
list.EventReceivers.Add(SPEventReceiverType.ItemAdding, _
    "[Fully-qualified Assembly name]", "Litware.MyReceiver")

WSS receiver classes use a naming convention for overridable methods that differentiates between synchronous and asynchronous events. For example, ItemUpdating is a cancelable, synchronous event that fires before the change is made to the content database. These synchronous events provide a great way to validate column values, as the event handler in Figure 10 shows.

There is a complementary asynchronous event named ItemUpdated that occurs after a change has been written to the content database. Unlike synchronous events, asynchronous events do not support canceling user actions. Instead they provide the means to perform a required operation after a change has been made to a document or list item, such as assigning the value of a calculated column or sending an e-mail notification.

In addition to supporting events on list items and documents, WSS 3.0 supports other new event types such as list events that fire when someone changes a list definition. For example, you can cancel the user’s action whenever someone tries to remove or rename a column in one of your custom lists. There are also events that fire whenever someone deletes or moves a site or an entire site collection.

Workflows in WSS

Workflow applications have been getting quite a bit of attention inside Microsoft lately. The WinFX® runtime components, to be released in concert with Windows Vista™, add a complete infrastructure—Windows Workflow Foundation—for building workflow-style applications. The workflow infrastructure includes an engine, pluggable components to persist workflow state, and a Visual Studio designer that makes it easy to create custom workflows by dragging and dropping components known as activities onto a workflow design surface.

WSS 3.0 builds on top of Windows Workflow Foundation to provide a structure for attaching business logic to list items and documents. It extends the basic workflow model by associating a task list and a history list with each workflow, adding a degree of responsibility and accountability to workflows that are human-oriented, such as a path for reviewing or approving a document.

Both WSS and MOSS 2007 ship with workflows that are installed and ready to use. WSS 3.0 includes some simple routing workflows for actions such as moderation and approval. MOSS 2007 supplies more complex workflows that support features such as the Web content management approval process.

Creating custom workflows represents an obvious extensibility point for developers building business solutions with WSS and MOSS 2007. In addition to the standard support of the Visual Studio extensions for Windows Workflow Foundation, there are plans to ship a WSS-specific workflow SDK and a starter kit that includes Visual Studio project templates for creating custom workflows for WSS sites.

SharePoint Designer also provides support for creating custom workflows in WSS 3.0 sites. This is really targeted at the power user as much or more than developers because it provides a means to step through a wizard and attach business logic to list items and documents in a production site.

Site Definitions, Features, and Solutions

Developers who have used WSS 2.0 as a platform to build business solutions have found that working with low-level site definitions provides the greatest amount of control and reusability. A site definition is a directory on the front-end Web server containing XML files and .aspx page templates that define a blueprint for the site, including list schemas and page layouts. The XML-based language that is used inside many site definition files is called Collaborative Application Markup Language (CAML).

However, there are some issues with this strategy in version 2. First, the XML files inside version 2 site definitions were poorly factored, making them unwieldy and hard to work with. Second, there was no support for evolving a site definition once it had been used to create sites. That means there wasn’t a supported technique involving site definitions that could be used to add functionality to an existing WSS 2.0 site. Third, site definitions and the custom assemblies they depend on created deployment issues because the files had to be pushed out to each front-end Web server in a farm without any assistance from the WSS infrastructure. Finally, WSS 2.0 provided no means to localize a site definition, a somewhat frustrating situation for companies wanting to internationalize their business solutions.

The first significant WSS 3.0 enhancement in this area is the introduction of features. A feature is like a site definition: a directory containing CAML-based XML files and page templates. But features offer a much more modular approach because you don’t need to define the entire blueprint for a site. Instead, a simple feature can define a single site element such as a custom list definition or menu commands to be displayed in one of the standard WSS menus.

Happily, features can be activated on an existing site. For example, you can create a feature that defines a custom list type, an instance of that list type, and an event handler or workflow on that list instance. Once the feature has been installed, it can be activated in any site within a farm and you can activate it through the WSS user interface, from the command line, or through custom code written against the WSS object model. Once the feature has been activated, the site will include your new custom list and any behavior you want to attach to it.

Features also make developing site definitions simpler. In WSS 2.0, each list definition had to be specified inside the context of a site definition. In version 3 every list definition can be factored out into its own feature. Site definitions are now easier to create as they can be composed using feature references. This is the approach taken with all the list types that ship as part of WSS collaboration services.

WSS 3.0 introduces a new deployment mechanism called a solution that is similar to Web Part packages in the sense that it is an aggregate .cab file containing XML instructions and files that need to be deployed on each front-end Web server. However, solutions go beyond Web Part packages to support the deployment of features, site definitions, and related assemblies used for event handlers and workflows.

WSS 3.0 support for solutions also assists in pushing deployment files to each Web server in a farm. An administrator adds a solution to a WSS farm, which copies the solution .cab file into the configuration database. Next, the administrator runs a command to deploy the solution, at which time WSS starts a timer job to push the solution .cab file out to each Web server and install it.

A very welcome change for WSS developers is the new support for localization. This functionality has been built on top of the localization infrastructure of ASP.NET 2.0, and it’s now possible (and recommended) to create site definitions and features in a language-neutral fashion and to maintain string literals in .resx files. Within the XML files and .aspx page template files of a feature or site definition you can acquire the value for a named string that has been localized into a .resx file using standard ASP.NET syntax.

<%$Resources:Litware,MyString%>

Internet-Style Security

Authentication in WSS 2.0 was based on Windows accounts and their associated Security IDs (SIDs). In a practical sense this meant that WSS 2.0 was tightly coupled with Active Directory® when used in anything but the smallest deployments. This dependency wasn’t a problem for companies that deployed Active Directory when they started using WSS 2.0 and SharePoint Portal Server 2003 for building intranet-based solutions. But for companies building extranet-based solutions, it was far more challenging. The tight coupling of WSS to Active Directory forced companies to create and maintain domain accounts for non-company users such as venders and customers.

All this changes with WSS 3.0 because authentication has been redesigned on top of the new authentication provider infrastructure introduced with ASP.NET 2.0. If you don’t want to maintain user accounts for WSS and MOSS 2007 sites inside Active Directory, you simply build or acquire an ASP.NET authentication provider that’s been designed to store and manage user accounts in a different identity repository.

For example, ASP.NET 2.0 ships with the forms authentication provider that allows you to maintain user accounts inside a SQL Server database. This authentication provider can be configured for use in a WSS site. With little effort, you can put a WSS site on the Internet that allows unknown users to register themselves as members. ASP.NET 2.0 provides convenient support for creating and maintaining user accounts and even allows users to change and reset their passwords.

Conclusion

I’ve covered many of the major advancements to WSS 3.0 targeted at developers, and you’ll discover even more once you begin to dig in and work with the product. It should be clear already that the WSS team has done a nice job listening to developer feedback about the previous version and responding with some significant new features.

Ted Pattison is an author and trainer who delivers hands-on training through his company, the Ted Pattison Group. Ted is currently researching and writing a new book focusing on Windows SharePoint Services 3.0 and Office 12 server technology.


Patterns &amp; Practices: Speed Development With Custom Application Blocks For Enterprise Library

Patterns & Practices

Speed Development With Custom Application Blocks For Enterprise Library

Mark Seemann
This article discusses:
  • Writing new application blocks for Enterprise Library
  • Building a plug-in loader application block
  • Understanding and implementing factories
  • Implementing configuration for custom application blocks
This article uses the following technologies:
Enterprise Library, Visual Studio, C#
Code download available at: AppBlocks2006_07.exe (315KB)
Contents
Creating an Application Block
Factories for Objects
Creating Factories
Defining Plug-In Configuration
Implementing PlugInProvider
Design-Time Behavior
Provider Nodes
Serializing and Deserializing XML
Conclusion

E
nterprise Library for Microsoft .NET Framework 2.0 is a library of application blocks, which are modular components designed to help developers deal with common development challenges. It provides an extensible framework for building robust, scalable applications. And while Enterprise Library contains several useful application blocks, you can also integrate your own reusable components. In this article, I will demonstrate how to build a sample application block that integrates with Enterprise Library.

The Microsoft patterns & practices team, which is the creator of Enterprise Library, coined the term "application block" to describe a library that is more than just a DLL file. Although an application block is indeed a DLL, it has further unique characteristics.

Application blocks are configuration-driven, meaning that configuration is used to define their behavior. This configuration can be defined in a normal .NET configuration file or it can be created and injected into the application block at run time. Some of the configuration will be typical configuration values, such as numbers defining conditions for certain behavior. But in regards to application blocks, configuration is also used to define extensions to the basic block.

Application blocks are also modular and extensible. The extensibility is usually achieved through a provider model; the block delegates some of its work to a provider, which implements a base class or interface that the block uses. Although a block typically comes with a few pre-packaged providers (for example, one that lets you log to the Windows® event log or one that lets you read and write data to SQL Server™), other providers can be developed and replaced by defining the provider type in the block’s configuration.

When would you want to create an application block instead of a normal library? If you have no need for configuration of the library and you don’t want to allow others to extend it, you do not need to create an application block. But if these capabilities sound useful, then you should consider creating an application block. And since Enterprise Library already offers a lot of configuration management infrastructure that you can make use of, you may benefit from creating an application block instead of a standalone, configurable library.

An application block adds complexity to the library, as well as a dependency on other blocks in Enterprise Library. In return, however, you gain a common configuration management system and you may be able to use other parts of Enterprise Library. For instance, data access is a very common need. If you integrate your application block with Enterprise Library, you can write an optional provider for your block that uses the Data Access Application Block. When done correctly, this creates an optional dependency on the Data Access Application Block; in cases where you do not need data access, you can avoid this dependency.

Creating an Application Block

In this article, I will show you how to build an application block that can be used for managing plug-ins for an application. Note that although I will cover many aspects of application block development, I will cut some corners in the part of the implementation that does not apply to Enterprise Library. As such, the solution I will create in this article is simplistic and not at all secure. For a thorough treatment on how to load plug-ins in a secure and robust fashion, see Shawn Farkas’s MSDN®Magazine article "Do You Trust It? Discover Techniques for Safely Hosting Untrusted Add-Ins with the .NET Framework 2.0". Because the Plug-In Loader application block I’m creating here is extensible, it’s possible to develop a new provider that utilizes the methods described in Shawn Farkas’s article and add it to this application block.

Now that the disclaimer is out of the way, let’s get started. First I will define the end goal of the application block. Since its purpose is to load plug-ins, I think a reasonable goal is to enable developers to write code like this:

PlugInManager<MyPlugIn> mgr =
    PlugInFactory.GetManager<MyPlugIn>();
IList<MyPlugIn> plugIns = mgr.GetPlugIns();
// Do something with the plug-ins...

The static PlugInFactory class creates a new PlugInManager<T>, which can be used to acquire the plug-ins. In this article, GetPlugIns is the only member of PlugInManager<T>. In a more complete implementation, other functionality—the ability to enable or disable each plug-in at run-time, for example—would likely be added. PlugInManager<T> delegates the real work to a provider specified in the application block’s configuration. This provider then loads and returns the plug-ins as defined in its implementation.

An application block consists of the run-time component and an optional design-time component. I’ll show you how to develop both of these. The run-time component is where the library’s logic is implemented. This is the file that you need to reference in your project when you want to utilize the application block. Developing the run-time component involves several steps in itself.

The design-time component is another library assembly that you can create and include. Just as the run-time component integrates with the Enterprise Library run-time framework, the design-time component integrates with Enterprise Library’s design-time tools. This design-time functionality exists primarily to make the developer’s job easier when using the application block. But it can also be used by system administrators or other users to provide a graphical tool for editing the configuration of an application.

An application block is a complex collection of code that involves a lot of interacting classes. As you read through this step-by-step guide, you may wonder why it is necessary to have so many moving parts. In part, this is because Enterprise Library provides a lot of flexibility regarding how you implement your application block. In many steps, Enterprise Library uses abstractions, such as interfaces and abstract classes, to accomplish certain tasks. For most such scenarios, an extendable default implementation is included, but you are rarely forced to use that default implementation. If you prefer, you can create your own implementation.

Each time Enterprise Library accomplishes a task—like reading configuration data from a persistent store or instantiating a concrete type when an abstract type is requested—you can extend or deviate from the default behavior. For each extensibility point in Enterprise Library there are typically a few interacting classes involved. Since Enterprise Library has a lot of extensibility points, there are many interacting classes—a fact you’ll soon discover for yourself.

Factories for Objects

Enterprise Library creates objects using factories. It has its own attribute-based approach in which you attribute classes with the type of their factory and then implement the factory by deriving from base classes that are provided by Enterprise Library. Figure 1 provides an overview of the typical object-creation process. As you can see, this is a complex process, but you must keep in mind that it accomplishes more than just creating an arbitrary instance of a type. You request an abstract type and, based on configuration data, you get the correct implementation of that type, appropriately configured and instrumented.

Figure 1 Object-Creation Process (Close Figure 1)

Step | Procedure | Notes
1 | Inspect the type to be created for the presence of a CustomFactory attribute. | This attribute identifies a type that must be used as a custom factory for the class. If this attribute is not present, an exception is thrown.
2 | Instantiate the custom factory identified in Step 1. | This factory must implement ICustomFactory.
3 | Call the custom factory’s CreateObject method to create an instance of the requested type. | The default implementation of ICustomFactory is the abstract AssemblerBasedCus-tomFactory<TObject, TConfiguration> class. If this class is used as a base for implementing ICustomFactory, Enterprise Library continues with the following steps.
4 | Call the AssemblerBasedCustomFactory<TObject, TConfiguration>.GetConfiguration method. | This configuration data is used to create the requested instance and identifies the implementation type to be created.
5 | Inspect the implementation type for the presence of the ConfigurationElementType attribute. | This attribute identifies the type that contains configuration data for the implementation type. If this attribute is not present, an exception is thrown.
6 | Instantiate the configuration type. | The configuration type is populated with configuration data from the configuration source.
7 | Inspect the configuration data class for the requested type, looking for the presence of the Assembler attribute. | This attribute identifies a type that must be used to create the requested type from its configuration data—a so-called assembler. If this attribute is not present, an exception is thrown.
8 | Instantiate the assembler. | The assembler identified in Step 7 must implement IAssembler<TObject, TConfiguration> or an exception is thrown.
9 | The assembler creates the requested object. | The configuration data defines the parameters for the object.

This object creation framework is highly extensible and you should use it as an object creation mechanism for your own application block, but you will soon find yourself creating a lot of factory classes. Figure 2 provides an overview of the interaction between these classes. When using the Plug-In Loader application block, the static PlugInFactory class is typically the entry point. This class is just a convenience class, which uses PlugInManagerFactory<T> to create instances of PlugInManager<T>. Most Enterprise Library application blocks provide such a static factory class as an entry point to the application block. Implementing a static factory class is optional when creating an Enterprise Library application block, but I’ve decided to include such a class for Plug-In Loader since it makes sense here.

Figure 2 Interaction of Object Factory Classes
Figure 2 Interaction of Object Factory Classes

Enterprise Library takes a factory-based approach to creating configurable objects, and although that framework is very flexible, it does not support generics since types are unknown at compile time. For that reason, the generic Plug-In Loader classes delegate most of their work to non-generic classes, which can be created and managed by Enterprise Library. PlugInManager<T> wraps the abstract PlugInProvider class, and PlugInManagerFactory<T> wraps PlugInProviderFactory. If your application block does not need to support generics, you don’t have to implement classes like PlugInManager<T> and PlugInManagerFactory<T>.

PlugInProviderFactory, which creates PlugInProvider instances, derives from a class provided by Enterprise Library. Using the CustomFactory attribute described in Step 1 of Figure 1, PlugInProvider identifies PlugInProviderCustomFactory, which Enterprise Library uses together with configuration classes to create instances of PlugInProvider.

This may seem complex, but Enterprise Library does most of the heavy lifting. You need to implement quite a few classes, but all of them are quite simple. You may be asking yourself why all this complexity is required. As I mentioned earlier, you are doing more than just newing up an instance of PlugInProvider. For starters, you cannot instantiate PlugInProvider, since it is an abstract class. When you request an instance of PlugInProvider, the application block must really serve up an implementation of that abstract class, based on configuration data. Enterprise Library uses this mechanism to identify what to create and how to create it.

Creating Factories

If you look at the intended target code I showed earlier, you can see that the first classes I need to implement are PlugInManager<T> and PlugInFactory. Both of these classes delegate most of their work to other classes. PlugInManager<T> uses PlugInProvider to perform the real work, as shown in Figure 3. Since the Enterprise Library object creation mechanism does not support generics, PlugInProvider is not a generic class, so to return IList<T>, the IList returned by PlugInProvider must be converted to the appropriate type-safe collection.

Figure 3 Getting a PlugInProvider (Close Figure 3)
public class PlugInManager<T>
{
    private PlugInProvider provider_;

    internal PlugInManager(PlugInProvider provider)
    {
        this.provider_ = provider;
    }

    public IList<T> GetPlugIns()
    {
        IList plugIns = this.provider_.GetPlugIns();
        List<T> returnList = new List<T>(plugIns.Count);
        foreach (T item in plugIns) returnList.Add(item);
        return returnList;
    }
}

PlugInProvider itself is the simple class shown in Figure 4. Since it is abstract, it provides the extensibility point for Plug-In Loader. If you want to extend Plug-In Loader, you can derive a new class from PlugInProvider and implement your custom logic there. The most notable feature of PlugInProvider is the CustomFactory attribute, which instructs Enterprise Library how to create a new instance of a class that derives from PlugInProvider. Also note the abstract GetPlugIns method, which is the method that inheritors have to implement.

Figure 4 Implementation of PlugInProvider (Close Figure 4)
[CustomFactory(typeof(PlugInProviderCustomFactory))]
public abstract class PlugInProvider
{
    private Type plugInType_;

    protected PlugInProvider(Type plugInType)
    {
        this.plugInType_ = plugInType;
    }

    public abstract IList GetPlugIns();

    protected Type PlugInType
    {
        get { return this.plugInType_; }
    }
}

PlugInProviderCustomFactory derives from AssemblerBased-CustomFactory<TObject, TConfiguration>, which is an abstract class provided by Enterprise Library. The only thing you have to do when deriving from this class is implement the GetConfiguration method. A couple of new classes, PlugInLoaderSettings and PlugInProviderData, make their first appearance here, as you can see in the following code:

public class PlugInProviderCustomFactory :
    AssemblerBasedCustomFactory<PlugInProvider, PlugInProviderData>
{
    protected override PlugInProviderData GetConfiguration(
        string name, IConfigurationSource configurationSource)
    {
        PlugInLoaderSettings settings =
            (PlugInLoaderSettings)configurationSource.
                GetSection(PlugInLoaderSettings.SectionName);
        return settings.PlugInProviders.Get(name);
    }
}
These classes are configuration classes and I will discuss them in more detail in the next section.

For now, the most important thing to notice is that the GetConfiguration method returns the appropriate configuration data, enabling Enterprise Library to construct a new PlugInProvider object, as described in Figure 1. With this custom factory in place, I can create a factory class that I can then use to create PlugInProvider instances, as shown here:

public class PlugInProviderFactory :
    NameTypeFactoryBase<PlugInProvider>
{
    public PlugInProviderFactory(IConfigurationSource configSource) :
        base(configSource) { }
}
Although this is yet another factory class, all I need to do is derive from the NameTypeFactoryBase<T> class and provide a public constructor. PlugInManagerFactory<T> simply wraps PlugInProviderFactory. PlugInFactory creates and maintains a dictionary of these factories, and delegates the work to the appropriate factory, as shown in the code in Figure 5.

Figure 5 Creating Factories and Sending Them Work (Close Figure 5)
public class PlugInManagerFactory<T>
{
    private PlugInProviderFactory factory_;

    public PlugInManagerFactory(IConfigurationSource
            configurationSource)
    {
        this.factory_ = new PlugInProviderFactory(configurationSource);
    }

    public PlugInManager<T> Create()
    {
        string name = typeof(T).AssemblyQualifiedName;
        return new PlugInManager<T>(this.factory_.Create(name));
    }
}

public static class PlugInFactory
{
    private readonly static Dictionary<Type, object> factories_ =
        new Dictionary<Type, object>();

    public static PlugInManager<T> GetManager<T>()
    {
        if (!PlugInFactory.factories_.ContainsKey(typeof(T)))
        {
            IConfigurationSource configSource =
                ConfigurationSourceFactory.Create();
            PlugInManagerFactory<T> factory =
                new PlugInManagerFactory<T>(configSource);
            PlugInFactory.factories_.Add(typeof(T), factory);
        }

        object piFactory = PlugInFactory.factories_[typeof(T)];
        return ((PlugInManagerFactory<T>)piFactory).Create();
    }
}

One item of note here is a naming convention that is specific to Plug-In Loader. Enterprise Library polymorphic collections are keyed by their items’ names, so each name must be unique within the collection. For Plug-In Loader, the type of plug-in would have been the most intuitive key. However, that would have required me to create my own collection class keyed on type, and thus I would not have been able to reuse the classes provided by Enterprise Library.

As the design-time component requires a unique name in any case, the convention for Plug-In Loader is for each PlugInProvider to be named by the assembly-qualified name of the plug-in type, which is what you see in effect in Figure 5. It is a bit of a hack, but the user will never notice, because I will also deal with this convention in the design-time component. On the other hand, if you feel like editing the raw XML, everything is a string in that world in any case.

That’s it for the first step of creating an application block. If you refer back to Figure 1, you may wonder why I have not defined any assembler classes. This is because assemblers are tied to implementations of PlugInProvider, not the abstract PlugInProvider class itself. Now I will describe how to define configuration classes, and then I will describe how to implement a PlugInProvider, which will include a discussion of creating the appropriate assembler.

Defining Plug-In Configuration

The configuration framework of Enterprise Library builds on top of System.Configuration and works very similarly. To define the configuration section for Plug-In Loader, I create the PlugInLoaderSettings class as shown in Figure 6. But instead of deriving directly from System.Configuration.ConfigurationSection, an application block’s configuration section should derive from Microsoft.Practices.EnterpriseLibrary.Common.Configuration.SerializableConfigurationSection. This adds Enterprise Library functionality to the class; among other things, this lets you store the configuration section somewhere other than the application configuration file.

Figure 6 PlugInLoaderSettings (Close Figure 6)
public class PlugInLoaderSettings : SerializableConfigurationSection
{
    public const string SectionName = "plugInConfiguration";

    private const string providersProperty_ = "plugInProviders";

    public PlugInLoaderSettings() : base() {}

    [ConfigurationProperty(PlugInLoaderSettings.providersProperty_)]
    public NameTypeConfigurationElementCollection<PlugInProviderData>
        PlugInProviders
    {
        get
        {
            return (NameTypeConfigurationElementCollection
                    <PlugInProviderData>)
                    this[PlugInLoaderSettings.providersProperty_];
        }
    }
}

The PlugInLoaderSettings class contains only a collection of PlugInProviderData classes. The PlugInProviderData class contains the data used to configure an instance of PlugInProvider, as shown in the following:

public class PlugInProviderData : NameTypeConfigurationElement
{
    public PlugInProviderData() : base() { }
    public PlugInProviderData(string name, Type type) :
        base(name, type) { }
}

This class represents a configuration element and derives indirectly from System.Configuration.ConfigurationElement. Had I wanted to create a simple configuration element, I could have derived PlugInProviderData directly from ConfigurationElement, but Enterprise Library gives me a couple of other options. One is the NamedConfigurationElement class, the other is NameTypeConfigurationElement. The former adds a name to the configuration element, which is useful when implementing the design-time features of the application block. This name also acts as the unique key in the generic configuration collection classes provided by Enterprise Library.

NameTypeConfigurationElement adds an additional Type property to the configuration element, and this is used to support polymorphic collections, which is exactly what I want in this case—to specify different plug-in providers, each with unique configuration settings, for different plug-in types. Where the name of the configuration element acts as a key to that element, the Type property identifies the type that the element configures. In the case of Plug-In Loader, the Type property identifies a type that implements PlugInProvider. Recall that, by convention, Plug-In Loader uses the Name property to store the assembly-qualified name of the type of plug-in. It is easy to mix up these two types, but the name identifies which plug-in type should be served by the provider, whereas the Type property identifies the type of the provider. Since the name is the key, you can only have a type of plug-in defined once, but many different plug-in types can be served up by the same provider type; in fact, most often they will probably all be served up by the same provider.

Due to the way Enterprise Library constructs these configuration element classes, it is not possible to make the PlugInProviderData class abstract, but you should think about it as being abstract. Notice that it does not really do anything, so in this particular implementation, I could have omitted it and created my configuration elements for the different plug-in providers by deriving them directly from NameTypeConfigurationElement. However, the abstract PlugInProvider class does contain some implementation, and it is easier to understand the structure of the application block’s code if there is a one-to-one relationship between providers and their configuration elements.

Implementing PlugInProvider

At this point, the abstract framework of the run-time component is complete, but there is still no functionality. It is now to time to create an implementation of PlugInProvider. This is a naïve implementation that is not secure and does not support plug-ins that can be unloaded from memory. For that reason, I will call it NaivePlugInProvider.

Shown in Figure 7, the NaivePlugInProvider class derives from PlugInProvider. Its main functionality is implemented in the GetPlugIns method, which simply loads and reflects over all types in all assemblies located in the configured folder. If a type implements the desired plug-in type, a new instance of that type is created and added to the list of plug-ins to return. Note that this implementation requires all plug-ins to have a default constructor; a more robust implementation might use a more sophisticated approach.

Figure 7 Implementing the Actual Plug-In Provider (Close Figure 7)
[ConfigurationElementType(typeof(NaivePlugInProviderData))]
public class NaivePlugInProvider : PlugInProvider
{
    private string plugInFolder_;

    public NaivePlugInProvider(Type plugInType, string plugInFolder)
        : base(plugInType)
    {
        this.plugInFolder_ = plugInFolder;
    }

    public override IList GetPlugIns()
    {
        ArrayList plugIns = new ArrayList();

        string[] assemblyFileCandidates =
            Directory.GetFiles(this.plugInFolder_, "*.dll");
        foreach (string assemblyFileCandidate in assemblyFileCandidates)
        {
            Assembly a = Assembly.LoadFrom(assemblyFileCandidate);
            foreach (Type t in a.GetTypes())
            {
                if((t.IsPublic) && (this.PlugInType.IsAssignableFrom(t)))
                {
                    plugIns.Add(Activator.CreateInstance(t));
                }
            }
        }

        return plugIns;
    }
}

NaivePlugInProvider has two other traits that are not quite so apparent, but that are interesting in the context of creating an Enterprise Library application block: the use of the ConfigurationElementType attribute, and the lack of a default constructor.

When configuring the Plug-In Loader application block, you should only need to be concerned about which PlugInProvider you want to use, not about which class provides configuration data for that provider. The ConfigurationElementType attribute contains that information, which means that the configuration data only contains information about which PlugInProvider to create, and the Enterprise Library infrastructure figures out which class contains configuration data for that provider. In this case, it is the NaivePlugInProviderData class shown in Figure 8. This class derives from PlugInProviderData and provides an extra configuration property that allows you to specify the folder that will contain the plug-in assemblies.

Figure 8 NaivePlugInProviderData and NaivePlugInProviderAssembler (Close Figure 8)
[Assembler(typeof(NaivePlugInProviderAssembler))]
public class NaivePlugInProviderData : PlugInProviderData
{
    private const string folderProperty_ = "plugInFolder";

    public NaivePlugInProviderData() : base()
    {
        this.Type = typeof(NaivePlugInProvider);
    }

    public NaivePlugInProviderData(string name) :
        this(name, "PlugIns")
    {
    }

    public NaivePlugInProviderData(string name, string plugInFolder) :
        base(name, typeof(NaivePlugInProvider))
    {
        this.PlugInFolder = plugInFolder;
    }

    [ConfigurationProperty(NaivePlugInProviderData.folderProperty_,
        IsRequired = true)]
    public string PlugInFolder
    {
        get
        {
            return (string)this[NaivePlugInProviderData.folderProperty_];
        }
        set { this[NaivePlugInProviderData.folderProperty_] = value; }
    }
}

public class NaivePlugInProviderAssembler :
    IAssembler<PlugInProvider, PlugInProviderData>
{
    public PlugInProvider Assemble(IBuilderContext context,
        PlugInProviderData objectConfiguration,
        IConfigurationSource configurationSource,
        ConfigurationReflectionCache reflectionCache)
    {
        NaivePlugInProviderData naiveConfiguration =
            objectConfiguration as NaivePlugInProviderData;
        Type plugInType = Type.GetType(naiveConfiguration.Name);
        string folder = naiveConfiguration.PlugInFolder;
        return new NaivePlugInProvider(plugInType, folder);
    }
}

The other interesting thing about NaivePlugInProvider is its lack of a default constructor. How does Enterprise Library create a new instance of NaivePlugInProvider when it has no default constructor? NaivePlugInProviderData is outfitted with an Assembler attribute. This attribute identifies a type that can create a NaivePlugInProvider instance from a NaivePlugInProviderData instance.

The NaivePlugInProviderAssembler class is also shown in Figure 8. An assembler class must implement IAssembler<TObject, TConfiguration>, which contains the single Assemble method. It uses the supplied configuration data to pull out the relevant information and create a new NaivePlugInProvider instance.

At this point, Plug-In Loader contains a fully functional, albeit simplistic, implementation and is ready for use. You can now go and create more providers to create different plug-in discovery behaviors for the application block. An obvious extension is a provider that follows secure practices for discovering and loading plug-ins. Another possible extension is one that pulls plug-ins from blobs in a SQL Server table, possibly creating an optional dependency on the data access application block.

If you do not mind having to write the entire configuration in XML by hand, you can stop right there and ship your application block. Otherwise, you can create a design-time component that takes care of that work for you.

Design-Time Behavior

The design-time component plugs into the Enterprise Library configuration application. This is an extensible Windows Forms application that allows you to edit application configuration files using a rich UI, instead of having to write raw XML. The component has three responsibilities: it must provide behavior for the application UI itself, it must allow the application to serialize the user’s settings, and it must be able to deserialize configuration data when the user opens an existing application configuration file.

Since application configuration files are based on XML, they are hierarchical in nature. The configuration application models this as a tree consisting of nodes. Each configuration element in the run-time component must be represented by a design-time node class, which provides additional design-time behavior to the configuration class. Figure 9 illustrates this relationship for the Plug-In Loader application block.

Figure 9 Mapping Run-Time and Design-Time Classes
Figure 9 Mapping Run-Time and Design-Time Classes

To integrate with the Enterprise Library configuration application, the design-time component must register with it. The assembly and its dependencies must be placed in the same directory as the configuration application itself, and it must be marked with the ConfigurationDesignManager attribute, like this:

[assembly: ConfigurationDesignManager(
    typeof(PlugInLoaderConfigurationDesignManager))]

This assembly-level attribute registers the PlugInLoaderConfigurationDesignManager with the configuration application. This class derives from the abstract ConfigurationDesignManager class.

Defining the design-time behavior involves specifying which actions are possible in which contexts. This is done by overriding ConfigurationDesignManager’s Register method:

public override void Register(IServiceProvider serviceProvider)
{
    PlugInLoaderCommandRegistrar cmdRegistrar =
        new PlugInLoaderCommandRegistrar(serviceProvider);
    cmdRegistrar.Register();

    // Node map code goes here...
}

PlugInLoaderCommandRegistrar derives from the abstract CommandRegistrar class; its purpose is to register design-time actions with the configuration application. The first action I need to implement is the command to add the application block to an application configuration file. When the Plug-In Loader application block is added to an application configuration, a PlugInLoaderSettingsNode, along with its child Plu gInProviderCollectionNode, must be added to the hierarchy.

First, these node classes must be defined, as such:

public class PlugInLoaderSettingsNode : ConfigurationNode
{
    public PlugInLoaderSettingsNode() :
        base("Plug-In Loader Application Block") {}

    [ReadOnly(true)]
    public override string Name
    {
        get { return base.Name; } set { base.Name = value; }
    }
}

PlugInProviderCollectionNode is almost identical, since PlugInLoaderSettingsNode does not contain other properties than the collection of PlugInProviders. While you may think I could have used a common class for both nodes, this is not the case. Both nodes occupy different places in the hierarchy, and I am going to attach different actions to each of them. If you wonder why I have overridden the Name property, it is merely so I can mark it with the Read-only attribute. This will make the nodes read-only in the configuration application.

When the command to add the Plug-In Loader application block to an application configuration file is invoked by the user, these two nodes must be added to the hierarchy. To accomplish this, I create the AddPlugInLoaderSettingsNodeCommand class, shown in Figure 10. It derives from AddChildNodeCommand and overrides the ExecuteCore method to implement the desired logic. The command class must be associated with a node class so that the base class knows that it should create an instance of PlugInLoaderSettingsNode and add it to the hierarchy. This is already done after the call to the base implementation of ExecuteCore, so all I need to do is create a new PlugInProviderCollectionNode and add it to the settings node.

Figure 10 Adding Nodes to the Hierarchy (Close Figure 10)
public class AddPlugInLoaderSettingsNodeCommand : AddChildNodeCommand
{
    public AddPlugInLoaderSettingsNodeCommand(
        IServiceProvider serviceProvider) : base(serviceProvider,
            typeof(PlugInLoaderSettingsNode)) {}

    protected override void ExecuteCore(ConfigurationNode node)
    {
        base.ExecuteCore(node);

        PlugInLoaderSettingsNode settingsNode =
            this.ChildNode as PlugInLoaderSettingsNode;
        if (settingsNode == null) return;

        PlugInProviderCollectionNode providersNode =
            new PlugInProviderCollectionNode();
        settingsNode.AddNode(providersNode);
    }
}

The AddPlugInLoaderSettingsNodeCommand class defines what happens when the user invokes the command. But I still need to define when and where this command is available. It should only be available when the user has selected the application configuration root node, and it should only be possible to invoke the command once. I will accomplish this from the PlugInLoaderCommandRegistrar class by overriding the abstract Register method:

public override void Register()
{
    this.AddPlugInLoaderCommand();
    // Add other commands here...
}
The AddPlugInLoaderCommand method only contains three statements, as shown in the following:
private void AddPlugInLoaderCommand()
{
    ConfigurationUICommand cmd =
        ConfigurationUICommand.CreateSingleUICommand(
        this.ServiceProvider, "Plug-In Loader Application Block",
        "Add the Plug-In Loader Application Block",
        new AddPlugInLoaderSettingsNodeCommand(this.ServiceProvider),
        typeof(PlugInLoaderSettingsNode));
    this.AddUICommand(cmd, typeof(ConfigurationApplicationNode));
    this.AddDefaultCommands(typeof(PlugInLoaderSettingsNode));
}

By calling CreateSingleUICommand, I specify that this command can only be invoked once. In this method call, I also provide display texts and an instance of AddPlugInLoaderSettingsNodeCommand, which is invoked if the user selects to perform this action. With the call to AddUICommand I associate this command with the ConfigurationApplicatonNode type, which is the type of the application configuration root node. The AddDefaultCommands method adds default commands, such as Add and Delete, to the newly created PlugInLoaderSettingsNode.

Provider Nodes

PlugInProviderData must be enhanced by PlugInProviderNode, and NaivePlugInProviderData by NaivePlugInProviderNode, as illustrated in Figure 9. The abstract PlugInProviderNode in Figure 11 provides design-time functionality to PlugInProviderData. Several attributes from the System.ComponentModel namespace are in play here: Category, Editor, and ReadOnly. These provide the same functionality as in the Visual Studio® property grid.

Figure 11 Design-Time Support for PlugInProviderData (Close Figure 11)
public abstract class PlugInProviderNode : ConfigurationNode
{
    private PlugInProviderData configurationData_;
    private Type plugInType_;

    protected PlugInProviderNode(PlugInProviderData configurationData) :
        base(configurationData == null ? null : configurationData.Name)
    {
        if (configurationData == null)
        {
            throw new ArgumentNullException("configurationData");
        }
        this.configurationData_ = configurationData;
    }

    [ReadOnly(true)]
    public override string Name
    {
        get { return base.Name; } set { base.Name = value; }
    }

    [BaseType(typeof(object), TypeSelectorIncludes.AbstractTypes |
        TypeSelectorIncludes.BaseType |
        TypeSelectorIncludes.Interfaces)]
    [Category("Plug-In Loader")]
    [Editor(typeof(TypeSelectorEditor), typeof(UITypeEditor))]
    public Type PlugInType
    {
        get { return this.plugInType_; }
        set
        {
            this.plugInType_ = value;
            if (value != null) this.Name = value.AssemblyQualifiedName;
        }
    }

    public abstract string Provider { get; }

    protected override void OnRenamed(
        ConfigurationNodeChangedEventArgs e)
    {
        base.OnRenamed(e);
        this.PlugInProviderData.Name = e.Node.Name;
    }

    internal PlugInProviderData PlugInProviderData
    {
        get { return this.configurationData_; }
    }
}
Figure 9 Mapping Run-Time and Design-Time Classes
Figure 9 Mapping Run-Time and Design-Time Classes

PlugInProviderNode wraps a PlugInProviderData instance, which provides and stores all the configuration data except the plug-in type. Plug-In Loader uses a special naming convention in which the configured name of the PlugInProvider is the assembly-qualified name of the plug-in type. Since there’s no guarantee that the Name property can be resolved to a type, PlugInProviderNode stores the plug-in type separately in a member variable.

The PlugInType property also contains the BaseType attribute, which is used by the TypeSelectorEditor identified in the Editor attribute to filter the available types. When this editor shows available types, only those that are abstract, base types, or interfaces are listed. When selecting a plug-in type, these are the only types worth listing since you cannot base a plug-in on a sealed type.

Another notable feature of PlugInProviderNode is the read-only Provider property. I find it is always nice to be able to inspect which provider is configured, and this provides the most direct way of informing the user. Otherwise, this can be really difficult to tell when using the configuration application.

The last thing to note about PlugInProviderNode is that I use OnRenamed to keep the name of the node and the name of the underlying data synchronized.

A NaivePlugInProviderNode class extends PlugInProviderNode by providing the PlugInFolder property. Adding a NaivePlugInProvider command to the configuration application is a bit simpler than adding the application block itself, since this command does not need to create additional sub-nodes under a NaivePlugInProviderNode. So I don’t need to create a separate command class for this action. Instead I can just let PlugInLoaderCommandRegistrar handle all of the command registration:

this.AddMultipleChildNodeCommand(
    "Naive Plug-In Provider", "Add a new naive Plug-In Provider",
    typeof(NaivePlugInProviderNode),
    typeof(PlugInProviderCollectionNode));
this.AddDefaultCommands(typeof(NaivePlugInProviderNode));

By calling AddMultipleChildNodeCommand, I specify that this command can be invoked an arbitrary number of times to create new NaivePlugInProviderNodes as child nodes of a PlugInProviderCollectionNode. Other PlugInProvider node types can be added using a similar mechanism.

Serializing and Deserializing XML

When you work with the configuration application, settings are retained only in memory until you choose to save the changes. Once you save changes, the data configured in each node must be serialized to XML. Fortunately, Enterprise Library and System.Configuration take care of serializing configuration elements to and from XML. The only thing you need to do is specify how to map from nodes to configuration classes.

This is done by overriding PlugInLoaderConfigurationDesignManager’s GetConfigurationSectionInfo method. This implementation acquires the PlugInLoaderSettingsNode from the hierarchy, but delegates the real work to the PlugInLoaderSettingsBuilder class shown in Figure 12. This internal class creates a new, empty PlugInLoaderSettings instance and then uses the node hierarchy to add configuration data to that instance.

Figure 12 Preparing Settings for Serialization (Close Figure 12)
internal class PlugInLoaderSettingsBuilder
{
    private PlugInLoaderSettingsNode settingsNode_;
    private IConfigurationUIHierarchy hierarchy_;

    internal PlugInLoaderSettingsBuilder(
        IServiceProvider serviceProvider,
        PlugInLoaderSettingsNode settingsNode)
    {
        this.settingsNode_ = settingsNode;
        this.hierarchy_ =
            ServiceHelper.GetCurrentHierarchy(serviceProvider);
    }

    internal PlugInLoaderSettings Build()
    {
        PlugInLoaderSettings settings = new PlugInLoaderSettings();
        this.BuildPlugInProviders(settings);
        return settings;
    }

    private void BuildPlugInProviders(PlugInLoaderSettings settings)
    {
        PlugInProviderCollectionNode providersNode =
            this.hierarchy_.FindNodeByType(this.settingsNode_,
                typeof(PlugInProviderCollectionNode))
                    as PlugInProviderCollectionNode;

        if (providersNode != null)
        {
            foreach (PlugInProviderNode pipNode in providersNode.Nodes)
            {
                settings.PlugInProviders.Add(pipNode.PlugInProviderData);
            }
        }
    }
}

Since Plug-In Loader’s configuration hierarchy is so shallow, this only involves looping through all PlugInProviders and adding the configuration data from their nodes. If the hierarchy had been deeper, this operation would have entailed traversing the node hierarchy and building up the equivalent hierarchy of configuration classes.

Just as Enterprise Library and System.Configuration take care of serializing to XML, the framework also performs the work of deserializing from XML to configuration class instances. However, you must provide the code that maps from configuration classes to a hierarchy of nodes. The initial step is to override PlugInLoaderConfigurationDesignManager’s OpenCore method:

protected override void OpenCore(IServiceProvider serviceProvider,
    ConfigurationApplicationNode rootNode, ConfigurationSection section)
{
    if (section != null)
    {
        PlugInLoaderSettingsNodeBuilder builder =
            new PlugInLoaderSettingsNodeBuilder(serviceProvider,
                (PlugInLoaderSettings)section);
        rootNode.AddNode(builder.Build());
     }
}

Like when serializing to XML, here I delegate the real work to another class: PlugInLoaderSettingsNodeBuilder. This class derives from NodeBuilder, which provides some utility methods for mapping between configuration classes and nodes. The idea is that the root node and all collection nodes have associated node builder classes, so a node builder class only contains code to create its own type of node, and delegates the work of creating the rest of the node hierarchy to other node builder classes. This is also the case for PlugInLoaderSettingsNodeBuilder, which creates a new PlugInLoaderSettingsNode and delegates the job of creating the collection of PlugInProviders to another class. The code download for this article shows how this is done.

This code uses a NodeCreationService to create a new PlugInProviderNode from the configuration data, and adds this node to the collection node. To enable the NodeCreationService to perform the mapping, a node map must be registered by PlugInLoaderConfigurationDesignManager in the Register method.

In order to create and register the node map, I create the PlugInLoaderNodeMapRegistrar class (which derives from NodeMapRegistrar) and override its abstract Register method. Here, I simply create the map between the configuration data class and the corresponding node class by calling the inherited AddMultipleNodeMap method:

this.AddMultipleNodeMap("Naive Plug-In Provider",
    typeof(NaivePlugInProviderNode), typeof(NaivePlugInProviderData));

Using the mechanism of node builder classes, I can build a hierarchy of nodes to an almost arbitrary depth and complexity while keeping the implementation of each node builder relatively simple. This approach may seem overly intricate for the very simple example shown here, but it scales very well to more complex configuration schemes where it becomes far more useful.

Conclusion

Creating an Enterprise Library application block is not trivial, but for the right kind of project, it may prove to be more than worth the effort. The core of creating an Enterprise Library application block, or turning a normal configuration-driven library into an application block, lies in developing the run-time component. But the optional extras are equally vital. I have described the design-time component, but this is only the first step among several.

Packaging the application block in a Microsoft® Installer (MSI) package is not difficult to do, and makes the block much easier to download and install. You should also consider creating comprehensive documentation—not only API documentation generated from XML comments, but also a quick introduction, an architectural overview, guidance on how to get started, and so on.

Most developers learn by example, so a few good QuickStart sample applications will make adoption of the application block easier. Even if your audience is only internal to your organization, samples may very well be worth the investment.

Mark Seemann works for Microsoft Consulting Services in Copenhagen, Denmark, where he helps Microsoft customers and partners architect, design and develop enterprise-level applications. He thanks Tom Hollander for providing valuable help for this article. Mark can be reached via his blog at blogs.msdn.com/ploeh.


Editor's Note: Get Ready for AJAX

Get Ready for AJAX

If you’re fortunate enough to attend a developer conference this year, look around at the variety of people. There are all kinds out there, a diverse ecosystem as the marketing folks might say. Even within a relatively narrow group like programmers, you’ll find a wide range of goals and interests.

Not everyone gets a charge out of the same things, and the best entrepreneurs know this. That must be why my Inbox is filled with all manner of spam, from one note with the subject "Mike Tyson wears Rolex You should TO!" to another that blares "Freddy Kruger wears Rolex." You really need to know your audience, not just generate random pitches.

Providing appropriate coverage in the vast yet limited space we have in MSDN Magazine presents a similar problem. What will our readers find interesting? For some, the flashy new features in Windows Vista get their hearts beating faster. Others prefer the gritty details down in the weeds of the server. Not to worry. The range of interesting topics we cover this month, from Windows Vista to Atlas to Enterprise Library, should provide something for everyone. We’ll resist the urge to compare these technologies to Mike Tyson and Freddy Kruger.

Instead, let’s focus on Atlas, the subject of this issue’s lead article. Atlas is the code name for a free framework that lets you take advantage of AJAX technologies in ASP.NET. Though Atlas is not named after Charles Atlas, the "World’s Most Perfectly Developed Man," that analogy may actually fit.

MSDN Magazine Podcast

Do you enjoy reading MSDN Magazine each month? Would you like to learn even more? This issue marks the launch of our new feature, the MSDN Magazine Podcast. Each month we will talk about the hot programing issues of the day with magazine authors and assorted luminaries. This time, we will discuss Atlas and ASP.NET with Matt Gibbs and Fritz Onion. Download or grab the RSS feed today at msdn.microsoft.com/msdnmag/podcast

AJAX stands for Asynchronous JavaScript and XML, a combination of existing technologies that makes it easier to create interactive Web sites. When you develop a site using AJAX, you typically use HTML and cascading style sheets for the display layer, JavaScript to interact with the document object model on the client side, an XmlHttpRequest object to provide a conduit between the client and server, and XML as the data transfer protocol.

Taken together, the technologies in AJAX let the server update pages on the fly, one of the closest things to a Holy Grail Web developers have had. This isn’t the first attempt at dynamic, updateable Web pages, however. Back in 1998, Microsoft released a technology called Remote Scripting (microsoft.com/mind/0498/cutting0498.asp), which let a client and server communicate using a Java-language applet. Remote Scripting made its way into some products, like the version of Outlook Web Access found in Exchange Server 2000.

Atlas is an extension of ASP.NET that takes advantage of AJAX. The ASP.NET community site (atlas.asp.net) has a big section devoted to Atlas where you can get downloads, tutorials, samples, and much more.

As more people adopt Atlas and AJAX, we’ll start to see more mash-ups (Web applications that combine content from multiple sources—maybe from RSS feeds or other public interfaces). Mash-ups are going to be big in the next couple of years. If you’re interested in the cool factor, this is where you want to go. Our advice is to take on the technology in two stages. First, get acquainted with Atlas. Matt Gibbs provides a comprehensive overview in this issue (page 48).

Next, download the CTP (which, as far as we can tell, is the new code name for "beta"). Get it up and running on your machine, and then dive in. The Atlas site provides some good walkthroughs to get you started, but, as you know, nothing works better than getting your hands dirty.

Finally, code code code. It’s the only way to become a mash-up celebrity. And besides, it’s a lot of fun.—J.T.

Thanks to Thanks to the following Microsoft technical experts for their help with this issue: Mike Ammerlaan, Rushi Desai, Andrew Downum, Nikola Dudar, Shawn Farkas, Dana Groff, Mike Harder, Gerald Hinson, Tom Hollander, Charlie Kaufman, Tom Laird-McConnell, Lukasz Pawlowski, Dmitry Robsman, Stefan Schackow, and Ayman Shoukry.

Excel, IntelliSense, Internet Explorer, Microsoft, MSDN, PowerPoint, Visual Basic, Visual C++, Visual SourceSafe, Visual Studio, Windows, Win32, WinFX, Windows Server, and Windows Vista are either registered trademarks or trademarks of Microsoft Corporation. Other trademarks or tradenames mentioned herein are the property of their respective owners. MSDN MAGAZINE DOES NOT MAKE ANY REPRESENTATION OR WARRANTY, EXPRESS OR IMPLIED WITH RESPECT TO ANY CODE OR OTHER INFORMATION HEREIN. MSDN MAGAZINE DISCLAIMS ANY LIABILITY WHATSOEVER FOR ANY USE OF SUCH CODE OR OTHER INFORMATION. MSDN and Microsoft are registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. MSDN Magazine, MSDN, and Microsoft logos are used by CMP under license from owner. MSDN Magazine is published by CMP Media LLC. CMP Media LLC is an independent company not affiliated with Microsoft Corporation. Microsoft Corporation is solely responsible for the editorial contents of this magazine.

MSDN Magazine does not make any representation or warranty, express or implied with respect to any code or other information herein. MSDN Magazine disclaims any liability whatsoever for any use of such code or other information.


Toolbox: Networking, Rich UIs, and More

Toolbox
Networking, Rich UIs, and More
Scott Mitchell

Contents
Perform Network-Level Operations
Slick UIs for ASP.NET Web Apps
Manage Your Font Library
The Bookshelf

Perform Network-Level Operations

End users today expect a level of interconnectivity between applications that was unheard of a mere five years ago. Many applications periodically determine if a new version is available and, if so, offer to download and install it. Financial applications will connect to the Internet for the latest stock quotes. Games seamlessly integrate with online services to keep track of a user’s high scores and how she ranks among her peers. These operations can be performed using a variety of technologies, from low-level TCP or UDP messages to HTTP or FTP requests; they may involve custom message-level protocols or industry standard ones.

Adding connectivity features can be daunting for developers who are mostly experienced in traditional application development paradigms. Network operations are built atop very precise, low-level specifications that leave no room for error. There are many different ways a network operation can fail, which introduces many edge cases that need to be handled. For these reasons, it’s often foolhardy for developers to tackle network-level operations on their own. They should instead rely on a framework or third-party component to provide high-level abstractions for the low-level networking needs.

The Microsoft® .NET Framework provides some classes for network-level operations in its System.Net namespace. There are classes designed for making HTTP and FTP requests and for working with socket-level communications, among others. For a more in-depth set of networking functions, you’ll need to turn to a third-party component. One such component suite that offers the gamut of networking functionality is IP*Works! Version 6 by /n software. IP*Works! is a collection of components that supports a wide array of low-level protocols.

IP*Works! simplifies working with these technologies through high-level libraries. Want to work with a Web site’s RSS feed? There’s no need to programmatically parse the XML; just use the IP*Works! Rss class’s GetFeed(url) method. This populates the Rss class with the items defined in the RSS feed. The particular elements of each item can be accessed with properties like ItemTitle and ItemPubDate.

Part of the challenge of creating networked apps is the inherent asynchrony. When receiving network traffic, the responses come in as chunks at undeterminable times; when transmitting a file over the network, it’s not clear when the transmission will complete. To further complicate matters, during these asynchronous processes, things may go awry. That connection to the FTP server may reset, or the network connection may be lost. IP*Works! simplifies these operations by offering various classes with an expansive list of events for which event handlers can be created.Virtually all of the components use the standard .NET asynchronous programming model to make starting and managing asynchronous operations a familiar exercise.

Price: $499 per developer workstation (royalty-free distribution).

www.nsoftware.com

Slick UIs for ASP.NET Web Apps

For me, designing the user interface for an application can be quite challenging. I am a backend ASP.NET developer by profession and, honestly, lack any and all semblance of artistic acumen. (My user interfaces have been compared to the green and black monochrome mainframe applications from yesteryear.)

Creating a user interface isn’t hard—ASP.NET ships with a number of controls that provide the common UI patterns. It’s a breeze to add a textbox, dropdown list, checkbox, or an editable grid of data to a Web page, but it’s not as easy to make it look good.

True, Visual Studio® offers its Auto Format wizard (to which my end users are eternally grateful), but that’s just window dressing. While visual aesthetics are important in UI design, the overall usability of the interface is equally important. Here, too, there is little help from the built-in ASP.NET controls, which provide the building blocks of a user interface but leave the design work to the page developer.

For design-challenged developers like me, it’s often more cost effective to turn to a third-party control vendor than invest the time and energy in attempting to create a usable, eye-pleasing user interface. One such ASP.NET control vendor is telerik, which offers the r.a.d. controls suite. The r.a.d. controls suite is a collection of 18 custom controls that provide polished, professional-looking results for a variety of common user interface scenarios.

The r.a.d. editor provides a rich, WYSIWYG editing experience in a Web page, the r.a.d. grid displays and modifies data, and various navigation and categorization controls are offered.

The r.a.d. controls use themes to easily manage the aesthetics of the rendered output, and ship with a number of predefined themes, making it easy to create UIs that look like they came from a highly paid design shop. The r.a.d. controls offer more than just eye candy, though, as their rich feature set, coupled with AJAX support, allows for the creation of very fluid and straightforward user interfaces.

For example, the r.a.d. treeview allows a page visitor to change the name of a treeview node in place, supports dragging and dropping (both for the repositioning of nodes in a tree and moving nodes from one treeview to another), keyboard support, and context menus. Best of all, adding the rich user interface and wiring up the backend code is incredibly simple. Configuring a control to use the more involved user interface features is as simple as setting a few properties, and handling the complex user interface typically involves just creating an event handler.

If you are an artistically-challenged Web developer like yours truly, check out telerik’s r.a.d. controls suite. It will help you move from ho-hum user interfaces to ones that will bedazzle.

Price: $799 per developer license for the control suite.

www.telerik.com

Manage Your Font Library

Choosing the right font for the user interface in an application can be a trying task for developers who don’t have an art department backing them up. Most developers opt for whatever font Visual Studio recommends, in part because seeing how a particular chunk of text looks in different fonts is not readily accessible in Visual Studio or even in many word processor programs.

To help with the task of choosing a font, check out Advanced Font Viewer 2.6 by Styopkin Software. Advanced Font Viewer allows you to enter text and then display it in all of the fonts installed on the system. You can make the text bold, italic, underlined, or strike-through, and can adjust the font size. Additionally, you may select a single font and view samples of the inputted string with differing font sizes, explore the font’s character map, or inspect the font’s embedded information. You can also easily print fonts or view them in a Web browser to see how they appear in other media. Advanced Font Viewer also makes it easy to install new fonts and remove existing ones.

Picking the ideal font for your application can help improve the user experience and should not be a random choice or one based on Visual Studio presets. While Advanced Font Viewer is no substitute for a trained graphics designer, it can help those less experienced at font selection to quickly view their font choices and make a more informed decision.

Price: $39 per developer license.

www.styopkin.com

The Bookshelf

In the past, a sure-fire way to determine the intended audience for a book was to peruse its figures. Beginner-level books were ones rife with screen shots showing step-by-step instructions; the advanced titles had a higher text-to-figure ratio and their figures were more likely to be fancy architectural-level diagrams. I say "in the past" because O’Reilly’s new series, "A Developer’s Notebook," turns that assumption on its head, providing step-by-step, visual tutorials from intermediate to advanced.

In ASP.NET 2.0: A Developer’s Notebook (O’Reilly, 2005), by Wei-Meng Lee, each chapter is broken down into a number of labs, with each lab focused on how to accomplish a particular task. You won’t find any high-level architectural discussions or best practices ruminations here—just a quick, step-by-step list and plenty of screen shots to help you follow along. Despite the straightforward step-by-step nature of these labs, they are clearly targeted for developers who know their stuff.

For example, the Chapter 4 labs focus on working with data in ASP.NET 2.0. You’ll find instructions on how to go from a blank ASP.NET page to one that lists database results filtered by the selected item in a dropdown list, with screen shots to walk you through the process.

The book contains over 50 labs spread out over eight chapters that cover Master Pages, Web Parts, data access, security, and performance, among other topics. My favorite chapters examine data access and security, respectively. In a combined 100 pages, Chapters 4 and 5 showcase ASP.NET 2.0 improvements in these two areas with labs that, in ASP.NET 1.x, would have required at least twice as many pages. While the whole book is a testament to ASP.NET 2.0 enhancements, these two chapters are likely to have ASP.NET 1.x developers most excited about the new possibilities.

A subtle, but very enjoyable touch found in the "A Developer’s Notebook" series is that the books really do look and feel like notebooks. The paper stock feels different than the standard computer trade book and has the same faded grid found in graph paper, and various notes appear in the margins in a font that looks like hand-written cursive. These nuances foster a sense of camaraderie with the author, as if he had let me borrow his personal notes. But, thankfully, these are notes I do not need to give back!

Price: $29.95.

Scott Mitchell, author of numerous books and founder of 4GuysFromRolla.com, is an MVP who has been working with Microsoft Web technologies since 1998. Scott is an independent consultant, trainer, and writer. Reach him at Mitchell@4guysfromrolla.com or via his blog at ScottOnWriting.NET.


Advanced Basics: Monitor Your Apps with System.Diagnostics

Advanced Basics
Monitor Your Apps with System.Diagnostics
Brad McCabe

Contents
Event Logs
Performance Counters
Process Class

It never fails. The application you just deployed ran great on your development machine—but stumbles in production. The problem might show up right away or maybe it creeps up over time. Now what?

All too often developers whip out the debug version of their code and redeploy that. This special build is often loaded with random writes to some file on disk for the developer to wade through later to try to figure out what’s going on. There has to be a better way.

The solution lies in the System.Diagnostics namespace, which provides classes for interacting with event logs, performance counters, and system processes. You can leave this monitoring code enabled in your production application and view information whenever a problem arises. Let’s see how you can use these classes in your applications.

Event Logs

Countless developers debug their applications by writing information out to external files. Besides leaving bits of trash around the hard drive, this approach often takes a long time, is not sortable or filterable, and rarely provides rich data. So why do so many people do it? They don’t realize how easy the event log is to use.

Let’s start by writing to the event log the long way to illustrate the concepts, then take a look at the quick and easy way. First, add an import statement to your code to reference the System.Diagnostics namespace, then insert the following code into your error handling routine:

‘ Create the source, if it does not already
‘ exist.
  If Not EventLog.SourceExists("MSDNDemo") Then
    EventLog.CreateEventSource("MSDNDemo", _
    "CustomLog")
End If

‘ Create an EventLog instance and assign its
‘ source.
Dim myLog As New EventLog()
myLog.Source = "MSDNDemo"

‘ Write an informational entry to the event log.
myLog.WriteEntry("Hello from the event log.")
The top section checks whether you’ve already created the source of your event. Most developers use the name of their application, making it easy to quickly pick out the application’s events from other entries in the event log.

If the source does not exist, it is created and associated with an event log. You can use existing sections of the event log such as the Application and System sections, or you can create your own section by specifying a name (as this demo does).

Once the source is created, adding events to the event log is as simple as creating a new EventLog object and associating it with the source. This example simply writes "Hello from the event log," but you can pass in detailed information—like a stack trace—to the WriteEntry method and place any information in the event log.

Figure 1 User-Generated Event Log Entries in a Custom Section
Figure 1 User-Generated Event Log Entries in a Custom Section

Let this code execute several times, and your event log will display the activity as shown in Figure 1. You can view the Windows® Event Viewer (%windir%\system32\eventvwr.exe), as shown in Figure 1. You can also view the event log from inside of Visual Studio® in the Server Explorer, as in Figure 2. If you double-click one of these entries, you can view the details and see your custom error message (see Figure 3).

Figure 2 Event Logs within Visual Studio
Figure 2 Event Logs within Visual Studio

The EventLog.WriteEntry method has 10 overloads that give you lots of flexibility and control. One of the most useful lets you specify the type of event you’re logging. Notice in Figure 1 that each item is of type Information. For a demo this works fine, but in the real world you might have lots of different events. The WriteEntry method lets you supply the event type, such as error, warning, information, success audit, or failure audit:

    myLog.WriteEntry("Your error here.", _
    EventLogEntryType.Error)

Figure 3 User-Generated Event Log
Figure 3 User-Generated Event Log

Now let’s try it the quick and easy way. Visual Basic® 2005 introduced the My object to act as a speed dial into the framework. You can use My to quickly add events to the event log:

My.Application.Log.WriteException( _
    "Written from My", TraceEventType.Warning, _
    "Additional Information")
And you can also use My via the WriteEntry method:
My.Application.Log.WriteEntry("Written from My", TraceEventType.Error)
But if you call this method and look in the event log, the message may not be there, because the My object lets you route your messages to different listeners based on settings in your configuration file. More discussion of this very powerful functionality is beyond the scope of this column, but you can find further information at "My.Application.Log Object".

Use the event log responsibly—there are few things more annoying than overly chatty applications that fill the event log with worthless noise. If you have an error, important warning, or information, you should log it. Specifying types on your entries will greatly help because most columns can be filtered in the event viewer in Windows.

Performance Counters

The event log is a great place to write messages, but what if you want to see what is going on inside your application while it is running? This is where performance counters come in.

Many developers are familiar with the Windows Performance Monitor for viewing CPU and memory utilization or hard drive information (see Figure 4). Did you know you can add your own counters and data points with the PerformanceCounter class? Used together, PerformanceCounter and the Windows Performance Monitor are like being able to hook an EKG machine up to your application at any time.

Figure 4 Standard Windows Performance Tool
Figure 4 Standard Windows Performance Tool

Before I talk about adding custom counters, let’s explore the counters that are already available. Open the Server Explorer in Visual Studio and navigate to the Performance Counters on one of your machines. You’ll see a huge list of categories for counters, which expand to reveal many individual counters for each category (see Figure 5).

Figure 5 Performance Counters
Figure 5 Performance Counters

You can use these system-managed counters in your application. Go ahead and expand the Memory category and then the Available Bytes counter. Drag the Available Bytes counter to a new Windows Form in your application.

On that form, drop a label and a new timer object with the Enabled property set to true. Add the following code:

Private Sub Timer1_Tick(ByVal sender As System.Object, _
     ByVal e As System.EventArgs) Handles Timer1.Tick
    Label1.Text = Me.PerformanceCounter1.NextValue
End Sub
This is a very simple example, not something you’re likely to do in a real application, but it shows how you can add basic monitoring or retrieving system counters to an application. In a real world scenario, you might take action based on the state of a value, log this information, or do any of a number of things.

More than likely, you are going to want to write some custom counters to monitor the health and performance of your apps at any given time. To add a new counter, right-click on the Performance Counters item in Server Explorer and choose Create New Category.

In this example, let’s create a category called TestCategory and add a new Counter named TestCounter. This time, you’ll access the counter from code.

Start by dragging and dropping a new timer and button to the form and add a private variable called counter to your form:

Private counter As PerformanceCounter
In the button click event, add the following code:
‘Init the counter object
counter = New PerformanceCounter("TestCategory", "TestCounter", False)
   ‘ Set its value to 0
   counter.RawValue = 0
   ‘ Start the time to update the counter
   Timer2.Enabled = True
In the timer event, add this code:
   Counter.IncrementBy(New Random().Next(-5, 10))
This causes the counter to increment by some random number from -5 to 10. The example uses a random number, but this could be the number of sales or orders processed, users logged on, errors logged, or anything else it makes sense to track in your application.

Before you start the sample application, load Performance Monitor from under Administrative Tools. This time, click on the + sign to add a new counter. Under Performance Object, you’ll now see the new TestCategory you created. You can select that from the dropdown and pick TestCounter from the list of counters.

With Performance Monitor running, click the button of the sample application. You should see a graph similar to Figure 6.

As you can see in the sample code, the IncrementBy method was used to increase or decrease the counter. If you wanted to move the counter up or down by one, you could simply call the Increment and Decrement methods.

Figure 6 TestCounter Results Displayed in Performance Monitor
Figure 6 TestCounter Results Displayed in Performance Monitor

For an example like this that uses the Server Explorer, some developers wonder if they can do it all with code. Yes, you could write code such as

   If Not
     PerformanceCounterCategory.Exists("MSDNSample")
   Then
     PerformanceCounterCategory.Create("MSDNSample", _
       "Built to show how to add custom counters", _
        PerformanceCounterCategoryType.SingleInstance, _
       "MSDNSampleCounter", _
       "Built to show how to add custom counters")
   End If
and add it to your button click event to create a different category and counter entirely from code. If you update your other code to point to this category and counter, you’ll see that the project runs and functions exactly the same as the sample.

Process Class

So far in this column you’ve seen how to use an event log and performance counters; now let’s dig into a third area of the System.Diagnostics namespace, the Process class.

First, let’s define a process: it’s simply a running application. With the Process class and the right permissions you can access information and perform actions, such as start or kill, on various processes on users’ systems.

The Process class gives you access to lots of data about your application or other processes. For the example here, let’s boost the base priority of the sample application.

Here’s the scenario. Your application is running on a server; it is the primary application and function on that server, but it’s not performing well because background tasks are consuming too much processor time. You can do one of two things; lower the priority of the background tasks or increase the priority of your tasks. (In most cases, you won’t need to adjust the priority of a process because Windows does a good job of running multiple processes and swapping them in and out.)

To work with the process class, you first need to fetch a reference to either your process or another process. Since you want to boost the current process, you’ll use:

   Dim myProcess As Process = _
       Process.GetCurrentProcess()
To access other processes on your machine or remote machines, you’d use the GetProcessessByName method instead:
   Dim theProcesses() As Process = _
       myprocess.GetProcessesByName( _
       "ProcessName", "MachineName")

Once you have a reference to a process, you can find out lots of information about it using its properties. For a complete list of properties, see ".NET Framework Class Library Process Members".

It takes just one line of code to boost the priority:

myProcess.PriorityClass = ProcessPriorityClass.RealTime
You boosted the application to RealTime. This is the highest level you can set in the OS and gives the app priority over just about everything This can lead to problems if your application hogs all of the system resources. Launch Task Manager to confirm that the OS sees the application in its boosted state, as shown in Figure 7.

Figure 7 Boosting to RealTime
Figure 7 Boosting to RealTime

You can use the process command to do lots of common tasks. Let’s say you wanted to start Microsoft Internet Explorer® minimized and navigate to the Visual Basic Developer Center. You would simply use the following code:

Dim startInfo As New ProcessStartInfo("IExplore.exe")
startInfo.WindowStyle = ProcessWindowStyle.Minimized
startInfo.Arguments = "msdn.microsoft.com/vbasic"
Process.Start(startInfo)
The Process class works well for printing, too.

Dim myDocumentsPath As String = _
    Environment.GetFolderPath(Environment.SpecialFolder.Personal)
myProcess.StartInfo.FileName = myDocumentsPath + "\MyFile.doc"
myProcess.StartInfo.Verb = "Print"
myProcess.StartInfo.CreateNoWindow = True
myProcess.Start()

As you can see, there are many uses for the System.Diagnostics namespace, from logging events to the event log, to reading and writing performance counters, to working with system processes. This is an incredibly useful and powerful set of tools that every developer should become familiar with and use in their work.

Send your questions and comments to basics@microsoft.com.

Brad McCabe is a program manager at Microsoft and, among other things, is responsible for the Visual Basic Developer Center on MSDN.


CLR Inside Out: Using Strong Name Signatures

CLR Inside Out
Using Strong Name Signatures
Mike Downen

Contents
A Short Refresher
Strong-Name Signing
Why Use Strong Names?
What Strong Names Can’t Do
Working with Strong Names
Delay Signing
Test Key Signing
Managing the Private Key
Working with Authenticode

Strong name signatures (and signing in general) are a key facet of Microsoft® .NET Framework security. But regardless of how well designed .NET signatures may be, they won’t offer the maximum benefit if you don’t know how to use them properly. This installment of CLR Inside Out talks about strong names, strong name signatures, and how to use them.

A Short Refresher

Digital signatures are used to verify the integrity of data being passed from the originator (the signer) to a recipient (the verifier). The signatures are generated and verified using public key cryptography. The signer of a message has a pair of cryptographic keys: a public key, which everyone in the world knows, and a private key, which is kept secret by the signer. The verifier knows only the public key, which is used to verify that the signer knew the private key and the message.

In some cases, when some additional infrastructure is in place, digital signatures can also be used to reliably learn the name of the signer, and to ensure some chunk of data (a message, some code, or so on) has not been modified after the signer created the signature for the data.

Various mechanisms are used to implement digital signatures. The current implementation of strong names in the .NET Framework relies on the RSA public key algorithm and the SHA-1 hash algorithm.

Strong-Name Signing

Strong names offer a powerful mechanism for giving .NET Framework assemblies unique identities. The strong name for an assembly consists of five parts: a public key, a simple name, a version, an optional culture, and an optional processor architecture. The public key is an RSA public key. The simple name is just a text string—usually the name of the file (without the extension) that contains the assembly. The version is a four-part version number, in the form of Major.Minor.Build.Revision (for example, 1.0.0.1).

To get a valid strong name, an assembly is strong-name signed during the build process. This is done using the private key that corresponds to the public key in the strong name. The strong name signature can then be verified using the public key. There are a few different techniques for creating a strong name signature and for managing the private key used in signing. I’ll discuss these various techniques later in the column.

When one assembly references a strong-named assembly, the referring assembly captures the strong name information for the referenced assembly. When the .NET Framework loads a strong-named assembly, it verifies the strong name signature. If the strong name signature of the assembly cannot be verified, the .NET Framework will not load the assembly.

One exception to this process has to do with strong-named assemblies that come from the Global Assembly Cache (GAC). These are not verified each time the .NET Framework loads them. This is because assemblies in the GAC are verified when installed in the GAC. Since the GAC is a locked-down, admin-only store, assemblies located here do not need to be verified each time they are loaded.

Why Use Strong Names?

Strong names prevent spoofing of your code by a third party (that is, of course, as long as you keep the private key secure). As mentioned, the .NET Framework verifies the signature either when loading the assembly or when installing it in the GAC. Without access to the private key, a malicious user cannot modify your code and successfully re-sign it.

Given that strong names prevent spoofing, they can be used to make certain security decisions. An enterprise, for example, could choose to trust all code that comes from the enterprise’s intranet and has been signed with the enterprise’s strong name key. But strong name signatures do not contain any reliable information about the publisher, so while it is safe to trust keys you control, it is dicier to trust keys from other organizations unless you have a secure channel to get their public key.

It is also important to note that strong names do not have any revocation mechanism that can be used if the private key is compromised. It goes without saying that if you plan to use your enterprise’s strong name for trust, keep the private key private! (I’ll have tips for this later in the column.) It’s a good idea to scope the trust of a strong name to a particular location (such as your enterprise’s application server) or at least a zone (such as your intranet zone). This provides some mitigation in case your private key is ever compromised or a buggy version of a signed assembly makes its way to some bad guys and they try to lure users into running it from outside the intranet.

What Strong Names Can’t Do

Strong-name signing is useful for preventing a malicious user from tampering with an assembly and then re-signing it with the original signer’s key. It cannot do anything, however, to prevent a malicious user from stripping the strong name signature entirely, modifying the assembly, re-signing it with his own key, and then passing off the assembly as his own code. Digital signatures in general and strong name signatures in particular cannot help with this problem. Watermarking is a more appropriate solution for protecting against this scenario.

Strong names are secure only when the strong name private key is kept secure. As I mentioned earlier, they have no revocation capabilities and they don’t expire. Nor do strong names inherently provide any way to securely map a public key to a particular publisher. Thus, strong names should be used very cautiously when making security decisions. Ideally, these security decisions should only be made when you can be sure the strong name public key corresponds to a particular publisher (for example, when the public keys are generated by your organization’s IT department). You should note that more sophisticated signing schemes (for example, Authenticode®) are available that do offer revocation and publisher verification capabilities.

Working with Strong Names

Strong-name signing is a good idea for most applications, especially those that are deployed over a network or any medium not fully controlled by the deployer. The anti-spoofing and anti-tampering benefits are quite valuable. However, there are some challenges to be aware of when using strong-name signing.

First, all assemblies referenced by a strong-named assembly must also be strong-named. If you reference an assembly written by a third party that is not strong-name signed, you cannot strong-name sign your assembly.

Second, strong-name signing makes servicing more complicated. Under current versioning policy, an assembly will always attempt to load the exact version of the assembly it was built against. If, for example, your application was built against version 1.0.0.0 of a strong-named assembly and you fix a bug in the assembly, bumping the version number to 1.0.0.1, the existing application will not find the updated assembly.

There are a few ways to deal with this situation. You can rebuild the application against the new assembly. Obviously, this is an annoying process for just picking up a bug fix and it may not be an option in some situations. Still, it works fine when your code is not widely deployed or the assembly being serviced is not widely shared across applications.

Another option is if the assembly is installed in the GAC, you can use publisher policy to redirect loads for version 1.0.0.0 to version 1.0.0.1. However, publisher policy is complicated.

Yet another option is to fix the bug and not change the assembly version number, so existing applications can still find the assembly. This is the approach the .NET Framework uses for bug fixes, although it’s not appropriate for new features or anything that breaks compatibility. If you are adding new features, you should bump the assembly version up and have applications rebuild to opt in to the new features.

Needless to say, strong-name signing adds some complexity to your development and build processes. But this is necessary to keep the private key secure. I’ll discuss how to deal with this complexity in a moment.

Now let’s take a quick look at a simple signing scenario: creating a random key pair and storing it in a file. I’m only using this particular approach as a way to introduce you to how strong-name signing works. Obviously, storing the private key in a file next to your code is not a good way to protect your key so you generally should not use this approach in production.

If you are using command-line tools, you can create a key pair with sn.exe in the SDK. Then you pass that key pair to the compiler when compiling your code. The following commands create a new random key pair using sn.exe and build a strong-name signed assembly with csc.exe:

sn –k mykey.snk
csc /keyfile:mykey.snk myapp.cs
This produces a strong-name signed application named myapp.exe. You can do the same thing with Visual Studio® 2005. The Signing pane on the project properties page controls strong-name signing for your project (see Figure 1). To create a simple key file, select the "Sign the assembly" option, and create a new key file that is not password protected. Visual Studio will add the key file to your project and use it to sign your application.

Figure 1 Signing the Assembly
Figure 1 Signing the Assembly

As I mentioned, storing the private key in a file next to your code is not a secure way to protect your private key. Fortunately, Visual Studio allows you to password-protect the key file. When you create a new strong name key for your project, just leave the "Protect my key file with a password" option selected and enter a password for your key file (see Figure 2). Visual Studio adds a personal certificate, a PFX file, to your project. This file includes your encrypted private key, protected by the password you selected.

Figure 2 Creating a Key
Figure 2 Creating a Key

While password-protecting your key files is a much better solution than storing them in the clear, it is still not ideal. You would still have to distribute the PFX file to all of your developers, and they would all have to know the password for the PFX file. Secrets that are widely shared like this do not tend to stay secret for very long. Ideally, you should not have to distribute the private key to build and test your code during development.

Delay Signing

This is where delay signing enters the picture. Delay signing allows you to generate a partial signature during development with access only to the public key. The private key can be stored securely out of the hands of the developers and used to apply the final strong name signature just before shipping your code.

To use delay signing, follow these steps:

  1. Extract the public key from the key pair. The public key can be distributed to developers in your organization without any security risk.
  2. Use the public key to delay sign your assemblies during development. Because the assemblies are not fully signed yet, you’ll also have to configure your development machines to skip strong name signature verification for your key—otherwise, the .NET Framework will not allow you to load the delay-signed assemblies.
  3. When you are ready to ship your code, use the (well guarded) private key to apply the final strong name signature to your delay-signed assemblies.

For step one, you can use the sn.exe tool to extract the public key from a key pair. For example, say you have your key pair stored in a securely configured key container named EnterpriseKey on a locked-down machine. You would use the following command to extract the public key to the EnterprisePublicKey.pk file:

sn –pc EnterpriseKey EnterprisePublicKey.pk
You can then distribute EnterprisePublicKey.pk to the developers in your organization. You can usually use the same public key for multiple assemblies because each will have a different simple name.

For step two, you use compiler options to delay sign your assemblies with the public key when building them. The example that follows creates a delay-signed application using options for the C# compiler:

csc /delaysign+ /keyfile:EnterprisePublicKey.pk myapp.cs

Now if you try to run myapp.exe, it will not run because it does not have a valid full strong name signature. You must configure the .NET Framework to skip verification for the delay-signed assemblies using your public key. To do this, you use the sn.exe tool again. The following example configures the .NET Framework to skip strong name signature verification for the myapp.exe assembly on your development machines:

sn –Vr myapp.exe

You can also use the sn.exe tool to skip signature verification for all assemblies signed with a particular key. For example, if you want to configure your machine to skip all assemblies delay signed with the same key as your application, you first do the following on your development machines:

sn –T myapp.exe
This prints out the value of the public key token, a shortened version of the public key that looks something like this:
Microsoft (R) .NET Framework Strong Name Utility  Version 2.0.50727.42
Copyright (c) Microsoft Corporation.  All rights reserved.

Public key token is b03f5f7f11d50a3a
(Note that the actual value of the public key token will vary depending on your public key.) You then execute the following command to skip strong name verification for any assembly using that public key token:
sn –Vr *,b03f5f7f11d50a3a

Any assembly delay signed with that public key will now skip strong name signature verification and run on your development machine. Note that skipping strong name signature verification is something that should only be done on development machines. It should never be done on production computers because it opens up those machines to assembly spoofing attacks.

Finally, for step three, you use the real private key to generate the final full strong name signature before shipping your code. Again, you use the sn.exe tool to do this. For example, if your key pair is stored in a key container named EnterpriseKey on your signing computer, you would use the following command to sign a delay-signed assembly:

sn –Rc myapp.exe EnterpriseKey

Now the assembly has a full signature. One of the nice things about delay signing is that it does not require any of the assemblies that use the delay-signed assembly to be rebuilt. Any assemblies that refer to the delay-signed assembly had access to its public key and were therefore able to create a full assembly reference, even though the assembly did not have a full signature.

Test Key Signing

There is a new feature in the .NET Framework 2.0 called test key signing. This is similar to delay signing. With test key signing, you sign your assembly with an alternate private key during development and then configure your development computers to verify the signatures against the alternate public key that corresponds to the alternate private key. Test key signing is valuable when you want almost the exact same behavior as full signing for testing purposes. For example, with delay signing, assemblies are not verified at all. This could hide performance issues if your application loads many strong-named assemblies because none of the strong name signatures would be verified during test runs when delay signing is turned on.

Test key signing involves the following steps:

  1. Delay sign your assembly.
  2. Create a test key pair with which to sign your assemblies. You can do this using sn.exe with the –k option.
  3. Test sign your assembly. You can do this using sn.exe and either the –TS or –TSc option. This will generate a signature for the assembly using the private key from your test key pair.
  4. Configure your development machines to use the test public key to verify the test key signed assemblies. You can use the –Vr option on sn.exe to do this.

When you are ready to ship your assemblies, you sign them with the real private key using sn.exe with either the –R or –Rc option, just as you would for a delay-signed assembly. The following series of commands creates a test key pair, test key signs a delay-signed assembly, extracts the test public key, and configures a development machine to verify the assembly against the test key:

sn –k TestKey.snk
sn –TS myapp.exe TestKey.snk
sn –p TestKey.snk TestPublicKey.pk
sn –Vr myapp.exe TestPublicKey.pk

When using test key signing, you should still protect the test private key, but you do not have to guard it as closely as the real private key. It is also a good idea to change the test key pair periodically.

Managing the Private Key

It is much easier to keep your private key secure if you are using delay signing or test key signing. You only have to have the private key installed on one or possibly a small number of locked-down signing computers. You can install the private key in a key container using restrictive access control lists (ACLs) to prevent unauthorized access to the key container. Alternatively, you can store the private key on a smart card or other separate hardware device. The sn.exe tool allows you to use a different cryptographic service provider (CSP) for signing. Take a look at the –c option in the documentation for sn.exe.

Working with Authenticode

Authenticode signatures provide more security semantics than strong name signatures. To generate an Authenticode signature, you have to obtain a code-signing certificate from a trusted certificate authority (CA). This can either be a commercial CA or a CA established by your IT department. The CA provides some level of assurance that the person or organization with the certificate is who they say they are, giving you some confidence that the software really came from the publisher listed in the signature. Authenticode also supports expiration and revocation, making it more flexible than strong name signatures. However, like strong name signatures, the presence of an Authenticode signature does not guarantee that you can trust the code. Authenticode simply provides some assurance about who published the code and that the code has not been tampered with since the publisher signed it.

Authenticode signatures can co-exist with strong name signatures. However, you must strong-name sign your assembly first, before applying the Authenticode signature. Adding the Authenticode signature will not invalidate a strong name signature.

Authenticode signatures can be used to trust applications deployed using the new ClickOnce model in the .NET Framework 2.0. You can use the tools in Visual Studio 2005 or in the SDK to apply an Authenticode signature to the deployment manifest of your ClickOnce application. You then install the certificate used to create the Authenticode signature in the trusted publisher certificate store on the client computers that will run your application; this can be done using Group Policy, Systems Management Server (SMS), or any other deployment mechanism available to you. Any ClickOnce application signed with this Authenticode certificate will run without any prompting. This is a convenient way to trust your enterprise’s applications.

Strong name signatures serve a valuable role in providing a form of identity for managed assemblies. Just remember that private key security is critical for strong names. If you do decide strong name signatures are appropriate for your organization, be sure to use some of the techniques discussed in this column to manage and protect the security of your private key.

Send your questions and comments to clrinout@microsoft.com.

Mike Downen is the program manager for security on the CLR team. He works on Code Access Security, the cryptography classes, and the ClickOnce security model. You can read Mike’s blog and contact him at blogs.msdn.com/CLRSecurity.


Data Points: Report Controls in SQL Server 2005 Reporting Services

Data Points
Report Controls in SQL Server 2005 Reporting Services
John Papa

Code download available at: DataPoints2006_07.exe (391KB)
Contents
Creating Charts
Embedding Expressions in Charts
Report Links
Creating the Link
Embedded Code
Accessing .NET Code
Security Policies
Accessing Code via a Report
Wrap Up

Reporting has always been one of the dark arts of development. The tools typically seem to do just enough to get you to a certain point, then leave you to find workarounds to solve more complex issues. Fortunately, reporting has evolved quite a bit over the past few years.

SQL Server™ 2005 Reporting Services lets you embed code in a report that can be accessed by various properties of that report, and it takes this concept a step further by allowing you to access external .NET libraries. It also offers a few techniques to integrate expressions and .NET code within a report, providing access to business logic and XML files.

Creating Charts

SQL Server 2005 Reporting Services includes a basic chart control that handles most situations quite nicely, and third-party chart controls are available as well. The chart controls let you set categories for the x axis and data points for the y axis, and you can add one or more series to the chart.

To create a chart, you can drag a chart control onto the report designer in the Layout tab. The chart control automatically goes into "set me up" mode, letting you drag the DataSet’s fields into the category, data, and series areas. The fields you drag into the category area are used for the x axis while those dragged to the data area are used as the data points that define the bars, pie chart, or lines (see Figure 1). You need at least one field in each of these areas to create a useful chart, though you can add more if you want.

Figure 1 Creating a Chart with Drag and Drop Chart Controls
Figure 1 Creating a Chart with Drag and Drop Chart Controls

You can also drag fields to the series area to show more than one set of data points on the chart. For example, if you want to show the sales for a particular customer by month, you could start by setting a filter to show only that customer. You’d then drag the sales field to the data area and the sale date field to the category area and change its value property to something like the following expression, which effectively groups the orders by month:

= Year(Fields!OrderDate.Value) & "-" & Month(Fields!OrderDate.Value)

If you wanted to build on this chart by comparing customer A to customer B, you could drag the customer field to the series area. Assuming you’re using a bar or column chart (which lend themselves nicely to this type of comparison), you’ll see both customers’ sales by month, with each customer identified by a different color bar or column. Series should be used only when necessary as they do dramatically increase the number of data points shown on the chart. You’ll have to weigh the value of showing more data against being able to quickly read and understand the graph.

Figure 2 shows the total sales for each customer in a given country. Each slice of the pie chart represents a customer’s sales in the selected country. The country is chosen by the user via a report parameter which gets a list of countries from a query separate from that which fills the pie chart. (For more details on how to create and establish report parameters, see my June 2006 Data Points column.

Figure 2 Parameter-Driven Pie Chart
Figure 2 Parameter-Driven Pie Chart

Switching the type of chart is as easy as going to the Layout tab of the report designer, right-clicking the chart control, and selecting the chart type from the popup menu where you’ll find a variety of charts. Figure 2 represents the data as a pie chart, but you could easily change this to a column, line, or bar chart by selecting the chart type and redeploying the report.

Embedding Expressions in Charts

One of the coolest features of SQL Server 2005 Reporting Services is how you can embed expressions in just about every property in a report. For example, you can set a chart’s title using a simple concatenation expression, as I did for the chart in Figure 2:

   = "Customer Sales for " & Parameters!country.Value

Notice that the title in Figure 2 says "Customer Sales for Italy." That’s because the user selected Italy as the country from the report parameter. You can use expressions to set just about any property of the chart, including the color of the bars, the data point labels, or the axis labels. Expressions are critical to reports since they are used throughout to configure so many aspects, including the properties of chart controls, filters, SQL parameters, and to create report links. (The report in Figure 2, CustomerSalesByCountryChart.rdl, is included in this month’s code download.)

Report Links

The ability to embed report links lets you enable users to drill into information on the fly without having to show everything on a single report. For example, suppose a manager is looking at a list of products and their inventories. While examining a product with a low inventory, the manager becomes curious as to how the inventory got so low and wants to be able to quickly see the sales totals for that product. By creating an embedded link within the product list report to another report that shows a chart of the product’s sales, the manager can quickly and easily get her answer.

Report links can be embedded within many elements of a report including header, details, and footer controls as well as in data points in a chart. Report links can navigate to a bookmark in the current report, a sub-report, an entirely different report, or a separate URL. The report links can also pass parameters to the destination, which is how in the previous example the manager clicking on a product in a product list report links to a chart showing that product’s sales.

To demonstrate report links, I created two reports: a product list report (ProductList.rdl) and a product sales chart report (ProductSalesChart.rdl). The product list report simply fills a DataSet with the ProductID and the ProductName fields form the Northwind Database’s Products table. The products are then displayed on the report using a table control containing textbox controls for the fields.

For the product sales chart report, I created a DataSet inside the report that gets the orders for all of the customers for a specific product and date range. Then I put a chart control on the report in the layout tab. I made the chart type "Simple Column" and set it to be 3D with the columns set to be Cylinders. (The latter two settings are on the 3D Effect tab on the chart control’s properties page.) I then dragged the Order Date field onto the category area and dropped the Extended Price field onto the data area. Since I wanted to show the total sales amount (sum of Extended Price) for each month, I set the value expression for the Order Date field to use the year and month of the order date.

When I created the DataSet that grabs the total sales from a stored procedure, the report created report parameters that match the parameters required by the stored procedure. Thus, the Report Parameters dialog already contains three parameters: startDate, endDate, and productID. For simplicity, to grab all of the data in Northwind, I set the start and end dates to 1/1/1996 and 1/1/2010. In a real-world situation I’d probably set the start date to one year ago today and the end date to today, so the report would show a year’s time by default.

The user should be able to choose the product that drives the chart, so I modified the productID report parameter to let the user select the ProductName from a dropdown list (for more information on report parameters, see the June Data Points column). Now the chart can run on its own and once the user selects a product, the chart will show the sales for the product by month.

Creating the Link

Once I had the two reports, I created a link in the product list report’s ProductName to connect to the product sales chart report and pass it the ProductID that was clicked. I went back to the product list report and selected the textbox control that displays the product’s name and went to its properties. Next I went to the Navigation tab, set the Hyperlink action to "Jump to a report," and selected the ProductSalesChart report from the dropdown list. Then I clicked on the Parameters button and passed the parameters, as shown in Figure 3. Notice that I’m sending the ProductID from the selected row (Fields!ProductID.Value) to the product sales chart report. At this point I can preview the product list report (see Figure 4), click on a product and the second report’s chart will appear (see Figure 5) showing that product’s sales by month.

Figure 3 Linking Report Parameters
Figure 3 Linking Report Parameters
Figure 4 Viewing a Report with Links
Figure 4 Viewing a Report with Links
Figure 5 Linked Product Sales Chart
Figure 5 Linked Product Sales Chart

Report links are very handy for creating drilldown charts that let users click on a bar or a pie slice and link to another report to show more detailed information. They’re also useful for scenarios in which you want to link to an external application. For example, you could set the link to point to a URL (instead of another report) where the URL links directly to a Web application and passes its parameters via a querystring. This is a nice feature to use on reports that show detailed data (such as individual sales reports) and that allow the user to click on the report to go directly to the sale in the Web application.

One caveat about report links is that finding where to set them can involve clicking into several sub-menus. Setting report links for a textbox is as simple as going to the Navigation tab in the property dialog. But setting them for a bar in a chart means going to the chart’s property dialog, clicking on the Data tab, selecting the Data Value from the list and clicking its Edit button, then going to the Action tab of the Edit Chart Value dialog that appears and, finally, setting the link.

Embedded Code

Often when I design reports I find I’m repeating certain expressions throughout different properties. For example, when I want to group a report based on the YYYY-MM format, I create the expression within the category’s "Group On" property. Then I might decide to use the same expression for the label of the tick marks of the category in the x axis, so I repeat the expression in the category’s Label property. Sometimes I use the same expression over and over throughout a report, and copy and paste just gets ugly.

Luckily, reports support embedded code that can be entered through the report’s property dialog on the Code tab. For example, formatting the date interval as YYYY-MM can be moved to a method in the Code window, which makes that method available everywhere within the report. To format the date interval, I could create the following function and put it in the Code window:

Function FormatDateInterval(ByVal value As DateTime) As String
    Return Year(value) & "-" & Month(value)
End Function
You might notice this code is written in Visual Basic®. That’s because at this time embedded code only supports Visual Basic.

The function can be accessed throughout the report as a member of the class called Code. The Code class is instantiated by the report once and its methods can be accessed throughout the report. In place of the expression I could use the following line of code to reference the FormatDateInterval method:

=Code.FormatDateInterval(Fields!OrderDate.Value)

One of the biggest advantages to using embedded code is that you can encapsulate methods required by a report in a single place within the report. The embedded code in the Code window is put inside a special XML element in the report’s RDL file. This means that the code is deployed automatically with the report wherever it goes.

If you prefer C# or another .NET language, you’ll have to access a separate .NET assembly. And keep in mind when using embedded code that the methods can be accessed only by the report that contains them. The embedded code window is merely a large textbox and offers no IntelliSense® or built-in debugging features. But for creating quick methods that are intended to be accessed by a single report, the code window is a very simple and effective tool.

If you want multiple reports to use the same code, you should consider accessing a .NET assembly from the embedded code, as I’ll demonstrate in the next section. Security policies prevent the report from performing anything beyond basic operations in embedded code, such as accessing external files, databases, or other resources. Your report can reference .NET assemblies from the References tab in the report properties dialog, but any referenced assembly will, by default, have only Execution permission.

Accessing .NET Code

It makes sense to access .NET code from a report when you have common logic you want multiple reports to share, when your existing .NET code contains business logic your reports can use, and when you want to access a file or another resource from your report.

Writing a custom .NET assembly is more involved than using the embedded code window, but it has some great advantages. You can use any .NET language to develop the custom code and you get all of the features built into Visual Studio®, such as a sophisticated debugger and IntelliSense. It’s also good practice to design your code to be reusable, and by encapsulating methods that will be accessed by your reports you are adhering to that principle. Plus, you can access files or other resources from the custom code and place appropriate security on the assembly under SQL Server 2005 Reporting Services.

My example assumes a business rule that specifies a customer’s level as gold, silver, or bronze based on their total sales over a year. I’ll define this business logic in a custom .NET assembly that will contain the rules that dictate a customer’s status. (All code in this example is also available in the download on the MSDN® Magazine Web site.)

Accessing code in a custom .NET assembly requires several steps. First, I’ll create the .NET class library that contains the business logic. I’ll add post-build events to the .NET project that will copy its assembly to the appropriate folders so the report can access the assembly when it’s executed either in the Report Manager Web interface or in the Visual Studio IDE. Then I’ll reference the custom .NET assembly in a report and create a Visual Basic method in the report’s Code window to invoke the custom assembly’s business logic. Finally, I’ll create an expression in the report’s detail rows that uses the method I created in the Code window (that calls the custom code).

The code in Figure 6 shows the custom .NET class in my MSDN200607 ReportLibrary project. The GetStatusLevel method accepts the total sales for a customer and returns a string value representing the status. I created a post-build event that copies the assembly to the folders that the reports have access to. (Of course the post build event is not necessary if you want to manually copy the files to the locations.)

Figure 6 Custom .NET Class (Close Figure 6)
using System;
using System.Security.Permissions;

namespace MSDN200607_ReportLibrary
{
    public class SalesRules
    {
        private enum _SalesLevel: int
        {
            Gold = 1,
            Silver = 2,
            Bronze = 3
        }

        public SalesRules(){}

        [PermissionSet(SecurityAction.Assert, Unrestricted = true)]
        public static string GetStatusLevel(decimal totalSales)
        {
            if (totalSales >= GetSalesLevelAmount(_SalesLevel.Gold))
            {
                return _SalesLevel.Gold.ToString();
            }
            else if (totalSales >=
                GetSalesLevelAmount(_SalesLevel.Silver))
            {
                return _SalesLevel.Silver.ToString();
            }
            else return _SalesLevel.Bronze.ToString();
        }

        private static decimal GetSalesLevelAmount(_SalesLevel
            salesLevel)
        {
            switch  (salesLevel)
            {
                case _SalesLevel.Gold: return 1000;
                case _SalesLevel.Silver: return 500;
                case _SalesLevel.Bronze:
                default: return 0;
            }
        }
    }
}

The custom .NET assembly must be copied to the C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies folder for the report to be able to access it when previewed in the report designer. (If your installation location of Visual Studio differs, adjust the location appropriately.) Reports running under SQL Server 2005 Reporting Services directly can access assemblies located in the C:\Program Files\Microsoft SQL Server\MSSQL.x\Reporting Services\ReportServer\bin. (Replace MSSQL.x with the appropriate folder where your installation of SQL Server 2005 Reporting Services is located.)

Security Policies

You may also need to modify the security policy to allow the report to access the custom .NET assembly. When reports run in the report designer’s preview tab, the security policy is set to FullTrust by default, so you may not need to modify the policy to allow additional permissions. In case you do, though, the policy configuration file for the report designer is located at C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies\RSPreviewPolicy.config. It’s a very good idea to back up the policy file before making any modifications.

The security policy file for SQL Server 2005 Reporting Services is located at C:\Program Files\Microsoft SQL Server\MSSQL.x\Reporting Services\ReportServer\rssrvpolicy.config. You’ll need to modify it if your custom .NET assembly requires more than Execution permissions. The easiest way is to create a named permission set that has the specific permissions my custom .NET assembly requires. For example, I created the following permission set and put it inside the rssrvpolicy.config file at the end of the other PermissionSet nodes:

<PermissionSet
    class="NamedPermissionSet"
    version="1"
    Unrestricted="true"
    Name="MSDN200607ReportLibraryPermissionSet"
    Description="Allows access to Sample Reporting resources">
<IPermission class="SecurityPermission"
        version="1" Flags="Execution, Assertion" />
</PermissionSet>
Then I added the MSDN200607ReportLibraryCodeGroup shown in Figure 7 to the same file at the end of the nested code groups. It uses the permission set I just created. This way my relaxed permissions as specified in the new permission set are limited to the new assembly.

Figure 7 Custom Code Group (Close Figure 7)
<CodeGroup
        class="UnionCodeGroup"
        version="1"
        PermissionSetName="MSDN200607ReportLibraryPermissionSet"
        Name="MSDN200607ReportLibaryCodeGroup"
        Description="This code group grants permissions to the Sample
                     Reports Library code full trust. ">
        <IMembershipCondition
              class="UrlMembershipCondition"
              version="1"
              Url="C:\Program Files\Microsoft SQL Server\MSSQL.3\
                   Reporting Services\ReportServer\bin\
                   MSDN200607-ReportLibrary.dll" />
</CodeGroup>

Accessing Code via a Report

Once I’ve created the assembly, copied it to the appropriate folders, and modified the security policy, the next step is to reference the assembly in a report. I referenced my assembly by opening the report (CustomerStatusCustomCode.rdl) in the report designer and going to the Layout tab. Then I opened the report properties, clicked the References tab, and browsed to find the assembly that my post-build event copied to the Reporting Services folder (see Figure 8).

Figure 8 Referencing an Assembly in a Report
Figure 8 Referencing an Assembly in a Report

At this point I could reference the custom code via an expression directly from the textbox in the details row. However, I prefer to consolidate all of the calls to custom code in the Code window. This way if the method’s signature (in the custom assembly) changes at some point in the future, I don’t have to search the report to find where I referenced the custom code. To do this I place the following code in the report’s Code window:

Function GetCustomerStatus(ByVal totalSales As Decimal) As String
    Return MSDN200607_ReportLibrary.SalesRules.
        GetStatusLevel(totalSales)
End Function
This code simply acts as a relay between the report and the custom code. It may not be so apparent how useful this is, because my example is so simple. However, when I’ve had several custom methods nested throughout a report it’s been very helpful to be able to look in one place to see what methods the report uses from a custom assembly.

Finally, I added a textbox to the details row of my customer status report and set its Value property to the following expression:

=Code.GetCustomerStatus(Fields!ExtendedPrice.Value)
Here I reference the Code instance, which represents an instance of the class that’s created from the report’s code window. The expression passes the sales total for each customer to my custom method and returns the gold, silver, or bronze status. I also added an interactive sort to the Sales Total header column so I could easily see if the customers were assigned the appropriate sales status.

Wrap Up

Creating charts and report links and embedding code are all simple, yet effective ways to enhance SQL Server 2005 Reporting Services reports. Adding report links to your reports that link to other reports or even to other applications via a URL are great ways to enhance the user experience. Whether your business rules are written in .NET libraries already or you create custom assemblies to contain business logic just for the reports, you are only a few steps from taking advantage of that .NET code.

Send your questions and comments for John to mmdata@microsoft.com.

John Papa is a Senior .NET Consultant with ASPSOFT (aspsoft.com) and a baseball fanatic who spends most of his summer nights rooting for the Yankees with his family and his faithful dog, Kadi. John, a C# MVP, has authored several books on ADO, XML, and SQL Server, and can often be found speaking at industry conferences such as VSLive or blogging at codebetter.com/blogs/john.papa.


Cutting Edge: Context-Sensitive PictureBox Controls

Cutting Edge
Context-Sensitive PictureBox Controls
Dino Esposito

Code download available at: CuttingEdge2006_07.exe (1940KB)
Contents
The Trick Unveiled
Architecting a Richer PictureBox Control
Defining Hot Regions
Adding the Necessary Event Handling
Putting the Pieces Together
Data Binding Support
An ASP.NET ImageMap Control

Great ideas are timeless. A long time ago in Microsoft Systems Journal Paul DiLascia demonstrated a neat trick to display context-sensitive tooltips floating over pictures. As the user moved the mouse over the picture, the tooltip control updated its text to reflect the name of the pointed figure.

How can a tooltip control be smart enough to support a non-rectangular area that, for example, precisely follows the natural shape of a human figure in a painting? Hot spots in images are nothing new, but you normally define them through common, easy-to-describe shapes—rectangles, circles, or perhaps polygons.

Paul solved the issue brilliantly by loading two copies of the image—the original image and a hot spot map. The original photo is displayed in a picture-box component, and the map image is kept hidden and used to map each pixel of the original image to a particular color. The map image is nearly identical to the original except that it fills each hot spot area with a color. This way, each hot spot area corresponds to a unique color and each unique color can be linked to a tooltip text. Before the tooltip pops up, the pixel underneath the mouse is mapped to the corresponding pixel in the hot-spot map. The color on the hidden copy of the image is mapped to the hot-spot list and the related text finally pops up.

In this month’s installment of Cutting Edge, I’ll apply Paul’s trick to the standard Windows® Forms PictureBox control so that it can display context-sensitive tooltips and fire proper events when the user clicks on particular areas. This way, you can easily implement clickable maps using real-world pictures and enlarge or shrink them at your leisure with little effort.

The Trick Unveiled

The photo on the left in Figure 1 is displayed to the user; the one on the right is used under the hood to quickly determine whether any clicked pixel on the displayed image belongs to hot areas, which in this photo are red, lime green, cyan, magenta, and yellow. I have deliberately chosen a real-world picture to illustrate my point. You can use virtually any image you like and still be able to create a map of context-sensitive regions inside of it.

Figure 1 Image User Sees and Underlying Image with Hot Spots
Figure 1 Image User Sees and Underlying Image with Hot Spots

Let’s briefly examine a common scenario. Your user wants to click on a world map to select a given region and then examine sales figures. How would you detect clicking on the various regions? You can partition the map into polygons made by a sequence of points. Once you have a region, a graphic API in the Microsoft® .NET Framework can tell you whether a point belongs to it. This approach is common in ASP.NET applications where the ImageMap control makes implementing it relatively effortless. However, you’ll run into trouble if the final bitmap size changes somehow. As the map size changes, you need to recalculate all the points for all regions. Applying a scale factor may help, but a more flexible solution would be welcome.

There are two key benefits to using a second image. First, you can more carefully control the boundaries of each region. Second, editing regions is trivial—just a matter of coloring proper areas using a simple graphics editor, such as Microsoft Paint (see Figure 2).

Figure 2 Colored Hot Spots Defined on a World Map
Figure 2 Colored Hot Spots Defined on a World Map

Using two images, especially in the context of Windows Forms applications, consumes negligible resources. In ASP.NET, memory consumption due to a double-image load is more critical. Fortunately, solutions can be architected to cache the image only once and share it among multiple requests and users.

The Windows Forms PictureBox control can display images in a variety of formats loaded from a variety of sources, including previously loaded Image objects, remote images at a known URLs, or images residing on disk. In the .NET Framework 2.0, PictureBox has been further enhanced to display a temporary picture while the control loads the main image and an error image when the selected image is not available.

In the .NET Framework 1.x, when the user clicks on the image, the control fires the Click event, but all that does is send notification of the user’s action; it provides no additional information about mouse position and so forth. Event handlers for the event receive a basic EventArgs:

Sub PictureBox1_Click(ByVal sender As Object, _
                      ByVal e As EventArgs) _
    Handles PictureBox1.Click
    ...
End Sub

In Windows Forms 2.0, you can take advantage of the new MouseClick event, which provides you with the position of the mouse at the time of the click:

Sub PictureBox1_MouseClick(ByVal sender As Object, _
                           ByVal e As MouseEventArgs) _
    Handles PictureBox1.MouseClick
    ...
End Sub
Similarly, if you want to display context-sensitive tooltips based on the portion of the image underneath the mouse, you can handle the MouseMove event from within the host form, process the position, and determine what to do.

Architecting a Richer PictureBox Control

The PictureBox control I’m going to implement in this column addresses both points. It accepts a collection of hot regions and fires client events if the user clicks on or moves over any such region. The overall programming interface mimics the programming interface of the ImageMap control in ASP.NET 2.0. The key difference is that the PictureBox control defines hot regions based on colors instead of points. The new PictureBox control derives from the Windows.Forms.PictureBox built-in control and features two essential properties: HotSpots and MapImage (see Figure 3).

Figure 3 Essential Properties of the PictureBox Control (Close Figure 3)
Public Class PictureBox : Inherits Windows.Forms.PictureBox
    Private m_tooltip As ToolTip
    Private m_bmp As Bitmap
    Private m_hotSpots As New HotSpotElementCollection

    Public Sub New()
        InitializeAsSmartPictureBox()
    End Sub

    Public Property MapImage() As Image
        Get
            Return DirectCast(m_bmp, Image)
        End Get
        Set(ByVal value As Image)
            m_bmp = New Bitmap(value)
        End Set
    End Property

    <DesignerSerializationVisibility(
     DesignerSerializationVisibility.Content)> _
    Public ReadOnly Property HotSpots() As HotSpotElementCollection
        Get
            Return m_hotSpots
        End Get
    End Property

    Private Sub InitializeAsSmartPictureBox()
        If Me.SizeMode = PictureBoxSizeMode.Normal Then
            m_tooltip = New ToolTip
            m_tooltipTitle = "You’re on"
            AddHandler Me.MouseClick, AddressOf OnClickInternal
        End If
    End Sub
    ...
End Class

The MapImage property is of type Image and represents the companion image of the displayed picture where hot regions are drawn using a well-known palette of colors. The Image is wrapped by a Bitmap object—a standard GDI+ object—that exposes methods to get the color of a given pixel.

Look at the map in Figure 2. The Image property of the PictureBox will be bound to the topmost image; the new MapImage property will be associated with the bottom image. The two images must meet a few requirements. First, images must have the same size. Second, the companion image must be saved using a non-lossy format such as BMP or perhaps PNG. JPEG images should be avoided because they use color approximation in their compression. If the color of some pixels in a hot region is modified, the whole point detection mechanism breaks. GIF files are fine as long as the colors you use to mark hot regions are within the color palette. Otherwise, like with JPEG images there’s the risk of color approximation. The displayed image can be any format.

The Windows Forms PictureBox control features a SizeMode property that indicates how the image is displayed. Valid values for this property come from the PictureBoxSizeMode enumeration. The default is Normal, meaning that the image is rendered starting from the upper-left corner of the control, and any part of the image that exceeds the PictureBox’s area is clipped. The StretchImage value, on the other hand, causes the displayed image to stretch or shrink to fit the PictureBox’s size. This poses a critical issue given the expected behavior of the extended PictureBox control: the companion image should be resized by the same factor. This can be done programmatically using the GDI+ classes. For example, you can use the GetThumbnailImage method on the Image class or, better yet, the DrawImage method on the Graphics class. To calculate the ratio, you compare the size of the PictureBox to the size of the image. Note that there are good reasons to avoid using GetThumbnailImage in this scenario. It may have very low fidelity relative to the original image, and in rare scenarios it may actually be a completely different image. Though this is very unlikely, it would be quite difficult to diagnose. The sample PictureBox control won’t provide additional features if its size mode is anything but Normal. If this does not meet your needs, you can augment the sample code available for download from the MSDN®Magazine Web site.

To promptly catch any value changes in the SizeMode property, and enable additional capabilities, you need to handle the SizeModeChanged event in the control:

Protected Overrides Sub OnSizeModeChanged(ByVal e As EventArgs)
    MyBase.OnSizeModeChanged(e)
    m_beSmart = (Me.SizeMode = PictureBoxSizeMode.Normal)
End Sub
An internal private member will track the working mode of the PictureBox. This member, the m_beSmart variable in the preceding code snippet, returns false if the SizeMode property is something other than Normal.

Defining Hot Regions

HotSpots is the second key property for the new PictureBox control. It is a collection of custom types, as you can see in Figure 4.

Figure 4 Defining Hot Spot Regions for the PictureBox Control (Close Figure 4)
Imports System.Collections.ObjectModel

Public Class HotSpotEventArgs : Inherits EventArgs
    Public HotSpot As HotSpotElement
    Public CancelTooltip As Boolean
End Class

Public Class HotSpotElement
    Private m_hotSpotColor As Color
    Private m_hotSpotID As Integer
    Private m_description As String
    Private m_title As String

    Public Property HotSpotColor() As Color
        Get
            Return m_hotSpotColor
        End Get
        Set(ByVal value As Color)
            m_hotSpotColor = value
        End Set
    End Property

    Public Property HotSpotID() As Integer
        Get
            Return m_hotSpotID
        End Get
        Set(ByVal value As Integer)
            m_hotSpotID = value
        End Set
    End Property

    Public Property Description() As String
        Get
            Return m_description
        End Get
        Set(ByVal value As String)
            m_description = value
        End Set
    End Property

    Public Property Title() As String
        Get
            Return m_title
        End Get
        Set(ByVal value As String)
            m_title = value
        End Set
    End Property

    Public Overrides Function ToString() As String
        Return String.Format("HotSpot: {0} ({1}-{2}-{3})", _
            HotSpotID, HotSpotColor.R, HotSpotColor.G, HotSpotColor.B)
    End Function
End Class

Public Class HotSpotElementCollection
        Inherits Collection(Of HotSpotElement)
    Public Function ContainsColor(ByVal clr As Color) As Boolean
        For i As Integer = 0 To Me.Count - 1
            Dim elem As HotSpotElement = Me.Item(i)
            If elem.HotSpotColor.ToArgb() = clr.ToArgb() Then Return True
        Next
        Return False
    End Function

    Public Function FindHotSpot(ByVal clr As Color) As HotSpotElement
        For i As Integer = 0 To Me.Count - 1
            Dim elem As HotSpotElement = Me.Item(i)
            If elem.HotSpotColor.ToArgb() = clr.ToArgb() Then Return elem
        Next
        Return Nothing
    End Function

    Protected Overrides Sub InsertItem( _
            ByVal index As Integer, ByVal item As HotSpotElement)
        If Not ContainsColor(item.HotSpotColor) Then
            MyBase.InsertItem(index, item)
        End If
    End Sub

    Protected Overrides Sub SetItem( _
            ByVal index As Integer, ByVal item As HotSpotElement)
        If Not ContainsColor(item.HotSpotColor) Then
            MyBase.SetItem(index, item)
        End If
    End Sub
End Class

HotSpots is of type HotSpotElementCollection, which is a generic type obtained from the Collection (Of T) type composed with the HotSpotElement type. The HotSpotElement class defines a region of color in a map image that is meaningful. In this context, a hot region is identified with a unique numeric ID, an RGB color, a title, and a description. The color is used to uniquely identify the region. If you’re interested in capturing user clicks, then the ID provides a numeric value to test on the client and determine what to do. It is likely that the color information would be enough to uniquely identify the clicked area; from this perspective, the ID information is redundant. However, it might be more useful than colors in rich data-binding scenarios where you want to associate regions on a map with the ID they have in the back-end database. In this way, the color information is not coupled with region information.

Title and description serve in another role as well—they can be used for context-sensitive tooltips that pop up as the user moves the mouse over the displayed picture.

You can populate the HotSpots property programmatically or declaratively from within the Visual Studio® 2005 designer. It’s nice that Visual Studio 2005 automatically recognizes collection properties and binds them to the built-in collection editor (see Figure 5).

Figure 5 Editing the Hotspots Collection Property
Figure 5 Editing the Hotspots Collection Property

The default collection editor adds collection members and displays them with any text that results from the member’s ToString method. In the rightmost grid, you see all the properties of the member type. Each property is editable as in the Visual Studio 2005 parent property grid.

If you’re familiar with ASP.NET control development, you know that changes entered through the designer are not persisted to the designer.vb or designer.cs codebehind file. If you don’t believe me, just try it. Drop the new PictureBox control on a form and populate the HotSpots collection. When you’re finished, save and start the form. The PictureBox control behaves as if the collection were empty, and the designer.vb file or the form has no hot spot elements saved. Why is that?

The answer to the problem is that you need to claim a particular designer serialization policy for the HotSpots collection property. In general, you need to do this for any collection properties in both Windows Forms and ASP.NET custom controls. The following attribute on the collection property does the trick:

<DesignerSerializationVisibility( _
    DesignerSerializationVisibility.Content)>
Without this attribute no changes made to the collection at design time are ever persisted to the designer.vb file in Windows Forms or the ASPX source page in ASP.NET.

The HotSpotElementCollection type inherits from the generic Collection type and extends it with a couple of finder methods—ContainsColor and FindHotSpot. Both take a color as their input and return a Boolean value or a HotSpotElement object respectively, based on their findings. (Look back at Figure 4.) FindHotSpot, in particular, returns the HotSpotElement that matches the specified color, if any.

In addition, the HotSpotElementCollection class overrides a couple of methods—InsertItem and SetItem—to make sure that the color doesn’t conflict with something already in the collection.

Armed with full support for image hot regions, let’s proceed to implement a couple of events for the host form. I’ll define two events—HotSpotFound and HotSpotClicked. HotSpotFound fires when the mouse is moving over a hot spot; HotSpotClicked fires when the user clicks on a hot spot.

Adding the Necessary Event Handling

The HotSpotFound event is declared using the new generic version of the EventHandler type, as follows:

Public Event HotSpotFound As EventHandler(Of HotSpotEventArgs)
You saw the HotSpotEventArgs data structure back in Figure 4. It basically extends the base EventArgs class with a couple of properties—HotSpot and CancelTooltip. The HotSpot property references the hot spot area that has been found or clicked. The CancelTooltip property indicates whether, in case of mouse movements, the corresponding tooltip should be canceled. This property gives client code the final word on the tooltip displayed for the hot spot. By handling the event, the client form can programmatically change or even cancel the tooltip.

The HotSpotClicked event follows an identical schema.

Public Event HotSpotClicked As EventHandler(Of HotSpotEventArgs)
In this case, though, the CancelTooltip property is not used by the PictureBox control once the event handler returns. Whatever value you assign to the property in the event handler is blissfully ignored by the PictureBox control.

The PictureBox control registers its own handler for the MouseClick event upon instantiation, as you saw in Figure 3. The internal handler, OnClickInternal, is shown in Figure 6.

Figure 6 Firing the HotSpotClicked Event (Close Figure 6)
Protected Overridable Sub OnClickInternal( _
            ByVal sender As Object, ByVal e As MouseEventArgs)
    PrepareAndRaiseClickEvent(e.X, e.Y)
End Sub

Private Sub PrepareAndRaiseClickEvent( _
        ByVal x As Integer, ByVal y As Integer)
    Dim elem As HotSpotElement = GetUnderlyingHotSpot(x, y)
    If elem Is Nothing Then
        HideTooltip()
        Return
    End If

    ‘ Raise an event to the host form with the underlying color
    Dim args As New HotSpotEventArgs
    args.HotSpot = elem
    RaiseEvent HotSpotClicked(Me, args)
End Sub

Private Function GetUnderlyingHotSpot( _
        ByVal x As Integer, ByVal y As Integer) As HotSpotElement
    If Not m_beSmart Then Return Nothing

    ‘ Ensure the point is inside the bitmap
    If x >= m_bmp.Width Or y >= m_bmp.Height Then Return Nothing

    ‘ Get the underlying color and check it against HotSpot collection
    Return HotSpots.FindHotSpot(m_bmp.GetPixel(x, y))
End Function

Private Sub HideTooltip()
    m_tooltip.SetToolTip(Me, String.Empty)
End Sub

When a point in the client area of the PictureBox control is clicked, the control receives the client coordinates of the point from the event data structure—the MouseEventArgs class.

The next step entails finding the color of the pixel at the specified position. Note that the internal MouseClick event fires even when the user clicks outside the image’s boundaries. You need to catch these clicks and return without calling GetPixel on the Bitmap. If you call GetPixel with coordinates outside the image’s size, an exception is thrown. GetPixel is a method on the Bitmap class that returns the color of the pixel at the given location. You then take this color and look up for a region associated with that color. If any is found, the corresponding HotSpotElement object is returned:

Dim elem As HotSpotElement = HotSpots.FindHotSpot(clr)
You use the FindHotSpot method on the hot spot collection to retrieve information about the found region.

But, what if your image naturally contains pixels with the same color as one of your hot spot colors? If the user were to move the mouse exactly over that pixel, a conflict would occur. To avoid that, for hot regions you should ideally choose colors that are not used elsewhere in the image. With over 16 million colors available, finding an unused color is possible, but not easy. However, because of the way colored pixels are distributed in a real image, your color will probably be found in one pixel here and there and may not become a problem.

A more elegant solution would consist in reserving a color (white or transparent perhaps) for all parts of the image that do not have a region defined. In this way, you color uniformly all the underlying image except hot regions. The reserved color might be exposed as a public property to let developers choose it at will.

Putting the Pieces Together

The PictureBox control allows you to define hot regions inside displayed images. Each region is painted on a copy of the image—the map image—using a distinct color, possibly a color not used in the rest of the image. After you drop the PictureBox control onto a form, you set the map image and populate the hot spot collection. The hot spot collection informs the control about "hot" colors in the image. By default, when the user clicks on a hot region, the HotSpotClicked event is fired. Here’s a typical event handler:

Sub PictureBox1_HotSpotClicked(ByVal sender As Object, _
                               ByVal e As HotSpotEventArgs) _
        Handles PictureBox1.HotSpotClicked
    MessageBox.Show(e.HotSpot.Description)
End Sub

If the PictureBox is enabled to capture mouse movements, it registers an internal handler for the MouseMove event. After the event is fired, the control will present a tooltip. Title and text of the tooltip usually reflect the values set in the hot spot element. However, these parameters can be changed in the client handler. As mentioned, the tooltip can also be canceled:

RaiseEvent HotSpotFound(Me, args)
If Not args.CancelTooltip Then
    m_tooltip.ToolTipTitle = args.HotSpot.Title
    m_tooltip.SetToolTip(Me, args.HotSpot.Description)
Else
    HideTooltip()
End If

Figure 7 shows the control in action and the standard tooltip it displays. The title of the tooltip is changed programmatically and the description of the hot region is also displayed on the form’s status strip:

Sub PictureBox1_HotSpotFound(ByVal sender As Object, _
                             ByVal e As HotSpotEventArgs) _
        Handles PictureBox1.HotSpotFound
    Info.Text = e.HotSpot.Description
    e.HotSpot.Title = "EMEA"
End Sub

Figure 7 Context-Sensitive Tooltips Displayed on a Map
Figure 7 Context-Sensitive Tooltips Displayed on a Map

Data Binding Support

The built-in PictureBox control features a bindable property called Image. The addition of the HotSpots collection raises the need of a stronger form of data binding. For example, it would be great if the collection could be populated from a database, thus saving developers from editing source files if a color or a description changed at some time. You might want to add a DataSource property and map its contents to the HotSpots collection or perhaps transform the HotSpots collection into a read/write property that can be set programmatically as an object. If you opt for the classic DataSource property, you also need to define a few mapper properties to indicate which fields on the bound data source map to bindable properties of the hot spot elements. For example, you might want to have DataDescriptionField, DataTitleField, DataColorField, DataValueField properties to map fields on the data source to hot spot elements. For DataColorField, you are responsible for inventing a translation algorithm that generates a .NET Framework color type based on a string or a number stored in the bound data source. This could be done easily using the ColorConverter class in the System.Drawing namespace.

The PictureBox control’s bindable Image property is marked with the Bindable attribute and added to the DataBindings collection. You might want to extend this behavior to the MapImage property, too. To make this happen, you can add the Bindable attribute to the MapImage property in the source code of the control:

<Bindable(True)> _
Public Property MapImage() As Image
  ...
End Property

An ASP.NET ImageMap Control

I discussed context-sensitive picture box controls in the context of a Windows Forms application; however, it’s easy to build a similar ASP.NET control starting from either the Image or ImageMap control.

In ASP.NET, the ImageMap control provides behavior similar to what I’ve described throughout this column and defines a number of HotSpot regions. When the user clicks a hot spot, the control either posts back or navigates to a specified URL. The ASP.NET framework comes with a number of predefined HotSpot objects including the CircleHotSpot, RectangleHotSpot, and PolygonHotSpot classes. Also, you can derive from the abstract HotSpot class to define your own custom hot spot object, which you may find handy in situations like the one I’ve described here.

Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is a mentor at Solid Quality Learning and the author of Programming Microsoft ASP.NET 2.0 (Microsoft Press, 2005). Based in Italy, Dino is a frequent speaker at industry events worldwide. Get in touch with Dino at cutting@microsoft.com or join the blog at weblogs.asp.net/despos.


Inside MSDN: Building the MSDN Aggregation System

Inside MSDN
Building the MSDN Aggregation System
John Mollman

Contents
Building the System
Architecture
Message Structure and Validation
Job Worker Infrastructure
Service Broker Plumbing
JobWorker Interfaces
JobWorker Factory
JobWorker Execution
Data Publishers and Subscribers
Associating Metadata with MSDN Content

Have you visited msdn2.microsoft.com? It’s the new online face of the MSDN® Developer Tools and Enterprise Server documentation. The infrastructure behind it includes a system developed by my team at Microsoft for aggregating information related to our content. By the time you read this, we expect that all of msdn.microsoft.com will have migrated to our new system and the msdn2 name will have been retired.

When my team set out to build our content aggregation system, we worked toward several high-level goals. The following explains what we wanted to accomplish:

  • Keep data providers and consumers loosely coupled using a publish/subscribe model and asynchronous messaging interfaces.
  • Let individual usage scenarios define and manage the structure and meaning of their data. We also wanted to keep those definitions opaque to the system and to unrelated scenarios.
  • Make it easy to plug new data providers and consumers into the system regardless of how complex their own internal logic may be.

These constraints were necessary because the universe of content and data we wanted to associate with MSDN content is widely varied and we couldn’t enforce structure or taxonomy upon it. For instance, we wanted to be able to recommend blog posts that refer to our documentation and allow users to tag documents to build up a "folksonomy" that describes MSDN content.

Building the System

After agreeing on our design goals we began looking for technologies to support them. It turned out that SQL Server™ Service Broker offered the asynchronous messaging support we needed and, since the message-queuing infrastructure is tightly integrated with the SQL Server database engine, our existing database backup, administration, and failover procedures could cover our messaging solution as well.

Fortunately, SQL Server Service Broker also supports significant scale out. For example, service endpoints (the senders and receivers of messages) can be remote or local and the database code does not need to know where endpoints are. This let us start out with all of our services on a single server, and then move them to remote servers as loads increased.

The common language runtime (CLR) integration in SQL Server 2005 allowed us to build a plug-in model for data providers and consumers and to implement complex business logic in the database layer. If you’ve written extended stored procedures to do things like pull data from a SOAP endpoint, you’ll appreciate how much more straightforward it is to implement this functionality using SQL Server CLR integration.

Architecture

Our system’s high-level architecture is shown in Figure 1.

Figure 1 System Architecture Overview
Figure 1 System Architecture Overview

The public interfaces to our system are message-based, and since the system is built on top of Service Broker, we were able to provide all the benefits of reliable asynchronous messaging without a lot of additional work. We expose two Service Broker services (which are the logical endpoints for Service Broker messaging) as data input points for the system.

Briefly, services are communication endpoints that use queues as their backing stores. Services specify what types of messages they can process by bundling them into contracts. The glossary on Service Broker Objects explains further. The following are the two services that we expose:

The Job Service receives messages that provide the system with the information it needs to execute a Job. For us, a Job is a unit of work that brings data related to one or more MSDN documents into the system. This is how we support the pull model of data acquisition. A job message contains the information a JobWorker needs to operate, which will differ for various JobWorkers. The data retrieved by the JobWorker is then submitted back into the system via a second Service Broker service called the DataSubmission service.

Service Broker Messaging Objects Glossary

These database objects are used in the basic design of an application that uses Service Broker.

  • Message types define the names of the messages exchanged between applications and optionally provide validation for the messages.
  • Contracts specify the direction and type of messages in a given conversation.
  • Queues store messages. This storage mechanism allows for asynchronous communication between services. Service Broker queues provide additional benefits, such as automatically locking messages in the same conversation group.
  • Services are addressable endpoints for conversations. Service Broker messages are sent from one service to another service. A service specifies a queue to hold messages, and the contracts for which the service can be the target. A contract provides a service with a well-defined set of message types.

The DataSubmission Service receives messages containing the data to be associated with MSDN content. DataSubmission messages can be submitted either by JobWorkers or by external data providers using a "push" model. In the latter case, data providers need to know the structure of the message envelope required by the DataSubmission service.

Message Structure and Validation

For either service, certain pieces of information used by the system to route messages and identify data sources must be included in all messages. So the message envelope needs to have a structure verifiable by the system, and the message payload needs to be structured according to a contract agreed upon by the provider and consumer while remaining opaque to the system. An example Job Message is shown in Figure 2.

Figure 2 Sample Job Message (Close Figure 2)
<?xml version="1.0" encoding="utf-16"?>
<env:job env:submittor="Test"
         env:submitted="2006-2-28 8:25:33.279"
         env:id="TestSendID"
         env:source="test_harness_sproc"
         env:sourceDomain="test_domain"
         xmlns:env="urn:msdn/aggregator/JobRequest">

  <!-- optional runtime overrides -->
  <env:providertype>MtpsMetadataJobProcessor</env:providertype>
  <env:initialization>
      payloadSchema=urn:mtpg-com:mtps/2004/1/key
  </env:initialization>

  <!-- optional scheduled job parameters -->
  <env:nextrun>2006-2-28 8:55:00.000</env:nextrun>
  <env:intervalminutes>30</env:intervalminutes>
  <env:lastrun>2006-2-28 8:24:33.279</env:lastrun>

  <!-- message payload -->
  <env:jobData>
    <k:key xmlns:k="urn:mtpg-com:mtps/2004/1/key">
      <k:contentId>ms164310</k:contentId>
      <k:contentGuid>41d332ea-9be2-4e6d-94f4-c4e85daf4272</k:contentGuid>
      <k:contentAlias>
          0a71b862-79b6-4480-a2a7-19574e2881de
      </k:contentAlias>
      <k:locale>en-us</k:locale>
      <k:version>SQL.100</k:version>

      <!-- reams of additional data omitted -->
    </k:key>
  </env:jobData>
</env:job>

We use XML for our messages. SQL Server Service Broker provides internal support for XML Schema Definitions (XSDs) to validate messages, so we adopted an XSD-based open content model to provide this multi-layer message validation. Using an open content model lets different consumers of XML data validate parts of an XML document while ignoring parts that aren’t relevant to their domain. Our system can validate the envelope data it needs without knowing the structure of the message payload, while data consumers can use domain-specific XSDs to validate the payload of messages they consume. This enables providers and consumers to agree on a schema-based contract defining their communications without the system (or other consumers) needing knowledge of that contract. The contract defining the structure of Job messages is described by the XML schema in Figure 3.

Figure 3 Job.xsd (Close Figure 3)
<?xml version="1.0" encoding="utf-16"?>
<xs:schema
    attributeFormDefault="qualified"
    elementFormDefault="qualified"
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:env="urn:msdn/aggregator/JobRequest"
    targetNamespace="urn:msdn/aggregator/JobRequest">

  <xs:element name="job">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="providertype" type="xs:string"
                    minOccurs="0" maxOccurs="1"/>
        <xs:element name="initialization" type="xs:string"
                    minOccurs="0" maxOccurs="1" />
        <xs:element name="nextrun" type="xs:string"
                    minOccurs="0" maxOccurs="1"/>
        <xs:element name="intervalminutes" type="xs:int"
                    minOccurs="0" maxOccurs="1"/>
        <xs:element name="lastrun" type="xs:string"
                    minOccurs="0" maxOccurs="1"/>
        <xs:element name="jobData" minOccurs="0" maxOccurs="1">
          <xs:complexType>
            <xs:sequence>
              <xs:any namespace="##any" minOccurs="0"
                  maxOccurs="unbounded" processContents="skip" />
            </xs:sequence>
          </xs:complexType>
        </xs:element>
      </xs:sequence>
      <xs:attribute name="submittor" type="xs:string" use="required" />
      <xs:attribute name="submitted" type="xs:string" use="required" />
      <xs:attribute name="id" type="xs:string" use="optional" />
      <xs:attribute name="source" type="xs:string" use="required" />
      <xs:attribute name="sourceDomain" type="xs:string" use="optional" />
    </xs:complexType>
  </xs:element>
</xs:schema>

In the Job.xsd schema we’ve defined an optional jobData element that can contain children from any namespace (xs:any namespace="##any"). An XML document can contain any valid XML in the jobData element and still be validated by this schema. The system uses this schema to enforce the structure of the envelope data it needs while ignoring the message payload. Individual JobWorkers, who share knowledge of the structure of their message payload with Job requesters, can use another XSD schema that the two parties share to validate the message payload. Note that when defining an open content model schema outside of SQL Server 2005, the common pattern is to define the xs:any element with a processContents="lax" attribute which indicates that the parser should make a best-effort attempt at validation, but not fail if unable to validate. SQL Server 2005 does not support "lax" schema validation, so we specify processContents="skip" instead. Once the schema has been defined we make it available to Service Broker and associate it with message types, as shown in Figure 4.

Figure 4 Defining Message Type with Schema Validation (Close Figure 4)
-- define schema collection
CREATE XML SCHEMA COLLECTION JobRequestMessageSchema AS
  N’&lt;xs:schema
    attributeFormDefault=&quot;qualified&quot;
    elementFormDefault=&quot;qualified&quot;
    xmlns:xs=&quot;http://www.w3.org/2001/XMLSchema&quot;
    xmlns:env=&quot;urn:msdn/aggregator/JobRequest&quot;
    targetNamespace=&quot;urn:msdn/aggregator/JobRequest&quot;&gt;

    &lt;!-- schema definition as shown in Figure 3 --&gt;
  &lt;/xs:element&gt;
&lt;/xs:schema&gt;’ ;

-- define message type and specify validation using our schema
CREATE MESSAGE TYPE [urn:msdn/aggregator/JobRequestMessage] VALIDATION = VALID_XML WITH SCHEMA COLLECTION JobRequestMessageSchema;

By creating the message type using WITH SCHEMA COLLECTION validation, we’re telling Service Broker to validate all messages of the specified type against our XML schema. If a message is received that does not conform to the schema, Service Broker will discard the message and return an error message to the Service that sent the message. This allows downstream code to consume messages without needing to validate them again, as Service Broker guarantees that all messages of this type on the queue conform to our schema.

In our job schema, root attribute values contain the identification information we carry throughout the system to uniquely identify data items. We use the combination of source, sourceDomain, and id values as our identifiers to mitigate namespace collisions across multiple providers. This protects against cases in which two different providers use the same identifier—an important safeguard since we can’t control how providers generate IDs. Envelope elements are generally used by scheduled jobs to pass and maintain information about when they should run, last-run status, and the like.

Job Worker Infrastructure

The MSDN Aggregation system supports a pull model of data acquisition through our JobWorker framework. A JobWorker is responsible for retrieving data related to MSDN content and adding it to the system via the data-submission messaging infrastructure. The design of the framework is intended to make it relatively easy to add new data providers to the system to support the rapid implementation of new data input scenarios. We needed to address both ease of implementation and ease of deployment and configuration in our design.

CLR integration enabled us to implement the JobWorker framework in C# using powerful object-oriented patterns while exposing its functionality to T-SQL and Service Broker code.

Before we look at how JobWorkers process messages, however, we need to get Service Broker set up to deliver those messages.

Service Broker Plumbing

We saw earlier (in Figure 4) how to define a Service Broker message type and associate an XML schema with it. Let’s take a look at the rest of the Service Broker setup needed to define our Job messaging infrastructure.We’ve already defined our JobRequest message. Now we’ll bundle this message into a Service Broker contract that specifies its directionality:

-- create Job contract
CREATE CONTRACT [urn:msdn/aggregator/JobRequests]
(
  [urn:msdn/aggregator/JobRequestMessage] SENT BY INITIATOR
);

This contract specifies that JobRequestMessages can be sent by the service delivering messages (the Initiator of the conversation).

Now we create the queues that will store these messages, as shown in Figure 5. We use unidirectional queues in our system, which means that a service receives messages from one queue and replies to another. This simplifies processing, as service activation code doesn’t need to filter out messages it doesn’t care about. Defining separate queues also enables scale-out, as it allows you to host the queues either locally or remotely without needing to rewrite your application code. Service Broker doesn’t restrict you from either sending or receiving on the same queue, however.

Figure 5 Creating the Queues (Close Figure 5)
-- JobPosting queue: receives JobRequests
CREATE QUEUE [urn:msdn/aggregator/JobPostingQueue]
  WITH STATUS = ON, RETENTION = OFF, ACTIVATION
(
  PROCEDURE_NAME = [ProcessJobRequests], -- activation sproc
  MAX_QUEUE_READERS = 1,
  EXECUTE AS OWNER
) ;

-- JobResponse queue: for response messages
CREATE QUEUE [dbo].[urn:msdn/aggregator/JobResponseQueue]
  WITH STATUS = ON , RETENTION = OFF, ACTIVATION
(
  PROCEDURE_NAME = [ProcessJobRequestResponses], -- activation sproc
  MAX_QUEUE_READERS = 1,
  EXECUTE AS OWNER
) ;

Now that we have our contract and queues in place, we can define the services that use them:

-- Service for handling (receiving) JobRequest messages
CREATE SERVICE [urn:msdn/aggregator/JobService]
AUTHORIZATION [dbo]
ON QUEUE [urn:msdn/aggregator/JobPostingQueue]
(
  [urn:msdn/aggregator/JobRequests]
) ;

-- Service for posting JobRequest messages (used by clients)
CREATE SERVICE [urn:msdn/aggregator/JobPostingService]
AUTHORIZATION [dbo]
ON QUEUE [urn:msdn/aggregator/JobPostingQueue];

The JobPostingService isn’t bound to a contract, as it only initiates Dialog Conversations. Only the receiving end of a conversation (the target) needs to be bound to a contract when defining the service, but the contract does need to be shared by the services on both ends of a conversation. The service on the sending end of a conversation uses the contract at run time when sending messages to the target. Here’s an example:

-- Begin dialog with JobService. Sender needs to know about
-- shared contract when sending.
DECLARE @dialog_handle UNIQUEIDENTIFIER;
BEGIN DIALOG CONVERSATION @dialog_handle
  FROM SERVICE [urn:msdn/aggregator/JobPostingService]
  TO SERVICE ‘urn:msdn/aggregator/JobService’
    ON CONTRACT [urn:msdn/aggregator/Jobs] ;

-- Send message on dialog conversation
SEND ON CONVERSATION @dialog_handle
    MESSAGE TYPE [urn:msdn/aggregator/JobRequestMessage]
    (@message) ;

So now we’ve got a pair of endpoints that can send and receive messages (the services). We’ve defined the messages we can understand and bound them to the services using a contract, and we’ve defined the backing store used to manage messages. That puts the Service Broker plumbing for our Job processing in place. Now let’s look at how we actually process the messages that come through this plumbing.

JobWorker Interfaces

With the Service Broker messaging infrastructure set up, we defined a set of CLR interfaces (shown in Figure 6), which represent the contract JobWorkers must implement to participate.

Figure 6 JobWorker Interfaces
Figure 6 JobWorker Interfaces

JobWorkers implement the IJobWorker interface, which defines a single Execute method and several properties for maintaining Job state data. Execute takes an IJobContext parameter, which is an interface defining the Submit method used to publish data back into the system. We provide a default implementation of IJobContext that implements basic submission functionality, and JobWorkers are free to provide their own implementation if the default doesn’t meet their needs. By exposing IJobContext as a parameter to IJobWorker.Execute, we’re not only enforcing a behavioral contract but also guaranteeing that submission functionality is always available to implementers.

Figure 7 shows an example of a simple implementation of IJobWorker.Execute, which uses XSLT to transform input XML data into the structure required by the DataSubmission service. In this case the message contains the data itself and the JobWorker only needs to transform it before submitting. In a more common scenario, where the JobWorker needs to acquire data from outside the system, the message parameter would contain information the JobWorker needs to retrieve the data (such as the URI of an RSS feed). In any case, the pattern is that the message parameter is the raw Job message received on the JobPosting queue. Our use of XSDs defining an Open Content Model means that the message data can be validated by JobWorkers as long as they and the submitter of the Job message have shared an XSD defining the message payload.

Figure 7 Simple Execute Implementation (Close Figure 7)
// Transform input data into DataSubmission structure using embedded
// resource XSLT stylesheet
public void Execute(IJobContext host, SqlXml message)
{
  XPathDocument doc = new XPathDocument(message.CreateReader());
  string xsltOut = string.Empty;
  using (MemoryStream buf = new MemoryStream())
  {
    // load XSLT from resource
    XslCompiledTransform xslt = GetFixupTransform();
    xslt.Transform(doc, null, buf);

    buf.Position = 0L;
    xsltOut = new StreamReader(buf).ReadToEnd();
  }

  // package data in Submission envelope
  XmlDocument outMsg = Utility.EnvelopSubmissionMessage(xsltOut, host);

  // Submit data message via IJobContext callback
  host.Submit(outMsg);
}

Once a JobWorker has been coded and compiled, it needs to be cataloged to SQL Server so that it is available to database application code. Cataloging an assembly stores the assembly’s Microsoft intermediate language (MSIL) code in a SQL Server table as varbinary data, and this is what the SQL Server CLR host uses when loading our assembly. An example of the syntax for cataloging an assembly is shown here:

-- catalog assembly from file location
CREATE ASSEMBLY [VeryUsefulJobWorker]
    FROM ‘d:\\AggregationBinaries\\VeryUsefulJobWorker.dll’
    WITH PERMISSION_SET = EXTERNAL_ACCESS;

For a more detailed description of this statement and its syntax, see msdn2.microsoft.com/library/ms189524.aspx

When you execute this statement, the SQL Server CLR host loads the assembly specified in the FROM clause, checks that the assembly contains valid MSIL, and loads any dependent assemblies referenced by the assembly being cataloged. If the PERMISSION_SET specified is SAFE or EXTERNAL_ACCESS, additional type safety verification is performed, and classes in the assemblies are checked for constructs such as writable static members and finalizer methods (neither of which are allowed in assemblies using these permission sets).

After an assembly has been cataloged, procedures, functions, and the like, which are to be exposed directly to database application code need to be cataloged and the appropriate CREATE statements, such as CREATE PROCEDURE, must be used. Since our JobWorkers don’t expose functionality to the database application code, we skip this step. If you’re wondering how useful JobWorkers really are if we don’t expose their functionality to database code, you’ll soon see.

JobWorker Factory

Now that we’ve got a handle on the interface contracts supporting the JobWorker infrastructure, let’s take a look at how we determine which JobWorker should handle a given Job message.

We saw earlier that Service Broker messages contain type information that is used to identify messages and can be used to differentiate messages from various sources or containing different information. In our system, Job messages can be sent by different services, but they’re all received on the same message queue. The MSDN Aggregation System uses Message Type data to determine which JobWorker implementation should handle a given Job message. The relevant T-SQL code for receiving and processing Job messages is shown in Figure 8.

Figure 8 Receiving Job Messages (Close Figure 8)
DECLARE @conversationHandle AS UNIQUEIDENTIFIER;
DECLARE @messageTypeName AS SYSNAME;
DECLARE @messageBody AS XML;

-- always RECEIVE in a transaction and COMMIT on success,
-- or ROLLBACK if unable to process the message
BEGIN TRANSACTION ;

-- get next message and its type from queue
RECEIVE TOP (1)
  @conversationHandle = conversation_handle,
  @messageTypeName = message_type_name,
  @messageBody = CAST (message_body AS XML)
FROM
  [urn:msdn/aggregator/JobPostingQueue]

. . .

-- Process job message
EXECUTE [ProcessJobRequest] @messageBody, @messageTypeName;
END CONVERSATION @conversationHandle;

-- COMMIT when message successfully processed
COMMIT TRANSACTION ;

We receive one message using the RECEIVE TOP (1) statement, and give it a column specifier indicating which columns should be included in the resultset that is returned. Service Broker is a reliable transactional messaging platform, and to be sure your application code extends the reliability guarantees Service Broker makes you should always enclose your RECEIVE and message processing logic in a SQL transaction. If you are unable to process a message, rolling back the transaction will put it back on the queue where it will be processed again if you have activation enabled. It is important to exercise caution about rolling back transactions, however, as once a message has failed to be processed and the processing transaction rolled back five times Service Broker will consider it a "poison message" and disable the queue. Rolling back the transaction is appropriate for transient failures, but if your application logic has no hope of ever successfully processing a message, it’s better to use another means to inform the sender. Our system commits the transaction as soon as it has received the message, and if we fail to process the message, sends an error message to the conversation initiator using END CONVERSATION WITH ERROR.

Our message-processing logic is in an activated stored procedure, which means it is called by the Service Broker infrastructure whenever a message is available on the queue. This lets us process messages one at a time and know that whenever it’s called there will be a message to process. An alternate model is to use WAITFOR (RECEIVE ...) TIMEOUT to periodically check for messages and wait if none are ready on the queue. There are some startup costs associated with activation of a stored procedure, so if your application receives multiple messages within a 2- to 3-second window, you’ll improve performance using WAITFOR.

Once we have the message and its Type information, we pass that to the ProcessJobRequest stored procedure. This is where SQL CLR integration really shines. It enables us to merge the power of Service Broker’s asynchronous messaging with a plug-in model built using a Factory Method software pattern and the IJobWorker interfaces we looked at earlier.

ProcessJobRequest is implemented as a SQL Server CLR stored procedure, meaning it is managed code hosted by the SQL Server runtime and can be called from T-SQL code as a regular stored procedure. You tell the SQL Server CLR runtime that your method should be exposed as a SQL Server CLR stored procedure by decorating it with a Microsoft.SqlServer.Server.SqlProcedure attribute. The method must be static and return either void or an integer Type. (See msdn2.microsoft.com/ms131094.aspx for details on coding SQL Server CLR stored procedures.) The following code shows the implementation of this procedure (with much important error handling code omitted).

[SqlProcedure( Name="ProcessJobRequest")]
public static void ProcessJobRequest(
    SqlXml jobData, string messageType)
{
    // get JobContext data from message envelope
    IJobContext ctxt = Utility.GetJobContextFromRequest(jobData);

    // use factory method to instantiate correct Job Worker
    IJobWorker processor = JobWorkerFactory.CreateWorker(messageType);
    processor.Execute(ctxt, jobData);
}

We first build up the job’s context data from the Job message envelope. (We don’t need to validate the structure of the envelope data here; Service Broker has already taken care of that by applying the XSD we specified when we created the message Type.) We then use our factory method to instantiate the correct JobWorker based on the type of message. We maintain a table in the database to map message type names to fully qualified CLR type names and use .NET reflection to instantiate the appropriate type. Adding a new JobWorker type requires cataloging the assembly into SQL Server and adding an entry in the mapping table specifying which assembly should process a given message type.

Once an appropriate JobWorker has been instantiated to process the message, its Execute method is called and it performs whatever work it needs to do.

JobWorker Execution

The choice of actions the JobWorker performs in its Execute method belongs to the JobWorker. There are some restrictions placed upon managed code by the SQL Server runtime, depending on which permission set the assembly containing the code uses. SAFE assemblies can’t access any resources outside of the database. EXTERNAL_ACCESS assemblies can make socket calls and access the file system and other system resources, but they are limited in their ability to use certain CLR functionality like threading libraries, synchronization primitives, and writable static members. UNSAFE assemblies are free to do just about anything they please (including executing unmanaged code and bypassing type safety validation). It is quite possible to shoot yourself in the foot using UNSAFE assemblies, and their use is generally discouraged. See msdn2.microsoft.com/ms189566.aspx for detailed information about SQL Server CLR Permission Sets. Most of our JobWorkers require access either to network resources such as sockets for making HTTP requests or to local or remote file systems, so they run using the EXTERNAL_ACCESS Permission Set.

Once a JobWorker has performed whatever it needs to retrieve and package its data, it submits the data back to the system by sending a message to the Data Submission service. The IJobContext implementation passed to the JobWorker’s Execute method contains a Submit method that is used for this purpose. This model lets the JobWorker remain unaware of the details of getting its data back into the system. By using the same service for receiving data from push providers and pull providers, we can centralize the distribution point for the publisher/subscriber implementation we use to route messages to data consumers. This makes it easy for data consumers to focus on the type of data they want to get and not on how that data is received.

Data Publishers and Subscribers

One of the core tenets of the MSDN Aggregation System is that information flowing into the system will not be of interest to all the consumers. To support this, our DataSubmission service uses a publisher/subscriber model, which provides a loose coupling between data providers and consumers. Data providers publish data into the system and data consumers "subscribe" to message types corresponding to the data they want to receive. As with Job messages, it is the responsibility of data providers and consumers to agree on the structure and semantics of the data they share via this mechanism. We use the same XSD-based open content model to allow the system and different data consumers to validate slices of message data according to shared schemas. Data consumers subscribe to message types, and we implemented a simple routing service that receives messages from the DataSubmission queue, checks their type, and forwards them to each consumer registered for that message type. Registration is managed using a database table that maps message types to subscribers. In our current release, subscribing to a message type requires manual configuration, but we’re investigating a more message-based subscription model that will support ad hoc subscriptions for a future release.

As with messages on the Job queue, Service Broker validates the message envelope using an open content model XSD, while the message payload is opaque to the system. It is the responsibility of data publishers and subscribers to agree upon the structure and meaning of the data they share, and manage the sharing of domain-specific XML schemas if they want to validate message payloads. The implementation logic of the pub/sub router service is similar enough to that of the JobWorker Factory Service that we can omit code examples in the interest of space.

Once subscribers receive data from the pub/sub router, how they process, structure, and store that data is entirely up to them. The Aggregation System doesn’t provide or prescribe this functionality as a matter of design. Just as we can’t predict all the different types of data coming into the system, we can’t predict all the ways in which consumers might want to manage that data. By leaving it open and providing an infrastructure that supports a wide variety of data producers and consumers, our hope is that we’ll see partners coming up with interesting scenarios we haven’t thought of.

Associating Metadata with MSDN Content

For data consumers that are tightly coupled with the MSDN Publishing System, the data they acquire through the Aggregation System is associated with MSDN content using SQL Server replication. The Publishing System provides a mechanism for associating arbitrary chunks of data with a given Content Item, and we piggyback on that functionality. Data consumers that use this mechanism format their data according to an agreed upon XML schema and insert it into a table for replication. The Publishing System then picks up the data and stores it separately from the Content Item itself, while making it easy for the Rendering Framework, which is the front edge of msdn2.microsoft.com, to retrieve it.

Other data consumers, which don’t have a tight coupling with the MSDN Publishing System, are responsible for providing access to their data according to their needs. The Aggregation System doesn’t in and of itself provide this functionality.

Send your questions and comments to insdmsdn@microsoft.com.

John Mollman is a Software Engineer in the MSDN/Technet Product Group at Microsoft. Previously he developed content publishing and distribution systems for Starwave and Walt Disney, and internet timing and scoring systems for the 2002 Summer Olympics. Reach him at blogs.msdn.com/mollman.


Extreme ASP.NET: Asynchronous Web Parts

Extreme ASP.NET
Asynchronous Web Parts
Fritz Onion

Code download available at: Onion2006_07.exe (176KB)
Contents
Web Part Congestion
Asynchronous Web Access
How It Works
Asynchronous Data Access

Building a customizable Web site complete with a collection of pluggable Web Parts is fairly easy with the portal infrastructure of ASP.NET 2.0. This model is very flexible, allowing users to easily place your Web Parts anywhere on the Web page so they are free to customize your site. However, these advantages can also lead to inefficiencies that may degrade the user experience, since you may not know beforehand which components will be used together, and therefore can’t make specific data retrieval optimizations for each individual component.

The most common inefficiency in a typical portal site occurs when multiple Web Parts simultaneously make network requests for data. Each request, whether to a Web service or a remote database, ends up adding to the overall time it takes to process the page even though the requests are typically independent of each other and could conceivably be issued in parallel.

Fortunately, ASP.NET 2.0 also introduces an easy-to-use asynchronous page model that, when used in combination with asynchronous Web service calls and asynchronous database access, can significantly improve the response time for a portal page as several independent Web Parts collect data in parallel. Here I’ll look at techniques for building Web Parts that perform their data retrieval asynchronously to make the portal pages that contain them more responsive and scalable.

Web Part Congestion

Let’s begin by considering the portal page shown in Figure 1. In this sample there are four Web Parts on a portal page, each retrieving data from a different source. The full source for this sample application is available for download on the MSDN®Magazine Web site and I encourage you to review the application as you read this column. In the sample, three of the Web Parts retrieve their data from a Web service, which intentionally waits for three seconds before returning. The fourth Web Part issues an ADO.NET query to a SQL Server database, which also waits three seconds before returning. This is an exaggerated example of the problem, but it’s not all that improbable.

Figure 1 Sample Portal Page
Figure 1 Sample Portal Page

Each of the Web Parts in the sample application is built with a user control and binds the results of the data retrieval to the controls that display it. The code and markup for each control is kept to a minimum so that the example is simple and lets you focus on making the Web Parts asynchronous.

Here’s the NewsWebPart.ascx user control file:

   <%@ Control Language="C#"
           AutoEventWireup="true"
           CodeFile="NewsWebPart.ascx.cs"
           Inherits="webparts_
           NewsWebPart" %>

   <asp:BulletedList ID="_newsHeadlines"
           runat="server">
   </asp:BulletedList>
And here’s the corresponding codebehind file for the news headlines sample Web Part:
public partial class webparts_NewsWebPart : UserControl
{
    protected void Page_Load(object sender, EventArgs e)
    {
        PortalServices ps = new PortalServices();
        _newsHeadlines.DataSource = ps.GetNewsHeadlines();
        _newsHeadlines.DataBind();
    }
}

Note how it interacts with a Web service to retrieve the sample news headlines. The stock quotes Web Part and the weather forecaster Web Part are implemented in much the same way and use different methods of the same Web service to retrieve their data. Similarly, Figure 2 shows the SalesReportWebPart.ascx user control file and the corresponding codebehind file for the sales report sample Web Part. Note how the control uses ADO.NET to retrieve the sales data from a database and then populates a GridView control with that data.

Figure 2 Sales Report Web Part (Close Figure 2)

SalesReportWebPart.ascx
<%@ Control Language="C#" AutoEventWireup="true"
            CodeFile="SalesReportWebPart.ascx.cs"
            Inherits="webparts_SalesReportWebPart" %>

Sales data for
<asp:TextBox ID="_yearTextBox" runat="server"
             Width="120px">2005</asp:TextBox>
<asp:Button ID="_setYearButton" runat="server"
            Text="Set year" /><br />

<asp:GridView ID="_salesGrid" runat="server"
        AutoGenerateColumns="False" DataKeyNames="id">
    <Columns>
        <asp:BoundField DataField="id" HeaderText="id"
                        Visible="False" ReadOnly="True" />
        <asp:BoundField DataField="quarter"
                        HeaderText="Quarter" />
        <asp:BoundField DataField="amount"
                        HeaderText="Amount"
                        DataFormatString="{0:c}"
                        HtmlEncode="False" />
        <asp:BoundField DataField="projected"
                        HeaderText="Projected"
                        DataFormatString="{0:c}"
                        HtmlEncode="False" />
    </Columns>
</asp:GridView>

SalesReportWebPart.ascx.cs
public partial class webparts_SalesReportWebPart : UserControl {
    protected void Page_Load(object sender, EventArgs e) {
        string dsn = ConfigurationManager.
             ConnectionStrings["salesDsn"].ConnectionString;
        string sql = "WAITFOR DELAY ‘00:00:03’ " +
                     "SELECT [id], [quarter], [year], [amount],
                     "[projected] FROM [sales] WHERE year=@year";

        using (SqlConnection conn = new SqlConnection(dsn))
        using (SqlCommand cmd = new SqlCommand(sql, conn))
        {
            cmd.Parameters.AddWithValue("@year",
                int.Parse(_yearTextBox.Text));
            conn.Open();
            _salesGrid.DataSource = cmd.ExecuteReader();
            _salesGrid.DataBind();
        }
      }
   }
}
Figure 3 Sequential Web Part Processing
Figure 3 Sequential Web Part Processing

As soon as the sample portal page is run, a problem becomes apparent. It takes more than 12 seconds for the request to process—a delay that will make most users avoid using the application. The reason for this lengthy delay is shown in Figure 3, which traces the path of execution a request takes when this page is executed. Like any other control in a page’s control hierarchy, each Web Part is loaded in turn, in the order defined by the control hierarchy of the page. Because this process is sequential, each Web Part must wait for the part preceding it in the hierarchy to complete before it can begin requesting its data and preparing its response. Because of the artificial 3-second delay introduced in each data retrieval, it is easy to see why it takes 12 seconds for the response to complete. Each Web Part is performing a completely independent data retrieval, one after the other. The important thing to realize is that all of these retrievals could be performed in parallel, cutting the response time by 75 percent. That is my goal here.

Asynchronous Web Access

In the example, three Web Parts use Web services to retrieve their data, and one uses ADO.NET to access a database. Let’s start by making the Web service invocations asynchronous, since there is some nice support in the Web service proxy classes generated by the Web Services Description Language tool WSDL.exe (or the Visual Studio 2005 Add Web Service Reference tool) for performing Web method invocation asynchronously.

When a Web service proxy class is created in ASP.NET 2.0, it actually generates three different ways of invoking any particular method, one synchronous and two asynchronous. For example, the Web service proxy that the Web Parts are using has the following methods available for invoking the GetNewsHeadlines Web method:

   public string[] GetNewsHeadlines()

   public IAsyncResult BeginGetNewsHeadlines(
       AsyncCallback callback, object asyncState)
   public string[] EndGetNewsHeadlines(       IAsyncResult asyncResult)

   public void GetNewsHeadlinesAsync()
   public void GetNewsHeadlinesAsync(       object userState)
   public event
       GetNewsHeadlinesCompletedEventHandler
       GetNewsHeadlinesCompleted;

The first method, GetNewsHeadlines, is the standard synchronous method. The next two, BeginGetNewsHeadlines and EndGetNewsHeadlines, can be used to invoke the method asynchronously and can be tied into any number of asynchronous mechanisms in .NET through the standard IAsyncResult interface.

But the most interesting method to use in this scenario is the last one: GetNewsHeadlinesAsync. In order to use this particular method, I must register a delegate with the proxy class’s event that was specifically generated to capture the results of async invocations (the GetNewsHeadlinesCompleted event in the example). The delegate signature is strongly typed to contain the return values of the method so that I can easily extract the results in the method implementation.

Using this event-based asynchronous method, rewriting the Web method invocation in the headline news Web Part to be asynchronous is easy, as shown in Figure 4. I first subscribe a delegate to the GetNewsHeadlinesCompleted event of the proxy class, and then call the GetNewsHeadlinesAsync method. In the implementation of the method subscribed to the completed event, I bind the results of the Web method call to the BulletedList to display to the client. One additional consideration is that these asynchronous methods only work if the Web Part is placed on a page with the Async="true" attribute set, which can be checked programmatically by looking at the IsAsync property of the containing page. If the page the Web Part is placed on is not async, then I need to resort to standard synchronous binding, as shown in Figure 4.

Figure 4 Asynchronous Data Binding (Close Figure 4)
public partial class webparts_NewsWebPart : UserControl {
    protected void Page_Load(object sender, EventArgs e)
    {
        // Instantiate Web service proxy for retrieving data
        PortalServices ps = new PortalServices();

        // If page has async enabled, invoke Web service
        // asynchronously and harvest results in callback.
        if (Page.IsAsync) {
            ps.GetNewsHeadlinesCompleted +=
              new GetNewsHeadlinesCompletedEventHandler(
                       ps_GetNewsHeadlinesCompleted);
            ps.GetNewsHeadlinesAsync();
        }
        else {
            // If not async, perform direct databinding
            _newsHeadlines.DataSource = ps.GetNewsHeadlines();
            _newsHeadlines.DataBind();
        }
    }

    // This callback is only invoked if the async web
    // service model was used.
    void ps_GetNewsHeadlinesCompleted(object sender,
        GetNewsHeadlinesCompletedEventArgs e) {
        _newsHeadlines.DataSource = e.Result;
        _newsHeadlines.DataBind();
    }
}

Now, for the asynchronous Web Part to perform its data retrieval asynchronously, it must be placed on a page with the Async attribute set to true, so I modify the Page directive of the portal page to look like the following:

<%@ Page Language="C#" AutoEventWireup="true"  Async="true" %>

Once I update the other two Web Parts that use Web services to retrieve their data asynchronously, the portal page is much more responsive. In fact, depending on the order in which the parts are loaded, it may render to the client in just over three seconds (if the sales Web Part is loaded first, it will take just over six seconds)! Even though the sales report Web Part is still sequentially accessing the database, the other three Web Parts are now performing their Web service invocations asynchronously so the primary request thread is no longer waiting for their completion. Of course, I ultimately want to have all of the I/O-bound work be asynchronous so that clients could use both Web service and database-driven Web Parts without unnecessary sequential blocking.

The other reason to push the I/O-bound work onto asynchronous I/O requests is to relinquish the primary thread back to the thread pool to service other requests. Currently I’m relinquishing the thread only after my sales report database query has completed, which means I’m sitting idly for three full seconds occupying a threadpool thread that could be used to service other requests. If I can make this last I/O-bound request for data asynchronous as well, my page will use the request thread only long enough to spool off all the asynchronous I/O requests and then return immediately back to the pool.

How It Works

If you’ve ever done any asynchronous programming, you probably have the feeling that the minor changes made to the Web service invocation can’t possibly be sufficient. I didn’t even have to touch an IAsyncResult interface, nor did I have to let the containing page know that I was performing asynchronous operations (by registering a task or some other technique), and yet it all seemed to work as I had hoped.

The secret lies in the Web service proxy class’s implementation of the asynchronous method, along with a helper class introduced in the Microsoft® .NET Framework 2.0 called the AsyncOperationManager. When I called the GetNewsHeadlinesAsync method of the proxy class, it mapped the call onto an internal helper method of the SoapHttpClientProtocol base class, called InvokeAsync, from which the proxy class derives. InvokeAsync does two important things—it registers the asynchronous operation by calling the AsyncOperationManager’s static CreateOperation method, and it then launches the request asynchronously using the WebRequest class’s BeginGetRequestStream method. At this point the call returns and the page goes on processing its lifecycle, but because the page has been marked with the Async="true" attribute, it will only continue processing the request up through the PreRender event and will then return the request thread to the thread pool. Once the asynchronous Web request completes, it will invoke the method I subscribed to the completed event of the proxy on a separate thread drawn from the I/O thread pool. If this is the last of the asynchronous operations to complete (kept track of by the synchronization context of the AsyncOperationManager), the page will be called back and the request will complete its processing from where it left off, starting at the PreRenderComplete event. Figure 5 shows this entire lifecycle when you use asynchronous Web requests in the context of an asynchronous page.

Figure 5 Asynchronous Web Requests in Asynchronous Pages
Figure 5 Asynchronous Web Requests in Asynchronous Pages

The AsyncOperationManager is a class that is designed to be used in different environments to help in the management of asynchronous method invocations. For example, if I called a Web service asynchronously from within a Windows® Forms application, it would also tie into the AsyncOperationManager class. The difference between each environment is the SynchronizationContext associated with the AsyncOperationManager. When running in the context of an ASP.NET-based application, the SynchronizationContext will be set to an instance of the AspNetSynchronizationContext class. The primary purpose here is to keep track of how many outstanding asynchronous requests are pending so that when they are all complete, the page request processing can resume. In contrast, when in a Windows Forms-based application, the SynchronizationContext will be set to an instance of the WindowsFormsSynchronizationContext class. Its primary purpose is to allow for easier marshaling of invocations from a background thread to the UI thread.

Asynchronous Data Access

Now, back to the problem of making the last Web Part asynchronous and the general issue of performing asynchronous data retrieval with ADO.NET. Unfortunately, there is no equivalent to the simple asynchronous mechanism exposed by Web service proxies for performing asynchronous data retrieval, so I’m going to have to do a little more work to get the final Web Part to participate in the asynchronous shuffle. I can work with the new asynchronous methods on the SqlCommand class and the asynchronous task feature of ASP.NET. Using SqlCommand, I can now invoke commands asynchronously using one of the following methods:

  • IAsyncResult BeginExecuteReader(AsyncCallback ac, object state)
  • IAsyncResult BeginExecuteNonQuery(AsyncCallback ac, object state)
  • IAsyncResult BeginExecuteXmlReader(AsyncCallback ac, object state)
And I can invoke the corresponding completion methods once the data stream is ready to begin reading:
  • SqlDataReader EndExecuteReader(IAsyncResult ar)
  • int EndExecuteNonQuery(IAsyncResult ar)
  • XmlReader EndExecuteXmlReader(IAsyncResult ar)

To use any of these asynchronous retrieval methods, "async=true" must be added to the connection string. For this scenario, I am interested in populating a GridView by binding it to a SqlDataReader, so I will use the BeginExecuteReader method to initiate the asynchronous call.

To tie this into the asynchronous page, ASP.NET 2.0 also allows you to register asynchronous tasks that need to be executed before the page completes rendering. This is a more explicit model than the one I used with the Web service proxies, but it also provides more flexibility. To register an asynchronous task, I create an instance of the PageAsyncTask class and initialize it with three delegates: begin handler, end handler, and timeout handler. The begin handler must return an IAsyncResult interface, so this is where I will launch my asynchronous data request using BeginExecuteReader. The end handler is called once the task is complete (when there is data ready to read in this example), at which point I can use the results. ASP.NET will take care of invoking the begin handler just before it relinquishes the request thread (immediately after the PreRender event completes). Figure 6 shows the updated implementation of the sales report Web Part performing asynchronous data access using the asynchronous tasks and the asynchronous BeginExecuteReader method of the SqlCommand class.

Figure 6 Asynchonous Data Access (Close Figure 6)
public partial class webparts_SalesReportWebPart : UserControl {
    // Local variables to store connection and command
    // for async data retrieval
    SqlConnection _conn;
    SqlCommand    _cmd;

    protected void Page_Load(object sender, EventArgs e) {
        string dsn = ConfigurationManager.
            ConnectionStrings["salesDsn"].ConnectionString;
        dsn += ";async=true"; // Allow for asynchronous operations

        string sql = "WAITFOR DELAY ‘00:00:03’ SELECT [id], " +
                     "[quarter], [year], [amount], [projected] FROM " +
                     "[sales] WHERE year=@year";

        _conn = new SqlConnection(dsn);
        _cmd = new SqlCommand(sql, _conn);
        _conn.Open();
        _cmd.Parameters.AddWithValue("@year",
            int.Parse(_yearTextBox.Text));

        // Launch data request asynchronously using page async task
        Page.RegisterAsyncTask(new PageAsyncTask(
            new BeginEventHandler(BeginGetSalesData),
            new EndEventHandler(EndGetSalesData),
            new EndEventHandler(GetSalesDataTimeout),
            null, true));
    }

    IAsyncResult BeginGetSalesData(object src, EventArgs e,
        AsyncCallback cb, object state)
    {
        return _cmd.BeginExecuteReader(cb, state);
    }

    void EndGetSalesData(IAsyncResult ar) {
        _salesGrid.DataSource = _cmd.EndExecuteReader(ar);
        _salesGrid.DataBind();
        _conn.Close();
    }

    void GetSalesDataTimeout(IAsyncResult ar) {
        // operation timed out, so just clean up by closing the connection
        if (_conn.State == ConnectionState.Open) _conn.Close();
        _messageLabel.Text = "Query timed out...";
    }

}

Note that I could use this same technique with my Web service requests by using the alternate asynchronous methods provided on the proxy class (BeginGetNewsHeadlines, for example). One potential advantage to this technique is that I can also specify a timeout handler. If the remote invocations fail to return in time, the associated timeout handler will be invoked. This timeout is specified in the Page directive using the AsyncTimeout attribute and defaults to 20 seconds. Also note that unlike when using the event-based asynchronous pattern, when using Page.RegisterAsyncTask I don’t have to branch to a synchronous invocation based on the result of Page.IsAsync. Asynchronous page tasks in Web Parts work just fine on synchronous pages, and even allow for parallel execution of Web Parts. The core difference is that on synchronous pages (ones without the Async="true" attribute), the main page thread won’t be released back to the thread pool during the execution of the asynchronous operations.

With all of the Web Parts now performing their data retrieval asynchronously, I can now use these parts in any page marked as asynchronous and know that the response time will no longer be the sum of the time it takes for all the Web Parts to retrieve their data, but the maximum amount of time taken by any one Web Part. By marking the page as asynchronous and using Web Parts that perform asynchronous I/O, I also increase the potential scalability of the site since the page will free up the primary request thread to service other clients while waiting for the data. The key takeaway here is that if you’re building portal sites with ASP.NET 2.0, you should keep in mind all of the new asynchronous features introduced in this release, and take advantage of them to improve both the responsiveness and scalability of your applications. For more information on asynchronous support in ASP.NET 2.0, see Jeff Prosise’s Wicked Code column in the October 2005 issue of MSDN Magazine.

Send your questions and comments for Fritz to xtrmasp@microsoft.com.

Fritz Onion is a cofounder of Pluralsight, a Microsoft .NET training provider, where he heads the Web development curriculum. Fritz is the author of Essential ASP.NET (Addison Wesley, 2003) and the upcoming Essential ASP.NET 2.0 (Addison Wesley, 2006). Reach him at pluralsight.com/fritz.


C++ at Work: Controlling Balloon Tips, and More

C++ At Work
Controlling Balloon Tips, and More
Paul DiLascia

Code download available at: CatWork2006_07.exe (217KB)
Contents
Upgrade Update
Triple-Click Update

Q I can’t find a way to make my balloon tips show whenever I want them to. That is, when my CEdit-based control has the focus, the balloon shows and then disappears whenever the control loses focus. I currently use the RelayEvent of the balloon in PreTranslateMessage event to make it pop up. How can I make my balloon tips show as I want?

Omid Manikhi

A For readers who don’t know what balloon tips are (there may be a few), they’re little yellow word balloons similar to tooltips that you can show when the user tabs to your edit control. Balloon tips were introduced with Windows® XP (_WIN32_WINNT>= 0x501). It’s your responsibility to show and hide balloon tips at the proper time by calling EM_SHOWBALLOONTIP and EM_HIDEBALLOONTIP.

At the same time they added balloon tips, the friendly Redmondtonians also added another edit-control feature—banner cues. A banner cue is a prompt like "Yo, type your name here!" that Windows displays in grayed text when the edit control is empty. As soon as the user tabs to the control, the banner cue disappears. Unlike balloon tips, which you must hide and show dynamically, you can set the banner cue once with EM_SETBANNERCUE when you initialize your form/dialog, and Windows will display banner cues at the proper time. Curiously, while MFC has CEdit wrappers for Get/SetBannerCue, it doesn’t have wrappers for balloon tips.

The symbols EM_SHOW/HIDEBALLOONTIP and EM_GET/SETBANNERCUE are defined in commctrl.h, but not the version of commctrl.h that ships with Visual Studio® 2005. You have to download the latest Windows SDK. In general, if you want the latest features, you should always build your projects with the Windows SDK files, not the ones that ship with Visual Studio. I’ll show you how to implement balloon tips and banner cues in just a moment, but first I must digress to explain a little about manifests.

The most likely reason you couldn’t get your balloon tips to work is that your app is loading an older version of comctl32.dll. Including the right header file is only half the story; you also must ensure your app loads the latest DLL. Balloon tips and banner cues require Common Controls version 6.0. By default, the version of comctl32.dll that’s installed in the \WINDOWS directory is version 5.82, which is a redistributable version. Version 6.0 is not redistributable. Users can only get it by installing Windows XP. The 6.0 version lives all by itself in its very own side-by-side directory, one with a long nasty name like C:\windows\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.0.0_x-ww_1382d70a\. On my machine, I have four—count ‘em, four!—such directories, one each for versions 6.0.0.0, 6.0.10.0, 6.0.2600.1579, and 6.0.2600.2180. So now we’ve progressed from DLL Hell to folder madness!

So now that you know where version 6.0 of comctl32.dll lives—how do you get your app to use it? This is where the manifest comes in. As you all know by now, a manifest is an XML file that describes your app’s dependencies—that is, which versions of which DLLs it needs to run. The manifest lets you say, "My app needs version 6.0 of comctl32.dll." You can specify which version of any DLL your app requires, and Windows will try to load it. Figure 1 shows the manifest I wrote for my test program, with a dependency that calls for version 6.0 of comctl32.dll. Note the public key token, which is required.

Figure 1 Manifest Using Common Controls V.6.0 (Close Figure 1)
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
    <assemblyIdentity
        version="1.0.0.0" processorArchitecture="X86"
        name="MSDN.DiLascia.TestForm" type="win32"
    />
    <description>TestForm -- Test program for RegexForm</description>
    <dependency>
        <dependentAssembly>
            <assemblyIdentity
                type="win32"
                name="Microsoft.Windows.Common-Controls"
                version="6.0.0.0"
                processorArchitecture="X86"
                publicKeyToken="6595b64144ccf1df"
                language="*"
            />
        </dependentAssembly>
    </dependency>
</assembly>

Writing the manifest is the easy part. The hard part is getting Windows to use it. There are two ways to add a manifest to your app. You can install a separate application.exe.manifest file along with your app.exe; or you can embed the manifest in your exe as a resource. Embedding is better because you don’t have to worry about the exe and manifest becoming separated. The simplest way to add a manifest to your app is to add a line in your resource file:

IDR_MAINFRAME RT_MANIFEST
    "res\\MyApp.manifest"

Another way is to use the manifest tool, mt.exe, which you can control from your project settings under Manifest Tool. For example, you can specify additional manifests to embed under Manifest Tool | Input and Output (see Figure 2). By default, Visual Studio generates an embedded manifest for whatever libraries you link with. The linker generates the manifest and the Manifest Tool embeds it in your exe. That’s why Omid couldn’t get his balloon tips to show: the default manifest stipulated an earlier version of Common Controls.

Figure 2 Specifying Additional Manifests (Close Figure 2)

If you’re not sure which version of a particular DLL your app actually loads, there’s an easy way to find out: call the DLL’s DllGetVersion function. Not all DLLs provide DllGetVersion, but most modern DLLs do, and fortunately ComCtl32.dll does. You can call GetProcAddress to see if the DLL implements DllGetVersion and, if so, call it. Way back in April 1998, I wrote a little class CModuleVersion with a static function DllGetVersion that encapsulates the process and can be used as follows:

DLLVERSIONINFO dvi;
CModuleVersion::DllGetVersion(_
    T("comctl32.dll"), dvi);
TestForm uses CModuleVersion to display the Common Controls version number in its main dialog (see Figure 3). Displaying version numbers in your About dialog is a good idea because it facilitates debugging and support. When a user calls to say your app isn’t working, the support folks can ask her to run the About dialog to quickly rule out DLL version problems.

Figure 3 TestForm Page
Figure 3 TestForm Page

And speaking of TestForm, now I can finally return to my main topic, which is how to implement balloon tips and banner cues. For my sample app, I modified the RegexForm program from my April and August 2005 columns. RegexForm uses the Active Template Library (ATL) Regex class (CAtlRegExp) to implement a table-driven form-validation system. For details, read the columns. FormTest uses my own home-grown CPopupText class to display pop-up tips that prompt the user what to type. For this month’s column, I modified CPopupText to use balloon tips. Figure 4 shows the before and after screen shots, and Figure 3 shows TestForm running with banner cues and balloon tips displayed.

Figure 4 Showing Balloon Tips
Figure 4 Showing Balloon Tips

Setting banner cues is straightforward. Just call MFC’s CEdit::SetBannerCue. RegexForm::Init does it when you initialize your regex form. CRegexForm already has the names of all the fields in the descriptor table you give it, so it’s easy to set each banner cue to "Enter [field-name]" where field-name is the actual field name. For example, "Enter Zip Code" or "Enter Phone Number." RegexForm::Init initializes the banner cues automatically from your form table if it detects version 6.0 common controls:

// in CRegexForm::Init
for (/* each fld=FLDINFO */) {
   ...
   if (m_iComCtlVer>=6) {
      CEdit* pEdit =        (CEdit*)pDlg->GetDlgItem(fld.id);
      CString s;
      s.Format(_T(" Enter %s here"), fld.name);
      pEdit->SetCueBanner(s);
   }
}
CRegexForm gets m_iComCtlVer from CModuleVersion::DllGetVersion as I already described. You don’t actually need to perform this check because older edit controls will ignore EM_SETBANNERCUE.

So much for banner cues—what about balloon tips? They’re slightly more work, but not much. The proper time to show the balloon tip is when your edit control gains focus. That is, when it receives WM_SETFOCUS or the main dialog receives WM_EN_SETFOCUS. Likewise, you should hide the tip when your edit control loses focus. For FormTest, I added a one-second delay before showing the balloon. If the user types immediately, why pester him with annoying screen junk?

CRegexForm and CPopupText already have all the logic to trap WM_EN_SET/KILLFOCUS and do delayed prompting with timeouts, so instead of implementing a new CBalloonTip class, I decided to ride on the back of my already-working CPopupText class. Why write code you don’t have to? Almost everything is exactly the same as before, except when it comes time to actually display the tip. This happens in CPopupText::OnTimer, when my show-the-tip alarm rings:

// in CPopupText::OnTimer
if (m_bUseBalloonTip) {
   CString text;
   GetWindowText(text);
   EDITBALLOONTIP bt;
   bt.cbStruct = sizeof(bt);
   bt.pszText  = text;
   bt.pszTitle = NULL;
   bt.ttiIcon = TTI_NONE;
   bRet = m_pWndEdit->SendMessage(EM_SHOWBALLOONTIP, 0, (LPARAM)&bt);
}
if (!bRet)
   // display popup text

CPopupText gets a new flag, m_bUseBalloonTip (default=TRUE). If the flag is set, CPopupText attempts to display a balloon tip. It gets the tip text from the pop-up text window, which CRegexForm has already set to the proper prompt. If the balloon tip fails (bRet=FALSE), CPopupText degrades gracefully back to its old-style popup tips. Piggy-backing on CPopupText has another benefit, too: I can route the WM_TIMER messages through the pop-up text window even though this window is hidden if balloon tips are operating. Figure 5 shows CPopupText with balloon tip support. The banner cues are in RegexForm.cpp, which you can download.

Figure 5 CPopupText with Balloon Tip Support (Close Figure 5)

puptext.h
#pragma once

//////////////////
// Get NONCLIENTMETRICS info: ctor calls SystemParametersInfo.
//
class CNonClientMetrics : public NONCLIENTMETRICS {
public:
    CNonClientMetrics() {
        cbSize = sizeof(NONCLIENTMETRICS);
        SystemParametersInfo(SPI_GETNONCLIENTMETRICS,0,this,0);
    }
};

//////////////////
// Popup text window, like tooltip.
//
class CPopupText : public CWnd {
public:
    CSize m_szMargins;       // extra space around text: change if you like
    CFont m_font;                // font: change if you like
    UINT  m_msecTimeout;         // auto-timeout
    BOOL    m_bUseBalloonTip;    // use balloon tip
    CWnd* m_pWndEdit;            // edit control for balloon tip, if any

    CPopupText();
    virtual ~CPopupText();

    BOOL Create(CPoint pt, CWnd* pParentWnd, UINT nID=0);
    void ShowDelayed(UINT msec, UINT msecTimeout=0, CWnd* pWndEdit=NULL);
    void Cancel();
    void UseBalloonTip(BOOL bUseBalloonTip)
    {
        m_bUseBalloonTip = bUseBalloonTip;
    }

protected:
    virtual void PostNcDestroy();
    virtual BOOL PreCreateWindow(CREATESTRUCT& cs);

    void DrawText(CDC& dc, LPCTSTR lpText, CRect& rc, UINT flags);

    afx_msg void OnPaint();
    afx_msg void OnTimer(UINT nIDEvent);
    afx_msg LRESULT OnSetText(WPARAM wp, LPARAM lp);

    DECLARE_DYNAMIC(CPopupText);
    DECLARE_MESSAGE_MAP();
};

puptext.cpp
#include "stdafx.h"
#include "puptext.h"

#ifdef _DEBUG
#define new DEBUG_NEW
#undef THIS_FILE
static char THIS_FILE[] = __FILE__;
#endif

enum {
    TIMER_SHOW = 1,
    TIMER_CANCEL,
};

IMPLEMENT_DYNAMIC(CPopupText,CWnd)
BEGIN_MESSAGE_MAP(CPopupText,CWnd)
    ON_WM_PAINT()
    ON_MESSAGE(WM_SETTEXT, OnSetText)
    ON_WM_TIMER()
END_MESSAGE_MAP()

CPopupText::CPopupText()
{
    CNonClientMetrics ncm;
    m_font.CreateFontIndirect(&ncm.lfMenuFont);
    m_szMargins = CSize(4,4);
    m_pWndEdit = NULL;
    // Default: use balloon tip. Note this will gracefully fail to normal tip
    // if commctl32 v6 is not installed.
    m_bUseBalloonTip = TRUE;
}

CPopupText::~CPopupText()
{
}

...

//////////////////
// Show window with delay. No delay means show now.
// Can also specify an auto-timeout to cancel tip.
//
void CPopupText::ShowDelayed(UINT msec, UINT msecTimeout, CWnd* pWndEdit)
{
    m_msecTimeout = msecTimeout;
    m_pWndEdit = pWndEdit;
    if (msec==0) {
        OnTimer(1);    // no delay: show it now
        if (msecTimeout>0) {
            SetTimer(TIMER_CANCEL, msecTimeout, NULL); // auto-cancel
        }

    } else {
        SetTimer(TIMER_SHOW, msec, NULL); // delay: set timer
    }
}

//////////////////
// Cancel: kill timer and hide window
//
void CPopupText::Cancel()
{
    if (m_hWnd) {
        KillTimer(TIMER_SHOW);
        KillTimer(TIMER_CANCEL);
        LRESULT bRet = FALSE;
        if (m_bUseBalloonTip && m_pWndEdit) {
            bRet = m_pWndEdit->SendMessage(EM_HIDEBALLOONTIP);
        }
        if (!bRet) {
            ShowWindow(SW_HIDE);
        }
    }
}

//////////////////
// Timer popped: display myself and kill timer
//
void CPopupText::OnTimer(UINT nTimer)
{
    if (nTimer==TIMER_SHOW) {
        LRESULT bRet = FALSE;
        if (m_bUseBalloonTip && m_pWndEdit) {
            // use balloon tip
            CString text;
            GetWindowText(text);
            EDITBALLOONTIP bt;
            bt.cbStruct = sizeof(bt);
            bt.pszText  = text;
            bt.pszTitle = NULL;
            bt.ttiIcon = TTI_NONE;
            bRet = m_pWndEdit->SendMessage(
                EM_SHOWBALLOONTIP, 0, (LPARAM)&bt);
        }
        if (!bRet) {
            ShowWindow(SW_SHOWNA);
            Invalidate();
            UpdateWindow();
        }
        KillTimer(TIMER_SHOW);
        if (m_msecTimeout>0) {
            SetTimer(TIMER_CANCEL, m_msecTimeout, NULL); // auto-cancel
        }
    } else if (nTimer==TIMER_CANCEL) {
        Cancel();
    } else {
        ASSERT(FALSE); // should never happen
    }
}

Upgrade Update

Last month I reported some of my experiences upgrading to Visual Studio 2005. Well wouldn’t you know it, the minute I shipped the column I ran into a whole new category of compiler changes I have to tell you about: the Safe C/C++ libraries. Even though we’re well into the new millennium, it appears some of my code still calls such old-world routines as strcpy and fopen. Well cover me in tar and drop me in a barrel of feathers! strcpy and fopen are just soooo retro.

To help folks like me get off their duffs and embrace modernity, Visual Studio 2005 introduces something called the Safe C/C++ libraries. These libraries provide replacement functions for many of the old C runtime libraries (CRT) functions you know and love. And why are they bad? Because they make it so easy to write unsafe code:

char mybuf[64];
strcpy(mybuf,otherstring);

What happens if otherstring is longer than 64 characters? Oops. Buffer overruns not only cause bugs, they open your program to exploits by malicious crackers. And what about fopen—why is fopen unsafe? Because it opens files in shared mode, which can lead to collisions if two programs write to the same file at the same time.

Starting with Visual Studio 2005, the unsafe functions are officially deprecated. Which is a fancy way to say don’t use them! If you do, the compiler will politely complain. Instead, the CRT library in Visual C++ ®2005 provides alternatives. The Safe C library has functions like strcpy_s and fopen_s, safe versions to replace the old equivalents:

char buf[64];
strcpy_s(buf, _countof(buf), src);

The Safe C functions do the sort of bounds-checking your mother always insisted on but you were too lazy to write. Of course, you have to modify your code to use them—but if you’re writing in C++, life is easy. You can #define a special symbol:

#define _CRT_SECURE_CPP_OVERLOAD_STANDARD_NAMES 1
And the compiler will use C++ function-overloading to generate safe templates that override the offending functions. There are only a few situations where template overloading fails and you have to explicitly call a safe function. And if you just want the compiler to shut up, you can always write:
#define _CRT_SECURE_NO_DEPRECATE
This turns off all Safe-C/C++ warnings. It’s a quick way to make your code compile if you’re in a hurry or you have millions of lines of legacy code you can’t bother to fix. I don’t recommend it. I know, when the compiler doesn’t like your code, it’s cause to gripe. But safe warnings really are your friends. You ignore them at your own peril. The warnings make it really easy to write safer code. Instead of searching every nook and cranny in your code for places where something naughty may happen, the friendly Redmontonians have done it for you. All you have to do is compile, then fix the warnings one by one. For C++ programmers, _CRT_SECURE_CPP_OVERLOAD_STANDARD_NAMES may be all you need.

I poked around in crtdefs.h, which is where all the _DEPRECATE warnings are defined, to see what else I could find. Visual Studio 2005 has lots of deprecated functions of all different kinds. For example, the compiler now complains of non-standard C names like strnicpy (it should be _strnicpy), and there are managed-heap deprecations for functions like _heapwalk. In all cases, the pattern is the same: there’s a macro the Redmondtonians use to declare the function and a symbol you can #define to suppress the warnings.

The Safe C/C++ libraries are, in my opinion, one of the best features in Visual Studio 2005. They’re not sexy, but they offer real benefits. They prod you to write better code, without forcing you to do so. To learn more about Safe C/C++, read Martyn Lovell’s excellent article in the May 2005 issue.

Triple-Click Update

In my April 2006 column, I showed how to implement a CTripleClick class to handle triple-clicks for your app. My implementation used the C standard library clock() function. But as reader Chris Depetris pointed out to me, the clock function is unreliable when there’s a background process that’s hogging the CPU. A better strategy is to use GetMessageTime. Right you are, Chris! GetMessageTime is more reliable. It reports the clock time of the current Windows message—that is, MSG::time for the current MSG. For the purposes of triple-click, that’s exactly what you want: the time when the mouse event occurred. You can find a revised version of Click3 in this month’s download.

As always, happy programming!

Send your questions and comments for Paul to cppqa@microsoft.com.

Paul DiLascia is a freelance software consultant and Web/UI designer-at-large. He is the author of Windows++: Writing Reusable Windows Code in C++ (Addison-Wesley, 1992). In his spare time, Paul develops PixieLib, an MFC class library available from his Web site, www.dilascia.com.


End Bracket: Refactor for Life

Refactor for Life
Eric N. Bush

There are certain fundamental laws of the universe that we just can’t escape. One of these is the Second Law of Thermodynamics as it applies to the general branches of science. It says that everything moves to a state of equilibrium and an increase in disorder, or entropy. This is even applicable in the field of computer science. Software systems tend to become increasingly disordered and complex as they are developed. When software reaches that critical point where it has aged and cannot be maintained, we bestow upon it the venerable title of Legacy Code. It might sound prestigious in another field, but when it comes to software, Legacy Code is a title no one is anxious to own.

The underlying reason for code becoming such a mess is that software development is an iterative process. Developers don’t start with a thorough blueprint that covers every detail to follow from start to finish. Instead, you start with a small kernel of code and expand it from there, adding to it from all sides until you have shaped the application into what you set out to create. But, if you’re like most developers, you don’t keep your code tidy as you rush towards shipping. This is made worse as project requirements and feature requests cause unforeseen changes in the code.

Development teams, of course, recognize this deficiency in their process and occasionally, when time permits, sneak in a little code cleanup work. This is typically done near the end of an implementation phase. But since developers want to be careful not to break anything that is already working, they don’t do the massive cleanup that is really needed. At this point it is all about managing risk and, as the saying goes, "if it ain’t broke, don’t fix it."

The more rigorous and official process of doing a cleanup is called refactoring. It has to be a way of life, something that is fully embraced and done right from the first day of implementation. This practice basically involves cleaning up your code as you write it, when you recognize ways to make it simpler or better structured without changing the functionality. Refactoring ranges from simple code cleanup to more complex design changes. It does not, however, involve fixing bugs. Strictly speaking, if you have a bug in your code and you refactor, the bug should remain because basic functionality (right or wrong) should not change. Before you start, you must first have a good set of tests to run. Without tests, you can’t be certain that the refactoring has not changed behavior.

If you have exhausted your own ideas on how to refactor your code, you can refer to Martin Fowler’s book Refactoring: Improving the Design of Existing Code. This has become the authoritative book on the topic. Fowler actually catalogs the different situations that require refactoring techniques and gives these names, such as Encapsulate Field, Extract Class, and Extract Method. For the Extract Method technique, you identify some code that can be taken out of an existing function and made a separate function. You might do this because the original function was too large or because you found out that the code was duplicated in more than one place. Visual Studio® 2005 contains a refactoring feature that makes it easy to perform several of the techniques. Here are some tips for successful refactoring:

  1. Have a comprehensive set of tests ready before you refactor.
  2. Make refactoring a regular part of your day-to-day coding practice.
  3. Make changes one at a time, continually verifying with tests that the code still works.
  4. Don’t jeopardize shipping; minimize the practice towards the end.
  5. If you inherit code that is poorly structured, consider refactoring the overall design first.

If you follow these practices, you will see many benefits. Having code that is less complex and better organized simplifies maintenance and makes future development more productive. Transfer of ownership is smoother. You usually end up with fewer lines of code. All of these benefits can translate into higher quality code. The crucial point is that you need to make refactoring a way of life. As messy code piles up, it gets harder and harder clean up. Don’t let yourself get to that point.

Perhaps we need to invoke the Third Law of Thermodynamics, which states that entropy in a system approaches a minimum value when the temperature is equal to absolute zero. Applied to the field of computer science, perhaps this law would state that software developers are kept from writing messy code when their office temperature is kept near absolute zero.

Eric N. Bush is a software development lead in the Engineering Excellence group at Microsoft. He works as an internal consultant and classroom instructor driving engineering and organizational improvements in the areas of people skills, process, and technology. Reach him at ericbush@microsoft.com.