 MSDN Magazine June 2004

MSDN Magazine June 2004

The Big Story: An Overview of the New Services, Controls, and Features in ASP.NET 2.0

ASP.NET has become a bit of a gold standard for Web programming. The upcoming version, ASP.NET 2.0 will have even more of the kinds of features that have made it the popular framework it is today. This article takes a broad look at those features, including what's new in data source controls, themes and skins, the DataGrid and its new functionality, managing roles, and other administrative tasks.

Jeff Prosise Security: Security Headaches? Take ASP.NET 2.0!

ASP.NET 2.0 provides significant advantages with respect to security, especially for folks developing Web sites that use Forms authentication. By providing a user profile repository with support for roles, Forms authentication will move beyond the purview of the ASP.NET internals guru, and should become much more broadly accessible. This article introduces security in ASP.NET 2.0 to give you a head start with upcoming features.

Keith Brown Portals: Unleash Your Site's Potential with Web Parts and Personalization in ASP.NET 2.0

ASP.NET 2.0 has addressed some of the most common problems developers face today with a suite of controls, components and IDE tools. One such issuer is maintaining preferences information about individual users of a Web application in a uniform manner. This article drills down into ASP.NET 2.0 to illustrate how it helps solve these problems far easier than is possible today, and demonstrates how these features can be combined to build powerful, personalized Websites in very little time.

Steven A. Smith Data: More Load, Less Code with the Data Enhancements of ASP.NET 2.0

A data source control is a server control that wraps some basic functions of a data source - be it a SQL Server database, an XML document, an Excel worksheet, or a sitemap description. Through the services of a similar component, data-bound controls can fetch data as well as insert new records or update and delete existing ones. Data source controls enable a consistent model across a variety of data sources and dramatically reduce the amount of code needed to implement a two-way data-binding scenario. This article provides an introduction to data source controls and other related data binding features.

Dino Esposito Master Pages: Master Your Site Design with Visual Inheritance and Page Templates

Master pages in ASP.NET 2.0 solve a problem many Web developers have been solving on their own with a variety of techniques for years - providing a single master template for an entire site. This article covers the details of master pages, discussing their usage and implementation and how they are a natural evolution of custom techniques developers are using today.

Fritz Onion Controls: Plan Your Migration to the Visual Studio 2005 Navigation Controls

Navigation is central to a well functioning Web application and it can be implemented numerous ways using today's technologies. This article highlights some of the key details of the new navigation controls that are part of the upcoming ASP.NET 2.0, and talks about how to design and implement one of these navigation controls today. This article contains a simple Web site that demonstrates the use of these navigation controls, pointing out some of the key items such as a breadcrumb control and things to watch for along the way. The article then describes how to implement a breadcrumb control today.

Dave Donaldson and Steven DeWalt Editor's Note: Fun with ASP.NET 2.0

As you could probably tell from the cover, we are bringing you early, in-depth coverage of ASP. NET 2. 0, the Web components of the next version of the Microsoft . NET Framework. ASP. NET 2. 0 was unveiled at the October 2003 Microsoft PDC, but until now has had limited availability.

New Stuff: Resources for Your Developer Toolbox

Theresa W. Carey Web Q&A: Visual Studio 2005, HTC Memory Problems, and More

Edited by Nancy Michell Data Points: Contrasting the ADO.NET DataReader and DataSet

John Papa Cutting Edge: Dress Your Controls for Success with ASP.NET 1.1 Themes, Part 2

Dino Esposito The XML Files: InfoPath 2003 SP1 Preview

Aaron Skonnard .NET Column: Unexpected Errors in Managed Applications

Jason Clark Bugslayer: Bad Code? FxCop to the Rescue

John Robbins Basic Instincts: Creating and Managing Secondary Threads

Ted Pattison .NET Matters: XML Comments, Late-bound COM, and More

Stephen Toub C++ Q&A: Performance Optimization, Controls versus Components

Paul DiLascia Resource File: Blogging Tools

Aaron Skonnard


The Big Story: An Overview of the New Services, Controls, and Features in ASP.NET 2.0

[ Editor's Update - 1/20/2006: This article refers to a beta version of Visual Studio 2005. An updated version of the article, reflecting features found in the final release of Visual Studio 2005, can be found at Web Apps: An Overview Of The New Services, Controls, And Features In ASP.NET 2.0.]
The Big Story

An Overview of the New Services, Controls, and Features in ASP.NET 2.0

Jeff Prosise
This article is based on the March 2004 Community Technology Preview of ASP.NET 2.0. All information contained herein is subject to change.
This article discusses:
  • New data controls
  • Administration and roles
  • Personalization and themes
  • Coding and compilation options
This article uses the following technologies:
ASP.NET 1.x and C#
Code download available at: ASPNET20Overview.exe (121KB)
Contents
Master Pages
Data Source Controls
Themes and Skins
New Controls
The GridView and DetailsView Controls
What's New in Administration
Membership Service
Login Controls
Role Manager
Personalization
SQL Cache Dependencies
New Dynamic Compilation Model
Precompiling and Deploying Without Source
New Code Separation Model
Client Callback Manager
Validation Groups
Cross-page Posting
Conclusion

I
n the four short years since it was first unveiled, ASP.NET has become the gold standard for Web applications run on servers powered by Windows®, adding runat="server" to the vocabularies of Web developers everywhere. It has also provided a revealing glimpse into the future of Web programming—a future that revolves around server-side controls that render HTML and script, and that fire events.

In the next major release of the Microsoft® .NET Framework, ASP.NET 2.0 sheds its new-kid-on-the-block status and matures into a full-blown adult. Its aim is to reduce the amount of code required to accomplish common Web programming tasks by 70 percent or more. The goal is a lofty one, but also one that's achievable thanks to a diverse assortment of new services, controls, and features that promise to make ASP.NET 2.0 almost as dramatic an improvement to ASP.NET 1.x as ASP.NET 1.x was to ASP.

Here I'll provide a broad overview of what you can expect to see in ASP.NET 2.0, with drill-downs in selected areas and sample programs to highlight key features. All code samples were built and tested against a pre-beta build of ASP.NET 2.0, and some code samples may have to be modified to work with the first beta.

Master Pages

One of the most glaring deficiencies of ASP.NET 1.x is its lack of support for page templates. Missing is the ability to define a "master page" that other pages inherit from. Developers make up for this by composing pages from user controls, which are easily replicated from page to page. Such tricks are no longer necessary in ASP.NET 2.0 thanks to a new feature known as Master Pages. Think of "visual inheritance" and you'll understand what Master Pages are all about. First you define a Master Page containing content that you want to appear on other pages and use ContentPlaceHolder controls to define the locations where subpages can plug in content of their own. Then you build subpages—ASPX files—that reference the master using directives like this one:

<%@ Page MasterPageFile="~/Foo.master" %>

In the subpages, you use Content controls to fill in the placeholders in the Master Page. Bring up a subpage in your browser and the content that appears is a perfect combination of the content defined in both the master and the subpage.

The application that is shown in Figure 1 uses a Master Page to define a header and footer that appear on every page. The subpage inserts content between the header and the footer by plugging a Content control into the master's ContentPlaceHolder. You should note the matching ID and ContentPlaceHolderID, and the @ Master directive in the Master Page.

Figure 1 Master Pages (Close Figure 1)

Master.master
<%@ Master %>

<html>
  <body leftmargin="0" topmargin="0" rightmargin="0"
    bottommargin="0" marginheight="0" marginwidth="0">
    <!-- Banner -->
    <table cellspacing="0" cellpadding="0"
      style="background-image: url('images/stripes.gif');
      background-repeat: repeat-x, repeat-y" width="100%">
      <tr><td align="center">
        <span style="font-family: verdana; font-size: 36pt;
          font-weight: bold; color: white">Master Pages</span><br>
        <span style="font-family: verdana; font-size: 10pt;
          font-weight: normal; color: white">This banner provided by
          Master.master</span>
      </td></tr>
    </table>

    <!-- Placeholder for content between banner and footer -->
      <asp:ContentPlaceHolder ID="Main" RunAt="server" />
    </form>

    <!-- Footer -->
    <table width="100%"><tr><td align="center">
      <span style="font-family: verdana; font-size: 8pt; color: red">
        Copyright (c) 2004 by Me Inc. All rights reserved<br>
        This footer provided by Master.master
      </span>
    </td></tr></table>
  </body>
</html>
Subpage.aspx
<%@ Page MasterPageFile="~/Master.master" %>

<asp:Content ContentPlaceHolderID="Main" RunAt="server">
  <table width="100%" height="256px"><tr><td align="center">
    <h2>This content provided by Subpage.aspx</h2>
  </td></tr></table>
</asp:Content>

Master Pages enjoy full support in the ASP.NET object model. The System.Web.UI.Page class features a new property named Master that permits a subpage to programmatically reference its master and the controls defined therein. Master Pages can be nested and can include default content that can be overridden in subpages:

<asp:ContentPlaceHolder ID="Main" RunAt="server">
    This is default content that will appear in subpages unless
    explicitly overridden
</asp:ContentPlaceHolder>
In addition, an application can designate a default Master Page in Web.config, as shown here:
<configuration>
  <system.web>
    <pages masterPageFile="~/Foo.master" />
  </system.web>
</configuration>
Individual subpages enjoy the freedom to override the default and designate Master Pages of their own.

The icing on the cake is the Master Page support in Visual Studio® 2005. When you load a subpage, the IDE shows a grayed-out, read-only version of the content defined in the master, and a full-color, fully editable version of the content defined in the subpage. Distinguishing between the two is easy, and if you want to edit content that belongs to the Master Page, all you have to do is simply open the master in the IDE.

For a more in-depth look at Master Pages, refer to Fritz Onion's article on the subject in this issue of MSDN® Magazine.

Data Source Controls

Data binding occupies a position of prominence in ASP.NET 1.x. A few well-placed lines of data-binding code can replace reams of ASP code that query a database and use repeated calls to the Response.Write method to render the query results into HTML. Data binding is even easier in ASP.NET 2.0, often requiring no code at all thanks to the new data source controls listed in the "New Controls Planned for ASP.NET 2.0" sidebar on the following page.

The following DataSource1.aspx page uses ASP.NET 2.0 data binding to display a portion of the SQL Server Pubs database:

<html>
  <body>
    <form runat="server">
      <asp:SqlDataSource ID="Titles" RunAt="server"
        ConnectionString="server=localhost;database=pubs;Integrated
                         Security=SSPI"
        SelectCommand="select title_id, title, price from titles" />
      <asp:DataGrid DataSourceID="Titles" RunAt="server" />
    </form>
  </body>
</html>
The SqlDataSource control defines the data source and the query performed on it, and the DataGrid's DataSourceID property points to the SqlDataSource. When the page loads, the SqlDataSource control performs the query and feeds the results to the DataGrid.

Of course, data binding in the real world is rarely this simple. Suppose you want to cache the query results or parameterize database queries using items selected in other controls. The page in Figure 2 uses one SqlDataSource to populate a dropdown list with the countries listed in Northwind's Customers table, and another SqlDataSource to populate a DataGrid with a list of customers in the country selected in the dropdown list. Note the <SelectParameters> element instructing the DataGrid's SqlDataSource to get the value for @country from the dropdown list. Also note the EnableCaching and CacheDuration attributes in the SqlDataSource bound to the dropdown list. These declarations cache the results of the SELECT DISTINCT query for 60 seconds.

Figure 2 DataSource2.aspx (Close Figure 2)
<html>
  <body>
    <form runat="server">
      <asp:SqlDataSource ID="Countries" RunAt="server"
        ConnectionString="server=localhost;database=northwind;
          Integrated Security=SSPI"
        SelectCommand="select distinct country from customers order by
                      country"
        EnableCaching="true" CacheDuration="60" />
      <asp:SqlDataSource ID="Customers" RunAt="server"
          ConnectionString="server=localhost;database=northwind;
            Integrated Security=SSPI"
          SelectCommand="select * from customers where country=@country">
        <SelectParameters>
          <asp:ControlParameter Name="Country"
            ControlID="MyDropDownList"
            PropertyName="SelectedValue" />
        </SelectParameters>
      </asp:SqlDataSource>
      <asp:DropDownList ID="MyDropDownList" DataSourceID="Countries"
        DataTextField="country" AutoPostBack="true" RunAt="server" />
      <asp:DataGrid DataSourceID="Customers" RunAt="server" />
    </form>
  </body>
</html>

These examples merely scratch the surface of what's possible with data source controls. For example, you can use stored procedures, parameterize queries using values extracted from query strings, user input, session state, and cookies, and you can specify whether the control should use a DataSet or DataReader. Because data source controls subsume the functionality of data adapters, you can even update databases using data source controls. Expect to see a lot written about data source controls as ASP.NET 2.0 nears final release. Dino Esposito covers them in much greater detail in his article in this issue of MSDN Magazine.

While I'm on the subject of data binding, you should also know that ASP.NET 2.0 supports a simplified data-binding syntax. ASP.NET developers can find expressions like this one imposing:

<%# DataBinder.Eval (Container.DataItem, "title") %>
In ASP.NET 2.0, the same expression can be written like this:
<%# Eval("title") %>
In addition to the Eval operator, ASP.NET 2.0 supports operators named XPath and XPathSelect that use XPath expressions to target data in XML documents.

Themes and Skins

ASP.NET pages can be pretty boring if you don't use attributes to dress up controls. The problem is that up to now, attributes had to be applied one at a time and you weren't able to "theme" controls by setting their visual attributes as a group. Enter themes and skins, a new feature of ASP.NET 2.0 that simplifies the task of building great-looking pages.

To see themes and skins in action, add the following directive to DataSource2.aspx, shown in Figure 2:

<%@ Page Theme="BasicBlue" %>
Now refresh the page. The result is shown in Figure 3.

Figure 3 Themes and Skins in Action
Figure 3 Themes and Skins in Action

Next, give the page a completely different look by adding a <head runat="server" /> element and changing the @ Page directive to indicate a page theme to use:

<%@ Page Theme="SmokeAndGlass" %>
The @ Page directive's new Theme attribute declaratively applies a theme to a page. Themes can also be applied programmatically using the Page class's Theme property. A theme is a collection of skins, and a skin is a set of visual attributes applied to a control type. BasicBlue and SmokeAndGlass are two of several predefined, or global, themes provided with ASP.NET 2.0. You'll find them in subdirectories underneath Microsoft.NET\Framework\...\ ASP.NETClientFiles\Themes.

You can define themes and skins of your own and privately deploy them in subdirectories stemming from an application's Themes directory. Each subdirectory constitutes a theme, and the theme name equals the subdirectory name. A theme subdirectory contains one or more .skin files as well as any other resources used by the theme, such as image files and style sheets. The actual skin definitions are contained in the .skin files and look very much like the tags used to declare control instances in ASPX files.

To demonstrate this, create a subdirectory named Themes in the folder where DataSource2.aspx is located. In the Themes directory, create a subdirectory named ShockingPink. Copy the following to a new .skin file in the ShockingPink directory:

<asp:DropDownList runat="server" BackColor="hotpink"
  ForeColor="white" />

<asp:DataGrid runat="server" BackColor="#CCCCCC" BorderWidth="2pt"
  BorderStyle="Solid" BorderColor="#CCCCCC" GridLines="Vertical"
  HorizontalAlign="Left">
  <HeaderStyle ForeColor="white" BackColor="hotpink" />
  <ItemStyle ForeColor="black" BackColor="white" />
  <AlternatingItemStyle BackColor="pink" ForeColor="black" />
</asp:DataGrid>
Next, change the @ Page directive in DataSource2.aspx to read:
<%@ Page Theme="ShockingPink" %>
The result of these actions is undoubtedly one of the most garish Web pages ever created!

ShockingPink.skin defines a default look for DropDownList and DataGrid controls. Skin files don't have to have the same names as the themes they belong to, although they often do. A theme can contain multiple .skin files, and a single .skin file can define attributes for any number of control types. Additionally, ASP.NET distinguishes between default skins and non-default skins. A skin that lacks a SkinID attribute is, by definition, a default skin. Non-default skins include SkinIDs that can be referenced with SkinID attributes in control tags.

New Controls

ASP.NET 2.0 will introduce about 50 new control types to help you build rich UIs while insulating you from the vagaries of HTML, client-side script, and browser Document Object Models (DOMs). The "New Controls" sidebar lists the new controls planned at the time of this writing, excluding Web Parts controls.

The DynamicImage control simplifies the task of displaying dynamically generated images in Web pages. In the past, developers often wrote custom HTTP handlers for dynamic image generation, or worse, generated images in ASPX files. DynamicImage renders both techniques obsolete. The code in Figure 4 uses a DynamicImage control to draw a pie chart. The key statement is the one that assigns the image bits to the control's ImageBytes array.

Figure 4 DynamicImage.aspx (Close Figure 4)
<%@ Import Namespace="System.Drawing" %>
<%@ Import Namespace="System.Drawing.Imaging" %>
<%@ Import Namespace="System.IO" %>

<html>
  <body>
    <asp:DynamicImage ID="PieChart" DynamicImageType="ImageBytes"
      RunAt="server" />
  </body>
</html>

<script language="C#" runat="server">
void Page_Load (Object sender, EventArgs e)
{
    // Create a bitmap and draw a pie chart
    Bitmap bitmap = new Bitmap (240, 180, PixelFormat.Format32bppArgb);
    Graphics g = Graphics.FromImage (bitmap);
    DrawPieChart (g, Color.White, new decimal[]
        { 100.0m, 200.0m, 300.0m, 400.0m }, 240, 180);
    g.Dispose();

    // Attach the image to the DynamicImage control
    MemoryStream stream = new MemoryStream ();
    bitmap.Save (stream, ImageFormat.Gif);
    bitmap.Dispose();
    PieChart.ImageBytes = stream.ToArray ();
}

void DrawPieChart (Graphics g, Color bkgnd, decimal[] vals,
    int width, int height)
{
    // Erase the background
    SolidBrush br = new SolidBrush (bkgnd);
    g.FillRectangle (br, 0, 0, width, height);
    br.Dispose ();

    // Create an array of brushes
    SolidBrush[] brushes = new SolidBrush[6];
    brushes[0] = new SolidBrush (Color.Red);
    brushes[1] = new SolidBrush (Color.Yellow);
    brushes[2] = new SolidBrush (Color.Blue);
    brushes[3] = new SolidBrush (Color.Cyan);
    brushes[4] = new SolidBrush (Color.Magenta);
    brushes[5] = new SolidBrush (Color.Green);

    // Sum the inputs
    decimal total = 0.0m;
    foreach (decimal val in vals)
        total += val;

    // Draw the chart
    float start = 0.0f;
    float end = 0.0f;
    decimal current = 0.0m;

    for (int i=0; i<vals.Length; i++) {
        current += vals[i];
        start = end;
        end = (float) (current / total) * 360.0f;
        g.FillPie (brushes[i % 6], 0.0f, 0.0f, width, height,
            start, end - start);
    }

    // Clean up and return
    foreach (SolidBrush brush in brushes)
        brush.Dispose ();
}
</script>

The DynamicImage control takes advantage of the new ASP.NET 2.0 image-generation service. Another way to access the image-generation service is to build images dynamically in ASIX files which are brand new in ASP.NET 2.0. The sample files accompanying this article (available on the MSDN Magazine Web site) include one named DynamicImage.asix that demonstrates the basics of ASIX files. To run it, copy DynamicImage.asix to a virtual directory on your Web server and call it up in your browser.

Another interesting and potentially very useful control debuting in ASP.NET 2.0 is the MultiView control. Paired with View controls, MultiView can be used to create pages containing multiple logical views. Only one view (the one whose index is assigned to the MultiView's ActiveViewIndex property) is displayed at a time, but you can switch views by changing the active view index. MultiViews are ideal for pages that use tabs or other controls to allow users to navigate between logical pages.

The page in Figure 5 uses a MultiView to show two different views of the Pubs database's Titles table: one rendered with a GridView and another with a DetailsView. View switching is accomplished by selecting from a dropdown list. Note the AllowPaging attribute in the <asp:DetailsView> tag permitting users to browse records in the DetailsView.

Figure 5 MultiView.aspx (Close Figure 5)
<%@ Page Theme="BasicBlue" %>

<html>
  <body>
    <form runat="server">
      <asp:SqlDataSource ID="Titles" RunAt="server"
        ConnectionString="server=localhost;database=pubs; Integrated
                          Security=SSPI"
        SelectCommand="select title_id, title, price from titles" />
      <asp:DropDownList ID="ViewType" AutoPostBack="true"
        OnSelectedIndexChanged="OnSwitchView" RunAt="server">
        <asp:ListItem Text="GridView" Selected="true" RunAt="server" />
        <asp:ListItem Text="DetailsView" RunAt="server" />
      </asp:DropDownList>
      <asp:MultiView ID="Main" ActiveViewIndex="0" RunAt="server">
        <asp:View RunAt="server">
          <asp:GridView DataSourceID="Titles" RunAt="server" />
        </asp:View>
        <asp:View RunAt="server">
          <asp:DetailsView DataSourceID="Titles" AllowPaging="true"
            RunAt="server" />
        </asp:View>
      </asp:MultiView>
    </form>
  </body>
</html>

<script language="C#" runat="server">
void Page_Load (Object sender, EventArgs e)
{
    if (IsPostBack)
        DataBind ();
}

void OnSwitchView (Object sender, EventArgs e)
{
    Main.ActiveViewIndex = ViewType.SelectedIndex;
}
</script>

The GridView and DetailsView Controls

The DataGrid is one of the most popular controls in ASP.NET, but in some ways it's a victim of its own success: it's so rich in functionality that it leaves ASP.NET developers wanting even more. The DataGrid control doesn't change much in ASP.NET 2.0, but two new controls named GridView and DetailsView offer features commonly requested in the DataGrid control, as well as adding a few surprises of their own.

GridViews render HTML tables like DataGrids do, but unlike DataGrids they can page and sort entirely on their own. GridViews also support a richer variety of column types (field types in GridView parlance) than DataGrids, and they have smarter default rendering behavior, automatically rendering Boolean values, for example, with checkboxes. GridViews are also easily paired with DetailsViews to create master-detail views. The chief deficiency of the GridView control is that, like DataGrid, it does most of its work by posting back to the server.

The page in Figure 6 combines a GridView and a DetailsView to create a simple master-detail view of the Pubs database's Titles table. SqlDataSource controls feed the controls their data and a SelectParameter in the SqlDataSource bound to the DetailsView control enables the DetailsView to display the record currently selected in the GridView. You select records by clicking the GridView's Select buttons, which are present because of the AutoGenerateSelectButton="true" attribute in the <asp:GridView> tag.

Figure 6 MasterDetail.aspx (Close Figure 6)
<%@ Page Theme="BasicBlue" %>

<html>
  <body>
    <form runat="server">
      <asp:SqlDataSource ID="Titles1" RunAt="server"
        ConnectionString="server=localhost;database=pubs;Integrated
          Security=SSPI"
        SelectCommand="select title_id, title, price from titles" />
      <asp:SqlDataSource ID="Titles2" RunAt="server"
        ConnectionString="server=localhost;database=pubs;Integrated
          Security=SSPI"
        SelectCommand="select title_id, title, price from titles where
          title_id=@title_id">
        <SelectParameters>
          <asp:ControlParameter Name="title_id" ControlID="MyGridView"
            PropertyName="SelectedValue" />
        </SelectParameters>
      </asp:SqlDataSource>
      <table><tr><td>
        <asp:GridView ID="MyGridView" DataSourceID="Titles1"
          Width="100%" RunAt="server" AutoGenerateColumns="false"
          SelectedIndex="0" AutoGenerateSelectButton="true"
          DataKeyNames="title_id">
          <Columns>
            <asp:BoundField HeaderText="Title ID"
              DataField="title_id" />
            <asp:BoundField HeaderText="Book Title" DataField="title" />
            <asp:BoundField HeaderText="Price" DataField="price"
              DataFormatString="{0:c}" NullDisplayText="TBD" />
          </Columns>
        </asp:GridView>
      </td></tr>
      <tr><td>
        <asp:DetailsView DataSourceID="Titles2" RunAt="server"
          AutoGenerateRows="false" Width="100%">
          <Fields>
            <asp:BoundField HeaderText="Title ID"
              DataField="title_id" />
            <asp:BoundField HeaderText="Book Title"
              DataField="title" />
            <asp:BoundField HeaderText="Price" DataField="price"
              DataFormatString="{0:c}" NullDisplayText="TBD" />
          </Fields>
        </asp:DetailsView>
      </td></tr></table>
    </form>
  </body>
</html>

Note the <Columns> and <Fields> elements defining field types in the GridView and DetailsView controls. These are morally equivalent to <Columns> elements in DataGrid controls. Figure 7 lists the supported field types. Of particular interest are ImageField and DropDownListField, which single handedly eliminate much of the code developers write today to include images and data-bound dropdown lists in DataGrids.

Figure 7 GridView and DetailsView Field Types (Close Figure 7)

Field Type | Description
AutoGeneratedField | Default field type
BoundField | Bound to a specific column in the data source
ButtonField | Displays a push button, image button, or link button
CheckBoxField | Displays a checkbox
CommandField | Displays buttons for selecting and editing items
DropDownListField | Displays a dropdown list
HyperLinkField | Displays a hyperlink
ImageField | Displays an image
TemplateField | Content is defined by HTML templates

What's New in Administration

Another glaring deficiency of ASP.NET 1.x that's been fixed in ASP.NET 2.0 is the complete lack of interfaces, either declarative or programmatic, for administering Web sites. In the past, changing a configuration setting meant firing up Notepad and editing Machine.config or Web.config. Not any more. ASP.NET 2.0 features a full-blown administration API that simplifies the task of reading and writing configuration settings. It also includes an administrative GUI which is displayed by requesting Webadmin.axd in your browser, as shown in Figure 8.

Figure 8 Administrative GUI
Figure 8 Administrative GUI

Though incomplete at the time of this writing, Webadmin.axd is designed to allow you to configure the various services included in ASP.NET 2.0 (such as the membership and role management services), view Web site statistics, and apply security settings.

Membership Service

One of the best new features of ASP.NET 2.0 is the new membership service, which provides an easy-to-use API for creating and managing user accounts. ASP.NET 1.x introduced forms authentication to the masses, but still required you to write a fair amount of code to do real-world forms authentication. The membership service fills the gaps in the ASP.NET 1.x forms authentication offerings and makes implementing forms authentication vastly simpler than before.

The membership API is exposed through two new classes: Membership and MembershipUser. The former contains static methods for creating users, validating users, and more. MembershipUser represents individual users and contains methods and properties for retrieving and changing passwords, fetching last-login dates, and the like. For example, the following statement takes a user name and password and returns true or false, indicating whether they're valid. It replaces calls to home-rolled methods in ASP.NET 1.x apps that validate credentials using Active Directory® or back-end databases, as shown here:

bool isValid = Membership.ValidateUser (username, password);

The next statement returns a MembershipUser object representing the user whose user name is "jeffpro":

MembershipUser user = Membership.GetUser ("jeffpro");
This statement retrieves a registered user's e-mail address (assuming an e-mail address is recorded):
string email = user.Email;

Where are user names, passwords, and other data managed by the membership service stored? Like virtually all state-management services in ASP.NET 2.0, membership is provider-based. Providers are modules that permit services to interact with physical data sources. ASP.NET 2.0 will include membership providers for Microsoft Access databases, SQL Server databases, Active Directory, and perhaps other data stores. You, or third parties, can write providers for others. Rob Howard, a program manager on the Microsoft Web Platform and Tools team, provides more detail on providers in his "Nothin' But ASP.NET" column.

By default, the membership service uses the Access provider and stores membership data in a file named AspNetDB.mdb in the application's Data subdirectory. Alternative providers can be selected through the <membership> section of Web.config. Rather than modify Web.config yourself, you can let Webadmin.axd modify it for you. The following excerpt is from Web.config after Webadmin.axd created a SQL Server™ database named WhidbeyLogin and configured the membership service to use it:

<membership defaultProvider="WhidbeyLogin">
  <providers>
    <add name="WhidbeyLogin"
      type="System.Web.Security.SqlMembershipProvider, ..."
      connectionStringName="webAdminConnection632112624221191376"
      applicationName="/Whidbey" requiresUniqueEmail="false"
      enablePasswordRetrieval="true" enablePasswordReset="false"
      requiresQuestionAndAnswer="false"
      passwordFormat="Encrypted" />
  </providers>
</membership>
The connectionStringName attribute references a connection string in the new <connectionStrings> section of Web.config. ASP.NET 2.0 will include the ability to encrypt that part of Web.config in order to safeguard database connection strings.

The uses of Webadmin.axd aren't limited to creating databases and selecting membership providers: it can also be used to create users, manage credentials, and more. Between Webadmin.axd and the membership API, both declarative and programmatic means exist for managing registered users of your site. That's a huge step forward from ASP.NET 1.x, which left the problem of credentials management largely up to you.

Login Controls

The membership service alone dramatically reduces the amount of code required to validate logins and manage users, but a new family of controls called the login controls makes forms authentication even easier. Login controls can be used with or without the membership service, but they integrate so well with that service that when login controls and membership are used together, fundamental tasks such as validating user names and passwords and e-mailing forgotten passwords can often be accomplished with zero lines of code. The "New Controls" sidebar includes a list of the login controls slated to ship with ASP.NET 2.0.

Figure 9 The Login Control
Figure 9 The Login Control

The Login control, shown in Figure 9, is the centerpiece of the family. In addition to offering a highly customizable UI, it's also capable of calling Membership.ValidateUser to validate user names and passwords. The Login control can also call FormsAuthentication.RedirectFromLoginPage to redirect users to the pages they were trying to get to when they were redirected to the login page. FormsAuthentication.RedirectFromLoginPage would then issue authentication cookies. You'll see Login and other login controls in action a little later in this article.

Role Manager

The membership service and login controls wouldn't be complete without support for role-based security. In ASP.NET 1.x, combining forms authentication with roles required you to write code to map role information onto each incoming request. The new role manager in ASP.NET 2.0, which can be used with or without the membership service, obviates the need for such code and simplifies the task of authorizing the user's access to various resources based on roles.New Controls Planned for ASP.NET 2.0

Simple Controls | Description
BulletedList | Displays bulleted lists of items
FileUpload | Permits files to be uploaded through Web pages
HiddenField | Represents hidden fields (<input type="hidden">)
ImageMap | Represents HTML image maps
Data Source Controls | Description
AccessDataSource | For data binding to Microsoft Access databases
DataSetDataSource | For data binding to non-hierarchical XML data
ObjectDataSource | For data binding to classes implemented in custom data layers
SiteMapDataSource | For data binding to XML site maps
SqlDataSource | For data binding to SQL databases using ADO.NET providers
XmlDataSource | For data binding to XML documents
Login Controls | Description
Login | Interface for logging in with user names and passwords
LoginName | Renders the names of authenticated users
LoginStatus | Interface for logging in and out
LoginView | Shows one view for authenticated users and another for unauthenticated users
ChangePassword | Interface for changing passwords
PasswordRecovery | Interface for e-mailing forgotten passwords
CreateUserWizard | Interface for creating new accounts
Rich Controls | Description
Content | Defines content for placeholders inherited from Master Pages
DetailsView | Renders individual records into HTML tables
DynamicImage | Represents dynamically generated images
FormView | Templated data-binding control with highly flexible UI
GridView | Super DataGrid; renders sets of records into HTML tables
Menu | Displays dropdown and fly-out menus
MultiView | Partitions a page into multiple logical views
SiteMapPath | Displays "cookie crumbs" showing paths to pages
TreeView | Renders hierarchical data as expandable/collapsible trees
View | Used with MultiView to define individual views
Wizard | Guides users through step-wise procedures
Mobile Controls | Description
Pager | Partitions pages for rendering to small devices
PhoneLink | Dials phone numbers on phone-enabled devices

Role management is provider based and is enabled through Web.config. The role manager exposes an API through the new Roles class, which exposes methods with names such as CreateRole, DeleteRole, and AddUserToRole. Significantly, you may never have to call these methods because Webadmin.axd is fully capable of creating roles, assigning users to roles, and so on. Once enabled, role-based security "just works" using the role information provided and URL authorization directives in Web.config files—the same URL authorizations that you're already familiar with from ASP.NET 1.x.

Now that you're acquainted with the membership service, the login controls, and the ASP.NET role manager, you might want to see an example that uses all three. The code samples that you can download for this article include a two-page application that demonstrates forms authentication in the style of Visual Studio 2005. To deploy the app and take it for a test drive, first copy PublicPage.aspx, LoginPage.aspx, and Web.config to a virtual directory on your Web server. Create a subdirectory named Secure in the virtual directory and copy ProtectedPage.aspx and the other Web.config file into it.

Fire up Webadmin.axd and configure the site to use forms authentication and the membership and role services to use the provider of your choice. Also create users named Bob and Alice and roles named Manager and Developer. Assign Bob to the role of Manager and Alice to the role of Developer. (I won't list all the steps because they'll probably change before you read this anyway. Fortunately, Webadmin.axd is reasonably intuitive, and it has wizards to guide you through the setup process.)

Next, call up PublicPage.aspx in your browser and click the "View Secret Message" button to view ProtectedPage.aspx. ASP.NET will redirect you to LoginPage.aspx, which uses a Login control to solicit a user name and password. Log in using Bob's user name and password. ProtectedPage.aspx should appear in the browser window because Bob is a Manager and the Web.config file in the Secure directory allows access to managers. Note the user name displayed by the LoginName control and the "Log out" link displayed by the LoginStatus control. Finally, close the browser, then reopen it and call up PublicPage.aspx again. Click "View Secret Message" and log in as Alice. This time you can't get to ProtectedPage.aspx because Alice isn't a manager.

I used a similar application to teach forms authentication in ASP.NET 1.x, but the 1.x version required significantly more code. The 2.0 version is remarkable for its brevity, particularly for the absence of any code to validate credentials typed into the login form or map user names to roles. If you're still not convinced, try implementing the same app in ASP.NET 1.x! Also, be sure to check out the changes made by Webadmin.axd to Web.config. Among other things, you should see a <roleManager> element enabling the role manager and perhaps designating a role management provider.

You might be curious to know whether the role manager makes a round-trip to the database where roles are stored in each and every request. The answer, fortunately, is no. It encodes roles in cookies and encrypts them for privacy. In the unlikely event that a user belongs to too many roles to fit in a cookie, the cookie contains a list of the most recently used roles and the database is only queried as a last resort.

Personalization

Another new service is personalization, which provides a ready-built solution to the problem of storing personalization settings for users of your site. Currently, such settings are typically stored in cookies, back-end databases, or a combination of the two. Regardless of where settings are stored, ASP.NET 1.x does little to help. It's up to you to set up and manage the back-end data stores and to associate personalization data using authenticated user names, cookies, or some other mechanism.

The ASP.NET 2.0 personalization service makes it easy to store per-user settings and retrieve them at will. It's based on user profiles, which you define in Web.config using the new <profile> element. The following code is from Web.config:

<profile>
  <properties>
    <add name="Theme" />
    <add name="Birthday" Type="System.DateTime" />
    <add name="LoginCount" Type="System.Int32" defaultValue="0" />
  </properties>
</profile>
It defines a profile containing three properties: a string named Theme, a DateTime value named Birthday, and an integer named LoginCount. The latter is assigned a default value of 0.

At run time, you access these properties for the current user using the page's Profile property, which refers to an instance of a dynamically compiled class containing the properties defined in the profile. For example, the following statements read the property values from the current user's profile:

string theme = Profile.Theme;
DateTime birthday = Profile.Birthday;
int logins = Profile.LoginCount;
Values can also be assigned to profile properties:
Profile.Theme = "SmokeAndGlass";
Profile.Birthday = new DateTime (1959, 9, 30);
Profile.LoginCount = Profile.LoginCount + 1;

An obvious benefit of the personalization service is strong typing. Another benefit is that personalization data is read and written on demand. Contrast that to session state, which is loaded and saved in every request whether it's used or not. But perhaps the biggest benefit of the personalization service is that you don't have to explicitly store the data anywhere; the system does it for you, and it stores the data persistently so it'll be around when you need it. Profiles don't time out as sessions do.

So where is personalization data stored? That depends. The personalization service is provider based, so you can configure it to use any of the available providers. ASP.NET 2.0 will ship with at least two personalization providers: one for Access and another for SQL Server. If you don't specify otherwise, the personalization service uses the Access provider, which by default stores personalization data locally in Data\AspNetDB.mdb. You can use a SQL Server database instead by modifying Web.config, either by hand or by using Webadmin.axd. If you don't want to store personalization data in either an Access database or a SQL Server database, you can write a provider of your own.

By default, ASP.NET uses authenticated user names to key stored personalization data, but you can configure it to support anonymous users as well. First, enable anonymous identification by adding the following statement to Web.config:

<anonymousIdentification enabled="true" />
Then add allowAnonymous="true" to the profile properties that you want to store for anonymous users:
<name="Theme" allowAnonymous="true" />
Now the Theme property will be available as a personalization setting regardless of whether callers to your site are authenticated.

By default, anonymous identification uses cookies to identify returning users. Attributes supported by <anonymousIdentification> allow these cookies to be configured in various ways. For example, you can specify the cookie name and indicate whether the cookie's contents should be encrypted. You can also configure the personalization service to use cookieless anonymous identification, whereupon it will rely on URL munging to identify returning users. There's even an autodetect option that uses cookies if the requesting browser supports them and URL munging if it doesn't.

To see personalization at work, run the Personalize.aspx sample in the download that accompanies this article. It lets each visitor to your site choose a theme, and then it records that theme and applies it each time the visitor returns. Observe that the theme is programmatically applied to the page in the page's PreInit event, which is a new event that fires even before Init.

Before you run the sample, you need to enable anonymous identification and define a profile that includes a string property named Theme. The following lines of code shows a Web.config file that performs those two tasks:

<configuration>
  <system.web>
    <anonymousIdentification enabled="true" />
    <profile>
      <properties>
        <property name="Theme" allowAnonymous="true" />
      </properties>
    </profile>
  </system.web>
</configuration>

SQL Cache Dependencies

Another feature sorely missing from ASP.NET 1.x is database cache dependencies. Items placed in the ASP.NET application cache can be keyed to other cached items and to objects in the file system, but not to database entities. ASP.NET 2.0 corrects this error-by-omission by introducing SQL cache dependencies.

SQL cache dependencies are represented by instances of the new SQLCacheDependency class. Using them is simplicity itself. The following statement inserts a DataSet named ds into the application cache and creates a dependency between the DataSet and the Northwind database's Products table:

Cache.Insert ("ProductsDataSet", ds,
    new SqlCacheDependency ("Northwind", "Products");
If the contents of the Products table changes, ASP.NET automatically removes the DataSet.

SQL cache dependencies can also be used with the ASP.NET output cache. The following directive instructs ASP.NET to cache output from the containing page until the contents of the Products table changes or 60 seconds elapse, whichever comes first:

<%@ OutputCache Duration="60" VaryByParam="None"
    SqlDependency="Northwind:Products" %>

SQL cache dependencies work with SQL Server 7.0, SQL Server 2000, and the upcoming SQL Server 2005. No preparation is required for SQL Server 2005, but SQL Server 7.0 and SQL Server 2000 databases must be configured to support SQL cache dependencies. Preparation involves creating database triggers and creating a special table which ASP.NET consults to determine whether changes have occurred. This table is periodically polled by a background thread using a configurable polling interval that defaults to five seconds. Neither the special table nor polling is required to detect changes in SQL Server 2005. In addition, SQL Server 2005 cache dependencies can be applied at the row level and SQL Server 7.0 and SQL Server 2000 cache dependencies work at the table level. You can prepare a database to support SQL cache dependencies using the Aspnet_regsqlcache.exe tool or Webadmin.axd.

New Dynamic Compilation Model

One of the many innovations introduced in ASP.NET 1.x was the ability for the system to compile your code on first access. Only pages were autocompiled, however, and auxiliary classes such as data access components had to be compiled separately.

ASP.NET 2.0 extends the dynamic compilation model so that virtually anything can be autocompiled. The bin directory is still there for backwards compatibility, but it's now complemented by directories named Code and Resources. C# and Visual Basic® files in the Code directory and RESX and RESOURCE files in the Resources directory are automatically compiled by ASP.NET and cached in system subdirectories. Furthermore, Web Services Description Language (WSDL) files dropped into the Code directory are compiled into Web service proxies, and XML Schema Definition Language (XSD) files are compiled into typed DataSets. Through Web.config, these directories can be extended to support other file types as well.

Precompiling and Deploying Without Source

Speaking of dynamic compilation, one of the most commonly asked questions regarding ASP.NET 1.x is whether it's possible to precompile pages to avoid the compilation delay that occurs the first time a page is accessed. While the question itself is something of a red herring (the delay is minimal, and the cost of the delay is amortized over the span of thousands or even millions of subsequent requests), Microsoft felt compelled to do something to alleviate developers' concerns. That "something" is the ability to precompile all the pages in an application by submitting a request for a phantom resource named precompile.axd.

But precompilation doesn't stop there. Another often-requested feature is the ability to precompile entire applications into managed assemblies that can be deployed without source code, a capability that is especially interesting in hosting scenarios. ASP.NET 2.0 includes a new command-line tool named Aspnet_compiler.exe that precompiles and deploys without source; Visual Studio 2005 will include a similar feature. The following command precompiles the application in the Web1 directory and deploys it without source code to Web2:

Aspnet_compiler -v /web1 -p c:\web1 c:\web2

Afterwards, the destination directory contains empty ASP.NET files (ASPX, ASCX, ASIX, and so on), plus copies of any static content present in the source directory such as HTML files, .config files, and image files. Deploying without source doesn't provide ironclad protection of your intellectual property since a clever ISP could still find out what makes the application tick by decompiling the generated assemblies, but it does raise the bar significantly for casual code snoopers.

New Code Separation Model

ASP.NET 1.x supports two programming models: the inline model, in which HTML and code coexist in the same ASPX file, and the codebehind model, which separates HTML into ASPX files and code into source code files (for example, C# files). ASP.NET 2.0 introduces a third model: a new form of codebehind that relies on the partial classes support in the Visual C#® and Visual Basic .NET compilers. The new codebehind fixes a nagging problem with the original: the fact that traditional codebehind classes must contain protected fields whose types and names map to the corresponding controls declared in the ASPX file.

Figure 10 shows the new codebehind model in action. Hello.aspx contains the declarative portion of the page and Hello.aspx.cs contains the code. You should take note of the CompileWith attribute in the @ Page directive. In addition, note the absence of any fields in the MyPage class providing mappings to the controls that are declared in the ASPX file. Old-style codebehind will still be supported, but the new style is the preferred programming model going forward. Not surprisingly, Visual Studio 2005 will support the new code separation model natively.

Figure 10 Codebehind Model (Close Figure 10)

Hello.aspx
<%@ Page CompileWith="Hello.aspx.cs" ClassName="MyPage" %>

<html>
  <body>
    <form runat="server">
      <asp:TextBox ID="Input" RunAt="server" />
      <asp:Button Text="Test" OnClick="OnTest" RunAt="server" />
      <asp:Label ID="Output" RunAt="server" />
    </form>
  </body>
</html>
Hello.aspx.cs
using System;

partial class MyPage
{
    void OnTest (Object sender, EventArgs e)
    {
        Output.Text = "Hello, " + Input.Text;
    }
}

Client Callback Manager

One of my favorite features in ASP.NET 2.0 is the "lightweight postback" capability offered by the new client callback manager. In the past, ASP.NET pages had to post back to the server to invoke server-side code. Postbacks are inefficient because they include all the postback data generated by the page's controls. They also force the page to refresh, resulting in an unsightly flashing.

ASP.NET 2.0 introduces a client callback manager that permits pages to call back to the server without fully posting back. Callbacks are asynchronous and are accomplished with XML-HTTP. They don't include postback data, and they don't force the page to refresh. (On the server side, the page executes as normal up to the PreRender event, but stops short of rendering any HTML.) They do require a browser that supports the XML-HTTP protocol, which generally means Microsoft Internet Explorer 5.0 or higher.

Using the client callback manager entails three steps. First, call Page.GetCallbackEventReference to obtain a reference to a function that can be called from client-side script to perform an XML-HTTP callback to the server. ASP.NET provides the function's name and implementation. Second, write a method in client-side script that will be called when the callback returns. The method name is one of the arguments passed to GetCallbackEventReference. Third, implement the ICallbackEventHandler interface in the page. The interface contains one method, RaiseCallbackEvent, which is called on the server side when a callback occurs. The string that RaiseCallbackEvent returns is returned to the method described in the second step.

The code in Figure 11 shows client callbacks in action and demonstrates one very practical use for them. The page displays a form that solicits a name and address. Type a 378xx or 379xx ZIP code into the Zip Code field and click the Autofill button, and a name appears in the City field. Significantly, the page goes back to the server to fetch the city name, but it does so using a client callback instead of a full postback. In real life, it could hit a database to convert ZIP codes into city names. Observe that the page doesn't repaint as pages tend to do when they post back to the server. Instead, the update is quick and clean!

Figure 11 Callback.aspx (Close Figure 11)
<%@ Implements Interface="System.Web.UI.ICallbackEventHandler" %>

<html>
  <body>
    <h1>Please Register</h1>
    <hr>
    <form runat="server">
      <table>
        <tr>
          <td>First Name</td>
          <td><asp:TextBox ID="FirstName" RunAt="server" /></td>
          <td></td>
        </tr>
        <tr>
          <td>Last Name</td>
          <td><asp:TextBox ID="LastName" RunAt="server" /></td>
          <td></td>
        </tr>
        <tr>
          <td>Address 1</td>
          <td><asp:TextBox ID="Address1" RunAt="server" /></td>
          <td></td>
        </tr>
        <tr>
          <td>Address 2</td>
          <td><asp:TextBox ID="Address2" RunAt="server" /></td>
          <td></td>
        </tr>
        <tr>
          <td>City</td>
          <td><asp:TextBox ID="City" RunAt="server" /></td>
          <td></td>
        </tr>
        <tr>
          <td>State</td>
          <td><asp:TextBox ID="State" RunAt="server" /></td>
          <td></td>
        </tr>
        <tr>
          <td>Zip Code</td>
          <td><asp:TextBox ID="Zip" RunAt="server" /></td>
          <td><asp:Button ID="AutofillButton" Text="Autofill"
            RunAt="server" /></td>
        </tr>
      </table>
    </form>
  </body>
</html>

<script language="javascript">
// Function called when callback returns
function __onCallbackCompleted (result, context)
{
    // Display the string returned by the server's RaiseCallbackEvent
    // method in the input field named "City"
    document.getElementById ('City').value = result;
}
</script>

<script language="C#" runat="server">
void Page_Load (Object sender, EventArgs e)
{
    // Get callback event reference (e.g., "__doCallback (...)")
    string cbref = GetCallbackEventReference (this,
        "document.getElementById ('Zip').value",
        "__onCallbackCompleted", "null", "null");

    // Wire the callback event reference to the Autofill button with
    // an onclick attribute (and add "return false" to event reference
    // to prevent a postback from occurring)
    AutofillButton.Attributes.Add ("onclick",
        cbref + "; return false;");
}

// Server-side callback event handler
string ICallbackEventHandler.RaiseCallbackEvent (string arg)
{
    if (arg.StartsWith ("378"))
        return "Oak Ridge";
    else if (arg.StartsWith ("379"))
        return "Knoxville";
    else
        return "Unknown";
}
</script>

Validation Groups

Validation controls are among the more brilliant innovations in ASP.NET 1.x. Controls such as RequiredFieldValidator and RegularExpressionValidator enable developers to do smart input validation on both the client and server without having to become experts in client-side scripting and browser DOMs. Unfortunately, version 1.x validation controls suffer from one fatal flaw in that there's no good way to group them together so that validators on one part of the page can override validators on another part of the page and allow postbacks to occur regardless of the state of the other validators.

The problem is illustrated by ValidationGroups1.aspx, which is included with the samples that you can download for this article. The page's designer intended for users to be able to fill in one group of TextBoxes and post back to the server without having to fill in the other group as well, but it doesn't work that way. Unless all the input fields are filled the validators complain, as shown in Figure 12.

Figure 12 Validation Controls in ASP.NET 1.x
Figure 12 Validation Controls in ASP.NET 1.x

The new validation-groups feature in ASP.NET 2.0 fixes this problem once and for all. Validation controls can now be grouped using the ValidationGroup property. Button controls can be assigned to groups the same way, and when all of the validators in a group are satisfied with the input, they permit postbacks to occur, provided, of course, the postbacks are generated by controls that belong to the same group as the validators. ValidationGroups2.aspx demonstrates the technique (see Figure 13). On the outside, this page is identical to ValidationGroups1.aspx. On the inside, it's very different. Now you can fill in either set of TextBoxes and post back by clicking the button in the TextBoxes' validation group.

Figure 13 ValidationGroups2.aspx (Close Figure 13)
<html>
  <body>
    <form runat="server">
      <h1>New Users</h1>
      <table>
        <tr>
          <td>User Name</td>
          <td><asp:TextBox ID="NewUserName" RunAt="server" /></td>
          <td><asp:RequiredFieldValidator ValidationGroup="NewUsers"
            ControlToValidate="NewUserName" ErrorMessage="Required"
            RunAt="server" /></td>
        </tr>
        <tr>
          <td>Password</td>
          <td><asp:TextBox ID="NewPassword1" TextMode="Password"
            RunAt="server" /></td>
          <td><asp:RequiredFieldValidator ValidationGroup="NewUsers"
            ControlToValidate="NewPassword1" ErrorMessage="Required"
            RunAt="server" /></td>
        </tr>
        <tr>
          <td>Retype Password</td>
          <td><asp:TextBox ID="NewPassword2" TextMode="Password"
            RunAt="server" /></td>
          <td><asp:RequiredFieldValidator ValidationGroup="NewUsers"
            ControlToValidate="NewPassword2" ErrorMessage="Required"
            RunAt="server" /></td>
        </tr>
        <tr>
          <td>E-Mail Address</td>
          <td><asp:TextBox ID="NewEMail" RunAt="server" /></td>
          <td><asp:RequiredFieldValidator ValidationGroup="NewUsers"
            ControlToValidate="NewEMail" ErrorMessage="Required"
            RunAt="server" /></td>
        </tr>
        <tr>
          <td></td>
          <td><asp:Button ValidationGroup="NewUsers"
            Text="Create Account" OnClick="OnCreateAccount"
            RunAt="server" /></td>
          <td></td>
        </tr>
      </table>
      <hr>
      <h1>Existing Users</h1>
      <table>
        <tr>
          <td>User Name</td>
          <td><asp:TextBox ID="UserName" RunAt="server" /></td>
          <td><asp:RequiredFieldValidator
            ValidationGroup="ExistingUsers"
            ControlToValidate="UserName" ErrorMessage="Required"
            RunAt="server" /></td>
        </tr>
        <tr>
          <td>Password</td>
          <td><asp:TextBox ID="Password" TextMode="Password"
            RunAt="server" /></td>
          <td><asp:RequiredFieldValidator
            ValidationGroup="ExistingUsers"
            ControlToValidate="Password" ErrorMessage="Required"
            RunAt="server" /></td>
        </tr>
        <tr>
          <td></td>
          <td><asp:Button ValidationGroup="ExistingUsers"
            Text="Log In" OnClick="OnLogIn" RunAt="server" /></td>
          <td></td>
        </tr>
      </table>
    </form>
  </body>
</html>

<script language="C#" runat="server">
  void OnCreateAccount (Object sender, EventArgs e) {}
  void OnLogIn (Object sender, EventArgs e) {}
</script>

Cross-page Posting

A common complaint about ASP.NET 1.x is that pages are only allowed to post back to themselves. That changes in version 2.0 with the introduction of cross-page posting. To set up cross-page posting, you designate the target page using the PostBackUrl property of the control that causes the postback to occur, as shown by the following code in PageOne.aspx:

<html>
  <body>
    <form runat="server">
      <asp:TextBox ID="Input" RunAt="server" />
      <asp:Button Text="Test" PostBackUrl="PageTwo.aspx"
        RunAt="server" />
    </form>
  </body>
</html>
When clicked, the button in PageOne.aspx posts back to PageTwo.aspx:
<html>
  <body>
    <asp:Label ID="Output" RunAt="server" />
  </body>
</html>

<script language="C#" runat="server">
void Page_Load (Object sender, EventArgs e)
{
    TextBox input = (TextBox) PreviousPage.FindControl ("Input");
    Output.Text = "Hello, " + input.Text;
}
</script>

PageTwo.aspx uses the new PreviousPage property of the Page class to acquire a reference to the originating page. A simple call to FindControl returns a reference to the TextBox declared in PageOne.aspx so the user's input can be retrieved.

By default, System.Web.UI.Page.PreviousPage returns a weakly typed reference to the page that originated the postback. If PageOne.aspx is the only page capable of posting to PageTwo.aspx, however, PageTwo.aspx could use the new @ PreviousPageType directive to gain strongly typed access to PageOne.aspx, as demonstrated in the code that follows:

<%@ PreviousPageType TypeName="ASP.PageOne.aspx" %>
  ...
Output.Text = "Hello, " + PreviousPage.Input.Text;

Conclusion

ASP.NET 2.0 includes other new features that I haven't written about. For example, a built-in site-counter service enables you to record site usage statistics and view them in Webadmin.axd or in custom GUIs of your own. The new Web Parts subsystem provides a framework for building SharePoint Server-style portals (more information on Web Parts and portals in ASP.NET 2.0 is available in Steven Smith's article in this issue of MSDN Magazine), while integrated mobile device support means you no longer have to install a separate toolkit to adapt output to PDAs and other small-footprint devices. A myriad of enhancements to existing controls make those controls more versatile than ever before in building component-based Web pages.

The time to learn about ASP.NET 2.0 is now because knowing what's coming (and what's not) is essential to planning an architecture today that will easily move forward tomorrow. Your ASP.NET 1.x applications should run without modification on version 2.0 because Microsoft has pledged that the new platform will be backward compatible with the old. But the future is ASP.NET 2.0, and that future is one of richer functionality and less code. What's not to like about that?

Jeff Prosise is a contributing editor to MSDN Magazine and the author of several books, including Programming Microsoft .NET (Microsoft Press, 2002). He's also a cofounder of Wintellect, a software consulting and education firm that specializes in Microsoft .NET.

© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


Security: Security Headaches? Take ASP.NET 2.0!

Security

Security Headaches? Take ASP.NET 2.0!

Keith Brown
This article is based on the March 2004 Community Technology Preview of ASP.NET 2.0. All information contained herein is subject to change.
This article discusses:
  • Security enhancements in ASP.NET 2.0
  • Server-side security controls
  • User and role databases
  • Cookieless forms authentication
This article uses the following technologies:
ASP.NET, Authentication
Contents
Forms Authentication Gets a Boost
Getting Started
Server-side Security Controls
Defining Roles
Password Recovery
Provider Tweaks
Control Tweaks
Programming Membership and Roles
Cookieless Forms Authentication
An Ounce of Prevention
Conclusion

N
ew security features are an important improvement in ASP.NET 2.0. These features include membership services that manage a database of user accounts, hashed passwords, a role manager for managing role membership for users, and five new server-side controls that make implementing forms authentication much easier. ASP.NET 2.0 also offers a provider model that gives you complete control over the implementation of the Membership and Role services and cookieless forms authentication. You also get easy Web-based administration for simple local and remote administration of user accounts and roles, as well as enhanced control of other non-security related settings.

Forms Authentication Gets a Boost

Forms authentication is one of the most popular features of ASP.NET 1.0 because it encapsulates best practices that were missing from many ad hoc implementations. For example, how many forms authentication implementations do you know of that protect the integrity of the cookie used to hold the client's credentials? Forms authentication not only writes the user's name into the cookie, it adds a message authentication code (a hash formed from the cookie and a secret value that only the Web server knows). This prevents a malicious client from elevating privileges or viewing another user's data by changing the name in her cookie.

If you follow the various newsgroups and list servers where .NET Web developers hang out, you'll read of people implementing the same things over and over: user databases, roles cached in cookies, controls for capturing user names and passwords, and administration tools for managing users and roles. The ASP.NET team has provided built-in solutions to virtually all of these problems. In researching the alpha build of ASP.NET 2.0, I was blown away by the sheer reduction in the amount of code that it takes to build a Web site that uses forms authentication in a manageable way.

Getting Started

You'll see how easy it is to get started using these new features as I walk you through some experiments that you can perform if you have the ASP.NET 2.0 bits, which is available to MSDN® Universal subscribers as a download.

To start, you'll need a virtual directory that points to an empty directory. You must make sure that your ASP.NET worker process has permissions to read, execute, and write to it. If you're running Windows® 2000 or Windows XP, you need to grant these permissions to the ASPNET local account, whereas under Windows Server™ 2003 you need to grant permissions to the Network Service account instead.

I'm going to be working with forms authentication, so I'll need to enable it with a web.config file. Now if I were showing you how to use ASP.NET 1.1, I'd tell you to open a text editor and start typing the XML by hand. But one of my favorite features in ASP.NET 2.0 is the interactive configuration file editor that's built right into the IIS management console, which you'll find on the "ASP.NET" tab on the property sheet for your virtual directory. Press the "Edit configuration" button to bring up the editor.

Figure 1 Configuration Editor
Figure 1 Configuration Editor

Figure 1 shows this new editor. You see I've chosen forms authentication instead of the default: Windows authentication. Do this for your own virtual directory. While you're in the configuration tool, set the default language for your Web app to C# since it'll save you some typing later. The Page Language Default setting is the first dropdown on the Application tab. After applying these changes, you'll find a web.config file in your directory with all your settings.

You'll need to register some users with the Membership service to get started, so the first page you'll be writing is one that allows you to add users. Here's a server control slated for the beta which allows you to implement this page in three lines of code:

<form runat='server'>
  <asp:createuser runat='server'/>
</form>
Since I'm using the alpha bits, though, I'll have to code this particular form by hand using the Membership class directly. For now, just use the ASPX page shown in Figure 2, and I'll discuss the Membership class later in the article. Figure 3 shows what you'll see when you point your browser to this page. Go ahead and add some users and passwords now. Your job should be much easier since this works right out of the box!

Figure 2 adduser.aspx (Close Figure 2)
<form runat='server'>
<table>
<tr><td>Name:</td><td><asp:textbox id='name' runat='server'/></td></tr>
<tr><td>Email address:</td><td><asp:textbox
  id='email' runat='server'/></td></tr>
<tr><td>Password:</td><td><asp:textbox textmode='Password'
  id='pwd1' runat='server'/></td></tr>
<tr><td>Retype password:</td><td><asp:textbox textmode='Password'
  id='pwd2' runat='server'/></td></tr>
</table>
<asp:button text='Submit' onclick='onSubmit' runat='server'/>
<p>
<asp:label id='msg' runat='server'/>
</form>

<script runat='server'>
void onSubmit(object sender, EventArgs args) {
  if (pwd1.Text.Equals(pwd2.Text)) {
    MembershipCreateStatus status;
    MembershipUser newUser = Membership.CreateUser(name.Text,
      pwd1.Text, email.Text, out status);
    msg.Text = status.ToString();
  }
  else msg.Text = "Passwords don't match, try again.";
}
</script>
Figure 3 Membership Page
Figure 3 Membership Page

Once you're finished adding users, take a close look at your virtual directory. You should see a new subdirectory called "DATA" that has a Microsoft® Access database inside. This is where the Membership and Role services store their data by default, but later I'll show how you can override the default storage mechanism to use SQL Server™ or your own custom data repository. Now it's time to use the security controls that are available in ASP.NET 2.0.

Server-side Security Controls

Figure 4 lists five new security controls available in ASP.NET 2.0. A good place to start exploring is the LoginStatus control. Start by creating a new ASPX page with this control on it. Call the new page default.aspx for simplicity:

<form runat='server'>
  <asp:loginstatus runat='server'/>
</form>

Figure 4 New Security Controls in ASP.NET 2.0 (Close Figure 4)

Control | Description
LoginStatus | Provides a button for logging in or logging out, depending on the user's current state
Login | Provides a form for collecting and validating login credentials against the Membership database
LoginName | Displays the name of the logged-in user
LoginView | Displays alternate content depending on whether the user is logged in or not, and optionally based on the roles in which the user is a member
PasswordRecovery | Provides a form for requesting a forgotten password

Point a browser to this page and you should see a Login link. If you view the source of the resulting page in your browser, you'll see that this hyperlink points to a page called login.aspx, which you've not yet written. This is yet another three-line Web page, so go ahead and create it now:

<form runat='server'>
  <asp:login runat='server'/>
</form>
If you've ever implemented forms authentication by hand, you'll appreciate these three lines of code. In the past, an equivalent implementation to perform a database lookup would have required a couple orders of magnitude more code.

Now go back to your browser and click the Login link, which should take you to the login page shown in Figure 5. Try logging in with an invalid user name or password, and notice that an appropriate default error message appears. The message doesn't give an attacker too much information. There's no opportunity for a naive developer to inadvertently send a message back to the user telling him he got the user name right, please try guessing another password!

Figure 5 Login Page
Figure 5 Login Page

Go ahead and type in a valid user name and password—one that you entered earlier through the adduser.aspx page—and you should be redirected back to the default.aspx page. Since you didn't supply any custom action to the login control, by default it simply logs you in using forms authentication, which means your browser now has an encrypted cookie holding your user name.

Now that you've been redirected back to default.aspx, do you see anything different? The login status control should now say Logout instead of Login. Because the forms authentication cookie was sent with the request, the FormsAuthenticationModule has created an authenticated user principal and associated it with the context of the request. The login status control notices this and changes to allow you to log out. Try logging out and logging back in to see this work.

Now let's add a bit more code to the default.aspx page:

<h3>User Name: <%= User.Identity.Name %>
</h3>
<h3>User Type: <%= User.GetType() %></h3>

Refresh the page and you should see the user name that you logged in with. Notice that the underlying object representing the user is of type GenericPrincipal, which is how FormsAuthenticationModule represents users. Once you turn on the Role Manager you'll see this type change, because when enabled, the new RoleManagerModule replaces the principal generated by FormsAuthentication with its own type.

Now let's add a LoginView control to default.aspx that displays alternate content depending on the user's login. The simplest way to use this control is to provide two blocks of content: one for an anonymous request (before the user has logged in), and another one for an authenticated request (after the user has logged in):

<asp:loginview runat='server'>
  <anonymoustemplate>
    <h4>If you see this, you've not yet logged in!</h4>
  </anonymoustemplate>
  <loggedintemplate>
    <h4>Welcome to my website, <asp:loginname runat='server'/>!</h4>
  </loggedintemplate>
</asp:loginview>
As you log in and log out, you should see the text in the LoginView control change as expected. It's a very simple idea, but it sure does make your code a whole lot clearer.

Defining Roles

I've crafted a simple page that allows you to add users to roles using the Role Manager, but before you can use it you'll need to enable the Role Manager for your app. Go back to the configuration tool and find the Authentication tab. Check the box that says "Role management enabled" and apply the change.

The code for the addrole.aspx page is shown in Figure 6, while Figure 7 shows what the form looks like. Put this page in your virtual directory and point your browser at it so that you can add some roles. Specify a user name (one that you added previously with the adduser.aspx form) along with a role name, and press the button to add the user to that role. The code first adds the role if it doesn't yet exist, then adds the user to the role. Under the covers, the Role Manager tracks these role mappings in the same Microsoft Access database that the Membership service used, but that's really just a coincidence. The Role Manager can store its data in SQL Server or any other store, and it doesn't have to use the same mechanism as the Membership service. The provider models for Membership and Role Manager are distinct in order to allow this.

Figure 6 addrole.aspx (Close Figure 6)
<form runat='server'>
<table>
<tr><td>Role:</td><td><asp:textbox id='role' runat='server'/></td></tr>
<tr><td>User:</td><td><asp:textbox id='user' runat='server'/></td></tr>
</table>
<asp:button text='Add user to role!' onclick='onSubmit' runat='server'/>
<p>
<asp:label id='msg' runat='server' viewstateenabled='false'/>
</form>

<script runat='server'>
void onSubmit(object sender, EventArgs args) {
  if (!Roles.RoleExists(role.Text)) {
    Roles.CreateRole(role.Text);
    msg.Text = "Created a new role.";
  }
  Roles.AddUserToRole(user.Text, role.Text);
}
</script>
Figure 7 Add Role
Figure 7 Add Role

If you've ever implemented custom roles in ASP.NET, you'll appreciate the built-in Role Manager for you no longer have to be a master of the ASP.NET HTTP pipeline in order to implement role-based security. Once you've added some roles, you can go back to default.aspx and have some fun with the LoginView control. Add another section after the <loggedintemplate/> element:

<rolegroups>
  <asp:rolegroup roles='ForumModerators'>
    <contenttemplate>
      <h4>Controls for forum moderators
          go here.</h4>
    </contenttemplate>
  </asp:rolegroup>
  <asp:rolegroup roles='Friends'>
    <contenttemplate>
      <h4>Welcome, friend!</h4>
    </contenttemplate>
  </asp:rolegroup>
</rolegroups>

You probably didn't choose the same roles that I did, so you'll need to replace my role names with your own, and tweak the content so it's appropriate for the role. Once you're finished, test out your new page by logging in under different user accounts in different roles, and watch how the content of the page changes as the roles change. Note that if two rolegroups match the user's role, the first one to match (from top to bottom) will always be displayed.

Although it's not new, remember that you can always test roles programmatically through User.IsInRole. Also bear in mind that you can use the <authorization/> section in web.config to grant or deny access to various pages, like so:

<authorization>
  <deny users='?'/>
  <allow roles='ForumModerators'/>
  <deny users='*'/>
</authorization>

The first entry tells ASP.NET to bounce any unauthenticated requests (force authentication). The second and third entries ensure that only ForumModerators can access the content in the directory tree where this web.config file resides. Remember that the authorization section can be used in web.config files placed in subdirectories, and can also be used in a <location/> element to control access to individual files.

Password Recovery

I haven't shown you the password recovery control in this introductory demo because its use requires careful consideration. You probably know what this control does: it lets the user request that his password be e-mailed to him. You need to do some risk assessment before deciding to e-mail cleartext passwords to users.

In fact, if you were to plunk down this control on a page in your existing site, it wouldn't operate because the Membership service refuses to disclose cleartext passwords by default. It couldn't even do so if it wanted to because by default it only stores a one-way hash of the password, not the password itself. When asked to validate a password, the Membership service hashes the presented password, then compares the hash value with its copy. If you want to recover cleartext passwords, you can reconfigure the Membership provider to store the passwords in encrypted form, in which case the provider will use the <machineKey/> to encrypt the password. Thus it can be decrypted and e-mailed to the user.

If you store hashed passwords, which is a really good idea, you need to prepare an alternate way of authenticating the user. You cannot e-mail a user his password, but if you've asked some questions ahead of time, such as "What is your favorite pet's name?" you can use the answers to authenticate the user and allow him to send you a new password. While the Membership service does support keeping a question and answer for each user, it's used only to decide whether to e-mail the password, so it can't be used with hashed passwords. This area could use some work, in my opinion.

A good model for password resets via question and answer was proposed by Viega and McGraw on page 95 of Building Secure Software (Addison-Wesley, 2002). It entails using a collection of hundreds of questions, and randomly picking a set of questions to ask the user when she first sets up her account. You can then ask the user a subset of these questions if she requests a password reset. This requires her to answer a number of the challenge questions correctly to proceed. If the user is successful, you choose a new set of random questions to ask to replace the questions previously used.

Provider Tweaks

Thus far, I purposely used default settings to keep it simple, but you'll need to tweak these settings for your own environment. For example, if you want the Membership service to store its data in SQL Server, you should select the AspNetSqlProvider instead of AspNetAccessProvider—the default. This setting is on the Authentication page of the configuration tool.

But what if you already have an existing user database that you need to integrate? It certainly won't have the tables and columns that the AspNetSqlProvider needs. Furthermore, what if it's on an AS/400 server or an Oracle installation? Fortunately, both the Membership and Role Manager systems are built on a layered model that I've shown in Figure 8. You can completely replace the Membership data store by extending the abstract MembershipProvider class, which is defined in the System.Web.Security namespace. Similarly, you can replace the Role Manager data store by extending RoleProvider. Rob Howard discusses the provider model in greater detail in his "'Nothin' But ASP.NET" column.

Figure 8 Provider Model
Figure 8 Provider Model

Certainly it's easiest to use the existing providers. In the alpha version, there are two. One works with an Access database and, as you've seen, it works fine right out of the box. The second is the SQL Server provider that I mentioned earlier. By the beta release there should also be a Membership provider that validates users against Active Directory®, and a Role provider that looks up roles from Authorization Manager.

Even if you choose one of the built-in providers, you can tweak its behavior in web.config. Figure 9 shows the provider settings for the SQL Server Membership provider. Note the passwordFormat setting where you can choose from three options: Hashed (the default), Encrypted, and Clear. Then you can choose your policy for password retrieval with the enablePasswordRetrieval and requiresQuestionAndAnswer attributes. Of course, if you choose to use hashed passwords, you'll have to set enablePasswordRetrieval to false. Otherwise, you can require the user to answer a challenge question before the system will e-mail his password.

Figure 9 Provider Settings
Figure 9 Provider Settings

The connection string for the database isn't stored in your web.config file; rather, it's referenced indirectly. Note that the property is called connectionStringName and points to a machine.config section specifically designed to hold connection strings. Keeping connection strings out of your web.config file is a good idea, especially if you can't use integrated authentication and are forced to use passwords. ASP.NET 2.0 is slated to support XML encryption for sensitive sections of configuration files—a really handy feature for the connection string section in machine.config.

The Role Manager can be configured to use cookies or URL munging and can cache roles in the cookie to reduce round-trips to the role database. The caching is intelligent: if the number of cached roles starts to get too large, the Role Manager will cache the most recently used roles in the cookie and dynamically look up the least-used roles. This feature was probably motivated by the need to support mobile devices with limited storage capacity.

There are a number of other settings you can tweak, but I'll leave you to explore them on your own. In the meantime, let's look at ways you can tweak the server-side security controls used earlier.

Control Tweaks

It's neat to be able to create a login page with three lines of code, but generally speaking, you'll want to customize the login control a bit to suit your application. Figure 10 shows some code you can use to replace the simple login page created earlier. In addition, the appearance of these controls can be modified with all the properties you'd expect on a Web control. And with the themes support in ASP.NET 2.0, you can maintain a consistent appearance across your entire Web site without having to change your code.

Figure 10 Customizing Login Control (Close Figure 10)
<form runat='server'>
  <asp:login runat='server'
    titletext='ACME.COM Security'
    instructiontext='You must log in to continue'
    usernamelabeltext='Email:'
    submitbuttontext='Continue'
    createusertext='register'
    createuserurl='adduser.aspx'
    displayrememberme='false'
    failuretext='Leave now, hacker!'
    helppagetext='help'
    helppageurl='http://www.success.com'
  />
</form>

One interesting feature of the login control is that it doesn't have to be stuck on its own page as I've done in this example. Rather, you can make it part of your master page, so it always shows up in the margin. Once the user logs in, you don't really want to see it anymore, so by default it disappears when it detects that an authenticated user is already present. You can tweak this behavior with the VisibleWhenLoggedIn property. This is an example of the features developers have been implementing manually with ASP.NET 1.1, and which are now built into ASP.NET 2.0.

The other controls have similar options. For example, if you would rather display a pretty button for the user to log in or out, set the Login(Out)ImageUrl properties on the login status control.

To get a feel for how it all works, use the Visual Studio® 2005 project wizard to create an Internet Web site. As of this writing, this wizard only shows up if you've imported the "Web.vssettings" IDE settings file into Visual Studio. You can do this via the Tools-Import/Export Settings dialog. The wizard includes all of the features I've talked about so far and allows plenty of UI customization to get the look and functionality you crave in your new Web site.

Programming Membership and Roles

While you're likely to get a lot of mileage out of the server-side security controls, it's good to know that the classes used to implement this high-level functionality are also available to you directly. There are two main classes you'll want to examine to learn the programming model for these services: Membership and Roles. I don't have room for an exhaustive treatment of them here and the details are sure to change as the product moves toward a final release, but let me give you a taste of what you'll find.

From the Membership class, you can create and manage users, each of whom is represented by an instance of the MembershipUser class. This class represents a user profile that includes properties such as Email, CreationDate, PasswordQuestion, and so on. When you create and update these user profiles, you do so via the Membership class which, due to its layered model, hides the details of where and how the profiles are stored (see Figure 8). Methods for changing the user's password and resetting the password to a random, computer-generated password are provided on this class, as is a timestamp that tracks the user's activity in order to maintain a count of current users (you can get this number by calling the GetNumberOfUsersOnline method on the Membership class).

To validate a user's password, just call ValidateUser on the Membership class, passing in the user name and password. The underlying provider will take care of all necessary password hashing or decryption. If a user forgets his user name, you can remind him by asking for an e-mail address and passing it to GetUserNameByEmail but that's not a secure option.

Cookieless Forms Authentication

One of the biggest complaints I get when I teach ASP.NET forms authentication is that it requires cookies. Fortunately, this restriction is gone in ASP.NET 2.0. There's a new "cookieless" attribute on the <forms/> element in web.config. You can set this to one of four values: UseCookies, UseUri, UseDeviceProfile, or AutoDetect.

UseCookies and UseUri force FormsAuthenticationModule to either use cookies or URL munging, respectively, for all requests. UseDeviceProfile says to look at the browser capabilities to determine which mode to use. Finally, AutoDetect will attempt to set a cookie and, if it fails, will use URL munging instead. Here's what a typical URL will look like once it's munged (the ellipses are mine since these URLs can get very long): http://www.acme.com/foo/(F(Cvc...A1))/default.aspx.

The parenthesized section of the URL contains the data the cookie would normally contain, and is stripped off by a module in the HTTP pipeline, so if you read the Request.Path property from an ASPX page, you won't see any of that extra goo in the URL. If you redirect the request, the URL will be munged automatically. In other words, this code will bring you (correctly) back to the same page you're currently looking at, with the URL correctly munged:

Response.Redirect(Request.Path)

This feature should make forms authentication considerably more widely implemented. However, as more Web sites use ASP.NET forms authentication, more attackers will attempt to find weaknesses, so it's a good idea to follow some ground rules.

An Ounce of Prevention

Forms authentication isn't very strong if it's not protected by secure sockets layer (SSL). At the very least, your login page should be sent to the user and shipped back to the Web server over a secure connection to prevent an eavesdropper from sniffing the user's cleartext password. But that's usually not enough. An eavesdropper who steals a forms authentication cookie has stolen the login because due to the way cookies work, there's no way to implement replay detection. Remember that cookies are normally sent with every request, even for things as simple as a request for a GIF file for a button on the page. Once stolen, an attacker can use the cookie to impersonate the user. To mitigate this risk you either need to shorten the cookie timeout drastically or run entire sections of the Web site (or better yet, the whole darn thing) over SSL.

I prefer the latter approach for sites that require high security, and when people complain that SSL is slow, I ask them why they haven't bought hardware to accelerate it. Some companies will insist on using SSL for only part of the site, however. If this is your situation, you can mitigate these cookie replay attacks by turning on the requireSSL attribute in the <forms/> element. This adds the "Secure" attribute to the forms authentication cookie, which indicates that the browser should only send the cookie back to the server over a secure channel. In other words, it won't be sent with requests that don't run over SSL. This feature was added in version 1.1 of the .NET Framework so it's not unique to ASP.NET 2.0. What is new in ASP.NET 2.0 is that this countermeasure can also be applied to session cookies:

<httpCookies requireSSL='true' httpOnlyCookies='true'/>

Since secure cookies won't be sent with requests that don't come over SSL, for pages that are accessible via raw HTTP, you can bet that User.Identity.IsAuthenticated will return false every time. In other words, you won't know who the user is on any pages that don't run over SSL. Note that even if you decide to run your entire site over SSL, it's a really good idea to turn on requireSSL, in case you accidentally allow access to a file or two over raw HTTP.

The httpOnlyCookies attribute is useful as a measure against cross-site scripting attacks; it indicates to the browser that the cookie should not be accessible from scripts. This uses a cookie attribute called HttpOnly that is currently only recognized by modern versions of Internet Explorer, but it's a great idea that I hope other browser vendors will adopt. To learn more, see Some Bad News and Some Good News.

Conclusion

ASP.NET 2.0 provides significant security advantages on Web sites that use forms authentication. By providing a user profile repository with support for roles, forms authentication will move beyond the purview of the ASP.NET internals guru to become much more widely implemented. It's going to be tough for me to go back and use the old stuff now!

Keith Brown is an independent consultant specializing in application security. Keith regularly teaches for DevelopMentor, where he oversees the security curriculum. He authored Programming Windows Security (Addison-Wesley, 2000) and is writing a new security book on .NET. Read it online at http://www.pluralsight.com/keith.


Portals: Unleash Your Site's Potential with Web Parts and Personalization in ASP.NET 2.0

Portals

Unleash Your Site's Potential with Web Parts and Personalization in ASP.NET 2.0

Steven A. Smith
This article is based on the March 2004 Community Technology Preview of ASP.NET 2.0. All information contained herein is subject to change.
This article discusses:
  • Easy portal development in ASP.NET 2.0
  • Quick customization techniques
  • Creating Web Parts
  • Personalization and other data stores
  • Credentials and personalization
This article uses the following technologies:
Visual Basic, .NET Framework
Contents
The Solution
The Five Minute Portal Application
Portal Features
Creating a Web Part
Security Controls
Customize the Portal
Personalize the Portal
Configuring Personalization
Storing and Retrieving Profile Data
Anonymous Personalization
Personalization Versus Other State Stores
Personalization Providers
Custom Providers
Conclusion

I
magine that you need to build an intranet site that allows everyone in your organization to view reports and other information based on their login. Imagine also that each user must be able to personalize the site, adding and removing modules that interest them and customizing pieces like weather or news sources to suit their locale and interests. Now think about how you would do this with ASP.NET 1.1 and Visual Studio® .NET 2003, and without any third-party software or tools. How much time do you think it would take? How many lines of code would you have to write? The answers to both of those questions almost certainly represent more time and effort than you or your client would like to spend. ASP.NET 2.0 and Visual Studio 2005 will address this problem with personalized portal solutions.

The Solution

Although the release of ASP.NET 2.0 is still quite a few months away, a preview was released at the PDC 2003. Despite being a very early example of what will eventually ship, this preview includes a powerful IDE and many exciting features, including support for personalized portals (a follow-up Community Technology Preview was released in March 2004 and was used to create the samples for this article). What's more, using many of these new features will require little or no additional code since all of the necessary settings and properties are configured declaratively. With ASP.NET 2.0, it will be possible to solve the problem I just described in a matter of hours (minutes in some cases) writing little more than a few lines of code—if any.

Before I tell you how ASP.NET 2.0 makes this possible, let me address two concerns that seem to always come up when developers first hear of this. The first concern is, "Oh no, there goes my job," which is understandable until you realize that there is still plenty of work left for the developer to do to add value to Web applications or address client needs. ASP.NET 2.0 helps you provide your clients with more value because you're spending less time doing the same repetitive tasks these common solutions call for. This way, you get to spend more time on the fun stuff! The second concern tends to be "Oh no, if I can't write the code, it won't be flexible enough to meet my needs." Fortunately, you still can write code to extend these features however you want—you just won't need to write as much, especially considering the default or typical usage of each feature.

The Five Minute Portal Application

If you have Visual Studio 2005 installed and you haven't already done this, I encourage you to follow along with this next task. I'm going to build a portal with all of the features I've laid out and it will only take about five minutes. (It might take you a bit longer if you're following along, but you'll be amazed at how little work is actually required.)

Figure 1 New Web Site Wizard
Figure 1 New Web Site Wizard

First, launch Visual Studio 2005 and select File | New Web Site. This will launch the New Web Site Wizard, which features several options for each language, as shown in Figure 1. Select the ASP.NET intranet site and name it "MSDN_Portal" in the default location. Notice that there is no need to set up an IIS Web or virtual directory or to specify an HTTP address. Upon clicking the OK button, the basic site is created, as shown in Figure 2. Notice that when the portal is browsed, the Visual Web Developer Web Server launches on a random port. This Web server comes with Visual Studio 2005, eliminating the need for IIS to be running on the developer's machine. For security reasons, only requests from the local computer are handled by this server.

Figure 2 Basic Home Page (Close Figure 2)

Believe it or not, you're finished! This application includes everything you need. As you can see, the site includes my login information (in this case, the Administrator account in the ISENGARD domain), which it gets from its Windows® authentication features. The site layout is encapsulated in a separate file, so that new pages added to the site can easily share the same appearance. Finally, the different sections of the homepage can be rearranged and customized by the user (and each user can modify them and persist them individually). Each of these sections, like Welcome, Announcements, and My Weather, is a Web Part. If you've used SharePoint™ Portal Server, you're probably familiar with Web Parts. Web Parts will be built into ASP.NET 2.0.

Portal Features

Out of the box, this portal provides a number of features that would require a great deal of code using ASP.NET 1.x. These include built-in authorization and authentication, per-user customization, common layout, dynamic menus based on an XML sitemap file, and more. The most exciting of these features for a portal site has to be the Web Parts and the customization they allow, so let's look at these in more detail.

Web Parts allow individual users to customize a page. Pages that support such customization simply expose a link that lets the user put the page into Edit Mode. Once in Edit mode, the user can modify the settings of individual Web Parts, as well as add, remove, and rearrange Web Parts by dragging and dropping them on the page. Figure 3 illustrates the drag and drop functionality within the portal, along with customized titles and weather information for the default portal application I just built. Take a look at the transparent "Kent OH Weather" Web Part that is being dragged down below "Steve's Links" in the bottom-right.

Figure 3 Customizing a Page (Close Figure 3)

Once the user has finished modifying the Web Parts and their locations on the page, clicking the End Personalization link returns the site to its normal state.

From the developer's perspective, adding Web Parts to an existing page is simple because they're just a special type of user control which are typically hosted within a WebPartZone control. Each page that includes Web Parts also needs a WebPartManager control. The WebPartZone control helps lay out Web Parts on the page and controls their layout, appearance, and colors. The portal that is shown in Figure 3 includes two WebPartZone controls, the Left Zone and the Right Zone.

The WebPartManager control is required on any page that uses Web Parts and is responsible for a lot of the plumbing required to make Web Parts work. It handles events, binding, communication between Web Parts, and calling the proper methods on the Web Parts on the page to ensure that their control tree is built and that they render properly. This control has no user interface and, in the case of the default portal, is simply declared as:

   <asp:webpartmanager id="Webpartmanager1"
      runat="server"></asp:webpartmanager>
The WebPartManager is also used to toggle the page between Edit mode and Normal mode, using its SetDisplayMode method.

Creating a Web Part

Writing your own Web Parts is easy. There are actually three different ways you can do it. If you only need the Web Part for one page, simply add a ContentWebPart Web control to the page and add your content to the control's ContentTemplate. Typically, though, you'll want to build Web Parts that can be reused on multiple pages. This can be accomplished either with a user control or with a custom control that inherits from System.Web.UI.WebPart. For an example of a Web Part that uses the custom control approach, take a look at the WeatherWebPart source code in the /src folder of the portal application.

Most of the time you'll probably create Web Parts from user controls. To demonstrate this, I'll create a "Hello, User" Web Part that can be customized to display any name the user chooses. First, create a new .ascx file, with content of "Hello," and a Label control for the name. Create a local variable, _yourName, as a String. Set the default value of _yourName to "YourName," then create a public property called YourName that gets and sets the value of _yourName. Add the Personalizable and WebBrowsable attributes to the class. Next, register the user control on the containing page (in Visual Studio .NET, drag it onto the page's design surface). Place the user control inside a WebPartZone control and then view the page in a browser.

To change the text of the Web Part, change the page to Edit mode and edit the Web Part. YourName should be listed as one of the custom settings for this control. Change the value of YourName to one of your liking and switch back to Normal view mode.

You should see your greeting, personalized to suit you. Imagine thousands of users being able to customize any individual part of your Web site and you'll have some idea of the potential this holds. The final code for the user control is shown in Figure 4.

Figure 4 User Control (Close Figure 4)
<%@ control language="VB" classname="Greeting"%>
<script runat="server">
    Private _yourName As String = "YourName"

    <Personalizable()> <WebBrowsable()> _
    Public Property YourName() As String
        Get
            Return _yourName
        End Get
        Set(ByVal Value As String)
            _yourName = value
        End Set
    End Property

    Sub Page_PreRender(ByVal sender As Object, _
             ByVal e As System.EventArgs)
        UserNameLabel.Text = YourName
    End Sub
</script>
Hello <asp:Label ID="UserNameLabel" Runat="Server" />

Security Controls

In addition to Web Parts, the portal also features zero-code authentication through the use of the new built-in security controls. These controls provide functionality to allow users to log in, view their login status, recover their password, or simply display their logged-in user name. All of these functions required custom code in ASP.NET 1.x, but now these common tasks are performed easily and in a very customizable fashion with little or no code required. Keith Brown covers these controls in greater depth in his article in this issue of MSDN® Magazine, but Figure 5 lists the controls and briefly describes their function. All of these security controls have support for skins, allowing them to be easily customized to fit within the look of a particular application or page.

Figure 5 Built-in Security Controls (Close Figure 5)

Control | Description
CreateUserWizard | Displays a user registration form wizard
Login | Displays a login name and password dialog, including support for remembering the user's login (via a cookie)
LoginName | Displays the name of the currently authenticated user
LoginStatus | Displays Log In or Log Out, as appropriate based on the user's status
LoginView | Allows separate content to be displayed, based on the user's authentication
PasswordRecovery | Provides the user with a wizard used to have a forgotten password sent and/or reset

Customize the Portal

There are, of course, many things left to be done to customize the sample portal for use by a real application. First, the look of the site and, at the very least, the My Company Name title in the header needs to be updated. Modifying the general layout and appearance of the default portal is easily accomplished by opening the Site.master file. ASP.NET 2.0 supports Master Pages, which provide visual inheritance. Pages can inherit their main appearance from a Master Page. Master Pages can inherit from other Master Pages. Modifications to a Master Page take effect immediately on every page that uses that Master Page.

Two of the best features of this implementation of visual inheritance are the IDE support within Visual Studio 2005 and the ease of implementation. Specifying a Master Page is as simple as adding a Master="site.master" parameter to a Page attribute, and a default application Master Page can be specified in the web.config. IDE support allows WYSIWYG editing of the site.master page and a read-only view of the Master Page layout on any page using that Master Page. Figure 6 shows the default.aspx page in design mode. Note the gray areas in the header and left navigation column. These sections are being displayed from the Master Page, and thus are not available for editing directly from the default.aspx page (although right-clicking and selecting "Edit Master" will open up the Master Page for editing). For some more details on creating Master Pages, see "Master Pages: Master Your Site Design with Visual Inheritance and Page Templates" by Fritz Onion in this issue of MSDN Magazine.

Figure 6 Default.aspx Page in Design Mode (Close Figure 6)

In addition to editing the Master Page directly to modify the look of the site, individual pages can modify the theme—another new feature in ASP.NET 2.0 that provides an easy way to define multiple different looks for a site. A theme is comprised of numerous skins, where each skin describes the look of a particular Web control. Themes and skins will, of course, be easy to create from scratch, but ASP.NET 2.0 will also ship with some global themes. Two such themes available now are BasicBlue and SmokeAndGlass. These are copied into the %SystemRoot%\InetPub\wwwroot\system_web\[version]\Themes folder when ASP.NET is installed. Adding "theme=SmokeAndGlass" to the default.aspx page <%@ Page %> directive results in quite a different look. Creating your own skins and themes is as easy as configuring the appearance of individual Web controls. In fact, it uses the exact same syntax.

One common requirement of portal systems is that users be able to personalize the look of the site by selecting a skin themselves. This capability and a great deal more are handled by the personalization features of ASP.NET 2.0.

Personalize the Portal

As mentioned before, Personalization refers to a user's ability to modify the application to suit them. Ideally, such modifications will persist, so that the user does not need to reapply his changes on each new visit to the site. You have already seen how users can modify the portal application by rearranging the Web Parts on each page. These changes are stored and kept for later visits. In addition to modifying and moving Web Parts, users may also be prompted to select a default theme for the site or for individual pages.

Personalization in ASP.NET 2.0 is very powerful and easy to use. Visual personalization is just the tip of the iceberg. Because Personalization follows the provider model, it is very easy to extend the personalization support in ASP.NET 2.0 to track all kinds of data about individual users and keep it in any data store for which a provider has been written. You can learn more about personalization techniques, including how to build support for personalization in ASP.NET 1.1, by checking out Dino Esposito's Cutting Edge column from March 2004.

ASP.NET 2.0 Personalization supports personalization of anonymous users as well as authenticated users. Access to personalization data is performed through the Profile object, available from the current HttpContext. Unlike Session and other state stores, the Profile object is strongly typed and information is only read as needed, making it both easier to develop against and more efficient. Configuring an application to use personalization and specifying which types of things are stored in the personalization engine is accomplished using a new section in the web.config file.

Configuring Personalization

In order to take advantage of the support for personalization in ASP.NET 2.0, you must first configure it. Luckily, in the case of the intranet sample Web application, this is done for you, with the user data stored in the ASPNetDB.mdb file in the /Data folder by default. To configure the personalization engine to use another data source, such as SQL Server™, you can either use the Web Administration tool or the ASP.NET SQL Server Setup Wizard.

At the time of this writing, the prerelease build of ASP.NET 2.0 that I have been using does not have the personalization section of the Web Application Administration tool implemented yet. At the moment, if you want to configure SQL Server as the provider for personalization, you need to use the aspnet_regsql.exe command-line tool. This tool is also used for Membership and setup of the planned beta feature of SQL Cache invalidation for SQL Server 2000. It can be run either in command-line mode by providing all details on the command line, or in GUI mode. Running the wizard in GUI mode is very straightforward, and after you provide it with the name and connection details for the installation of SQL Server you want to configure, the wizard will run the necessary SQL scripts in order to set up the tables and other objects needed for Personalization.

Once the data source has been set up, the next task is to configure the ASP.NET application for Personalization. This involves editing the web.config file and adding a new element within the <system.web> element, as follows:

<profile>
  <properties>
    <add name="NickName" />
    <add name="College" type="System.String" />
    <add name="BirthDate" type="System.DateTime" />
  </properties>
</profile>
Note that each <add/> subelement of <properties> in the profile defaults to the type System.String, but you can also explicitly set the type. Properties can also be arranged into groups by specifying a <group name = "GroupName">...</group> element within the <properties> element.

Storing and Retrieving Profile Data

Storing information in the current user's Profile is simple since the Profile object is now accessible directly from any ASP.NET page. Unlike other state containers, like Session and Cache, which use key/value pairs and store everything as an Object, the Profile object is strongly typed and exposes all supported properties as actual properties of the class. Thus, to set the current user's NickName property to the value passed in from a TextBox, the code would be something like this:

Profile.NickName = TextBox1.Text;

Another great thing about the Profile object is that it has full IntelliSense® statement completion support. Instead of having to flip back to web.config to see what you named a particular property, you can just type "Profile." in Visual Studio and all of its available properties and methods will be displayed in a pop-up dialog.

Retrieving data from the Profile object is as simple as adding data to it. For instance, to display the user's BirthDate in a Label, you might do this:

Label1.Text = Profile.BirthDate.ToShortDateString();
Note once again that unlike the Session or Cache collections, the Profile returns a strongly typed object, so no casting is necessary.

Anonymous Personalization

Normally, Personalization works because the user has logged into the application, so his preferences can be stored and attached to his user credentials. However, sometimes you want to track information about a particular user without requiring that he first log in. This is a common requirement for an e-commerce store, allowing the user to browse and add items to his shopping cart without having to log in, and then collecting all of the needed information when the user is ready to check out. The ASP.NET 2.0 Personalization features provide support for this kind of scenario through anonymous personalization.

Anonymous personalization is disabled by default, and thus requires a few configuration changes to set up. Anonymous identification must be enabled and then each property in the profile that will support anonymous usage must be configured explicitly. Building on my current sample, adding support for anonymous access to a shopping cart would look something like the code that is shown in Figure 7.

Figure 7 Shopping Cart Access (Close Figure 7)
<system.web>

<anonymousIdentification enabled="true" />

        <profile >
            <properties>
                <add name="NickName" type="System.String" />
                <add name="College" type="System.String" />
                <add name="BirthDate" type="System.DateTime" />
                <add name="Cart"
                   type="Msdn.ShoppingCart, Msdn.ShoppingCart"
                   serializeAs="XML" />
            </properties>
        </profile>
. . .
</system.web>

Note the serializeAs attribute, which in this case has been set to XML for the Msdn.ShoppingCart object. It defaults to String, but can also be set to XML, Binary, or ProviderSpecific to support whatever persistence method is being used.

By default, anonymousIdentification uses a cookie to track the anonymous user. However, if the user's browser does not support cookies, there are supported alternatives including an AutoDetect option (cookieless="AutoDetect") which will use cookies if supported, or store the anonymous ID in the URL otherwise. Finally, anonymous identification supports two events which can be handled to provide additional control over the process.

The AnonymousIdentification_OnCreate event is raised when the anonymous ID is created. It allows the generated anonymous ID to be overwritten in case you need to use a custom scheme for specifying the IDs of anonymous users. The AnonymousIdentification_OnRemove event is raised when an anonymous user authenticates (and thus is no longer anonymous). Its purpose is to let you clean up any data associated with the anonymous ID.

Just by configuring site settings this way, anonymous users will be able to store their personal information in their profile and have this data persist between requests and visits (from the same machine, assuming they haven't cleared their cookies). However, another necessary feature of anonymous personalization is the ability to transfer settings from the anonymous profile to an authenticated profile once the user registers or logs in. This is done by handling the Personalization_OnMigrateAnonymous event, which is raised after the AnonymousIdentification_OnRemove event. To migrate the anonymous user's shopping cart once they log in (perhaps during the checkout process), you would use something like this in your global.asax file:

void Personalization_OnMigrateAnonymous(Object sender,
    PersonalizationMigrateEventArgs e)
{
    Profile.Cart = Profile.GetProfile(e.AnonymousId).Cart;
}

Personalization Versus Other State Stores

How does the Personalization engine compare to other state storage mechanisms? The obvious comparison is between the Profile object and the Session object, since both have per-user scope. The Profile object offers several key advantages over the Session object that make it an ideal place to store user information that you know you'll need at design time.

The Profile object is strongly typed, which makes it perform better than the Session object (Session requires explicit casts whenever objects are read from it). The Profile object also makes it much easier to work within an integrated development environment, where statement completion and compile-time checking ensure that the proper values are being accessed, unlike the Session object, which uses simple string values as keys.

Profile data is persisted beyond the user's session and can be stored in any of a variety of data stores. Session state, on the other hand, only lasts until the user is inactive for some period of time. In addition, Profile data is only read when needed, and session data is requested on every user request. (Session usage can be optimized on a page-by-page basis by setting the page directive EnableSessionState to either false or readonly as needed by the page.) Session data also continues to use memory resources long after the user has stopped using the application, whereas Profile data does not cause this memory to be held onto.

For all of those negatives, Session does have some advantages. Personalization requires some configuration and setup, though only once and minimally so. Session just works by default, though it supports configuration. Session data will generally be read and written faster than Profile data, assuming it is configured using the default InProc settings, because it is simply stored in memory. However, if you're using a state server or SQL Server for session state, this advantage disappears completely.

Personalization Providers

Personalization uses Microsoft® Access by default for its data store, though it can be configured to use SQL Server as well as custom user-written providers. A provider is simply a class that handles moving the profile data from the Web application to some kind of data store, and back again. The provider design pattern is a key feature of the ASP.NET 2.0 release and can be seen in a number of features. Custom providers can be used to map the Profile data to existing Microsoft databases (rather than using the personalization SQL schema) or to reference a data store that is not supported by default, such as Oracle or XML files. Multiple providers can be used within the same application. Each property can specify a provider that is used to access it, or properties can be arranged into groups, and each group associated with a separate provider.

For example, the following configuration will store some user data using the default AspNetAccessProvider provider, and some data using the AspNetSqlProvider. Note that each of these is defined in the machine.config file, and the AspNetAccessProvider is used as the default simply because it appears first in the machine.config file. It is recommended that you have each application specify its default provider by setting it on the <profile> element, as shown in Figure 8.

Figure 8 Personalization Provider (Close Figure 8)
<profile defaultProvider="AspNetAccessProvider">
  <properties>
    <group name="Commerce">
      <add name="Cart"
        type="Msdn.ShoppingCart, Msdn.ShoppingCart"
        serializeAs="XML"
        provider="AspNetSqlProvider"
      />
    </group>
    <group name="UserInfo">
      <add name="NickName" />
      <add name="College" />
    </group>
  </properties>
</profile>

The UserInfo data in Figure 8 will be accessed using the AspNetAccessProvider, which is the default, while the Commerce data will be accessed via the AspNetSqlProvider. Even if you are only going to use a single provider for all of your Personalization settings, it can be useful to group like items together, since they are then referenced by group on the Profile object. For instance, to store something in the College property defined in Figure 8, you would use code like this:

Profile.UserInfo.College = CollegeTextBox.Text;

Custom Providers

In addition to the providers that ship with ASP.NET 2.0, you will also be able to write your own providers. This is fairly straightforward and will make some of the new features like Personalization and Membership usable by many more people than would have otherwise been possible, since many organizations will not want to have to use a new data store for this information. By writing your own provider, you have complete control over how the data is stored and retrieved for personalization. To store profile data in an alternate data store, such as an XML file on the Web server, you must create a class that inherits from the System.Web.Profile.ProfileProvider class. This base class provides some common functionality and specifies all of the properties and methods that your custom provider (XmlProfileProvider, for example) would need to implement in order to work as a provider for the Profile feature. Unfortunately, the full implementation of an XmlProfileProvider is beyond the scope of this article, but once it's been written, the following configuration snippet demonstrates how one would reference it:

<profile defaultProvider="XmlProfileProvider">
  <providers>
    <add name="XmlProfileProvider"
type="Msdn.XmlProfileProvider, XmlProfileProvider" />
  </providers>
  <properties>
    <add name="NickName" />
    <add name="College" />
  </properties>
</profile>
The type attribute in the <add> element of the <providers> element should be formatted so that it specifies the fully qualified class name of the provider, followed by the name of the assembly in which it is located (without the .dll extension), which should be in the application's /bin folder or in the Global Assembly Cache.

For information on the Provider design pattern, see Rob Howard's "Provider Model Design Pattern and Specification, Part 1,".

Conclusion

The ASP.NET 2.0 portal framework is an excellent example of how a variety of the new features can work together to provide a powerful, complete solution. The new security controls make user authentication a breeze. Master Pages and themes make it child's play to manage the appearance of all the pages in the site. Web Parts provide a powerful way to allow users to personalize the site to suit their individual needs, and the Personalization/Profile features of ASP.NET 2.0 provide an easy-to-use API for tracking details about individual users within the application. It's clear that the ASP.NET team at Microsoft has targeted this common Web application archetype with many productivity-enhancing features that work in concert to produce a whole that is more than just the mere sum of its individual (Web) parts.

Steven A. Smith is president of AspAlliance.com and DevAdvice.com, two .NET-oriented developer communities. He is co-author of the ASP.NET Developer's Cookbook (SAMS, 2003) and is working on a 2.0 edition. He is a Microsoft ASP.NET MVP, and provides training on .NET through his company, ASPSmith.com.


Data: More Load, Less Code with the Data Enhancements of ASP.NET 2.0

Data

More Load, Less Code with the Data Enhancements of ASP.NET 2.0

Dino Esposito
This article is based on the March 2004 Community Technology Preview of ASP.NET 2.0. All information contained herein is subject to change.
This article discusses:
  • New data source components in ASP.NET 2.0
  • Binding to business objects and XML data
  • GridView and DetailsView controls
This article uses the following technologies:
ASP.NET, Data, C#
Code download available at: ASPNET20Data.exe (134KB)
Contents
Data Source Components at a Glance
SQL-based Data Source Controls
Data Source Controls and Caching
Binding to Business Objects
Binding to XML Data
A Quick Look Under the Hood
Simplified Data-binding Syntax
Conclusion

T
he vast majority of Web applications consume data of some sort, and one of the most common uses of ASP.NET is to bind that data to user interface elements. ASP.NET 1.x provides extremely flexible, generic data binding optimized for performance and can give developers full control of the page lifecycle. Any collection of data that implements the IEnumerable interface (such as the DataView), or any objects that support the members of the IListSource interface (such as DataSet and DataTable), can be easily and quickly associated with server controls and used to populate a page.

Such a flexible approach works well as long as you implement simple read-only functions. More complicated read-only functions, such as paging, filtering, and sorting, require a little more attention. For example, to sort a data-bound control you need to know a lot about the page's architecture and behavior, including viewstate management and postback events. A deeper knowledge of the ASP.NET infrastructure is needed to enable data editing.

Page developers have to continually reimplement the same pattern: locate input controls and retrieve fresh data, prepare parameterized calls to SQL commands or stored procedures, execute the statement, and, finally, refresh the user interface. In many cases, this is boilerplate code, but for this very reason it may be more annoying to write than any other code.

The ASP.NET 1.x data-binding model allows binding to any data source that is enumerable, leaving you responsible for any other operations, such as insert, update, and sort. The data-binding model you find in ASP.NET 2.0 defines a second type of input format for data-bound controls. In addition to all objects that implement the IEnumerable interface, ASP.NET 2.0 data-bound controls accept data source controls, which are a brand new family of components just introduced in this version.

In this article, I'll take you on a tour of the main data source controls available in ASP.NET 2.0 and discuss their programming interface, usage, and implementation.

Data Source Components at a Glance

A data source control is basically a server control that wraps some basic functions of a particular data source (such as SQL Server™, Microsoft® Access, or XML documents). Basic functions include select, insert, update, and delete. A data source control has no UI and doesn't render any markup (note that there are a number of controls that have no UI, yet do render markup). It can be declaratively or programmatically bound to a data control. Through the services of a data source component, a data-bound control can insert new records or update and delete existing ones along with fetching data from a specific data source. The interface to the control is always the same regardless of the particular data source you're using, be it a SQL Server database, an XML document, a Microsoft Excel worksheet, or a sitemap description.

The data source object is self describing and lets the bound control know about the supported functionalities of the underlying data source. In this way, the control can easily adapt its own user interface based on the capabilities of the data source to which it's connected. For example, a grid component can show an Edit column only if the underlying data source is editable.

Data source controls are by far the most important change in the ASP.NET 2.0 data-binding model. In ASP.NET 2.0, data source controls are the recommended tool for performing data binding. It is important to note that data source controls work side by side with objects that expose the IEnumerable interface. In no way do data source controls replace DataView and arrays in data-driven applications. Also, backward compatibility is guaranteed. With data source controls, you now have an alternative way of binding data to any new and existing data-bound controls. Data source controls don't present any performance problems; performance is about the same as it was in version 1.x or even slightly better.

ASP.NET 2.0 adds a new property to all data-bound controls so that each control can be successfully bound to a data source control. The new property, DataSourceId, should be set to the name of a data source control defined in the same page. The following code snippet shows how to populate a DataGrid control with the results of a query against a SQL Server database:

<asp:SqlDataSource runat="server" ID="MySource"
    ConnectionString="...;"
    DataSourceMode="DataSet"
    SelectCommand="..." />

<asp:DataGrid runat="server" ID="data"
    DataSourceId="MySource" />

SqlDataSource is the data source control that exposes the contents of a SQL relational database. (It is important to note that SqlDataSource is not specific to SQL Server, but more on that later.) The ConnectionString property identifies the source database while the SelectCommand property is set to the query string. As mentioned earlier, you can pass data to a data-bound control using either the classic DataSource property or the new DataSourceId property. Note that the two properties are mutually exclusive. If you set both, an exception will be thrown.

What do you gain by using data source controls over classic enumerable objects? First of all, you can declare your data source in the .aspx page using a simple tag. This enables automatic instantiation of the data source object and reduces the amount of code you need to write to completely set up the page. No longer do you need to explicitly manipulate objects like SqlConnection and SqlCommand. Data sources reduce your dependence on server data components (connection, adapters, typed DataSets) as these components are heavily reliant on the code-generation capabilities of Visual Studio® .NET. In Visual Studio 2005, the amount of auto-generated code insulated in don't-change-this-code regions diminishes significantly. This doesn't mean that design-time features are being dropped, but quite the opposite. Data source controls promote a direct and implicit binding between controls and data components. This architecture allows the development of smart designers that permit dynamic discovery of schema and data and, thus, a more accurate representation of the runtime appearance of the data-bound control. Figure 1 shows the design-time SqlDataSource control and the menu used to connect to a store.

Figure 1 SqlDataSource Control at Design Time (Close Figure 1)

At least for common scenarios such as selecting, sorting, paging, deleting, and basic updating, you set up data binding by simply connecting and configuring a pair of controls. Figure 2 shows the Data tab of the Visual Studio 2005 toolbox. It contains a few data source controls and data-bound controls—the only tools you need to deal with in many situations. When this is the case, your page is virtually free of data-binding code. As the page requires more complex data-binding features, though, you would need to add little bits of code.

Figure 2 Data Tab
Figure 2 Data Tab

Data source controls enable a consistent binding model across a variety of data sources. (The controls in Figure 2 are only a subset of those that will be available by the time ASP.NET 2.0 ships.) As a page developer, you will work with the same properties regardless of whether the data source is a relational table (regardless of the database system), an XML document, a custom class, or an Excel file.

Another advantage you get by using data source controls relates to data caching. Many books and articles about ASP.NET coding strategies and optimization put caching data among the best practices for building high-performance, scalable Web applications. No doubt that the caching of data represents a critical performance enhancement, even though it is not a magic wand that works by design for all pages and applications. For example, when you manage lots of session-specific, volatile data, or your requirements dictate that fresh data must always be displayed, extensive caching can be a suboptimal approach. Rest assured that in most cases, caching data is a way to improve applications.

Data source controls integrate caching capabilities too, and turning default caching features on and off is as easy as setting a Boolean property. Caching is transparent to data-bound controls and the data source control manages some aspects of it, such as the generation of the cache key and the expiration policy (time and key dependency timestamp). Other settings are left to the decision of the page developer, including the lifetime of data in the cache. Note that data source controls maintain a separate cache for each distinct combination of connection string, select query, parameters, and cache settings.

An important side note is that some data source controls, specifically the SqlDataSource, support data cache invalidation—the ability to detect database changes and invalidate the currently cached data. As you'll see later, this feature requires specific support from the underlying SQL store.

There are clearly many advantages to using data source controls over classic enumerable objects, but let's look at some of the drawbacks. As mentioned, in ASP.NET 2.0 each data-bound control supports a dual API for data binding. These APIs are quite insulated, sharing few common points. It may appear at first that data source controls tout codeless binding and visual programming done by merely pointing and clicking. It's absolutely correct that you can create a data-driven page without writing a single line of code. However, this doesn't mean that data-bound controls don't let you hook up internal events. The new architecture is more automatic, but preserves all aspects of the explicit data-binding model used in ASP.NET 1.x.

All in all, the key difference between the two models lies in the fact that data source controls act as a proxy when it comes to performing data access. If a live data source object is essential to your application, consider that a couple of pre and post events are fired for each supported data operation (such as Deleting/Deleted events). This gives you exactly the same control over the data flow that you have in ASP.NET 1.x, but through a much simpler and more compact syntax.

Finally, bear in mind that data source controls are just a set of classes. As such, they can be instantiated and manipulated programmatically under your total control. In doing so, you use data source controls as a more abstract API working atop the raw ADO.NET classes. In a way, data source controls represent the evolution of the Data Application block, an API introduced for the .NET Framework 1.x to implement common ADO.NET best practices and reduce the code you need to write.

SQL-based Data Source Controls

ASP.NET 2.0 differentiates data source controls as tabular or hierarchical components based on the nature of the data they represent. Figure 3 lists the tabular components and hierarchical controls supported in the Community Technology Preview build of ASP.NET 2.0 released in March 2004. Let's review the data source controls in more detail starting with the one you'll probably use most often: SqlDataSource.

Figure 3 Tabular and Hierarchical Components (Close Figure 3)

Tabular Components | Description
AccessDataSource | Represents a connection to an Access database. Inherits from the SqlDataSource control but omits the ConnectionString and ProviderName properties in favor of a simpler DataFile property that will point to the MDB file. The control uses the Jet 4.0 OLE DB provider to connect to the database. If required for the MDB file, you can specify a user name and password. A ShareMode property allows you to specify whether the MDB file is read-only or read-write.
DataSetDataSource | Works with the XML representation of a DataSet object. The XML data can be specified as a string or through a file name. You can't bind this control to a DataSet object, not even programmatically. The class features methods to retrieve the corresponding DataSet object and set schema information. Although XML based, the control supports only the tabular interface and can only be bound to list controls. Mainly used to display XML data in read-only scenarios, the control also supports editing of the underlying XML data.
ObjectDataSource | Allows binding to a custom .NET business object that returns data. The class is specified by name through the TypeName property. The control allows developers to structure applications using a three-tier architecture and still take advantage of the ASP.NET 2.0 declarative data-binding model. The class is expected to follow a specific design pattern and include, for example, a parameterless constructor and methods with a well-known behavior.
SqlDataSource | Represents a connection to an ADO.NET data provider that returns SQL data, including data sources accessible through OLE DB and ODBC. The name of the provider and the connection string are specified through properties. Do not use this class to connect to an Access database.
Hierarchical Controls | Description
SiteMapDataSource | Gets data from an ASP.NET 2.0 sitemap source. The actual source depends on the site provider that the application uses. In the default case, the data source is an XML file with a .sitemap extension. The control pumps site map information into hierarchical data-bound controls such as the new TreeView control.
XmlDataSource | Loads XML input data from a file, URL, or as a string containing the XML content. If the XML data does not also contain schema information, the XML schema may be additionally specified as a separate file or string. The control can be used to populate hierarchical controls such as TreeView or Menu.

SqlDataSource is a data source control that represents a connection to a relational data store such as SQL Server, Oracle, DB2, or any data source accessible through OLE DB and ODBC bridges. For the control to work, it is essential that a .NET managed data provider exists and that the data source can return SQL resultsets. As mentioned in Figure 3, a separate data source control is available to connect to Access databases.

You set up the connection to the data store using two main properties: ConnectionString and ProviderName. The former represents the connection string to the actual data source; the latter specifies the fully qualified name of the ADO.NET managed provider class to use for the operation. The ProviderName property defaults to "System.Data.SqlClient", which means that the default data store is SQL Server. To target an Oracle database, you use the "System.Data.OracleClient" string. If you change the value of the ProviderName property, the DataSourceChanged event will be raised. Note that this event causes any controls bound to the SQLDataSource object to rebind.

The DataSourceMode property controls how the Select command retrieves data. The property decides whether a data adapter or a command object is used and accepts values defined by the SqlDataSourceMode enumeration: DataSet and DataReader. In the former case, the results of the query are cached in a DataSet object and stored on the Web server. When working in DataSet mode, the data source control supports advanced scenarios in which caching, sorting, and filtering are enabled. When the DataSourceMode property is set to DataReader, a data reader is used to read the rows one at a time in a read-only, forward-only way. The value of the DataSourceMode property has no effect on other operations such as insert, update, or delete.

The data source supports a few data operations such as insert, delete, update, and select. Figure 4 lists the properties involved with the operations. Each xxxCommand property is a string that contains the command text (or the stored procedure name) to be used. The command—with the exception of SelectCountCommand—can optionally contain parameters listed in the associated parameter collection. A parameter collection is a collection class named ParameterCollection which stores objects whose base class is Parameter. The Parameter class represents a parameter in a parameterized query, filter expression, or command executed by a data source control.

Figure 4 Data Operations (Close Figure 4)

Properties | Description
DeleteCommand, DeleteParameters | Gets or sets the SQL statement (and related parameters) used to delete rows in the underlying data store
InsertCommand, InsertParameters | Gets or sets the SQL statement (and related parameters) used to insert new rows in the underlying data store
SelectCommand, SelectParameters | Gets or sets the SQL statement (and related parameters) used to retrieve data from the underlying data store
SelectCountCommand | Gets or sets the SQL statement used to retrieve a row count from the underlying data store
UpdateCommand, UpdateParameters | Gets or sets the SQL statement (and related parameters) used to update rows in the underlying data store

The managed provider and its underlying relational engine determine the exact syntax of the SQL to use and the syntax of the embedded parameters. For example, if the data source control points to SQL Server, command parameter names must be prefixed with the @ symbol. If the target data source is an OLE DB provider or an ODBC driver, parameters are identified with a question mark symbol (?) and located by position. The code that is listed in Figure 5 shows a SQL data source that exposes the contents of the Customers database.

Figure 5 Connecting to Customers Database (Close Figure 5)
<asp:SqlDataSource runat="server" ID="MySource"
   ConnectionString="SERVER=...;DATABASE=...;Integrated Security=SSPI;"
   SelectCommand="SELECT * FROM customers"
   UpdateCommand="UPDATE customers SET country=@country"
   DeleteCommand="DELETE FROM customers WHERE customerid=@custid">
   <UpdateParameters>
     <asp:QueryStringParameter Name="country"
          QueryStringField="Country" DefaultValue="" />
   </UpdateParameters>
   <DeleteParameters>
     <asp:QueryStringParameter Name="custid"
          QueryStringField="ID" DefaultValue="" />
   </DeleteParameters>
</asp:SqlDataSource>

How a parameter is bound to a value depends on the type of the parameter. ASP.NET 2.0 supports quite a few parameter types, which (see Figure 6). The QueryStringParameter used in Figure 5 represents an input data coming from the URL query string. The DefaultValue attribute represents the default value to use if the parameter isn't specified. Other sources of input parameters are cookies, session data, form input fields, and other server controls.

Figure 6 Parameter Types (Close Figure 6)

Parameter | Description
ControlParameter | Gets the parameter value from any public property of a server control
CookieParameter | Sets the parameter value based on the content of the specified HTTP cookie
FormParameter | Gets the parameter value from the specified input field in the HTTP request form
ProfileParameter | Gets the parameter value from the specified property name in the profile object created from the application's personalization scheme
QueryStringParameter | Gets the parameter value from the specified variable in the request query string
SessionParameter | Gets the parameter value based on the content of the specified Session slot

Note that this little feature also guards against a significant security lapse. It shields you from writing bad, potentially insecure code, such as the following:

MySource.SelectCommand += TextBox1.Text
This particular line of code opens up a serious security hole in the application because it allows SQL injections to occur if the content of the TextBox control is not properly validated. The content of the SelectParameters collection (as well as the content of any other data source parameter collection) is automatically flushed into the Parameters collection of the SqlCommand object that is actually used to perform the data operation.

As mentioned, SelectCommand and other command properties are strings, not objects. Is there a way to distinguish SQL commands from stored procedures? If you want to use stored procedures, just set the command property of choice to the name of the stored procedure (assuming of course that the target database supports stored procedures). The data source control will internally parse the string, looking for the SELECT keyword. Any text that doesn't begin with SELECT is assumed to be the name of a stored procedure.

Another pair of properties that characterize the SqlDataSource control are FilterExpression and FilterParameters. A filter expression is a string that creates a filter on top of the data retrieved using the specified Select command. The syntax used for the FilterExpression property is the same as the syntax used for the RowFilter property of the DataView class and is similar to that used with the SQL WHERE clause. The FilterExpression property can also contain parameters. Filter parameters are prefixed with the @ character and enclosed in single quotation marks. FilterParameters represents the collection of parameters that are evaluated for the placeholders found in the filter expression.

To fully understand the potential of data source controls and the new data-binding model of ASP.NET 2.0, let's analyze the code in Figure 7. The page it renders is shown in Figure 8. The page comprises a GridView and a DetailsView control joined together to form a master/detail schema. The record selected in the GridView shows up in edit mode in the DetailsView and allows you to enter changes. As you can see, no Visual Basic® or C# code is needed to achieve this. It's possible because all the glue code, properly parameterized and generalized, is buried in the data source controls. The page includes two SqlDataSource controls. The first one retrieves the customers list that is then displayed by the GridView control. The second data source control retrieves a filtered view on the Customers table that the DetailsView control binds to:

<asp:sqldatasource runat="server"
   •••
  selectcommand="SELECT * FROM customers"
  filterexpression="customerid='@customerid'">
  <filterparameters>
    <asp:ControlParameter Name="customerid"
      ControlId="MyGrid"
      PropertyName="SelectedValue" />
  </filterparameters>
</asp:sqldatasource>

Figure 7 GridView and DetailsView Using Data Binding (Close Figure 7)
<%@ page language="C#" theme="SmokeAndGlass" %>
<%@ import namespace="System.Data" %>
<%@ import namespace="System.Data.SqlClient" %>

<html>
<head id="Head1" runat="server">
    <title>Edit Customers</title>
</head>
<body>
    <form id="Form1" runat="server">
        <asp:sqldatasource runat="server" id="Customers"
            connectionstring="SERVER=(local);DATABASE=northwind;
                Integrated Security=SSPI;"
            selectcommand="SELECT customerid, companyname FROM customers"
        />
        <asp:sqldatasource runat="server" id="CustomerDetails"
            connectionstring="SERVER=localhost;DATABASE=northwind;
                Integrated Security=SSPI;"
            selectcommand="SELECT * FROM customers"
            filterexpression="customerid='@customerid'">
            <filterparameters>
                <asp:ControlParameter Name="customerid"
                     ControlId="MyGrid"
                     PropertyName="SelectedValue" />
            </filterparameters>
        </asp:sqldatasource>

        <table style="border:black 1px solid;"><tr>
            <td valign="top">
                <asp:gridview runat="server" id="MyGrid"
                     datakeynames="customerid"
                     datasourceid="Customers"
                     allowpaging="true"
                     autogenerateselectbutton="true">
                     <pagersettings mode="NextPreviousFirstLast" />
                </asp:gridview>
            </td>

            <td valign="top">
                <asp:detailsview runat="server" id="MyDetail"
                     defaultmode="Edit"
                     autogeneraterows="false"
                     autogenerateeditbutton="true"
                     datasourceid="CustomerDetails">
                   <Fields>
                      <asp:boundfield datafield="companyname"
                           headertext="Company" />
                      <asp:boundfield datafield="contactname"
                           headertext="Contact" />
                      <asp:boundfield datafield="address"
                           headertext="Address" />
                      <asp:boundfield datafield="city" headertext="City"
                           />
                      <asp:boundfield datafield="country"
                           headertext="Country" />
                   </Fields>
                </asp:detailsview>
            </td>
        </tr></table>
    </form>
</body>
</html>

The parameter that restricts the query to just one customer comes from the SelectedValue property of the MyGrid control—the GridView control on the page. When the user selects a new customer on the grid, the SelectedValue property changes and the DetailsView gets automatically bound to a new record. For this to work, it is essential that you set the DataKeyNames property of the GridView to the name of the field used in the details query—customerid in this case.

Figure 8 GridView and DetailsView Controls
Figure 8 GridView and DetailsView Controls

In Figure 8, the DetailsView control is in edit mode and displays only a few of the table columns:

<asp:detailsview ...
   defaultmode="Edit"
   autogeneraterows="false"
   autogenerateeditbutton="true">
  <fields>
    •••
  </fields>
</asp:detailsview>
Through the DefaultMode property, you can decide if the view initially displays in read-only, edit, or insert mode. The AutoGenerateEditButton property adds the Edit buttons to the bottom of the details view. If the user clicks on the Update button, the control executes the update command associated with the data source control, if any. Likewise, you can enable delete and insert buttons on the DetailsView interface and have the control automatically resolve them through the command properties on the bound data source component. What really happens when a command executes on a data source control depends on the nature of the underlying data source, be it a SQL Server table, an XML document, or a managed business object.

In the code shown in Figure 7, the data source component bound to the DetailsView control lacks an UpdateCommand property. As a result, if users click the Update button, an exception is thrown. Another little wrinkle with the code in Figure 7 is that when the details view displays, no input field is automatically given focus. In ASP.NET 1.x, you had to inject script code manually to have this done. In ASP.NET 2.0, the Page class features a brand new SetFocus method that is more convenient to use to set the input focus to a particular control. In addition, each control in ASP.NET 2.0 exports a Focus method that assigns the browser focus to itself. Avoiding exceptions and setting the focus are two extra tasks that require code to be written. Let's tackle exceptions first.

As obvious as it may seem, it's worth a reminder that enabling updates without first defining a valid update command is definitely not a good programming practice. However, it is interesting to walk the stack trace at the end of which you find the exception. Data source controls wrap data operations in a pair of events that fire before and after the operation. You may think that handling the data source's Updating event would brilliantly solve the issue. In the event handler, you could check the UpdateCommand string and cancel the event if the string is empty. While theoretically correct, this approach would only work on paper. In reality, the exception I referred to earlier is thrown before the Updating event fires, but what is really happening?

When you click the Update button, the DetailsView control fires its own preliminary event, the ItemUpdating event. If the operation is not canceled, the control then determines the data source object it is bound to, and invokes the Update method on the data source control. The method checks the UpdateCommand string and, if empty, throws the exception. Only if UpdateCommand is not empty does the Update method proceed and fire the Updating event to the page. In summary, the event to hook up is the DetailsView's ItemUpdating event, like so:

void ItemUpdating(object sender, DetailsViewUpdateEventArgs e)
{
    if (CustomerDetails.UpdateCommand == string.Empty)
       e.Cancel = true;
}
To fully set up the update operation, you add the UpdateCommand property to the data source control:
<asp:sqldatasource runat="server"
   •••
  updatecommand="UPDATE customers
     SET country=@country WHERE customerid=@customerid">
</asp:sqldatasource>

It is interesting to note that you don't strictly need to define update parameters. New data-bound controls (like GridView and DetailsView) are smart enough to extract values from textboxes and other input controls and to create an ad hoc parameter list. By default, the parameter name matches the name of the bound field. For example, the @country parameter is bound to the textbox used to show the content of the country column. What about the customerid primary key field? Add the DataKeyNames attribute to the DetailsView control and you're finished:

<asp:detailsview runat="server" id="MyDetail"
   •••
   datakeynames="customerid">

A realistic scenario for update is when you want to check the result of the operation and verify that it completed successfully. In this case, you can handle the Updated event of the SqlDataSource control, as shown here:

void ItemUpdated(object sender, SqlDataSourceStatusEventArgs e)
{
   if (e.RowsAffected == 0)
   {
     Response.Write("No rows affected");
   }
}
The event argument class for the event provides a member named RowsAffected that indicates how many rows have been affected by the executed command.

To give focus to a particular textbox when the DetailsView gets into edit mode, use the following code:

void ItemCreated(object sender, EventArgs e)
{
   TextBox txt;
   txt = MyDetail.Rows[0].Cells[1].Controls[0] as TextBox;
   if (txt != null)
      SetFocus(txt.UniqueID);
}
You write an ItemCreated event handler for the DetailsView control and then retrieve the row that contains the textbox you want to consider. The Rows collection returns the collection of rows that represents the data fields. Each row is made of two cells. The first displays the header text so I'll take the second one in the code snippet just shown. The first control in this cell is the textbox unless a custom template is used. But if templates are used, things get even easier because you know the ID of the control you're looking for. Once you've got the ID of the textbox, you call the page's new SetFocus method.

Data Source Controls and Caching

A data source control retrieves data that will be made available to other components within the application. When multiple pages need to access this information, an up-to-date cache provides for a significantly faster response. The ASP.NET Cache object is the preferred place to store in-memory data. You can instruct the SqlDataSource control to cache the results of a query for a certain amount of time, but only if the data source mode is DataSet.

Caching is disabled by default in the SqlDataSource control, but you can enable it by setting the EnableCaching property to true. You should also give the CacheDuration property a nonzero value. The CacheDuration property indicates (in seconds) how long the contents of the data source are maintained in memory. The following code shows a SqlDataSource control that caches data to expire every five minutes (300 seconds):

<asp:SqlDataSource runat="server"
   ConnectionString="..."
   SelectCommand="SELECT * FROM products"
   EnableCaching="true"
   CacheDuration="300" />

An absolute expiration policy is used by default, but you can configure the control to use a sliding policy by setting CacheExpirationPolicy. A unique cache key is created for each combination of caching parameters, connection string, and the values of SelectParameters and SelectCommand. The cache key is then hashed to protect the source data used to generate it.

The SqlDataSource control also supports a form of cache expiration based on a logical dependency set between its contents and a table in a SQL Server database. The class that makes this possible is SqlCacheDependency. There are two separate implementations for the feature—one for SQL Server 2005, and one for SQL Server 2000 and SQL Server 7.0. The implementation of the two are completely independent from one another. At this time, the feature is not supported by other data relational stores you can access through the SqlDataSource control.

In SQL Server 7.0 and SQL Server 2000, the dependency is specified as a string property of the form "database:table". The database part of the string must refer to a database listed under the new <sqlCacheDependency> section of the web.config file, and the table part must be the name of a table in that database. You can specify multiple table dependencies by separating them with semicolons, as in the following example:

<asp:SqlDataSource runat="server"
   ConnectionString="..."
   SelectCommand="sp_getdata"
   SqlCacheDependency="pubs:Authors;pubs:TitleAuthor"
   EnableCaching="true"
   CacheDuration="300" />
Any change to Authors or TitleAuthor tables would automatically invalidate cached content retrieved by the data source control through the sp_getdata stored procedure. The possibility that a change in the database can invalidate some values in the cache is particularly helpful in Web farm scenarios when an update on a machine can automatically force a refresh on the data of all others.

Binding to Business Objects

Data-bound controls inherently force a direct binding between the presentation layer and the back end of a Web application. What if you encapsulate the data-retrieval logic, and optionally the business logic too, into a suite of classes? Can you connect these objects to data-bound controls or should you just consider business objects and data binding as mutually exclusive choices? The ObjectDataSource control allows you to design the application using a three-tier architecture and still benefit from the declarative data-binding model. Figure 9 shows how ObjectDataSource enables business components to associate their contents to data-bound controls.

Figure 9

The ObjectDataSource control takes the name of the class through the TypeName attribute. It also supports declarative attributes like SelectMethod, InsertMethod, UpdateMethod, and DeleteMethod, which let you specify the name of the method to use for a given operation. Each method has its own collection of parameters. Here's an example of how to use the ObjectDataSource control:

<asp:objectdatasource runat="server"
   id="MyObjectDS"
   updatemethod="Update"
   typename="Samples.Employee">
   <updateparameters>
     <asp:controlparameter name="empID"
        controlid="ddEmpList" propertyname="SelectedValue" />
     <asp:controlparameter name="address"
        controlid="txtAddress" propertyname="Text" />
     •••
   </updateparameters>
 </asp:objectdatasource>

In that code snippet, the data source control maps to a class named Samples.Employee. It is your responsibility to link the proper assembly to the application. The easiest way, but not necessarily ideal in all situations, is placing the source code of the class in the Code subdirectory of the application. Any source file deployed in that folder gets dynamically compiled to an assembly and loaded in the Web space. The class is instantiated through reflection. Note that the class used by this data source component must meet a few requirements; you can't use an arbitrary class in the .NET Framework. To bind successfully, classes must have a default, parameterless constructor, be stateless, and have methods that easily map to select, update, insert, and delete semantics. In the code just shown, the Update method is expected to have the following signature:

public void Update(string empID, string address, ...)

The name of the parameter tag must match the formal name of a parameter in the method's signature. In addition, the object must perform updates one item at a time; objects that update their state using batch operations are not supported.

The ObjectDataSource control is one of the most compelling new entries in the ASP.NET 2.0 arsenal of controls and components. It promotes the use of a strong object model in the middle tier of distributed applications and makes it easier than ever to set up a direct link between business logic and presentation. In a future article, I'll cover the ObjectDataSource control.

Binding to XML Data

There are two ways you can bind data source controls to XML data: using any XML document and using the XML representation of a DataSet object. In the former case, you use a hierarchical data source control (XmlDataSource) while in the latter, you use a tabular data source component (DataSetDataSource). The XML data can be specified either through a URL (the DataFile property) or a string (the Data property). Schema information can be provided in the same way using the SchemaFile or the Schema property.

The XmlDataSource control can be bound to both tabular and hierarchical data-bound controls. The tabular view of XML data is just a list of nodes at a given level of the hierarchy, whereas the hierarchical view shows the complete hierarchy but requires a multilevel control like TreeView, Menu, or any other control that inherits the HierarchicalDataBoundControl class.

The DataSetDataSource class, instead, can only be bound to GridView and other list controls. You can't populate DataSetDataSource with a live instance of the DataSet class. However, once you've successfully initialized the control you can retrieve the internal DataSet object using the GetDataSet method.

   <asp:DataSetDataSource ID="MySource" runat="server"
      DataFile="data.xml" />

A Quick Look Under the Hood

Data source controls inherit the base class Control and come in two flavors: tabular and hierarchical. The DataSourceControl abstract class serves as the base class for all data source controls and defines the interface between data-bound controls and the underlying data. Although the data source control has no visual rendering, it is implemented as a control to allow for "declarative persistence" (automatic instantiation during the request processing) as a native part of the .aspx source code and to gain access to the page view state.

A data source control exposes the contents of its underlying data source through a set of properties and methods. Some of these members are specific to the control, while others are common to all source controls and are defined as part of the IDataSource interface. All data source controls implement the IDataSource interface and use the interface's properties and methods to expose the bound content as a set of named views.

The IDataSource interface is simple and includes one event (DataSourceChanged) and a few methods (GetView and GetViewNames). The DataSourceChanged event is raised when the bound data source changes, such as when you change the connection string. GetView takes the name of the data source view to retrieve and returns it as a DataSourceView object. GetViewNames returns a collection of names representing the list of view objects associated with the control. As a result, the internal architecture of a data source control looks like a collection of named views.

A named view is represented with an instance of the DataSourceView class, which is similar to the ADO.NET DataView class. DataSourceView represents a customized view of data in which special settings for sorting, filtering, and other data operations have been defined. At its core, a data source control simply manages views of data loaded from the underlying data source.

The DataSourceView class is the base class for all views associated with a data source control. The number of views in a data source control depends on the connection string, the characteristics, and the actual contents of the underlying data source. Figure 10 lists the members of the DataSourceView class. Each data source control derives its own view class from DataSourceView.

Figure 10 DataSourceView Class Members (Close Figure 10)

Member | Description
CanDelete | Boolean property. Indicates whether deletions are allowed on the underlying data source. The deletion is performed by invoking the Delete method.
CanInsert | Boolean property. Indicates whether insertions are allowed on the underlying data source. The insertion is performed by invoking the Insert method.
CanPage | Boolean property. Indicates whether the underlying data source supports paging.
CanRetrieveTotalRowCount | Boolean property. Indicates whether the underlying data source can retrieve the total number of data rows instead of the data.
CanSort | Boolean property. Indicates whether the data in the view can be sorted.
CanUpdate | Boolean property. Indicates whether updates are allowed on the underlying data source. The update is performed by invoking the Update method.
Name | Readonly string property. Returns the name of the current view.
SortExpression | String property. Gets and sets the sort expression used to create a sorted view on the underlying data.
Delete | Method that performs a delete operation on the data associated with the view.
Insert | Method that performs an insert operation on the data associated with the view.
Select | Method that returns an enumerable object filled with the data contained in the underlying data store.
Update | Method that performs an update operation on the data associated with the view.

Simplified Data-binding Syntax

Admittedly, the big change in data binding is the introduction of data source controls. So what happened with the previously supported <%# ... %> data-bound expressions? The functionality remains the same in ASP.NET 2.0, but the syntax that is used has been simplified and made less verbose than it was originally in ASP.NET version 1.x.

In ASP.NET 1.x, you typically use the static method DataBinder.Eval for the late binding of data store fields to object properties, as in this example:

<%# DataBinder.Eval(Container.DataItem, fieldName, formatString) %>

The Container.DataItem expression references the object upon which the expression is evaluated. The expression is typically a string with the name of the field to access on the data item object. In general, it can be an expression that includes indexes and property names. This commonly used code can frequently be found repeated in the same form in pages. Only the expression and format string change from place to place. ASP.NET 2.0 supports a simplified syntax, as in the following code snippet:

<%# Eval(fieldName, formatString) %>
When the page is compiled for use, the Eval call is inserted in the source code of the page as a standalone call. The following code gives you an idea of what really happens:
object o = Eval(fieldName);
string result = Convert.ToString(o);

The result of the call is converted to a string and assigned to a data-bound literal control, which is an instance of the DataBoundLiteralControl class. Finally, the data-bound literal is inserted in the page's control tree. In ASP.NET 2.0, the Page class is enriched with a new protected method named Eval which determines the current data item object and calls into the original DataBinder.Eval static method.

Side-by-side with the DataBinder class, ASP.NET 2.0 provides an object capable of binding to the result of XPath expressions executed against an object that implements the IXPathNavigable interface. This class is XPathBinder and it plays the same role as DataBinder, except that it works on XML data:

<%# XPath("Orders/Order/Customer/LastName") %>

Like Eval, the XPath keyword in the expression is a new protected method on the Page class. It calls into the static method XPathBinder.Eval. Internally, XPathBinder.Eval gets a navigator object from the data source and evaluates the specified expression.

The XPathBinder class also features a Select method. The method executes an XPath query and retrieves a nodeset (an enumerable collection of XML nodes). This collection can be assigned as a late-bound value to data-bound controls such as the Repeater control. An equivalent simplified syntax exists for this scenario too:

<asp:Repeater runat="server"
     DataSource='<%# XPathSelect("orders/order/summary") %>'>
•••
</asp:Repeater>

XPathSelect is the keyword you use in data-binding expressions to indicate the results of an XPath query conducted on the container object. If the container object does not implement IXPathNavigable, an exception is thrown.

Conclusion

The advent of data source controls do not make ADO.NET objects unnecessary; they are still essential pieces of the .NET Framework, but they have been removed from prime time and pushed to the back-end infrastructure of most common data binding operations. In ASP.NET 2.0, you need to use raw ADO.NET objects much less frequently.

Data source controls make codeless binding possible, but seldom practical in real-world situations. Some code must always be written to fine tune the page, but data source controls significantly reduce the quantity. Data source controls also integrate caching capabilities and allow you to enable caching declaratively. Everything happens behind the scenes so you can deliver best-practice code with almost no effort.

Data source controls enable a consistent model across a variety of data sources. You can use the same data-binding model regardless of whether the data source is a SQL table, an XML document, a business object, a site map, or even an Excel worksheet. The new architecture is more automatic in nature, but works side-by-side with the old model based on enumerable objects.

Dino Esposito is an instructor and consultant based in Rome, Italy. Author of Programming ASP.NET and Introducing ASP.NET 2.0, he also teaches classes on ADO.NET and ASP.NET and speaks at conferences. Dino cofounded www.vb2themax.com. Get in touch with him at cutting@microsoft.com or join the blog at http://weblogs.asp.net/despos.


Master Pages: Master Your Site Design with Visual Inheritance and Page Templates

Master Pages

Master Your Site Design with Visual Inheritance and Page Templates

Fritz Onion
This article is based on the March 2004 Community Technology Preview of ASP.NET 2.0. All information contained herein is subject to change.
This article discusses:
  • Building Web page templates the old way
  • Creating Master Pages in ASP.NET 2.0
  • Master Pages implementation details
This article uses the following technologies:
ASP.NET
Code download available at: ASPNET20MasterPages.exe (135KB)
Contents
Template Design
Building Templates in Classic ASP
Building Templated Sites in ASP.NET 1.x
Master Pages
Implementation Details
Details of Usage
Conclusion

O
ver the years Web developers have struggled with a variety of techniques to "templatize" their sites, but none have proven to be a truly generic, reusable way of maintaining a standardized appearance across an entire site. Master Pages, one of the most anticipated features coming in ASP.NET 2.0, finally provide a sanctioned and elegant way of designing pages in an application based on templates.

Template Design

It is desirable in Web site design to define a standard appearance for all pages. This may include common headers, footers, and menus that provide a core set of features and appearance throughout the site. For dynamic sites, built with technologies like ASP or ASP.NET, it is extremely useful if these common features on all pages are factored out into some type of page template, allowing each page to consist only of its own unique content and providing a central location for making site-wide changes in appearance and behavior. As a simple, concrete example of a site that would benefit from some type of page template technique, see Figure 1.

Figure 1 Simple Templated Site
Figure 1 Simple Templated Site

This particular page has a header at the top, a footer at the bottom, a navigation bar on the left, and an area for page-specific content filling out the remainder of the space. Ideally, the header, footer, and navigation bar should be defined only once and somehow propagated to all pages in the site.

This is precisely the problem that Master Pages in ASP.NET 2.0 solve simply and cleanly. By defining one Master Page and then creating many Content Pages based on it, you can very easily create sites with a common appearance driven by a single template (the Master Page). Before I explore the details of Master Pages, however, it is useful to understand how developers are building templated sites today in both ASP and ASP.NET 1.x.

Building Templates in Classic ASP

Many sites that were built using ASP 3.0 use the concept of templates, typically through the use of server-side include (SSI) directives. SSI directives provide the ability to insert contents of files into specific locations within an ASP page. One common technique is to use the SSI directive to bring in frequently used elements like footers, headers, and navigation bars. Figure 2 demonstrates this practice.

Figure 2 Server-side Includes in ASP
Figure 2 Server-side Includes in ASP

While this technique is a step in the right direction, it still means that each page has to generate the surrounding layout elements and overall structure in addition to any page-specific content. Another more template-like approach taken by many sites is to specify two common include files, such as pageStart.asp and pageEnd.asp, which contain not only the common navigation bar, header, and footer, but also the surrounding layout elements and overall page structure. All that is left for an individual page to add is its specific content.

This approach provides the control many developers want in a centralized template for the entire site; however, it also has several drawbacks. First, there is no designer support for editing files that have SSI directives, so designers have to work on a merged rendering to see the full page appearance, only to have it later separated out again. Second, the include mechanism is rather simplistic in that it only inserts the contents of one file into another file prior to server-side evaluation by the ASP parser. This means that it is easy to make mistakes in matching HTML element closing tags and choosing the include locations correctly to prevent incorrect rendering. It is also difficult to customize portions of the included files. In the asp "technique2" sample available for download on the MSDN® Magazine Web site, one of the immediate drawbacks you will notice is that the title of each page is intentionally left blank. This happens because the title is specified only once in the top-level page_start include file and there is no clean way to have pages that include the template to specify what should appear in the title element. Finally, there is overhead associated with the use of include files because they increase the overall size of the ASP Script Engine Cache.

Based on these simple approaches using server-side include directives, a truly generic templating mechanism should ideally have the following five features:

  • Some facility to let the developer create portions of a page that are defined independently and reused across multiple pages
  • The ability to define the "shell" of a page which other pages then use as their template
  • The ability for pages to alter various elements within an inherited template page, such as changing the title element
  • The ability to specify alternate page templates declaratively within a page
  • The ability to actually view the rendered version of the page using its associated template
The use of server-side include files with ASP 3.0 satisfies the first of these criteria, and arguably the second up to a point, but fall flat on the last three of the features.

Building Templated Sites in ASP.NET 1.x

ASP.NET brought a completely new object-based programming model with the potential for a newer, more elegant templating mechanism. Instead of having to rely on the rather rudimentary SSI mechanism used in ASP 3.0, ASP.NET gave you a server-side representation of each page with a complete hierarchical object model of elements that are to be rendered back to the client. The feature that most closely mirrors the functionality of SSI files is the user control. The user control provides a declarative way of creating custom controls whose content and behavior can be plugged into other pages at arbitrary locations, so using them as a generic way of defining reusable portions of a page is quite straightforward, as illustrated in Figure 3.

Figure 3 Reusable Page Portions with User Controls
Figure 3 Reusable Page Portions with User Controls

In addition to acting as simple placeholders for content, user controls are also full-fledged controls that can expose properties, methods, and events. Unlike server-side include directives, you can instrument a user control with functionality that can influence its rendering on a page. For example, you could create a property on your header control called ShowBreadcrumbs that would be able to selectively turn a breadcrumb feature on or off at your discretion.

There are, however, some limitations and drawbacks to the user control model of reuse. To begin with, many developers find it cumbersome to have to add a Register directive for each user control to every page that will use them. Ironically, many ASP.NET developers have resorted to using SSI directives to import a collection of Register directives into all their pages, replacing several lines with just one SSI directive. More importantly, however, there is no built-in way to use user controls to satisfy any of the five criteria I listed earlier for developing a templating mechanism (except the first, since you can create reusable portions of pages).

ASP.NET does provide a much richer object model, and many people have built their own templating mechanism to overcome the lack of a native one in ASP.NET 1.x. With a little ingenuity and some clever coding, you can build a generic templating mechanism. Most implementations rely on the fact that you have the opportunity to manipulate the control hierarchy that is automatically constructed for you by the Page class prior to it being sent back to the client. For example, one technique is to create a common Page-derived base class which extracts the controls in the generated page hierarchy, dynamically loads a user control which serves as the page template, and then injects the extracted controls into a well-known location within the user control hierarchy. A complete implementation of this technique is included with the code download for this article.

Using a technique like this, I can come very close to achieving all five of my desired criteria for a truly useful templating mechanism. I have user controls for defining reusable portions of pages. I have a page template mechanism with which I can define a single template and apply it to any number of pages in my system. There are ways of defining "replaceable" elements within the template using this technique, and it is possible to specify alternate templates for each page. The one missing feature is designer integration, which is not truly possible without some help from Visual Studio® .NET in recognizing this templating technique.

The other disadvantage with custom template techniques like this is that there is no one sanctioned way to do it, and there is nothing built into the .NET Framework that supports it. What this means is that each site may be using a completely different mechanism for templating.

While ASP.NET gives you a much more powerful programming model that is extremely flexible, it still does not have a built-in mechanism for templating a site. In particular, while some developers have built templating mechanisms through clever use of control hierarchy replacement and user controls, they require extra configuration steps and, more importantly, have no designer support. Enter Master Pages in ASP.NET 2.0.

Master Pages

The arrival of Master Pages in ASP.NET 2.0 represents the first templating mechanism from Microsoft that meets all of the criteria I outlined earlier. They provide site-level page templates, a mechanism for fine-grained content replacement, programmatic and declarative control over which template a page should use, and perhaps most compelling of all, integrated designer support. Technically, Master Pages are implemented in much the same way that I described custom template mechanisms in ASP.NET 1.1, but add to that designer support from Visual Studio 2005 and the fact that it is now a sanctioned and supported way to construct templated sites with visual inheritance, and you finally have a complete template solution.

The implementation of Master Pages in ASP.NET 2.0 consists of two conceptual elements: Master Pages and Content Pages. Master Pages act as the templates for Content Pages, and Content Pages provide content to populate pieces of Master Pages that require "filling out." A Master Page is essentially a standard ASP.NET page except that it uses the extension of .master and a <%@ master %> directive instead of <%@ page %>. This Master Page file serves as the template for other pages, so typically it will contain the top-level HTML elements, the main form, headers, footers, and such. Within the Master Page you add instances of the ContentPlaceHolder control at locations where you want Content Pages to supply page-specific content, as shown in Figure 4.

Figure 4 Adding Placeholder Instances (Close Figure 4)
<!-- file: sitetemplate.master -->
<%@ master language="C#" %>

<html>
  <head>
    <title>
      <asp:contentplaceholder runat="server" id="_titleContent">
        Standard title
      </asp:contentplaceholder>
    </title>
  </head>
  <body>
    <form runat="server">
      <h2>Common header</h2>

      <asp:contentplaceholder  runat="server" id="_mainContent" />

      <h2>Common footer</h2>
    </form>
  </body>
</html>

In contrast, Content Pages are just ordinary .aspx files that specify an associated Master Page in their page directive using the masterpagefile attribute. These pages must contain only instances of the Content control as their sole purpose is to supply content for the inherited Master Page template. Each Content control must map to a specific ContentPlaceHolder control defined in the referenced Master Page, the contents of which will be inserted into the Master Page's placeholder at rendering time. The following Content Page provides content for the sitetemplate.master Master Page shown in Figure 4:

<!-- file: default.aspx -->
<%@ page language="C#"
    masterpagefile="sitetemplate.master" %>

<asp:content contentplaceholderid="_titleContent" runat="server">
  Main page
</asp:content>
<asp:content contentplaceholderid="_mainContent" runat="server">
  This is the content for the default page.
</asp:content>

Note that with this mechanism you can specify content to be placed at very specific locations in the Master Page template. The example just shown describes how the subtle problem of generating unique page titles with templates is solved easily by providing a ContentPlaceHolder control within the title element of the Master Page and a corresponding Content control with a matching ContentPlaceHolderId in the Content Page. This example also illustrates how Master Pages can supply default content for placeholders, so if the Content Page decides not to provide a Content control for a particular placeholder, it will have a default rendering.

With the fundamental mechanics of Master Pages in place, I can now revisit the templated example that I built earlier using other techniques. Recall that this example employs reusable page content elements (user controls in ASP.NET) to define the header, navigation bar, and footer. The template page lays out the page using these elements, and the Content Page supplies the inner content for the page. Figure 5 shows the rendering of this example using Master and Content Pages.

Figure 5 Page Rendering with Master Pages in ASP.NET 2.0 (Close Figure 5)

Figure 6 Design Support for Master Pages
Figure 6 Design Support for Master Pages

Even more compelling is the fact that Master Pages are understood by the designer in Visual Studio 2005, so that when you are visually editing a Content Page it displays the content of the inherited Master Page in a grayed out region so it is obvious what the ultimate rendering of the page will look like. Figure 6 shows my continuing example using Master Pages as it would appear when editing a Content Page that is affiliated with my Master Page.

Implementation Details

As mentioned earlier, the implementation of Master and Content Pages is quite similar to the approach taken by many developers building their own custom templating mechanism in ASP.NET 1.x. In particular, the MasterPage class derives from UserControl and thus inherits the same generic container functionality that user controls provide. Much like the custom implementation I discussed earlier, the templates defined by Master Pages are injected into the generated control hierarchy for the requested page. This injection happens just prior to the Init event of the Page class so that all of the controls will be in place prior to Init, when it is common to perform programmatic manipulation of controls.

The actual merging of the Master Page's control hierarchy and the page's control hierarchy is done in a manner similar to the approach I outlined earlier. The top-level control of the Master Page (which will be named the same as the file containing the Master Page) will be inserted as the root control in the new page hierarchy. Then the contents of each Content control in the page are injected as a collection of child controls underneath the corresponding ContentPlaceHolder control. Figure 7 shows a sample Content Page with an associated Master Page and the resulting merged control hierarchy that is created just prior to the Init event during the page processing. Note the color-coded affiliation indicating the origin of each control in the resulting hierarchy.

Figure 7 Master Page and Content Page Merged Hierarchy (Close Figure 7)

One of the implications of this implementation is that the Master Page itself is just another control in your page class's hierarchy, and you can perform any of the tasks with the Master Page directly that you are used to performing on controls. The current Master Page associated with any given page is always available via the Master property accessor. As an example of interacting with the Master Page, within the default.aspx page shown in Figure 7, you could add code to programmatically access the HtmlForm that was implicitly added by the Master Page, as shown in the following code snippet:

void Page_Load(object sender, EventArgs e)
{
  HtmlForm f = (HtmlForm)Master.FindControl("_theForm");
  if (f != null)
  {
    // use f here...
  }
}

In addition to accessing the Master Page, you can also change the Master Page affiliation of a Content Page at run time. The MasterPageFile property is exposed as a public property on the Page class and can be modified within the code for any page. Any modifications to this property must be made in a handler for the new PreInit event of the Page class for it to take effect, since the creation of and merging with the Master Page happens just prior to the firing of the Init event. The following override of the OnPreInit method could be added to any page class using a Master Page to programmatically change the Master Page affiliation:

protected override void OnPreInit(EventArgs e)
{
  this.MasterPageFile = "othertemplate.master";
  base.OnPreInit(e);
}

Details of Usage

As you begin to use Master Pages in your site design, you will run into some issues that may not have occurred before if you have never used a site-level templating mechanism. The first issue is that of relative paths in referenced resources like images or stylesheets. When you are creating a Master Page you must keep in mind that the directory from which relative paths are going to be evaluated may very well change based on the page being accessed. Consider the directory structure of a site shown in Figure 8.

Figure 8 Site Directory Structure
Figure 8 Site Directory Structure

If you were to add a reference to the Check.gif image in the images directory from the Site.master in the masterpages directory, you might be tempted to add a simple image element, as shown here:

   <img src="../images/check.gif" />

Unfortunately, this would only work for pages that were in a relative directory location similar to where the image as the Master Page was, like page1.aspx. Any other page, like default.aspx, would not correctly resolve the relative path. One solution to this is to use the root path reference syntax in ASP.NET and ensure that all relative references are made from server-side controls, which is the only place this syntax works. The image reference I mentioned earlier would become:

<img src="~/images/check.gif" runat="server" />

Another option is to rely on the fact that relative path references in server-side controls are evaluated relative to the Master Page in which they are placed. This means that it would also be sufficient to change the image reference to the following:

<img src="../images/check.gif" runat="server" />
Server-side path references in pages that reference Master Pages are still relative to the page itself, so you should not have to change any techniques you may already have in place to deal with relative references in pages.

Another common request that ASP.NET developers make when they first encounter Master Pages is that they would like to have the ability to require that all pages within an application be Content Pages that reference a specific Master Page. While no such "must-use" attribute exists, you can designate a Master Page to be used by default for all pages in an application by adding a pages element to your web.config file specifying a common Master Page, as shown in the following code snippet:

<!-- file: web.config -->
<configuration>
  <pages masterPageFile="~/sitetemplate.master" />
</configuration>

Like any settings specified at the application level, individual pages can elect to override the default masterPageFile attribute, but adding this to your configuration file will guarantee that no pages will be added to your application accidentally without an associated Master Page.

Finally, you may find that it is useful to have a "meta" Master Page—that is, a Master Page for a set of Master Pages. Master Pages support arbitrarily deep nesting, so you can create whatever level of Master Pages you decide makes sense for your application. Just like pages that have Master Pages, Master Pages that have Master Pages must consist exclusively of ContentPlaceHolder controls at the top level. Within these ContentPlaceHolder controls, Master Pages can add additional ContentPlaceHolder controls for the actual pages to use. Note that pages that reference a Master Page which itself has a Master Page can only provide content elements for ContentPlaceHolder controls on the immediate parent Master Page. There is no way to directly populate placeholders on a Master Page two or more levels up from a particular page. As an example, take a look at the Master Page definition (metatemplate.master) that is shown in the code in Figure 9.

Figure 9 Master Page Definition (Close Figure 9)
<%@ master %>

<html>
  <head>
    <title>
      <asp:contentplaceholder runat="server"
           id="_titleContent">
        Standard title
      </asp:contentplaceholder>
    </title>
  </head>
  <body>
    <form id="_theForm" runat="server">
      <h2>Header</h2>
      <asp:contentplaceholder runat="server" id="_mainContent" />
      <h2>Footer</h2>
    </form>
  </body>
</html>

You can now define another Master Page which in turn specifies that Master Page as its master and provides content elements for each of the ContentPlaceHolder controls in the parent Master Page (sitetemplate.master), as shown in Figure 10.

Figure 10 Defining a Parent Master Page (Close Figure 10)
<%@ master masterpagefile="~/metatemplate.master" %>

<asp:content runat="server"
             contentplaceholderid="_titleContent">
  <asp:contentplaceholder runat="server" id="_title">
    Default title
  </asp:contentplaceholder>
</asp:content>

<asp:content runat="server"
             contentplaceholderid="_mainContent">
  <table>
    <tr>
      <td><asp:contentplaceholder id="_leftContent"
               runat="server" /></td>
      <td><asp:contentplaceholder id="_rightContent"
               runat="server" /></td>
    </tr>
  </table>
</asp:content>

Note that in order to grant pages access to the _titleContent placeholder in the parent Master Page, I had to declare a Content control and nest a new ContentPlaceHolder control within it to give pages access to that location in the parent Master Page.

Conclusion

The addition of Master Pages in ASP.NET 2.0—the long-awaited feature of site-level templates—represents a new era of easy Web site creation for Web developers. No longer do developers have to resort to clumsy server-side include directives in ASP or complicated control hierarchy-manipulating techniques in ASP.NET. Now creating a page template that all your other Web pages can be based upon is just as easy as designing any other ordinary type of Web page. I'll bet you can't wait to get started.

Fritz Onion is an independent consultant, author, and instructor specializing in ASP.NET. He is the author of Essential ASP.NET (Addison Wesley, 2003) and is working on a second edition covering ASP.NET 2.0. He writes and teaches courses for DevelopMentor and is also a regular speaker at industry conferences.


Controls: Plan Your Migration to the Visual Studio 2005 Navigation Controls

Controls

Plan Your Migration to the Visual Studio 2005 Navigation Controls

Dave Donaldson and Steven DeWalt
This article is based on the March 2004 Community Technology Preview of ASP.NET 2.0. All information contained herein is subject to change.
This article discusses:
  • The SiteMapDataSource and SiteMapPath controls
  • The TreeView control
  • A breadcrumb navigation control for ASP.NET 1.1
This article uses the following technologies:
ASP.NET and Visual Basic .NET
Code download available at: NavigationControls.exe (145KB)
Contents
A Simple Web Site
The Web.sitemap File
The SiteMapPath Control
The SiteMapDataSource Control
The TreeView Control
A Breadcrumb Control Today
The Breadcrumb Class
Conclusion

W
hen building the UI for a Web app, one of the first items of business is designing the overall structure of the Web site, which includes all site navigation. Navigation controls are typically placed on the left or along the top of Web pages and might include the increasingly popular breadcrumb navigation (which shows the path the user has walked through), creating an overall menu system. To implement navigation, there are several technologies available, but they are often too labor-intensive for developers. Examples are DHTML, include files (for those using classic ASP), and user controls in ASP.NET. Sometimes navigation elements are hardcoded, but more often than not, navigation items are kept in a data store for read-only access at run time, whether in a database (and cached) or in XML configuration files.

In ASP.NET 2.0, most of the work required to implement site navigation is built in so you can take advantage of the rich feature set without having to write a single line of server-side code.

A Simple Web Site

To demonstrate how easily you can build a Web site using the ASP.NET 2.0 navigation controls, we've created a fictional, jack-of-all-trades computer company: Northwind Traders. The company's Web site has a menu on the left and a breadcrumb at the top-right. Looking at the design of a page in the Visual Studio® 2005 designer, shown in Figure 1, you can see that three controls are circled. Circled in blue is the SiteMapPath control, which effectively displays the common breadcrumb navigation. Circled in red is the TreeView control, currently set to display a flat view and not to expand. Circled in green is the SiteMapDataSource control, which acts as the data source for the TreeView control. Keep this picture in mind as we walk you through the new navigation controls, their properties, and other items of interest.

Figure 1 A Page in the Visual Studio 2005 Designer
Figure 1 A Page in the Visual Studio 2005 Designer

The Web.sitemap File

The key to the new navigation controls is the new web.sitemap file, shown in Figure 2. This is a standard XML file that acts as the default data store for the navigation items. When creating a new Web site in Visual Studio 2005, the web.sitemap file is not added by default so you have to add it yourself manually (this will most likely be changed before final release). To add it, right-click on the Web site and select Add New Item. Under Standard Templates, select XML File and rename it to web.sitemap (the final release will support adding a sitemap file as a unique type).

Figure 2 Web.sitemap (Close Figure 2)
<?xml version="1.0" encoding="utf-8" ?>
<siteMap>
   <siteMapNode title="Home" description="Home" url="Home.aspx" >
      <siteMapNode title="Products" description="Our products"
           url="Products.aspx">
          <siteMapNode title="Hardware" description="Hardware choices"
             url="Hardware.aspx" />
          <siteMapNode title="Software" description="Software choices"
             url="Software.aspx" />
      </siteMapNode>
      <siteMapNode title="Services" description="Services we offer"
          url="Services.aspx">
         <siteMapNode title="Training" description="Training classes"
            url="Training.aspx" />
         <siteMapNode title="Consulting"
            description="Consulting services"
              url="Consulting.aspx" />
         <siteMapNode title="Support" description="Supports plans"
              url="Support.aspx" />
      </siteMapNode>
   </siteMapNode>
</siteMap>

Once you have created the file, you need to complete it according to the schema for web.sitemap (see Figure 3), which is not at all difficult or cumbersome. Basically, you need a root element (<siteMap>) and zero or more nested <siteMapNode> elements. Note that the structure of the <siteMapNode> elements is logical and does not have to map to the physical directory structure of your Web site. For example, the files for the Northwind Traders Web site are all located in the root directory. Another thing to keep in mind is that for each <siteMapNode> element, the URL attribute must be unique within the sitemap file. Also note that in the March 2004 Technical Preview build of Visual Studio 2005, there is no IntelliSense® to work from when filling out your web.sitemap file.

Figure 3 Web.sitemap Schema (Close Figure 3)
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
   elementFormDefault="qualified">
  <xs:element name="siteMap">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="siteMapNode" maxOccurs="unbounded" />
      </xs:sequence>
    </xs:complexType>
  </xs:element>
  <xs:element name="siteMapNode">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="siteMapNode" minOccurs="0" maxOccurs="unbounded"
        />
      </xs:sequence>
      <xs:attribute name="url" type="xs:string" />
      <xs:attribute name="title" type="xs:string" />
      <xs:attribute name="description" type="xs:string" />
      <xs:attribute name="keywords" type="xs:string" />
      <xs:attribute name="roles" type="xs:string" />
      <xs:attribute name="siteMapFile" type="xs:string" />
      <xs:attribute name="Provider" type="xs:string" />
    </xs:complexType>
  </xs:element>
</xs:schema>

To make the web.sitemap file available for consumption by the navigation controls, a default site map provider is available. In the machine.config file installed Visual Studio 2005, the default site map provider is named AspNetXmlSiteMapProvider and uses the class type System.Web.XmlSiteMapProvider. The provider has an attribute that determines the default site map file. This attribute, siteMapFile, equals "web.sitemap". A new abstract base class named SiteMapProvider, from which XmlSiteMapProvider derives, has been added to the next version of the Microsoft® .NET Framework, which allows you to implement your own site map providers.

You should note that it's possible that the name of the web.sitemap file could change before Visual Studio 2005 is released. For instance, in keeping with the .config naming convention, the file might end up being called sitemap.config.

The SiteMapPath Control

Before we get into the details of this control, let's take some time to understand the importance of a breadcrumb trail in a Web site. In medium to large Web sites, it's very easy for users to quickly get lost in the maze of links, and it's often hard for users to find their way back to the beginning of their trail. This is where the breadcrumb is so important to the user experience. Breadcrumb navigation allows users to understand quickly where they are within the site without having to go back to the home page and start over.

To create this type of navigation in the next version of Visual Studio, you use the SiteMapPath control. By default, this control uses the web.sitemap file as its data store so once you've created that file and filled it in, you're already halfway finished implementing breadcrumb navigation on your Web site.

Located under the Navigation tab of the designer toolbox, the SiteMapPath control is available to drag and drop onto your Web form in the designer. Once you've done that, you'll have a basic breadcrumb trail in your Web site based on the elements in your web.sitemap file. Chances are you'll want to adjust the properties of the SiteMapPath control to fit the overall design of your site. There are too many properties of the SiteMapPath control to cover them all in depth here, but we'll describe a handful of properties you should become familiar with right away.

PathDirection The default is RootToCurrent, which displays the breadcrumb in this fashion: Home > Products > Software. The other option is CurrentToRoot, which reverses the order: Software > Products > Home.

PathSeparator This is whatever text you want to display between the nodes of the breadcrumb. Don't forget to include the leading and trailing space. For instance, to display "Home > Products > Software", you would set this property to " > ", not ">".

RenderCurrentNodeAsLink This basically determines whether or not to display the current page you are on as a link to itself in the breadcrumb. The default is False.

Each item in this next list is a set of properties to define all font and style items as well as general appearance. Each set of properties contains the same property subset.

CurrentNodeStyle Gives the current node in the breadcrumb its appearance (overrides NodeStyle).

HoverNodeStyle Determines the appearance of a breadcrumb node when you hover over it.

NodeStyle Gives all nodes in the breadcrumb a consistent style when the breadcrumb is displayed (unless it is overridden by RootNodeStyle or CurrentNodeStyle).

PathSeparatorStyle Sets the appearance for whatever text you defined in PathSeparator.

RootNodeStyle Determines the style of the root node of the breadcrumb (overrides NodeStyle).

As you can see, you have complete control over how you want your breadcrumb to behave and look, and it's all provided for you without writing a single line of server-side code. For instance, if you want users to quickly recognize the current node in the breadcrumb of your Web site, you can set the font bold property to True under CurrentNodeStyle, or maybe set the font size to Large, while at the same time, the font bold property under NodeStyle can be set to False and its font size set to Small. A quick look at the source of the home page for the Northwind Traders Web site shows how we implemented the SiteMapPath control:

<asp:sitemappath id="SiteMapPath1" runat="server" pathseparator=" -> ">
        <hovernodestyle cssclass="crumbHoverNode" />
        <nodestyle cssclass="crumbNode" />
        <pathseparatorstyle cssclass="crumbPathSeparator" />
</asp:sitemappath>

The SiteMapPath control also has a property named SiteMapProvider. When left blank (the default), the control uses the AspNetXmlSiteMapProvider mentioned earlier (or XmlSiteMapProvider), which in turn reads the web.sitemap file. If you created your own site map provider and added the entry in machine.config or web.config, this is the property you use to make that change.

The SiteMapDataSource Control

This control is very important when you need to view hierarchical data, such as common navigational structures. A SiteMapDataSource control will often be used as the data source for other controls to bind to, such as the TreeView or DropDownList controls. To use a SiteMapDataSource control, find it under the Data tab of the designer toolbox and drag it onto your Web form.

Figure 4 SiteMapDataSource Properties
Figure 4 SiteMapDataSource Properties

If you take a look at Figure 4, you'll see that the property list for a SiteMapDataSource control is not very extensive, so let's go into some detail about each of its properties.

EnableViewState If you're already using ASP.NET, this property will be familiar. It is used to turn ViewState on or off. The value is set to True by default.

FlatDepth This property specifies how deep in the hierarchical site map to retrieve nodes from (provided by the SiteMapProvider, which by default is the web.sitemap file). The default value is -1, which indicates there is no limit to the depth. A value of 0 corresponds to the root node level of the hierarchy, while a positive integer corresponds to that many levels below the root node. This is useful if you have a deep hierarchy and want to limit the display to a set number of levels.

ParentLevelsDisplayed Specifies the number of levels of parent nodes to retrieve, relative to the current node. The default is -1, which means there is no limit of parent levels retrieved.

SiteMapProvider Determines the site map provider for the control to bind to. The default is blank, which means the control uses the default site map provider discussed earlier. If you want to use your own site map provider instead of the default, you would type it in here.

SiteMapViewType Determines how the data source is viewed. The default is Tree, which depicts data in the same hierarchical structure that the site map is specified in. The other values are Flat and Path. Flat shows only a flat view of the nodes with no hierarchical structure while Path displays data as a hierarchical path between the current node and root node, in the same way a SiteMapPath control renders its site map data.

StartingDepth Specifies the depth at which the control begins retrieving <siteMapNode> elements from the web.sitemap file. The default value is -1, which indicates that there is no restriction on the depth that the data source retrieves nodes from. A value of 0 is used to indicate the root node.

StartingNodeType By default, this property is set to Root, which means the starting node is the root node of the hierarchy, but you can set the starting node to be any node. The starting point can be set to a node relative to the current position in the site map by setting this property to Parent or Current. If you use Parent, the control always begins from the parent node of the currently displayed page, unless the currently displayed page corresponds to the root node. If this is set to Current, the control always begins from the node that represents the currently displayed page.

StartingNodeUrl Instead of setting the starting node to the Root, Current, or Parent node, you can specify the URL of an element in the web.sitemap file as the starting point. Setting this property takes precedence over both the StartingDepth and StartingNodeType properties.

ID Note what the ID of the control is when using it as the data source for other controls, such as the TreeView or DropDownList controls.

A typical declaration of the SiteMapDataSource control on the Northwind Traders Web site, which uses the default values for all properties except for SiteMapViewType, looks like this:

<asp:sitemapdatasource id="SiteMapDataSource1" runat="server"
    sitemapviewtype="Flat" />
There's no limit to the number of SiteMapDataSource controls you can have. For instance, you can have a TreeView control on the left side of Web pages that binds to one SiteMapDataSource control and flat navigation on the right side that binds to another SiteMapDataSource control.

The TreeView Control

For those of you who have developed Windows®-based applications using Windows Forms, a TreeView control is a familiar item; however, using TreeViews has not been common practice in the realm of Web applications. To implement something similar to a TreeView in a Web site today, you typically have to write a large amount of DHTML on the client to get it to work properly. This proves to be very painful because you have to account for all the idiosyncrasies among the various Web browsers and their versions, and testing becomes a nightmare. All that complexity has gone away in ASP.NET 2.0, encapsulated in the new TreeView control.

The easiest way to use the TreeView control for site navigation is to drag the control from the designer toolbox (located under the Core tab) and bind its data source to a SiteMapDataSource control. Once you've done that, the rest of the implementation involves setting the properties, all the while not having to write any server-side code. Just as with the SiteMapPath control, there are more properties than we can cover in this article, but Figure 5 lists the main ones. We'll describe a handful of them in more depth here.

Figure 5 TreeView Properties
Figure 5 TreeView Properties

ImageSet This defines a set of images to use in your TreeView control. There are numerous image set styles provided, most notable are Custom, MSDN, XPFileExplorer, and Windows_Help. Picking one of the numerous predefined sets allows your TreeView to use familiar images and inherit those styles.

ShowExpandCollapse At first, this might seem a bit odd because we're so used to seeing TreeViews with the expand/collapse indicators (the + and - signs); however, this property gives you the option of not showing those indicators. The default, however, is True.

ShowLines This property determines whether or not to display the lines connecting child nodes in the tree to their parent node. The default value is False.

EnableClientScript This is a fairly important property, especially if you want your TreeView control to expand and collapse, which will usually be the case (although we chose not to do that in the Northwind Traders Web site). This property indicates whether or not to render client-side script to handle the events for expanding and collapsing nodes, which it does by default. When this is set to True, expensive trips to the server are avoided, but if you set this property to False, a postback to the server is required every time a user clicks a node in the tree.

ExpandDepth Sets the number of expanded tree levels when the TreeView control is first displayed. For example, setting this property to 2 expands the root node and all parent nodes immediately under the root node. The default is -1, which indicates that all nodes are fully expanded.

MaxDataBindDepth When binding this control to a data source, such as a SiteMapDataSource control, use this property to limit the number of tree levels to bind to the control. For example, setting this property to 2 will bind only the root node and any nodes immediately under the root node to the TreeView control. All remaining nodes in the data source (the SiteMapDataSource control) are ignored, which could be handy in a deep hierarchical site map. The default is -1, which binds all of the tree levels in the data source to the control.

PopulateNodesFromClient By default this property is set to True, which means the TreeView is statically predefined by what's in the web.sitemap file. Because of this, round-trips to the server are eliminated. In order for this property to be True, the EnableClientScript property must also be set to True; otherwise, postbacks to the server will take place.

DataSourceID This is the ID for the data source to bind to. If you've created a SiteMapDataSource control and want to use it as the data source for your TreeView, type the ID of the SiteMapDataSource control here.

There are many more properties for TreeView controls than those detailed here, including the same node style properties (and subproperties) found in the SiteMapPath control, thus allowing you very fine-grained control over how you want your TreeView to behave and appear. We'll leave the rest of the exploration of the TreeView properties up to you. Figure 6 shows the source of one of the Web pages in the Northwind Traders Web site, which gives you an idea of how we implemented the TreeView control.

Figure 6 Using the TreeView Control (Close Figure 6)
<asp:treeview id="TreeView1" runat="server" datasourceid="SiteMapDataSource1"
    font-underline="False" font-names="Verdana" font-size="8pt" font-
    italic="False"
    font-bold="False" imageset="MSDN" forecolor="#000000" nodeindent="30"
    backcolor="#EEEEEE" showexpandcollapse="False">

    <selectednodestyle verticalpadding="1" bordercolor="#999999"
        horizontalpadding="3"
        backcolor="White" font-underline="False" font-italic="False"
        font-bold="False" borderwidth="1px" borderstyle="Solid" />

    <hovernodestyle verticalpadding="1" bordercolor="#999999"
        horizontalpadding="3"
        backcolor="#C7C7C7" font-underline="True" font-italic="False"
        font-bold="False" borderwidth="1px" borderstyle="Solid" />

    <parentnodestyle font-underline="False" font-italic="False" font-
        bold="False" />

    <leafnodestyle font-underline="False" font-italic="False" font-
        bold="False" />

    <nodestyle font-underline="False" forecolor="Black"
        verticalpadding="2" nodespacing="1"
        horizontalpadding="4" font-names="Verdana" font-size="8pt" font-
        italic="False" font-bold="False" />

    <rootnodestyle font-underline="False" font-italic="False"
        font-bold="False" />
</asp:treeview>

A Breadcrumb Control Today

Now you're probably thinking: these controls are all well and good, and I can't wait to use them, but I want this functionality right now. Well, we've taken the liberty of creating a simple breadcrumb control using ASP.NET 1.1, basing its design loosely on the Visual Studio 2005 SiteMapPath control. By following the design of this control, we've created an easy migration path when the time comes to switch from this control to the new one. Naturally, we called the control Breadcrumb.

For the Breadcrumb control, we thought it would be a good idea to keep the web.sitemap file (and its internal structure) as the default data source loaded from the root of the Web site. This sample should give you a general idea about how to roll your own control, but the actual APIs and implementations in the final release of ASP.NET 2.0 could be different.

Because the web.sitemap file can contain any number of non-nested and nested <siteMapNode> elements, we created the SiteMapNode class to hold the attributes (properties) for each one, such as Title, Url, and Description. Since each <siteMapNode> element can contain other <siteMapNode> elements, we added a property in the class called NodeList, which is a collection of other SiteMapNodes. Figure 7 contains the code for this class.

Figure 7 SiteMapNode Class (Close Figure 7)
Namespace DoughLibrary
    Public Class SiteMapNode
        Private m_strUrl As String = String.Empty
        Private m_strTitle As String = String.Empty
        Private m_strDescription As String = String.Empty
        Private m_blnHasLeaf As Boolean = False
        Private m_objNodeList As SiteMapNodeList

        Public Sub New()
            Me.New(False)
        End Sub

        Public Sub New(ByVal hasLeaf As Boolean, ByVal url As String, _
                       ByVal title As String, ByVal description As _
                       String)
            Me.New(hasLeaf)
            Me.m_strUrl = url
            Me.m_strTitle = title
            Me.m_strDescription = description
        End Sub

        Public Sub New(ByVal hasLeaf As Boolean)
            If (hasLeaf = True) Then
                Me.m_blnHasLeaf = True
                Me.m_objNodeList = New SiteMapNodeList
            Else
                Me.m_blnHasLeaf = False
            End If
        End Sub

        Public Property HasLeaf() As Boolean
            Get
                Return Me.m_blnHasLeaf
            End Get
            Set(ByVal Value As Boolean)
                Me.m_blnHasLeaf = Value
            End Set
        End Property

        Public Property NodeList() As SiteMapNodeList
            Get
                Return Me.m_objNodeList
            End Get
            Set(ByVal Value As SiteMapNodeList)
                Me.m_objNodeList = Value
            End Set
        End Property

        Public Property Url() As String
            Get
                Return Me.m_strUrl
            End Get
            Set(ByVal Value As String)
                Me.m_strUrl = Value
            End Set
        End Property

        Public Property Title() As String
            Get
                Return Me.m_strTitle
            End Get
            Set(ByVal Value As String)
                Me.m_strTitle = Value
            End Set
        End Property

        Public Property Description() As String
            Get
                Return Me.m_strDescription
            End Get
            Set(ByVal Value As String)
                Me.m_strDescription = Value
            End Set
        End Property
    End Class
End Namespace

Figure 8 displays the code for the SiteMapNodeList class, which is your basic collection that inherits from ArrayList, but it is strongly typed to contain only SiteMapNodes. The SiteMapNodeList class is constructed when a SiteMapNode is determined to contain other SiteMapNodes and is populated by simply adding some SiteMapNode objects to it.

Figure 8 SiteMapNodeList Class (Close Figure 8)
Namespace DoughLibrary
    Public Class SiteMapNodeList : Inherits ArrayList
        Default Public Shadows Property Item(ByVal index As Integer) _
                As SiteMapNode
            Get
                Return CType(MyBase.Item(index), SiteMapNode)
            End Get
            Set(ByVal Value As SiteMapNode)
                MyBase.Item(index) = Value
            End Set
        End Property

        Public Shadows Function Add(ByVal value As SiteMapNode) _
               As Integer
            MyBase.Add(value)
        End Function
    End Class
End Namespace

Similar to SiteMapNodeList, the SiteMapPath class is a strongly typed collection of SiteMapNode objects and is used by the Breadcrumb control to obtain the actual nodes to render to the browser (in fact, the code is identical to the SiteMapNodeList class except for the name of the class itself). For example, if the path to display to the user should be Home > Services > Training, the SiteMapPath object would contain the Home SiteMapNode, the Services SiteMapNode, and the Training SiteMapNode.

Note that the order of the SiteMapPath collection is always root node to current node. Because of this, the Breadcrumb control can easily determine the direction in which to render the nodes in the browser. If the PathDirection property is set to RootToCurrent the control reads the SiteMapPath collection from first element to last element, and if the PathDirection property is set to CurrentToRoot the control reads the collection from last element to first element.

The SiteMap class is where all the magic happens; it takes care of loading all nodes from the web.sitemap file and builds the SiteMapPath object, which is then exposed as a read-only property (see Figure 9). Because we know that the Breadcrumb control only cares about rendering a navigation path, providing the SiteMapPath property makes sense because, in essence, the SiteMapPath is the only thing the control cares about. Note, too, that our implementation has a hardcoded dependency on the shape of the sitemap file shown in Figure 2. For robustness, the GetSiteMapPath method should instead use recursion.

Figure 9 SiteMap Class (Close Figure 9)
Imports System.Xml

Namespace DoughLibrary
    Public Class SiteMap
        Private m_objSiteMapNode As SiteMapNode
        Private m_objSiteMapPath As SiteMapPath
        Private m_uriSiteMapFile As Uri
        Private m_uriCurrentUrl As Uri
        Private m_objXmlReader As XmlTextReader
        Private m_intNodeDepth As Integer = 0

        Public Sub New(ByVal siteMapFile As Uri, ByVal currentUrl As Uri)
            Me.m_uriSiteMapFile = siteMapFile
            Me.m_uriCurrentUrl = currentUrl
            Me.m_objSiteMapNode = New SiteMapNode(True)
            Me.LoadSiteMapNodes()
            Me.m_objSiteMapPath = GetSiteMapPath()
        End Sub

        Public ReadOnly Property SiteMapPath() As SiteMapPath
            Get
                Return Me.m_objSiteMapPath
            End Get
        End Property

        Private ReadOnly Property CurrentUrl() As Uri
            Get
                Return Me.m_uriCurrentUrl
            End Get
        End Property

        Private ReadOnly Property SiteMapFile() As Uri
            Get
                Return Me.m_uriSiteMapFile
            End Get
        End Property

        Private Property Depth() As Integer
            Get
                Return Me.m_intNodeDepth
            End Get
            Set(ByVal Value As Integer)
                Me.m_intNodeDepth = Value
            End Set
        End Property

        Private Sub LoadSiteMapNodes()
            Me.m_objXmlReader = New XmlTextReader(SiteMapFile.ToString())
            Me.LoadSiteMapXmlFile()
        End Sub

        Private Sub LoadSiteMapXmlFile()
            ' Ignore whitespace in web.sitemap file
            Me.m_objXmlReader.WhitespaceHandling = WhitespaceHandling.None

            ' Skip xml declaration and any xml prolog
            Me.m_objXmlReader.MoveToContent()

            If (Me.m_objXmlReader.IsStartElement("siteMap") = True) Then
                ' Read past <siteMap> document root element
                Me.m_objXmlReader.ReadStartElement("siteMap")
                ReadSiteMapNode()
            End If
        End Sub

        Private Sub ReadSiteMapNode()
            ' Temp branch and leaf nodes for tree traversal
            Dim objBranch As SiteMapNode
            Dim objLeaf As SiteMapNode

            ' Read the root node's attributes
            ReadNodeAttribs(Me.m_objSiteMapNode)

            ' Cycle through the web.sitemap file nodes
            While (Me.m_objXmlReader.Read())
                If (Me.m_objXmlReader.NodeType = XmlNodeType.Element) Then
                    If (Me.m_objXmlReader.IsEmptyElement() = False) Then
                        Me.Depth += 1

                        ' On a branch node, so create NodeList for leaves
                        objBranch = New SiteMapNode(True)
                        ReadNodeAttribs(objBranch)
                        Me.m_objSiteMapNode.NodeList.Add(objBranch)
                    Else
                        ' At leaf, so don't create NodeList
                        objLeaf = New SiteMapNode(False)
                        objBranch.NodeList.Add(objLeaf)
                        ReadNodeAttribs(objLeaf)
                    End If
                Else
                    If (Me.m_objXmlReader.NodeType = _
                           XmlNodeType.EndElement) Then
                        Me.Depth -= 1
                    End If
                End If
            End While
        End Sub

        Private Sub ReadNodeAttribs(ByRef node As SiteMapNode)
            ' Read title, description, and url attributes for the passed
            ' node
            node.Title = Me.m_objXmlReader.GetAttribute("title")
            node.Description = _
                Me.m_objXmlReader.GetAttribute("description")
            node.Url = Me.m_objXmlReader.GetAttribute("url")
        End Sub

        Private Function GetSiteMapPath() As SiteMapPath
            Dim objPath As SiteMapPath = New SiteMapPath
            Dim objCurrentNode As SiteMapNode = GetCurrentNode()
            Dim objTempNode As SiteMapNode = New SiteMapNode
            Dim i As Integer = 0
            Dim blnFound As Boolean = False

            ' Always add the root node
            objPath.Add(Me.m_objSiteMapNode)

            If Not (objCurrentNode Is Me.m_objSiteMapNode) Then
                ' This loop takes advantage of knowing that the web.sitemap
                ' file is 3 levels deep. For robustness, this loop should
                ' make use of recursion instead.
                While Not blnFound
                    objTempNode = Me.m_objSiteMapNode.NodeList.Item(i)
                    If (objTempNode Is objCurrentNode) Then
                        objPath.Add(objTempNode)
                        blnFound = True
                    Else
                        If (objTempNode.HasLeaf) Then
                            For Each objNode As SiteMapNode In _
                                        objTempNode.NodeList
                                If (objNode Is objCurrentNode) Then
                                    objPath.Add(objTempNode)
                                    objPath.Add(objNode)
                                    blnFound = True
                                    Exit For
                                End If
                            Next
                        End If
                    End If
                    i += 1
                End While
            End If

            Return objPath
        End Function

        Private Function GetCurrentNode() As SiteMapNode
            Dim objResultNode As SiteMapNode = New SiteMapNode
            Dim objTempNode As SiteMapNode = New SiteMapNode
            Dim i As Integer = 0
            Dim blnFound As Boolean = False

            If (Me.m_objSiteMapNode.Url.Equals( _
                    Me.CurrentUrl.PathAndQuery)) Then
                objResultNode = Me.m_objSiteMapNode
            Else
                ' This loop takes advantage of knowing that the web.sitemap
                ' file is 3 levels deep. For robustness, this loop should
                ' make use of recursion instead.
                While Not blnFound
                    objTempNode = Me.m_objSiteMapNode.NodeList.Item(i)
                    If (objTempNode.Url.Equals( _
                            Me.CurrentUrl.PathAndQuery)) Then
                        objResultNode = objTempNode
                        blnFound = True
                    Else
                        If (objTempNode.HasLeaf) Then
                            For Each objNode As SiteMapNode In _
                                    objTempNode.NodeList
                                If _
                                    (objNode.Url.Equals( _
                                    Me.CurrentUrl.PathAndQuery)) Then
                                    objResultNode = objNode
                                    blnFound = True
                                    Exit For
                                End If
                            Next
                        End If
                    End If
                    i += 1
                End While
            End If

            Return objResultNode
        End Function
    End Class
End Namespace

As the control is loaded, it creates a new instance of the SiteMap class by passing it the current URL and the name of the site map file. All work in the SiteMap class is done in the constructor, first by loading all <siteMapNode> elements into their own SiteMapNode objects (and linking them together) and then by building the SiteMapPath object. Once the control has instantiated a SiteMap, it can just ask for the SiteMapPath property to use for rendering to the browser. The code for doing this is very short:

Dim uriFile As Uri = New Uri("http://localhost/nwt/web.sitemap")
Dim uriUrl As Uri = New Uri("http://localhost/nwt/products.aspx")
Dim objSiteMap As SiteMap = New SiteMap(uriFile, uriUrl)
Dim objPath As SiteMapPath = objSiteMap.SiteMapPath

The loading of all the SiteMapNodes is done with help from the XmlTextReader, which is used by a few private helper methods. By checking if each <siteMapNode> in the web.sitemap file is an empty element, we can determine which nodes are branches and which ones are leaves (where branches equate to nested nodes and leaves are non-nested nodes). If a node is determined to be a branch, it's added to the NodeList of the parent node, but if the node is a leaf it's added to the NodeList of the current branch. This also allows us to capture the depth of the tree.

Once all SiteMapNodes are loaded, the SiteMapPath is built, which is done by first determining which SiteMapNode is the current node. Like the Visual Studio 2005 version of the breadcrumb implementation, uniqueness is guaranteed by the given URL. This is compared to the URL property of each SiteMapNode while looping through them until a match is found, after which the loop is exited. Now that the current SiteMapNode has been found, we loop through all nodes again to track the path, always from root node to current node, as shown in the GetSiteMapPath method.

The Breadcrumb Class

All server-side code for the Breadcrumb control is found in its custom control class and handles the loading of the web.sitemap file (OnLoad) and the rendering of the HTML (Render). It exposes an enumeration, PathDirection, which has the following values: RootToCurrent (the default) and CurrentToRoot. Also available are the properties listed in Figure 10, whose names are the same as in ASP.NET 2.0. Note that each of the "style" properties are named like their ASP.NET 2.0 cousins, but they behave slightly differently. Instead of being a set of properties, each style is a discrete property that can be set via the stylesheet.

Figure 10 Breadcrumb Control Properties (Close Figure 10)

Property | Description
PathDirection | Same as the ASP.NET 2.0 version
PathSeparator | Same as the ASP.NET 2.0 version
RenderCurrentNodeAsLink | Same as the ASP.NET 2.0 version
CurrentNodeStyle | Gives the current node in the breadcrumb its appearance (overrides NodeStyle)
HoverNodeStyle | Determines the appearance of a breadcrumb node when you hover over it
NodeStyle | Gives all nodes in the breadcrumb a consistent style when the breadcrumb is displayed
PathSeparatorStyle | Sets the appearance for any text you defined in the PathSeparator property
RootNodeStyle | Determines the style of the root node of the breadcrumb (overrides NodeStyle)

During the control's load event, an instance of SiteMap is created and the control then gets the SiteMapPath as just described. Next, the overriding Render method outputs the navigation path to the browser in the proper PathDirection with all associated styles to complete its appearance. Figure 11 shows the code for this class, and the code snippet that follows demonstrates how to use this custom control on a Web form. Take a look at the similarities between this code and the code that is necessary to use the ASP.NET 2.0 SiteMapPath control:

<nwt:breadcrumb id="Breadcrumb1" runat="server" pathseparator=" -> "
    currentnodestyle="crumbNode"
    nodestyle="crumbNode"
    pathseparatorstyle="crumbPathSeparator"
    rootnodestyle="crumbNode"
    rendercurrentnodeaslink="True" />

Figure 11 Breadcrumb Class (Close Figure 11)
Imports System.ComponentModel
Imports System.Web.UI
Imports System.Web.UI.WebControls
Imports System.Web

Namespace DoughLibrary
    Public Enum PathDirection
        RootToCurrent
        CurrentToRoot
    End Enum

    Public Class Breadcrumb : Inherits WebControl
        Private m_enmPathDirection As PathDirection = _
            PathDirection.RootToCurrent
        Private m_strPathSeparator As String = String.Empty
        Private m_blnRenderCurrentNodeAsLink As Boolean = False
        Private m_strCurrentNodeStyle As String = String.Empty
        Private m_strHoverNodeStyle As String = String.Empty
        Private m_strNodeStyle As String = String.Empty
        Private m_strPathSeparatorStyle As String = String.Empty
        Private m_strRootNodeStyle As String = String.Empty
        Private m_SiteMapPath As SiteMapPath

        Private Property SiteMapPath() As SiteMapPath
            Get
                Return Me.m_SiteMapPath
            End Get
            Set(ByVal Value As SiteMapPath)
                Me.m_SiteMapPath = Value
            End Set
        End Property

        Public Property PathDirection() As PathDirection
            Get
                Return Me.m_enmPathDirection
            End Get
            Set(ByVal Value As PathDirection)
                Me.m_enmPathDirection = Value
            End Set
        End Property

        Public Property PathSeparator() As String
            Get
                Return Me.m_strPathSeparator
            End Get
            Set(ByVal Value As String)
                Me.m_strPathSeparator = Value
            End Set
        End Property

        Public Property RenderCurrentNodeAsLink() As Boolean
            Get
                Return Me.m_blnRenderCurrentNodeAsLink
            End Get
            Set(ByVal Value As Boolean)
                Me.m_blnRenderCurrentNodeAsLink = Value
            End Set
        End Property

        Public Property CurrentNodeStyle() As String
            Get
                Return Me.m_strCurrentNodeStyle
            End Get
            Set(ByVal Value As String)
                Me.m_strCurrentNodeStyle = Value
            End Set
        End Property

        'Other properties removed for brevity

        Public Property PathSeparatorStyle() As String
            Get
                Return Me.m_strPathSeparatorStyle
            End Get
            Set(ByVal Value As String)
                Me.m_strPathSeparatorStyle = Value
            End Set
        End Property

        Public Property RootNodeStyle() As String
            Get
                Return Me.m_strRootNodeStyle
            End Get
            Set(ByVal Value As String)
                Me.m_strRootNodeStyle = Value
            End Set
        End Property

        Protected Overrides Sub OnLoad(ByVal e As System.EventArgs)
            Dim strFile As String = Page.Request.MapPath("web.sitemap")
            Dim strPage As String = Page.Request.Url.ToString.ToLower
            Dim objSiteMap As SiteMap = New SiteMap(New Uri(strFile), _
                New Uri(strPage))
            Me.m_SiteMapPath = objSiteMap.SiteMapPath
        End Sub

        Protected Overrides Sub Render(ByVal output As _
                System.Web.UI.HtmlTextWriter)
            ' SiteMap Class always returns the SiteMapPath in RootToCurrent
            ' Order. If PathDirection attribute is set to CurrentToRoot,
            ' reverse the SiteMapPath direction before rendering.
            If Me.PathDirection = PathDirection.CurrentToRoot Then
                Me.SiteMapPath.Reverse()
            End If

            ' outermost span start element for breadcrumb control
            output.AddAttribute(HtmlTextWriterAttribute.Id, Me.ID)
            output.RenderBeginTag(HtmlTextWriterTag.Span)

            For Each objCrumb As SiteMapNode In Me.SiteMapPath
                ' start crumb span
                output.RenderBeginTag(HtmlTextWriterTag.Span)

                ' start crumb hyperlink anchor
                If Me.SiteMapPath.IndexOf(objCrumb) = 0 Then
                    If Me.PathDirection = PathDirection.RootToCurrent Then
                        ' set css of 1st crumb: root if RootToCurrent
                        ' direction
                        output.AddAttribute( _
                           HtmlTextWriterAttribute.Class, Me.RootNodeStyle)
                    Else
                        ' set css of 1st crumb: current if CurrentToRoot
                        ' direction
                        output.AddAttribute( _
                          HtmlTextWriterAttribute.Class, Me.CurrentNodeStyle)
                    End If
                ElseIf Me.SiteMapPath.IndexOf(objCrumb) = _
                        (Me.SiteMapPath.Count - 1) Then
                    If Me.PathDirection = PathDirection.RootToCurrent Then
                        ' set css of last crumb: current if RootToCurrent
                        ' direction
                        output.AddAttribute(HtmlTextWriterAttribute.Class, _
                            Me.CurrentNodeStyle)
                    Else
                        ' set css of last crumb: root if CurrentToRoot
                        ' direction
                        output.AddAttribute(HtmlTextWriterAttribute.Class, _
                            Me.RootNodeStyle)
                    End If
                Else
                    ' set css of all other crumbs to the NodeStyle property
                    ' value
                    output.AddAttribute(HtmlTextWriterAttribute.Class, _
                        Me.NodeStyle)
                End If

                If RenderCurrentNodeAsLink = False Then
                    If ((Me.PathDirection = PathDirection.RootToCurrent) _
                           AndAlso (Me.SiteMapPath.IndexOf(objCrumb) = _
                           (Me.SiteMapPath.Count - 1))) Or _
                           ((Me.PathDirection = _
                           PathDirection.CurrentToRoot) AndAlso _
                          (Me.SiteMapPath.IndexOf(objCrumb) = 0)) Then

                        output.RenderBeginTag(HtmlTextWriterTag.Span)
                        ' <span>
                        output.Write(objCrumb.Title)
                        output.RenderEndTag() ' </span>
                    Else
                        output.AddAttribute(HtmlTextWriterAttribute.Title, _
                            objCrumb.Description)
                        output.AddAttribute(HtmlTextWriterAttribute.Href, _
                            objCrumb.Url)
                        output.RenderBeginTag(HtmlTextWriterTag.A) ' <a>
                        output.Write(objCrumb.Title)
                        output.RenderEndTag() ' </a>
                    End If
                Else
                    output.AddAttribute(HtmlTextWriterAttribute.Title, _
                        objCrumb.Description)
                    output.AddAttribute(HtmlTextWriterAttribute.Href, _
                        objCrumb.Url)
                    output.RenderBeginTag(HtmlTextWriterTag.A) ' <a>
                    output.Write(objCrumb.Title)
                    output.RenderEndTag() ' </a>
                End If

                output.RenderEndTag() ' </span> (end crumb span)

                ' append path separator if not on last crumb
                If (Me.SiteMapPath.IndexOf(objCrumb) _
                        < (Me.SiteMapPath.Count - 1)) Then
                    output.AddAttribute(HtmlTextWriterAttribute.Class, _
                        Me.PathSeparatorStyle)
                    output.RenderBeginTag(HtmlTextWriterTag.Span)
                    output.Write(Me.PathSeparator)
                    output.RenderEndTag() ' </span>
                End If
            Next

            output.RenderEndTag() ' </span> (outermost span)
        End Sub
    End Class
End Namespace

All the code we've described for the Breadcrumb control provides you a simple way to add navigation paths to your Web site. And even though there is room to increase its robustness (through caching, error handling, and recursion), the control is designed and written for easy extensibility.

Conclusion

As you can see, implementing a SiteMapPath (breadcrumb) control today that has some of the functionality provided by its ASP.NET 2.0 counterpart took several hundred lines of code. To implement a TreeView control today that's as powerful as the ASP.NET 2.0 version would take a significant amount of time and thousands of lines of code. That alone should have you clamoring for the new navigation controls coming in ASP.NET 2.0 because all that functionality and more is provided to you out of the box with minimal configuration and, more importantly, without the need to write the code yourself. With ASP.NET 2.0 in your arsenal you'll achieve navigational goals you didn't think possible.

Dave Donaldson and Steven DeWalt are both consulting IT architects for a large insurance company. When not working for the man, they can be found building .NET-based applications and services for their small software shop, LoudCarrot Software. You can contact them at http://www.loudcarrot.com.


Editor's Note: Fun with ASP.NET 2.0

Editor's Note
Fun with ASP.NET 2.0

As you could probably tell from the cover, we are bringing you early, in-depth coverage of ASP.NET 2.0, the Web components of the next version of the Microsoft .NET Framework. ASP.NET 2.0 was unveiled at the October 2003 Microsoft PDC, but until now has had limited availability. Basically, if you went to the PDC and brought the DVDs home, you had an early copy of the release. At press time, however, all the technology we cover in this issue is available to MSDN Universal subscribers; a larger distribution is scheduled for Tech•Ed this May.

So why have we chosen to devote an entire issue to the features of ASP.NET 2.0 this month? The answer is simple—it's really cool. You'll see what we mean as you read the articles we've gathered, but for those of you who are new to the world of ASP.NET, it might be informative to take a look at the evolution of Windows-based Web development over the years.

ISAPI was the first API offered by Microsoft for programmatically handling Internet traffic, both HTTP-based and otherwise. The interface was relatively basic, offering little more than a wrapper around what some coders were familiar with in CGI (Common Gateway Interface) libraries. You could get basic session data, and that was about it. You had to write modules that were, frankly, kind of annoying and picky. It often felt like you were patching bits together and keeping your fingers crossed. Debugging was a bear—and not one of those friendly circus bears who balance a ball on their nose. We're talking the kind that you see on the nature specials, where the salmon jumps out of the water and right into the bear's mouth. And that salmon is your DLL.

Compared with ISAPI, ASP (Active Server Pages) represented a great leap forward. Though ASP was really implemented as an ISAPI DLL, its purpose was to take the grunge work out of going to the Web. ASP let you interlace your HTML pages with script code—VBScript or JavaScript—and create useful output. This approach grew in popularity, spawning dozens of dedicated user groups and community sites. Of course, there were still some issues with ASP code, like its tendency to lead the developer towards hard-to-read spaghetti code that could quickly become hopelessly interwoven with display elements.

The first preview edition of ASP.NET 1.0 hit the streets in 2000. It had a couple of huge advantages over ASP—server-side programming could be separated from HTML, and the .NET Framework could be easily accessed from code. Well before it was officially released in 2002, thousands of sites were using ASP.NET for production code; since then, ASP.NET has become so popular that its adoption has recently overtaken that of Java servlets (JSP pages), according to NetCraft.

So where is there to go from 1.0? ASP.NET 2.0 represents a further enhancement of the technology in a number of areas. In fact, it's built upon several pillars: security, data, personalization, master pages, and so on. Each pillar has been created as a way to make it significantly easier for coders to create powerful Web sites by using well-tested patterns, so they don't have to reinvent the wheel each time.

The enhancements around ASP.NET 2.0 security provide good task encapsulation. In ASP.NET 1.0, if you wanted security, you had to build it yourself. And you know what's a really bad idea? Homebrewed security schemes. You remember when you were in grade school and your dad gave you a haircut in the kitchen? With the newspaper spread out under the chair? And it looked crooked as heck and you had to wear a Mets cap to school for the next three weeks? Well, maybe that's our own personal painful memory, but the lessons we learned in the kitchen chair can be applied to ASP.NET and security: let the guys who do it for a living give you the basics and you'll be much happier with the results.

The point we're trying to make here is that ASP.NET 2.0 provides ready-made security modules. If you want to add authentication services, you can just snap in a component and all the hard work is done for you. The same logic suffuses the other new elements of ASP.NET 2.0. Data is easier than ever. Creating page templates has gone from a do-it-yourself project to a feature that's supported in the Visual Studio 2005 page designer.

In short, we've had a lot of fun playing with the ASP.NET 2.0 bits, and we hope you do too.

—J.T.

Thanks to the following Microsoft technical experts: Scott Berry, Catherine Brooker, Christopher Brumme, Claudio Caldato, Pablo Castro, Michael Fanning, Jonathan Keljo, Ronald Laeremans, Gerrard Lindsay, Ivan Medvedev, Michael Murray, Stefan Schackow, Dan Vallejo, and Jeffrey Van Gogh. Special thanks to Rob Howard and the ASP.NET team.

Active Directory, ActiveX, IntelliSense, JScript, Microsoft, MSDN, Outlook, Visual Basic, Visual C++, Visual C#, Visual Studio, Windows, Windows NT, and Win32 are registered trademarks of Microsoft Corporation. SharePoint and Windows Server are trademarks of Microsoft Corporation. Other trademarks or tradenames mentioned herein are the property of their respective owners.

MSDN Magazine does not make any representation or warranty, express or implied with respect to any code or other information herein. MSDN Magazine disclaims any liability whatsoever for any use of such code or other information.


New Stuff: Resources for Your Developer Toolbox

New Stuff
Resources for Your Developer Toolbox
Theresa W. Carey

Contents
Testing the User Experience
Configuration Management System
Generate Code and Database Documentation
Visualize Your Data
Push Those Patches
Convert from Speech to Text
Troubleshoot Task Resources
Smart Components
The Bookshelf

Testing the User Experience

Techsmith recently rolled out Morae, which records and synchronizes user and system data for the analysis of human-to-computer interaction. It enables users to record, log, analyze, and share user information, application details, and system data with its automatically synchronized screen and camera video, and can even be run on a laptop.

The default settings record screen video, camera video, and audio of the user (with installed camera and microphone), as well as Web page changes, mouse clicks, keyboard entry, onscreen text, and various application events.

The company's Rich Recording Technology platform synchronizes these input streams into one recording file. Recorder can be set to start and stop manually, set for a specific event, or specific time. Start and stop options such as event-based triggers (including mouse clicks) or the launching or closing of an application can be modified independently. For example, you could set Recorder to begin recording each time Microsoft® Internet Explorer is launched, then stop recording after a specified period.

The search results can be viewed in a time-stamped list view or interactive graph format, synchronized with the video. Your search results can be exported to a comma-delimited file so that they can then be imported into other applications.

http://www.techsmith.com

Configuration Management System

Perforce Software announced version 2003.2 of its Fast Software Configuration Management (SCM) System, which includes the Perforce Plug-In for Windows® Explorer (P4EXP). The plug-in gives one-click access to Perforce SCM functionality when developers use Windows Explorer in order to browse files and folders.

Perforce tracks and manages software development, handles distributed development, and supports developers across more than 50 operating systems.

The system groups multifile updates into atomic changes, enables concurrent development, and manages multiple software releases by using its own Inter-File Branching system.

SCM users can access source control through the file management environment of Windows Explorer. The newly designed look of the Perforce Web Client (P4Web) is consistent with the appearance of Perforce Software's Windows, Linux, and Macintosh interfaces.

Additional server enhancements in Performance Software version 2003.2 include improved performance, more versatile file diff layouts for mixed-platform environments, and new options for integrating files and viewing annotated file content.

http://www.perforce.com

Generate Code and Database Documentation

SharpPower Corp. released RapTier 1.4, a C#, Visual Basic® .NET, ASP.NET, and SQL code generator for database-driven applications. The latest version introduces ASP.NET UI generation, support for FoxPro and Sybase databases, stored procedure wrapping, and database documentation generation.

The data access code generated by RapTier provides efficient isolation of SQL code and business logic, supports both DataSets and typed Data Transfer Objects, and represents database stored procedures as regular .NET code. Generated ASP.NET and Windows Forms applications allow the user to edit, sort, insert, and delete records in the database.

RapTier 1.4 allows users to generate database documentation that contains information about tables, views, stored procedures, indexes, triggers, and table relations. SQL Server™, MSDE, Microsoft Access, Oracle, MySQL, FoxPro, Sybase SE, and Sybase Anywhere are all supported. RapTier is available in three editions: Lite, Professional, and Enterprise.

http://www.SharpPower.com

Visualize Your Data

Tecplot Inc. recently announced the release of Tecplot Edge 2.0, a data visualization product designed for application developers. Tecplot Edge is a customizable version of Tecplot that can be integrated with other third-party applications.

With Tecplot Edge, developers can integrate XY, 2-D, and 3-D, plotting capabilities into their solver, simulation code, or analysis application. New in Tecplot Edge 2.0 is the ability to modify the entire menu system and replace the Tecplot sidebar. This lets developers tailor the interface and deliver a solution with a customized appearance. Also new is support for bitmapped buttons in the Tecplot GUI Builder and enhanced development tools. Tecplot Edge 2.0 is available for Windows, Mac OSX, Linux, and Unix. A suite of development tools for extending functionality and building platform-independent GUI code is also included.

http://www.tecplot.com

Push Those Patches

Executive Software has unveiled new patch management functionality available in Sitekeeper 3.1 via a plug-in that utilizes inventory and "push" deployment technology. This update determines which versions of software are running on which machines, and allows you to download and update software with just a few clicks. You can determine the machines that need specific software versions, updates, or patches.

The redesigned interface contains direct links for help, so if you run into a snag, you can get assistance without having to abort your task.

Sitekeeper can give your company an enterprise class solution to ensure that you keep current with all the latest security updates. This update is available for existing Sitekeeper 3.0 customers and is included as part of the Sitekeeper Inventory and Deployment modules.

http://www.execsoft.com

Convert from Speech to Text

Cool Conversations (Cool C) has released the Cool C Read Writer as a new component for their Cool C Interactive Speech software. This upgrade module, utilizing the Microsoft Speech Engine, makes it possible for a Windows-based PC equipped with the Cool C Talking Computer Software to convert spoken words into printed text. Cool C Read Writer will also read aloud text that is pasted into the application. These functions complement the program's ability to have conversations with computer users and deliver both verbal reminders and special broadcast messages.

The basic Cool C Talking Computer software converts most any computer into a talking computer. The program makes it possible for any Windows-based PC to verbally respond to spoken questions or comments from users. Each Cool C Broadcast Group has the ability to send private messages with the Cool C interactive speech application functioning as a receiver for broadcast information. Messages are delivered directly to the PC of this defined group of users controlled by the broadcaster. Cool C will automatically search for messages each time a user's computer connects to the Internet.

When a message is received by Cool C, a box will appear asking the user if they want to hear the message. Clicking on the box will open the main program window, which will deliver the message verbally and also provide scrolling text. If you want, the window can be customized to include advertising or sponsorship graphics with links to the Web.

The Cool C Read Writer upgrade module provides users with two utilities: pasting text from a Web page or document into an application to be read aloud, and having dictated content converted to text to be saved or pasted into other applications.

http://www.coolconversations.com

Troubleshoot Task Resources

The Warecase Software Group launched its eXtended Task Monitor for Windows 2000 and later, an auxiliary tool designed for troubleshooting task resources and workstation problems that sometimes arise during software development.

XTM can provide solutions for problems which may develop in a product's functioning without having to reinstall it from the user workstation or close the application. The XTM toolkit provides the developer with direct access to the flow of processes, allowing data collection and problem solving. XTM integrates its task-monitoring tools with a suite of tools for hands-on management of data resources and the workstation while maintaining product functionality.

Its features include the ability to monitor processes, modules, handles, threads, files, registry keys, memory, windows, COM+ components, services, and environment variables. In addition, XTM can monitor process performance and statistics, manage process resources, filter processes by particular criteria, free resources previously allocated, manage synchronization objects, find the required resources, perform workstation services management, workstation power management, and tracing.

http://www.warecase.com

Smart Components

Smart Components recently introduced Flat File To Xml Pro, which converts textual flat files into XML using a specification file. Many older applications such as banking or payroll systems output flat files for various purposes from printing to reporting. Developing software to process these files can be time consuming and difficult, requiring a rewrite of the application as the format of the flat file inevitably changes.

Flat File To Xml Pro eliminates these problems by converting the flat file into XML using a specification file as defined in XML. Once converted into XML, the data can be processed easily in a multitude of ways, giving all the advantages of XML without requiring a rewrite of existing legacy code.

Flat File To Xml Pro includes all the features of Flat File To Xml including a flat file described in an XML specification file, a fast, efficient parser, outputting the XML as a file or as an XML doc, and a demo application with complete source code and several transformation examples.

Flat File To Xml Pro also includes the following powerful features: the use of Regular Expressions to identify different types of flat file lines so they can be formatted differently, complex customizable formatting of data when creating output XML, outputting lines as parent or child nodes, and outputting data as child nodes or attributes.

In addition, the product provides three different event logging modes to help debug transformations, and you can input XML as a file or a document.

http://www.smartcomponents.plus.com

The Bookshelf
Anchor Point Books

In How to Fail at Software Development (Anchor Point Books), Arthur Griffith explains that the principal cause of code failure is that no one involved asks the simple, yet crucial questions that can detect and avoid disastrous results.

Some questions that could prevent implosion are: Does anybody really want this thing? Do folks talk about finishing the project or just talk about continuing work? Is the written design document treated with respect? If the overall design is a source of comedy, disdain, or is completely ignored, then you will never get anywhere. Does everybody know exactly what is to be accomplished? Do you have everything you need to build it? This book examines what happens when the team does not address these and many more questions.

http://www.anchorpointbooks.com

Addison-Wesley Professional recently published two new books in their Microsoft .NET Development Series: .NET Framework Standard Library Annotated Reference, Volume 1: Base Class Library and Extended Numerics Library by Brad Abrams, and The Visual Basic .NET Programming Language by Paul Vick.

The Visual Basic .NET Programming Language describes the language, beginning with simple concepts and moves toward the more advanced concepts. Paul Vick is one of the lead architects of Visual Basic .NET and author of the specification.

NET Framework Standard Library Annotated Reference, Volume 1: Base Class Library and Extended Numerics Library is the first of a two-volume set providing developers with a detailed reference to the most useful parts of the .NET Framework Class Library. The book and CD set uses annotations and code samples to move beyond the online documentation in order to provide dictionary-style reference to the most used parts of the Framework.

This volume covers a subset of the ISO CLI standards, including the Base Class Library and the Extended Numerics Library. The printed book contains overviews of each namespace and an alphabetic reference of types in the standard. The CD is comprised of the book contents as well as member-level descriptions, sample code, and all source code in a buildable format.

http://www.awprofessional.com

Send your New Stuff to Theresa at newstuff@microsoft.com.

Theresa W. Carey is a freelance writer who lives in California. Her byline has appeared in Barron's and PC World.


Web Q&amp;A: Visual Studio 2005, HTC Memory Problems, and More

Web Q&A
Visual Studio 2005, HTC Memory Problems, and More
Edited by Nancy Michell

Q In Visual Studio® 2005, when I create a new Visual C# Project from the File menu, I don't see the template ASP.NET Web Application in the template window. If I choose to create a new Web site following the menu path Files | New | Website | General C# and I choose the template ASP.NET Web site, I'm missing many files such as Webform1.aspx.cs, Global.asax, Web.config, AssemblyInfo.cs, and so forth. Why is this happening?

A In Visual Studio 2005 the project model has changed quite a bit (for the better). Now you start with a simplified Web Application into which you can add the things that you really need. In the past, many users didn't even know what Global.asax and web.config were used for.

The codebehind model has also been enhanced to support a better model—code-beside—that uses new features in Visual Studio 2005 such as partial classes to create pages. By default, depending on the build you have, Visual Studio 2005 will add the pages using this new model. Incidentally, you will also notice that Visual Studio 2005 no longer requires IIS to be installed on the developer machine to create Web sites as its own built-in ASP.NET-enabled Web server is included.

Q Where can I find ASP.NET 2.0 tutorials and samples?

A Take a look at http://www.asp.net/whidbey/pdc.aspx. You'll find information on building server controls, data controls, Master Pages, themes, and more. Also, see the various articles in this issue (June 2004) which discuss developing in ASP.NET 2.0.

Q Removing an HTML Component (HTC) from an HTML page does not seem to release memory. I tested this using two functions. The first adds an HTC component to a page 1000 times. The second removes the added components one by one. Adding the components to the page increased Microsoft® Internet Explorer memory usage, but removing the component did not bring down the memory usage. However, whenever I minimize the browser, memory usage comes down. What am I doing wrong?

A According to the Knowledge Base article "Memory Leak Occurs When Behaviors Are Dynamically Removed," your problem is caused by a synchronization issue between Internet Explorer and the script engine.

This fix is included in Internet Explorer 6.0 SP1, but it is turned off by default. To turn on this fix, set the following registry value to either a string value of yes or a binary value of 1:

HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main\Cleanup HTCs

If that doesn't help, you could have a different problem, which is not fixed. According to the Knowledge Base article "Memory Leak Occurs When You Remove Behaviors", a solution is to programmatically refresh the page or browse to another page.

Q How can I lock the Internet Explorer home page seen by my field sales force so that if people change it, it will revert back to the original home page? The guys in the field are running Windows® XP Professional. Can this be done by administering Group Policy locally and over the network when they log on? My users have full administrator rights.

A This option is configurable via the Internet Explorer Administration Kit (IEAK) or Group Policy. The registry setting is:

HKCU,"Software\Policies\Microsoft\Internet Explorer\Control
    Panel","HomePage",0x10001,1,0,0,0

For more information check out Knowledge Base article 320159, "Home Page Setting Changes Unexpectedly, or You Cannot Change Your Home Page Setting".

One way to set the Group Policy is to first set the home page using Group Policy through User Configuration | Windows Settings | Internet Explorer Maintenance | URLs | Important URLs | Homepage. Then set the Group Policy so that it cannot be changed using the GUI (as just described) by selecting User Configuration | Admin. Templates | IE. Since the users are administrators (and might be clever), they could change the settings back after the Group Policy is applied because by default as long as the Group Policy doesn't change afterward it won't be applied again.

To remedy this, activate the policy for Internet Explorer Maintenance (Computer Configuration | Admin. Templates | System | Group Policy) and set it so that it will be applied even if the Group Policy did not change. This way the next time the user logs on, the Group Policy check will be performed and the settings will be reapplied, reverting the user's settings to the required settings.

Q Can I attach more than one event to window.onload in JScript®?

A Yes. You can fake multicasting by calling any number of methods from your one event handler. To allow the UI to continue processing, you can use setTimeout:

<script language="jscript">
    // you should do this on multiple lines

    function handleOnLoad() { handlerOne(); handlerTwo(); handlerThree(); }
</script>
<body onLoad="handleOnLoad()" ... />

Also, it is worth noting that there are good and bad ways to fake dynamic multicasting. For a look at some bad ways, see Eric Lippert's article "Multicast Delegates The Evil Way".

Note that this kind of fake multicasting can be done entirely with Internet Explorer-style event handlers. Also note that the comments in that article point out a number of good alternatives, including the use of "attachEvent" in Internet Explorer.

Q I'm trying to run the following snippet in an ASP page:

<%
Set Parent = GetObject("IIS://localhost/W3SVC")

For Each Child In Parent
  If IsNumeric(Child.Name) Then
     Response.Write "ProgrammticID: " & Child.Name & "Friendly Name: " &
         Child.ServerComment & "<BR>"

   End If
Next
%>

I keep getting an access denied error on the GetObject request. The page currently runs under the context of an anonymous user who has very little access. What is the minimum amount of security I can apply in order to get this working?

A You have to be at least in the Server Operators group, perhaps even in the Administrators group, to get GetObject to work. Do not add your anonymous user to these groups. Place Access Control Lists on the file so that only administrators of the system can access the file. When an administrator requests the file, it will then be running in the correct security context to complete the task (as long as you are connecting to the local machine). You can then configure the custom error for the HTTP 401 error for this page (or application, as appropriate) to relay to the user that they don't actually have permission to perform the operation, or whatever message is appropriate. You could also do all of this in ASP by using "On Error Resume Next," adding various error checking statements around the GetObject command.

Q I'm looking for some best practices regarding JavaScript and ASP.NET development. For example, what should I be concerned about to ensure proper functioning in multiple browsers? When should I use JavaScript for validation? Do you recommend using validators in all situations?

A If you want to perform validation and you're using ASP.NET, you can use custom validator controls on the server side. Thus, you can avoid client-side validation, which should never replace actual server-side validation, as the client could be spoofed.

On the client side, even if you're using Internet Explorer, you may run into JavaScript compatability issues if you are creating any app targeted at multiple browsers and languages. However, some important recommendations and best practices include:

  • Using the least amount of script code possible
  • Placing JavaScript code in a .js file and using <script src="myfile.js" /> tags so the scripts can be cached and the code separated from the UI
  • Avoiding the generation of HTML from your script
  • Using semicolons
  • Deciding on a browser test matrix ahead of time, and using that to test your app
  • If at any point you think you've found a clever trick to exploit, think again

But what you should care about in the JavaScript to support multiple browsers is the wrong question to ask. Eric Lippert wrote about why this is the wrong question in a recent blog entry.

Having a big list of browser differences isn't going to help much. What you need is a method that lets you write portable software. One possibility is to get machines that can run all the browsers you want to support. Then write a little bit of HTML and script, and test it in every browser. If something breaks in one place but not in another, congratulations—you've found a browser incompatibility. Work around it. That is much better than listing every known difference between every version of every browser, comparing every line of code against that list, and hoping for the best. You're going to have to test at some point—so test early and test often.

As far as using JavaScript for validation, that's a good question. For performance you should validate on the client. For security, you should validate on the server.

Since security is more important than performance, you first need to get the server validation right. To reduce the load on the server and decrease the time between when the user enters bad data and when you catch it, adding client-side validation might be a good idea. Before making performance improvements, you need to have measurable goals. See How Bad Is Good Enough? for more guidance on this.

Q I'm using forms authentication, and when the user's session times out, he is sent back to the login.aspx page. At that point, if the user reenters his login information, he is taken back via FormsAuthentication.RedirectFromLoginPage to the page he was last viewing. How can I put a message on the login page to tell them that their session has timed out and they must log in again?

A Two distinct concepts are actually at play here: session state and forms authentication. When using in-process session state, Session_End does not map to your forms authentication session timing out. Session_End will fire silently on the server side x minutes after the last request to an ASPX page in the application (x being the session state timeout). Based on the description you've provided, what seems to be happening is that the authentication ticket embedded within the forms authentication cookie is expiring. Once that happens, the user will be redirected to the login page, but Session_End will not fire.

You really can't do a Response.Redirect or similar command in Session_End for the simple reason that this happens totally on the server side (without any HttpContext, since no request really caused the session to end).

If you're using in-memory nonpersistent cookies, you could try using something like this in Page_Load of your Login.aspx.cs file:

if(Request.Cookies["SyprosCom"]!=null)
{
    Response.Write("Redirected due to forms. Auth. timeout");
}
However, this logic will fail if you are using persistent cookies and the cookie timeout is equal to the forms authentication ticket timeout. For more information on ASP.NET session state take a look at Understanding session state modes + FAQ, ASP.NET Session State, and Using ASP.NET Session State in a Web Service.

Q I am using session variables to display a navigation bar on a Web page. When I create a child window (Ctrl-N), the session variables are passed on as well. So, on both the pages, the same navigation bar appears. Is there a way to restrict the session variables or figure out if this is a new browser instance or not?

A A child window of Internet Explorer (Ctrl-N) will always use the same session because it is from the same Internet Explorer process. You can't restrict the session variables; you'll have to partition them in your own logic. So taking the second part of your question (figuring out if this is a new browser instance or not), it depends how your navigation works. If it's postback-driven from standard controls you could do the following:

  1. When you paint your page the first time (not a postback), put a unique key (a GUID would do it) into either a hidden form field or ViewState, and put the same key into your session state; call this your "subsession ID."
  2. Check for this key whenever you get a postback. If it's not a match for the one in the session, don't allow access to the session variables you want to restrict to the first subsession, and plop a new GUID into the Viewstate/hidden form field and session state for matching.

To make this work you need some sort of hashtable-style data structure stored in session that maps variables associated with this new "subsession ID." Then you can write a simple class that wraps access to session variables so that it keys off that structure, allowing or denying access as appropriate.

Q I'm new to the .NET Framework and I've found that code in the InitializeComponent function is deleted when I change the EnableViewState property value of a page. Why is this happening?

A If your .aspx/.ascx page stops working for no apparent reason and you've pored over your code, and everything looks exactly as you left it the night before, check the hidden "Web Form Designer generated code" block at the bottom of the codebehind file. Make sure the InitializeComponent method looks like this:

private void InitializeComponent()
{
    this.Load += new System.EventHandler(this.Page_Load);
}
If it doesn't, Visual Studio .NET 2003 may have deleted this code. See the Knowledge Base article "FIX: Code in the InitializeComponent Function Is Deleted When You Change the EnableViewState Property Value of a Page".

Got a question? Send questions and comments to webqa@microsoft.com.

Thanks to the following Microsoft developers for their technical expertise: Greg Bybee, Hitesh Chaudhary, Sachin Goregaoker, Mark Ingalls, Andre Kronmaier, Dan Liebling, Eric Lippert, Nathan Manis, Carlos Aguilar Mares, Benjamin Moules, Patrick Ng, Kalmadi Santhosh Rao, Dan Reynolds (Excell Data Corporation), Darren Ross (ConsultNet), Carol Roy, Spyros Sakellariadis, Peter Scheuermann, Dan Sellers, Sean Sutherland, Anurag Vyas, Mark Wilson-Thomas, and Jon Wong.


Data Points: Contrasting the ADO.NET DataReader and DataSet

Data Points
Contrasting the ADO.NET DataReader and DataSet
John Papa

Contents
Being Connected
Supporting Multiple Resultsets
The Disconnected Side
When to Use the DataSet

Iam often asked by developers whether the ADO.NET DataReader or the DataSet is the better tool. Some developers say the DataReader is better because it is lightweight, while still others say they prefer the DataSet for its inherent flexibility. The truth is that both have their place in Microsoft® .NET development as their usefulness depends on the situation.

The ADO 2.x recordset object is able to operate in either a connected or a disconnected mode. It can remain connected to the underlying database while traversing a forward-only rowset or it can retrieve a rowset into a client-side, in-memory cursor and disconnect itself from the database. Among the hurdles you may well encounter in migrating from classic ADO to ADO.NET is gaining a full understanding of how the operations that the ADO recordset performed are now handled in ADO.NET.

Instead of a single rowset container, ADO.NET offers two distinctly separate data storage objects: the DataReader and the DataSet. This month I'll focus on the purpose of these two data retrieval classes in ADO.NET and help you to decide which is the best choice to use in a particular situation. I will explore how to retrieve data into both the DataReader and the DataSet, beginning by discussing the DataReader's unique capabilities. I will also compare the connected DataReader to the disconnected DataSet, weighing the pros and cons of using each in different scenarios.

Being Connected

Before deciding when to use a DataReader, it is smart to understand its features and limitations. The DataReader has a defined set of operations that revolve around its connected, forward-only, read-only nature (the read-only DataReader is also known as the firehose cursor of ADO.NET). A DataReader is a stream of data that is returned from a database query. When the query is executed, the first row is returned to the DataReader via the stream. The stream then remains connected to the database, poised to retrieve the next record. The DataReader reads one row at a time from the database and can only move forward, one record at a time. As the DataReader reads the rows from the database, the values of the columns in each row can be read and evaluated, but they cannot be edited.

Unlike the DataSet, the DataReader is not implemented directly in the System.Data namespace. Rather, the DataReader is implemented through a specific managed provider's namespace such as System.Data.SqlClient.SqlDataReader. Because all DataReaders, including the OleDbDataReader, the SqlDataReader, and other managed provider's DataReaders implement the same IDataReader interface, they should all provide the same base set of functionality. Each DataReader is optimized for a specific data provider. If the database you are developing against has a managed provider for ADO.NET, then you should take advantage of it. Otherwise, you can use the System.Data.OleDb or the System.Data.Odbc namespaces, which expose more generic managed providers that can access a variety of data sources. If you are developing against SQL Server™ or Oracle, it would be more efficient to use the provider that was made specifically for these databases. In this column, I will query the SQL Server Northwind database using the System.Data.SqlClient namespace.

The fact that the SqlDataReader is part of a specific managed provider's feature set further differentiates it from the DataSet. The SqlDataReader can only retrieve one row at a time from the data source and in order for it to get the next record, it has to maintain its connection to the data source. The DataSet, however, doesn't need to know about where it gets its data. The DataReader can only get its data from a data source through a managed provider. The DataSet can also get its data from a data source via a managed provider, but the data source can also be loaded manually, even from an XML file on a hard drive. If the .NET Framework does not provide a managed provider that is specifically designed for your database, it is certainly worth checking to see if the manufacturer or a third party has one available since they should perform better than the generic OLE DB and ODBC providers.

In ASP.NET, DataReader objects can be used for more robust situations such as binding themselves to an ASP.NET DataGrid or a DropDownList server control. The following code demonstrates how to retrieve a list of products from the Northwind database using a SqlDataReader object:

string sSQL = "SELECT * FROM Products";
string sConnString =
    "Server=(local);Database=Northwind;Integrated Security=SSPI;";
using (SqlConnection oCn = new SqlConnection(sConnString))
{
    SqlCommand oSelCmd = new SqlCommand(sSQL, oCn);
    oSelCmd.CommandType = CommandType.Text;
    oCn.Open();
    SqlDataReader oDr = oSelCmd.ExecuteReader();
    DataGrid1.DataSource = oDr;
    DataGrid1.DataBind();
}
Both a SqlConnection and a SqlCommand object are created. The SqlConnection is opened and the SqlCommand object executes the SQL query, returning the first row to the SqlDataReader. At this point the connection to the database is still open and associated with the SqlDataReader. This code shows how a SqlDataReader can be bound to a bindable object such as an ASP.NET DataGrid.

Alternatively, a DataReader could be used to retrieve the rows and then loop through them manually, one by one. It can support several resultsets as well. For example, a list of products and categories could be retrieved from a database. The following code retrieves a SqlDataReader and loops through its rows, writing the first column's value for each row to the console:

SqlDataReader oDr = oCmd.ExecuteReader();
while(oDr.Read()) {
    Console.WriteLine(oDr[0]);
}

Supporting Multiple Resultsets

The DataReader supports access to multiple resultsets, one at a time, in the order they are retrieved. This code is easily modified to handle multiple resultsets. The following code retrieves a SqlDataReader and loops through its rows, again writing the first column's value for each row to the console:

SqlDataReader oDr = oCmd.ExecuteReader();
    do {
        while(oDr.Read())
        {
            Console.WriteLine(oDr[0]);
        }
        Console.WriteLine(oDr[0]);
    }
    while(oDr.NextResult());
Once all of the rows from the first resultset are traversed, the rowset from the second query is retrieved and its rows are traversed and written. This process can continue for several resultsets using a single SqlDataReader.

The Read method of the SqlDataReader loads the next record so that it can be accessed and moves the position of the cursor ahead. It returns a Boolean value indicating the existence of more records. This feature can help circumvent a common problem in classic ADO development: the endless loop. In classic ADO, when looping through a recordset object developers would often omit the MoveNext method and run the code only to remember a second too late that this would cause the recordset to loop infinitely on the same record. ADO.NET is kind to developers as its DataReader object's Read method automatically moves the position to the next record so this situation can't occur. The NextResult method of the SqlDataReader object retrieves the subsequent rowset and makes it accessible to the SqlDataReader. It also returns a Boolean value indicating if there are additional resultsets to traverse, like the Read method does.

The DataReader in the previous code sample shows how to get the value for a column from the DataReader using its ordinal index position. Can the DataReader be indexed by the column name or can the index of the column be retrieved? The answer to both of these questions is yes. The code in Figure 1 shows how a DataReader retrieves data and displays the CompanyName for each row in the diagnostics' output window.

Figure 1 Retrieving and Displaying Data (Close Figure 1)
string sConnString =
    "Server=(local);Database=Northwind;Integrated Security=SSPI;";
using(SqlConnection oCn = new SqlConnection(sConnString))
{
    SqlCommand oCmd = new SqlCommand("SELECT * FROM Customers", oCn);
    oCn.Open();
    SqlDataReader oDr =
        oCmd.ExecuteReader(CommandBehavior.CloseConnection);
    while(oDr.Read())
    {
        System.Diagnostics.Debug.WriteLine("Company Name " +
            oDr["CompanyName"]);
        System.Diagnostics.Debug.WriteLine("Company Name: name  = " +
            oDr.GetName(1));
       System.Diagnostics.Debug.WriteLine("Company Name: index = " +
            oDr.GetOrdinal("CompanyName"));
    }
}

To demonstrate these techniques, this code displays the CompanyName value, column name, and index. The first line in the loop writes the CompanyName using the value representing the name of the CompanyName column. This could also have been accomplished by passing the index of 1. I avoid this as it is less clear which column you are accessing, although using the string value is slower than using the index. The second line in the loop writes the name of the column at position 1 (CompanyName) using the GetName method. The third line gets the index of the CompanyName column using the GetOrdinal method of the SqlDataReader. You might also notice that the CommandBehavior is set to CloseConnection, ensuring that the connection will be closed when the SqlDataReader is closed. These methods are simple but they make the SqlDataReader a power tool.

The Disconnected Side

The DataSet is the main data storage tool in the ADO.NET disconnected architecture. Unlike the DataReader, the DataSet is not connected directly to a database through a Connection object when you populate it. Instead, to fill a DataSet from a database you first create a DataAdapter object (such as a SqlDataAdapter) for the provider and associate it with a SqlConnection object. Then the SqlDataAdapter can broker the data retrieval for the DataSet by issuing a SqlCommand against the database through the SqlConnection, retrieving the data, and filling the DataSet.

You can think of the SqlDataAdapter as a bridge between the connected and disconnected objects. One of its purposes is to serve as the route for a rowset to get from the database to the DataSet. For example, when the SqlDataAdapter's Fill method is executed it opens its associated SqlConnection object (if not already open) and issues its associated SqlCommand object against the SqlConnection. Behind the scenes, a SqlDataReader is created implicitly and the rowset is retrieved one row at a time in succession and sent to the DataSet. Once all of the data is in the DataSet, the implicit SqlDataReader is destroyed and the SqlConnection is closed.

The following code shows how a DataSet can be filled from the Products table of the Northwind database. Notice that there is no explicit SqlDataReader object in this code sample:

string sSQL = "SELECT * FROM Products";
string sConnString =
    "Server=(local);Database=Northwind;Integrated Security=SSPI;";
SqlDataAdapter oDa = new SqlDataAdapter();
DataSet oDs = new DataSet();
using(SqlConnection oCn = new SqlConnection(sConnString))
{
    SqlCommand oSelCmd = new SqlCommand(sSQL, oCn);
    oSelCmd.CommandType = CommandType.Text;
    oDa.SelectCommand = oSelCmd;
    oDa.Fill(oDs, "Products");
}
The DataSet can read and load itself from an XML document as well as export its rowset to an XML document. Because the DataSet can be represented in XML, it can be easily transported across processes, a network, or even the Internet via HTTP. Unlike the DataReader, the DataSet is not read-only. A DataSet can be modified, and rows can be added or deleted. Changes to a DataSet can be sent to the database via a managed provider's objects. Another key difference between the DataSet and the DataReader is that the DataSet is fully navigable. Its rows can be traversed forward or backward. The DataReader can be traversed forward only. In addition, a DataSet is highly flexible in that its DataTable objects can be filtered vertically or horizontally and they can be sorted or even searched. The DataSet is independent of any one data provider as it relies on a DataAdapter specific to each provider to broker the data between the DataSet and the database.

Not only can the DataSet be loaded from XML, it can also be loaded manually. Notice in Figure 2 how a DataTable is created and its columns added manually. A primary key constraint is established as well as an auto-incrementing value for the key field. Then the DataTable is added to an empty DataSet (though is doesn't have to be empty) and the rows are added one by one to the DataTable, all without ever connecting to a data source.

Figure 2 Creating a DataSet Manually (Close Figure 2)
//-- Create the table
DataTable oDt = new DataTable("Employees");
DataRow oRow;
oDt.Columns.Add("EmployeeID", System.Type.GetType("System.Int32"));
oDt.Columns.Add("FirstName", System.Type.GetType("System.String"));
oDt.Columns.Add("LastName", System.Type.GetType("System.String"));
oDt.Constraints.Add("PK_Employees", oDt.Columns["EmployeeID"], true);
oDt.Columns["EmployeeID"].AutoIncrement = true;
oDt.Columns["EmployeeID"].AutoIncrementSeed = -1000;
oDt.Columns["EmployeeID"].AutoIncrementStep = -1;
oDs.Tables.Add(oDt);

//-- Add the rows
oRow = oDs.Tables["Employees"].NewRow();
oRow["FirstName"] = "Haley";
oRow["LastName"] = "Smith";
oDs.Tables["Employees"].Rows.Add(oRow);
oRow = oDs.Tables["Employees"].NewRow();
oRow["FirstName"] = "Madelyn";
oRow["LastName"] = "Jones";
oDs.Tables["Employees"].Rows.Add(oRow);

//-- Bind it to a DataGrid
grdEmployees.DataSource = oDs.Tables["Employees"];

Because the DataSet is disconnected, its use can reduce the demand on database servers. It does, however, increase the memory footprint in the tier where it is stored, so be sure to account for that when designing around a DataSet as your data store. Scaling up on the middle tier using parallel servers and load balancing is a common way to handle the increased load so that session-based information can be stored in objects such as the DataSet.

When to Use the DataSet

Your situation will dictate when and where you'll use a DataSet versus a DataReader. For example, the DataSet's disconnected nature allows it to be transformed into XML and sent over the wire via HTTP if appropriate. This makes it ideal as the return vehicle from business-tier objects and Web services. A DataReader cannot be serialized and thus cannot be passed between physical-tier boundaries where only string (XML) data can go.

DataSet objects are a good choice when the data must be edited or rows added or deleted from the database. It is not the only choice, however, as a DataReader could be used to retrieve the data and changes would be sent to the database via a SqlDataAdapter through a separate, self-maintained DataRow array. That process can be quite messy because the SqlDataReader cannot allow edits as the DataSet can. As mentioned earlier, the DataSet is also a good choice when you need data manipulation such as filtering, sorting, or searching. Since the DataSet can be traversed in any direction, all of these features are available. This flexibility also makes the DataSet an easy choice when the situation calls for multiple iterations of the rowset. A DataReader can move forward only, so to loop through it more than once, the DataReader would be closed and reopened and the query would hit the database a second time.

If a rowset is intended to be bound to a single ASP.NET server control and the data is read-only, the DataReader could suffice. If a rowset is intended to be bound to more than one read-only ASP.NET server control, you should consider using a DataSet instead. If a DataReader was bound to more than one control (such as three DropDownList controls), the same query would hit the database three times since the DataReader can only move forward. The DataSet also works well when a rowset must be persisted between page calls to the Session or Cache objects.

Of course, because the DataReader is associated with a specific data source, it cannot be created, filled, or traversed without a connection to the data source. Unlike the DataReader, a DataSet can be created manually without a connection to the source. In a situation such as an online shopping cart in which a custom data store is required, a DataSet could be created manually and its rows added.

Another good use of the DataSet is the situation in which data must be retrieved and a complex action performed on each row. For example, an application might retrieve a hundred stock and mutual fund symbols from a 401k table that needs to be edited. This data might have to include the stock and mutual fund prices on screen, as well. A DataSet could be used to store the rowset and some code could loop through the DataSet and perform a lookup of each stock's price through a third-party Web service. Finally, one of the more compelling reasons to use a DataSet instead of a DataReader is that the DataSet can be serialized when the rowset needs to be passed around a network or the Internet. A DataReader cannot be serialized to XML due to its connected nature.

The DataSet is not the best solution in every situation, and there are several situations in which DataReaders should really be considered. One is when an application implements a .NET architecture without data binding—situations in which manual updates to the database are performed and controls are loaded by looping through rowsets. DataReaders are a good choice when an application has to be sensitive to changes in the underlying database.

There are other times when a DataReader can be the right choice, such as when populating a list or retrieving 10,000 records for a business rule. When a huge amount of data must be retrieved to a business process, even on a middle tier, it can take a while to load a DataSet, pass the data to it on the business tier from the database, and then store it in memory. The footprint could be quite large and with numerous instances of it running (such as in a Web application where hundreds of users may be connected), scalability would become a problem. If this data is intended to be retrieved and then traversed for business rule processing, the DataReader could speed up the process as it retrieves one row at a time and does not require the memory resources that the DataSet requires.

When output or return parameters need to be retrieved, a DataReader will not allow you to get them until the DataReader and its connection are closed. If data must be bound to a read-only list in a Web Form, a DataReader is a very good choice. Binding a DataReader to a DropDownList or even a read-only DataGrid in ASP.NET works well as the data can be retrieved and displayed in the list but does not need to be persisted for editing. DataSets are ideal if data needs to be edited, sorted, filtered, or searched.

As I have shown, when data must be passed without a connection to a database or when rich features for manipulating the data are required, a DataSet fits the bill nicely. The DataReader works well when a simpler purpose for the data exists such as populating a dropdown list or retrieving tens of thousands of rows for processing. Your decision should be based on the particular factors of the situation of the application.

Send your questions and comments for John to mmdata@microsoft.com.

John Papa is a baseball fanatic who spends most of his summer nights rooting for the Yankees with his two little girls, wife, and faithful dog, Kadi. He has authored several books on ADO, XML, and SQL Server and can often be found speaking at industry conferences such as VSLive.


Cutting Edge: Dress Your Controls for Success with ASP.NET 1.1 Themes, Part 2

Cutting Edge
Dress Your Controls for Success with ASP.NET 1.1 Themes, Part 2
Dino Esposito

Code download available at: CuttingEdge0406.exe (143KB)
Contents
The ASP.NET 2.0 Compilation Model
Setting Up a Theme Compiler
Detecting Theme File Changes
Generating the Theme Class
Generating the Dynamic Assembly
Applying the Theme
Adding Support for CSS Styles
Parsing the ASP.NET 2.0 Theme Format
A Word on Reflection
Using the Source Code

Last month I demonstrated an easy way to add theme support to ASP.NET 1.1 applications (see Cutting Edge: Dress Your Controls for Success with ASP.NET 1.1 Themes). I wrote an XML file to represent the theme settings and then I compiled it into a Visual Basic® .NET class and into a C# class using a tool I wrote in Visual Basic. Finally, I added the resulting file to a Web project. Once compiled to a class file, theme settings are used to configure control properties using strongly typed, early-bound code.

You may notice a significant drawback to this model—if you modify anything in the theme file, a new compile step is required to apply and reflect changes. There are a couple of ways to work around this. One is to read from the theme file at run time and use .NET reflection to update any control properties programmatically. No compilation is required, but this would mean that theming would be implemented through late-bound access to controls.

While this doesn't have to slow down your application because it could be designed to happen only once per session, if you're trying to squeeze every little bit of performance out of the code, it's definitely something to avoid. So where can you turn if you don't use .NET reflection?

You could use the ASP.NET runtime built-in mechanism that detects changes to page files and other critical files such as web.config and global.asax. When an .aspx page is modified, its source code is parsed and a class file is generated. The class file is then compiled into an assembly and loaded into the current AppDomain. This procedure takes place dynamically in the background while the application is up and running. As a result, the application promptly detects changes to any of its pages and refreshes them.

You could design a similar mechanism for theme files. Instead of compiling theme files offline, the theme manager component could detect changes to monitored files and create all necessary assemblies on the fly. This model would be more effective because it combines extreme flexibility and early-bound code.

In this column, I'll design and implement a component that parses theme files in memory and builds dynamic assemblies much like the HTTP runtime does for .aspx resources. In addition, I'll briefly discuss the solution based on .NET reflection, which is much easier to code.

The ASP.NET 2.0 Compilation Model

Implementing a form of dynamic compilation in ASP.NET 2.0 will be much easier than it is in ASP.NET 1.1 because the infrastructure exposes a reliable built-in layer. The new build system automatically manages quite a few file types, including theme files. In addition, you can extend it by writing custom build providers to take care of your own resources.

Since its inception, ASP.NET has compiled a few file types on the fly—Web pages (.aspx), Web services (.asmx), HTTP handlers (.ashx), and embedded user controls (.ascx). These files are automatically compiled on demand when first requested by an application. Any changes made to the source of a dynamically compiled file automatically invalidates the corresponding assembly, which will then be recreated. This mechanism greatly simplifies application development as developers only need to save the file and refresh the page to immediately apply changes to the application.

The new ASP.NET build system eliminates the need for an explicit pre-compilation step within the Visual Studio® IDE and provides an extensibility model that allows new file types to be added. By default, C# and Visual Basic class files are detected, as are themes, Web Services Description Language (WSDL) scripts, and XSD schemas for DataSets. WSDL scripts generate Web service proxy classes whereas XSD schemas originate typed DataSets.

In ASP.NET 2.0, there are a few new predefined folders that the build system manages—/Code, /Resources, and /Themes. Their contents are managed and monitored by ASP.NET, which processes files and generates and links assemblies as needed.

Theming and personalization support in ASP.NET 2.0—two features based on dynamic code compilation—are built entirely using the new build provider model. If you're looking for a head start on the ASP.NET 2.0 compilation model, have a look at New Code Compilation Features in ASP.NET Whidbey.

The ASP.NET 2.0 build system is highly customizable and can be extended to support custom files. The key to this change is the <buildProviders> section in the web.config file. The <add> section lets you define a new build provider associated with a given file extension and folder.

<buildProviders>
  <add extension="*.template" appliesTo="Templates"
       type="Samples.MyBuildProvider" />
</buildProviders>

A build provider object is derived from the BuildProvider base class. The appliesTo attribute indicates one or more folders that the provider will monitor and manage. In the end, to implement dynamic compilation of certain resources in ASP.NET 2.0 you only need to write and register a build provider. Although writing a build provider may not be trivial, at least you have a clear path to take and the certainty that the surrounding environment fully supports the feature you're going to build.

In ASP.NET 1.1, on the other hand, you're on your own and must generate the code and manage dependencies. At the end of the day, both in ASP.NET 2.0 and ASP.NET 1.1 the main task of the code you're called to write is the same—the generation of a CodeDOM tree. However, in ASP.NET 1.1 you need to build some infrastructure that decides whether to compile or just load from an already existing assembly.

Setting Up a Theme Compiler

Figure 1 outlines the steps needed to build a dynamic theme compiler for ASP.NET 1.1. The ThemeManager class is the connection point between the application and the infrastructure. A client application just calls the Init method of the ThemeManager class, passing in the name of the theme file:

void Page_Load(object sender, EventArgs e)
{
  if (!IsPostBack) {
    LoadData();
    ThemeManager.Init("brown", this);
  }
}

The Init method triggers a procedure that gets an instance of the specific theme class. Once the theme manager holds that object, it applies the theme and all of its associated CSS style sheets. For the purposes of this column, a theme is an XML file that contains code snippets describing property-value pairs for a number of themeable server controls. Figure 2 shows an abbreviated sample theme file.

Figure 2 Sample Theme File (Close Figure 2)
<Flat>
    <Button>
        <Property>BackColor</Property>
        <Value>Color.WhiteSmoke</Value>
    </Button>
    <Button>
        <Property>BorderStyle</Property>
        <Value>BorderStyle.Solid</Value>
    </Button>
    <Button>
        <Property>BorderWidth</Property>
        <Value>Unit.Pixel(1)</Value>
    </Button>
    <Button>
        <Property>BorderColor</Property>
        <Value>Color.Gray</Value>
    </Button>
    <Button>
        <Property>Font.Name</Property>
        <Value>"Tahoma"</Value>
    </Button>
    <TextBox>
        <Property>Font.Name</Property>
        <Value>"Tahoma"</Value>
    •••
    <DataGrid>
        <Property>Font.Name</Property>
        <Value>"Tahoma"</Value>
    </DataGrid>
    <DataGrid>
        <Property>Font.Size</Property>
        <Value>FontUnit.Point(8)</Value>
    </DataGrid>
    <DataGrid>
        <Property>BorderStyle</Property>
        <Value>BorderStyle.Solid</Value>
    </DataGrid>
    <DataGrid>
        <Property>BorderWidth</Property>
        <Value>Unit.Pixel(1)</Value>
    </DataGrid>
    <DataGrid>
        <Property>BorderColor</Property>
        <Value>Color.Gray</Value>
    </DataGrid>
</Flat>
Figure 1 Building the Compiler
Figure 1 Building the Compiler

The initialization of the theme manager passes through a couple of distinct steps, as you can see here:

public static void Init(string theme, Control root)
{
HandleThemeInfo(theme, root);
HandleCssInfo(theme, root.FindControl("Stylesheet"));
}
The string parameter is the name of the theme. This name is mapped to a theme file using a fixed naming convention. The Control parameter indicates the root of the control's tree to which the theme will be applied. If you set it to the this keyword (the equivalent of the Me keyword in Visual Basic), it indicates the whole page.

The HandleThemeInfo routine first determines if the theme file has changed since last time. If it hasn't, HandleThemeInfo verifies that the corresponding DLL is available in the ASP.NET temporary folder (the same folder where dynamically compiled pages are stored). If all went well, the Init method attempts to load the assembly and grab a reference to the theme object. If either the DLL is missing or the theme source file has changed, the Init method parses the theme contents and generates a C# (or Visual Basic) class. The class is then compiled into a dynamic assembly and loaded into memory, as you can see in Figure 1.

Detecting Theme File Changes

In Windows®, changes to files and folders can be detected in real time using a built-in system component known as the file notification engine. In the Microsoft® .NET Framework, the component is wrapped by the FileSystemWatcher class. An instance of this class works under the covers of the ASP.NET runtime and monitors changes to all the source .aspx files involved with an application. If you embed a file notification handler in the theme manager class, you will be able to catch all changes to the theme file that occur while the Web application is up and running. But what if you stop the application and then update the theme files? In this case, no changes will be detected unless you persist the timestamp of monitored files. Note that a similar mechanism is also employed by ASP.NET to track changes across restarts of the application. To implement a tracking mechanism for theme files, you can borrow some of the ideas and the solutions used within the ASP.NET HTTP runtime.

ASP.NET stores all of its temporary files in a directory below the Microsoft.NET\Framework\v1.1.4322\Temporary ASP.NET Files folder which is, in turn, rooted in the Windows folder. The Temporary ASP.NET Files folder has as many subdirectories as there are Web applications running. Each of these subdirectories contains a couple of child directories whose names are randomly generated strings. At the end of the path you find all the temp files and assemblies that make the application run. You can programmatically obtain the path to this folder using the following call:

string aspNetTempPath = HttpRuntime.CodegenDir;
The folder you get contains an XML file for each page you access in the application. The file has the same name of the ASPX source, plus some extra information and the XML extension:
Sample.aspx.XXX.xml
The contents look like the following code snippet:
<preserve assem="..." type="..." hash="...">
    <filedep name="c:\inetpub\wwwroot\xxx.aspx" />
</preserve>

The hash attribute and the XXX string in the file name contain the timestamp of the page file (the <filedep> node) to which the specified assembly (the assem attribute) refers. Guess what the ASP.NET runtime does at this point? The ASP.NET runtime compares the current timestamp of the dependency (the file in the <filedep> node) with the timestamp persisted in the XML file. If the two match, the assembly is up to date; otherwise, the assembly is obsolete and must be recreated. For effective theming support, you need to implement a similar mechanism. You should note that in ASP.NET 2.0 all of this machinery is still needed, but it is implemented in the new compilation model.

The full source code of the HandleThemeInfo method is shown in Figure 3. If the XML companion file doesn't exist in the temporary folder, the theme file must be parsed and a new assembly must be created. If the companion XML file exists, you read the timestamp of the current assembly and compare it to the current timestamp of the source theme file. The timestamp is expressed as the number of ticks in the DateTime object returned by the LastWriteTime property of the file. If the original theme file is not changed, the HandleThemeInfo method checks if the assembly still exists in the temporary folder. If it does and it's up to date, it gets loaded through the Activator object. File cache change dependencies could also be used to further enhance this process.

Figure 3 HandleThemeInfo (Close Figure 3)
Private Shared Sub HandleThemeInfo(ByVal theTheme As String, _
        ByVal root As Control)
    Dim themeName As String = theTheme.ToLower()
    Dim currentThemeTypeName As String = GetThemeTypeName(themeName)
    Dim themeAssembly As String = GetThemeAssemblyName(themeName)
    Dim infoFile As String = GetThemeInfoFileName(themeName)
    Dim shouldCompile As Boolean = False
    Dim currentTheme As Theme = Nothing

    ' Look for a [theme].xml file in the HttpRuntime.CodegenDir folder
    If Not File.Exists(infoFile) Then
        shouldCompile = True
    End If

    ' Parse the info-file to extract the assembly name and the file
    ' dependency
    If Not shouldCompile Then
        Dim reader As New XmlTextReader(infoFile)
        reader.Read()   ' on the root <info>
        Dim themeObjAssembly As String = reader("assem").ToString()
        Dim themeObjType As String = reader("type").ToString()
        reader.Read()
        reader.Read()   ' on the <filedep> node
        Dim themeOrigPath As String = reader("name").ToString()
        Dim themeTimeStamp As Long = Long.Parse(reader("time").ToString())
        reader.Close()

        ' Compare the timestamp of the XML file and the .theme11 file
        ' If no match is found, the theme object is recreated
        Dim fiTheme As New FileInfo(themeOrigPath)
        If fiTheme.LastWriteTime.Ticks <> themeTimeStamp Then
            shouldCompile = True
        End If
    End If

    ' For some reasons, the DLL doesn't exist
    If Not File.Exists(themeAssembly) Then
        shouldCompile = True
    End If

    ' Compile the theme into a class/assembly in the
    ' HttpRuntime.CodegenDir folder
    If shouldCompile Then
        CreateTheme(themeName)
    End If

    ' At this point, we can safely load the assembly found at the earlier
    ' step
    Dim handle As ObjectHandle = _
        Activator.CreateInstanceFrom(themeAssembly, currentThemeTypeName)
    currentTheme = CType(handle.Unwrap(), MsdnMag.Theme)

    ' Apply the theme
    currentTheme.ApplyRecursive(root)
End Sub

Private Shared Function GetThemeTypeName(ByVal themeName) As String
    Return String.Format("MsdnMag.{0}_theme11", themeName)
End Function

Private Shared Function GetThemeInfoFileName(ByVal themeName) As String
    Return String.Format("{0}\{1}.theme11.xml", HttpRuntime.CodegenDir, _
                         themeName)
End Function

Private Shared Function GetThemeAssemblyName(ByVal themeName) As String
    Return String.Format("{0}\{1}_theme.dll", HttpRuntime.CodegenDir, _
                         themeName)
End Function

Generating the Theme Class

The theme file contains the desired settings for some control properties. These settings must be turned into a class which will then recursively apply them to the specified page. A theme gets into the game when all the controls that make up the page are initialized with their default values. Applying a theme to a control means overriding the properties that are part of the theme after initialization. This is what I did in last month's column and it serves as a summary of how themes work in ASP.NET 2.0. First, a theme class inherits a base abstract class named Theme:

public abstract class Theme
{
  public virtual void ApplyRecursive(Control ctl) {
     Apply(ctl);
     foreach (Control child in ctl.Controls)
        ApplyRecursive(child)
  }
  public abstract void Apply(Control ctl)
}

I compile this base class in a separate assembly and store it in the Global Assembly Cache (GAC). You can also copy the assembly to a public path like the ASP.NET runtime root folder (the v1.1.4322 directory where all system assemblies reside). Note that an assembly can be copied to the GAC only if it is strongly named. In this month's source code, which is available on the MSDN® Magazine Web site, the BaseTheme project is already configured to generate a strongly named assembly.

The code that parses the theme source file and creates the corresponding class is shown in Figure 4. The parsing stage is pretty simple here because I assume that the theme source file has a DataSet-compliant schema. In the end, parsing is reduced to calling ReadXml on a new DataSet object. The CodeDOM code to create the theme class is nearly identical to the code developed last month. I won't cover its details here; check the May 2004 Cutting Edge installment for more information. The code in Figure 5 gives you an idea of the final results in C#. The theme class is named after the source theme and overrides the Apply method to apply settings to an individual control.

Figure 5 Generated Code (Close Figure 5)
//-----------------------------------------------------------------------
// <autogenerated>
//     This code was generated by a tool.
//     Runtime Version: 1.1.4322.573
//
//     Changes to this file may cause incorrect behavior and will be lost
//     if the code is regenerated.
// </autogenerated>
//-----------------------------------------------------------------------

namespace MsdnMag {
    using System.Web.UI.WebControls;
    using System.Web.UI;
    using System.Drawing;

    public class brown_theme11 : Theme {

        public override void Apply(Control ctl) {
            //
            //  ------------------------------------
            //  Class:: Button
            //  ------------------------------------
            if (ctl is Button) {
                Button _button = ((Button)(ctl));
                _button.BackColor = Color.Brown;
                _button.ForeColor = Color.Linen;
                _button.BorderStyle = BorderStyle.Outset;
                _button.BorderWidth = Unit.Pixel(1);
                _button.BorderColor = Color.PeachPuff;
                _button.Font.Name = "Arial";
                _button.Font.Bold = true;
            }
            //
            //  ------------------------------------
            //  Class:: TextBox
            //  ------------------------------------
            if (ctl is TextBox) {
                TextBox _textbox = ((TextBox)(ctl));
                _textbox.Font.Name = "Tahoma";
                _textbox.ForeColor = Color.Maroon;
                _textbox.BackColor = Color.Linen;
                _textbox.BorderStyle = BorderStyle.Inset;
                _textbox.BorderWidth = Unit.Pixel(1);
                _textbox.BorderColor = Color.Chocolate;
            }
            //
            //  ------------------------------------
            //  Class:: DataGrid
            //  ------------------------------------
            if (ctl is DataGrid) {
                DataGrid _datagrid = ((DataGrid)(ctl));
                _datagrid.Font.Name = "Arial";
                _datagrid.Font.Size = FontUnit.Point(9);
                _datagrid.BorderStyle = BorderStyle.Outset;
                _datagrid.BorderWidth = Unit.Pixel(2);
                _datagrid.BorderColor = Color.Brown;
            }
        }
    }
}
Figure 4 Create Theme Class (Close Figure 4)
Private Shared Sub CreateTheme(ByVal themeName As String)
    ' Define the language to be used internally to generate
    ' source/assembly
    Dim lang As String = "c#"    ' or vb

    ' Check the theme file for existence
    Dim fileName As String = GetThemeSourceFile(themeName)
    If Not File.Exists(fileName) Then
        Throw New ArgumentException(String.Format( _
            "The specified theme file ('{0}') _
            does not exist.", themeName))
    End If

    ' Read the content of the file to a DataSet
    ' Assume that the .theme11 file is an XML file that can be read as a
    ' DataSet
    ' Each table in the DataSet has two columns: Property, Value
    Dim themeContents As New DataSet
    themeContents.ReadXml(fileName)

    ' Create a class that inherits from BaseTheme
    Dim sourceFile As String = CodeGenHelper.ParseThemeCode( _
        themeName, themeContents, lang)

    ' Create the XML companion file
    Dim infoFile As String = GetThemeInfoFileName(themeName)
    CodeGenHelper.CreateInfoFile(infoFile, themeName, fileName)

    ' Compile the class to the ASP.NET temp folder
    Dim comp As CodeDomProvider
    Select Case lang.ToLower()
        Case "vb"
            comp = New VBCodeProvider
        Case "c#", "cs"
            comp = New CSharpCodeProvider
    End Select

    CompileThemeClass(comp, sourceFile, themeName)
    Return
End Sub

' *****************************************************************
' Compile the autogenerated theme class
Private Shared Sub CompileThemeClass(ByVal comp As CodeDomProvider, _
        ByVal source As String, ByVal themeName As String)
    Dim asmName As String = GetThemeAssemblyName(themeName)
    If File.Exists(asmName) Then
        SafeRemove(asmName)
    End If

    Dim icc As ICodeCompiler = comp.CreateCompiler()
    Dim cp As CompilerParameters = New CompilerParameters

    cp.ReferencedAssemblies.Add("system.dll")
    cp.ReferencedAssemblies.Add("system.web.dll")
    cp.ReferencedAssemblies.Add("system.drawing.dll")
    cp.ReferencedAssemblies.Add("BaseTheme.dll")
    cp.OutputAssembly = asmName
    cp.IncludeDebugInformation = False
    cp.GenerateExecutable = False
    cp.GenerateInMemory = False

    Dim results As CompilerResults
    results = icc.CompileAssemblyFromFile(cp, source)

    If results.Errors.Count > 0 Then
        Dim msg As String = ""
        For Each err As CompilerError In results.Errors
            msg += String.Format("{0} at {1},{2} {3}", err.ErrorText, _
                                 err.Line, err.Column, vbCrLf)
        Next

        Throw New ApplicationException("Can't compile the theme: " + msg)
        Return
    End If

    Return
End Function

' *****************************************************************
' Remove/rename old assemblies to be replaced
Private Shared Sub SafeRemove(ByVal asmName As String)
    ' Try to delete any old copy of the DLL
    If File.Exists(asmName + ".DELETE") Then
        File.Delete(asmName + ".DELETE")
    End If

    ' Try to delete the DLL
    Try
        File.Delete(asmName)
    Catch ex As Exception
        File.Move(asmName, asmName + ".DELETE")
    End Try
End Sub

Once you have a source file, you should compile and load it in the current AppDomain. In the sample code for this column, I use C# to generate the interim source code of Figure 5. This is totally arbitrary; you can change it to Visual Basic .NET with a little tweak to the code that is shown in Figure 4:

Private Shared Sub CreateTheme(ByVal themeName As String)
   Dim lang As String = "vb"
   •••
End Sub

It is important to note, though, that the language used at this level has nothing to do with the language you use to develop the whole project. You can use C# to generate interim theme classes in a Web application developed with Visual Basic.

The code in Figure 4 controls the creation of the XML companion file, which is needed to track changes to the source theme file. Figure 6 shows how the companion XML is created.

Figure 6 Creating the XML (Close Figure 6)
Public Shared Sub CreateInfoFile(ByVal infoFile As String, _
        ByVal themeName As String, ByVal themeFile As String)
    ' Create the info file
    ' <info assem="xyz" type="MsdnMag.Sample_Theme11">
    '    <filedep name="c:\inetpub\wwwroot\app\themes\sample.theme11"
    '     time="ticks" />
    ' </info>

    Dim xml As New XmlTextWriter(infoFile, Nothing)
    xml.Formatting = Formatting.Indented
    xml.Indentation = 1
    xml.IndentChar = vbTab
    xml.WriteStartElement("info")
    xml.WriteAttributeString("assem", "theme")
    xml.WriteAttributeString("type", "MsdnMag." + themeName + "_theme11")
    xml.WriteStartElement("filedep")
    xml.WriteAttributeString("name", themeFile)
    xml.WriteAttributeString("time", New _
        FileInfo(themeFile).LastWriteTime.Ticks)
    xml.WriteEndElement()
    xml.WriteEndElement()
    xml.Close()
End Sub

Note that you don't need to take any special measures to create, edit, or delete files in the ASP.NET temporary folder (the file returned by the HttpRuntime.CodegenDir property). The reason is that the default ASP.NET account (ASPNET on Windows 2000 and Windows XP and Network Service on Windows Server™ 2003) is always granted full privilege on that folder. If you are impersonating another user, then you should make sure that account can read and write files in that folder. If you don't, themes won't work and the whole ASP.NET application may fail. Note that using the ASP.NET temporary files isn't necessarily a recommended approach in general scenarios, but in this case it makes sense.

Generating the Dynamic Assembly

To programmatically compile a class file, you first get hold of a CodeDomProvider object—specifically, a CSharpCodeProvider or VBCodeProvider object. Next, you must obtain a compiler object by calling the CreateCompiler method, after which you configure the compiler. You reference assemblies, define the output assembly name, choose whether you're compiling to a library or to an executable, and decide whether to include debug information. Basically, you have as many properties and collections to work with as there are switches for the command-line versions of the C# and Visual Basic compilers.

Because you're compiling a class that inherits from a base type, you must ensure that the base type assembly is referenced. In this case, the assembly is BaseTheme, as you saw in Figure 4. A base theme class also exists in ASP.NET 2.0 but it's part of the .NET Framework and as such is brought in by the System.Web assembly.

This code compiles the specified source file into an assembly:

results = icc.CompileAssemblyFromFile(cp, source)

In this month's sample code, I use a fixed convention for the name of a theme assembly—the name of the theme followed by a constant. For example, brown_theme.dll. This is totally arbitrary; you can name the assembly using a randomly generated string if you want. Random names, though, are dangerous because they can lead to a proliferation of useless assembly files, one for each version of the theme you update in the lifetime of the app. Always using the same name ensures that the file will be overwritten.

So, what if you modify the theme while the application is running? In this case, you can't just delete or overwrite the DLL because it is locked by the ASP.NET worker process. But you can rename the file to .DELETE (this is always possible) and create a new one with the same name. Next time the assembly is generated, just get rid of all .DELETE files. The ASP.NET runtime employs a similar procedure to replace its temp files.

Applying the Theme

Now you have an assembly with a theme object that's ready to use. How do you load the assembly and type into memory? Take a look at the following code snippet:

ObjectHandle handle;
handle = Activator.CreateInstanceFrom(themeAssembly,
    currentThemeTypeName);
currentTheme = (MsdnMag.Theme) handle.Unwrap();
The CreateInstanceFrom method on the Activator object loads the assembly from the specified path and returns a reference to the specified type—the type of the dynamically created theme class. CreateInstanceFrom doesn't return the direct object, though. It returns an ObjectHandle object. ObjectHandle boxes the actual object into a reference so that the object can be passed between multiple AppDomains more effectively—that is, without loading the object's metadata in each AppDomain. In this case, you don't really take advantage of this feature because a single AppDomain is involved. In order to get the wrapped object, you call Unwrap and cast to the right type. You should note that all overloads of CreateInstanceFrom that accept an assembly path return ObjectHandle objects. So unboxing the theme object is a necessity, rather than a choice.

Once you have obtained a reference to the dynamically created theme object, you cast it to the base class and call the ApplyRecursive method, as seen here:

currentTheme.ApplyRecursive(root)
As shown earlier, ApplyRecursive recursively calls into the overridden Apply method for each control in the specified tree of controls. Figure 7 shows a sample page skinned in two different ways.

Figure 7 Two Skins
Figure 7 Two Skins

The theme manager component I've discussed so far doesn't require you to bind your app to a fixed number of themes. New theme files can be deployed at all times and are automatically compiled and used on demand when the next page request arrives.

Adding Support for CSS Styles

In ASP.NET 2.0, a theme is made of a skin (control properties) and one or more CSS style sheets. Adding support for CSS styles is also easy in ASP.NET 1.1. Figure 8 lists all the necessary code. To link a CSS style sheet to a Web page you need to define a <link> tag in the <head> section:

<head>
  <link rel=stylesheet href=... />
</head>

Figure 8 Add CSS Support (Close Figure 8)
Private Shared Sub HandleCssInfo(ByVal theme As String, _
                                 ByVal link As HtmlGenericControl)
    Dim themeName As String = theme.ToLower()

    ' Retrieve the CSS file (if any): same name, .css extension
    Dim cssFile As String = GetCssThemeSourceFile(themeName)
    If Not File.Exists(cssFile) Then
        Return
    End If

    ' No <head> control provided
    If link Is Nothing Then
        Return
    End If

    link.Attributes("rel") = "stylesheet"
    link.Attributes("href") = "themes\" + theme + ".css"
End Sub

If you want to add or replace CSS style sheets programmatically, ASP.NET 2.0 comes with a new HTML control named HtmlHead that represents the <head> tag. An instance of this control is automatically created if you add the runat="server" attribute to the tag. Using an instance of the control, the ASP.NET 2.0 theme engine appends as many stylesheet references as needed. A similar trick is used here but involves the <link> tag instead of <head> for the sake of simplicity. The requirement is that any page that can sport a theme includes the following markup code:

<head>
  <link runat="server" id="Stylesheet" />
</head>

The ID set to Stylesheet is arbitrary, but I preferred to keep it fixed for simplicity. The name of the CSS is fixed too and matches the name of the theme file, but with the .css extension. This aspect of the CSS support in themes is arbitrary and can be easily generalized. In ASP.NET, any tag marked with the runat="server" attribute is transformed in a server control. If a tag-specific control does not exist, the tag is mapped to the HtmlGenericControl class. For example, the <head> tag is an instance of HtmlGenericControl in ASP.NET 1.1 but maps to HtmlHead in ASP.NET 2.0. The <link> element is mapped to HtmlGenericControl:

link.Attributes("rel") = "stylesheet"
link.Attributes("href") = "themes\" + theme + ".css"
Using the Attributes collection of the class, you can bind CSS documents dynamically to the page.

Parsing the ASP.NET 2.0 Theme Format

The theme format used so far is radically different from that used in ASP.NET 2.0. I opted for the XML schema in Figure 2 for simplicity. The ASP.NET 2.0 format is smarter and easier to use. In fact, it consists of snippets of ASPX controls markup, as shown in the following lines of code:

<asp:Button runat="server"
     Font-Bold="true"
     BorderColor="#585880"
     BorderWidth="1pt"
     ForeColor="#585880"
     BackColor="#F8F7F4" />

Basically, you could design the skin of a control in Visual Studio. Then you could just copy and paste the control's markup to the theme file. Parsing a text file made of a sequence of markup blocks is not as easy as building a DataSet from some XML text. However, with some help from regular expressions and a little effort on your own, you can make it through. It is interesting to note that in this case you can use any new compelling ASP.NET 2.0 themes in your ASP.NET 1.1 applications.

A Word on Reflection

At the beginning of this column I mentioned that themes can also be implemented using reflection, thus avoiding parsing and class compilation. Let me show you a quick trick that can be used to import ASP.NET 2.0 themes in ASP.NET 1.1 using reflection. The prerequisite to the trick is that you own a component that parses the ASP.NET 2.0 format and returns a collection of property/value pairs for each themeable control. Given a page control, the following code sets the BackColor property using the string representation of a value:

   Dim desc As PropertyDescriptor
   desc = _
   TypeDescriptor.GetProperties(ctl) _
      ("BackColor")
   Dim c As TypeConverter = desc.Converter
   Dim value As Object = _
      c.ConvertFromString("Cyan")
   desc.SetValue(ctl, value)
This code snippet is logically equivalent to the following strongly typed code:
   ctl.BackColor = Color.Cyan

If you dynamically parse an ASP.NET 2.0 theme file and set control properties through reflection you get the same effect I discussed earlier. If you want to generate code from an ASP.NET 2.0 theme format you must slightly modify the CodeDOM tree built in this column. In particular, you should replace code snippet expressions with code primitive expressions. (See this month's code download.)

Using the Source Code

The code download consists of three projects: BaseTheme, ThemeManager, and a sample Web application. To add theming capabilities to a new application, you add a reference to the ThemeManager which, in turn, requires a reference to BaseTheme. Next, in the Page_Load event, or wherever else it's appropriate in your application, you call the Init method on the ThemeManager class. Any theme file you plan to use must be deployed in a Themes folder below the application's root virtual folder. You should deploy the BaseTheme either in the GAC or in a path where the compiler can locate it. This requirement doesn't apply to ThemeManager, though having both assemblies installed in the same location wouldn't be a bad idea. Both class library projects are configured to output an assembly with a strong name, as shown in Figure 9. The key pair is stored in the BaseTheme folder and the AssemblyInfo file references it.

Figure 9 ThemeManager
Figure 9 ThemeManager

Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is a Wintellect instructor and consultant based in Italy. Author of Programming ASP.NET and Introducing ASP.NET 2.0 (both from Microsoft Press) , he spends most of his time teaching classes on ASP.NET and ADO.NET and speaking at conferences. Reach Dino at cutting@microsoft.com or join the blog at http://weblogs.asp.net/despos.


The XML Files: InfoPath 2003 SP1 Preview

The XML Files
InfoPath 2003 SP1 Preview
Aaron Skonnard

Code download available at: XMLFiles0406.exe (131KB)
Contents
Getting Started
Compatibility
Features and Enhancements
Tablet PC Support
Security Enhancements
E-mail Deployment
Extensibility and Integration
Toolkit for Visual Studio .NET
Sample InfoPath Project
Summing It Up

The recently released InfoPath 2003 SP1 Preview provides a sneak peek at the InfoPath enhancements that will ship with Microsoft® Office 2003 SP1 around the middle of 2004. The preview is packed with new features, others which improve the user experience and some which improve the developer experience by increasing flexibility and control. I'll walk through several of these new features with a focus on those that apply to developers.

Getting Started

The initial InfoPath 2003 release helped businesses begin to realize the power and flexibility that comes from using XML and Web services throughout the enterprise. Most already understood the basic interoperability benefits of these technologies, but it wasn't until the release of InfoPath 2003 that they were able to visualize how members across their organizations could tap into their XML resources.

In a nutshell, InfoPath 2003 makes it extremely easy to build forms based on XML Schema or Web Services Description Language (WSDL) definitions. It's so easy that anyone, regardless of their technical background, can do it. Within minutes you can have a form up and running that's capable of producing documents or SOAP messages that conform to a specific WSDL contract. You can check out the September 2003 MSDN® Magazine article titled "Turn User Input into XML with Custom Forms Using Office InfoPath 2003" for additional background information on the basic InfoPath 2003 features.

Although the initial release was a major breakthrough in many ways, it was wanting in a few areas. The recently released service pack preview provides several major improvements right away rather than waiting for the next major release. The significance of these improvements makes this release feel less like a service pack and more like a major upgrade.

To get started, you can download and install the InfoPath 2003 SP1 Preview from Microsoft Office Online. It's a full installation so you have to remove your current version of InfoPath before running the installer (unlike the process with a typical service pack).

Compatibility

Compatibility is one of the most common concerns when installing a service pack or an upgrade. If you've built large systems using the current InfoPath 2003 release you'll want to make sure that installing the service pack won't break anything. In the case of InfoPath 2003 SP1 Preview, there are several options available that allow you to maintain compatibility with existing forms while experimenting with the new features. Forms created with the current version of InfoPath 2003 are 100 percent compatible with the SP1 Preview.

After running the installation, InfoPath 2003 can take advantage of the new service pack features, but you don't have to use them. You can disable them by selecting "Disable Service Pack features" from Tools | Options | Advanced (a global program setting). The next time you start InfoPath 2003, you won't be able to use the new service pack features in any of the forms you're designing. In fact, you won't even be able to open forms that employ SP1 features until you reenable support through this dialog. This ensures compatibility with users who do not have SP1 installed. Even with the feature enhancements disabled, you'll still be able to take advantage of the performance updates, bug fixes, and so on.

Assuming you haven't disabled the service pack features globally, you can also control support for service pack features on a form basis. For example, when you open a form created using the current version of InfoPath 2003, a dialog will ask you if you want to enable the service pack features in the existing form. If you choose "Yes," you'll be able to take advantage of the new features, but only users with SP1 installed will be able to work with your form. If you choose "No," the form will remain compatible with the current release, but you won't be able to use the new features. If you decide later that you'd like to add the service pack features to your form, you can always select "Add Service Pack Features to Form" from the Tools menu.

Figure 1 Form Options Dialog
Figure 1 Form Options Dialog

The new service pack features are automatically enabled when you create new forms with SP1 installed (unless, of course, you disable the new service pack features globally). If you create a form that uses service pack features and later decide that you need a version that can be used without SP1 installed, you can always export such a version from the Form Options dialog (check Figure 1 to see where the options are located in this dialog). However, one thing to keep in mind is that the exported copy will lose all of the enhanced service pack features.

Features and Enhancements

As with most service packs, the InfoPath 2003 SP1 Preview includes various bug fixes, performance improvements, and security enhancements. In addition to these standard service pack items, SP1 is packed with compelling functional enhancements that improve the user experience and give developers more control over form design and functionality.

These enhancements include a variety of new controls and layout options. For example, SP1 introduces a new master/detail control that models traditional master/detail relationships commonly found in data sources. In addition, SP1 introduces a file attachment control that allows users to attach local files to a form easily. There are several other new layout controls available including a repeating recursive section, a choice section, a choice group, and a repeating choice group, among others.

There's also a new condition builder feature inside that enables form designers to dynamically change form options available to users based on roles or other criteria. In addition to InfoPath's built-in controls, SP1 makes it possible to author and use custom controls in your forms.

The InfoPath 2003 SP1 Preview also gives you more control over form display and printing. Form designers can insert page breaks that take effect only during printing but not while a user is filling out the form. There are also new font enhancements that make for better digital form reproduction.

Other enhancements include better support for secondary data sources, better DataSet (and DiffGram) support for sending updates back to a database, and increased protection against data loss through auto-save and auto-recovery functionalities that are available when the user is filling out forms.

However, some of the most compelling enhancements revolve around Tablet PC inking, security and signatures, improved e-mail deployment, form extensibility and integration with other technologies, and the new support for Visual Studio .NET development. I'll cover each of these areas in more detail.

Tablet PC Support

SP1 comes with many enhancements that benefit Tablet PC users. For starters, InfoPath automatically detects when you're filling out a form on a Tablet PC and allows you to enter data in "ink" mode. You simply write over the form's controls by hand and InfoPath uses an ink-to-text transformation to fill it in. This process is completely automatic at run time and transparent to the form developer at design time.

Figure 2 Ink Picture Control
Figure 2 Ink Picture Control

You can also design forms specifically for Tablet PC users with the Ink Picture control. This control allows users to save ink drawings or handwriting directly into a form. The form shown in Figure 2 illustrates how you could use the Ink Picture control to store a handwritten signature on a form field entered using the "inking" capabilities of a Tablet PC. When the user fills out this form and saves it, the signature will be stored as encoded binary data in an XML element (of type xsd:base64Binary) so that other applications can then process it.

Security Enhancements

SP1 introduces three security levels (restricted, domain, and full trust) that specify the level of trust required by a given form. You can access this setting by selecting Tools | Form Options | Security. Restricted trust means the form cannot access data outside the form, domain trust means the form can access data from the domain in which it's located, and full trust means the form has access to files and data on the user's computer.

Form designers can enable full trust by digitally signing a form with a digital certificate. Then users filling out the form can define which publishers they fully trust, thus allowing only trusted form publishers access to files and settings on their computers. However, users can always deny all form access to their local files and settings when they so desire.

You can also require users to sign their filled-out forms before submitting them. You can have users sign the entire form or just portions of it. You can even specify elements of your form that you want signed. Each form element may be signed independently—a feature known as co-signing. Or you can have each new signature sign the preceding signatures, using a feature known as counter-signing. These flexible signature techniques make it possible to provide verification and non-repudiation in a variety of ways to meet your legal needs.

For example, assume you have a simple form that contains several elements, but only two—SSN and CreditCard—need to be signed. You can specify this while designing the form by going to Tools | Form Options | Digital Signatures and indicating which elements need to be signed (see Figure 3).

Figure 3 Defining Elements to Be Signed
Figure 3 Defining Elements to Be Signed

Then when a user fills out this form he can sign these specified elements simply by pressing the Digital Signatures button on the toolbar (the little red ribbon). Performing this action opens the Digital Signatures dialog, which allows the user to add and remove signatures from a given form. Then when the user presses the Add button, the Digital Signature Wizard allows him to select a particular form element that he would like to sign.

After going through the wizard, you'll have a new signature saved into the form. The wizard allows you to select a certificate with which to do the signing and it allows the user to enter a comment to include in the signature. Adding extra information to the signature provides you with some flexibility that can help indicate the intention of the signer.

You can even see the form as it existed when a user digitally signed it. Simply press the View Signed Form button on the Digital Signatures dialog and you will see the form exactly as it looked when it was originally signed.

E-mail Deployment

When you e-mail a form template with InfoPath 2003 and other people try to open it, they'll typically get an error saying that the file has been moved or copied from its original location because they don't have access to the original form template. This is a result of the way the template's path is saved in the .xsn file for versioning purposes.

The preview release improves support for the scenario in which you want to deploy forms via e-mail and update trusted solutions without worrying about versioning. You can simply choose "Send as Attachment" from the file menu and InfoPath will package the form template into an attachment that others will be able to open without having access to the original form template.

Extensibility and Integration

SP1 makes it possible to extend and integrate with InfoPath forms in a variety of ways. One new way to extend the functionality of InfoPath forms is through support for ActiveX® controls, which allows you to take advantage of the wide variety of UI controls developed in the COM era. On a similar note, SP1 also introduces improved support for ADO.NET DataSets and its native DiffGram format for representing changes to a resultset.

InfoPath 2003 introduced an Expression Box control for evaluating simple XPath expressions across different fields in the form. SP1 enhances that capability by providing a palette of built-in functions including avg, sum, max, min, now, today, concat, and several others (some of these are provided by XPath while others are InfoPath extension functions). Before SP1 you had to write macros and script to accomplish these tasks.

You can also use XPath expressions to enhance form validation at run time. InfoPath will automatically provide XML Schema validation while users fill out forms. SP1 also introduces support for declarative (XPath) business rules that you can associate with any control to enhance run-time validation logic. XPath business rules allow you to express constraints that are not possible to describe in XML Schema.

In addition to these features, SP1 provides better support for XML Schema extensibility. For example, the current version of InfoPath has a hard time dealing with changes to the form's underlying schema after you've begun designing the form. This forces you to start over after making a change to the underlying schema. SP1 now provides built-in support for schema changes and complete changes to a form's underlying data source. This makes it quite easy to evolve an InfoPath solution over time.

Toolkit for Visual Studio .NET

The last new feature I'm going to discuss is by far the most compelling. SP1 makes it possible to integrate InfoPath with Visual Studio® .NET through a plug-in called the InfoPath 2003 Toolkit for Visual Studio .NET. This is a separate download that you can get from Microsoft Office Online. It requires you to have SP1 already installed.

Once you've installed the toolkit, you'll see some new project types in the Visual Studio .NET New Project dialog (Microsoft Office InfoPath Projects). You can create a new InfoPath Form Template project in either C# or Visual Basic® .NET.

When you create a new InfoPath Form Template project, the Microsoft Office Project Wizard runs and asks if you want to create a new InfoPath form template or base the project on an existing template that you've already designed. In the latter scenario, the form is converted to managed code, but the developer may need to rewrite business logic as script code is not converted to C# or Visual Basic .NET.

Once you press Finish, the wizard creates your new InfoPath form template and integrates it with your new Visual Studio .NET project. You'll see a new InfoPath window open with the new form you need to design as well as your Visual Studio .NET project.

If you browse the Visual Studio .NET Solution Explorer window, you'll notice that various InfoPath artifacts are part of your project (see Figure 4). These artifacts include the form manifest, template, view, and the data source the form is based on. In this case, I based the InfoPath form on an existing XML Schema definition named invoice.xsd.

Figure 4 InfoPath Artifacts
Figure 4 InfoPath Artifacts

The project also contains a C# source file, FormCode.cs, where you can put code to respond to InfoPath form events. When you build the project, Visual Studio compiles the code into an assembly, which is embedded in the InfoPath form template in order to simplify deployment.

To open the InfoPath form from the Solution Explorer, right-click on the solution node and select Open InfoPath. Then you can design your form using traditional InfoPath techniques.

You can add code to handle form events in a variety of ways. You can select from several events listed in the Tools | Programming menu including On Load, On Switch Views, On Context Change, and On Sign. You can also add code to respond to an event for a particular control. Simply open the control's property window and press the Edit Form Code button or navigate to the control's validation event handlers and select the one you want to handle. For example, Figure 5 shows the property window for a button control on an InfoPath form.

Figure 5 Property Window
Figure 5 Property Window

When you press the Edit Form Code button, InfoPath takes you back into Visual Studio .NET and adds an event handler that looks like the one shown here:

[InfoPathEventHandler(MatchPath="Button1",
  EventType=InfoPathEventType.OnClick)]
public void Button1_OnClick(DocActionEvent e)
{
   // Write your code here
   •••
}
The event handler methods are annotated with an [InfoPathEventHandler] attribute to help InfoPath figure out which method to call at run time. The code you write inside the event handler can access the underlying XML document to read or modify values as necessary. So extending an InfoPath form with custom logic is simply an exercise in XML DOM programming.

When you use the F5 function key, Visual Studio .NET loads your form for you and allows you to fill it out. You can set breakpoints in your event handlers to step through your code when the events fire. This tight integration with Visual Studio .NET provides a whole new level of control and flexibility to the development of InfoPath forms.

Sample InfoPath Project

I've provided a complete sample project for you to download from the MSDN Magazine Web site. It contains the sample invoice form shown in Figure 6. This form allows a customer to fill out an invoice and it dynamically displays the total along the way. It needs to calculate the extended price (quantity * price) of each line item and then sum the results in order to display the invoice total (displayed at the bottom). This is something that you cannot accomplish with the XPath sum function alone, but it's quite easy to do by writing just a few a lines of C# code.

Figure 6 Sample Invoice Form
Figure 6 Sample Invoice Form

The total needs to be recalculated whenever a quantity or price value changes in the list of line items. You can do this by adding an OnAfterChange event handler for each data source field. Browse to the data source view in InfoPath and navigate to the Quantity and Price fields. Right-click on the field, select Properties, and navigate to the Validation and Event Handlers tab. Then select the OnAfterChange event handler (at the bottom of the dialog) and press Edit. This will take you back to Visual Studio .NET and it will insert a new event handler method in your code (see Figure 7).

Figure 7 Event Handlers for Invoice Sample Form (Close Figure 7)
// The following function handler is created by Microsoft
// Office InfoPath. Do not modify the type or number of
// arguments.
[InfoPathEventHandler(
    MatchPath="/tns:Invoice/tns:LineItems/tns:LineItem/tns:Price",
    EventType=InfoPathEventType.OnAfterChange)]
public void Price_OnAfterChange(DataDOMEvent e)
{
    // Write code here to restore the global state.

    if (e.IsUndoRedo)
    {
        // An undo or redo operation has occurred
        // and the DOM is read-only.
        return;
    }

    // recalculate total
    double total=0;
    IXMLDOMNodeList nl =
        e.Source.ownerDocument.selectNodes("//tns:LineItem");
   foreach (IXMLDOMNode n in nl)
   {
      string q, p;
      q = n.selectSingleNode("tns:Quantity").text;
      p = n.selectSingleNode("tns:Price").text;
      if (!q.Equals("") && !p.Equals(""))
      total += double.Parse(q) * double.Parse(p);
   }
   e.Source.ownerDocument.selectSingleNode("//tns:Total").text =
                                           total.ToString();
}

// The following function handler is created by Microsoft
// Office InfoPath. Do not modify the type or number of
// arguments.
[InfoPathEventHandler(
    MatchPath="/tns:Invoice/tns:LineItems/tns:LineItem/tns:Quantity",
    EventType=InfoPathEventType.OnAfterChange)]
public void Quantity_OnAfterChange(DataDOMEvent e)
{
   Price_OnAfterChange(e);
}

Notice that the event handler looks different from the button example. In this case, the InfoPathEventHandler attributes contain a complete XPath expression to specify the field in the data source to which this event responds. I've added one of these for each field, but the implementation for Quantity_OnAfterChange simply calls Price_OnAfterChange.

Notice that the implementation of Price_OnAfterChange accesses the XML document using the DOM API and several XPath expressions through calls to selectSingleNode. This is how I extract the quantity and price of each line item. Then after I've calculated the total, I inject it back into the underlying XML document using the same technique.

It's important to note that InfoPath does not use the managed System.Xml APIs, but rather a build of MSXML 5.0, which is unmanaged. The toolkit comes with several InfoPath interop assemblies that simplify the interaction with these unmanaged components in your event handlers.

Summing It Up

InfoPath 2003 SP1 is packed with new features, many of which make it possible for developers to build more usable, secure, and powerful solutions that are easier to maintain over time. The most compelling new feature is the integration with Visual Studio .NET, which allows you to write managed code that extends the capabilities of your InfoPath forms.

Send your questions and comments for Aaron to xmlfiles@microsoft.com.

Aaron Skonnard teaches at Northface University in Salt Lake City. Aaron coauthored Essential XML Quick Reference (Addison-Wesley, 2001) and Essential XML (Addison-Wesley, 2000), and frequently speaks at conferences. Reach him at http://www.skonnard.com.


.NET Column: Unexpected Errors in Managed Applications

.NET
Unexpected Errors in Managed Applications
Jason Clark

Code download available at: NET0406.exe (123KB)
Contents
How .NET Deals with Unhandled Exceptions
Exceptions and Code Access Security
Some Best Practices
Enforcing Best Practices with FxCop

In this installment of the .NET column I am going to present a handful of tips for dealing with unexpected errors. While it is common practice to catch exceptions to deal with expected failure cases, it is also common for managed applications to encounter errors for which the application is completely unprepared. In an ideal world you will establish clear policies for how your application discovers and responds to unhandled exceptions, reduced security permissions, and other related edge-cases, but in practice these are often overlooked.

Error handling infrastructure is a deceptively complex topic, and you can easily be overwhelmed by details, so I'll address a few relatively simple operations that are broadly applicable for dealing with unexpected errors.

How .NET Deals with Unhandled Exceptions

The common language runtime (CLR) has a set of rules on managing unhandled exceptions. But if you are using the Windows® Forms classes, the Windows Forms message pump applies its own unhandled exception policy, which differs from the policy of the CLR. Add to that the fact that .NET Framework hosts such as ASP.NET impose their own standards on unhandled exceptions, and you can see where the topic begins to get complicated. Still, it's important that managed applications take deliberate steps to deal with unhandled exceptions.

To help you understand unhandled exceptions, let's first take a brief look at handled exceptions. The code in Figure 1 shows two methods, one that calls the other. Admittedly, this code is strange, but it conveys a point. When MethodA calls MethodB, MethodB throws an exception of type InvalidOperationException. This causes normal execution to stop at the point of the thrown operation, and the execution engine searches up the call stack for a compatible catch handler. Because there is a catch clause in MethodA that handles this type of exception, the execution engine finds the compatible catch clause in MethodA and executes it. Once the catch block is completed, MethodA continues running normally, starting with the code following the try/catch block. This is an example of a handled exception; it is indicative of the type of exception handling that most managed applications must perform in response to expected error cases when calling into a class library and managed application code.

Figure 1 Handling Exceptions (Close Figure 1)
void MethodA() {
   try {
      MethodB(1);
   } catch (InvalidOperationException e) {
      // place handling code here
   }
}

void MethodB(Int32 num) {
    if (num == 1) {
       throw new InvalidOperationException();
    } else {
       throw new ArgumentException();
    }
}

However, if MethodA called MethodB with an argument of any value not equal to 1, MethodB would throw an ArgumentException, for which MethodA does not provide a handler. Now if none of the methods further up in the call stack provide an ArgumentException-compatible handler either, then that exception is an unhandled exception.

The code in Figure 2 throws an unhandled exception in response to the user pressing any of its five buttons. Each button represents a different threading context from which an exception can go unhandled. To get a feel for how managed code responds to unhandled exceptions, compile the code in Figure 2 as a console application, run it (but not under the debugger), and start pressing buttons here and there.

Figure 2 ThrowsUnhandledExceptions.cs (Close Figure 2)
using System;
using System.Threading;
using System.Drawing;
using System.Windows.Forms;

class ExceptionForm : Form {

   public ExceptionForm() {

      String[] buttonStrings = {
         "Throw From This Button's Event Handler",
         "Close Window and Throw After the Fact",
         "Create a Thread and Throw From It",
         "Throw From a Thread-pool Thread",
         "Throw From a Finalizer Method"
         };

      for (Int32 index = 0; index < buttonStrings.Length; index++) {
         Button button = new Button();
         button.Text = buttonStrings[index];
         button.Size = new Size(224, 24);
         button.Location = new Point(8, 11 + index * 32);
         button.Click += new EventHandler(OnButtonClick);
         Controls.Add(button);
      }

      ClientSize = new Size(240, 174);
      Text = "Exception Form";
   }

   private void OnButtonClick(Object sender, EventArgs args) {

      switch (((Button)sender).Text) {
      case "Throw From This Button's Event Handler":
         throw new InvalidOperationException();

      case "Close Window and Throw After the Fact":
         App.shouldThrowOnExit = true;
         Close();
         break;

      case "Create a Thread and Throw From It":
         Thread t = new Thread(new ThreadStart(ThreadMethod));
         t.Start();
         break;

      case "Throw From a Thread-pool Thread":
         ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadPoolMethod));
         break;

      case "Throw From a Finalizer Method":
         CreateObjectThatThrowsOnFinalize();
         GC.Collect();
         GC.WaitForPendingFinalizers();
         break;
      }
   }

   void ThreadMethod() {
      throw new InvalidOperationException();
   }

   void ThreadPoolMethod(Object param) {
      throw new InvalidOperationException();
   }

   void CreateObjectThatThrowsOnFinalize() {
      new ThrowsOnFinalize();
   }

   class ThrowsOnFinalize {
      ~ThrowsOnFinalize() {
         throw new InvalidOperationException();
      }
   }
}

class App {

   internal static Boolean shouldThrowOnExit = false;

   public static void Main() {
      Application.Run(new ExceptionForm());
      if (shouldThrowOnExit)
         throw new InvalidOperationException();
   }
}

The behavior represented in this application is surprising the first time you run across it. In particular, you might notice that pressing any of the bottom three buttons seems to have no effect on the application whatsoever, even though they are causing unhandled exceptions to be thrown! In certain cases, exceptions that are unhandled are swallowed whole by the CLR (although this behavior is changing in the .NET Framework 2.0). Here is a summary of the unhandled exception handler default behaviors that are executing in this application.

  • Unhandled exceptions that occur on the application's main thread cause the application to terminate.
  • Unhandled exceptions that occur in threads other than the application's main thread are swallowed by the CLR. This includes manual threads, thread pool threads, and the CLR's finalizer thread. If your application is a console application, the CLR outputs exception text to the console (though your application keeps running). If your application is not a console application, there is no outward indication when an exception occurs and your application keeps running.
  • Unhandled exceptions that occur on a thread that is pumping window messages via the Windows Forms classes are subject to the Windows Forms unhandled exception handler. A debug dialog box is produced by default, but this behavior can be overridden (more on this in a moment).

Now let's take a look at what applications should do, in abstract terms, in the event of an unhandled exception.

In an attempt to reduce the number of unhandled exceptions in their applications, many developers catch the base Exception type, like I've done in the following snippet:

try {
   •••
} catch (Exception e) {
   // handle any CLS-compliant exception
}

Counter to what you may think, code like this is almost always undesirable and actually reduces the robustness of the application. Here's why: there are only a miniscule number of cases in which your code can be sure that it knows how to appropriately handle any possible exception that escapes a try block. There are far more unexpected exception cases than you might imagine. In managed code, even code that doesn't call any methods could throw a variety of exceptions, including security exceptions, out-of-memory exceptions, execution-engine exceptions, and the like. It is best for your applications to always catch specific exceptions and leave all other unexpected exceptions as unhandled. I will look at this concept later in this column.

When an exception goes unhandled, it is a good rule of thumb to shut down the application. An unhandled exception is a bug in your application, and continuing to execute in an undefined state runs the risk of data corruption and other ugly problems. Certain server-side applications are sophisticated enough to unload only the state and threads for a particular client in the event of unhandled exceptions. But most client-side applications, and many server-side applications, are not as sophisticated and the process should be shut down (and perhaps automatically restarted).

Of course, this means that your application cannot live with the default unhandled exception behavior exhibited by the application in Figure 2. This behavior leaves a fair number of unhandled exceptions ignored by the application, and in one of the cases a dialog box that is guaranteed to mystify the user is displayed. Fortunately, very little code is required to improve things dramatically.

The code in Figure 3 shows a boilerplate implementation of the Main entry point that sets up an unhandled exception handler event for both the CLR unhandled exception handler and the Windows Forms unhandled exception handler. This is done by providing handlers for the AppDomain.UnhandledException event and the Application.ThreadException event.

Figure 3 Handling Unhandled Exceptions (Close Figure 3)
class App {

   public static void Main() {
      try {
         SubMain();
      } catch (Exception e) {
         HandleUnhandledException(e);
      }
   }

   public static void SubMain() {

      // Setup unhandled exception handlers
      AppDomain.CurrentDomain.UnhandledException += // CLR
         new UnhandledExceptionEventHandler(OnUnhandledException);

      Application.ThreadException += // Windows Forms
         new System.Threading.ThreadExceptionEventHandler(
             OnGuiUnhandedException);

      // Start application logic
      // Perhaps call to Application.Run(...);

   }

   // CLR unhandled exception
   private static void OnUnhandledException(Object sender,
      UnhandledExceptionEventArgs e) {
      HandleUnhandledException(e.ExceptionObject);
   }

   // Windows Forms unhandled exception
   private static void OnGuiUnhandedException(Object sender,
      ThreadExceptionEventArgs e) {
      HandleUnhandledException(e.Exception);
   }

   static void HandleUnhandledException(Object o) {
      Exception e = o as Exception;

      if (e != null) { // Report System.Exception info
         Debug.WriteLine("Exception = " + e.GetType());
         Debug.WriteLine("Message = " + e.Message);
         Debug.WriteLine("FullText = " + e.ToString());
      } else { // Report exception Object info
         Debug.WriteLine("Exception = " + o.GetType());
         Debug.WriteLine("FullText = " + o.ToString());
      }

      MessageBox.Show("An unhandled exception occurred " +
         "and the application is shutting down.");
      Environment.Exit(1); // Shutting down
   }
}

Notice that in this example the exception information is only reported as debug output. However, for a shipping application it would make more sense to report the error information to an event log, a file, or to isolated storage. The companion code to this column includes an implementation of the application in Figure 2 that reports its unhandled exception data to a file (the download is available on the MSDN® Magazine Web site).

In addition to reporting failure information in such a way that you can fix the bug in future releases of your software, using unhandled exception notifications provides a way to make your application behave the same in the event of any unhandled exception. This improves the determinism and robustness of your applications.

Note that for ASP.NET applications the analog to the Windows Forms Application.ThreadException event is the HttpApplication.Error event. You can define a handler for the event in your Global.asax file by defining an Application_Error method, or you may register for the event directly in code. ASP.NET applications have an additional feature that redirects to a page in the case of an unhandled exception. To set this to a page other than the default, you may specify a <customErrors defaultRedirect="..." /> element and attribute in your web.config file.

Exceptions and Code Access Security

All managed applications can be subject to a security sandbox known as Code Access Security (CAS). The idea behind CAS is that your code can execute as partially trusted, which means that the CLR's just-in-time compiler, verifier, and class library impose certain restrictions on the actions that your code can perform.

The CAS features of the CLR are quite powerful, and for cases in which you want to locally execute code that you don't trust, it is great. But many developers aren't aware of just how easy it is to bump into CAS restrictions when you don't expect them. It surprises me how little the tools and documentation call out this aspect of managed code. Let's look at an example:

class App {
   public static void Main() {
      String[] entries = Directory.GetFileSystemEntries(@"C:\");
      foreach (String s in entries) Console.WriteLine(s);
   }
}

This code lists the names of all of the files and subdirectories in the root of your C: drive. Compile and run this code as a console application, and you should see a list of directory entries. Now copy the executable file (no need to rebuild) to a share somewhere on your network, and run the application. You should see output to your console that looks something like Figure 4.

Figure 4 Security Exception Console Output (Close Figure 4)
Unhandled Exception: System.Security.SecurityException: Request for the
permission of type System.Security.Permissions.FileIOPermission,
mscorlib, Version=1.0.5 000.0, Culture=neutral,
PublicKeyToken=b77a5c561934e089 failed.
   at System.Security.CodeAccessSecurityEngine.CheckHelper(PermissionSet
      grantedSet, PermissionSet deniedSet, CodeAccessPermission demand,
      PermissionToken permToken)
   at System.Security.CodeAccessSecurityEngine.Check(PermissionToken
      permToken, CodeAccessPermission demand, StackCrawlMark& stackMark,
      Int32 checkFrames, Int32 unrestrictedOverride)
   at System.Security.CodeAccessSecurityEngine.Check(CodeAccessPermission
      cap, StackCrawlMark& stackMark)
   at System.Security.CodeAccessPermission.Demand()
   at System.IO.Directory.GetFileSystemEntries(String path, String
      searchPattern)
   at System.IO.Directory.GetFileSystemEntries(String path)
   at App.Main()

The state of the failed permission was:
<IPermission class="System.Security.Permissions.FileIOPermission,
mscorlib, Version=1.0.5000.0, Culture=neutral,
PublicKeyToken=b77a5c561934e089"
             version="1"
             PathDiscovery="C:\."/>

The most immediately useful information in this exception output are the first four lines which state that "Request for the permission of type ... FileIOPermission ... failed" and the last line which shows the resource to which the code was denied access: "C:\."

What surprises many developers is that just copying their app to another location in the network causes the app—which was previously working just fine—to crash! No code or system configuration changes are required to see this behavior. This is because the default security policy of the CLR is to grant only partial trust to any code that originates from a network location. In other words, by default your application is only fully trusted if it originates from a hard drive location in your local system. When partially trusted code attempts an action that it is not permitted to perform, the results are a SecurityException.

If this information is new to you, it can be a useful learning experience to copy one of your working applications to a network share and try launching it. Unless your application deliberately addresses CAS, there will likely be problems due to security exceptions. Finding out which kinds of actions are not permitted for your own partially trusted code can be very educational.

In the future, it is likely that Windows, the CLR, and developer tools will make more direct use of the sandbox features of the CLR (and perhaps your application is one of the few that deliberately utilizes the sandbox today). However, most managed client applications and many managed Web applications will not function correctly unless they run with full trust.

If your application requires full trust to run, then as one of its startup actions you might consider testing to see if it is granted full trust. Doing so will provide your code the opportunity to alert the user to the configuration or installation problem, and shut down gracefully. This is better than subjecting the user to the default behavior—a flurry of security exceptions, one of which will probably activate the unhandled exception behavior.

The code in Figure 5 modifies the preceding example to first test for permissions, and then perform the application logic. The gist of this code is that a System.Security.PermissionSet object is created in such a way as to represent an unrestricted set of permissions. Then the code calls the Demand method on the permission set, which results in a SecurityException if the executing code (or any other code in the call stack) is not granted these permissions. This is perhaps the simplest policy that your application can employ to require full-trust execution while allowing the application to report failure to the user.

Figure 5 Check for Full Trust Permissions (Close Figure 5)
class App {
   public static void Main() {
      try {
         // Demand full trust permissions
         PermissionSet fullTrust = new
            PermissionSet(PermissionState.Unrestricted);
         fullTrust.Demand();

         // Perform normal application logic
         String[] entries = Directory.GetFileSystemEntries(@"C:\");
         foreach (String s in entries) Console.WriteLine(s);

      // Report that permissions were not full trust
      } catch (SecurityException) {
         Console.WriteLine("This application requires full-trust " +
            "security permissions to execute.");
      }
   }
}

Other alternatives to this approach exist, but they are more complicated. For example, you can find all of the places in your application where security exceptions occur and wrap them in try/catch blocks. This option results in code that can be run from more security contexts. However, the downside is that this approach requires analysis for each failure case to determine what kind of backup action to perform when the ideal action isn't permitted by the sandbox.

Another option is to use a feature of CAS called declarative security, which will cause the runtime to fail to load your code if it is not sufficiently permitted to perform its duties. Of course, this option will not allow your code to report its shortcomings to the user because the CLR will refuse to even load the code. That ends up looking like an application crash to the user.

In practice, today's .NET-based applications fall into two categories: those that have a specific use for partially trusted execution, and those that don't. In the cases where partial trust provides benefits to your user, the many features of CAS can be useful, including declarative security, catching of security exceptions, testing for specific permissions using SecurityManager.IsGranted, and so forth. However, if your application's scenario does not require or benefit from CAS, then the simple test for permissions on application startup gives your user an experience that's much better than an application crash, and your code investment is minimal. In addition, Visual Studio® 2005 will include a tool called PermCalc which will help determine which permissions are required for your application to run successfully.

Some Best Practices

Exception handling best practices and design is a topic that generates lots of debate, both inside Microsoft and throughout the software industry. I may one day distill into a column some of the best practices that we have captured on the Microsoft CLR team regarding effective use of exception handling in managed code. Space does not permit full coverage of exceptions best practices here. However, there are two best practices that enjoy wide acceptance and provide significant bang for your buck. Plus, they relate nicely to the use of unhandled exception handlers in your apps:

  • Don't catch or throw base exception types
  • Use finally blocks liberally

I touched on this first guideline earlier by stating that you should not catch the base exception type System.Exception. Now that I have looked at unhandled exceptions, let's rethink that.

It is natural for developers to make the following mental progression when thinking about that:

  1. Exceptions represent failures
  2. Catching the base exception type increases the likelihood of catching a failure to nearly 100 percent
  3. Catching the base exception is good

The reason that this logic does not follow is that catching an exception is not fundamentally the same thing as handling a failure. Catching specific exception types makes it far more likely that the developer will be able to write appropriate handling code. Catching base exceptions, though, has the effect of hiding unexpected failures, while very likely failing to cover all of the bases that it should in terms of handling logic.

There is plenty of historical data with managed code to back up the guidance that you should allow unexpected exceptions to propagate up the call stack for most situations. Meanwhile, with a proper unhandled exception-handling policy, you will learn of these unexpected cases from your exception logs. Catching too aggressively masks the ability to learn of the failures that were unexpected (but are now expected).

Now let's look at the guideline surrounding finally blocks. In general, well-written managed code contains more try/finally constructs than it does try/catch constructs. The reason for this is that finally blocks guarantee the ability to perform method-level clean-up operations in both exceptional and non-exceptional executions. Many methods include code that qualifies as "clean-up operations." This includes the closing of files and other resources that were opened in the method, unlocking synchronization locks that were taken during the method, and so forth. Meanwhile, if you follow the first guideline—only catching expected exceptions—the number of methods that actually catch any exceptions at all is reduced in the end.

These two guidelines go hand in hand. By not over-catching, you increase the likelihood that exceptions will travel further up the call stack in your applications. Sometimes these exceptions will go unhandled; other times code further up the stack will catch the exception. Either way, liberal use of finally blocks reduces the chaos in your application when exceptions travel up your call stack.

Enforcing Best Practices with FxCop

While I am on the topic of best practices, if you are not already familiar with a tool called FxCop, you probably should be. It's useful and free online at http://www.gotdotnet.com/team/fxcop.

The CLR team at Microsoft maintains a set of guidelines and best practices for writing managed code libraries. They also maintain the FxCop tool, which analyzes managed assemblies for guideline and best-practice violations. Most of the teams at Microsoft writing production managed code use FxCop, and many include it as part of the daily build process. Microsoft routinely releases updates of FxCop which include new useful rules.

Like any analysis tool, FxCop is not a substitute for human intelligence. However, it does help you identify places where you code violates specific guidelines. For example, FxCop will tell you where you catch System.Exception. FxCop guidelines are factored into an extensible mechanism called a rule. The tool lets you enable and disable rules independently of one another on a per-analysis bases.

FxCop also supports the creation of custom rules (see the Bugslayer column by John Robbins in this issue for a detailed look at creating custom rules). Some of the rules enforced by the tool (such as naming conventions) may seem unnecessarily strict, but remember that Microsoft developers use this tool internally to promote the consistency of their class libraries (a benefit to all developers using managed code). I suggest that you disable the rules that don't benefit your project. But I do believe that taking a couple of hours to investigate all the uses of FxCop will pay off in your managed projects.

Send your questions and comments for Jason to dot-net@microsoft.com.

Jason Clark provides training and consulting for Microsoft and Wintellect and is a former developer on the Windows NT and Windows 2000 Server team. He is the coauthor of Programming Server-side Applications for Microsoft Windows 2000 (Microsoft Press, 2000). You can get in touch with Jason at JClark@Wintellect.com.


Bugslayer: Bad Code? FxCop to the Rescue

Bugslayer
Bad Code? FxCop to the Rescue
John Robbins

Code download available at: Bugslayer0406.exe (167KB)
Contents
The Gestalt of FxCop
Common Errors
Setting Up Your Project
Introduction to FxCop Rules
Using the Introspection engine
Before You Begin Analysis
The Important Check Method
Wrap-up
Tips

When you've got bad code, there's only one thing you can do when they come for you: call FxCop. Some of you may have already used this excellent free tool from Microsoft, but many developers haven't because, for some unexplained reason, it's hidden deep down in the bowels of the GotDotNet Web site at http://www.gotdotnet.com/team/fxcop. (FxCop, if you don't know, is a code analysis tool that checks managed code assemblies for conformance to Microsoft® .NET Framework design guidelines.) The big news is that the industrious FxCop team has just released a new version, 1.30, that has many nice improvements and is considerably faster at analyzing large applications. To see the complete list of new features, surf over to the FxCop team blog. The FxCop team let me take an early peek at the new version so I could write this column close to the time they release it to the world.

FxCop is so important that I expect to spend two columns discussing how to use it and, most importantly, how to write your own rules. This month I want to start out with an introduction to FxCop. I will talk mainly about how to properly analyze FxCop errors in your code and how to integrate it into your daily build. Before I'm finished, I'll show you how to build your own FxCop analysis rules. Because rule development is such an extensive topic, I'll discuss how to build advanced rules in my next column.

The Gestalt of FxCop

Microsoft has obviously made a huge investment in .NET and in promulgating best .NET coding practices through the .NET Framework Design Guidelines, which you can find at Design Guidelines for Class Library Developers. Internally, Microsoft wanted to make sure their own developers followed the rules for consistency and maintainability. To enforce the Design Guidelines they devised a solution, out of which sprang FxCop.

While the FxCop team could have taken a source-code-analysis approach, like the tools of old, they decided to use the strength of .NET to do much of the heavy lifting. The first versions of FxCop used reflection so they didn't have to worry about finding and reading classes in an assembly. The bulk of the infrastructure work was involved in cracking the Microsoft Intermediate Language (MSIL) and providing an extensible scheme to write rules.

What's new in FxCop 1.30 is that it now performs analysis through a technique called Introspection. The use of the Introspection engine allows for much faster analysis and supports multithreaded analysis. Unlike the Reflection engine from previous versions, in the Introspection engine the assemblies you're analyzing are not locked so you won't need to shut down FxCop to do a fix and recompile of those assemblies. Finally, the Introspection engine offers a much richer analysis infrastructure compared to the Reflection engine. Those of you who have built rules using the Reflection engine from previous versions will be glad to know that your code investment is safe as FxCop still fully supports the Reflection engine. To update your rules, you'll need to change a few of the base class names you derived from and do a recompile.

FxCop has over 200 rules that cover everything about your assemblies from ensuring that you use COM interop correctly, to ensuring proper globalization, to enforcing rules for writing high-performance code. The FxCop help file, which is part of the installation, documents all of the existing rules; I'll refer you there to read about them. While you might think that the existence of more than 200 rules would mean that there's no need for more, you'll see in a minute that's not the case at all.

If you've never used FxCop, you should probably stop reading right now and give it a whirl because for the rest of the column I'll assume you've at least run it once across your application's binaries. If you have any application more complicated than "Hello World" don't be surprised if you're staring at a ton of errors in the message pane the first time you run FxCop. The good news is you found some errors; the bad news is you have to fix them. Figure 1 shows FxCop pointed at one of my assemblies.

Figure 1 FxCop in Action (Close Figure 1)

When it comes to fixing those errors the first time you run FxCop, it's just like the old joke about how you eat an elephant: one bite at a time. Figure 2 defines the icons shown in the message pane sorted in order of most to least importance. The trick to getting a handle on FxCop-reported errors is to sort the Level column in the message pane so that the critical errors are at the top of the list. Start working through those and move on to the errors. In the big scheme of things, the difference between a critical error and an error is a subjective one. I wish the FxCop team had used a simpler scheme consisting only of errors, warnings, and informational messages as it would have simplified the UI and made life easier for rule developers.

Figure 2 FxCop Error Levels (Close Figure 2)

Icon | Name | Description
 | Critical error | Errors that prevent the code from working. These are the errors you always fix first; they must never be excluded.
 | Error | General errors that don't affect usability and behavior. Only exclude these errors in extremely rare circumstances.
 | Critical warning | Problems that don't affect code behavior but can cause issues with maintainability and compliance with the Design Guidelines. You should not exclude these errors.
 | Warning | Lesser problems that affect stability, maintainability, and correctness as dictated by the Design Guidelines.

FxCop offers excellent advice on how to fix problems, but one thing that confuses folks is determining when an error that's reported by FxCop should be excluded completely. In general, you should exclude errors only very rarely. If you do exclude an error, FxCop now makes it mandatory to include a note about why you are excluding it. On my teams, the Project Manager/Team Lead is responsible for checking that every excluded error has a solid reason for its exclusion.

Common Errors

Some of the errors you'll see in your assemblies will not be your fault. For instance, some of the code generated by the IDE forms designer does not follow the Design Guidelines at all. If you see errors in the Windows® Form's InitializeComponent method, you can safely exclude them. Some other errors concern ASP.NET naming conventions. The word is that for ASP.NET 2.0 and Visual Studio® 2005, Microsoft is scrubbing all the wizards and code generators with FxCop to avoid these clashes.

I want to point out a couple of tricks for fixing particular errors faster and understanding their consequences. When you run FxCop against your assembly, the first error you'll probably see is the "Assemblies specify permission requests" error. Fixing this error is not as easy as most because you'll need to inspect your code closely to determine exactly which permissions you need and which you can refuse. When Visual Studio 2005 ships, it will have a security analysis tool that will identify the minimum security permissions your app will need, making permissions trivial to set. Until this tool ships, it's a good idea to have a coworker analyze your code to ensure your permissions are set appropriately.

FxCop has an aggressive spell checker and will probably generate many errors simply because it doesn't know your company name and other names in your code. Fortunately, in the FxCop directory is a file called CustomDictionary.xml where you can add words and acronyms. (Obviously, you'll want to keep your team's version of CustomDictionary.xml in your version control system along with your FxCop file so everyone can benefit.) You can also flag words as deprecated or inappropriate. Earlier versions of FxCop only read the CustomDictionary.xml file out of the FxCop installation directory; version 1.30 looks for the file in the same directory as your project file. Figure 3 shows CustomDictionary.xml using a variety of fields.

Figure 3 Dictionary (Close Figure 3)
<Dictionary>
 <Words>
  <Recognized>
   <Word>Bugslayer</Word>
   <Word>uncategorize</Word>
   <Word>fxcopbug</Word>
   <Word>stylesheet</Word>
   <Word>gotdotnet</Word>
   <Word>uninstantiated</Word>
   <Word>rethrows</Word>
   <Word>aspx</Word>
   <Word>ndpsec</Word>
   <Word>typeof</Word>
   <Word>readonly</Word>
   <Word>Sta</Word>
   <Word>Callee</Word>
   <Word>Callees</Word>
   <Word>Pdb</Word>
   <Word>Opcode</Word>
   <Word>&amp;amp</Word>
   <Word>&amp;gt</Word>
   <Word>&amp;lt</Word>
  </Recognized>
  <Deprecated/>
  <Inappropriate/>
 </Words>
 <Acronyms>
  <CasingExceptions>
   <Acronym>CJK</Acronym>
  </CasingExceptions>
 </Acronyms>
</Dictionary>

You can see the Deprecated and Inappropriate elements under the Words element, but at the time of this writing the FxCop rules supplied by Microsoft do not read those sections. However, there's nothing stopping an ambitious developer from writing a new rule that will read and process them.

Setting Up Your Project

Once you've taken care of the initial round of errors and warnings in your assemblies, you'll want to examine how your projects are set up. Along with the GUI version of FxCop, there's a console version, FxCopCMD, which you should use so you can have FxCop run after every build to ensure no additional problems have edged their way into your code. In fact, teams at Microsoft are now running FxCop against their daily builds. That alone should make it mandatory for you as well.

While FxCopCMD will work just fine by pointing it at an assembly, you'll want to use a previously created FxCop project with FxCopCMD so your exclusions and additional rule files are always used. There's a big question as to what type of project you want for your application. On one hand, it's quite convenient to have a single project for all the assemblies in your application. That way there's a single call you'll have to make as part of your build script. The one master project is also the one place all exclusions reside for your whole application. It's also very convenient if you want to add a new rule and have everyone use it; you only have to add it to the one FxCop project and everyone will automatically be using it.

The biggest drawback to using one large master FxCop project is that if you have an application that has lots of assemblies, running FxCop will be a lengthy operation when you want to check only a few assemblies. While you can always uncheck assemblies you don't want analyzed (in the Targets tree control), that can cause a big problem. For example, if you pull the master FxCop project for the application from version control to make some exclusions, you might unintentionally leave those assemblies unchecked when you return the file. Then your daily build will analyze only those few assemblies and nothing else, defeating the purpose of FxCop.

While going to individual FxCop projects for each assembly is faster, it has its drawbacks as well. First, it is now a bigger hassle to check all those individual files to ensure they contain additional rules files and that exclusions have been used properly. Second, you have to account for all those individual FxCop project files, and it's easy to forget to add new assemblies to the mix.

What I would really like is a tool that looks in your build directories and consolidates the individual assembly FxCop projects into a single master FxCop project. That would mean moving over all targets and exclusions from the individual files to the master file. My original plan was to write such a tool as part of this column, but I found that it was going to be more complicated than writing code to merge project files. Since I haven't built the merging utility yet, if you're interested in working on it, send me an e-mail and I can coordinate the project between us and other interested parties.

Until a merge utility exists, the best practice, in my opinion, is to have a single FxCop project for your entire application. In the meantime, I wrote a quickie tool, NoPartialAnalysis, to check that all targets have full analysis turned on. It performs a check of the Targets node in an FxCop project and if any of the nodes are unchecked, the tool exits with a return value of 1. Your daily build will fail on the bad return value so you know you're not getting the full FxCop check. Once you check the version control logs and find who made the bad move, you can apply the proper motivation so that person never does it again.

Overall, there's nothing too exciting about NoPartialAnalysis. It's really just XML manipulation. Note that NoPartialAnalysis properly handles both the normal compressed saved files and the uncompressed. The default is compressed, and I strongly suggest you use it. The only benefit to the uncompressed files is that you will fill up your hard disk and have an excuse to get a new one.

For many people, simply using FxCop with its default rules supplied with the download is sufficient enough. However, real developers will immediately think of a whole bunch of errors that they would love a tool to check for them automatically, so they'll be itching to write their own rules.

Introduction to FxCop Rules

If you've peeked at the FxCop SDK section of its help file, you've seen that there's only documentation for the Reflection engine. It mentions that the SDK for the Introspection engine will be posted later. While you can write powerful rules with the Reflection SDK, the Introspection engine has all the great features and it's drastically faster than the Reflection engine.

While I could have taken the easy way out and used the Reflection SDK to build reflection-based rules, I felt it was important to get started with Introspection. The obvious drawback is that the FxCop team did not document the Introspection engine at all, but that never stopped a determined developer armed with Lutz Roeder's phenomenal Reflector and its awesome .NET decompiler. Once I started poking at the Introspection engine and the rules that come with FxCop (see the <FxCop installation>\Rules directory for those assemblies), I quickly learned that it represents a giant step up in complexity.

While the Reflection engine provides a few helper functions, the bulk of the work those rules perform is through normal .NET reflection and in string manipulation as the underlying MSIL is returned as a string buffer. With Introspection, you can rely on the Introspection engine to do all of that work. That means I had to spend a lot more time looking at the existing rules in Reflector to see how they performed their magic in order to get started on specific rules. As you get into more complicated rules, that becomes a daunting task. Fortunately, with Reflector, it really becomes an exercise in code reading.

While you certainly can build cool rules without documentation, as you'll see there is one potentially more serious reason that the FxCop team might not have documented the Introspection engine. They might be planning to change the Introspection engine interfaces in the future and don't want folks using the existing interfaces. That's a perfectly valid reason and could present problems in the future. However, I'll take that chance, as I'll get the benefit of faster execution and no file locking today with the Introspection engine. Additionally, I'm willing to bet that the interfaces won't change too drastically because that would require the FxCop team to rewrite more than 200 existing rules for the Introspection engine. As anyone who's had to fix programs written five years ago knows, once you write some code it has a habit of sticking around for quite a while. The FxCop team was kind enough to confirm for me that the Introspection engine interfaces, except for two methods I'll discuss later, will be around for a long time. There are also plans to document the Introspection engine.

Using the Introspection engine

I wanted to point out all the issues with using the Introspection engine for building rules so you'd understand the ramifications. If the FxCop team does change the rules in the future, I'll make sure to update any rules I've developed and mention it in future Bugslayer columns. Since I use these rules in all of my production code, I have a stake in updating them on every FxCop release.

What you can do with a rule is limited only by your imagination. What makes FxCop so interesting is that you get access to everything in an assembly, from all attributes (applied to anything), right down to an individual parameter of a method. Even better is that you get very cool support from FxCop to get at the MSIL for a method as well as the call graph for callers and callees. You'll just need to make sure that the rule you want makes sense from the standpoint of analyzing the binary. For example, if you are thinking about considering a performance analysis rule, unless you can look for an exact MSIL or call pattern, it might be better to use source analysis for that type of rule. A good example of a performance rule that works because you're looking for a fixed pattern is the TestForEmptyStringsUsingStringLength rule you'll find in FxCop's PerformanceRules.DLL.

The basics of getting an Introspection engine rule started are straightforward and if you're familiar with the Reflection engine resource-based rules, you should feel right at home. In Figure 4, I combined the three files of the supplied EmptyFxCopRule file so you could follow along with me. The simple base class, BaseEmptyFxCopRule, exists solely to derive from the required BaseIntrospectionRule from the Microsoft.Tools.FxCop.Sdk.Introspection namespace. BaseIntrospectionRule assumes that the description text and related data resides in an embedded resource. Consequently, the three parameters required by the BaseIntrospectionRule constructor are the type name of the rule, the embedded resource file name that contains the rule descriptions, and the assembly that contains the embedded resource file. Since you'll most likely have more than one rule per assembly, having a simple base class like BaseEmptyFxCopRule means that you can hide the common resource information manipulation in the base class and require derived classes to specify the type name in calling the base class constructor.

Figure 4 Minimal FxCop Rule (Close Figure 4)

BaseEmptyFxCopRule.CS
using System ;

using Microsoft.Cci ;
using Microsoft.Tools.FxCop.Sdk ;
using Microsoft.Tools.FxCop.Sdk.Introspection ;

namespace EmptyFxCopRule
{
    [CLSCompliant(false)]
    public abstract class BaseEmptyFxCopRule : BaseIntrospectionRule
    {
        protected  BaseEmptyFxCopRule ( String name )
            : base ( name                                ,
                     "EmptyFxCopRule.RuleData"           ,
                     typeof(BaseEmptyFxCopRule).Assembly  )
        {
        }
    }
}
EmptyAssemblyRule.CS
using System ;
using Microsoft.Cci ;
using Microsoft.Tools.FxCop.Sdk ;
using Microsoft.Tools.FxCop.Sdk.Introspection ;

namespace EmptyFxCopRule
{
    /// <summary>
    /// Summary description for EmptyRule.
    /// </summary>
    [CLSCompliant(false)]
    public class EmptyAssemblyRule : BaseEmptyFxCopRule
    {
        public EmptyAssemblyRule ( ) : base ( "EmptyAssemblyRule" )
        {
        }

        public override Problem[] Check ( AssemblyNode theAssembly )
        {
            return ( null ) ;
        }
    }
}
RuleData.XML
<?xml version="1.0" encoding="utf-8" ?>
<Rules>
   <Rule TypeName="EmptyAssemblyRule">
      <Name>
         EmptyAssemblyRule: The prose name of the rule goes here.
      </Name>
      <Description>
         EmptyAssemblyRule: The short description goes here.
      </Description>
      <LongDescription>
         EmptyAssemblyRule: The long description goes here.
      </LongDescription>
      <GroupOwner>
         EmptyAssemblyRule: This is optional, but is generally the
                            developer/team/company.
      </GroupOwner>
      <DevOwner>
         EmptyAssemblyRule: The team or individual responsible for the
                            rule.
      </DevOwner>
      <Url>
         EmptyAssemblyRule: The URL for help containing the rule.
      </Url>
      <Resolution>
         EmptyAssemblyRule: The string that appears in the message panel
                            pointing out the error.
      </Resolution>
      <Email>
         EmptyAssemblyRule: The optional email address of the rule author.
      </Email>
      <Message Certainty="99">
         Warning
      </Message>
   </Rule>
</Rules>

The second file, EmptyAssemblyRule.CS, shows a minimal rule that looks at assemblies. I'll talk more about the Check method later. The main thing to notice is that the name of the class is EmptyAssemblyRule. You can see that in its constructor, EmptyAssemblyRule passed that string into the base class.

The final file in Figure 4 is the embedded resource XML file. I did a little format tweaking so you could see the individual elements more easily. The key text is at the top where the TypeName attribute lists the name of the rule that the data applies. While you can probably discern the meaning of all the other elements, I found it valuable to add the EmptyFxCopRule assembly to an empty project, expand the tree for EmptyFxCopRule, and double-click on the EmptyAssemblyRule so you can see where the Rules Details dialog displays the various text elements.

Before You Begin Analysis

Before I get into the details of rule development, I want to point out a few things I encountered. First note that adding an XML file to the project does not add it as an embedded resource. Consequently, after adding the XML resource file that describes the rule, make sure you set the Build Action to Embedded Resource instead of Content. When creating new rules, even if they are additional rules in an existing assembly, always test to make sure it's set up correctly. It's simple to test a rule infrastructure by building your rule shell and adding it to an empty FxCop project. If FxCop reports an exception when loading your rule, it usually means you have an error in your XML resources. Unfortunately, FxCop just reports that you have bad XML, not which element is wrong, so finding it will take some trial and error.

When I first put the EmptyFxCopRule file together for this column, I couldn't get the actual EmptyAssemblyRule to show up underneath the assembly in the FxCop Rules tab. Since I was creating an empty rule, I had mistakenly used "Informational" instead of "Information" as the message type. Once I fixed my mistake, the rule showed up just fine. The Message element is not very forgiving when it comes to the name you specify—it must exactly match what's in the Microsoft.Tools.FxCop.Sdk.MessageLevel enumeration or you'll get an XML loading error in FxCop. Just so you have it, here's the definition from Reflector:

   public enum MessageLevel
   {
       CriticalError = 1,
       CriticalWarning = 4,
       Error = 2,
       Information = 16,
       Warning = 8,
   }

One last thing I want to talk about here is the protection-level properties, TypeProtectionLevel, MemberProtectionLevel, and NestedTypeProtectionLevel, which you can override from BaseIntrospectionRule. Those members return an OR'd level specified by the Microsoft.Tools.FxCop.Sdk.ProtectionLevels enumeration (see Figure 5).

Figure 5 Protection Levels (Close Figure 5)
public enum ProtectionLevels
{
   Public = 1,
   Family = 2,
   Private = 4,
   Assembly = 8,
   FamilyAndAssembly = 16,
   FamilyOrAssembly = 32,
   NestedPublic = 64,
   NestedFamily = 128,
   NestedPrivate = 256,
   NestedAssembly = 512,
   NestedFamilyAndAssembly = 1024,
   NestedFamilyOrAssembly = 2048,
   All = 4095,
   None = 1073741824,
}

The default OR'd values returned by those properties are listed in Figure 6 and you can see that they generally relate to the visibility levels associated with those items you can access from another assembly. You'll almost always go with the defaults, but I want to point out the ProtectionLevels enumeration properties as it's easy to forget they're there and you'll wonder why that private method you want to test isn't being passed to your Check method.

Figure 6 BaseIntrospectionRule Default Protection Levels (Close Figure 6)

Property | Default Protection Levels
TypeProtectionLevel | Public
NestedTypeProtectionLevel | Public | NestedPublic | NestedFamily | NestedFamilyOrAssembly
MemberProtectionLevel | Public | Family | FamilyOrAssembly

The Important Check Method

As you've probably guessed, the Check method is where all the action is. The BaseIntrospectionRule class defines 15 different Check methods you can implement in your rules, most of which I've listed in Figure 7. Two overloaded Check methods not shown—one that takes a Method and Statement as a parameter and one that takes nine parameters—are deprecated and will be removed from future versions of the Introspection engine. As you scan down the list of check methods, you can see, for example, that if you want a rule that works on properties, you simply override the version Check that takes the Property type as a parameter. All the types such as Method, Property, and so forth come out of the Microsoft.Cci namespace defined in Microsoft.Metadata.Prototype.DLL, which can be found in the FxCop directory.

Figure 7 BaseIntrospectionRule Check Methods (Close Figure 7)
public virtual Problem[] Check ( AssemblyNode theAssembly ) ;
public virtual Problem[] Check ( Event theEvent ) ;
public virtual Problem[] Check ( Field field ) ;
public virtual Problem[] Check ( InstanceInitializer initializer ) ;
public virtual Problem[] Check ( Member member ) ;
public virtual Problem[] Check ( Method method ) ;
public virtual Problem[] Check ( Method method , Parameter parameter );
public virtual Problem[] Check ( Module theModule ) ;
public virtual Problem[] Check ( Property theProperty ) ;
public virtual Problem[] Check ( Resource resource ) ;
public virtual Problem[] Check ( StaticInitializer initializer ) ;
public virtual Problem[] Check ( string       namespaceName ,
                                 TypeNodeList types          ) ;
public virtual Problem[] Check ( TypeNode type ) ;

When I started to write FxCop rules, the first ones I wanted to add involved assemblies and, fortunately, those are the easiest to get started with. All the rules I'll be talking about are in the Bugslayer.FxCop.DesignRules assembly in the code download available at the MSDN® Magazine Web site, so you may want to pull up this project and follow along. To create it, I simply followed the same techniques I found by using Reflector on the FxCop rules.

One point of contention I've had with the Design Guidelines is that it's not mandatory to include an XML documentation file with your assemblies. My guess is that since Microsoft did not include full documentation comments as a feature of Visual Basic® .NET and managed C++, it couldn't make it a hard requirement. However, all languages will support XML documentation files in Visual Studio 2005, and for Visual Basic .NET today there are numerous add-ins with that capability. I've made it a requirement in all of my projects, so I wanted an FxCop rule that will check that an assembly always has documentation associated with it.

This very important rule, AssembliesHaveXmlDocCommentFiles, turns out to be extremely simple to write. You can find the code in the eponymous .cs file. To show you how simple the rule is, Figure 8 shows the Check method that does all the work. As you can see, if you return null/Nothing in the Check method, that tells FxCop that you didn't find any errors. Returning an allocated Problems array signals that there are errors and tells you the text of the error. In Figure 8, I use the BaseIntrospectionRule.GetResolution method to get the Resolution element out of the AssembliesHaveXmlDocCommentFiles rule's XML file. In this case, the string is: " '{0}' is missing or has an invalid XML Doc Comments file." The format item allows me to put the name of the assembly in the error text, which I do by calling the Problem property's (a ProblemsCollection type from BaseIntrospectionRule) AsArray method. What's nice about the Problem property in the BaseIntrospectionRule is that if you have a rule that is recursive, you can add all the problems you've found in the code and take care of getting them all. With most rules only returning a single error string, you'll usually use code like what's specified in the error portion of Figure 8 to do the work.

Figure 8 Check Method for Assembly XML Comments (Close Figure 8)
public override Problem[] Check ( AssemblyNode theAssembly )
{
    // First, check if this is a DLL library.
    if ( false == IsLibraryAssembly ( theAssembly.Location ) )
    {
        // It's not a DLL so there's no need for an XML file.
        return ( null ) ;
    }

    // Build up the name to the XML file, which should  be in the same
    // directory where the DLL is located.
    String XmlFile = Path.ChangeExtension ( theAssembly.Location ,
                                            XMLExtension          ) ;

    // The return value.
    Problem[] retvalue = null ;
    bool failedTests = false ;

    if ( true == File.Exists ( XmlFile ) )
    {
        // The file exists so let's see if it's a proper and happy XML
        // file.
        try
        {
             XmlDocCommentsFileDictionary tempFile =
                         new XmlDocCommentsFileDictionary ( XmlFile ) ;

             // I guess you could optionally check that the timestamps
             // are close here, but since the Visual Basic .NET comment
             // file can be built at a different time, I can't always be
             // assured the timestamps are close.
        }
        catch ( XmlException )
        {
            failedTests = true ;
        }
    }
    else
    {
        failedTests = true ;
    }

    if ( true == failedTests )
    {
        // Build up the error string to return.
        String[] arr = new String[1] ;
        arr[0] = Path.GetFileName ( theAssembly.Location ) ;
        retvalue = Problems.AsArray ( GetResolution ( arr ) ) ;
    }
    return ( retvalue ) ;
}

While the AssembliesHaveXmlDocCommentFiles rule really doesn't do much with the Introspection engine, most of your code will. After struggling to figure out the gyrations necessary to use the Introspection engine code, it finally dawned on me how to think about it. I happened to look at the version comment for the Microsoft.Metadata.Prototype.DLL assembly, which contains the base class Standard Visitor for BaseIntrospectionRule, and it says: compiler-oriented replacement for System.Reflection and System.Reflection.Emit. That's when it hit me that the Introspection engine is simply another way of doing reflection. Of course, there's no documentation, but once I realized that fact, it became easier to visualize how to use the Introspection engine. Where reflection loads up the assembly and calls the types and other information in the assembly, the Introspection engine maps the assembly into memory and goes through the grind of walking all the internal tables by hand so it avoids locking the assembly like reflection does.

While my first rule, AssembliesHaveXmlDocCommentFiles, is relatively easy, I wanted to create other rules that would look to see if an assembly had the AssemblyTitleAttribute, AssemblyCompanyAttribute, AssemblyDescriptionAttribute, and AssemblyCopyrightAttribute all applied and not set to null values. Again, I feel all those attributes should always be set on any assembly in order for the assembly to be correct. At first I thought it was going to be pretty easy to do this as I just had to decompile the Microsoft rule AssembliesHaveVersionNumbers in DesignRules.DLL. A quick peek at the rule showed that the AssemblyNode passed to the Check method had a Version property on it. My hope was that there were either properties to get the attributes I wanted or a simple way to get attributes for an assembly. Of course, life wasn't so easy, but it turned out to be a good way to learn about the Introspection engine usage.

In a previous Reflector exploration mission, I saw that the Microsoft.Tools.FxCop.Sdk.Introspection.RuleUtilities class had several GetCustomAttributes methods that pointed out an Attributes property on the various Microsoft.Cci namespace types. Looking through the various rules files supplied with FxCop, you'll see that all the calls to any RuleUtilities.GetCustomAttribute method was using a set of predetermined attribute types defined in the Microsoft.Cci.SystemType class. As that particular GetCustomAttribute method takes a Microsoft.Cci.Module (a base class of Microsoft.Cci.AssemblyNode) and an attribute type, it looked like the one to use in my case. Therefore, my next mission was to find an example of getting a type out of a Module or AssemblyNode class to see what method to call.

I thought I might have to do a lot of searching with Reflector, but the RuleUtilities class had what I needed so I didn't have to look far. There is a public field called WebMethodAttribute that looked to be a custom attribute lookup. Looking through the RuleUtilities.GlobalBeforeAnalysis method, I saw towards the bottom of the method that it called the AssemblyNode.GetType property, passing in the Microsoft.Cci.Identifier classes for the namespace and the type. That was the hint I needed because in my spelunking of the Microsoft.Cci.SystemTypes class, the SystemAssemblies property returns the MSCORLIB.DLL AssemblyNode.

The Identifier class is, as the name implies, the Introspection identifier for an element (a TypeNode in Introspection terms). In almost all cases, you can get the identifier for a particular TypeNode either by passing the string name of the identifier you want to the Identifier constructor or by using the static Identifier.For method. If you happened to follow along with Reflector for the last few paragraphs, you saw that the description is far more complex than the actual gyrations I went through.

To tie everything together, Figure 9 shows the Check method that implements the actual rule and the AttributeType method to get the type. Since all the assembly attributes rules I wrote (AssembliesHaveCompanyAttributesRule, AssembliesHaveCopyrightAttributesRule, AssembliesHaveDescriptionAttributesRule, and AssembliesHaveTitleAttributesRule) are almost identical they all share a base class, BaseAssemblyAttributeRule, and simply define the abstract AttributeType method to return the particular method to look up. You can see all the action in full in the Bugslayer.FxCop.DesignRules project.

Figure 9 Checking if Assembly Attributes Exist (Close Figure 9)
public override TypeNode AttributeType ( )
{
    if ( null == typeCopyrightAttribute )
    {
        typeCopyrightAttribute = SystemTypes.SystemAssembly.GetType (
                         Identifier.For("System.Reflection"),
                         Identifier.For("AssemblyCopyrightAttribute"));
        Debug.Assert ( null != typeCopyrightAttribute,
                       "null != typeCopyrightAttribute"  ) ;
    }
    return ( typeCopyrightAttribute ) ;
}

public override Problem[] Check ( AssemblyNode theAssembly )
{
    // Assume everything will work.
    Problem [] retvalue = null ;

    // Get the custom attribute type.
    AttributeList attribList = RuleUtilities.GetCustomAttributes (
                               theAssembly, AttributeType ( ) ) ;
    if ( 1 == attribList.Length )
    {
        // Get the value of the attribute since it does exist.
        String strValue = attribList[0].Expressions[0].ToString() ;
        if ( 0 == strValue.Length )
        {
            // The attribute is there but has an empty value.
            retvalue=Problems.AsArray(GetResolution(theAssembly.Name));
        }
    }
    else
    {
        // The attribute is missing.
        retvalue = Problems.AsArray (GetResolution (theAssembly.Name));
    }
    return ( retvalue ) ;
}

Before I end the column, I want to point out a few things that'll help you develop your rules. The first should be obvious—you definitely need to take a long look at the Microsoft.Tools.FxCop.Sdk.Introspection.RuleUtilities class in Reflector so you can start learning how to utilize the Microsoft.Cci classes in your rules. The second item is that while there is a logging and tracing system built into FxCop, debugging your code through print statements is passé. To debug your rules, set the Debug Mode to Program and the Program field (both in the Debugging property page) to FXCOP.EXE and you'll debug your rule with ease.

Wrap-up

Now when the bad code comes for you, you've got FxCop to fight back. I hope you find the rules and NoPartialAnalysis I presented here useful and that they will help in your development efforts. I strongly encourage you take a look at the code and spend some time with Reflector looking at the Introspection engine rules in FxCop so you can see how all the magic is done. If you're looking for a rule to learn about writing rules, here are two ideas:

  • A rule to read and process the Deprecated and Inappropriate sections in CustomDictionary.XML. Take a look at the MemberNamesShouldBeSpelledCorrectly rule in UsageRules.DLL for some hints on how to start.
  • A rule that checks the spelling in embedded resource files.

If you're curious about what's coming up in my future advanced rules discussion, I've already written a couple of cool rules. The first shows how to do configurable rules and checks to see if you have too much boxing and unboxing going on in any method. I've also created a rule that checks the exceptions thrown directly from a method and ensures you've documented them with <exception> tabs in the XML documentation comments for that method. I've also got some other interesting rules in the works that I'll refine before my next installment.

Tips

Tip 63 At this point, I've only seen one other person who's posted rules on the Web, Anand M, at http://www.gotdotnet.com/team/fxcop. His rules are based on the older Reflection engine, but do show you how to write those rules in Visual Basic .NET.

Tip 64 If you're starting to do the very cool Tablet PC development and need another test machine, but can't afford a second Tablet PC, I ran across a great idea on how to mimic one at Can't Afford a Tablet PC? Try This!. Wei-Meng Lee and Brian Jepson point out that you can install Windows Tablet PC Edition on a laptop and hook up a graphics tablet. That gives you an instant test machine for about $100!

Send your questions and comments for John to slayer@microsoft.com.

John Robbins is a cofounder of Wintellect, a software consulting, education, and development firm that specializes in programming for .NET and Windows. His latest book is Debugging Applications for Microsoft .NET and Microsoft Windows (Microsoft Press, 2003). You can contact John at http://www.wintellect.com.


Basic Instincts: Creating and Managing Secondary Threads

Basic Instincts
Creating and Managing Secondary Threads

Contents
Creating a Thread
Passing Parameters to a Secondary Thread
When to Create New Threads
Conclusion

In my January 2004 column I showed you how to use delegates to execute a method in an asynchronous fashion. There you learned how to dispatch an asynchronous method call by simply calling BeginInvoke on a delegate object. Executing methods asynchronously using delegates is easy because the common language runtime (CLR) creates and manages a pool of worker threads for you behind the scenes. When you call BeginInvoke, the CLR takes care of dispatching your request to one of these worker threads from its built-in thread pool.

I recommend that you use asynchronous delegates and the programming techniques shown in my January 2004 column in the majority of cases where you need to execute code asynchronously from a desktop application. However, there are certain occasions when using asynchronous delegates doesn't provide as much flexibility or efficiency as you need.

This month I'll show you how to create and manage your own threads using the Thread class of the CLR. I'll also discuss the scenarios in which it's better to create and manage a secondary thread than to execute a method asynchronously with delegates.

Creating a Thread

When you want to create a new thread, you create an instance of the Thread class defined in the System.Threading namespace. The constructor of the Thread class accepts a reference to a ThreadStart delegate object. You can see two examples of how to create a new thread in the code in Figure 1.

Figure 1 Creating a Secondary Physical Thread (Close Figure 1)
Imports System
Imports System.Threading

Class MyApp
  Shared Sub Main()

    '*** long-hand syntax for creating a thread
    Dim MyHandler As New ThreadStart(AddressOf TedsCode.MyAsyncTask)
    Dim ThreadA As New Thread(MyHandler)
    ThreadA.Start()

    '*** short-hand syntax for creating a thread
    Dim ThreadB As New Thread(AddressOf TedsCode.MyAsyncTask)
    ThreadB.Start()

  End Sub
End Class

Class TedsCode
  Shared Sub MyAsyncTask()
    '*** code to run on secondary thread
  End Sub
End Class

As you can see, there is a longhand syntax and a shorthand syntax for creating a new thread object. The technique for creating ThreadA in Figure 1 demonstrates how to explicitly create a ThreadStart delegate object that is bound to the method named MyAsyncTask. The technique for creating ThreadB accomplishes the same goal with less typing. That's because the Visual Basic® .NET compiler will implicitly create a ThreadStart delegate object when you write the following line of code:

Dim ThreadB As New Thread(AddressOf TedsCode.MyAsyncTask)

With version 1.0 and version 1.1 of the CLR, each Thread object is associated with its own physical Win32® thread. However, creating an object from the Thread class doesn't actually create a physical thread. Instead, you must call the Thread object's Start method for the CLR to call to the Windows® OS and create a physical thread. Note that future versions of the CLR are likely to provide an optimization whereby it will not be necessary to create a separate physical thread for each Thread object.

Once the CLR has created the physical thread, it uses it to execute the target method bound to the ThreadStart delegate object. The lifetime of this physical thread is controlled by the target method's execution. When execution of the method completes, the CLR gives control of the physical thread back to Windows. At this point, the OS destroys the physical thread.

You should realize that there is measurable overhead involved in creating and destroying physical threads. This overhead isn't all that significant in a desktop application where a thread is typically created in response to a user's action. However, creating new threads is something you should avoid in server-side code that's written for performance and scalability.

Remember that executing methods asynchronously using delegates leverages the built-in thread pool of the CLR. Therefore, achieving asynchronous execution using delegates scales better in a server-side application because it doesn't require the continual creation and destruction of physical threads.

Passing Parameters to a Secondary Thread

When you create a new Thread object to execute a method asynchronously, you must use a method that matches the calling signature of the ThreadStart delegate. That means you must create a method that is a Sub procedure that accepts no parameters (see the MyAsyncTask sub that is shown in Figure 1). This makes it a bit tricky to pass parameters to a method that is going to execute on a newly created thread.

There is a common technique you can use for passing parameters to a secondary thread; it involves creating a custom thread class. Examine the class definition for TedsThreadClass, shown in Figure 2. You can see how it has been designed with a custom set of instance fields and a parameterized constructor. When you create an object from a custom thread class such as TedsThreadClass, you can initialize it with whatever parameter values are required in your particular situation.

Figure 2 Passing Parameters to an Asynchronous Method (Close Figure 2)
Imports System
Imports System.Threading

Class MyApp
  Shared Sub Main()

    Dim ThreadA As New TedsThreadClass(1, "Bob")
    ThreadA.Start()

    Dim ThreadB As New TedsThreadClass(2, "Betty")
    ThreadB.Start()

  End Sub
End Class

Class TedsThreadClass

  '*** define custom fields and constructor with parameterized data
  Protected x As Integer
  Protected y As String
  Sub New(ByVal X As Integer, ByVal Y As String)
    Me.x = X
    Me.y = Y
  End Sub

  '*** instance method used to run asynchronous task
  Sub MyAsyncTask()
    '*** this code has access to x and y
  End Sub

  '*** encapsulate threading support
  Protected InnerThread As New Thread(AddressOf Me.MyAsyncTask)
  Public Sub Start()
    InnerThread.Start()
  End Sub

End Class

Note that TedsThreadClass is different from the example shown in Figure 1 because in Figure 2 the MyAsyncTask method is defined as an instance method instead of a shared method. That means that the MyAsyncTask method in Figure 2 can access the custom instance fields x and y. The key point of this technique is that it provides the means for passing a customized set of parameters to a method executing on a new secondary thread.

TedsThreadClass contains a protected field named InnerThread, which holds a Thread object that's bound to the instance method MyAsyncTask and a public Start method. This allows the custom thread class to encapsulate the implementation details for creating and managing Thread objects. To use this custom thread class, simply instantiate an object and call its Start method, like this:

Dim ThreadA As New TedsThreadClass(1, "Bob")
ThreadA.Start()

You have just seen what's involved with a custom thread class that's been designed to pass parameters to an asynchronous method call. Remember that delegates make passing parameters in an asynchronous method call much easier because you can define a custom delegate type with whatever parameter list you like. Note that making an asynchronous method call by creating and managing a thread usually requires more work during design and coding.

You should also consider that executing a method asynchronously with delegates provides an easy way to harvest the return value and output parameters: the delegate's EndInvoke method. Executing methods asynchronously with a new Thread object is not as convenient. You must devise a custom scheme to pass data from secondary threads back to the application's primary thread.

When you are creating and managing secondary threads, it also takes more effort to coordinate thread synchronization. For example, what should you do when you need the application's primary thread to wait until a secondary thread has completed its work? You should call the Join method supplied by the Thread class, as shown in the following code:

Dim ThreadA As Thread
ThreadA = New Thread(AddressOf TedsCode.MyAsyncTask)

'*** do some work on primary thread

'*** now block until secondary thread is done
ThreadA.Join()

The Join method is a blocking call. The thread that calls Join is blocked until the other thread has finished its work. In other words, the Join method is used to take two threads representing parallel paths of execution and join them together into a single path.

When to Create New Threads

I have already outlined several reasons why executing a method asynchronously on a new Thread object is more difficult and not as efficient as using an asynchronous delegate. Because asynchronous delegates are easier to use and they provide the efficiency of using the built-in thread pool, I recommend you use them when you have to make asynchronous method calls.

There are, however, a few design situations in which delegates should not be used. The following is a list of circumstances in which you should most likely use secondary threads rather than delegates for asynchronous execution.

  • You need to execute a long-running task
  • You need to adjust a thread's priority
  • You need a foreground thread that will keep a managed desktop application alive
  • You need a single-threaded apartment (STA) thread to work with apartment-threaded COM objects

When you need to dedicate a thread to a task that's going to take a long time, you should create a new thread. For example, imagine you need to dedicate a secondary thread to watch for updates to a file or to listen for incoming data on a network socket, and you need it for the lifetime of the application. It would be considered bad style to use delegates because you would effectively be taking a thread out of the CLR thread pool and never returning it. Asynchronous method execution using delegates should only be used for relatively short-running tasks.

When you need to change a thread's priority, you should create a new thread. The Thread class exposes a public Priority property that allows you to increase or decrease the priority of a thread. However, you should not change the priority of threads from the CLR thread pool. You should only adjust the priority of the threads you have created by calling New on the Thread class.

Let's look at a simple example involving a desktop application. Imagine you want to run an asynchronous method on a secondary thread with a lower priority to reduce its impact on the responsiveness of the application's user interface. You can adjust the Thread object's Priority property before calling the Start method:

Dim ThreadA As Thread
ThreadA = New Thread(AddressOf TedsCode.MyAsyncTask)
ThreadA.Priority = ThreadPriority.BelowNormal
ThreadA.Start()

The settings for a thread's priority level are Highest, AboveNormal, Normal, BelowNormal, and Lowest. You should generally only lower thread priorities and try to refrain from raising them. Be careful of choosing the Highest and AboveNormal settings because they can have unpredictable effects on the application and the system as a whole.

Creating a new thread is also a good idea when you want to keep a managed desktop application alive while a secondary thread is running. Imagine a scenario in a Windows Forms application where the user closes the main form while a secondary thread is performing a task in the background. Is the fact that this secondary thread is still running important enough to keep the application alive? If the secondary thread is a background thread, the answer is no. The application will shut down right away. However, if the secondary thread is a not a background thread, the app will keep running.

Each Thread object has an IsBackground property that indicates whether it is a background thread. All the threads in the CLR thread pool are background threads and should not be modified in this respect. That means a task executing asynchronously as a result of a call to BeginInvoke on a delegate is never important enough by itself to keep an application alive.

When you create a new Thread object, it is by default a foreground thread because its IsBackground property is set to false. That means your application will continue to run while a secondary thread is executing. If you would like to create a new thread and make it a background thread, you assign a value of True to its IsBackground property before you call the Start method, like so:

Dim ThreadA As Thread
ThreadA = New Thread(AddressOf TedsCode.MyAsyncTask)
ThreadA.IsBackground = True
ThreadA.Start()

Remember that whether a Thread object is a foreground or background thread is really only important when writing managed EXE-based applications such as a console application or a Windows Forms application. The IsBackground property of a Thread object has no effect on applications that have been started using a non-managed EXE such as the ASP.NET worker process.

Finally, when you need to initialize a thread for COM interoperability to run within a single-threaded apartment you should create a new thread. To explain why this is important, I'll provide a little background on COM.

In COM, there are STAs and multithreaded apartment (MTA) threads. The majority of COM components, including all those created with Visual Basic 5.0 and Visual Basic 6.0, are apartment-threaded. Apartment-threaded components are restricted to running on STA threads due to issues related to thread safety and affinity. A small percentage of performance-oriented COM components have been written to run safely on MTA threads.

Before code written with Visual Basic .NET can create and interact with a COM object, the CLR must first initialize the calling thread for COM interoperability. In particular, the CLR must make a system call to initialize the thread as either MTA or STA.

If you create apartment-threaded objects using an MTA thread, you will experience thread switching. Thread switching involves a measurable performance hit because each call to the apartment-threaded COM object must be marshaled from the MTA thread over to the object's STA thread. Thread switching can be avoided by using an STA thread instead of an MTA thread. Therefore, you should always use STA threads when creating objects from apartment-threaded COM components.

By default, the primary thread for an application created with Visual Basic .NET will initialize to an STA thread. When you compile a Windows Forms app or a console app, the Visual Basic .NET compiler automatically adds the STAThread attribute to the Main method that serves as the application's entry point:

Class MyApp
  <System.STAThread()> Shared Sub Main()
    '*** application code here
  End Sub
End Class

The presence of the STAThread attribute forces the application's primary thread to initialize as an STA thread the first time the application does any interoperability work with COM. This is what you want when working with COM components because it is likely that they will be apartment-threaded. In rare cases when you need to write a Visual Basic .NET-based application that uses COM components that are all freethreaded, you can force the application's primary thread to initialize to an MTA thread by explicitly adding the MTAThread attribute to the application's Main method:

Class MyApp
  <System.MTAThread()> _
  Shared Sub Main()
    '*** application code here
  End Sub
End Class

I've just discussed the issues that affect whether an application's primary thread gets initialized as an STA or an MTA thread. Now it's time to look into how things work with secondary threads. The first important point to understand is that you have no control over the threads from the CLR thread pool. They are always initialized as MTA threads. For this reason, you should avoid using asynchronous delegates to execute code which interoperates with apartment-threaded COM components due to the performance degradation that occurs with thread switching.

When you create and manage a secondary thread that is going to interoperate with COM components, you can control whether it is initialized as an STA thread or an MTA thread. After you create an object from the Thread class, you can explicitly set its ApartmentState property to STA before calling Start, as you see here:

Dim ThreadA As Thread
ThreadA = New Thread(AddressOf TedsCode.MyAsyncTask)
ThreadA.ApartmentState = ApartmentState.STA
ThreadA.Start()

'*** When you create a COM object, the CLR
'*** will initialize ThreadA as an STA thread.

You now have a secondary thread which can create and make calls to apartment-threaded COM objects without incurring the overhead of thread switching. This makes it possible to interact with apartment-threaded COM components from a secondary thread in a more efficient manner.

Conclusion

This month I have examined why and how you would create secondary threads. You saw that creating and managing secondary threads to execute methods asynchronously is something you should do only when you cannot use delegates. Such situations include handling long-running tasks, adjusting thread priority, using foreground threads, and using STA threads for more efficient interoperability with apartment-threaded COM components.

My last three columns have examined how to use asynchronous delegates and how to create secondary threads. In short, I've taught you how to get in trouble by introducing multithreaded behavior in your apps. In the next installment of Basic Instincts, I'll show you how to stay out of trouble by designing and writing thread-safe code that is robust in the face of concurrency and that can run dependably in a multithreaded world.

Send your questions and comments for Ted to instinct@microsoft.com.

Ted Pattison is a cofounder of Barracuda .NET, a developer education company that assists companies building collaborative applications using Microsoft technologies. Ted is the author of several books including Building Applications and Components with Visual Basic .NET (Addison-Wesley, 2003).


.NET Matters: XML Comments, Late-bound COM, and More

New information has been added to this article since publication.
Refer to the Editor's Update below.

.NET Matters
XML Comments, Late-bound COM, and More
Stephen Toub

Code download available at: NETMatters0406.exe (174KB)

Q I'd like to use the XML documentation for a method from within that method. Is this possible? I'd like to be able to do the following:

/// <summary>Usage: myApp.exe file1.dat file2.dat ... fileN.dat</summary>
public static void Main(string [] args)
{
        if (args.Length == 0)
        {
            Console.WriteLine(XmlComments.Current.Summary.InnerText);
            return;
        }
        •••
}
Does any class like this exist in the Microsoft® .NET Framework?

A The C# compiler is able to extract XML comments from C# source files. If at compile time, the /doc command-line option (or the equivalent option in the Visual Studio® .NET IDE) is used to specify a target documentation file, all of the comments are extracted from the source files and are written in an XML format to the target file. Visual Studio .NET can then use this file to display IntelliSense® when the XML file is in the same directory as the compiled assembly (the documentation file must also be named the same as the assembly, with a .xml extension). Both the Visual C++® and Visual Basic® compilers will support this feature in Visual Studio 2005.

There are no classes in the .NET Framework written specifically for the purpose of mining and working with these XML comments, but functionality exposed from the System.Xml namespace can be used to make that task quite straightforward. In fact, in the code download for this column I've included a class that lets you use the syntax you've described.

In order to retrieve the XML documentation for a specific type or class member, you first need to find the XML file containing that documentation. To do that, you need to know in which assembly the type or class member is declared. Given the correct MemberInfo, you can use the MemberInfo.DeclaringType to get the Type that declares the member (in the case where you want the documentation for a Type, you don't need to obtain its DeclaringType as you already have a valid Type object). The assembly that declared this type can then be found using the Type's Assembly property, and the Assembly instance returned from that property exposes its file location on disk through the Location property. You can then substitute .xml for the extension of the assembly file and load the XML document from that location into an XmlDocument. Note that most of the assemblies in the .NET Framework are loaded from the Global Assembly Cache (GAC), but their XML documentation files are in the runtime directory. If the system fails to load the documentation from the assembly's folder, try the location returned by RuntimeEnvironment.GetRuntimeDirectory (see last month's column for more information).

Figure 1 Accessing XML Comments Programmatically
Figure 1 Accessing XML Comments Programmatically

Once you have the XmlDocument loaded with the assembly's XML comments, XPath expressions can be used to find the comments for the specified MemberInfo. This approach is diagrammed in Figure 1 and the code that retrieves the XmlNode is shown in Figure 2, creating an XPath expression based on the documentation at Processing the XML File.

Figure 2 Using XPath to Retrieve Comments (Close Figure 2)
private XmlNode GetComments(MemberInfo mi)
{
    Type declType = (mi is Type) ? ((Type)mi) : mi.DeclaringType;
    XmlDocument doc = LoadAssemblyComments(declType.Assembly);
    if (doc == null) return null;
    string xpath;

    // Handle nested classes
    string typeName = declType.FullName.Replace("+", ".");

    // Based on the member type, get the correct xpath query
    switch(mi.MemberType)
    {
        case MemberTypes.NestedType:
        case MemberTypes.TypeInfo:
            xpath = "//member[@name='T:" + typeName + "']";
            break;

        case MemberTypes.Constructor:
            xpath = "//member[@name='M:" + typeName + "." +
                "#ctor" + CreateParamsDescription(
                    ((ConstructorInfo)mi).GetParameters()) + "']";
            break;

        case MemberTypes.Method:
            xpath = "//member[@name='M:" + typeName + "." +
                mi.Name + CreateParamsDescription(
                    ((MethodInfo)mi).GetParameters());
            if (mi.Name == "op_Implicit" || mi.Name == "op_Explicit")
            {
                xpath += "~{" +
                    ((MethodInfo)mi).ReturnType.FullName + "}";
            }
            xpath += "']";
            break;

        case MemberTypes.Property:
            xpath = "//member[@name='P:" + typeName + "." +
                mi.Name + CreateParamsDescription(
                    ((PropertyInfo)mi).GetIndexParameters()) + "']";
            break;

        case MemberTypes.Field:
            xpath = "//member[@name='F:" + typeName + "." + mi.Name + "']";
            break;

        case MemberTypes.Event:
            xpath = "//member[@name='E:" + typeName + "." + mi.Name + "']";
            break;

            // Unknown member type, nothing to do
        default:
            return null;
    }

    // Get the node from the document
    return doc.SelectSingleNode(xpath);
}

I've created the XmlComments class so that the constructor takes the MemberInfo for which you want the comments. The constructor retrieves the XmlNode for the member's comments if one is available and uses XPath queries to parse the results looking for nodes and node lists that represent the summary, method parameters, exceptions that can be thrown, and so on. These nodes and node lists are then made available through public properties on XmlComments such as Summary and Exceptions. In order to support the syntax in your question, I've added a static Current property that uses the StackTrace class to retrieve the frame for the calling method and then uses the associated MethodInfo in order to construct an XmlComments instance:

public static XmlComments Current
{
        get { return new XmlComments(
                  new StackTrace().GetFrame(1).GetMethod()); }
}

There are a few corner cases I needed to be aware of when writing this, the most noticeable being accessor methods for properties and events. Accessors don't have their own XML comments, so given a MethodInfo for an accessor I need to be able to access the parent MemberInfo. One approach would be to use simple string manipulation to parse the name of the property or event from the accessor's name (for example, trimming off the "get_" or "set_" from the beginning of the name). I opted for a slightly more robust approach. For a given type, I store a Hashtable that maps accessor MethodInfo objects to their parent MemberInfo (which I create by looping over all properties and events in a Type, adding to the table all accessors found). When I need to check if a method is an accessor, I can query this table; if the MethodInfo exists as a key in the table, then not only is it an accessor but I also have immediate access to the MemberInfo for the parent which I can pass to the GetComments method shown in Figure 2 instead of the original MethodInfo for the accessor.

Of course, this whole solution only works if you build the XML comments for your application. And it's important to remember that the XML comments are not compiled into the assembly but rather into a separate .xml file, which means any solution that uses this technique should be tested with and without the .xml file present, in the correct location, and correctly named.

Q I'm trying to access a COM component at run time, but I don't have an interop assembly for it. Is it still possible for me to use it? I've considered dynamically generating an assembly when I need it, but that seems like overkill. The problem is that I'm writing a generic utility class in .NET and I won't know the ProgID of the COM component at compile time. It does implement IDispatch.

A The Type class exposes a method, GetTypeFromProgID, that you can use for this very scenario (you can use Type.GetTypeFromCLSID if you know the CLSID rather than the ProgID). You supply it with the ProgID of your component and it'll return to you a Type that describes it. You can then use the Activator class to create an instance of the component and various methods on Type, such as InvokeMember, SetProperty, and GetProperty, to access it.

To make it easier to work with, I've created the helper class shown in Figure 3 that makes performing the most common operations easier. As an example, the following code creates an instance of the SharePoint.StssyncHandler COM component and queries its GetStssyncAppName method to determine which application on your machine handles the stssync protocol:

using(LateBoundComHelper lb =
    new LateBoundComHelper("SharePoint.StssyncHandler"))
{
    string stsSyncHandler = (string)lb.Invoke("GetStssyncAppName");
}
With Microsoft Outlook® 2003 installed, stsSyncHandler will most likely be set to "Outlook".

Figure 3 Accessing Late-bound COM Objects (Close Figure 3)
public class LateBoundComHelper : IDisposable
{
    private Type _type;
    private object _obj;

    public LateBoundComHelper(string progId) :
        this(Type.GetTypeFromProgID(progId, true))
    {
    }

    public LateBoundComHelper(Type t)
    {
        if (t == null) throw new ArgumentNullException("t");
        _type = t;
        _obj = Activator.CreateInstance(t);
    }

    public object Invoke(string methodName, params object [] args)
    {
        ThrowIfDisposed();
        return _type.InvokeMember(methodName,
            BindingFlags.InvokeMethod, null, _obj, args);
    }

    public object this[string propertyName]
    {
        get
        {
            ThrowIfDisposed();
            return _type.InvokeMember(propertyName,
                BindingFlags.GetProperty, null, _obj, null);
        }
        set
        {
            ThrowIfDisposed();
            _type.InvokeMember(propertyName, BindingFlags.SetProperty,
                               null, _obj, new object[]{value});
        }
    }

    private void ThrowIfDisposed()
    {
        if (_obj == null) throw new ObjectDisposedException(GetType().Name);
    }

    public void Dispose()
    {
        if (_obj != null)
        {
            if (_type.IsCOMObject)
            {
                while (Marshal.ReleaseComObject(_obj) > 0);
            }
            else if (_obj is IDisposable) ((IDisposable)_obj).Dispose();
            _obj = null;
        }
    }
}

As an aside, a similar helper class exists buried in the depths of System.Web. Using your favorite disassembler or decompiler, find the System.Web.Mail.SmtpMail class. SmtpMail declares a nested class named LateBoundAccessHelper that it uses to provide late-bound access to CDONT.NewMail on Windows NT® and also to CDO.Message on Windows® 2000 and higher.

If for some reason you still want to dynamically generate an assembly from a COM type library, you can use the ConvertTypeLibToAssembly method on the System.Runtime.InteropServices.TypeLibConverter class to generate the assembly for you. In fact, the tlbimp.exe and tlbexp.exe utilities included with the .NET Framework SDK are wrappers around this class (tlbimp.exe uses ConvertTypeLibToAssembly, while tlbexp.exe uses ConvertAssemblyToTypeLib). An example of its usage is available at TypeLibConverter.ConvertTypeLibToAssembly Method.

Q My team uses Debug.Assert throughout our Windows Forms app. When an assertion fails, an assertion dialog is displayed. I'd like to be able to control when this dialog is displayed based on certain environmental conditions, but I don't want to have to change every call to Assert to account for this. Is this possible?

A As with any programming challenge, there are most likely multiple ways to solve this problem. One solution which works well is to write your own trace listener which derives from DefaultTraceListener. Override its Fail method and in your implementation only call to the base method when you want to allow the assertion to be displayed. Remember that the DefaultTraceListener implementation already only displays the dialog if the calling code has the UIPermissionWindow.SafeSubWindows permission and if the AssertUIEnabled setting is true in the application's diagnostics configuration. Once you have your custom trace listener, you can remove the DefaultTraceListener and substitute yours in its place. You can do it programmatically, like so:

System.Diagnostics.Debug.Listeners.Clear();
System.Diagnostics.Debug.Listeners.Add(new MyCustomTraceListener());
Or you can use a configuration file, as shown here:
<system.diagnostics>
    <trace autoflush="true" indentsize="0">
        <listeners>
            <clear/>
            <add name="MyTraceListener"
                type="NetMatters.MyCustomTraceListener, MyAssembly" />
        </listeners>
    </trace>
</system.diagnostics>

If you decide to take another approach and want to be able to display the assertion dialog at will without relying on the Debug or TraceListeners collection, an easy way is to again use the functionality built into the DefaultTraceListener:

new DefaultTraceListener().Fail();
This will display the assertion regardless of which listeners are currently configured in the listeners collections. Remember, too, that referring to the Debug and Trace listeners collections as separate entities is a bit of a misnomer. Internally, both Debug.Listeners and Trace.Listeners wrap TraceInternal.Listeners, and thus a listener added to one of them is actually added to both. In fact, the real difference between the Trace and Debug classes is the value passed as a parameter to the ConditionalAttribute attached to each method on the classes. This attribute controls the circumstances under which the call sites are compiled into the target Microsoft intermediate language (MSIL).

Q For my application, I need a buffer that only keeps the last n objects added to it. I checked the System.Collections namespace but I couldn't find anything appropriate. Does such a class exist? Is there any way to reuse existing functionality in the .NET Framework to make my implementation easier?

A As far as I know, System.Collections.Queue is the closest class in the Framework to what you want. In fact, it's so close that with only a few added lines of code you can create your own class that behaves the way you require.

The Queue class implements the classic First In First Out (FIFO) data structure of the same name. Internally, it maintains an array of objects and two integers that mark the head and tail positions within that array. When the Enqueue method is used to add an item to the queue, the internal capacity of the queue is checked to determine whether the array has enough room to store the new item. If it doesn't, the size of the array is increased. After it has confirmed there's enough room, the new item is added at the position specified in the tail integer. The tail integer is then incremented, possibly causing it to wrap around to the beginning of the array; this can happen if items have been dequeued, incrementing the head pointer and leaving a gap at the beginning.

In order to implement the circular buffer, all you need to do is ensure that there's a maximum capacity for the queue and that you never exceed that limit. One solution is to override the Enqueue method and first check to determine whether you've reached the limit. If you have, simply remove an item using the Dequeue method, then proceed by delegating to the base class. The code in Figure 4 shows the basics of the implementation. Note that in order to make this truly robust, you should override and implement other methods like Queue.Clone, which returns a Queue instance, but I'll leave that as an exercise for you.

Figure 4 Implementing a Circular Buffer (Close Figure 4)
public sealed class CircularBuffer : Queue
{
    private int _capacity;

    public CircularBuffer(int capacity) : base(capacity)
    {
        if (capacity <= 0)
                 throw new ArgumentOutOfRangeException("capacity");
        _capacity = capacity;
    }

    public override void Enqueue(object obj)
    {
        if (base.Count == _capacity) base.Dequeue();
        base.Enqueue(obj);
    }

    public override void TrimToSize()
    {
        throw new NotSupportedException();
    }
}

Q I want to write a regular expression that will detect strings that have only legal combinations of '{' and '}' characters (where every '{' is at some point followed by a matching '}'), but I can't figure out how to express it. Is this even possible? For example, the string "abcd{1234{5678}ef}g" is valid, as is "{}", but "}{" and "abcd{1234{5678{ef}g}" are invalid. Help!

A Matching all strings with equal numbers of '{' and '}', regardless of their order, requires a grammar more powerful than regular grammars (those that can be processed by regular expressions). It can easily be done with context-free grammars (CFG), but there are no classes in the Framework for parsing those.

If all you want to do is find out if there are legally matching '{' and '}' characters, you can write some simple code to walk through each character in the string and enforce the rule manually. Start with a count of 0. Every time you see a '{', increment the count by 1; every time you see a '}', decrement the count by 1. If you ever get a negative count at any step along the way, your string is not a match. If at the end the count equals 0, then it is a match:

private bool IsMatch(string str)
{
    int count=0;
    for(int i=0; i<str.Length; i++)
    {
        if (str[i] == '{') count++;
        else if (str[i] == '}') count--;
        if (count < 0) return false;
    }
    return count == 0;
}

Even if there were Framework classes that provided CFG support, you can't get much better performance than something like what I've just shown, and what you're doing should be clear to anyone maintaining the code (especially if you add comments). [ Editor's Update - 7/20/2004: The System.Text.RegularExpressions.Regex class supports grouping constructs with lookahead and lookbehind modifiers. This allows it to support some grammars more powerful than regular grammars (for more information, see Grouping Constructs). As such, matching all strings with an equal number of properly-ordered left and right brackets is possible using a pattern like the following:

string pattern =
    @"^((?<openBracket>\{) | [^\{\}] |" +
    @"(?<closeBracket-openBracket>\}))*" +
    @"(?(openBracket)(?!))$";
Regex r = new Regex(pattern, RegexOptions.IgnorePatternWhitespace);
However, the IsMatch method shown previously will still perform better than this equivalent Regex.]

Q I'm writing an application that makes HTTP requests to various Web sites using the URLs provided to me by a user. I only want to make the request if the URL is on my local intranet. Is there an easy way to perform this check?

A The System.Security.Policy namespace provides a Zone class that is used by the common language runtime (CLR) when enforcing Code Access Security (CAS). You can explicitly use the Zone class to perform the desired check:

private static bool IsIntranet(Uri url)
{
    System.Security.Policy.Zone zone =
        System.Security.Policy.Zone.CreateFromUrl(url.ToString());
    return zone.SecurityZone == System.Security.SecurityZone.Intranet;
}
URL membership in a zone is based on your Internet Explorer configuration. To configure zones, use the Security tab in the Internet Explorer options panel.

A safer approach might be to run the code that makes these requests in a CAS sandboxed environment that only allows requests to intranet URLs. You can read more about sandboxing at Sandboxing code dynamically.

Send your questions and comments to netqa@microsoft.com.

Stephen Toub is the Technical Editor for MSDN Magazine.


C++ Q&amp;A: Performance Optimization, Controls versus Components

C++ Q&A
Performance Optimization, Controls versus Components
Paul DiLascia

Code download available at: CQA0406.exe (181KB)

Q What's the best and fastest way to allocate a large number of small class objects (for example, 10,000 small records)? MFC serialization, of course, streams the objects in and out and does everything necessary. However, lots of time is taken with memory allocation and deallocation. Is there some way to improve this?

Dave Kerrison

A I can't tell you the best way because that depends on the specifics of your application and how it's used. Performance and memory allocation are such huge topics that books have been written about them. There's no perfect solution that works for all situations. Optimization always requires making intelligent trade-offs between speed and other resources. For example, you can make searches very fast if you're willing to build a huge index. Or you might make display very fast at the expense of load time. That said, I can give you an overview of some issues to consider, as well as some tools and approaches to help you discover the answers yourself.

If you're unhappy with your program's performance, the first thing to do is get a clear picture of exactly where the bottlenecks are located. You can buy a sophisticated profiler that produces all kinds of performance reports, but if all you need is a quick idea of where your code is spending its time, you may be able to make do with a few home-grown tools. I wrote a little class, ShowTime, that reports how long it takes for some section of your code to execute. To use it, all you have to do is instantiate a ShowTime object on the stack at the top of the code block you want to clock:

void CalculatePi()
{
   ShowTime st("Calculating pi");
   // do it
}
This will generate a TRACE message that looks something like:
Calculating pi: 342 msec

How does ShowTime work? It uses the familiar C++ constructor/destructor pattern for smart pointers and any other situation where you want to automatically do something at the beginning of a code block and something else at the end. ShowTime's constructor saves the clock time (number of clock ticks since the process started) in a data member; the destructor then subtracts this from the end time and generates a message. Since the ctor/dtor are invoked at the beginning/end of your code block, this measures the total elapsed time. Figure 1 shows the code.

Figure 1 ShowTime (Close Figure 1)
//////////////////
// ShowTime displays diagnostics that tell how long an operation took.
// To use, instantiate a ShowTime object at the top of the code block
// you want to clock. If you create a PerfLog, ShowTime will use it.
//
//        // in some block of code:
//        {
//            ShowTime("time for operation X is");
//            // do operation X
//        }
//
class ShowTime {
protected:
    LPCTSTR msg;            // display message
    clock_t start;          // start time
    clock_t end;            // end time

    // convert clock ticks to milliseconds
    static long msec(clock_t clocktime)
    {
        return (long)(clocktime * CLOCKS_PER_SEC / 1000);
    }

public:

    // ctor: save message, record start time.
    ShowTime(LPCTSTR m)
    {
        msg = m;
        start = msec(clock());
    }

    // dtor: record end time and log results.
    ~ShowTime()
    {
        end = msec(clock());
        TCHAR buf[512];
        _snprintf(buf, 512, _T(" %s: %d msec\n"), msg, end-start);
        PerfLog::Write(buf);
    }
};

ShowTime isn't very sophisticated. For example, it doesn't account for multithreading and it doesn't report how much time is spent in each function the way a profiler would. But for everyday purposes it can give you a good idea of where your app is spending time. Don't forget to run your performance tests on the release build! After all, that's the one you ship. Moreover, differences between release and debug builds can skew your results. For example, depending how you've set things up, the debug builds may do extra stack-checking that slows your app down. Since there's no TRACE in release builds, I added another class, PerfLog, that lets you direct the performance stats to a text file:

// open log file
PerfLog mylog("MyResults.log");
Now ShowTime writes to the file MyResults.log, as well as the TRACE stream. Also, don't forget to remove the performance monitoring before you ship!

With ShowTime in hand, I can begin to answer your question. I wrote a program, PerfTest, that's a typical MFC doc/view app with a document that allocates a linked list of 20,000 fixed-length records using three different methods.

Method 1 is the vanilla MFC way. The list is implemented as an MFC CObList which uses a separate list cell for each item in the list. The list cell is nothing more than a tiny structure that holds pointers to the next/previous cell and the object itself. So CObList has a 12-byte overhead for every item, but list cells are necessary if you need several lists pointing to the same items (for example, if you want to keep your objects sorted several different ways).

Method 2 represents the first performance improvement. Here, the records themselves store the next pointer, so there's no separate list cell. This method is possible only if you have one list—that is, if you don't need several lists pointing to the same objects sorted in different ways. In this scenario, an array might be even more efficient. But even with one list, if you alter the order frequently, a list is faster than an array because changing pointers is faster than moving objects in memory.

Methods 1 and 2 both allocate each record/object separately with a call to new. If you allocate 20,000 objects, that's 20,000 calls to new. Method 3 allocates all 20,000 objects as a single array with a single call, as shown here:

m_array = new CMyRecord[20000];

The records are then linked by setting each record's next pointer to the next record in the array. The allocation is quick since there's only one function call, but it requires a single contiguous chunk of memory large enough to hold 20,000 records. Of course, the compiler still has to make sure your objects get initialized. When you use the vector form of new, the compiler generates code to invoke each object's constructor, so that's 20,000 calls to the ctor. Likewise when you delete [], you'll get 20,000 calls to the dtor. If your ctor/dtor are empty, these calls will be optimized away. But if they actually do something, the code will require finite time to execute. In this case, you might be able to further speed performance by allocating the array as raw bytes (thus avoiding the ctor/dtor calls), then initialize the objects using some handmade code—but now you're living on the fringe.

Yet another whole class of allocation performance improvement strategies—which I will mention but not explore—involves overloading operator new, as shown in the following code:

class CMyRecord {
public:
  void* operator new (size_t nbytes) {
    return FancyAlloc(nbytes);
  }
  void operator delete (void* p) {
    FancyFree(p);
  }
};

You can implement FancyAlloc and FancyFree however you like, as long as they allocate/free a block of memory that's the right size. If you have a particular object that's used frequently throughout your program, one of the most common tricks is to maintain a free pool of objects. Instead of calling free, your delete operator adds the freed object to a list called the free pool. The allocator then looks for an object in the free pool before calling malloc. This can make allocation/deallocation extremely fast, but you have to be careful you don't work at cross-purposes with the built-in allocators, which may already be doing something similar and which work pretty well for most purposes.

In any case, as I said at the outset, a full discussion of memory allocation is beyond the scope of one column. There's an almost endless number of tricks you can play. I chose these three methods not because they are exhaustive, but to illustrate some ways in which you can optimize your program.

To decide which method is fastest, the key question is: fastest for what? If all you're looking at is allocation, then you'd expect Method 1 (CObList) to be the slowest since it allocates the most objects and Method 3 (bulk alloc) to be the fastest since it allocates with one call. But what about reading and writing? Each method implies a different serialization strategy. For Methods 1 and 2, each record is serialized with a separate call to CMyRecord::Serialize (see Figure 2). For Method 3, I write the entire array in one fell swoop. Here again, this is an arbitrary choice I made for educational purposes. I could just as easily serialize the giant array as individual records, as in Method 2. Note that things get a little complex when you implement unorthodox serialization strategies.

Figure 2 Timing Serialization Strategies (Close Figure 2)
////////////////////////////////////////////////////////////////
// MSDN Magazine — June 2004
// If this code works, it was written by Paul DiLascia.
// If not, I don't know who wrote it.
// Compiles with Visual Studio .NET 2003 on Windows XP. Tab size=3.
//
#include "StdAfx.h"
#include "PerfTest.h"
#include "Doc.h"
#include "ShowTime.h"
•••
const BIGNUMALLOC = 20000; // allocate this many objects

PerfLog mylog(_T("PerfTest.log")); // open log file

// Typical fixed-length record object.
class CMyRecord : public CObject {
protected:
    CMyRecord* next; // next record in list
    TCHAR name[64];
    int num;
    BOOL flag;
    friend class CMyDoc;
public:
    CMyRecord() { }
    CMyRecord(int i)    { InitRec(i); }

    void InitRec(int i) {
        _snprintf(name, 64, _T("rec %d"),i);
        num = i;
        flag = TRUE;
    }
    virtual void Serialize(CArchive& ar);
    DECLARE_SERIAL(CMyRecord)
};
IMPLEMENT_SERIAL(CMyRecord, CObject, 1)

// Serialize one record: write/read string and other data.
void CMyRecord::Serialize(CArchive& ar) {
    if (ar.IsStoring()) {
        ar.WriteString(name);
        ar.WriteString(_T("\n")); // don't forget!
        ar << num << flag;
    } else {
        ar.ReadString(name,countof(name));
        ar >> num >> flag;
    }
}

// Serialize document. This fn serializes each record list array using a
// different technique to illustrate the performance of each.
void CMyDoc::Serialize(CArchive& ar) {
    CString msg;
    msg.Format(_T("CMyDoc::Serialize: %s\n"),
        ar.IsStoring() ? _T("Write") : _T("Read"));
    PerfLog::Write(msg);

    CFile& file = *ar.GetFile();

    {
        // Method 1: Serialize record list the vanilla MFC way (CObList)
        ShowFileUsed fs(file);
        ShowTime st(_T("M1: Serializing CObList"));
        m_recObList.Serialize(ar);
    }
    {
        // Method 2: Serialize individual records (no CObList/list cell)
        ShowFileUsed fs(file, _T("Total size"));
        ShowTime st(_T("M2: Serializing individual records"));
        if (ar.IsStoring()) {
            ar << m_nRecList;
        } else {
            ar >> m_nRecList; // read number of records
            AllocateRecordsIndividual(m_nRecList); // allocate them
                                                   // (link too)
        }
        for (CMyRecord* prec=m_pRecList; prec; prec=prec->next) {
            prec->Serialize(ar);
        }
    }
    {
        // Method 3: Read/write all records as single memory block
        ShowFileUsed fs(file, _T("Total size"));
        ShowTime st(_T("M3: Serializing bulk array"));
        if (ar.IsStoring()) {
            ar << m_nArray;
            ar.Write(m_pArray,m_nArray*sizeof(CMyRecord));
        } else {
            ar >> m_nArray;
            AllocateRecordsBulk(m_nArray);
            ar.Read(m_pArray,m_nArray*sizeof(CMyRecord));
            LinkRecords(m_pArray, m_nArray);
        }
        TRACE(_T("len=%d\n"),m_nArray);
    }
}

// Create new document: allocate three large record arrays using the three
// different methods described above.
BOOL CMyDoc::OnNewDocument()
{
    PerfLog::Write(_T("CMyDoc::OnNewDocument\n"));

    VERIFY(CDocument::OnNewDocument());
    {
        ShowTime st(_T("M1: Allocating for CObList"));
        AllocateRecordsObList(BIGNUMALLOC);
    }
    {
        ShowTime st(_T("M2: Allocating individual records"));
        AllocateRecordsIndividual(BIGNUMALLOC);
    }
    {
        ShowTime st(_T("M3: Allocating bulk array"));
        AllocateRecordsBulk(BIGNUMALLOC);
        LinkRecords(m_pArray, m_nArray);
    }
    return TRUE;
}

// Allocate records using Method 1: MFC CObjList
void CMyDoc::AllocateRecordsObList(int nrec) {
    for (int i=0; i<nrec; i++) {
        m_recObList.AddTail(new CMyRecord(i));
    }
}

// Allocate individual records using Method 2: Manual linked list.
// Also links the records.
void CMyDoc::AllocateRecordsIndividual(int nrec) {
    CMyRecord* prev = NULL;
    for (int i=0; i<BIGNUMALLOC; i++) {
        CMyRecord* rec = new CMyRecord(i);
        ASSERT(rec);
        if (prev) {
            prev->next = rec;
        } else {
            m_pRecList = rec;
        }
        prev = rec;
    }
    prev->next = NULL;
    m_nRecList = nrec;
}

// Allocate records using Method 3: allocate array as one memory block.
void CMyDoc::AllocateRecordsBulk(int nrec) {
    m_pArray = new CMyRecord[nrec];
    ASSERT(m_pArray);
    for (int i=0; i<nrec; i++) {
        m_pArray[i].InitRec(i);
    }
    m_nArray = nrec;
}

// Link bulk-allocated records (make each record point to the next)
void CMyDoc::LinkRecords(CMyRecord* array, int nrec) {
    if (array && nrec>1) {
        for (int i=0; i<nrec-1; i++) {
            array[i].next = &array[i+1];
        }
        array[nrec-1].next = NULL;
    }
}

Any time you serialize data that contains pointers, you have to somehow convert the pointers to something meaningful on disk because the odds that your data will be loaded into exactly the same memory location the next time you read the file are about as low as your chances of winning the lottery without buying a ticket. MFC performs a lot of magic to convert pointers to and from disk IDs. For PerfTest, I save the records in list order, so there's no need for IDs. I can simply relink the list after it's loaded (Methods 2 and 3). Of course, this means Method 3 fails if you change the list order (I've implicitly assumed the list is really an array).

Finally, another serialization issue is: should you serialize your records as fixed or variable length? CMyRecord contains a 64-character array. CMyRecord::Serialize uses CArchive::ReadString/WriteString to serialize only the characters used, not all 64. If the string is "foo", only four characters are serialized ("foo" plus '\n' required at the end). Method 3 writes the entire array. It serializes all characters, even if the string is empty. How wasteful is that? It depends. If your string is a 10-character phone number or 16-character credit card number, chances are most records have the full complement of characters, so it may be OK to serialize them all. But if the string is an address or optional field, you may have megabytes of zeroes on disk. Just something to consider. And it's not just disk space that's wasted, but speed too since it takes more time to read/write more bytes. Method 3 lets you read/write the entire list with one call—but is it really faster?

To find out, I sprinkled PerfTest with ShowTime objects to show how long various operations take. I then ran the program, created a new file, saved it, read it back in, and quit. Figure 3 shows the log generated by ShowTime, annotated with comments to explain the action. As expected, Method 1 (CObList) is the slowest for allocation (130 ms) and Method 3 the fastest (70 ms). The difference is more pronounced for deleting the objects. What about serialization? For writing, there's not much difference between Methods 2 and 3—60 versus 61 ms. Apparently the time gained by having only one call to write is lost by writing so much data—536KB for Method 2 versus a whopping 2.9MB for Method 3. (I wrote another class, ShowFileUsed, that reports the difference between the start and end positions in the archive's CFile during serialization.) For reading, Method 3 fares better, but this could also be a side-effect of disk caching—another factor you have to consider when you embark upon your performance tests.

Figure 3 Log (Close Figure 3)
// Start PerfTest program. This is an MFC SDI app, so MFC creates a new
// doc. Program allocates 20,000 records using 3 different methods.
CMyDoc::OnNewDocument
  M1: Allocating for CObList: 130 msec
  M2: Allocating individual records: 90 msec
  M3: Allocating bulk array: 70 msec

// Save the file (write test).
CMyDoc::Serialize: Write
  M1: Serializing CObList: 200 msec
  file size = 577457 bytes
  M2: Serializing individual records: 60 msec
  Total size = 536514 bytes
  M3: Serializing bulk array: 61 msec
  Total size = 2879488 bytes

// Open the same file (read test). MFC called DeleteContents first.
CMyDoc::DeleteContents
  M1: Delete CObList: 80 msec
  M2: Delete individual records: 51 msec
  M3: Delete bulk array: 10 msec

CMyDoc::Serialize: Read
  M1: Serializing CObList: 240 msec
  file size = 581553 bytes
  M2: Serializing individual records: 170 msec
  Total size = 536514 bytes
  M3: Serializing bulk array: 90 msec
  Total size = 2877516 bytes

// Quit the app. This calls DeleteContents again.
CMyDoc::DeleteContents
  M1: Delete CObList: 100 msec
  M2: Delete individual records: 50 msec
  M3: Delete bulk array: 10 msec

ShowTime provides raw performance figures, but you have to interpret them with a heavy dose of common sense. It took almost twice as long to allocate 20,000 records using CObList (Method 1) than allocating them as a bulk array (Method 3)—but will anyone ever notice it takes 70 ms longer to open a file? That's literally less than the time it takes to blink. And is the time saved by doing a bulk-read really worth using five times as much disk space? For PerfTest, the answer is a resounding no. For some other app, it might be yes. The bottom line is that you can't be sure until you experiment. You can always make your program faster, but usually only at the expense of some other resource like memory, disk space, complexity (which translates into reliability, robustness, and programmer-hours), or speed in some other area. Performance optimization is an art. The trick is to understand your app well, and buy or write some tools that let you see what your app is really doing. You may be surprised what you discover.

Q I'm learning about the Microsoft® .NET Framework and I don't understand the difference between a control and a component. I see these terms used interchangeably, but when do I derive from Control and when do I derive from Component?

Linda Berno

A Good question! The short answer is that a control is a component that has a user interface. The long answer has historical roots in the earliest days of Windows®, when a control was any child window—button, listbox, edit control, or static text—in a dialog. The idea was that these windows are like the knobs and buttons—controls—you use to operate a radio or electronic gadget. As more kinds of controls were added (comboboxes, date/time controls, and so on), a control became more generally known as any child window, whether used in a dialog or any other kind of main window. Pretty soon even BASIC programmers were writing their own specialized controls and, naturally, people wanted to share them. One way to share code is to mail the source on a floppy, but that's so retro even I wouldn't do it. What was needed was some mechanism whereby developers could build controls that other programmers could plug into their apps with a minimum of fuss. This was the motivation for VBX controls, OLE controls, OCX and eventually ActiveX® controls.

This is where the confusion between control and component comes in. Because in order to solve the problem of reusable controls, all of these technologies had to first solve the more general problem of reusable components. (COM, if you still remember it, stands for Component Object Model.) In software lingo, the term component refers to any reusable object or body of code that interacts with other objects. Ever since subroutines were invented, programmers have been pursuing the holy grail of software engineering: a unified theory of programming that lets programmers build large systems from off-the-shelf building blocks—that is, components—written in the programming language of your choice. Every new programming paradigm from subroutines to OOP to DLLs to COM to the .NET Framework represents a different scheme for providing reusability. VBX used DLLs with hardwired names. COM uses interfaces and IUnknown. The .NET Framework uses its Microsoft intermediate language (MSIL) layer and common language runtime (CLR) to provide the unifying glue.

So while controls are a prime example of components (and historically what drove much of their development) they are not the only kind of component. A component need not display anything or have a user interface. A component might perform scientific calculations, collect performance data, count the number of milliseconds since January 1, 1970, or read the number of dollars in President Bush's campaign coffer. Figure 4 shows examples of Visual Studio® .NET components that aren't controls.

Figure 4 Components
Figure 4 Components

In the .NET Framework, the terms control and component have added meaning specific to .NET. The Component class provides the base implementation for objects that can be used on a design surface such as the Windows Forms Designer or Web Forms Designer. A Component is anything you can drag onto a form. The Component class implements IComponent, one of three interfaces used to support reusable software components in .NET: IComponent, ISite, and IContainer. These interfaces are vastly simpler than their COM cousins from the days of OLE. IContainer is little more than a list of Components with Add/Remove methods and a Components property to get the Components as a ComponentCollection.

IComponent derives from IDisposable and has just one property, Site, to get the component's ISite interface. A Component may or may not have a Site. ISite has just four properties including the site's Name and DesignMode, which controls whether the component is in—what else?—design mode. ISite derives from another interface, IServiceProvider, which has a single method, GetService. In COM, IServiceProvider is kind of like QueryInterface—it lets you query an object for some interface by ID ("Fetch me your IWebBrowser2, please"), but differs from QueryInterface in that the object itself doesn't necessarily implement the interface itself, it merely knows where and how to get it. Likewise in the .NET Framework, IServiceProvider is a generic way that you can get some other interface or object—service—that an object might know about without having to implement it.

The .NET Framework makes it easy to write reusable components. No more IDL, no more type-definition languages, no more onerous design-time support. Through the magic of reflection, the CLR already knows everything it needs to know about your classes just from the code itself. To add design-time support, all you have to do is tag your properties with extra designer attributes. For example in managed C ++:

// in CMyControl
[Category(S"Appearance")]
[Description(S"Specifies widget foreground color.")]
_property Color get_ForeColor() { ... }
_property void set_ForeColor(Color value) { ... }
Now the Forms Designer lists your ForeColor property under "Appearance" and uses your Description in help text. For more information about design-time attributes, read the section "Design-Time Attributes for Components" in the .NET Framework documentation.

Figure 5 Class Hierarchy
Figure 5 Class Hierarchy

Figure 5 shows part of the .NET Framework class hierarchy that illustrates what I've been saying. As you can see, Control derives from Component. This is another way of saying that a Control is a Component (but not vice versa). More specifically, a Control is a Component that has a user interface—that can draw stuff and interact with the user. The Control class is also the base class for all managed window classes—Forms, Buttons, Grids, Panels, Toolbars, and so on. The Control class is where WndProc and ClientSize are defined, as are all the standard window events like GotFocus and Click. Web controls (System.Web.UI.Control) are also components, but not in the strict sense of deriving from the System.ComponentModel.Component class. (For Web controls, the System.Web.UI. Control itself implements IComponent.)

In addition to implementing IComponent, System.ComponentModel.Component also provides all the marshaling support that components need, but it does so by deriving from MarshalByRefObject. If you want to build a component that marshals by value, you can derive from MarshalByValueComponent (which implements IComponent, IDisposable, and IServiceProvider). System.Data.DataColumn, DataSet, and DataTable are examples of marshal-by-value components. These objects pass their actual data across machine/process boundaries.

If you're writing a reusable widget that you want people to be able to drag and drop onto their forms using the Forms Designer, you must derive from Component. If your widget also has a user interface—if it creates a window, paints, or interacts with the user—you should derive from Control. Got it?

Send your questions and comments for Paul to cppqa@microsoft.com.

Paul DiLascia is a freelance writer, consultant, and Web/UI designer-at-large. He is the author of Windows++: Writing Reusable Windows Code in C++ (Addison-Wesley, 1992). Paul can be reached at http://www.dilascia.com.


Resource File: Blogging Tools

Resource File
Blogging Tools
Aaron Skonnard

Whether you're looking for a tool to help you author your first weblog, a hosting service to host it, a customizable weblog framework, or desktop authoring tools, there are plenty to choose from. For reading other people's weblogs, there are a variety of RSS aggregators at your disposal.

The most popular hosted weblog services include Blog*Spot (via Blogger.com), TypePad.com, LiveJournal.com, Xanga.com, and DiaryLand.com. All of these sites make it easy to set up a new weblog in minutes, after which you can begin posting entries using a simple Web UI. All of these services offer similar features that can include automatic syndication using RSS, template customization, WYSIWYG editing, spell checking, commenting, searching, photo albums, mobile blogging, automatic surveys, and some interesting social networking features. Most offer free limited accounts as well as full-featured commercial accounts.

The point of these services is to make blogging a whole lot easier. However, if you use a service, you'll always have a limited feature set because you don't own the code. Another major drawback is that your weblog content resides on someone else's server. Although some services make it possible to export weblog content, the overall approach doesn't let you move your weblog to a new location (or into a new application) at a later time.

Blogger.com is one site that provides an alternative, allowing you to publish your weblog to another Web site of your choice (Blog*Spot being the default option). Blogger.com stores the weblog content on its server and on the target location, so you can change publication destinations whenever you want.

If you're interested in creating your own weblog hosting service, check out Userland's Manila 9.0, a commercial content management system designed around RSS, or one of the various open source weblog frameworks. You get the source code with the latter, which allows you to customize your weblog application to fit your needs, plus you can benefit from enhancements made by others. The most popular open source weblog frameworks are probably MovableType and Greymatter, both of which require Perl and CGI support on the server. The most popular among Windows-based developers are .Text and dasBlog, both of which are implemented using the Microsoft .NET Framework.

Both the .Text and dasBlog frameworks support skinning, multiple versions of RSS, the new ATOM syndication format, categories, trackbacks, referrals, SMTP notifications, and proprietary Web service APIs. The main difference between them is how they store information. dasBlog stores content in flat XML files while .Text stores content in a database (either SQL Server or MSDE). This can be attractive for users who don't have access to a database through their ISP. However, the .Text data model allows you to host multiple blogs from a single installation (see http://blogs.msdn.com for an example). dasBlog is definitely easier to install and configure but .Text has plenty of documentation and community support.

Another popular approach is to author your weblog on your desktop and then publish the HTML files to a Web site (similar to the Blogger.com approach). Radio Userland is by far the most popular tool in this camp. One of its biggest advantages is that you can blog while offline; all of your weblog content is stored on your local hard drive. You can publish your weblog to a different server whenever you see fit and if something happens to your server, you're still OK. Furthermore, since Radio Userland doubles as an RSS aggregator, you get a weblog reader that lets you post comments on your own.

There are plenty of RSS aggregators available today that make reading weblogs a lot like reading e-mail. Bloglines.com and NewsIsFree.com provide online aggregation services that aggregate your feeds into a single Web page that you can access from anywhere. Of course, you have to be online to get to it. Desktop aggregators download feeds locally for offline reading, but then you have to be on the computer where the aggregator is installed to read them. Some of the most popular no-cost desktop aggregators include SharpReader, RSS Bandit, and Syndirella, which provide similar features including multiple RSS version support, advanced search capabilities, OPML import/export, and custom groupings.

In addition to Radio Userland, one of the most popular commercial desktop aggregators is NewsGator (shown in the Figure), which plugs directly into Outlook 2000 or later. Newsgator also provides online services that synchronize with your Outlook subscriptions in order to give you the best of both worlds.

For more info on blogging technologies and standards, take a look at the XML Files in the April 2004 issue.