 MSDN Magazine July 2005

MSDN Magazine July 2005

XML Comments: Document Your Code in No Time At All with Macros in Visual Studio

Starting in Visual Studio 2005, XML documentation support will become available in Visual Basic and C++ as it has been for C#. However, making full use of the many comment tags will take a little extra work In this article, the author shows how Visual Studio macros can be used to automate the creation of XML documentation and creates macros that write large sections of comments with just a keystroke.

Tony Chow Smart Tags: Simplify UI Development with Custom Designer Actions in Visual Studio

Windows Forms 2.0 components expose smart tags by utilizing Designer Actions. By taking advantage of this feature, you can add smart tags to your own custom components to make setting their properties and other actions much easier and developer-friendly.

Michael Weinhardt Reflection: Dodge Common Performance Pitfalls to Craft Speedy Applications

With reflection in .NET, you can load types, understand their members, make decisions about them, and execute, all within the safety of the managed runtime. But to use this power wisely, it's important to understand the associated costs and pitfalls to keep performance impact at a minimum. This article explains how.

Joel Pobar Visual Basic: Simplify Common Tasks by Customizing the My Namespace

Underlying the My namespace's APIs is a fully extensible architecture you can leverage to customize the behavior of My and to add new services to its hierarchy to adapt to specific application needs. the My namespace is optimized to provide simple solutions to common problems. But if your requirements are different, you can use the My namespace's customization model to modify the behavior of individual My namespace members to meet specific application requirements.

Joe Binder Behind the Scenes: Discover the Design Patterns You're Already Using in the .NET Framework

In this article, the author presents a basic overview of several common design patterns and how they are used in the .NET Base Class Library and other areas of the .NET Framework. You'll discover some of the motivation for why the Framework is designed the way it is, and come away with a better understanding of the abstract concepts of the patterns themselves.

Rob Pierry Debugging: Root Out Elusive Production Bugs with These Effective Techniques

Errors happen. But in production, error logs often provide little or no help in pinpointing the exact line of code in which the problem originates. You can't usually isolate the root cause by modifying code in a production environment because recreating the exact same environment is very difficult due to the large number of potential software and hardware configurations. This article provides some concrete steps you can take to get at the heart of the problems in your production code.

Matt Adamson Editor's Note: Designing the PC of the Future

While spending some idle moments poking around on the Web between working on articles the other day, we ran across a design challenge that Microsoft is co-sponsoring. Along with the IDSA (Industrial Designers Society of America), Microsoft is inviting all you designers out there to draw up plans for the next-generation PC.

New Stuff: Resources for Your Developer Toolbox

r.a.d.controls by telerik gives you an impressive set of great looking interface components, Visual Studio® .NET design-time support, cross-browser compatibility, keyboard support, and data binding.

Marnie Hutcheson Web Q&A: Web Page Layout, Quirks Mode, and More

Edited by Nancy Michell Advanced Basics: Creating A Breadcrumb Control

Hansel and Gretel had the right idea when "they followed the pebbles that glistened there like newly minted coins, showing them the way. " The deeper you get into the forest or into your data, the more likely you are going to need help to find your way back out again.

Duncan Mackenzie Cutting Edge: DHTML-Enabled ASP.NET Controls

In the past, I've covered some core aspects of the interaction between DHTML behaviors, the browser, and ASP. NET runtime (see Cutting Edge: Extend the ASP. NET DataGrid with Client-side Behaviors and Cutting Edge: Moving DataGrid Rows Up and Down).

Dino Esposito Data Points: The Enterprise Library Data Access Application Block, Part 1

A solid data access later (DAL) can benefit an application by hiding redundant tasks, helping handle exceptions more gracefully, helping clean up resources more efficiently, and providing a layer of abstraction from the database.

John Papa Bugslayer: Unhandled Exceptions and Tracing in the .NET Framework 2.0

By now, you've certainly heard about the big changes coming in Visual Studio® 2005, but when it's time to move your code over it will be the small things that trip you up. In this column, I want to cover two of the many excellent changes that you could easily overlook as you make the move to the new runtime and development tools.

John Robbins .NET Matters: StringStream, Methods with Timeouts

Stephen Toub C++ at Work: Enum Declarations, Template Function Specialization

Paul DiLascia { End Bracket }: Experimenting with F#

After five years of guiding generics for the Microsoft® . NET Framework into Visual Studio® 2005, I've turned my attention to attempting to achieve a synthesis between type-safe, scalable, math-oriented scripting and programming for .

Don Syme


XML Comments: Document Your Code in No Time At All with Macros in Visual Studio

XML Comments

Document Your Code in No Time At All with Macros in Visual Studio

Tony Chow
Parts of this article are based on a prerelease version of Visual Studio 2005. Those sections are subject to change.
This article discusses:
  • Macros and the Visual Studio extensibility model
  • How to document a constructor automatically
  • Documenting exceptions thrown from methods and properties automatically
  • How to automatically copy XML documentation into a descendent class from its base type
This article uses the following technologies:
Visual Basic, C#, Visual Studio
Code download available at: XMLComments.exe (119KB)
Contents
Documenting Constructors
Getting Started
Getting at the Constructor
Generating the XML Comment Block
Injecting the XML Comment Block
What About the Parameters?
Documenting Exceptions
Obtaining the Source Code
Newly Thrown Exceptions
Rethrown Exceptions
From Theory to Practice
Emulating Documentation "Inheritance"
Walking the Inheritance Tree
Conclusion

O
ne of the most widely acclaimed features of the C# programming language is XML-based code annotation. Starting in Visual Studio® 2005, XML documentation support will also become available in Visual Basic® and C++. Yet in everyday practice, making full use of the many comment tags can be a tedious chore, and as a result, most managed code in the field remains woefully under-documented.

In this article, I will demonstrate how Visual Studio macros can be used to automate the creation of XML documentation. I'll create macros that write large sections of comments with just a keystroke. At the end of the day, nothing can completely automate code commenting, but a few well-placed macros will make your job substantially easier, eliminating much boilerplate work and allowing you to concentrate only on substantive comments that must be written by hand. And, you can create commenting macros for specific tasks based on the techniques I describe here.

Documenting Constructors

Let's begin with a relatively simple example that will introduce the essential features of Visual Studio extensibility. When writing XML documentation, the summary statements for constructors are among the most formulaic. Microsoft documentation typically observes the following format:

Initializes an instance of the <see cref="TypeName"/> class.
Any additional information is simply appended to the end of the sentence. The routine nature of constructor summaries makes them a prime candidate for automation, as well as the objective of the first macro I'll discuss here. When invoked, the macro will determine whether the current editing caret in the active code file is positioned within a class constructor. If it is, the macro will automatically create the summary statement and apply it to the constructor method.

With Visual Studio 2005 looming on the horizon, I'll make sure that this macro works on all versions of C# as well as in Visual Basic 2005. At the time of writing, Visual C++® 2005 does not support Visual Studio extensibility as well as the other languages do, so I won't be covering it here.

Getting Started

To start authoring macros, you first need to fire up the macros IDE. This is done from the main Visual Studio window with the menu item Tools | Macros | Macros IDE. The resulting window, shown in Figure 1, looks very much like Visual Studio itself, except that it shows a special macros project called MyMacros.

Figure 1 The Macros IDE
Figure 1 The Macros IDE

All macros in Visual Studio .NET are written in Visual Basic .NET. Within the macros IDE, they are organized into Visual Basic modules. Each public parameterless Sub method in a public module is recognized as a macro by Visual Studio. In the "MyMacros" project you may find an existing module named RecordingModule. This is the module into which recorded macros are stored.

As a container for all the macros and helper routines I'll be writing, I'll create a new module and name it XmlCommentMacros. If you look inside the module file, you should see that the EnvDTE namespace is imported by default. This is the COM automation library for the Visual Studio extensibility model, and you'll need it. For the needs of the macros, you should import the following additional namespaces: System, System.Xml, System.IO, System.Collections, System.Text, and System.Text.RegularExpression. In order to support the System.Xml namespace, you will also need to add a reference to the System.Xml assembly to the macros project. You'll then create a public Sub method named DocumentConstructor. So far, the module should look like the following:

Imports EnvDTE
Imports System
Imports System.IO
Imports System.Collections
Imports System.Xml
Imports System.Text
Imports System.Text.RegularExpressions

Public Module XmlCommentMacros
    Public Sub DocumentConstructor()
    End Sub
End Module

At this point, if you return to the main Visual Studio window and open the Macros Explorer tool window, you should see the DocumentConstructor macro displayed nested within the XmlCommentMacros module node. You have created the skeleton of your constructor-documenting macro and can now proceed to fill it with logic.

Getting at the Constructor

When the macro is executed, the first order of business is to determine whether or not the editing caret in the active code file currently resides in a constructor. The position of the editing caret is discovered with the following line of code:

Dim caretPosition As TextPoint = DTE.ActiveDocument.Selection.ActivePoint
So far, things are pretty self-explanatory. The DTE global object is the entry point into the Visual Studio extensibility model. Within it, the DTE.ActiveDocument object represents the currently active code file in the Visual Studio IDE. Using the Selection property of the ActiveDocument object, you are then able to retrieve the position of the editing caret as a TextPoint object.

The TextPoint object is your ticket to the most powerful feature of Visual Studio extensibility, namely, CodeModel. CodeModel maintains a structured, language-agnostic view of all code within a project, organizing code elements such as classes, methods, properties, and namespaces into interlinked trees of CodeElement objects that are updated in real-time as you modify your code. All code elements implement the CodeElement interface, which provides generic information such as the name of the element. In addition, each object also implements at least one interface that exposes functionalities specific to its type. For example, code elements that represent classes implement the CodeClass interface, property elements implement the CodeProperty interface, and, in turn, namespace elements implement the CodeNameSpace interface. Figure 2 illustrates CodeModel's most important interfaces and its relationship to the other components of the extensibility model.

Figure 2 General Structure of CodeModel (Close Figure 2)

With the TextPoint object, access to CodeModel is provided by the CodeElement property. This read-only property accepts a single enumerated parameter of type vsCMElement, identifying the type of code element that you are interested in retrieving. When invoked, the CodeElement property examines the code file from the current point upwards and returns the first CodeElement object, if any, that is of the specified type and which encapsulates the TextPoint. In this case, you need to obtain a reference to an object constructor, and since object constructors are identified as "functions" by CodeModel, you should attempt to retrieve a code element of type vsCMElementFunction:

Dim element As CodeElement = _
    caretPosition.CodeElement(vsCMElement.vsCMElementFunction)

A call to TextPoint.CodeElement returns a null reference if no matching code element is found. If the "element" variable turns out to be a non-null value, you will know that the editing caret does indeed reside in a method declaration. What you don't know yet is whether this method is a constructor. This bit of information is easily obtainable by casting the variable into a CodeFunction interface, and examining its FunctionKind property. If the "element" variable is null, or if the object method it represents does not happen to be a constructor, the macro should inform the user and then gracefully exit:

Dim func As CodeFunction = element
If func is Nothing OrElse func.FunctionKind <> _
        vsCMFunction.vsCMFunctionConstructor Then
    MsgBox("Constructor not found.")
    Exit Sub
End If

Generating the XML Comment Block

If the macro has not terminated itself by now, you have successfully acquired a CodeFunction object representing a constructor method, for which you are now poised to generate the XML documentation. Since there are no better tools at dealing with XML than those built into the Microsoft® .NET Framework, you'll be editing the XML comment block as a System.Xml.XmlDocument object. Ideally, the macro should also be able to detect the presence of an existing summary statement, and get confirmation from the user before overwriting its contents.

In this, you are greatly aided by the availability of the DocComment property on most code element interfaces, including CodeFunction. This string property sets or returns the XML comment block associated with the code element. Since later you'll need to extract existing comment tags from the code and place them in an XmlDocument, you encapsulate the logic in a private helper method named GetCommentXml, shown in Figure 3. From the main macro method, GetCommentXml is invoked and its return value placed in a variable named doc:

Dim doc As XmlDocument = GetCommentXml(func)

Figure 3 The GetCommentXml Helper (Close Figure 3)
Private Function GetCommentXml(ByVal element As CodeElement) _
        As XmlDocument

    Dim doc As XmlDocument = New XmlDocument()

    Dim xmlStr As String = element.DocComment
    If Not xmlStr Is Nothing AndAlso xmlStr.Trim() <> String.Empty Then
        If xmlStr.StartsWith("<doc>") = False Then
            ' Ensure single root node
            xmlStr = "<doc>" & xmlStr + "</doc>"
        End If
        xmlStr = Regex.Replace(xmlStr, "^(\s*<doc>\s*){2,}", "<doc>")
        xmlStr = Regex.Replace(xmlStr, "(\s*<\/doc>\s*){2,}$", "</doc>")
        doc.LoadXml(xmlStr)
    Else
        Dim rootTag = doc.CreateElement("doc")
        doc.AppendChild(rootTag)
    End If

    Return doc
End Function

Let's examine what just occurred in GetCommentXml. In a nutshell, the method creates an XmlDocument, examines the DocComment property on the code element, and populates the former with the existing XML comment block. If existing comments are not found, a single root element named "doc" is created and appended to the XmlDocument object. No surprise here—a well-formed XML document must have a single root node.

You will notice, however, that some minor chicanery was required when dealing with the output of DocComment. This is necessitated by an unfortunate inconsistency between the way C# implements DocComment and the way Visual Basic does it in Visual Studio 2005, at least in Beta 2. Whereas the C# implementation returns the XML comment block already in a well-formed state (with a root <doc> node), Visual Basic simply gives you the naked comment block without the root node. This makes it compulsory in the latter case to manually wrap the return value of DocComment between <doc> tags, before you can pass it into the XmlDocument.LoadXml method. Also in Beta 2, in certain circumstances DocComment may mistakenly return a comment block with not one, but two <doc> tags, one nested inside the other. This necessitates a second workaround in GetCommentXml, where I use a regular expression to catch repeat occurrences of "<doc>" and "</doc>" and replace them with a single instance.

Having thus obtained the comment block as an XmlDocument object, you must now retrieve the <summary> tag from within the root node or, if one does not exist, create a new one. Should the <summary> tag already exist and have contents, the user must be warned about the impending overwrite before proceeding. You therefore continue the macro, using an XPath query to determine the existence of a <summary> tag:

Dim summaryTag As XmlElement = doc.SelectSingleNode("/doc/summary")
If summaryTag Is Nothing Then
    ' Create <summary> tag
    summaryTag = doc.CreateElement("summary")
    doc.FirstChild.PrependChild(summaryTag)
ElseIf summaryTag.InnerXml <> String.Empty Then
    ' Overwrite confirmation
    If MsgBox("Do you wish to overwrite the contents of the summary tag?", _
            MsgBoxStyle.YesNo) = MsgBoxResult.No Then
        Exit Sub
    End If
End If

With the XmlElement object representing the <summary> tag in hand, you can now create the summary statement for the constructor. Follow the established Microsoft format, with a cross reference to the class or structure served by the constructor:

<para>Initializes an instance of the <see cref="TypeName"/>
class/structure.</para>

Determining the name of the parent type in CodeModel is as simple as accessing the Parent property of the CodeFunction object. As a nice touch, the summary should distinguish between a "class" and a "structure" when referring to the parent type. The following code wraps up the summary generation process:

Dim typeName As String = func.Parent.Name
Dim typeCategory As String = _
    IIf(func.Parent.Kind = vsCMElement.vsCMElementStruct, _
    "structure", "class")

Dim summary As String = String.Format( _
    "<para>Initializes an instance of the <see cref=""{0}""/> {1}.
     </para>", typeName, typeCategory)

summaryTag.InnerXml = summary

Injecting the XML Comment Block

Now that you've created the summary, all that's left to do is to inject the XML comment block back into the DocComment property. Naturally you'll need the output to be properly indented, and the easiest way to make it so is to employ the System.Xml.XmlTextWriter class. Like GetCommentXml, this is an operation that will be needed by all of your macros. Create a new helper method named SetCommentXml, as shown in Figure 4. This helper takes two arguments: a CodeElement and an XmlDocument. It first writes the contents of the XmlDocument into a properly indented string, and then assigns it to the code element's DocComment property. As in GetCommentXml, SetCommentXml needs to accommodate the different behaviors of C# 2.0 and Visual Basic 2005, in that the former demands a <doc> root node when setting the DocComment property, and the latter does not. This forces the code to vary the node at which XmlTextWriter starts writing, depending on the current language, which is determined by examining the CodeElement.Language property. Within the main method, an invocation to SetCommentXml completes the macro:

SetCommentXml(func, doc)

Figure 4 SetCommentXml Helper (Close Figure 4)
Private Function SetCommentXml(ByVal element As CodeElement, _
    ByVal doc As XmlDocument)
  Dim sb As StringBuilder = New StringBuilder()
  Dim sw As StringWriter = New StringWriter(sb)
  Dim xw As XmlTextWriter = New XmlTextWriter(sw)

  Try
    xw.Indentation = 1
    xw.IndentChar = Char.Parse(vbTab)
    xw.Formatting = Formatting.Indented

    Dim rootNode As XmlNode
    If element.Language = CodeModelLanguageConstants.vsCMLanguageVB Then
      rootNode = doc.FirstChild
    Else ' CSharp
      rootNode = doc
    End If

    rootNode.WriteContentTo(xw)
    element.DocComment = sb.ToString()
  Finally
    xw.Close()
    sw.Close()
  End Try
End Function

From the Visual Studio IDE, running the DocumentConstructor macro with the editing caret positioned within a constructor method written in C# or Visual Basic 2005 produces the respective outcomes shown in Figure 5. For even greater convenience, you can map the macro to a keystroke in the Visual Studio Options dialog. Documenting a constructor is now as convenient as hitting a key.

Figure 5 Running DocumentConstructor Macro (Close Figure 5)

C#
public class Foo
{
    /// <summary>
    ///    <para>Initializes an instance of the <see cref="Foo"/> class.
    /// </summary>
    public Foo()
    {
    }
}
Visual Basic 2005
Public Class Foo
    ''' <summary>
    '''    <para>Initializes an instance of the <see cref="Foo"/> class.
    ''' </summary>
    Public Sub New()
    End Sub
End Class

What About the Parameters?

Alert readers might protest at this point, and rightly so, that the macro doesn't take into account parameterized constructors. Indeed, in both C# and Visual Basic 2005, as soon as you type the three slashes or apostrophes over a code element, IntelliSense® creates an XML comment skeleton for you, complete with tags for all parameters. Your macro here, in contrast, is blissfully unaware of any parameters. In this regard, it's less functional than the IDE's default behaviors. Fortunately, these functionalities are easily duplicated by a simple helper method. In addition to framing the <param> tags, this helper will also add other essential tags depending on the type of code element you pass to it, making it capable of generating comment skeletons on any code element, and not just constructors. Let's call it EnsureBasicTags. Figure 6 provides the complete code for the helper.

Figure 6 EnsureBasicTags Helper (Close Figure 6)
Private Sub EnsureBasicTags(ByVal element As CodeElement, _
        ByVal doc As XmlDocument)

    ' <summary>
    If doc.SelectSingleNode("/doc/summary") Is Nothing Then
        Dim summaryTag As XmlElement = doc.CreateElement("summary")
        summaryTag.AppendChild(doc.CreateTextNode(""))
        doc.FirstChild.PrependChild(summaryTag)
    End If

    ' <param>
    If element.Kind = vsCMElement.vsCMElementFunction Then
        For Each param As CodeParameter in element.Parameters
            Dim xpath As String = _
                String.Format("/doc/param[@name='{0}']", param.Name)
            If doc.SelectSingleNode(xpath) Is Nothing Then
                Dim paramTag As XmlElement = doc.CreateElement("param")
                paramTag.SetAttribute("name", param.Name)
                paramTag.AppendChild(doc.CreateTextNode(""))
                doc.FirstChild.AppendChild(paramTag)
            End If
        Next
    End If

    ' <returns>
    If element.Kind = vsCMElement.vsCMElementFunction _
            AndAlso element.Type.TypeKind <> vsCMTypeRef.vsCMTypeRefVoid _
            AndAlso doc.SelectSingleNode("/doc/returns") Is Nothing Then
        Dim returnsTag As XmlElement = doc.CreateElement("returns")
        returnsTag.AppendChild(doc.CreateTextNode(""))
        doc.FirstChild.AppendChild(returnsTag)
    End If

    ' <value>
    If element.Kind = vsCMElement.vsCMElementProperty _
            AndAlso doc.SelectSingleNode("/doc/value") Is Nothing Then
        Dim valueTag As XmlElement = doc.CreateElement("value")
        valueTag.AppendChild(doc.CreateTextNode(""))
        doc.FirstChild.AppendChild(valueTag)
    End If
End Sub

The EnsureBasicTags helper method accepts two parameters: the CodeElement to document, and the XmlDocument containing the XML comment block being worked on. It creates the <summary> tag in all cases, <param> tags if the code element is a method, the <returns> tag for methods with a return value, and the <value> tag for properties. When creating the <param> tags, the method parameters are discovered by enumerating the Parameters collection of the CodeFunction interface. Using XPath queries, you'll also ensure that each tag is only added if it's not already present. All along the way, you have to append an empty TextNode object to every tag that is created in order to prevent XmlTextWriter from omitting the closing tag when rendering an XML element that has no contents.

Modify the DocumentConstructor method to invoke EnsureBasicTags right above the SetCommentXml call, and the macro is truly complete, as shown here:

EnsureBasicTags(func, doc)
In addition to being functional, this macro demonstrates many core concepts in Visual Studio extensibility. Armed with these insights, you can now venture into deeper waters.

Documenting Exceptions

Thorough documentation of all thrown exceptions can greatly enhance the usability of any class library. It lets users anticipate anomalous conditions and apply appropriate mitigative measures, without going through the pain of trial and error. In the .NET Framework, exception documentation is also the standard means of conveying acceptable values for method arguments. To encourage documenting exceptions, XML documentation formalizes the concept in the form of the <exception> tag. In spite of this, the <exception> tag remains one of the most underused in real-world coding. Part of the reason is the iron discipline required; it's a lot of work typing up a tag for each and every exception and describing the conditions that may cause it to be thrown.

Things are about to be made easier, however, by the second macro. When completed, this macro will examine a method or property and produce an <exception> skeleton for each type of exception explicitly thrown in the code. For the most commonly used exceptions, it will even write entire lines of comments, saving your time and sanity. Let's see how this is accomplished.

Like the first macro, the new macro will begin by obtaining the CodeElement object over which the editing caret is positioned. The specific requirements are a bit different, however. In the first macro, the only code elements of interest were methods. But since you are documenting exceptions, and exceptions can be thrown in methods and properties alike, you will now be interested in obtaining a CodeElement that represents either a method or a property. This service is best provided by another helper method. This helper must accept a parameter array of vsCMElement values, and call TextPoint.CodeElement repeatedly in a loop, looking for a code element that matches any of the specified types. Let's call the helper GetCurrentCodeElement, and define it as shown in Figure 7. As before, the helper has to work around the bug in the Visual Studio 2005 Beta by wrapping each call to TextPoint.CodeElement in a try/catch construct.

Figure 7 GetCurrentCodeElement Helper (Close Figure 7)
Private Function GetCurrentCodeElement( _
        ByVal ParamArray kinds() As vsCMElement) As CodeElement
    Dim caretPosition = DTE.ActiveDocument.Selection.ActivePoint

    For Each kind As vsCMElement in kinds
        Dim element As CodeElement = caretPosition.CodeElement(kind)
        If Not element Is Nothing Then
            Return element
        End If
    Next

    Return Nothing ' No matching element found.
End Function

With this hurdle out of the way, you can begin writing the macro. In the XmlCommentMacros module, declare a new macro method named DocumentExceptions. The first line of this macro will invoke the GetCurrentCodeElement helper. If the helper does not find a method or property at the editing caret, the macro shall inform the user and exit. Otherwise, you will retrieve the XmlDocument for the CodeElement object by calling the GetCommentXml helper that was created earlier, followed by a call to EnsureBasicTags, so that in addition to documenting exceptions, you also create a skeleton of basic XML comment tags. The opening lines of the DocumentExceptions macro are shown in the code in Figure 8.

Figure 8 DocumentExceptions Excerpt (Close Figure 8)
Public Sub DocumentExceptions()
    Dim element As CodeElement = GetCurrentCodeElement( _
        vsCMElement.vsCMElementFunction, _
        vsCMElement.vsCMElementProperty)

    If element Is Nothing Then
        MsgBox("This macro is only applicable to methods and properties.")
        Exit Sub
    End If

    Dim doc As XmlDocument = GetCommentXml(element)

    EnsureBasicTags(element, doc)
End Sub

Obtaining the Source Code

In the first macro, you relied solely on the language-agnostic CodeModel objects to provide the necessary information about a constructor and the class it serves. But CodeModel only goes as deep as member declarations; it doesn't analyze what goes on within procedural code. This means that in your effort to discover thrown exceptions, you have to somehow obtain the actual source code and parse the code in a language-specific fashion.

This isn't as daunting a task as it may sound. With CodeModel, it's quite easy to access the source code specific to a code element. On each CodeElement object there are two properties, StartPoint and EndPoint, denoting the points in the code file where the code element starts and ends, respectively. Calling the CreateEditPoint method on the StartPoint object yields an EditPoint object on which the GetText method can then be invoked (with the EndPoint object as a parameter) to obtain the complete text of the source code. The following code appended to the macro does just that and stores the output in a variable:

Dim editPoint As EditPoint = element.StartPoint.CreateEditPoint()
Dim sourceCode As String = editPoint.GetText(element.EndPoint)

When executed on C#, the sourceCode variable will contain the source code from the declaration line of the code element, all the way to its closing bracket. For Visual Basic 2005, it holds the code from the member declaration to the "End Function/Sub/Property" statement of the same member.

Source code in hand, you are now faced with the more challenging part of the process: parsing the code. This is something that a few well-thought-out regular expressions can easily take care of, however. Let's consider the various ways exceptions can be thrown in code, and design a regular expression to match each scenario. Note that you're only looking for exceptions explicitly thrown in the code body itself; you won't look for exceptions that might be thrown from invocations within the method or property.

Newly Thrown Exceptions

When an exception is thrown in your code, it is usually done by instantiating an exception object and passing it to the "throw" keyword, all in a single statement. The syntax is very similar in C# and Visual Basic. Of the newly thrown exceptions, the most commonly used are those derived from the ArgumentException base class, including ArgumentNullException, ArgumentOutOfRangeException, and ArgumentException itself. These also happen to be the exceptions for which the macro can generate entire comments.

With these exception classes, the constructor invocation usually accepts an argument that specifies the name of the faulty parameter that has caused the exception to be thrown. The name of the exception, together with the affected parameter name, are all that you need to formulate reasonably informative comments. Take, for instance, the following C# method, which accepts four param-eters and which throws an ArgumentNullException for two of its parameters, an ArgumentOutOfRangeException for the third parameter, and an ArgumentException for the fourth:

public void Bar(string arg1, string arg2, int index, object arg4)
{
    if (arg1 == null) throw new ArgumentNullException("arg1");
    if (arg2 == null) throw new ArgumentNullException("arg2");
    if (index < 0) throw new ArgumentOutOfRangeException("index");
    if (arg4 == this) throw new ArgumentException("Invalid.", "arg4");
}

If the macro could identify the exceptions thrown in this code and extract the name of each exception class along with the name of the affected parameter in each case, it wouldn't take much effort to generate the XML documentation shown in Figure 9.

Figure 9 DocumentExceptions Sample Output (Close Figure 9)
<exception cref="ArgumentNullException">
    <para>The argument <paramref name="arg1"/> is <langword
          name="null"/>.</para>
    <para>-or-</para>
    <para>The argument <paramref name="arg2"/> is <langword
          name="null"/>.</para>
</exception>
<exception cref="ArgumentOutOfRangeException">
    <para>The argument <paramref name="index"/> is out of range.</para>
</exception>
<exception cref="ArgumentException">
    <para>The argument <paramref name="arg4"/> is invalid.</para>
</exception>

Notice that, in keeping with the Microsoft practice, only one <exception> tag is created for each type of exception thrown. If an exception appears more than once, an "-or-" paragraph is used to delimit the various conditions under which it can be incurred. Admittedly, the comment that was generated for ArgumentException is less informative than the other comments, but it would still save a lot of typing by providing a template to which more specific information can be added.

Instances of the ArgumentNullException and ArgumentOutOfRangeException, along with the values of their paramName arguments, can be identified and captured with a single regular expression that works with code written in either C# or Visual Basic, as you see in the following:

(?i:\bthrow\s+new\s+(?<exception>Argument(Null|OutOfRange)Exception)\s*\
(\s*"(?<param>\w+)")
This expression extracts the exception information that follows each "throw new" phrase, while ignoring whitespace. The "exception" group captures the exact name of the exception, while the "param" group captures the name of the affected parameter. Visual Basic support is achieved with a (?i:) grouping construct that makes the whole expression case-insensitive.

Capturing instances of ArgumentException demands a separate regular expression, for, unlike the other two exception classes, the constructor of ArgumentException accepts the name of the affected parameter as its second, rather than first argument. Once again, this regular expression applies to either C# or Visual Basic:

(?i:\bthrow\s+new\s+(?<exception>ArgumentException)\s*\
(\s*".*?",\s*"(?<param>\w+)")

Here, the first constructor argument is ignored, leaving the second argument to be captured as the name of the affected parameter. It is worth pointing out that the liberal use of "\s*" and "\s+" patterns in both of these expressions makes them highly tolerant of extra spaces that may appear in the source code by accident or design. This is a good practice that you should always follow when creating regular expressions.

For other types of exceptions, it's more difficult to derive sufficient information from the source code to generate complete comments. Nonetheless, it would still be tremendously labor-saving to emit <exception> skeletons in the XML comment block, one for each exception type. To facilitate this, I use another regular expression that captures only the name of the exception class:

(?i:\bthrow\s+new\s+(?<exception>[\w\.]*Exception\b))

As before, by virtue of being case-insensitive, this regular expression works with C# and Visual Basic alike. For the time being, you need not worry about overlapping the exceptions already captured by the other two regular expressions.

Rethrown Exceptions

Aside from newly instantiated exceptions, the macro also needs to handle exceptions that are captured by a try/catch clause in either C# or Visual Basic and then rethrown. In this usage, C# and Visual Basic are sufficiently different to warrant a separate regular expression for each. For C#, the expression to recognize and capture a rethrown exception can be written as follows:

(?s:(?<indent>\n\s*)catch\s*\(\s*(?<exception>[\w\.]+)\b.*?\).*?\bthrow\s*
(\s+\w+)?;.*?\k<indent>\})
This regular expression is more complex than the ones you've seen thus far and merits some explanation.

Since the pattern spans multiple lines, you need to switch on the single-line mode by encapsulating the entire expression in a (?s:) grouping construct, so that the new line character "\n" is matched by the "." and "\s" character classes. In essence, the responsibility of this expression is to match any block of text that begins with a "catch" keyword, contains a "throw" keyword, and ends with the first occurrence of a closing bracket that has the same level of indentation as the aforementioned "catch". The matching of indentation levels is accomplished by first capturing the indenting spaces ahead of the "catch" keyword in the "indent" group, and then back-referencing the captured characters with a "\k" clause ahead of a closing bracket. Along the way, the name of the exception is captured by the "exception" group.

The Visual Basic version of this regular expression follows much of the same principles, with two distinct characteristics of the language necessitating modifications. Instead of a closing bracket, you have to match an "End Catch" phrase. Also, in the Visual Basic version of the try/catch clause, the variable to store the exception object is not optional. Of course, Visual Basic is also case-insensitive. Taking these various factors into account, you end up with the following expression:

(?si:(?<indent>\n\s*)Catch\s+\w+\s+As\s+(?<exception>[\w\.]+).*?\bthrow\s*
(\s+\w+)?(?=\s*\n.*?)\k<indent>End\s+Try)

From Theory to Practice

These, then, are all the regular expressions that you need to perform basic parsing of the source code. The remaining code of the DocumentExceptions macro is available in the code download. You first gather all the regular expressions needed for the current language into an ArrayList collection. Then you enumerate this collection, running each expression over the source code of the method or property being documented. For each type of exception found, your macro uses an XPath query to ascertain whether an <exception> tag already exists for it. If none exists, a new tag is created, ensuring that at the very least an XML skeleton is generated for each exception thrown. The XPath query also makes sure that each exception type is never documented twice, even if it's caught by more than one regular expression.

If a "param" group is found within a regular expression match, there's more work to be done because it signals that you've encountered an argument exception for which a comment can be automatically generated. To prevent the same parameter name from being documented more than once for each exception type, the code should issue another XPath query to determine whether a reference to the parameter, in the form of a <paramref> tag, already exists within the <exception> tag. If it does not, proceed to construct the comment pertaining to the parameter. The finished comment is appended to the <exception> tag's InnerXml property, preceded by an "-or-" paragraph if needed, in order to delimit it from other comments.

After all of the regular expressions have been executed over the source code, the macro should end by calling the SetCommentXml helper method to emit the finished XML comment block back into the code file.

Run the DocumentExceptions macro over a method or property written in either C# or Visual Basic 2005, and the resulting XML documentation should account for most exceptions explicitly thrown in the code. For programmers already steeped in sound exception practices and who aspire to develop the documentation skills to match, this macro is certain to save a tremendous amount of time and labor. Comprehensive exception documentation has suddenly become a more viable proposition.

Emulating Documentation "Inheritance"

For the third and final exercise, you'll muster everything you've learned so far to tackle one of the greatest stumbling blocks in achieving comprehensive documentation coverage of your code—the absence of a "documentation inheritance" feature in Visual Studio. If you've dealt with XML documentation for any length, you are more than likely irritated by the fact that when overriding a method or property in a child class, or implementing an interface member, the XML documentation from the base type is not carried over. This is frustrating, because it forces you to either document the member from scratch, or hand-copy comments from the base type. While you may not necessarily want an overridden member to be documented with exactly the same text as in the base type, it's at least handy to have the base XML documentation as a template to work from.

Here as before, the rich capabilities of CodeModel make it almost trivial to provide this feature, the goal of the third macro. When this macro is invoked over a method or property, it will walk the inheritance hierarchy, find the original XML comments for that member from the base type, and copy it into your code.

What enables you to do all this is the fact that CodeModel keeps track of the inheritance relationships of all types declared or referenced in the project. This information is exposed through two properties available on both the CodeClass interface and the CodeStruct interface: the Bases property and ImplementedInterfaces property. The Bases property is a collection of all direct parent classes and never has more than one item in it, since .NET only permits single implementation inheritance. The ImplementedInterfaces property is a collection of all interfaces directly implemented by the current type. By enumerating these two collections in a recursive fashion, you can discover all of the base types associated with the current class or structure, identify the base implementation of the property or method being documented, and extract the documentation from it.

With this in mind, let's begin writing the macro, which you'll call InheritDocumentation. The macro starts much as the others did, with an invocation to the GetCurrentCodeElement helper to obtain either a method or property code element at the editing caret. As before, if a method or property cannot be found, the user is warned and the macro exits. At this point, the macro should look like the following:

Public Sub InheritDocumentation()
    Dim element As CodeElement = GetCurrentCodeElement( _
        vsCMElement.vsCMElementFunction, vsCMElement.vsCMElementProperty)

    If element Is Nothing Then
        MsgBox("This macro is only applicable to methods, and properties.")
        Exit Sub
    End If
End Sub

Even if a method or property is successfully obtained, some additional filtering can be performed to ensure that you don't waste time on an ineligible code element. A private member, for instance, cannot possibly derive from a base type. Likewise, a static member is not applicable for documentation inheritance either. The accessibility of the member can be determined by examining the Access property, which is exposed by both the CodeFunction and CodeProperty interfaces.

Determining whether a member is static is a bit more difficult. To be sure, there's an IsShared property on the CodeFunction interface, but the same cannot be said of the CodeProperty interface. This seeming inconsistency exists because the code model reflects the fact that a property is actually an amalgamation of two accessor methods. In turn, the accessor methods are available as CodeFunction objects through the CodeProperty.Getter and CodeProperty.Setter properties, and it is by examining the value of the IsShared property on either accessor that you can tell whether a property is static. Instead of writing a contorted "If" statement, it is better to create a simple helper function that takes either a method element or a property element as the argument and returns a simple Boolean value indicating whether the member is static. The code for the IsMemberStatic helper is as follows:

Private Function IsMemberStatic(ByVal member As CodeElement) As Boolean
    If member.Kind = vsCMElement.vsCMElementFunction Then
        Return member.IsShared
    ElseIf Not member.Getter Is Nothing Then
        Return member.Getter.IsShared
    ElseIf Not member.Setter Is Nothing Then
        Return member.Setter.IsShared
    Else
        Return False
    EndIf
End Function

If, with the help of IsMemberStatic, the macro finds that the current member is either private or static, it should inform the user and then gracefully exit, like so:

If element.Access = vsCMAccess.vsCMAccessPrivate _
        OrElse IsMemberStatic(element) Then
    MsgBox("Not applicable to static or private members.")
    Exit Sub
End If
Of course, the macro will also need to warn the user if any comments exist on the member since they will be overwritten with the base documentation. The easiest way to tell whether there are existing comments is to make a call to the GetCommentXml helper method and see if the root node of the returned XmlDocument contains any child nodes:
If GetCommentXml(element).FirstChild.ChildNodes.Count > 0 Then
    If MsgBox("Overwrite the current documentation " _
            + "with inherited documentation?",
            MsgBoxStyle.YesNo) = MsgBoxResult.No Then
        Exit Sub
    End If
End If

Walking the Inheritance Tree

After taking all possible precautions and obtaining the CodeElement object representing the current method or property, it is time to walk up the inheritance tree of the class or structure and look for the base member that it implements or overrides. This service is best provided by a helper method which I'll name FindBaseMember, shown in its entirety in Figure 10. The FindBaseMember helper is meant to be invoked recursively and takes two arguments: a CodeElement representing the method or property being documented, and another CodeElement representing the class, structure, or interface that is currently being examined.

Figure 10 The FindBaseMember Helper Method (Close Figure 10)
Private Function FindBaseMember(ByVal descendentMember As CodeElement, _
        ByVal typeElement As CodeElement) As CodeElement
    If descendentMember.Parent.FullName <> typeElement.FullName Then
        For Each member As CodeElement In typeElement.Members
            If IsMemberStatic(member) = False _
                    AndAlso member.Name = descendentMember.Name _
                    AndAlso member.Kind = descendentMember.Kind _
                    AndAlso member.Type.AsString = _
                        descendentMember.Type.AsString Then
                If member.Kind = vsCMElement.vsCMElementFunction Then
                    ' Compare method parameters
                    If member.Parameters.Count <> _
                            descendentMember.Parameters.Count Then
                        GoTo MemberEnd
                    End If
                    For i As Integer = 1 To member.Parameters.Count
                      Dim paramX As CodeParameter = _
                          member.Parameters.Item(i)
                      Dim paramY As CodeParameter = _
                          descendentMember.Parameters.Item(i)
                      If paramX.Type.AsString <> paramY.Type.AsString Then
                          ' Different overloads: skip
                          GoTo MemberEnd
                      End If
                    Next
                End If
                ' Matching base member found
                Return member
            End If
MemberEnd:
        Next
    End If

    ' Matching base member not found, keep looking
    Dim baseTypeList As ArrayList = New ArrayList()
    For Each baseType As CodeElement In typeElement.Bases
        baseTypeList.Add(baseType)
    Next
    If Not TypeOf (typeElement) Is CodeInterface Then
        For Each baseInterface As CodeElement In _
                typeElement.ImplementedInterfaces
            baseTypeList.Add(baseInterface)
        Next
    End If

    For Each baseType As CodeElement In baseTypeList
        Dim baseMember As CodeElement = _
            FindBaseMember(descendentMember, baseType)
        If Not baseMember Is Nothing Then
            Return baseMember
        End If
    Next

    ' No base member found
    Return Nothing
End Function

The first invocation of FindBaseMember is made within the macro method, with the class or structure of the documented member as the second argument:

Dim baseMember As CodeElement = FindBaseMember(element, element.Parent)
This sets the ball rolling. Each time FindBaseMember is invoked, it enumerates the members of the specified type and looks for one with the same name, accessibility, and signature as the method or property being documented. If no matching member is found on the specified type, or if the specified type is the same as the one on which the documented member occurs, FindBaseMember climbs further up the inheritance tree. It enumerates all direct base types gathered from the Bases and ImplementedInterfaces collections and invokes itself recursively with each base type as the argument. The final output of each FindBaseMember recursion is either a CodeElement representing the first matching base member found in the inheritance tree, or a null reference if no matching base member can be found.

Once the return value of FindBaseMember is in the bag, the rest of the macro is straightforward, as shown in Figure 11.

Figure 11 Using Results of FindBaseMember (Close Figure 11)
If baseMember Is Nothing Then
    MsgBox("Base member not found.  No documentation can be copied.")
    Exit Sub
End If

Dim doc As XmlDocument = GetCommentXml(baseMember)

If doc.FirstChild.ChildNodes.Count = 0 Then
    MsgBox("Base documentation not found.")
    Exit Sub
End If

SetCommentXml(element, doc)

The macro informs the user and exits if FindBaseMember has returned a null reference. Otherwise, the GetCommentXml helper is invoked on the base member to retrieve its XML documentation. If the base member itself turns out to be undocumented, the macro informs the user and exits. Finally, SetCommentXml is invoked, effectively copying the documentation from the base class to the active code file.

The InheritDocumentation macro is now complete. Simply execute it over a method or property that overrides or implements an XML-documented base member, and the base documentation is instantly cloned. While you may still want to tweak the text to your liking, the amount of work saved by this macro is significant.

Conclusion

A new version of Visual Studio is upon us, and with great new IDE features like Code Snippets getting all the press, it's easy to forget that macros have always offered awesome automation capabilities, many of which no canned features can ever hope to match. When applied to the task of creating XML documentation, macros are especially rewarding, resulting in much better code documentation with far less effort. In the preceding discussion, I've only scratched the surface of what you can accomplish using macros. The rest is for you to discover.

Tony Chow is a developer and consultant specializing in the .NET Framework. He is based in Los Angeles, California. You can reach Tony at tony@bluetentacle.com.

© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


Smart Tags: Simplify UI Development with Custom Designer Actions in Visual Studio

Smart Tags

Simplify UI Development with Custom Designer Actions in Visual Studio

Michael Weinhardt
This article is based on a prerelease version of Visual Studio 2005. All information contained herein is subject to change.
This article discusses:
  • The role of designer actions in Visual Studio
  • Designer action property and method items
  • How you can build your own custom designer actions and accompanying smart tags
This article uses the following technologies:
Visual Studio 2005
Code download available at: DesignerActions.exe (194KB)
Contents
An Introduction to Designer Actions
Building a Designer Action List
Creating Designer Action Property Items
Creating a Custom Designer
Rendering Designer Action Property Items
Design-Time-Only Designer Action Property Items
Multivalue Designer Action Property Items
Designer Action Method Items
Adding Designer Action Method Items
Toggling the Designer Action Method Item Display Name
Automatically Show the Smart Tag Panel
Categories and Descriptions
Headers and Labels
Where Are We?

T
he Properties window, which has been a mainstay of the Windows® Forms design-time experience, supports the configuration of an entire component or control from one location. Visual Studio® 2005 complements this experience with a new feature, known as smart tags, which exposes key configurations right on the design surface to enhance overall design-time productivity. Many Windows Forms 2.0 components expose smart tags by utilizing a subset of the Microsoft® .NET Framework 2.0 known as Designer Actions. In the same way, you can add smart tags to your own custom components and reap the same benefits.

An Introduction to Designer Actions

When components that support smart tags are selected on the Windows Forms design surface, a smart tag anchor is displayed at the top-right edge of a component. The anchor is a button that, when clicked, opens the smart tag panel. A smart tag panel is a designer-managed UI that exposes component configurations as a set of smart tag entries. Figure 1 illustrates these key elements.

Figure 1 Smart Tag Anchor, Panel, and Tasks
Figure 1 Smart Tag Anchor, Panel, and Tasks

In the designer action infrastructure of Visual Studio 2005, smart tag entries are known as designer action items and are shuttled around the design time in groups known as designer action lists. If a component requires smart tag support, it must pass at least one designer action list containing at least one designer action item to the Windows Forms designer. The same designer action list is then passed to the smart tag panel, which converts each designer action item into an equivalent visual task.

Building a Designer Action List

Since designer action items need to be packaged within designer action lists, the best way to begin adding smart tag support to your component is by building a designer action list. The designer action infrastructure implements a special class to represent designer action lists, appropriately called DesignerActionList. Like all designer action classes, DesignerActionList is located in the System.ComponentModel.Design namespace. Physically, however, they reside in the System.Design.dll assembly which you'll need to reference from your project before going any further.

The DesignerActionList class is designed to associate one or more designer action items with a single component. The association is made via its constructor, which expects a component's IComponent reference that is also exposed from the read-only Component property:

public class DesignerActionList {
  public DesignerActionList(IComponent component);
  public IComponent Component { get; }
  ...
}
DesignerActionList isn't a list in the sense that it implements IList or ICollection. Instead, it returns a list of designer action items stored in a DesignerActionItemCollection from its GetSortedActionItems method. The way you return your own list of designer action items is to derive from DesignerActionList, override GetSortedActionItems, and create, fill, and return a DesignerActionItemCollection instance (if you don't override it, the default implementation of GetSortedActionItems will use reflection to wrap all public methods and properties in DesignerActionItems and return them sorted alphabetically):
public class ClockControlDesignerActionList : DesignerActionList {
  ...
  public override DesignerActionItemCollection GetSortedActionItems() {

    // Create list to store designer action items
    DesignerActionItemCollection actionItems =
      new DesignerActionItemCollection();
    ...
    // Fill list of designer action items
    // Return list of designer action items
    return actionItems;
  }
}

DesignerActionItemCollection is a strongly typed collection that you fill with one or more DesignerActionItem objects. DesignerActionItem is an abstract class that captures functionality common to all designer actions. However, because DesignerActionItem is abstract, it can't be instantiated directly and, even if it could, it is too generic to be useful. Instead, the designer action infrastructure provides several DesignerActionItem derivations, each of which is specialized to support a particular style of designer action item. One of those derivations, the DesignerActionPropertyItem, models what is possibly the most common type of item displayed from a smart tag panel—a property entry.

Creating Designer Action Property Items

As an example, I'll be using the ClockControl that Chris Sells and I developed in our two-part series from the April 2003 and May 2003 issues of MSDN®Magazine. ClockControl had a Face property, which specifies whether the ClockControl's face is rendered as analog, digital, or both. Because it's one of the properties most likely to be configured at design time, it is an ideal smart tag candidate.

To turn it into a designer action property item, you need to create a DesignerActionPropertyItem instance and add it to DesignerActionItemCollection, like so:

public class ClockControlDesignerActionList : DesignerActionList {
  ...
  public override DesignerActionItemCollection GetSortedActionItems() {
    ...
    // Add Face designer action property item
    actionItems.Add(new DesignerActionPropertyItem("Face", "Face"));
    ...
  }
  ...
}

The DesignerActionPropertyItem class constructor expects two string arguments: member name and display name. Member name is the name of a property that's exposed by the custom DesignerActionList. Display name is a text value that the smart tag panel displays as a smart tag task's label.

The primary reason for the DesignerActionList to provide a property implementation, rather than using a component's own implementation, is that the smart tag panel is geared to operate on DesignerActionList objects rather than directly with the components themselves. As such, DesignerActionLists must implement a property that exposes the component's property, by proxy, for the smart tag panel to read from and write to. The role of the proxy property is illustrated by the diagram in Figure 2.

Figure 2 Proxy Property Implementation Exposed from DesignerActionList
Figure 2 Proxy Property Implementation Exposed from DesignerActionList

In light of this requirement, there is another consideration to make. The proxy property should not directly set a component's properties using code like this:

public class ClockControlDesignerActionList : DesignerActionList {
  ...
  // Face property
  public ClockFace Face {
    get { return this.ClockControl.Face; }
    set { this.ClockControl = value; }
  }
  ...
}
As outlined in the documentation, such code circumvents design-time services like the Properties window refreshing to reflect the changed property, or supporting undo for property. You can avoid this by coding the property's set accessor to use a property descriptor, as shown in Figure 3. You could use the Windows Forms designer's component change service to set these changes, although the technique that is suggested in the documentation is arguably much more concise.

Figure 3 Safe Implementation of the Face Property (Close Figure 3)
public class ClockControlDesignerActionList : DesignerActionList {
  ...
  // Face property
  public ClockFace Face {
    get { return this.ClockControl.Face; }
    set { SetProperty("Face", value); }
  }

  // Helper method to safely set a component's property
  private void SetProperty(string propertyName, object value) {
    // Get property
    PropertyDescriptor property =
      TypeDescriptor.GetProperties(this.ClockControl)[propertyName];
    // Set property value
    property.SetValue(this.ClockControl, value);
  }

  // Helper property to acquire a ClockControl reference
  private ClockControl ClockControl {
    get { return (ClockControl)this.Component; }
  }
}

Both the proxy Face property and the GetSortedActionItems method comprise the minimum implementation required by a designer action list. GetSortedActionItems will be called from the smart tag panel when it opens and when it is refreshed, in both cases retrieving the single designer action property item and displaying it as an equivalent smart tag task. The initial value for the smart tag property is retrieved from the component's Face property through the ClockControlDesignerActionList class's proxy property, and vice versa to set the Face property if it's changed from the smart tag panel.

Of course, the problem is that we haven't yet passed ClockControlDesignerActionList to the Windows Forms designer and, consequently, ClockControl won't even be provided designer action support. This little hole is filled by a custom designer.

Creating a Custom Designer

Designer actions are classified as design-time-only functionality, which means they are provided to and used by a component solely from within the design-time environment. If you're familiar with the design-time infrastructure in general, you'll be aware that it is the responsibility of the designer class to provide design-time-only functionality such as this. By default, all components are provided a default designer when they are hosted in the Windows Forms designer, unless they use their own custom designers. A custom designer is a class that implements the IDesigner interface, although most derive from .NET Framework-provided IDesigner base implementations specific to the component type, including ComponentDesigner for components and ControlDesigner for controls. A custom designer is associated with a component using the DesignerAttribute. In the original implementation of ClockControl, ClockControlDesigner was created to add a design-time-only property to the ClockControl. Following are the key implementation details:

public class ClockControlDesigner : ControlDesigner { ... }

[Designer(typeof(ClockControlDesigner))]
public class ClockControl : Control, ... { ... }

With the advent of the .NET Framework 2.0 and smart tag support, the ComponentDesigner class was updated with the new read-only ActionLists property, shown here:

public class ComponentDesigner : IDesigner, ... {
  ...
  public virtual DesignerActionListCollection ActionLists { get; }
  ...
}

The ActionLists property is of type DesignerActionListCollection, an object that stores one or more DesignerActionList objects. Although discussion of multiple designer action lists are beyond the scope of this article, this support for them allows you to break a single custom designer action list into several discrete versions, which you may consider if the original is becoming too large to easily maintain, or if you need to selectively show subsets of designer action items based on a component's design-time context.

By default, the base ComponentDesigner class's ActionList property returns an empty DesignerActionListCollection, which is viewed by the Windows Forms designer as "Smart Tags Not Required" for the component in question. Consequently, you'll need to override ActionList and return your own DesignerActionListCollection, to which you'll add your custom DesignerActionList implementations. Since ClockControlDesigner indirectly derives from ComponentDesigner via ControlDesigner, you can override ActionLists directly, as shown in Figure 4.

Figure 4 Overriding ActionLists (Close Figure 4)
public class ClockControlDesigner : ControlDesigner {
  ...
  public override DesignerActionListCollection ActionLists {
    get {
      // Create action list collection
      DesignerActionListCollection actionLists =
        new DesignerActionListCollection();

      // Add custom action list
      actionLists.Add(
        new ClockControlDesignerActionList(this.Component));

      // Return to the designer action service
      return actionLists;
    }
  }
  ...
}

That's all the implementation you need to be able to edit a property from the smart tag panel. After rebuilding the updated ClockControl solution and then dragging the new ClockControl onto the form, you should be able to edit the Face property from the smart tag panel for the ClockControl, as shown in Figure 5.

Figure 5 Face Designer Action Property Item
Figure 5 Face Designer Action Property Item

While you can't influence what's displayed as the smart tag panel title, the Windows Forms designer uses a reasonable default with the following naming convention: "ComponentTypeName Tasks". The Windows Forms designer also takes care of determining what controls are used to provide display and editing of designer action property items.

Rendering Designer Action Property Items

From Figure 5, you can see that the smart tag panel renders enumeration types, like the Face property, as dropdown lists. The heuristics exercised by the smart tag panel to translate other property types to controls includes the following:

  • If a property has an associated UI type editor, as specified automatically by the property type, or by manually specifying one by applying the EditorAttribute to the proxy property, the editor is shown.
  • If a property is a Boolean, a checkbox is shown.
  • Properties other than enumerations and Booleans are displayed as textboxes. If a property is of a type that doesn't support conversion from a string, it's displayed as read-only text sourced from the type's ToString method.

While you have no control over this process, beyond choosing a property's type, you can add the icing to the heuristics cake with the same special design-time attributes to provide enhanced property editing support. One of these is the EditorAttribute, and it allows developers to associate a specialized UI with a property. The ClockControl library already contains a UI type editor, FaceEditor, designed specifically for properties of type ClockFace. Since the Face property is of such a property type, its ClockControl implementation can, and should, be adorned with EditorAttribute to associate the FaceEditor with it. Unfortunately, the smart tag panel doesn't know about it because it refers to the ClockControlDesignerActionList class's proxy Face property. However, the situation is quickly rectified by adding the EditorAttribute to the proxy Face property, as shown here:

// Face proxy property
[Editor(typeof(FaceEditor), typeof(UITypeEditor))]
public ClockFace Face {
  get { return this.ClockControl.Face; }
  set { SetProperty("Face", value); }
}

FaceEditor provides a dropdown style of UI type editor, but the same technique can also be used with modal UI type editors, such as the DigitalTimeFormatEditor that ClockControl associates with the DigitalTimeFormat property, as shown in Figure 6. A modal UI type editor behaves the same way from the smart tag panel as it would from the Properties window.

Figure 6 Using DigitalTimeFormatEditor (Close Figure 6)
public class ClockControlDesignerActionList : DesignerActionList {
  ...
  public override DesignerActionItemCollection GetSortedActionItems() {
    ...
    // Add DigitalTimeFormat designer action property item
    actionItems.Add(new DesignerActionPropertyItem(
        "DigitalTimeFormat","Digital Time Format"));
    ...
  }
  ...
  [Editor(typeof(DigitalTimeFormatEditor), typeof(UITypeEditor))]
  public string DigitalTimeFormat {
    get { return this.ClockControl.DigitalTimeFormat; }
    set { SetProperty("DigitalTimeFormat", value); }
  }
  ...
}

If a proxy property is one of the .NET Framework 2.0 intrinsic types, such as DateTime, that have UI type editors associated with them, the smart tag panel will automatically allow access to UI type editors. This is the case for ClockControl's BackupAlarm and PrimaryAlarm properties.

While adding a UI type editor to a designer action property item is relatively straightforward, more exotic scenarios are supported and I'll take a look at some of them now.

Design-Time-Only Designer Action Property Items

The ClockControl's original designer exposed the design-time-only ShowBorder property, shown here:

public class ClockControlDesigner : ControlDesigner {
  ...
  // Provide implementation of ShowBorder to provide
  // storage for created ShowBorder property
  [CategoryAttribute("Design")]
  [DesignOnlyAttribute(true)]
  [DefaultValueAttribute(true)]
  [DescriptionAttribute("Show/Hide a border at design-time.")]
  public bool ShowBorder { get; set; }
  ...
}

Because it's implemented on a designer rather than a component, turning ShowBorder into a designer action property item is a little different than what you've seen, although the creation of a DesignerActionPropertyItem for it is the same:

public class ClockControlDesignerActionList : DesignerActionList {
  ...
  public override DesignerActionItemCollection GetSortedActionItems() {
    ...
    // ShowBorder designer action property item
    actionItems.Add(
      new DesignerActionPropertyItem("ShowBorder", "Show Border"));
    ...
  }
  ...
}

The difference here is that the proxy ShowBorder property will need to read from and write to ClockControlDesigner's property implementation. First this means acquiring a reference to ClockControlDesigner from ClockControlDesignerActionList. Unfortunately, DesignerActionList doesn't provide this support natively, but it's quite simple to add:

public class ClockControlDesignerActionList : DesignerActionList {
  ...
  // Helper method to acquire a ClockControlDesigner reference
  private ClockControlDesigner Designer {
    get {
      IDesignerHost designerHost = (IDesignerHost)
        this.ClockControl.Site.Container;
      return (ClockControlDesigner)
        designerHost.GetDesigner(this.ClockControl);
    }
  }
  ...
}
Now, the proxy ShowBorder property can easily interact with the ClockControlDesigner class's actual ShowBorder property:
public class ClockControlDesignerActionList : DesignerActionList {
  ...
  // ShowBorder proxy property
  public bool ShowBorder {
    get { return this.Designer.ShowBorder; }
    set { this.Designer.ShowBorder = value; }
  }
  ...
}

You may have noticed that this proxy property doesn't use the SetProperty helper introduced earlier. Instead, it leans on the actual ShowBorder implementation's use of the design-time's component change service to achieve the same result (see Figure 7).

Figure 7 ShowBorder Implementation (Close Figure 7)
public class ClockControlDesigner : ControlDesigner {
  ...
  public bool ShowBorder {
    get { return _showBorder; }
    set {
      // Change property value
      PropertyDescriptor property =
        TypeDescriptor.GetProperties(
          typeof(ClockControl))["ShowBorder"];
      this.RaiseComponentChanging(property);
      _showBorder = value;
      this.RaiseComponentChanged(property, !_showBorder, _showBorder);
      ...
    }
  }
  ...
}

The only thing left to do is rebuild the solution, with the result shown in Figure 8.

Figure 8 ShowBorder Designer
Figure 8 ShowBorder Designer

Since the ShowBorder property is Boolean, it appears as a checkbox in the smart tag panel. In the sample, you'll notice that not only does the border appear and disappear as the ShowBorder designer action property is toggled, but the Properties window's ShowBorder field is also updated immediately to reflect the change. This is the expected result of the actual ShowBorder property's component change service integration.

Multivalue Designer Action Property Items

One other designer action property item that's worth looking at is one that is multivalued, like the HourHand, MinuteHand, and SecondHand properties implemented by ClockControl which, if displayed from the smart tag panel, would look like Figure 9.

Figure 9 Not So Pretty
Figure 9 Not So Pretty

Unfortunately, the smart tag panel doesn't allow the expandable style of property editing that is supported by the Properties window. At best, the smart tag panel can use the custom Hand class's type converter, HandConverter, which supports conversion between a Hand object and a multivalued string representation of a Hand object. However, as you can see from Figure 9, the string format is not as user friendly as it could be.

Because expandable property editing is not supported from the smart tag panel, you'll need to come up with some alternatives if string editing won't cut it. For example, you might consider providing a separate designer action property item for each constituent property. While this would work, you would need to be conscious of using too much UI real estate, which isn't the best use of the smart tag panel. Alternatively, you could create a new UI type editor specifically for the Hand type, which would consume no more UI real estate than the current solution does now.

Designer Action Method Items

If you consider the physical configuration of the HourHand, MinuteHand, and SecondHand properties to really represent a single, logical unit of configuration, you can change tack and use designer action method items instead. Designer action method items provide single-click access to complex functionality that will likely involve more than a single property set. This functionality would typically be presented via a specialized UI to reduce any complexity. As such, designer action method items are an ideal way to roll the physical configuration of the three hand properties into a single, easy, logical step.

To add a task to the smart tag panel that supports editing all clock hands at once, you would begin by updating GetSortedActionItems in your custom DesignerActionList class, only this time adding a DesignerActionMethodItem object to the DesignerActionItemCollection, as shown here:

public class ClockControlDesignerActionList : DesignerActionList {
  ...
  public override DesignerActionItemCollection GetSortedActionItems() {
    ...
 // EditClockHands designer action method item
    actionItems.Add(new DesignerActionMethodItem(
        this, "EditClockHands",
        "Edit Clock Hands"));
    ...
  }
  ...
}

The DesignerActionMethodItem class's constructor accepts three arguments: a reference to the designer action list object that the designer action method item is associated with, the name of a method implementation that resides in the custom designer action list, and the smart tag panel text for this task. In this case, the method implementation isn't really a proxy method because the smart tag window calls it to do whatever needs to be done. Internally, the method will do all the property reads and writes as necessary to complete the task. With regard to configuring all three clock hands at once, this means grabbing the ClockControl's three hand property values, passing them to a custom form for editing, and writing the property values back to the ClockControl if they were changed. The code that performs these steps is shown in Figure 10.

Figure 10 Configuring All Three Clock Hands (Close Figure 10)
public class ClockControlDesignerActionList : DesignerActionList {
  ...
  private void EditClockHands() {
    // Create form
    HandsEditorForm form = new HandsEditorForm();

    // Set current hand values
    form.HourHand = _ this.ClockControl.HourHand;
    form.MinuteHand = _ this.ClockControl.MinuteHand;
    form.SecondHand = _ this.ClockControl.SecondHand;

    // Update new hand values if OK button was pressed
    if( form.ShowDialog() == DialogResult.OK ) {
      SetProperty("HourHand", form.HourHand);
      SetProperty("MinuteHand", form.MinuteHand);
      SetProperty("SecondHand", form.SecondHand);
    }
  }
  ...
}

With a DesignerActionMethodItem object and a method implementation in place, a rebuild is required to have the designer action method item appear on the smart tag panel.

As you can tell from Figure 11, designer action method items are displayed as link labels on the smart tag panel. They are also displayed on the Property window's Command panel; if the Command panel is not visible, right-click the Properties window and choose Commands.

Figure 11 Edit Clock Hands Designer Action Method Item
Figure 11 Edit Clock Hands Designer Action Method Item

You should note that unlike designer action property items, the method implementation of a designer action method item can be set to private, protected, internal, and public. Also, the method must not accept any arguments as the smart tag panel doesn't provide a mechanism to capture and pass them to the designer action method implementation. Conversely, the designer action method implementation must not return a value as the smart tag panel can't receive or process it.

Adding Designer Action Method Items

The sphere of influence exerted by designer action method items extends beyond the smart tag panel. In fact, they can be configured to be displayed in a design-time component's context menu and implicitly in the Properties window. This is achieved through one of the DesignerActionMethodItem class's constructor overloads which accept an additional Boolean argument. When true, the designer action infrastructure adds a menu item to the underlying component's context menu, as well as adding a link label in the Properties window's Description pane. A little tweak to the existing solution is required:

public override DesignerActionItemCollection GetSortedActionItems() {
  ...
  // EditClockHands designer action method item
  actionItems.Add(new DesignerActionMethodItem(
      this, "EditClockHands", "Edit Clock Hands" , true));
  ...
}

Note that custom designers implement a Verbs property that you can use to add and update context menus and the Properties window. If you built a custom designer from a version of the .NET Framework earlier than version 2.0 that does use the Verbs property, the Windows Forms designer will automatically turn them into smart tag methods without any effort on your part. Unfortunately, you won't be able to categorize them or lay them out as nicely as native designer action items, which I'll explore shortly.

Toggling the Designer Action Method Item Display Name

One of the common designer action method items you'll find on rectangular controls in Windows Forms 2.0 is the ability to dock and undock those controls to and from their parent container from a designer action method item available from the smart tag panel (this smart tag item can be provided for any control automatically by adding the DockingAttribute to a control). The trick with the Dock/Undock property is to toggle the display name to reflect the current component's Dock state whenever the designer action method item's link label is clicked, ultimately looking like Figure 12.

Figure 12 Dock/Undock Property
Figure 12 Dock/Undock Property

To begin with, you'll need to create a new designer action method item, with an accompanying method implementation that toggles the Dock property between a DockStyle of Fill and a DockStyle of None, as shown in Figure 13.

Figure 13 Toggling the Dock Property (Close Figure 13)
public class ClockControlDesignerActionList : DesignerActionList {
  ...
  public override DesignerActionItemCollection GetSortedActionItems() {
    ...
    // Dock/Undock designer action method item
    actionItems.Add(new DesignerActionMethodItem(
        this, "ToggleDockStyle", "Dock/Undock in parent container"));
    ...
  }
  ...
  private void ToggleDockStyle() {

    // Toggle ClockControl's Dock property
    if( _clockControl.Dock != DockStyle.Fill ) {
      SetProperty("Dock", DockStyle.Fill);
    }
    else {
      SetProperty("Dock", DockStyle.None);
    }
  }
  ...
}

To toggle the designer action method item's display name, you need two things: a helper method that calculates and returns the appropriate text, and a way to have it called when the dock style changes. With regard to the latter, the smart tag panel is refreshed whenever the DesignerActionService.Refresh method is used, which results in a subsequent call to GetSortedActionItems. Consequently, you can invoke the helper method from the DesignerActionMethodItem's constructor. The updated constructor and new helper method are both shown in Figure 14.

Figure 14 Constructor and Helper Method (Close Figure 14)
public class ClockControlDesignerActionList : DesignerActionList {
  ...
  public override DesignerActionItemCollection GetSortedActionItems() {
    ...
    // Dock/Undock designer action method item with display name
    // generated from GetDockStyleText helper method
    actionItems.Add(new DesignerActionMethodItem(
        this, "ToggleDockStyle", GetDockStyleText()));
    ...
  }
  ...
  // Helper method that returns an appropriate display name for the
  // Dock/Undock property, based on the ClockControl's current Dock
  // property value
  private string GetDockStyleText() {
    if( this.ClockControl.Dock == DockStyle.Fill ) {
      return "Undock in parent container";
    }
    else {
      return "Dock in parent container";
    }
  }
  ...
}

Automatically Show the Smart Tag Panel

If you wanted to give developers the option to dock to the parent container or to perform any sort of configuration, as soon as a component is dragged onto a form, you can make use of the DesignerActionList class's AutoShow property. By default, the base implementation of AutoShow will return false, which turns off AutoShow (a user can also disable AutoShow completely through an option in the Tools | Options page). Consequently, you'll need to override it to return:

public class ClockControlDesignerActionList : DesignerActionList {
  ...
  public ClockControlDesignerActionList(ClockControl clockControl)
    : base(clockControl) {
    // Automatically display smart tag panel when design-time component
    // is dropped onto the Windows Forms Designer
    this.AutoShow = true;
    ...
  }
  ...
}

Whether your smart tag panel is automatically shown or shown on demand, you'll want it to be a little more approachable. A variety of support for improving the appearance of your smart tag panel is available from the designer action infrastructure.

As you may have noticed, the only order applied to the appearance of the designer action items is that they are listed in the order they are added to the DesignerActionItemCollection in the GetSortedActionItems method. Unfortunately, the default appearance doesn't really provide developers with much additional visual assistance to understand what each designer action item does. Fortunately, you can use several designer action techniques to turn your smart tag panel from drab to fab.

Categories and Descriptions

By default, designer action property and method items are placed into a default category. To help developers distinguish between designer actions and then determine what the specific actions do, you can add the designer actions to custom categories and give them informative text descriptions. Both DesignerActionPropertyItem and DesignerActionMethodItem classes provide constructor overloads that allow you to supply both of these:

public override DesignerActionItemCollection GetSortedActionItems() {
  ...
  // Add DigitalTimeFormat designer action property
  actionItems.Add(new DesignerActionPropertyItem( "DigitalTimeFormat",
      "Digital Time Format", "Appearance", // Category string argument
      "The digital time format, ..." // Description string argument
    ));
  ...
}

When categories are applied, the order in which designer action items are rendered to the smart tag panel is determined first by category and second by the order in which they are added to the designer action item collection. The description is displayed as a tooltip when the mouse hovers over the task.

Usually, custom components like the ClockControl will apply both CategoryAttribute and DescriptionAttribute to their properties to influence their appearance in the Properties window. Consequently, the designer action list class's proxy implementation of these properties should use the same string values. One way to get them is with reflection, as shown in Figure 15.

Figure 15 Using Reflection (Close Figure 15)
public class ClockControlDesignerActionList : DesignerActionList {
  ...
  public override DesignerActionItemCollection GetSortedActionItems() {
    ...
    // Add DigitalTimeFormat designer action property
    actionItems.Add(new DesignerActionPropertyItem(
        "DigitalTimeFormat", "Digital Time Format",
        GetCategory(this.ClockControl, "DigitalTimeFormat"),
        GetDescription(this.ClockControl, "DigitalTimeFormat")));
  ...
  }

 // Helper method to return the Category string from a CategoryAttribute
 // assigned to a property exposed by the specified object
 private string GetCategory(object source, string propertyName) {
    PropertyInfo property = source.GetType().GetProperty(propertyName);
    CategoryAttribute attribute = (CategoryAttribute)
      property.GetCustomAttributes(typeof(CategoryAttribute), false)[0];
    if( attribute == null ) return null;
    return attribute.Category;
  }

  // Helper method to return the Description string from a
  // DescriptionAttribute assigned to a property exposed by the specified
  // object
  private string GetDescription(object source, string propertyName) {
    PropertyInfo property = source.GetType().GetProperty(propertyName);
    DescriptionAttribute attribute = (DescriptionAttribute)
      property.GetCustomAttributes(
        typeof(DescriptionAttribute), false)[0];
    if( attribute == null ) return null;
    return attribute.Description;
  }
}

Both GetCategory and GetDescription take two arguments: an object and the name of the property implemented by the object that is adorned with either CategoryAttribute or Description. Then, the TypeDescriptor is employed to retrieve the desired attribute and return the string value. Both GetCategory and GetDescription return null if the source object's property implementation is not adorned with the expected attribute. Null strings passed to the DesignerActionPropertyItem class's constructor are treated as if they weren't provided. The corresponding smart tag task will be placed into the default category and won't have a tooltip. In general, properties that you expose from your components that can be configured from the design time should be adorned with both CategoryAttribute and DescriptionAttribute, especially as they influence the Properties window in the same manner to provide the same benefits.

The reason that both the GetCategory and GetDescription methods expect an object parameter to describe the source object rather than something more specific, like a ClockControl parameter, is to cater to properties located on different types like ClockControlDesigner.ShowBorder. Here's how GetCategory and GetDescription are passed a reference to ClockControlDesigner:

public override DesignerActionItemCollection GetSortedActionItems() {
  ...
  // ShowBorder designer action property item
  actionItems.Add(new DesignerActionPropertyItem(
      "ShowBorder", "Show Border",
      GetCategory(this.Designer, "ShowBorder"),
      GetDescription(this.Designer, "ShowBorder")));
  ...
}

In general, this technique works well for designer action properties since, as we know, all design-time configurable component or designer properties are likely to be adorned with CategoryAttribute or DescriptionAttribute. Designer action methods, on the other hand, are highly likely to be implemented in their entirety on the custom designer action list, rather than acting as proxies to underlying implementations. Consequently, you'll have to provide category and description strings when instantiating DesignerActionMethodItem, like so:

public override DesignerActionItemCollection
   GetSortedActionItems() {
   ...
   // EditClockHands designer action method item
    actionItems.Add(new DesignerActionMethodItem(
        this, "EditClockHands",
        "Edit Clock Hands","Appearance",
        "Configure the ClockControl's hour,
         minute and second hands.",
        true));
   ...
}
Fortunately, these are unlikely to be repeated anywhere else. In those few cases where they might be repeated, you should easily be able to refactor both GetCategory and GetDescription methods to operate against methods in addition to properties.

Headers and Labels

If you want categories to be more recognizable, you could assign each one a text header using another designer action item, DesignerActionHeaderItem, whose constructor accepts a single string value, as shown in the following:

public override DesignerActionItemCollection GetSortedActionItems() {
   ...
  actionItems.Add(new DesignerActionHeaderItem("Appearance"));
  // Appearance category designer action items added here ...
  actionItems.Add(new DesignerActionHeaderItem("Behavior"));
  // Behavior category designer action items added here ...
  actionItems.Add(new DesignerActionHeaderItem("Design"));
  // Design category designer action items added here ...
   ...
}

Even more descriptive would be to provide a piece of descriptive text, which you can do with the DesignerActionTextItem, whose constructor accepts a string category name value plus the text string, as shown here:

public override DesignerActionItemCollection GetSortedActionItems() {
  ...
  actionItems.Add(new DesignerActionHeaderItem("Appearance"));
  actionItems.Add(new DesignerActionTextItem(
      "Properties that affect how the ClockControl looks.",
      "Appearance"));
  // Appearance category designer action items added here ...
  ...
}
The two parameter constructors ensure that labels are sorted first by the category they are assigned to, then in the order in which they are added to the designer action item collection.

Figure 16 ClockControl Smart Tag Pane
Figure 16 ClockControl Smart Tag Pane

The result of applying categories, descriptions, headers, and labels is the fabulous smart tag panel, shown in Figure 16. You can easily mix and match subsets of these UI elements to support a wide variety of scenarios.

Where Are We?

Your custom components can utilize the .NET Framework 2.0 designer action infrastructure to provide smart tag support. This process is relatively simple, particularly when you need to support more exotic scenarios such as toggling text labels, integrating with the context menu and Properties window, and optimizing smart tag panel usability. Smart tags create a more polished, contemporary, and productive design-time experience for custom components. It is definitely worth your time to learn all about them.

Michael Weinhardt is currently co-updating the book, Windows Forms Programming in C# by Chris Sells and writes a monthly column for MSDN online called Wonders of Windows Forms. Visit www.mikedub.net for further information.


Reflection: Dodge Common Performance Pitfalls to Craft Speedy Applications

Reflection

Dodge Common Performance Pitfalls to Craft Speedy Applications

Joel Pobar
Parts of this article are based on a prerelease version of the .NET Framework 2.0. Those sections are subject to change.
This article discusses:
  • How to get good reflection performance
  • Early-bound and late-bound invocation
  • Member caching and handles
  • Best practices for using reflection
This article uses the following technologies:
.NET Framework, C#
Contents
What's Slow and What's Not?
When Should You Use Reflection?
Invoking or Calling a Member
Early-Bound and Late-Bound Invocation
Hybrid Late-Bound to Early-Bound Invocation
I Want My MemberInfo
The Reflection MemberInfo Cache
Handles and Handle-Resolution APIs
Implementing Your Own Cache
Wrapping It Up

U
sing reflection efficiently is like haggling with an API. You have to pay to play and make some concessions. But it's worth it. Reflection in .NET is one of the most powerful features you can employ to achieve application extensibility. With reflection, you can load types, understand their members, make decisions about them, and execute, all within the safety of the managed runtime. But to use this power wisely, it's important to understand the associated costs and pitfalls.

In this article I'll show you which reflection tasks are costly and which ones are not. I'll guide you in determining the cost/benefit trade-offs and will dive deep into the framework to get a close look at the internals. Then I'll review what's new for reflection in the Microsoft® .NET Framework 2.0 and how it makes certain functionality even less costly.

What's Slow and What's Not?

Figure 1 outlines some commonly used reflection APIs. As you'll see, some are fast and light while others are heavyweights.

Figure 1 Common Reflection Tasks (Close Figure 1)

Fast and Light Functions
typeof
Object.GetType
typeof == Object.GetType
Type equivalence APIs (including typehandle operator overloads)
get_Module
get_MemberType
Some of the IsXX predicate APIs
New token/handle resolution APIs in the .NET Framework 2.0
Costly Functions
GetXX  APIs (MethodInfo, PropertyInfo, FieldInfo, and so on)
GetCustomAttributes
Type.InvokeMember
Invoke APIs (MethodInfo.Invoke, FieldInfo.GetValue, and so on)
get_Name (Name property)
Activator.CreateInstance

Generally, lightweight functions are fast for one of two reasons: either the reflection infrastructure has the information it needs already stored in super-fast runtime data structures, or the just-in-time (JIT) compiler detects these method calls and code patterns and generates special optimizations to speed things up a bit. Two good examples of a reflection JIT optimization are the C# typeof method and the base class library's (BCL) Object.GetType method. Both are heavily used in the BCL for type equality, and as a result they had to be special cased to ensure optimal performance. In this article, however, I'll focus on the costly reflection APIs.

When Should You Use Reflection?

You should always think carefully about how you're using reflection. Using reflection occasionally without enforcing strict performance criteria is probably fine. If reflection APIs are only invoked when you're calling the part of your app that loads and invokes a third-party plug-in, then the cost should be reasonable. However, if your scenario involves a high-volume ASP.NET Web site that requires good throughput and response times, and if it makes significant use of the heavy reflection APIs in your "fast path" (the code in the application that must run very fast and is used repeatedly), you should really consider an architectural review to decide if you've made the right decisions on your use of reflection.

As a classic example, imagine you're writing an app that acts as a host for third-party plug-ins. Do calls into these plug-ins occur mostly in areas of you code that don't get much action or will they be accessed frequently? And what if you require reflection but your application has to be fast?

The first thing to think about when considering reflection is whether the extensibility point of your application can be statically defined as an interface or as a base class. The cleanest and best-performing way to call members on an extensibility point is to statically define the contract for consumers to implement, and then use that contract to invoke safely. Usually this contract is defined by an interface or base class, and the consumer has access to the contract to implement against it. To illustrate, I've created a sample with an extensible way to plug in a logging infrastructure.

First, I define a contract for my application's logging requirement. My app should call on three different methods to log three different types of messages: security, application, and error messages, so I define the contract to be an interface with three methods:

public interface LogInterface {
    void WriteErrorEvent(string errorMessage);
    void WriteApplicationEvent(string applicationMessage);
    void WriteSecurityEvent(string securityMessage);
}
This interface could be compiled as a DLL and distributed to third parties that want to provide logging infrastructure plug-ins for my app. Third parties could then implement their logging infrastructure using the contract (the interface); making their plug-in libraries ready for my app to consume. This code shows a third-party logging plug-in that writes all log messages to the console:
public class JoelsLogger : LogInterface {
    public void WriteErrorEvent(string errorMessage) {
        Console.WriteLine("error: " + errorMessage);
    }
    public void WriteApplicationEvent(string applicationMessage) {
        Console.WriteLine("application: " + applicationMessage);
    }
    public void WriteSecurityEvent(string securityMessage) {
        Console.WriteLine("security: " + securityMessage);
    }
}

Now that I have a contract and a library that implements the contract, I need code for the discovery and loading of the plug-in, discovery of the interface, type instantiation, and invocation. Here is a trivial example of this application logic:

Assembly asm = Assembly.LoadFrom(@"c:\myapp\plugins\joelslogger.dll");
LogInterface logger = null;

foreach (Type t in asm.GetTypes()) {
    if (t.GetInterface("LogInterface") != null) {
        logger = (LogInterface)Activator.CreateInstance(t);
        break;
    }
}

if (logger != null) logger.WriteApplicationEvent("Initialized...");
Obviously this is not optimal. It's a clunky way to discover what type implements the interface since it iterates over each type and asks the loader to load it. A better approach is to declare that all plug-ins specify the type to locate during the library load stage. This can be done via an assembly-level custom attribute or through a configuration file. Unfortunately, there is no way to enforce this "lookup" contract via code (though exceptions can be thrown from my application at run time if a contract is not met), so my app will have to rely on good solid documentation for its extensibility point to make sure third parties implement this piece of code correctly. Here's how you can add an assembly-level custom attribute:
[AttributeUsage(AttributeTargets.Assembly)]
public class LogInterfaceTypeAttribute : System.Attribute {
    public readonly string TypeName;

    public LogInterfaceTypeAttribute(string typeName) {
        TypeName = typeName;
    }
}

LogInterfaceTypeAttribute has only one field, the string name of the type that implements the "LogInterface" interface. In my application docs, I ask that all third parties add my LogInterfaceType attribute to their library at the assembly level to tell my app which type in that library implements my interface:

[assembly: LogInterfaceType("JoelsLogger")]
class JoelsLogger : LogInterface { ... }

Now I discover the attribute after I have loaded the assembly:

LogInterfaceType logtype = (LogInterfaceType)
    asm.GetCustomAttributes(typeof(LogInterfaceType), false)[0];
LogInterface log = (LogInterface)
    Activator.CreateInstance(asm.GetType(logtype.TypeName));
log.WriteApplicationEvent("Initialized...");
This approach avoids the inefficient loop-and-check previously used for contract implementation discovery. The performance gain is quite significant because it only loads the exact type that is needed to use the third-party logging library.

Invoking or Calling a Member

The reflection APIs are divided by their basic goals: invoking a member and inspecting a member. Now I'll consider the implementation behind reflection to discover exactly where the costs lie.

Reflection invocation is generally performed through two APIs in the reflection namespace: MethodBase.Invoke and Type.InvokeMember. This invocation is sometimes referred to as "late-bound" because the call to the member is analyzed at run time instead of at compile time. In contrast, call instructions emitted by the compiler are called "early-bound" calls. Using the late-bound APIs to perform invocation on members is significantly slower than using the early-bound call instruction counterparts. To put the performance implications of these APIs in perspective, it's worthwhile to contrast the cost with the other mechanisms for member invocation used in the runtime. Figure 2 illustrates the spectrum of invocation mechanisms for methods and the relative performance overhead of each. For a closer look, read Eric Gunnerson's great article, "Calling Code Dynamically".

Figure 2 Relative Performance of Invocation Mechanisms
Figure 2 Relative Performance of Invocation Mechanisms

These metrics are based on an abstract scenario and should only be used as a starting point, since actual scenarios may have different performance characteristics. The best way to get accurate numbers for your scenario is to measure, measure, and measure again.

Early-Bound and Late-Bound Invocation

Invocation mechanisms can be divided into three categories: early-bound, late-bound, and a hybrid of early- and late-bound invocation. Early-bound invocation mechanisms result from direct intermediate language (IL)-level instructions (call, callvirt, and calli), interface calls, and delegate calls. These early-bound call mechanisms are typically emitted statically by compilers at compile time, where the compiler is supplied with all of the required information about the target to be invoked. These early-bound cases are significantly faster than their late-bound and hybrid counterparts as they map down to a few x86 instructions emitted by the JIT or native code generation (NGEN) compilers. Aggressive optimizations can also be made by the JIT because the call site and method are unchanging and are well known to the runtime.

The late-bound cases are MethodBase.Invoke, DynamicMethod via Invoke, Type.InvokeMember, and late-bound delegate calls (calls on delegates via Delegate.DynamicInvoke). All of these methods come with significantly more negative performance implications than the early-bound cases. Even in the best case, they're typically an order of magnitude slower than the slowest early-bound case. Type.InvokeMember is the slowest of the late-bound invocation mechanisms because there are two functions that InvokeMember needs to perform to properly invoke a member. First, it must figure out the exact member it's supposed to invoke, as determined by the user's string input, and then perform checks to be sure the invocation is safe. MethodBase.Invoke, on the other hand, doesn't need to figure out which method it must call because the MethodBase already holds the identity of the method.

To further explore how reflection sets up and invokes a member, consider a typical usage of MethodBase.Invoke:

public class D {
    public void MyMethod(string arg) { ... }
}
...
MethodInfo mi = typeof(D).GetMethod("MyMethod");
mi.Invoke(dobj, new object[] { "testing" });
Reflection needs to take the user's string representation of MyMethod and create a MethodInfo for the method that matches that name. Because the common language runtime (CLR) stores information about the method's name in metadata, reflection must look inside metadata to learn which method on type "D" has the specified name. This logic alone is expensive. Once you know which method has the string name MyMethod, a MethodInfo is created (MethodInfo derives from MethodBase).

On the call to Invoke, reflection must perform all the checks required to make the call safe and secure. Argument and parameter type matching is generally done first. In the code snippet just shown, you can see that MyMethod accepts a string as a parameter, and the call to invoke has an object array that contains the corresponding string to be used as the method's argument. This is a match and is therefore type safe. If the types in the argument array and the types of the method parameters don't match exactly, the reflection framework then checks to see if it can perform type coercion to force the types of the arguments and parameters to match. After argument and parameter matching and coercion are complete, reflection checks any code access security (CAS) demands that are located on the method and invokes the security subsystem as needed. Finally, after the security subsystem has vouched for the safety of the call, reflection tells the runtime to start execution of the call.

Hybrid Late-Bound to Early-Bound Invocation

If you think about it, late-bound really means doing all the work at run time that a compiler would do at compile time. The fundamental problem is that you may repeat the same work every time you invoke the method, while compilers do all that heavy lifting once. With the code-generation features of the runtime, however, you can bridge the gap between late-bound and early-bound invocation by acting like a compiler at run time. I call this the hybrid approach. Late-bound invocation can do the work to find the method, then emit code to statically patch up the call site to the method, much like a compiler would. In the .NET Framework 2.0, there's a new reflection feature built into the runtime called Lightweight Code Generation (LCG), which is optimal for enabling this hybrid scenario. The hybrid approach can still be accomplished in the .NET Framework 1.x, albeit with a slightly heavier implementation.

I Want My MemberInfo

To create a MemberInfo, there are generally two places that reflection looks to get information about a member: metadata and runtime data structures. When the runtime spins up to run a program, it populates its own runtime data structures with some of the information from metadata found on disk in the PE file. If you recall, metadata is a set of data tables that describe your assemblies and the entities contained in them. Using methods as an example, you can find the method names and the attributes of a method, like its visibility, in metadata. The runtime data structures only store information from metadata that is constantly and consistently referenced by its runtime services (JIT compilation, security checks, and so on). The CLR will lazily populate its data structures from metadata to get the currently executing job done. You can think of the information in the runtime's data structures as a very small subset of the metadata.

To create a MemberInfo, reflection must retrieve metadata information. It finds this data by consulting the runtime data structures or by going to metadata and computing it. Making use of the runtime data structures is fast since the data structures are optimized and have great locality of reference. However, if the runtime data structures don't have the required information, reflection is used to look in the metadata— a significantly slower operation.

Touching any sort of metadata is unpredictable and painful because you may end up with a page fault while bringing in the metadata from disk. Accessing metadata significantly increases your working set and can involve a great number of temporary allocations on the garbage collector (GC) heap. Working set refers to the amount of memory your application consumes in its running state. Rico Mariani describes it in his posting "My mom doesn't care about space".

Working set can have a significant impact on your performance. What's even worse is that reflection doesn't have any invariants or policy for accessing metadata or for the impact on the working set that happens with each call. The reason for this undefined performance behavior is that reflection and the runtime have a number of caches in place to reduce the number of times the reflection runtime has to travel to metadata to pick up the bits. At a high level, the two basic caches that reflection may access when it is asked to build a MemberInfo are the metadata cache and the reflection MemberInfo cache. Unfortunately, the working set increase comes not only from pulling in metadata pages, but also from the fundamental design of the reflection MemberInfo cache.

The Reflection MemberInfo Cache

If you want a lesson in bad design, the reflection MemberInfo cache is it. In all releases, this caching solution attempts to reduce the cost of the repeated metadata access that happens when you ask for the same MemberInfo more than once. There are a few surprises associated with the cache design in the .NET Framework 1.x which may negatively impact the working set.

Figure 3 B and D
Figure 3 B and D

First, let's look a little closer at the way this cache works. In .NET Framework 1.x, each type has its own MemberInfo cache. The main entry point for creation of the cache comes from the GetXX APIs in the reflection namespace (I use GetXX as the name of the collective MemberInfo obtainers: GetMethod, GetProperty, GetEvent, and so on). There are two forms of these APIs, the non-plural, which return one MemberInfo (such as GetMethod), and the plural APIs (such as GetMethods), which return an array of members contained in the type. In the .NET Framework 1.x, reflection enforced a policy that MemberInfo objects would be created for all members of a specific kind on both the target type and the type's inheritance hierarchy on calls to either the plural or non-plural APIs. I call this eager caching; regardless of whether you asked for one member or all members, reflection would cache all members by default. For example, if you call GetProperty looking for a particular property on a type, all properties from that type will be cached.

Figure 3 helps illustrate the eager caching algorithm. Here you can see a simple class hierarchy, where type D derives from type B and where each type has a few methods on it. Consider now an application that invokes the following code:

MethodInfo mi = typeof(D).GetMethod("MyMethod");

When this code requesting the MethodInfo for MyMethod on type D is executed for the first time, the .NET Framework 1.x would eagerly create MethodInfos for all methods including non-visible methods on types D and B, and store them into the type's MemberInfo cache. Figure 4 shows the result. Once this cache is populated, subsequent calls to GetMethod and GetMethods on the same type would pull the MethodInfo from the cache, removing the requirement to requery the metadata.

Figure 4 .NET Framework 1.x MemberInfo Cache
Figure 4 .NET Framework 1.x MemberInfo Cache

Unfortunately, types that have a deep inheritance hierarchy and a lot of members are good examples of where this cache design can negatively affect performance. On a single call to GetMethod where the user is requesting one method only, reflection will create the cache and then eagerly spend a lot of time reading metadata and populating the cache for all the other methods, even if they won't be used immediately or ever.Improving Reflection Performance

There are various reasons why an application can't define a static extensibility contract, but it's usually a result of the flexibility the architect built in. Here are some ways to reduce some of the cost of reflection and of the MemberInfo cache.

Avoid using Type.InvokeMember to call late-bound methods Type.InvokeMember is the slowest way to invoke a member via reflection. If your scenario doesn't involve COM interop invocation (something which Type.InvokeMember makes easier), consider other alternatives for setting up calls to members obtained through late-bound methods.

Avoid case-insensitive member lookups When calling both the plural and non-plural GetXX APIs, avoid specifying BindingFlags.IgnoreCase as a binding flag argument as it has a significant invocation and working set costs. Metadata is case sensitive and to perform an "ignore case" operation, reflection needs to obtain the metadata string names for all members of a type and then do a case-insensitive comparison. In addition, due to the design of the reflection cache, a second MemberInfo cache is created for the case-insensitive members, which basically doubles an application's working set.

Use BindingFlags.ExactMatch whenever possible Specify this BindingFlag when calling MethodBase.Invoke if you know that the types of the arguments you are passing exactly match the types of the method's parameters. It will allow reflection to go down an optimized, more efficient code path that does not perform argument type-coercion. If the argument types don't match the method's parameter types, an exception will be thrown.

Call the non-plural GetXX method if you know the name of the instance you care about (.NET Framework 2.0 only) The MemberInfo cache is lazily populated in the .NET Framework 2.0, which means lower working set cost, and less time to retrieve the method. If you know the name of the particular method you want to obtain, use the non-plural GetXX method.

Cache only the handle to a member (.NET Framework 2.0 only) In the .NET Framework 2.0, developers have the ability to define a cache policy themselves. The .NET Framework 2.0 introduces new APIs called the token handle resolution APIs. This set of APIs exposes some of the fundamental member identity concepts of the runtime and allows a user to set up and execute a MemberInfo caching policy on their own.

Another problem with eager caching is the cache growth policy. The .NET Framework 1.x MemberInfo cache is allowed to continue to grow indefinitely, without any reclamation. The more members you reflect on, the bigger the cache grows, even if you're application is finished using reflection. The resulting working set growth can cause problems for all sorts of long-running apps that use reflection heavily. For example, tools such as object browsers that continuously reflect on user-loaded types can be severely affected. The cache, and the working set, can get big fast.

A better cache policy was needed for the .NET Framework 2.0 reflection cache. Specifically, a consumer should be able to call a non-plural GetXX API and not have to worry that the API will create MemberInfos for rest of the members on that type. Plus, the reflection cache growth problem also needed a solution.

The .NET Framework 2.0 delivers a better reflection MemberInfo back-end cache. As a result, there is a better policy around obtaining members as well as a much better plan for growth and reclamation. There are two main changes to the MemberInfo cache policy in the .NET Framework 2.0. First, the cache creation algorithm is lazy so reflection only retrieves and caches data it is asked for. Secondly, the MemberInfo cache for each type has a solution for growth and reclamation. The cache has been rebuilt in managed code (as opposed to unmanaged in the .NET Framework 1.x) and sits in the GC heap, so the user can control cache reclamation. Such user control can significantly reduce the working set of an application.

With the new lazy policy, calling a non-plural GetXX API now does what you would expect—it creates a cache (if a cache for the type's members doesn't already exist) and populates it with just one member. Of course, calling the plural GetXXs API will result in the same eager algorithm described for the .NET Framework 1.x cache because the call is asking for all members of a specific kind on a type. Thus, executing the same code you executed earlier

MethodInfo mi = typeof(D).GetMethod("MyMethod");
will result in the situation depicted in Figure 5 rather than that shown in Figure 4.

Figure 5 .NET Framework 2.0 MemberInfo Cache
Figure 5 .NET Framework 2.0 MemberInfo Cache

Figure 5 shows that, as before, a MethodInfo is allocated in the MemberInfo cache and a reference is created and then returned to the requester of the MethodInfo. This time, however, both are in the GC heap. Separate from the MemberInfo cache is the Type cache, another cache created by the loader and populated every time a type is loaded. This cache holds a weak reference to the relevant MemberInfo cache, so even though the Type cache is never reclaimed, it won't keep a MemberInfo cache alive. It has an insignificant impact on the working set, and you needn't worry about the implications of the Type cache; just remember that it exists.

Now if an application no longer references a MethodInfo and there are no existing references to any other MethodInfos in the relevant MemberInfo cache, the GC will be able to collect and reclaim the memory for that cache. So if you drop all your MemberInfo references, there's a good chance that you'll drop all MemberInfo caches, as well. Notice I said "good chance." There's always a caveat, of course, and here is the one for this situation: the .NET Framework 2.0 MemberInfo cache is process-wide, so another application in the same process (running in a different application domain) with a reference to a MemberInfo on the same type as your application will keep the relevant cache alive. The chance of that happening depends on your particular code base.

For more on tweaking reflection performance, see the sidebar "Improving Reflection Performance."

Handles and Handle-Resolution APIs

A handle is a small, lightweight structure that, in association with a type context, defines the identity of a member. You can think of handles as trimmed down MemberInfos, foregoing most of the methods and data, and without the back-end cache. In the .NET Framework 2.0, reflection offers APIs that get the handle from a MemberInfo and resolve the handle back to the MemberInfo, as shown here:

// Obtaining a Handle from an MemberInfo
RuntimeMethodHandle handle = typeof(D).GetMethod("MyMethod").MethodHandle;

// Resolving the Handle back to the MemberInfo
MethodBase mb = MethodInfo.GetMethodFromHandle(handle);

Figure 6 Costs for Handle Resolution and GetXX Calls
Figure 6 Costs for Handle Resolution and GetXX Calls

Handles don't keep the members in the cache alive but they still represent the identity of a MemberInfo. A user has the ability to set up his own cache based on these two invariants. The cost of going from a handle to a MemberInfo is about the same as using one of the GetXX methods if the appropriate MemberInfo is already in the cache. If the MemberInfo is not in the cache, resolving a Handle to a MemberInfo is approximately twice as fast as using one of the GetXX methods. You'll get consistent results, you won't need to worry about whether the cache is alive, and you'll keep working set down. Figure 6 compares the differences between methods.

Implementing Your Own Cache

Given the efficiency of handles, you might consider implementing your own cache on top of the one that exists in the Framework. But good cache design is hard, as Rico Mariani points out in his blog "Caching Implies Policy." If you find it difficult to set the policy of your cache consider avoiding the pain of implementing one. Rico's cache policy posting is a good read if you're thinking about developing your own caching strategy.

If you want to try anyway, a quick and easy exercise is to cache a call to GetMethod. For that API, you need a System.Type, the string-based method name, and an optional type array for the method's parameters (since overloading can introduce multiple methods with the same name on the same type). The obvious trick is to make the key to your cache unique. Here's a cache key implementation for all GetMethod call sites:

class CacheKey
{
    public CacheKey(RuntimeTypeHandle th, string methodName, Type[] args)
    {
        InstanceHandle = th;
        MethodName = methodName;
        TypeArguments = args;
    }
    ...
}

This key can then be used to store the associated MethodHandle. Whenever a call to GetMethod is required, you can hit your cache with the RuntimeTypeHandle of the type, the string-based method name, and the array of type parameters, and then get back the appropriate MethodHandle. A call to MethodInfo.ResolveMethodFromHandle will quickly retrieve a MethodInfo from that MethodHandle, ready for use.

If you get a cache miss on your own cache, you'll need to call GetMethod anyway to obtain the handle from the resulting MethodInfo. After making use of the MethodInfo, you'll end up throwing the MethodInfo away, keeping only the handle and hoping that the GC soon reclaims the MemberInfo cache that was temporarily created. A cache hit, however, never results in a call to GetMethod. You can go directly from the handle to the MethodInfo, perform whatever operations you like, and then drop the MethodInfo again.

After you've retrieved the appropriate MemberInfo, you'll need to be able to invoke it. If your scenario requires that you call Invoke on a MemberInfo multiple times and you can't define a static contract for some reason, you can consider code generation to remove the cost of the Invoke method call. To recap, a call to Invoke involves a number of security checks, type parameter checks, and walks along metadata, all of which can become expensive quickly. It doesn't make sense to repeat these checks if you are calling the same method multiple times. Enter the new Lightweight Code Generation (LCG) feature in the .NET Framework 2.0. This feature bridges the gap between purely dynamic invocations and early-bound calls. It hangs off of the System.Reflection.Emit namespace and provides the ability to generate new methods at run time. These dynamic methods are fully reclaimable by the GC, which is important for control over an application's working set. The LCG feature uses the existing Reflection.Emit.ILGenerator class and a new DynamicMethod class to emit IL for a new method, and it provides two ways to perform invocation, through delegates and through DynamicMethod.Invoke.

As an introductory example to using DynamicMethod, consider the following code with generates a method that displays "Hello World" to the console:

RuntimeMethodHandle myMethodHandle =
    typeof(Console).GetMethod("WriteLine"),
    new Type[]{typeof(String)})).MethodHandle;
DynamicMethod dm = new DynamicMethod(
    "HelloWorld",          // name of the method
    typeof(void),          // return type of the method
    new Type[]{},          // argument types for the method
    typeof(LCGHelloWorld), // type in module with which to associate
    false);                // skip JIT visibility checks
ILGenerator il = dm.GetILGenerator();
il.Emit(OpCodes.Ldstr, "Hello, world");
il.Emit(OpCodes.Call, myMethodHandle);
il.Emit(OpCodes.Ret);

There are two ways to invoke this LCG method: using DynamicMethod.Invoke, or DynamicMethod.CreateDelegate. Delegate invocation is the faster, recommended approach:

delegate void MyHelloWorldDelegate();
...
MyHelloWorldDelegate del = (MyHelloWorldDelegate)
    dm.CreateDelegate(typeof(MyHelloWorldDelegate));
...
del();

With that brief overview in hand, let's take a step back and consider the problem we're trying to solve with code generation.

When a method is not known at compile time, you cannot use the extremely fast call instructions because the compiler is unaware of how to set up the CLR for dynamic call sites. Here's an example where a compiler has all of the information it needs:

public static int GetEmployeeIdNumber(string employeeName) { ... }

// callsite
int empNum = GetEmployeeIdNumber("Joel");

At compile time, the compiler will emit IL to set up the stack for this call site. Everything about this particular call site is well known to the compiler. It can check that the types of the arguments and the types of the parameters match, and it can make sure that the method is "visible" to the call site (in this case, it is because the method is public). After these checks, the compiler can safely emit IL that pushes the string "Joel" onto the stack and emits a "call method" instruction to invoke the method. The IL for this call site is illustrated here:

  ldstr      "Joel"
  call       int32 EmployeeData::GetEmployeeIdNumber(string)
  stloc.0

Contrary to this example, most uses of reflection invocation are purely dynamic. You can use the reflection APIs to invoke methods in a run-time world where you don't know or have compile-time access to the signature or location of the method that you're trying to invoke. Unfortunately, you pay the price at invocation time because reflection has to do a lot of these checks at run time while a compiler would do them at compile time.

You may be asking why reflection doesn't simply perform these checks once. Reflection and its various Invoke methods repeat these checks every time you invoke, even if the arguments are the same. This is done for a variety of reasons, including security. LCG can be used to bridge the gap between the compiler and reflection for invocations over a particular method. Using LCG, you can bypass the checks that reflection performs each time you invoke if you know that the method you are calling is safe.

For LCG to bridge between a call site and a method, you'll first need to define a delegate that can be used statically in code to reference the LCG method. For the cases where the extensibility point's method is totally unknown, you'll need to make a delegate signature that is generic enough to apply to all method signatures. This is fairly easy since the delegate signature can return an object and take an object array:

delegate object CallSiteDelegate(object[] args);

Now you need to set up the LCG method to match this generic signature. The LCG MethodWrapper method returns an object, and takes an object array as its parameters:

DynamicMethod dm = new DynamicMethod("MethodWrapper", typeof(object),
    new Type[] { typeof(object[]) }, ..., ...);

The code for building the LCG method is complicated and is beyond the scope of this article. As a summary though, the method needs to perform several operations before finally calling the intended method. It must check that there are enough arguments in the object array that is supplied to the LCG wrapper method. It then must pull out the instance or "this" pointer if the method being called is an instance method. Next, it unpacks the argument array and lays out the arguments on the stack, after which it calls the method and returns its result.

The most interesting part of the LCG method wrapper is the last few IL instructions to do the call once the arguments are laid out on the stack. The instructions end up being a straight call or callvirt. This is the basic building block for bridging late-bound and early-bound. Looking at Figure 2, you can see that these IL instruction calls will be super fast.

Finally, calling the dynamically generated method is done via the CallSiteDelegate delegate, with arguments supplied in the object array. For GetEmployeeNumberId it looks like the following:

DynamicMethod dm = ...;
CallSiteDelegate getEmployeeIdMethod = (CallSiteDelegate)
    dm.CreateDelegate(typeof(CallSiteDelegate));
getEmployeeIdMethod(new object[] { "Joel" });
The performance of the whole invocation of the method will be slightly slower than a delegate call plus a call or callvirt instruction.

Wrapping It Up

Here I've tried to provide a clear explanation of how and when to use reflection and what it looks like under the covers, along with some tricks and tips on how to get the most out of it. I'm hoping that I've delivered enough information to get you thinking about your reflection scenarios, and enough depth for you to make some solid decisions about your designs.

Joel Pobar is a Program Manager on the common language runtime (CLR) team at Microsoft. He primarily works on the more dynamic features of the CLR. You can reach him at joelpob@microsoft.com.


Visual Basic: Simplify Common Tasks by Customizing the My Namespace

Visual Basic

Simplify Common Tasks by Customizing the My Namespace

Joe Binder
This article is based on a prerelease version of Visual Basic 2005. All information contained herein is subject to change.
This article discusses:
  • Customizing and configuring My.Settings, My.Application.Log, My.Computer.Network, and more
  • Techniques and guidelines for extending the My namespace
This article uses the following technologies:
Visual Basic 2005
Code download available at: My.exe (177KB)
Contents
Configuring Existing My Namespace Members
Customizing My.Application.Log
Configuring My.Computer.Network
Customizing My.Settings
Extending the My Namespace
Adding Members
Extending the My.Application Event Model
The MyType
Redefining My.User
My Namespace Customizations
Design Guidelines
Designing Class Libraries for My
Packaging and Deploying Extensions
Conclusion

T
he My namespace in Visual Basic® 2005 is designed to help you easily write compelling applications. Its task-based APIs, intuitive hierarchy, and application framework allow you to harness the power of the Microsoft® .NET Framework often with only a single line of code to complete a difficult task. Underlying the My namespace's APIs is a fully extensible architecture you can leverage to customize the behavior of My and to add new services to its hierarchy to adapt to specific application needs.

The My namespace exposes properties and methods that "just work." By solving common programming problems, its solution-driven hierarchy and preconfigured services alleviate the need to navigate the entire .NET Framework. Unlike many of the types and members of the .NET Framework, the My namespace is optimized to provide simple solutions to common problems. However, you needn't abandon the simplicity that the My namespace provides because your requirements are different. Using the My namespace's customization model, you can modify the behavior of individual My namespace members to meet specific application requirements.

Configuring Existing My Namespace Members

One of the biggest problems developers face when using general-purpose infrastructures is configuration. For this reason, the My namespace's members contain a default configuration that is optimized to accommodate the most common scenario.

For example, the System.Security namespace contains a powerful, highly extensible infrastructure for performing role-based security. However, performing a common task, such as configuring the required classes to run as the current Windows® user, can be a bit complicated, as you can see:

System.Threading.Thread.CurrentPrincipal = _
    New System.Security.Principal.WindowsPrincipal( _
        System.Security.Principal.WindowsIdentity.GetCurrent)

My.User encapsulates the Windows user by default so this initialization code is unnecessary. Like many members in the My namespace, however, the default configuration of My.User is not hardwired and can be overridden as needed:

My.User.CurrentPrincipal = _
    New WindowsPrincipal(WindowsIdentity.GetAnonymous())

Setting the CurrentPrincipal property on My.User effectively changes the user that the application will run as. My.User, in turn, returns information about the user specified.

My.User is a logical encapsulation of the current logged on user. By default, My.User is initialized to the current Windows user. However, running as the Windows user is not always appropriate: it is sometimes necessary to rely on an external infrastructure such as a SQL database to retrieve user information. Using the System.Security.Principal infrastructure, My.User can be reconfigured to use IPrincipal and IIdentity implementations designed to interact with alternative authentication and authorization models:

My.User.CurrentPrincipal = CustomPrincipal

Once My.User.CurrentPrincipal has been set, subsequent calls to My.User will query the custom implementation for the user name, user's roles, and so on. Although designing custom IPrincipal and IIdentity implementations is beyond the scope of this article, you can find more information at Authentication and Authorization.

Customizing My.Application.Log

Many applications require detailed tracing information to be maintained for health monitoring and audit purposes. My.Application.Log provides straightforward methods for writing categorized trace information to a log file:

Private Sub MyApplication_UnhandledException( _
        ByVal sender As Object, ByVal e As UnhandledExceptionEventArgs) _
        Handles Me.UnhandledException
    My.Application.Log.WriteException(e, TraceEventType.Critical, _
        "The application has crashed due to an unhandled exception")
 End Sub

Because My.Application.Log is built atop the System.Diagnostics.TraceSource infrastructure, you can alter its default configuration to route and filter trace messages as needed using a .config file. For example, it's often necessary to write critical error messages to the Application Event Log so system administrators can identify application failures. Adding the code in Figure 1 to the App.config file ensures that critical messages will be routed to the Application Event Log while messages of all severities will continue to be routed to a file.

Figure 1 Routing Trace Messages (Close Figure 1)
<system.diagnostics>
    <sources>
        <!-- The DefaultSource is used by My.Application.Log
             to write messages. -->
        <source name="DefaultSource" switchName="DefaultSwitch">

            <!-- The listeners section defines where messages
                 will be written.  Here I'm writing to a file
                 and the event log -->
            <listeners>
                <add name="FileLog"/>
                <add name="EventLog"/>
            </listeners>

        </source>
    </sources>

    <!-- The switches determine when the TraceSource is
         enabled/disabled.  Here I'm stating that
         My.Application.Log is enabled for all messages
         with a severity greater than or equal to Information. -->
    <switches>
        <add name="DefaultSwitch" value="Information" />
    </switches>

    <!-- Listeners are responsible for writing the log messages. -->
    <sharedListeners>
        <add name="FileLog" type=
            "Microsoft.VisualBasic.Logging.FileLogTraceListener"
            initializeData="FileLogWriter"/>
        <add name="EventLog" type=
            "System.Diagnostics.EventLogTraceListener"
            initializeData="My Customization Example">

            <!-- By providing a filter I can ensure that only
                 a subset of messages will be written by the listener.
                 Here I'm specifying that only Critical messages are
                 written to the Event Log.-->
            <filter type="System.Diagnostics.EventTypeFilter"
                initializeData="Critical"/>

        </add>
    </sharedListeners>
</system.diagnostics>

Since My.Application.Log is driven by a .config file, it has the added benefit of being modifiable after the application has been deployed because the application does not need to be recompiled.

Configuring My.Computer.Network

My.Computer.Network provides straightforward methods for determining network availability as well as uploading and downloading files, as shown here:

If My.Computer.Network.IsAvailable Then
    My.Computer.Network.DownloadFile( _
        "http://myserver.com/log.txt", _
        "c:\archives\serverlog.txt")
End If
It is built using the System.Net classes and inherits their runtime configuration characteristics. For example, administrators may add the following section to a machine or application configuration file to use the default system proxy for all network requests:
<system.net>
    <defaultProxy>
        <proxy usesystemdefault="true"/>
    </defaultProxy>
</system.net>
My.Computer.Network, in turn, will automatically detect these settings and use them for network requests. For more information on System.Net configuration, see <system.net> Element.

Customizing My.Settings

My.Settings encapsulates a powerful infrastructure for manipulating application settings, such as Web service URLs and user preferences. It enables you to add and remove settings at design time and to retrieve those settings at run time without writing error-prone, often tedious code to locate, parse, and update configuration (.config) files. (For more information on My.Settings, see Using My.Settings in Visual Basic 2005). Because many applications require settings to be persisted in alternative locations (such as the Windows registry or .ini files) you can implement custom logic that tells how and where My.Settings should read and write settings.

The My.Settings configuration model employs the Provider Design Pattern, which is used extensively in ASP.NET 2.0. (See Provider Model Design Pattern and Specification, Part 1 for more information on the Provider Pattern). The Settings Provider performs the logic for reading and writing settings using a specific medium, such as a Web service or the Windows registry. A Settings Provider can be specified for individual settings or all settings in My.Settings.

Within Visual Studio® 2005, you can specify Settings Providers using the property grid for the setting. When specifying the provider, the following format is used: Namespace.Type. Assembly. As an example, Figure 2 shows how to customize My.Settings to use the RegistrySettingsProvider that is included in the source code for this article.

Figure 2 Using the RegistrySettingsProvider (Close Figure 2)

Extending the My Namespace

The My namespace exposes solutions for many common programming tasks that surface in application development, but if you're working in a specific domain you can encounter problems that the My namespace does not solve. For example, My.Computer contains support for reading and writing to serial ports but it does not support USB ports; My.Application includes tracing functionality but it doesn't expose crash reporting services; Data intensive applications may benefit from a top-level "My.Data" node for performing common data tasks. Using the extensibility model included in the My namespace, you can add new members to the My namespace to meet growing application needs. Deploying My namespace extensions to other developers working in the same domain allows them to use the extensions as easily as the other members of the My namespace.

You might be asking why you should extend at all. To date, developers have relied on external class libraries to solve specific problems, so why write the code if someone else has already done it? Often times developers will use third-party and community-owned class libraries, which involves a certain learning curve. What problem does the library solve? How is it intended to be used? And how do I actually use it? These are the sorts of questions you grapple with. The existences of varying design standards only complicates the learning process. The My namespace's extension model seeks to add a level of consistency and simplicity to the process of sharing solutions among developers. Following its straightforward design guidelines, you can expose solutions in a manner that ensures consumers of the solution will understand its intended usage pattern and that developers of all experience levels can be productive with the solution.

Unlike other .NET Framework namespaces, which reside in external assemblies (DLLs), the My namespace actually resides below the root namespace of the Visual Basic project in an in-memory file. Figure 3 shows some contents of the My namespace.

Figure 3 My Namespace Overview (Close Figure 3)

WindowsApplication1
Namespace My
    Module MyProject
        Public ReadOnly Property Application() As MyApplication
        Public ReadOnly Property Computer() As MyComputer
        Public ReadOnly Property User() As User
        ...
    End Module

    Partial Class MyApplication
        Inherits ApplicationServices.WindowsFormsApplicationBase
    End Class
    Partial Class My Computer
        Inherits Devices.Computer
    End Class
End Namespace
Microsoft.VisualBasic.dll
Namespace Devices
    Public Class Computer
        Public ReadOnly Property Ports() As Ports
        ...
    End Class
End Namespace

Namespace ApplicationServices
    Public Class WindowsFormsApplicationBase
        Public ReadOnly Property Info() As AssemblyInfo
        ...
    End Class
End Namespace

The Application, Computer, and User property definitions are manifested as My.Application, My.Computer, and My.User, respectively. Interestingly, My.Application and My.Computer actually return instances of the MyApplication and MyComputer classes, which are also defined within the My namespace. These classes, and the fact that they're declared with the Partial keyword, are essential to the extension of My.Application and My.Computer.

Adding Members

Because My is just a namespace, adding top-level properties to it is relatively straightforward. Start by adding a module to the project and enclosing the module within the My namespace. Next, annotate the module with the HideModuleName attribute. The HideModuleName attribute tells IntelliSense to hide the module name when users navigate the My namespace.

Finally, add the properties as appropriate to the module and for each property added to the My namespace, add a backing field of type ThreadSafeObjectProvider(Of T). This field is used to create thread-safe instances inside the property, such that each thread that is accessing the extended property receives its own instance of the returned type. The following example adds a SampleExtension extension to the My namespace:

Namespace My
    <HideModuleName()> _
    Module MyProject
        Private m_extension As _
            New ThreadSafeObjectProvider(Of SampleExtension)
        Public ReadOnly Property SampleExtension() As SampleExtension
            Get
                Return m_extension.GetInstance()
            End Get
        End Property
    End Module
End Namespace

Because the types returned from My.Application and My.Computer are defined as Partial in the project, extending My.Application and My.Computer is straightforward. The process for extending My.Computer is as follows. First, add a Friend class named MyComputer and mark the class with the Partial keyword. Enclose the MyComputer class in the My namespace and then add properties and methods to the MyComputer class. As properties and methods are added they are immediately available under My.Computer:

Imports System.Net.NetworkInformation
Namespace My
    Partial Class MyComputer
        Public ReadOnly Property IPAddresses() As IPAddressCollection
            ...
        End Property
    End Class
End Namespace
Extending My.Application follows the same process. The only difference is that the Partial class name is MyApplication.

Extending the My.Application Event Model

My.Application includes a number of events in Windows Forms projects that encapsulate changes to the overall application state, such as application startup and shutdown, changes to network connectivity, and application crashes. Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase, the base class from which My.Application derives, includes entry points for extensibility that you can use to perform work before and after these events fire. Here is how these methods are invoked:

  1. OnInitialize. The first entry point to be called. Any preprocessing of command-line arguments can be done here. If OnInitialize returns False, the event sequence will not continue and the application will exit.
  2. OnCreateSplashScreen. Called after the initialization work is complete. This method is where the splash screen for the application is set.
  3. OnStartup. Called just before the Startup event is fired. Returning false from this method cancels the event sequence and exits the application.
  4. OnRun. Called after the startup event has been handled and before the main form is shown.
  5. OnCreateMainForm. The last of the startup events to be fired. It's where the MainForm property on My.Application is set.
  6. OnShutdown. Called just before the Shutdown event is fired.

Figure 4 demonstrates how to extend the My.Application event model by adding logic to ensure that the splash screen is shown for the time specified as a command-line argument.

Figure 4 Extending the MyApplication Event Model (Close Figure 4)
Class MyApplication
    Private m_splashTimeout As Integer = 2000
    Private m_timerThread As Thread

    ''' <summary>
    ''' Searches the command-line arguments for a splash screen
    ''' timeout.
    ''' </summary>
    Protected Overrides Function OnInitialize( _
            ByVal commandLineArgs As ReadOnlyCollection(Of String)) _
            As Boolean
        For Each argument As String In commandLineArgs
            If (argument.Contains("/splashtimeout")) Then
                m_splashTimeout = Integer.Parse( _
                   argument.Split(":")(1)) * 1000
            End If
            m_timerThread = New Thread(AddressOf RunTimer)
            m_timerThread.IsBackground = True
            m_timerThread.Start()
        Next
        Return MyBase.OnInitialize(commandLineArgs)
    End Function

    ''' <summary>
    ''' Before showing the main form, make sure that the splash
    ''' screen has shown for the requested length of time.
    ''' </summary>
    ''' <remarks></remarks>
    Protected Overrides Sub OnRun()
        If SplashScreen IsNot Nothing AndAlso _
           SplashScreen.Visible = True AndAlso _
           m_timerThread IsNot Nothing AndAlso _
           m_timerThread.IsAlive Then
               m_timerThread.Join(m_splashTimeout)
        End If
        MyBase.OnRun()
    End Sub

    ''' <summary>
    ''' This method executes for roughly m_splashTimeout.
    ''' </summary>
    Private Sub RunTimer()
        Dim runningTime As Integer = 0
        While runningTime < m_splashTimeout
            Thread.Sleep(100)
            runningTime += 100
        End While
    End Sub
End Class

In addition to adding top-level members to the My namespace, members can also be removed and redefined as needed. For example, if a specific application domain requires an application object other than the default Microsoft.VisualBasic.ApplicationServices.ApplicationBase, My.Application can be redefined to return the more appropriate application object. Similarly, members such as My.WebServices can be removed from the My namespace to accommodate scenarios in which Web services are not used.

The MyType

The MyType construct tells the Visual Basic compiler which top-level members should appear for a given project type. For example, My.Forms is not very useful in a Windows Service project so it is omitted from the My namespace. Extenders of the My namespace can use MyType to reshape the My namespace in order to provide a highly customized experience for their consumers.

MyType is effectively a predefined compiler definition that is used to conditionally compile members of the My namespace. As an example, consider the following code, which the compiler uses to construct the My namespace in Windows Forms projects:

#If _MyType = "WindowsForms" Then
    'My.Forms will be included in the My Namespace
#Const _MYFORMS = True
    'My.Application will include Windows Forms specific
    'events and members
#Const _MYAPPLICATION = "WindowsForms"
    'My.Computer will contain members for Windows client applications
#Const _MYCOMPUTER = "Windows"
    'My.User will return the Windows user
#Const _MYUSER = "Windows"
#ElseIf _MyType = "Console" Then
    '...
#End If

When a custom MyType value is provided to the Visual Basic compiler, the My namespace is effectively empty and individual members must be added back as needed. Adding new members to the top-level of the My namespace follows the process outlined in the section on adding members to the top-level My namespace; predefined, top-level members, such as My.Forms, can be added back to the My namespace using compiler constants.

Each predefined, top-level member of the My namespace has a compiler constant associated it. For example, a Boolean compilation constant called _MYFORMS is used to specify whether or not "Forms" should be included in the My namespace. Each predefined, top-level member of the My namespace and its associated compilation constant is described in Figure 5.

Figure 5 My Namespace Compilation Constants (Close Figure 5)

Member Name | Compilation Constant | Available Values | Returns
My.Application | _MYAPPLICATIONTYPE | Windows | ApplicationServices.ApplicationBase
 |  | Console | ApplicationServices.ConsoleApplicationBase
 |  | WindowsForms | ApplicationServices.WindowsFormsApplicationBase
My.Computer | _MYCOMPUTERTYPE | Windows | Devices.Computer
 |  | Web | Devices.ServerComputer
My.Forms | _MYFORMS | True | My.Forms is included
 |  | False | My.Forms is excluded
My.User | _MYUSERTYPE | Windows | ApplicationServices.User
 |  | Web | ApplicationServices.WebUser
My.WebServices | _MYWEBSERVICES | True | My.WebServices is included
 |  | False | My.WebServices is excluded

Because predefined, top-level members of the My namespace are configured using compilation constants, their values must be provided to the Visual Basic compiler via command-line arguments. Within Visual Studio, compiler constants are specified on the Compile page of the Project Designer.

Redefining My.User

To demonstrate the process for defining a custom MyType, I'll configure My.User such that it returns a custom type. The custom type, ExtendedUser, contains utility methods called Login and Logout for performing authentication. Start by creating a class library and then add the following class to it:

Imports System.Security
Public Class ExtendedUser
    Inherits Microsoft.VisualBasic.ApplicationServices.User

    Public Function Login(ByVal username As String, _
                          ByVal password As SecureString) As Boolean
        'TODO: Perform login logic
    End Function

    Public Sub Logout()
        'TODO: Perform logout logic
    End Sub
End Class

Next, create a Windows Forms Project and add a reference to the Class Library in the Windows Forms Project. Add the following custom definition for My.User:

#If _MyType = "Custom" Then
Namespace My
    <HideModuleName()> _
    Module MyExtensionModule
        Private m_userInstanceProvider As New _
            ThreadSafeObjectProvider(Of ExtendedUser)
        Public ReadOnly Property User() As ExtendedUser
           Get
               Return m_userInstanceProvider.GetInstance()
           End Get
        End Property
    End Module
End Namespace
#End If

Set _MyType="Custom" and include the predefined, top-level members of the My namespace using command-line arguments to the compiler. To include all predefined members except My.User (as just defined), here are the command-line arguments:

MyType="Custom", _MYAPPLICATIONTYPE="WindowsForms",
_MyType
_MYCOMPUTERTYPE="Windows", _MYFORMS=True, MYWEBSERVICES=True
Once you have completed these steps, My.User will return ExtendedUser, wherever it used in the Windows Forms Project.

My Namespace Customizations

When you're considering strategies for designing, deploying, and maintaining your My namespace extensions, it is easiest to divide the problem into two pieces: the My namespace extension and the class library. The My namespace extension is the code that is going to be used to expose your class library in My, as was demonstrated earlier. The class library is the assembly (DLL) that contains the types that will be exposed in the extensions.

Design Guidelines

The following guidelines will help minimize the maintenance costs for My namespace extension components.

Only include the extension logic The logic included in the My namespace extension itself should only include the plumbing that will be used to expose functionality in the My namespace. Because this component resides in consumers' projects as source code, updating the extension component is very expensive and should be avoided when possible.

Minimize project assumptions Assuming a set of references, project-level Imports, or specific compiler settings (for example, Option Strict is off) is generally unwise. Instead, minimize dependencies, fully qualify all type references using the Global keyword, and ensure the extension compiles with Option Strict to minimize errors in the extension.

Isolate the extension code Placing all of the My namespace extension code in a single file or a separate folder in the project will help users isolate the My namespace extension.

Designing Class Libraries for My

Like most object models, some design patterns work well in the My namespace and others do not. When designing an extension to the My namespace, consider the following principles:

Stateless APIs Methods in the My namespace should provide an end-to-end solution to a specific problem. The parameter values provide the only input required to complete the task. Methods that rely on previous state tend to be confusing in the context of My. My.Computer.FileSystem includes good examples of stateless APIs that work well in the context of My.

Global instances The only state that is maintained within the My namespace is global to the project. For example, My.Application.Info encapsulates state that is shared through the app.

Simple parameter types In general, the easiest methods to call do not require complex types as parameters but instead take Strings, primitive types, and so on.

Factory methods Some types are necessarily difficult to instantiate. Providing factory methods as extensions to the My namespace enables you to more easily discover and consume types that fall into this category. An example of a factory method that works well is My.Computer.FileSystem.OpenTextFileReader. There are several Stream types, but this method helps demystify which Stream to use and how to create it.

These guidelines do not preclude general design principles for class libraries. Rather, they are recommendations that are optimized for developers using Visual Basic and the My namespace. A discussion of the .NET Framework Class Design Guidelines is available at Designing .NET Class Libraries.

Packaging and Deploying Extensions

The process for packaging and deploying My namespace extensions is similar to deploying a standard class library or third-party controls. The only other step is to provide a straightforward mechanism for adding the extension to each project. Visual Studio 2005 includes an improved template format that is well-suited to packaging and distributing My namespace extensions.

A Visual Studio Template (.vstemplate) file encapsulates content that can either be used as a baseline for new projects (a project template) or added to an existing project (an item template). Packaging My namespace extensions as template files integrates the extension seamlessly into Visual Studio, allowing consumers to add them to their projects using the Add New Item and New Project dialogs. A benefit of this approach is that Visual Studio templates can be installed per-user or per-machine. A detailed example of a My namespace extension packaged as a Visual Studio Item template is included in the code download.

Conclusion

The My namespace includes a full-featured set of members and services designed to make the most common programming scenarios the easiest to implement. Although extensive, the core set of APIs included in the My namespace is merely the starting point for future innovation. Its extensibility model enables you to create, share, and continually evolve the My namespace as application requirements change and new solutions are discovered.

Joe Binder is a program manager on the Visual Basic team at Microsoft where he works on the My namespace, the Visual Basic runtime, and the Visual Basic Application Framework. He can be reached at jbinder@microsoft.com.


Behind the Scenes: Discover the Design Patterns You're Already Using in the .NET Framework

Behind the Scenes

Discover the Design Patterns You're Already Using in the .NET Framework

Rob Pierry
This article discusses:
  • Common design patterns used in .NET Framework classes
  • Patterns used to implement the ASP.NET programming model and request pipeline
  • How patterns make programming tasks faster and easier
This article uses the following technologies:
.NET Framework and ASP.NET
Code download available at: DesignPatterns.exe (122KB)
Contents
Observer Pattern
Iterator Pattern
Decorator Pattern
Adapter Pattern
Factory Pattern
Strategy Pattern
Composite Pattern in ASP.NET
Template Method Pattern
Patterns in the ASP.NET Pipeline
Intercepting Filter Pattern
Page Controller Pattern
Other Web Presentation Patterns in ASP.NET
Conclusion

R
ecently, Microsoft has placed increasing emphasis on design patterns. If you are unfamiliar with patterns, suddenly being inundated with new terms and foreign-looking UML diagrams can be overwhelming. This emphasis on patterns, however, doesn't represent a shift in methodology so much as a change in vocabulary. The Microsoft® .NET Framework base class library (BCL) already makes extensive use of patterns, and you are probably familiar with the most common ones, even though you might not realize it yet.

In this article, I'll cover a basic overview of several common design patterns and how they are used in the BCL and other areas of the .NET Framework. In doing so, you can discover some of the motivation for why the Framework is designed the way it is, as well as make the abstract concepts of the patterns themselves more intuitively understandable.

Most of the patterns I'll be covering come from the canonical reference, Design Patterns by Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides, (Addison-Wesley, 1995). These authors are collectively known as the Gang of Four. The Gang of Four didn't invent these patterns, but they documented and formalized the good work others had been doing since the beginning of software development.

If you are already familiar with some of these patterns, feel free to read about those you aren't familiar with, since each section is relatively independent. The section on ASP.NET-related patterns requires familiarity with the request pipeline, a basic overview of which is provided in that section.

Within the .NET Framework, the use of some patterns is so prevalent that they have been built into programming languages themselves, instead of just represented by the class libraries. The first two patterns I will discuss, the Observer and the Iterator, are supported by various language features of both C# and Visual Basic® .NET. As you will see, they are an integral part of many common programming tasks.

Observer Pattern

Good object-oriented design emphasizes both encapsulation and loose coupling. In other words, classes should keep internal details private and also minimize their strict dependencies. In most applications, classes don't work in isolation; they interact with many other classes. A common scenario of class interaction occurs when one class (the Observer) needs to be notified when something changes in another (the Subject). For example, several Windows® Forms controls might need to update their display after a button is clicked. A simple solution would be to have the Subject call a specific method of the Observer whenever a change in state occurs. This introduces a host of problems, however. Now, since the Subject needs to know which method to call, it is tightly coupled to that specific Observer. Furthermore, if you need to add more than one Observer, you have to continue to add code for each method call to the Subject. If the number of Observers changes dynamically, this gets even more complex. You'll quickly end up with a brittle mess that's difficult to maintain.

Applying the Observer pattern helps to resolve this problem efficiently. You can decouple the Subject from the Observers so that Observers of any variety can easily be added and removed, at both design time and run time. The Subject maintains a list of interested Observers. Each time the Subject's state changes, it calls the Notify method on each Observer. Figure 1 shows a sample implementation. All classes designed to act as Observers implement the ICanonicalObserver interface, and all Subjects must derive from CanonicalSubjectBase. If a new Observer wants to monitor a Subject, the Add method easily handles this without having to change any code in the Subject class. Note also that each Subject only directly depends on the ICanonicalObserver interface, not any specific Observer.

Figure 1 Observer Implementation (Close Figure 1)
public abstract class CanonicalSubjectBase
{
    private ArrayList _observers = new ArrayList();

    public void Add(ICanonicalObserver o)
    {
        _observers.Add(o);
    }

    public void Remove(ICanonicalObserver o)
    {
        _observers.Remove(o);
    }

    public void Notify()
    {
        foreach(ICanonicalObserver o in _observers)
        {
            o.Notify();
        }
    }
}

public interface ICanonicalObserver
{
    void Notify();
}

While the Gang of Four's Observer pattern solves some of these problems, there are still some roadblocks, since subjects must inherit from a specific base class and Observers must implement a special interface. Thinking back to the Windows Forms button example, a solution emerges. The .NET Framework introduces delegates and events to solve these problems. If you have done any programming for ASP.NET or Windows Forms, then you have probably worked with events and event handlers. Events act as the Subject, while delegates act as Observers. Figure 2 shows an example of the Observer pattern, making use of events.

Figure 2 Observer Pattern Using Events and Delegates (Close Figure 2)
public delegate void Event1Hander();

public delegate void Event2Handler(int a);

public class Subject
{
    public Subject(){}

    public Event1Hander Event1;
    public Event2Handler Event2;

    public void RaiseEvent1()
    {
        Event1Handler ev = Event1;
        if (ev != null) ev();
    }

    public void RaiseEvent2()
    {
        Event2Handler ev = Event2;
        if (ev != null) ev(6);
    }
}

public class Observer1
{
    public Observer1(Subject s)
    {
        s.Event1 += new Event1Hander(HandleEvent1);
        s.Event2 += new Event2Handler(HandleEvent2);
    }

    public void HandleEvent1()
    {
        Console.WriteLine("Observer 1 - Event 1");
    }

    public void HandleEvent2(int a)
    {
        Console.WriteLine("Observer 1 - Event 2");
    }
}

The Windows Forms Button control exposes a Click event which gets raised whenever the button is clicked. Any class designed to react to this event just needs to register a delegate with that event. The Button class doesn't depend on any of the potential Observers, and each Observer only needs to know the correct type of the delegate for the event (EventHandler, in this case). Since EventHandler is a delegate type and not an interface, each Observer doesn't need to implement an extra interface. Assuming it already contains a method with a compatible signature, it only needs to register that method with the event of the Subject. Through delegates and events, the Observer pattern lets you decouple Subjects and their Observers.

Iterator Pattern

Many programming tasks involve manipulating collections of objects. Whether these collections are simple lists or something more complex such as binary trees, you'll often need some way to get access to each object in the collection. In fact, depending on the collection, you may want several ways to access each object such as front to back, back to front, preorder or postorder. To keep the collection simple, the traversal code itself is often in its own class.

One of the most basic ways to store a list of objects is in an array. Array types are built into both Visual Basic .NET and C#. Both languages also have a loop structure that aids in iterating over arrays: foreach in C# and For Each in Visual Basic .NET. Here's an easy example of iterating over arrays:

int[] values = new int[] {1, 2, 3, 4, 5};
foreach(int i in values)
{
    Console.Write(i.ToString() + " ");
}
These statements make use of the iterator for the array behind the scenes. All you need to know is that you are guaranteed to have the loop run exactly once for each item in the array.

To make those statements work, the object referenced in the In expression must implement IEnumerable. Any collection of objects that implements the IEnumerable interface can be traversed (enumerated). This interface has a single method, GetEnumerator, that returns an object which implements IEnumerator. The IEnumerator class contains the code necessary to iterate through the collection. It has a property for the current object (Current) and methods to advance to the next object as well as start over (MoveNext, and Reset). All of the collection classes in the System.Collections namespace, as well as arrays, implement IEnumerable and can therefore be iterated over.

If you examine the Microsoft intermediate language (MSIL) generated by the C# compiler for any code that uses foreach, you can see that in most cases it just uses the IEnumerator to do the iteration (certain types, such as arrays and strings, are special cased by the compiler). Here you can see the IEnumerator approach for iterating over the same array shown previously:

int[] values = new int[] {1, 2, 3, 4, 5};
IEnumerator e = ((IEnumerable)values).GetEnumerator();
while(e.MoveNext())
{
    Console.Write(e.Current.ToString() + " ");
}

The .NET Framework uses the IEnumerable and IEnumerator interfaces to implement the Iterator pattern. The Iterator pattern lets you easily traverse a collection without exposing the inner workings of that collection. An Iterator class, an implementer of IEnumerator, is a separate class from the collection, which implements IEnumerable. The Iterator class maintains the state of the traversal (including what the current item is and whether or not there are more items to be traversed) outside of the collection itself. The algorithm for the traversal is contained in the Iterator as well. This way you can simultaneously have multiple Iterators, each traversing the same collection in wildly different ways, without adding any complexity to the collection class itself.

Decorator Pattern

Any useful executable program involves either reading input, writing output, or both. Regardless of the source of the data being read or written, it can be treated abstractly as a sequence of bytes. .NET uses the System.IO.Stream class to represent this abstraction. Whether the data involves characters in a text file, TCP/IP network traffic, or something else entirely, chances are you will have access to it via a Stream. Since the class for working with file data (FileStream) and the class for working with network traffic (NetworkStream) both inherit from Stream, you can easily write code that processes the data independent of its origins. Here's a method for printing out some bytes from a Stream to the console:

public static void PrintBytes(Stream s)
{
    int b;
    while((b = fs.ReadByte()) >= 0)
    {
        Console.Write(b + " ");
    }
}

Reading a single byte at a time is typically not the most efficient way to access a stream. For example, hard drives are capable of (and optimized for) reading continuous blocks of data from the disk in a big chunk. If you know you are going to be reading several characters, it is better to read a chunk from the disk all at once and then consume the chunk from memory byte by byte. The Framework includes the BufferedStream class for doing just that. The constructor for BufferedStream takes as the parameter whatever stream you would like buffered access to. BufferedStream overrides the main methods of Stream, such as Read and Write, to provide more functionality. Since it is still a child class of Stream, you can use it the same as any other Stream (note that FileStream includes its own buffering capabilities). Similarly, you can use System.Security.Cryptography.CryptoStream to encrypt and decrypt Streams on the fly, without the rest of the application needing to know anything more than the fact that it is a Stream. Figure 3 shows several calls to my printing method using different Streams.

Figure 3 Using Different Streams (Close Figure 3)
MemoryStream ms = new MemoryStream(new byte[] {1, 2, 3, 4, 5, 6, 7, 8});
PrintBytes(ms);

BufferedStream buff = new BufferedStream(ms);
PrintBytes(buff);
buff.Close();

FileStream fs = new FileStream("../../decorator.txt", FileMode.Open);
PrintBytes(fs);
fs.Close();
Figure 4 Using the Decorator Pattern
Figure 4 Using the Decorator Pattern

This ability to dynamically attach new functionality to objects transparently using composition is an example of the Decorator pattern, as shown in Figure 4. Given any instance of Stream, you can add the capability for buffered access by wrapping it in a BufferedStream, without changing the interface to the data. Since you are just composing objects, this can be done at run time, rather than using a technique like inheritance, which is a compile-time decision. The core functionality is defined either by an interface or by an abstract class (like Stream) from which all the Decorators derive. The Decorators themselves implement (or override) the methods in the interface (or base class) to provide the extra functionality. BufferedStream, for example, overrides Read to read from a buffer fed by the wrapped Stream, instead of reading from that Stream directly. As Figure 3 illustrates, any composition of Decorators, no matter how complex, can still be used as if it were only the base class.

Adapter Pattern

One of the strengths of the .NET Framework is backward compatibility. From .NET-based code you can easily call legacy COM objects and vice versa. In order to use a COM component in your project, all you have to do is add a reference to it via the Add Reference dialog in Visual Studio .NET. Behind the scenes, Visual Studio® .NET invokes the tlbimp.exe tool to create a Runtime Callable Wrapper (RCW) class, contained in an interop assembly. Once the reference has been added (and the interop assembly has been generated for you), the COM component can be used like any other class in managed code. If you were looking at code someone else had written without seeing the list of references (and without examining metadata associated with the classes or their implementation), you would be unable to tell which classes were written in a .NET-targeted language and which were COM components.

The magic that makes this happen is contained in the RCW. COM components have different error handling mechanisms and also make use of different data types. For example, strings in the .NET Framework use the System.String class while COM might use a BSTR. When calling a COM component with a string parameter from .NET-based code, though, you can pass in a System.String just like you would to any other similar managed code method. Inside the RCW, this System.String is converted into a format that the COM component expects, like a BSTR, before the COM call is made. Similarly, a method call on a COM component typically returns an HRESULT to indicate success or failure. When a COM method call returns an HRESULT that indicates that the call failed, the RCW turns this into an exception (by default), so it can be handled like all other managed code errors.

By allowing managed classes and COM components to interact despite their interface differences, RCWs are an example of the Adapter pattern. The Adapter pattern lets you adapt one interface to another. COM doesn't understand the System.String class, so the RCW adapts it to something that it can understand. Even though you can't change how a legacy component works, you can still interact with it. Adapters are frequently used like this.

The Adapter class itself wraps an Adaptee, translating all calls from the client into the appropriate format and sequence of calls. Though this sounds similar to the Decorator, there are several key differences. With a Decorator, the interfaces of the objects you're composing are the same, while the entire point of an Adapter is to allow you to change interfaces.Adapters also have a definite sequence to them; the Adaptee must be contained by the Adapter. A Decorator class doesn't need to know whether it is wrapped by 1 or 500 other classes, since the interfaces are all the same. As a result, the use of Decorators can be transparent to the application, while the use of Adapter cannot.

Factory Pattern

There are many cases in the Framework where you can obtain a new instance of a struct or class without calling its constructor yourself. The System.Convert class contains a host of static methods that work like this. To convert an integer to a Boolean, for example, you can call Convert.ToBoolean and pass in the integer. The return value of this method call is a new Boolean set to "true" if the integer was non-zero and "false" otherwise. The Convert class creates the Boolean for you with the correct value. Other type conversion methods work similarly. The Parse methods on Int32 and Double return new instances of those objects set to the appropriate value given only a string.

This strategy for creating new object instances is known as a Factory pattern. Rather than invoking the object's constructor, you can ask the object factory to create the instance for you. That way, the factory class can hide the complexity of object creation (like how to parse a Double out of a string). If you wanted to change the details of creating the object, you'd only have to change the factory itself; you would not have to change every single place in the code where the constructor is called.

These type conversion methods are one variant of this pattern, since you're not required to use the factory to create the objects in question. A more pure example of this pattern is the System.Net.WebRequest class, used to make a request and receive a response from a resource on the Internet. FTP, HTTP, and file system requests are supported by default. To create a request, call the Create method and pass in a URI. The Create method itself determines the appropriate protocol for the request and returns the appropriate subclass of WebRequest: HttpWebRequest, FtpWebRequest (new to the .NET Framework 2.0), or FileWebRequest. The caller doesn't need to know the specifics of each protocol, only how to invoke the factory and work with the WebRequest that gets returned. If the URI changes from an HTTP address to an FTP address, the code won't have to change at all. This is another common use of the Factory pattern. The parent class acts as a factory and returns a specific derived class based on parameters the client passes in. As in the WebRequest example, this hides the complexity of selecting an appropriate derived class from the caller.

Strategy Pattern

Both Array and ArrayList provide the capability to sort the objects contained in the collection via the Sort method. In fact, ArrayList.Sort just calls Sort on the underlying array. These methods use the QuickSort algorithm. By default, the Sort method will use the IComparable implementation for each element to handle the comparisons necessary for sorting. Sometimes, though, it is useful to sort the same list in different ways. For example, arrays of strings might be sorted with or without case sensitivity. To accomplish this, an overload of Sort exists that takes an IComparer as a parameter; IComparer.Compare is then used for the comparisons. This overload allows users of the class to use any of the built-in IComparers or any of their own making, without having to change or even know the implementation details of Array, ArrayList, or the QuickSort algorithm.

Leaving the choice of comparison algorithm up to the user of the class like this is an example of the Strategy pattern. The use of Strategy lets a variety of different algorithms be used interchangeably. QuickSort itself only requires a way to compare objects to each other. By calling Compare through a provided interface, the caller is free to substitute whatever particular comparison algorithm fits its specific needs. The code for the QuickSort can remain unchanged.

Figure 5 Strategy in Action
Figure 5 Strategy in Action

One of the new generic collections in version 2.0 of the .NET Framework, List<T>, also makes heavy use of the Strategy pattern, shown in Figure 5. In addition to the updated Sort method, the find-related methods, BinarySearch, and others all take parameters that allow parts of the respective algorithms to vary based on the needs of the caller. The use of a Predicate<T> delegate in the FindAll<T> method lets the caller use any method as a filter for the List<T> so long as it takes the appropriate object type and returns a Boolean. Combined with anonymous methods (another new C# language feature in version 2.0), clients can easily filter lists based on properties and methods of the objects in the list, without introducing dependencies into the List<T> class itself. Using the Strategy pattern lets complicated processes like sorting be easily modified to fit a caller's specific purpose, meaning you'll be able to write and maintain less code.

Composite Pattern in ASP.NET

The ASP.NET request/response pipeline is a complex system. Patterns are used in the design of the pipeline itself and in the control architecture to effectively balance its performance with extensibility and ease of programming. Before delving into the pipeline, however, I'll examine the patterns used in the programming model itself.

When dealing with collections of objects, there are often operations that are appropriate for both a single object and the entire collection. Think about an ASP.NET control. A control may be a simple single item like a Literal, or it could be composed of a complex collection of child controls, like a DataGrid is. Regardless, calling the Render method on either of these controls should still perform the same intuitive function.

When each item in the collection might itself contain collections of other objects, the use of the Composite pattern is appropriate. Composite is an easy way to represent tree-like collections without having to treat parent and leaf nodes differently.

The canonical example of Composite relies on an abstract base class, Component, that contains both methods for adding and removing children, and the operations common among parents and children. ASP.NET uses this formulation exactly with System.Web.UI.Control. Control represents the Component base class. It has operations for dealing with children (such as the child Controls property) as well as standard operations and properties like Render and Visible. Each object, whether a primitive object (like Literal) or a composite of several objects (like DataGrid), inherits from this base class.

Because the domain of controls is so diverse, there are several intermediate derived classes like WebControl and BaseDataList that serve as base classes for other controls. Though these classes expose additional properties and methods, they still retain the child management functions and core operations inherited from Control. In fact, the use of the Composite pattern helps to hide their complexity, if desired. Regardless of whether a control is a Literal or a DataGrid, the use of Composite means you can just call Render and everything will sort itself out.

Template Method Pattern

When the standard library of ASP.NET controls doesn't meet your needs, you have several options on how to create your own. For simple controls that only need to be used in a single project, a user control is often the best choice. When the control is to be used in several Web applications or requires more functionality, a custom server control may be a better fit.

When dealing with custom controls, there are two general types: controls that combine the functionality of several existing controls (called composite controls), and controls with a unique visual representation. The process for creating both of these types is very similar. For composite controls, you create a new class that inherits from one of the control base classes (like Control or WebControl) and then override the CreateChildControls method. In this method you add the controls whose functionality you are combining to the collection of child controls, called Controls. For other custom controls, you override Render instead and use the HtmlTextWriter parameter to output the HTML for your control directly.

Regardless of which style of custom control you choose, you don't have to write any code to handle the functionality that's common to all controls, like loading and saving ViewState at the right time, allowing PostBack events to be handled, and making sure the control lifecycle events are raised in the correct order. The main algorithm for how a control should be loaded, rendered, and unloaded is contained in the control base class.

The specifics of your particular control are handled at well-defined places in the control algorithm (the CreateChildControls or Render methods). This is an example of the Template Method pattern. The main algorithm skeleton is defined in a base class and subclasses can then plug in their own details without affecting the algorithm itself, as shown in Figure 6. A composite control and a custom control both share the same general lifecycle, but they can end up with drastically different visual representations.

Figure 6 Template Method Pattern
Figure 6 Template Method Pattern

This pattern is similar to the Strategy pattern. These two patterns differ in scope and in methodology. Strategy is used to allow callers to vary an entire algorithm, like how to compare two objects, while Template Method is used to vary steps in an algorithm. Because of this, Strategy is more coarsely grained. There can be vast differences between different client implementations, while with Template Method the algorithm remains fundamentally the same. The other main difference is that Strategy uses delegation while Template Method uses inheritance. In the sorting example of Strategy, the comparison algorithm is delegated to the IComparer parameter, but with custom controls you subclass the base and override methods to make changes. Both, however, let you easily alter processes to fit your specific needs.

Patterns in the ASP.NET Pipeline

When a client requests an ASPX page from the Web server, the request travels through many steps before ultimately ending up as HTML displayed by the client's browser. First, the request is processed by IIS and routed to the appropriate ISAPI extension. The ISAPI extension for ASP.NET (aspnet_isapi.dll) routes the request to the ASP.NET worker process.

Figure 7 ASP.NET Request Pipeline
Figure 7 ASP.NET Request Pipeline

At this point, the request begins to interact with classes that you are used to dealing with. The request is passed to an HttpApplication. Usually this is the class created in the codebehind file for Global.asax. The HttpApplication then passes the request through any number of HTTP Modules. These classes implement the IHttpModule interface and have a chance to modify the request (or even halt the processing of it) before it gets passed on to the next module. ASP.NET provides some standard modules which provide functionality you're probably familiar with, including FormsAuthenticationModule, PassportAuthenticationModule, WindowsAuthenticationModule, and SessionStateModule, all of which provide exactly the functionality that their names imply.

Ultimately, the request ends up at an IHttpHandler, the most common of which is System.Web.UI.Page. Inside the IHttpHandler.ProcessRequest method, Page raises appropriate events (like Init, Load, and Render), handles ViewState, and provides the programming model for ASP.NET. Figure 7 shows an outline of this process.

Several of the patterns employed by this process are more thoroughly documented in another standard reference for patterns, Martin Fowler's Patterns of Enterprise Application Architecture (Addison-Wesley, 2002).

Intercepting Filter Pattern

Once a request has made it into an HttpApplication, it passes through any number of IHttpModules. Each module is independent and has only a limited amount of control over the order in which it is invoked. The HttpApplication class exposes a sequence of events that get raised as the request makes its way through processing. These events include BeginRequest, AuthenticateRequest, AuthorizeRequest, and EndRequest. When the HttpApplication loads a module, it calls the Init method of the IHttpModule interface, allowing the module to register for any of the events it cares about. As a given request is handled, the events are raised in the appropriate order and all registered modules get a chance to interact with the request. The module can therefore control the stage at which it gets invoked, but not the exact order within that stage.

These modules are an example of the Intercepting Filter pattern. This pattern represents a chain of filters that are each in turn given a chance to modify a request (or message) that passes through them. Figure 8 shows a simplified flow diagram of this process. The key ideas of this pattern are that the filters are independent; filters can modify the request as it passes through.

Figure 8 Request Flow
Figure 8 Request Flow

There are several different implementation variations of the Intercepting Filter pattern, one of which is the event-based model employed by ASP.NET. A simpler variant involves maintaining a list of filters and iterating over it, invoking a method on each one in turn. This is how the Web Services Enhancements (WSE) for ASP.NET Web services uses this pattern. Each filter either extends SoapInputFilter (for request messages) or SoapOutputFilter (for responses), overriding the ProcessMessage method to perform the work of the filter.

Another option is to implement Intercepting Filter via the Decorator pattern. Then each filter would wrap its successor, performing preprocessing, invoking the successor, and then performing post-processing. The chain would be built using recursive composition, from back to front. This pattern is used to implement .NET Remoting channel sinks.

Regardless of implementation, the result is a dynamically configurable chain of independent filters. Since they are independent, these filters can easily be reordered and reused in other applications. Common tasks like authentication or logging can be encapsulated in a filter and used over and over. These tasks can then be handled by the filter chain before a request even reaches the HttpHandler, keeping the handler code cleaner.

Page Controller Pattern

System.Web.UI.Page implements a core part of the programming model for ASP.NET. Whenever you want to add a logical page to a Web application, you can create a Web Form (represented by an ASPX file and its codebehind). You can then write code to handle the specific demands of the new page, whether through handling page-level events, displaying a set of controls, or loading and manipulating data. Each logical page in the application has a corresponding Web Form that controls its behavior and regulates its presentation.

The practice of having one controller for each logical page is an example of the Page Controller pattern. This idea is fundamental to ASP.NET. When a logical page is requested via its URI, the ASP.NET runtime resolves the address to the corresponding subclass of Page and uses that class to handle the request. All the details about what a page looks like, what user input it can handle, and how it responds to that input are contained in a single place. When a logical page in the application requires changes, no other pages are affected. It's an abstraction that's so common we don't even think about it.

One of the drawbacks usually associated with the pure implementation of Page Controller is that common code must be repeated for each page. ASP.NET avoids this drawback by including other patterns in the pipeline implementation, as well as providing System.Web.UI.Page as a common base class for all Page Controllers. Cross-cutting concerns like authentication and session state are handled by the HttpModule Intercepting Filters and raising page lifecycle events, and other activities, are handled by the base class.

Other Web Presentation Patterns in ASP.NET

In addition to Intercepting Filter and Page Controller, ASP.NET makes use of variations of several other Web presentation patterns. When ASP.NET determines which HttpHandler to pass a request to, it uses something similar to a Front Controller. Front Controller is characterized by a single handler for all requests (like System.Web.UI.Page). Once the request reaches the Page class, though, the Page Controller pattern takes over.

Within the ASP.NET implementation of Page Controller, there are elements of the Model View Controller pattern. Model View Controller separates the model (business objects, data, and processes) from the view (the display of information). The controller responds to user input and updates the model and view. Roughly speaking, an ASPX page represents the View while its codebehind file represents a hybrid Model-Controller. If you were to pull all business and data-related code out of the codebehind and leave only event handling code, this would turn the codebehind into a pure Controller while the new class containing the business logic would be the Model.

Since these variations diverge from the canonical form of the patterns, they won't be covered here. Martin Fowler's book and the Microsoft Patterns site are good sources for more discussion.

Conclusion

Now that I have examined instances of common patterns in use throughout the .NET Framework and BCL, it should be even easier to recognize those same patterns in code you work with every day. Hopefully highlighting the design patterns underlying common classes and functionality has given you a better sense of what those patterns are and the benefits they provide. Trying to imagine UI programming without the Observer pattern or collections without an Iterator shows how indispensable these frameworks really are. Once you understand what each does, it becomes another valuable tool to add to your toolbox.

Rob Pierry is a Senior Consultant with Geniant, a leading enterprise IT consulting company specializing in Service Oriented Architecture. He can be reached at rpierry+msdn@gmail.com.


Debugging: Root Out Elusive Production Bugs with These Effective Techniques

Debugging

Root Out Elusive Production Bugs with These Effective Techniques

Matt Adamson
This article discusses:
  • ADPlus and dumps
  • First-chance and second-chance exceptions
  • Debugging managed and unmanaged hangs and crashes
This article uses the following technologies:
C++, C#
Code download available at: Debugging.exe (194KB)
Contents
Debugging Tools for Windows
Using ADPlus
Debugging Symbols
First-Chance Exceptions
Unmanaged First-Chance Exceptions
Managed First-Chance Exception
Unmanaged Thread Executing Endlessly
Managed Thread Executing Endlessly
Deadlocks
Unmanaged Deadlock Application
Managed Deadlock Application
Crashing
Conclusion

E
rrors in code are inevitable. No matter how much testing you have done, when your application is deployed in a production environment, errors will occur. These problems can manifest as standard exceptions, as hangs where the CPU is being used 100 percent of the time, as deadlocks, where a couple of threads are locked and will never release the other's resources, and as crashes, where the application dies a silent death.

In these cases, error logs often provide little or no help in pinpointing the exact line of code in which the problem originates. As you can't usually isolate the root cause by modifying code in a production environment, most developers try to recreate an identical production environment on their workstations. They copy the specific components, the customer's database (if any), and then try to reproduce the problem in their debugger. At best the error will be reproduced and the failing code found; at worst the same error won't occur at all. Recreating the exact same production environment is very difficult due to the large number of potential software and hardware configurations. Furthermore, it may not be feasible or permitted to copy large secure databases, or to even get the necessary configuration information from the client.

In this article, I will demonstrate how to solve each of the problems described using managed and unmanaged applications without having to recreate the production environment. For each example, I will show you how to find the root cause using the appropriate debugger commands. So that you can start using these techniques right away, I'll cover only the most important commands for these situations; if you're interested in digging further, refer to the other resources mentioned throughout the article.

Debugging Tools for Windows

The debugging tools for Windows® can be downloaded from Debugging Tools for Windows. The package includes a number of tools that will help developers with post-mortem debugging, the most useful of which is ADPlus. This is a console-based VBScript that can automatically create memory dump and log files. ADPlus effectively calls the console debugger (CDB) with a number of command-line options to obtain its output. The most useful feature is that you can create dump files for production issues when they occur.

Dump files contain a snapshot of the memory space of a process, and are most useful when that process is failing—hanging, crashing, or simply throwing an exception. There are two common types of dump files: full and mini dumps. A full mode dump is typically much larger than a mini dump because it contains all committed pages of memory owned by the application. A mini dump only includes selected parts of a process's memory (basically just call stacks). As an example, a full dump of notepad.exe editing a small text file may be about 20MB, whereas a mini dump would be around 7KB.

The best feature of dump files is the ability to capture a call stack. You can see exactly what your application was doing at the time the problem occurred. There is no ambiguity; the call stack reveals exactly what is happening within the process. Most of the time you can quickly identify the problem code and resolve the issue.

Using ADPlus

ADPlus can be started in either hang or crash mode. In hang mode, ADPlus takes a snapshot of the process at that point in time, so it should be run when the application is actually hanging. When the process memory is being dumped, the process is frozen. The debugger non-invasively attaches to the process; when the dump finishes, the debugger detaches and the process resumes execution.

A debugger attached to a process can receive two types of notifications for each thrown exception: first and second chance. The debugger gets the first chance to handle the exception. If the debugger allows the execution to continue and does not handle the exception, the application will see the exception as usual. If the application does not handle the exception, the debugger gets a second chance to see the exception. In this case the application would normally crash if the debugger was not present. Figure 1 shows the interaction between the debugger and the application to which it is being attached (the debuggee).

Figure 1 Interaction Between Debugger and Debuggee
Figure 1 Interaction Between Debugger and Debuggee

Crash mode starts the CDB and attaches to the process specified. It remains attached to the process until either a second-chance exception is caught or the user presses Ctrl+C to detach from the debugger. In this case a full dump is taken of the process before it terminates. ADPlus can be left running in crash mode until a crash occurs. You should not notice any performance degradation when running in this mode unless many first-chance exceptions are being raised, causing mini dumps to be created frequently. This is not the only use for crash mode, however. One of the most useful and lesser known features of ADPlus is the creation of mini dump files when first-chance exceptions are raised. By capturing dump files for each exception raised, you can quickly find the root cause of the problem. ADPlus provides a configurable option called OutputDir to specify the folder in which to create dump files. Because some full dump files are huge, you must set up this folder on a drive with a lot of available space. It is painful to have ADPlus running in crash mode waiting weeks for a crash, and then have it fail to capture a dump because of insufficient space for the dump file.

WinDBG, one of the debuggers in the toolkit, is a powerful tool capable of both user-mode and kernel-mode debugging. It is effectively a GUI to the lower-level debuggers CDB/NTSD and KD. Although it is usually used to debug kernel mode device drivers and system services, WinDBG can also be used for common user-mode applications. When I discuss each error later on, I will show the specific WinDBG commands that are required to read and analyze the dump files.

Debugging Symbols

Debugging symbols are essentially data containing source, line information, data types, variables, and functions that allow the debugger to provide symbolic names rather than the hex addresses of all the functions. Symbols are distributed in files with a PDB extension. Make sure to always create and store symbols for every build of your application, including release builds.

When opening a dump file in WinDBG, the debugger will look for the correct PDB file using an internal timestamp stored in the PE header of the DLL. This is why it is essential to create the PDB files as you release new builds. Options for enabling this within all configurations of a C# project are shown in Figure 2. The steps to enable this for C++ applications are available at Generating and Deploying Debug Symbols with Microsoft Visual C++ 6.0.

Figure 2 Generating Debugging Symbols for C# Projects
Figure 2 Generating Debugging Symbols for C# Projects

_NT_SYMBOL_PATH is an environment variable containing the path where debuggers, including WinDBG, will look for symbols. This path should point to both operating system symbols and any application symbols you've already created. To make it easy to obtain the correct symbols for the version of Windows you're using, including any installed service packs and hotfixes, Microsoft has created a symbol server which allows for the correct versions of all operating system symbols to be downloaded automatically. Simply set _NT_SYMBOL_PATH to the following:

SRV*\\<path to download symbols to>*http://msdl.microsoft.com/download/
symbols

If you use a shared network drive, all other developers on your team can access the symbols you've downloaded and vice versa. Once they are downloaded, they will always be obtainable locally from the file system, unless new versions of DLLs are released.

When analyzing a dump file, you will also need to ensure that the debugger can find your application debugging symbols. The .sympath+ command can be used from within WinDBG to append the path of directories where your application debugging symbols are located:

0:000> .sympath+ C:\Assemblies\Tranmit.Sprinter.Invoicing\bin\release
Symbol search path is: SRV*\\d:\windows\symbols*http://msdl.microsoft.com/
download/symbols;C:\Assemblies\Tranmit.Sprinter.Invoicing\bin\release

Now that you have created separate PDB files for each DLL version, you will need to know which versions of your components were running when the dump was created. The simplest way to do this is by using the lmvm debugger command:

0:000> lmvm UnmanagedProductionDebuggingEngine
start    end        module name
01650000 0165e000   UnmanagedProductionDebuggingEngine (deferred)
    Image path: d:\UnmanagedProductionDebuggingEngine.dll
    Timestamp: Sat Oct 23 11:40:43 2004 (417A352B)  Checksum: 00000000
    File version:     1.0.0.0
    Product version:  1.0.0.0
    File flags:       0 (Mask 3F)
    File OS:          4 Unknown Win32
    File type:        2.0 Dll
    File date:        00000000.00000000
You can easily see that version 1.0.0.0 of UnmanagedProductionDebuggingEngine.dll was running; therefore, you can point the symbol path to the correct PDB folder for this version.

A lot of articles on production debugging mention the importance of copying the symbols onto the client's server both for the operating system and the application. I strongly recommend against this. First, your application can be more easily reverse-engineered by clients with your PDBs. Second, this provides very little benefit anyway, as most of the dump analysis you do will be performed on your own development workstation and not on the client's site. The only advantage of having correct symbols installed on the server is that the text log files produced by ADPlus are more meaningful because they display call stacks with symbolic names. However, analysis of dump files usually requires more information than simply seeing the stack; it requires information such as variables and arguments on the stack.

First-Chance Exceptions

It is often useful to analyze exceptions that are thrown but not caught correctly, and there are situations in which exception information is lost. Perhaps you are using Visual Basic® in ASP pages and are making use of the On Error Resume Next statement. If the code doesn't handle errors in all cases, error information can be lost and useful information overwritten by other errors. Some exceptions may be caught and handled incorrectly by the application, which then subsequently causes other errors to occur. You may also have some .NET-based code that catches an exception and rethrows it, losing all important error information. Analyzing the contents of these initial exceptions may be crucial to pinpointing the cause of the error.

Before I look at an unmanaged application throwing a first-chance exception, there are some important points to go over. ADPlus does not by default create mini dumps for first-chance exceptions of type "unknown" and "C++." The documentation states that this is because it is quite common for applications to generate a significant number of these legitimately. If you're performing production debugging for C++ and Visual Basic components, you most certainly will want to enable mini dump creation for such exceptions. Visual Basic applications raise unknown exceptions when errors occur and C++ applications frequently throw C++ exceptions. Enabling mini dumps for these exceptions can only be done through XML configuration files, which ADPlus uses. An ADPlus config file is provided to show you the options that need to be set for some of the sample apps; other samples only require simple ADPlus command-line configuration.

Unmanaged First-Chance Exceptions

My first example is an unmanaged application, UnmanagedFirstChanceException, which throws an exception when attempting to read information from a configuration file using the C++ CStdioFile class and handles it (see Figure 3).

Figure 3 Unmanaged FirstChanceException (Close Figure 3)
try {
    // Load the configuration file into the first stream
    CStdioFile configurationFile1(
        _T("Configuration.xml"), CFile::modeRead);

    if (IsWindows2000()) {
        // Load the configuration file into a second stream
        CStdioFile configurationFile2(
            _T("Configuration.xml"), CFile::modeRead);
    }

    ... // parses configuration file to obtain all settings
}
catch(CFileException *pException) {
      cout << _T("Failed to find configuration file") << endl;
}

The code seems odd because it tries to read the config file into a CStdioFile object twice if it detects that it is running under Windows 2000. And since configuration information for the app isn't loaded in all cases, it is quite possible that this code block has never been tested prior to its release into production.

When the app is run in production, the message "Failed to find configuration file" is output. After you've verified that the file does indeed exist in the same location as the application, it is difficult to know where to start your diagnosis. On further reading through the code, it is clear that the CFileException object can be thrown from within the CStdioFile constructor for a variety of reasons other than file not found. If an attempt to reproduce this problem on your own developer workstation proves to be unsuccessful, you'll then need to determine why the exception is being thrown on the production system.

You need to run ADPlus in crash mode to obtain the first-chance exception for the CFileException object. But first, you need to create the configuration file ADPlus will use to create the mini dump file. Remember that you must use configuration files rather than command-line options because C++ exceptions need to be captured. The UnmanagedFirstChanceExceptionConfig.xml file provides the configuration required for ADPlus to catch the exception dump. Most of the time ADPlus will be used to attach to already running processes; in some cases this will not catch exceptions that occur when the process is starting. In this example, if the process starts and then exits very quickly, ADPlus will not be available to attach to the process in time. The ADPlus spawn option is therefore a good choice. It allows the debugger to spawn the process internally and begin monitoring it as soon as it starts. You will need to modify the spawn element in the configuration file to point to the full path of the executable on your machine.

When the configuration file has been created, ADPlus should be run using the following options:

adplus –c full_path_to_config_file
The UnmanagedFirstChanceExceptionConfig.xml config file, available on the MSDN®Magazine Web site in the code download for this article, will inform ADPlus to effectively start the UnmanagedFirstChanceException application in a suspended state, attach the debugger to the process, and then notify the primary thread of the process to resume execution. ADPlus will create another folder in the output directory that includes the process name, process identifier, and date/time stamp. Within this folder you should see two dump files, one created when the first-chance exception was raised, and the other when the process was shut down.

Open up WinDBG and press Ctrl+D to select and open the first-chance exception dump file you've created. Before you do anything else, you need to set the symbol path to the folder containing your application .pdb file. In a production system you have to find the version of the component the dump was created with, and then set the path to the folder with the correct .pdb file for this version. The lm v command will list all modules loaded into the process together with the version of the module. In the example, the symbol path can simply be set by using the !sympath+ command and appending the full path to the .pdb file found in the release folder of your application.

Once the dump file is loaded, enter kb to see a full call stack. Providing that all the symbol paths have been set correctly, the stack should look like the following:

0:000> kb
ChildEBP RetAddr  Args to Child
WARNING: Stack unwind information not available. Following frames may be
wrong.
...
0012fec4 7c1cd441 0000000B 00000002 003252d8
MFC71!AfxThrowFileException+0x45
[f:\vs70builds\3077\vc\mfcatl\ship\atlmfc\src\mfc\filex.cpp @ 118]
0012ff00 00401034 00402110 00000000 00000000
MFC71!CStdioFile::CStdioFile+0x60
[f:\vs70builds\3077\vc\mfcatl\ship\atlmfc\src\mfc\filetxt.cpp @ 58]
012ff58 004010d7 7c90ee18 004013c8 0000001
UnmanagedFirstChanceException!LoadConfigurationData+0x34
[d:\unmanagedfirstchanceexception\unmanagedfirstchanceexception.cpp @ 53]
...

Child Extended Base Pointer (ChildEBP) points to the beginning of the local information for the current stack frame, RetAddr tells each call where to return when it's finished, and Args to Child contains three columns that represent the arguments passed to the called child or function. It is clear an exception is thrown from the CStdioFile constructor after the call to AfxThrowFileException. The documentation for this method reveals that it takes the cause of the exception as the first parameter. Now all you need to do is find the value of this parameter from the dump. The first argument to most routines shown on the stack is in the first column of DWORDs under the "Args to Child" heading in the stack trace. Here you can see the value of the first parameter to AfxThrowFileException is 0xB (11 in decimal).

If you review the Afx header file in the MFC source code, you can see the different causes for an exception to be thrown. The value 11 indicates a sharing violation:

class CFileException : public CException {
    DECLARE_DYNAMIC(CFileException)
  public:
    enum {
          none, generic, fileNotFound, badPath, tooManyOpenFiles,
          accessDenied, invalidFile, removeCurrentDir, directoryFull,
          badSeek, hardIO, sharingViolation,
          lockViolation, diskFull, endOfFile
    };

Because the symbols contain line number information, you can also see from the call stack that the exception was thrown on line 53 of the LoadConfigurationData method. It is now very clear that a sharing violation was thrown because the Configuration.xml file was already opened at the point the second CStdioFile was being created. You have now pinpointed why the exception was raised, so you can proceed to fix the issue.

This shows that because of environment conditions it's not always possible to reproduce problems. The developer was trying to reproduce the problem on his Windows XP workstation rather than the Windows 2000 operating system used in production. This meant that two calls were not being made to open the same file.

Managed First-Chance Exception

The next example I'm going to look at is a first-chance exception generated from a .NET managed application. I've included a sample ASP page that uses COM interop to access a managed object (using Server.CreateObject) in the download with this article. To reproduce the issue, click on the "Load a purchase order" link in the managed issues section of the default.asp page. The page attempts to initialize a purchase order given an ID of 200 using the PurchaseOrder.InitializeFromID method, as shown in Figure 4. Although you can see the argument passed is incorrect, the client of this method will not have a clue from the message passed to the BusinessObjectException constructor.

Figure 4 Managed Exception Thrown (Close Figure 4)
try {
    if(id <= 1000)  // ID must always be greater than 1000
        throw new ArgumentOutOfRangeException( "id", id,
            "InitializeFromID expects ID value to be greater than 1000");
}
catch(ArgumentOutOfRangeException exception) {
    throw new BusinessObjectException( String.Format( "Failed to
        initialize purchase order ID={0} from database", id), exception);
}

The techniques used to analyze these kinds of exceptions are very similar to those used for the unmanaged application, with a few differences. Since WinDBG doesn't support viewing internal common language runtime (CLR) data structures and call stacks, you'll need to use a WinDBG extension called Son of Strike (SOS). All dumps created to analyze .NET exceptions through SOS should be created as full dumps, even if only first-chance exceptions are raised, as SOS requires certain well-defined subsets of a process's memory to do its work. For the Microsoft .NET Framework 2.0, these sections are included in mini dumps.

Often exceptions in managed code are nested within other exceptions as internal exceptions, as exceptions are thrown from one layer to another and wrapped with an exception appropriate to that level. To avoid generating dumps for all of these inner exceptions, you can create a more targeted configuration file, which simply creates dumps only when a particular type of exception is thrown. This is shown in Figure 5.

Figure 5 ADPlus Config File (Close Figure 5)
<ADPlus>
    <Settings> <RunMode>CRASH</RunMode> </Settings>
    <PreCommands> <Cmd> !load clr10\sos </Cmd> </PreCommands>
    <Exceptions>
        <Option>  NoDumpOnFirstChance  </Option>
        <Option>  NoDumpOnSecondChance  </Option>
        <Config>
            <Code> clr </Code>
            <Actions1> Log </Actions1>
                <CustomActions1> !clr10\sos.cce
                  ManagedProductionDebuggingAssembly.BusinessObjectException
                  1; j ($t1 = 1) '.dump /ma /u
                  ManagedFirstChanceExceptionFull.dmp;gn' ; 'gn'
                </CustomActions1>
                <ReturnAction1> GN </ReturnAction1>
                <Actions2> Void </Actions2>
                <ReturnAction2> GN </ReturnAction2>
        </Config>
    </Exceptions>
</ADPlus>

If you start ADPlus using this configuration file and reproduce the problem, you should see only one dump file created for the code in Figure 4. After running the code, you will notice a slight pause while the dump file is created in the folder where the debugging tools were installed.

I will now use commands similar to those John Robbins highlighted in his excellent SOS column in the June 2003 issue of MSDN Magazine. To load the SOS extension, use the command !load clr10/sos.dll. Since you know the dump file will be stopped in the current thread that caused the exception, you can execute the !dumpstackobjects command to find the exception detail (see Figure 6).

Figure 6 !dumpstackobjects Command (Close Figure 6)
0:022> !dumpstackobjects
 succeeded
Loaded Son of Strike data table version 5 from "D:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorwks.dll"
ESP/REG  Object   Name
0147e920 058b41d8 ManagedProductionDebuggingAssembly.BusinessObjectException
0147e938 058b41d8 ManagedProductionDebuggingAssembly.BusinessObjectException
0147e94c 058b41d8 ManagedProductionDebuggingAssembly.BusinessObjectException
0147e970 058b41d8 ManagedProductionDebuggingAssembly.BusinessObjectException
0147e9b4 058b41d8 ManagedProductionDebuggingAssembly.BusinessObjectException
0147e9c8 058b4268 System.String    Failed to initailise purchase order ID=2
0147e9cc 058b41d8 ManagedProductionDebuggingAssembly.BusinessObjectException
0147e9d0 058b4390 System.Collections.Specialized.NameValueCollection

Inspect the BusinessObjectException exception object by executing !dumpobj 058b41d8 (see Figure 7). As I am interested in the innermost exception's content, I'll also need to review the inner exception of this exception. The memory address of the inner exception object can be used to execute a further !dumpobj command against this address which outputs the inner exception, as shown in the code in Figure 8.

Figure 8 Inner Exception Info for Managed Exception (Close Figure 8)
Name: System.ArgumentOutOfRangeException
MethodTable 0x79b984bc
EEClass 0x79b9854c
Size 72(0x48) bytes
mdToken: 0200003a  (d:\windows\microsoft.net\framework\v1.1.4322\mscorlib.dll)
FieldDesc*: 79b985b0
      MT  Field  Offset Type   Attr     Value     Name
79b8fd3c  400001d   4  CLASS   instance 00000000  _className
79b8fd3c  400001e   8  CLASS   instance 00000000  _exceptionMethod
79b8fd3c  400001f   c  CLASS   instance 00000000  _exceptionMethodString
79b8fd3c  4000020  10  CLASS   instance 058abf90  _message
79b8fd3c  4000021  14  CLASS   instance 00000000  _innerException
79b8fd3c  4000022  18  CLASS   instance 00000000  _helpURL
...
Figure 7 Viewing Managed Exception Information (Close Figure 7)
0:022> !dumpobj 058b41d8
Name: ManagedProductionDebuggingAssembly.BusinessObjectException
MethodTable 0x017c57ec
EEClass 0x07a24b30
Size 92(0x5c) bytes
mdToken: 02000004  (d:\windows\system32\managedproductiondebuggingassembly.dll)
FieldDesc*: 00000000
      MT    Field   Offset   Type  Attr     Value    Name
...
79b8fd3c  4000020  10        CLASS instance 058b4268 _message
79b8fd3c  4000021  14        CLASS instance 058b416c _innerException
79b8fd3c  4000022  18        CLASS instance 00000000 _helpURL
79b8fd3c  4000023  1c        CLASS instance 00000000 _stackTrace
79b8fd3c  4000024  20        CLASS instance 00000000 _stackTraceString
...

As you can see, there are no further inner exceptions because the _innerException property is null (all zeros); use !dumpobj 058abf90 to view the actual message. This produces the output shown in Figure 9, with the message contents highlighted.

Figure 9 Inner Exception Message Property Value (Close Figure 9)
0:022> !dumpobj 058abf90
Name: System.String
MethodTable 0x79b8db58
EEClass 0x79b8dea4
Size 132(0x84) bytes
mdToken: 0200000f  (d:\windows\microsoft.net\framework\v1.1.4322\mscorlib.dll)
String: InitializeFromID expects ID value to be greater than 1000
FieldDesc*: 79b8df08
      MT  Field   Offset  Type           Attr     Value   Name
79b8db58  4000013   4     System.Int32  instance   58     m_arrayLength
79b8db58  4000014   8     System.Int32  instance   57     m_stringLength
79b8db58  4000015   c     System.Char   instance   49     m_firstChar
79b8db58  4000016   0     CLASS         shared    static  Empty
    >> Domain:Value 001676b0:0583126c <<
79b8db58  4000017   4     CLASS         shared    static  WhitespaceChars
    >> Domain:Value 001676b0:05831280 <<

The output at the top of Figure 8 indicates that the name of the innermost exception is ArgumentOutOfRangeException. The message shown in Figure 9 mentions a condition that the ID value passed must be greater than 1000. You can therefore conclude that the client ASP page is causing the error because bad values are being passed into the component. Why not just capture any ArgumentOutOfRangeException types in the ADPlus configuration file? You may not always have the source code available for third-party components. This approach works whether you have source code available or not.

A side note: sometimes when analyzing .NET exceptions, you may not have ADPlus running in crash mode while a .NET exception is thrown that only occurs occasionally. In this case you may still be able to examine the contents of .NET exceptions—they might be lying around in the managed heap because garbage collection hasn't occurred. Try taking a hang dump just after reproducing this issue and then run the !DumpHeap -type BusinessObjectException command. You can then use the address shown for the object and follow the same set of commands as for the address found when !dumpstackobjects was run.

Unmanaged Thread Executing Endlessly

Some applications take over the processor entirely for a moment, which is fine. However, when processor usage remains high for more than a few seconds, it can indicate a hung process. Another symptom of a hung application is unresponsiveness.

Hanging applications are usually caused by one of two things: an endless loop in code leaves a thread executing endlessly while eating up CPU cycles (called a live lock), or a thread in a state where it is waiting for a resource that never becomes available (called a dead lock). In both cases troubleshooting needs to focus on isolating the path of code executing leading up to the hang.

The next example I'll look at is an unmanaged app stuck in an endless loop. The code that runs an endless loop will be created in a COM object call Bank. To show also how dump files can be used in IIS, this component will be called from an ASP page.

ADPlus can create hang mode dumps for specific processes, by ID or name, or by using the IIS option. The IIS option takes dumps of all IIS-related processes, which includes Web apps run in low protection (inside INETINFO.exe), medium protection (in a single DLLHOST process for all apps), and high protection (in a separate DLLHOST process for each app). The IIS option may be impractical on production systems with many Web applications, so it is best to target only the specific Web application's DLLHOST process. You can use the tlist command-line tool with the –k option to list all dllhost processes including the Web application name. You will probably know which Web application is being unresponsive because your users will tell you! In any case, you can confirm this using the Performance Monitor Process and Thread objects.

Since many different instances of DLLHOST can exist for separate Web apps run in high protection, by default Performance Monitor differentiates them using the # symbol and a sequence number: DLLHOST, DLLHOST#1, DLLHOST#2, and so on. Threads within a process also use the same scheme to differentiate themselves. It would be better to use the actual identifiers for the process and threads when selecting counters to add, which you can do by running the following registry script:

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PerfProc\Performance]
"ThreadNameFormat"=dword:00000002
"ProcessNameFormat"=dword:00000002

The next thing to do is to ensure the Web application is started and the DLLHOST process is created by browsing to the default Web page. Open PerfMon and add the "% Processor Time" counter for all DLLHOST instances from the Process object.

Now reproduce the issue by navigating to the Get Bank Balance page in the unmanaged issues section. You should notice immediately that one of the DLLHOST instances is using up all of the processor time (see Figure 10). At this point you can take a hang dump of this specific process using the command-line option:

adplus –hang –p process id

Figure 10 CPU Utilization in DLLHOST Process
Figure 10 CPU Utilization in DLLHOST Process

Once you have the process ID, clear the existing counter set, and create a new one for all threads in this process to find which thread is using all the CPU time within this process.

If you have collected a hang dump of a process where no other Perfmon data is available, there is another method you can use to find the hanging thread. The !runaway extension displays information about the time consumed by each thread in the process. Because the runaway extension sorts by descending CPU time, the offending thread is usually the one at the top of the list. However, in long running processes the top threads are not always the offending threads, it can just be they are the most active. A better (but more laborious) way to identify a runaway thread is to compare two or more dumps a minute or so apart and try to spot those that have increased the most. As you can see in the code that follows, thread id 0x22c (decimal ID 556) is consuming the most CPU time, the same as the thread ID you found with the Performance Monitor counters.

0:000> !runaway
 User Mode Time
 Thread    Time
 22c       0 days 0:03:17.218
 248       0 days 0:00:00.015
 7a0       0 days 0:00:00.015
 fa0       0 days 0:00:00.000
 29c       0 days 0:00:00.000

Now that you have identified the hanging thread as 556, you need to find the corresponding thread number in WinDBG. Unfortunately, this mapping is required because of the way the threads are assigned and used in WinDBG. The ~ command prints the status for all threads in the current process:

0:000> ~
.  0  Id: f18.f28 Suspend: 1 Teb: 7ffdf000 Unfrozen
   1  Id: f18.7a0 Suspend: 1 Teb: 7ffde000 Unfrozen
  ...
  18  Id: f18.72c Suspend: 1 Teb: 7ffa5000 Unfrozen
  19  Id: f18.22c Suspend: 1 Teb: 7ffa4000 Unfrozen
  20  Id: f18.d60 Suspend: 1 Teb: 7ffa3000 Unfrozen

The columns from this command indicate, in order, the decimal WinDBG thread identifier, hex process ID, hex thread ID, the suspend count of the thread, address of thread environment block, and the thread status. The period before the 0 indicates this is the current thread. You can quickly see that the thread number for the hanging thread is number 19 (hex thread ID 22c).

Now that you have the thread number, look at the call stack for this thread by running the ~19kb command. ~19 means run the command for thread 19. Here are the results.

0:000> ~19kb
ChildEBP RetAddr  Args to Child
0143f330 771273d0 016c2954 0003e948 0143f394
UnmanagedProductionDebuggingEngine!CBank::GetBalance+0x12
[d:\unmanagedproductiondebuggingengine\bank.cpp @ 34]
0143f350 771279e0 016c2954 0000001c 00000004 oleaut32!DispCallFunc+0x16a
0143f3e0 01653dc8 0017c284 016c2954 00000000
oleaut32!CTypeInfo2::Invoke+0x234
0143f404 7332ae0e 016c2954 00000001 73304dd0
 ...
You can clearly see that the function at the top of the call stack is GetBalance, currently executing line 34. Although your production components will require careful analysis to determine why it's entering an endless loop, this allows you to find the loop quickly.

Managed Thread Executing Endlessly

The managed application is very similar to the previous examples. A class Bank in the ManagedProductionDebuggingAssembly assembly contains the same behavior as the Bank COM object. Follow the same steps as in the "Unmanaged Thread Executing Endlessly" section to identify the thread using all the CPU time. When you've identified the thread, change the context in the debugger as before using ~<thread number>s and run the !clrstack command to see a managed call stack. The following shows the output for one test run:

0:022> !clrstack succeeded
Loaded Son of Strike data table version 5 from
"D:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorwks.dll"
Thread 22
ESP       EIP
0147ea98  07d50078 [DEFAULT] [hasThis]
ValueClass System.Decimal
ManagedProductionDebuggingAssembly.Bank.GetBalance(String) at [+0x0] [+0x0]
d:\managedproductiondebuggingassembly\bank.cs:41
0147eb44  79240167 [FRAME: GCFrame]
...

Deadlocks

The next application to explore reproduces an unresponsive component. Although the net effect is also a hung application, the symptom is different because the CPU is not overutilized. This usually indicates a deadlock, but before I get into that, I need to examine multithreading.

Let's say you have created a class that performs work on CSV (comma-separated values) files, reading the data in and compiling statistics such as total number of records. If the class can be accessed by multiple clients at the same time, without a synchronization mechanism in place, the data could very easily become corrupted. For example, one thread, A, could be halfway through reading through all records in one CSV file when the operating system preempts it and schedules thread B to run. Thread B starts processing another file using the same shared variable as thread A to access the file. Thread B then gets preempted and Thread A is allowed to run again. At this point Thread A's file pointer is now pointing to a completely different file at a different location. The code on this thread will then run with undefined behavior, possibly causing the process to crash or data to become inconsistent.

You need to use a synchronization mechanism to ensure that while Thread A is reading the file, Thread B cannot change the shared variable until Thread A completes. A critical section is a mechanism that allows you to do this. At the point in the code that needs to be protected, you should enter the critical section; when you've finished running this section of code, you should leave the critical section. Only one thread in the process can be in the critical section at a particular time. Only when the thread leaves the critical section will another thread be allowed to enter. The Win32 API provides the EnterCriticalSection and LeaveCriticalSection methods and the .NET Framework provides the Monitor.Enter and Exit methods to allow you to do this. The Monitor class uses an internal CLR structure to provide synchronization; native Windows critical sections are not used.

Here's how a textbook deadlock occurs. Thread A acquires a resource using a critical section called CS1 to protect access. Thread B then runs and acquires a different resource using critical section CS2. Thread A runs and attempts to acquire critical section CS2 to access the resource that Thread B is using, blocking until Thread B releases it. If Thread B runs and attempts to acquire the critical section CS1 that Thread A owns, both threads will deadlock, as both are waiting for resources that are owned by each other. The result is a hung process—no code can execute on either thread now.

Note that many deadlocks occur due to other synchronization primitives such as events and mutexes. These require more elaborate debugging techniques and are outside the scope of this article.

If you find an application is unresponsive and you suspect a deadlock, you can take a hang dump when the application becomes unresponsive. Analysis of the dump file should reveal whether a deadlock has indeed occurred and which two threads in the process are deadlocked. Once you have identified the two threads, you can review their call stacks and hopefully determine why the deadlock has occurred.

Unmanaged Deadlock Application

To produce a sample unmanaged deadlock, open up the default.asp page in two browser windows. With both windows side by side, press the "Approve a purchase order" link in the unmanaged section in one window, and then the "Reject a purchase order" link in the unmanaged section in the other. You have 10 seconds in which to press on the second link to reproduce a deadlock. Now both requests from each window will be deadlocked. Now you can take a standard IIS hang dump.

Each page creates an instance of a PurchaseOrder object and calls either the Reject or Approve method. The implementation of the Approve method is shown in Figure 11.

Figure 11 Unmanaged Approve Method (Close Figure 11)
STDMETHODIMP CPurchaseOrder::Approve()
{
    AFX_MANAGE_STATE(AfxGetStaticModuleState());
    {
        CAccessLock<> lockLineItems(m_lockLineItems);
        Sleep(10000);
        {
            CAccessLock<> lockObjectStatus(m_lockObjectStatus);
            ... // Approve the purchase order
        }
    }

    return S_OK;
}

The CAccessLock class in the Approve method is effectively a wrapper around the CCriticalSection class, which calls the Lock method in the constructor and UnLock in the destructor. Two critical sections are acquired before the purchase order is approved, one to protect concurrent access to line items and the other one to protect the object state. The Reject method is similar; in this case the critical section m_lockObjectStatus is locked first and then m_lockLineItems is locked.

The first thing to do when analyzing a situation such as this is to verify the issue isn't 100 percent CPU utilization by looking at the results of the runaway extension. As you can see in the following, no threads are monopolizing the CPU:

0:000> !runaway
 User Mode Time
 Thread    Time
 1198      0 days 0:00:00.015
 10f4      0 days 0:00:00.000
 10dc      0 days 0:00:00.000
 33c       0 days 0:00:00.000
 c34       0 days 0:00:00.000
 1628      0 days 0:00:00.000

Now you need to look at the critical sections in the process. You can view critical sections either by using WinDBG commands such as !cs to view all critical sections in the process, and then reviewing the call stacks of the other threads in the process waiting on these critical sections, or you can use a very handy WinDBG extension called SIEExtPub (available for download at Debugging Tools Download). The command !critlist displays a list of all critical sections in the process together with the thread owning the critical section and a list of all threads waiting on it. It will also identify deadlock situations:

0:000> !critlist
CritSec at   a6a3c4.  Owned by thread 16.  Deadlocked on CritSec a6a3a4.
  Waiting Threads: 15
CritSec at   a6a3a4.  Owned by thread 15.  Deadlocked on CritSec a6a3c4.
  Waiting Threads: 16
You can see that thread 16 owns a critical section that thread 15 is waiting on, but thread 16 is also waiting on a critical section held by thread 15, creating a classic deadlock situation. Once you have the thread numbers you can review the call stacks using the command ~<thread number>s to change status to the thread, then view the call stack. The stack for thread 15 is shown in Figure 12. The call stack for thread 16 is almost identical, the difference being a call to Reject instead of Approve.

Figure 12 Thread 15 Call Stack (Close Figure 12)
0:015> kp
ChildEBP RetAddr
0143f254 7c90e9c0 ntdll!KiFastSystemCallRet
0143f258 7c91901b ntdll!ZwWaitForSingleObject+0xc
0143f2e0 7c90104b ntdll!RtlpWaitForCriticalSection+0x132
0143f2e8 00a659d7 ntdll!RtlEnterCriticalSection+0x46
0143f318 00a65ab3 UnmanagedProductionDebuggingEngine!CCriticalSection::Lock(void)+0x37 [d:\program files\microsoft visual studio .net 2003\vc7\atlmfc\include\afxmt.inl @ 72]
0143f338 771273d0 UnmanagedProductionDebuggingEngine!CPurchaseOrder::Approve(void)+0x63 [d:\unmanagedproductiondebuggingengine\purchaseorder.cpp @ 42]

From these call stacks, you can clearly see that the Reject method is waiting on a critical section called m_lockLineItems that the Approve method has acquired. The Approve method is also waiting on the critical section m_lockObjectStatus, which the Reject method has acquired (see Figure 13). Thread 15 is trying to acquire a lock that thread 16 owns and thread 16 is trying to acquire a lock that thread 15 owns. The solution is to make sure all critical sections are acquired in the same order; that is, m_lockLineItems is acquired followed by m_lockObjectStatus. This is a design pattern known as lock hierarchy. You can actually write code to enforce this, such that if you try to take a lock that violates the hierarchy, the application can throw an exception instead of deadlock.

Figure 13 Unmanaged Deadlock Scenario
Figure 13 Unmanaged Deadlock Scenario

This example is quite simple. In the real world you may have cycles of locks indirectly with more than two threads deadlocked. Consider thread 1 waiting on thread 2 waiting on thread 3 waiting on thread 1.

Managed Deadlock Application

Finding two deadlocked threads in a managed application can often be done using the SOS !SyncBlk command. Follow the same steps to reproduce the problem as for the unmanaged deadlock application, except use the links "Approve a purchase order" and "Reject a purchase order" in the managed issues section from default.asp. Take a hang mode dump after the deadlock occurs and open the dump file. Use the !SyncBlk command to list all synchronization blocks with locks:

0:000> !syncblk
Loaded Son of Strike data table version 5 from
"D:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorwks.dll"
Index SyncBlock MonitorHeld Recursion   Thread  ThreadID     Object
Waiting
    4 0x01726ffc        3         1 0x1761ce8 0x1960  22   0x0596943c
ManagedProductionDebuggingAssembly.LockClass
        Waiting threads:  21
    5 0x01727028       3         1 0x184e58 0x1534  21   0x05969430
ManagedProductionDebuggingAssembly.LockClass
        Waiting threads:  22
The output shows that threads 22 and 21 own a lock on objects of type LockClass. Thread 21 is waiting on 22 and Thread 22 is waiting on 21. Now you can use the ~*e !clrstack command to see a list of managed call stacks for threads 22 and 21 (see Figure 14).

Figure 14 Managed Call Stacks for Deadlocked Threads (Close Figure 14)
Thread 21
ESP         EIP
0x0143e9ac  0x7c90eb94 [FRAME: GCFrame]
0x0143ea60  0x7c90eb94 [FRAME: HelperMethodFrame] 0x79404fa0 is not a MethodDesc

0x0143ea8c  0x07c00136 [DEFAULT] [hasThis] Void ManagedProductionDebuggingAssembly.PurchaseOrder.Approve()
  at [+0x3e] [+0x22] d:\managedproductiondebuggingassembly\purchaseorder.cs:16707566
...
Thread 22
ESP         EIP
0x0147e9ac  0x7c90eb94 [FRAME: GCFrame]
0x0147ea60  0x7c90eb94 [FRAME: HelperMethodFrame] 0x79404fa0 is not a MethodDesc

0x0147ea8c  0x07c001bd [DEFAULT] [hasThis] Void ManagedProductionDebuggingAssembly.PurchaseOrder.Reject()
  at [+0x3d] [+0x22] d:\managedproductiondebuggingassembly\purchaseorder.cs:16707566
...

In Figure 14, there are two instances of LockClass within the PurchaseOrder object, one for each of the locked objects. To identify which lock object is being waited on in thread 22, you need to dump the contents of the lock object. You can use !DumpObj 0x0596943c followed by !DumpObj 0x05969404 to dump the contents of the lockName field within the LockClass (see Figure 15).

Figure 15 Value of Lock Name (Close Figure 15)
0:000> !DumpObj 0x05969404
Name: System.String
MethodTable 0x79b8db58
EEClass 0x79b8dea4
Size 44(0x2c) bytes
GC Generation: 0
mdToken: 0x0200000f  (d:\windows\microsoft.net\framework\v1.1.4322\mscorlib.dll)
String: objectStatus
FieldDesc*: 0x79b8df08
MT         Field     Offset Type         Attr     Value    Name
0x79b8db58 0x4000013 0x4    System.Int32 instance 13       m_arrayLength
0x79b8db58 0x4000014 0x8    System.Int32 instance 12       m_stringLength
0x79b8db58 0x4000015 0xc    System.Char  instance 0x6f     m_firstChar
0x79b8db58 0x4000016 0      CLASS        shared   static   Empty
    >> Domain:Value 0x001676f0:0x058f126c <<
0x79b8db58 0x4000017 0x4    CLASS        shared   static   WhitespaceChars
    >> Domain:Value 0x001676f0:0x058f1280 <<

If you review the lock for thread 21 you will see it acquired the lineItems lock. Now focus on these locks in both the Approve and Reject methods in the PurchaseOrder class. The order the locks are acquired should be changed so they are both acquired in the same order in both methods.

Crashing

The most common cause of a crash in unmanaged apps is an access violation caused by illegal reads and writes of memory. If ADPlus is run in crash mode, a full dump will be created which can then be analyzed using the same techniques I discussed for a first-chance exception.

Although managed applications can call unmanaged code through either COM interop or platform invoke, second-chance exceptions that cause the application to crash rarely occur because access violations are converted into managed exceptions. In this case, the !dumpstack command can prove very useful to show a call stack with both managed and unmanaged calls.

Conclusion

This article has discussed the importance of postmortem analysis of production issues and its utility when using the dump files and WinDBG. A methodical approach to any issues discovered via dump files should be able to isolate the cause of most issues that are difficult or impossible to reproduce in-house. I suggest you walk through the examples using the download available for this article to familiarize yourself with the tools, and then apply these same techniques to the next production issue you look at that is difficult to reproduce. Once you start creating dumps, you will never look back!

Matt Adamson is a senior software engineer at Tranmit plc where he designs and develops n-tier applications using C# and C++. You can reach him at adamson_matthew@hotmail.com and mattadamson.blogspot.com.


Editor's Note: Designing the PC of the Future

Editor's Note
Designing the PC of the Future

While spending some idle moments poking around on the Web between working on articles the other day, we ran across a design challenge that Microsoft is co-sponsoring. Along with the IDSA (Industrial Designers Society of America), Microsoft is inviting all you designers out there to draw up plans for the next-generation PC. This isn't software-related—it's a quest for the best new PC design. You can win up to $125,000USD if you submit the winning entry at www.startsomethingpc.com.

Now, it's widely known that editors don't always have the top design skills one would hope for in a contest like this. Frankly, we thought there was no hope of snatching top honors in this contest, but then we saw an interesting clause in the ground rules:

I'm not a designer, can I enter?

Sure, but you're going to want to do some studying up on some engineering concepts. Designs that clearly would require a heat sink the size of a mid-sized sedan or cost more than an SUV probably won't get very far. Our tip is to be practical yet innovative. Think about the pain points today for users and how you would solve them.

Perfect! This contest is open until October, so there's plenty of time to create the perfect PC. We've sketched out a few ideas for this next-gen PC, so please feel free to send us your critique. (Or better yet, try the contest itself.)

  1. The biggest pain point for users is when the hard drive crashes, taking down months of work that we were "just about to" back up. Doesn't matter whether they're big or small—hard drives will always die without warning at the worst time. That's why in the next-gen PC, the hard drive will be replaced with an array of hundreds of compact flash cards.
  2. Current PCs come in drab colors—usually gray or black. The next-generation PC will come in a variety of garish colors that will not match any decorating scheme devised after 1977. Fuchsia, neon yellow, electric blue—all excellent options that let people know that this machine has truly been designed.
  3. On/off switches are so passé. They should be replaced by a built-in clapper device.
  4. The next-gen PC will have fifty—no, sixty—USB sockets. If we're using the machine in the kitchen, we want to be able to power all sorts of 5VDC appliances with it as well.
  5. What could be more functional than a CD tray that works as a cup holder? And not one of those little ones, either—it should accommodate anything from a can of soda to a Super Big Gulp.
  6. Since landfills are filling up with abandoned electronics, the next-gen PCs should be made out of recyclable paper products.
  7. The PC should have a red eye on the front. Every time the user tries to do something on the machine that's not allowed by group policy, the eye will beep and the machine will say "I'm afraid I can't do that, domain-slash-username."
  8. Ultimately, a PC should consist of nothing but a large screen that smoothly descends from the ceiling whenever you walk into the room. It will be controlled entirely by the wonderment shown by a small child who accesses the Internet by touching the screen with a single finger. But in reality, you'll need an extra three feet of ceiling clearance for this design, the screen will get stuck in the down position after about a week, and it'll be covered with grubby chocolate fingerprints in no time.

So there you have it—our editorial dream for the perfect PC of the future. If you think you can design something even better, why not head over to the site today and download an entry form?

—J.T.

Have you ever wanted your own archive of MSDN Magazine articles, but were afraid to ask? Well, do we have excitement for you! The MSDN Magazine 2004 CD is available now. This spiffy CD contains all of our content from 2000 through 2004, plus all the source code. And if that's not enough, we've also included an index of all articles appearing in Microsoft Systems Journal and Microsoft Internet Developer from 1986 to 2000. You know you want one. Just go to our site and place your order today!

Thanks to the following Microsoft technical experts for their help with this issue: Brad Abrams, Christopher Brumme, Israel Burman, Tom Christian, Scott Densmore, Michael Harsh, David Janson, Chris King, Ashwin Kowtha, Ronald Laeremans, Bertrand Le Roy, Gerrard Lindsay, Stan Lippman, Haibo Luo, Subhag Oak, Peter Provost, Stefan Schackow, Kent Sharkey, Craig Skibo, Mike Stall, and Yiru Tang.

FrontPage, IntelliSense, Microsoft, MSDN, MSN, Outlook, Visual Basic, Visual C++, Visual C#, Visual Studio, Windows, Win32, Xbox, and Windows Server are either registered trademarks or trademarks of Microsoft Corporation. Other trademarks or tradenames mentioned herein are the property of their respective owners.

MSDN Magazine does not make any representation or warranty, express or implied with respect to any code or other information herein. MSDN Magazine disclaims any liability whatsoever for any use of such code or other information.


New Stuff: Resources for Your Developer Toolbox

New Stuff
Resources for Your Developer Toolbox
Marnie Hutcheson

Contents
Stunning ASP.NET Web Sites
Create Help Files Inside Visual Studio .NET
Bridging the World of Java/J2EE and .NET
Include Interactive Diagrams in Your UI
A Fast Data Grid Control
Add Custom Functions to Your PDF Files
The Bookshelf

Stunning ASP.NET Web Sites

r.a.d.controls by telerik gives you an impressive set of great looking interface components, Visual Studio® .NET design-time support, cross-browser compatibility, keyboard support, and data binding.

The first thing that struck me when I went through the demos for this toolset was how professional the Web pages created with these controls can look. This new release includes major version updates of the r.a.d.treeview, r.a.d.chart, and r.a.d.panelbar components, as well as a minor update of r.a.d.spell.

The scrolling news box and the dynamic charts are cool, but I was most impressed by the tree, panel, menu, and editor controls. The r.a.d.chart component gives you five new chart types and supports image maps and postback events so your users can drill down in your charts.

The r.a.d.treeview component has a client-side load on demand mechanism that lets you populate child nodes on demand without a full page postback to the server. There is also a new node sorting feature and the ability to bind the treeview component to data sources.

Forget "graphically challenged" and don't worry about what a multiple-line menu option is going to do to your menu layouts. The r.a.d.menu control takes care of those details for you. You just pick the style and skin you want (for example, Microsoft® Windows® or Outlook®) and presto—you get great looking menus that you can drag and drop into your ASP.NET project. r.a.d.menu generates code that can be crawled and indexed by search engines with the menu rendering a list of hyperlinks even when JavaScript is disabled or not installed.

The r.a.d.editor component looks and works just like Microsoft Word, so there is no scare factor for office folks who are tasked with entering information in your application or updating their own Web sites. They can paste or type in their content, spell check it, strip out proprietary formatting, and submit it as RTF or HTML. In addition, the r.a.d.editor supports localization and is compliant with Section "508" Accessibility standards.

You get a large set of configuration options for the r.a.d.editor that let you control things like who can edit what, where content is going to be saved, what content is going to be saved as, and so on. The r.a.d.controls use a role-based security model that you configure.

The r.a.d.designer component lets you set up your own page layouts called content templates. You specify what goes in each area—the menu in the left border, the top banner and page footers, and content areas. When you start a new page, you just pick the layout you want to use and the page is built with all your components and graphics already in place. You just tweak, edit, or add what you need to the template to finish your new page.

Price: $799 per single-user license.

www.telerik.com

Create Help Files Inside Visual Studio .NET

Help Publisher for Visual Studio 2003 by Indigoware gives you all the necessary project templates, editors, import wizards, and content tools to create, compile, and preview help files from inside Visual Studio .NET. In fact, you can compile your help files as part of your application build.

Help Publisher gives you new templates to create both HTML Help 1.x and Microsoft Help 2.0 format help files for your applications. You can convert existing content into HTML help files and you can publish your help topics to Word. Help Publisher for FrontPage® and Dreamweaver interoperate with the new Help Publisher for Visual Studio so you can import existing help content from HTML Help Workshop projects or FrontPage and Dreamweaver.

Help Publisher also gives you tools that allow you to link topic pages to Context IDs, simplifying the task of adding context-sensitive Help to applications created in Visual C++®, Visual Basic® .NET, and Visual C#® .NET.

Price:$99 per single-user license.

www.indigoware.com

Bridging the World of Java/J2EE and .NET

When you use J-Integra for .NET by Intrinsyc your .NET software can access existing Java/J2EE applications as though they were actually .NET-based applications. It is fully bidirectional, so your Java/J2EE software can also access .NET components as though they were written with Java.

The J-Integra for .NET plug-in for Visual Studio .NET lets you incorporate J-Integra for .NET into your client applications from inside the Visual Studio .NET environment. It provides all the functionality necessary to create a working client, including generating .NET proxies for Java classes, and generating a .NET Remoting configuration file that requires no manual editing.

J-Integra supports the use of C++, Visual Basic .NET, C#, and other .NET-based languages with standard Java objects. You can use Microsoft .NET Remoting from Java (fast binary format, HTTP/TCP channels), which means that you can do things such as pass objects by reference. It has support for events and transparently maps .NET Collections to and from Java collections.

If you are using Microsoft technology for client software applications and Java technologies for back-end systems, J-Integra will let you continue to grow both technologies without having to retrain staff or rewrite existing components.

Price: $1,199 per end-user J2EE license.

www.intrinsyc.com

Include Interactive Diagrams in Your UI

If you need to write .NET-based applications with interactive graphical diagrams in the interface, then GoDiagram by NorthWoods can save you tons of effort. GoDiagram makes it easy to build applications where users can see and manipulate graphs of nodes connected by links.

GoDiagram supports multiple styles of links and nodes, automatic layout, multiple document layers, and subgraphs. In addition, it gives you the ability to include dials and gauges within your graphical diagrams.

When you drop the GoDiagram component onto a Windows Form you can use its default behaviors, such as clipboard, zoom, scroll, selection and multiple selections, and drag and drop between applications, to build graphical applications quickly. You can customize GoDiagrams just by setting properties on views, documents, and objects, or by providing event handlers for the controls.

Free evaluation kits are available for each flavor of GoDiagram, including the basic controls, the AutoLayout option, and user guides.

Price: $199 and up per single-user license.

www.nwoods.com/go

A Fast Data Grid Control

FlyGrid.Net 1.0 by 9Rays.Net is a data grid control for Windows Forms that features Shift/Ctrl cell block selection, a split view capability so users can split the grid into several horizontal and vertical split views and resize them, full support of RightToLeft languages, and numerous drag and drop options. FlyGrid .NET 1.0 supports unlimited levels of nested grids within each row to display master/detail views or hierarchical data from several data sources. It works in bound, unbound, and virtual modes and it can mix these modes. You can group and sort data on several columns. You can also customize summaries and filters at run time.

Price: $179.95 and up.

www.9rays.net

Add Custom Functions to Your PDF Files

AcroButtons Version 2.0 by WindJack Solutions Inc. lets you create, manage, and embed custom toolbar buttons in your PDF documents. These buttons travel with your document. They show up as custom toolbar buttons (with the style and image of your choice) when someone opens your PDF file in Acrobat and the free Acrobat Reader.

No matter how the document is zoomed and scrolled, your buttons will always be visible on the toolbar. You can place Submit, Navigate, Search, and Help buttons (and even your company logo with a link to your Web site) on the toolbar where they will always be visible.

Toolbar buttons created with AcroButtons are built entirely with Acrobat JavaScript. Once you build them, they are independent of the AcroButtons tool. They can be used in every Acrobat product (6.0 or greater) that executes JavaScript. You build it once, use it everywhere, and distribute it to anyone.

This release includes icon images and button actions to help you get started. The Image Prep Tool helps you create the best button from images you select or import with cropping, masking, scaling, and automatic sizing features. You can import button icons from various image formats, and you can create your own button actions from scratch, or by customizing the predefined button actions that come with the product.

Price: $99 per single-user license.

www.windjack.com

The Bookshelf

Any book that makes it to a fourth edition is doing something good for its readers. Jesse Liberty, the author of Programming C#, 4th Edition (O'Reilly, 2005), says "C# builds on the lessons learned from C (high performance), C++ (object-oriented structure), Java (garbage collection, high security), and Visual Basic (rapid development) to create a new language ideally suited for developing component-based, n-tier distributed Web applications."

The book begins with a quick but thorough tour of C# language syntax and introduces the keywords, concepts, and details that make C# and .NET an effective environment for building desktop and Web applications. Then, the author takes you into the code and shows you how to use C# to write typical Web and Windows applications using, ASP.NET, Windows Forms, and ADO.NET.

He also explains how to build standards-based Web services, how to manage deployment with assemblies, work with metadata, and more.

Price: $44.95, 644 pages.

www.oreilly.com

All prices were confirmed at press time and are subject to change.

Send your New Stuff to newstuff@microsoft.com.

Marnie Hutcheson is president of Internet Development Associates (Ideva), a firm in Ocala, Florida, that specializes in Internet and intranet Web application design and development. She has published technical papers and books on various computing topics. You can reach her at marnie@ideva.com.


Web Q&amp;A: Web Page Layout, Quirks Mode, and More

Web Q&A
Web Page Layout, Quirks Mode, and More
Edited by Nancy Michell

Contents
Debugging Without Admin Rights

Q I've heard it's better to use absolute/relative positioning as opposed to a table-driven layout on a Web page. Is that true? I've also heard that it's really better to use cascading style sheets (CSS) only and try to avoid the use of JavaScript altogether. What are the issues I will most likely encounter when I make the choice?

A There was one case here at Microsoft in which a site had to be localized to work with 31 different languages. The site was originally written for English only and had absolute/relative positioning all over the place. Once all the strings were translated, the site looked really horrendous because everything was positioned based on the length of strings in English. To fix the problem, all the HTML had to be rewritten to use table layouts to produce a consistent look across languages.

On the other hand, table layout is generally not compliant with accessibility standards; you need to use CSS for that. Tables are appropriate for table data such as financial data and other lists of information, but only in specific scenarios. You should use a text reader on your site to see if it is accessible—it's a good exercise for anyone who is not vision-impaired.

You can recreate pretty much any table layout using CSS. It's not always fun, but it will work. There's been a move away from tables and towards CSS layout in a few places in Microsoft, most notably at www.msn.com. You can view the source if you're interested. There are also some findings at www.danboe.net.

You also need to consider whether or not you want to use "quirks" mode (explained in the next question). When you don't use quirks mode in Microsoft® Internet Explorer and other browsers, the layout challenges really begin.

Some Web controls (like the DataGrid and file upload) require additional CSS tricks (horizontal line only, row height, and so on) and extra layout work for browsers other than Internet Explorer to meet the compliance requirements.

Q Can you tell me what the terms "quirks mode" and "standards mode" refer to?

A In the early days of the Web, different browsers exhibited differ-ent behavior for the same HTML code. Along with the later browser versions (Internet Explorer 6.0, for example), came more rendering standardization. One major player in the standards game was the Document Type Definition (DTD) which helped pages comply with W3C rendering recommendations.

However, when these new standards and new browser interpretations are enforced, older Web pages often render incorrectly and can be both unappealing and dysfunctional because the new rendering engines often can't make sense of the convoluted coding these pages employed to make the layout look the way the designer wanted it. As you may remember, some strange contortions were required in the old days to make pages look nice.

Under this new system, when one of these later browsers receive a document with a DTD indicated in the DOCTYPE element, the browser renders the page according to these new standards—this is called standards mode. If there is no DOCTYPE element, the DOCTYPE is unclear, or if any one of many other conditions exist, such as DOCTYPEs that state the version of HTML that should be rendered, then the browser renders in quirks mode. Of course there are other modes as well. Content that is identified to the browser as text/xml or other XML or XHTML MIME types is rendered in strict mode.

While there is more to this topic, the basic answer is that the mode tells the browser how to interpret the layout code for the page. For more information on rendering compatibility, see compatMode Property.

Q More and more I'm hearing about DTDs. Can you explain what they're used for?

A A DTD is a Document Type Definition, and a DOCTYPE is the HTML element that indicates to a browser exactly which DTD it should use. As noted in the previous answer, a DTD gives the browser information about how to render the page. The following shows an example:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
   "http://www.w3.org/TR/html4/loose.dtd">

This DOCTYPE element points the browser to the exact location of the DTD. In each DTD, all elements and text styles, and what they mean, are defined. Numbering style, bullet styles, alignments, margins (in pixels), and tons of other specific rendering instructions are spelled out. If you design a Web page using a particular DOCTYPE definition but then refer to a different DTD, the page will not look as you intended. But choosing a DTD does eliminate the problem of different browsers rendering pages differently and unexpectedly.

Remember also that if you want to validate your HTML using a tool, you will need to tell the tool what to measure against, and you do that by indicating a DTD. The W3C's HTML validator can be found at validator.w3.org, and other validators are available from www.w3.org/QA/Tools/#validators for your validation needs.

Web pages are much more complex today than they were a few years ago, and readers expect a professional look. Using DTDs keeps you from leaving the appearance of your Web site and the location and presentation of all its elements to chance. There's lots more to explore on this topic at Working with HTML DOCTYPE Declarations in FrontPage.

Q I want to prevent users from clicking a button in my application multiple times. Each time this button is clicked, a new record is added to the database and I don't want duplicate records added.

I've tried various ways to accomplish this, but with little success. The main issue I've encountered is having the server-side code still execute when I disable the button on the client side, while still allowing the validation controls to work. Is this possible?

A The code in Figure 1 illustrates how you can disable the controls upon submission. One thing you should take into consideration is that if the user clicks the Stop button while the page is posting, then you will end up with a page that might have the buttons disabled and may be unusable. This code has been tested with validator controls and, with their current implementation, it seems to work fine.

Figure 1 Prevent Repeated Submission (Close Figure 1)

DisableCtrlOnSubmit
public class DisableHelper {
    public static void DisableCtrlOnSubmit(
            System.Web.UI.Page Page, params string[] strControls) {
        StringBuilder sbJS = new StringBuilder(200);
        sbJS.Append ("<script language='javascript'>");
        sbJS.Append (@
          "function MSDisable() {
             try {
               if (Page_IsValid==false) return;
               window.setTimeout('MSDisableCtrls()',0);
             }
             catch(e) {
               window.setTimeout('MSDisableCtrls()',0);
             }
           }");
         sbJS.Append ("function MSDisableCtrls() { ");
         foreach (string strControl in strControls) {
             sbJS.Append (strControl);
             sbJS.Append (".disabled=true;");
         }

         sbJS.Append ("}</");
         sbJS.Append ("script>");
         if (!Page.IsClientScriptBlockRegistered("MSDisableImpl")) {
             Page.RegisterClientScriptBlock(
                 "MSDisableImpl", sbJS.ToString ());
         }
         Page.RegisterOnSubmitStatement ("MSDisable", "MSDisable();");
     }
 }
Using DisableCtrlOnSubmit in a Page
<html>
    <head>
        <title>Disable Ctrls</title>
    </head>
    <body>
        <form id="Form1" method="post" runat="server">
            <asp:textbox text="0" id="TextBox1" runat="server" />
            <asp:requiredfieldvalidator id="RFV1" runat="server"
                errormessage="Required" controltovalidate="TextBox1" />
            <asp:comparevalidator id="CFV1" runat="server"
                errormessage="Numeric" operator="DataTypeCheck"
                controltovalidate="TextBox1" type="Integer" />
            <br />
            <asp:textbox text="0" id="TextBox2" runat="server" />
            <asp:requiredfieldvalidator id="rfv2" runat="server"
                errormessage="Required" controltovalidate="TextBox2" />
            <asp:comparevalidator id="cfv2" runat="server"
                errormessage="Numeric" operator="DataTypeCheck"
                controltovalidate="TextBox2" type="Integer" />
            <br />
            <asp:label id="lblResult" runat="server" text="Result:" />
            <asp:button id="cmdAdd" onclick="Add"
                runat="server" text="Add" />
            <asp:button id="cmdSubstract" onclick="Substract"
                runat="server" text="Substract" />
            <br />
            <asp:validationsummary showmessagebox="True"
                id="ValidationSummary1" runat="server" />
        </form>
        <script runat="server" language="C#">
            void Page_Load (System.Object sender, System.EventArgs e) {
                DisableHelper.DisableCtrlOnSubmit (
                this, "Form1.cmdAdd", "Form1.cmdSubstract");
            }
            void Add (System.Object sender, System.EventArgs e) {
                lblResult.Text = (Int32.Parse (TextBox1.Text) +
                    Int32.Parse(TextBox2.Text)).ToString();
                System.Threading.Thread.Sleep (2000);
            }
            void Substract (System.Object sender, System.EventArgs e) {
                lblResult.Text = (Int32.Parse (TextBox1.Text) –
                    Int32.Parse(TextBox2.Text)).ToString();
                System.Threading.Thread.Sleep (2000);
            }
        </script>
    </body>
</html>

The idea is that you only disable controls you want to disable (of course it could be used to disable a Div or a Panel or something that contains other controls), so in the Page_Load you just add any controls that support the HTML-disabled attribute that you want to disable on postback, like so:

DisableHelper.DisableCtrlOnSubmit(this, "Form1.cmdAdd",
    "Form1.dropDownList1", "Form1.cmdOk", "Form1.cmdCancel");

You might want to consider changing the code so that you pass a reference to the button and use GetPostBackEventReference so that your controls inside other naming containers work.

Q I want to be able to develop and debug ASP.NET solu-tions without giving developers admin rights on either the client or the server. I realize that there are a couple of Knowledge Base articles on debugging without admin rights but my developers need to be able to set up new solutions in Visual Studio® .NET 2003 without admin rights to the Windows Server™ 2003 installation.

A The question of which rights to allow developers is one of risk versus productivity, in that there are tasks that developers commonly perform that may require administrator rights. Restricting developers to only user-level rights (even on their own machine) means that someone else will need to perform these tasks (such as installing new tools, for example) for them. Alternatively, you can provide your developers with both a low-privileged account and a local administrator account, and have them use RunAs to elevate their privileges where necessary to accomplish administrative tasks (this is automated when running Setup.exe packages, so a low-privileged user will automatically be prompted for elevated credentials for the install). Of course, the developers should be strongly encouraged to run using the low-privileged account by default.

It's important to balance risk against productivity. If developers have too few privileges, they will be unable to replicate the target user environment, and will not be able to test their own code. With too many privileges, they will not be able to test how well their code behaves under the target user's privilege, and the application produced may require unnecessary privilege levels in order to run. These kinds of bugs, sometimes referred to as LUA bugs (for least-privilege user account), result from developing with elevated privilege, and can be very painful and expensive to track down and fix.

Please note that Visual Studio 2005 will ship with out-of-the-box support for creating and debugging ASP.NET applications as a least-privileged user, without requiring IIS at all, using a local-only Web server that comes with Visual Studio 2005. It's important to note that this will require additional testing to ensure that your code works in your deployment environment, but this feature is designed to support precisely the scenario described.

Debugging Without Admin Rights

Here are some resources for developing and debugging ASP.NET applications without admin rights:

  • Configuring ASP.NET to run using the developer's account info
  • Developing software in Visual Studio .NET with Non-Administrative Privileges
  • Setting Up a Shared Development Server for ASP.NET on Windows Server 2003 (MSDN® Academic Alliance)

Send your questions and comments to webqa@microsoft.com.

Thanks to the following Microsoft developers for their technical expertise: Gaylon Blank, Daniel Boerner, Greg Bybee, Jim Cheshire, John Conwell, G. Andrew Duthie, Sean Gephardt, Brian Hitney, Alun Jones, Carlos Aguilar Mares, Eduard Rusanovschi, Michael Schmidt, Matthew Smolen, and James Whittred.


Advanced Basics: Creating A Breadcrumb Control

Advanced Basics
Creating A Breadcrumb Control
Duncan Mackenzie

Code download available at: AdvancedBasics0507.exe (152KB)
Contents
Building a Breadcrumb Control
Drawing and Clicking
All the Bells and Whistles
Conclusion

Hansel and Gretel had the right idea when "they followed the pebbles that glistened there like newly minted coins, showing them the way." The deeper you get into the forest or into your data, the more likely you are going to need help to find your way back out again. On the MSDN® Web site, and many others, the way out is represented by a navigational element in the header of the page that shows your current position in the site. Clicking on any of the links to the left of your current position takes you to that location, providing a quick path to find more related information or to get back to the starting point of the site.

This type of navigation is often referred to as "breadcrumbs" and is most appropriate when your information is organized in a hierarchical structure, with many levels, and works well in the same situations in which a TreeView would be used. Data often maps well to a tree-like structure, so this style of data representation is often found in all sorts of apps, both Windows® and Web-based.

My own applications are no exception; I'm currently working on a new app in which the main method of navigation will be through a tree that displays the structure of a complete Web site, and another tree that represents a single page. Because I am so familiar with the breadcrumb control on MSDN, I decided that I would employ that functionality in my application.

Building a Breadcrumb Control

When the user selects a single page deep within the site, I would like to show them their current position and, at the same time, make it easy to move back up the hierarchy to any point. They will have a TreeView, of course, but I don't think that a tree is as clear and easy as a horizontal list of past locations. So, I decided that I would create a breadcrumb control in Windows Forms.

When you are looking at the code sample, you'll notice that my control is called an "Eyebrow" instead of a breadcrumb. Eyebrow is the name used in MSDN code, and it just stuck in my mind. You're probably wondering how this control relates to an eyebrow. So am I. I know my eyebrows don't have any information about my current position, and they certainly don't help me get around, but that's what they're called in the code, so that's how it'll be.

Back to the control. The control works by being associated with a TreeView. You can configure that association programmatically or through the property grid in Visual Studio® at design time. The control's rendering is then based on information in the associated TreeView, namely the currently selected node.

By hooking the tree's selection changed event (AfterSelect), the control is notified whenever it needs to be redrawn. The associated TreeView is accessed directly to find the currently selected node, to navigate up through all of the parent nodes, and to change the selected node when the user clicks on one of the hyperlinked items. By obtaining all of its navigational information from the TreeView, the breadcrumb control doesn't have to know anything about your particular application. As long as you set up and populate your TreeView and handle the tree's events, this control should work within your application.

Drawing and Clicking

The real core of the control falls into two areas: rendering the control and tracking which areas of the control should be clickable. The first task, the rendering, isn't doing anything really special if you are used to GDI+ coding. I build up a collection of tree nodes in the current node's path, then walk through them from top to bottom, and write out the appropriate text, as shown in Figure 1.

Figure 1 Rendering the Control (Close Figure 1)
Dim bounds As Rectangle = Me.ClientRectangle()

If Me.ClickAreas Is Nothing Then
    Me.ClickAreas = New ClickAreaCollection
Else
    For Each ca As ClickArea In Me.ClickAreas
        ca.Dispose()
    Next
    Me.ClickAreas.Clear()
End If

Dim nodes As New myTreeNodeCollection
Dim parentNode As TreeNode = m_tree.SelectedNode

Do While Not parentNode Is Nothing
    nodes.Insert(0, parentNode)
    parentNode = parentNode.Parent
Loop

Dim startingPos As Point = bounds.Location

For Each n As TreeNode In nodes
    Dim p As String = n.Text

    If Not n Is m_tree.SelectedNode Then
        Dim r As Region = DrawText(p, startingPos, g)
        Dim ca As New ClickArea
        ca.Region = r
        ca.Node = n
        Me.ClickAreas.Add(ca)
        DrawDelim(Me.m_delim, startingPos, g)
    Else
        DrawDelim(p, startingPos, g)
    End If
Next

As I go through this drawing loop, I'm building up a collection of ClickArea objects, which associate a specific tree node with an area of the control. This process, and the collection it creates, is the key to properly handling mouse over and click events.

When the user brings the mouse over the control, a search through the collection of ClickArea objects is executed (shown in Figure 2), and each one in order is checked to see if the current mouse position is contained within the associated Region. If the mouse is over a clickable region, the pointer is changed to the Hand cursor, indicating to the user that something should happen if they click on this point (see Figure 3). The code that changes the cursor is shown in Figure 4.

Figure 4 Changing the Cursor (Close Figure 4)
Protected Overrides Sub OnMouseMove(ByVal e As MouseEventArgs)
    Dim pos As New Point(e.X, e.Y)
    Dim node As TreeNode
    node = FindNode(pos)

    If Not node Is Nothing Then
        Me.Cursor = Cursors.Hand
    Else
        Me.Cursor = Cursors.Default
    End If
End Sub

Private Function FindNode(ByVal pos As Point)
    For Each ca As ClickArea In Me.ClickAreas
        If ca.Region.IsVisible(pos) Then
            Return ca.Node
        End If
    Next
    Return Nothing
End Function
Figure 2 Defining the ClickArea Collection (Close Figure 2)
Public Class ClickArea
    Implements IDisposable

    Private m_node As TreeNode
    Private m_region As Region

    Public Property Node() As TreeNode
        Get
            Return m_node
        End Get
        Set(ByVal Value As TreeNode)
            m_node = Value
        End Set
    End Property

    Public Property Region() As Region
        Get
            Return m_region
        End Get
        Set(ByVal Value As Region)
            m_region = Value
        End Set
    End Property

    Public Sub Dispose() Implements System.IDisposable.Dispose
        m_node = Nothing
        If Not m_region Is Nothing Then
            m_region.Dispose()
            m_region = Nothing
        End If
    End Sub
End Class
Figure 3 Hovering Over a Link
Figure 3 Hovering Over a Link

In the OnClick procedure of the control, the same search is performed, but in this case if a node is found, the current selection of the tree is updated to point to the clicked item:

Protected Overrides Sub OnClick(ByVal e As System.EventArgs)
    Dim pos As Point = Me.PointToClient(Me.MousePosition())
    Dim node As TreeNode
    node = FindNode(pos)

    If Not node Is Nothing Then
        Me.m_tree.SelectedNode = node
    End If
End Sub
Updating the SelectedNode of the tree will in turn cause the tree's AfterSelect event to fire, causing the Eyebrow control to be redrawn.

All the Bells and Whistles

Never content to leave something in its simplest state, I added some additional properties to the control: AutoSize, Wrap, and ShowImages. The first two are closely related, because AutoSize only works if Wrap is set to true. Setting both these properties to true causes the control to wrap text to a new line if necessary (Wrap) and to increase its height as required to display all the text (AutoSize). If a single node's text is wider than the control, then it is trimmed instead of wrapped because my code only supports wrapping between complete items. This is shown in Figure 5.

Figure 5 AutoSizing and Wrapping (Close Figure 5)
'within the DrawItem routine
Dim sz As SizeF = g.MeasureString(p, f)
If Not img Is Nothing Then
    sz.Width += img.Width + Spacer
    sz.Height = Math.Max(sz.Height, img.Height)
End If
Dim w As Integer = sz.Width
Dim lineSize, remainingSpace As Integer
lineSize = w + Spacer
remainingSpace = clientBounds.Width - startingPos.X
If (lineSize > remainingSpace) AndAlso _
   (lineSize <= clientBounds.Width) Then
     If m_wrap Then
        startingPos.X = clientBounds.Left
        startingPos.Y += sz.Height + Spacer
     End If
End If

ShowImages may be useful if you use images in your tree and if those images are meaningful to your users. If you set it to true, then when drawing out the individual item, the image is pulled from the TreeView's associated ImageList and drawn next to its text on the Eyebrow control, as shown here:

'grabbing the image from a TreeNode in OnPaint
Dim img As Image
If m_ShowImages AndAlso Not Me.Tree.ImageList Is Nothing Then
    img = Me.Tree.ImageList.Images(n.ImageIndex)
End If

'drawing the image next to each item
If Not img Is Nothing Then
    g.DrawImageUnscaled(img, startingPos)
End If

Including support for images complicates the sizing and wrapping code, but the visual effect will be worth it in most applications. The end result is shown in Figure 6.

Figure 6 AutoSizing, Wrapping, and Images
Figure 6 AutoSizing, Wrapping, and Images

Conclusion

As usual, I haven't built everything into this control that you will probably want; it is a sample after all. I can think of three key additions that you may eventually want: the ability to control the image position (upper left, middle right, and so on), an option to use a separator image between items, and more complex display options for when space is limited. The control in its present state is a good place to start, though, and should open up some interesting navigation options for your tree-based apps.

Send your questions and comments for Duncan to basics@microsoft.com.

Duncan Mackenzie is a developer for MSDN and the author of the Coding 4 Fun column on MSDN online. He can be reached through his personal site at www.duncanmackenzie.net.


Cutting Edge: DHTML-Enabled ASP.NET Controls

Cutting Edge
DHTML-Enabled ASP.NET Controls
Dino Esposito

Code download available at: CuttingEdge0507.exe (131KB)
Contents
Anatomy of a Postback
The Client-Side Counterpart
DHTML Behaviors
A DropDownList Example
The Extended Object Model
Summing It Up

In the past, I've covered some core aspects of the interaction between DHTML behaviors, the browser, and ASP.NET runtime (see Cutting Edge: Extend the ASP.NET DataGrid with Client-side Behaviors and Cutting Edge: Moving DataGrid Rows Up and Down). But I haven't covered the intricacies of DHTML behaviors and advanced client-side scripting so I'll do that here. I'll show how to make ASP.NET code and the Internet Explorer DHTML Document Object Model (DOM) work together and discuss how you set up the communication between the ASP.NET runtime and a server-side instance of an ASP.NET control.

Anatomy of a Postback

To design an effective mechanism for cooperation between DHTML and server-side controls, you need a solid understanding of the ASP.NET postback mechanism. Imagine you have a page with a couple of textboxes and a Submit button. When a user clicks the button, the page posts back. The post can be initiated in one of two ways—through a Submit button or through script. A Submit button is represented by the HTML: <INPUT type="submit">. Most browsers also support posting via the submit method in a <form> element. In ASP.NET, the second approach is used for LinkButtons and auto-postbacks.

When a submit operation is initiated, the browser prepares and sends an HTTP request according to the form's contents. In ASP.NET, the "action" attribute of the sending form is set to the URL of the current page; the "method" attribute, on the other hand, can be changed at will and even programmatically. Possible methods include GET and POST.

The postback for an ASP.NET page that contains a couple of textboxes, a dropdown list, and a Submit button looks like this:

_VIEWSTATE=%D...%2D
&TextBox1=One
&TextBox2=Two
&DropDownList1=one
&Button1=Submit

The contents of all input fields, including hidden fields, are sent as part of the payload. In addition, the value of the currently selected item in all list controls is added, as is the name of the Submit button that triggered the post. If there are one or more LinkButtons on the page, two extra hidden fields called __EVENTTARGET and __EVENTARGUMENT are added to the payload:

__EVENTTARGET=
&__EVENTARGUMENT=
&_VIEWSTATE=%D...%2D
&TextBox1=One
&TextBox2=Two
&DropDownList1=one
&Button1=Submit
Both of these hidden fields are empty if the page posts back through a Submit button. If you post back through LinkButton in the page, the payload changes as follows:
__EVENTTARGET=LinkButton1
&__EVENTARGUMENT=
&__VIEWSTATE=%D ... %2D
&TextBox1=One
&TextBox2=Two
&DropDownList1=one

In this case, the __EVENTTARGET field contains the name of the LinkButton that initiated the post. ASP.NET uses this information when constructing the server-side representation of the requested page and in determining what caused the postback.

On the server, IIS picks up the request and forwards it on to the ASP.NET runtime. A pipeline of internal modules processes the request and instantiates a Page-derived class. The page class is an HTTP handler and, as such, implements the IHttpHandler interface. The runtime calls the Page's ProcessRequest method through the IHttpHandler interface and the server-side processing starts. Figure 1 provides an overall view of the request process.

Figure 1 ASP.NET Postback Process
Figure 1 ASP.NET Postback Process

Once the server-side processing of the page has begun, the Page object goes through a sequence of steps, as outlined in Figure 2. As its first step, the Page object creates an instance of all server controls that have a runat="server" attribute set in the requested ASPX source file. At this time, each control is created from scratch and has exactly the same attributes and values outlined in the ASPX source. The Page_Init event is fired when all controls have been initialized. Next, the page gives all of its controls a chance to restore the state they had last time the posting instance of that page was created. During this step, each control accesses the posted view state and restores its state as appropriate.

Figure 2 Page Lifecycle Events (Close Figure 2)

Step | Point of Interception
1. Page initialization | Page_Init
2. Viewstate restoration | LoadViewState overridable method on the Control class
3. Process posted data | IPostBackDataHandler.LoadPostData on input controls
4. Page loading | Page_Load
5. Data changed event | IPostBackDataHandler.RaisePostDataChangedEvent on controls
6. Postback event | IPostBackEventHandler.RaisePostBackEvent on controls
7. Prerendering phase | Page_PreRender
8. Viewstate creation | SaveViewState overridable method on the Control class
9. Rendering | Render overridable method on the Control class
10. Page unloading | Page_Unload

At this point, each control's state must be updated with any data posted by the browser. For this to happen, a special conversation is set up between individual controls and the ASP.NET runtime. This is an important point to consider in light of client-side interaction.

The ASP.NET Page class looks up the Form or QueryString collection, depending on the HTTP verb that was used to submit the request. The collection is scanned to find a match between a posted name and the ID property of a server-side control created to serve the request. For example, if the HTTP payload contains TextBox1=One, the Page class expects to find a server-side control named TextBox1. Each ASP.NET control lives on the server but retains a counterpart on the client. The link between them is the string containing the control's ID.

While the ASP.NET Page class can successfully locate a server control with a given name, it has no idea of the type of that control. In other words, from the page perspective, TextBox1 can be either a TextBox, a DropDownList, a DataGrid, or a custom control. For this reason, the Page class processes the control only if it adheres to an expected contract—the IPostBackDataHandler interface. If the control implements that interface, the page invokes its LoadPostData method. The method receives the name of the control (TextBox1, in the example) plus the collection of posted values—that is, Form or QueryString. As an example, a TextBox control will extract the corresponding value ("One", in the example) and compare it to its internal state. This behavior is common to all input controls and to all controls that expect to receive input data from the browser. For example, a DataGrid control that allows users to change the order of columns using drag and drop will, at this point, receive the modified order of columns.

The LoadPostData implementation depends on the characteristics and expected behavior of the particular control. The TextBox control compares the posted string to the value of its Text property. The DropDownList control compares the incoming data to the value of the currently selected item. If the compared values coincide, the method returns false. If the values differ, then the relevant control properties are updated and the method returns true. Figure 3 shows an implementation for a TextBox control.

Figure 3 IPostBackDataHandler Implementation (Close Figure 3)
bool IPostBackDataHandler.LoadPostData(
    string name, NameValueCollection postedValues)
{
    string oldValue = this.Text;
    string newValue = postedValues[name];
    if (!oldValue.Equals(newValue))
    {
        this.Text = newValue;
        return true;
    }

    return false;
}

void IPostBackDataHandler.RaisePostDataChangedEvent()
{
    this.OnTextChanged(EventArgs.Empty);
}

LoadPostData for a TextBox control compares the value posted for a given control with the current value of the Text property. Note that at the time this comparison is made, the Text property contains the value just restored from the view state. From now on, the state of the control is up to date and reflects the old state and the input coming from the client. The Boolean value that LoadPostData returns indicates whether or not the second method on the interface—RaisePostDataChangedEvent—must be invoked later. A return value of true means that the value of Text (or the property or properties a control updates with posted values) has been refreshed and subsequently the TextBox raises a server-side data-changed event. For a TextBox control, this event is TextChanged.

Once this step has been accomplished, the Page_Load event is fired and a second check is made on the control that appears to be responsible for the postback (based on the information sent from the browser). If this control implements IPostBackEventHandler, the RaisePostBackEvent method is invoked to give the control a chance to perform the postback action. The following pseudocode illustrates the implementation of this method for the Button class:

void IPostBackEventHandler.RaisePostBackEvent(string eventArgument)
{
   if (CausesValidation) Page.Validate();
   OnClick(new EventArgs());
   OnCommand(new CommandEventArgs(
       CommandName, CommandArgument));
}

As you can see, when a button is clicked and the host page has completed its restoration process, the OnClick event is invoked, followed by OnCommand. A similar piece of code serves the LinkButton class. Code like this is used for any custom controls that require the post action to be started on the client.

The Client-Side Counterpart

Each server control outputs some markup that is sent down to the client. The browser then uses that information to build a DOM rooted in the outermost tag of the control's markup. Simple server controls such as TextBox map directly to HTML elements; more complex controls like the DataGrid map to a subtree of HTML elements, in many cases rooted in an HTML table tag. The root tag, or the most significant tag in the HTML, is given a name (the name HTML attribute) that matches the ID of the server control. This guarantees that the ASP.NET runtime can correctly match up client HTML elements with instances of server-side controls.

When you use or build an ASP.NET control with rich client-side functionalities you end up with at least two related problems. First, you have to figure out how to transfer to the server any input generated on the client. Second, you must make sure that the server control retrieves and properly handles that chunk of information. A third issue revolves around the format you use to send data across the wire.

There might be many ways to solve these issues and, frankly, any approach that works is valid. But when writing code for an ASP.NET control, why not do as the ASP.NET team did. That's where that anatomy of a postback fits in.

In the two articles that I mentioned at the beginning of this piece, I create a custom DataGrid control and collect some user input through drag and drop and other client-side operations. The input is then serialized to a string and packed into a hidden field. The hidden field is like any other <INPUT> tag except that it doesn't show up in the user interface. The hidden field is part of the form and its contents are picked up and used to prepare the HTTP payload when a postback is made. The hidden field is created by the server-side control and given the same ID of the control.

For example, a custom DataGrid named DragDropGrid1 will create its own personal hidden field with the same name. Any client-side action that is relevant to the behavior of the grid is persisted to the hidden field. When the page posts back, that information is carried to the server and consumed by ASP.NET in the manner described earlier. The matching ID determines the link between the contents of the input field and a server-side control. The control-specific implementation of the IPostBackDataHandler interface does the rest, giving the control a chance to modify its server state in light of client-side user actions.

If you get to consider simple and basic controls such as TextBox and DropDownList, then the <INPUT> element and the displayed user interface are the same thing. Any user interface-related operation automatically modifies the contents associated with the input element. This is much less automatic with more complex and advanced controls. Again, think of a DataGrid control that allows row movements or columns by drag and drop. The user interface of a DataGrid is a mere HTML table padded with plain text. The hidden field to carry data is silently created as part of the markup and injected in the page. Some additional code is needed to capture UI events and persist results to the hidden field.

What do you think this additional code should look like? Can it really be different from mere script code? It has to be pure JavaScript code at its core, but if the browser supports it, you can wrap it up in a more elegant and neater object model—that's mostly what a DHTML behavior is all about.

DHTML Behaviors

DHTML behaviors are a feature of Internet Explorer 5.0 and later. They're not supported by any other browser. A DHTML behavior component can be written in any Internet Explorer-compatible scripting language (usually JavaScript) and supplies dynamic functionality that can be applied to any element in an HTML document through CSS style sheets. DHTML behaviors use CSS to separate script and content in a document using an .htc that incorporates all the DHTML-based functionality needed to describe and implement a given behavior. This behavior, in turn, can be attached to a variety of HTML elements via a new CSS style. Put another way, DHTML behaviors bring the benefits of reuse to the world of scripting.

What's in a DHTML behavior? First, a behavior component can define an object model—a collection of methods, properties, and events that describe the provided behavior and supply tools to control it programmatically. In addition, a DHTML behavior needs to capture some page- and element-level events and handle them. You code this through classic HTML event handlers. You have access to the whole page DOM and can read and write attributes throughout the page. Figure 4 shows an HTC component that allows expanding and collapsing the children of the element to which it is applied.

Figure 4 DHTML Behavior Component (Close Figure 4)
<PROPERTY NAME="Expanded" />
<ATTACH EVENT="onreadystatechange" HANDLER="Init" />
<ATTACH EVENT="onclick" HANDLER="HandleClick" />

<script language="javascript">

// Handles the initialization phase
function Init() {
   if (Expanded == null)
       Expanded = true;

   // Toggle visibility for all children of THIS element
   for (i=0; i<children.length; i++) {
      if (Expanded == true)
         children[i].style.display = "";
      else
         children[i].style.display = "none";
   }
}

// Handles the OnClick event on the current element
function HandleClick() {
   var i;
   var style;

   // Make sure the sender of the event is THIS element
   if (event.srcElement != element)
      return;

   // Toggle visibility for all children of THIS element
   for (i=0; i<children.length; i++)
   {
       style = children[i].style;
       if (style.display == "none")
       {
           style.display = "";
       }
       else
       {
           style.display = "none";
       }
   }
}

</script>

In DHTML, the expand/collapse functionality is achieved by toggling the value of the display attribute in the style object. A value of "none" keeps the element hidden; a value of "" (empty string) makes the element visible.

The core functionality is found in a <script> tag that collects public event handlers as well as internal functions and classes. Outside of the <script> tag, you define the object model of the behavior and the internal events it wants to handle:

<PROPERTY NAME="Expanded" />
<ATTACH EVENT="onreadystatechange" HANDLER="Init" />
<ATTACH EVENT="onclick" HANDLER="HandleClick" />

The preceding code snippet declares a variable named Expanded and a couple of handlers for the onclick and onreadystatechange DOM events. Properties can be assigned a value in the HTML source through the mechanism of attributes. Event handlers must be defined in the <script> tag. The onreadystatechange event is a common presence in many DHTML behaviors because it represents the initialization phase of the component. In Figure 4, you check the value of Expanded in the initializer and, based on that, you toggle the visibility value of child elements.

To attach a behavior, you use CSS notation (behaviors are ignored in browsers that do not support CSS):

<style> .LIST {behavior:url(expand.htc);} </style>

The CSS attribute is named "behavior". It is assigned a URL that ultimately points to the HTC file. Once you have defined a LIST class, you assign it to any HTML element that requires it:

<ul class="LIST" style="cursor: hand;" expanded="false">

As you see, any public properties defined on the behavior can be initialized as an attribute in any tags that contain style attributes.

Other than the public object model, DHTML behavior offers nothing that you can't get through plain scripting. But with a single attribute you can attach a certain behavior to a given HTML element or to the root of a HTML element subtree, as is the case with ASP.NET controls. A DHTML behavior can encapsulate a lot of details regarding the internal implementation of the behavior and it has full access to the page's DOM.

A DropDownList Example

In both aforementioned articles, I glossed over the code that specifically handles browser/server communication. Now it's time to focus on what the control needs to do in order to receive and properly process on the server any client-side input. The sample ExtendedDropDownList control is a custom control that is derived from the basic DropDownList control:

public class ExtendedDropDownList :
    System.Web.UI.WebControls.DropDownList
{
    ...
}

The most important difference between the basic and extended dropdown control is that the extended one exposes a client-side object model to let script code add elements dynamically. Newly added elements are added to the items collection and sent to the server out-of-band, that is outside the classic format that the HTTP payload takes when a dropdown control is involved.

In ASP.NET, the DropDownList control is designed to be read-only across postbacks. In other words, any list items dynamically added through DHTML code are lost once the page posts back to the server. The canonical HTTP payload doesn't include the items in the dropdown list. It only mentions the ID of the currently selected item. Control-specific information generated on the client can get to the server only in a hidden field. The hidden field can be given an arbitrary name, but in general you give the worker hidden field the same ID as the server control. As explained in the earlier "Anatomy of a Postback" section, this guarantees that the ASP.NET runtime invokes the methods of the IPostbackDataHandler interface on the control to post incoming data. For a custom DropDownList control, though, things are a little bit different. In fact, the base DropDownList control already requires an input element with the same name as the control's ID. This is a <SELECT> element:

<SELECT name="DropDownList1">
   <OPTION value="one">One
   <OPTION value="two">Two
   <OPTION value="three">Three
</SELECT>

A custom and enhanced dropdown list control requires an extra hidden field to carry the text and IDs of the additional items appended at the client. To avoid naming conflicts, this hidden field must have a different name. In my code, I use "_My" to postfix the ID. This is arbitrary, but once you choose a naming convention you must stick with it. In the source code of the customized ExtendedDropDownList control (available in the code download), the control defines a Boolean property to enable client-side insertion and overrides the OnPreRender method. The OnPreRender method simply registers the additional hidden field with the predetermined name. The hidden field is empty when the page is rendered to the client and will be filled as the user works with the control adding items dynamically to the list.

What about the contents of the hidden field? Should you use any specific format or convention? The data being passed to the server should be laid out according to a format that is known to both the server control and the client-side DHTML behavior. The format you choose is arbitrary as long as it achieves the expected goals. In this example, I'll use a pipe-separated pair of strings for each dynamically added item (note that without proper escaping, this prohibits the use of the pipe character in the actual values). The left part of the pair represents the text; the right part is for the ID. Here's an example:

One|id1,Two|id2

The custom ExtendedDropDownList control must implement the IPostBackDataHandler interface from scratch. The implementation on the base class is marked as private and can't be invoked from within a derived class.

The code in the LoadPostData method serves two main purposes. First, it manages the index of the selected item. The ID of this item comes through the HTTP payload and is matched against the current contents of the dropdown list. The index found (if any) is then used to overwrite the value of the SelectedIndex property. If this results in a change to the existing value, the host page gets a SelectedIndexChanged event. It is worth noting here that the event is not raised by the ASP.NET runtime. To be more precise, when LoadPostData returns true, the ASP.NET runtime invokes the RaisePostDataChangedEvent method on the same IPostBackDataHandler interface. By implementing this method, a control can fire a proper event.

The second goal of LoadPostData for the ExtendedDropDownList control is populating the Items collection with the new elements just added on the client. As mentioned, text and ID of these new elements are stored in the control's hidden field.

You can decide to raise an ad hoc event to signal that new elements have been added on the client. To do so, you define a private Boolean variable (_addedNewElements in the example) that is set during the execution of LoadPostData only if new elements have been added, as shown here:

void IPostBackDataHandler.RaisePostDataChangedEvent()
{
   OnSelectedIndexChanged(EventArgs.Empty);

   if (_addedNewElements)
      OnNewItemsAdded(EventArgs.Empty);
}

NewItemsAdded is a custom server-side event that is fired right after SelectedIndexChanged and before the postback event caused by the submit control:

public event EventHandler NewItemsAdded;

With the control's implementation discussed so far, when the Page_Load event is fired to the page the dropdown list control has been fully rebuilt to reflect the changes on the client—the new items are now definitely part of the control's state.

The Extended Object Model

When the dropdown list is displayed on the client, it takes the form of a <SELECT> tag. The DHTML object model designs a tree of objects around these elements and gives you the tools to add or remove items programmatically via JavaScript. The following code shows what you really need to execute:

var oOption = document.createElement("OPTION");
element.options.add(oOption);
oOption.innerText = text;
oOption.value = id;

By employing a DHTML behavior, you can wrap the previous code in a new method that's easier to use. Figure 5 details my dropdownlistex.htc behavior. It contains a Boolean property to enable support for client-side insertions as well as a client-side method named AddItem. The code for this method actually extends the DHTML tree for a regular dropdown element and simplifies the insertion of a new item. When attached to a client-side button, the following code adds a new item to the specified dropdown list entirely on the client. As you can see, it leverages the AddItem method defined on the behavior:

<SCRIPT lang="javascript">
function InsertTheNewItem() {
   var obj = document.getElementById("NewElement");
   var text = obj.value;
   var id = obj.value;
   document.getElementById("DropDownList2").AddItem(text, id);
}
</SCRIPT>

Figure 5 The DropDownListEx.htc Behavior (Close Figure 5)
<PROPERTY NAME="Modifiable" />
<METHOD NAME="AddItem" />
<ATTACH EVENT="onreadystatechange" HANDLER="Init" />

<script language="javascript">

// Handles the initialization phase
function Init()
{
   if (Modifiable == null)
       Modifiable = false;
}

// Add a new item programmatically
function AddItem(text, id)
{
    if (!eval(Modifiable))
    return false;

    var oOption = document.createElement("OPTION");
    element.options.add(oOption);
    oOption.innerText = text;
    oOption.value = id;

    var hiddenField = GetHiddenField(element.id + "_My");

    // Add a separator
    var tmp = hiddenField.value;
    if (tmp != "")
       hiddenField.value += ",";

    hiddenField.value += text + "|" + id;
}

function GetHiddenField(fieldName)
{
    // Go up in the hierarchy until the FORM is found
    var obj = element.parentElement;
    while (obj.tagName.toLowerCase() != "form")
    {
    obj = obj.parentElement;
    if (obj == null)
       return null;
    }

    if (fieldName != null)
    return obj[fieldName];
}

</script>

Summing It Up

DHTML behaviors are Internet Explorer client-side components that encapsulate a given behavior and attach it to an HTML element. From the ASP.NET perspective, you can utilize these components to enrich server controls with advanced, browser-specific capabilities. It is important to realize that DHTML behaviors are not strictly necessary to endow server controls with powerful client capabilities; their use, however, makes it possible to better encapsulate all of the required features in a reusable and easily accessible object model.

In addition to learning the internal mechanics of DHTML behaviors, you should become familiar with the postback interfaces of ASP.NET controls—in particular, IPostBackDataHandler. This interface lets developers handle any posted data whose format and layout is entirely up to you. A deep understanding of this interface is key to implementing effective interaction between browser and server environments within the boundaries of ASP.NET controls.

Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is a Wintellect instructor and consultant based in Italy. Author of Programming ASP.NET and the new book Introducing ASP.NET 2.0 (both from Microsoft Press), he spends most of his time teaching classes in ASP.NET and ADO.NET and speaking at conferences. Get in touch with Dino at cutting@microsoft.com or join the blog at weblogs.asp.net/despos.


Data Points: The Enterprise Library Data Access Application Block, Part 1

Data Points
The Enterprise Library Data Access Application Block, Part 1
John Papa

Contents
Enterprise Library and the Seven Blocks
Connection Management
Configuration Settings
Security
The Factory Pattern
On Deck

A solid data access later (DAL) can benefit an application by hiding redundant tasks, helping handle exceptions more gracefully, helping clean up resources more efficiently, and providing a layer of abstraction from the database. When you want to install a DAL in your architecture, you need to either find or build a data access component. Building one takes precious time away from a project's design, development, and testing timeline. What if you could find one that was already tested, follows best practices, and has full source available? If you are currently considering a DAL, you should take a look at the Enterprise Library Data Access Application Block (DAAB). This Application Block was created by the Microsoft® Patterns and Practices group, and since it ships with all of the source and relevant unit tests, the code can be enhanced or modified to suit different needs.

If you are like me, you are skeptical of any pre-built component but still open to the possibility. I brought my skepticism into an examination of the DAAB last summer when I had the opportunity to get an early glimpse of Enterprise Library. Since then I have tested and used the Enterprise Library DAAB in a variety of situations to see how it holds up as a DAL in applications.

This month I will share how the Enterprise Library DAAB fits into an architecture as well as some best practices in using the block. I will review the Enterprise Library model and dive right into the DAAB and its configuration and cryptography features, when and where the DAAB can be used, and lots more.

Enterprise Library and the Seven Blocks

The Microsoft Patterns and Practices group has developed several application blocks to perform functions ranging from caching to data access. The goal is to create code libraries that most enterprise solutions require. In January 2005 the Patterns and Practices group released the Enterprise Library. This most recent incarnation of the application blocks includes updated versions of many of the older blocks as well as some new ones.

Note: after you install the Enterprise Library, go to the Start menu and select Programs | Microsoft Patterns and Practices | Enterprise Library | Install Services. This will complete the Enterprise Library installation by creating its specific performance counters. Then you will be able to view the Enterprise Library performance counters via PerfMon. Hopefully, future releases of Enterprise Library will include this step in the installation.

All of the Enterprise Library blocks use the Configuration Application Block as well as a common library. The blocks integrate with each other easily but they are also built to stand independently. For example, Enterprise Library's Logging and Instrumentation Application Block can log messages to an event log or to a file. However, it can also log messages to a database by using the DAAB. All seven of the Enterprise Library blocks contain instrumentation using performance counters, Windows® Management Instrumentation (WMI) events, and Event Log messages. This helps administrators monitor the status of applications through tools like the Event Log, Performance Monitor, or Microsoft Operations Manager (MOM). The seven blocks are:

  • Caching Application Block
  • Configuration Application Block
  • Cryptography Application Block
  • Data Access Application Block
  • Exception Handling Application Block
  • Logging and Instrumentation Application Block
  • Security Application Block

At a minimum, these seven Enterprise Library blocks rely on the Configuration Application Block (as shown in Figure 1) and the common library, which contains a limited but commonly required set of functionality. For example, it exposes a bit of cryptography so that the Configuration Application Block can optionally encrypt its entries without having to reference the entire Cryptography Application Block (which references the Configuration Application Block, as do all Enterprise Library blocks).

Figure 1 Enterprise Library's Application Blocks
Figure 1 Enterprise Library's Application Blocks

The beauty of the Enterprise Library blocks is that you can choose which blocks to use without having to develop them from scratch. It is a best practice to use a DAL in an enterprise solution. Developing and testing one that abstracts ADO.NET and specific database provider objects, and which handles common data practices, takes time. The DAABs save you that time.

Connection Management

Far too often I see applications that have problems with connection state and connection pooling as well as inconsistencies in the way connection strings are stored. While the connection object may have a simple purpose (connect to a data provider), it is often the source of scalability and resource problems. As shown in Figure 2, the Enterprise Library DAAB stores connection strings in its own config file, which can be edited using the Enterprise Library Configuration Tool, making editing and maintaining the configuration settings easier than editing an XML configuration file by hand.

Figure 2 Enterprise Library Configuration Tool
Figure 2 Enterprise Library Configuration Tool

The DAAB also helps manage connection state for you. For example, if you want to fill a DataSet or insert some rows by executing a stored procedure, the DAAB automatically opens and closes the connection when needed. It uses the open late, close early approach so that connection pooling is maximized. Of course, you could write this code yourself, but it is easy to forget to close the connection. If you use a DataReader, the DAAB's methods will keep the connection open as the DataReader requires. This code leaves the connection open after the ExecuteReader method executes so the resulting IDataReader object can still access the data:

string sSql = "SELECT CustomerID, CompanyName, City, Region " +
    "FROM Customers WHERE City LIKE 'M%'";
Database dbNorthwind = DatabaseFactory.CreateDatabase();
DBCommandWrapper cmdCust = dbNorthwind.GetSqlStringCommandWrapper(sSql);
IDataReader rdrCust = dbNorthwind.ExecuteReader(cmdCust);

This code creates a Database object using the DatabaseFactory object's CreateDatabase method. It then creates a DBCommandWrapper object (which represents the SQL statement) and executes the command using the Database object's ExecuteReader method. This returns an IDataReader object. The DAAB keeps the connection open as required. However, if you replaced the last line of code with the following, the DAAB would have opened and closed the connection for you:

DataSet dsCust = dbNorthwind.ExecuteDataSet(cmdCust);

Configuration Settings

The configuration tool that comes with Enterprise Library provides an easy way to adjust and validate database settings for the DAAB. After you install Enterprise Library, this tool is in Program Files\Microsoft Enterprise Library\bin\EntLibConfig.exe. Like the previous version of the DAAB, Enterprise Library uses config files to store its connection string settings. By doing so, the information can be modified without having to recompile the application.

Once you open the Enterprise Library Config tool, choose File | Open Application from the menu. Then navigate to and select the Web.config file or the App.config file for your application. To create a config file for the DAAB, right-click on the Application node and choose New | Data Access Application Block. Two new complex nodes will be added beneath the Application to represent the Configuration Application Block and the DAAB. (The Configuration Application Block is used to help determine how to store and transform the other Enterprise Library blocks' config files.) This also causes XML to be added to the Web.config (or App.config) that links it to a separate config file called dataConfiguration.config. This file can store database connection string information for multiple connections.

The Enterprise Library Config Tool automatically creates entries for connection strings, database instances, and database types. If you are using SQL Server™, then you do not need to add another entry for the database type. If you want to use Oracle, however, you can add a new entry here and point it to the Oracle data provider. The dataConfiguration.config file allows for multiple database types so you can refer to connections to different database providers within the same application. After you create the new database type by right-clicking on Database Types and selecting New | Database Type, you can select the Oracle data provider by clicking on the ellipses on the right side of the TypeName field (as shown in Figure 3). Or, you can add a different data provider by clicking on the Load an Assembly button.

Figure 3 Type Selector
Figure 3 Type Selector

The Database Instances node is used to associate one of the database types with one of the connection strings. The name of the database instance is what you use to refer to a particular connection string for a particular provider via code. If you have more than one database instance, make sure you set the DefaultInstanceNode attribute under the root DAAB node. (The default is the database instance that is used by the DatabaseFactory.CreateDatabase method if no argument is passed to it.)

You can edit the connection string settings under the Connection Strings node. By default you get a connection string that has a database, server, and integrated security attributes. For example, you can change the SQL Server instance (localhost), the name of the database (Northwind), and the Integrated Security setting (True). If you don't want to use integrated security, you can add the User ID and Password connection string settings. When you right-click on the connection string node and choose New | Parameter, an attribute is created called Parameter, which you can rename User ID and set its value. You can also add a Password attribute by right-clicking the connection string node and selecting New | Password Parameter. I recommend using integrated security, but if you choose to use SQL Security and pass in the User ID and Password attributes, consider using the Enterprise Library cryptography features to encrypt the connection string.

Security

Database credentials must always be protected. The DAAB stores connection string information in a configuration file, which can then be encrypted. When you choose an encryption algorithm, that algorithm applies to all of the blocks. You can, however, tell each block whether or not it should use the encryption.

To set up encryption, right-click the Encryption Settings node and select New | File Key Algorithm Storage Provider. You can then either create a new key algorithm or choose an existing one. Let's assume you want to create a new one and you click the Next button. You can then select the algorithm you want to use, and generate the algorithm's key in the next window of the wizard. Finally, specify a file name for the file that will store the algorithm's key.

At this point I have created an encryption algorithm key file, but I have not yet told the Enterprise Library Config tool to encrypt the dataConfiguration.config file. To apply the encryption to the block, simply navigate to the Configuration Application Block node and select its dataConfiguration child node. Then, set the Encrypt attribute to True and save your changes. If you then try to open the dataConfiguration.config file, you will not be able to read the file as it is now encrypted (see Figure 4). The DAAB uses the algorithm's key file to decrypt the contents of the file.

Figure 4 dataConfiguration.config Before and After Encryption
Figure 4 dataConfiguration.config Before and After Encryption

It might seem odd that the cryptography features I demonstrated work without having to use Enterprise Library's formal Cryptography Application Block. The reason that the Cryptography Application Block is not needed in this case is that the common library assembly that the blocks refer to contains this limited cryptography feature without all of the overhead of the entire Cryptography Application Block.

If, instead of Windows Authentication, you choose to use SQL authentication (which means using a specific SQL Server user ID and password combination for each user) keep in mind that if the connection string is different for each user, users will not share the same connection pool. If you use SQL Authentication, consider using a fixed SQL Server account so that the connection string stays the same and you can take advantage of connection pooling.

The Factory Pattern

Before you can retrieve or update data you must first establish a connection to a data provider such as SQL Server. In ADO.NET you create a SqlConnection object, passing a connection string to its constructor. Then you can open the connection and fill a DataSet or retrieve data through a DataReader. If you want to connect to Oracle, you need to create the Oracle-specific connection object. If you want to connect to another data provider, you use its specific connection object or you can use the more generic OleDbConnection. One of the nice features of the DAAB is that you can easily abstract your code from a specific data provider. For example, check out the following code:

private void Foo(string sSql)
{
    string sDatabaseInstance = "NorthwindDBInstance";
    Database db = DatabaseFactory.CreateDatabase(sDatabaseInstance);
    DBCommandWrapper cmdCust = db.GetSqlStringCommandWrapper(sSql);
    IDataReader rdrCust = db.ExecuteReader(cmdCust);
    ...
}
This code creates an instance of the Database-derived object, which in this case will be a SqlDatabase object. One line of code could be modified to make the same CreateDatabase method return an instance of an OracleDatabase object as follows:
string sDatabaseInstance = "MyOracleDBInstance";

The CreateDatabase method of the DatabaseFactory accepts an argument that represents a DatabaseInstance entry in the dataConfiguration.config file. So by changing the database instance, I can change the type of Database object that the DatabaseFactory.CreateDatabase method returns. The cool part is that I do not have to declare a SqlDatabase or an OracleDatabase object. Rather, I can use their base class, the Database object, thus keeping my code abstracted from data provider specificity. All of this is made possible through the Factory pattern that the DAAB uses.

I can also build my own provider and derive it from the DAAB's abstract Database class (you cannot create an instance of it directly because it's abstract). If you crack open the code for Enterprise Library's DAAB and look in the Sql folder, you will see the SqlDatabase class. This class inherits and uses most of the methods of the abstract Database class, but it overrides certain properties and methods. The Database class does not refer to the System.Data.SqlClient namespace, but the SqlDatabase class does. For example, the abstract Database class implements an abstract ParameterToken property that is intended to be overridden in any class that inherits from the Database class. The SqlDatabase class overrides this property to return the @ symbol (the SQL Server-specific character that precedes parameter names). Another example of an overridden method is the GetConnection method. From the SqlDatabase class, this method should return an instance of a SqlConnection object as an IDBConnection.

The DatabaseFactory class is sealed, so it cannot be inherited. Its constructor is private, so the DatabaseFactory cannot be directly instantiated either. This allows me to call the static CreateDatabase method without allowing me to create an instance of the class itself. When you call the CreateDatabase method and pass it a database instance name, the DAAB uses reflection to figure out what specific type of Database object to create. This is all handled under the covers inside of the source of the DAAB and Enterprise Library. Way down in the call stack there is a method in the Configuration Application Block ConfigurationFactory class called CreateObject. This method basically takes an instance of the Type class, which represents the Database-derived class you want to create. It gets the constructor of the class by using the following line of code through reflection:

ConstructorInfo constructor = type.GetConstructor(new Type[] {});

Once it has the constructor, it then tries to invoke the constructor by using reflection once again:

createdObject = constructor.Invoke(null);

In this scenario, the createdObject variable then represents the instance of the SqlDatabase object. There is quite a bit of code surrounding this whole process, but the abstract classes, the classes that derive from them and override their methods and properties, and reflection are at the heart of the Database Factory pattern. Unless you are truly intrigued by the inner workings of the DAAB, you don't need to delve deep into its code. However, if you are as curious as I am, you'll likely find it very interesting.

On Deck

The DAAB exposes several ways to retrieve and modify data using features of ADO.NET. For example, you can execute a stored procedure and return a DataSet using the ExecuteDataSet method or you can execute an UPDATE statement and return nothing by using the ExecuteNonQuery method. The Database class has dozens of overloaded methods that do the following:

  • Execute stored procedures or SQL statements
  • Return a DataSet, a DataReader, a Scalar value, an XmlReader, or nothing at all
  • Allow for specific parameters to be created and passed in
  • Determine which parameters that it needs, create them, and cache them
  • Involve commands in a transaction

In the next installment of Data Points I will explore many of these features, and demonstrate how to set up a project to use the DAAB and retrieve data in a variety of ways. In the final installment of this three-part series, I will demonstrate techniques used to save data as well as how to take advantage of the DAAB's parameter caching. I will also review the Test Driven Development (TDD) practices that Enterprise Library uses and show how to write TDD code and test it using a tool such as NUnit.

Some advantages of using the DAAB as a DAL are that it reduces repetitive lines of ADO.NET code, provides for database abstraction, and provides instrumentation. The configuration tool, the separation of the config files from the Web.config (or App.config), and the general ability to easily store and encrypt database settings are also good features. The Enterprise Library DAAB contains a lot of code, but if you are not going to need database abstraction and your application is small in scale, you might not want to use it. You might want to consider using ADO.NET directly if you have an application that is not tiered or does not warrant a true DAL. However, if you are building an enterprise solution that needs to be scalable, follow best practices, can secure its connection settings, is used by many developers around the world, or includes database abstraction, or if you want to be able to use instrumentation to monitor the health of your application, Enterprise Library's DAAB is probably a better choice.

Send your questions and comments for John to mmdata@microsoft.com.

John Papa is a Senior .NET Consultant with ASPSOFT and a baseball fanatic who spends most of his summer nights rooting for the Yankees with his family and his faithful dog, Kadi. He has authored several books on ADO, XML, and SQL Server and can often be found speaking at industry conferences such as VSLive or blogging at codebetter.com/blogs/john.papa.


Bugslayer: Unhandled Exceptions and Tracing in the .NET Framework 2.0

Bugslayer
Unhandled Exceptions and Tracing in the .NET Framework 2.0
John Robbins

Code download available at: Bugslayer0507.exe (148KB)
Contents
What Happened to My Thread!?
A Better Way
Tracing the New Way
Tracing Sample
More Tracing Improvements
Wrap Up
Tips

By now, you've certainly heard about the big changes coming in Visual Studio® 2005, but when it's time to move your code over it will be the small things that trip you up. In this column, I want to cover two of the many excellent changes that you could easily overlook as you make the move to the new runtime and development tools. Don't get me wrong. I don't mean to make the transition sound dangerous. The truth is, your ported code will probably work just great with nothing more than a recompile. But there are things you should know. When you finish reading, you should also check out the "What's New" sections in the documentation.

The first change I'll talk about is how the common language runtime (CLR) in the Microsoft® .NET Framework 2.0 now handles unhandled exceptions. Then I'll discuss some of the excellent enhancements to the tracing system so you can get the best information from the field.

What Happened to My Thread!?

In native Windows® development, the SetUnhandledExceptionFilter API from KERNEL32.DLL makes it trivial for the OS to notify you when any thread in your application has an unhandled exception. If you've been reading Bugslayer over the years, you'll know I've spent my share of ink on using and extending the magic SetUnhandledExceptionFilter.

When I turned to .NET development, it looked like the AppDomain.UnhandledException event had the same sort of functionality. If you register a delegate with this event, the common language runtime notifies you that an unhandled exception occurred, but there's a serious twist. What happens after you return from your delegate depends on the thread type.

To demonstrate the exception-handling problems in the .NET Framework 1.x, download and compile the ConsoleExceptions program (included with this month's code download) in Visual Studio .NET 2003. The program takes two parameters; the first is a "y" to install an AppDomain.UnhandledException delegate, or an "n" to use the default handlers. The second is the thread type in which you want to cause an exception ("m" for the main thread, "t" for a managed thread, "p" for a pool thread, and "g" for the garbage collector finalizer thread).

Running ConsoleException.exe with the parameters "n m" brings up the just-in-time (JIT) debugging dialog and gives you an opportunity to debug the application in the managed debugger of your choice. While I did not include an option to create a native thread from a C++ DLL in ConsoleException, you would see the exact same behavior as you get from a crash in the main thread.

Things get more interesting if you specify a crash in any other type of thread in ConsoleException. For example, if you use the command-line parameters "n t" to cause an exception in a managed thread, you won't see a JIT debugging dialog. In fact, instead you'll see the call stack from the default exception filter in the console, followed by Console.WriteLine output indicating that the finally block in Main executed, and "Done!" to indicate the application ran to completion.

When I first saw this behavior, which is the same for the garbage collector and all pool threads as well, my eyes popped out of my head and bounced off my monitor. What's happening is the CLR is silently killing those threads! Interestingly, there is an undocumented option to have the JIT debugger pop up when an unhandled exception occurs on threads other than the main thread. If you set the upper 6 bytes of the DWORD value DbgJITDebugLaunchSetting to 0xFFFFFF in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework, that tells the CLR to bring up the debugger. Set the low byte to the default 0x02 to spawn the debugger specified in the DbgManagedDebugger value in the same key.

After setting the DbgJITDebugLaunchSetting to 0xFFFFFF02, and passing "n t" as the command-line arguments to ConsoleException, you'll see the fine JIT debugging dialog. Before you press "Yes" to debug ConsoleException, take a peek at the console window in which ConsoleException is running. You'll see something very odd. You'll see "Done!", indicating that the application has ended and you'll see the blinking cursor at the prompt. Of course, if you press "Yes" on the JIT debugging dialog, you'll get an error stating that the application is invalid. If you haven't guessed, the rest of your application keeps running on its merry way when a thread is blocked waiting in the JIT debugging dialog.

If you do add a delegate to AppDomain.UnhandledException, the CLR will notify you that there's been an exception so you can at least log that fact and possibly kill the process. The UnhandledExceptionEventArgs parameter has a read-only Boolean property, IsTerminating, that is true if the application is ending. This indicates the exception came out of the main thread or a Win32® native thread. As you can guess, a value of false in IsTerminating indicates the exception came from one of the other types of threads.

A Better Way

The great news about the .NET Framework 2.0 is that unhandled exceptions, no matter where they come from, now cause termination of the app. You also have the chance to debug the unhandled exception if you have a JIT debugger installed. If you haven't installed an AppDomain.UnhandledException delegate and you're using multiple threads in your console or Windows Forms applications, you'll definitely want to add one when you port your code to the .NET Framework 2.0 because you may have been silently losing threads all along.

I can't thank the Redmondtonians enough for finally changing this design. While a breaking change from the behavior in the .NET Framework 1.x, this will definitely lead to better applications because you'll immediately know when there's a problem (if you really want to keep the old behavior, there is a configuration switch that will let you). While a mini dump and Son of Strike (SOS) is the next best thing to being there when your app takes a dive, getting those giant full-memory dumps can be a problem. In those cases, judicious use of tracing can save the day. Fortunately, the .NET Framework 2.0 has a bunch of great tracing enhancements.

Tracing the New Way

The number-one sentiment about tracing is that it's very cool that you set a TraceSwitch externally in the config file, but getting the TraceSwitch to reread the config file while the app is running is a big concern. In the .NET Framework 1.x, you have to manually do all the work yourself. But in the .NET Framework 2.0, the Trace object has a static method, Refresh, that will force a reread of the configuration file and update any trace switches as appropriate.

Instead of everyone writing their own file change watcher, I whipped up a little class you can drop into your apps to get the TraceSwitch updates easily (see Figure 1). The only hitch is that Visual Studio 2005 console and Windows Forms applications by default run inside a special named process, <Program Name>.VSHOST.EXE, which allows for faster managed debugging.

Figure 1 ConfigTraceSwitchWatcher.cs (Close Figure 1)
/*-----------------------------------------------------------------------
 * MSDN Magazine - Bugslayer Column - John Robbins
 * A .NET Framework 2.0 class to refresh the TraceSwitch values whenever
 * app's configuration file changes.
-----------------------------------------------------------------------*/
using System;
using System.IO;
using System.Text;
using System.Threading;
using System.Reflection;
using System.Diagnostics;
using System.Collections.Generic;

namespace Bugslayer
{
    /// <summary>
    /// A class that watches the application config file and refreshes all
    /// <see cref="TraceSwitch"/> values.
    /// </summary>
    class ConfigTraceSwitchWatcher
    {
        // The file watcher itself.
        private FileSystemWatcher watcher;
        // The name of the configuration file for reference.
        private String appConfigFile;

        /// <summary>Creates the watcher for the config file.<summary>
        public ConfigTraceSwitchWatcher ( )
        {
            appConfigFile = AppDomain.CurrentDomain.SetupInformation.
                ConfigurationFile.ToUpper().Replace(".VSHOST", "");
            Initialize ( );
        }

        /// <summary>Creates the watcher for the file.</summary>
        /// <param name="fileName">The full path to watch.</param>
        public ConfigTraceSwitchWatcher ( String fileName )
        {
            appConfigFile = fileName;
            Initialize ( );
        }

        /// <summary>Shuts off all configuration monitoring.</summary>
        public void Close ( )
        {
            // Clear the notifications.
            watcher.EnableRaisingEvents = false;
            watcher = null;
            appConfigFile = null;
        }

        // The common initialization.
        private void Initialize ( )
        {
            Debug.Assert(appConfigFile != null, "appConfigFile != null");
            Debug.Assert(appConfigFile.Length > 0,
                "appConfigFile.Length > 0");
            if ((null == appConfigFile) || (appConfigFile.Length == 0))
            {
                throw new ArgumentException ( "appConfigFile" );
            }

            // Set the directory to watch and only watch for writes
            String path = Path.GetDirectoryName ( appConfigFile );
            watcher = new FileSystemWatcher ( path );
            watcher.NotifyFilter = NotifyFilters.LastWrite;

            // Set up the event handlers.  Make sure to watch
            // changes made to file, including creation and deletion.
            watcher.Changed +=
                new FileSystemEventHandler ( watcher_Changed );
            watcher.Deleted +=
                new FileSystemEventHandler ( watcher_Changed );
            watcher.Created +=
                new FileSystemEventHandler ( watcher_Changed );

            // Let fly.
            watcher.EnableRaisingEvents = true;
        }

        void watcher_Changed ( object sender , FileSystemEventArgs e )
        {
            // Check to see if this is the real application config.
            if (0 == String.Compare (appConfigFile, e.FullPath,
                true, Thread.CurrentThread.CurrentCulture))
            {
                // Can this get any simpler?
                Trace.Refresh( );
            }
        }
    }
}

Tracing Sample

When I first developed ConfigTraceSwitchWatcher, I saw to my dismay that my App.Config files were copied to both <Program Name>.EXE.CONFIG and <Program Name>.VSHOST.EXE.CONFIG. The good news is that the <Program Name>.VSHOST.EXE that is used when debugging uses the <Program Name>.VSHOST.EXE.CONFIG for startup, but after that, all the configuration information is read from the proper <Program Name>.EXE.CONFIG. That's why you see the code to strip out ".VSHOST" in the ConfigTraceSwitchWatcher constructor.

The code to ConfigTraceSwitchWatcher is in the TraceRefreshDemo program included with this month's source code so you can play with it interactively. If you don't want to debug using the Visual Studio hosting process, you can go into the Debug property page and uncheck the Enable the Visual Studio hosting process. Based on experiments with builds prior to Beta 2, managed debugging is definitely faster with the Visual Studio hosting process enabled. So, with almost no code changes at all, you can get the benefits of ConfigTraceSwitchWatcher in your existing app.

Another new set of classes that you can easily take advantage of are the new TraceListener-derived classes. The first of these, WebPageTraceListener, mimics my ASPTraceListener class, which I talked about when I first covered .NET in October 2001 (see Bugslayer: Handling Assertions in ASP .NET Web Apps), and puts any System.Diagnostics.Trace.Write* output on the page. The WebPageTraceListener class does not appear automatically in the Trace.Listeners collection, so if you want to use it, you'll need to either add WebPageTraceListener to the collection programmatically, or add it through the <system.diagnostics> section of your Web.config. For more information, see "Walkthrough: Integrating ASP.NET Tracing with System.Diagnostics Tracing" in the Visual Studio 2005 Help.

On a side note, part of the motivation for the October 2001 column was to provide a functional assertion for ASP.NET applications. Now that your Edit-Compile-Debug cycle for your ASP.NET applications will be spent using the integrated Web server rather than IIS, you'll get to see the DefaultTraceListener assertion message box in all its stack trace glory with nothing extra on your part. There's still no assertion that will work while running inside IIS, so you still might want to consider my ASPTraceListener. But being able to see most of your assertions during coding and unit testing is a huge boon for productivity.

For those of you building console applications, you can see your tracing directly in the console window with the new ConsoleTraceListener. The default is for the ConsoleTraceListener to send its output to standard output, but you can specify that the output go to standard error instead by passing false to the constructor. If you're into delimited output, the new DelimitedTraceListener will separate the output with semicolons by default. Of course, you can change the character by setting the DelimitedListTraceListener.Delimiter property.

While tracing to a raw text file is helpful, I can't count the number of teams that have implemented their own XML output TraceListener so they can add some more useful data to the tracing output. If you wanted one, but didn't have the time to write it, you can now use Microsoft-supplied XmlWriterTraceListener.

Figure 2 is an example of the output. Microsoft has not yet released the schema, but looking at the output (which at the time of this writing is not well-formed XML as there are multiple root elements), you see that there's more interesting data than just simple strings in there.

Figure 2 Example XmlWriterTraceListener Output (Close Figure 2)
<E2ETraceEvent xmlns="http://schemas.microsoft.com/2004/06/E2ETraceEvent">
  <System xmlns=
   "http://schemas.microsoft.com/2004/06/windows/eventlog/system">
    <EventID>0</EventID>
    <Type>3</Type>
    <SubType Name="Information">0</SubType>
    <Level>8</Level>
    <TimeCreated SystemTime="2005-03-29T16:09:23.9844772-08:00" />
    <Source Name="Trace" />
    <Correlation ActivityID="{00000000-0000-0000-0000-000000000000}"/>
    <Execution ProcessName="NewTraceListeners.vshost"
      ProcessID="2932" ThreadID="8" />
    <Channel/>
    <Computer>TESTBOX</Computer>
  </System>
  <ApplicationData>
    Hello from the first string in the output!!
  </ApplicationData>
</E2ETraceEvent>

<E2ETraceEvent xmlns="http://schemas.microsoft.com/2004/06/E2ETraceEvent">
  <System xmlns=
   "http://schemas.microsoft.com/2004/06/windows/eventlog/system">
    <EventID>0</EventID>
    <Type>3</Type>
    <SubType Name="Information">0</SubType>
    <Level>8</Level>
    <TimeCreated SystemTime="2005-03-29T16:09:24.1407252-08:00" />
    <Source Name="Trace" />
    <Correlation ActivityID="{00000000-0000-0000-0000-000000000000}" />
    <Execution ProcessName="NewTraceListeners.vshost"
      ProcessID="2932" ThreadID="8" />
    <Channel/>
    <Computer>TESTBOX</Computer>
    </System>
    <ApplicationData>About to go into loop...</ApplicationData>
</E2ETraceEvent>

The tracing enhancements I've been talking about require either very minor code changes or tweaking your application's configuration file. However, you can see from the output of the XmlWriterTraceListener, there are a few more changes to tracing in the .NET Framework 2.0. These changes help solve a problem that the existing System.Diagnostics.Trace object exhibits when you start working on larger applications.

More Tracing Improvements

As it stands in the .NET Framework 1.x, there's only one Trace object that's global to the application domain. For most applications that's fine because you can add different output options through TraceListener-derived classes added to the Trace.Listeners collection. However, as you move into larger industrial-strength applications, the global, one-size-fits-all nature of the Trace object becomes a hindrance. While you can designate TraceSwitch options to control output, if you want to see only three assemblies tracing (of the multitude of assemblies you may have in your application), all that output still goes to the same TraceListeners so you will have tons of output to sort through.

Another problem with the existing Trace object in larger applications is that the output is limited to just the string passed to Trace.Write*. In real-world apps, many times, such as when tracing into error handling code, you want to add lots of additional output, such as the call stack, process information, thread information, and so on. While you could programmatically add that data into every place where you think you might need it in your trace output, it would be much better if you could dynamically change the data output in the configuration file.

With the new TraceSource class, Microsoft has pushed the Trace object down a level. A TraceSource has the same basic capabilities as the Trace object in that the TraceSource offers methods for tracing as well as Listeners properties so you can have extensible output. Now if you want specific tracing for those multiple assemblies, you'd simply configure each assembly's TraceSource in the application configuration file. You can add the specific TraceListeners to each assembly's TraceSource so that all output will be independent of the others, making analysis a lot easier. Additionally, TraceSource supports the TraceSwitch classes that make it easy to control the level of output: Error, Warning, Informational, and Verbose.

Instead of writing about all the new pieces, just showing you the code will be worth thousands of words. Figure 3 uses the new TraceSource in a programmatic fashion along with plenty of comments about the output and features. You should read this as it wasn't until I wrote the code in Figure 3 that the awesome new tracing features made sense.

Figure 3 The TraceSource and TraceListener New Features (Close Figure 3)
using System;
using System.Text;
using System.Diagnostics;
using System.Collections.Generic;

class Program
{
    static void ShowAllTraceMethods ( TraceSource trc )
    {
        // The simplest method.  Notice it takes parameters, which is a
        // big improvement over the Trace object.
        trc.TraceInformation ( "The simplest trace method with {0}{1}" ,
                               "params" , "!" );

        // The method to trace an event based on the TraceEventType enum.
        trc.TraceEvent ( TraceEventType.Error , 100 , "Pretend error!" );

        // The method to make dumping out data easy.
        trc.TraceData ( TraceEventType.Information ,
                        50 , "Some" , "pretend" , "data." );

        // The method to record a transfer.  This method is primarily for
        // the XmlWriterTraceListener.  See the column text for more
        // discussion.
        trc.TraceTransfer ( 75 , "What was transfered?" ,
            new Guid ("7b5fcdbc-913e-43bd-8e39-ee13c062ecc3") );
    }

    static void Main ( string[] args )
    {
        // Create the TraceSource for this program.  Like the Trace
        // object, the TraceSource.Listeners collection starts out with
        // the DefaultTraceListener.
        TraceSource trc = new TraceSource ( "HappySource" );

        // Set the switch level for this TraceSource instance so
        // everything is shown. The default for TraceSource is to *not*
        // trace.  The default name of the switch is the same as the
        // TraceSource. You'll probably want to be sharing Switches across
        // TraceSources in your development.
        trc.Switch.Level = SourceLevels.All;

        // Trace to show the default output.
        ShowAllTraceMethods ( trc );

        // The TraceListener class has a very interesting new property,
        // TraceOutputOptions, which tells the TraceListener the additional
        // data to automatically display.
        trc.Listeners[ "Default" ].TraceOutputOptions =
            TraceOptions.Callstack | TraceOptions.DateTime |
            TraceOptions.ProcessId | TraceOptions.ThreadId |
            TraceOptions.Timestamp;

        // Now all the trace calls in the Debug Output window will show
        // all the data included in the TraceOutputOptions.

        ShowAllTraceMethods ( trc );

        // Filtering allows you to apply a limiter to the TraceListener
        // directly.  That way you can turn on tracing, but apply more
        // smarts to the actual output so you can better separate the
        // wheat from the chaff on a production system.
        EventTypeFilter evtFilt =
            new EventTypeFilter ( SourceLevels.Error );

        // Apply the filter to the DefaultTraceListener.
        trc.Listeners[ "Default" ].Filter = evtFilt;

        // The only output in the Debug Output window will be from the
        // TraceEvent method call in ShowAllTraceMethods.
        ShowAllTraceMethods ( trc );

        trc.Flush ( );
        trc.Close ( );
    }
}

After I'd written that code, I ran across a wonderful blog entry (A Tracing Primer - Part I) from Mike Rousos, a member of the BCLTeam, talking about the TraceSource. The blog had a graphic that showed where the various classes fit together. For those of you who are graphical thinkers, I took Mike's idea and expanded it so you could see how a trace flows through the new tracing system.

Figure 4 shows three different TraceSource instances in use by an app. When a Trace* method is called, it checks the Switch-derived class in the TraceSource.Switch property to see if the condition on the Trace* method matches. If not, nothing is traced. Notice that a Switch class can be shared between TraceSource instances (see B and C in the diagram). If the Switch-derived condition is met, the TraceSource loops through its Listeners collection calling the appropriate TraceListener.Trace* method. (In the .NET Framework 2.0 documentation you'll see that the TraceListener class has numerous new properties and methods.)

Figure 4 Data Flow of TraceSource to Output
Figure 4 Data Flow of TraceSource to Output

If the individual TraceListener class has a TraceFilter instance in the TraceListener.Filter property, before any output is sent, the TraceListener checks to see if TraceFilter.ShouldTrace returns true. If it does not, no output is sent. If ShouldTrace returns true, the output is sent to that TraceListener's internal writing method.

In Figure 4, TraceSource instance B has a single TraceListener instance called X1 set in its TraceListeners collection and TraceSource C has a single TraceListener called Y1. However, the TraceSource objects are sharing a Switch 2, so any initial matching on the TraceSource.Trace* methods for B or C will be filtered identically. In the case of the two TraceListeners, they both share a filter, Filter 2, so any filtering will apply to both.

At the top of Figure 3, the ShowAllTraceMethods calls each of the methods in a TraceSource instance dedicated to tracing. The first method, TraceInformation, is for the simplest tracing, and after I talk about the other methods, you probably won't want to use it for your tracing as it has no flexibility for controlling output externally. The main method you'll want to use is TraceEvent as it allows you to assign a TraceEventType enumeration to indicate the level of tracing. The second parameter is the numeric identifier for the trace. With that parameter you either uniquely identify tracing, or assign a specific value to a class to help identify tracing output at a glance. There are three overloads to TraceEvent, but the big news is that one takes variable-length parameters so you can pass a formatting string and get your data displayed nicely in the trace.

The TraceData method accepts the trace identifier as well, but its main purpose in life is to allow you to pass either an object array or multiple parameters so the trace output shows them separated by commas. If you are dealing with data objects whose ToString method does the heavy lifting, the TraceData call can be your dependable friend. In the broader scheme of things, you'll nearly always use TraceEvent.

The final method, TraceTransfer, was designed to work hand in glove with the XmlWriterTraceListener. If you look back at the output of XmlWriterTraceListener in Figure 2, you might have noticed a Correlation element with an ActivityID attribute that looks like a GUID. TraceTransfer sets the internal GUID that the XmlWriterTraceListener will use until the next time a call to TraceTransfer sets a new GUID. You can assign a GUID to major activities in your application, such as login handling, so you can quickly scan the XmlWriterTraceListener output looking for logical operations in your tracing.

I'm willing to bet that only the largest applications will use TraceTransfer and the ActivityID. That's because the first parameter of both the TraceEvent and TraceData methods is a TraceEventType, and it's worth mentioning how much this has grown in the .NET Framework 2.0. In the .NET Framework 1.x, you were limited to Error, Warning, Info, and Verbose with a TraceSwitch. Figure 5 shows all the values of the TraceEventType enum and what they are for. As you can see, Start, Stop, Resume, and Transfer are all called activity types. Since one of the primary uses of tracing is to determine which operations occurred and the order in which they occurred, these new tracing events make that extremely easy. To see just the activity tracing, you can set the TraceSource.Switch.Level to SourceLevels.ActivityTracing.

Figure 5 The TraceEventType Values (Close Figure 5)

TraceEventType Value | Usage
Critical | Unrecoverable errors and exceptions in the application
Error | Recoverable errors the application handled, such as invalid logins
Warning | Unusual activity that may need exploration, such as data not being in the proper format
Information | Normal operation information, such as a user logging in or out
Verbose | Debugging information, such as the entering and exiting of a method
Start | The activity type indicating a logical operation was started
Stop | The activity type indicating a logical operation was stopped
Suspend | The activity type indicating a logical operation was suspended
Resume | The activity type indicating a logical operation was restarted
Transfer | The activity type indicating a change in the correlation ID (for example, a call to TraceTransfer)

While programmatically manipulating TraceSource instances is possible, as shown back in Figure 3, the real action is in manipulating the tracing from the application's configuration file. Figure 6 shows the App.config of the ConfigTraceSource program included with this month's source code. The program has two TraceSource instances, HappySource and GiddySource. The sources element is where you configure the individual TraceSource instances in a source element for each instance you want to configure. The two required attributes to the source element are the self-explanatory name and switchName. By associating a Switch instance with the TraceSource, you can control the output for the TraceSource instance. Keep in mind that Switch instances can be shared across multiple TraceSource instances.

Figure 6 HappySource and GiddySource (Close Figure 6)
<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <system.diagnostics>
        <sources>
            <source name="HappySource" switchName="HappySwitch">
                <listeners>
                    <add name="myTextListener"
                         type="System.Diagnostics.TextWriterTraceListener"
                         initializeData="TextWriterOutput.txt"/>
                    <remove name="Default"/>
                </listeners>
            </source>
            <source name="GiddySource" switchName="GiddySwitch">
                <listeners>
                    <add name="myConsoleListener"
                         type="System.Diagnostics.ConsoleTraceListener"/>
                    <remove name="Default"/>
                </listeners>
            </source>
        </sources>
        <sharedListeners>
            <add name="myXmlListener"
                 type="System.Diagnostics.XmlWriterTraceListener"
                 initializeData="SharedOutput.XML"
                 traceOutputOptions="DateTime, Timestamp, Callstack">
                <filter type="System.Diagnostics.EventTypeFilter"
                        initializeData="Warning"/>

            </add>
        </sharedListeners>
        <switches>
            <add name="HappySwitch" value="All" />
            <add name="GiddySwitch" value="Error"/>
        </switches>
    </system.diagnostics>
</configuration>

Underneath the source element, you can specify any TraceListeners you want to associate with that TraceSource. You can add and remove any items from the TraceSource.Listeners property as well as dynamically create new TraceListeners directly from the configuration file, just like you can in the .NET Framework 1.x.

What's most interesting in Figure 6 is the sharedListeners element where you can add TraceListener instances to all TraceSource instances in the application. In the XmlWriterTraceListener added in Figure 6, I also show how to apply a TraceFilter to a TraceListener. The new traceOutputOptions attribute allows you to specify the TraceOptions you want applied to the TraceListener.

While I added that filter to the TraceListener being shared across all TraceSource instances, you can also add those TraceFilters to TraceListeners you add to an individual TraceSource instance.

The last piece of Figure 6 is the switches element, where you configure individual Switch instances. The only difference in the switches element in the .NET Framework 2.0 from the .NET Framework 1.x is that the value attribute can now take string values that are passed to the switch constructor to set the value.

As you can see, the amount of control that you have over your tracing in the configuration with the .NET Framework 2.0 is simply amazing. The icing on the cake is the new ability in the .NET Framework 2.0 to tell you the exact line in the configuration file that's not correct so those of us who have spent hours pulling our hair out looking for problems in our configuration file can now start growing some of it back.

Wrap Up

While it's long overdue, the "Dude! Where did my thread go?" fix makes the .NET Framework 2.0 a mandatory upgrade in my book. All the working set reductions and performance boosts are secondary. In fact, I've been moving my heavily multithreaded applications over to the .NET Framework 2.0 for testing even if they are due to ship on the .NET Framework 1.x just to take advantage of the fixed behavior.

Microsoft spent quite a while looking at different applications to see what folks needed. Of course, the documentation for Visual Studio 2005 Beta 2 says almost nothing about the new tracing capabilities so I knew you could use this information. As I was playing with the new tracing, I kept looking for features that have been left out, but I have yet to find any. That's a great sign!

Tips

Tip 69 As all good Bugslayers know, a symbol server is a mandatory tool in your development shops. Prior to Visual Studio 2005, getting the pesky _NT_SYMBOL_PATH environment variable setup was an accident waiting to happen. Symbol server is now drastically easier to set up. Simply go to the Options dialog, Debugging, Symbols property page. In the Symbol files location control, paste in the following string that specifies the location of the Microsoft public symbol server:

http://msdl.microsoft.com/download/symbols
For your own symbol servers, simply enter \\server\share where server and share are the names inside your network.

Tip 70 A good statistic to keep track of is the churn on your publicly accessible types, classes, methods, and properties between versions of your apps. This can help you pinpoint areas you need to test and alert you to potential breaking changes. The .NET team is obviously very concerned about this, so they produced a tool called LibCheck to generate those statistics. You can read more about it at Using reflection to List version to version assembly changes: Libcheck, one of the best tools ever!.

Send your questions and comments for John to slayer@microsoft.com.

John Robbins is a cofounder of Wintellect, a software consulting, education, and development firm that specializes in .NET and Windows. His latest book is Debugging Applications for Microsoft .NET and Microsoft Windows (Microsoft Press, 2003). You can contact John at www.wintellect.com.


.NET Matters: StringStream, Methods with Timeouts

.NET Matters
StringStream, Methods with Timeouts
Stephen Toub

Code download available at: NETMatters0507.exe (132KB)

Q I'm trying to use a method that accepts a Stream as a parameter. However, the data I want to pass to that method is currently stored only as a string. Can you tell me what is the best way for me to pass the string to this method?

A There's not necessarily a best way, but there are a few solutions. And as with many scenarios in which you're provided with multiple approaches, you need to weigh space demands against time demands to choose the most appropriate solution. For example, is the string in question small? If so, consider copying that string into a System.IO.MemoryStream. With a small string, the duplicate data probably won't break the memory bank:

MemoryStream memStream = new MemoryStream();
byte [] data = Encoding.Unicode.GetBytes(theString);
memStream.Write(data, 0, data.Length);
memStream.Position = 0;

If you plan on doing this frequently, you might consider refactoring this functionality into its own class. The easiest way is to derive a class from MemoryStream, and then take advantage of MemoryStream's constructor that accepts the byte array of data to be contained within the stream. Moreover, you can use a related constructor that also accepts a Boolean value:

public MemoryStream(byte [] buffer, bool writable) { ... }
Setting the writable parameter to false forces the MemoryStream to return false from its CanWrite property and to prevent any write operations on the stream. This makes the stream immutable, which makes sense given that the source string is, by definition, also immutable. Such a class is shown here:
public class StringStream : MemoryStream
{
    public StringStream(string str) :
        base(Encoding.Unicode.GetBytes(str), false) {}
}
To pass your string to the method that accepts a Stream, now all you have to do is create a StringStream (passing your string to its constructor) and send that stream into the method.

Of course, this approach effectively doubles the amount of memory in use for the string's data, which might be prohibitive if the string is big. Moreover, if the string is sufficiently large, there's a good chance the allocated byte array could end up on the Large Object Heap. This means that even if the method call is brief and the array isn't used after the method call, the garbage collector won't clean up the array until it performs a full collection.

Figure 1 shows an alternative: a custom Stream-derived class. As opposed to the simple StringStream implementation that I showed previously, this implementation is meant to minimize the amount of overhead required to wrap a string as a stream.

Figure 1 Alternate StringStream Implementation (Close Figure 1)
public class StringStream : Stream {
    private readonly string _str;
    private readonly long _byteLength;
    private int _position;

    public StringStream(string str) {
        if (str == null) throw new ArgumentNullException("str");
        _str = str;
        _byteLength = _str.Length*2;
        _position = 0;
    }

    public override bool CanRead { get { return true; } }
    public override bool CanSeek { get { return true; } }
    public override bool CanWrite { get { return false; } }

    public override long Length { get { return _byteLength; } }

    public override long Position {
        get { return _position; }
        set
        {
            if (value < 0 || value > int.MaxValue)
                throw new ArgumentOutOfRangeException("Position");
            _position = (int)value;
        }
    }

    public override long Seek(long offset, SeekOrigin origin) {
        switch(origin)
        {
            case SeekOrigin.Begin: Position = offset; break;
            case SeekOrigin.End: Position = _byteLength + offset; break;
            case SeekOrigin.Current: Position = Position + offset; break;
        }
        return Position;
    }

    public override int Read(byte[] buffer, int offset, int count) {
        if (_position < 0) throw new InvalidOperationException();

        int bytesRead = 0;
        while(bytesRead < count)
        {
            if (_position >= _byteLength) return bytesRead;

            char c = _str[_position / 2];
            buffer[offset + bytesRead] = (byte)((_position % 2 == 0) ?
                c & 0xFF : (c >> 8) & 0xFF);
            Position++;
            bytesRead++;
        }
        return bytesRead;
    }

    public override void Write(byte[] buffer, int offset, int count) {
        throw new NotSupportedException();
    }

    public override void Flush() {
        throw new NotSupportedException();
    }

    public override void SetLength(long value) {
        throw new NotSupportedException();
    }
}

Deriving from Stream requires the implementation of 10 abstract methods and properties: CanRead, CanSeek, CanWrite, Flush, Length, Position, Read, Seek, SetLength, and Write. For an immutable stream, CanWrite should return false, and the implementations of Flush, Write, and SetLength should all throw a NotSupportedException (they all deal with changes to the stream's data). CanRead should return true; otherwise, there would be no point to this wrapper.

The constructor accepts as a parameter the string to be wrapped and stores it. It then computes the length of the stream, which is the length of the string multiplied by two, since a System.String is a series of Unicode UTF-16 code points and each code point is 2 bytes. This value is exposed by the Stream's Length property.

The Position property is very straightforward, providing access to the current byte position within the stream. The Seek method then takes advantage of the Position property, using it to compute a new position based on the provided arguments and then storing that new value back to Position.

That just leaves the Read method, the core of the class. A client calls the Read method, passing in a byte array to which the read data should be stored, an offset into the array at which the data should start, and the number of bytes to be read. The method then returns the number of bytes read. To make the implementation simple, I loop until the requested number of bytes has been read (or until there are no more to read), storing 1 byte each time through the loop. This loop consists of picking up the next character from the string and extracting from it the appropriate byte by using some simple bit manipulation. I could have used System.BitConverter.GetBytes, but that would result in a 2-byte array being allocated each time I retrieved the bytes for a character, and since the whole point of this exercise is to avoid extraneous allocations for large strings, that would have been a bit counterintuitive.

Q I'd like to execute a method, but only allow it to run for a certain amount of time. Is there any way to do this?

A From a reliability perspective, a good way to do this is to incorporate knowledge of the time limit into the method you're calling. Internally, that method can poll for the amount of time it's used. Alternatively, if it blocks for a resource to become available, it can wait for only a specified period of time if that capability is available for the resource. One example of this is using Monitor.TryEnter instead of Monitor.Enter, or using WaitHandle.WaitOne(TimeSpan, bool) instead of WaitHandle.WaitOne(). Of course, if doing so is infeasible, if you don't want to mess with the method's logic to account for this, or if the code in the method to be executed is out of your control, you need an alternate approach.

One solution comes in the form of the Abort method on System.Threading.Thread. Thread.Abort raises a ThreadAbortException in the thread corresponding to the Thread object on which it is invoked, beginning the process of terminating that thread. Thus, you can create a separate thread to run the method in question and then wait the amount of time allocated for that method to complete. If it doesn't complete in the time allotted, you can abort the thread, which in most situations will cause the method to stop executing. I've implemented this solution in the TimedOperation class, shown in Figure 2.

Figure 2 Implementing Timeouts for Method Calls (Close Figure 2)
public class TimedOperation {
    public static object Invoke(TimeSpan time, out bool aborted,
        Delegate method, params object[] parameters)
    {
        if (method == null) throw new ArgumentNullException("method");
        if (time.TotalSeconds <= 0)
            throw new ArgumentOutOfRangeException("time");

        Operation op = new Operation();
        op.Method = method;
        op.Parameters = parameters;

        Thread t = new Thread(new ThreadStart(op.Run));
        t.IsBackground = true;
        t.Start();

        if (!t.Join(time))
        {
            aborted = true;
            t.Abort(); // dangerous!
            return null;
        }
        else
        {
            aborted = false;
            if (op.Exception != null) throw op.Exception;
            return op.Result;
        }
    }

    private class Operation {
        public volatile Delegate Method;
        public volatile object [] Parameters;
        public volatile Exception Exception;
        public volatile object Result;

        public void Run()
        {
            try
            {
                this.Result = Method.DynamicInvoke(Parameters);
            }
            catch(ThreadAbortException) { throw; }
            catch(Exception exc)
            {
                this.Exception = exc;
            }
        }
    }
}

Let's say you want to invoke the method YourMethod, allowing it only five seconds to run. You could use TimedOperation to do this, as shown in the following code:

bool timeExpired;
object result = TimedOperation.Invoke(TimeSpan.FromSeconds(5),
    out timeExpired, new SomeDelegate(YourMethod), new object[]{...});
if (timeExpired) Console.WriteLine("Ran out of time.");
else Console.WriteLine("Result: {0}", result);
The first parameter is the amount of time for which you want to let the method run. When TimedOperation.Invoke returns, the second parameter, an out Boolean, indicates whether the called method returned on its own accord or due to the thread being aborted when time ran out. The third and fourth parameters are the delegate to be executed and the parameters to be used for this delegate's invocation.

TimedOperation.Invoke first checks to make sure that the parameters are valid, namely that a delegate was supplied and that the amount of time allowed is positive (it wouldn't make much sense to allow the method negative time). The delegate and the parameters are then packaged into an instance of the internal Operation class. This class provides the method that will be run on the temporary thread. The user-supplied delegate can't be used directly as the ThreadStart method for several reasons. First and foremost, doing so would only allow TimedOperation to work with methods that accept no parameters and that return no value. Second, it would be more difficult to rethrow any exceptions thrown from this method to the caller of TimedOperation.Invoke.

With this Operation instance in hand, a new background thread is started with the Operation's Run method. The Run method simply executes the user-supplied delegate within a try block: any exceptions besides ThreadAbortException thrown from the delegate are caught and stored in the Operation instance, along with any value returned by the delegate, if any. After starting the thread, the Invoke method waits for the worker thread to finish processing by using the Thread.Join overload that accepts a timeout. If Thread.Join returns true, it means that the worker thread completed within the time specified. In this case, Invoke rethrows any exceptions that might have been thrown by the delegate, or it returns the value returned by the delegate (any out or ref parameters supplied to the delegate will be accessible to the caller of TimedOperation.Invoke through the parameters argument). If, however, Thread.Join returns false, it means that the worker thread did not complete in the time allotted. As a result, Invoke calls Abort on the thread object and returns null.

This technique is used by the Microsoft® .NET Framework in a couple of places, the most prominent in ASP.NET. ASP.NET implements an execution timeout for all requests, which by default is 90 seconds. The HttpContext for each request, along with the Thread that's processing it, is stored on a list which is periodically culled by a worker thread in the ASP.NET runtime. If a request is found on this queue past its execution timeout, the worker thread cancels the request using Thread.Abort to abort the relevant thread.

I mentioned earlier that TimedOperation.Invoke will in most situations cause the thread to stop executing, but there are, in fact, situations in which it will not. ThreadAbortException is a special exception in that it can be caught but, when the catch block finishes processing, the exception is rethrown by the runtime. However, if code has been granted the ControlThread security permission, it is able to explicitly ignore any ThreadAbortExceptions thrown, using the Thread.ResetAbort method to prevent the runtime from rethrowing the exception. In this scenario, the method you invoke could continue to run even after you try to stop it. Additionally, ThreadAbortException will still allow the relevant catch and finally blocks to run. Any one of these blocks could enter into an infinite loop, preventing the thread from being aborted in a graceful manner. As a third example, the common language runtime (CLR) is only able to abort threads when they are executing managed code; any thread a user tries to abort that is currently executing unmanaged code won't be aborted until it returns to the managed world.

Unfortunately, there are possibly worse problems with this approach. The point of Thread.Abort is to stop a thread running, regardless of what it's doing, and in general it's very good at its job. Consider the following code:

IntPtr someMemory = Marshal.AllocHGlobal(1024);
try
{
    ...
}
finally
{
    Marshal.FreeHGlobal(someMemory);
}

If this code is executing on the thread you're aborting, and it just so happens that the abort happens after the call to AllocHGlobal and before the pointer to that allocated memory is stored to the someMemory local variable, or if it happens after the value is stored to someMemory but before the try block is entered, congratulations, you have a memory leak. Or what if the thread in question entered a lock (such as a Mutex) and was aborted before it could exit it? Congratulations, you've now orphaned the lock, deadlocking other threads that later attempt to acquire that lock. These types of constructs don't even need to be in your own code in order to run into these types of problems. Using any class from the .NET Framework that relies on creating Win32® objects and receiving handles to them, such as System.IO.FileStream, puts you at risk.

The .NET Framework 2.0 introduces new reliability features, including SafeHandles and Constrained Execution Regions, which will help to make your applications much more reliable in the face of asynchronous exceptions such as ThreadAbortException. However, you will still need to be careful when writing code that uses Thread.Abort, as it can cause unexpected failures.

Consider a type that makes use of a static constructor. If this static constructor throws an exception, any future attempts to run that constructor will result in the same exception being thrown, wrapped in a TypeInitializationException. So, if you abort a thread while it is running a static constructor, a ThreadAbortException will be raised within the constructor, thereby most likely preventing the use of that type in the current application domain. If you knew there was a chance that a specific type's static constructor might be aborted in this fashion, you could attempt a preemptive strike by using the RunClassConstructor method on the System.Runtime.CompilerServices.RuntimeHelpers class to run the type's static constructor before the thread is aborted, but that's nothing if not a hack.

If you have some knowledge of the method you want to run with a timeout, you might be able to avoid using Thread.Abort and instead opt for Thread.Interrupt, which is much safer from a reliability perspective but isn't as immediate in its results as Thread.Abort. Thread.Interrupt throws a ThreadInterruptException on the target thread, but only when the thread is in a wait, sleep, or join state; if the target thread is in none of these states, an exception won't be thrown until the thread enters one of them. This is safer than Thread.Abort because a thread will not be in one of these states unless it is explicitly coded to be there (such as with a call to Monitor.Enter, Thread.Sleep, or Thread.Join), and thus the previous example using Marshal.AllocHGlobal would be more reliable, unless of course it was coded poorly, such as in the following (purposely silly) way:

IntPtr someMemory = Marshal.AllocHGlobal(1024);
Thread.Sleep(10);
try
{
    ...
}
finally
{
    Marshal.FreeHGlobal(someMemory);
}

The flip side to this is that the target thread may never exit. Consider the following method:

void LoopForever() { while(true); }
This method immediately enters an infinite loop, and while a Thread.Abort would cause its operation to cease, it never enters the WaitSleepJoin thread state, and thus a Thread.Interrupt will be unable to stop it.

The moral of the story is that it's best to avoid Thread.Abort whenever possible, but on the other hand there's no better way to stop a method's execution if you have no control over the implementation of the method in question. Just be aware that by using Thread.Abort without extreme care, you could possibly be putting the current application domain, or even worse the whole process, into an inconsistent state.

Send your questions and comments to netqa@microsoft.com.

Stephen Toub is the Technical Editor for MSDN Magazine.


C++ at Work: Enum Declarations, Template Function Specialization

C++ At Work
Enum Declarations, Template Function Specialization
Paul DiLascia

Code download available at: CAtWork0507.exe (204KB)

Q I have just read your article, "Wrappers: Use Our ManWrap Library to Get the Best of .NET in Native C++ Code," in the April 2005 issue, but the utility DumpEnum was not provided. Where may it be obtained?

Jarrad Waterloo

Given the name of a Microsoft® .NET Framework class like MenuItem or Form, is there some way to programmatically find which assembly contains that class?

Several Readers

A I'm happy to explain how I wrote DumpEnum and I'll even give you the code. In the process I'll answer the second question, too. But first let me explain for other readers just what DumpEnum does. One of the things I had to do in my April 2005 article was write a C++ enum to exactly match the .NET Framework type RegexOptions. RegexOptions is an enumerated (enum) type you can use with methods like Regex::Match and Replace to control matching. For example, you can call Regex::Match with RegexOptions::IgnoreCase to ignore case or RegexOptions::Singleline in order to treat the input string as one line. Figure 1 shows the values in RegexOptions.

Figure 1 RegexOptions (Close Figure 1)

Member Name | Value
Compiled | 8
CultureInvariant | 512
ECMAScript | 256
ExplicitCapture | 4
IgnoreCase | 1
IgnorePatternWhitespace | 32
Multiline | 2
None | 0
RightToLeft | 64
Singleline | 16

To pass RegexOptions from native C++ code, you need a C-style enum with the same values. The quickest way to write such code if you only need to do it once is to copy from the documentation into your C++ file, then edit it to follow C syntax. But what if you make a typo, or the options ever change, or you want to wrap several enum types? Better and more reliable is to write a tool that generates the C++ code automatically—especially since reflection makes it easy: the .NET Framework provides all the information needed to describe itself. So I wrote a little program called DumpEnum that you can run from a command prompt, like so:

DumpEnum RegexOptions

DumpEnum writes the name/value pairs as C/C++ code to the standard output. You can redirect the output to a file by typing

DumpEnum RegexOptions > regopt.h
then inserting regopt.h into your header file. That's what I did for RegexWrap.h in my article. Figure 2 shows the actual file DumpEnum generates for RegexOptions.

Figure 2 RegexOptions in C++ (Close Figure 2)
//////////////////
// Enumeration for System.Text.RegularExpressions.RegexOptions
// in System.dll. Underlying type is System.Int32.
// Automatically generated by DumpEnum.
//
enum RegexOptions {
   None = 0,
   IgnoreCase = 1,
   Multiline = 2,
   ExplicitCapture = 4,
   Compiled = 8,
   Singleline = 16,
   IgnorePatternWhitespace = 32,
   RightToLeft = 64,
   ECMAScript = 256,
   CultureInvariant = 512,
};

So now that you know what DumpEnum does, you should see how I implemented it.

Every Framework class is described by a System.Type class. It has properties like Name to get the name of the type and IsEnum to tell if the type is an enumeration. The first thing DumpEnum has to do is get hold of the Type for whatever type name was passed on the command line—for example, RegexOptions. It turns out this isn't as easy as you might hope. The two most common ways to get the Type are to invoke obj->GetType if you have an object instance, or __typeof if you don't. For example:

#using <System.dll>
using namespace System::Text::RegularExpressions;
...
Type *t = __typeof(RegexOptions);

In C++ you have to use __typeof for managed types because typeof is already defined (in the new C++\CLI you would use typeid<>). But __typeof only works with a symbol name, not a string, which presupposes you know the name at compile time, as well as which assembly and namespace it inhabits. DumpEnum doesn't have this information at compile time; it gets the type name as a command-line parameter. DumpEnum doesn't have an object instance either, so it can't call Object::GetType. What to do?

There is another way to get the Type. If you know which assembly the type lives in, you can load it and call Assembly::GetType, which takes a string. But Assembly::GetType requires the full type name, and you have to know which assembly to load. My first stab at DumpEnum required me to type this information on the command line, as shown in the following:

DumpEnum System.dll System.Text.RegularExpressions.RegexOptions

You know RegexOptions lives in System.dll and you know the namespace is System.Text.RegularExpressions because the documentation says so. But looking in the documentation is such a bore, and typing the full namespace is enough to give you carpal tunnel syndrome. I'd rather spend three hours writing a program that lets me type 12 characters instead of 54, than spend 30 seconds looking up the DLL/namespace and typing the full type name. This may seem like a laziness paradox, but it's what the good-programmer ethos demands because you end up with a cool tool you can use forever—and can share with your friends!

But before I can write DumpEnum the way I want, I have to solve the more general problem, the one the second question poses: given a (possibly unqualified) type name like RegexOptions, how do I find which Framework DLL contains that type? There's no central database or registry entry that holds this information (and a good thing, too), or method to call, so it appears hopeless. But wait a minute—think again! All the Framework DLLs live in one folder, and there were only 70 or so of them the last time I counted. Why not use brute force? Just load every assembly in the known Framework universe, looking for ones that have a type whose name matches the one desired. This is straightforward and only takes a few seconds on my lowly 1 GHz P3. I wrote a program called FindType that does the job:

FindType MenuItem

Figure 3 shows the results. FindType lists all the Framework DLLs that export a type with MenuItem in the name. FindType looks for types that contain the text appearing as a word. In other words, if you run "FindType Control", FindType reports System.Windows.Forms.Control and System.Web.UI.Control, but not System.Web.UI.WebControls.WebControl. The idea is you're supposed to enter the short name, the one you'd use if you were #using the right namespace. FindType accomplishes this by building a regular expression of the form "\bControl\b". The special anchor character (atomic zero-width assertion) '\b' matches a word break without including the break character in the match.

Figure 3 FindType in Action
Figure 3 FindType in Action

How does FindType know which DLLs to search? All the Framework DLLs live in a single folder with a name like C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322. You can get the actual path for your machine by concatenating the environment variables %FrameworkDir% (which resolves to "C:\WINDOWS\Microsoft.NET\Framework") and %FrameworkVersion% ("v1.1.4322"). To make life easy, I encapsulated the process in a class CFindType. To use it, you have to derive your own specialization and implement the virtual function OnMatch:

class CMyFindType : public CFindType {
protected:
   virtual BOOL OnMatch(LPCTSTR typName, LPCTSTR asmPath)
   {
      // print it
   }
};
Then you can instantiate and call the method FindExact:
CMyFindType ft;
int nFound = ft.FindExact("MenuItem");

CFindType grovels through the assemblies, looking for ones with types whose names match the one passed. Whenever it finds a type that matches, it calls your virtual OnMatch handler with the type and assembly names. If you want to use a different regular expression, you can call CFindType::Find instead of FindExact. CFindType, the class, makes implementing FindType, the program, easy: Just write an OnMatch handler that displays the information on stdout. For details, download the code available on the MSDN®Magazine Web site.

CFindType is itself derived from an even more general class, CEnumTypes, which enumerates all the types in the Framework, calling a virtual OnType function for each one. CFindType::OnType uses Regex to compare the type name with the one requested, and calls OnMatch if it matches. CEnumTypes uses _tgetenv to get the environment variables to build a filespec of the form "FrameworkDir\FrameworkVersion\*.dll", and it uses MFC's CFileFind class to enumerate the DLLs. CEnumTypes attempts to load each DLL as an assembly. If the load fails (perhaps the DLL is not a managed assembly), CEnumTypes ignores it and keeps searching. If the load succeeds, it calls Assembly::GetExportedTypes to get an array of Types the assembly exports, and calls OnType for each one. Figure 4 shows the code.

Figure 4 FindType.h (Close Figure 4)
////////////////////////////////////////////////////////////////
// MSDN Magazine — July 2005
// If this code works, it was written by Paul DiLascia.
// If not, I don't know who wrote it.
// Compiles with Visual Studio .NET 2003 (V7.1) on Windows XP. Tab size=3.
//
#pragma once

using namespace System::Reflection;
using namespace System::Text::RegularExpressions;

//////////////////
// Class to enumerate all the types in all the assemblies in
// the Framework.You must derive from this and override OnType
// to do something with each type found.
//
class CEnumTypes {
protected:
   BOOL m_bAbort; // derived classes can set this to stop enumerating

   // pure virtual function—you must override
   virtual BOOL OnType(LPCTSTR typName, LPCTSTR asmPath) = 0;

public:
   CEnumTypes() : m_bAbort(FALSE) { }
   virtual ~CEnumTypes() { }

   // enumerate all types, calling virtual OnType for each one.
   int Enumerate()
   {
      CFileFind finddlls;
      int nfound=0;

      // Create filespec: look for *.dll in Framework dir
      CString dllspec;
      dllspec.Format(_T("%s\\%s\\%s"),
         _tgetenv(_T("FrameworkDir")),
         _tgetenv(_T("FrameworkVersion")),
         _T("*.dll"));

      BOOL bMore = finddlls.FindFile(dllspec);
      while (bMore && !m_bAbort) {
         // for each DLL:
         bMore = finddlls.FindNextFile();
         CString asmPath = finddlls.GetFilePath();
         try {
            // load assembly, enumerate all types
            Assembly* a = Assembly::LoadFile(asmPath);
            Type* types[] = a->GetExportedTypes();
            int n = types->Length;
            for (int i=0; i<n; i++) {
               if (OnType(CString(types[i]->FullName), asmPath))
                  nfound++;
            }
         } catch (Exception* /*e*/) {
            // couldn't load: skip (do nothing)
         }
      }
      return nfound;
   }
};

//////////////////
// Derived specialization to find all the types/assemblies that contain
// a type whose name matches a given string or regex. You must derive
// from this and override OnMatch to do something with each type found.
//
class CFindType : public CEnumTypes {
protected:
   CString m_regex; // regular expression to match type name

   // pure virtual function—you must override
   virtual BOOL OnMatch(LPCTSTR typName, LPCTSTR asmPath) = 0;

   // override CEnumTypes to count only types whose name matches regex.
   virtual BOOL OnType(LPCTSTR typName, LPCTSTR asmPath)
   {
      return Regex::Match(typName, m_regex)!=Match::Empty
         && OnMatch(typName, asmPath);
   }

public:
   // Search for the type appearing as a word by itself—use the regex
   // "\btypename\b" (typename surrounded by word breaks). So if you
   // search for "Menu" CFindType will find System.Foo.Menu but not
   // "System.Foo.MenuItem."
   //
   int FindExact(LPCTSTR typname)
   {
      m_regex.Format(_T("\\b%s\\b"), typname);
      return Enumerate();
   }

   // Search using arbitrary regex
   int Find(LPCTSTR regex)
   {
      m_regex = regex;
      return Enumerate();
   }
};

Finally, now that I have CFindType, I can use it to solve my original problem: fixing DumpEnum so I don't have to tell it the assembly and full type names. DumpEnum uses CFindType with a different OnMatch handler. The handler in DumpEnum checks that the type is in fact an enumerated type (Type::IsEnum returns True) and, if so, dumps the enum's name/value pairs as C++ code, as in Figure 2. The DumpIt function that actually does the work is shown in Figure 5. DumpIt uses Enum::GetUnderlyingType to get the underlying type of the enumeration (usually System.Int32), Enum::GetValues to get the enumeration values, and Convert::ChangeType to convert the enumeration values to their underlying type. Here's the code that prints the name/value pairs:

Type* entype = // managed enum type
Array* values = Enum::GetValues(entype);
for (int i=0; i<values->Length; i++) {
   Object* enval = values->GetValue(i);
   Object* unval = Convert::ChangeType(enval, untype);
   _tprintf(_T("\t%s = %s,\n"), enval, unval);
}

Figure 5 DumpIt.cpp (Close Figure 5)
//////////////////
// Dump enumerated Framework type as C/C++ code.
// By this point, entype is guaranteed enum type.
//
void DumpIt(Type* entype, LPCTSTR asmPath) {
   // Dump  name/value pairs as C code
   Type* untype = Enum::GetUnderlyingType(entype);
   _tprintf(_T("//////////////////\n"));
   _tprintf(_T("// Enumeration for %s in %s.\n"),
      CString(entype->FullName), ShortName(asmPath));
   _tprintf(_T("// Underlying type is %s. Automatically "
      "generated by DumpEnum.\n"), CString(untype->ToString()));
   _tprintf(_T("//\n"));
   _tprintf(_T("enum %s {\n"), CString(entype->Name));

   Array* values = Enum::GetValues(entype);
   for (int i=0; i<values->Length; i++) {
      Object* enval = values->GetValue(i);
      Object* unval = Convert::ChangeType(enval, untype);
      _tprintf(_T("\t%s = %s,\n"), CString(enval->ToString()),
         CString(unval->ToString()));
   }
   _tprintf(_T("};\n"));
}

So now you have two programs for the price of one: FindType lets you find which DLL/assembly contains a specific type; DumpEnum generates C/C++ code to wrap Framework enums.

Before I move on, there's a footnote to the story. After I wrote FindType, I discovered to my dismay there's already a FindType program that comes with Visual Studio® .NET—it even has the same name! The Microsoft version does the same thing as mine, only better. (Ohmygosh, the Redmondtonians are a step ahead of me; I must be slipping!)

Microsoft FindType has options to specify exact or partial match, which directories and namespaces to search, whether to show methods, properties, events, and so on. Figure 6 shows the help screen listing all the options. FindType is a great program to study if you want to learn more about reflection, assemblies, and types, or if you have nothing better to do on a Friday night. You can find FindType in VS.NET\SDK\v1.1\Samples\Applications\TypeFinder, with projects for C# and Visual Basic. I built the C# version and copied the EXE to my bin directory; I use FindType all the time now to look up types.

Figure 6 FindType from Visual Studio (Close Figure 6)

Does the Redmontonian FindType render my own version totally useless? Of course not. For one thing, CFindType and CEnumTypes are classes, not programs, which means you can use them to write your own tools that need to look up type information, programs like DumpEnum. And my classes come in the best programming language of all: C++. The Microsoft version only comes in Visual Basic® and C#. If you decide to use CFindType within a larger application of your own, you should be aware that CFindType makes no attempt to unload the assemblies as they're enumerated. Since each assembly can consume a fair amount of memory, you could consider changing the implementation to load each assembly into its own application domain, and then unload the application domain when you're done searching the assembly.

Q I'm using the source code from a template-based library. This library includes some specializations of a template function for a specific type. The class template, function template, and template function specialization are all in header files. I #included the headers into my .cpp file and my project compiled and linked. But to use the library in my whole project I #included the headers in stdafx.h. Now I get multiply defined symbol errors for the specialized template functions. How should I organize the header files to avoid multiply defined symbol errors? At the moment, I'm using /FORCE:MULTIPLE, but I would like a more elegant solution.

Lee Kyung Jun

A Indeed, there is a more elegant solution. I'll explain in a moment, but first let me review how template function specializations work. Suppose you have a template function that compares two objects based on operator> and operator==:

template <typename T>
int compare(T t1, T t2)
{
   return t1==t2 ? 0 : t1 > t2 ? 1 : -1;
}

The function this template generates returns zero or +/-1 depending on whether the first argument is equal to, greater than, or less than the second argument. It's the typical sort of function you need for sorting collections. It assumes the type T has operator== and operator> and works fine for ints, floats, doubles, or DWORDS. But it doesn't work as expected if you use it to compare character strings (char* pointers) because the function generated compares pointers, not the strings themselves:

LPCTSTR s1,s2;
...
int cmp = compare(s1,s2); // s1<s2? Oops!
To make compare work for strings, you need a template specialization that uses strcmp or its TCHAR equivalent, _tcscmp:
// specialization for strings
template<>
int compare<LPCTSTR>(LPCTSTR s1, LPCTSTR s2)
{
   return _tcscmp(s1, s2);
}

So far, so good. Now the question is: where should you put this specialization? The obvious place is in the header file with the template. But this can lead to the multiply defined symbol errors that Lee encountered. The reason is clear if you remember that a template specialization is a function, not a template. It's the same as if you'd written:

int compare(LPCTSTR s1, LPCTSTR s2)
{
   return _tcscmp(s1, s2);
}
There's no reason you can't define a function in a header file—but if you do, you can't then #include the header in multiple files. At least, not without the linker complaining. So what should you do?

If you hold tight to the concept that a template function specialization is a function, not a template, you'll realize there are three options, exactly the same as for ordinary functions: you can make the specialization inline, extern, or static. For example, take a look at the following:

template<>
inline int compare<LPCTSTR>(LPCTSTR s1, LPCTSTR s2)
{
   return _tcscmp(s1, s2);
}

This is the easiest and most common solution for most template libraries. Since the compiler expands inline functions directly without generating an external symbol, it's okay to #include them in multiple modules. The linker won't complain because there are no symbols to be multiply defined. For small functions like compare, inline is what you want anyway (it's faster).

But what if your specialization is long, or you don't want to make it inline for some reason? Then you can make it extern. The syntax is the same as for normal functions:

// in .h header file
template<>
extern int compare<LPCTSTR>(LPCTSTR s1, LPCTSTR s2);

Of course, now you have to implement compare somewhere. Figure 7 shows part of a program I wrote to illustrate the details. I implemented the specialization in a separate module Templ.cpp that's linked with the main project. Templ.h is #included in stdafx.h, which is included in both Templ.cpp and the main module—but the project builds with no link errors. Go ahead, download the source and try it yourself.

Figure 7 Extern Template Specialization (Close Figure 7)

templ.h
#pragma once

//////////////////
// Basic function template to compare two types based on operators.
// Returns zero or +/-1 depending on whether t1 is equal to, greater
// than, or less than t2.
//
template <typename T>
int compare(T t1, T t2)
{
   return t1 == t2 ? 0 : t1 > t2 ? 1 : -1;
}

//////////////////
// Specialization for LPCTSTR: note only DECLARED here.
// Implementation is in Templ.cpp
//
template<> extern int compare<LPCTSTR>(LPCTSTR s1, LPCTSTR s2);
templ.cpp
#include "stdafx.h"
#include "Templ.h"

//////////////////
// Implementation for LPCTSTR template specialization.
//
template<>
int compare<LPCTSTR>(LPCTSTR s1, LPCTSTR s2)
{
   return _tcscmp(s1, s2);
}

If you're writing a template library for other developers, the extern approach can be a pain because now you have to create a link library (.lib) with the object module that contains the specialization. If you already have a .lib, it's no big deal; if you don't, you might wish for a way to avoid introducing one. It's more elegant (and less trouble) to implement your templates using only header files. The easiest way is to use inline, but you could also put your specializations in a separate header file from their declarations and instruct developers to #include the specializations in only one module. Alternatively, you can keep everything in one file and use a preprocessor symbol to control instantiation:

#ifdef MYLIB_IMPLEMENT_FUNCS
template<>
int compare<LPCTSTR>(LPCTSTR s1, LPCTSTR s2)
{
   return _tcscmp(s1, s2);
}
#endif

With this approach, all modules include the header file, but only one #defines MYLIB_IMPLEMENT_FUNCS before including it. This approach won't work with precompiled headers because the compiler loads the precompiled version with whatever value MYLIB_IMPLEMENT_FUNCS had in stdafx.h.

The last and definitely least way to avoid multiply defined symbol errors is to make your specialization static:

template<>
static int compare<LPCTSTR>(LPCTSTR s1, LPCTSTR s2)
{
   return _tcscmp(s1, s2);
}
This keeps the linker quiet because static functions aren't exported outside their modules, and it lets you keep everything in one header file without introducing preprocessor symbols. But it's inefficient because now every module gets a copy of the function. If the function is small, it doesn't matter—but then why not use inlining?

So the short answer is: make your specialization inline or extern. Usually inline is the way to go. Either way, you have to edit the header file. If this isn't possible because you're using a third-party library and you can't or don't want to edit the source, you have no choice but to use the linker option /FORCE:MULTIPLE. And while you're waiting for your project to build, you can send the folks who wrote the library a nastygram explaining why it's not cool to define function template specializations that aren't inline or extern. Tell them I sent you.

Send your questions and comments for Paul to cppqa@microsoft.com.

Paul DiLascia is a freelance software consultant and Web/UI designer-at-large. He is the author of Windows++: Writing Reusable Windows Code in C++ (Addison-Wesley, 1992). In his spare time, Paul develops PixieLib, an MFC class library available at www.dilascia.com.


{ End Bracket }: Experimenting with F#

{ End Bracket }
Experimenting with F#
Don Syme

After five years of guiding generics for the Microsoft® .NET Framework into Visual Studio® 2005, I've turned my attention to attempting to achieve a synthesis between type-safe, scalable, math-oriented scripting and programming for .NET. I believe the industry needs to do more for scientists and mathematicians, where the demand for this kind of language is great.

Scientists love their scripting languages, which they use to drive tools such as MatLab and Mathematica to great effect, but they also need languages that scale with high performance. Math-oriented programming is also big business, from the modern trading floors that depend on trained mathematicians modeling financial products to the Xbox® games with their machine learning AI algorithms. At the moment, the gap between scientific scripts and applications seems too large.

Many readers may be familiar with the F# language from our group at Microsoft Research, Cambridge. F# is a Meta Language (ML) family language that aims to satisfy many of these requirements. It has a core language similar to that of OCaml's, which itself was based on Edinburgh ML. ML is a rather misunderstood programming language: it originated in the 1970s because logicians needed a type-safe scripting language for compositional, extensible symbolic manipulations.

F# is a research language made available to demonstrate the power of the .NET Framework. It is not a Microsoft product.

Scripting is only feasible if you don't have to write many types, so type inference was essential. The logicians also wanted to embed many mini-languages into their scripting language. During the 1980s and 1990s ML spawned into several general-purpose (GP) languages such as Standard ML, OCaml, and Haskell. However, in the context of .NET it seems that languages may gain more by focusing on the niches where they have the most to offer. There are many good GP languages on .NET, but there are not so many statically type-safe, math-oriented languages with the feel of script programming but the performance of .NET managed code.

.NET has brought a transformation: languages can now complement each other, rather than merely compete.A language that fills a niche can thrive—F# is a small language with a small library, but it's very powerful because F# can use all the libraries, tools, and runtime support of .NET and because other .NET-based languages can make use of F# code.

Math-oriented implies several things, among them values, functions, arrays, streams, tuples, discriminated unions, matrices, closures, pattern matching, succinct algorithms, and avoiding mutation-oriented programming except where truly needed. These are precisely the concerns of "functional" programming languages, along with type inference.

One of the great things about F# is that you write fewer classes—an important requirement for scripting languages. For example, the following F# code uses a simple inner function "diff". Its translation into C# uses additional explicit argument passing:

F#
let DistanceFromAverage (a,b,c) =
let avg = (a+b+c) / 3.0 in
let diff(x) = (avg - x)*(avg - x) in
Math.Sqrt(diff(a) + diff(b) + diff(c))

C#
private static double diff(double avg, double x)
{
    return (avg - x) * (avg - x);
}
public static double DistanceFromAverage(double a, double b, double c)
{
    double avg = (a+b+c) / 3;
    return Math.Sqrt(diff(avg,a) + diff(avg,b) + diff(avg,c));
}
Inner functions are not new, but type inference makes them convenient to use, and they save the programmer from manually "closure converting" her code. Likewise, function types reduce the number of explicit delegate and interface types, and in C# a simple discriminated union requires an encoding into four classes:
F#
type expr =
| Sum of expr * expr
| Product of expr * expr
| Const of double

C#
abstract class Expr { ... }
class Sum : Expr { Expr left; Expr right; ... }
class Product : Expr { Expr left; Expr right; ... }
class Const : Expr { Double const; ...}
Finally, object expressions can replace many classes that represent instantiations of extension points. These are like inner classes, but type inference gives a succinct notation. F# can be downloaded from research.microsoft.com/projects/ilx/fsharp.aspx. In short, ML-like languages would appear to have a lot to offer in the context of .NET as type-safe scripting languages, especially given that they can also scale to writing large programs.

Don Syme is a Researcher at Microsoft Research, located in Cambridge, England. He initiated the project leading to the design and implementation of generics for the CLR.