 MSDN Magazine May 2004

MSDN Magazine May 2004

C# 2.0: Create Elegant Code with Anonymous Methods, Iterators, and Partial Classes

C# 2.0 introduces a wealth of exiting new features, such as generics, iterators, partial classes and anonymous methods. While generics are the most talked-about feature especially for former classic C++ developers, the rest of the new features are important additions to your .NET development arsenal, enhancing power and improving overall productivity. This article is dedicated to all the new C# 2.0 capabilities besides generics to give you a good overall picture of the upcoming features.

Juval Lowy C++: Write Faster Code with the Modern Language Features of Visual C++ 2005

The next version of Visual C++ has a new syntax that is both elegant and powerful. It has new optimization technology that has improved the speed of Microsoft. It has new compilation modes that ensure Common Language Infrastructure compliance and verifiability for the .NET Framework, and it has new models for interop. In this article Stephen Toub explains these and other improvements to Visual C++.

Stephen Toub Visual Basic: Navigate the .NET Framework and Your Projects with "My"

The next version of Visual Basic, Visual Basic 2005, will include some powerful new features. One of the most interesting is the My language extensions: My.Application, My.Computer, My.Forms, My.Resources, My.Settings, My.User, and My.WebServices. The My language extensions take the idea of helper functions to a whole new level because they include so much functionality out of the box. Called "a speed-dial for the .NET Framework" by the author, the My extensions are a feature you won't want to overlook.

Duncan Mackenzie .NET Framework 2.0: Craft a Rich UI for Your .NET App with Enhanced Windows Forms Support

The upcoming version of the .NET Framework offers a host of enhancements an order of magnitude over and above existing versions. In particular, developers writing Windows Forms benefit from a variety of new and improved features targeting development, deployment, increased productivity, and auto-generated code. This article covers some of the key new features including designer enhancements, new controls, data binding, and deployment to give you a taste of what's to come.

Michael Weinhardt and Chris Sells ClickOnce: Deploy and Update Your Smart Client Projects Using a Central Server

ClickOnce is a new deployment technology that allows users to download and execute Windows-based client applications over the Web, a network share, or from a local disk. Users get the rich interactive and stateful experience of Windows Forms, but still have the ease of deployment and updates available to Web applications. ClickOnce applications can be run offline and support a variety of automatic and manual update scenarios.Learn all about it here.

Brian Noyes Debugging: DataTips, Visualizers and Viewers Make Debugging .NET Code a Breeze

There is a whole host of new goodies in upcoming release of Visual Studio 2005 that will enhance your debugging experience. One such improvement will make it easy to visualize types within the debugger. This article discusses those improvements and covers debugger attributes and type visualizers. In addition, the author shows you how to extend the display of your own types with custom attributes and how to plug in an assembly to visualize.NET types built into the framework.

Morgan Skinner Editor's Note: Introducing Our Online Corrections Page

One of the great things about working for a print publication like MSDN Magazine is the sense of permanence that the printed word provides. Sure, there are fringe benefits—the trade show groupies and expensive vacation junkets paid for by advertisers are great—but at the end of the day, once something appears in print most people consider it "official" in a way that a Web site can never replicate.

New Stuff: Resources for Your Developer Toolbox

Guardsoft has announced the availability of VSGuard, which allows you to compare internal data between two executing programs. This technique can reduce time spent on debugging and testing. It is useful whenever a program is modified during the development process or ported to a different environment, such as when moving the program to a new platform or during system upgrades.

Theresa W. Carey Web Q&A: Request Timeouts, Byte Array Conversion, and More

Edited by Nancy Michell Data Points: Saving Parent-child Data in a Multitiered App Using ADO.NET

John Papa Cutting Edge: Dress Your Controls for Success with ASP.NET 1.1 Themes

Dino Esposito The XML Files: XML Data Migration Case Study: GEDCOM

Aaron Skonnard Advanced Basics: Create a Graphical Editor Using RichTextBox and GDI+

Ken Spencer Basic Instincts: Updating the UI from a Secondary Thread

Ted Pattison .NET Matters: Finalizers, Assembly Names, MethodInfo, and More

Stephen Toub C++ Q&A: Updating Views in MFC, Spying on Events in the .NET Framework

Paul DiLascia Resource File: Microsoft Newsgroups for Developers

Microsoft offers more than 1,900 public and 500 private newsgroups that cover all aspects of Microsoft products and technologies. An individual newsgroup might get up to 15,000 postings per month, with developers answering customer questions, asking about specific product features, and getting product feedback.


C# 2.0: Create Elegant Code with Anonymous Methods, Iterators, and Partial Classes

[ Editor's Update - 1/20/2006: This article refers to a beta version of Visual Studio 2005. An updated version of the article, reflecting features found in the final release of Visual Studio 2005, can be found at C#: Create Elegant Code With Anonymous Methods, Iterators, And Partial Classes.]
C# 2.0

Create Elegant Code with Anonymous Methods, Iterators, and Partial Classes

Juval Lowy
This article was based on a pre-release version of Microsoft Visual Studio 2005, formerly code-named "Whidbey." All information contained herein is subject to change.
This article discusses:
  • Iterating over collections
  • Spanning class definitions across files
  • Anonymous methods for use with delegates
  • Other new C# features in Visual Studio 2005
This article uses the following technologies:
C# and Visual Studio
Code download available at: C20.exe (164KB)
Contents
Iterators
Iterator Implementation
Recursive Iterations
Partial Types
Anonymous Methods
Passing Parameters to Anonymous Methods
Anonymous Method Implementation
Generic Anonymous Methods
Anonymous Method Example
Delegate Inference
Property and Index Visibility
Static Classes
Global Namespace Qualifier
Inline Warning
Conclusion

F
ans of the C# language will find much to like in Visual C#® 2005. Visual Studio® 2005 brings a wealth of exciting new features to Visual C# 2005, such as generics, iterators, partial classes, and anonymous methods. While generics is the most talked-about and anticipated feature, especially among C++ developers who are familiar with templates, the other new features are important additions to your Microsoft® .NET development arsenal as well. These features and language additions will improve your overall productivity compared to the first version of C#, leaving you to write cleaner code faster. For some background information on generics, you should take a look at the sidebar "What are Generics?"

Iterators

In C# 1.1, you can iterate over data structures such as arrays and collections using a foreach loop:

string[] cities = {"New York","Paris","London"};
foreach(string city in cities)
{
   Console.WriteLine(city);
}

In fact, you can use any custom data collection in the foreach loop, as long as that collection type implements a GetEnumerator method that returns an IEnumerator interface. Usually you do this by implementing the IEnumerable interface:

public interface IEnumerable
{
   IEnumerator GetEnumerator();
}
public interface IEnumerator
{
   object Current{get;}
   bool MoveNext();
   void Reset();
}

Often, the class that is used to iterate over a collection by implementing IEnumerable is provided as a nested class of the collection type to be iterated. This iterator type maintains the state of the iteration. A nested class is often better as an enumerator because it has access to all the private members of its containing class. This is, of course, the Iterator design pattern, which shields iterating clients from the actual implementation details of the underlying data structure, enabling the use of the same client-side iteration logic over multiple data structures, as shown in Figure 1.

Figure 1 Iterator Design Pattern
Figure 1 Iterator Design Pattern

In addition, because each iterator maintains separate iteration state, multiple clients can execute separate concurrent iterations. Data structures such as the Array and the Queue support iteration out of the box by implementing IEnumerable. The code generated in the foreach loop simply obtains an IEnumerator object by calling the class's GetEnumerator method and uses it in a while loop to iterate over the collection by continually calling its MoveNext method and current property. You can use IEnumerator directly (without resorting to a foreach statement) if you need explicit iteration over the collection.

But there are some problems with this approach. The first is that if the collection contains value types, obtaining the items requires boxing and unboxing them because IEnumerator.Current returns an Object. This results in potential performance degradation and increased pressure on the managed heap. Even if the collection contains reference types, you still incur the penalty of the down-casting from Object. While unfamiliar to most developers, in C# 1.0 you can actually implement the iterator pattern for each loop without implementing IEnumerator or IEnumerable. The compiler will choose to call the strongly typed version, avoiding the casting and boxing. The result is that even in version 1.0 it's possible not to incur the performance penalty.

To better formulate this solution and to make it easier to implement, the Microsoft .NET Framework 2.0 defines the generic, type-safe IEnumerable<ItemType> and IEnumerator<ItemType> interfaces in the System.Collections.Generics namespace:

public interface IEnumerable<ItemType>
{
   IEnumerator<ItemType> GetEnumerator();
}
public interface IEnumerator<ItemType> : IDisposable
{
   ItemType Current{get;}
   bool MoveNext();
}

Besides making use of generics, the new interfaces are slightly different than their predecessors. Unlike IEnumerator, IEnumerator<ItemType> derives from IDisposable and does not have a Reset method. The code in Figure 2 shows a simple city collection implementing IEnumerable<string>, and Figure 3 shows how the compiler uses that interface when spanning the code of the foreach loop. The implementation in Figure 2 uses a nested class called MyEnumerator, which accepts as a construction parameter a reference back to the collection to be enumerated. MyEnumerator is intimately aware of the implementation details of the city collection, an array in this example. The MyEnumerator class maintains the current iteration state in the m_Current member variable, which is used as an index into the array.

Figure 3 Simple Iterator (Close Figure 3)
CityCollection cities = new CityCollection();
//For this foreach loop:
foreach(string city in cities)
{
   Trace.WriteLine(city);
}
//The compiler generates this equivalent code:
IEnumerable<string> enumerable = cities;
IEnumerator<string> enumerator = enumerable.GetEnumerator();
using(enumerator)
{
   while(enumerator.MoveNext())
   {
      Trace.WriteLine(enumerator.Current);
   }
}
Figure 2 Implementing IEnumerable<string> (Close Figure 2)
public class CityCollection : IEnumerable<string>
{
   string[] m_Cities = {"New York","Paris","London"};

   public IEnumerator<string> GetEnumerator()
   {
      return new MyEnumerator(this);
   }
   //Nested class definition
   class MyEnumerator : IEnumerator<string>
   {
      CityCollection m_Collection;
      int m_Current;
      public MyEnumerator(CityCollection collection)
      {
         m_Collection = collection;
         m_Current = -1;
      }
      public bool MoveNext()
      {
         m_Current++;
         if(m_Current < m_Collection.m_Cities.Length)
            return true;
         else
            return false;
      }
      public string Current
      {
         get
         {
            if(m_Current == -1)
               throw new InvalidOperationException();
            return m_Collection.m_Cities[m_Current];
         }
      }
      public void Dispose(){}
   }
}

The second and more difficult problem is implementing the iterator. Although that implementation is straightforward for simple cases (as shown in Figure 3), it is challenging with more advanced data structures, such as binary trees, which require recursive traversal and maintaining iteration state through the recursion. Moreover, if you require various iteration options, such as head-to-tail and tail-to-head on a linked list, the code for the linked list will be bloated with various iterator implementations. This is exactly the problem that C# 2.0 iterators were designed to address. Using iterators, you can have the C# compiler generate the implementation of IEnumerator for you. The C# compiler can automatically generate a nested class to maintain the iteration state. You can use iterators on a generic collection or on a type-specific collection. All you need to do is tell the compiler what to yield in each iteration. As with manually providing an iterator, you need to expose a GetEnumerator method, typically by implementing IEnumerable or IEnumerable<ItemType>.

You tell the compiler what to yield using the new C# yield return statement. For example, here is how you use C# iterators in the city collection instead of the manual implementation of Figure 2:

public class CityCollection : IEnumerable<string>
{
   string[] m_Cities = {"New York","Paris","London"};
   public IEnumerator<string> GetEnumerator()
   {
      for(int i = 0; i<m_Cities.Length; i++)
         yield return m_Cities[i];
   }
}
You can also use C# iterators on non-generic collections:
public class CityCollection : IEnumerable
{
   string[] m_Cities = {"New York","Paris","London"};
   public IEnumerator GetEnumerator()
   {
      for(int i = 0; i<m_Cities.Length; i++)
         yield return m_Cities[i];
   }
}

In addition, you can use C# iterators on fully generic collections, as shown in Figure 4. When using a generic collection and iterators, the specific type used for IEnumerable<ItemType> in the foreach loop is known to the compiler from the type used when declaring the collection—a string in this case:

LinkedList<int,string> list = new LinkedList<int,string>();
/* Some initialization of list, then  */
foreach(string item in list)
{
   Trace.WriteLine(item);
}
This is similar to any other derivation from a generic interface.

Figure 4 Providing Iterators on a Generic Linked List (Close Figure 4)
//K is the key, T is the data item
class Node<K,T>
{
   public K Key;
   public T Item;
   public Node<K,T> NextNode;
}
public class LinkedList<K,T> : IEnumerable<T>
{
   Node<K,T> m_Head;
   public IEnumerator<T> GetEnumerator()
   {
      Node<K,T> current = m_Head;
      while(current != null)
      {
         yield return current.Item;
         current = current.NextNode;
      }
   }
   /* More methods and members */
}

If for some reason you want to stop the iteration midstream, use the yield break statement. For example, the following iterator will only yield the values 1, 2, and 3:

public IEnumerator<int> GetEnumerator()
{
   for(int i = 1;i< 5;i++)
   {
      yield return i;
      if(i > 2)
         yield break;
   }
}

Your collection can easily expose multiple iterators, each used to traverse the collection differently. For example, to traverse the CityCollection class in reverse order, provide a property of type IEnumerable<string> called Reverse:

public class CityCollection
{
   string[] m_Cities = {"New York","Paris","London"};
   public IEnumerable<string> Reverse
   {
      get
      {
         for(int i=m_Cities.Length-1; i>= 0; i--)
            yield return m_Cities[i];
      }
   }
}
Then use the Reverse property in a foreach loop:
CityCollection collection = new CityCollection();
foreach(string city in collection.Reverse)
{
   Trace.WriteLine(city);
}

There are some limitations to where and how you can use the yield return statement. A method or a property that has a yield return statement cannot also contain a return statement because that would improperly break the iteration. You cannot use yield return in an anonymous method, nor can you place a yield return statement inside a try statement with a catch block (and also not inside a catch or a finally block).

Iterator Implementation

The compiler-generated nested class maintains the iteration state. When the iterator is first called in a foreach loop (or in direct iteration code), the compiler-generated code for GetEnumerator creates a new iterator object (an instance of the nested class) with a reset state. Every time the foreach loops and calls the iterator's MoveNext method, it begins execution where the previous yield return statement left off. As long as the foreach loop executes, the iterator maintains its state. However, the iterator object (and its state) does not persist across foreach loops. Consequently, it is safe to call foreach again because you will get a new iterator object to start the new iteration. This is why IEnumerable<ItemType> does not define a Reset method.

But how is the nested iterator class implemented and how does it manage its state? The compiler transforms a standard method into a method that is designed to be called multiple times and that uses a simple state machine to resume execution after the previous yield statement. All you have to do is indicate what and when to yield to the compiler using the yield return statement. The compiler is even smart enough to concatenate multiple yield return statements in the order they appear:

public class CityCollection : IEnumerable<string>
{
   public IEnumerator<string> GetEnumerator()
   {
      yield return "New York";
      yield return "Paris";
      yield return "London";
   }
}

Let's take a look at the GetEnumerator method of the class shown in the following lines of code:

public class MyCollection : IEnumerable<string>
{
   public IEnumerator<string> GetEnumerator()
   {
      //Some iteration code that uses yield return
   }
}
When the compiler encounters a class member with a yield return statement such as this, it injects the definition of a nested class called GetEnumerator$<random unique number>__IEnumeratorImpl, as shown in the C# pseudocode in Figure 5. (Remember that all of the features discussed in this article—the names of the compiler-generated classes and fields—are subject to change, in some cases quite drastically. You should not attempt to use reflection to get at those implementation details and expect consistent results.)

Figure 5 The Compiler-generated Iterator (Close Figure 5)
public class MyCollection : IEnumerable<string>
{
   public virtual IEnumerator<string> GetEnumerator()
   {
      GetEnumerator$0003__IEnumeratorImpl impl;
      impl = new GetEnumerator$0003__IEnumeratorImpl;
      impl.<this> = this;
      return impl;
   }
   private class GetEnumerator$0003__IEnumeratorImpl :
      IEnumerator<string>
   {
      public MyCollection <this>; // Back reference to the collection
      string $_current;
      // state machine members go here
      string IEnumerator<string>.Current
      {
         get
         {
            return $_current;
         }
      }
      bool IEnumerator<string>.MoveNext()
      {
         //State machine management
      }
      IDisposable.Dispose()
      {
         //State machine cleanup if required
      }
   }
}

The nested class implements the same IEnumerable interface returned from the class member. The compiler replaces the code in the class member with an instantiation of the nested type, assigning to the nested class's <this> member variable a reference back to the collection, similar to the manual implementation shown in Figure 2. The nested class is actually the one providing the implementation of IEnumerator.

Recursive Iterations

Iterators really shine when it comes to iterating recursively over a data structure such as a binary tree or any complex graph of interconnecting nodes. With recursive iteration, it is very difficult to manually implement an iterator, yet using C# iterators it is done with great ease. Consider the binary tree in Figure 6. The full implementation of the tree is part of the source code available with this article.

Figure 6 Implementing a Recursive Iterator (Close Figure 6)
class Node<T>
{
    public Node<T> LeftNode;
    public Node<T> RightNode;
    public T Item;
}
public class BinaryTree<T>
{
   Node<T> m_Root;
   public void Add(params T[] items)
   {
      foreach(T item in items)
         Add(item);
   }
   public void Add(T item)
   {...}
   public IEnumerable<T> InOrder
   {
      get
      {
         return ScanInOrder(m_Root);
      }
    }
   IEnumerable<T> ScanInOrder(Node<T> root)
   {
      if(root.LeftNode != null)
      {
         foreach(T item in ScanInOrder(root.LeftNode))
         {
            yield return item;
         }
      }

      yield return root.Item;

      if(root.RightNode != null)
      {
         foreach(T item in ScanInOrder(root.RightNode))
         {
            yield return item;
         }
      }
   }
}

The binary tree stores items in nodes. Each node holds a value of the generic type T, called Item. Each node has a reference to a node on the left and a reference to a node on the right. Values smaller than Item are stored in the left-side subtree, and larger values are stored in the right-side subtree. The tree also provides an Add method for adding an open-ended array of values of the type T, using the params qualifier:

public void Add(params T[] items);

The tree provides a public property called InOrder of type IEnumerable<T>. InOrder calls the recursive private helper method ScanInOrder, passing to ScanInOrder the root of the tree. ScanInOrder is defined as:

IEnumerable<T> ScanInOrder(Node<T> root);
It returns the implementation of an iterator of the type IEnumerable<T>, which traverses the binary tree in order. The interesting thing about ScanInOrder is the way it uses recursion to iterate over the tree using a foreach loop that accesses the IEnumerable<T> returned from a recursive call. With in-order iteration, every node iterates over its left-side subtree, then over the value in the node itself, then over the right-side subtree. For that, you need three yield return statements. To iterate over the left-side subtree, ScanInOrder uses a foreach loop over the IEnumerable<T> returned from a recursive call that passes the left-side node as a parameter. Once that foreach loop returns, all the left-side subtree nodes have been iterated over and yielded. ScanInOrder then yields the value of the node passed to it as the root of the iteration and performs another recursive call inside a foreach loop, this time on the right-side subtree.

The InOrder property allows you to write the following foreach loop to iterate over the entire tree:

BinaryTree<int> tree = new BinaryTree<int>();
tree.Add(4,6,2,7,5,3,1);
foreach(int num in tree.InOrder)
{
   Trace.WriteLine(num);
}
// Traces 1,2,3,4,5,6,7
You can implement pre-order and post-order iterations in a similar manner by adding additional properties.

While the ability to use iterators recursively is obviously a powerful feature, it should be used with care as there can be serious performance implications. Each call to ScanInOrder requires an instantiation of the compiler-generated iterator, so recursively iterating over a deep tree could result in a large number of objects being created behind the scenes. In a balanced binary tree, there are approximately n iterator instantiations, where n is the number of nodes in the tree. At any given moment, approximately log(n) of those objects are live. In a decently sized tree, a large number of those objects will make it past the Generation 0 garbage collection. That said, iterators can still be used to easily iterate over recursive data structures such as trees by using stacks or queues to maintain a list of nodes still to be examined.

Partial Types

C# 1.1 requires you to put all the code for a class in a single file. C# 2.0 allows you to split the definition and implementation of a class or a struct across multiple files. You can put one part of a class in one file and another part of the class in a different file, noting the split by using the new partial keyword. For example, you can put the following code in the file MyClass1.cs:

public partial class MyClass
{
   public void Method1()
   {...}
}
In the file MyClass2.cs, you can insert this code:
public partial class MyClass
{
   public void Method2()
   {...}
   public int Number;
}

In fact, you can have as many parts as you like in any given class. Partial type support is available for classes, structures, and interfaces, but you cannot have a partial enum definition.

Partial types are a very handy feature. Sometimes it is necessary to modify a machine-generated file, such as a Web service client-side wrapper class. However, changes made to the file will be lost if you regenerate the wrapper class. Using a partial class, you can factor those changes into a separate file. ASP.NET 2.0 uses partial classes for the code-beside class (the evolution of codebehind), storing the machine-generated part of the page separately. Windows® Forms uses partial classes to store the visual designer output of the InitializeComponent method as well as the member controls. Partial types also enable two or more developers to work on the same type while both have their files checked out from source control without interfering with each other.

You may be asking yourself, what if the various parts define contradicting aspects of the class? The answer is simple: a class (or a struct) can have two kinds of aspects or qualities: accumulative and non-accumulative. The accumulative aspects are things that each part of the class can choose to add, such as interface derivation, properties, indexers, methods, and member variables.

For example, the following code shows how a part can add interface derivation and implementation:

public partial class MyClass
{}
public partial class MyClass : IMyInterface
{
   public void Method1()
   {...}
   public void Method2()
   {...}
}

The non-accumulative aspects are things that all the parts of a type must agree upon. Whether the type is a class or a struct, type visibility (public or internal) and the base class are non-accumulative aspects. For example, the following code does not compile because not all the parts of MyClass concur on the base class:

public class MyBase
{}
public class SomeOtherClass
{}
public partial class MyClass : MyBase
{}
public partial class MyClass : MyBase
{}
//Does not compile
public partial class MyClass : SomeOtherClass
{}
In addition to having all parts define the same non-accumulative parts, only a single part can override a virtual or an abstract method, and only one part can implement an interface member.

C# 2.0 supports partial types as follows: when the compiler builds the assembly, it combines from the various files the parts of a type and compiles them into a single type in Microsoft intermediate language (MSIL). The generated MSIL has no recollection which part came from which file. Just like in C# 1.1 the MSIL has no record of which file was used to define which type. Also worth noting is that partial types cannot span assemblies, and that a type can refuse to have other parts by omitting the partial qualifier from its definition.

Because all the compiler is doing is accumulating parts, a single file can contain multiple parts, even of the same type, although the usefulness of that is questionable.

In C#, developers often name a file after the class it contains and avoid putting multiple classes in the same file. When using partial types, I recommend indicating in the file name that it contains parts of a type such as MyClassP1.cs, MyClassP2.cs, or employing some other consistent way of externally indicating the content of the source file. For example, the Windows Forms designer stores its portion of the partial class for the form in Form1.cs to a file named Form1.Designer.cs.

Another side effect of partial types is that when approaching an unfamiliar code base, the parts of the type you maintain could be spread all over the project files. In such cases, my advice is to use the Visual Studio Class View because it displays an accumulative view of all the parts of the type and allows you to navigate through the various parts by clicking on its members. The navigation bar provides this functionality as well.

Anonymous Methods

C# supports delegates for invoking one or multiple methods. Delegates provide operators and methods for adding and removing target methods, and are used extensively throughout the .NET Framework for events, callbacks, asynchronous calls, and multithreading. However, you are sometimes forced to create a class or a method just for the sake of using a delegate. In such cases, there is no need for multiple targets, and the code involved is often relatively short and simple. Anonymous methods is a new feature in C# 2.0 that lets you define an anonymous (that is, nameless) method called by a delegate.

For example, the following is a conventional SomeMethod method definition and delegate invocation:

class SomeClass
{
   delegate void SomeDelegate();
   public void InvokeMethod()
   {
      SomeDelegate del = new SomeDelegate(SomeMethod);
      del();
   }
   void SomeMethod()
   {
      MessageBox.Show("Hello");
   }
}
You can define and implement this with an anonymous method:
class SomeClass
{
   delegate void SomeDelegate();
   public void InvokeMethod()
   {
      SomeDelegate del = delegate()
                         {
                             MessageBox.Show("Hello");
                         };
      del();
   }
}
What are Generics?

Generics allow you to define a type-safe data structure or a utility helper class without committing to the actual data types used. This results in a significant performance boost and higher-quality code because you get to reuse data processing algorithms without duplicating type-specific code. For example, instead of defining a linked list in terms of System.Object, you can define it in generic terms, using generic type parameters:

public class LinkedList<T>
{
   T[] m_Items  =  new T[100];
   public void AddHead(T item)
   {...}
   public T RemoveHead(T item);
   {...}
}

T is a merely a placeholder for the actual type to be used. When the client declares and instantiates the list, it provides the type to use instead of the generic type parameter:

LinkedList<int> list = new LinkedList<int>();
list.AddHead(3);

You can use multiple type parameters, such as K for the list key and T for the underlying data type stored:

   public class LinkedList<K,T>
   {...}

You can define generic interfaces and have the subclass either provide a particular type or you can keep the subclass generic:

   public interface IList<T>
   {...}
   public class NumberList :  IList<int>
   {...}
   public class LinkedList<T> :  IList<T>
   {...}

For additional information about generics, see Jason Clark's .NET columns in the September 2003 and October 2003 issues of MSDN® Magazine. There is also an MSDN whitepaper, "An Introduction to C# Generics," available.

The anonymous method is defined in-line and not as a member method of any class. Additionally, there is no way to apply method attributes to an anonymous method, nor can the anonymous method define generic types or add generic constraints.

You should note two interesting things about anonymous methods: the overloaded use of the delegate reserved keyword and the delegate assignment. You will see later on how the compiler implements an anonymous method, but it is quite clear from looking at the code that the compiler has to infer the type of the delegate used, instantiate a new delegate object of the inferred type, wrap the new delegate around the anonymous method, and assign it to the delegate used in the definition of the anonymous method (del in the previous example).

Anonymous methods can be used anywhere that a delegate type is expected. You can pass an anonymous method into any method that accepts the appropriate delegate type as a parameter:

class SomeClass
{
   delegate void SomeDelegate();
   public void SomeMethod()
   {
      InvokeDelegate(delegate(){MessageBox.Show("Hello");});
   }
   void InvokeDelegate(SomeDelegate del)
   {
      del();
   }
}
If you need to pass an anonymous method to a method that accepts an abstract Delegate parameter, such as the following
void InvokeDelegate(Delegate del);
first cast the anonymous method to the specific delegate type.

A concrete and useful example for passing an anonymous method as a parameter is launching a new thread without explicitly defining a ThreadStart delegate or a thread method:

public class MyClass
{
   public void LauchThread()
   {
      Thread workerThread = new Thread(delegate()
                            {
                                MessageBox.Show("Hello");
                            });
      workerThread.Start();
   }
}
In the previous example, the anonymous method serves as the thread method, causing the message box to be displayed from the new thread.

Passing Parameters to Anonymous Methods

When defining an anonymous method with parameters, you define the parameter types and names after the delegate keyword just as if it were a conventional method. The method signature must match the definition of the delegate to which it is assigned. When invoking the delegate, you pass the parameter's values, just as with a normal delegate invocation:

class SomeClass
{
   delegate void SomeDelegate(string str);
   public void InvokeMethod()
   {
      SomeDelegate del = delegate(string str)
                         {
                             MessageBox.Show(str);
                         };
      del("Hello");
   }
}

If the anonymous method has no parameters, you can use a pair of empty parens after the delegate keyword:

class SomeClass
{
   delegate void SomeDelegate();
   public void InvokeMethod()
   {
      SomeDelegate del = delegate()
                         {
                             MessageBox.Show("Hello");
                         };
      del();
   }
}
However, if you omit the empty parens after the delegate keyword altogether, you are defining a special kind of anonymous method, which could be assigned to any delegate with any signature:
class SomeClass
{
   delegate void SomeDelegate(string str);
   public void InvokeMethod()
   {
      SomeDelegate del = delegate
                         {
                             MessageBox.Show("Hello");
                         };
      del("Parameter is ignored");
   }
}

Obviously, you can only use this syntax if the anonymous method does not rely on any of the parameters, and you would want to use the method code regardless of the delegate signature. Note that you must still provide arguments when invoking the delegate because the compiler generates nameless parameters for the anonymous method, inferred from the delegate signature, as if you wrote the following (in C# pseudocode):

SomeDelegate del = delegate(string)
                   {
                       MessageBox.Show("Hello");
                   };
Additionally, anonymous methods without a parameter list cannot be used with delegates that specify out parameters.

An anonymous method can use any class member variable, and it can also use any local variable defined at the scope of its containing method as if it were its own local variable. This is demonstrated in Figure 7. Once you know how to pass parameters to an anonymous method, you can also easily define anonymous event handling, as shown in Figure 8.

Figure 8 Anonymous Method as Event Handler (Close Figure 8)
public class MyForm : Form
{
   Button m_MyButton;
   public MyForm()
   {
      InitializeComponent();
      m_MyButton.Click += delegate(object sender,EventArgs args)
                          {
                             MessageBox.Show("Clicked");
                          };
   }
   void InitializeComponent()
   {...}
}
Figure 7 Local Variable in Anonymous Method Code (Close Figure 7)
class SomeClass
{
   string m_Space = " ";
   delegate void SomeDelegate(string str);
   public void InvokeMethod()
   {
      string msg = "Hello";
      SomeDelegate del = delegate(string name)
                         {
                            MessageBox.Show(msg + m_Space + name);
                         };
      del("Juval");
   }
}

Because the += operator merely concatenates the internal invocation list of one delegate to another, you can use the += to add an anonymous method. Note that with anonymous event handling, you cannot remove the event handling method using the -= operator unless the anonymous method was added as a handler by first storing it to a delegate and then registering that delegate with the event. In that case, the -= operator can be used with the same delegate to unregister the anonymous method as a handler.

Anonymous Method Implementation

The code the compiler generates for anonymous methods largely depends on which type of parameters or variables the anonymous methods uses. For example, does the anonymous method use the local variables of its containing method (called outer variables), or does it use class member variables and method arguments? In each case, the compiler will generate different MSIL. If the anonymous method does not use outer variables (that is, it only uses its own arguments or the class members) then the compiler adds a private method to the class, giving the method a unique name. The name of that method will have the following format:

<return type> __AnonymousMethod$<random unique number>(<params>)
As with other compiler-generated members, this is subject to change and most likely will before the final release. The method signature will be that of the delegate to which it is assigned.

The compiler simply converts the anonymous method definition and assignment into a normal instantiation of the inferred delegate type, wrapping the machine-generated private method:

SomeDelegate del = new SomeDelegate(__AnonymousMethod$00000000);
Interestingly enough, the machine-generated private method does not show up in IntelliSense®, nor can you call it explicitly because the dollar sign in its name is an invalid token for a C# method (but a valid MSIL token).

The more challenging scenario is when the anonymous method uses outer variables. In that case, the compiler adds a private nested class with a unique name in the format of:

__LocalsDisplayClass$<random unique number>
The nested class has a back reference to the containing class called <this>, which is a valid MSIL member variable name. The nested class contains public member variables corresponding to every outer variable that the anonymous method uses. The compiler adds to the nested class definition a public method with a unique name, in the format of:
<return type> __AnonymousMethod$<random unique number>(<params>)

The method signature will be that of the delegate to which it is assigned. The compiler replaces the anonymous method definition with code that creates an instance of the nested class and makes the necessary assignments from the outer variables to that instance's member variables. Finally, the compiler creates a new delegate object, wrapping the public method of the nested class instance, and calls that delegate to invoke the method. Figure 9 shows in C# pseudocode the compiler-generated code for the anonymous method definition in Figure 7.

Figure 9 Anonymous Method Code with Outer Variables (Close Figure 9)
class SomeClass
{
   string m_Space = " ";
   delegate void SomeDelegate(string str);

   private sealed class __LocalsDisplayClass$00000001
   {
      public SomeClass <this>; //Back pointer, name is valid in MSIL
      public string msg;       //Outer variable
      public void __AnonymousMethod$00000000(string name)
      {
         MessageBox.Show(msg + <this>.m_Space + name);
      }
   }
   public void InvokeMethod()
   {
      string msg = "Hello";
      __LocalsDisplayClass$00000001 locals;
      locals = new __LocalsDisplayClass$00000001();
      locals.<this> = this;
      locals.msg = msg;
      SomeDelegate del = new
          SomeDelegate(locals.__AnonymousMethod$00000000);
      del("Juval");
}

Generic Anonymous Methods

An anonymous method can use generic parameter types, just like any other method. It can use generic types defined at the scope of the class, for example:

class SomeClass<T>
{
   delegate void SomeDelegate(T t);
   public void InvokeMethod(T t)
   {
      SomeDelegate del = delegate(T item){...}
      del(t);
   }
}

Because delegates can define generic parameters, an anonymous method can use generic types defined at the delegate level. You can specify the type to use in the method signature, in which case it has to match the specific type of the delegate to which it is assigned:

class SomeClass
{
   delegate void SomeDelegate<T>(T t);
   public void InvokeMethod()
   {
      SomeDelegate<int> del = delegate(int number)
                              {
                                 MessageBox.Show(number.ToString());
                              };
      del(3);
   }
}

Anonymous Method Example

Although at first glance the use of anonymous methods may seem like an alien programming technique, I have found it quite useful because it replaces the need for creating a simple method in cases where only a delegate will suffice. Figure 10 shows a real-life example of the usefulness of anonymous methods—the SafeLabel Windows Forms control.

Figure 10 The SafeLabel Control (Close Figure 10)
public class SafeLabel : Label
{
   delegate void SetString(string text);
   delegate string GetString();
   override public string Text
   {
      set
      {
         if(InvokeRequired)
         {
            SetString setTextDel = delegate(string text)
                                           {base.Text = text;};
            Invoke(setTextDel,new object[]{value});
         }
         else
            base.Text = value;
      }
      get
      {
         if(InvokeRequired)
         {
            GetString getTextDel = delegate(){return base.Text;};
            return (string)Invoke(getTextDel,null);
         }
         else
            return base.Text;
      }
   }
}

Windows Forms relies on the underlying Win32® messages. Therefore, it inherits the classic Windows programming requirement that only the thread that created the window can process its messages. Calls on the wrong thread will always trigger an exception under Windows Forms in the .NET Framework 2.0. As a result, when calling a form or a control on a different thread, you must marshal that call to the correct owning thread. Windows Forms has built-in support for solving this predicament by having the Control base class implement the interface ISynchronizeInvoke, defined like the following:

public interface ISynchronizeInvoke
{
   bool InvokeRequired {get;}
   IAsyncResult BeginInvoke(Delegate method,object[] args);
   object EndInvoke(IAsyncResult result);
   object Invoke(Delegate method,object[] args);
}

The Invoke method accepts a delegate targeting a method on the owning thread, and it will marshal the call to that thread from the calling thread. Because you may not always know whether you are actually executing on the correct thread, the InvokeRequired property lets you query to see if calling Invoke is required. The problem is that using ISynchronizeInvoke complicates the programming model significantly, and as a result it is often better to encapsulate the interaction with the ISynchronizeInvoke interface in controls and forms that will automatically use ISynchronizeInvoke as required.

For example, instead of a Label control that exposes a Text property, you can define the SafeLabel control which derives from Label, as shown in Figure 10. SafeLabel overrides its base class Text property. In its get and set, it checks whether Invoke is required. If so, it needs to use a delegate to access the property. That implementation simply calls the base class implementation of the property, but on the correct thread. Because SafeLabel only defines these methods so that they can be called through a delegate, they are good candidates for anonymous methods. SafeLabel passes the delegate, wrapping the anonymous methods to the Invoke method as its safe implementation of the Text property.

Delegate Inference

The C# compiler's ability to infer from an anonymous method assignment which delegate type to instantiate is an important capability. In fact, it enables yet another C# 2.0 feature called delegate inference. Delegate inference allows you to make a direct assignment of a method name to a delegate variable, without wrapping it first with a delegate object. For example, take a look at the following C# 1.1 code:

class SomeClass
{
   delegate void SomeDelegate();
   public void InvokeMethod()
   {
      SomeDelegate del = new SomeDelegate(SomeMethod);
      del();
   }
   void SomeMethod()
   {...}
}
Instead of the previous snippet, you can now write:
class SomeClass
{
   delegate void SomeDelegate();
   public void InvokeMethod()
   {
      SomeDelegate del = SomeMethod;
      del();
   }
   void SomeMethod()
   {...}
}

When you assign a method name to a delegate, the compiler first infers the delegate's type. Then the compiler verifies that there is a method by that name and that its signature matches that of the inferred delegate type. Finally, the compiler creates a new object of the inferred delegate type, wrapping the method and assigning it to the delegate. The compiler can only infer the delegate type if that type is a specific delegate type—that is, anything other than the abstract type Delegate. Delegate inference is a very useful feature indeed, resulting in concise, elegant code.

I believe that as a matter of routine in C# 2.0, you will use delegate inference rather than the old method of delegate instantiation. For example, here is how you can launch a new thread without explicitly creating a ThreadStart delegate:

public class MyClass
{
   void ThreadMethod()
   {...}
   public void LauchThread()
   {
      Thread workerThread = new Thread(ThreadMethod);
      workerThread.Start();
   }
}

You can use a double stroke of delegate inference when launching an asynchronous call and providing a completion callback method, as shown in Figure 11. There you first assign the method name to invoke asynchronously into a matching delegate. Then call BeginInvoke, providing the completion callback method name instead of a delegate of type AsyncCallback.

Figure 11 Using Delegate Inference (Close Figure 11)
class SomeClass
{
   delegate void SomeDelegate(string str);
   public void InvokeMethodAsync()
   {
      SomeDelegate del = SomeMethod;
      del.BeginInvoke("Hello",OnAsyncCallBack,null);
   }
   void SomeMethod(string str)
   {
      MessageBox.Show(str);
   }
   void OnAsyncCallBack(IAsyncResult asyncResult)
   {...}
}

Property and Index Visibility

C# 2.0 allows you to specify different visibility for the get and set accessors of a property or an indexer. For example, it is quite common to want to expose the get as public, but the set as protected. To do so, add the protected visibility qualifier to the set keyword. Similarly, you can define the set method of an indexer as protected, (see Figure 12).

Figure 12 Public Get and Protected Set (Close Figure 12)
public class MyClass
{
   public string this[int index]
   {
      get
      {
         return m_Names[index];
      }
      protected set
      {
         m_Names[index] = value;
      }
   }
   string[] m_Names;
   //Rest of the class
}

There are a few stipulations when using property visibility. First, the visibility qualifier you apply on the set or the get can only be a stringent subset of the visibility of the property itself. In other words, if the property is public, then you can specify internal, protected, protected internal, or private. If the property visibility is protected, you cannot make the get or the set public. In addition, you can only specify visibility for the get or the set, but not both.

Static Classes

It is quite common to have classes with only static methods or members (static classes). In such cases there is no point in instantiating objects of these classes. For example, the Monitor class or class factories such as the Activator class in the .NET Framework 1.1 are static classes. Under C# 1.1, if you want to prevent developers from instantiating objects of your class you can provide only a private default constructor. Without any public constructors, no one can instantiate objects of your class:

public class MyClassFactory
{
   private MyClassFactory()
   {}
   static public object CreateObject()
   {...}
}

However, it is up to you to enforce the fact that only static members are defined on the class because the C# compiler will still allow you to add instance members, although they could never be used. C# 2.0 adds support for static classes by allowing you to qualify your class as static:

public static class MyClassFactory
{
   static public T CreateObject<T>()
   {...}
}

The C# 2.0 compiler will not allow you to add a non-static member to a static class, and will not allow you to create instances of the static class as if it were an abstract class. In addition, you cannot derive from a static class. It's as if the compiler adds both abstract and sealed to the static class definition. Note that you can define static classes but not static structures, and you can add a static constructor.

Global Namespace Qualifier

It is possible to have a nested namespace with a name that matches some other global namespace. In such cases, the C# 1.1 compiler will have trouble resolving the namespace reference. Consider the following example:

namespace MyApp
{
   namespace System
   {
      class MyClass
      {
         public void MyMethod()
         {
            System.Diagnostics.Trace.WriteLine("It Works!");
         }
      }
   }
}

In C# 1.1, the call to the Trace class would produce a compilation error (without the global namespace qualifier ::). The reason the error would occur is that when the compiler tries to resolve the reference to the System namespace, it uses the immediate containing scope, which contains the System namespace but not the Diagnostics namespace. C# 2.0 allows you to use the global namespace qualifier :: to indicate to the compiler that it should start its search at the global scope. You can apply the :: qualifier to both namespaces and types, as shown in Figure 13.

Figure 13 Using the Global Namespace Qualifier (Close Figure 13)
namespace MyApp
{
   class MyClass
   {
      public void MyMethod()
      {
         ::MyClass obj = new ::MyClass();
         obj.MyMethod(); // Traces "Hello" instead of recursion
      }
   }
}
public class MyClass
{
   public void MyMethod()
   {
      Trace.WriteLine("Hello");
   }
}

Inline Warning

C# 1.1 allows you to disable specific compiler warnings using project settings or by issuing command-line arguments to the compiler. The problem here is that this is a global suppression, and as such suppresses warnings that you still want. C# 2.0 allows you to explicitly suppress and restore compiler warnings using the #pragma warning directive:

// Disable 'field never used' warning
#pragma warning disable 169
public class MyClass
{
   int m_Number;
}
#pragma warning restore 169

Disabling warnings should be generally discouraged in production code. It is intended only for analysis when trying to isolate a problem, or when you lay out the code and would like to get the initial code structure in place without having to polish it up first. In all other cases, avoid suppressing compiler warnings. Note that you cannot programmatically override the project settings, meaning you cannot use the pragma warning directive to restore a warning that is suppressed globally.

Conclusion

The new features in C# 2.0 presented in this article are dedicated solutions, designed to address specific problems while simplifying the overall programming model. If you care about productivity and quality, then you want to have the compiler generate as much of the implementation as possible, reduce repetitive programming tasks, and make the resulting code concise and readable. The new features give you just that, and I believe they are an indication that C# is coming of age, establishing itself as a great tool for the developer who is an expert in .NET.

Juval Lowy is a software architect providing consultation and training on .NET design and migration. He is also the Microsoft Regional Director for the Silicon Valley. His latest book is Programming .NET Components (O'Reilly, 2003). Contact Juval at http://www.idesign.net.

© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


C++: Write Faster Code with the Modern Language Features of Visual C++ 2005

C++

Write Faster Code with the Modern Language Features of Visual C++ 2005

Stephen Toub
This article was based on a pre-release version of Microsoft Visual Studio 2005, formerly code-named "Whidbey." All information contained herein is subject to change.
This article discusses:
  • .NET C++/CLI syntax
  • Interop technologies
  • Profile Guided Optimization
  • MSIL optimization
  • OpenMP support
  • Enhanced buffer security checks
This article uses the following technologies:
Visual C++ .NET 2003, Visual C++ 2005
Contents
C++/CLI—The New Syntax
Interop Options
It Just Works
Optimizations
Security
Conclusion

T
he introduction of the Visual Studio® .NET 2003 C++ compiler was a mouthwatering experience for enthusiasts of the C++ language. With 98 percent conformance to the ISO C++ standard, Visual C++® .NET 2003 was truer to these standards than any previous version and incorporated language support for features such as partial template specialization. It also included enhanced buffer security checks and improved compiler diagnostics. C++ developers joined the ranks of developers using C# and Visual Basic® .NET who are able to use the drag and drop forms designer to build robust Windows® Forms applications. The compiler also included optimizations targeting the Intel Pentium 4 and AMD Athlon processors.

If you were excited about Visual C++ .NET 2003, you'll be in a frenzy over the next version, Visual C++ 2005. Visual C++ 2005 has a new syntax for development in .NET that is both elegant and powerful. It has new optimization technology that has improved the speed of Microsoft products up to 30 percent. It has new compilation modes that ensure Common Language Infrastructure (CLI) compliance and verifiability for the Microsoft® .NET Framework, and it has new models for interop that provide a seamless merging of the native and managed worlds as well as complete control over when these boundaries are crossed. The compiler includes an enhanced version of the buffer security check option present in the previous two versions, and it includes new security-focused versions of libraries in prevalent use by C++ applications. It has support for the OpenMP standard as well as for 64-bit platforms, including the Intel Itanium and AMD64 chips. It fixes the mixed DLL loading issue and provides automatic runtime elimination of the Double P/Invoke performance problem. The list of enhancements and improvements goes on and on. As one of the architects on the C++ team told me, "C++ is where it's at, man!"

C++/CLI—The New Syntax

How many of us found working with the Managed Extensions syntax of the previous two versions of C++ cumbersome and fraught with error? How many of us felt that Visual C++ wasn't being treated as a first-class .NET-based language? Apparently, most of us did (including the development group itself, if you read their blogs). The Visual C++ team heard our cries, and starting with Visual C++ 2005, the Managed Extensions for C++ syntax that was introduced with Visual Studio .NET 2002 is going the way of the dinosaurs as a revised language definition is introduced that brings with it an attractive new syntax.

The design team had a few major goals for this release in regard to the language design. First (and probably most important to those of us who think code is art), they wanted to make sure that writing C++ felt natural and that it provided an elegant syntax with pure extensions to the ISO C++ standard. They wanted to make it easy to write verifiable code in C++ in order to enable partial-trust scenarios such as ClickOnce deployment, Form design support and managed-code hosting in SQL Server™ 2005. They wanted to leave no room for a language "lower" than C++. They wanted to bring the complete power of .NET to C++, and at the same time bring the power of C++ to .NET. They've succeeded brilliantly in all aspects.

The specification for the new extensions is called C++/CLI and is now being standardized. To get a taste of the new language extensions, see the candidate base document that was posted online on November 21, 2003, and which is available for download at C++/CLI Language Specification.

Most noticeably for anyone reading code in the new syntax, the common double-underscore keywords prevalent in Managed Extensions for defining garbage-collected classes, properties, and so on, are a thing of the past. While a few of these keywords remain and a few more are being introduced, they are infrequently used and won't muddy up the readability of the code. These double-underscore keywords are being replaced with two new types of keywords: context-sensitive and spaced. Context-sensitive keywords are only keywords when used in certain contexts, and spaced keywords are only keywords when used in combination with other keywords. For example, the __property keyword from Managed Extensions is replaced with the property keyword. (Not only that, but the entire syntax for defining a property and its accessors has been dramatically refined, making the declaration look very similar to what you might write in C#. See Figure 1 for an example.) This doesn't prevent you from using "property" as the name of a variable in your code. A token parsed as "property" is only treated as a keyword when in the context of declaring a property on a type.

Figure 1 Syntax Comparison (Close Figure 1)

Managed Extensions Syntax
public __gc __sealed class Student
{
    private:
        double m_grade;
        String* m_name;
    public:
        __property double get_Grade() { return m_grade; }
        __property void set_Grade(double newGrade) { m_grade = newGrade; }

        __property String* get_Name() { return m_name; }
        __property void set_Name(String* newName) { m_name = newName; }
}
C++/CLI Syntax
public ref class Student sealed
{
    private:
        double m_grade;
    public:
        // standard property syntax
        property double Grade
        {
            double get() { return m_grade; }
            void set(double newGrade) { m_grade = newGrade; }
        }

        // trivial property
        // compiler can generate accessors and backing store
        property String^ Name;
}

Types in the new syntax are declared with the form "adjective class", where the adjective describes what type of class you're creating, as shown in the following:

class            N  { /*...*/ };  // native type
ref class        R  { /*...*/ };  // CLR reference type
value class      V  { /*...*/ };  // CLR value type
interface class  I  { /*...*/ };  // CLR interface type
enum class       E  { /*...*/ };  // CLR enumeration type
In previous versions of the language, the way a type was declared controlled where and how it lived. Only native classes or structs and managed value types could be created on the stack. Managed ref classes always lived on the managed heap. In Visual C++ 2005, all types, be they native or managed, can be created on the stack, with stack-based deterministic cleanup semantics.

To instantiate an object of type T to live on the native heap, use "new T". This returns a pointer to the location of the object on the native heap (a concept referred to as a __nogc pointer in Visual Studio .NET 2002 and Visual Studio .NET 2003). To instantiate an object of type T that should live on the managed heap, Visual C++ 2005 introduces the gcnew keyword to be used in the same manner as the new keyword. Calling "gcnew T" returns a handle to a whole object on the managed heap. A handle is a new construct introduced in Visual C++ 2005 that resembles the __gc pointer of Managed Extensions. To instantiate a type T on the stack, the standard "T t;" declaration suffices.

To be fair, I'm taking some liberties in how I define instantiation. Managed reference classes always live on the managed heap, and native types always live on the stack or on the native heap. When a managed reference type is declared to live on the stack, the compiler is in fact instantiating it on the managed heap, as shown in Figure 2.

Figure 2 Managed Ref Type on the Stack
Figure 2 Managed Ref Type on the Stack

This begs some questions. What happens when my instance on the stack goes out of scope? How is it cleaned up? Many C# developers have complained about the C# language's lack of deterministic cleanup. The C# language provides the using keyword to make it easy to dispose of an IDisposable object, but this requires extra code and is clunky compared to the destructor pattern familiar to C++ developers. In C#, safe cleanup is off by default and requires explicit coding. For example, consider the first C# code snippet in Figure 3. The StreamReader object is declared on the managed heap. When this method finishes execution, no more references to the StreamReader instance remain. However, the object won't be finalized until the garbage collector runs. Until that time, the file will not be closed and the application will continue to hold onto its open file handle. To add deterministic cleanup, you must use the IDisposable interface implemented by classes that use unmanaged resources.

Figure 3 Deterministic Cleanup (Close Figure 3)

Implemenation Code
C# without deterministic cleanup
string ReadFirstLineFromFile(string path)
{
  StreamReader reader = new StreamReader(path);
  return reader.ReadLine();
})
C# with deterministic cleanup
string ReadFirstLineFromFile(string path)
{
  using (StreamReader reader = new StreamReader(path))
  {
    return reader.ReadLine();
  }
}
Visual Basic .NET with deterministic cleanup
Function ReadFirstLineFromFile( _
    ByVal path As String) As String
  Dim reader As StreamReader
  Try
    reader = New StreamReader(path)
    ReadFirstLineFromFile = reader.ReadLine()
  Finally
    If Not reader Is Nothing Then _
      CType(reader, IDisposable).Dispose()
  End Try
End Function
C++ with deterministic cleanup
String^ ReadFirstLineFromFile(String^ path)
{
  StreamReader reader(path);
  return reader.ReadLine();
}

The second code sample in Figure 3 shows what the new code in C# looks like. Not horrible, and still readable, but as you start introducing more objects that require cleanup, your code becomes harder and harder to read. Also, any objects you forget to dispose will put additional pressure on the finalizer thread when the garbage collector eventually does run. In the meantime, you could possibly lock up valuable resources. This becomes even uglier when viewing the equivalent implementation in Visual Basic .NET, also shown in Figure 3 (although Visual Basic 2005 gains a Using statement similar to that in C#).

Visual C++ 2005 now provides the ability to have destructors and/or finalizers on any type, regardless of whether it is managed or native. In the case where the type is managed, the compiler maps the destructor to the IDisposable::Dispose method. This means that you can write the same method in C++ using the fourth snippet in Figure 3, where reader's destructor/Dispose method will automatically be called just as if you were using the "using" statement in C#. When a type is created on the stack, its destructor is invoked when it goes out of scope.

One of the biggest problems with Managed Extensions was working with pointers. Pointers were used for a variety of tasks and in a variety of situations, and as such were very difficult to understand. Deciphering which kind of pointer you were dealing with in any given piece of code required intelligence somewhere on the level of genius. That complexity is removed in this next version. In Visual C++ 2005, pointers are plain old C++ pointers. They point to an object that is stable, and you can perform arithmetic with them. Pointers refer to objects whose lifetime must be explicitly managed by the developer. When dealing with a pointer, it is not the runtime's responsibility to clean up after it.

Now let's see how the designers of Visual C++ 2005 have pulled this off. Unlike the new operator which in both Visual Studio .NET 2003 and Visual Studio 2005 returns a pointer, the gcnew operator returns a "handle," a new construct represented in the syntax by a caret (^). Handles refer to whole objects on the managed heap. That is, they cannot be used to point to the interior of types, and the compiler has a number of restrictions on their usage that enforce this behavior and that help the developer to use them correctly and safely. Handles do not allow pointer arithmetic, nor can they be cast to a void pointer or to any integral type. The asterisk and arrow operators are still used to dereference it, however.

This doesn't mean you can no longer acquire a pointer to something on the garbage collected heap. Similar to a combination of the & operator and the fixed keyword in C#, the pin_ptr abstraction in Visual C++ 2005 allows you to retrieve a pinning pointer to an object on the managed heap. As long as this pointer is in existence, the object on the managed heap will be pinned, preventing the garbage collector from moving it around during a collection. Visual C++ 2005 also introduces the tracking reference operator, notated with a percent sign (%). When introduced to the native & reference operator in C++, most developers learn to understand it as a pointer to an object that is automatically dereferenced by the compiler upon usage. In most respects, % is to ^ as & is to *.

In the managed world, having native references to managed objects is just as dangerous as having native pointers to managed objects. The fundamental reasoning behind pointers and references is that the referenced objects don't move around. The tracking reference is similar to the native reference except that it references objects on the managed heap and "tracks" them even if they are moved by the garbage collector. The percent sign operator is also used to "take the address of" managed objects, so just as the & operator when applied to a native type returns a pointer to that object, the % operator when applied to a managed reference type returns a handle to that object.

In general, C++ developers are at peace knowing that standards control their language. For this reason, to promote adoption by third parties, and to ensure the stability of the language going forward, this new syntax has been pulled together into a proposed standard known as C++/CLI. In October of 2003, ECMA voted to create a special task force, called TG5, dedicated to the analysis and adoption of this standard, just as WG21 serves as the governing body for ISO C++. In fact, key players in WG21 are also serving in TG5. The plan is to have this standardized by the end of 2004.

Interop Options

Visual C++ 7.1 provided the best interop functionality of any of the .NET Framework-based languages in Visual Studio .NET 2003. It has the necessary power to implement real-world interop scenarios, as exemplified by the port of Quake II to the .NET Framework, available at http://www.vertigosoftware.com/Quake2.htm. Visual C++ 2005 extends this functionality further.

There are four main ways to do interop in .NET between the managed and native worlds. COM interop can be accomplished using Runtime Callable Wrappers (RCW) and COM Callable Wrappers (CCW). The common language runtime (CLR) is responsible for type marshaling (except in the rare scenarios where a custom marshaler is used) and the cost of these calls can be expensive. You need to be careful that the interfaces are not too chatty; otherwise a large performance penalty could result. You also need to ensure that the wrappers are kept up to date with the underlying component. That said, COM interop is very useful for simple interop scenarios where you are attempting to bring in a large amount of native COM code.

The second option for interop is to use P/Invoke. This is accomplished using the DllImport attribute, specifying the attribute on the method declaration for the function you want to import. Marshaling is handled according to how it has been specified in the declaration. However, DllImport is only useful if you have code that exposes the required functions through a DLL export.

When you need to call managed code from native code, CLR hosting is an option. In such a scenario, the native application has to drive all of the execution: setting up the host, binding to the runtime, starting the host, retrieving the appropriate AppDomain, setting up the invocation context, locating the desired assembly and class, and invoking the operation on the desired class. This is definitely one of the most robust solutions in terms of control over what and when things happen, but it is also incredibly tedious and requires a lot of custom code.

The fourth option, and quite possibly the easiest and the most performant, is to use the interop capabilities in C++. By throwing the /clr switch, the compiler generates MSIL instead of native machine code. The only code that is generated as native machine code is code that just can't be compiled to MSIL, including functions with inline asm blocks and operations that use CPU-specific intrinsics such as Streaming SIMD Extensions (SSE). The /clr switch is how the Quake II port to .NET was accomplished. The Vertigo software team spent a day porting the original C code for the game to code that successfully compiled as C++ and then threw the /clr switch. In no time they were up and running on the .NET Framework. Without adding any additional binaries and simply by including the appropriate header files, managed C++ and native C++ can call each other without any additional work on the part of the developer. The compiler handles the creation of the appropriate thunks to go back and forth between the two worlds.

This resulted in some issues for C++ developers. One of these issues was the now infamous mixed DLL loading problem that affected users of Visual Studio .NET 2002 and Visual Studio .NET 2003. If you're running native code inside the loader lock and you reference a managed type in an assembly that hasn't yet been loaded, the CLR will kindly load the assembly for you. It does this by calling LoadLibrary. Of course, LoadLibrary attempts to acquire the loader lock, at which point you're deadlocked. Developers and product managers alike will be happy to hear that this problem has been solved in the upcoming version.

The /clr switch was a huge boon for C++ developers, but it did have some downsides. As mentioned earlier, the images produced with the /clr switch contained both native and managed code, which sometimes caused problems. First, these mixed images are not CLI-compliant (meaning that, for example, they will not run on Rotor). They have native entry points, and there are often significant transition penalties as you continually cross the managed boundaries. But most importantly, the existence of native entry points can wreak havoc on tools that consume the assemblies, including reflection. In order for reflection to inspect an image, it must first load the assembly and execute it. Only once all initialization has taken place is reflection able to inspect the metadata. Unfortunately, reflection cannot correctly load a managed assembly that contains native entry points.

Moreover, Visual Studio .NET 2003 rarely produced verifiable code, and even when it did, the effort it took in nontrivial cases was often significant. While MSIL has first-class support for unverifiable instructions (you can do pointer arithmetic, perform indirect loads, and access the native heap), verifiable code gives you the ability to engage in partial trust scenarios which, in turn, enables the rich set of features available in Visual Studio 2005. ClickOnce deployment relies on partial trust, as does managed code hosting in SQL Server 2005. One of the major goals the C++ development team had for Visual C++ 2005 was to enable the compiler to aid the developer in the production of non-mixed and verifiable images. They've done so with the introduction of two new compiler switches: /clr:pure and /clr:safe. But before I delve into these new switches, I need to examine how C++ interop functions.

It Just Works

In Visual Studio .NET 2003, the C++ interop technology was known as IJW, or "It Just Works." For the upcoming version, this has been renamed to the more descriptive "Interop Technologies." So how does it "just work?" For every native method used by the application, the compiler creates both a managed and an unmanaged entry point. One of them is the actual method implementation, while the other is a forwarding thunk that creates the appropriate transition and performs any necessary marshaling. The managed entry point is almost always the actual method implementation, except when the code for that method can't be expressed in MSIL or when a developer uses the "#pragma unmanaged" compiler directive to force the implementation of the entry point to be native machine code.

When an IJW forwarding thunk is used (for example, when the native entry point is the forwarding thunk), the compiler provides the implementation for the thunk and the call into the actual implementation through an offset or an Import Address Table (IAT) jump. A reasonable amount of time for an IJW thunk is somewhere on the order of 50 to 300 cycles, though contrived test cases can be developed to get that number down to as little as 10. When the forwarding thunk is MSIL, a managed P/Invoke is used. P/Invokes consist only of a declaration and no actual method implementation; the CLR supplies the functionality for the thunk at run time. These forwarding thunks are often slightly slower than their native equivalents.

As mentioned, a consequence of IJW is two entry points for each function, one managed and one unmanaged. But certain constructs require call sites to these entry points to be populated at compile time (function pointers and vtables, for example). Which entry point should the compiler choose, given that it can't know at compile time the managed state of the call site at run time? In Visual Studio .NET 2003, the unmanaged entry point was always selected by the compiler. This, of course, creates a problem if the caller is indeed managed, an issue known as the Double P/Invoke problem, illustrated in Figure 4. In such a scenario, the managed call transitions to the unmanaged thunk only to transition again right back to managed code, operations resulting in a significant amount of unnecessary overhead.

Figure 4 Double P/Invoke Problem
Figure 4 Double P/Invoke Problem

Visual C++ 2005 provides a few solutions. The first is the __clrcall keyword which lets you specify whether to emit an unmanaged entry point on a method-by-method basis. Augmenting a function declaration with this keyword will prevent the generation of an unmanaged entry point (a downside to this is that the function can no longer be called directly from native code). The __clrcall keyword can also be placed on a function pointer, causing it to be populated with a managed entry point should the compiler have a choice. The second solution provided by Visual C++ 2005 is the automatic elimination of the Double P/Invoke using runtime checks and a cookie that will help the runtime determine whether the unmanaged thunk can be skipped, forwarding the call directly to the managed entry point. This feature, however, may not make it into the final bits.

The third solution is pure MSIL. The new /clr:pure compiler flag instructs the compiler to generate a pure managed image with no native constructs. Not only does this produce CLI-compliant assemblies that enable partial trust scenarios, it also solves the Double P/Invoke problem by preventing the generation of the unmanaged thunk. As a result, there's only one entry point for every function (the managed entry point), so vtables and function pointers are never populated with an unmanaged entry point.

Just because code is CLI-compliant, however, doesn't mean it's verifiable, which is an important goal for enabling low trust scenarios such as when code is loaded from a file share. To that end, Microsoft has introduced an even stricter compiler flag called /clr:safe. This is the holy grail of verifiability for C++ developers. Throwing this switch causes the compiler to ensure that the generated assembly is completely verifiable; any unverifiable constructs will result in a compile-time error. For example, attempting to compile with an integer pointer as a variable will result in the error "int* = this type is not verifiable" referencing the specific line containing the invalid construct. There are a some scenarios where it's appropriate to go to this extreme. For example, all managed C++ code to be run as a stored procedure in SQL Server 2005 should be compiled with this flag.

Figure 5 Compilation Modes
Figure 5 Compilation Modes

Figure 5 diagrams the managed and unmanaged worlds of data and code and shows which of them are targeted with the different compiler flags. Not including any of the /clr flags will result in the generation of a completely native image. Using /clr will result in a mixed image that can contain both managed and unmanaged code and data. Pure MSIL, generated by using the /clr:pure flag, will not contain any unmanaged code, though it is still not guaranteed to be verifiable and can contain native types. Safe MSIL is the ultimate in verifiability, targeting only the .NET Framework. In short, these two new compilation modes will enable a wide variety of scenarios that either weren't possible before or that were very difficult to accomplish.

Optimizations

All good software developers want to ensure that their software performs. Compiler writers are a special kind of developer; not only must their code perform, but the code that their code generates must be as efficient as is possible. For this reason, a good optimizing back end is essential to the success of any compiler. Here, Visual C++ 2005 strikes gold.

Visual Studio .NET 2002 and Visual Studio .NET 2003 saw some terrific optimizations added to the C++ compiler as lots of work was done to improve the performance of native code. Support was added for targeting the Intel Pentium IV chip as well as the SSE and SSE2 architectures. Most notably, Whole Program Optimization (WPO) was added, allowing the linker to optimize the entire program when presented with the .obj files for each of the compiled .cpp files. These object files were different from normal object files because rather than containing native machine code, they contained an intermediate language used to communicate between the front and back ends of the compiler. The linker was then able to optimize all of these files as a large unit, providing for more inlining opportunities, better stack alignment options, and the possibility of using custom calling conventions in various scenarios, among other optimizations. Visual C++ 2005 improves upon WPO with a new feature referred to as Bottom-Up, Top-Down analysis. But the big improvement comes in the form of Profile Guided Optimization (POGO), a brand new feature in the compiler that is going to make some performance waves.

Static analysis of source code leaves many open questions for a compiler. Given an if statement that compares two variables, how often will the first be greater than the second? Which of the cases in a switch statement is targeted the most? Which functions are used the most, and which code is often the coldest? If the compiler could know at compile time how the code was going to be used at run time, it could optimize for common scenarios. That's exactly what the Visual C++ 2005 compiler can do.

Figure 6 Profile Guided Optimization
Figure 6 Profile Guided Optimization

The process for compiling for POGO is shown in Figure 6. The first steps involve compiling the code and linking it into an instrumented build filled with a set of profiling probes. As with WPO, the object files generated by the compiler and fed to the linker are comprised of an intermediate language rather than native machine code. These probes come in two flavors: value probes and count probes. Value probes are used to construct a histogram of the values that variables hold, while count probes are used to keep track of the number of times you traverse a particular path through the application. When the application is run and put through normal use, data is gathered from all of these probes and written to a profile database. Along with the original .obj files fed to the linker, this profile data can then be fed back into the linker. The linker is able to analyze the profile data, determine additional optimizations that should be applied, and spit out a new non-instrumented build of the application. It is this compiled version and not the instrumented version that should be distributed to customers.

Profile Guided Optimization enables a wide variety of optimizations. Based on the count probes, inlining decisions can be made at each function call site. Value probes enable switch and if-else constructs to be reordered so as to pull out the most frequent values and avoid the extra unnecessary checks before hitting the frequent case. Sections of code can be reordered so that the most frequent paths fall through rather than forcing unnecessary jumps around the code. This avoids costly Translation Lookaside Buffer (TLB) thrashing and paging.

Cold code can be placed in special sections of the module, which also helps to avoid these issues. Virtual call speculation can be performed so that virtual call sites that often result in a call made on a specific type can avoid the vtable lookup in the common cases. Partial inlining can be performed, whereby only the hot sections of functions can be inlined and the decision to do so made on a call site by call site basis. Additionally, certain sections of code can be compiled with certain optimization goals in mind, while other sections are compiled with different goals. For example, hot and/or small functions can be compiled to maximize speed (/O2), while colder and/or bigger functions can be compiled to minimize space (/O1).

If you understand your real-world scenarios and can put your apps through common use cases while instrumented, the performance gains can be tremendous. Recently, SQL Server was recompiled using POGO and obtained a whopping performance boost of up to 30 percent in many common scenarios. Moving forward, you can bet that Microsoft will start compiling many of its products with this technology. Note that it is very important not to attempt complete code coverage when profiling your instrumented build. The whole point of POGO is to determine how the common use cases can be optimized. By attempting complete code coverage, you'll be severely penalizing yourself.

Visual C++ 2005 also adds support for OpenMP, an open specification for building multithreaded programs. It consists of a set of pragmas used to instruct the compiler as to which sections of your code can be parallelized. Code with large loops lacking dependencies on previous iterations is most suited for OpenMP. Take a look at the following simple copy function that adds the values from arrays a and b and stores the results in c:

void copy(int a[], int b[], int c[], int length)
{
    #pragma omp parallel
    for(int i=0; i<length; i++)
    {
        c[i] = a[i] + b[i];
    }
}

For a multiprocessor machine, the compiler will generate multiple threads to execute the iterations of this loop, and each thread will perform a subset of the copy operations. Note that the compiler cannot verify whether the loop has dependencies and thus will not stop you from using these pragmas in improper scenarios. If there are dependencies, you'll most likely get incorrect results according to what you expect from your code, though they will be correct with regards to the specification.

While the biggest gains with OpenMP often come from parallelizing loops as in the example just shown, straight-line code can also exhibit improvement. The "#pragma omp section" directive can be used to demark non-dependent sections in a piece of code, allowing the developer to specify regions that can be run in parallel. The compiler can then generate multiple threads to execute these sections on different processors.

In an important change for developers using .NET, the Visual C++ 2005 optimizer performs most of the same optimizations when targeting MSIL as it does when targeting the native platform, though it does so with different tuning. While the just-in-time (JIT) compiler today analyzes for optimizations at run time, allowing the C++ compiler to optimize during the initial compilation can still provide significant performance benefits (the C++ compiler has much more time to perform its analysis than does the JIT). The Visual C++ 2005 compiler optimizes managed types for the first time, performing loop optimizations, expression optimizations, and inlining. There are places, though, where the compiler can't optimize .NET-based code. For example, it has problems with strength reduction due to the unverifiability of pointer arithmetic, and certain code can't be inlined due to the strict type and member accessibility requirements of the CLR, though it does do significant analysis for legal inlining opportunities. In addition, optimizing MSIL introduces the need to make trade-offs concerning what is presented to the JIT compiler. For example, you wouldn't want to unroll a loop and expose a plethora of variables to the JIT compiler, whereupon it would have to perform register allocation (an NP-complete problem). The Visual C++ team is working through these issues and will have a very well-tuned optimizing solution by the time the system is released.

Security

The Trustworthy Computing initiative, announced by Bill Gates in 2002, has had a noticeable impact on all products developed by Microsoft. Developers for the Windows operating system spent months in security training and code reviews, which resulted in making Windows Server™ 2003 the most secure operating system the company has ever released. Microsoft Office 2003 includes many security features, such as Information Rights Management (IRM), better macro security, and HTML download blocking in Outlook®. The compiler teams are also taking strides towards making their compilers and the code they generate more secure.

Visual Studio .NET 2002 introduced a buffer security check /GS compiler option. This flag causes the compiler to allocate space on the stack before the return address for functions that the compiler decides are susceptible to buffer overrun attacks. On function entry, a security cookie with a known computed value is placed into this buffer, and on function exit it is checked to ensure that the cookie has not been corrupted. A change in cookie value could signify a potential overwriting of the return address, and as a result an error will be generated and the application terminated.

Of course, this doesn't prevent all buffer overrun attacks. Visual Studio .NET 2003 augmented the /GS feature by sorting local variables on the stack so that arrays were allocated in memory addresses higher than the rest of the local variables, shielding these local variables from overruns. This can deter attacks based on vtable hijacking and other pointer-based hacks.

Figure 7 0ld /GS

Visual C++ 2005 brings another update to this powerful feature. When a function call is made, the function activation record is laid out as shown in Figure 7. If one of the local buffers is overrun, an attacker could potentially overwrite everything above it on the stack, including the exception handler record, the security cookie, the frame pointer, the return address, and the function's arguments. Most of these values are protected by various mechanisms (such as safe exception handling), however it is still possible to exploit a buffer overrun in a function which has function pointer parameters. If a function takes a function pointer (or a struct or class that contains a function pointer) as an argument, an attacker could potentially overwrite the value in this pointer and cause the code to execute any function he or she wants to run. To combat this, the Visual C++ 2005 compiler analyzes all function parameters for this vulnerability and lays out the function activation record as shown in Figure 8. Copies of the vulnerable parameters are placed below the local variables on the stack, and these copies are then used instead of the originals. Should the originals be overwritten by a buffer overrun, they won't be exploitable as the copy being used will still retain the original value.

Figure 8 New /GS

In accordance with the "secure by default" Trustworthy Computing directive, the Visual C++ 2005 compiler now enables the buffer security check option by default. This will help make all products compiled with Visual C++ more secure. In fact, Microsoft is now building all of its products, including Windows, Office, and SQL Server, with this option enabled.

Other strides are being taken in Visual C++ 2005 to ensure that code is developed with security in mind. A large majority of C++ applications have dependencies on the C Runtime libraries (CRT) and on the Standard Template Libraries (STL). When these were first designed, code security was not a high priority, nor were many of the attacks in use today commonplace. As a result, much of the functionality provided by these libraries is often used in an insecure manner, opening up applications to potential attacks. Recent books like Michael Howard's Writing Secure Code (Microsoft Press® 2002) have stressed the importance of certain coding practices not exemplified by these libraries.

With Visual C++ 2005, Microsoft is introducing new versions of these libraries that are rewritten in an effort to identify all functions that could lead to common security issues and to provide alternate versions that are more secure. The long term goal of this effort is to deprecate all "insecure" versions in favor of their more robust counterparts. Over 400 new "safe" functions are being introduced in this new version of the CRT alone, ensuring that all pointer parameters are checked for null values, and all functions that perform memory copy operations take the number of bytes to be copied in addition to the destination and source buffers.

Conclusion

There is so much more available in Visual C++ 2005 that it's hard to stop writing about it all: delayed CLR loading for mixed images, the native AppDomains API, new declspecs for better global variable support with respect to AppDomains and processes, module constructors, linker support for object files and .NET modules, implicit boxing, XML comments using the same syntax C# developers have come to love, a brand new version of STL targeting the .NET Framework, param arrays, alias hints, a new floating point model, operator overloading, and on and on.

A new version of any .NET Framework-based language often causes people to ask "what language should my team use if we're writing an app targeting .NET?" Today, if you're doing a lot of native interop work, it's simple. C++ is the easiest language to use for native interop and is often the most performant. Additionally, if you're bringing an existing C++ app forward to .NET, there's really no better way. In fact, Visual C++ is the path most recommended by Microsoft when converting your existing apps to the .NET Framework.

As for new applications, you might ask why developers uninitiated in the world of .NET choose one .NET-based language over another. There is no straightforward answer as each language has its own strengths, but for pure .NET-based applications, the experience in C#, Visual Basic, and C++ is basically the same. If, as a developer, you are already comfortable in a particular language, there is no significant reason to switch to another one.

You might choose C++ over other languages, however, if you're doing any sort of interop. The experience in C++ is almost certain to be nicer than other languages due to the extensive interop support built directly into the language. In addition, the deterministic cleanup it provides through destructors is invaluable when it comes to eliminating resource leakage and ensuring the correctness of your applications. C++ also has many powerful features that can be used in combination with those provided by the CLR. For example, C++ not only supports both templates and generics, but it supports them in combination. This allows for more expressiveness and power than using the individual features alone. In particular, a useful library-writing technique is to write a template that implements a generic interface. This gives you all of the flexibility and power of templates, such as specialization, while still giving other languages the ability to directly use the objects instantiated from the template through the generic interface. All in all, C++ really is where it's at.

Stephen Toub is the Technical Editor for MSDN Magazine. You can reach him at stoub@microsoft.com.


Visual Basic: Navigate the .NET Framework and Your Projects with &quot;My&quot;

[ Editor's Update - 1/20/2006: This article refers to a beta version of Visual Studio 2005. An updated version of the article, reflecting features found in the final release of Visual Studio 2005, can be found at Visual Basic: Navigate The .NET Framework And Your Projects With The My Namespace.]
Visual Basic

Navigate the .NET Framework and Your Projects with "My"

Duncan Mackenzie
This article was based on a pre-release version of Microsoft Visual Studio 2005, formerly code-named "Whidbey." All information contained herein is subject to change.
This article discusses:
  • Visual Basic in Visual Studio 2005
  • My.Application, My.Computer, My.Forms, My.Resources, My.Settings, My.User, and My.WebServices
This article uses the following technologies:
Visual Basic 2005 and the .NET Framework
Code download available at: VB2005.exe (134KB)
Contents
Why My?
Exploring My
My.Application
My.Computer and My.User
Exposing Settings, Resources, and More From Your Project
Conclusion

I
set out to write this article about Visual Basic® 2005, but that left too many things to choose from. I could wander through the entire feature list, giving you a bit of information on cool IDE features like Edit and Continue, IntelliSense® Code Snippets, or the Exception Assistant; skim over the advanced language enhancements like generics, partial classes, and unsigned types; and barely touch on all the cool Microsoft® .NET Framework enhancements around Windows® Forms, data binding, ClickOnce deployment, and more. While these are all interesting features, I'm going to dig into one of the coolest features of Visual Basic 2005, the namespace called My.

Why My?

One of the biggest issues that programmers run into is the sheer breadth of .NET, which means that finding the best class for a particular task can be quite daunting. I have read many newsgroup postings from developers working very hard to call a Win32® API function from their Visual Basic .NET or C# code, when that exact function already exists in .NET. Why are they trying to call the API? They didn't find, notice, or perhaps understand how they could use existing parts of the Framework. My provides an intuitive navigation hierarchy that exposes existing .NET functionality through easily understood root objects, targeting the practical tasks and concepts that you grapple with on a regular basis.

Wow, that sounds great, but what does it mean? To quickly illustrate the benefit of My, consider a very common question from the forums on GotDotNet: "How do I read the entire contents of a text file into a string?" The answer for the .NET Framework 1.1 is this:

Dim sr As New IO.StreamReader("c:\mytextfile.txt")
contents = sr.ReadToEnd
sr.Close()
It doesn't require a lot of code, but it certainly isn't intuitive. For many people, the path to StreamReader would have led them through the Stream class, the FileStream class, and more before ending up with that code snippet. That exact same code will work just fine in Visual Basic 2005, but by using My I can get to the same information much more quickly by writing this:
contents = My.Computer.FileSystem.ReadAllText("c:\mytextfile.txt")
The My.Computer.FileSystem class is just one of many classes included in the My language extensions, so the first thing to do is get familiar with what is available.

My is best described as a speed-dial for the .NET Framework. This type of simplification—making it easier to find and execute the right operation—is a common practice in programming. When I run into a multistep process that will be used repeatedly by me or by my development team, I generally create a helper function. The helper function is usually just a simple call that takes in the parameters that matter for the most common usage and then performs the multistep process under the covers, smoothing the development process. Most software development shops build such helper functions, and it is quite a common practice to combine groups of them into some set of code snippets or form of shared library such as MyCompany.Utilities or Duncan.NetworkUtilities.

There is a lot of duplication of effort going on in the creation of these libraries though, as many functions would be useful to almost any development group. My takes this idea of helper functions to a whole new level because they include so much functionality and because they are being shipped with Visual Basic. Just think of it as a library full of productivity-enhancing code that you don't have to write.

Exploring My

My exposes several distinct classes which organize a variety of functions into one of seven general areas: My.Application, My.Computer, My.Forms, My.Resources, My.Settings, My.User, and My.WebServices.

In general, the classes within My provide easy access to information in one of two categories, either the underlying .NET Framework or elements of the current project. My.Application, My.Computer, and My.User are all focused on Framework functionality, while My.Forms, My.Resources, My.Settings, and My.WebServices all deal with the contents of your current project. As I walk through each of these classes, starting with those that are focused on the underlying Framework, I will provide quick samples along with information on the properties and classes they contain.

As I mentioned earlier, it can sometimes be difficult to find the exact functionality that you need when you're working with the .NET Framework, but finding it isn't the only issue you need to be aware of. In some particular circumstances, even after you have found the class you need, the specific steps and constructor parameters needed to get it ready can take several lines of code and can be difficult to understand. The My classes focused on the Framework (My.Application, My.Computer, and My.User) are designed to help you navigate directly to the class you need, and to set it up so that you are ready to do your work.

My.Application

My.Application provides developers with an intuitive grouping of application information and services. Individual nodes under My.Application include AssemblyInfo (the application's copyright, title, description, and so on), OpenForms (a collection of all Forms currently open in the project), Log (a feature-rich, highly configurable logging facility that is integrated with the System.Diagnostics classes), and more.

Using the My.Application.Log class, logging exceptions is easier than it probably should be. How are developers supposed to feel like true code warriors when they can write code like this:

Dim winINIFile As String

Try
    winINIFile = My.Computer.FileSystem.ReadText("c:\windows\wind.ini")
Catch ex As IO.FileNotFoundException
    My.Application.Log.WriteException(ex, "Error Accessing INI File")
End Try

One of the more common feature requests that have come out of discussions with users of Visual Basic 6.0 and earlier has been answered with the addition of the OpenForms collection. This collection, which was available through the keyword Forms in Visual Basic 6.0, provides you with an easy way to loop through all of the currently open forms in your application, without the hassle of maintaining your own global list, as shown here:

For Each f As Form In My.Application.OpenForms
    Debug.WriteLine(f.Text)
    f.WindowState = FormWindowState.Minimized
Next

My.Computer and My.User

My.Computer allows you to navigate to services and data related to the host computer. The functionality in this class alone deserves its own article, but I'll discuss a few of the highlights.

The FileSystem classes provide a very simple API for working with and inquiring about files; this should banish any lingering yearning for the FileSystemObject library. The following code uses the FileSystem class to copy all of the pictures from the current user's My Pictures folder to a new folder (C:\Desktop Wallpaper), displaying a progress bar if the file copy takes more than a few moments to complete:

Dim myPics As String = _
    My.Computer.FileSystem.SpecialFolders.MyPictures.Path

My.Computer.FileSystem.CopyFolderContents( _
    "C:\Desktop Wallpaper", myPics, False, True)

MsgBox( My.Computer.FileSystem.GetFiles( _
    myPics, "*.jpg", "*.bmp").Count)

To accomplish the same file copy in the .NET Framework 1.1 would require quite a bit more code (see Figure 1), and that is without including the ability to show a progress dialog, which you would have to create yourself.

Figure 1 Copying Files in the .NET Framework 1.1 (Close Figure 1)
Dim myPics As String = Environment.GetFolderPath( _
        Environment.SpecialFolder.MyPictures)

Dim pictureFiles As String() = IO.Directory.GetFiles(myPics)

Dim targetDirectory As String = "C:\Desktop Wallpaper"
Dim newPath As String
Dim jpgAndbmpCount As Integer

For Each pic As String In pictureFiles
    newPath = IO.Path.Combine(targetDirectory, pic)
    IO.File.Copy(IO.Path.Combine(myPics, pic), newPath, False)
    If IO.Path.GetExtension(pic) = "jpg" Or _
        IO.Path.GetExtension(pic) = "bmp" Then
        jpgAndbmpCount += 1
    End If
Next

MsgBox(jpgAndbmpCount)

My.Computer.Ports (which is not available in the PDC preview of Visual Studio® 2005) turns reading and writing to a serial port into a simple process. This is one of the most common and difficult tasks to perform using the current version of the .NET Framework. The snippet shown here previously took an entire article to build! (See SERIAL COMM: Use P/Invoke to Develop a .NET Base Class Library for Serial Device Communications.)

Dim comport As IO.Ports.SerialPort
comport = My.Computer.Ports.OpenSerialPort("COM1")
AddHandler comport.ReceivedEvent, AddressOf DataReceived

The My.Computer.Audio class allows you to play user-defined or system sounds, putting an end to the practice of P/Invoking the PlaySound API, as shown here:

Dim musicFile As String
musicFile = My.Computer.FileSystem. _
    GetFiles("C:\WINDOWS\Media", "*.wav")(0).Path
My.Computer.Audio.Play(musicFile)

My.Computer.Network makes a wide variety of networking calls easy to use, including receiving and sending Pings to a remote machine, uploading or downloading a file, determining if you are currently connected, and quite a bit more.

If My.Computer.Network.IsConnected Then
    If My.Computer.Network.Ping("www.duncanmackenzie.net") Then
        Debug.WriteLine("Site Available")

        My.Computer.Network.DownloadFile( _
            "http://www.duncanmackenzie.net/Articles/", _
            My.Computer.FileSystem.SpecialFolders.MyDocuments, _
            ShowProgress:=True)
    End If
End If

My.Computer.Printer works together with the new Printing classes in the Windows Forms namespace to make printing a cinch. As an example, producing the report shown in Figure 2 can be created with code familiar to those of you who've programmed with GDI+. Here are the first few lines of code:

With My.Computer.Printers.DefaultPrinter
    .HorizontalAlignment = HorizontalAlignment.Center
    .WriteLine("Document Title")
    .WriteImage(New Bitmap("z:\msdn-logo.gif"))
    .HorizontalAlignment = HorizontalAlignment.Left
    .Write("Company Name")
    .HorizontalAlignment = HorizontalAlignment.Right
    .WriteLine("Report Title")
    .WriteHorizontalLine(0.05)
    .WriteLine()
 •••
If you are wondering what's involved in producing the same type of report using version 1.1 of the Framework, you can check out my printing article on MSDN® Online at Printing Reports in Windows Forms.

Figure 2 Creating Reports with My
Figure 2 Creating Reports with My

There are quite a few more classes that I am not going to even start describing, including My.Computer.Keyboard, My.Computer.Registry, and My.Computer.Screen. I encourage you to explore these on your own.

There is so much functionality exposed through the My.Computer section of My that its object model is going to be one of the first bits of documentation I look at as each new beta and Community Drop of Visual Studio 2005 is released.

My.User is one of the simplest classes in My, but don't let its simplicity fool you; it provides access to quite a few useful bits of information about the current user of the app. This info includes properties such as DisplayName, UserName, and DomainName, along with methods for querying role membership, and status functions like IsAuthenticated:

If My.User.IsAuthenticated Then
    If My.User.IsInRole("BUILTIN\Administrators") Then
        MsgBox("tsk, tsk... running as Admin are we?")
    End If
End If

Note that the My.User properties and methods use Windows authentication by default, but will work seamlessly with custom security infrastructures as well, keeping the coding model the same regardless of the type of authentication you use.

Exposing Settings, Resources, and More From Your Project

The project-focused classes in My include Settings, Resources, WebServices, and Forms. The first two deal with your project's associated data while the clearly named Forms and WebServices classes provide direct access to forms and Web services that you have added to your project.

Figure 3 Settings Designer
Figure 3 Settings Designer

Working through the IDE of Visual Studio 2005, you can set up a variety of resource elements and a strongly typed group of user or application-level settings for your application (through the designer interface shown in Figure 3).

Figure 4 Resource Editing
Figure 4 Resource Editing

In both cases, you have to configure the settings or resources before the corresponding classes will become available through My, but once they have been added to your project, the My.Resources and My.Settings classes (respectively) make it easy for you to interact with either type of information. The following routine retrieves and modifies the value of a stored user setting, then uses the My.Resources class to retrieve a localized string (created using the IDE tools shown in Figure 4) for use in a message box:

Dim lastRun As Date
lastRun = My.Settings.LastRun
My.Settings.LastRun = Now()

Dim myMessage As String = _
    String.Format(My.Resources.LastRunMessage, _
    lastRun.ToShortDateString)

MsgBox(myMessage)

In addition to the two areas just described, all of the forms in your project are available through My.Forms and any Web services you have referenced in your project are exposed through the My.WebServices class. For Web services, this means you have the option of referencing them directly without manually creating an instance. So, after adding a reference to the TemperatureService from xmethods.com, you can retrieve the current temperature using the following code:

Dim tempService As New net.xmethods.www.TemperatureService()
MsgBox(tempService.getTemp("98052"))
This can be expressed more simply as:
MsgBox(My.WebServices.TemperatureService.getTemp("98052"))

My.Forms is different from the My.Application.OpenForms collection that I discussed earlier because My.Forms exposes a default instance of each of your Form classes, not the currently open instances of those classes. If you have a background in programming in Visual Basic 6.0 or earlier, then these default instances will be extremely useful to you as they enable you to show, hide, or otherwise access your forms using the Form's class name directly:

Private Sub showForm2_Click(ByVal sender As System.Object, _
        ByVal e As System.EventArgs) Handles showForm2.Click
    My.Forms.Form2.Show()
End Sub

Private Sub updateForm2_Click(ByVal sender As System.Object, _
        ByVal e As System.EventArgs) Handles updateForm2.Click
    My.Forms.Form2.Text = "Updated..."
End Sub

This was standard behavior in Visual Basic before the release of the .NET Framework, and it will be a welcome restoration to developers who became familiar with that style of programming in Visual Basic 6.0.

Conclusion

My in Visual Basic 2005 provides a quick and easy method for accessing some of the deeper areas of functionality in the .NET Framework without preventing you from using the Framework directly in any way you choose. In addition to exposing Framework functionality, some areas of My, such as the Forms collection, bring back familiar and convenient programming concepts from earlier versions of Visual Basic. Try it out in the available builds of Visual Studio 2005, and I'm confident that you will find it useful.

Duncan Mackenzie is the Visual Basic and C# Content Strategist for MSDN Online, author of MSDN's Coding 4 Fun column and the author of a variety of books and articles on Visual Basic and .NET. You can contact Duncan at http://www.duncanmackenzie.net.


.NET Framework 2.0: Craft a Rich UI for Your .NET App with Enhanced Windows Forms Support

[ Editor's Update - 1/20/2006: This article refers to a beta version of Visual Studio 2005. An updated version of the article, reflecting features found in the final release of Visual Studio 2005, can be found at Smart Clients: Craft A Rich UI For Your .NET App With Enhanced Windows Forms Support.]
.NET Framework 2.0

Craft a Rich UI for Your .NET App with Enhanced Windows Forms Support

Michael Weinhardt and Chris Sells
This article was based on a pre-release version of Microsoft Visual Studio 2005, formerly code-named "Whidbey." All information contained herein is subject to change.
This article discusses:
  • What's new in Windows Forms controls
  • Working with themes
  • Binding and displaying data
  • Other data-specific features
This article uses the following technologies:
C# and Windows Forms
Contents
System.Windows.Forms
Windows Themes Support
Control Layout and Configuration
Controls and Data Binding
The GridView
Typed DataSets
DefaultInstance
DataContainer Control
Master-detail Binding
ClickOnce Deployment
Configuring ClickOnce Deployment
Deployment
ClickOnce Versioning and Rollback
Conclusion

M
icrosoft really hit a sweet spot with the .NET Framework 1.0. Not only did .NET signify a fundamental shift in the way developers build and distribute Windows®-based applications, but it also created a great technological platform for scaling features upwards and outwards. Visual Studio® 2005 is a very solid case in point and does both exceptionally well with a smorgasbord of new and improved functionality. The scope of these changes most definitely covers the Windows Forms subset of the Microsoft® .NET Framework, starting with the core System.Windows.Forms and reaching out across a wide variety of supporting technologies, both old and new alike, including ADO.NET, ObjectSpaces, and ClickOnce deployment.

Productivity improvements are found in every nook and cranny of Visual Studio 2005. They include IDE enhancements, improved control and data designers, and code generation. Code junkies may be disheartened by how much less code needs to be written for a variety of scenarios, while the rest of us will have more time for Battlestar Galactica reruns.

System.Windows.Forms

Significant work has gone into System.Windows.Forms, the beating heart of the technologies that comprise Windows Forms. At the time of this writing, the namespace's usable surface areas, consisting of public types and members, has increased by 67 percent and 127 percent respectively, indicating that Visual Studio 2005 is most definitely a major release. While we simply don't have enough space in this article to cover everything, Figure 1 gives you an overview of the major new areas of improvement with additions and enhancements covering a wide range of areas, such as better theme support, improved data binding, and the GridView control.

Figure 1 New Windows Forms Features (Close Figure 1)

Category | Description
Layout controls | SplitContainer, FlowLayout, and TableLayout container controls for managing controls at design time and run time.
Data binding controls | GridView control for enhanced tabular data presentation, navigation, and processing. DataContainer control for simplifying data binding, data-bound UI design, native VCR-style UI support, and master-detail UI support.
UI building blocks | MainMenu component now supports menu item images. WinBar control implements a fully-featured tool strip like you find in Microsoft Office. WebBrowser control is a managed Web browser wrapper over shdocvw.dll.
Control and component | ControlArray makes a comeback to provide a single interface
management | for managing multiple controls, not necessarily of the same type. ComponentArray is a component-oriented alternative to the ControlArray.
Asynchronous call support | BackgroundWorker component wraps worker-thread creation and management (technically not from System.Windows.Forms, but certainly a Windows Forms must). Sound file loading in the sound component. Image file loading in the PictureBox control.
General improvements | Autocompletion support for the TextBox and ComboBox controls from a variety of system sources including Internet Explorer history and Start | Run. Most recently used List item formatting support for the ComboBox and ListBox controls.

Windows Themes Support

While creating a new Windows Forms project is the same as it has always been, the result gives rise to one of the first changes you may notice: default support for Windows themes. A Windows theme is a shell-wide user setting that dictates the look of the Windows UI. It can be specified by right-clicking on the desktop and then clicking Properties | Themes from the Display Properties dialog box.

Because Windows themes are user-specified, it is the duty of Windows Forms apps to reflect them. Themes are supported in existing versions of .NET, but version 1.0 of the .NET Framework requires the developer to add a specially named manifest file to a particular location in the file system. The .NET Framework version 1.1 simplifies the process by providing the Application.EnableVisualStyles method to enable support, and by setting the FlatStyle property to "System" on each control that implemented the method. Visual Studio 2005 further improves the process by including the call to EnableVisualStyles by default in a new Windows Forms project's default form:

public class Form1 : System.Windows.Forms.Form {
  •••
  [STAThread]
  static void Main() {
    Application.EnableVisualStyles();
    Application.Run(new Form1());
  }
  •••
 }

Traditionally, setting a control's FlatStyle to System indicates that the operating system dictates how a control will look when rendered. Now, a majority of Visual Studio 2005 controls render according to the Windows theme presently in use when FlatStyle is set to the default value of "Standard." This is semantically equivalent to a System setting and consequently saves you the effort of setting it in the designer. Whether the FlatStyle is Standard or System, the benefit for developers is theme sensitivity, including support for dynamically responding to a theme change at run time without requiring a single line of code.

Control Layout and Configuration

Windows themes support requires no effort beyond creating a project or form and dragging the necessary controls onto it. However, laying controls out on a form and configuring them can be a time-consuming process, especially for complex forms. In order to make this task easier, the Windows Forms designer comes with several new features that are intended to curtail design-time chores, including Snap-To Alignment, Edit Properties Mode, and smart tags, which are accessible in situ with the control.

While you can still use the Visual Studio Layout toolbar to align controls horizontally and vertically, you can only do so once the controls are on the form. Alternatively, you can use Snap-To Alignment to achieve the same effect as you drop controls onto a form and drag them around, avoiding the Layout toolbar and associated design-time expense, including control selection and toolbar button clicking. Snap-To Alignment is manifested as one or more sticky blue lines that guide controls into alignment with other controls in close proximity.

Figure 2 Edit Properties
Figure 2 Edit Properties

Another new feature, Edit Properties Mode, allows you to tab through controls and change their properties directly on the page. Edit Properties Mode is initiated by selecting it from any context menu on the Windows Forms designer or by clicking the menu item of the same name. The next step is to select and tab through each property, editing as you go. The process is illustrated in Figure 2. When finished, return to normal editing by clicking "Return to Layout Mode." Setting a property is a simple way to alter a control. Completely replacing one control with another more relevant control is not as simple; you will need to delete an existing control, drop a new one on the form, and reconfigure it. Complex multistep designer tasks like this are the perfect use for smart tags, which are design-time tasks packaged into a single step exposed as a menu item icon near the related control. Smart tags also expose simple, although frequently performed, design-time tasks, including setting a TextBox's Text property the moment it's dropped onto a form.

Controls and Data Binding

In addition to the IDE improvements, Windows Forms in Visual Studio 2005 offers new controls as well as improvements to old favorites such as the TextBox and ComboBox. As an example of some of the new and improved controls in use, Figure 3 shows the UberBrowser, built using much less code than would have been required in previous versions of Windows Forms.

Figure 3 Web Browser Using New Controls
Figure 3 Web Browser Using New Controls

This simple example demonstrates how the enhanced toolbox in Visual Studio 2005 allows you to build more with less. For example, the fully-functional UberBrowser took about 10 minutes to design and required only 85 lines of code to hook up the traditional gamut of Web navigation events and to implement URL navigation history. The actual work of Web navigation and HTML rendering is performed by the WebBrowser control in the right pane. The splitter bar and right and left panes are provided by a single SplitContainer.

The ComboBox, as shown in Figure 3, sports new autocompletion support. Autocompletion is implemented across three properties, starting with AutoCompleteMode, which allows you to set the autocompletion style from an enumeration of the same name:

enum AutoCompleteMode {
  None = 0x0,        // No autocompletion
  AutoSuggest = 0x1, // Possible matches chosen from drop-down list
  AutoAppend = 0x2,  // Possible matches
                     // appended to text
                     // while typing
  AutoSuggestAppend = 0x3 // AutoSuggest and
                          // AutoAppend combined
}

An option other than "None" requires you to set the AutoCompleteSource property to one of several system autocompletion sources that have been specified by the AutoCompleteSource enumeration:

enum AutoCompleteSource {
  FileSystem = 0x1,       // File system
  HistoryList = 0x2,      // All URLs from History list
  RecentlyUsedList = 0x4, // All URLs from Recently Used list
  AllURL = 0x6,           // HistoryList + RecentlyUsedList
  AllSystemSources = 0x7, // FileSystem + AllURL
  CustomSource = 0x40,    // AutoCompleteCustomSource
  None = 0x80             // No source
}

CustomSource instructs the ComboBox that you will provide autocompletion options from a collection of items stored in the AutoCompleteCustomSource property. Currently, the ComboBox and TextBox are the only controls that offer autocompletion.

The GridView

Most controls in the Toolbox support data binding to an arbitrary data source through a native data-binding infrastructure. In a variety of scenarios, particularly prototyping and RAD development, data binding is critical. The Windows Forms team has dramatically enhanced the data-binding space with improvements to typed DataSets, new controls including the GridView and the DataContainer, and richer design-time support for speedier development. These features allow you to build either a field-style or tabular-style UI with equal ease.

Sometimes a tabular-style UI is the most appropriate for presenting and processing bound data and, to that end, Microsoft included the DataGrid control in previous versions of the .NET Framework. For Visual Studio 2005, the Windows Forms team responded to community feedback on the DataGrid and decided to construct a new grid control, System.Windows.Forms.GridView (see Figure 4).

Figure 4 GridView Columns (Close Figure 4)

Possibly the most striking difference between the DataGrid and GridView is the GridView object model, which has been abstracted into a natural grid structure built upon columns, rows, and cells that allow developers to logically drill down through tons of intuitively located functionality, including:

  • Rich UI customization support through styles, formatting, layout, and item selection
  • The ability to natively display a broader variety of data than the DataGrid allowed, through a richer set of column types, including images
  • Nifty features like frozen columns (as in Microsoft Excel) and run-time column reordering (as in Microsoft Outlook®)
  • More than 100 events for granular control over navigation, editing, validation, custom painting, and error handling

In combination with a comprehensive design-time experience, the rich feature set supports rapid customization with less reliance on code—in some cases due to the folding of common DataGrid coding scenarios into native GridView methods, properties, and events. One such example is row selection on cell click, which is now exposed by the GridView as the SelectionMode property. Of course, no control can be everything to everyone. When they fall short, developers lean on extensibility to incorporate custom features. The GridView infrastructure doesn't disappoint with a variety of cell, row, and column base implementations you can derive from and plug into. The overall result for the GridView is a much more compelling grid-style control than the DataGrid.

Typed DataSets

The GridView control is part of a family of controls that can bind to data sources such as arrays, collections, and typed DataSets. The advantage of typed DataSets is that the .NET Framework plumbing allows controls to bind within the designer to help developers visualize the UI at design time. The GridView supports this by autogenerating a column for each field on the data source at design time. Visual Studio 2005 enhances typed DataSets to make your life simpler. You will spend less time in the designer as typed DataSets now fully decipher data source metadata, including foreign-key constraints and autoincrement seeds and steps, as shown in Figure 5.

Figure 5 Default Typed DataSet
Figure 5 Default Typed DataSet

The designer also generates a Fill query, which wraps the table's SELECT statement and is implemented as a method of the same name on the typed DataSet. Sometimes, the SELECT, INSERT, UPDATE, and DELETE statements you specify initially are not useful in all situations. For example, the Fill method will return all Employees, although there are times when you may only need to return a single employee. In this particular situation, it makes sense for you to use the same table and fill it with a more specific query. The typed DataSet designer accommodates by allowing you to add your own queries from the appropriate table's context menu by selecting Add | Query from the table's context menu.

The "DbTable Query Wizard" dialog opens to help you construct your query, after which the table is updated with the new data. When you complete the configuration of a typed DataSet, you typically bind it to one or more controls in the user interface. In Visual Studio 2005, typed DataSets can be bound directly to controls that implement DataSource and DataMember properties without the need for adding a DataSet to the form.

DefaultInstance

Anyone who has made heavy use of typed DataSets knows that, in general, each new typed DataSet class is used in one place in an application. Because the Windows Forms team knows this and wants to make our lives easier, they've put a static property, called DefaultInstance, into each generated typed DataSet class to hold that single instance. It's implemented like so:

   public static EmployeeTDS DefaultInstance {
     get {
       if (_defaultInstance == null) {
         _defaultInstance = new Northwind();
       }
       return _defaultInstance;
     }
   }
You use the DefaultInstance property this way:
public class Form1 : System.Windows.Forms.Form {
  •••
  private void Form1_Load(
    object sender, System.EventArgs e) {
    // Load data
    EmployeeTDS.DefaultInstance.LoadData();
  }
  •••
}

Databound controls set their DataSource property with the default instance's type name that by default is ProjectNamespace.TypedDataSetName, while the subsequent process of specifying a DataMember remains unchanged.

DataContainer Control

The GridView control is an example of a complex bound control, which means that it knows how to manage the data binding to support navigation, inserts, updates, and deletes. On the other hand, simple bound controls don't know this and require you to implement equivalent support through the data source's binding manager, typically with a VCR-style user interface. You would use simple bound controls when your preference for presenting list-style data is as a set of fields rather than a grid. This approach requires you to add fields to the form manually as a set of appropriate controls, such as Labels and TextBoxes, resulting in something that looks like Figure 6.

Figure 6 Basic UI
Figure 6 Basic UI

While you can implement such Details Forms by hand, in Visual Studio 2005 you can implement them in one fell swoop with a DataContainer bound to a typed DataSet. Simply select the Details Form menu item from the DataContainer's smart tag.

This feature will automatically generate all the fields from the selected data source for you. In addition, the DataContainer will load and save the data and adds VCR-style navigation and editing by default, without writing code. The result is shown in Figure 7.

Figure 7 Field-style UI with VCR Controls
Figure 7 Field-style UI with VCR Controls

If your UI requirements change and a tabular-style UI is more suitable, you can use the DataContainer to do the hard work of changing the UI for you by selecting the appropriate option from the DataContainer's smart tag. The designer removes all existing controls and replaces them with a single DataGrid (although in future versions of Visual Studio this will most likely be a GridView).

Master-detail Binding

The multifunctional DataContainer also happens to be a great tool for constructing master-detail UIs, which provide a Forms-based visualization of the one-to-many relationships used in relational databases. One example of this is the Employees-Orders relationship, which is correctly configured for you by simply adding the Orders table to the typed data.

Figure 8 Master-detail Form
Figure 8 Master-detail Form

Since we already have the master UI generated (shown in Figure 7) all that we need to do is drag a GridView onto the form, set its DataSource to the DataContainer, and its DataMember property to the Orders-OrderDetails relationship—FK_Orders_Employees in this case—resulting in Figure 8.

ClickOnce Deployment

When you finally have all the controls you need on your forms and the code needed to make them work together, you'll have an application that you still need to deploy. Deploying an enterprise-wide Windows Forms application to a user base of any nontrivial size can be quite an installation and versioning challenge, so developers of previous generations have been more than happy to trade rich Windows UIs for Web applications to gain the accompanying deployment benefits.

With .NET, Microsoft began the process of creating an infrastructure that allows Windows Forms applications to be deployed with the same ease as ASP.NET applications. Prior to Visual Studio 2005, the .NET Framework supported no-touch deployment, which allowed Windows Forms applications to be deployed from either a URL or a Universal Naming Convention (UNC) file path and executed within a code access security (CAS) controlled sandbox on the client machine. No-touch deployment has evolved into ClickOnce deployment in Visual Studio 2005, incorporating several new features along the way to offer Windows Forms developers a deployment infrastructure as slick as Web applications. In particular, ClickOnce offers improved support for application delivery, versioning and rollback, greater control over client-side application installation, permission elevation, and design-time configuration support all included in the structure of Visual Studio.

Configuring ClickOnce Deployment

At design time, ClickOnce deployment can be configured through Visual Studio 2005 from the Publish property page in a project's Configuration Properties. Depending on your infrastructure and user base, different deployment sources (file servers, Web servers, or FTP servers) make sense in different scenarios. [ Editor's Update - 7/1/2004: FTP servers can be used as publishing targets, but they cannot be used as deployment sources (i.e. clients can't install from an FTP server).] ClickOnce deployment allows you to specify which of these your application will be published to and installed from and the mechanism by which it is installed (through a Web page, for example). Installation can also be controlled by configuring whether the application is physically installed to the client machine. If installed to the client, you can then stipulate whether the application checks for updates before it loads or as it runs, how frequently it does so, and enforce required updates—a useful option to deploy critical updates. It is also possible to specify application prerequisites, the most likely being the .NET Framework itself. Several security settings can also be set from the Security property page.

When it comes to publishing, you can build and publish the application in one step by clicking Project | Publish Project, which will both build and publish the application. The process of publishing is managed by the Publishing Wizard, which basically revisits your publishing configuration settings, allowing you to change them on a per-publish basis. The final form in the publishing process presents you with a list that confirms your choices and gives you an option to either publish the application or go back and alter the settings. The Publishing Wizard then enacts your choices and publishes the application as required.

Deployment

Deployment begins when the user navigates to the deployment Web page and clicks the appropriate hyperlink for the prerequisites and/or the application. The application link refers to a .deploy file rather than to the application executable itself. The .deploy file is one of two manifests required by ClickOnce deployment that specifies application deployment and update requirements. The second of the two files, .manifest, specifies the files required by the application to execute. This is the semantic equivalent of an assembly manifest. Both .deploy and .manifest files are processed on the client machine by the ClickOnce deployment service, System.DeploymentFramework.Service.exe. This service is installed with the .NET Framework and starts the first time a .deploy file is requested. While the user doesn't see any of this background activity, he is presented with a dialog to confirm the download. [ Editor's Update - 7/1/2004: If the online application is running in the sandbox, the user might not be presented with the dialog.] When the download has completed, the actual application will be loaded into and begin executing from AppHost.exe. The application is stored in a local cache unless you specify no client-side installation. [ Editor's Update - 7/1/2004: The application is always cached locally.] While subsequent application invocations will always need .deploy, the application will execute from the local cache until .deploy specifies that a new version is available on the server.

If the application's security requirements are greater than those configured for the zone from which the application is executed (Internet or intranet, for example), ClickOnce provides the user with the opportunity to elevate application permissions to the required level, ensuring correct execution. If not, the application is never started, giving it no opportunity to run at all. [ Editor's Update - 7/1/2004: If the application is not granted the necessary permissions, it won't be downloaded.]

ClickOnce Versioning and Rollback

If requested, an icon is added to the Start menu and an entry is added in Control Panel | Add/Remove Applications. From Add/Remove Applications, you can either completely uninstall the application or revert to the previous version, if there was one. ClickOnce versioning depends on the version numbers you specify for your application using the AssemblyVersionAttribute, which is automatically generated into the AssemblyInfo.cs file of every Windows Forms project by default. It looks like this:

[assembly: AssemblyVersion("1.0.*")]

The version number is used to determine when a new version is ready for download. It is also used to name both the deployment folder and the client-side cache folder in which the application resides. The naming convention for both folders is ApplicationName_major#.minor#.build#.revision#.

ClickOnce deployment is a powerful technology that offers many more benefits than we can describe in this article. However, the key motivation for using ClickOnce is the ease of deployment, installation, and versioning for your Windows Forms applications. For more information on ClickOnce, see the article "ClickOnce: Deploy and Update Your Smart Client Projects Using a Central Server" by Brian Noyes in this issue of MSDN® Magazine.

Conclusion

When you consider the large number of new features in Visual Studio 2005, you'll realize that we have barely begun to discuss the wide variety of technologies that you will likely use in Windows Forms development. These include improvements in ADO.NET and support for an entirely new way to access relational data via object-relational mapping with the brand new ObjectSpaces. What we've done here is give you a taste of the most important new features and explain the corresponding increase in productivity that they facilitate, beginning with IDE and designer improvements in Visual Studio and moving on to controls that are either improved from previous versions, like the TextBox and the ComboBox, or just plain new, like the GridView and the WinBar. The GridView and DataContainer controls, in conjunction with typed DataSet enhancements, will help you incorporate richer data presentation and processing into your applications more rapidly.

Finally, ClickOnce deployment simplifies the entire Windows Forms deployment process while providing deployment capabilities comparable to traditional installers.

Michael Weinhardt is currently co-updating Chris Sells's book, Windows Forms Programming in C# and writes a monthly column for MSDN Online called "Wonders of Windows Forms." Visit http://www.mikedub.net for further information.

Chris Sells is a Content Strategist for MSDN online, currently focused on "Longhorn," the next version of Windows (http://msdn.microsoft.com/longhorn). More information about Chris and his various projects is available at http://www.sellsbrothers.com.


ClickOnce: Deploy and Update Your Smart Client Projects Using a Central Server

ClickOnce

Deploy and Update Your Smart Client Projects Using a Central Server

Brian Noyes
This article was based on a pre-release version of Microsoft Visual Studio 2005, formerly code-named "Whidbey." All information contained herein is subject to change.
This article discusses:
  • Using ClickOnce to deploy real apps
  • Versioning benefits
  • Programmatic and automatic updating
This article uses the following technologies:
ClickOnce, XML, C#
Code download available at: ClickOnce.exe (502KB)
Contents
First Time App Deployment
Designing ClickOnce Applications
ClickOnce Sample App
Programmatic Updates
The Shape of Things to Come
Conclusion

A
pplication architects often struggle with two important but competing goals. They want to write a Windows®-based rich client application that runs on the desktop because that provides the best stateful, interactive experience for the user. However, they also want to minimize the effort required to deploy and update their apps—a goal best accomplished using a thin client model. Users care about applications that are easy to use and that will not interfere with other apps on their machine.

ClickOnce, part of version 2.0 of the Microsoft® .NET Framework, allows you to deploy Windows-based rich client apps to a desktop by placing the application files on a Web or file server accessible to the client and providing the user with a link.

When the user clicks the link on a Web page or in an e-mail, the application files are downloaded to the user's machine and executed. The program executes in a secure sandbox provided by .NET code access security (CAS) limiting what it's allowed to do based on the evidence it presents. When a new version of the application is deployed to the server, it can be automatically detected by connected clients, and the update can be downloaded and applied in a variety of ways based on optional deployment settings. ClickOnce allows applications to be deployed offline as well, making the app behave as if it had been installed on the local machine.

ClickOnce relies on a number of capabilities that already exist in the .NET Framework and runtime, as well as a few new ones that will be introduced specifically for ClickOnce. To understand the security protections of ClickOnce, you need to have a good understanding of CAS. If you have been exposed to no-touch deployment in .NET 1.x, or the Updater Application Block from the Microsoft Patterns and Practices series, you will have a basic understanding of what ClickOnce can do for you. ClickOnce will be further enhanced in the next version of Windows, code-named "Longhorn," so this is a deployment model that it would be wise to get your arms around for the future.

A key goal of ClickOnce is to provide a trustworthy deployment model for users to be able to download and execute applications from centrally managed servers without requiring administrator privileges on the client machine. They are deployed in a safe manner that prevents ClickOnce deployed applications from interfering with or corrupting any other applications or data on the client. Applications deployed using ClickOnce also need to run in a secure execution context whose permissions are limited based on where the application is coming from or the trust assigned to the originator of that application.

The initial deployment model of ClickOnce is straightforward and applies to any ClickOnce application. When it comes to updating ClickOnce applications, there are a bunch of options that modify the way updates occur depending on choices that are made when the application is published to the deployment server. To clarify the terms, I will refer to the process of deploying application files to a deployment server as "publishing" the application, whereas the process of deploying application files to a client machine from the deployment server via ClickOnce mechanisms will be referred to as "deploying." To differentiate between the two types of ClickOnce applications, I will refer to applications that are deployed to support offline execution as "installed" applications, and those that can be run only while online as "launched" applications.

First Time App Deployment

The basic process to perform a first-time deployment of an application via ClickOnce is shown in Figure 1. Once the application is developed and tested, you publish the application to the deployment server. This involves simply copying all the application files into a folder on the Web or file server, along with the application and deployment manifest files that describe the deployment to the ClickOnce runtime. Once the files have been published to the deployment server, a link to the deployment manifest is provided to the user in the form of a URL (as a hyperlink or shortcut). When a user clicks on that link, the .NET runtime on the client machine will recognize the file extension and perform a ClickOnce deployment of the application, downloading and executing the application on the user's desktop.

Figure 1 ClickOnce Deployment Process
Figure 1 ClickOnce Deployment Process

The files are downloaded to a cache under the logged-on user's profile on the local machine in an obfuscated location. The application then executes with the CAS security context determined by the location of the deployment server, such as the Internet or Local Intranet zone. You can override the default location-based policy with a custom security policy on the user's machine that depends on content-based evidence (hash, publisher, or strong name). If you have content-based evidence that matches the deployed application, the permissions of the corresponding code group will apply instead. I will discuss security protections and configuration options as I step through some of the deployment scenarios throughout the rest of this article.

If the ClickOnce application was published to be a launched application, then that is pretty much the end of the story. The user will have to launch the application using the link provided. Typically, this will be a link to a Web site, which launches the application that provides functionality related to the site's purpose. Every time the user clicks the link, the runtime on the user's machine will check the version cached locally against the version on the deployment server. If new files are available, they will be downloaded before the application is launched. The locally cached version will then be executed on the user's desktop. This is very much like no-touch deployment using .NET 1.x today.

However, if the application was published as an installed application, in addition to launching the first time the user clicks the provided link, a shortcut will be added to the Start menu (for example, Start | Programs | MyCompany | MyClickOnceApp), and the program can be launched from there in the future. Additionally, an item will be added to the Add/Remove Programs Control Panel applet to allow the user to uninstall the application, a process which simply removes the Start menu shortcut and clears the files from the cache under the user's profile. The Add/Remove Programs item will also let the user roll back to a previous version if updates have occurred.

The update story for installed applications gets a little more complicated because of the many available options. These include whether updates are checked for automatically, whether they are required, when the application checks for available updates, and whether the update is downloaded before launching the application or in the background while the app is running. Additionally, you can perform updates programmatically using the ClickOnce API and skip the automatic update mechanisms. I'll step through several of these scenarios later with the sample application.

Designing ClickOnce Applications

ClickOnce is focused on deploying presentation-tier Windows-based applications. It is not a replacement for Windows Installer (MSI) technology, nor is it suitable for all types of applications. As mentioned earlier, if a program needs to carry out privileged operations that could affect other applications or data on the target machine, such as performing unrestricted file access or accessing the registry, then it may not be suitable for deployment using ClickOnce. Additionally, if an application needs privileged access to the local machine at install time, then it is definitely not a candidate for ClickOnce deployment because by design there are no mechanisms in ClickOnce to add registry entries, install shared assemblies in the Global Assembly Cache (GAC), install services, and so on.

However, if you can design your application so that it does not need to do anything special at install time, and its primary purpose is to present a rich user interface that talks to back-end services on the network, then you have a great candidate for ClickOnce deployment. ClickOnce allows you to request additional runtime execution privileges at install time by prompting the user, and you can also deploy a trust license from a trusted license authority that will allow elevated runtime privileges without prompting the user. This allows your ClickOnce app to do meaningful things that go beyond what the default security policy would allow, but does not require an administrator to touch each client machine to deploy custom security policy every time you roll out a new application. As with the design of any application, however, you should address your security requirements early to determine whether ClickOnce is a suitable option.

Applications deployed with ClickOnce have no specific implementation requirements in the most common scenarios. You don't have to write any ClickOnce-specific code, derive from any base classes, or implement any interfaces. The basic ClickOnce capabilities are provided by the runtime, so the client application itself does not necessarily have to be aware of ClickOnce. There are optional programmatic approaches to performing updates that would require writing some ClickOnce-specific code in your application, but that only applies if the ClickOnce automatic update capabilities do not meet your requirements.

Any client machines will need to have the .NET Framework 2.0 installed on them for ClickOnce to work. When you publish a ClickOnce app, a bootstrapper setup.exe and supporting files are also published to the server, and allow you to easily deploy the .NET Framework and any other Microsoft required components, such as the Microsoft Data Access Components (MDAC), Microsoft Installer, DirectX®, and so on. However, installing those items is a privileged operation and will require administrator permissions to succeed. The server itself does not have any specific requirements and can be any kind of Web server or file server.

ClickOnce is targeted toward line of business applications such as Customer Resource Management (CRM) applications, personnel management applications (timecards, training, pay, and benefits management, contact info, and so on), or business data analysis applications. These applications typically need to present a rich interactive UI for the user, but act upon data and services that are accessed on middle-tier applications, Web servers, or database servers. They are also frequently deployed in enterprise environments in which there may be thousands of workstations, and deploying apps and updates to all of those machines is often too costly if an administrator has to perform a normal installation on each machine. ClickOnce could also potentially be used for consumer applications over the Web, such as multimedia applications, games, communications tools, and other forms of apps that do not have extensive install-time requirements, especially if those applications might have updates frequently available.

ClickOnce Sample App

The best way to get a sense of what ClickOnce can do for you is to take a walking tour by performing a number of deployments and updates using a sample application. The TimecardManager sample application allows you to enter and view time entries for working on a job or project. There are probably a few more features you would add before using it for real-world time tracking, but it's a good example of a simple application that you'd deploy to large numbers of intranet or Internet users to be available both online and offline. I will evolve the application over the course of the article from a launched application that simply stores time entries in a local file using isolated storage to a final installed application that stores entries in a database when connected and performs updates at the explicit request of the user.

I'll show you some dialogs that you will see in the Tech Preview version of Visual Studio® 2005 (formerly code-named "Whidbey") to demonstrate the basic process of publishing and updating applications. Keep in mind that these UIs will probably change significantly for the better by the time Visual Studio 2005 is released, and even by Beta 1, so don't get too wrapped up in the particular layouts and options. Focus mainly on the flow.

TimecardManager is a simple Windows Forms application consisting of a couple of forms, as shown in Figure 2. As I mentioned earlier, except for certain advanced scenarios, you do not have to write any ClickOnce-specific code within the application itself. So for the first couple versions of the TimecardManager application, I am not going to focus on the code at all.

Figure 2 A Windows Forms App
Figure 2 A Windows Forms App

To publish this application, you need to place the compiled executable (and any supporting assemblies and resources) into a folder on the Web server or file server that you will use as the deployment server. In addition to the application files themselves, you need an application manifest and a deployment manifest. The manifest files are just XML files with a particular schema that the common language runtime (CLR) understands and can use to determine what to deploy and how to deploy it using ClickOnce.

The application manifest (.manifest file) describes the assemblies and files that comprise the application, including identity information of the assemblies (name, hash/public key token, version, and locale), application dependencies, and trust information that will affect the security policy under which the application is run. The deployment manifest (.deploy file), shown in Figure 3, contains summary information about the deployment itself, including version information, where to find the application manifest, and a number of options regarding the kind of deployment and update behavior to be performed.

Figure 3 The Deployment Manifest (Close Figure 3)
<?xml version="1.0" encoding="utf-8"?>
<assembly
  xsi:schemaLocation="urn:schemas-microsoft-com:asm.v1
  assembly.adaptive.xsd" manifestVersion="1.0"
  xmlns="urn:schemas-microsoft-com:asm.v1"
  xmlns:asmv2="urn:schemas-microsoft-com:asm.v2"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <assemblyIdentity name="http://localhost/appdeploy/TimecardManager/
       TimecardManager.deploy" version="2.0.0.0"
       publicKeyToken="ef176fbfd3ea4d64" processorArchitecture="x86"
       asmv2:culture="en-US" />
  <description asmv2:publisher="Software Insight"
    asmv2:product="TimecardManager">
  </description>
  <deployment isRequiredUpdate="false"
    xmlns="urn:schemas-microsoft-com:asm.v2">
    <install shellVisible="true" />
    <subscription>
      <update>
        <beforeApplicationStartup />
        <periodic>
          <minElapsedTimeAllowed time="0" unit="hours" />
        </periodic>
      </update>
    </subscription>
    <deploymentProvider codebase="TimecardManager.deploy" />
  </deployment>
  <dependency>
    <dependentAssembly>
      <assemblyIdentity name=" http://localhost/appdeploy/
         TimecardManager/TimecardManager.manifest"
         version="2.0.0.0" publicKeyToken=" ef176fbfd3ea4d64"
         processorArchitecture="x86" asmv2:culture="en-US" />
    </dependentAssembly>
    <asmv2:installFrom
    codebase="TimecardManager_2.0.0.0/TimecardManager.manifest" />
  </dependency>
</assembly>

You could hand-craft these files, but Visual Studio 2005 will automatically create them when you use its built-in publishing capabilities. When you select Project | Publish Project, a wizard steps you through the final options involved in actually publishing the application to a deployment server so that it can be presented to ClickOnce clients. There are additional ClickOnce options that affect the way the application will be updated, and these options can be accessed through the Project Properties dialog.

Under Configuration Properties, there are two new subsections of project settings: Security and Publish (shown in Figure 4 and Figure 5). The Security section allows you to specify which permissions the application needs to execute. These requirements get published as part of the application manifest and are used on the client to determine whether to prompt the user to allow elevated permissions for the application. If the permissions required by the application are greater than the permissions that the security policy would grant based on the application evidence, then the user will be prompted to allow the elevated permissions that the application needs to run.

Figure 5 Publish Project Settings (Close Figure 5)

Figure 4 Security Project Settings (Close Figure 4)

In the PDC Technology Preview version, the functionality in the Security project settings is incomplete. In future versions, you will have fine-grained control to specify individual permissions that the application requires. In the Tech Preview version, you can only select from the built-in permission sets in .NET. However, you can tweak the resulting application manifest by hand after publishing, as I will do in version 2.0 of the sample application. The Publish project settings allow you to configure additional fine-grained options regarding how the application gets deployed and updated by ClickOnce, including whether the application can be run offline or online only, whether updates are required or optional, and how updates are downloaded and applied.

If you open up version 1.0 of the TimecardManager project in Visual Studio and select Project | Publish Project, you will see the Publish Wizard, which allows you to specify the location to which you want to publish the application. Typically, this location will be a Web server, but it could be a network file share, an FTP server, or a local disk path as well.

I will assume you have IIS on your development machine, so first enter a publishing location of http://localhost/appdeploy/TimecardManager. Then the wizard will confirm whether you want this application to be an installed application (available offline) or a launched application (available online only), and will match what was selected in the Publish project settings. For version 1.0 of the application, you will deploy it as a launched application. The final step just summarizes the actions that will be taken.

When you press Finish, Visual Studio 2005 will recompile the application to make sure it is up to date, and will then copy the project build outputs to the deployment server. It will also generate the appropriate application and deployment manifests based on the project settings and will place them on the deployment server as well. Finally, it will launch an instance of Microsoft Internet Explorer opened to a generated publish.htm page that contains a link to the deployment manifest so that you can test out the ClickOnce deployment process.

If you click the deployment manifest link, the client runtime checks the application information on the server and the application will launch without prompting the user. If you run the app again using the link to the .deploy file, you will see a brief flash as the runtime rechecks the application information to determine that the cached version is current, and then the application will launch again with no additional prompting.

Now that you've built, deployed, and executed your first ClickOnce application, the next logical step is to take a look at some of the update behaviors of ClickOnce. You should be aware of a couple of issues in the Technology Preview version of ClickOnce that could trip you up. First, you must have a network connection, even if you are deploying from the local machine. Second, you must close the browser that contains the publish page before publishing a new version of the application, or the caching of that page in Internet Explorer will confuse ClickOnce on the client machine.

Version 1.1 of the sample app has no additional functionality, but its deployment model is changed to an installed application instead of a launched application. You can actually version the deployment separately from the versioning of the assembly or application by changing the version attribute on the assemblyIdentity element in the .deploy file. For demonstration purposes and to have Visual Studio do the dirty work, I just bumped the version number of the assembly in the AssemblyInfo.cs file. This will also result in an updated deployment version number that matches the executable assembly version when you publish with Visual Studio. If you open version 1.1 of the TimecardManager, select Project | Publish Project, and specify the same location for the publishing server as used for version 1.0, you will see in Step 2 of the Publish Wizard that this version of the app is configured for both online and offline use.

If you launch the application using the link on the publish.htm Web page after publishing, you will see a slightly different prompt. Because this version of the application will be installed on the user's machine (meaning it adds a Start menu item and an Add/Remove Programs item), the user is prompted to confirm the installation. If you click the Install button, the application will be downloaded and run as before. If you look in the Start menu after this process, you will find a new menu group under Programs with the name of the company under which your Visual Studio environment is running. Under that will be a shortcut to the deployment manifest that lets you launch the application whether you are online or offline.

In Visual Basic® .NET projects, there is a Version section in the project properties that lets you set the company and description information that will be used for publishing. In the Technology Preview, these settings are completely separate from those specified using assembly attributes, and C# has no corresponding project properties. I would expect to see a more unified approach in later beta versions. You can always manually set the company name that the app is published under by editing the publisher attribute of the description element within the deployment manifest file after publishing to the deployment server. In fact, you can expect that administrators will want to control what goes in the deployment manifest, and tools will be made available in later releases to allow them to easily edit the settings through a GUI tool.

The ClickOnce project settings for version 1.1 of the TimecardManager specified that ClickOnce should check for application updates before the application starts. The settings also specified that application updates are required. If you are connected to the network, each time you launch version 1.1 of the TimecardManager from the Start menu shortcut, it will check the server to see if there is a new version available. If there is, it will require that you download and run that new version, otherwise the application will not be allowed to run. You will be prompted when it detects a new version, but aside from that the connected behavior with these settings is almost identical to a launched application. If you are offline, the app will just run like a normal local application.

Let's move on to yet another version of TimecardManager to demonstrate both the update behavior and some elevated permissions requirements. Version 2.0 of the application allows you to save your timecard entries to a database over the network if you are connected, as well as cache the entries in isolated storage when you are disconnected. It also puts the data access code into a separate class library assembly, so the application then consists of more than one assembly. Because it talks to a database, it will require permissions that are not part of the Local Intranet code group that is applied based on launching from a URL like http://machinename/folder/app.deploy. Note that the code group is applied at the application level, which is a new feature in version 2.0 of the .NET Framework. In version 1.x of the .NET Framework, the lowest level that code groups were applied to was the individual assembly. Because a ClickOnce application manifest provides an identity for an application as a single entity, regardless of which assemblies and files it is comprised of, security policy can be scoped to the application instead of individual assemblies.

To allow the application to obtain the permissions it needs without administrator intervention on the client machine, ClickOnce can prompt the user at installation time to ask them to grant the application the elevated permissions. Once permission is granted, the user will not be prompted again on subsequent runs. Although this capability circumvents some of the security protections provided by the CLR by allowing a user to elevate security permissions for that application, you can prevent the user from being allowed to do this through security policy in an enterprise environment. Allowing the user to elevate permissions is an important capability for applications deployed over the Internet, where no administrators may be available.

The next step is to open the project for version 2.0 of the TimecardManager and publish the project. Because the Technology Preview version of Visual Studio 2005 does not allow you to tweak the individual permissions needed by a ClickOnce-deployed application, you will have to edit the application manifest after publishing. This way, you can make it prompt for the additional database permission that this version of the application needs. You can find the application manifest in a subfolder of the location to which you published. For example, if you published to http://localhost/appdeploy/TimecardManager, the application files and manifest for version 2.0 will be located in C:\Inetpub\wwwroot\appdeploy\Timecard Manager\TimecardManager_ 2.0.0.0.

Based on the project security settings, the manifest will contain permission elements for the Local Intranet zone, but version 2.0 of the application requires the additional SqlClientPermission to make calls to the database. To do this, you add an IPermission element to the security section of the application manifest, specifying the permission type required as shown here (removing the line breaks):

<IPermission class="System.Data.SqlClient.SqlClientPermission,
   System.Data, Version=1.2.30703.27, Culture=neutral,
   PublicKeyToken=b77a5c561934e089" version="1"
   Unrestricted="true"/>
Once you add that element to the security element in the manifest and save it, you can launch version 1.1 of the application using the Start menu shortcut that was created at install time. When you do, the application will check the deployment server, see that a new version (2.0) is available and updates are required, and prompt you to apply the update. When you do, you will see a slightly different installation dialog confirming both installation and elevation of permissions. If you allow it to install, the app should download the updates and run version 2.0, getting the time entries from the database rather than from isolated storage. The demo application keeps it simple and works with either offline or online storage, and does not try to synchronize the two data sources. For a real-world application you would need to come up with a design to merge the locally cached time entries with those on the database server when you run in a connected mode following disconnected operations.

In later versions of the Visual Studio beta, there will be one additional way to handle elevated permissions without needing to prompt the user. In an intranet environment, you can have an administrator configure a trust license authority within the security policy on each user's machine. Once this is done, a trust license can be deployed with a ClickOnce application to elevate permissions to those required by the application without prompting the user. Basically, if the trust license comes from an authority that is already trusted by the user's machine, then the ClickOnce engine will be allowed to elevate the permissions on the user's machine based on the credentials of the trust license authority deployed with the application.

Programmatic Updates

The automatic updating features in ClickOnce are very flexible and can address the vast majority of application update requirements. There may be times, though, where you want full control of how and when updates occur. Version 3.0 of the sample application does not use the automatic update features of ClickOnce at all. Instead, it uses the ClickOnce programmatic API to manually check for available updates and command those updates when necessary. In the Technology Preview, the classes that support these features live in the System.DeploymentFramework namespace. In later versions, they will be moved to System.Deployment. This namespace lives in a separate assembly, so version 3.0 of the application references System.DeploymentFramework.dll to bring in the code for updates. You can try out the by-request update feature by running version 3.0 of the TimecardManager application and selecting File | Check for Updates. The application code will use the ClickOnce API to see if any updates are available from the deployment server and, if so, will download them.

In addition to changing the update model, version 3.0 also corrects a bug that you may have noticed. Each time you updated to a new version, any records saved in isolated storage disappeared. In .NET 1.x, you only had two options for using isolated storage: isolation by user and assembly, or isolation by user and domain.

Using isolated storage for a ClickOnce application is the right thing to do because it prevents you from needing elevated file I/O permissions for many scenarios and keeps your application's files from interfering with other applications' files. However, the isolation levels available in .NET 1.x were insufficient for a ClickOnce scenario because as soon as you apply an update, the assembly and domain identity will change, and your existing application files stored in isolated storage will be orphaned. As a result, version 2.0 of the .NET Framework includes a new level of isolation: isolation by user and application. This allows files stored using this isolation level to be accessible by multiple versions of the same application, where the application version is determined by the application manifest, as opposed to the normal individual assembly versioning scheme for .NET-based applications.

To use programmatic updating, you code against the ApplicationDeployment class (previously named ApplicationUpdaterService in the Technology Preview). This class exposes several methods and properties that can help you determine whether updates are available and how to apply them. The static CurrentDeployment property will get you back an instance of the ApplicationDeployment that encapsulates information and interaction with the deployment on the client machine. When you call the CheckForUpdate method, the deployment server will be checked for an update. If one is available, you can get the new files either synchronously with the Update method or asynchronously through a separate API. Figure 6 shows the on-demand update code from the sample application.

Figure 6 Updates on Demand (Close Figure 6)
private void checkForUpdatesMenuItem_Click(object sender,
   System.EventArgs e)
{
  ApplicationDeployment updater = ApplicationDeployment.CurrentDeployment;
  Version verDepServer = updater.CheckForUpdate();
  if (verDepServer != null) // Update available
  {
    DialogResult res = MessageBox.Show(
     "A newer version of the TimecardManager application is available."
     + " Do you wish to update the application now?",
     "TimecardManager   Updater", MessageBoxButtons.YesNo);
    if (res == DialogResult.Yes)
    {
      updater.Update();
      MessageBox.Show("Please shutdown and restart the application" +
      " to start using the new version.");
    }
  }
}

With the Technology Preview bits, once the update is downloaded, you will need to prompt the user to restart the application for the new version to take effect. The update is simply downloaded to a separate set of folders under the user's cache and the Start menu shortcut is updated to point to that version. In later builds of Visual Studio 2005, I expect to see the ClickOnce API include a way to tell the application to shut down and restart using the new version that has just been downloaded using the API, without having to prompt the user to restart manually.

The Shape of Things to Come

I have touched on a lot of the features available in ClickOnce. Several features are not yet available, but should be by the time Visual Studio 2005 is released. I mentioned trust licenses earlier, which will allow permissions to be elevated in deployments coming from a trusted source without prompting the user. In future builds of Visual Studio 2005, XML signatures will be applied to all manifests and will be required to verify the authenticity of the deployment source and ensure that the app files have not been tampered with since they were deployed. Visual Studio 2005 will include a "Debug in zone" feature that allows you to launch a debug session in a specified security context so that you can debug the app with the permissions that it will have granted in the targeted runtime environment, such as the Local Intranet or Internet zones. It will also include a code analysis tool to determine exactly which permissions it requires so that you can truly specify the minimum set of permissions required when you deploy with ClickOnce.

ClickOnce in the next release of Windows, code-named "Longhorn," will contain further enhancements. In order to support all the platforms that the .NET Framework supports, ClickOnce in Visual Studio 2005 just uses simple file transfer mechanisms to download files. In Longhorn, ClickOnce will take advantage of the Background Intelligent Transfer Service (BITS) to allow a trickle-feed update of files in a way that is less obtrusive to the user. Additionally, the execution context in Longhorn will be very different and more robust, allowing apps to either be hosted in a browser or run externally, and they will run within a Secure Execution Environment that will provide even better protections than will be available on the .NET Framework 2.0 runtime. The new UI features of Longhorn will provide an even more compelling reason to run a Windows client instead of a browser-based application. ClickOnce will make this possible in a distributed environment. Look for other enhancements not yet announced once initial feedback on ClickOnce is obtained.

Conclusion

ClickOnce provides a new deployment technology that gives you the best of a rich Windows-based application user experience and the deployment and maintenance benefits of Web applications. It allows a user to download and execute a rich client application over the Web, off a network file share, or from local media, and the app can subsequently be run offline. It offers a full Windows user interface running on a desktop, while allowing single server deployment of application files and updates. Client applications are automatically deployed and updated on the user's machine from the deployment server in a safe way and will not affect other applications or data that already exist on the machine. Those applications run in a security context that can prevent them from doing malicious things at run time. ClickOnce includes a number of options that give application architects and designers a lot of flexibility in addressing real-world deployment and execution scenarios and requirements.

The power and flexibility provided by ClickOnce make it possible to use Windows client applications in many scenarios where browser-based applications are the only feasible solution. I recommend you take a close look at the capabilities of ClickOnce and consider how it can deliver your distributed application's presentation tier when Visual Studio 2005 is released. I'm certain that you'll find that ClickOnce makes updating and deploying your own applications faster, smoother, and more user friendly.

Brian Noyes is a software architect with IDesign Inc., a .NET-focused architecture and design consulting firm. Brian specializes in designing and building data-driven distributed apps for Windows and the Web. Contact him at brian.noyes@idesign.net.


Debugging: DataTips, Visualizers and Viewers Make Debugging .NET Code a Breeze

Debugging

DataTips, Visualizers and Viewers Make Debugging .NET Code a Breeze

Morgan Skinner
This article was based on a pre-release version of Microsoft Visual Studio 2005, formerly code-named "Whidbey." All information contained herein is subject to change.
This article discusses:
  • Enhanced debugging in Visual Studio 2005
  • Customizing the debugger display with your own attributes
  • Extending the types the debugger drills into with custom proxies and visualizers
This article uses the following technologies:
C#, Visual Studio 2005, DataSets
Code download available at: VS2005Debugging.exe (157KB)
Contents
DataTips and Type Viewers
DebuggerDisplayAttribute
DebuggerBrowsableAttribute
DebuggerTypeProxy
Type Visualizers
Writing a Visualizer
Getting the Data
Visualizing the Hashtable
Rendering the Visualizer
Conclusion

T
here is a host of new goodies in Visual Studio® 2005 to make your coding more productive. These include generics, anonymous methods in C#, Master Pages in ASP.NET, and various new controls in Windows® Forms, to name just a few. Underpinning these advancements is another batch of improvements which are less likely to grab the limelight but are nonetheless very important. Regardless of how good a programmer you are or how many flashy tools are available in your toolbox, you will always have to jump into the debugger at some point in the process to see what's going wrong. The only problem with breaking into the debugger in such a scenario is that the debugging environment isn't currently as rich in features as it could be.

As an example, suppose you have an application that retrieves some data from a database and stores it locally within a DataSet. Could you go through a DataSet within the debugger and tell exactly what's in it? How about after some modifications? Can you tell which rows have been updated, and what the old and new values are? This scenario is common because there is often some object within the debugger which is not simple to decompose manually. Luckily, Visual Studio 2005 comes to the rescue with a raft of new debugger features. There are DataTips, which are nicely formatted tooltip windows, displayed while debugging, that can expand to drill into more detail on a particular object. The next level of enhancement comes with Type Viewers, which consist of a number of new attributes that can alter the display of a particular type within the current debug windows. At the top of the pile are Type Visualizers, which allow you to write a custom viewer window for a particular type.

In this article, I'll briefly discuss DataTips, and then go on to discuss Type Viewers and Type Visualizers in more depth.

DataTips and Type Viewers

When you're debugging, you often want to explore a particular variable to see what it contains. As an example, consider an Employee class, which might contain several member variables and properties. A sample C# definition of such a class is shown in Figure 1.

Figure 1 Employee Class (Close Figure 1)
using System;

public class Employee
{
    public Employee ( int employeeID , string forename , string surname )
    {
        _employeeID = employeeID ;
        _forename = forename ;
        _surname = surname ;
    }

    public int EmployeeID
    {
        get { return _employeeID ; }
    }

    public string FullName
    {
        get { return string.Format ( "{0} {1}" , _forename ,
                                                 _surname ) ; }
    }

    public string Forename
    {
        get { return _forename ; }
        set { _forename = value ; }
    }

    public string Surname
    {
        get { return _surname ; }
        set { _surname = value ; }
    }

    private int _employeeID ;
    private string _forename ;
    private string _surname ;
}

A DataTip is displayed within the code window when the mouse hovers over a particular variable. The data displayed is a dynamic, expandable tooltip, which expands as the mouse is moved over the display. Figure 2 shows the display of data for an object as the mouse hovers over the variable.

Figure 2 A DataTip
Figure 2 A DataTip

This display is similar to that shown within the Quick View or Watch windows, but it's now possible to get this display without switching away from the code window. So far all of this is for free—DataTips just work, so any current classes you have will exhibit this behavior in the debugger.

The displayed data, however, contains both the internal class members and public properties, which produces a confusing display as data is replicated. If you could somehow filter the data to show only necessary items then that would be a real step in the right direction. Also, displaying "{Employee}" is rather crude. It would be better to be able to control what's displayed for a given item to something under the programmer's control. For both of these facilities, you need to look at Type Viewers.

With the DataTip shown in Figure 2, there is too much information, so you need some way to show only the public properties. This customized view will be used in DataTips and also Watch windows and the Quick Watch window. To get this to work, you must attribute your type using one or more of the new Debugger attributes. In the following discussion, I'll describe these attributes and show the effect of these within the Watch window.

DebuggerDisplayAttribute

This attribute is used to provide a single-line summary of a type within the data windows of the debugger. The text displayed is now under your complete control, so that the following could be used on the employee type to display the FullName property:

[DebuggerDisplay("Employee ( {FullName} )")]
public class Employee
  {
    •••
  }

The debugger uses this string as a format specification, and replaces items within curly braces with the corresponding data from the object being displayed. Within the braced expression, you get implicit access to the object's this pointer. The data within the braces can be a field, property, or method, and can have any visibility, so using private members here is permitted. The syntax examples shown in Figure 3 would result in the output shown in Figure 4. Note that I've constructed this image from three different runs to show them all in one image. The use of expressions within the DebuggerDisplay attribute should be kept to a minimum, as the expansion of the attribute is done by the debugger and is language specific.

Figure 3 Debugger Output (Close Figure 3)

Syntax Description
[DebuggerDisplay(
  "{_forename} {_surname}")]
Uses the private _forename and _surname members
[DebuggerDisplay(
  "Employee- {ToString()}")]
The ToString method is called to provide the textual representation to be displayed as the value of the object
[DebuggerDisplay("Employee
  ( {Forename} {(_wizard) ?
  \"Is a Wizard\" : \"Is
  \"Is not a Wizard\" } )")]
An expression is evaluated in order to provide a differing representation to the user based on the value of a flag
Figure 4 Types in the Debugger
Figure 4 Types in the Debugger

The DebuggerDisplay attribute allows you to customize the display of a given object. However, when the object is expanded you're still rewarded with private data and properties. There are two ways to change what is displayed when a given type is expanded. The first is to attribute some fields as being invisible to the debugger, and the second is to use a proxy class.

DebuggerBrowsableAttribute

DebuggerBrowsableAttribute is added to members of a given class to specify whether they are visible, and if so, what state they should show up in. The attribute has one parameter, that being a value from the DebuggerBrowsableState enumeration. The chart in Figure 5 describes the four possible values for this enumeration, and describes the output that is displayed in the data window when that value is used on a member.

Figure 5 DebuggerBrowsableState Values (Close Figure 5)

Value | Description
Collapsed | The item will be shown initially collapsed within the data window. This is the same as the default behavior of the debugger.
Expanded | This will expand a hierarchical type when its containing type is expanded. If you had a Customer class with an internal list of Orders, then that list of orders could be automatically expanded for the user when the Customer was expanded.
Never | This attribute hides the member so that it is not displayed at all within the debugger data windows. It would be used in my Employee class to hide the internal members, such as _forename and _surname.
RootHidden | This value indicates that the attributed member should not be shown, but should instead be expanded to show submembers.

With these attributes, it is possible to create a simple display for a given type, but there may be cases where simply hiding fields isn't enough. This is where the DebuggerTypeProxy attribute is useful.

DebuggerTypeProxy

When added to a type, the DebuggerTypeProxy attribute specifies a separate class used as a proxy when displaying that type. You can think of this attribute as a custom view for a type. It is common to define the proxy for a type as an internal nested class, since this kind of class has full access to all private details of the containing type and, as such, can use these details when constructing the display.

The proxy class must have a public constructor which accepts one argument of the type that is displayed by the proxy. All public properties of the proxy will be used in constructing the display and any private or protected variables will not be shown.

The code in Figure 6 shows a typical Address class, which contains fields for five address lines and a ZIP code. The proxy class has been designed to output only those address lines that are non-null. Note that the DebuggerTypeProxy is defined on the outer Address class.

Figure 6 Address Class (Close Figure 6)
[DebuggerTypeProxy(typeof(AddressProxy))]
public class Address
{
    public Address ( string line1 , string line2 , string line3 , string
                     line4 , string line5 , string zip )
    {
        _addressLine1 = line1 ;
        _addressLine2 = line2 ;
        _addressLine3 = line3 ;
        _addressLine4 = line4 ;
        _addressLine5 = line5 ;
        _zipcode = zip ;
    }

    private string _addressLine1 ;
    private string _addressLine2 ;
    private string _addressLine3 ;
    private string _addressLine4 ;
    private string _addressLine5 ;
    private string _zipcode ;

    internal class AddressProxy
    {
        public AddressProxy ( Address address )
        {
            _address = address ;
        }

        public string Address
        {
            get
            {
                StringBuilder sb = new StringBuilder (
                    _address._addressLine1 ) ;

                if ( null != _address._addressLine2 )
                   sb.AppendFormat ( ", {0}" , _address._addressLine2 ) ;
                if ( null != _address._addressLine3 )
                   sb.AppendFormat ( ", {0}" , _address._addressLine3 ) ;
                if ( null != _address._addressLine4 )
                   sb.AppendFormat ( ", {0}" , _address._addressLine4 ) ;
                if ( null != _address._addressLine5 )
                   sb.AppendFormat ( ", {0}" , _address._addressLine5 ) ;
                if ( null != _address._zipcode )
                   sb.AppendFormat ( ", {0}" , _address._zipcode ) ;

                return sb.ToString ( ) ;
            }
        }

        private Address _address ;
    }
}

The resulting display within the Watch window is shown in Figure 7. In addition to showing the simplified view, it is also possible to expand the Raw View node to see all of the internal members of the object. I'll discuss the magnifying glasses later.

Figure 7 Filtered Addresses
Figure 7 Filtered Addresses

There are many classes in the Framework marked with the DebuggerTypeProxy attribute; it is especially useful for complex classes in which there are a large number of internal member variables that aren't usually vital to have when debugging. The view displayed is somewhat customizable in that with the addition of a proxy you can decide which data to display and which to hide. However when displaying more rich types such as DataSets and Images, it would be better to be able to "see" the object.

When debugging in Visual Studio, it's common to stop on a breakpoint and look through a number of local variables to see what they contain. A problem arises when that data does not have a simple representation, such as a bitmap or other complex data structure. In this situation, the debugger still presents a textual view of the data type, which at best can be used to find information such as the pixel dimensions of a bitmap, but won't really help with viewing the actual contents of that bitmap. What's needed is some way to graphically display an item in the debugger so that a custom window can display the data. It is now possible to perform this custom rendering in Visual Studio 2005 using Type Visualizers.

Type Visualizers

A Type Visualizer is a class in the Microsoft® .NET Framework that implements the IDebugVisualizer interface, which I'll discuss shortly. An assembly-scoped attribute is used to identify that the assembly contains a custom visualizer and to provide information as to what type of visualizer it applies to and the name displayed in the UI. I'll show an example of this attribute later in this article.

Figure 8 Visualizable Types
Figure 8 Visualizable Types

Any type that has a visualizer defined is displayed with a magnifying glass and dropdown menu, as shown in Figure 8. Clicking the magnifying glass will display the last chosen visualizer for that type. A small number of visualizer classes ship with the .NET Framework, most notably for the string class. There are three styles available for strings: a multiline text box showing the text, a version which displays the string as XML, and another that displays the string as HTML within a browser. These displays provide a much richer environment and should save a great deal of time when debugging, especially when you're constructing large segments of HTML or XML manually. The display shows a menu which lists all visualizers defined for the selected type (in this case a Hashtable, listing the visualizer from this article). The user can then click this menu item to display a modal UI.

Writing a Visualizer

A visualizer usually consists of three classes: one that implements the IDebugVisualizer interface, another that provides the user interface, and a third that serializes the .NET type from the debugee process to the debugger, since you need to copy the object from the debugee to the debugger in order to construct the visualization (you don't necessarily need to copy the whole object—just the necessary data). The code in Figure 9 is a minimal implementation of a visualizer for a Hashtable.

Figure 9 Hashtable Visualizer (Close Figure 9)
using System;
using System.Diagnostics ;
using System.Drawing;
using System.Windows.Forms;

[assembly : DebuggerVisualizer ( typeof (HTVisualizer.MinimalVisualizer ) ,
 VisualizerUIType.Modal , Target = typeof (System.Collections.Hashtable ) ,
 Description = "Minimal Hashtable" )]

namespace HTVisualizer
{
    /// <summary>
    /// Constructs a minimal implementation of a visualizer
    /// </summary>
    public class MinimalVisualizer : IDebugVisualizer
    {
        void IDebugVisualizer.Show ( IServiceProvider provider ,
             IVisualizerObjectProvider objectProvider , VisualizerUIType
             uiType )
        {
            Form blank = new Form ( ) ;

            blank.Text = "Put some controls on here!" ;

            IModalDialogVisualizerService modalService =
                provider.GetService ( typeof
                (IModalDialogVisualizerService ) ) as
                IModalDialogVisualizerService ;

            modalService.ShowDialog ( blank ) ;
        }
    }
}

Writing visualizers in this way means that they can be added to classes that already exist within the .NET Framework and are hooked up to the class they apply to at run time. The interfaces used for a visualizer are defined within the System.Diagnostics namespace, but they are buried within another assembly currently called meehost which ships with Visual Studio and is installed in the Common7\IDE directory. By the time Visual Studio 2005 ships, this assembly will be renamed and installed in the Global Assembly Cache (GAC).

In Figure 9 the DebuggerVisualizer attribute is applied with assembly scope and indicates that there is a visualizer which targets the Hashtable and is of type MinimalVisualizer. The Description property is used as the caption on the dropdown menu within Visual Studio. The code then requests the IModalDialogVisualizerService from the passed service provider, which is used in this example to show a blank form to the user.

The VisualizerUIType enumeration defines three values: Modal, DropDown, and ToolWindow. The latter two are not currently supported by the Visual Studio IDE but may well be implemented in a later version of the product.

Once the visualizer is written, it needs to be installed so that Visual Studio can access it within a debug session. In the present version, the assembly that contains the DebuggerVisualizer attribute should be copied into the Common7\Packages\Debugger\attribcache directory. This directory is probed by the debugger each time a debug session is begun, so files dropped in here will be evaluated the next time the debugger runs.

Getting the Data

So far I've discussed how to construct a visualizer, but I haven't described how to get hold of the object that is being displayed in order to do something useful with it.

The IDebugVisualizer.Show method is passed an object that implements the IVisualizerObjectProvider, which is used to obtain this serialized object, as shown in the following code snippet:

Hashtable hash = objectProvider.GetObject ( ) as Hashtable ;
Under the covers, this operation is serializing an object from the process being debugged, transporting it into the memory space of the debugger, and then deserializing it for use within the visualizer. The opposite is also possible in that a visualizer can replace an object within the debugee by calling IVisualizerObjectProvider.SetObject. This facility opens up some wonderful possibilities for tools. For years, developers have been updating an object within the debugee manually in the data window, but now there's a way for you to write your own custom code to do it.

Suppose you're testing some boundary condition that breaks the application. Rather than wading through a bunch of screens in order to set things up correctly, you could write a visualizer that would load and save objects such as a serialized DataSet. Without altering the source code whatsoever, you could set a breakpoint, load in a DataSet from disk, and continue with your debug session.

Even though the class that implements IDebugVisualizer is called a visualizer, your code doesn't have to provide a UI for the user. You could dump the contents of the currently selected data type to disk, or you could validate a string against an XML schema. You might also want to take a snapshot of an object to look at it later, or possibly revert an object to some state stored on disk. Feel free to come up with your own creative ideas because debugging just became much more fun!

Being able to read from as well as write to the object in the debugee is a great capability, but now that you have the hashtable, what do you do with it?

Visualizing the Hashtable

The impetus for writing this article was a late night session I had with the debugger and a hashtable. I've written my own hashtables in the past and was eager to display the data graphically within a .NET Hashtable. Along the way I unearthed the inner workings of this data structure, which made writing the visualizer all the more interesting.

For those of you who don't spend time working on the internals of data structures, a brief recap on how hashtables work may be in order. The main idea of a hashtable is to provide access to a particular object based on a key value. Each object added to a hashtable is allocated a slot based on the hash of this key, where a hash is the output of a function that generates a number based on some input data and which always generates that same number when provided with the same input. To retrieve an object, all you need to know is the key value, which when rehashed will return the same slot within the data structure so the object can be retrieved quickly, usually in O(1) (pronounced "order one") time.

A typical hashtable is made up of a number of "buckets." When an item is inserted into a hashtable, a bucket is chosen based on the hash of the key used for that item. A bucket is often implemented as a linked list into which the new item is inserted. This is called collision chaining, as multiple items that hash to the same bucket are chained together with a list. When searching a linear hashtable, a sequential linear search is performed on the linked list found in the bucket selected by the hash. Figure 10 shows an example hashtable, which consists of four buckets and some entries.

Figure 10 Simple Hashtable
Figure 10 Simple Hashtable

Here the buckets are represented on the left, and each is the head of a linked list of nodes. In this simplified example, when an entry is added to the hashtable, the bucket selected is calculated as the remainder when the Key value is divided by 4 (in C# this would be Key % 4). When a new entry is added to the hashtable, its bucket number is calculated and it is then added into the list. In the case of keys 4 and 12, both fall into the same bucket, which is called a collision and which leads to a chain of entries.

A hashtable that has a good spread of buckets can provide optimal lookup time for a node, but one that has many chained objects isn't going to provide good random access to nodes because once a bucket has been found, the linked list needs to be traversed to find the particular node, resulting in O(n) (pronounced "order-en") performance.

The .NET Hashtable isn't implemented in as simplistic a manner as the one just described, and it performs better than the example shown in Figure 10 because it uses a nonlinear rehashing algorithm. Simply put, when an item is inserted, a candidate bucket is chosen, which involves a calculation based on the hashcode. If the bucket is full, another calculation is performed, which effectively jumps to another bucket in the hashtable. This occurs until an empty bucket is found, at which point the inserted item is stored in this bucket. Figure 11 shows a hashtable with three entries on the left with no collisions. In the hashtable on the right, there is a collision with the third element of the array, so the hash_coll term is updated to indicate this collision.

Figure 11 Hashtable
Figure 11 Hashtable

The value stored in the hash_coll entry is not a pointer but a discrete value that the algorithm uses to determine whether to continue searching for a free slot. The value will have the high bit set if there is a collision, whereas it will be clear if no collision has occurred. In Figure 11, when the final item is added into the hashtable, the third entry would have the collision bit set. Deleting entries is also simple with the .NET Hashtable because the algorithm simply updates the key to a predefined value in order to indicate that the slot is vacant.

In order to work out how Hashtable was implemented, I used Lutz Roeder's excellent Reflector tool, available for download at http://www.aisto.com/roeder/dotnet. I find this tool invaluable and use it practically every day. You can also see the implementation of Hashtable as implemented in the Shared Source CLI, available at .NET 247.

So, armed with the knowledge of how the Hashtable class was implemented, the next step was to implement a proxy class that would read a hashtable and convert the internal representation into something that could be serialized and consumed by the debugger. For the purposes of this example visualizer, only the string representation of the key and value are being used, as this cuts down on the amount of data being passed from the debugee to the debugger, as well as being all that can be displayed by the visualizer.

In order to decompose the hashtable into data that could then be passed to the debugger, I needed to do a small amount of reflection. In most circumstances, I would argue against the use of reflection, however sometimes there simply is just no other way. In this instance, the hashtable contains an internal array which contains the bucket information described previously. The visualizer uses code to read the data from the buckets array and converts it into a representation that can then be serialized. A bucket is defined as seen in Figure 12, which shows a slightly modified display from the Reflector tool.

Figure 12 A Bucket in the Reflector
Figure 12 A Bucket in the Reflector

Getting the values contained within a particular bucket is a two-step process. The first step is to retrieve the private buckets collection, which is returned as the Array type. Retrieving a particular bucket is then accomplished by calling the GetValue method on the Array, which returns an object from which further code can be used to retrieve individual fields from the bucket. This example code is shown in Figure 13. The code presented in the download uses the same method, but in a slightly different manner as it has been split into two different functions. One function retrieves the buckets array and the other function retrieves values from a given bucket.

Figure 13 Get Buckets Array (Close Figure 13)
Array buckets ;

FieldInfo fi = _hashtable.GetType().GetField ( "buckets" ,
    BindingFlags.NonPublic | BindingFlags.Instance ) ;

buckets = fi.GetValue ( _original ) as Array ;

// Next get a particular bucket, based on the passed bucketNumber
object bucket = buckets.GetValue ( bucketNumber ) ;

// From the bucket, read the key
FieldInfo info = bucket.GetType ( ).GetField ( "key" , BindingFlags.Public
                                               | BindingFlags.Instance ) ;

object key = info.GetValue ( bucket ) ;
// Then read the value
info = bucket.GetType ( ).GetField ( "val" , BindingFlags.Public |
                                     BindingFlags.Instance ) ;

object val = info.GetValue ( bucket ) ;

// Lastly get the hash_coll
info = bucket.GetType ( ).GetField ( "hash_coll" , BindingFlags.Public |
                                     BindingFlags.Instance ) ;

int hash_coll = (int)info.GetValue ( bucket ) ;

To read the value of a private instance variable, you need to obtain the FieldInfo object for the field using reflection and then call its GetValue method. Because reflection is being used instead of strongly typed access during development (which you can't use to access the private fields of another class), the code in the download contains considerable error checking.

Once all of the code to decompose the hashtable into constituent parts has been completed, the next decision is how best to represent that data so that it can be used by the visualizer. Armed with the knowledge of how the hashtable was implemented, I decided upon the scheme that is illustrated in Figure 14.

Figure 14 The Hashtable Deconstructed
Figure 14 The Hashtable Deconstructed

The left-hand column shows the hash value generated for each discrete key in the hashtable. There may be fewer hashes created than entries within the hashtable as two keys might well produce the same hashcode. Similarly, a given hashcode might lead to a bucket that contains an object with a different hash due to the probing algorithm used by the implementation.

The third line displays the buckets that are traversed when looking for the item with a key value of 4. The hashcode of 4 is computed, and is then used to index into the buckets collection. Here it finds an entry which does not match the key, so it is shown with a greyed out caption (to indicate a collision). The hashtable algorithm then re-indexes into the array based on the current position, which results in bucket 39. This has the appropriate key, which is indicated by the yellow caption.

You can see that looking up the node with a key value of 4 is slower than the other items, due to this collision with the node in bucket 23. The following code produces this (abridged) data:

Hashtable ht = new Hashtable();
for(int loop = 0; loop < 20; loop++)
{
  ht.Add (loop.ToString(), loop.ToString());
}

There are only 20 entries in this collection, and when I tested this I received four collisions. The number of collisions isn't easy to predict as it is dependent on the key values, the number of items in the hashtable, and the current size of the buckets array (which is dynamically resized based on its content). It is always best to size any collection before adding items into it, and the hashtable is no exception to this rule, even though it will dynamically change size.

The code that displays the hashtable is fairly lengthy and space doesn't permit me to display it all, so feel free to download it and dissect it as necessary. There are two vital sections: deconstructing the hashtable and displaying it within the debugger process.

Rendering the Visualizer

When deciding how to display the hashtable, I wanted to be able to display some of the useful parts of the hashtable without going into too much complexity. I chose the display in Figure 14 because it's easy to read and it displays problems with hash collisions immediately. During testing I wrote some code that would cripple any keys passed to the hashing function so that only 1 of 10 possible hashes was returned. This proved instructive of how many collisions will occur if a poor hashcode generator is used. This visualizer will therefore be helpful if you have created your own hashing functions, as you can see how well your hashing algorithm is partitioning the items.

A number of features of the control are worth looking into. First, the control supports zooming, so the user can use the mouse wheel to zoom in or out of the control at run time. This is very useful when there are a large number of items within the hashtable. Quite a bit of code is required to support zooming, so I wrapped this in a reusable class and derived the Hashtable visualizer from it.

Each object displayed on screen is an instance of a Node class I've created, containing details of the display element, such as the text displayed in the title and details section. When I first wrote the code, each node also contained brush and font objects. However when displaying a large hashtable with many objects, this became too memory-intensive, so I added in a behavior object.

This behavior object is shared among many nodes, and it contains information such as the gradient fill colors and fonts to use when rendering the nodes. When a node is rendered, it uses its behavior object to obtain the appropriate visual styles. Using this pattern has made the rendering process not only faster but also less memory hungry. The rendering of each node uses a graphics path that specifies the rounded corners, and each node is built up from a number of different operations. One way to make this process faster still would be to pre-render each different node style into an offscreen bitmap, blit these onto the screen, and then draw the text.

Conclusion

In this article I have described some of the new debugging features that are in Visual Studio 2005. The DataTips feature shows debugging information right in the code where it's needed, without requiring you to open another window or select the appropriate object within the watch window. Being able to expand an object here is a fantastic step forward, and the ability to filter the displayed information for the appropriate data by using the new debugger attributes is further evidence of how hard the debugger team has been working to make debugging much less of a chore.

In my opinion, the best new features in Visual Studio 2005 are the custom visualizers. Using them, you can now freely extend part of the debugger experience, and display custom views of objects that are live within your debug session.

The example code for this article provides a visualizer for the hashtable and is available from the link at the top of this article.

Morgan Skinner is an application development consultant for Microsoft in the UK, specializing in Visual C#, Windows Forms, and ASP.NET. He's been working with .NET since PDC 2000. His home page is http://www.morganskinner.com.


Editor's Note: Introducing Our Online Corrections Page

Editor's Note
Introducing Our Online Corrections Page

One of the great things about working for a print publication like MSDN Magazine is the sense of permanence that the printed word provides. Sure, there are fringe benefits—the trade show groupies and expensive vacation junkets paid for by advertisers are great—but at the end of the day, once something appears in print most people consider it "official" in a way that a Web site can never replicate.

Of course, the impermanence of online content provides some benefits that we don't get in a print vehicle. For instance, if an online article has a bug or needs a typo fixed, the site master can just dive into the page, fix what's broken, and within five minutes no trace of the problem remains. Not so with print, and our policy has always been that the MSDN Magazine Web site is an exact replica of the printed magazine. Even if an article is out of date, we leave it on the site for archival purposes.

Over time, we've come to the realization that however well-intentioned this policy was, it could potentially lead to magazine content providing less-than-optimal guidance to our online readers. We've worked to create a sensible update policy that would maintain the integrity of the original article while fixing any sections that could lead to unexpected problems. Through this process, the MSDN Magazine Corrections Page was born.

There are several types of updates that might need to be applied to articles. The most common are fixes for outdated information, incorrect statements, and security updates. For instance, in 2001 we ran a great article about ActiveX interop using the .NET Framework Beta 1. It was a well-written piece that did a great job of explaining the topic. The problem, however, was that the features didn't make it into the final release of the Framework version 1.0!

With our updated corrections policy, we note that the article contains outdated information on the updates page, and then provide a warning at the top of the article, plus pointers to newer MSDN Library content that will help you solve the problem that this article originally addressed. With that fix, we've saved dozens of developers a support call. In addition, we hope we're saving developers the time they might have spent searching for a feature in the Framework that ultimately wasn't included in their version.

As we learn more about the right ways to write hack-resistant applications, many techniques that were formerly part of a product's documentation are now understood to be security holes. The most common example is a database connection call that uses the sa account with a blank password. Everyone's done this in the past, right? We are working to remove these instances from our back issues content and noting exactly why we are doing this with each instance.

There is little question that our editors and authors are the tops in the industry. We put each article through extensive review, but once in a while a nagging little error can inadvertently creep into a piece. If we catch one, we will correct it online and clearly note the change so that you don't have to rely upon blind luck to find a fix that's been slipstreamed into our content.

We invite you to visit our online corrections page frequently at http://msdn.microsoft.com/msdnmag/correct.aspx. If you can't remember that URL, we also offer it as a menu choice from our homepage. If you find what you think is a problem with an article, please contact the MSDN Magazine Ombudsman at mmombud@microsoft.com.

It's not enough to produce zero-defect articles for our readers. We are committed to maintaining the quality of all our content, both in print and online. And if you want to reward us with an expensive vacation junket, you've got our e-mail address!

—J.T.

Thanks to the following Microsoft technical experts: Scott Berry, Brandon Bray, Christopher Brumme, Simon Calvert, Felix Cheung, Scott Currie, Nick Hodapp, Anson Horton, Rob Howard, Polita Huff, Christopher Jones, Ronald Laeremans, Gerrard Lindsay, Enzo Lombardi, Duncan Mackenzie, Scott Nonnenberg, Joel Pobar, Grant Richins, Jay Roxe, Scott Seely, Herb Sutter, Dan Vallejo, and Feng Yuan.

DirectX, IntelliSense, Microsoft, MSDN, MSN, Microsoft Press, Outlook, Visual Basic, Visual C++, Visual C#, Visual Studio, Windows, and Win32 are registered trademarks of Microsoft Corporation. Windows Server is a trademark of Microsoft Corporation. Other trademarks or tradenames mentioned herein are the property of their respective owners.

MSDN Magazine does not make any representation or warranty, express or implied with respect to any code or other information herein. MSDN Magazine disclaims any liability whatsoever for any use of such code or other information.


New Stuff: Resources for Your Developer Toolbox

New Stuff
Resources for Your Developer Toolbox
Theresa W. Carey

Contents
Compare Internal Data for Debugging
Document Your .NET-based Apps
Reengineer and Migrate Apps to .NET
Test Distributed Applications
Tab Control for C and C++
.NET Database Enterprise Server
Windows Forms Style Sheets
Emulate Hardware Devices in .NET
Include Autocomplete in Your .NET-based App
BASIC Game Development
The Bookshelf

Compare Internal Data for Debugging

Guardsoft has announced the availability of VSGuard, which allows you to compare internal data between two executing programs. This technique can reduce time spent on debugging and testing. It is useful whenever a program is modified during the development process or ported to a different environment, such as when moving the program to a new platform or during system upgrades.

VSGuard is fully integrated with Visual Studio® .NET. One of its best features is migrating apps from Visual Studio 6.0 to the Microsoft® .NET Framework. VSGuard runs Visual Basic® 6.0 and Visual Basic .NET projects concurrently, automatically comparing data and reporting the differences it identifies.

VSGuard links seamlessly with industry standard visualization packages such as OpenDX, which can display the differences between the internal data of programs. You can test whether the program performs the same task as a previous version and take advantage of tools to isolate any differences you may find.

VSGuard is able to identify errors down to an individual line in the source code. It does not require large trace files, which saves both space and time. VSGuard can be used with Visual Studio .NET and versions 1.0 and 1.1 of the .NET Framework, on Windows® 2000, and Windows XP.

http://www.guardsoft.com

Document Your .NET-based Apps

Innovasys announced Document! X 4.0, which automatically produces technical documentation and online help for Microsoft .NET Framework assemblies, COM components, Visual Basic for Applications source code, and XSD schemas. Document! X 4.0 includes full-page edit and visual comment designers for Visual C#® .NET and Visual Basic .NET source code that allow you to edit source comments in a preview of the generated documentation.

Also new is documentation of XSD schemas, including full graphical schema diagrams. With Document! X 4.0, documentation can be produced throughout the design and development phases, allowing development teams to work with an up-to-date reference.

http://www.innovasys.com

Reengineer and Migrate Apps to .NET

Simplex has released Transport Suite 2.0, which is comprised of two products: Visual Expert and Architect. Visual Expert is a source code analyzer for legacy apps built with Visual Basic and PowerBuilder. It enables development teams to acquire and transfer application knowledge by referencing application components, identifying business rules, creating system documentation, and exporting the application code structure to XML for analysis and integration using Unified Modeling Language (UML).

Architect streamlines design and development cycles by providing a decompiled view of Visual Basic, PowerBuilder, C/C++, Java application baselines and other code and database structures using UML. Architect automatically generates foundation code for database schema, table maintenance stored procedures, data access class wrappers, inter-application workflow, and class structures directly from the UML designs.

Architect is an extensible and customizable environment, supporting a diverse range of analysis, documentation, and generation requirements.

http://www.simplexng.com

Test Distributed Applications

Shunra Software Ltd. recently announced Shunra\Storm Solution Suite 3.1 designed to emulate any enterprise environment, including the global network and remote user load in the lab. It enables testing, networking, and performance professionals to evaluate the functionality, robustness, performance, and scalability of any distributed n-tier application from the remote user perspective well before global deployment takes place.

Shunra/Storm 3.1 can be used for testing any distributed application such as ERP/CRM, eBusiness, VoIP, Citrix, offshore outsourcing, .NET, and Web services. It features integration with automated load testing tools, scripting tools, shell and control processes, Tcl, SQL tools, VBScripts, and many console applications such as FTP and Telnet.

Shunra\Storm 3.1 includes the ability to save the lab configuration in a single file, including instructions for all third-party tools. Using these stored files, users can test different application versions under the same parameters each time to baseline the application and ensure accurate testing. Shunra\Storm 3.1 provides detailed reports that combine and correlate results from all test lab equipment. Through the central console, stored test scenarios can be rerun automatically whenever required.

http://www.shunra.com

Tab Control for C and C++

Softel vdm has released SftTabs/DLL 4.5, its latest tab control version for C, C++, and C++/MFC. SftTabs/DLL 4.5 includes 60 basic tab control styles that can be further customized to suit your needs with multiple tab rows, scroll buttons, bitmaps, and colors.

This new version introduces hidden tabs, which can be convenient when an application wants to suppress certain tabs to simplify the user interface. Hidden tabs are best used in situations where certain groups of users should not have access to all tabs of a tab control without being aware that any additional tabs may exist. Tabs can also be disabled so they are still visible but not accessible to the user.

SftTabs/DLL 4.5 supports development using Visual Studio 6.0 and Visual Studio .NET and with C and unmanaged C++ applications. The included controls can be used on all versions of Windows 95 through Windows Server™ 2003, allowing migration of applications across the supported platforms. SftTabs/DLL 4.5 is available for 32-bit/Pentium-class and for Itanium (2) 64-bit systems.

http://www.windowscontrols.com

.NET Database Enterprise Server

Ong-Tech has launched its Enterprise Server, which helps developers implement enterprise applications by providing services that handle all database transactions. In addition, Enterprise Server provides a service brokerage functionality allowing custom services to be hosted inside the server to take advantage of its built-in features.

Enterprise Server is implemented with .NET Framework technology and is designed for extensibility, scalability, and interoperability. By exposing the server as XML-based Web services, code can be written to consume these services on virtually any platform using any programming language.

Enterprise Server currently supports SQL Server™, MySQL, PostgreSQL, ODBC, OLE, and Oracle-compatible databases. Additional database vendors are supported through the extension API.

http://www.ongtech.com

Windows Forms Style Sheets

.net PowerTools has released PerfectForm, a component for Visual Studio .NET that creates consistent and stylish Windows Forms applications. PerfectForm allows you to change the properties of all controls from a single style document without the need to recompile.

Support is built in for custom controls and third-party controls. A Visual Studio wizard is included for creating new Windows Forms, or your existing application can be modified with a few code changes. With PerfectForm, you can centralize your style definitions across an application or across the enterprise.

http://www.dotnetpowertools.com

Emulate Hardware Devices in .NET

Instrumentation Widgets, published by 9Rays.Net, is a library of meters, indicators, scales, sliders, gauges, knobs, buttons, dials, tanks, thermometers, switches, matrixes, LEDs, and other .NET-based components that emulate hardware devices.

They are all Microsoft .NET Framework managed components written in C#. All controls use double buffering to produce smooth animation and avoid flicking. There are currently more than 30 components in the library, with more on the way. The collection of components also includes compasses, oscilloscopes, odometers, toggles, multimedia buttons, and buttons with skin support. All the device emulators are customizable and scalable, and users can change size, color, font types, value range, and other properties of each component. All the components include a data binding feature as well. The program comes in Standard and Professional editions.

http://www.9rays.net

Include Autocomplete in Your .NET-based App

Dinky AutoComplete Tool (DACT), published by LaMarvin, s.r.o. is a small component that provides an autocomplete feature to any Windows Forms application. Once the component is integrated into an application by placing it onto a Form, the DACT tool automatically adds autocomplete to all System.Windows.Forms.TextBox and combobox controls (and all of their descendants) on the form.

The built-in designer support provides for design-time customization of the autocomplete-related properties. DACT supports autocompletion of user-entered strings (saved in the registry), file system paths, user's history lists, and the user's most recently used files as well as URLs.

http://www.lamarvin.com/dact_default.asp

BASIC Game Development

ethosBasic.com has released an updated version of ethosBASIC, a modular development system designed for the creation of video games and animations that runs on Windows.

ethosBASIC is based on the BASIC computer language platform with enhanced functionality (based on DirectX® technology) that can handle animations, graphics, sounds, and music flexibly.

The user interface is based on the familiar Windows editor, and consists of special function windows for handling the databanks of special effects, debugging, or editing the code and the animations.

http://www.ethosbasic.com

The Bookshelf

C# Cookbook, by Stephen Teilhet and Jay Hilyard (O'Reilly, 2003) is a collection of solutions and practical examples for the C# language. Teilhet and Hilyard teach by example, modeling good programming practice. The 200-plus recipes in the book, which cover performance, versioning, readability, maintainability, logic, reliability, and other topics, are in accordance with best practices. Most "recipes" contain a complete, documented code sample showing readers how to solve a specific problem, including a discussion of how and why the underlying technology works, and offering alternatives, limitations, and other considerations where appropriate.

Topics include performing numeric and data type con-versions, using and comparing strings and characters, creating and maintaining structures and classes, throwing and handling exceptions, and implementing diagnostic code in applications. You'll also find information regarding the use of delegates and events, working with regular expressions, using and extending .NET collection classes, managing files and directories, creating data structures and algorithms, maximizing security, and working with XML documents.

Organized by the types of problems likely to be encountered as developers improve their expertise in C#, the coding solutions in the C# Cookbook can be used, adapted, and modified to accommodate your application goals.

http://www.oreilly.com/catalog/csharpckbk

Send your New Stuff to Theresa at newstuff@microsoft.com.

Theresa W. Carey is a freelance writer who lives in California. Her byline has appeared in Barron's and PC World.


Web Q&amp;A: Request Timeouts, Byte Array Conversion, and More

Web Q&A
Request Timeouts, Byte Array Conversion, and More
Edited by Nancy Michell

Q I have an ASP.NET Web service and I continually get a ThreadAbortException. Is it possible that this is happening because I'm taking too long to read data from a database and the ASP timeout has been reached?

A Yes. ASP.NET has a class called the RequestTimeoutManager whose job is to enforce the timeouts. At the beginning of the request, ASP.NET uses the current HttpContext object to create a RequestTimeoutEntry for each request, adding it to a list, and then removing it when the request has finished processing.

However, if the request is in the list and it is scheduled to time out, then the runtime calls Thread.Abort to abort the thread executing the request. You can see this in the function TimeoutIfNeeded if you are looking at the Microsoft® intermediate language in ILDASM for the private System.Web.RequestTimeoutManager.RequestTimeoutEntry class. Depending on how long you want to allow for your database query, it might make sense to increase the timeout for that page so that you don't hit the limit.

Q How can I convert a byte array into usable info in VBScript?

A This question is asked rather frequently. Though VBScript has no ability to manipulate a byte array, OLE Automation does. If you pass the data to the CStr method, that should call OLE Automation's "convert this thing to a string" method, which will turn an arbitrary byte array "thing" into a Unicode string. You can then extract the bytes with the LenB, InstrB, and similar methods. It's a clunky solution, but it will work.

A better solution would probably be to either use a language that supports the data type you want or use an object model that returns the data type you want, but you don't always have those choices available.

Q What exactly causes additional instances (additional processes) of iexplore.exe to run with Microsoft Internet Explorer 6.0? On some machines I only get one instance of Internet Explorer listed in Task Manager no matter how many windows are open. On other machines, when I look at Task Manager I find many instances.

I have a related question as well. I am having difficulty getting the following line of code to work:

top.window.open("http://intranet", "name");
I am using Internet Explorer 6.0 and have much more than 32MB free. Any idea why this doesn't work? I just want to open a browser window in a new process.

A The most common reason for the failure of code like yours is popup-blocking applications. If any of these apps were installed on the target machine, that would cause this problem.

As far as how a new process gets started, if you open new instances of Internet Explorer using either the Start menu, the quick launch, or some other method that runs iexplore.exe, then in reality a new process is being launched (see Figure 1 for an example).

Figure 1 Launching a New Process
Figure 1 Launching a New Process

If you open a new Internet Explorer window from within an existing one using Ctrl+N, the window opens in the same process. In that case you are instantiating a window, and you'll only see one iexplore process running. When you launch an Internet Explorer window from an Internet Explorer process, there is still only one process running, regardless of the number of child windows. When you create a new process in Windows®, you create a new instance of an application that runs in its own main thread, gets allocated its own memory, and so forth.

An external application launches Internet Explorer by creating a new process. When you click on, say, a hyperlink in an MSN® Messenger window, a new Internet Explorer process is launched (which is a new instance of iexplore.exe). Since the HTTP protocol is registered with the shell to launch the default browser, that app is just ShellExecuting the href of the link, or even better (and more secure), passing the href as a parameter to an invocation of iexplore.exe (via CreateProcessEx or some other process-creation API).

Internet Explorer used to offer an option called "Launch Browser Windows in a Separate Process" but it's not available in newer versions of Internet Explorer, as the document at "Launch Browser Windows in a Separate Process" Setting Is Not Available in Internet Explorer 5.01 or Internet Explorer 5.5 explains.

Q I have some Microsoft Installer (MSI) packages and MSP patch packages in an IIS virtual directory. When I try to open these packages in Internet Explorer, the MSI packages open fine, but I get a "Page not found" error when I try to open MSP packages, even if I register the .msp extension with a MIME type of IIS. Do I need to add or modify any settings in IIS 6.0 or Internet Explorer 6.0 to fix this?

A The only way to get IIS to do anything with the MSP file is to provide a mapping for the .msp extension, which is the way IIS handles requests for .asp files with asp.dll. Otherwise, you'll have the option to choose how you want the client (Internet Explorer) to act on the file. You probably don't really want to execute the MSP on the server, unless you intend to install an application on the server when a user browses to an MSP file. In the case of the MSI file, it should be executing on the client; when the user browses to somefile.msi they are prompted with a dialog to Open, Save As, or Cancel. If they choose Open, Internet Explorer will download the file to the user's Temporary Internet Files folder and then execute the MSI package. Choosing Save As will invoke the same download process, but the user will choose the folder they want to save the download into; it will not be executed. For more information see MIME Type Handling in Microsoft Internet Explorer.

Q When I select the "Open link in a new window" option within Internet Explorer, I want the new window maximized. I tried setting this registry key, but had no luck:

HKCU\Software\Microsoft\Internet Explorer\Document Windows\Maximized=yes
The following registry entry works, but it's equivalent to pressing F11—not what I want:
HKCU\Software\Microsoft\Internet Explorer\Main\Full Screen=yes

A The only place you commonly see the "Maximized" registry value checked in code is during CD setup for autorun. By default, Internet Explorer will use the currently opened window size when launching a new window. Of course, this means it will also use the last valid window size for the Internet Explorer process. So if you open Internet Explorer, then open a new window, maximize that new window, close the old window, then close your maximized window, Internet Explorer will load maximized. This works for the main window but does not save the maximized state when you right-click on a link and choose to open in a new window.

Try this: set up a Web page that has hyperlinks on it. With no other iexplore.exe processes running, call iexplore.exe -k from the Start | Run dialog (this will load Internet Explorer in kiosk mode). Right-click on a shortcut and open the link; it should load in a new full-screen window. Close both the new window and the full screen window with Alt+F4. Now load Internet Explorer using the desktop icon. It should load full screen, as should the hyperlinked pages.

Q Are there any limitations on DataSet size in ADO.NET?

A There is no arbitrary limitation on the DataSet size. However, it was optimized for small rowsets, in the order of a few dozen to a few hundred rows. It's not recommended that you store 50,000 rows in your DataSet, for example, as performance will degrade more than linearly. Particularly for Web applications and Web services, it's best to keep the DataSet size as small as you can. However, what is reasonable performance for one application may be out of the question for another, so you should definitely test the likely usage scenarios before making any individual decisions.

Q A client has placed his Visual Basic® 6.0 COM+ components on a Network Load Balanced app server because these servers also run an IIS Web app. On the rich client (also written with Visual Basic 6.0), they configure the "stubs" to call these components via the virtual IP address of the load-balanced server. Will this work? The client is encountering performance problems on his WAN.

A It looks like the client is using DCOM between client and server. DCOM is a connection-oriented protocol and cannot be properly load balanced using Network Load Balancing. There will be load balancing when a client first connects, but after that the client requests will always go back to the same physical server. DCOM caches the actual IP address of the target machine. Therefore if the server should fail, the client will not automatically failover to a new server unless there is client-side code to detect the failure and reestablish the connection. New servers added to the server farm will not get any load at all until clients disconnect and reconnect or until new clients connect. Therefore, servers added at a later time will end up being very lightly loaded. When there are many clients and few servers you should see a reasonable distribution of load between the servers. But once the clients are up and running, that load will never be redistributed, even when a server is overutilized.

.NET Remoting over TCP/IP works in a similar manner. The real way to observe load balancing is to see what happens when one of the load balancing servers goes down. In the scenario you describe, it should cause creation of their components to fail.

As an alternative, you can load balance COM+ components using Application Center 2000; SP2 introduces support for the Microsoft .NET Framework. They can be installed on top of Windows Server™ 2003 (downloads are available at http://www.microsoft.com/applicationcenter/downloads).

Q I am trying to recover an XML string that's generated in a stored procedure through a FOR XML EXPLICIT clause, using the SqlCommand object. However, the string is truncated after approximately 2000 characters rather than the expected 9000. I used the ExecuteScalar method, then tried to use a DataSet filled using the SqlDataAdapter object, but neither worked as I expected.

A When you select XML output through regular ExecuteXxx methods, the server returns the data in chunks of up to 2048 nchars each. Since ExecuteScalar returns only the first value, you will only get the first 2048 nchars from this output.

If you were reading this output through ExecuteDataReader, you would have to keep reading and concatenating the first column values in each row in order to get the whole output. ExecuteXmlReader, on the other hand, processes these chunks itself and returns the whole output at once.

Got a question? Send questions and comments to webqa@microsoft.com.

Thanks to the following Microsoft developers for their technical expertise: Olivier Arbey, Aaron Barth, Pablo Castro, Fabricio Catae, Sushil Chordia, Tom Cox, Jeff Davis, Denny Dayton, Ira Deshpande (Volt), Lale Divringi, Nick Duane, Omar Fadl, Naphtaly Friedman, Brent Goodpaster, Paul Harper, Ron Howe, Michael Joseph, Joonas Karjalainen, Marcin Kojlo, Eric Lippert, Tony Muniz, Michael Nelson, Jaret Russell (Volt), Joshua Schroeder, Malcolm Stewart, Charles Torre, Stephen Toub, Dino Viehland, Chek Yoon Wong, and Steven Wort


Data Points: Saving Parent-child Data in a Multitiered App Using ADO.NET

Data Points
Saving Parent-child Data in a Multitiered App Using ADO.NET
John Papa

Code download available at: DataPoints0405.exe (146KB)
Contents
ASP.NET Parent-child Web Form
Marshaling Data Changes
The Transaction and Sequence
Details
Wrapping It Up

In this month's column I will focus on some data management techniques commonly required in enterprise applications. These include saving parent-child data in a multitier application using ADO.NET, ADO.NET transactions, merge techniques, and a number of other ADO.NET features. I'll also discuss the best process for incorporating consistent and informative exception-handling techniques and debugging strategies into your multitier app.

One of the most robust features of the ADO.NET DataSet is that you can store multiple related rowsets using DataTable and DataRelation objects as well as the original values and changes to them. Through the DataSet, an application can have a parent-child relationship represented by a single data store. This allows the application to store changes locally to both the parent and the child rowsets in the DataSet. The DataSet's changes can be cached locally and then later saved to a database all at once. For example, through a front-end application, a user could make changes to data, including deletions, updates, the addition of new records, and the addition of detail records. The changes would be cached locally and then could either be canceled entirely or sent back to the database all in one transaction.

In the Data Points column in the July 2003 issue of MSDN® Magazine I focused on managing batch inserts from a DataSet, detailing how to maintain the integrity of the parent-child relationship and IDENTITY values. This month I'll demonstrate how to manage inserts, updates, and deletes from both parent and child tables all in one batch. Using the Orders and [Order Details] tables of the SQL Server™ Northwind database, I'll walk through a sample application that can manipulate several Order and Order Detail records, storing changes in a relational ADO.NET DataSet that contains DataTable objects. The DataSet will be stored locally and only its changes will be marshaled to a business services object. I will discuss how to enforce and relax constraints on the DataSet, the SqlDataAdapter's RowUpdated event handler, how to implement transactions via ADO.NET, and how to merge DataSets together. I will also discuss one of the most critical factors in saving all types of changes from a hierarchical DataSet: the sequence in which the action queries are executed against the database.

ASP.NET Parent-child Web Form

Figure 1 shows the application that I will be referring to throughout this column. It loads all orders for a given customer (the application assumes the customer is always called "AROUT") and allows four basic functions to be performed on the local DataSet containing Orders and their Order Details: viewing, adding, deleting, and updating. The application also allows the user to either save all the DataSet changes to the database or cancel them.

Figure 1 Load Orders
Figure 1 Load Orders

The UI should be intuitive as there are links and buttons labeling the main functions. These links and buttons all hook into a series of events that are defined in the WebForm1.aspx file. The following code snippet shows the properties of the Orders DataGrid—specifically, the event handlers that will be fired when the listed events occur:

<asp:DataGrid id="grdOrder" Runat="server"
  •••
  OnPageIndexChanged="grdOrder_OnPageIndexChanged"
  OnItemCommand="grdOrder_OnItemCommand"
  OnEditCommand="grdOrder_OnEditCommand"
  OnUpdateCommand="grdOrder_OnUpdateCommand"
  OnCancelCommand="grdOrder_OnCancelCommand"
  >

The OnItemCommand event fires when the user clicks the Delete or View link in any Order row. In the same manner, the OnEditCommand, OnUpdateCommand, and OnCancelCommand events fire when the user clicks the Edit, Save, or Cancel link, respectively. Both the grdOrder and grdOrderDetail grids include various events, which are linked to corresponding event handlers located in the codebehind file called WebForm1.aspx.

The grdOrder_OnUpdateCommand event handler, which fires when the user has finished, grabs the values in the editable controls for the current row. The code that follows shows how this event handler finds the controls using the FindControl method:

Label lblOrderID =
   (Label)e.Item.FindControl("lblOrderID");
int nOrderID =
   Convert.ToInt32(lblOrderID.Text);
TextBox txtOrderDate = (TextBox)e.Item.FindControl("txtOrderDate");
DateTime dtOrderDate = Convert.ToDateTime(txtOrderDate.Text);
TextBox txtCity = (TextBox)e.Item.FindControl("txtCity");
string sCity = txtCity.Text;
TextBox txtCountry = (TextBox)e.Item.FindControl("txtCountry");
string sCountry = txtCountry.Text;
I used this method because the controls are dynamically created based upon which row is being edited at the time. The lblOrderID Label is located first and is cast to a Label control. I do this to retrieve the primary key of the Orders DataTable, which will be used to locate the current row in the DataTable. The remaining values are then retrieved in succession.

Once the values are retrieved and stored in local variables, the corresponding DataRow is located in the Orders DataTable by passing the primary key of the DataTable to the Find method. Then the edited values are set to the DataRow in the DataSet:

DataRow oRow = oDs.Tables["Orders"].Rows.Find(nOrderID);
oRow["OrderDate"] = dtOrderDate;
oRow["ShipCity"] = sCity;
oRow["ShipCountry"] = sCountry;
Afterwards, the grids are bound once again to the DataSet and the applied changes are shown to the user in the form. After this code executes, the current value of the OrderDate DataColumn for this DataRow is set to the new value. This feature of the DataSet allows both an original and a current value to be stored in a DataSet. The DataSet can compare these values and retrieve all rows that have any columns with different original and current values using the DataSet's or DataTable's GetChanges method or the DataTable object's Select method. Either of these methods can be used to retrieve changed rows which can then be marshaled back to a business tier and then on to the database.

The following code is a snippet from the DeleteOrder section of the grdOrder_OnItemCommand event handler:

nOrderID =
    Convert.ToInt32(((Label)e.Item.FindControl("lblOrderID")).Text);
DataRow oDelRow = oDs.Tables["Orders"].Rows.Find(nOrderID);
oDelRow.Delete();
It similarly grabs the OrderID value for the current row (which is being deleted), locates the corresponding DataRow in the Orders DataTable, and deletes the row from the DataSet. The Delete method does not actually remove the row from the DataTable. When a row is deleted, the current values of the columns of the DataRow are wiped out, leaving only the original values.

When a row is added to the DataSet, the original values of the columns of the DataRow do not exist. The new values are all represented by the current values of the columns. The following code sample is a snippet of what is executed when the user adds an order to the DataSet:

//-- Create an Order table DataRow
DataRow oRow = oDs.Tables["Orders"].NewRow();
//?- Set the values for the new row
oRow["CustomerID"] = sCustomerID;
oRow["OrderDate"] = dtOrderDate_New;
oRow["ShipCity"] = sCity_New;
oRow["ShipCountry"] = sCountry_New;
//-- Add the new row to the Orders DataTable
oDs.Tables["Orders"].Rows.Add(oRow);
The NewRow method creates an empty new row from the Orders DataTable. The values for the row are set and the new DataRow object is added to the Orders DataTable. Notice that the OrderID DataColumn's value is never set explicitly in this code sample. Because the Northwind Orders table's OrderID column has an IDENTITY constraint, it will automatically be assigned an incremental value. Since the application does not know what value to set due to concurrency issues and the disconnected nature of scalable applications, an alternative is available in ADO.NET. The DataColumn has a series of properties that, when initiated, work together to increment the column automatically. The following code is from the GetData method of the Order class:
oDa.Fill(oDs, "Orders");
//-- Set the auto incrementing aspects
oDs.Tables["Orders"].Constraints.Add("PK_Orders",
    oDs.Tables["Orders"].Columns["OrderID"], true);
oDs.Tables["Orders"].Columns["OrderID"].AutoIncrement = true;
oDs.Tables["Orders"].Columns["OrderID"].AutoIncrementSeed = -1;
oDs.Tables["Orders"].Columns["OrderID"].AutoIncrementStep = -1;

After the Orders DataTable is filled by a SqlDataAdapter, a primary key constraint is placed on the OrderID DataColumn. Keep in mind that this constraint is in ADO.NET, not on the actual table in the underlying database. After the primary key constraint is established, the AutoIncrement property is set to true for the OrderID DataColumn. The first new row that is added to the DataTable will have its value automatically set to -1. All subsequent rows added before either RejectChanges or AcceptChanges are called will increase by the AutoIncrementStep value. The technique shown starts the new OrderID values at -1 and increases them by -1. Thus new rows will have the values of -1, -2, -3, and so on. Initializing them in a negative range ensures that the values will not collide with actual OrderID values in the database. The application does not care what the OrderID's value is as long as it can be used to uniquely identify a pending row and all of its related order detail records. Thus, the OrderID values that are generated by the DataColumn for new pending rows will not be the same as for any existing row in the database.

There are other controls besides the DataGrids in this application, such as buttons and labels. The button controls have event handlers that are fired when the button controls are clicked. There is a button for saving all changes to the database and one for canceling all local changes and refreshing from the database. There is also a Label server control called lblErrorMessage that is used to display any error messages thrown by the application in either the presentation or the business tier. Solid exception-handling techniques can be invaluable in finding and fixing the causes of errors.

In this application I have incorporated an exception-handling technique that creates and throws a CustomException object around the application's tiers. (This technique and the CustomException object are explained in detail in my March 2004 column.)

If an exception occurs, it is trapped and a custom exception is created and thrown back up the call stack. Once it reaches the originating event handler, the exception is then logged for posterity and displayed on the Web page in the lblErrorMessage Label. The message and its call stack are displayed, so it is easy to identify and pinpoint where the exception occurred. Figure 2 shows an example of an exception that this code reported. It occurred when I clicked the "Reload from Database" button after I removed EXECUTE permissions from the SqlTestUser SQL Login on the prGet_Order stored procedure.

Figure 2 Execute Permission Error
Figure 2 Execute Permission Error

In order to examine the form's codebehind and the business classes (that could comprise the middle tier), I will first make several changes to the Orders and Order Details rowsets via their respective ASP.NET DataGrid server controls. To truly test the application, a series of tests with a thorough quality assurance process would be necessary. Since I have limited space here, I will throw one test scenario at the application by entering a few inserts, deletes, and updates for Orders and Order Details. I will then save the entire batch to the database in a single transaction by clicking the Save All Changes button. Now let's walk through the code that executes after the Save All Changes button is clicked.

Marshaling Data Changes

Before saving changes to the DataSet, it is wise to make sure there actually are changes to save. Thus, as soon as the user tries to save the changes, the code checks the Boolean value of the DataSet's HasChanges method. Now I'm ready to send the changes back to the middle tier so I create an instance of the Order class and call its SaveData method. This method accepts the user ID, password, and a DataSet to save as parameters. While the application would work if the entire DataSet was sent to the business tier, there is no need to pass any more than is necessary. The DataSet exposes a GetChanges method which returns another DataSet containing the same DataTable and DataRelation schema as the original DataSet. Since only the rows that were added, edited, or deleted will be contained within the DataSet, this DataSet is much smaller. This is especially important if the DataSet contains DataTable objects that have hundreds of rows, only a handful of which were actually modified in previous transactions.

This does pose a minor problem that will have to be dealt with after the changes have been applied to the database. The problem is that now there are two DataSet objects with the same schema: the original DataSet and the delta DataSet, which contains all rows that have changes from their original values. Keep in mind that the added rows in the Orders DataTable have artificial OrderID values such as -1 and -2. After the new order rows have been saved to the database, the new values will be returned to the delta DataSet in the Order class. I will tackle the techniques of merging these DataSet objects and their data back together after I review the actual saving of the data in the business tier.

The Transaction and Sequence

The Order class's SaveData method accepts the delta DataSet and promptly establishes a connection to the database. Before the connection is opened, oDa_Handle_RowUpdated is added as an event handler for the SqlDataAdapter's RowUpdated event, which is invoked when a row is modified by the SqlDataAdapter. The event handler is critical to the success of this application as it is used to assist in the preparation and synchronization of the delta DataSet with its originating DataSet back in the WebForm1.aspx.cs codebehind. When an Order row is modified via the SqlDataAdapter, this event will fire and evaluate if the modification was an insert or a delete. If it was either of those, it will set the status of the event to UpdateStatus.SkipCurrentRow. This effectively prevents the AcceptChanges method from affecting this row. Thus the original value of the OrderID would still be the artificial ID of -1, for example, and the current value for the OrderID would be the next IDENTITY value that the database assigned (such as 17004). If this row is not skipped, then the values would both be set to the new value. Why skip the row? The row is left with different original and current values so that when I merge the delta DataSet with the originating DataSet, they will sync with each other properly, using the original values of the OrderID DataColumns. The OrderDetail class's version of this event is even more critical, as I will demonstrate in the following code:

private void oDa_Handle_RowUpdated(object o, SqlRowUpdatedEventArgs e)
{
    if (e.StatementType == StatementType.Insert ||
        e.StatementType == StatementType.Delete)
    {
        e.Status = UpdateStatus.SkipCurrentRow;
    }
}

Since many changes can be made to the database separately (updates, inserts, and deletes for both orders and order details) a transaction is necessary to make sure that either all changes are applied or that none are. In the SaveData method (the core of which is shown in Figure 3) the BeginTransaction method of the SqlConnection object is invoked. This begins a transaction on the existing SqlConnection object and on the underlying database. A SqlTransaction object is created and stored to a local variable. This oTrn variable can then be used to commit or roll back the transaction as needed. It can also be used to retrieve a reference to the SqlConnection that it was started on.

Figure 3 Order's SaveData Method (Close Figure 3)
oDa.RowUpdated += new SqlRowUpdatedEventHandler(oDa_Handle_RowUpdated);

//=============================================================
//--Set up the Connection
//=============================================================
oCn = new SqlConnection(sConnString);

//=============================================================
//-- Open the Connection and create the Transaction
//=============================================================
oCn.Open();
oTrn = oCn.BeginTransaction();

//=============================================================
//-- Set up the INSERT Command
//=============================================================
sProcName = "prInsert_Order";
oInsCmd = new SqlCommand(sProcName, oCn, oTrn);
oInsCmd.CommandType = CommandType.StoredProcedure;
oInsCmd.Parameters.Add(new SqlParameter("@sCustomerID",
                       SqlDbType.NChar, 5, "CustomerID"));
oInsCmd.Parameters.Add(new SqlParameter("@dtOrderDate",
                       SqlDbType.DateTime, 8,"OrderDate"));
oInsCmd.Parameters.Add(new SqlParameter("@sShipCity",
                       SqlDbType.NVarChar, 30, "ShipCity"));
oInsCmd.Parameters.Add(new SqlParameter("@sShipCountry",
                       SqlDbType.NVarChar, 30, "ShipCountry"));
oInsCmd.Parameters.Add(new SqlParameter("@nNewID", SqlDbType.Int,
            0, ParameterDirection.Output, false, 0, 0, "OrderID",
            DataRowVersion.Default, null));
oInsCmd.UpdatedRowSource = UpdateRowSource.OutputParameters;
oDa.InsertCommand = oInsCmd;

//=============================================================
//-- Set up the UPDATE Command
//=============================================================
sProcName = "prUpdate_Order";
oUpdCmd = new SqlCommand(sProcName, oCn, oTrn);
oUpdCmd.CommandType = CommandType.StoredProcedure;
oUpdCmd.Parameters.Add(new SqlParameter("@nOrderID", SqlDbType.Int, 4,
                                        "OrderID"));
oUpdCmd.Parameters.Add(new SqlParameter("@dtOrderDate",
                       SqlDbType.DateTime, 8,"OrderDate"));
oUpdCmd.Parameters.Add(new SqlParameter("@sShipCity",
                       SqlDbType.NVarChar, 30, "ShipCity"));
oUpdCmd.Parameters.Add(new SqlParameter("@sShipCountry",
                       SqlDbType.NVarChar, 30, "ShipCountry"));
oDa.UpdateCommand = oUpdCmd;

//=============================================================
//-- Set up the DELETE Command
//=============================================================
sProcName = "prDelete_Order";
oDelCmd = new SqlCommand(sProcName, oCn, oTrn);
oDelCmd.CommandType = CommandType.StoredProcedure;
oDelCmd.Parameters.Add(new SqlParameter("@nOrderID", SqlDbType.Int, 4,
                                        "OrderID"));
oDa.DeleteCommand = oDelCmd;

//=============================================================
//-- Update the Order record(s)
//=============================================================
oDa.Update(oDs.Tables["Orders"].Select("", "",
    DataViewRowState.ModifiedCurrent));

//=============================================================
//-- Add the Order record(s)
//=============================================================
oDa.Update(oDs.Tables["Orders"].Select("", "", DataViewRowState.Added));

//=============================================================
//-- Update, Insert and Delete the Order Detail records
//=============================================================
OrderDetail oOD = new OrderDetail();
oOD.SaveData(sUserID, sPassword, oTrn, oDs);

//=============================================================
//-- Delete the Order record(s)
//=============================================================
oDa.Update(oDs.Tables["Orders"].Select("", "", DataViewRowState.Deleted));

oTrn.Commit();
oCn.Close();

The SqlCommand objects are then created and their SqlParameter objects set appropriately for the insert, update, and delete types of action queries. The SqlCommand objects are created using the three-argument constructor that in this method accepts the name of stored procedure, the SqlConnection object to use, and the SqlTransaction object to enlist in its transaction scope. Notice that in the code to create the InsertCommand (called oInsCmd) the oInsCmd object's UpdatedRowSource property is set to UpdateRowSource.OutputParameters. In this application, this will be used to pass the new OrderID value back to the SqlCommand object from the database, which is how the OrderID DataColumn gets its original and current values to be different (in other words, -1 and 17004). This code (along with the code directly preceding it that creates the output parameter directly) is used to grab the new OrderID from the database by way of the stored procedure's output parameter. Then the value is taken and set to the OrderID column in the DataTable.

Now that the SqlConnection is opened, the SqlTransaction is initiated and the series of SqlCommand objects are established, the SaveData method begins to make the changes to the database. The sequence of the action queries is important since I'm dealing with a relational data structure of orders and their order detail records. For example, it would not be prudent to insert an [Order Details] record into the database without a corresponding and valid Orders table record. A good rule of thumb for sequencing your action queries via SqlCommand objects is as follows:

  1. Update parent rows
  2. Insert parent rows
  3. Update child rows
  4. Insert child rows
  5. Delete child rows
  6. Delete parent rows

I found that when inserting the parent rows prior to updating them, the DataViewRowState of the new rows was changed from Added to ModifiedCurrent. This happened because the OrderID value was changed by the insert SqlCommand (oInsCmd), thus changing the row's state to be ModifiedCurrent. So why is this important? I found that new rows being created were then being marked as modified, which caused them to be issued to the database a second time via the update SqlCommand (oUpdCmd). There are a couple of ways to avoid this, but the simplest that I found was to perform updates before the inserts.

The SqlDataAdapter pushes the changes to the database via the Update method. It can accept a variety of arguments through its polymorphic nature, but in this code it accepts an array of DataRow objects from the Select method of the Orders DataTable. The Select method is used here to retrieve all DataRow objects that match the type of SQL action query that will be issued. To issue updates to the database, the DataRows returned from the Select method will have a DataViewRowState of ModifedCurrent. The deletes and inserts will have a DataViewRowState of Deleted and Added, respectively. Once all of the changes are issued to the database, the SqlTransaction object's Commit method is called and all of the changes are committed to the database.

Details

Before the transaction can be committed, any changes to the OrderDetail DataTable have to be sent to the database, too. After the parent rows were updated and inserted into the database, an instance of the OrderDetail class is created and its SaveData method is invoked. Figure 4 shows the core of the SaveData method of the OrderDetail class. Note that it turns off the enforcement of constraints on the DataSet. This effectively tells the DataSet that an OrderID in the parent and the child DataTable objects are allowed to be out of sync. This is turned off so that the oDa_Handle_RowUpdated event handler for the OrderDetail class can set the OrderID values appropriately, as I will demonstrate. Once the constraints are off, the series of updates, inserts, and deletes are issued in sequence. When any rows in the delta OrderDetails DataTable are sent to the database, the event handler is invoked (see Figure 5).

Figure 5 RowUpdated Event Handler (Close Figure 5)
//-- See if the change is an insert
if (e.StatementType == StatementType.Insert)
{
    //--Prevent AcceptChanges from being called on this row
    e.Status = UpdateStatus.SkipCurrentRow;
    //--Get the current primary key value
    int nCurrentKey = (int)e.Row["OrderID"];
    //--Get the pseudo key value from the parent and set it as the child
    //  row's primary key
    e.Row["OrderID"] =
        e.Row.GetParentRow("OrdersToOrderDetails")["OrderID",
        DataRowVersion.Original];
    //--Make the Order ID the pseudo ID (-1) for both Original and
    //  Current value.
    e.Row.AcceptChanges();
    //--Reset the primary key value in the foreign key column of the
    // child row
    //--Makes the current value the value in the database.
    e.Row["OrderID"] = nCurrentKey;
}
Figure 4 OrderDetail's SaveData Method (Close Figure 4)
//=============================================================
//-- Turn constraints off because we may have to synchronize
//-- parent and child rows in the update row handler.
//=============================================================
oDs.EnforceConstraints = false;
//=============================================================
//-- Update the Order Details record(s)
//=============================================================
oDa.Update(oDs.Tables["OrderDetails"].Select("", "",
           DataViewRowState.ModifiedCurrent));

//=============================================================
//-- Add the Order Details record(s)
//=============================================================
oDa.Update(oDs.Tables["OrderDetails"].Select("", "",
           DataViewRowState.Added));

//=============================================================
//-- Delete the Order Details record(s)
//=============================================================
oDa.Update(oDs.Tables["OrderDetails"].Select("", "",
           DataViewRowState.Deleted));
//=============================================================
//-- Turn constraints back on
//=============================================================
oDs.EnforceConstraints = true;

The code in Figure 5 prevents the current row from having AcceptChanges called on it, thus keeping the DataRow's original and current values separate. If the row is not skipped, then the original value would be set to be the same as the Current value (if AcceptChanges was allowed to proceed). After the row is skipped, the current value of the OrderID is retrieved and stored to a local variable. This is the OrderID value that was created by the database. This OrderID value is then used to travel up to the parent DataTable (Orders) and retrieve the original value of the OrderID. For example, the nCurrentKey variable would be set to a value such as 17004 and then this would be used to find the DataRow in the parent DataTable that has its current value as 17004 as well.

Once the row is found using the GetParentRow method, the original value of the linked OrderID DataColumn in the Orders DataTable is retrieved. This value (for example, -1, -2, and so forth) is set to be the child DataTable's OrderID current value. The AcceptChanges method is then issued so that the current value is copied to the original value. Now the original and current values are both set to -1. Finally, the current value of the DataRow's OrderID column is set to the database's value (17004). Basically, all of the fuss in this event handler exists to toggle the original and current values of the OrderID DataColumn. This allows the constraints to be enforced again as the child records are now in sync with the relationship established between it and its parent DataTable.

Once the database changes are complete, the SqlTransaction is committed and the delta DataSet is returned to the WebForm1.aspx.cs codebehind event that called it originally. What is important to remember is that the delta DataSet contains the changed records including the new and original OrderID values.

The following code snippet shows the retrieval of the delta DataSet (oDsDelta) returned from SaveData. A generic function (Common.DisplayDataSet) is used to display the contents of both the original and delta DataSets before the merge to the output window when it is in debug mode:

DataSet oDsDelta = oOrder.SaveData(oDs.GetChanges());
oOrder = null;
Common.DisplayDataSet(oDs, "*** BEFORE MERGE - ORIGINAL ***");
Common.DisplayDataSet(oDsDelta, "*** BEFORE MERGE - DELTA ***");
oDs.EnforceConstraints = false;
oDs.Merge(oDsDelta.Tables["Orders"]);
oDs.Merge(oDsDelta.Tables["OrderDetails"]);
oDs.AcceptChanges();
oDs.EnforceConstraints = true;
This function is a crude technique but is valuable in locating problems as it is easy to see the contents of the entire delta DataSet and its originating DataSet to compare them.

The originating DataSet (oDs) then turns off its constraints so that any violations that may have occurred during the merge are completely ignored. Then the two DataSet objects are merged into the originating DataSet using the original values of the OrderID DataColumn objects in each respective DataSet.

Now that all of the data is merged into a single DataSet, the OrderID values should be set to the new values (such as 17004). Because of this, the AcceptChanges method is called on the oDs DataSet and the constraints are enforced once again. Finally, each of the ASP.NET DataGrid objects are bound again and the changes that were made to the records are displayed to the user.

Wrapping It Up

When you're using a DataSet that represents a parent-child relationship via two related DataTable objects, the child table's foreign key that references the parent table's primary key must stay in sync with its values to maintain the relationship. This is fairly straightforward to manage, but it gets a bit more involved when the parent table's primary key is an auto-incrementing column. When a row is added to the parent DataTable in the DataSet, a new ID value must be generated. Then, the value must be propagated to the child DataTable. Since all of this happens locally to the DataSet, when the changes are sent back to the database, a new ID value must be retrieved and synchronized with the DataSet's DataTable objects to maintain the DataSet's parent-child relationships. Also, assuming that only the changes in the DataSet are marshaled to the database via a middle tier, possibly in a physically separate server, there is the matter of synchronizing the original DataSet with the marshaled DataSet. ADO.NET makes it possible to implement this process across multiple classes, and even multiple tiers, by using events such as SqlDataAdapter RowUpdated.

In this month's column I focused on how to implement a parent-child series of data modifications through ASP.NET and ADO.NET, taking into account marshaling only modified data, instituting atomic transactions, retrieving IDENTITY values, and keeping the parent-child DataTables in sync. The business tier could be set up as a separate assembly containing Order and OrderDetail classes. For the sake of simplicity I have included all of the code in the same application, but in an enterprise application these classes could be split into their own projects, built into separate assemblies and referenced by the Web Form project. The sample application contains more exception handling and debugging code. You can download it and a readme instruction file for setting it up on the MSDN Magazine Web site.

Send your questions and comments for John to mmdata@microsoft.com.

John Papa is a baseball fanatic who spends most of his summer nights rooting for the Yankees with his two little girls, wife, and faithful dog, Kadi. He has authored several books on ADO, XML, and SQL Server and can often be found speaking at industry conferences such as VSLive.


Cutting Edge: Dress Your Controls for Success with ASP.NET 1.1 Themes

Cutting Edge
Dress Your Controls for Success with ASP.NET 1.1 Themes
Dino Esposito

Code download available at: CuttingEdge0405.exe (138KB)
Contents
The "Theme" of Themes
A Quick Tour of Themes in ASP.NET 2.0
Designing Themes in ASP.NET 1.1
Creating Themes
.NET Code Generation
The ThemeManager Class
Issues and Gripes

Themes and skins are relatively new concepts in the world of Windows®. Windows XP was the first version of Windows to allow users to change the look of their system components. The Microsoft® .NET Framework 1.x does not natively support themes, nor do ASP.NET or Windows Forms applications. In Visual Studio® 2005, on the other hand, themes officially join the arsenal of programming tools, at least for Web developers. In ASP.NET 2.0, page developers can select the page theme using a directive, and all controls will render according to the settings in the theme. The theme in use can also be replaced programmatically at run time. When this happens, the page will reflect the new settings and redraw its controls accordingly.

Themes are a system feature that sit between the core code of the host system (the ASP.NET runtime) and the base implementation of individual components. Ideally, application developers should never need to deal with themes except to select one by name. When a system is built with themes in mind, components detect the current theme and reflect any settings in their UIs. The programmer is not involved in the process and everything takes place at a level below the application's code.

On the other hand, adding theme support to an existing framework not specifically designed to support themes can be a difficult challenge. In this column, I'll demonstrate one way to extend ASP.NET 1.1 controls to support theming. By the time you finish reading, you'll know how to create XML-driven theme files and apply their style attributes to all controls on a page, dressing your controls for success.

The "Theme" of Themes

A golden rule of Web usability states that Web pages on a site must have a consistent look. The use of cascading style sheets (CSS) helps fulfill this requirement, but it is not enough. A CSS file can dictate the appearance of some page elements such as body, anchors, headings, and input fields. ASP.NET controls need the same flexibility that CSS classes provide for visual styles. An ASP.NET theme is like a superset of a CSS file because it addresses all visual properties of the control including those properties that can't be addressed with a CSS style. For example, consider the Button server control. You can set borders, fonts, and colors for a button using either CSS or theme properties. However, the Button class also exposes properties such as CausesValidation and EnableViewState that you can't set using CSS. Those properties aren't visual properties, so it makes sense that they should not be set with a stylesheet. What about a DataGrid then? ItemStyle, ShowFooter, and PagerStyle are all visual properties. How can you set them through CSS?

Unlike Button, the DataGrid control doesn't map conveniently to any of the base HTML elements. This means that any CSS settings can address only a few of its properties (Gridlines, CellSpacing, font, colors) but not all the more abstract and wider set of visual properties supported by this rich control.

A theme is a collection of property settings that are applied to all controls simultaneously with little effort from the developer. The big benefit of themes is that once you have associated a page with a theme, all the controls that you drop on the page surface will automatically take the defined settings—at least at run time.

When are themes applied? This depends on the actual implementation. However, a best practice is to apply theme settings to controls immediately after their instantiation. As a result, each control is first created with the attributes set in the .aspx markup and then assigned the settings stored in the theme file. In this way, theme settings override corresponding markup settings. Unthemed properties retain the original value set in the .aspx file.

A Quick Tour of Themes in ASP.NET 2.0

Before I begin my implementation of themes for ASP.NET 1.1, let's take a quick tour of themes in the upcoming version. In the ASP.NET 2.0 Technology Preview, a theme is described by a .skin file that relies on other resources such as CSS files and images. These files are saved in a special location under the installation path of the .NET Framework 2.0 or under the Themes folder of the application. In the former case, you have global themes visible to all applications running on the machine. Themes stored in an application-specific folder are local to the application and not available outside of it. The path of global themes is: %SystemRoot%\Microsoft.NET\Framework\vx.x.xxxx\ASP.NETClientFiles\Themes. The actual name of the subdirectory labeled vx.x.xxxx changes according to the build of ASP.NET 2.0 that is in use. Under the folder named Themes you'll find multiple child paths, each identifying a distinct theme. The theme's main directory contains .css and .skin files. The name of the directory is the name of the theme. A theme is normally set at design time but, in some cases, can also be set or modified programmatically.

A theme contains skins and CSS styles defined for a variety of server controls. A skin is like a stylesheet for ASP.NET-specific properties. You can use classic CSS styles and skins together. Associating a theme with a page is very easy, as you can see here:

<%@Page Language="C#" Theme="SmokeAndGlass" %>
All you do is assign the name of the theme to the Theme attribute of the @Page directive. Bear in mind that the name of the theme must match the name of a subdirectory under the local or global path for themes. Themes can be set on a per-page basis or automatically assigned to all the pages in the site. To assign them to all pages, you create a setting in the web.config file.

To create a theme, you first create any optional CSS files you plan to use. This allows you to style HTML elements such as tables, anchors, input fields, and so forth. Then you create a skin file in which you define the settings you want to be applied to the ASP.NET server controls. The following code snippet is an ex-cerpt from one of the ASP.NET 2.0 standard themes—the SmokeAndGlass theme:

<asp:TextBox runat="server"
     BackColor="#FFFFFF" ForeColor="#585880"
     BorderStyle="Solid" BorderColor="#585880"
     BorderWidth="1pt" Font-Size="0.9em"
     Font-Names="Verdana" />
<asp:Button runat="server"
     Font-Bold="true" BorderColor="#585880"
     BorderWidth="1pt" ForeColor="#585880"
     BackColor="#F8F7F4" />
Once you select the SmokeAndGlass theme, any button and textbox controls will render as described (see Figure 1). Note that an ASP.NET 2.0 theme configures the visual properties of a control. Properties that influence the runtime behavior of the control should not be used.

Figure 1 SmokeAndGlass Color Scheme
Figure 1 SmokeAndGlass Color Scheme

The skin file can be written in a variety of ways. In ASP.NET 2.0, it contains chunks of ASPX markup; in my simple ASP.NET 1.1 implementation I'll discuss here, it is an XML file. In general, using ASPX markup simplifies the writing of themes because you can design your control in Visual Studio .NET and then just cut and paste the corresponding markup to the skin file. However, this requires a parser for the ASPX markup code, which is not publicly available to developers and which would need to be written by hand. For now, I'll express an ASP.NET 1.1 theme file as raw XML.

Designing Themes in ASP.NET 1.1

The major components of themes in ASP.NET 1.1 are an XML file to skin controls, an autogenerated proxy file created with a command-line tool, and an assembly that provides the base class for the proxy along with a couple of static methods that are used to initialize the theme engine.

As mentioned, a theme represents a collection of assignments in which the left part is a control property and the right part is a typed value. These settings are read from the theme file and are known only at run time. As it turns out, you have two basic ways to implement that—using reflection or via a dynamically generated theme object. Personally, I have a love-hate relationship with .NET reflection. I love the flexibility that reflection provides, as much as I dislike the messy code it forces you to write. Reflection-based code is often hard to read and maintain if you use it for purposes other than checking the metadata and the types of a given assembly. To set a property on a control using reflection, you must first get a PropertyDescriptor object for the property, then get a typed value for it, and make the assignment. However, consider the following snippet:

<button>
   <property>BackColor</property>
   <value>Color.Gainsboro</value>
</button>
You know the property name—BackColor—and a string that represents the value to assign. Using reflection, you must be able to transform "Color.Gainsboro" into an object of type Color. It is possible, but it is not trivial.

Let's explore the alternate approach—code generation. Code generation consists of generating a Visual Basic® or C# class that parses the theme file and produces instructions like the following:

ctl.BackColor = Color.Gainsboro;
Parsing is simpler and there's no runtime overhead because of code indirection and late binding. In the .NET Framework 1.x, proxy classes for Web services provide a good example of static code generation. When you add a Web reference to your project, Visual Studio .NET generates a source file and silently adds it to the project. When you compile the project, that file is compiled and merged with the resulting assembly. On the downside, you need to regenerate the class whenever the Web Services Description Language (WSDL) file changes.

A third possibility would be dynamic code generation. Dynamic code generation entails parsing and compiling the theme file at run time in much the same way as ASP.NET pages are processed by the runtime. This is how themes are implemented in ASP.NET 2.0 (an assembly for the referenced theme is generated on the first request and used for all subsequent requests). I'll show how to implement this mechanism in ASP.NET 1.x in a future column. For now, let's go for static code generation Web-services style.

To support themes in ASP.NET 1.1, you need to create an XML document that represents the skin of the various controls, write a command-line tool that creates a Visual Basic or C# source class out of the theme, and create a theme manager object to ensure that all controls in the page are properly skinned.

Creating Themes

A theme is an XML document with a schema that can be mapped to a DataSet object. The theme name is the DataSet name and each table stores the settings for a particular control. Figure 2 presents a sample theme that skins Button, TextBox, and DataGrid controls. You can save this file to any location on the server machine—for example, in a Themes folder below the root of the Web application.

Figure 2 Sample Theme (Close Figure 2)
<Flat>
    <Button>
        <Property>BackColor</Property>
        <Value>Color.WhiteSmoke</Value>
    </Button>
    <Button>
        <Property>BorderStyle</Property>
        <Value>BorderStyle.Solid</Value>
    </Button>
    <Button>
        <Property>BorderWidth</Property>
        <Value>Unit.Pixel(1)</Value>
    </Button>
    <Button>
        <Property>BorderColor</Property>
        <Value>Color.Gray</Value>
    </Button>
    <Button>
        <Property>Font.Name</Property>
        <Value>"Tahoma"</Value>
    </Button>
    <TextBox>
        <Property>Font.Name</Property>
        <Value>"Tahoma"</Value>
    </TextBox>
    <TextBox>
        <Property>ForeColor</Property>
        <Value>Color.Navy</Value>
    </TextBox>
    <TextBox>
        <Property>BackColor</Property>
        <Value>Color.WhiteSmoke</Value>
    </TextBox>
    <TextBox>
        <Property>BorderColor</Property>
        <Value>Color.Gray</Value>
    </TextBox>
    <TextBox>
        <Property>BorderStyle</Property>
        <Value>BorderStyle.Ridge</Value>
    </TextBox>
    <TextBox>
        <Property>BorderWidth</Property>
        <Value>Unit.Pixel(1)</Value>
    </TextBox>
    <DataGrid>
        <Property>Font.Name</Property>
        <Value>"Tahoma"</Value>
    </DataGrid>
    <DataGrid>
        <Property>Font.Size</Property>
        <Value>FontUnit.Point(8)</Value>
    </DataGrid>
    <DataGrid>
        <Property>BorderStyle</Property>
        <Value>BorderStyle.Solid</Value>
    </DataGrid>
    <DataGrid>
        <Property>BorderWidth</Property>
        <Value>Unit.Pixel(1)</Value>
    </DataGrid>
    <DataGrid>
        <Property>BorderColor</Property>
        <Value>Color.Gray</Value>
    </DataGrid>
</Flat>

Settings in the theme file will be processed by a theme manager object—a custom class generated by parsing the theme file. This class will contain code like the following for each skinned control:

If TypeOf(ctrl) Is Button
   ctrl.BackColor = Color.Gainsboro
   •••
End If
During the page loading, the theme manager will invoke this class to override the default settings of the control, thus ensuring that each skinned control has the expected graphical attributes.

.NET Code Generation

The System.CodeDom namespace contains a bunch of classes that you can use to easily create a code graph to render in either Visual Basic .NET or C#. You outline the structure of the code you want to obtain and then render it in the language of choice. The System.CodeDom classes let you indicate the type of statement you want and its parameters. When it comes to generating code, that abstract description is turned into the real syntax.

Figure 3 illustrates the source code of the command-line utility used to parse theme files into a C# or Visual Basic class. The command line accepts up to three arguments—the file name, the language (which defaults to Visual Basic), and the namespace (which defaults to MsdnMag). A really simple CmdLineParser class is used to parse the arguments. If you're looking for a more complete one, take a look at Command Line Parser Library.

Figure 3 Parsing Theme Files (Close Figure 3)
Imports System.IO
Imports System.Data

Module MsdnMag

    Sub Main(ByVal args() As String)
        ' Get command line parameters
        If (args.Length = 0) Then
            ShowUsage()
            Return
        End If
        Dim params As New CmdLineParams(args)

        ' Check the theme file for existence
        If Not File.Exists(params.FileName) Then
            Throw New ArgumentException( _
                "The specified theme file does not exist.")
        End If

        ' Read the content of the file to a DataSet
        ' Assume the theme file is an XML file that can be read as a
        ' DataSet
        ' Each table in the DataSet has two columns: Property, Value
        Dim themeContents As New DataSet
        themeContents.ReadXml(params.FileName)

        ' Create a source file class XxxTheme that inherits from Theme
        Dim info As New ThemeInfo
        info.UserData = params
        info.Contents = themeContents
        info.Name = themeContents.DataSetName + "Theme"
        info.UserData.ClsName = info.Name
        Dim sourceFile As String = CodeGenHelper.ParseThemeCode(info)

        ' Show output
        ShowBanner()
        ShowParams(params)
        Console.WriteLine("File created: " + sourceFile + ".")
        Console.ReadLine()
    End Sub
    •••
End Module

Public Class CmdLineParams
    Public FileName As String
    Public Language As String
    Public NsName As String
    Public ClsName As String

    Public Sub New()
        Language = "VB"
        NsName = "MsdnMag"
    End Sub

    Public Sub New(ByVal args() As String)
        Language = "VB"
        NsName = "MsdnMag"

        ' At least one argument
        FileName = args(0)
        Dim i As Integer = 0
        For Each token As String In args
            If i > 0 Then
                If token.StartsWith("/l:") Then
                    Language = token.Substring("/l:".Length)
                End If
                If token.StartsWith("/ns:") Then
                    NsName = token.Substring("/ns:".Length)
                End If
            End If
            i += 1
        Next
    End Sub
End Class

Public Class ThemeInfo
    Public Name As String
    Public Contents As DataSet
    Public UserData As CmdLineParams
End Class

The XML theme file is assumed to be in a format that can be easily turned into a DataSet object using the ReadXml method:

Dim themeContents As New DataSet
themeContents.ReadXml(params.FileName)
In addition to the schema of Figure 2, you can use the following syntax, which is more compact:
<Flat>
  <Button Property="BackColor" Value="..." />
  <Button Property="ForeColor" Value="..." />
  •••
  <DataGrid Property="Gridlines" Value="..." />
  •••
</Flat>

The theme compiler reads those settings in and produces a new class named after the root node of the theme. If the theme is rooted in a node named Flat, the resulting class will be FlatTheme. This class will inherit from a Theme base class defined in a companion assembly (more on this in a moment).

Public Class FlatTheme : Inherits Theme

The class overrides one of the methods on the base class—Apply. The following is the signature of the Apply method:

Public Overrides Sub Apply(ByVal ctrl As Control)
The Apply method takes a control and compares its actual type against all the skinned controls in the current theme. If the type of control matches one of the skinned types, all the settings defined in the theme are applied to the control.

Figure 4 shows the source code of the method that governs the code generation process. The CodeCompileUnit object represents the root of the code graph you are going to generate. You start by adding a namespace (an instance of the CodeNamespace class) and then all the necessary imports. You add the namespace to the graph using the Namespaces collection, like so:

Dim ns As New CodeNamespace(info.UserData.NsName)
unit.Namespaces.Add(ns)

Figure 4 Code Generation Process (Close Figure 4)
Public Shared Function ParseThemeCode(ByVal info As ThemeInfo) As String
    ' Declare the unit of code being created
    Dim unit As New CodeCompileUnit

    ' Declare the type (class) to be created
    Dim cls As New CodeTypeDeclaration(info.UserData.ClsName)

    ' Bind to the base class
    cls.BaseTypes.Add("Theme")

    ' Add a new method to the class
    Dim methodApply As New CodeMemberMethod
    cls.Members.Add(methodApply)

    ' Define namespace, imports, and method parameters
    SetupClass(unit, cls, methodApply, info)

    ' Create the body of method [named Apply(ctl)]
    For Each t As DataTable In info.Contents.Tables
        ' Add comments
        AddSeparator(methodApply, t.TableName)

        ' Add IF
        AddIfStatement(methodApply, t, info.UserData.Language)
    Next

    ' Generate the source file
    Return GenerateGraph(unit, info)
End Function

Private Shared Sub SetupClass(ByRef unit As CodeCompileUnit, _
            ByRef cls As CodeTypeDeclaration, _
            ByRef methodApply As CodeMemberMethod, ByVal info As ThemeInfo)

    ' Define the MsdnMag namespace
    Dim ns As New CodeNamespace(info.UserData.NsName)
    unit.Namespaces.Add(ns)

    ' Define some Imports
    Dim imp1 As New CodeNamespaceImport("System.Web.UI.WebControls")
    Dim imp2 As New CodeNamespaceImport("System.Web.UI")
    Dim imp3 As New CodeNamespaceImport("System.Drawing")
    ns.Imports.Add(imp1)
    ns.Imports.Add(imp2)
    ns.Imports.Add(imp3)

    ' Define the theme class
    ns.Types.Add(cls)

    ' Define the Apply method
    methodApply.Attributes = MemberAttributes.Public Or _
        MemberAttributes.Override
    methodApply.Name = "Apply"
    methodApply.Parameters.Add( _
           New CodeParameterDeclarationExpression("Control", "ctl"))
End Sub

An import is represented by the CodeNamespaceImport class. You add it to the Imports collection of the namespace object:

Dim imp As New CodeNamespaceImport("System.Web.UI")
ns.Imports.Add(imp)
The class must be added to the Types collection of the namespace. Any base type for the class (including, of course, interfaces) is added to the BaseTypes collection of the CodeTypeDeclaration class. In Visual Basic .NET, the first element is associated with the Inherits keyword; all the others are associated with the Implements keyword. The code snippet here shows how to generate the declaration of a class that contains a method:
Dim cls As New CodeTypeDeclaration(clsName)
ns.Types.Add(cls)
cls.BaseTypes.Add("Theme")
Dim methodApply As New CodeMemberMethod
cls.Members.Add(methodApply)

For methods, you need to specify formal parameters and the function body. Here's how to declare the Apply method's signature:

met.Attributes = MemberAttributes.Public Or MemberAttributes.Override
met.Name = "Apply"
met.Parameters.Add( _
    New CodeParameterDeclarationExpression("Control", "ctrl"))
The body is a collection of variable declarations and statements. The idea is that you generate a comment and an If statement for each table in the DataSet. Remember that a table corresponds to a control to be skinned where the table is the class name of the control. You'll find all the code that generates the FlatTheme class in the code download on the MSDN® Magazine Web site. Again, the underlying theme file was shown in Figure 2.

The System.CodeDom namespace lets you declare the code you want to be generated. However, it doesn't cover all possible instructions you might want to write. For example, there's no way to obtain a For...Each loop. The namespace provides iteration capabilities but only through the Do...While statement. (Iterative commands are all equivalent if you look at the core functionality, but each has a different degree of usefulness, depending on the scenario.) Once the code graph is complete, you can generate the output in either Visual Basic .NET or C# (or any other language that has an ICodeGenerator implemented for it). Figure 5 shows the code that creates the source class.

Figure 5 Source Class (Close Figure 5)
Private Shared Function GenerateGraph(ByVal unit As CodeCompileUnit, _
            ByVal info As ThemeInfo) As String
        ' What's the language to use?
        Dim targetFile As String = info.UserData.FileName
        Select Case info.UserData.Language.ToLower()
            Case "vb"
                targetFile = targetFile + ".vb"
                GenerateCode(unit, New VBCodeProvider, targetFile)
            Case "c#", "cs"
                targetFile = targetFile + ".cs"
                GenerateCode(unit, New CSharpCodeProvider, targetFile)
        End Select
        Return targetFile
End Function

Private Shared Sub GenerateCode(ByVal unit As CodeCompileUnit, ByVal _
            provider As CodeDomProvider, ByVal fileName As String)
        ' Obtains an ICodeGenerator from a CodeDomProvider class
        Dim gen As ICodeGenerator = provider.CreateGenerator()

        ' Creates a StreamWriter to an output file
        Dim writer As New StreamWriter(fileName, False)

        ' Generates source code using the code generator
        gen.GenerateCodeFromCompileUnit(unit, writer, _
            New CodeGeneratorOptions)

        ' Closes the output file
        writer.Close()
End Sub

The ThemeManager Class

The mechanism just described is quite similar to the "Add a Web Reference" feature in Windows Forms and Web Forms projects. When you choose to add a Web service, Visual Studio .NET generates a source file and binds that file to the project. Likewise, you generate the theme class and add it to an ASP.NET project.

Basic operation and interaction with any control tree is managed through the code in the theme class. Despite the warning at the top of the autogenerated file, you can edit the file as long as you know what you're doing.

The dynamically generated class inherits from a base class—Theme. The Theme class is part of an external assembly that you need to bind to your main application. The source code of the Theme class is shown here:

Public MustInherit Class Theme
  Public Overridable Sub ApplyRecursive(ByVal ctl As Control)
     Apply(ctl)
     For Each child As Control In ctl.Controls
         ApplyRecursive(child)
     Next
  End Sub

  Public MustOverride Sub Apply(ByVal ctl As Control)
End Class
It defines an ApplyRecursive method that loops through all controls in the specified container and skins each individually. The Apply method is abstract and is properly overridden in the theme class generated by the theme compiler.

Which component is physically responsible for applying the selected theme? The theme manager is a class that contains a handful of static methods to register multiple themes with a single application and to select a particular theme to apply.

The theme manager has two main methods—Register and Apply. The Register method takes the name and type of the theme and creates an instance of the specified class. The object is then placed in the ASP.NET cache with an autogenerated key. The Apply method takes the name of the theme, retrieves the object from the cache, and calls ApplyRecursive. In turn, ApplyRecursive invokes the object's specific Apply method, which was tailor-made for the theme XML source file in question:

Shared Function Register(name As String, t As Type) As Theme
   Dim key As String = GetCacheKeyName(name)
   Dim obj As Theme = Activator.CreateInstance(t)
   HttpContext.Current.Cache(key) = obj
End Function

Shared Function Apply(name As String, root As Control)
   Dim key As String = GetCacheKeyName(themeName)
   Dim obj As Theme = HttpContext.Current.Cache(key)
   obj.ApplyRecursive(root)
End Function

The elements of the puzzle are all set now. I'll briefly summarize the steps you need to take to extend ASP.NET 1.1 applications with themes. You start by storing property settings into an XML file and then parse this file into a Visual Basic .NET or C# class. The class must be added to the project; one class for each theme you want to support. Obviously, you could write the class files directly without using the tool.

Once the theme classes are part of the project, you modify the Page_Init handler as follows:

Sub Page_Init(sender As Object, e As EventArgs) Handles MyBase.Init
   If Not IsPostBack Then
      ThemeManager.Register("Sample", GetType(MsdnMag.SampleTheme))
      ThemeManager.Register("Flat", GetType(MsdnMag.FlatTheme))
      ThemeManager.Apply("Sample", Me)
   End If
End Sub
You register the theme objects and apply the one of your choice. If you don't allow users to change the theme at run time, then you have no need to register more than one theme class. Note that Visual Studio .NET hides the Page_Init handler in the Web designer-generated region.

To change the theme at run time, add a dropdown list to your page and fill it with all the available themes. (You can also add a new method to ThemeManager to return them all.) Turn on the AutoPostBack attribute of the dropdown list so that the page posts back whenever the user makes a new selection. The event handler will contain the following, surprisingly simple, code:

ThemeManager.Apply(ListOfThemes.SelectedValue, Me)

I recommend that you bind the data again after you change the theme. I also suggest that you do whatever you do in the Page_Load handler that relates to the controls subject to skins. This is especially important if you skin data-bound controls like the DataGrid. Figure 6 shows a sample page.

Figure 6 Changing a Theme with a Dropdown List
Figure 6 Changing a Theme with a Dropdown List

Issues and Gripes

As you may have guessed already, the Web theming solution I've shown is not right for everyone and is primarily targeted toward using just one theme across all pages of a Web site. The first issue is the syntax of the theme file. The XML document is tedious to cre-ate and must be turned into a source class before use. As a shortcut, you can use the XML file to create the source class and then update the Visual Basic .NET or C# file directly in case of further changes.

Since you bring in plain classes, debugging is not an issue. Visual Studio .NET can easily step into the theme classes as in any other class in the project. I've created a registry file to add some shell support to theme files. If you install the shell support, you can then right-click on a theme file and create the corresponding source file in the preferred language (see Figure 7).

Figure 7 Compiling a Theme
Figure 7 Compiling a Theme

Another subtle issue has to do with the viewstate restoration and applies if you change themes dynamically. Suppose you support two themes, Blue and Green, where the Blue theme doesn't contain any background color for textboxes. Suppose, also, that Blue is the default theme. The first time you display the page, the background of textboxes is white—or any other color you set in the designer. When you change the theme to Green, the background color changes accordingly. At this point, if you move back to the original theme—Blue—the textbox retains the background color of the Green theme. Why? Because of the viewstate restoration and because no explicit background color setting was forced to the controls by the current theme.

In my themes implementation for ASP.NET 1.x, you change the theme using a postback event after the page loading stage has completed. This will give the theme the last word on control settings. However, it is important that each theme assigns a value to each property it cares about; otherwise, the value of these properties might change from time to time.

Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is a Wintellect instructor and consultant based in Rome, Italy. Author of Programming ASP.NET (Microsoft Press, 2003), he spends most of his time teaching classes on ADO.NET and ASP.NET and speaking at conferences. Get in touch with Dino at cutting@microsoft.com or join the blog at http://weblogs.asp.net/despos.


The XML Files: XML Data Migration Case Study: GEDCOM

The XML Files
XML Data Migration Case Study: GEDCOM
Aaron Skonnard

Code download available at: XMLFiles0405.exe (143KB)
Contents
GEDCOM 101
Mapping GEDCOM to XML
Designing the Transformation
GedcomReader Implementation
Parsing GEDCOM 5.5
Using GedcomReader
XSLT for GEDCOM 6.0
Where Are We?

XML's ubiquity and continually improving tool support has created a magnetism that attracts organizations everywhere. As organizations move to XML, they must also provide a coherent data migration strategy that allows their users to bring old files forward. This is a nontrivial problem that typically requires tedious code to implement the transformation process. The System.Xml namespace in the Microsoft® .NET Framework, however, can greatly simplify these data migration challenges through its extensible APIs and support for XSLT.

An example of data migration is what's currently happening around genealogy data formats. Genealogists have long relied on the Genealogical Data Communications (GEDCOM) 5.5 format for sharing genealogical information. GEDCOM 5.5 is text based but not XML based. A beta version of GEDCOM 6.0 is available and is completely based on XML. But what about the gigabytes of genealogical information that can still be found in GEDCOM 5.5 format? This presents an interesting data migration challenge that really should not be ignored.

In this column I'll walk you through solving this data migration problem using System.Xml. This process can serve as a blueprint for other data migration problems you may face.

GEDCOM 101

GEDCOM was developed to facilitate exchanging genealogical data across different genealogy programs and systems. A common format like GEDCOM allows users to share their work with others regardless of the program they're using.

Version 5.5 is the most widely used version of the various GEDCOM specifications (see The GEDCOM Standard Release 5.5). GEDCOM 5.5 relies on a simple text-based grammar that leverages line delimiters and level numbers to structure family tree information. Figure 1 provides a sample GEDCOM 5.5 file.

Figure 1 GEDCOM 5.5 (Close Figure 1)
0 HEAD
1 SOUR AcmeGen
2 VERS 5.0
2 NAME Acme Genealogy
1 DATE 13 February 2004
2 TIME 03:50:51
1 GEDC
2 VERS 5.5
2 FORM LINEAGE-LINKED
1 CHAR ANSEL
1 SUBM @SUB01@
1 SUBN @N01@
0 @SUB01@ SUBM
1 NAME Aaron Skonnard
1 ADDR 456 Main
2 CONT Salt Lake City, Utah 84150
0 @N01@ SUBN
1 DESC 1
1 ORDI N
0 @I100294220729@ INDI
1 NAME Bernt Olaf /Skonnord/
1 SEX M
1 BIRT
2 DATE 14 JUN 1860
2 PLAC Snertingdal, Oppland, Norway
1 FAMC @F306458249@
1 SOUR @S01@
•••

Each GEDCOM line contains a level number, a tag, and an optional value. Multiple lines constitute a GEDCOM record. A level of 0 marks the beginning of a new GEDCOM record. Every line that follows is part of the record until you reach another line with a level of 0. The tag name conveys meaning about the information on the line as defined in the specification.

In the example shown in Figure 1, HEAD is the first record and it contains six children (SOUR, DATE, GEDC, CHAR, SUBM, SUBN). SOUR contains two children (VERS and NAME) while DATE contains one child (TIME). The increasing level numbers indicate parent-to-child relationships. A line may also contain a unique identifier, as shown in the following snippet:

0 @SUB01@ SUBM
1 NAME Aaron Skonnard
1 ADDR 456 Main
2 CONT Salt Lake City, Utah 84150
In this example, the SUBM record has a unique identifier of SUB01. This ID must be unique within the scope of the document. Identifiers make it possible to establish links between records. For example, the HEAD record referenced this SUBM record on the following line of code:
1 SUBM @SUB01@

These are the basics of the GEDCOM 5.5 grammar. GEDCOM 5.5 is capable of representing sophisticated tree structures through its use of level numbers, identifiers, and cross-referencing capabilities. I don't have enough space in this column to cover GEDCOM semantics in more detail. Suffice it to say that GEDCOM 5.5 makes it possible to express a wide range of genealogy-related information.

Since GEDCOM was primarily designed to represent tree structures (family trees) in an interoperable manner, moving GEDCOM to an XML format seems like a perfect fit.

Mapping GEDCOM to XML

The GEDCOM 5.5 grammar maps nicely to XML. One way to define a simple XML mapping is to convert each GEDCOM line into an XML element with the same name. The line's optional data, if any, can be placed in an attribute named "value". Identifiers can be placed in an attribute named "id" and references to other records can be placed in an attribute named "idref". Figure 2 may help you visualize the mapping. Applying this simple mapping to the GEDCOM 5.5 file shown in Figure 1 produces the XML document shown in Figure 3.

Figure 3 A GEDCOM XML Sample (Close Figure 3)
<GEDCOM>
  <HEAD>
    <SOUR value="AcmeGen">
      <VERS value="5.0"/>
      <NAME value="Acme Genealogy"/>
    </SOUR>
    <DATE value="13 February 2004">
      <TIME value="03:50:51"/>
    </DATE>
    <GEDC>
      <VERS value="5.5"/>
      <FORM value="LINEAGE-LINKED"/>
    </GEDC>
    <CHAR value="ANSEL"/>
    <SUBM idref="SUB01"/>
    <SUBN idref="N01"/>
  </HEAD>
  <SUBM id="SUB01">
    <NAME value="Aaron Skonnard"/>
    <ADDR value="456 Main">
      <CONT value="Salt Lake City, Utah 84150"/>
    </ADDR>
  </SUBM>
  <SUBN id="N01">
    <DESC value="1"/>
    <ORDI value="N"/>
  </SUBN>
  <INDI id="I100294220729">
    <NAME value="Bernt Olaf /Skonnord/"/>
    <SEX value="M"/>
    <BIRT>
      <DATE value="14 JUN 1860"/>
      <PLAC value="Snertingdal, Oppland, Norway"/>
    </BIRT>
    <FAMC idref="F306458249"/>
    <SOUR idref="S01"/>
  </INDI>
•••
Figure 2 Mapping GEDCOM 5.5 to GEDCOM XML
Figure 2 Mapping GEDCOM 5.5 to GEDCOM XML

The XML version conveys the same information, but now it can be processed by a much wider range of tools and technologies. For example, you could process the XML file with your favorite XML API (such as DOM, SAX, XmlTextReader, or XPathNavigator), a query language like XPath or XQuery, or a transformation language like XSLT. Ultimately, once you have GEDCOM data in XML format, you can do just about anything with it.

Designing the Transformation

So how do you transform the GEDCOM 5.5 file into XML format? You can't use XSLT because the input format, GEDCOM 5.5, is not XML based. Instead, you have to write code that manually parses the GEDCOM 5.5 format and produces the XML format shown in Figure 3. Using the .NET Framework, the easiest way to accomplish both of these tasks at the same time is to write a custom XmlReader class.

I've provided a complete sample that illustrates how this is done (you can download the sample from the link at the top of this article). The sample contains several classes that work together in translating the GEDCOM 5.5 information into a logical XML document. To start, I defined an interface called IGedcomNode that several other classes implement. This interface models a logical XML node that contains information from a GEDCOM source. It allows the consumer to access the node's XML name, value, and type, as well as the GEDCOM 5.5 level number that appears on each GEDCOM line. I'll need the level number later on in order to figure out when a node ends and when a new node begins.

Then I defined several concrete classes to model the different types of XML nodes that will make up the resulting XML document (see Figure 4). These include GedcomElement, GedcomAttribute, and GedcomText. Each of these classes implements the IGedcomNode interface. I also defined a class called IGedcomLine whose purpose is to manage a sequence of IGedcomNode objects that came from a particular GEDCOM line.

Figure 4 GEDCOM 5.5 to XML Classes (Close Figure 4)
// standard interface for all XML nodes that we'll be
// creating from the GEDCOM 5.5 file

public interface IGedcomNode
{
    string Name {get;}
    string Value {get;}
    int LineNumber {get;}
    XmlNodeType NodeType {get;}
}

public class GedcomAttribute : IGedcomNode
{
    ... // represents a value from the GEDCOM file that will
        // now become an XML attribute node
}

public class GedcomElement : IGedcomNode
{
    ... // represents a tag from the GEDCOM file that will
        // now become an XML element node
}

public class GedcomEndElement : IGedcomNode
{
    ... // a node that marks the end of an XML element node
}

public class GedcomText : IGedcomNode
{
    ... // represents a value from the GEDCOM file that will
        // now become an XML text node
}

public class GedcomLine
{
    ... // represents a GEDCOM 5.5 line that has been parsed
        // into an array of IGedcomNode objects (see above)
}

GedcomReader Implementation

The meat of the implementation is found in the GedcomReader class. GedcomReader derives from System.Xml.XmlReader and overrides its abstract members. The fundamental purpose of GedcomReader is to take care of parsing an underlying GEDCOM 5.5 file and turn it into an XML document. Hence, the consumer doesn't need to know anything about GEDCOM 5.5. The consumer only needs to be familiar with the XML format as shown in the code in Figure 3.

My implementation of GedcomReader has a single constructor that takes a GEDCOM 5.5 file name. The constructor opens the supplied GEDCOM 5.5 file with a System.IO.StreamReader object and prepares to begin parsing. It also provides a Close method that closes the underlying file stream when the user is finished.

The Read method (see Figure 5) does the bulk of the work because it advances the logical cursor through the document. Therefore, the implementation of Read must deal with the translation between the GEDCOM 5.5 format and the target XML format that I'm trying to simulate (see Figure 3).

Figure 5 GedcomReader Implementation with GEDCOM 5.5 Parsing (Close Figure 5)
public class GedcomReader :    XmlReader
{
  •••
  private GedcomLine ParseGedcomLine(string lineText)
  {
    string xref_id="", tag="", pointer="", linevalue="";
    int nextPart = 0;

    // split GEDCOM line into parts
    string[] lineParts = lineText.Split(' ');
    // first part is always the line number
    int lineNumber = int.Parse(lineParts[nextPart++]);
    // check to see if line has an ID and get tag name
    if (lineParts[nextPart].StartsWith("@"))
    {
       xref_id = lineParts[nextPart++].Replace("@", "");
       tag = lineParts[nextPart++];
    }
    else
      tag    = lineParts[nextPart++];

    // check to see if value is a pointer or text
    if (lineParts.Length > nextPart)
      if (lineParts[nextPart].StartsWith("@"))
        pointer    = lineParts[nextPart++].Replace("@", "");
    if (lineParts.Length > nextPart)
      linevalue = GetRemainingValue(lineParts, nextPart);

    GedcomLine line = new GedcomLine(lineNumber);
    GedcomElement e = new GedcomElement(tag, lineNumber);
    if (xref_id    != "")
      e.AddAttribute(new GedcomAttribute("id", xref_id, lineNumber));
    if (pointer    != "")
      e.AddAttribute(new GedcomAttribute("idref", pointer, lineNumber));
    line.Add(e);
    if (linevalue != "")
      e.AddAttribute(new GedcomAttribute("value", linevalue, lineNumber));
    return line;
  }

  public override bool Read()
  {
    try
    {
      switch (readState)
      {
        case ReadState.Initial:
        {
          nodeStack.Push(new GedcomElement("GEDCOM", -1));
        readState =    ReadState.Interactive;
        return true;
        }
        case ReadState.Interactive:
        {
          // deal with attributes and end element nodes
          ...
          if (currentLineNodes != null && !currentLineNodes.EOF)
          {
            currentLineNodes.MoveNext();
            nodeStack.Push(currentLineNodes.Current);
          }
          else
          {
            // need to parse a new GEDCOM line
            currentLineText = fileReader.ReadLine();
            if (currentLineText == null)
            {
              readState = ReadState.EndOfFile;
              return false;
            }
            currentLineNodes = ParseGedcomLine(currentLineText);
            ...
            nodeStack.Push(currentLineNodes.Current);
          }
          return true;
        }
        default:
          return false;
      }
    }
    catch(Exception e)
    {
      readState = ReadState.Error;
      throw e;
    }
  }
  ... // remaining methods omitted
}

The implementation uses a stack to keep track of the current XML node and its ancestors. Read will periodically push nodes onto the stack and eventually pop them off as the cursor moves through the underlying document. The node at the top of the stack is considered the current node.

The implementation of Read first checks the ReadState to see if this is the first call to Read. If so, it creates an XML element named "GEDCOM" (using a GedcomElement object) and pushes it onto the stack—this is the root element of the target format (see Figure 3). Then it sets the ReadState to "Interactive".

On future calls to Read, the implementation will read a line from the underlying GEDCOM 5.5 file and call ParseGedcomLine, which then produces a GedcomLine object containing several IGedcomNode objects. The GedcomLine object serves as a cache. This is necessary because each call to Read only processes a single XML node. So after a line has been parsed, future calls will process the remaining nodes in the GedcomLine object until the end of the cached collection is reached. At this point, a new line is parsed and the process repeats itself.

The other tricky thing that Read has to deal with is inserting XmlNodeType.EndElement nodes in the correct sequence.

Parsing GEDCOM 5.5

The parsing code (see ParseGedcomLine) takes care of the GEDCOM 5.5 parsing details and generating the proper XML format using the various IGedcomNode-derived classes. You can change the code in this method to customize the target XML format. The code shown in Figure 5 produces the XML format shown in the code in Figure 3.

With this code in place, overriding the remaining XmlReader members becomes quite simple (see Figure 6). For example, the implementations of LocalName and Name pull the current node from the top of the stack and ask for its name through the IGedcomNode interface. The override for NodeType does the same thing. As you can see, the core transformation functionality takes place in the implementation of Read. You can download the complete sample for more details on the remaining implementations.

Figure 6 XmlReader Property Implementation (Close Figure 6)
public class GedcomReader :    XmlReader
{
  ... // remaining methods omitted

  public override string LocalName
  {
    get
    {
      if (readState != ReadState.Interactive)
        return String.Empty;
      IGedcomNode node = nodeStack.Peek() as IGedcomNode;
      return node.Name;
    }
  }

  public override string Name
  {
    get
    {
      if (readState != ReadState.Interactive)
        return String.Empty;
      IGedcomNode node = nodeStack.Peek() as IGedcomNode;
      return node.Name;
    }
  }

  public override XmlNodeType NodeType
  {
    get
    {
      if (readState != ReadState.Interactive)
        return XmlNodeType.None;
      IGedcomNode node = nodeStack.Peek() as IGedcomNode;
      return node.NodeType;
    }
  }

  public override string Value
  {
    get
    {
      IGedcomNode node = nodeStack.Peek() as IGedcomNode;
      return node.Value;
    }
  }
}

Using GedcomReader

With the completed GedcomReader implementation, you can now process GEDCOM 5.5 documents as if they were really XML documents structured like the one shown in Figure 3. For example, you can begin writing XmlReader-based code that streams through the file and inspects each XML node one at a time:

GedcomReader gr = new GedcomReader("skonnord.ged");
while (gr.Read())
   Console.WriteLine("{0}: {1}", gr.NodeType, gr.Name);
Or you could load the document into an XmlDocument and process it using XPath. For example, the following code snippet identifies the INDI elements that have a NAME element containing 'Bernt' in its value:
GedcomReader gr = new GedcomReader("skonnord.ged");
XmlDocument doc = new XmlDocument();
doc.Load(gr);
gr.Close(); // done using GedcomReader
XmlNode indi = doc.SelectSingleNode(
  "/GEDCOM/INDI[contains(NAME/@value, 'Bernt')]");

After loading the GEDCOM 5.5 source into an XmlDocument object, you can serialize it to XML 1.0 using the Save method:

GedcomReader gr = new GedcomReader("skonnord.ged");
XmlDocument doc = new XmlDocument();
doc.Load(gr);
gr.Close(); // done using GedcomReader
doc.Save("skonnord.xml");

The generated skonnord.xml file will look like the file shown in Figure 3. Loading the GEDCOM 5.5 file and saving it achieves the complete file transformation. You can even take the transformation one step further by using XSLT to translate the intermediate XML format into something else.

XSLT for GEDCOM 6.0

The latest GEDCOM specification, version 6.0, defines a full-fledged XML format for GEDCOM information. The specification even provides a Document Type Definition (DTD) that defines the elements and attributes that make up the complete GEDCOM 6.0 vocabulary.

The GEDCOM 6.0 format is much different from the one my GedcomReader simulates. In order to migrate to GEDCOM 6.0, you have to either modify the GedcomReader implementation in order to simulate the new format or write an XSLT that performs the transformation in a subsequent step.

The GEDCOM 6.0 format is much more complex than the simple mapping I simulated in GedcomReader. Consequently, trying to simulate GEDCOM 6.0 in the GedcomReader code would be extremely difficult and error prone. Using an XSLT transformation to accomplish this step is a more tractable problem.

In Figure 7 I've provided an XSLT in the sample project that illustrates how to generate a GEDCOM 6.0 file from the intermediate XML format shown in Figure 3. The XSLT covers the most common GEDCOM 6.0 use cases, and it produces files that pass validation against the GEDCOM 6.0 DTD.

Figure 7 XSLT Transformation to GEDCOM 6.0 (Close Figure 7)
<xsl:stylesheet version="1.0"
   xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

  <xsl:output indent="yes" omit-xml-declaration="yes"/>

  <xsl:template match="GEDCOM">
    <GEDCOM>
      ...
      <xsl:apply-templates select="INDI"/>
      ...
    </GEDCOM>
  </xsl:template>

  <xsl:template match="INDI">
    <IndividualRec Id="{@id}">
      <IndivName>
        <xsl:call-template name="OutputLang"/>
        <xsl:value-of select="NAME/@value"/>
      </IndivName>
      <Gender><xsl:value-of select="SEX/@value"/></Gender>
      <xsl:if test="DEAT|BURI|CREM">
        <DeathStatus>dead</DeathStatus>
      </xsl:if>
      <xsl:call-template name="TransformAttributes"/>
      <xsl:for-each select="ASSO">
        <AssocIndiv>
          <Link Target="IndividualRec" Ref="{@idref}">
          <xsl:choose>
            <xsl:when test="TYPE='FAM'">
              <xsl:attribute name="Target">FamilyRec
              </xsl:attribute>
            </xsl:when>
            <xsl:when test="TYPE='INDI'">
              <xsl:attribute name="Target">IndividualRec
              </xsl:attribute>
            </xsl:when>
            ...
          </xsl:choose>
          </Link>
          <Association>
            <xsl:value-of select="RELA/@value"/>
          </Association>
          <xsl:call-template name="TransformNote"/>
        </AssocIndiv>
      </xsl:for-each>
      <xsl:call-template name="TransformRecordCom"/>
    </IndividualRec>
  </xsl:template>

  ... <!-- remainder omitted for clarity, download file for more
           details -->

</xsl:stylesheet>

You can use this XSLT by taking advantage of the System.Xml.Xsl.XslTransform class, as shown here:

GedcomReader gr = new GedcomReader(gedcomFileName);
XmlDocument doc = new XmlDocument();
doc.Load(gr);
gr.Close(); // done using GedcomReader

XslTransform tx = new XslTransform();
tx.Load("gedcom6.xsl");
FileStream fs =
   new FileStream("skonnord6.xml", FileMode.Create);
tx.Transform(doc, null, fs, null);

The output file, skonnord6.xml, will now be GEDCOM 6.0-compliant. At this point it's also possible to write other XSLT transformations that change the intermediate XML format into another format of your choosing. For example, you could write an XSLT transformation that produces human-readable HTML pages for viewing and navigating the GEDCOM family tree information.

As you can see, it didn't take much code to implement a complete GEDCOM migration path. The ability to programmatically move between GEDCOM 5.5 and GEDCOM 6.0 greatly simplifies the migration scenarios that are involved in building a complete genealogy system around the new GEDCOM 6.0 data model.

Where Are We?

I've walked through a real-world data migration scenario using the System.Xml classes in .NET. I started with the GEDCOM 5.5 format and moved to an intermediate XML format that can be processed in a variety of ways. This is made possible by a custom XmlReader implementation called GedcomReader.

The intermediate XML format can also be transformed into a variety of other formats. I was able to migrate to GEDCOM 6.0 by using an XSLT transformation (see Figure 8).

Figure 8 Migrating to GEDCOM 6.0
Figure 8 Migrating to GEDCOM 6.0

The extensibility points provided by System.Xml facilitate dealing with a variety of data migration scenarios like this one. If you have old data formats lying around that would benefit from migrating to XML, follow the approach discussed in this column to implement your own migration path.

Send your questions and comments for Aaron to xmlfiles@microsoft.com.

Aaron Skonnard teaches at Northface University in Salt Lake City. Aaron coauthored Essential XML Quick Reference (Addison-Wesley, 2001) and Essential XML (Addison-Wesley, 2000), and frequently speaks at conferences. Reach him at http://www.skonnard.com.


Advanced Basics: Create a Graphical Editor Using RichTextBox and GDI+

Advanced Basics
Create a Graphical Editor Using RichTextBox and GDI+
Ken Spencer

Code download available at: AdvancedBasics0405.exe (130KB)
Contents
Approach 1: The Subform
Approach 2: The Transparent Form
Approach 3: GDI+ and RichTextBox
One More Solution
Conclusion

Irecently was asked a question by a reader who is developing a graphical editor that supports non-Windows® objects (rectangle, line, circle, image) as well as a Windows object (a text control derived from the rich text control). He wanted to define the Z-order of these objects, but discovered that the text window always hides other objects.

Figuring out how to implement the Z-order took some time and research. I've seen discussions about making controls transparent, but I couldn't find a good answer, so I tried a few approaches of my own to solve the problem. To test solutions, I decided to focus on the problem of hosting graphics such as lines, boxes, circles and such. Let's walk through a few approaches and see what happens.

Approach 1: The Subform

Using a Windows Form to host the objects allows me to put the items on the form and then make the form transparent, thus rendering the items beneath it except for those objects that are on the form itself. I tried this by creating two forms and making one of the forms transparent (by setting the TransparencyKey to the value of the form's BackColor). This seemed to work fine. Then I put this form on the master form as a subform.

To create a subform on a Windows Form you must set the TopLevel property of the subform to False and then create an instance of the subform in the parent, and add the subform to the controls collection of the parent. This will place the subform on the parent form. This approach is pretty slick and it even works in design view in Visual Studio® .NET so you can see your form design while building the application. You might notice that I have not shown you any code yet. I will get to that in a minute when I start describing what works best.

The problem with using a subform approach is that the transparent form functionality only works if the form has its TopLevel property set to True. As I mentioned, when you use a form as a subform you must set the TopLevel property to False and thus you cannot use transparency with that form. I was quite disappointed as this would have been an elegant solution that would allow you to do really creative things with your applications.

Approach 2: The Transparent Form

The second approach also uses a Windows Form as a host for child objects. I had more success with this one than with the subform. This time I set up the second form with a transparent background but the problem is that you have to keep these forms in sync with the main form. For instance, if the user minimizes the main form, you must minimize all of the palette forms. If the main form moves, you must move the palette forms. This requires lots of code and much fiddling to get it right. If you must host items such as non-Windows objects that you can render with GDI+, then this form approach might be appropriate. It can be done, but it will take a ton of work to keep the many forms synchronized. Although the transparent form requires a lot of effort, it does work. In the process of debugging this approach, however, I hit on a solution that seems to work even better.

Approach 3: GDI+ and RichTextBox

This approach does everything in one form except for a drawing palette which is implemented by a second form. It works great but it still needs some fine tuning, which I'll leave as an exercise for you to do on your own.

First, I created a simple form that has two controls. The first control is a button called Drawing Palette. Then I added a RichTextBox and a MainMenu (File Open and Close). So far, this approach is really simple. Now for the fun.

I wanted the application to allow me to open a rich text file (.rtf), then add graphics to it. I could have just used GDI+, but that would not allow the user to add graphics at run time. So, I decided to add a drawing palette as another form, place it at a specific location over the main form, and allow users to draw on it. After much playing around with this extra form approach, I finally found a solution for handling graphics, which I will show you now.

Figure 1 Simple Text File
Figure 1 Simple Text File

As you can see, Figure 1 shows a simple text file opened in the application with simple graphics displayed over the text. The text is shown in the RichTextBox.

Figure 2 Drawing Palette Over Text
Figure 2 Drawing Palette Over Text

Figure 2 shows the drawing palette on the Grapher form overlaying the textbox. Grapher is a simple form with one key setting. Its Opacity property is set to 50 percent. This allows the underlying text in the RichTextBox to show through this form. Grapher also has all the widgets (such as border, control box, and so on) turned off, resulting in a borderless form. The toolbar is actually a panel control housing three pictureboxes with the graphics for the buttons.

The version I wrote only supports drawing a straight line and a rectangle. The user can see the underlying text beneath the palette. This allows the user to position the graphics over the text visually. Once finished, the user can click the X to close the palette or display the palette later to add more graphics.

Figure 3 shows the key code from Form1. I have omitted most of the generic code in Figure 3 and Figure 4, such as variable definitions and opening files, to focus on the relevant code. The DrawingPaletteButton_Click event is used to start a drawing operation by allowing the user to drag out the size and location of the drawing palette over the RichTextBox. The DrawingPaletteButton_Click code sets up the cross-hair cursor and sets the DrawingMode variable to True.

Figure 4 Key Code Elements from Grapher.vb (Close Figure 4)
Friend Structure DrawingItem
        Dim DrawingMode As DrawingModes
        Dim StartPostion As Point
        Dim EndPostion As Point
        Dim BoxSize As Size
        Dim PaletteX As Integer
        Dim PaletteY As Integer
    End Structure
    Friend Enum DrawingModes
        Line
        Rectangle
        None
    End Enum
Private Sub CloseEditModePicturebox_Click( _
    ByVal sender As System.Object, ByVal e As System.EventArgs) _
    Handles CloseEditModePicturebox.Click
        Dim ThisDrawItem As New DrawingItem

        If IsNothing(ThisParentForm.DrawingItems) Then
            ThisParentForm.DrawingItems = New Collection
        End If

        For Each ThisDrawItem In ThisDrawingItems
            ThisParentForm.DrawingItems.Add(ThisDrawItem)
        Next
        If Not IsNothing(ThisParentForm.DrawingItems) Then
            ThisParentForm.PaintGraphics()
        End If
        Me.Hide()
    End Sub
End Class
Figure 3 Key Code Elements from Form1 (Close Figure 3)
Private Sub DrawingPaletteButton_Click(ByVal sender As System.Object, _
        ByVal e As System.EventArgs) Handles DrawingPaletteButton.Click
    Try
        Me.MainTextRichTextbox.Cursor = Cursors.Cross
        DrawingMode = True
    Catch exc As Exception
    End Try
End Sub

Private Sub MainTextRichTextbox_MouseDown(ByVal sender As Object, _
        ByVal e As System.Windows.Forms.MouseEventArgs) _
        Handles MainTextRichTextbox.MouseDown
    DrawBoxX = e.X
    DrawBoxY = e.Y
    If DrawingMode Then
        MouseDownDraw = True
    End If
End Sub

Private Sub MainTextRichTextbox_MouseMove(ByVal sender As Object, _
        ByVal e As System.Windows.Forms.MouseEventArgs) _
        Handles MainTextRichTextbox.MouseMove
    If Not MouseDownDraw Then
        Exit Sub
    End If

    If e.X > DrawBoxX Then
        BoxWidth = e.X - DrawBoxX
        StartX = DrawBoxX
    Else
        BoxWidth = DrawBoxX - e.X
        StartX = e.X
    End If

    If e.Y > DrawBoxY Then
        BoxHeight = e.Y - DrawBoxY
        StartY = DrawBoxY
    Else
        BoxHeight = DrawBoxY - e.Y
        StartY = e.Y
    End If

    Me.Refresh()
    MeGraphics.DrawRectangle(LinePen, _
        StartX, StartY, BoxWidth, BoxHeight)
End Sub

Private Sub MainTextRichTextbox_MouseUp( ByVal sender As Object, _
        ByVal e As System.Windows.Forms.MouseEventArgs) _
        Handles MainTextRichTextbox.MouseUp
    If DrawingMode Then
        Me.MainTextRichTextbox.Cursor = Cursors.Default
        frmGrapher.StartPosition = FormStartPosition.Manual
        frmGrapher.XFormOffset = StartX
        frmGrapher.YFormOffset = StartY
        GraphicxLoc = Me.Location.X + _
            Me.MainTextRichTextbox.Location.X + StartX + XOffset
        GraphicyLoc = Me.Location.Y + _
            Me.MainTextRichTextbox.Location.Y + StartY + YOffset
        Debug.WriteLine("MainTextRichTextbox_MouseUp:" & _
            " graphicxloc= " & GraphicxLoc & _
            " graphicyloc= " & GraphicyLoc)
        frmGrapher.Location = New Point(GraphicxLoc, GraphicyLoc)
        frmGrapher.Size = New Size(BoxWidth, BoxHeight)
        frmGrapher.ResetDrawingItems()
        frmGrapher.ShowDialog()
    End If
    DrawingMode = False
    MouseDownDraw = False
End Sub

Private Sub MainTextRichTextbox_SelectionChanged(_
        ByVal sender As Object, _
        ByVal e As System.EventArgs) _
        Handles MainTextRichTextbox.SelectionChanged
    If DrawingMode Then
        MainTextRichTextbox.Select(0, 0)
    End If
End Sub

Private Sub Form1_Paint( ByVal sender As Object, _
        ByVal e As System.Windows.Forms.PaintEventArgs) Handles MyBase.Paint
    PaintGraphics()
End Sub

Friend Sub PaintGraphics()
    Dim ThisDrawItem As New Grapher.DrawingItem
    Dim xLoc, yLoc As Integer
    Dim xEndLoc, yEndLoc As Integer

    MainTextRichTextbox.Refresh()
    If Not IsNothing(DrawingItems) Then
        For Each ThisDrawItem In DrawingItems
            With ThisDrawItem
                .PaletteX = .PaletteX
                .PaletteY = .PaletteY
                xLoc = .PaletteX + .StartPostion.X
                xEndLoc = (.PaletteX) + .EndPostion.X
                yLoc = .PaletteY + .StartPostion.Y + (YOffset - 50)
                yEndLoc = .PaletteY + .EndPostion.Y + (YOffset - 50)

                If Not DebugOutputComplete Then
                    DebugOutputComplete = True
                End If

                Select Case ThisDrawItem.DrawingMode
                    Case Grapher.DrawingModes.Rectangle
                        MeGraphics.DrawRectangle(LinePen, _
                            xLoc, yLoc, .BoxSize.Width, .BoxSize.Height)
                    Case Grapher.DrawingModes.Line
                        MeGraphics.DrawLine(LinePen, xLoc, _
                            yLoc, xEndLoc, yEndLoc)
                    Case Else
                End Select
            End With
        Next
    End If

    'Test guidelines
    If ShowGuidesMenuItem.Checked Then
        MeGraphics.DrawLine(LinePen, 60, 0, 60, Me.Height)
        MeGraphics.DrawLine(LinePen, 0, 60, Me.Width, 60)
    End If
End Sub

Private Sub MainTextRichTextbox_TextChanged( _
        ByVal sender As Object, _
        ByVal e As System.EventArgs) _
        Handles MainTextRichTextbox.TextChanged
    PaintGraphics()
End Sub

Private Sub Form1_Resize(ByVal sender As Object, _
        ByVal e As System.EventArgs) _
        Handles MyBase.Resize
    If Not MeGraphics Is Nothing Then
        MeGraphics.Dispose()
    End If
    MeGraphics = Me.MainTextRichTextbox.CreateGraphics
End Sub

The MainTextRichTextBox_MouseDown sets up the drag operation when the user clicks the mouse button. The MainTextRichTextBox_MouseMove event actually draws the rectangle. When the user releases the mouse button, MainTextRichTextBox_MouseMove fires and the palette is drawn by displaying the Grapher form, as you saw in Figure 2.

Now, let's move to the code in Figure 4. When implemented, the user can click either the line or rectangle graphic shown on the toolbar in Figure 2 and then draw that object with the mouse. Grapher also uses the DrawingMode enumeration to track which type of graphic to draw (line or rectangle). The user can draw as many items on a single palette as they want. Then the user clicks the X on the toolbar.

The drawing operations in Grapher are similar to those in Form1. However, instead of throwing away the rectangle, the definition for the drawing operation is stored in the DrawingItem structure. Then when the user releases the mouse, the instance of the structure is stored in the ThisDrawingItems collection. Finally, when the user clicks the X, the palette is closed by calling CloseEditModePicturebox_Click. This event uses the DrawingItems collection in Grapher to update the DrawingItems collection in Form1.

Refer back to the Paint event in Figure 3. This event calls the PaintGraphics procedure to actually perform the work. The Paint event in Grapher is very similar, except that it does not call a subroutine to handle it. The code in both events processes the DrawingItems (Form1) and ThisDrawingItems (Grapher) and uses GDI+ to display the graphics on the RichTextBox.

The collection of drawing items allows you to easily move drawing items from the palette form to the main form. Thus, you do not need to worry about trying to have a transparent form or control to host the graphics. Instead, the code just draws the graphics on the main form's RichTextBox.

One More Solution

Let's consider drawing everything using GDI+. You could treat each block of text and each graphic as an object and store them in a collection. I would create a custom structure and enum like this:

Enum ObjectTypes
    Text = 0
    Bitmap = 1
    EMF = 2
End Enum

Structure EditorObject
    Dim ThisObject
    Dim ThisObjectType As ObjectTypes
    Dim X As Integer
    Dim Y As Integer
    Dim Z As Integer
End Structure

Now, I could create an object and put it in a collection. Next I would iterate through the collection of items and draw them. Since each object has a Z value, I could use that to determine the object's placement relative to the other objects.

Conclusion

As you can see, sometimes the solution to a problem is rather simple. Since just about everything is an object in .NET, you can often get around limitations by using a collection to hold the object and manipulate it. Since I am using a collection of graphics items, I could even save the items to disk alongside the text file and reload them later. I like the third approach I presented as it allows me to separate text and graphics and it makes it easy to use the RichTextBox as the text engine. This way I don't have to devise a cool way to handle the text; I just draw the graphics over the text. The Microsoft® .NET Framework does not limit you in this respect; it usually lets you do it either way.

One last note: the GDI+ code shown in this little app needs some tweaking before it is put into production. I did not fine-tune the coordinate mapping or provide other features that you could easily add to meet your needs.

Send your questions and comments for Ken to basics@microsoft.com.

Ken Spencer works for 32X Tech (http://www.32X.com), where he provides training, software development, and consulting services on Microsoft technologies.


Basic Instincts: Updating the UI from a Secondary Thread

Basic Instincts
Updating the UI from a Secondary Thread
Ted Pattison

Code download available at: BasicInstincts0405.exe (130KB)
Contents
Asynchronous Execution Using Delegates
Marshaling to the UI Thread
Control.InvokeRequired

In my January 2004 column I discussed the use of delegates to execute a method asynchronously. I showed you that it's relatively easy to call BeginInvoke on a delegate object from the code behind a Windows® Form in order to dispatch an asynchronous method call. You also saw how to set up a callback method that fires automatically when the asynchronous method call completes its execution. However, I stopped short of talking about how you should update the UI to tell the user that the work is complete.

Updating user interface elements from a secondary thread in Windows Forms is tricky because a secondary thread is not allowed to read or write property values directly from a form or any of its child controls. This restriction exists because form objects and control objects in Windows Forms are not thread-safe. The only thread that's allowed to directly access a property value of a form or one of its controls is the primary UI thread. You must learn how to update the user interface without breaking this rule.

The technique you're going to learn here involves writing code to switch control to the primary UI thread from the secondary thread where it's executing—a process known as marshaling. This will allow you to update the UI safely and reliably. Once you switch control from a secondary thread to the primary UI thread, you are free to change any property values of a form and its child controls.

Asynchronous Execution Using Delegates

I am going to start by examining the code behind the Windows Form named Form1 (see Figure 1). This code demonstrates how to execute a method asynchronously. The technique involves binding a target method (GetCustomerList) to a delegate object and then calling BeginInvoke on that delegate object to start the method's asynchronous execution.

Figure 1 Asynchronous Method Execution (Close Figure 1)
Class Form1 : Inherits Form

  '*** create delegate object to execute method asynchronously
  Private TargetHandler As GetCustomerListHandler _
                           = AddressOf DataAccessCode.GetCustomerList

  '*** create delegate object to service callback from CLR
  Private CallbackHandler As AsyncCallback _
                             = AddressOf MyCallbackMethod

  Sub cmdExecuteTask_Click(sender As Object, e As EventArgs) _
                           Handles cmdExecuteTask.Click
    '*** execute method asynchronously with callback
    TargetHandler.BeginInvoke("CA", CallbackHandler, Nothing)
  End Sub

  '*** CLR calls this method when asynchronous method completes
  '*** note this method runs on worker thread (not the UI thread)
  Sub MyCallbackMethod(ByVal ar As IAsyncResult)
    Dim retval As String()
    retval = TargetHandler.EndInvoke(ar)
    '*** add code here to trigger updating of user interface
  End Sub

End Class

Class DataAccessCode
  Shared Function GetCustomerList(ByVal State As String) As String()
    '*** call across network to DBMS or Web services to retrieve data
    '*** imagine; this method takes several seconds to complete
  End Function
End Class

The technique used here also shows how to bind a callback method (specifically, MyCallbackMethod) to a second delegate object. A reference to this delegate object is then passed as the second parameter in the call to BeginInvoke:

TargetHandler.BeginInvoke("CA", CallbackHandler, Nothing)

A key benefit to using a callback method is that it eliminates the need for polling to determine when the asynchronous method has completed. When the secondary thread has finished executing the GetCustomerList method, it will then execute MyCallbackMethod before returning to the pool of worker threads maintained by the common language runtime (CLR).

It's critical for you to remember that a callback method such as MyCallbackMethod executes on a secondary thread, not on the primary UI thread. That means you should never attempt to update the UI directly from a callback method such as this. Instead, you must write code to switch the flow of execution from the secondary thread running the callback method back to the primary UI thread. I'll take you step-by-step through a design to accomplish exactly that.

Marshaling to the UI Thread

I'll begin by creating a new custom method named UpdateUI that will be called when the callback method needs to update the user interface. The calling signature of the UpdateUI method has been designed for this application-specific scenario:

'*** to be called when update to user interface required
Sub UpdateUI(StatusMessage As String, Customers As String())
  '*** this method will switch control over to primary UI thread
End Sub

The UpdateUI method accepts a string parameter named StatusMessage, which is used to pass a status message that will be displayed to the user. A second parameter, named Customers, is based on a string array. The Customers parameter is used to pass the string array of customer names that has been returned from the asynchronous call to the GetCustomerList method. If you would like to use a similar design in your application, there is no need to have the exact same parameter list. You are free to create a method named UpdateUI with a different calling signature that accepts any list of parameters that makes sense for your application.

Next, I will rewrite the implementation of MyCallbackMethod to call the UpdateUI method. This allows the application to begin the process of updating the user interface at the completion of the asynchronous call to GetCustomerList. Examine the method implementation in Figure 2.

Figure 2 Calling UpdateUI (Close Figure 2)
'*** this method fires at completion of asynchronous call
Sub MyCallbackMethod(ByVal ar As IAsyncResult)
 Try
    '*** retrieve return value from async call
    Dim retval As String()
    retval = TargetHandler.EndInvoke(ar)
    '*** begin process to update UI
    UpdateUI("Task complete", retval)
  Catch ex As Exception
    Dim msg As String
    msg = "Error: " & ex.Message
    UpdateUI(msg, Nothing)
  End Try
End Sub

As you can see, the call to UpdateUI is made just after the call to EndInvoke from within the same Try statement. Making the call to EndInvoke within a Try statement is important because it is the only reliable way to determine whether the asynchronous call completed successfully. Immediately after a successful call to EndInvoke, MyCallBackMethod calls the UpdateUI method to start the process of updating the user interface.

Now let's take a look at an implementation for the UpdateUI method. In the scenario I have just explained, MyCallbackMethod calls UpdateUI in a synchronous fashion. That means that both of these methods will be running on a secondary thread instead of on the primary UI thread. Therefore, the UpdateUI method cannot access the form or any of its controls directly. Instead, this method must be written to switch the flow of execution from a secondary thread to the primary UI thread.

When you want to switch the flow of execution from a secondary thread to the primary UI thread, you should call one of two different public methods supplied by Windows Forms controls. These two methods, Invoke and BeginInvoke, are defined by the Control class inside the System.Windows.Forms namespace. You can call Invoke or BeginInvoke on any type of Control object, including a Form object. The code in my example will call the BeginInvoke method on the Form object.

Things can be a little confusing at first when you start calling the Invoke method or the BeginInvoke method on a Form object or a Control object. That's because delegate objects also expose a pair of methods named—you guessed it—Invoke and BeginInvoke. However, the Invoke and BeginInvoke methods supplied by the Control class are very different from the Invoke and BeginInvoke methods supplied by a delegate object.

It is the act of calling Invoke or BeginInvoke on a Form object or a Control object that switches the flow of execution from a secondary thread to the primary UI thread. The difference between the two methods is that a call to Invoke is a blocking call while a call to BeginInvoke is not. In most cases it is more efficient to call BeginInvoke because the secondary thread can continue to execute without having to wait for the primary UI thread to complete its work updating the user interface. In the example in this month's column I use BeginInvoke for this reason.

Now it's time to write the implementation of the UpdateUI method using a call to BeginInvoke. Take a look at the code in Figure 3. As you can see, I have written UpdateUI to call the Form object's BeginInvoke method. At a high level, the secondary thread calls BeginInvoke in order to trigger the primary UI thread to run a method named UpdateUI_Impl. Note that the call to BeginInvoke on the Form object requires two parameters:

Dim handler As New UpdateUIHandler(AddressOf UpdateUI_Impl)
Dim args() As Object = {StatusMessage, Customers}
Me.BeginInvoke(handler, args)

Figure 3 Calling BeginInvoke (Close Figure 3)
Sub UpdateUI(StatusMessage As String, Customers As String())
  '*** switch control over to primary UI thread
  Dim handler As New UpdateUIHandler(AddressOf UpdateUI_Impl)
  Dim args() As Object = {StatusMessage, Customers}
  '*** call BeginInvoke method of Form object
  Me.BeginInvoke(handler, args)
End Sub

Delegate Sub UpdateUIHandler(StatusMessage As String, _
                             Customers As String())

Sub UpdateUI_Impl(StatusMessage As String, Customers As String())
  '*** update user interface controls from primary UI thread
  Me.sbMain.Panels(0).Text = StatusMessage
  Me.lstCustomers.DataSource = Customers
End Sub

The first parameter passed in the call to BeginInvoke is a delegate object. This parameter allows the secondary thread to pass a reference telling the primary UI thread which method to execute. Note that the delegate object in this example points to the method named UpdateUI_Impl. The second parameter passed in the call to BeginInvoke is an Object array. This is a very flexible parameter because it allows the secondary thread to pass an arbitrary number of parameters that can each be of any type. In my example, the Object array passed to BeginInvoke contains the following two elements: a string containing the status message and a string array holding the names of customers.

Here's where things get interesting. When the secondary thread calls BeginInvoke, Windows Forms automatically calls the UpdateUI_Impl method on the primary UI thread. When Windows Forms calls the UpdateUI_Impl method, it passes the two parameters that are passed in the Object array. Therefore, the implementation of UpdateUI_Impl has access to the status message and the string array of customer names. Since this method is executing on the primary UI thread, it can use these parameter values to directly update the property values of any controls on the form:

Sub UpdateUI_Impl(StatusMessage As String, Customers As String())
  '*** update user interface controls from primary UI thread
  Me.sbMain.Panels(0).Text = StatusMessage
  Me.lstCustomers.DataSource = Customers
End Sub

Control.InvokeRequired

At this point I have a design that updates the user interface in a safe and reliable fashion. However, there is one more technique I would like to discuss here that would make this design a little better. It would be really convenient if the UpdateUI method was written so that it can be called from any method on the form. However, the current design isn't particularly efficient if you call the UpdateUI method from code that is already running on the primary UI thread. That's because UpdateUI will create a delegate object and call BeginInvoke unnecessarily.

Let's make a modification to the UpdateUI method to check whether it's currently executing on the primary UI thread or on a secondary thread. You can perform this check by querying a public property of the Control class named InvokeRequired. This coding technique gives the UpdateUI method an opportunity to call UpdateUI_Impl synchronously on the same thread whenever it is appropriate to do so:

Sub UpdateUI(ByVal StatusMessage As String, ByVal Customers As String())
  If Me.InvokeRequired Then
    '*** switch over to primary UI thread
    Dim handler As New UpdateUIHandler(AddressOf UpdateUI_Impl)
    Dim args() As Object = {StatusMessage, Customers}
    Me.BeginInvoke(handler, args)
  Else
    '*** direct call - already running on primary UI thread
    UpdateUI_Impl(StatusMessage, Customers)
  End If
End Sub

You have now seen all the steps required to execute a method asynchronously on a secondary thread and to update the user interface in a safe and reliable manner. The complete process is shown in Figure 4. If you want to download a sample application written in Visual Basic® .NET along with all the source code for this month's column, you can find it at the link at the top of this article.

Figure 4 Switching Control Between Threads (Close Figure 4)
Public Class Form1 : Inherits Form

  '*** create delegate object to execute method asynchronously
  Private TargetHandler As GetCustomerListHandler _
                           = AddressOf DataAccessCode.GetCustomerList

  '*** create delegate object to service callback from CLR
  Private CallbackHandler As AsyncCallback _
                             = AddressOf MyCallbackMethod

  Sub cmdExecuteTask_Click(sender As Object, e As EventArgs) _
                           Handles cmdExecuteTask.Click
    '*** execute method asynchronously with callback
    UpdateUI("Starting task...", Nothing)
    TargetHandler.BeginInvoke("CA", CallbackHandler, Nothing)
  End Sub

  '*** callback method runs on worker thread and not the UI thread
  Sub MyCallbackMethod(ByVal ar As IAsyncResult)
    Try
      Dim retval As String()
      retval = TargetHandler.EndInvoke(ar)
      UpdateUI("Task complete", retval)
    Catch ex As Exception
      Dim msg As String
      msg = "Error: " & ex.Message
      UpdateUI(msg, Nothing)
    End Try
  End Sub

  '*** can be called from any method on form to update UI
  Sub UpdateUI(StatusMessage As String, Customers As String())
    '*** check to see if thread switch is required
    If Me.InvokeRequired Then
      Dim handler As New UpdateUIHandler(AddressOf UpdateUI_Impl)
      Dim args() As Object = {StatusMessage, Customers}
      Me.BeginInvoke(handler, args)
    Else
      UpdateUI_Impl(StatusMessage, Customers)
    End If
  End Sub

  '*** delegate used to switch control over to primary UI thread
  Delegate Sub UpdateUIHandler(StatusMessage As String, _
                               Customers As String())

  '*** this method always runs on primary UI thread
  Sub UpdateUI_Impl(StatusMessage As String, Customers As String())
    Me.sbMain.Panels(0).Text = StatusMessage
    Me.lstCustomers.DataSource = Customers
  End Sub

End Class

Send your questions and comments for Ted to instinct@microsoft.com.

Ted Pattison is a cofounder of Barracuda .NET, an education company that assists companies building collaborative applications using Microsoft technologies. Ted is the author of several books including Building Applications and Components with Visual Basic .NET (Addison-Wesley, 2003).


.NET Matters: Finalizers, Assembly Names, MethodInfo, and More

.NET Matters
Finalizers, Assembly Names, MethodInfo, and More
Stephen Toub

Code download available at: NETMatters0405.exe (139KB)

Q When do I need to implement a finalizer on my class? Should I always do it, or only when I'm holding onto unmanaged resources? Should I always implement the IDisposable interface when I have a finalizer, and vice versa?

A Finalizers only need to be implemented when you hold onto resources that need cleaning up. For example, FileStream holds onto a native file handle and implements a finalizer so as to release the handle when the FileStream is garbage collected. Unfortunately, finalizers place a significant amount of pressure on the garbage collector and should be used only when absolutely needed.

The IDisposable interface announces to the world that your class holds onto resources that need to be disposed and allows users of your class to deterministically release them. Therefore, any class that implements a finalizer should also implement IDisposable (if the garbage collector can free resources, a developer should be allowed to do it explicitly). However, the opposite is not necessarily true; not all classes that implement IDisposable need to or even should implement a finalizer.

Consider the scenario where my managed class has a FileStream as a private member. FileStream holds onto unmanaged resources and implements both IDisposable and a finalizer. When no more references to the instance exist, this FileStream will be similarly unreachable and will be available for finalization. There's no reason for my class to be in the queue of objects that have registered for finalization since the instance of FileStream will be there. But my class should provide a way for a user to immediately release all resources it holds onto, either directly or indirectly, and therefore my class should implement IDisposable. My implementation of Dispose will simply call the FileStream's Dispose method. Note, though, that you need to be careful not to dispose of shared resources (resources used by other instances, for example). If you write a class that will dispose of a resource made available to it from an external source, make sure your documentation is clear on the subject so that others know whether they're handing over control of any resources they give you.

If you do need to implement a finalizer in your class, your Dispose method should use the GC.SuppressFinalize method to ensure that finalization of your instance is suppressed. This will remove the instance from the set of objects that require finalization, reducing the pressure on the garbage collector during a collection. A common pattern implemented throughout the Microsoft® .NET Framework is to add to a class a Dispose method that takes a Boolean as a parameter. This Boolean indicates whether the class is being disposed because the IDisposable.Dispose method is called or because the finalizer is run (both the finalizer and IDisposable.Dispose delegate to this method). If it's being disposed deterministically, GC.SuppressFinalize is invoked. If it's being called from a finalizer, avoid using managed members of your class that implement finalizers as they may have already been finalized.

Figure 1 provides guidelines on when it is appropriate to implement these constructs in your classes.

Figure 1 When to Implement IDisposable and/or a Finalizer (Close Figure 1)

Your Class Holds Onto | IDisposable | Finalizer
Only managed resources that don't implement IDisposable or have any way of being closed |  |
Only managed resources, but some of them implement IDisposable or can be closed in some way |  |
Both managed resources and unmanaged resources |  |
Only unmanaged resources |  |

Q I'd like to base some operations that my application performs on the amount of memory available in the computer. What's the easiest way to ask the operating system for this information?

A While I'm sure there are other ways to obtain this information, I've found Windows® Management Instrumentation (WMI) to be perfect for these types of tasks. The Win32_OperatingSystem class provides a wealth of information about the happenings of the OS, and the System.Management namespace provides a very nice set of classes to access WMI data. You can use the ManagementObjectSearcher class as shown in Figure 2 to query for Win32_OperatingSystem.TotalVisibleMemorySize. Since ManagementObjectCollection (as returned by ManagementObjectSearcher) does not expose a method to access individual items in the collection, I've used a foreach loop to access its contents through an enumerator. I only care about one value, so I stop after the first iteration.

Figure 2 Using WMI from .NET (Close Figure 2)
private static ulong PhysicalMemory
{
    get
    {
        SelectQuery query = new SelectQuery("Win32_OperatingSystem");
        using (ManagementObjectSearcher searcher =
               new ManagementObjectSearcher(query))
        {
            foreach(ManagementObject mo in searcher.Get())
            {
                ulong memAvailable;
                try
                {
                     memAvailable = (ulong)mo["TotalVisibleMemorySize"];
                }
                finally
                {
                    mo.Dispose();
                }
                return memAvailable;
            }
        }
        throw new InvalidOperationException(
            "Couldn't get physical memory size.");
    }
}

Note that the value reported by TotalVisibleMemorySize might not be the total amount of physical memory in the computer, but rather what is reported to the OS as being available to it. You can learn more about the information available through the Win32_OperatingSystem WMI class at Win32_OperatingSystem.

Q I'm trying to get the strong name of an assembly without loading the assembly into my AppDomain. Is this possible?

A Absolutely. The System.Reflection.AssemblyName class has a static method, GetAssemblyName, that can be used to retrieve the AssemblyName for an assembly on disk. This method will briefly open the assembly file, but the assembly will not be loaded into the AppDomain. The following code will write out to the console the name of an assembly whose path is specified as a command-line argument, as shown here:

static void Main(string [] args) {
    if (args.Length > 0) {
        try {
            AssemblyName a = AssemblyName.GetAssemblyName(args[0]);
            Console.WriteLine(a.Fullname);
        }
        catch(Exception exc) {
            Console.WriteLine(exc.Message);
        }
    }
}

Note that this same technique can be used to determine which DLLs and EXEs are native and which are managed. You can loop over all of the files to be tested and query for their AssemblyName. If no exception is thrown and you get back a valid name, the file is managed. However, this does have some performance drawbacks as all native files will cause exceptions to be thrown. Another way doesn't rely on reflection and doesn't attempt to load the Portable Executable (PE). Rather, it parses the PE header of the DLL or EXE in order to determine if the appropriate bits are set demarking the DLL as managed. The FAQ at Managed Extensions for C++ Frequently Asked Questions provides C++ code to do this. Equivalent C# code is shown in Figure 3.

Figure 3 Determining if a Portable Executable is Managed (Close Figure 3)
private static bool IsManaged(string fileName)
{
    byte [] data = new byte[4096];
    using (FileStream file =
                new FileStream(fileName, FileMode.Open, FileAccess.Read))
    {
        file.Read(data, 0, data.Length);
    }
    if ((data[1] << 8 | data[0]) != 0x5a4d) return false;

    int winNtHdr = data[63]<<24 | data[62]<<16 | data[61]<<8 | data[60];
    if ((data[winNtHdr+3]<<24 | data[winNtHdr+2]<<16 |
        data[winNtHdr+1]<<8 | data[winNtHdr]) != 0x4550) return false;

    int lightningAddr = winNtHdr + 24 + 208, sum=0, top=lightningAddr+8;
    for(int i=lightningAddr; i<top; i++) sum |= data[i];
    return sum != 0;
}

Q I have a bunch of foreach loops in my C# application. When I look at the compiler-generated Microsoft intermediate language (MSIL), I see that some include a try/finally block, but I didn't have any in my original code. Why were these added?

A Good question. Remember that a foreach loop is used to iterate over collection types. This is accomplished by retrieving an enumerator and using its MoveNext operation to continually walk the list and its Current property to retrieve the current item from the list. The enumerator is retrieved using the collection's GetEnumerator method. Since this enumerator might implement IDisposable, the C# compiler needs to ensure that it is disposed as soon as the iteration has completed. As a result, it generates the loop in a try block and then attempts to dispose of the enumerator in the finally block. If the return type of the GetEnumerator method is one that implements IDisposable, the compiler will generate a finally block containing code similar to this:

((IDisposable)enumerator).Dispose();

However, if the enumerator type returned does not implement IDisposable, the compiler still has to attempt to dispose the object since it can't be sure that a derived type (which can be returned from the function) doesn't implement IDisposable. Consequently, the compiler will generate a finally block containing several lines of code similar to the following:

IDisposable disposable = enumerator as IDisposable;
if (disposable != null) disposable.Dispose();

Since most enumerable classes have a GetEnumerator method that returns IEnumerator (which does not implement IDisposable), the previous code is the most commonly generated. In the rare case where the return type does not implement IDisposable and is sealed (meaning that no types are derived from the return type), the compiler doesn't need to implement a try/finally since there is nothing to be disposed.

Q I want to enumerate all of the objects in an object hierarchy, but there's a good chance there are cyclical references and I'm not sure how to avoid them. Does the .NET Framework provide anything to help me out in this situation?

A Iterating over all of the objects in an object hierarchy and serializing them out in some fashion is exactly what the remoting formatters like BinaryFormatter do. As a result, there are some helpful classes in the Framework that make this task simple. The ObjectIDGenerator class in the System.Runtime.Serialization namespace is a specialized table that's able to keep track of objects and unique identifiers for them. You can ask it for the ID of an object, and it will either give you the existing ID for an object it's already seen or generate a new ID for an unseen object. This makes it very easy to avoid cycles because it prevents you from iterating over the same object twice. In addition, the collection classes in the System.Collections namespace make it easy to track objects that still need to be examined.

The code in Figure 4 implements my solution. The ObjectWalker class provides an implementation for an IEnumerator that makes it easy to walk the entire object hierarchy using a foreach loop to enumerate all of the objects:

foreach(object obj in new ObjectWalker(hierarchy))
{
    Console.WriteLine("{0}:\t{1}", obj.GetType(), obj);
}

Figure 4 Enumerating an Object Hierarchy (Close Figure 4)
public class ObjectWalker : IEnumerable, IEnumerator
{
    private object _obj, _current;
    private Stack _toWalk;
    private ObjectIDGenerator _idGen;

    public ObjectWalker(object obj) {
        if (obj == null) throw new ArgumentNullException("obj");
        _obj = obj;
        Reset();
    }

    public IEnumerator GetEnumerator() { return this; }

    public void Reset() {
        _toWalk = new Stack();
        _idGen = new ObjectIDGenerator();
        Schedule(_obj);
    }

    public object Current { get { return _current; } }

    private void Schedule(object toSchedule) {
        if (toSchedule == null) return;
        bool firstOccurrence;
        _idGen.GetId(toSchedule, out firstOccurrence);
        if (firstOccurrence)
        {
            if (toSchedule.GetType().IsArray)
            {
                foreach(object arrItem in ((Array)toSchedule))
                    Schedule(arrItem);
            }
            else _toWalk.Push(toSchedule);
        }
    }

    public bool MoveNext()
    {
        if (_toWalk.Count == 0) return false;
        if (!IsTerminalObject(_current = _toWalk.Pop()))
        {
            foreach(FieldInfo fi in _current.GetType().GetFields(
                BindingFlags.Instance | BindingFlags.Public |
                BindingFlags.NonPublic))
            {
                object toSchedule = fi.GetValue(_current);
                if (toSchedule != null) Schedule(toSchedule);
            }
        }
        return true;
    }

    private bool IsTerminalObject(object data)
    {
        Type t = data.GetType();
        return t.IsPrimitive || t.IsEnum || t.IsPointer || data is string;
    }
}

Iteration begins by pushing the root object onto a stack. Each time MoveNext is called, the code pops an object from the stack and stores it as the next value to be returned from the Current property. In addition, it uses reflection to determine whether there are any fields on the object that need to be enumerated. If there are, it grabs all valid objects from those fields and uses the ObjectIDGenerator to determine if they've already been processed. If they haven't, they're added to the processing stack for future retrieval. Before checking for fields, however, the code first checks to see if the current object represents a termination condition. For this implementation, I stop recurring when a primitive (such as int or byte), a string, an enum, or a pointer is found. I stop on pointers so that I don't attempt to dig around in unmanaged memory.

As the MoveNext method uses reflection to access the contents of potentially non-public fields, this class requires the appropriate ReflectionPermission to operate successfully.

Q In my GUI application, I have a MethodInfo object that represents a method that modifies a control on my main form. This MethodInfo is obtained by another piece of the system using reflection and can represent methods with a wide variety of signatures. As this code is running on a worker thread, I need to use the control's Invoke method to execute the specified method on the thread that owns the control's underlying window handle, but to do so requires a delegate for the method I want executed. How can I convert my MethodInfo object to a delegate so that I can pass it to the Invoke method?

A If you already have a delegate class whose signature matches that of the method represented by the MethodInfo, and if that method is static, then the static Delegate.CreateDelegate method will do exactly what you need. Pass it the Type object for the delegate to instantiate and the MethodInfo for the static method to wrap, and it'll return to you an appropriate delegate instance. This delegate can then be passed to the Invoke method of the control, as shown in the following lines of code:

MyDelegate d = (MyDelegate)Delegate.CreateDelegate(
    typeof(MyDelegate), theMethodInfo);
theControl.Invoke(d, new object[]{param1, param2, ...});

If the method is not static, you can use a different overload of Delegate.CreateDelegate that takes the delegate type as before, along with the name of the method and the instance of the object on which the method should be invoked:

MyDelegate d = (MyDelegate)Delegate.CreateDelegate(
    typeof(MyDelegate), targetObj, theMethodInfo.Name);
theControl.Invoke(d, new object[]{param1, param2, ...});

However, since this still requires a delegate for the MethodInfo signature and since you may not have the correct delegate type on hand, this will not work for all scenarios. A better solution is to create a delegate class that can work for any MethodInfo. To do this, you can create a delegate whose signature matches that of MethodInfo.Invoke, which takes the target instance as the first argument (or null for static methods), and the parameters for the call in an object array as the second argument. The delegate declaration can be written as follows:

private delegate object MethodInvokeHandler(
    object target, object [] parameters);
You can then create an instance of this delegate that wraps your MethodInfo's Invoke method and pass that delegate to the Control's Invoke method, as shown here:
MethodInvokeHandler d = new MethodInvokeHandler(theMethodInfo.Invoke);
object [] parameters = new object[]{param1, param2, ...};
theControl.Invoke(d, new object[]{targetObj, parameters});
Using this approach and the delegate defined previously, you can get Control.Invoke to invoke any method for which you have a MethodInfo object.

Another solution is to dynamically generate a delegate class that works with methods that have the matching signature (Paul DiLascia and Dino Esposito discuss code generation in their columns this month). You can then use the Delegate.CreateDelegate method as described previously. Why would you want to dynamically generate a delegate when you have the solution just presented? Because the solution given is relatively slow. Rather than being able to call to the method directly (a delegate is, in a sense, an object-oriented function pointer and is efficient, although it's still much more expensive than directly calling a method), the solution has you invoking a delegate that executes an invocation that does a late-bound call to the actual method.

While dynamically generating code can incur a performance penalty, you only have to do it once up front and from then on you get the speed benefits on every call. If you'll be making many calls, it's worth the startup time you spend. In my testing where the target method performed simple arithmetic on integer arguments, using the dynamically generated delegate improved performance more than five fold. Of course, this argument is moot if the target method does any kind of real work. The overhead involved in making the call could be insignificant compared to the time taken in the method, in which case simplicity is best.

As you should do with most software projects, write the code first so that it's clean and it works. Then if it does not perform up to par, use a profiler to determine the hotspots and fix those until performance is adequate.

Q What algorithm is used by the System.Random class? Is the algorithm cryptographically secure?

A In current versions of the .NET Framework, System.Random uses an algorithm based on the subtractive method described by Donald Knuth in The Art of Computer Programming, Volume 2: Seminumerical Algorithms (Addison-Wesley, 1997). You can see the source for the System.Random class in the Shared Source Common Language Infrastructure (SSCLI), available online at System.Random Class.

System.Random is not cryptographically secure and should not be used for any applications where predictability can pose a threat. The RNGCryptoServiceProvider class in the System.Security.Cryptography namespace is, however, a cryptographically secure random number generator and can be used in situations where cryptographically strong random values are required.

Q In last month's column you answered a question on how to obtain the path to the common language runtime (CLR) installation directory. You used P/Invoke to access the unmanaged GetCORSystemDirectory function exposed from mscoree.dll. Why did you choose that approach as opposed to the other ones that are available to you?

A There are two other simple ways that I know of to get at the same directory path. The first, and easiest, is the GetRuntimeDirectory method on the System.Runtime.InteropServices.RuntimeEnvironment class. The second way is to get the type of a class that lives in mscorlib.dll (such as the object class) and then use that Type object to get the fully qualified name of the mscorlib module. On a machine running Windows XP with version 1.1 of the .NET Framework, this resolves to %windir%\microsoft.net\framework\v1.1.4322\mscorlib.dll. The System.IO.Directory.GetParent method can then be used to get the path of the parent folder.

While the RuntimeEnvironment.GetRuntimeDirectory method is certainly the easiest to use, it is not the fastest. Tests on my machine show this method to perform approximately 5 times slower than the equivalent interop approach, and the type approach to perform approximately 15 times slower. Of course, there's no need for any of these functions to be called in a long-running tight loop and, as such, efficiency isn't a huge issue. But the speed of the interop option coupled with the fact that interop is just really cool made me want to show that approach.

Send your questions and comments to netqa@microsoft.com.

Stephen Toubis the Technical Editor for MSDN Magazine.


C++ Q&amp;A: Updating Views in MFC, Spying on Events in the .NET Framework

C++ Q&A
Updating Views in MFC, Spying on Events in the .NET Framework
Paul DiLascia

Code download available at: CQA0405.exe (198KB)

Q I want to update all the child windows on one timer event which is in CMainFrame in my MDI app. The views are displaying many graphs. I am able to update only the active window with the following line of code:

GetActiveWindow()->GetActiveView()->GetDocument()
Is there any way that I can get all the child windows or all documents from the CMDIFrame class?

Makarand

A Your situation is not uncommon. Many programs that collect real-time data need to update their screens periodically. Even if your program doesn't collect data in real time, you still have to update your views any time the user does something to change the document. Fundamental to MFC's doc/view model (and all object/view models) is the idea that data and presentation are separate. Users—or real-world events—change the underlying object, data, or document; the changes are then propagated to the presentation via some sort of update-the-view-now event.

If you have several views on the same document, MFC already has a mechanism to update all the views in one fell swoop. The function to call is CDocument::UpdateAllViews, which calls CView::OnUpdate for each view open on the document. You can pass your own app-specific "hint" describing what kind of update operation to perform. For example, if you know that only the title of the document has changed, you could define an enum value CHANGED_TITLE and send it as the hint code. If your document has graphics and text you can define enums CHANGED_TEXT and CHANGED_GRAPHICS. The purpose of these hint codes is to improve performance. By giving the view a hint as to what has changed, it can more intelligently repaint only those areas of the screen that actually need repainting, potentially avoiding lengthy paint operations or screen flicker.

UpdateAllViews updates all the views attached to a document, but how do you update all documents? There's no UpdateAllDocuments function in MFC, so you have to enumerate all the docs yourself. This requires a nested traversal of document templates and documents, as shown here:

for (/* each CDocTemplate in app */) {
  for (/* each CDocument in CDocTemplate */) {
    // do something
  }
}

Since enumerating the docs is so useful, I wrote a little class, CDocEnumerator, that hides all the MFC mechanics of templates and POSITIONs. I actually wrote such a class way back in September 1995—but hey, that was almost 10 years ago! Figure 1 shows the code. CDocEnumerator makes it easy to enumerate all the open documents in your application:

CDocEnumerator it;
CDocument* pdoc;
while ((pdoc=it.Next())!=NULL) {
   // do something
}

Figure 1 CDocEnumerator (Close Figure 1)

DocEnum.h
// CDocEnumerator — Enumerates all the documents in an MFC application.
// To use:
//
//    CDocEnumerator it;
//    CDocument* pdoc;
//    while ((pdoc=it.Next())!=NULL) {
//       // do something
//    }
//
class CDocEnumerator {
private:
   CPtrArray m_doclist; // array of all docs
   int m_iPos;          // current position in array
public:
   CDocEnumerator();
   CDocument* Next();
   void Reset() { m_iPos=-1; }
};
DocEnum.cpp
// CDocEnumerator Enumerates all the documents in an application.

#include "StdAfx.h"
#include "DocEnum.h"

//////////////////
// ctor loads all doc pointers into private array.
//
CDocEnumerator::CDocEnumerator() {
   CWinApp* pApp = AfxGetApp();
   POSITION pos = pApp->GetFirstDocTemplatePosition();
   while (pos!=NULL) {
      CDocTemplate* pDocTemplate = pApp->GetNextDocTemplate(pos);
      POSITION pos2 = pDocTemplate->GetFirstDocPosition();
      while (pos2!=NULL) {
         m_doclist.Add(pDocTemplate->GetNextDoc(pos2));
      }
   }
   Reset();
}

//////////////////
// Fetch next document.
//
CDocument* CDocEnumerator::Next() {
   if (++m_iPos < m_doclist.GetCount()) {
      return (CDocument*)m_doclist[m_iPos];
   }
   return NULL;
}

What could be easier? To show how it all works in practice, I wrote a little program, UpdView, that simulates your real-time data collection program. Each UpdView document counts the seconds since it was opened. Figure 2 shows UpdView in action. If you download, build, and run UpdView, you can see that each view updates every second to show how long its document has been open. In Figure 2, the document named file2.dat has two views, both of which display the same underlying document. Each document maintains its own time-since-opened value (data); the view merely displays it (presentation). As in your own program, UpdView works by setting a timer in the main frame. The timer handler uses CDocEnumerator to tell each document to collect more data, as shown here:

void CMainFrame::OnTimer(UINT_PTR nIDEvent) {
   CDocEnumerator it;
   CDocument* pdoc;
   while ((pdoc=it.Next())!=NULL) {
      ((CMyDoc*)pdoc)->CollectMoreData();
   }
}

Figure 2 UpdView in Action
Figure 2 UpdView in Action

CMyDoc::CollectMoreData increments a simple counter. In a real program, CollectMoreData would fetch recent stock quotes, download the latest images from Mars, or read the temperature in Bill Gates's bathtub. Whatever. The important thing is that after collecting the data, CMyDoc tells its views to update themselves:

void CMyDoc::CollectMoreData()
{
  iData++; // time waits for no man...
  UpdateAllViews(NULL, 0, NULL);
}

Now MFC calls each view's OnUpdate method, which calls Invalidate/UpdateWindow to repaint the view. Since UpdView is so simple, there's no need for hints. In a real program, you might pass hint info to help the view repaint its window more efficiently.

Q I'm writing an app in C# using the Microsoft® .NET Framework. I understand how events work, but sometimes it's hard to see which events get raised when. When I'm programming in C++, I can always run Spy++ to see when windows messages get sent. Even without Spy++, I can write a window procedure that displays the message as TRACE output. For example:

LRESULT MyWndProc(HWND hwnd, UINT msg, WPARAM wp, LPARAM lp)
{
   TRACE(_T("Message: %04x, wp=%04x, lp=%04x\n"), msg, wp, lp);
   •••
}

I tried to do something like this for events in C#, but I couldn't figure it out because there's no central event handler. Is there some way to see the events, or is there a third-party Spy tool for .NET?

Francine Wiggins

A I'm not aware of any event spy tool for the .NET Framework, but it's not too hard to write some code that spies on events. Notice I said not too hard—because it turns out it's a bit harder than the WndProc example for messages.

Just to make sure everyone understands, let me underscore the difference between window messages and Framework events. In Windows®, the operating system communicates with your program by sending messages to your window procedure (WndProc) when something happens. It's easy to spy on messages because every message goes through a single callback: namely, the window procedure. This is true whether you're programming in C/C++ or the .NET Framework. In .NET, you can spy on window messages by overriding Control.WndProc, the same as in your C example. But while window messages are conceptually a kind of event (and many programmers call them events), and while many Framework events correspond to window messages (for example, OnClick is the same as WM_COMMAND/BN_CLICKED), Framework events use a different mechanism entirely, one based on delegates instead of a universal callback procedure. Whereas every window message goes through the same WndProc, every Framework event has its own delegate. So how can you spy on events when there's no central handler? How do you even know which events exist to be handled?

Finding the events is relatively easy. One of the most powerful features of the .NET Framework is reflection, which is a fancy way to say that .NET is self-conscious. For example, to find all the public events the Form class can raise, you can write:

EventInfo[] allEvents = typeof(Form).GetEvents();

This fetches an array of EventInfo objects, one for each event defined in the Form class. Each EventInfo object contains information describing the event. EventInfo.Name is the name of the event (for example, Click or Closing). EventInfo.EventHandlerType is the Type of the handler (delegate) required to handle the event. You can even call EventInfo.AddEventHandler to add another handler. This might lead you to suspect that one way to spy on events would be to write a generic handler and hook it up to every event in the class you want to spy on. This approach is on the right track, but the details are somewhat complex. How do you write the generic handler? What's its signature? The obvious answer would be:

// generic event handler—for any event
public void OnAnyEvent(Object sender, EventArgs e);

However, not all handlers use EventArgs. For example, the Form.Closing event requires a handler with CancelEventArgs:

void OnClosing(Object sender, CancelEventArgs e);
CancelEventArgs is, of course, derived from EventArgs, so it's perfectly legal to pass CancelEventArgs to a method that expects EventArgs—but delegates in version 1.1 of .NET don't work that way. When you add an event handler with += or EventInfo.AddEventHandler, you must supply a delegate whose type exactly matches the event handler type. This means you can't use a single generic handler for all events. At minimum, you'll need one handler for each event delegate signature.

But it's even worse than that. Even if you could use a generic handler, how will the handler know which event was fired? The whole point of a spy tool is to report which event fired. If nothing else, the spy tool should display the name of the event. But the Framework doesn't pass information about which event was fired because this information is implicit in the handler itself. You know it's the Fooble event because your OnFooble handler was called. If you use the same handler for multiple events, you lose the ability to distinguish them. So the only way to spy on events is to write a different handler for each event. This would seem to present an insurmountable problem: how can you possibly write the code when you have no idea until run time which object you're expected to spy on and what events it exposes?

By writing the code itself at run time, of course! Reflection doesn't only mean you can query objects in the system; it also means you can create them. There are several ways to generate code in the .NET Framework. System.CodeDom provides a high-level, language-independent code model you can use to create code objects such as assemblies, modules, classes, and methods, then emit them in your favorite language using classes like Microsoft.CSharp.CSharpCodeProvider or Microsoft.VisualBasic.VBCodeProvider. Another way to generate code is to explicitly write out the code in a language of your choice to a file or into a StringBuilder and use the Process class to invoke the appropriate command-line compiler (such as C# compiler csc.exe). In fact, the friendly Redmondtonians assure me "this is done all over the Framework." For example, XmlSerializer uses csc.exe to compile dynamically generated C# that is optimized for serializing and deserializing specific types.

The Code Document Object Model (CodeDOM) is designed for code generators that need to deal with code abstractly in order to emit it in multiple languages from a single internal representation and optionally compile it. Running a compiler like csc.exe entails a performance hit, which may nevertheless be justified by repeatedly invoking the code—the same way you might compile a regular expression (RegEx) you plan to use frequently. But there's still another way to generate code on the fly: you can use System.Reflection.Emit.ILGenerator to generate low-level Microsoft intermediate language (MSIL) instructions directly. It's definitely not for the fainthearted, but it's quick and efficient and occasionally just the ticket to pull some swift software trick—like writing an event spy.

To show how, I wrote a class, EventSpy, that reports every event raised by some target object, the spyee. I also wrote a program called SpyTest that shows how to use it. (Both EventSpy and SpyTest are available for download from the MSDN® Magazine Web site). Using EventSpy is easy: just instantiate and add a handler for the SpyEvent event, like so:

// spy on myself
spy = new EventSpy("MySpy", this);
spy.SpyEvent += new SpyEventHandler(OnEventSpy);

EventSpy raises a SpyEvent event every time the target object (the spyee) raises an event of any kind. When this happens, SpyEventArgs.EventName is the name of the event and SpyEventArgs.EventArgs holds the original event args. What you do with the event is up to you. SpyTest reports the event to the diagnostic stream:

// in main form
private void OnEventSpy(Object sender,
  SpyEventArgs e)
{
   Trace.WriteLine(String.Format("{0}: On{1}: {2}",
      sender.GetType(), e.EventName, e.EventArgs));
}

Figure 3 shows a typical run. Go ahead, download the code, and try it yourself—it really works! EventSpy also has a DumpEvents function that lists all the events your target class exposes.

Figure 3 SpyTrace
Figure 3 SpyTrace

If all you want to do is use EventSpy to spy on events, you can quit reading and go directly to the MSDN Magazine Web site to download the source. For those masochists among you who may be tempted to implement something like EventSpy on your own (good luck!), I offer a brief overview of how EventSpy works.

The upshot of the event discussion is that in order to spy on events, you have to dynamically generate a class that looks something like Figure 4. Before you create the class, you have to create an assembly and module for it. System.Reflection.Emit has the AssemblyBuilder, ModuleBuilder, TypeBuilder, FieldBuilder, ConstructorBuilder, and MethodBuilder classes to build everything you need. Here's how to create an assembly on the fly:

AssemblyName an = new AssemblyName();
an.Name = "EventSpyHelper";
AssemblyBuilder asm = AppDomain.CurrentDomain
   .DefineDynamicAssembly(an,AssemblyBuilderAccess.Run);

Figure 4 EvSpyImpl (Close Figure 4)
//////////////////
// This is what the dynamic helper class would look like if it were
// written in C#
//
public class EvSpyImpl : Object
{
   private EventSpy spy;

   public EvSpyImpl(EventSpy s)
   {
      spy = s;
   }

   public void OnEventFoo(object o, EventFooArgs e)
   {
      spy.ReportEvent("EventFoo", o, e);
   }

   public void OnEventBar(object o, EventBarArgs e)
   {
      spy.ReportEvent("EventBar", o, e);
   }

   ••• // ditto for all events in target object type.

}

Creating the module is even simpler. Creating the class, fields, and methods is likewise mostly a straightforward matter of giving each thing a name and perhaps some flags and calling the appropriate builder method. The hardest part comes when it's time to generate the actual code—in this case, the constructor and event handlers for each event. The key is a class called ILGenerator. Here's how EventSpy writes the event handlers in Figure 4:

// create a new method OnEventXxx
MethodBuilder mthd = helperClass.DefineMethod(...);
// generate its code
ILGenerator il = mthd.GetILGenerator();
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldfld,fld);
il.Emit(OpCodes.Ldstr,ev.Name);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Ldarg_2);
il.Emit(OpCodes.Callvirt, miReportEvent);
il.Emit(OpCodes.Ret);

Each call to ILGenerator.Emit emits a single MSIL instruction. But how did I know which instructions to emit? Do you really think I know how to program in MSIL? Of course not! You don't have to speak MSIL to emit code. All you have to do is write the code you want in C# (or any other .NET-targeted language), compile it, and use ILDASM to inspect the generated MSIL. This is what I did; Figure 5 shows part of the ILDASM dump for the code in Figure 4. As you can see, the MSIL is the same as what appears in the previous snippet.

Figure 5 ILDASM Dump
Figure 5 ILDASM Dump

Once you know the trick to discovering the right MSIL, the rest is fairly straightforward, albeit rather bug-prone: any little mistake you make is bound to send your system spiraling into psychosis with a death message that does little to suggest where your error lies. Like I said, MSIL is not for the fainthearted. At least nobody's life is hanging in the balance. Your job, maybe.

EventSpy is fairly primitive. You can't turn spying on or off, you can only spy on instance (as opposed to static) events and you have to create a new EventSpy instance for every object you want to spy on. Well, what do you expect from a free download? But EventSpy gets the job done when you're debugging, and I've used it successfully a number of times to see what's happening in the event stream. I leave it to you to add more features. My main mission is to tip you off about System.Reflection.Emit and dynamic code generation. Figure 6 summarizes the different ways to generate code dynamically in .NET. To learn more, check out the documentation or the article "Bring the Power of Templates to Your .NET Applications with the CodeDOM Namespace" by Adam J. Steinert. Happy programming!

Figure 6 Generating Code Dynamically in .NET (Close Figure 6)

Technique | Description
CodeDOM | For code generators that need to deal with code abstractly, then emit high-level source code in multiple languages. Very high-level.
csc.exe, vbc.exe | You can invoke the C# compiler for C# code that you generate dynamically. Entails a small performance hit. Use when your code is too complex for manually written MSIL.
ILGenerator | Use to generate MSIL instructions on the fly. Most efficient, but also most low-level, error-prone, and difficult to debug. Use when the code you want to generate is very simple.

Send your questions and comments for Paul to cppqa@microsoft.com.

Paul DiLascia is a freelance writer, consultant, and Web/UI designer-at-large. He is the author of Windows++: Writing Reusable Windows Code in C++ (Addison-Wesley, 1992). Paul can be reached at http://www.dilascia.com.


Resource File: Microsoft Newsgroups for Developers

Resource File
Microsoft Newsgroups for Developers

Contents
Using Outlook Express as a Newsreader

Microsoft offers more than 1,900 public and 500 private newsgroups that cover all aspects of Microsoft products and technologies. An individual newsgroup might get up to 15,000 postings per month, with developers answering customer questions, asking about specific product features, and getting product feedback. The newsgroups have unlimited reach; public newsgroups are fed to Usenet (the collection of all posts publicly distributed through the NNTP protocol) and live on servers outside of Microsoft. Private newsgroups for MSDN members are available by connecting directly to Microsoft newsgroup servers. These require an account and password for each set of newsgroups. MSDN newsgroups are a part of the Usenet hierarchy and are available locally to customers via ISPs around the world, as well as through an online reader at MSDN Online.

The newsgroup community is made up of expert Microsoft developers, Microsoft support professionals, MSDN subscribers, Microsoft Most Valuable Professionals (MVPs), IT professionals, and other developers exchanging ideas and providing programming solutions.

The MSDN Managed Community Newsgroups are made up of about 220 newsgroups. In managed newsgroups, MSDN subscribers are guaranteed priority service, receiving responses to their technical questions within two business days.

Managed newsgroups have been established for discussions ranging from Microsoft .NET development and data access to Web services and graphics. MSDN Managed Client Newsgroups include microsoft.public.platformsdk.msi, microsoft.public.dotnet.framework.windowsforms, and microsoft.public.win32.programmer.networks. Platformsdk.msi experts provide feedback on issues ranging from basic MSI customization questions to programmatically accessing the MSI using .NET-based applications. In the dotnet.framework.windowsforms newsgroups, users will find answers to questions regarding data adapters, DataGrids, and Windows Forms. Issues in the win32.programmer.network relate to the networking API, Peer-to-Peer (P2P), RAS, Remote Procedure Call (RPC), SNMP, Synchronization Manager and System Event Notification Service, UPnP Device Control/Host API, and more.

In addition, MSDN Managed Architecture Newsgroups include microsoft.public.vsnet.general and microsoft.public.dotnet.framework. The vsnet.general (Visual Studio .NET) newsgroup provides resources for debugging .NET-based applications, assistance with Web projects, including project creation, programming, and troubleshooting in Visual Basic and C#, and other application deployment concerns.

Just as in any public area of the Internet, it pays to be careful when disseminating your personal information. Since newsgroups are publicly available, any real e-mail address could be harvested for spam mail lists. Microsoft advises all news-group and community participants to use a modified version of their address to mitigate floods of unwanted e-mail, for example mmeditor_NO_SPAM@microsoft_NOSPAM.com. Used wisely, newsgroups provide an extremely efficient and valuable resource for quickly getting tech-nical help from individuals who have encountered the same problems that you have.

Using Outlook Express as a Newsreader

Outlook Express, which is part of Windows XP, can serve as a full-featured newsreader. To connect to the Microsoft public newsgroups, take the following steps:

  1. Go to Tools | Accounts, click the News tab, and click Add | News.
  2. Type in your name, then enter an e-mail address that's clear to other people but confusing to an automated reader.
  3. When asked for the NNTP (news) server, enter msnews.microsoft.com and click OK.

You will be prompted to download newsgroups from the account you just added. Say yes; the first time you do this might be slow, but on most machines it'll take just a few seconds. This produces a list of all Microsoft newsgroups; you can filter by keyword and subscribe to newsgroups. Subscribing means only that you've told your reader that a newsgroup is of interest to you, and that you want it to be checked regularly for new messages. Double-click the groups to which you want to subscribe.

Now whenever you select the msnews.microsoft.com account you created, then click the Synchronize Account button, all the new messages in all of your subscribed newsgroups will be downloaded for local reading. You can easily change these settings; see your Outlook Express documentation for more information.