 MSDN Magazine April 2005

MSDN Magazine April 2005

Wrappers: Use Our ManWrap Library to Get the Best of .NET in Native C++ Code

By now you're probably used to reaching into the .NET Framework using Managed Extensions with your C++ code. But what do you do if you have legacy apps that use older versions of the compiler, or if you want to avoid some of the CLR requirements? Well, you can wrap Framework classes in a native way so you can use them in any C++/MFC app without /clr. Paul DiLascia shows you how.

Paul DiLascia Security: Unify Windows Forms and ASP.NET Providers for Credentials Management

The .NET Framework 2.0 provides custom credentials management to ASP.NET apps out of the box. Using it, you can easily authenticate users without using Windows accounts. In this article the author presents a set of helper classes that let a Windows Forms application use the ASP.NET credentials management infrastructure as easily as if it were an ASP.NET application.

Juval Lowy Speed: NGen Revs Up Your Performance with Powerful New Features

In the .NET Framework 2.0, NGen has been greatly improved and offers a number of compelling new features to make it easier and faster to deploy high-performance managed applications. This article introduces some of these new features and explains how and when you should use NGen to improve the performance of your apps.

Reid Wilkes Escape DLL Hell: Simplify App Deployment with ClickOnce and Registration-Free COM

DLL Hell that results from versioning conflicts is generally not a problem in .NET because components do not require registration, and are either totally isolated to an application or are managed in a well-defined side-by-side way with the help of the Global Assembly Cache. But if you're still using COM, wouldn't it be great if you could deploy your existing COM components using a model similar to the .NET Framework? In Windows XP, you can, and this article explains how.

Dave Templin Flex Your Data: Teach Old Data New Tricks with the Fully Loaded Advances in ADO.NET 2.0

ADO.NET 2.0 will streamline your data access development efforts. Its new features let you work better with XML and SQL Server, and they offer lots of other improvements as well. This article takes a detailed look at some of the more interesting enhancements and focuses on performance and flexibility.

Julia Lerman Team Up!: Get All Your Devs in a Row with Visual Studio 2005 Team System

Visual Studio 2005 Team System includes tools to help project managers, architects, developers, testers, and even development managers. In this article, the author provides an overview of Team System using the Visual Studio 2005 December Community Technology Preview, shows you how to set up a development project, and explores the typical development process, from inception through testing.

Chris Menegay Editor's Note: 20 Years and Still Going Strong!

Microsoft Systems Journal (MSJ) was founded in 1986. At the time, no other publication focused specifically on the needs of programmers who were using Microsoft tools and technologies. Two issues were produced in the first year, each with five articles.

New Stuff: Resources for Your Developer Toolbox

DbNetGrid. Net 2. 71 by DbNetLink Limited, is a Web-based grid component that works with DHTML behaviors that support both ASP and ASP. NET. The integrated DbNetGrid GUI designer application lets you create your grids without writing any code.

Marnie Hutcheson Web Q&A: Get Authentication Type, Get Screen Resolution, and More

Edited by Nancy Michell Advanced Basics: Remembering User Information in Visual Basic .NET

Many applications need to store user-specific settings to be persisted between sessions. But how do you go about saving and restoring these settings in your Microsoft® . NET Framework-based application? It's not all that easy to find the correct answer.

Duncan Mackenzie Cutting Edge: A New Grid Control in Windows Forms

Grid controls are essential in many of today's apps. Until now, though, most developers using Visual Basic® have had to buy third-party components to get an effective, easy to use grid component. The Windows® Forms DataGrid turned out to lack too many features for the average developer.

Dino Esposito Test Run: Lightweight UI Test Automation for ASP.NET Web Apps

The release of ASP. NET revolutionized Web development and made it easy to create full-featured Web applications. Visual Studio® 2005 and ASP. NET 2. 0 will let you add even more functionality to your applications, but the more features a Web application has, the more important testing becomes.

James McCaffrey Service Station: Developing .NET Web Services with Beta 2

Version 2. 0 of the Microsoft® . NET Framework makes numerous improvements at various levels in the Web services protocol stack. In addition, better tool support and an increased focus on interoperability make your life easier.

Aaron Skonnard .NET Matters: ICustomTypeDescriptor, Part 1

Stephen Toub Pure C++: Generic Programming Under .NET

Visual Studio® 2005 brings the type parameter model of generic programming to the Microsoft® . NET Framework. Parameterized types are, of course, standard fare for C++ programmers. So, for those who are unfamiliar with them, I'll give a brief introduction to generic programming in my next few columns.

Stanley B. Lippman C++ at Work: Form Validation with Regular Expressions in MFC

I thought I'd use this month's column to describe an interesting app I built using the RegexWrap library described in my article "Wrappers: Use Our ManWrap Library to Get the Best of . NET in Native C++ Code" in this issue.

Paul DiLascia { End Bracket }: Creating a Custom Metrics Tool

Metrics play an important role in our lives. Even if we don't realize it or characterize it as such, many daily activities have the potential to be quantified to some degree. So it's not surprising that metrics play an even greater role in the workplace, where there are goals and a bottom line and where much of a day's activity can be summarized in numbers.

Stephen Toub


Wrappers: Use Our ManWrap Library to Get the Best of .NET in Native C++ Code

Wrappers

Use Our ManWrap Library to Get the Best of .NET in Native C++ Code

Paul DiLascia
This article discusses:
  • Using managed classes from native C++ without /clr
  • GCHandle, gcroot, and building mixed-mode DLLs
  • Regular expressions in the .NET Framework
This article uses the following technologies:
C++ and the .NET Framework
Code download available at: C.exe (248KB)
Contents
A Simple Question
ManWrap
Hiding the Managed Stuff
RegexWrap
Coping with Collections
Dealing with Delegates
Handling Exceptions
Wrapping Arrays
Encapsulating Enums
Building Mixed-Mode DLLs
Fun with RegexWrap
Conclusion

T
he Managed Extensions for C++ make it possible to mix native and managed code freely, even in the same module. Wow, life is good! Compiling with /clr, however, has consequences you may not want. It forces multithreading and dispenses with some useful runtime checks. It interferes with MFC's DEBUG_NEW, and some .NET Framework classes may conflict with your namespace. And what if you have a legacy application that uses an older version of the compiler that doesn't support /clr? Isn't there some way to reach into the Framework without the Managed Extensions? Yes!

In this article, I'll show you how to wrap Framework classes in a native way so you can use them in any C++/MFC app without /clr. As my test case, I'll wrap the regex classes from the .NET Framework in a DLL and implement three MFC programs using it. You can use RegexWrap.dll to add regular expressions to your own C++/MFC applications, or use my ManWrap tools to wrap your own favorite Framework classes.

A Simple Question

It all started when reader Anirban Gupata sent me a simple question: is there some regular expression library he could use in his C++ app? My response was "Sure, there are several, but .NET already has a Regex class. Why not use that?" Regular expressions are so fabulously useful (see the sidebar "Regular Expressions") they're enough to make even the most diehard C++ fan yearn for the .NET Framework. So I wrote a little program called RegexTest to see what Regex can do. Figure 1 shows it running. You enter a regular expression and input string, press a button, and RegexTest displays the Matches, Groups, and Captures. It all happens in a single function called FormatResults(see Figure 2), which formats a big CString when the user presses OK. FormatResults is the only function in RegexTest that calls the Framework, so it's easy to put in its own module compiled with /clr.

Figure 2 FormatResults.cpp (Close Figure 2)
////////////////
// Format match results into MFC CString.
// From original version of RegexTest not using wrappers.
// This shows the vanilla way to call managed objects using Managed
// Extensions.
//
CString FormatResults(LPCTSTR lpszRegex, LPCTSTR lpszInput)
{
   CString result;
   CString temp;
   Regex* r = new Regex(lpszRegex);
   MatchCollection* mc = r->Matches(lpszInput);
   int n = mc->Count;
   temp.Format(_T("Number of Matches: %d\n"), n);
   result += temp;
   for (int i=0; i<n; i++) {            // for each lpszInput:
      Match *m = mc->Item[i];
      temp.Format(_T("Match %d at %d: %s\n"), i, m->Index,
         CString(m->Value));
      result += temp;

      // Show groups
      GroupCollection *gc = m->Groups;
      for (int j=0; j<gc->Count; j++) {
         Group *g = gc->Item[j];
         if (g->Success) {
            temp.Format(_T(" Group %d match at %d: %s\n"), j, g->Index,
               CString(g->Value));
         } else {
            temp.Format(_T(" Group %d failure\n"), j);
         }
         result += temp;
      }

      // Show captures
      CaptureCollection *cc = m->Captures;
      for (j=0; j<cc->Count; j++) {
         Capture *c = cc->Item[j];
         temp.Format(_T(" Capture at %d: %s\n"), c->Index,
            CString(c->Value));
         result += temp;
      }
      result += _T("\n");
   }

   // Use Regex to convert all newlines to \r\n for edit control
   r = new Regex("\n");
   result = r->Replace(result,"\r\n");

   return result;
}
Figure 1 RegexTest
Figure 1 RegexTest

If my only goal was to write RegexTest, I'd be finished. But writing RegexTest got me thinking. Real apps need to hold their objects longer than the duration of a function call. Suppose I want to store my regular expression in my window class? Good idea, but unfortunately you can't store __gc pointers in unmanaged memory:

class CMyWnd ... {
protected:
  Regex* m_regex; // NOT!
};
Instead, you need GCHandle or its templatized cousin, gcroot:
class CMyWnd ... {
protected:
  gcroot<Regex*> m_regex; // swell!
};

GCHandle and gcroot are well described in the docs and elsewhere (see "Tips and Tricks to Bolster Your Managed C++ Code" by Tomas Restrepo in the February 2002 MSDN®Magazine), so all I'll say here is that gcroot employs templates and C++ operator overloading, which makes handles look and act like pointers. You can copy, assign, and convert; and gcroot's dereference operator-> lets you use pointer syntax to invoke your managed object:

m_regex = new Regex("a+");
Match* m = m_regex->Match("S[aeiou]x");
Managed objects, C++, and you—all together in one happy family. What's not to love?

The only possible complaint in this situation is that to use gcroot, you need /clr. How else can the compiler even know what gcroot/GCHandle are? This doesn't mean your code must be managed; you can use #pragma unmanaged to generate native code. But as I mentioned earlier, using /clr has repercussions. It forces multithreading (which breaks some shell functions like ShellExecute), and it's incompatible with options like /RTC1, which generates useful runtime checks for stack errors and buffer under- and overruns (see John Robbins' August 2001 Bugslayer column). If you use MFC, you've probably already had problems with /clr and DEBUG_NEW. You can also have namespace collisions with functions like MessageBox, which exists in the .NET Framework, MFC, and the Windows® API.

In my January column, I showed how to create a project where only one module uses /clr. That works fine when your Framework calls are localized to a few functions like FormatResults that you can put in a separate file, but it fails if you have broadly used classes with gcroot members, because many more modules #include your class. So you flip the /clr switch and—whoosh!—your whole app is beamed to Managedland. It's not that /clr is so terrible, it's that there are times when you may prefer to stay native. Can't you have your Framework classes and native compilation too? Yes, but you need a wrapper.

ManWrap

ManWrap is a set of tools I built to wrap managed objects in native C++ classes. The idea is to create classes that use the Managed Extensions internally to call the Framework, but expose a purely native interface to the outside world. Figure 3 illustrates this approach. You need the Managed Extensions to build the wrapper itself, and apps using the wrappers need the Framework to run, but the apps themselves compile in native mode without /clr. How does ManWrap accomplish this feat? The best way I can explain is to simply begin blithely wrapping and see what happens. Since every .NET object derives from Object, I'll start there:

class CMObject {
protected:
  gcroot<Object*> m_handle;
};

Figure 3 ManWrap
Figure 3 ManWrap

CMObject is a native C++ class that holds a handle to a managed object. To do anything useful, however, I need some standard constructors and operators, and while I'm at it, I'll wrap Object::ToString, which will come in handy. Figure 4 shows my first stab. CMObject has three constructors: a default constructor, a copy constructor, and a constructor from Object*. There's also an assignment operator from CMObject and a ThisObject method that returns the underlying Object, which the dereference operator-> uses. These are the minimal methods every wrapper class needs. The wrapper methods themselves (in this case, ToString) are trivial:

CString CMObject::ToString() const
{
  return (*this)->ToString();
}

Figure 4 CMObject—First Pass (Close Figure 4)
//////////////////
// First-attempt implementation for CMObject, base for all wrappers.
// Class declaration — .h file
//
class CMObject {
protected:
   gcroot<Object*> m_handle; // handle to managed object
public:
   // fns whose signatures use managed types
   CMObject(Object* o) : m_handle(o) { }
   Object* ThisObject() const { return (Object*)m_handle; }
   Object* operator->() const { return ThisObject(); }

   // fns whose signatures use native types
   CMObject() { }
   CMObject(const CMObject& o) : m_handle(o.m_handle) { }
   CMObject& operator=(const CMObject& r) {
      m_handle = r.m_handle;  // copies underlying GCHandle.Target
      return *this;
   }

   // wrapped methods/properties
   CString ToString() const {
      return (*this)->ToString();
   }
};

It's only one line of code, but there's more happening than meets the eye: (*this)-> invokes gcroot's dereference operator-> which casts the underlying GCHandle.Target to an Object*, whose ToString method returns a managed String. The Managed Extensions and IJW (It Just Works) interop magically convert the string to LPCTSTR, which the compiler uses to construct a CString on the stack automatically, because CString has a constructor-from-LPCTSTR. Isn't C++ a really amazing language?

So far, CMObject is pretty useless because all I can do is create null objects and copy them. How much good is that? But CMObject isn't designed for the single life; it's designed to serve as the base for more wrappers. Let's try another class. The Framework's Capture class is a very simple class that represents a single subexpression match in a regular expression. It has three properties: Index, Value, and Length. In order to wrap it, I'll do the obvious thing: Capture derives from object, so I'll derive CMCapture from CMObject, as shown here:

  class CMCapture : public CMObject {
    // now what?
  };

CMCapture inherits m_handle from CMObject, but m_handle is gcroot<Object*>, not gcroot<Capture*>. So, do I need a new handle? No. Capture derives from Object, so a gcroot<Object*> handle can hold a Capture object too.

   class CMCapture : public CMObject {
   public:
     // call base ctor to initializes m_handle
     CMCapture(Capture* c) : CMObject(c) { }
   };

CMCapture needs all the same constructors and operators as CMObject, and I have to override ThisObject and operator-> to return the new type.

Capture* ThisObject() const
{
  return static_cast<Capture*>((Object*)m_handle);
}
The static_cast is safe because my interface guarantees that the underlying object can only be a Capture object. Wrapping the new properties is again trivial. For example:
int CMCapture::Index() const
{
  return (*this)->Index;
}

Hiding the Managed Stuff

So far, so good. I have what seems like a brainless way to wrap managed objects in C++. But my C++ classes still require /clr to compile. My whole goal is to build native wrappers so apps using them don't need /clr. To get rid of the /clr requirement I have to hide all the managed stuff from native clients. For example, I have to hide the gcroot handle itself because native code doesn't know what a GCHandle is. What to do?

I once had a math professor who said every proof is either a bad joke or a cheap trick. What I'm about to describe is definitely in the cheap trick category. The key to ManWrap is the special predefined compiler symbol _MANAGED, which is 1 when compiling with /clr and otherwise undefined. _MANAGED makes hiding the handle trivial, as shown here:

#ifdef _MANAGED
# define GCHANDLE(T) gcroot<T>
#else
# define GCHANDLE(T) intptr_t
#endif
Now I can revise CMObject like so:
class CMObject {
protected:
  GCHANDLE(Object*) m_handle;
  •••
};

Modules compiling with /clr (namely, the wrapper itself) see a gcroot<T> handle. C++ apps not using /clr see only a (possibly 64-bit) raw integer. Pretty clever, eh? I told you it was a cheap trick! If you're wondering why intptr_t, it's because gcroot's one-and-only data member, its GCHandle, is specifically designed to operate as an integer, with op_Explicit conversions to and from IntPtr. intptr_t is merely the C++ equivalent of IntPtr, so whichever way you compile CMObject (either native or managed), it has the same size in memory.

Size is one thing, but there's more to going native than just size. What about the other managed stuff, like the methods labeled "signatures use managed types" in Figure 4? I can use _MANAGED to hide them too:

#ifdef _MANAGED
// managed-type methods here
#endif

By "managed-type methods" I mean methods whose signatures use managed types. Enclosing them in #ifdefs makes them invisible to native clients. In Nativeland, these functions don't exist. These are things like a constructor that takes an argument of type X, where X is managed, and operator-> that native code has no use for, nor ability to comprehend or compile. I need these methods only inside the wrapper itself—which compiles with /clr.

So I've hidden the handle and all the "managed-type" functions. Is there anything else? What about the copy constructor and operator=? Their signatures use native types, but their implementations access m_handle:

class CMObject {
public:
  CMObject(const CMObject& o) :
    m_handle(o.m_handle) { }
};

Suppose I have a CMObject obj1 and I write CMObject obj2=obj1. The compiler invokes my copy constructor. This works fine in managed code where m_handle is gcroot<Object*>, but in native code m_handle is intptr_t, so the compiler copies the raw integer. Oops! You can't copy a GCHandle as if it were an integer. You have to go through proper channels by reassigning the GCHandle's Target or letting gcroot do it for you. The problem is my copy constructor is defined inline. All I have to do is make it a true function and move the implementation to the .cpp file:

// in ManWrap.h
class CMObject {
public:
  CMObject(const CMObject& o);
};
// in ManWrap.cpp
CMObject::CMObject(const CMObject& o)
  : m_handle(o.m_handle) {
}

Now when the compiler invokes the copy constructor, it calls into ManWrap.cpp, where everything executes in managed mode and treats m_handle like the gcroot<Object*> it truly is, not the lowly intptr_t native clients see; gcroot sets the GCHandle's Target. The same applies to the operator= and the wrapper functions themselves, like CMObject::ToString or CMCapture::Index. Any member function that accesses m_handle must be a true function, not inline. The function call is the price you pay for going fully native. (Life is tough, I know.) But you can't have everything and the overhead is insignificant in all but the most performance-critical situations. If you need to manipulate 17 trillion objects in real time, by all means, don't use wrappers! But if all you want is to access a few .NET Framework classes without /clr, the function call is relatively insignificant.

Figure 5 shows the final CMObject from ManWrap. Once you understand how CMObject works, creating new wrappers is merely a matter of cloning the process: derive from CMObject, add the standard constructors and operators, use _MANAGED to hide the ones that use managed types, and implement the rest as true functions. The only difference with derived objects is you can leave the copy constructor and operator= inline because they can invoke their base class without accessing m_handle directly:

class CMCapture : public CMObject {
public:
  CMCapture(const CMCapture& o) : CMObject(o) { }
};

Figure 5 CMObject (Close Figure 5)

CMObject.h
//////////////////
// Wrapper for Object, base of .NET hierarchy.
// CMObject doesn't use DECLARE_WRAPPER because it has no base class.
//
class CMObject {
protected:
   GCHANDLE(Object*) m_handle;
public:
#ifdef _MANAGED
   // visible to managed clients only: anything that deals with __gc
   // objects
   CMObject(Object* o) : m_handle(o) { }
   Object* ThisObject() const { return (Object*)m_handle; }
   Object* operator->() const { return ThisObject(); }
#endif
   // visible to all clients
   CMObject();
   CMObject(const CMObject& o);
   BOOL operator==(const CMObject& r) const;
   BOOL operator!=(const CMObject& r) const { return ! operator==(r); }
   CMObject& operator=(const CMObject& r);
   CString ToString() const;
   CString TypeName() const;
};
CMObject.cpp
////////////////
// CMObject implementation, from ManWrap.cpp
// If this code works, it was written by Paul DiLascia.
// If not, I don't know who wrote it.
// Compiles with Visual Studio .NET 2003 on Windows XP. Tab size=3.
//

CMObject::CMObject()
{
   // default ctor must be here and not inline, so m_handle is
   // appropriately initialized to a NULL GCHandle. Otherwise it will
   // have unpredictable values since native code sees it as intptr_t
}

CMObject::CMObject(const CMObject& o) : m_handle(o.m_handle)
{
   // See remarks above
}

CMObject& CMObject::operator=(const CMObject& r)
{
   m_handle = r.m_handle;  // copies underlying GCHandle.Target
   return *this;
}

BOOL CMObject::operator==(const CMObject& r) const
{
   return (*this)->Equals(r.ThisObject());
}

CString CMObject::ToString() const
{
   return (*this)->ToString();
}

CString CMObject::TypeName() const
{
   return (*this)->GetType()->Name;
}

CMCapture's copy constructor can be inline because it simply passes its native argument to CMObject. You have to pay the function toll when you construct an object, but at least you don't have to pay it twice.

Following the rules I've sketched, writing wrappers is so easy you can do it in your sleep. Or better yet, write some macros to streamline the process as I did for ManWrap. Here's the final CMCapture, from RexexWrap.h:

class CMCapture : public CMObject
{
   DECLARE_WRAPPER(Capture, Object);
public:
   // wrapped properties/methods
   int Index() const;
   int Length() const;
   CString Value() const;
};
It uses the macro DECLARE_WRAPPER defined in ManWrap.h, to save typing. There's another macro IMPLEMENT_WRAPPER that makes the implementation equally trivial (see the code download). The macros declare and implement all the basic constructors and operators I've described. In case you hadn't noticed, the names are designed to feel familiar to MFC programmers. DECLARE/IMPLEMENT_WRAPPER assumes you follow my naming convention: CMFoo is the native wrapper for managed Foo objects. (I'd have used CFoo, but that collides with the MFC CObject for Object, so I added the M in CM for Managed.) Figure 6 shows DECLARE_WRAPPER. IMPLEMENT_WRAPPER is similar. You can download and ponder it for homework.

Figure 6 declare_wrapper.h (Close Figure 6)
// DECLARE_WRAPPER macro from ManWrap.h

#ifdef _MANAGED
// This declares the managed part of DECLARE_WRAPPER. For internal use
// only. Use DECLARE_WRAPPER.
//
#define DECLARE_WRAPPER_M(MT,BT)                            \
public:                                                     \
   CM##MT(MT* o) : CM##BT(o) { }                            \
   MT* ThisObject() const                                   \
   {                                                        \
      return static_cast<MT*>((Object*)m_handle);           \
   }                                                        \
   MT* operator->() const                                   \
   {                                                        \
      return ThisObject();                                  \
   }                                                        \
   CM##MT& operator=(MT* o)                                 \
   {                                                        \
      m_handle = o;                                         \
      return *this;                                         \
   }                                                        \

#else // NOT _MANAGED

// native code doesn't see the managed stuff: macro does nothing
#define DECLARE_WRAPPER_M(MT,BT)

#endif // _MANAGED

//////////////////
// Use this to declare a wrapper class. Declares the basic ctors and
// operators needed for type-safeness usage. MT=managed type, BT=base
// (managed) type. You must follow the naming convention CMFoo = wrapper
// for Foo.
//
#define DECLARE_WRAPPER(MT,BT)                              \
public:                                                     \
   CM##MT() { }                                             \
   CM##MT(const CM##MT& o) : CM##BT(o) { }                  \
   CM##MT& operator=(const CM##MT &r)                       \
   {                                                        \
      CM##BT::operator=(r);                                 \
      return *this;                                         \
   }                                                        \
   DECLARE_WRAPPER_M(MT,BT)                                 \

Astute readers may have noticed something odd. The only constructors I've written to this point are default constructors, copy constructors, and constructors from pointer-to-managed-type. The last is hidden from native code, so it seems all native clients can do is create Null objects and copy them. What good is that? The lack of constructors is simply an unfortunate result of my choice of classes. You never create an Object by itself, and Capture objects only come from other objects like Match or Group. But Regex has a real constructor that takes a String, so CMRegex wraps it like so:

// in RegexWrap.h
class CMRegex : public CMObject {
  DECLARE_WRAPPER(Regex,Object);
public:
  CMRegex(LPCTSTR s);
};
// in RegexWrap.cpp
CMRegex::CMRegex(LPCTSTR s)
  : CMObject(new Regex(s))
{ }
Here again the constructor must be a true function because it calls "new Regex," which requires the Managed Extensions and /clr. In general, DECLARE/IMPLEMENT_WRAPPER only declare and implement the canonical constructors and operators you need to manipulate wrapper objects in type-safe fashion. If the class you're wrapping has "real" constructors, you have to wrap them yourself. DECLARE_WRAPPER is cool, but it's not clairvoyant.

If you're wrapping a method that returns a managed object of some other type, then you have to wrap that type too, because you obviously can't return the managed object directly to native code. For example, Regex::Match returns a Match*, so wrapping Regex::Match requires wrapping Match:

CMMatch CMRegex::Match(LPCTSTR input)
{
  return CMMatch((*this)->Match(input));
}
This is where the constructor from pointer-to-managed-type comes in. Just as the compiler automatically converts the String from Object::ToString into a CString, here it automatically converts the Match* from Regex::Match into a CMMatch object, because CMMatch has the proper constructor (automatically defined by DECLARE/IMPLEMENT_WRAPPER). So while native code can't see the construct-from-pointer-to-managed-type ctors, they're indispensable for writing wrappers.

Regular Expressions

Some newbies out there may not be fully familiar with regular expressions, so a brief review seems in order. If you're a regular expressions savant, feel free to skim.

A regular expression is a string that describes a family of strings. For example, the regular expression "Mic*" describes all strings that contain "Mic" followed by zero or more characters. Mickey, Microsoft, Michelangelo, or just Mic are all examples. Dot (a period) matches any character; + is like * but requires at least one character, so "Mic.+" matches all the previous examples except "Mic". [a-z] matches a range, so [a-zA-Z_0-9] matches a letter, digit, or underbar. Regex calls these word characters, which you can write as \w. So "\w+" matches sequences of word characters with at least one letter—in other words, C tokens. Well, almost. C tokens can't start with a digit, so here's a regular expression that works: "^[a-zA-Z_]\w*$". The special character ^ means "begins with" (unless it comes inside a range, in which case it means NOT) and $ means "ends", so "^[a-zA-Z_]\w*$" means: a string of letters, digits, or underbars that begins with a letter or underbar.

Regexes are extremely useful for input validation. \d matches digits and {n} matches n repetitions of something, so ^5\d{15}$ matches 16-digit numbers that begin with 5, in other words, MasterCard numbers. And, ^[45]\d{15}$ adds Visa cards too, which begin with 4. You can use parens to group expressions, so here's a test. What does the following regular expression describe?

^\d{5}(-\d{4}){0,1}$
Hint: {0,1} matches 0-1 repetitions (which you can abbreviate as ?). Have you figured it out yet? The expression reads: five digits followed by zero or one repetitions of: (a dash followed by four digits). This matches 02142 and 98007-4235, but not 3245 or 2345-98761. In other words, US ZIP codes. The parens group the ZIP+4 part so the {0-1} modifier applies to the whole group.

I've only scratched the surface of what regular expressions can do. I haven't mentioned replace, and I don't dare describe what happens in Unicode because I have no clue. But you can get a sense of how powerful regular expressions are. They've been a mainstay of UNIX for ages, and made a big comeback with Web programming and languages like Perl, where serving HTML is almost entirely an exercise in text manipulation. Not until the .NET Framework have regular expressions become a full-fledged Windows citizen. Better late than never!

Framework Regex

The .NET Framework implements regular expressions with a Regex class and three supporting classes: Match, Group, and Capture (see Figure A). Typically, you create a Regex and call Regex::Match with your input string to get the first Match, or Regex::Matches to get them all together:

Regex *r = new Regex("\b\w+\b");
MatchCollection* mc =
  r->Matches("abc ,_foo ,<& mumble7");
for (int i=0; i<mc->Count; i++) {
   Match *m = mc->Index(i);
   Console.WriteLine(m->Value);
}
This should display "abc", "_foo", and "mumble7", each on a line. This example introduces another special character, \b, called an anchor or atomic zero-width assertion, just like ^ (begin) and $ (end). \b designates a word boundary, so "\b\w+\b" means one or more word characters separated by word breaks.

Figure A Regex Classes
Figure A Regex Classes

Each parenthesized expression in a regular expression constitutes a Group. Regex::Groups returns all the Groups as a collection, which is never empty because the entire regular expression itself is a group. Groups are important because they let you do logical OR matches like "(ying|yang)", they let you apply quantifiers to subexpressions, and they let you extract individual parts of matches. Figure 1 in the main article shows my RegexTest program running with the zip example to reveal the groups.

The most powerful function of all is Regex::Replace, which makes regular expressions amazingly powerful. Like many developers, I've had to manually convert "\n" to "\r\n" before passing strings to a multiline edit control too many times, but with Regex::Replace, it's trivial:

s = Regex::Replace(s,"\n","\r\n");

Regex::Match and Replace have static overloads so you can do quick regular expressions without creating objects. One of my favorite Regex::Replace overloads is one that takes a delegate that lets you compute replacement text on the fly using procedural code—see the main article for a fun example of this.

Some words of caution: every regular expressions implementation is slightly different. For example, the Perl I'm familiar with allows {,1} as shorthand for {0,1}, but the Redmontonians don't do it that way. Just beware of minor differences. The definitive .NET Regex reference is "Regular Expressions as a Language" in the MSDN Library.

RegexWrap

In honor of MSDN Magazine's 20th anniversary, now that I've explained ManWrap, it's time to do some wrapping! I used ManWrap to wrap the .NET Regex class in a DLL called RegexWrap.dll. Figure 7 shows the abridged header file. I won't explain all the details since they're mostly trivial, but here's a typical wrapper:

CString CMRegex::Replace(LPCTSTR input, LPCTSTR replace)
{
  return (*this)->Replace(input, replace);
}
In practically every case, the implementation is one line: just call the underlying managed method and let the compiler convert the parameters. Isn't interop lots of fun? This works even when the parameter is another wrapped class, as I have already explained for CMRegex::Match.

Figure 7 RegexWrap.h (Close Figure 7)
////////////////////////////////////////////////////////////////
// MSDN Magazine — February 2004
// If this code works, it was written by Paul DiLascia.
// If not, I don't know who wrote it.
// Compiles with Visual Studio .NET 2003 on Windows XP. Tab size=3.
//

// This file declares all the wrappers for RegexWrap. The implementation
// is in RegexWrap.cpp
//
#pragma once
#include "ManWrap.h"
#ifdef _MANAGED
using namespace System::Runtime::InteropServices;
using namespace System::Text::RegularExpressions;
#endif

//////////////////
// You MUST instantiate one of these somewhere in your app
// BEFORE you call any RegexWrap DLL functions, to initialize
// RegexWrap.dll. DLLs that use Managed Extensions and MFC/ATL require
// special initialization because they don't use the standard startup
// code that calls DllMain. The best thing to do is create a static
// instance somewhere (outside function scope), or in your CWinApp-
// derived application class (MFC).
//
class WREXPORT CRegexWrapInit {
public:
   CRegexWrapInit();
   ~CRegexWrapInit();
};

//////////////////
// Wrapper for .NET Capture class
//
class WREXPORT CMCapture : public CMObject
{
   DECLARE_WRAPPER(Capture, Object);
public:
   // wrapped properties/methods
   int Index() const;
   int Length() const;
   CString Value() const;
};

//////////////////
// Wrapper for Group.
//
class WREXPORT CMGroup : public CMCapture
{
   DECLARE_WRAPPER(Group, Capture);
public:
   // wrapped properties/methods
   bool Success() const;
   CMCaptureCollection Captures() const;
};

//////////////////
// Wrapper for Match.
//
class WREXPORT CMMatch : public CMGroup
{
   DECLARE_WRAPPER(Match, Group);
public:
   // wrapped properties/methods
   CMMatch NextMatch() const;
   CString Result(CString sReplace) const;
   CMGroupCollection Groups() const;
   static const CMMatch Empty; // constant empty match
   typedef CString (CALLBACK* evaluator)(const CMMatch&, void* param);
};

//////////////////
// Wrapper for Regex. This one has all the good stuff.
//
class WREXPORT CMRegex : public CMObject {
   DECLARE_WRAPPER(Regex,Object);
public:
   enum Options {
      None = 0,
      .
      .
   };

   CMRegex(LPCTSTR s);
   CMRegex(LPCTSTR s, Options opt);
   CMMatch Match(LPCTSTR input);
   static CMMatch Match(LPCTSTR input, LPCTSTR pattern);
   .
   . // lots more
   .

};

Of course, nothing is ever totally trivial. I did encounter some bumps and snags along the way to RegexWrap: collections, delegates, exceptions, arrays, and enums. I'll describe how I dealt with each one in turn.

Coping with Collections

Collections are ubiquitous throughout the Framework. For example, Regex::Matches returns all the matches as a MatchCollection and Match::Groups returns all the Groups as a GroupCollection. My first thought on how to deal with collections was to convert them to STL containers of wrapped objects. On second thought, I realized this was a bad idea. Why create a new set of handles that point to objects that are already in the collection? While .NET Collections resemble STL containers in some respects, they're not exactly the same. For example, you can access a GroupCollection by integer index or by string name.

Rather than using an STL vector or map, a simpler approach is to use the system I've already built, namely, ManWrap. Figure 8 shows how I wrapped GroupCollection. It's what you'd expect, only now there's a new macro, DECLARE_COLLECTION, that does the same thing as DECLARE_WRAPPER, but adds three methods that all collections have: Count, IsReadOnly, and IsSynchronized. Naturally, there's IMPLEMENT_COLLECTION to implement these. Since GroupCollection lets you index by integer or string, the wrapper has two operator[] overloads.

Figure 8 GroupCollection.cpp (Close Figure 8)
//////////////////
// ManWrap wrapper for GroupCollection.
//

// In header, RegexWrap.h
class WREXPORT CMGroupCollection : public CMObject
{
   DECLARE_COLLECTION(GroupCollection, Object);
public:
   CMGroup operator[](int i);
   CMGroup operator[](LPCTSTR name);
};

// In module, RegexWrap.cpp
IMPLEMENT_COLLECTION(GroupCollection, Object)

CMGroup CMGroupCollection::operator[](int i)
{
   return (*this)->Item[i];
}

CMGroup CMGroupCollection::operator[](LPCTSTR name)
{
   return (*this)->Item[name];
}

Once I wrap Match, Group, and CaptureCollections, I can wrap the methods that use them. Regex::Matches returns a MatchCollection, so here's the wrapper:

CMMatchCollection CMRegex::Matches(LPCTSTR input)
{
  return (*this)->Matches(input);
}
CMMatch::Groups and CMGroup::Captures are entirely similar. Once again, the compiler silently performs all the type conversions. I love C++ and interop!

Dealing with Delegates

One of the most significant innovations in programming history is the concept of callbacks. They let you call a function that isn't known until runtime. Callbacks provide the basis for virtual functions and all forms of event programming. But in the managed world, one doesn't say "callback," one says "delegate." For example, one form of Regex::Replace lets you pass a MatchEvaluator:

MatchEvaluator* delg = // create one
String *s = Regex::Replace("\\b\\w+\\b",
  "Modify me.", delg);

Regex::Replace calls your MatchEvaluator delegate with each successive Match. Your delegate returns the replacement text. Later, I'll show you a cute example using MatchEvaluator. Right now, I'm focused on wrapping it. The Framework speaks delegates whereas C++ speaks callbacks. To make them talk, I first need a typedef:

class CMMatch ... {
public:
  typedef CString (CALLBACK* evaluator)(const CMMatch&, void* param);
};

CMMatch::evaluator is a pointer-to-function that takes a CMMatch and void* param, and returns CString. Putting the typedef inside CMMatch is a matter of style more than anything else, but it does avoid cluttering the global namespace. The void* param provides a way for native callers to pass their state. Delegates always have an associated object (which can be null if the method is static), but in C/C++ all you get is a function pointer, so callback interfaces usually add a void* that gives you a way to pass state information. Such is the lowly life of C. With my new typedef and these remarks in mind, I can declare CMRegex::Replace, like so:

class CMRegex ... {
public:
  static CString Replace(LPCTSTR input,
    LPCTSTR pattern,
    CMMatch::evaluator me,
    void* param);
};

My wrapper resembles the real Replace method (both are static), with an extra void* param. But how do I implement it?

CString CMRegex::Replace(...)
{
  MatchEvaluator delg = // how to create?
  return Regex::Replace(..., delg);
}

To create the MatchEvaluator, I need a __gc class with a method that calls the caller's native callback with the caller's void*. I wrote a little managed class called WrapMatchEvaluator that does just that (you can download the code for details). To save typing, WrapMatchEvaluator has a static Create function that returns a new MatchEvaluator, so CMRegex::Replace is still just one line:

CString CMRegex::Replace(LPCTSTR input,
  LPCTSTR pattern,
  CMMatch::evaluator me,
  void* lp)
{
  return Regex::Replace(input, pattern,
    WrapMatchEvaluator::Create(me, lp));
}
Well, it's one line in the source. Here I had to split it to fit in the printed magazine. Since native code has no use for WrapMatchEvaluator (it's a __gc class), the implementation is inside RegexWrap.cpp, not the header.

Handling Exceptions

Sooner or later, the .NET Framework is going to complain about something you did. Rude, I know, but if you pass Regex a bad expression, what do you expect? Native code can't handle managed exceptions, so I've got to do something. Dumping the user into the CLR debugger certainly won't win me any brilliance points so I'll just wrap the Exceptions too. I'll catch them at the border and make them don their wrappers before escaping into Nativeland. Trap-and-wrap is tedious but necessary. Regex's constructor can throw an exception, so I need to revise my wrapper, as shown here:

Regex* NewRegex(LPCTSTR s)
{
  try {
    return new Regex(s);
  } catch (ArgumentException* e) {
    throw CMArgumentException(e);
  } catch (Exception* e) {
    throw CMException(e);
  }
}
CMRegex::CMRegex(LPCTSTR s) : CMObject(NewRegex(s))
{
}

The basic formula involves trapping the exception inside the wrapper and re-throwing it in wrapped form. The reason for introducing NewRegex is so I can use initializer syntax instead of assigning m_handle inside the constructor (which would be less efficient because it sets m_handle twice). Once I trap-and-wrap Exceptions, native code can handle them the native way. Here's how RegexTest handles it when the user types a bad expression:

// in FormatResults
try {
  // create CMRegex, get matches, build string
} catch (const CMException& e) {
  result.Format(_T("OOPS! %s\n"), e.ToString());
  MessageBeep(0);
  return result;
}

One thing to consider when wrapping exceptions is whether you need to wrap every kind of exception thrown. For Regex, there's only ArgumentException, but .NET has dozens of exception types. Which ones you wrap and how many catch blocks you add depends on how much information your app needs. Whatever you do, be sure to always trap the base exception class in the last catch block, so nothing slips out to crash your app.

Wrapping Arrays

I've dealt with collections, delegates, and exceptions. Now it's time for arrays. Regex::GetGroupNumbers returns an array of ints whereas Regex::GetGroupNames returns an array of Strings. I have to convert managed arrays to native types before passing them to Nativeland. C-style arrays are one option, but there's just no reason to use C-style arrays when there's STL. ManWrap has a template function that converts a managed array of Foo objects into an STL vector of CMFoo. CMRegex::GetGroupNames uses it, as you can see here:

vector<CString> CMRegex::GetGroupNames()
{
  return wrap_array<CString,String>((*this)->GetGroupNames());
}

Again, the wrapper is still one line. There's another wrap_array to convert int arrays, because the compiler needs a __gc declarator to tell the difference between native and managed int arrays. I leave you to download and ponder the source for homework.

Encapsulating Enums

Finally, we come to enums, the last problem child in the RegexWrap family. Not a problem really, just a typing headache. Some Regex methods allow RegexOptions to control what you want to do. For example, if you want to ignore case, you can call Regex::Match with RegexOptions::IgnoreCase. To let native apps access these options, I defined my own native enum with the same names and values as RegexOptions, as you can see in Figure 7. In order to save typing and eliminate errors, I wrote a little utility called DumpEnum, which spits out C code for any .NET Framework enum class.

Building Mixed-Mode DLLs

With all the programming problems solved, the last step is to package RegexWrap as a DLL. You have to do the usual __declspec(dllexport) or __declspec(dllimport) on all your classes (which I simplify with macros), and you have to play some tricks to build a managed DLL. Managed DLLs require special initialization because they can't use the normal DllMain startup code and they require /NOENTRY and manual initialization. See my February 2005 C++ At Work column for details. The bottom line for RegexWrap is that to use RegexWrap.dll, you must instantiate a special DLL-initialization class somewhere at global scope as shown in the following line of code:

// somewhere in your app
CRegexWrapInit libinit;

I also encountered a minor problem debugging. To debug wrapper DLLs from your native app, you need to set "Debugger Type" to "Mixed" in your project's Debug settings. The default (Auto) determines which debugger to load by looking at the EXE. With ManWrap apps, the EXE is native so the IDE uses the native debugger and you can't step into managed code. If you select Debugger Type=Mixed, the IDE loads both debuggers.

Figure 9 File and Module Relationships
Figure 9 File and Module Relationships

Once you overcome these little nuisances, RegexWrap works like any other C++ DLL. Clients #include the header file and link the import library. Naturally, you need RegexWrap.dll in your PATH and the .NET Framework installed to run. Figure 9 shows the relationships among the files and modules in a typical client application like RegexTest.

Fun with RegexWrap

With Regex finally wrapped, it's time for some fun! The whole reason I wrote RegexWrap was to bring the power of regular expressions to native MFC apps.

The first thing I did was port my original mixed-mode RegexTest program with its managed FormatResults function to a purely native version using RegexWrap. Every pointer to Regex, Match, Group, and Capture is now a CMRegex, CMMatch, CMGroup, or CMCapture object. The same applies for collections (you'll need to download the code for the details). The important thing is that now RegexTest is fully native, there's not a /clr to be found in any project or make file. If you're new to regular expressions, RegexTest is a great way to explore them.

My next sample is a fun program that turns English into semi-readable gibberish. Linguists have long observed the following curious phenomenon: if you scramble the letters in the middle of each word in a sentence, but preserve the letters at the beginning and end, the result is more readable than you might expect. Apparently our brains read by scanning the beginning and end of words and fill in the rest. I used RegexWrap to implement a WordMess program that demonstrates this phenomenon. Type a sentence and WordMess scrambles it as described. Figure 10 shows it running. Here's what WordMess did to the first sentence in this paragraph: my nxet sapmle is a fun prgaorm taht tnurs Ensiglh itno smei-reabldae gibbiserh.

Figure 10 WordMess
Figure 10 WordMess

WordMess uses the MatchEvaluator delegate form of Regex::Replace (through its wrapper, of course):

// in CMainDlg::OnOK
static CMRegex MyRegex(_T("\\b[a-zA-Z]+\\b"));
CString report;
m_sResult = MyRegex.Replace(m_sInput, &Scrambler, &report);
MyRegex is a static CMRegex object that matches words, that is, sequences of one or more letters (no digits) surrounded by word breaks. (The hardest part of writing regular expressions in C++ is remembering to type two backslashes every time you want one.) So CMRegex::Replace calls my Scrambler function once for each word in the input. Figure 11 shows the Scrambler function. Look how easy it is with STL strings and the swap and random_shuffle algorithms. This would have been tons of code without STL. Scrambler uses its void* param as a CString to which it appends each substitution it makes. WordMess adds the report to its results, as Figure 10 shows. Amzanig!

Figure 11 WordMess Scrambler Function (Close Figure 11)
//////////////////
// Match evaluator callback: This function receives a match and returns
// the text to replace it with. The algorithm is: scramble the middle of
// the word.
//
// - Words with fewer than four letters are unchanged.
// - Four letter words have their middle two letters swapped.
// - Five letter words preserve their first and last letters.
// - Longer words preserve the second letter as well.
//
// Look how easy this is with STL algorithms!!—swap and random_shuffle.
// Scrambler reports its activity in a CString passed as the void* param.
//
CString CALLBACK Scrambler(const CMMatch& m, void* param)
{
   tstring word = m.Value(); // STL string easier to manipulate than CString
   size_t len = word.size();
   if (len>=4) {
      if (len==4) {
         swap(word[1],word[2]);         // use STL swap algorithm!
      } else {
         // STL shuffle algorithm!
         random_shuffle(word.begin()+(len <=5 ? 1 : 2), word.end()-1);
      }
   }
   if (param) {
      CString temp;
      temp.Format(_T("Scramble: '%s' -> '%s'\n"), m.Value(),
         word.c_str());
      (*(CString*)param) += temp;
   }
   return word.c_str();
}

For my final application, I chose something more serious and practical, a program called RegexForm that validates various kinds of input: ZIP code, Social Security number, telephone number, and C tokens. See this month's C++ at Work column for a discussion of RegexForm.

Conclusion

Well, that's a wrap! I hope you've enjoyed wrapping Regex with me, and I hope you'll find ways to use ManWrap to wrap other Framework classes you want to call from native code. ManWrap isn't for everyone: you only need it if you want to call the .NET Framework while keeping your code native. Otherwise, just use /clr and call the Framework directly.

Paul DiLascia is a freelance writer, consultant, and Web/UI designer-at-large. He is the author of Windows++: Writing Reusable Windows Code in C++ (Addison-Wesley, 1992). Paul can be reached at www.dilascia.com.

© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


Security: Unify Windows Forms and ASP.NET Providers for Credentials Management

Security

Unify Windows Forms and ASP.NET Providers for Credentials Management

Juval Lowy
This article is based on prerelease versions of the .NET Framework 2.0 and Visual Studio 2005. All information contained herein is subject to change.
This article discusses:
  • .NET user credentials management
  • The ASP.NET 2.0 security providers model
  • Architecting custom security for Windows Forms
  • Code access security considerations for client applications
This article uses the following technologies:
Windows Forms, ASP.NET, Web services, C#
Code download available at: Security.exe (220KB)
Contents
ASP.NET Security Infrastructure
Solution Architecture
Implementing IPrincipal
The LoginControl Class
Authenticating the User
The AspNetLoginControl
Implementing IUserManager
AspNetLoginControl and Code Access Security
The UserManager Web Service
The WSLoginControl
WSLoginControl and Code Access Security
The Sample Application
The LoginDialog Class
Conclusion

W
indows® Forms intranet applications often resort to storing user credentials in a database even when deployed in a homogenous Windows environment. This stems from the limitations of using Windows accounts. For ASP.NET applications, the Microsoft® .NET Framework 2.0 provides out-of-the-box custom credentials management. You can easily authenticate users and authorize them, without ever having to use Windows accounts. This saves developers valuable time and effort, not to mention providing a high-quality, secure solution.

This article presents a set of interacting helper classes that enable a Windows Forms application to use the ASP.NET credentials management infrastructure, with the same ease as if it were an ASP.NET application. Doing so provides the productivity benefits of ASP.NET as well as a unified credentials store, regardless of the application user interface that is being employed. I'll also present some .NET programming techniques and various other design and programming best practices.

ASP.NET Security Infrastructure

Before showing you how to take advantage of the ASP.NET user credentials management, you need to learn a bit about it. Out of the box, ASP.NET applications can store their custom user credentials in either SQL Server™ or SQL Server Express. That said, the credentials management architecture is that of a provider model, and you can easily add other storage options, such as a Microsoft Access database. ASP.NET developers can configure their application directly from within Visual Studio® 2005. When selecting ASP.NET Configuration from the Web site menu, Visual Studio 2005 browses to the ASP.NET administration pages and allows you to configure various security parameters, as shown in Figure 1. Here you have access to the site management provider where you can select which store to use, such as a SQL Server or SQL Server Express database. The administration pages also allow you to create new users and delete existing ones, create new roles and delete existing ones, and allocate users to roles. You should note that the same database tables are used to store the user information from multiple ASP.NET applications. As a result, each user or role record is also associated with a particular application name.

Figure 1 ASP.NET Application Security Configuration
Figure 1 ASP.NET Application Security Configuration

To use the SQL Server provider, run the setup file aspnet_regsql.exe found under \WINDOWS\Microsoft.NET\Framework\Version\. The setup program will create a new database called aspnetdb, which contains the tables and stored procedures required to manage the credentials.

At run time, ASP.NET will authenticate the callers using the credentials in the database. The easiest way to allow a user to provide the credentials is to drop a Login control on the Web Form. The Login control collects the user name and password from the user and authenticates using a class called MembershipProvider, defined as the following:

public abstract class MembershipProvider : ProviderBase
{
   public abstract string ApplicationName{get;set;}
   public abstract bool ValidateUser(string name, string password);
   ...
}

The goal of MembershipProvider is to encapsulate the actual provider used and the details of the actual data access, as well as to enable changing the membership provider without affecting the application itself. Depending on the configured security provider, the Login control will use a concrete data access class such as SqlMembershipProvider, which derives from the MembershipProvider class, when using SQL Server:

public class SqlMembershipProvider : MembershipProvider
{...}

However, the Login control interacts only with the MembershipProvider base class. The Login control obtains the required membership provider by accessing the Provider static property of the Membership class, defined as:

public sealed class Membership
{
   public static string ApplicationName{get;set;}
   public static MembershipProvider Provider{get;}
   public static bool ValidateUser(string userName, string password);
   ...
}

The Membership class offers many members, which support every aspect of user management. Membership.Provider returns an instance of the configured provider based on settings in the Web application configuration file.

I'll be focusing on two members of the Membership class. The first is the ApplicationName property, which is used to set and retrieve the application name. The second is the ValidateUser method, which authenticates the specified credentials against the store, returning true if they match and false otherwise. The static Membership.ValidateUser method is shorthand for retrieving the configured provider from Membership.Provider and using its instance ValidateUser method.

You can also apply role-based security to authorize operations or access to resources. The aspnetdb database stores the mapping of users to roles. Once the user is authenticated, ASP.NET will set the User property of the HTTP context and the page to a custom security principal object called RolePrincipal:

public sealed class RolePrincipal : IPrincipal
{...}
RolePrincipal uses the abstract class RoleProvider:
public abstract class RoleProvider : ProviderBase
{
   public abstract string ApplicationName{get;set;}
   public abstract bool IsUserInRole(string username, string roleName);
   public abstract string[] GetRolesForUser(string userName);
   ...
}
The ApplicationName property of RoleProvider binds the role provider to the particular application. The IsUserInRole method verifies the user's role membership. The GetRolesForUser method returns all the roles that a specified user is a member of.

Like the membership providers, depending on the configured security provider, RolePrincipal uses a corresponding data access class such as SqlRoleProvider to authorize the caller:

public class SqlRoleProvider : RoleProvider
{...}

You can obtain the required role provider by accessing the Provider static property of the Roles class, defined as:

public sealed class Roles
{
   public static string ApplicationName{get;set;}
   public static string[] GetRolesForUser(string username);
   public static bool IsUserInRole(string username, string roleName);
   public static RoleProvider Provider{get;}
   ...
}
Both Roles.GetRolesForUser and Roles.IsUserInRole are shorthand, using the Roles.Provider property internally. Roles.Provider returns an instance of the configured provider based on settings in the Web application configuration file.

Solution Architecture

While my primary goal was to take advantage of the ASP.NET 2.0 credentials management infrastructure, I also wanted to provide a general-purpose custom authentication and authorization infrastructure for Windows Forms. Such infrastructure should not necessarily be coupled to ASP.NET 2.0, and could easily use any custom credentials store, such as Access or a Lightweight Directory Access Protocol (LDAP) database. The first step was to decouple it from the actual credentials store by defining the IUserManager interface, as follows:

public interface IUserManager
{
   bool Authenticate(string applicationName,string userName,
       string password);
   bool IsInRole(string applicationName, string userName, string role);
   string[] GetRoles(string applicationName, string userName);
}

The Authenticate method is used to authenticate the specified user credentials against the credentials store. IsInRole authorizes the user when you're using role-based security. IUserManager also provides the GetRoles method which returns all the roles of which a specified user is a member. GetRoles is useful when caching role membership, which I'll discuss later.

Authenticate is used by an abstract Windows Forms custom control called LoginControl. LoginControl is used in a similar fashion to its ASP.NET cousin in that you add it (or rather, a subclass of it) to your Windows Forms application. LoginControl obtains an implementation of IUserManager and authenticates the supplied credentials using the Authenticate method. If the user specified valid credentials, LoginControl instantiates an implementation of IPrincipal called CustomPrincipal. LoginControl provides CustomPrincipal with the implementation of IUserManager, and attaches CustomPrincipal to the current thread. The CustomPrincipal class can use the IsInRole or GetRoles methods of IUserManager for authorizing the user. This architecture is shown in Figure 2. Both LoginControl and CustomPrincipal are defined in the WinFormsEx.dll class library assembly, available with the code download for this article.

Figure 2 LoginControl Architecture
Figure 2 LoginControl Architecture

You should note that CustomPrincipal never authenticates the user since it implicitly trusts LoginControl to do so. This means that you should not allow CustomPrincipal to be attached to a thread without going through valid authentication. In order to enforce that, the CustomPrincipal class is an internal class, called only by LoginControl. While an instance of CustomPrincipal could still be created using reflection, that would require a high level of trust. Thus I recommend that you follow the principles of least privilege and not grant the assembly the ReflectionPermission that would allow this to happen.

Implementing IPrincipal

The sole purpose of CustomPrincipal is to replace the Windows security principal and service the PrincipalPermissionAttribute and PrincipalPermission classes. CustomPrincipal should only be installed after successful authentication. To further enforce and automate this design decision, CustomPrincipal doesn't have a public constructor, so its clients have no direct way to instantiate it. Instead, the clients use the Attach public static method. Attach first verifies that the identity provided is that of an authenticated user:

Debug.Assert(user.IsAuthenticated);

If the user is authenticated, Attach creates an object of type CustomPrincipal, providing it with the security identity and the implementation of IUserManager to use, as well as with the role caching policy, shown in Figure 3.

Figure 3 CustomPrincipal Class (Close Figure 3)
internal class CustomPrincipal : IPrincipal
{
   IIdentity m_User;
   IPrincipal m_OldPrincipal;
   IUserManager m_UserManager;
   string m_ApplicationName;
   string[] m_Roles;
   static bool m_ThreadPolicySet = false;

   CustomPrincipal(IIdentity user,string applicationName,
                   IUserManager userManager,bool cacheRoles)
   {
      m_OldPrincipal = Thread.CurrentPrincipal;

      m_User = user;
      m_ApplicationName = applicationName;
      m_UserManager = userManager;

      if(cacheRoles)
      {
         m_Roles = m_UserManager.GetRoles(m_ApplicationName,m_User.Name);
      }
      //Make this object the principal for this thread
      Thread.CurrentPrincipal = this;
   }

   static public void Attach(IIdentity user,string applicationName,
                             IUserManager userManager)
   {
      Attach(user,applicationName,userManager,false);
   }

   static public void Attach(IIdentity user,string applicationName,
                             IUserManager userManager,bool cacheRoles)
   {
      Debug.Assert(user.IsAuthenticated);

      IPrincipal customPrincipal = new
          CustomPrincipal(user,applicationName,
              userManager,cacheRoles);

      //Make sure all future threads in this app domain use this principal
      //In addition, because default principal cannot be set twice:
      if(m_ThreadPolicySet == false)
      {
         AppDomain currentDomain = AppDomain.CurrentDomain;
         currentDomain.SetThreadPrincipal(customPrincipal);
         m_ThreadPolicySet = true;
      }
   }

   public void Detach()
   {
      Thread.CurrentPrincipal = m_OldPrincipal;
   }

   public IIdentity Identity
   {
      get { return m_User; }
   }

   public bool IsInRole(string role)
   {
      if(m_Roles != null)
      {
         return Array.Exists(m_Roles,
            delegate(string roleToMatch) { return roleToMatch == role; });
      }
      else
      {
         return m_UserManager.IsInRole(
            m_ApplicationName, m_User.Name,role);
      }
   }
}

The constructor of CustomPrincipal saves the identity provided, as well as a reference to the previous principal associated with that thread. Most importantly, the constructor replaces the default principal by setting the CurrentPrincipal property of the current thread to itself, as shown in the following line of code:

Thread.CurrentPrincipal = this;

To support log-out semantics, CustomPrincipal provides the Detach method for detaching itself from the thread and restoring the old principal saved during construction.

In addition, Attach sets the default thread principal object to CustomPrincipal, so that it will be attached to new threads automatically. However, since you can only set the thread principal object once per app domain, Attach verifies first that it was not done already (it is possible to attach and detach the principal), using the static flag m_ThreadPolicySet:

if(m_ThreadPolicySet == false)
{
   m_ThreadPolicySet = true;
   currentDomain.SetThreadPrincipal(customPrincipal);
}

While authentication is a one-off cost, authorization can be a frequent operation. Since verifying role membership may potentially be expensive (such as when querying a database or calling a Web service), CustomPrincipal can cache all the roles the user is a member of, by saving the roles in the m_Roles member array. The constructor of CustomPrincipal takes a Boolean parameter called cacheRoles. If cacheRoles is true, the constructor will initialize m_Roles by calling the GetRoles method of the provided user manager. While this will enable almost instant role membership verifications, unfortunately the downside is that when caching roles, CustomPrincipal will not detect changes to the roles repository, such as removing the user from a particular role. Use caching only in cases when the allocation of users to roles is a relatively infrequent event and when the performance and scalability goals absolutely mandate it. This is why the Attach version that does not take a cacheRoles parameter defaults to no caching.

Implementing IPrincipal is straightforward. In its implementation of the Identity property, CustomPrincipal returns the saved identity. To implement IsInRole, CustomPrincipal checks if there are any cached roles. If so, it searches the roles array using the static Exists method of the Array type. Exists takes a delegate of type Predicate, defined as the following:

public delegate bool Predicate<T>(T t);

Exists will evaluate the method targeted by the predicate for each item in the array and will return true if a match is found. IsInRole initializes the predicate with an anonymous method that compares the role specified in the call to IsInRole with the role passed in as a parameter. If there are no cached roles, IsInRole delegates the query to the provided implementation of IUserManager, returning its results.

The LoginControl Class

LoginControl provides two textboxes for capturing the user name and password. In addition, the control uses the ErrorProvider component. LoginControl will only authenticate if the user provides both user name and password, and the error provider will alert the user to missing input. Authentication takes place in the Click event-handling method for the Log In button. Figure 4 shows a partial listing of LoginControl (with some of the mundane code omitted in the interests of space).

Figure 4 Partial Listing of LoginControl (Close Figure 4)
using LoginEventHandler = EventHandler<LoginEventArgs>;

public class LoginEventArgs : EventArgs
{
   public LoginEventArgs(bool authenticated);
   public bool Authenticated{get;internal set;}
}

[DefaultEvent("LoginEvent")]
[ToolboxBitmap(typeof(LoginControl),"LoginControl.bmp")]
public abstract partial class LoginControl : UserControl
{
   string m_ApplicationName = String.Empty;
   bool m_CacheRoles = false;
   public event LoginEventHandler LoginEvent = delegate{};

   [Category("Credentials")]
   public bool CacheRoles //Gets and sets m_CacheRoles
   {...}

   [Category("Credentials")]
   public string ApplicationName //Gets and sets m_ ApplicationName
   {...}

   string GetAppName()
   {
      if(ApplicationName != String.Empty)
      {
         return ApplicationName;
      }
      Assembly clientAssembly = Assembly.GetEntryAssembly();
      AssemblyName assemblyName = clientAssembly.GetName();
      return assemblyName.Name;
   }

   static public void Logout()
   {
      CustomPrincipal customPrincipal =
         Thread.CurrentPrincipal as CustomPrincipal;
      if(customPrincipal != null)
      {
         customPrincipal.Detach();
      }
   }

   static public bool IsLoggedIn
   {
      get { return Thread.CurrentPrincipal is CustomPrincipal; }
   }

   protected virtual void OnLogin(object sender,EventArgs e)
   {
      string userName = m_UserNameBox.Text;
      string password = m_PasswordBox.Text;

      /* Validation of userName and password using the error provider */

      string applicationName = GetAppName();
      IUserManager userManager = GetUserManager();

      bool authenticated;
      authenticated = userManager.Authenticate(
         applicationName,userName,password);
      if(authenticated)
      {
         IIdentity identity = new GenericIdentity(userName);
         CustomPrincipal.Attach(
            identity,applicationName,userManager,CacheRoles);
      }
      LoginEventArgs loginEventArgs = new LoginEventArgs(authenticated);
      LoginEvent(this,loginEventArgs);
   }
   protected abstract IUserManager GetUserManager();
}

When using LoginControl, you need to provide it with the credentials provider to use, the application name, and the role caching policy. Specifying the credential provider is done by subclasses of LoginControl. The subclasses need to override GetUserManager and return an implementation of IUserManager.

LoginControl provides the properties ApplicationName and CacheRoles. Because LoginControl derives from UserControl, it natively integrates with the Windows Forms Designer. These two properties are available for visual editing during the design time of a form or a window that uses LoginControl. To enrich the Designer support, the properties are decorated with the Category attribute, as shown here:

[Category("Credentials")]
When you select the Categories view of the control properties in the Designer, these properties will be grouped together under the Credentials category. Subclasses of LoginControl can add their own properties to this category, too. The CacheRoles property can be set to true or false, and LoginControl simply passes it as-is to CustomPrincipal.Attach. Unaltered, CacheRoles defaults to false.

The LoginControl cannot be used stand-alone, but it must be contained in another form or dialog. That container can, in turn, be used by different applications with different application names. You therefore have two options for supplying the application name. You can simply use the ApplicationName property to specify the application name. Or, if you do not know that name in advance (if you're developing a general-purpose container), LoginControl can retrieve the application name from the Windows Forms entry application assembly used to launch the control. During authentication, if the LoginControl has no value set in the ApplicationName property, LoginControl will use the friendly name of the entry assembly as the application name. This logic is encapsulated in the private helper method GetAppName.

Authenticating the User

The OnLogin method is called when the user clicks the Log In button. After validating the user name and password, OnLogin calls GetAppName to retrieve the application name. It then calls GetUserManager to obtain an implementation of IUserManager. Authentication itself is done simply by calling the Authenticate method of the user manager. If authentication is successful, OnLogin wraps a generic identity object around the user name and attaches the custom principal. Note that OnLogin never interacts with the custom principal directly. All OnLogin does is provide it with the IIdentity object, the application name, the caching policy, and the implementation of IUserManager to use.

The question now is what LoginControl should do after authentication. The control has no knowledge of the required behavior of its hosting container. If authentication fails, perhaps it should present a message box, or perhaps it should throw an exception. If authentication succeeds, maybe it should close the hosting dialog, or move to the next screen in a wizard, or something else. Since only the hosting application knows what to do after both successful and failed authentication, all LoginControl can do is inform it by firing an event. LoginControl declares the event LoginEvent of the type EventHandler<LoginEventArgs>. LoginEventArgs contains a Boolean property called Authenticated, indicating the outcome of the authentication. To avoid a multithreaded race condition where another thread removes all subscribers from the LoginEvent just before publishing, LoginControl initializes LoginEvent with an anonymous method.

LoginControl also provides two handy helpers: the static Boolean property IsLoggedIn, and the static method Logout. IsLoggedIn allows the caller to query if a user is logged in. LoginControl retrieves the current principal and checks if it is of the CustomPrincipal type. If the user is logged in, this of course will be the principal used. The Logout method allows the user to log out. Logout retrieves the current principal, and if it is of the CustomPrincipal type, detaches it from the current thread. As explained previously, calling Detach on CustomPrincipal will also restore the previous principal. Note that if multiple threads are involved, you will need to log out on each one of them.

The AspNetLoginControl

Let's take a look now at the definition of my AspNetLoginControl, as shown here:

public partial class AspNetLoginControl : LoginControl
{
   protected override IUserManager GetUserManager()
   {
      return new AspNetUserManager();
   }
}
AspNetLoginControl derives from LoginControl, and in its overriding of GetUserManager it returns my AspNetUserManager implementation of IUserManager, which uses the ASP.NET providers directly, as shown in Figure 5.

Figure 5 AspNetLoginControl
Figure 5 AspNetLoginControl

AspNetUserManager is capable of using any valid ASP.NET 2.0 provider, hence its name. To use AspNetLoginControl, you will need to add to the application's configuration file the same values you would have placed in a Web application configuration file, indicating which provider to choose, as well as any provider-specific values and settings.

For example, to use the SQL Server providers, you must add the settings that are shown in Figure 6 to the application configuration file. The connection string value shown in Figure 6 is used to connect to the aspnetdb database on the local machine after default installation. Note the use of the enabled attribute of the roleManager tag to enable authorization.

Figure 6 Settings for AspNetLoginControl (Close Figure 6)
<?xml version="1.0"?>
<configuration>
   <system.web>
      <membership defaultProvider="AspNetSqlProvider" />
      <roleManager enabled="true" defaultProvider="AspNetSqlProvider" />
   </system.web>
   <connectionStrings>
      <remove name="LocalSqlServer" />
      <add name="LocalSqlServer" connectionString="
         data source=(local);Integrated Security=SSPI;
         Initial Catalog=aspnetdb" />
   </connectionStrings>
</configuration>

By default, the configuration file generated by Visual Studio 2005 uses NTFS security to enforce access permission security to the application configuration file, so only application administrators can modify the provider's settings.

Implementing IUserManager

AspNetUserManager contains as member variables instances of the ASP.NET membership and role providers. All the functionality of IUserManager is accomplished by delegating to the ASP.NET providers, as shown in Figure 7.

Figure 7 AspNetUserManager Class (Close Figure 7)
class AspNetUserManager : IUserManager
{
   public bool Authenticate(string applicationName, string userName,
      string password)
   {
      Membership.ApplicationName = applicationName;
      return Membership.ValidateUser(userName,password);
   }

   public bool IsInRole(string applicationName, string userName,
      string role)
   {
      Roles.ApplicationName = applicationName;
      return Roles.IsUserInRole(userName,role);
   }

   public string[] GetRoles(string applicationName,
      string userName)
   {
      Roles.ApplicationName = applicationName;
      return Roles.GetRolesForUser(userName);
   }
}

To implement Authenticate, AspNetUserManager calls the ValidateUser method of Membership. To implement IsInRole and GetRoles, AspNetUserManager calls the IsUserInRole and GetRolesForUser methods of Roles.

One thing that you should be aware of is that my code is not thread-safe in the face of another thread changing membership.ApplicationName. As a result, the code shown here is fine as long as multithreaded access is not a requirement.

AspNetLoginControl and Code Access Security

AspNetLoginControl works well, and you can certainly use it, but it does have one important shortcoming. The ASP.NET providers were designed to be used on the server, and they require a few nontrivial permissions, including unmanaged code access permisions, unrestricted SQL Server client access permissions, and minimal ASP.NET Hosting permissions. Typically, server-side applications run in an elevated trust environment, potentially even full trust, and will have no problem obtaining those permissions and executing properly.

The AspNetLoginControl, on the other hand, is going to be used by Windows Forms applications. It is quite likely that such applications will be executing in a partial trust environment, perhaps as ClickOnce applications. The client environment may very well not grant the applications using AspNetLoginControl the permission they require to operate.

Figure 8 lists the permissions currently required for AspNetLoginControl and lists who demands them. These permissions are the product of both the control itself and the components it uses: CustomPrincipal and AspNetUserManager.

Figure 8 Security Permissions Required by AspNetLoginControl (Close Figure 8)

Permission Type | Specific Permission Value | Demanded By
Security | Execution | Any managed application in order to run
Security | Unmanaged code access | The ASP.NET providers for their internal implementation
ASP.NET hosting | Minimal | The ASP.NET providers such that any party that hosts them has permission to do so
SQL Server client | Unrestricted | The ASP.NET providers for accessing the database
User interface | Safe subwindows | LoginControl in order to display itself
Security | Control principal | CustomPrincipal to be able to set principal policy

You have a number of options in choosing how to deal with these permission demands. Although you can grant full trust to the WinFormsEx.dll assembly and all its clients, this is inadvisable because it opens the way for the assemblies to be lured into doing things they are not supposed to do.

You can grant most of these permissions using the .NET Configuration tool to both the WinFormsEx.dll assembly and to every client application that wants to use it. You can even list these permissions in the ClickOnce application manifest. To ease the task of granting the permissions, the source files accompanying this article contain the WinFormsEx.xml file. This is a custom permission set file that you can add to the .NET Configuration tool and grant the necessary permissions.

However, the best solution is to avoid the permissions demanded by the ASP.NET providers altogether, and to use a different implementation of IUserManager, one that does not demand server-side permissions in a client environment. I'll present this option next. Also note that the ASP.NET team is investigating shortening this list of required permissions prior to the final product release.

The UserManager Web Service

The solution to the partial-trust problem is to wrap the ASP.NET providers with a Web service. When using a Web service, none of the security permission demands made by the providers will ever make their way back to the client.

Using a Web service also has the advantage of better scalability, since only the Web service will be using the connection to the database, rather than each individual client application. Another benefit of a Web service is that it avoids some potential security issues with clients authenticating themselves against SQL Server and with secure connection string management on the client side.

There are, however, a few downsides to using a Web service. For example, you should secure the communication between the clients and the Web service, because the clients will be sending credentials over the wire. This can easily be done using HTTPS. You also have the problem of additional call latency, which should be resolved using role caching. Authenticating against the Web service itself may be an issue, but this is mitigated if you can sustain anonymous access to the service in your intranet environment. Finally, authorizing the Web service calls may present problems. The Web service allows callers to retrieve role information about a user. Role membership information may be sensitive information in its own right authenticate. This can be dealt with by adding role-based security to the Web service and authorizing the callers. Note, though, that authorization requires authentication.

Using the WebServiceBinding attribute, you can define IUserManager as a Web service interface and implement it, as shown in Figure 9. The UserManager class in Figure 9 uses both Membership and Roles to obtain the configured providers from the Web service configuration file. Note that each Web service method of UserManager also accepts the application name to use, so that a single Web service can support multiple Windows Forms applications.

Figure 9 Implementing IUserManager on a Web Service (Close Figure 9)
[WebServiceBinding]
public interface IUserManager
{
   [WebMethod(Description="Authenticates the user.")]
   bool Authenticate(string applicationName, string userName,
      string password);

   [WebMethod(Description="Verifies user role's membership.")]
   bool IsInRole(string applicationName, string userName, string role);

   [WebMethod(Description="Returns all roles the user is a member of.")]
   string[] GetRoles(string applicationName, string userName);
}

[WebService(Namespace = "http://SecurityServices",
   Description = "Wraps with a web service the ASP.NET providers. " +
                 "This web service should be accessed over https.")]
class UserManager : IUserManager
{
   public bool Authenticate(string applicationName, string userName,
      string password)
   {
      if(HttpContext.Current.Request.IsSecureConnection == false)
      {
         HttpContext.Current.Trace.Warn(
           "You should use HTTPS to avoid sending cleartext passwords.");
      }
      Membership.ApplicationName = applicationName;
      return Membership.ValidateUser(userName,password);
   }

   public bool IsInRole(string applicationName, string userName,
      string role)
   {
      Roles.ApplicationName = applicationName;
      return Roles.IsUserInRole(userName,role);
   }

   public string[] GetRoles(string applicationName, string userName)
   {
      Roles.ApplicationName = applicationName;
      return Roles.GetRolesForUser(userName);
   }
}

The WSLoginControl

WinFormsEx.dll contains the definition of WSLoginControl, shown in the following code snippet:

public partial class WSLoginControl : LoginControl
{
   protected override IUserManager GetUserManager()
   {
      return new UserManager();
   }
}

WSLoginControl derives from LoginControl, and in its overriding of GetUserManager, it returns UserManager, a client-side Web service proxy class used to invoke the UserManager Web service. WSLoginControl can use any Web service that manages user credentials as long as it supports the IUserManager interface, hence its name. To generate the proxy class, add a Web reference to the UserManager Web service. Then you should change the machine-generated UserManager Web service proxy class to derive from IUserManager. Since the proxy class is a partial class and is machine-generated, it's preferable to add that code in a separate file:

partial class UserManager : IUserManager
{}
WinFormsEx.dll already contains the definition of the UserManager Web service proxy class. The machine-generated code in the proxy class looks up the Web service address from the application configuration file. Add to the application configuration file under the appSettings section a key called UserManager whose value is the Web service address:
<?xml version="1.0"?>
<configuration>
   <appSettings>
      <add key="UserManager"
           value="http://localhost/SecurityServices/UserManager.asmx"/>
   </appSettings>
</configuration>

Figure 10 WSLoginControl and Its Supporting Classes
Figure 10 WSLoginControl and Its Supporting Classes

Figure 10 shows WSLoginControl, CustomPrincipal, and their interactions with the UserManager Web service.

WSLoginControl and Code Access Security

Using a Web service keeps the ASP.NET providers permission demands on the server. The trade-off is that you will need to grant the clients Web access permission to connect to the user management Web service. You will also need to grant the rest of the permissions required by LoginControl and CustomPrincipal. Figure 11 lists the reduced permissions required when using a credentials management Web service.

Figure 11 Security Permissions Required by WSLoginControl (Close Figure 11)

Permission Type | Specific Permission Value | Demanded By
Security | Execution | Any managed application in order to run
Web access | Connect to the user manager Web service | The UserManager proxy class to be able to use the Web service
User interface | Safe subwindows | LoginControl in order to display itself
Security | Control principal | CustomPrincipal to be able to set principal policy

You can grant all the permissions from Figure 11 using the .NET Configuration tool to both the WinFormsEx.dll assembly and to every client application that wants to use it. You can also use Visual Studio 2005 to list these permissions in the ClickOnce application manifest. For example, in Visual Studio 2005, go to the Security tab in the project settings of a ClickOnce application that uses WSLoginControl. Check the "Enable ClickOnce Security Settings" checkbox and the "This is a partial trust application" checkbox. Under "Zone your application will be installed from," select (Custom). This will remove all permissions except execution permission. Next, select SecurityPermission, and under Settings, select Include. Click Properties to bring up the Permission Settings dialog. Select assembly execution and principal control permission, and click OK. To grant permission to call the Web service, include the WebPermission, and in its properties, specify the UserManager's URL and allow the application to connect to it.

In a similar manager, grant the user interface permission that's listed in Figure 8. When you publish your ClickOnce application, its application manifest will include these permissions. You can even use Visual Studio 2005 to debug the application under these partial trust settings.

The Sample Application

Figure 12 shows a sample Windows Forms multi-document interface (MDI) application that uses WSLoginControl. Under its Security tab, the application requests the permissions listed in Figure 11. The application requires users to log in and authenticate themselves before creating a new document window. When the user selects the File | New menu item, it is handled by OnFileNew method, which demands that the user be a member of the Manager role:

[PrincipalPermission(SecurityAction.Demand,Role = "Manager")]
void OnFileNew(object sender,EventArgs args)
{...}
Before using the application, you will need to create the users and roles in the database. You can and should use the support Visual Studio 2005 offers for managing the credentials stored in the aspnetdb database.

Figure 12 Sample Application
Figure 12 Sample Application

Create a new ASP.NET Web site. Select ASP.NET Configuration from the Web site menu to bring up the Web Site Administration pages, shown in Figure 1. Select the Provider tab and then click the "Select a single provider for all site management data" link. Under the Provider list, click AspNetSqlProvider. Next, click the Security tab. Under Users, click Select authentication type, and on the next page click From the Internet. Go back to the Security tab.

Under Roles, select Enable Roles, followed by Create roles. Add a new role called Manager, and go back to the Security tab. Under Users, click Create user, and provide the user name and password as well as the other requested information. Make sure to check the Manager checkbox under Roles to make the new user a member of the Manager role. Click the Create User button and close Visual Studio 2005 (unless you would like to add more users and roles). Visual Studio 2005 uses a forward slash ("/") for the application name by default.

The LoginDialog Class

The sample application provides the Security menu item. When the user selects Security | Log In, the app brings up the LoginDialog dialog. The code for LoginDialog is shown in Figure 13.

Figure 13 LoginDialog Class Uses LoginControl (Close Figure 13)
partial class LoginDialog : Form
{
   LoginControl m_LogInControl;
   bool m_Authenticated;
   public LoginDialog()
   {
      Authenticated = false;
      InitializeComponent();
   }

   public bool Authenticated
   {
      get { return m_Authenticated; }
      protected set { m_Authenticated = value; }
   }

   void OnLogin(object sender,LoginEventArgs args)
   {
      bool successful = args.Authenticated;
      if(successful == false)
      {
         MessageBox.Show(
            "Invalid user name or password. Please try again",
            "Log In",MessageBoxButtons.OK,MessageBoxIcon.Hand);
      }
      else
      {
         Authenticated = true;
         Close();
      }
   }

   static public void Logout()
   {
      LoginControl.Logout();
   }

   static public bool IsLoggedIn
   {
      get { return LoginControl.IsLoggedIn; }
   }
}

LoginDialog contains the WSLoginControl. When you open LoginDialog in the Windows Forms Designer, you can set the WSLoginControl properties. LoginDialog sets the ApplicationName property to "/" and CacheRoles to False. The application configuration files set the UserManager Web service address to http://localhost/SecurityServices/UserManager.asmx.

LoginDialog subscribes to the LoginEvent event of LoginControl, providing the dialog's OnLogin as the event-handling method, as shown here:

m_LogInControl.LoginEvent += OnLogin;

In the OnLogin event-handling method, LoginDialog alerts the user with a message box if the login attempt failed. If the login was successful, LoginDialog sets a public property called Authenticated to true, and closes itself. Authenticated is used by the client of LoginDialog to find out the authentication outcome. Authenticated will be false if the user closed LoginDialog without logging in. Note that Authenticated uses public get and protected set accessors to allow clients to retrieve the value, but not set it. The Security menu of the sample app also contains a Log Out option. The implementation calls the Logout static method of LoginDialog, which delegates to LoginControl.Logout, thus detaching CustomPrincipal from the current thread.

I also wanted to constantly inform the user of the login status. The sample application uses a timer to update its status bar. To find out if the user is logged in, on every timer tick event the application checks the value of the LoginDialog.IsLoggedIn static property and updates the status bar accordingly. LoginDialog.IsLoggedIn simply delegates to LoginControl.IsLoggedIn.

Conclusion

Providing a credentials management solution requires a holistic approach, which takes into account the application deployment and code access security needs, as well as scalability, extensibility, design-time integration, and reuse contexts.

Whatever the reason your Windows Forms application cannot use Windows accounts, you now have access to the well-designed ASP.NET 2.0 security credentials store, providing the latest best practices for credentials management. The LoginControl provides you not only with the same productivity-oriented programming model as ASP.NET but also with a unified credentials store. Your middle tier can easily use different front ends with the same authentication and authorization infrastructure.

Juval Lowy is a software architect providing .NET architecture consultation and advanced training. He is also the Microsoft Regional Director for the Silicon Valley. This article contains excerpts from his upcoming book, Programming .NET Components 2nd Edition (O'Reilly, 2005). Contact Juval at www.idesign.net.


Speed: NGen Revs Up Your Performance with Powerful New Features

Speed

NGen Revs Up Your Performance with Powerful New Features

Reid Wilkes
This article is based on a prerelease version of the .NET Framework 2.0. All information herein is subject to change.
This article discusses:
  • How NGen can improve the performance of managed apps
  • How to know when NGen is right for you
  • New NGen features in the .NET Framework 2.0
This article uses the following technologies:
The .NET Framework and the CLR
Contents
Enter NGen 2.0
Is NGen Right for You?
NGen Hardbinding
Using the New NGen
Tying It All Together with an Example
Shutting Down

M
ost managed code developers have experimented with or at least heard of the NGen tool which ships with the Microsoft® .NET Framework. As a matter of fact, NGen has been around since the .NET Framework 1.0 release, and Microsoft has always used NGen on its own core Framework assemblies at .NET Framework installation time.

So why write an article on NGen now? There are two answers to this question. With the upcoming release of the .NET Framework 2.0, NGen has been greatly improved and offers a number of compelling new features to make it easier and faster to deploy high-performance managed applications. This article will introduce and describe some of these new features. In addition, this article will explain how and when you should consider using NGen to improve the performance of your apps.

Figure 1 NGen Execution
Figure 1 NGen Execution

Let's start by quickly going over the basics of what NGen does. As you know, the just-in-time (JIT) compiler is used for compiling the Microsoft intermediate language (MSIL) code in a .NET assembly into native code for the local machine immediately before the code is executed. This is temporary code; it is created and stored in the running processes's memory space, which is reclaimed by the OS when the process dies. Thus, it will be regenerated each time a new process needs it.

Pre-compilation, or pre-JITing, is the creation of native code from MSIL on the client machine, but instead of being generated at run time it is done as a separate isolated step. The output from pre-JITing is persisted such that the native code can later be used in lieu of running the JIT at program execution time. NGen is the term used generically for pre-JIT technology in the common language runtime (CLR), and specifically for the command-line utility used to create and store the native code. The native code produced by NGen is stored in true Win32® PE files which are called "native images" or "NGen images" which are in turn stored in the "native image cache" or "NGen cache." "NGen cache" is also used more generally to refer to all the information that NGen stores—not only the actual native images but also the information about which assemblies have native images.

When creating native images, NGen simply loads the JIT compiler and invokes it to create native code for the assembly, which is then persisted into the native image and stored in the native image cache. Subsequently, when the assembly is used by a managed application, the CLR loader will locate, load, and use the native images. Figure 1 and Figure 2 illustrate all the steps involved in these mechanisms. (See the sidebar "Troubleshooting Native Images".)

The NGen technology in versions 1.0 and 1.1 of the .NET Framework has some serious drawbacks. All managed assemblies have dependencies, even if the only dependency is on mscorlib.dll. NGen images contain code and data structures which are based on assumptions about the dependencies of the assembly. They are therefore highly fragile to changes in those dependencies. If those dependencies change, it invalidates the code in the native image and the runtime will fail to use the native image, instead reverting to JIT compilation. This problem is particularly bad when considering that if a component shared by many applications is updated, for instance System.dll, then this will cause the native images for many applications on the machine to be invalidated.

Enter NGen 2.0

Fixing the fragility issues surrounding native images was one of the primary goals for the .NET Framework 2.0 NGen technology. There are a couple of approaches which were available from the start. The first is to actually remove the fragile interdependencies from native images. Most of these interdependencies come from compiler optimizations such as cross-assembly inlining of methods, so this option would have hurt performance enough to make it an unacceptable tradeoff. The other approach, and the one which has been taken, is to make the management of native images and their dependencies as automatic as possible in order to relieve developers from having to worry about whether their application performance will be affected if a shared component is updated.

Figure 2 How Native Images Are Used
Figure 2 How Native Images Are Used

To achieve these goals, the NGen tool interface was changed to expose a new usage model. Instead of invoking NGen on an individual assembly, NGen is now invoked on an application. This is done by running NGen on the top-level assembly in the application (the top-level assembly is the assembly that is not depended on by any other assemblies). NGen will automatically discover all the available dependencies of the application and make sure that all the dependencies also have native images (this only applies to static dependencies that are recorded in the assemblies' manifest—any assembly explicitly loaded using methods like Assembly.Load will not be discovered by NGen). In addition, NGen keeps a record of all the dependencies of the application so that if a shared component is later updated, NGen can discover all applications that are affected by the update and regenerate native images where needed.

In addition to this tracking and updating mechanism, NGen now also supports deferred commands. This means that if you want to use NGen with a large application (which may take a significant amount of time to complete), you can queue this command to be completed by the new NGen service. The NGen service is a true Windows® service process that runs in the background and waits for idle time on the machine, at which point it will complete any outstanding compilation jobs that have been waiting in the queue.

The NGen queue supports a simple priority scheme to help establish the order in which applications are pre-compiled when submitting multiple jobs to the queue. The priorities also determine whether the service should wait for idle time on the machine to do the background compilation. This queuing functionality is especially useful in conjunction with the feature to update all native images that have been invalidated by the servicing of some shared component. Because this update may take a significant amount of time, you may not want to wait for the NGen update command to complete before exiting the component update installer. Instead, you may choose to queue the update command and allow the NGen service to take care of the compilation in the background.

In addition to the major new features, there have been a number of other changes that make NGen more usable and reduce the occasions when native images are thrown away in favor of running the JIT. For instance, native images created by NGen 2.0 can be shared across application domains in a process. In previous versions, native images could only be used by a single application domain. This meant that the performance gains achieved through NGen were largely lost for any application that used multiple application domains. One key place where this change will be realized is in ASP.NET applications, which previously were unable to use native images for this very reason.

Another improvement is that reflection is fully supported using only native images. This eliminates another entire class of scenarios in which native images could not be used. Also, the new generics feature is almost 100 percent supported for NGEN images. There are a few rather obscure cases in which the JIT might be pulled in, but these are rare. This particular feature was quite a feat and if you're interested in learning more about it, the implementation is well described at Combining Generics, Precompilation and Sharing Between SoftwareBased Processes. In addition, there have been quite a few small but effective performance tricks implemented in the .NET Framework 2.0 version of the CLR that focus specifically on the NGen case. Using NGen images is the primary way to achieve maximum performance with apps targeting the .NET Framework 2.0.

Is NGen Right for You?

So, NGen has some great features—but how do you know when to use it? Well, there are a couple of things NGen does for applications. It can reduce the private memory pages used by applications, which can, in turn, have a positive impact on system-wide memory pressure. It can also improve the startup time of the application. I'll look at each of these in turn.

When managed applications execute in the standard JIT case, they load an assembly containing MSIL code into memory and then invoke the JIT compiler on that MSIL code. The JIT compiler allocates private memory (which cannot be shared with other processes) and then writes the compiled machine code into that memory. There are several ways that native images improve the memory usage in the JIT case. First, the JIT compiler itself is not loaded into memory. The JIT compiler is implemented in a separate DLL from the rest of the runtime and is only loaded by the runtime if needed. This reduces memory consumption by a constant amount for any app using NGen.

Second, native images are loaded by the CLR just like any other DLL. This means that the actual memory pages containing the machine code can be shared by the OS between multiple processes. The effect of this is that any app using one of these shared native images is able to use these code pages already loaded into memory and thus reduce the number of new pages committed for that process. Correspondingly, the total amount of memory needed system-wide for all instances of the app using those native images is reduced. This benefit is only visible when considering shared components, such as the .NET Framework assemblies or apps that are used in multiple sessions on Terminal Server (see Figure 3).

Figure 3 Shared Memory
Figure 3 Shared Memory

Application startup time can also be decreased by NGen under certain conditions. During startup, modules are loaded, memory is allocated, and the CLR executes a tremendous amount of code to initialize itself and, in some cases, JIT-compile the user code. Measurements in the CLR performance labs have shown significant startup time improvements when using native images instead of JIT compilation. These gains don't necessarily apply to all applications, though, and are usually limited to the warm startup case. Warm startup occurs when a managed program has been run previously and hence much of the code for the runtime and the native images has already been loaded from disk into memory. Cold startup—launching a managed application for the first time immediately after starting the computer—does not benefit as much from NGen because costly disk accesses dominate the startup time.

Despite these NGen benefits, it is not always the best choice. The performance issues can become very complicated, involving code and data access patterns, how code and data are laid out in a file, how many calls are made across module boundaries, and how much of the necessary infrastructure has already been loaded by other applications and is resident in memory. These and many other considerations make analysis difficult, but what matters are the real numbers derived from actually measuring your application. The CLR performance architects stress that doing careful, repeated measurements of your app in its various possible configurations is the only way to know the best course to take.

One more detail is also worth mentioning. If you are using strong-named assemblies, NGen is probably not a gain for you unless they are in the Global Assembly Cache (GAC). The CLR loader does extra validation on strong-named assemblies that aren't in the GAC, which causes nearly every page in the native image to get touched, thereby usually completely eliminating any startup gains that would be realized through the use of NGen.

There are a number of far more detailed discussions of NGen performance implications available online and easily located with a simple search. The authoritative reference is in Chapter 5 of "Improving .NET Application Performance and Scalability".

NGen Hardbinding

Another new feature of .NET Framework 2.0 NGen is hardbinding. In some cases, the JIT has a natural advantage over NGen because the JIT has a more complete knowledge of the virtual addresses of code and data across all modules and thus can create code that makes calls and loads data using actual virtual addresses as the targets. NGen, on the other hand, will traditionally build stubs into the code when writing calls or data accesses whose targets are in another assembly. These stubs are basically calls into a very small, tight routine in the runtime which calculates the actual virtual address of the desired method or data structure and then "backpatches" the call site by overwriting the call to the stub with a call to the actual target virtual address. This backpatching will cause a previously read-only code page to become writeable, and thus no longer shareable. Hardbinding allows NGen to create the same kind of code as the JIT and eliminate this backpatching (and subsequent reduction in shareable pages) at the cost of forcing native images to be very tightly bound. This tight binding means that all the native images that are hardbound to each other must be loaded at the same time. In terms of startup performance, this may turn out to be a detrimental effect if under normal circumstances not all of your assemblies are loaded at startup time.

The decision to use hardbinding is ultimately left to the runtime and NGen, but you can supply hints using the System.Runtime.CompilerServices.DependencyAttribute and also the System.Runtime.CompilerServices.DefaultDependencyAttribute. The former attribute is placed on an assembly and tells NGen how likely it is for a specific dependency to be loaded. The latter attribute is placed on a dependency and states how frequently that dependency will be loaded. If no DependencyAttribute is specified on a parent, then the DefaultDependencyAttribute of the child is used. Both attributes get their values from the System.Runtime.CompilerServices.LoadHint enumeration, which has values of Always, Default, and Sometimes. Using the value of Always is a hint to the runtime and NGen to use hardbinding for that dependency. The other values mean the same—don't use hardbinding.

If you are considering using these attributes, you need to carefully weigh the data from doing performance experiments to make sure it is going to give you what you need. Whether this is the right choice depends heavily on the particular application in question. Microsoft uses hardbinding attributes on only a few of the base .NET Framework assemblies like mscorlib.dll. These assemblies contain a DefaultDependencyAttribute set to LoadHint.Always, which means that unless overridden, other assemblies will hardbind to them by default. Also, be aware that it's important to get base addresses right with hardbinding since hardbound assemblies bear a higher cost for base address relocations. For more information, see the sidebar, "NGen Image Base Addresses."

Using the New NGen

Now let's look at how to actually use these new features. Note that as of this writing, the information here is more current than the information in the .NET Framework documentation for Beta 2.

You can only access NGen functionality through the NGen.exe command-line tool. The syntax for using this tool has changed dramatically with the .NET Framework 2.0. Although the old syntax is still supported, it does not make full use of the new functionality. I recommend moving completely to the new syntax, so I won't discuss how the old commands work.NGen Image Base Addresses

Getting the base address right for NGen images is at least as important as it is for unmanaged code modules. If an NGen image misses its preferred base address, the CLR has to execute special code to patch up the native image at its new address. This effect is even more pronounced for hardbound native images since they require more work to patch up and are more sensitive to relocations. If any hardbound dependency gets rebased then all the callers into that assembly must also be fixed up at load time. The way to set the base address for a native image is by setting the base address on the assembly created by the language-specific compiler. The C#, Visual Basic®, and C++ compilers all have options that allow for controlling the base address of the assembly produced. In addition, the ".imagebase" directive in MSIL will cause ILASM to produce an assembly with the specified base address. NGen will retain this base address for the native image except when the assembly is an executable (.exe). In this case, the base address on the native image will be offset enough to allow room for the executable itself to load at its preferred base address. Because the native images are true Win32 PE files, it is possible to view the preferred base address of a native image in the cache using a tool such as dumpbin.exe.

Every NGen command line now begins with an action specified as the first parameter. The actions available are shown in Figure 4. The install action adds the native image for a top-level assembly, known as a root, to the NGen cache. NGen will also automatically find all the dependencies of the root and add native images for those assemblies to the cache as well. The command line needed to install the root AccountingApp.exe into the NGen cache would look like the following:

ngen install AccountingApp.exe

Figure 4 NGen.exe Command-Line Interface (Close Figure 4)
NGEN INSTALL <root name> [<build configuration flags>] [<config flags>]
[<option flags>] [/queue:<priority>]
NGEN UNINSTALL (<root name> | * ) [<build configuration flags>] [<config flags>]
[<option flags>]
NGEN DISPLAY [<root name>] [<build configuration flags>] [<config flags>]
[<option flags>] [/verbose]
NGEN UPDATE [ * ] [<option flags>] [/queue]
NGEN EXECUTEQUEUEDITEMS <priority> [<option flags>]
NGEN QUEUE ( pause | continue ) [<option flags>]

Scenario Flags:
    /Debug
    /Profile
    /NoDependencies
Config Flags:
    /AppBase:<path>
    /ExeConfig:<full path to exectable>
Option Flags:
    /nologo
    /silent
    /verbose
Priority Values:
    1, 2, or 3.
Root Names:
    Either a file name or assembly display name for assembly in GAC.

When locating the dependencies of AccountingApp.exe, NGen will use the same assembly-probing logic that the CLR uses at run time. NGen defaults to using the directory containing AccountingApp.exe as the AppBase. This means that assembly probes will be performed starting in this directory. This is different from previous versions of NGen, where the AppBase was set to the current directory. For the most part, this new behavior works as expected and does the right thing.

There are, however, times when NGen needs some special help to locate dependencies. The /appbase switch allows you to override the AppBase from the directory containing the assembly to any directory you specify. The /execonfig switch can be used when installing a DLL assembly to the native image cache. This switch allows you to specify the name of an executable that has a configuration file associated with it. NGen will use the information in that executable's configuration file to guide the lookup of dependencies for the DLL being installed.

Assemblies in the GAC can be installed to the NGen cache as well. The syntax required to do this involves using the display name of the assembly instead of the file name:

ngen install "payroll, Version=1.0.1.2, Culture=neutral,
    PublicKeyToken=ab7e26f746a3c28e, processorArchitecture=MSIL"

NGen also supports various build configuration flags, which can be supplied with the install action. The most useful of these are debug and profiling. The debug configuration flag tells NGen to create a set of native images for the root which can be used in a debugger. If you try to launch an application under a debugger using default native images, the runtime will instead use the JIT to create debuggable code. Passing the debug flag to NGen causes the debuggable code to be created in the native image. This allows you to launch a program in the debugger and still have that program use code from the native images. This is the only way to attach to a running program using native images with a managed debugger and be able to see call stacks, locals, and all the other familiar debugging tricks. Similarly, the profiling switch tells NGen to create native images, which can be used under a profiler. By using these switches, it is possible to create multiple build configurations for a single root in the NGen cache. For instance, I could run the following two commands:

ngen install AccountingApp.exe
ngen install AccountingApp.exe /debug
After issuing these two commands I would then have both a default and debug set of native images for AccountingApp in the native image cache.

The uninstall command is used much like the install command but, quite obviously, removes roots and their dependencies from the NGen cache. Because NGen tracks the dependencies of roots in the cache, you don't have to worry about accidentally removing the native image for a shared component that some other root has a dependency on. NGen will make sure that as long as some installed root has a dependency on a shared component, the native image for that shared component will remain in the cache. Uninstall, like install, supports the build configuration (debug, profile) and assembly probing (appbase, execonfig) flags. This allows you to selectively remove the native images for a debug configuration of a root without removing the images for the default configuration of that same root. You should note, however, that running uninstall without any flags tells NGen to uninstall all the configurations for the specified root. Here are some examples of uninstall commands:

ngen uninstall Payroll.dll

ngen uninstall "payroll, Version=1.0.1.2, Culture=neutral,
    PublicKeyToken=ab7e26f746a3c28e, processorArchitecture=MSIL"

ngen uninstall "payroll, Version=1.0.1.2, Culture=neutral,
    PublicKeyToken=ab7e26f746a3c28e, processorArchitecture=MSIL" /debug

ngen uninstall payroll

The first command removes the root Payroll.dll, which is located in the current directory. This would include any build configurations for that root. The second removes the root for the assembly in the GAC that matches the given display name. This command would also remove all build configurations for the root. The third command only removes the debug configuration for the root "payroll, Version=1.0.1.2, Culture=neutral, PublicKeyToken=ab7e26f746a3c28e, processorArchitecture=MSIL". The final command is the simplest form of uninstall and will remove all roots with the simple name payroll, which would include all the roots in the previous examples.

The display command is how you can peer into the native image cache. It can be used with no arguments, like so:

ngen display
This would display information about all the roots in the cache. It can also be used with a whole or partial assembly name:
ngen display payroll.dll
ngen display payroll

Both of these commands would display only the information for the root Payroll.dll. The first command only works if you either specify the full path to Payroll.dll, or are in the same directory as Payroll.dll. The second command would also display the information for the root named "payroll, Version=1.0.1.2, Culture=neutral, PublicKeyToken=ab7e26f746a3c28e, processorArchitecture=MSIL". Display outputs two sections of output: the first is a list of all root assemblies, and the second is a flat list of every single native image in the system's cache. The /verbose switch will force much more state information to be printed, but I won't go into those details here. NGen display will also print a list of roots which depend on a given assembly's native image if you use syntax like the following:

ngen display payroll

This command will display all the roots with a simple assembly name of payroll, all roots that depend on an assembly with a simple assembly name of payroll, and all the native images for assemblies with a simple assembly name of payroll. This can be useful for determining how many roots will need to be regenerated after an update to a shared component.

The next NGen action is "update." The update action takes no arguments (that have been covered yet here) and simply forces NGen to recalculate the dependencies of all roots in the cache, and generate new native images for anything that is not up to date. Update is primarily intended to be used after a shared component on the system has been updated. The command simply looks like:

ngen update
If there are a large number of roots in the cache, then the NGen update command may take a significant amount of time to complete. It is primarily for this reason that the deferred NGen capabilities were added for the .NET Framework 2.0.

Deferred NGEN allows you to issue a command to NGen, through the command-line tool, which tells NGen to do some action using the service in the background. This is accessed through the /queue switch, which is supported for both the install and update actions. For instance, issuing

ngen update /queue
tells NGen to do a full update action, but to wait to do all the work until the machine is idle. The command itself would return immediately, and thus a script that called it would be able to finish even though the NGen work had not yet completed. The deferred command will be completed by the NGen service process following the idle time detection rules. These are subject to fine-tuning before the final product releases so I want to avoid providing too much detail. In general, though, the service notices that there has been no user input on the machine for a timeout period, then the service declares the machine idle and will begin launching normal priority worker processes to perform any necessary compilation.

The install command has an additional feature: the /queue switch accepts a numeric argument between 1 and 3 that represents the priority with which to perform the deferred install command. Priority 1 and 2 commands are completed by the service in the background, but without waiting for idle time. Instead, workers are immediately launched by the service to begin doing the compilation work. Priority 3 commands follow the same idle time rules as update commands do. As a matter of fact, update commands are always implicitly set at Priority 3. An install command might look like this:

ngen install AccountingApp.exe /queue:2

This command would add a new root to the cache for AccountingApp.exe. It would then tell the service to immediately begin compiling native images for AccountingApp.exe and all of its various dependencies.

The difference between priority 1 and priority 2 is that the service ensures that commands issued with a priority of 1 are completed before commands issued with a priority of 2 and so on. Other than that, priorities 1 and 2 behave the same way in that neither wait for idle time on the machine, but rather immediately cause the service to begin the background compilation. The priorities can also be useful in certain scenarios when used in conjunction with the executequeueditems action. This action forces commands in the queue to be immediately executed. It is a synchronous command, so it will not return until all the work specified is completed. Executequeueditems takes a priority number as an optional argument, and will only process queued commands whose priority numbers are less than or equal to that particular level. For instance,

ngen executequeueditems 2
will force NGen to synchronously process all commands in the queue with priorities 1 and 2. The obvious conclusion, then, is that
ngen executequeueditems 3
is the way to force every queued command to be completed synchronously. The default priority for executequeueditems is 3.

One thing to note is that synchronous commands are carried out without going through the service at all. This means that NGen commands without the /queue switch, including executequeueditems, are completed entirely by ngen.exe and the worker processes it launches. These commands will work even if the service is disabled, and executing these commands will not cause the service to be started. The service itself is a Windows service process whose normal state is to be completely shut down. When commands are queued to the service through ngen.exe, the service is started and remains started even through reboots until there are no further queued commands to process. At this point, the service sets itself to not start up at reboot time and shuts itself down. The service will then remain shut down until yet another queued NGen command is issued.

The last action I need to cover makes for a nice segue into a discussion of some common usage scenarios. The queue action (not to be confused with the /queue switch) is the way to control the activity of the service. There are three supported arguments to the queue action: pause, continue, and status. It is recommended that any installer issue a queue pause command before even beginning to copy files to the machine:

ngen queue pause
This will ensure that the service does nothing in the background while the installer is copying files, putting assemblies into the GAC, or pre-compiling them. The purpose is to avoid conflicts between the installer and the service trying to access the same resources. At its completion, the installer can issue the command,
ngen queue continue
which will allow the service to resume processing any queued commands, or go back into a shutdown state if there is no more work to do. The status argument to ngen queue causes diagnostic information about the service process to be printed out to the console.

Tying It All Together with an Example

Let's put all this together by examining a scenario and looking at what happens behind the scenes by using the display command. In this particular scenario, we are building a deployment package for a suite of tools used by small businesses. The core tools are Inventory.exe, TaxWizards.exe, and Books.exe. In addition to these core tools, the package deploys a DLL which is shared by all applications in the suite, Payroll.dll, and a component from a third-party development toolkit called Chart.dll which could potentially be used by other applications on the user's system. The installation requirements allow a relatively short time for installation which prevents running NGen on every component at installation time. However, these applications are relatively large and the startup time gains from using NGen have been shown to be significant. The decision has been made to finish the pre-JITing of the flagship application, Books.exe, during installation and then queue the commands to pre-compile the rest of the applications after installation. Although somewhat contrived, this example scenario gives us a broad coverage of all the core types of tasks supported.

The install script for this package might look this:

ngen queue pause
... (xcopy all files to application directory)
... (install Payroll.dll to GAC)
... (install Charts.dll to GAC)
... (add registry keys for application)
ngen install Books.exe /queue:1
ngen install Inventory.exe /queue:2
ngen install TaxWizards.exe /queue:2
ngen update /queue
ngen executequeueditems 1
ngen queue continue
First, the script pauses the service to prevent any resource conflicts, then copies all files, puts the shared components into the GAC, and sets up necessary registry keys. The script then installs the three top-level assemblies as roots in the native image cache by using the queued NGen install commands. After this, the script issues a queued NGen update. The purpose of this is to ensure that any native images that depend on the shared component Chart.dll are updated if necessary. This would only be necessary if an assembly that depends on Chart.dll is automatically upgraded to the new Chart.dll currently being installed. That could be the case if the update to Chart.dll differs only in the build revision part of the version number (for example, 1.0.0.1 to 1.0.0.3) or if the binding policy affecting this component was updated. At any rate, the safe bet is to always run the NGen update command if installing a component to the GAC.

Next, in the script, the executequeueditems command is issued. The argument to this command is 1, which tells it to only force the processing of queued commands with priorities of 1. Finally, the NGen queue is reactivated with the "queue continue" command. After this script completes the last command, the service will automatically begin processing the priority 2 commands, which were queued. Remember that priority 2 commands do not require the service to wait for idle time, but rather are processed eagerly in the background. At some point in the future when the service detects that the machine is idle, it will complete the queued update command, which is implicitly at priority level 3.

Let's look at the state of the cache before the script is run:

>ngen display
Microsoft (R) CLR Native Image Generator - Version 2.0.41203.00
Copyright (C) Microsoft Corporation 1998-2002. All rights reserved.

Roots:

C:\WINDOWS\Microsoft.NET\Framework\v2.0.x86ret\mscorlib.dll

Native Images:

mscorlib, Version=2.0.0.0, Culture=neutral,
PublicKeyToken=b77a5c561934e089
Note that the display output is trimmed in these examples to include only the relevant information. In this output there is only one root added to the cache: mscorlib.dll. You can also see under the "Native Image:" heading that there is one native image—the native image for the mscorlib root. Now look at Figure 5, which shows the output from the display command after running through all three install commands in the script. You can see that there are four roots: the mscorlib root, which existed previously, and a new root from each of the ngen install commands. You'll also see that the new roots are annotated with the tag "StatusPending". This means that the installation of these new roots has only been queued and no actual work has yet been done. This is also evidenced by the fact that there is still only one native image listed—the native image for mscorlib.

Figure 5 Output of Display Command (Close Figure 5)
>ngen display
Microsoft (R) CLR Native Image Generator - Version 2.0.41203.00
Copyright (C) Microsoft Corporation 1998-2002. All rights reserved.

Roots:

C:\WINDOWS\Microsoft.NET\Framework\v2.0.x86ret\mscorlib.dll
D:\Scratch\SBApp\Books.exe (StatusPending)
D:\Scratch\SBApp\Inventory.exe (StatusPending)
D:\Scratch\SBApp\TaxWizards.exe (StatusPending)

Native Images:

mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

Figure 6 shows the display output after running the queued update command. The difference in this output versus Figure 5 is that now even the mscorlib.dll root has been marked as "StatusPending". This gives us a little insight into how the update command is implemented. Update begins by marking all the roots in such a way that tells the service it needs to look at the root again, regenerate a list of the dependencies, and generate native images for all assemblies in that list that don't have valid native images already in the cache. This explains the new status you see on the mscorlib.dll root. You can also see that there is still a native image for mscorlib.dll. Update does not get rid of native images unless they prove to be invalid. When this update is processed by the service it will detect that there is already a valid native image for mscorlib.dll in the cache and will not regenerate a new one.

Figure 6 More Display Command (Close Figure 6)
>ngen display
Microsoft (R) CLR Native Image Generator - Version 2.0.41203.00
Copyright (C) Microsoft Corporation 1998-2002. All rights reserved.

Roots:

C:\WINDOWS\Microsoft.NET\Framework\v2.0.x86ret\mscorlib.dll (StatusPending)
D:\Scratch\SBApp\Books.exe (StatusPending)
D:\Scratch\SBApp\Inventory.exe (StatusPending)
D:\Scratch\SBApp\TaxWizards.exe (StatusPending)

Native Images:

mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089

After running the executequeueditems command you can take another snapshot look at the cache, shown in Figure 7. The root for Books.exe is now complete as it no longer has the "StatusPending" tag attached to it. The other roots that were added are still pending. There are now native images created for Books.exe and its dependencies: Chart.dll and Payroll.dll. The final command in the script tells the service to once again begin doing background compilation. As stated earlier, the service will process any queued commands in decreasing priority order. Therefore the queued priority 2 installs for Inventory.exe and TaxWizards.exe will be completed first (and without waiting for idle time). The queued update command will be executed last (at idle time) and by then, provided no other commands are executed, the only work left to be done for the update is to make sure there are good native images for the root mscorlib.dll. The final state of the cache after all work has been done is shown in Figure 8.Troubleshooting Native Images

The native images produced by NGen are stored in a private cache maintained by the CLR, and the CLR uses them automatically. Overall, the system is very much a black box and is Just Supposed to Work. It's useful, however, to be able to confirm that native images are actually being used by your application to make sure that you are not hitting an edge condition which causes the native images to be discarded. You can see whether native images are being used in your process by attaching a debugger and looking to see if the native image PE file is loaded into the process. You must attach with the debugger to do this test. Unless you created debug native images, the native images will not be loaded at all if you launch the program under the debugger. The name of the native image file is the same as the name of the original assembly, with ".ni." inserted between the file name and the file extension. For instance, mscorlib.dll's native image will be named "mscorlib.ni.dll". Another way to examine how native images are being used is through the fuslogvw.exe tool which ships with the .NET Framework SDK. This tool has a mode enabled by a radio button on the main panel to log binding information for native images. Fuslogvw also provides the useful feature of logging information about why native images are rejected when that is an issue.

Figure 8 Final Cache State (Close Figure 8)
>ngen display
Microsoft (R) CLR Native Image Generator - Version 2.0.41203.00
Copyright (C) Microsoft Corporation 1998-2002. All rights reserved.

Roots:

C:\WINDOWS\Microsoft.NET\Framework\v2.0.x86ret\mscorlib.dll
D:\Scratch\SBApp\Books.exe
D:\Scratch\SBApp\Inventory.exe
D:\Scratch\SBApp\TaxWizards.exe

Native Images:

Books, Version=1.0.1.1, Culture=neutral, PublicKeyToken=3a0f3302b4d4c63a
mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
Payroll, Version=1.0.1.1, Culture=neutral, PublicKeyToken=3a0f3302b4d4c63a
Chart, Version=3.1.0.0, Culture=neutral, PublicKeyToken=3a0f3302b4d4c63a
Inventory, Version=1.0.1.1, Culture=neutral, PublicKeyToken=3a0f3302b4d4c63a
TaxWizards, Version=1.0.1.1, Culture=neutral, PublicKeyToken=3a0f3302b4d4c63a
Figure 7 The Cache (Close Figure 7)
>ngen display
Microsoft (R) CLR Native Image Generator - Version 2.0.41203.00
Copyright (C) Microsoft Corporation 1998-2002. All rights reserved.

Roots:

C:\WINDOWS\Microsoft.NET\Framework\v2.0.x86ret\mscorlib.dll (StatusPending)
D:\Scratch\SBApp\Books.exe
D:\Scratch\SBApp\Inventory.exe (StatusPending)
D:\Scratch\SBApp\TaxWizards.exe (StatusPending)

Native Images:

Books, Version=1.0.1.1, Culture=neutral, PublicKeyToken=3a0f3302b4d4c63a
mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
Payroll, Version=1.0.1.1, Culture=neutral, PublicKeyToken=3a0f3302b4d4c63a
Chart, Version=3.1.0.0, Culture=neutral, PublicKeyToken=3a0f3302b4d4c63a

Shutting Down

Once the service completes all the work queued during the installation script, it will shut itself down so that it is no longer consuming any resources. It will only start running again if another command is queued through NGen. It has been one of the primary design goals of this feature that it remain as invisible and unobtrusive as possible. As a whole, in fact, the goal with the NGen technology in the CLR is to provide a robust, hands-off infrastructure for gaining the performance benefits of pre-JITing. The .NET Framework 2.0 features of NGen go a long way towards meeting this goal.

Now that you've learned about the new features of NGen that are available in the .NET Framework 2.0 release, you're all ready to incorporate this technology into your own managed applications. Remember, measure your application carefully in order to decide whether NGen is right for both your application and your customer's scenarios based on the data that you collect. With all of the improvements in the upcoming .NET Framework 2.0, NGen may be just the way to give your managed applications the performance edge that you need.

Reid Wilkes works for Microsoft. He is a member of the quality assurance team for the common language runtime and focuses on the NGen technology.


Escape DLL Hell: Simplify App Deployment with ClickOnce and Registration-Free COM

Escape DLL Hell

Simplify App Deployment with ClickOnce and Registration-Free COM

Dave Templin
This article is based on a prerelease version of Visual Studio 2005. All information herein is subject to change.
This article discusses:
  • The benefits of Reg-Free COM
  • How to isolate existing COM components and ActiveX controls using Visual Studio 2005
  • The limitations of Reg-Free COM
This article uses the following technologies:
Visual Basic, COM, ClickOnce
Code download available at: RegFreeCOM.exe (1,392KB)
Contents
Reg-Free COM Overview
Isolating COM Components
Isolating a Simple COM Component
A More Complex Example
Native Assemblies
Conclusion

I
f you're using Business Objects or UserControls written in Visual Basic® 6.0 or Active Template Library (ATL), then you're using COM. Prior to the Microsoft® .NET Framework, COM was the principal component model that applications in Windows® were built upon. But deploying applications with COM components can be difficult because in order for them to work, they need to be globally registered on the machine. Unfortunately, this global registration can cause unwanted side effects over the lifecycle of applications that use these components, unless they are managed very carefully. This is called DLL Hell.

These factors are generally not a problem in .NET because components do not require registration, and are either totally isolated to an application or are managed in a well-defined side-by-side way with the help of the Global Assembly Cache (GAC). Various migration wizards can help to move your code forward to the .NET Framework, but this isn't always practical. There's also the school of thought that says, "If it ain't broke, don't fix it!" Wouldn't it be great if you could deploy your existing COM components using a model similar to the .NET Framework? Well, starting in Windows XP, it turns out that you can!

Windows XP introduces a new COM activation model called registration-free COM, or Reg-Free COM for short. Simply stated, Reg-Free COM is a registry replacement for COM components. It works by expressing all of the standard component registration information that is typically installed into the system registry in a file that can be stored in the same folder as the application itself.

This article will show you how to deploy applications that use existing COM components using the new support in Visual Studio® 2005 for Reg-Free COM. This functionality allows such applications to be deployed with simpler deployment models such as XCOPY and ClickOnce, overcoming the traditional factors that make the deployment of COM components sometimes difficult.

ClickOnce is a new deployment technology for Windows Forms apps that is being introduced in the .NET Framework 2.0. It is a simplified application deployment model that works over the Web, from network shares, or from distributable media such as CDs. It also provides auto-updating capabilities, safe and isolated application management, and configurable application security. In a nutshell, it makes deploying and updating a Windows Forms application as easy and safe as a Web application. For more information, see the article on ClickOnce by Brian Noyes in the May 2004 issue of MSDN®Magazine.

Reg-Free COM Overview

Traditionally, when an application uses a COM component, the component must be registered for the operation to succeed, as shown in Figure 1. This is true whether the application is native or based in the .NET Framework. When an application instantiates a COM component, the activation generally boils down to the CoCreateInstance API. In simple terms, CoCreateInstance looks in the registry for the identifier (CLSID) specified by the application, finds the associated component module, and activates the corresponding class.

Figure 1 Traditional COM Activation
Figure 1 Traditional COM Activation

The benefit of describing components in the system registry is that it enables them to be shared among multiple applications. It's also possible to register a component such that it can be installed and activated on another machine over the network. If your application is not using any of these capabilities, then this global machine registration can also be a huge liability for your application. First, ensuring every component is properly registered usually requires the overhead of a separate setup program, such as an MSI or EXE installer. Also, component registration may involve anywhere from dozens to thousands of registry keys, depending on the size of the component. If any one of these entries is corrupted, it can cause the application to fail. Having applications, components, and registration scattered all over the machine can make it very hard to properly manage application updates and uninstallation, particularly if some of the components are shared between multiple applications. This is probably COM's most prevalent problem as it makes some components and applications extremely fragile and difficult to maintain.

Reg-Free COM is different from traditional COM in that it doesn't require the component to be described separately in the system registry. It eliminates all of the previously mentioned problems, at the expense of maintaining the component definition within the scope of the application itself. It works by expressing all of the information that is typically installed into the system registry in an XML manifest. This is just another file in the application folder, as shown in Figure 2. If a component has a manifest, then during activation of the component, the operating system will look at the manifest before looking in the registry.

Figure 2 Isolated COM Activation
Figure 2 Isolated COM Activation

The presence of an application manifest (such as the WindowsApplication1.exe.manifest) in the same folder as the program (WindowsApplication1.exe) is all that is required to enable the Reg-Free COM component model. This works because each time a program is run, the Native Assembly Loader looks for a manifest file based on the name of the executable. Like the component model for .NET assemblies, there is no extraneous component definition required outside of the application folder. When one or more manifests are present, Windows builds an internal table that relates each identifier (CLSID) to its corresponding component file. This is done early in CreateProcess, before any application code is run. Later, when the application instantiates a component, CoCreateInstance looks in this internal table before looking in the system registry. If all components are fully described in one or more manifests, absolutely no component registration is required in the system registry for the application to function properly!

The principal benefit of this model is that the COM component is totally isolated within the application. Even if other applications that use the same COM component (or a different version of it) require it to be registered, it will not interfere with this application in any way. There is also no way that the COM component for this application can be accessed by other applications. Therefore, problems due to version conflicts or the need to change a component's GUID may be sidestepped by using Reg-Free COM exclusively, although this should not be used as an excuse to dispense with any versioning strategies. Finally, update and uninstallation couldn't be simpler—just remove or replace the application folder.

Incidentally, existing code does not generally need to be altered to take advantage of Reg-Free COM. Setting up the proper configuration of manifests is all that may be required. In fact, the original intent of Reg-Free COM was to enable existing native applications, such as ones written in Visual Basic 6.0, C++, or some combination of languages. The focus of this article, though, is on using Reg-Free COM from .NET-based applications.

As I mentioned, if the Native Assembly Loader fails to bind to a COM component using the manifest, it will fall back to the global component registration, if present. This behavior can give a false sense that an improperly configured Reg-Free COM application is working properly. Therefore, when verifying that an application is correctly configured for Reg-Free COM, it's best to test on a clean machine. Otherwise, you'll have to take care to unregister all of the constituent components before testing to ensure that the application is running fully Reg-Free.

Isolating COM Components

Visual Studio 2005 introduces a new feature that allows you to isolate any given COM component with the simple flip of a switch. It works by automatically generating a manifest from the component's type library and component registration. Therefore, it is important to note that isolating a COM component requires that it be registered on the developer's machine. The requirement to register the component on end-user machines is removed.

Every COM reference in Visual Studio 2005 has a new property called Isolated. By default, this property is False, indicating that it should be treated like a normal, registered COM reference. If this property is True, it causes a manifest to be generated for the component at build time. It also causes the corresponding component files to be copied to the application folder. When the manifest generator encounters an isolated COM reference, it enumerates all of the CoClass entries in the component's type library, relating each entry with its corresponding registration data. In this manner, manifest definitions are automatically generated for all the COM classes in a file.

The ability to isolate a COM component was primarily designed to enable Reg-Free COM deployment of Visual Basic 6.0 COM components within Visual Basic .NET or C# applications. This feature is not limited to Visual Basic 6.0 components, though, and may work with any COM component that is suitable for use with Reg-Free COM. Not all application and component scenarios are suitable for use with Reg-Free COM, however. See the sidebar, "The Limitations of Reg-Free COM" for more information.

In the next two sections, I'll demonstrate exactly how this works. First, I'll create a simple COM component in Visual Basic 6.0 and show how to deploy it using Reg-Free COM. Then, we'll look at a slightly more sophisticated example that shows how to make several ActiveX® controls work under Reg-Free COM as well.

Isolating a Simple COM Component

Let's start by creating a simple COM component in Visual Basic 6.0. If you have Visual Basic 6.0, the steps to create the component manually are trivial. Alternatively, you can simply copy the VB6Hello.dll file from the sample code for this article and register it by typing "regsvr32 VB6Hello.dll" from a command prompt. Or, you could use ATL or any development environment capable of producing a COM DLL.

Using Visual Basic 6.0, create a new ActiveX DLL named "VB6Hello" and add the following code into the Class1 code module:

Public Sub SayHello()
    MsgBox "Hello from Visual Basic 6.0"
End Sub
Build VB6Hello.dll by selecting the File menu and choosing the Make command. Note that only ActiveX DLL and ActiveX Control project types are supported with Reg-Free COM. ActiveX EXE and ActiveX Document project types cannot be used with Reg-Free COM, as discussed in the sidebar.

Next, we'll reference this COM component from a Windows Forms application. Start Visual Studio 2005, and create a new Visual Basic WindowsApplication project named "RegFreeComDemo1". You can also use C# or J# if you prefer, since the code will be trivial and easily adaptable to the language of choice. Select the Project menu, choose the Add Reference command, and browse to the VB6Hello.dll component produced from the code. Add a button to the form named "Say Hello", double-click on it to go to the code window, and add the following code:

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As _
    System.EventArgs) Handles Button1.Click
    Dim Vb6Obj As New VB6Hello.Class1
    Vb6Obj.SayHello()
End Sub

If you press F5 to run your application, you should see the appropriate message box when the button is clicked, just like the one shown in Figure 3.

Figure 3 RegFreeComDemo1
Figure 3 RegFreeComDemo1

Before we get to the Reg-Free COM part, let's take a moment to look at a few things. First, examine the set of files that were produced by the build: Interop.VB6Hello.dll, RegFreeComDemo1.exe, and RegFreeComDemo1.exe.config. RegFreeComDemo1.exe is, of course, the main program file. Interop.VB6Hello.dll is a COM interop assembly generated automatically by the build. The other files not shown are supporting files and do not pertain to our discussion. Notice, though, that the VB6Hello.dll COM component is not present in the folder.

Try unregistering the COM component (for example, type regsvr32 /u VB6Hello.dll from a command prompt) and then running RegFreeComDemo1.exe from outside of the Visual Studio IDE. You should see that the application now fails when the button is clicked. If you unregistered your COM component, you should reregister it before moving on, or you will not be able to rebuild your application within Visual Studio 2005.The Limitations of Reg-Free COM

Reg-Free COM provides some pretty clear advantages over traditional deployment techniques. There are, however, a few limitations and caveats that should also be pointed out before you get started.

Of course, the biggest limitation of Reg-Free COM is that it only works on the Windows XP operating system or higher. It also requires changes to the way in which components are loaded in the core operating system. Unfortunately, there is no down-level support layer for Reg-Free COM.

Not every component is a suitable candidate for use under Reg-Free COM. A component is not considered suitable if any of the following are true:

  • The component is an out-of-process (ActiveX EXE) server. Only DLLs are supported.
  • The component is a system component or part of the operating system, such as XML, Data Access, Internet Explorer, or DirectX® components. For example, you should not attempt to isolate components such as Microsoft XML (MSXml.dll) or Microsoft Internet Controls (SHDocVw.dll). These components are either part of the operating system, or can be installed with a separate redistributable package.
  • The component is part of an application, such as Microsoft Office. For example, you should not attempt to isolate components such as the Microsoft Word Object Model or Microsoft Excel Object Model. These two components are part of Office and can only be used on a machine that has the full Office product installed on it.
  • The component is intended for use as an add-in or a snap-in, such as an Office add-in or a control in a Web browser. Such components typically require some kind of registration scheme defined by the hosting environment that is beyond the scope of the manifest itself. The other problem is an arbitrary application may not be designed to recognize isolated components, as it probably doesn't have a way to reference your component through a manifest.
  • The component manages a shared physical or virtual system resource. For example, it could manage some kind of data connection shared between multiple applications or a device driver for a print spooler.

Data applications generally require a separate Data Access redistributable to be installed before they can be run. You should not attempt to isolate components such as the Microsoft ADO Data Control, Microsoft RemoteData Control, Microsoft DAO Object Library, Microsoft OLE DB, or Microsoft Data Access Components (MDAC). Visual Studio 2005 simplifies the process of deploying data applications with the appropriate redistributables using the new Visual Studio 2005 Bootstrapper feature. If you are using MDAC or SQL Server Express, you can simply check the corresponding item in the Prerequisites dialog on the Publish page in project properties.

In some cases it may be possible for the developer of the component to redesign it for suitable use under Reg-Free COM. If you have components that simply are not suited for use with Reg-Free COM, you can still build and publish ClickOnce applications that depend on them through the standard registration scheme using the Bootstrapper. For more information, see Sean Draine's article on the Visual Studio 2005 Bootstrapper in the October 2004 issue of MSDN Magazine.

As mentioned in the main body of this article, the isolation feature works by examining the component's type library and registration. A build warning will result if any nonstandard component registration is encountered while isolating a COM component. The manifest schema is a closed model that only supports standard COM registration semantics. Such warnings indicate that the component may not be suitable for use under Reg-Free COM.

A COM component can only be isolated once per application. For example, you can't isolate the same COM component from two different ClassLibrary projects that are part of the same application. Doing so will result in a build warning, and the application will fail to load at run time. In order to avoid this problem, Microsoft recommended that you encapsulate COM components in a single class library.

There are several scenarios in which COM registration is required on the developer's machine, even though the deployment of the application does not require registration. The Isolated feature always requires that the COM component be registered on the developer's machine in order to auto-generate the manifest during the build. There are no registration-capturing capabilities that invoke the self-registration during build. Also, any classes not explicitly defined in the type library will not be reflected in the manifest. When using a COM component with a preexisting manifest, such as a native reference, the component may not need to be registered at development time. However, registration is required if the component is an ActiveX control and if you'd like to work with it in the Toolbox and the Windows Forms designer.

Each COM component used by your application is represented in your project as a COM Reference. These references are visible under the References node in the Solution Explorer window. If this node is not visible, click the Project menu and select the Show All Files command.

Now let's isolate this COM component. Expand the References node in the Solution Explorer window. Select the VB6Hello item under the References node and set the Isolated property to True in the Properties window, as shown in Figure 4. If the Properties Windows is not visible, press F4.

Figure 4 Isolation
Figure 4 Isolation

When you press F5, the application works as expected, although it is now running under Reg-Free COM. In order to prove this, try unregistering the VB6Hello.dll component and running RegFreeComDemo1.exe outside of the Visual Studio IDE as before. This time when the button is clicked, it still works! If you temporarily rename the application manifest, it will again fail.

Let's take another look at the set of five files that were produced by the build: Interop.VB6Hello.dll, RegFreeComDemo1.exe, RegFreeComDemo1.exe. con- fig, RegFreeComDemo1.exe.manifest, and VB6Hello.dll. The last two files in this list are new after isolating the COM component. VB6Hello.dll is now copied to the application folder, as is an application manifest file named RegFreeComDemo1.exe.manifest. It is this manifest file that makes Reg-Free COM work. If you XCOPY this set of files to another machine running Windows XP with just the .NET Framework 2.0 installed, the application will run without any registration. Since Windows XP includes the Visual Basic 6.0 runtime libraries, you don't have to worry about having to install MSVBVM60.dll, OLEAUT32.dll, and so on.

The application manifest is actually an XML document, so you can examine it using any text editor. The contents of this file are shown in the code in Figure 5. The hash and GUID identifiers may be different depending on whether you used the exact same files I did. Note the <file> element with <comClass> and <typelib> subelements. As you can see, the class and type library GUIDs are declared in the manifest. This is the same information that traditionally appears in the registry.

Figure 5 Reg-Free COM Application Manifest (Close Figure 5)
<?xml version="1.0" encoding="utf-8"?>
<assembly xsi:schemaLocation="urn:schemas-microsoft-com:asm.v1
assembly.adaptive.xsd" manifestVersion="1.0"
xmlns:asmv1="urn:schemas-microsoft-com:asm.v1"
xmlns:asmv2="urn:schemas-microsoft-com:asm.v2"
xmlns:dsig="http://www.w3.org/2000/09/xmldsig#"
xmlns="urn:schemas-microsoft-com:asm.v1"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <assemblyIdentity name="RegFreeComDemo1.exe"
   version="1.0.0.0" type="win32" />
  <file name="VB6Hello.dll" asmv2:size="20480">
    <hash xmlns="urn:schemas-microsoft-com:asm.v2"> •••  </hash>
    <typelib tlbid="{ea995c49-e5d0-4f1f-8489-31239fc9d9d0}" version="2.0"
      helpdir="" resourceid="0" flags="HASDISKIMAGE" />
    <comClass clsid="{97b5534f-3b96-40a4-88b8-19a3bf4eeb2e}"
      threadingModel="Apartment" progid="VB6Hello.Class1"
      tlbid="{ea995c49-e5d0-4f1f-8489-31239fc9d9d0}" />
  </file>
</assembly>

If you've used ClickOnce before, you may have noticed that the ClickOnce application manifest is stored in the same file as the one I've discussed. It turns out that ClickOnce and RegFree COM definitions can coexist in the same file. Let's see what this same application looks like when published using ClickOnce. In Visual Studio 2005, right-click the project in the Solution Explorer and select the Publish command to publish the project using the Publish Wizard. The contents of the ClickOnce version of the application manifest are available in the download for this article.

The application manifest includes a superset of the definitions from the simple application manifest. It contains the same <file> element with <comClass> and <typelib> subelements, but it also includes a definition for all the files in the entire application. In addition, ClickOnce requires a hash for all application files and assemblies so that it can guarantee application integrity. Another key difference is that all ClickOnce applications must be signed, whereas a manifest intended only for XCOPY deployment does not need to be signed. The application security is defined as full trust. Any applications that directly reference native code (including any COM components) are full trust by definition. Visual Studio 2005 will emit a build warning if you are using partial trust with one or more COM references.

Lastly, you should be aware that the minimum required OS version is automatically set to Windows XP when using Reg-Free COM, so end-users will be blocked from installing the application with a descriptive error message telling them that they need to install a newer version of Windows in order to use the application.

Seeing an application manifest that contains both ClickOnce and Reg-Free COM definitions demonstrates an interesting point about the structure of ClickOnce application manifests. It shows that they are an extension of the existing manifest format that was introduced in the operating system starting with Windows XP.

This example demonstrated how to call into a simple component using Reg-Free COM, though this could just as easily have been a more complex component such as a Business Object. In the next example, you'll see how ActiveX controls work just as well under Reg-Free COM.

A More Complex Example

In this example we'll use a pre-built component called "VB6ControlTest" supplied with the sample code for this article. This is a Visual Basic 6.0 UserControl that uses several other ActiveX controls that come with the Visual Basic 6.0 development environment. Copy VB6ControlTest.ocx from the sample code to your machine now and register it.

Start Visual Studio 2005 and create another new Visual Basic WindowsApplication project named "RegFreeComDemo2". Next you'll add the VB6ControlTest component to your project, but since this is an ActiveX control, you need to add it through the Toolbox. Click on the View menu and select Toolbox. Right-click the General tab and select the Choose Items command. From the Choose Toolbox Items dialog, click the COM tab and browse to VB6ControlTest.ocx. Drag the VB6ControlTest.UserControl1 control to your form and resize it so that the entire tab control is visible. When you've completed all of this, you should have something that looks like Figure 6.

Figure 6 RegFreeComDemo2 Program
Figure 6 RegFreeComDemo2 Program

Now let's isolate all of the ActiveX controls. Expand the References node in the Solution Explorer window. Set the Isolated property to True for both VB6ControlTest and AxVB6ControlTest items. We've isolated the UserControl, but we'd also like to isolate the controls referenced by the UserControl. To achieve this, you simply add a reference to these additional controls from the .NET-based application and isolate them from there. Using the Add Reference command, go ahead and add the following items from the COM tab, and then isolate them:

  • Microsoft Common Dialog Control 6.0 (SP6)
  • Microsoft Rich Textbox Control 6.0 (SP4)
  • Microsoft Tabbed Dialog Control 6.0 (SP4)
  • Microsoft Windows Common Controls 6.0 (SP6)
  • Microsoft Windows Common Controls-2 6.0 (SP3)

Be careful which controls you choose to isolate. As a guideline, any controls that are purely UI widgets are likely to work just fine under Reg-Free COM, but you should always fully test them with your application to be sure. See the sidebar for more information.

Publish the project using the Publish Wizard. If you have another machine with just the .NET Framework 2.0 installed, go there now and install the ClickOnce application. There you have it—a .NET application with several ActiveX controls deployed smoothly using ClickOnce. A limited user could install it with no fuss, and there's no DLL Hell to worry about.

Native Assemblies

The Isolated property makes it really easy to isolate and deploy existing COM components that don't already have a manifest. However, if a component is supplied with a manifest from the vendor or component owner, you can reference the native assembly directly. In fact, you should always prefer using the manifest supplied by the author of the component wherever possible over the Isolated flag.

Visual Studio 2005 supports references to native assemblies from Visual Basic .NET or C#. A native reference is created when a reference's File Type property is set to Native in the Properties window. To add a native reference, select Project | Add Reference, then click the Browse tab and navigate to the location of the manifest. Some components place the manifest inside the DLL. In this case, you can simply choose the DLL itself and Visual Studio will add it as a native reference if it detects that the component contains an embedded manifest.

Incidentally, native assemblies can define additional files associated with the component, and may refer to other dependent native assemblies. It is possible, then, to encapsulate an arbitrarily complex dependency graph of native assemblies and files in a custom authored manifest. Visual Studio 2005 will automatically include any such files or dependent native assemblies expressed in the manifest if they are in the same folder as the referenced component.

The format of a manifest is plain XML. Therefore, if you are a component owner and you wish to make your component into a native assembly, you can author a manifest manually. You can use the MT tool in the Platform SDK to perform tasks such as computing hash codes, manifest embedding, and generation of Reg-Free COM entries from an ATL RGS script.

You can, of course, use Reg-Free COM and native references from within a Visual Basic .NET or C# class library. Interestingly, the resulting component is made up of both a managed assembly (the .dll) and a native assembly (the .manifest). When referencing the component from another project, you need two references to consume the component properly. When using a project-to-project reference (for example, choosing Add Reference and selecting the component as a project in the same solution), both assemblies are referenced automatically. Otherwise, you must choose Add Reference twice. The first time choose Add Reference and browse to the DLL; then choose Add Reference again and browse to manifest.

Conclusion

Reg-Free COM is a great way to overcome the headaches that typically go along with deploying applications that use COM components. It works for ActiveX controls, but it can also work with nonvisual business objects. It brings native and COM components to level of parity with managed components in terms of both application isolation and ease of deployment. The only significant drawback is the higher minimum-platform requirement of Windows XP. But as more and more desktops are migrated to newer platforms, the benefits offered by these new capabilities become increasingly more compelling.

Dave Templin is a developer on the Visual Basic team at Microsoft and is the lead for all of the great new ClickOnce support in Visual Studio 2005. Dave can be reached online at blogs.msdn.com/dtemp.


Flex Your Data: Teach Old Data New Tricks with the Fully Loaded Advances in ADO.NET 2.0

New information has been added to this article since publication.
Refer to the Editor's Update below.

Flex Your Data

Teach Old Data New Tricks with the Fully Loaded Advances in ADO.NET 2.0

Julia Lerman
This article is based on a prerelease version of the .NET Framework 2.0. All information herein is subject to change.
This article discusses:
  • ADO.NET 2.0 classes that will simplify coding
  • Faster reading and writing of data
  • Interacting with SQL Server 2005
  • Threading, reusing connections, and other optimizations
This article uses the following technologies:
.NET Framework, ADO.NET, SQL Server
Contents
The DataTable as a First-Class Citizen
IDataReader
DataTableReader
Batch Updates
Binary Remoted DataSets
RowState
The ProviderFactory
SQL Server Updates
Threading, Notifications, and Bulk Copy

T
he Microsoft® .NET Framework 2.0 delivers an updated ADO.NET that will streamline your data access development efforts. The ADO.NET team has worked with both the System.Xml team and the SQL Server™ team to provide a consistently rich developer experience, crossing technology boundaries from ADO.NET to XML and back. This will all be apparent as you dig into ADO.NET 2.0. There are improvements on so many fronts that you will certainly be pleased.

Rather than attempt to list every new and updated feature of ADO.NET 2.0, I will discuss in detail some of the more interesting improvements and focus on performance and flexibility.

The DataTable as a First-Class Citizen

In the .NET Framework 1.x, DataTables did not support XML serialization on their own. Specifically, the DataTable did not support the ReadXml and WriteXml methods native to the DataSet. In addition to leveraging these methods to persist data, the ability to send a DataSet to a Web service (or return one from a Web service) is quite convenient; however, you could not do this with a DataTable. And because Microsoft chose not to document and publicly support the IXmlSerializable interface in the .NET Framework 1.x, it's not clear in the documentation that DataSet inherits that interface. With the .NET Framework 2.0, not only are you encouraged to use the IXmlSerializable interface, but System.DataTable class now implements IXmlSerializable as well. This will make life simpler because you will not be forced to use a DataSet when all you need is a table. ReadXml, WriteXml, and GetSchema, previously unique to the DataSet, are now part of the DataTable.

Because of this, you can now pass DataTables to and from .NET-based Web services. Although these changes will not make earth-shattering differences in your code, working with DataTables will be much easier. It will also alleviate some confusion regarding when to use a DataTable versus a DataSet. Interestingly, if you were to compare the resulting XML from a persisted DataTable to the XML of a persisted DataSet that contains only that one DataTable, you would see that the XML is quite similar, as both work directly with the DataTable.

Another DataSet method that DataTable now supports is the Merge method. The DataTable.Merge method allows you to pull in another DataTable (with the same PreserveChanges and MissingSchemaAction from the .NET Framework 1.x) but you can't bring in a DataRow collection like you can with the DataSet. If a DataRow collection contains rows from separate DataTables, the DataSet.Merge method can combine the various rows and create a new DataTable when necessary. But if that same DataRow collection was used with DataTable.Merge, there would be no way to handle the rows from other DataTables. Therefore, to avoid problems, the overload is not supported by DataTable.

IDataReader

If you are familiar with the SqlDataReader, then you have likely already worked with streamed data. Classes like SqlDataReader implement the IDataReader interface, which lets you read data in a read-only, forward-only, firehose-style manner. The DataReader is used to stream data from a database. The DataAdapter.Fill method actually uses a DataReader in the background to stream the data into a DataTable or DataSet. Although the DataReader is more efficient than a DataSet, it cannot be used for binding data to controls (such as a DataGrid) because they require an object that supports IList or IListSource. [ Editor's Update - 3/15/2005: ASP.NET data-bound controls can bind to data sources that support IListSource or IEnumerable. While the IDataReader interface itself does not derive from either IListSource or IEnumerable, all of the .NET Framework classes that implement IDataReader also implement IEnumerable, including SqlDataReader. Thus, while you can't count on being able to bind an arbitrary IDataReader implementation to a control like the DataGrid, you can in the case of SqlDataReader.] In the .NET Framework 1.x, this means you have to create a DataTable or DataSet in the data access layer and then pass it back to the UI if you want to perform data binding.

In ADO.NET 2.0, Microsoft has responded to customer feedback and exposed the functionality behind Fill so that you can load data into a DataSet or DataTable without a DataAdapter using the Load method. With Load, the process of populating a DataTable or DataSet is no longer explicitly tied to a DataAdapter and therefore no longer requires a connection to the database in some scenarios. This means you can return the more efficient DataReader from your data layer and still benefit from the functionality of the DataTable and DataSet, which you can create in the layer where you need to use it. You now have better control over where to expend your resources. As always, it's important to remember that a DataReader maintains a connection to your database and you must close that connection, especially if you are moving it out of the data layer. Load also works with a new disconnected relative of the DataReader, DataTableReader, which I discuss later.

The Load method also has a LoadOption parameter which lets you choose the merging mode if there are already rows in the object you are loading into. These options are OverWriteRow, PreserveCurrentValues, and UpdateCurrentValues. The Beta 1 documentation explains these options in great detail. Microsoft has made merging data much easier with this functionality. In fact, these same options have been added to the DataSet.Merge method and the new DataTable.Merge method, which I previously discussed.

DataTableReader

The DataTableReader is a new class that inherits from another new base class, DbDataReader (a nondatabase-specific implementation of an IDataReader). DataTable and DataSet both have the CreateDataReader method (called GetDataReader in earlier releases of the Beta), which will return a DataTableReader with the same number of resultsets that is represented by the container. Deleted rows will not be pushed into the DataTableReader. You can use the DataTableReader to iterate through data, to stream data elsewhere, and to load data into a DataSet or DataTable.

DataTableReader is a great way to iterate through a DataSet or DataTable without the overhead of those larger classes. This is advantageous because if any rows are deleted from the source DataTable while you are in the middle of iterating, the record pointer will remain in its correct position. This is quite different from SqlDataReader, which puts a reference constraint on the source data, preventing deletions. Additionally, if new rows are added ahead of the DataTableReader's record pointer, they will be available as you continue to iterate through your DataTable.

If you inspect this object in the debugger you'll note that although you can't see the Tables property of a DataTableReader in code, you can see it in the debugger. The schema of the DataTable(s) are absorbed into the DataTableReader, which is how the class is able to iterate not only through the tables, but through the columns and rows. Note that DbDataReader implements IEnumerable, and thus has the functionality of a Collection, which is how you can drill into this data in the debugger.

Figure 1 shows part of the Visual Studio® QuickWatch window for a DataTableReader. Note the objects with the lock icons. The DataTableReader has knowledge of the DataTable from which it has been derived, though the DataTable's interface is not exposed.

Figure 1 DataTableReader in the Visual Studio QuickWatch Window
Figure 1 DataTableReader in the Visual Studio QuickWatch Window

Like the DataReader, however, the information is both forward-only and accessed by relative position. If there are multiple resultsets, you can move forward from one to the next with the NextResultset method. You jump into the DataTableReader with the Read method which will position the pointer on the first row of the first resultset. As with the .NET Framework 1.x SqlDataReader and related classes, data is accessed using the Item property or one of the format specific methods, such as GetDateTime.

It is important to remember that unlike the DataTableReader, database-specific DataReaders stream directly from the database and maintain an open connection. Passing a DataReader from one application layer to another is not recommended as you are dragging the connection along with you. A DataReader whose connection is not properly closed can wreak havoc on your resources. You won't need to worry about this with the DataTableReader.

Batch Updates

If you fancy yourself a hard-core developer and find the DataAdapter commands to be too much like wizards, the DataAdapter's new UpdateBatchSize property (available in the SqlDataAdapter and OracleDataAdapter classes) may change your mind.

In the .NET Framework 1.x, the need to update multiple records has forced you to choose among efficiency, performance, and security. Using the DataAdapter's built-in Update command can save significant coding, but in the background it was still performing one database call per record.

Enter the UpdateBatchSize property of the DataAdapter. When batch updating is used, the DataAdapter builds a series of parameterized commands and sends them all at once to the server during an update. This could be done manually, but it is complex and as a result is not always the best option for many developers.

There has been a big change in batch updating since the earlier betas. In Beta 1, a query similar to Figure 2 (as captured by the SQL Profiler receiving a batch of three update queries) was passed to the database. Unfortunately, the SQL Server limit of 2,100 parameters could be quickly reached with this approach. The ADO.NET team has revised the internals of the function so that there is no more concern about the number of parameters in the query. If you send a batch of more than one update query to SQL Server, you can now see in SQL Profiler that each update query is run individually.

Figure 2 Batch Update Query in Beta 1 (Close Figure 2)
exec sp_executesql N'BEGIN
exec sp_executesql @P1, @P2, @P3, @P4, @P5, @P6, @P7, @P8;
exec sp_executesql @P1, @P2, @P9, @P10, @P11, @P12, @P13, @P14;
exec sp_executesql @P1, @P2, @P15, @P16, @P17, @P18, @P19, @P20;
END
',N'@P1 ntext,@P2 ntext,@P3 smallint,@P4 int,@P5 int,@P6 money,@P7
smallint,@P8 real,@P9 smallint,@P10 int,@P11 int,@P12 money,@P13
smallint,@P14 real,@P15 smallint,@P16 int,@P17 int,@P18 money,@P19
smallint,@P20 real',
@P1=N'UPDATE [Order Details] SET [Quantity] = @p1 WHERE (([OrderID] =
@p2) AND ([ProductID] = @p3) AND ([UnitPrice] = @p4) AND ([Quantity] =
@p5) AND([Discount] = @p6))',
@P2=N'@p1 smallint,@p2 int,@p3 int,@p4 money,@p5 smallint,@p6 real',
@P3=2,@P4=10284,@P5=60,@P6=27.20,@P7=1,@P8=0.25,@P9=2,@P10=10284,@P11=67,
@Pz12=11.20,@P13=1,@P14=0.25,@P15=2,@P16=10285,@P17=1,@P18=14.40,@P19=1,@P20=0.2

You can control the UpdateBatchSize property yourself, defining the number of rows to update in one batch which can be fine-tuned for efficiency based on your resources. The greatest benefit of batch updating is that you will have fewer round-trips to the database server if you have a large number of rows to update.

Bigger, however, is not always better with UpdateBatchSize. For example, with a local database it might be better to make more round-trips to the server with smaller batch sizes, whereas connecting to a remote database might demand the inverse. There are many factors you will need to consider relating to your resources, from the speed of the client machines to the latency of your network when you are determining how large to make the batches. Tuning batch sizes based on your own variables is recommended.

The new ToTable method of the DataView lets you create a new table based on a sorted or filtered DataView of another DataTable. If the original DataTable that the DataView is based on is part of a DataSet, the DataTable returned by this method still has an association with the originating DataSet. Because of this, you cannot add this DataTable to another DataSet directly. You will get an ArgumentException stating that the DataTable already belongs to another DataSet. With the .NET Framework 1.x, you can use a copy of the DataTable instead:

myDataSet.Tables.Add(MyNewTable.Copy)

Figure 3 Visualizer
Figure 3 Visualizer

One of the overloads of ToTable accepts a Boolean that filters distinct rows, a feat which has been difficult to accomplish in the past. For example, if you have a DataTable whose rows contain author names, addresses, and contracts, it would be quite a chore to create a list of the unique states represented by those authors or a list of authors with contracts and without in each state. You would need to create a routine that iterates through the rows and compares values from one row to another. Although the DataView itself can't filter on the unique values, the ToTable method allows you to create a separate table that will filter out rows where the key values are the repeated values. Along with the Boolean, you need to pass in a parameter array of the columns that you would like to filter. Lastly, another overload of ToTable accepts an output table name as an additional parameter:

Public Function DVtoNewTable() As DataTable
    Dim ds As DataSet = GetAuthors()
    Dim dt As DataTable = ds.Tables(0)
    Dim dv As DataView = dt.DefaultView
    Return dv.ToTable("StatusbyState", True, _
        New String() {"state", "contract"})
End Function
Figure 3 shows the Visual Studio 2005 DataSet/DataTable Visualizer examining the result of this method.

Binary Remoted DataSets

One of the benefits of remoting over XML Web services is that data can be transported not only in XML format, but in binary format as well, unless it's data in a DataSet. Even though their encoding mechanism is binary, DataSets are serialized as XML before being sent through the pipe, causing a performance hit. In ADO.NET 2.0, this has been fixed. Remoted DataSets are now truly serialized as binary data. For more information, see Cutting Edge: Binary Serialization of DataSets.

RowState

RowState can sometimes be a mysterious beast, especially when you are adding and modifying rows. Consider the following states as you step through a typical procedure to create a DataRow, add it to a DataTable, and modify it:

Create New DataRow via DataTable.NewRow   Detached
Modify Data in DataRow                    Detached
Add Row via DataTable.Rows.Add            Added
Modify Row Again                          Added
Update DataTable via AcceptChanges        Unchanged
Modify Row again                          Modified
The resulting RowStates are as you would expect. But what if you receive data from a Web service or through remoting and you need to use it to update your local datastore? If you are using the DataAdapter Update method, these records will, in most cases, be ignored because they will likely be flagged as Unchanged.

Two new methods have been added to the DataRow class to deal with this: SetAdded and SetModified. These methods allow you to change the state of an Unchanged row to either Added or Modified, giving you a little more granular control over the RowState. Note that in the current version of the Beta (the November Beta 1 Refresh at the time of this writing) the IntelliSense® is not quite correct in its descriptions. SetAdded tells you that it will "reject changes to a row since AcceptChanges was last called." SetModified is actually displaying the description for the method directly above it, SetColumnError, so beware. While these two methods can be misused, they can help you solve some concurrency problems.

The ProviderFactory

There is a collection of new classes inside of the System.Data.Common namespace that make it easier to handle a variety of databases. Previously, if you wanted to accommodate different databases, you might have written separate blocks of code for each database type. In the .NET Framework 1.x, SqlConnection and OledbConnection have no common data-related ancestor in their inheritance. The same is true for SqlCommand and OledbCommand. The best option for writing database-independent code was to use interfaces.

The new ProviderFactory classes let you create provider-independent connections, commands, adapters, and more. You can then code against those and pass in the name of the actual namespace you want to work with at run time. Here's a quick look at how that works:

Public Sub ProviderTests(ByVal providerName As String)
    Dim provider As DbProviderFactory = _
        DbProviderFactories.GetFactory(providerName)
    Dim connection As DbConnection = provider.CreateConnection()
    Dim adapter As DbDataAdapter = provider.CreateDataAdapter()
End Sub

This is incredibly simple to create and use. In this example, provider name strings (like "System.Data.SqlClient") could be stored in a resource file, the application configuration file, or with any other mechanism you use for storing configuration data.

SQL Server Updates

Because the ADO.NET team is part of the SQL Server organization at Microsoft, it should come as no surprise that special attention has been paid to the interactions between ADO.NET and SQL Server. Much has and will be written about the fact that SQL Server 2005 is capable of hosting the CLR. The System.Data.Sql and System.Data.SqlServer namespaces contain the tools for interacting with .NET from SQL Server 2005. Note that the System.Data.SqlServer namespace from earlier releases of SQL Server 2005 betas is targeted to be removed in the Beta 3 version. Its functionality will be wrapped into the SqlClient namespace. But let's take a look at a few of the many ways in which ADO.NET 2.0 now makes use of some specific SQL Server functionality. All of these functions are available solely in the SqlClient namespace.

One of the big changes in SQL Server 2005 is the addition of Multiple Active Resultsets (MARS). You have probably heard the puns such as "Data is from Venus, Results are from MARS."

With MARS, the SQL Server team changed how a resultset is delivered from SQL Server and the ADO.NET team has taken advantage of this. The result is that you can now get multiple DataReaders from the same connection, one of the most requested features of ADO.NET. MARS is enabled by default when connecting to SQL Server 2005, but can be configured with the Boolean parameter MultipleActiveResultSets in the connection string, which you can use to deactivate MARS if you want.

It is important to remember that MARS requires separate commands to generate multiple DataReaders from one connection, though you can use the same command to execute Updates and Inserts, and so on. Also remember that MARS comes with resource overhead and you should consider comparing the performance of reused SqlConnection objects (which are simpler to code) to the performance that connection pooling already achieves. MARS will not always improve performance. But to be able to wrap multiple commands into one transaction process, a performance hit may be an acceptable price to pay.

When using transactions with MARS, creating multiple transactions within one connection is not allowed and will throw a System.InvalidOperationException. Though database transactions (such as SqlTransaction and OledbTransaction) are still available, you should definitely explore the powerful TransactionScope class in the new namespace, System.Transaction. TransactionScope can be leveraged for local transactions as well as distributed transactions. For more information on the System.Transactions namespace, see John Papa's February 2005 Data Points column.

This type of connection reuse was not a problem for DataAdapters prior to ADO.NET 2.0. When you fill a DataAdapter, the data is basically pushed all the way out of the pipe so that it is free and clear for another call.

Threading, Notifications, and Bulk Copy

Writing multithreaded applications can be complex but the benefits can be huge. One of the innovative new features in ADO.NET 2.0 is true asynchronous processing capabilities in the SqlCommand class. The SqlCommand's asynch methods are a great addition for middle-tier processing and are handy when, for example, you have to execute many queries in a row (given that they are not dependent on each other). These queries can now run simultaneously. For an in-depth look at these features, see the MSDN online article "Asynchronous Command Execution in ADO.NET 2.0" by Pablo Castro. This is a powerful addition to ADO.NET and Pablo's article outlines its use.

At this point, while it may seem that combining asynchronous commands with MARS would create some extremely impressive performance scenarios, the performance gain would, as a matter of fact, be quite minimal and probably not worth the complexity.

Two more features specific to SQL Server in ADO.NET 2.0 are Data.Sql.SqlNotification and Data.SqlClient.SqlDependency. SQL Server 2005 can now push change notifications into a queue by hooking the SqlNotification or SqlDependency object to your SqlCommand. Then you can either subscribe to the queue through a special RECEIVE query or just poll the queue yourself. If you use the RECEIVE query, you can create an event handler to capture the messages from SQL Server when changes have been made.

SqlDependency is a higher-level abstraction of SqlNotification that uses the default behaviors of SqlNotification and is a little easier to code against. For implementation details, check the Beta 1 docs or some of the other resources found in this article.

There are some important rules for using notification. First, when referencing a table in your query, you must use a two-part name for the table. If the table has a single name, such as the authors table in the pubs database, then you would append the owner name (such as dbo.authors). In a case where the table already has a two-part name, such as person.contact in AdventureWorks, person.contact is sufficient. Second, you must specify the columns that are being returned rather than querying with "*" as in "select * from authors". Third, notification is designed to scale for large numbers of common queries in which you have only a few parameters changing and it would significantly impair the performance of your application if you were to use numerous ad hoc queries.

There are more rules listed in SQL Books Online under Query Notifications. Additionally, there are some very specific rules to follow (granting permissions, setting datatable properties, and so on) in SQL Server 2005. (See "Query Notifications in ADO.NET 2.0".)

It is important not to get the SqlDependency and SqlNotification classes confused with the System.Web.Caching.SqlCacheDependency class. This ASP.NET class can not only use SQL Server 2005 query notification, but can also work with SQL Server 2000 and SQL Server 7.0. The ADO.NET classes will not know how to work with the previous versions of SQL Server, however. For more information on caching in ASP.NET 2.0, take a look at "Caching Improvements in ASP.NET Whidbey," by G. Andrew Duthie and "Improved Caching in ASP.NET 2.0," by Stephen Walther. You can also find a thorough examination of notification and SQL Server 2005 in A First Look at ADO.NET and System.Xml v.2.0, by Alex Homer, Dave Sussman, Mark Fussell (Addison Wesley, 2004).

If you have ever considered using ADO.NET to get data from one source into SQL Server, you have probably been shocked by how much longer it takes than a simple Data Transformation Services (DTS) transfer. That's because with ADO.NET, you end up writing one record at a time to the database in the background. The new SqlBulkCopy class enables bulk inserts of data into SQL Server from within your .NET-based application. Even leveraging the new batch updates will not come close to the SqlBulkCopy class in ADO.NET 2.0, which is nearly as fast as DTS.

SqlBulkCopy lets you push data from a DataTable, a collection of DataRows, or a class that implements IDataReader with just a few lines, as shown here:

Dim connDest As New SqlConnection(sqllocal)
connDest.Open()
Dim dr As SqlDataReader = MyRoutinetoPopulateaDataReader
Dim oBCP As New SqlBulkCopy(connDest)
oBCP.DestinationTableName = "MyDestinationTable"
oBCP.WriteToServer(dr)
dr.Close()
oBCP.Close()
connSrc.Close()
connDest.Close()
In my own tests when transferring a SQL Server 2000 table of over a hundred thousand records into a SQL Server 2005 database, SqlBulkCopy performance was almost identical to DTS, whereas using batch updates was orders of magnitude slower. So now it is quite conceivable to have a programmatic solution to moving large amounts of data without having to rely on external processes to make it happen.

Every application that you write can benefit from the advances in ADO.NET 2.0. Here I've mentioned just some of the new and enhanced features. You will find that the MSDN Documentation that is supplied with Beta 1 (which can also be accessed at msdn2.microsoft.com/library) has numerous articles of interest as well.

Julia Lerman is a consultant who has been developing database applications and consulting to businesses since 1984. She is also a conference speaker, a Microsoft .NET MVP, an ASPInsider, leader of the Vermont .NET User Group, and an INETA board member. Julia lives in Vermont and blogs at thedatafarm.com/blog.


Team Up!: Get All Your Devs in a Row with Visual Studio 2005 Team System

[ Editor's Update - 1/20/2006: This article refers to a beta version of Visual Studio 2005. An updated version of the article, reflecting features found in the final release of Visual Studio 2005, can be found at Team Up: Get All Your Devs In A Row With Visual Studio 2005 Team System.]
Team Up!

Get All Your Devs in a Row with Visual Studio 2005 Team System

Chris Menegay
This article is based on the Visual Studio 2005 December Community Technology Preview. All information contained herein is subject to change.
This article discusses:
  • Software development processes
  • Team System's support for the entire development team
  • Work item tracking, unit and load testing, static code analysis, and source control
This article uses the following technologies:
Visual Studio 2005
Contents
Project Methodology
MSF Agile
Creating a Team Project
Project Plans and Work Items
Project Documentation
Design Your Application
Team Foundation Version Control
Writing Better Code
Managed Code Analysis
Unit Tests
Load Testing
Testers, Manual Testers, and Bug Tracking
Team Build
The Team Site and Reporting
Conclusion

S
oftware development is generally recognized as a difficult process. Numerous studies have been conducted and volumes of books have been written on how to improve the process of developing applications to yield better and more consistent results. The difficulty has never been in coming up with new and better ideas to develop software. Rather, the difficulty is in taking those ideas and implementing them in meaningful ways in the real world. With Visual Studio® 2005 Team System, Microsoft is taking an important step in helping development teams build robust software systems.

Team System extends the power of Visual Studio 2005 with a new source control system. Team System also includes unit testing and code analysis tools to help developers. However, Microsoft has broadened its focus from just developers, and now includes tools to support the entire development team. Team System includes tools to help project managers, architects, developers, testers, and even development managers. Team System contains a new work item tracking system to manage development tasks, as well as the implementation of process, and a Web portal to allow a level of transparency to the development process.

In this article, I'll give an overview of Team System using the Visual Studio 2005 December Community Technology Preview (CTP). I'll show you how to set up a development project and explore all the steps of the development process through to testing. Installing the version of Team System that is included with the December CTP is a slightly more difficult process than the one that will be required when the product ships in its final form. The CTP supports a very specific environment that requires multiple machines, or multiple virtual machines, to install. The installation guide that comes on the DVD or in the download is accurate, but you may need some additional help to get things working properly. You can also refer to my notes, as well as those of a few others, posted at the following blogs: weblogs.asp.net/cmenegay, blogs.msdn.com/robcaron, and blogs.msdn.com/askburton.

Project Methodology

In the past, Visual Studio was solely a tool for developers. Because of this, it provided little value for the other phases of a development project, such as requirements gathering, design, and testing. However, Team System was designed to support more than just the developers. It is meant to support the whole development lifecycle, and the people involved in that lifecycle.

One of the best things about Team System is that it is built with an understanding of process. Beyond the idea that having some form of process might be a good thing, Microsoft made very few assumptions about what your process would be, and has therefore built in a great amount of flexibility. Team System uses what Microsoft refers to as "methodology templates" to define processes. You can develop your own methodology, use one of the methodologies that will ship with Team System, or even acquire a third-party methodology template.

In the past, many development groups have not implemented formal processes because the investment in both time and money to adopt and enforce a formal process was too great. With Team System, the process will be a part of the tools that teams use every day, making it much more approachable for a broader set of development teams.

Microsoft currently has an established development process called the Microsoft [ Editor's Update - 3/24/2005: MSF Formal has been renamed to MSF for CMMI Process Improvement, and MSF Agile has been renamed to MSF for Agile Software Development.] Both of these methodologies are implemented as templates and integrated into Visual Studio. The December 2004 CTP only included support for MSF Agile. It is quite likely that organizations that have a formal process will move their existing processes into Team System, while organizations that have not previously implemented a formal methodology will use either MSF Formal or MSF Agile.

Not all members of the development process, however, will have, or even desire, Visual Studio. To meet the needs of non-developers, both a Project Portal and a Team Foundation Client are also slated in order to access many of the features of Team System that would be desired by a broader audience. Team Foundation Client will essentially be a version of Visual Studio 2005 with all the development features removed while retaining all the Team System features. This means that most of the processes covered in this article should apply to Team Foundation Client as well.

MSF Agile

MSF Formal is a process designed to help achieve CMMI Level 3 Compliance, while MSF Agile is meant to be more flexible and is iterative by design. No single process will be an ideal fit for all projects, so it is possible that organizations will adopt a process based on the needs of a particular development effort. An early view of MSF Agile is available for download at MSF for Agile Software Development.

MSF Agile supports the following five roles: Architect, Business Analyst, Developer, Project Manager, and Tester. As you read through the upcoming paragraphs about process and team development, keep in mind these five distinct roles, as well as business users/sponsors and IT management. Team System has features that will appeal to all of these unique types of individuals.

Creating a Team Project

Now that I've laid some groundwork for the overall purpose behind Team System, I'd like start walking through the process of setting up what is called a portfolio project in the Beta 1 Refresh (it will be called a team project in subsequent releases). After loading up the Visual Studio 2005 December CTP, you first need to connect to a Team Foundation Server (TFS), which should be running on some other system or virtual machine. TFS is a server platform that provides many of the team features of Team System. Some of the features that TFS provides are the new source control system, work item tracking and the team portal site.

To connect Visual Studio to a TFS machine, under the Tools menu is an option to "Connect to Team Foundation Server." The installation notes will assist you in getting connected to your TFS machine. Once connected, you will see the Team Explorer window. Team Explorer is your view into TFS, much like Server Explorer provides insight into a SQL Server database. Team Explorer is a great tool. You will want to become familiar with it as soon as possible, as you will be using it regularly.

Once you are connected to TFS, you can create your team project. Team projects are one of the key benefits of TFS. A team project is a single place to access and work with all of your project-related artifacts, including design documents, work items, and project plans. You will create one team project for each development project on which you are working. To do this from the Team Explorer, use the "New Team Project" toolbar button. Alternately, from Team Explorer you can right-click your TFS Server and create the team project from there. You also have the option of adding an existing team project to your Team Explorer window. When setting up the team project, you must choose which methodology template you would like to use, as shown in Figure 1. In addition, you have the option of setting up a new source control folder, or branching an existing code base. I will use MSF Agile for my examples.

Figure 1 Creating a New Team Project
Figure 1 Creating a New Team Project

When you set up a team project several things happen. One key item is the creation of a Windows SharePoint® Services (WSS) team site. TFS integrates with WSS to provide many of its document management capabilities. WSS also has collaboration features that can be leveraged by TFS. If you open your browser and go to http://tfsserver/sites/project, you will see the home page for the team site created by TFS. This is a WSS site, which is accessible by anyone with the proper permissions. It is a great way for business users and management to gain access and insight into the status of a project.

Once you have a team project, you may want to configure your project before going any further. You do this via the Team Explorer by right-clicking your project and selecting Team Project Settings | Classifications. The Settings menu option allows you to set security permissions and source control policy as well as set up your project structure. If you select MSF Agile, you currently have three iterations added for your project by default. You can rename these, remove them, or add more. For example, if you want to have six phases of incremental rollouts, you can set these up as iterations. Later on, when you are creating work items, they can be associated with a particular iteration in your overall project. Remember that the methodology templates are under your control, so don't feel confined by the process.

Project Plans and Work Items

One of the key benefits that Team System provides from a project management perspective is a feature called work item tracking. Features such as unit testing, better source control, and code analysis are what get developers excited. Project managers, business stakeholders, and IT managers, on the other hand, are going to be especially excited by the work item tracking feature which does exactly what the name implies. Each and every task that needs to take place in the software development process can be treated as a work item. These include documentation tasks, design tasks, development tasks, bugs, or requirements. At this point you may be thinking that you already manage your work items as tasks in Microsoft Project, or even Excel. But the truth of the matter is that most people are really only creating tasks in these tools, and not actually tracking or managing them.

Taking into consideration your current tracking process, think about your answers to the following questions: Are the design documents complete? How many tasks are assigned to a particular developer? What tasks have been completed? Which of the tasks are furthest behind? What is the impact on on the schedule when tasks are not completed on time? Granted, many organizations already have the ability to answer these questions. However, in order to do so, a good deal of communication must take place between the project manager and the rest of the team, including developers, business analysts, and testers. The project manager must talk to all of these different people to accurately assess project status. Keeping track of the status of tasks is usually accomplished with project status meetings, individual conversations, or via e-mail. One of the goals of Team System is to streamline this process of data collection, tracking, and reporting.

Once you select a methodology template to use (as you'll recall, I chose MSF Agile), several tasks (work items) are automatically created on your behalf. This is done because in the MSF Agile process certain tasks need to be accomplished. Figure 2 shows an example of the work items that were created when I set up my team project using MSF Agile.

Figure 2 Work Items in MSF Agile
Figure 2 Work Items in MSF Agile

Work items are very similar to tasks in Microsoft Project in that they can be assigned to people. They have a status as well as duration. As people complete their work items, they can change the status of those work items and have that status reflected on the WSS team site. The WSS site utilizes SQL Server™ Reporting Services to show various reports based on the data stored in SQL Server by TFS. You will have the ability to create your own reports as well that are then integrated into SharePoint using custom Web Parts. Note, however, that these reports are not available as part of the December 2004 CTP.

Project Documentation

Based on the MSF Agile approach, one of the first steps in the process is to start creating some documentation. Depending on where you are in your process, this documentation could be a business case, a scenario, or even a project plan. The documentation you create will depend on your methodology. To create these documents in the context of Team System, you use the Team Explorer (alternately, you could do this through the team portal site). If you drill down into the Documents node, you will be able to right-click and select Add a Document. When you specify which methodology to use for your project, Team System configures the appropriate templates that will be used for the various documents of your project. When you then select Add a Document, you will be prompted to choose the type of document you want to add by selecting the appropriate template. Templates include options for Design, Lifecycle, Plans, Project Management, and Requirements Scenarios and Features. Once you've selected your template, you will edit your document in the appropriate tool (Microsoft Word or Excel, for example).

The documents that are created are stored in the WSS site that hosts your project. This allows people within your organization to access the documentation simply by navigating their browser to the appropriate site. Documents added directly to the portal site will be shown in Team Explorer. To see newly added documents in Team Explorer, right-click the Document node and select Refresh.

At some point, the project manager will want to create an actual project plan. With Team System, there is some real fun to be had doing this, assuming of course that you find creating project plans and tracking tasks to be loads of fun. At the very least, Team System makes the management of your project much more simple, which to me translates to exciting!

The process of creating a project plan is fairly straightforward in Team System. You do this from Team Explorer by navigating to Documents | Project Management and right-clicking. Here, the context menu contains an option to Create Microsoft Project Plan. Team System also supports tracking your project tasks in Excel spreadsheets, and you would create a spreadsheet for your project tasks in a manner similar to this.

A real benefit of Team System is its relationship with Project or Excel. When you install Visual Studio 2005 December CTP on a computer that has Microsoft Project or Excel installed, you will get some additional features added to these applications. Specifically, you will have a Work Items menu as well as a new toolbar that allows you to interact with a Team Foundation Server. The key feature is that you can synchronize your project plan tasks with work items within Team System.

The process begins with creating a project plan and importing the original work items that were created as part of the team project creation. You then add and assign additional tasks, and synchronize the project plan tasks with Team System's work item database. As developer's work on tasks and check code into source control, they will update the status of the work items from within Visual Studio. Work item status data will be available on the team portal site via work item reports. Project managers can then synchronize their project plans with the work item database to keep their information current and acquire the latest status.

Figure 3 shows Microsoft Project with the work items from Team System imported in and used as Project tasks. You will notice a new toolbar for interacting with a Team Foundation Server. This integration is available in Excel as well as in Project. This was added based on customer feedback, to accommodate users who choose to manage their project tasks in Excel rather than in Project.

Figure 3 Importing Work Items into Microsoft Project (Close Figure 3)

Once the project plan is ready, it should be synchronized with Team System so that the project team members will know what their assignments are. If you are a developer, the elegance of this is that you receive your work item list without ever leaving Visual Studio. You simply navigate to Work Items in Team Explorer and open up My Work Items or My Active Work Items. This streamlined approach puts little burden on the developer as well as requiring fewer meetings between project managers and developers.

Design Your Application

From the developer perspective, once work items for development tasks have been assigned, you are ready to begin designing your application and writing code. Hopefully, one or more of the tasks on the project plan actually accounts for the design phase of your application. Notice that I mention designing before writing code. Team System comes with some great design tools that you should consider using. I won't go into the designers here; instead for more information I will refer you to Brian A. Randell and Rockford Lhotka's article, "Bridge the Gap Between Development and Operations with Whitehorse," in the July 2004 issue of MSDN®Magazine.

Team Foundation Version Control

Before creating any designs or code, you will want to set up source control using Team Foundation Version Control. Formerly known by its code name "Hatteras," Team Foundation Version Control is a much more robust platform than Microsoft Visual SourceSafe®. All of the content you store in this new source control system is automatically backed up in a SQL Server 2005 database for easy administration. The design goals for this new source control system include high scalability and performance, two features that are lacking in Visual SourceSafe. The new source control system will include more robust branching and merging capabilities. Because of this, multi-checkout will be turned on by default.

A new feature called "shelving" is also available. This feature allows you to take checked-out code that you are working on and check it into source control, but not into the primary branch, allowing developers to properly back up their work without forcing incomplete changes onto other developers on their team.

Visual SourceSafe will still be available and updated to work with Visual Studio 2005. Visual SourceSafe may prove useful on smaller projects with smaller teams that don't require the power of Team Foundation Server and that don't want to put forth the extra effort and expense of installing it. To select which source control engine you want to use, simply open Visual Studio, go to Tools | Options and select Source Control options.

Once you are configured to use Team Foundation Version Control, any time you create a new project in Visual Studio, you will have the option to add the project to source control. At that time, you will select the Team Foundation Server you wish to use. As a developer, when you are working on tasks you can mark work items to associate the work items with code that you are checking into source control. This association is then stored in the database and can be used later to create build reports. In addition, the new source control system also allows the creation of check-in policies. The purpose of a check-in policy is to restrict what code is allowed to be checked into the source control repository.

Writing Better Code

Visual Studio 2005 Team System integrates within the IDE several tools that will help you write better code. These include a profiler to analyze potential performance problems in your application, code analysis tools for both managed and unmanaged code, and unit testing tools with code coverage analysis. The code analysis tools are based on technology that has been in use at Microsoft for some time, but never so nicely integrated with Visual Studio. Native code is analyzed using PREfast, while managed code is analyzed using FxCop. The unit testing tools are very similar to what is available in tools such as NUnit, but there is greater ease of use and the time to create tests has decreased. This makes unit testing much more approachable. The unit tests also integrate nicely with code coverage, which is essentially a reporting mechanism to let you know how much of your code was actually exercised by your unit tests.

Managed Code Analysis

If you are familiar with FxCop, you'll be comfortable with the managed code analysis capabilities of Team System. The integration with the IDE is excellent. To enable Code Analysis, simply go to the properties for your project, then go to the Code Analysis tab. From there you can enable Code Analysis as well as select which rules you wish to run. Most organizations will likely disable a number of the provided rules, as the rules that come with Code Analysis are very strict. Indeed, it is a humbling experience to run FxCop against one of your existing applications to see how noncompliant you are with its rules.

The great part about Code Analysis being integrated with the IDE is that, once turned on, it will automatically run when your application is compiled and give you warnings or errors as part of the build process. You do have the ability to turn Code Analysis on for Release builds and off for Debug builds, which is a nice feature. Be aware that running Code Analysis can take a bit of time on larger projects. The other nice feature that's available is the option of running Code Analysis as a check-in policy for source control. This allows you to prevent developers from checking code that has not passed the FxCop validation rules into the source control repository. Check-in policy comes with an override mechanism to allow code to be checked in, but the override can be logged.

Unit Tests

The general concept behind unit testing is to run isolated pieces of code in your application and test to see if they yield the desired result. This usually equates to calling a method with a variety of controlled inputs, and testing the returned data. This can easily be done without any kind of tool support, but to make it productive, some assistance is needed. Some traditional problems with unit testing are the creation of the tests, the organization of the tests, and having a mechanism of easily reporting the success or failure of those tests. Team System addresses all of these problems.

Team System provides the capability to automatically create unit tests for your methods. [ Editor's Update - 3/24/2005: This capability is only available for managed code.] Attributes are used to differentiate unit test code from regular code. Within the IDE are tools to both organize your tests into test lists as well as to execute tests based on those test lists. When you execute your tests, the results are shown in Visual Studio in a Test Results window.

To implement unit tests in your project, simply right-click in your code and select Create Tests, as shown in Figure 4. You will be prompted to select which methods on which objects you want to create tests. You can create multiple tests at one time by selecting methods from multiple different classes. The tests that are automatically generated will contain code to instantiate the object that contains the method to be tested, declare variables for method parameters, and finally call the method to be tested. By default, the tests will return a result of Inconclusive. You can change this under the Configuration tab when creating the tests. Inconclusive means that it is not known whether the test succeeded or failed. This is done to remind you that you need to actually put some logic into your tests. Consider the following example of a method to be tested:

Public Class Math
  Public Function Add(ByVal Value1 As Integer, _
    ByVal Value2 As Integer) As Integer
    Return Value1 + Value2
  End Function
End Class
In this example, the Add method simply returns the result of adding two integers. When you first create your unit test, the generated test code will look similar to that shown in Figure 5.

Figure 5 Generated Test Code (Close Figure 5)
''' <summary>
''' AddTest is a test case for Public Function Add(As
''' Integer, As Integer)
''' </summary>
<TestMethod()> Public Sub AddTest()
  Dim target As NewOrderEntry.Math = New NewOrderEntry.Math

  ' TODO: Initialize to an appropriate value
  Dim Value1 As Integer
  ' TODO: Initialize to an appropriate value
  Dim Value2 As Integer

  Dim expected As Integer
  Dim actual As Integer

  actual = target.Add(Value1, Value2)
  Assert.AreEqual(expected, actual)

  Assert.Inconclusive("Verify the correctness of this test method.")
End Sub
Figure 4 Creating Unit Tests (Close Figure 4)

To run your test, you must first compile your project and then open up your Test Manager window. Test Manager can be found under the Test | Windows menu, and allows you to view your various tests and select which tests you wish to run. The Test Manager also allows you to add your tests to the various test lists that you create. For instance, you may want to group together all tests that are used to exercise your business logic, or your data access code. Alternately you could group your tests by functional areas within your application.

When you run your tests, you have several options. One option you should consider is turning on code coverage, which lets you capture and analyze data showing what code was executed by your unit tests. The code coverage option is part of the test run configuration. From the Test Manager, the toolbar button Edit Test Run Configuration allows you to select and edit a set of configurable options for running your tests. After running your tests, there are two things you should review: the test results (see Figure 6) and the code coverage results (see Figure 7).

Figure 7 Code Coverage Results (Close Figure 7)

Figure 6 Test Results (Close Figure 6)

You will notice that the Test Results window indicates that my one test was inconclusive. This is because I didn't change the generated test to do anything useful. The test should be changed to something similar to that shown in Figure 8.

Figure 8 Revised Test Code (Close Figure 8)
''' <summary>
''' AddTest is a test case for Public Function Add(As Integer,
''' As Integer)
''' </summary>
<TestMethod()> Public Sub AddTest()
  Dim target As NewOrderEntry.Math = New NewOrderEntry.Math

  Dim Value1 As Integer = 32
  Dim Value2 As Integer = 10

  Dim expected As Integer = 42
  Dim actual As Integer

  actual = target.Add(Value1, Value2)
  Assert.AreEqual(expected, actual)
End Sub

In Figure 8, the code assigns the two parameter values to two integers and checks the return value of the Add method to be sure that it is as expected. If the value returned from the Add method is 42, the correct answer, the test passes; if not, the test fails. You also have the ability to create data-driven unit tests. This will allow you to use actual data from a database, rather than hardcoding values.

Unit tests are only useful if they are written to accurately exercise your code. If you have 1000 lines of code and only execute 10 lines with your unit tests, then your unit tests don't give an accurate representation of how good your code is. The code coverage capabilities of Team System are a great resource to determine if your unit tests are covering the majority of your existing code, and conversely to determine if there's any existing code that you'll never use and that can thusly be removed.

In addition to the Code Coverage Results view, you can also turn on color coding of your source code. Figure 9 shows two pieces of code. The code in red was not executed by the unit tests; the code in green was executed.

Figure 9 Code Coverage Source Highlighting (Close Figure 9)

The test results and code coverage results can be published to your Team Foundation Server which, in turn, stores the data in the database. This allows reports shown on the portal site to use this data to give a representation of where the project stands from a testing and code quality perspective.

Load Testing

In addition to unit testing, Team System also provides the ability to create, manage, and run Web load tests. This functionality is very similar to what was previously provided by Application Center Test, only much more robust, scalable, and entirely integrated with Visual Studio. You have the ability to create your own tests from scratch, as well as to record a Web browser session for later playback on one or multiple PCs. Some nice features are included in the load test tool. In particular, the tool has the ability to handle ASP.NET forms authentication in addition to the ability to properly handle ASP.NET ViewState.

To use the load testing capabilities, the easiest way is to create a new Test Project. This is a new project type that you can add to your solution. Once that is done, you can add multiple new tests by right-clicking the project in Solution Explorer and selecting Add New Item. Recording a Web load test is as easy as adding the appropriate type of test to your project and selecting Record to initiate the record process. As you create new tests, they will show up in the Test Manager window along with your unit tests, providing you with one complete view of the various tests for your project. Web tests are distinct from load tests in Visual Studio 2005. A Web test is the script that exercises a particular portion of a Web application. A load test is a separate test that contains the appropriate data used to simulate load, such as number of users to simulate. You therefore create Web tests and then use them in a load test to stress your Web application.

Testers, Manual Testers, and Bug Tracking

While Team System provides unit testing and load testing of Web applications, it does not provide an automated mechanism to test non-Web user interfaces (though Compuware has announced the integration of its TestPartner product to provide testing for rich client user interfaces). Your QA team is supported by having built-in test management tools, as well as a bug-tracking system. Bugs are simply a particular type of work item, and are entered into Team System and assigned as a task for someone to work on. The status of bugs, as well as bug counts, will be viewable from the project portal site.

Another great feature is the ability to store and manage the various manual tests that exist for a project. You are likely familiar with the various Word documents and manual scripts that often get created to detail the steps required to manually test a system. Team System allows these scripts to be stored as manual tests in TFS and managed alongside your Web tests and unit tests. These scripts can be actual Word documents or plain text. There is also the capability to execute these tests from within Visual Studio.

When these tests are selected to be run, their status is pending until a tester steps through the manual process and marks the test as having passed or failed. Obviously, a bug would likely be entered if the test fails. The manual tests can also be grouped using test lists to provide for some great organization capabilities. Having the tests in TFS allows developers easy access to these tests as well if they need to repeat the steps that were followed to duplicate a bug. These new tools really pull the testers and the developers into the same environment to allow them to work better together.

Team Build

Another tool you have in your toolkit with Visual Studio 2005 is Team Build. Team Build utilizes MSBuild, which is the new extensible build engine that ships with the .NET Framework 2.0. Team Build supports the concept of a build server. This build server will listen to network requests that will direct the build server to build the application using build scripts that are stored in Team Foundation Version Control. The build scripts can be generated from Visual Studio using a wizard to select various build options. These options include having FxCop or unit tests run as part of the build process. You can also have work items updated and notifications sent as part of the build process. For instance, you could have a nightly build process to automatically compile your application, run unit tests and static analysis, and then finally put the new build on a test server and notify testers of the availability of a new build.

The Team Site and Reporting

Aside from the developer-oriented features of Team System, the important visibility it brings to those outside the immediate development process should be noted. This includes managers, project managers, testers, business users, analysts, and anyone else that may have interest in the status of a development project. While testers and project managers may have access to Team Foundation Server by using the Team Foundation Client, it isn't realistic to assume that business users and IT management would be running Team Foundation Client.

The SharePoint home page for your project is the perfect tool for these individuals. From this site they are able to access the current status of the project, review the number and severity of bugs, and access the project documentation. In addition, various reports will be available. These will include reports for outstanding work items, outstanding bug reports, test results, and many others. This added visibility should provide a much greater insight into the development process than what has previously been available.

Before I finish, I should make one point about extensibility. For organizations that have existing development processes already in place, Team System was designed to support customization. Team System was also designed with the idea that third-party companies would be able to integrate and extend it. Borland has already announced that they will be delivering a version of their CaliberRM requirements management tool that integrates with Team System. This fills a gap in the Microsoft product suite. If Team System doesn't meet all of your needs, however, be sure to look for the appropriate tools that integrate with and extend it.

Conclusion

Visual Studio 2005 Team System will be purchased in a different manner than Visual Studio is purchased today. I want to make this distinction since the beta builds don't accurately reflect the variety of products that will be available when the product is officially released. Most notably, Team Foundation will be a separately purchased server product. There are already several feature walkthroughs available and more will be released during the beta cycles. If you have been considering putting more "process" into your development efforts, then you should consider investigating Team System to see if it meets your needs. For more information, visit the product site at Visual Studio 2005 Team System.

Chris Menegay is a founder and principal consultant with Notion Solutions. Notion Solutions is a training and consulting firm based in Dallas, Texas that specializes in Microsoft technologies. You can reach Chris at weblogs.asp.net/cmenegay.


Editor's Note: 20 Years and Still Going Strong!

Editor's Note
20 Years and Still Going Strong!

Microsoft Systems Journal (MSJ) was founded in 1986. At the time, no other publication focused specifically on the needs of programmers who were using Microsoft tools and technologies. Two issues were produced in the first year, each with five articles. From this experiment 20 years ago, we have grown to the magazine you have in front of you today.

Along the way, we have reported on some of the most important technologies of an ever-changing, always fascinating industry. If you are looking for a history of programming for the past two decades, you'll find it in the pages of MSJ, Microsoft Internet Developer (MIND), and finally MSDN Magazine. In this, our 20th year of publication, we thought we'd step back and look at some of the key topics we've brought you over the years.

1986 A year after the launch of Windows 1.0, MSJ launches its first issue. Key topics include Windows 1.0, MS-DOS, and Microsoft XENIX. Typical articles include "Aldus: Preparing PageMaker for the Move to Windows" and "DDE: A Public Protocol for Advanced Application Linkages."

"Working on MSJ and later MSDN Magazine was, like the magazine itself, always stimulating, decidedly cutting edge, and an amazing opportunity to learn from the best in the industry."—Joanne Steinhart, Editor-in-Chief

1987 In March, MSJ becomes a bimonthly publication. You would've learned about Windows 2.0, Windows 386, and OS/2, as well as brand new CD-ROM technology. Articles like "Microsoft OS/2: A Foundation for the Next Generation" and "Microsoft Excel for Windows: Meeting the Demands of a New Generation" really showed our love for the word "generation."

1988 Branching out a bit from pure systems coverage, MSJ discusses SQL Windows, QuickBasic 4.0, and other languages. May brings the first discussions of DLLs in the article "Design Concepts and Considerations in Building an OS/2 Dynamic-Link Library." The C language is pretty much your only choice for serious systems-level programming without assembler.

1989 MSJ still has a systems-level focus, with articles about emulating UNIX serial I/O interfaces and using 80286 protected mode functions. Charles Petzold writes his famous series "Programming for Presentation Manager," including a program called Checkers. Disappointingly, it makes no references to Richard Nixon's dog.

1990 A watershed year for users, developers, and MSJ. In July MSJ runs its first introduction to Windows 3.0. This is generally considered the release in which Windows really takes hold with consumers and, not coincidentally, with developers.

"When I was designing the prototype of MSJ in August of 1986, Jon Lazarus told me that my main mission was to create something new for developers—a magazine that would present code as clearly and prominently as the article text, which was a radical notion at the time." —Michael Longacre, Editor-in-Chief

1991 Microsoft begins joint publication of MSJ with M&T Publishing. 15 years later, its descendant, CMP Media, still publishes MSDN Magazine in partnership with our editorial staff here at Microsoft. 1991 is a banner year for development. In May, MSJ runs its first C++ coverage. In July, MSJ provides the first look at a new language, Visual Basic, which revolutionizes the way people write accounts payable systems with bright red window backgrounds. In September, Windows 3.1 is discussed, along with innovations like OLE and TrueType. September also marks the last discussion of the ill-fated OS/2. November provides the first discussion of the Windows 32-bit API, later known simply as Win32.

1992 As Microsoft prepares to release its first version of C++, MSJ looks at MFC for the first time in March. This is followed by the first coverage of Windows NT in July, and the biggest event of all in September—MSJ becomes a monthly publication.

1993 C++ is quickly becoming the standard language for Windows-targeted programming, and in February, Paul DiLascia writes his first C/C++ column. Paul is still writing a monthly column for MSDN Magazine today.

1994 Windows 95, code-named "Chicago," is debuted in the January issue. The Internet is quickly becoming the hot new pastime for consumers, leading to our September coverage "Getting Wired Into the Internet: A Crash Course on FTP, Gopher, Web, and More."

1995 In May, Jeff Prosise files the last installment of his MS-DOS Q&A column. He writes a series on programming for Windows 95, followed by the debut of his Wicked Code column. In June, Don Box writes his first OLE Q&A column. Windows 95 is released, providing millions of users with a built-in TCP/IP stack and simplified Internet access. It is followed closely by the release of Visual Basic 4.0, which bridges the gap between the 16-bit world of Windows 3.x and the 32-bit world of Windows 95.

"Al Gore may have helped create the information superhighway, but in starting Microsoft Interactive Developer in 1996, we built its first rest stop—complete with facilities."—Eric Maffei, Editor-in-Chief

1996 Microsoft Interactive Developer (MIND) is launched in the spring, featuring an interview with Bill Gates and an overview of "Sweeper," the code name for a package of technologies that becomes WinInet, part of ActiveX. The MSJ Web site launches. In November, Don Box's column reflects the evolution of the times by becoming ActiveX/OLE Q&A. It will later change into ActiveX Q&A before becoming House of COM in July 1998.

1997 MSJ continues to diversify its focus. February brings Visual Basic 5.0, which lets you create ActiveX controls. April's issue provides the first coverage of Windows CE, the version of Windows for mobile devices. June brings the first look at Active Template Library, as well as a sneak peek at Windows NT 5.0, which becomes Windows 2000. Finally, our October issue features John Robbins's first Bugslayer column as well as a look at Active Desktop for Internet Explorer 4.0.

"In my first week of work in 1989, I went to see how OS/2 was being used at KFC and shared a bucket with Steve Ballmer." —Laura Euler, Senior Editor

1998 With new technology comes new fears, and in the wake of the building Y2K Fever, technical editor Joe Flanigen sets off a small wave of terror in July by explaining how dates in MFC only run through the year 2038. MIND changes its name, but not its acronym, to Microsoft Internet Developer. Meanwhile, MSJ continues to become more Internet focused. Articles like "How to Design Reusable HTTP Components by Exploiting WinInet and COM" vie for space alongside assembly language and Win32. In November, Keith Brown writes his first Security Briefs column.

1999 After more than a decade with the publications, editor-in-chief Eric Maffei retires, taking his many references to "2001: A Space Odyssey" with him. C++ is not the only game in town anymore—OLE DB and ADO, along with Visual Basic, are now seen as mainstream programming and receive lots of coverage in MSJ.

2000 As the worlds of traditional desktop, server, and Internet-based programming converge, we reflect this trend by combining MSJ and MIND to form MSDN Magazine. The first issue covers topics from SQL Server 2000 to SOAP. In July, MSDN Magazine scores a coup by publishing a special edition in conjunction with the Microsoft PDC in Orlando, where the .NET Framework is unveiled. We become the first magazine to provide technical coverage of the .NET Framework and C#.

2001 Having overseen the successful launch of MSDN Magazine, editor-in-chief Michael Longacre retires to his gracious country estate. Windows XP is the hot topic, both in the pages of MSDN Magazine and at the Microsoft PDC in Los Angeles. With in-depth monthly coverage, .NET has become the heart of MSDN Magazine.

2002 MSDN Magazine becomes part of the MSDN subscription program in the United States. We are careful to temper our .NET coverage with "guru" stories like "Inside Windows: An In-Depth Look into the Win32 Portable Executable File Format." In a nod to the increasingly important role that security plays in the world of programming, MSDN Magazine launches its first annual security- themed issue in November.

"What strikes me most over the last six years is the dedication of every member of the team involved. We all believe in what we are doing and the goals we have set together."—Kerry Gates, Publisher

2003 Building upon the success of our first "mini-mag" in 2000, MSDN Magazine publishes a special edition for the Microsoft PDC in Los Angeles. The 48-page supplement focuses on the next version of Windows, code-named "Longhorn," and the next version of SQL Server, code-named "Yukon."

2004 After more than 15 years with the group, editor-in-chief Joanne Steinhart retires. The first holiday season without her famous cookies is indeed bleak for many staffers. Building on the success of MSDN Magazine, we launch TechNet Magazine for the IT professional market and we begin to talk about "Whidbey," the code name for both Visual Studio 2005 and the .NET Framework 2.0.

2005 MSDN Magazine is now received by about 150,000 readers a month in 25 countries, with eight local editions. For 20 years, we have maintained one simple goal—we want to be your first source for understanding how to develop with Microsoft tools and technologies. We've been successful by keeping this in mind every month, but we really couldn't do it without your loyal readership over the years. From the bottom of our hearts, we thank you for all you've helped us become, and we're looking forward to the next 20 years.

—J.T.

Thanks to the following Microsoft technical experts for their help with this issue: Pablo Castro, Felix Cheung, John Durant, Erick Ellis, David Fries, Robert Green, Sam Guckenheimer, Kelvin Hoover, Elena Kharitidi, Ronald Laeremans, Gerrard Lindsay, Rico Mariani, Randy Miller, Brian Pepin, Jennifer Perret, Elliot Rapp, Mark Rideout, Stefan Schackow, Joe Stegman, Anson Tsao, Maura Van Der Linden, and Ravinder Vuppula.

ActiveX, DirectX, IntelliSense, JScript, Microsoft, MSDN, SharePoint, Visual Basic, Visual C++, Visual SourceSafe, Visual Studio, Windows, Windows NT, Win32, and Windows Server are either registered trademarks or trademarks of Microsoft Corporation. Other trademarks or tradenames mentioned herein are the property of their respective owners.

MSDN Magazine does not make any representation or warranty, express or implied with respect to any code or other information herein. MSDN Magazine disclaims any liability whatsoever for any use of such code or other information.


New Stuff: Resources for Your Developer Toolbox

New Stuff
Resources for Your Developer Toolbox
Marnie Hutcheson

Contents
A Web-Based Grid Component
A Wish List of Controls
Advanced Data Display Controls
A New Content Management Tool
Protecting Your Web Apps
The Bookshelf

A Web-Based Grid Component

DbNetGrid.Net 2.71 by DbNetLink Limited, is a Web-based grid component that works with DHTML behaviors that support both ASP and ASP.NET. The integrated DbNetGrid GUI designer application lets you create your grids without writing any code. It features a tab-based interface that lets you define and modify your grid's methods and properties.

The designer generates fully encapsulated grids that you can embed in your Web pages. You can also link and nest these grids on the same page to create drill-down capabilities in the browser. Since it's pure DHTML, there are no client-side installation or Java language requirements. These grids can be searched, sorted, edited, exported, uploaded, printed, and copied. You can save the results in HTML, Word, Excel, or XML formats.

The server-side portion of the component runs as a C# compiled library in the .NET managed environment so there is no interpreted script on the server. It will automatically select the SqlClient or OleDb ADO.NET libraries depending on the database you are accessing. In most cases you can upgrade your grids created in the previous version, DBNetGRID to DbNetGrid.Net without making any coding changes.

Price: A single-user license is $199.

www.dbnetgrid.com

A Wish List of Controls

Xtreme Toolkit Professional Edition 9.51 for Visual Studio .NET, by Codejock Software, includes just about every type of control you can wish for. All the controls are fully customizable user interface components for the development of Windows®-based applications. Two packages are available: Xtreme Suite Professional Edition for ActiveX and Visual Studio .NET, and Xtreme Toolkit Professional Edition for C++ and MFC.

All the components, dialogs, and displays come in multiple styles so you can get the appearance that you want in your application. You get animation behaviors, task panel drag and drop to move options up and down in listboxes, hot tracking (mouseover events), and flicker-free resizing. The toolkit supports multiple languages, including right to left languages.

There is a cool new style docking and toolbox control like the one in Visual Studio® 2005 that gives developers using MFC the ability to provide advanced visual cues for window placement when the docking position is changed within the application.

Price: Xtreme Toolkit Professional Edition one-year subscription is $595 and Xtreme Suite Professional Edition one-year subscription is $395.

www.codejock.com

Advanced Data Display Controls

SftTree/OCX by Softel vdm Inc. gives you hierarchical data displays for your applications. The control combines features found in tree view, list view, and grid controls and includes support for new cell editing mechanisms that allow most Windows controls to be used for cell editing. There is a SftPrintPreview/OCX control, available separately, that enables printing and print preview right from the display.

You get a lot of grid features and flexibility such as multiple columns, multiple selection, multiple text lines per item, cell word-wrap, built-in row and column headers (with optional buttons and bitmaps), user resizable columns, column reordering, grid lines, sorting and optimal column width, and horizontal scroll-area calculation.

SftTree/OCX 5.0 includes online help, printable documentation (PDF file), technical support (via telephone, fax, or e-mail), and free downloadable maintenance updates.

SftTree/OCX supports most popular development environments including .NET, Visual Basic®, Microsoft® Access, Delphi, and C++ Builder. It runs on Windows 98, Windows Me, Windows 2000, Windows XP, and Windows Server™ 2003.

Price: A developer license is $439.

www.windowscontrols.com

A New Content Management Tool

XStandard Pro by XStandard is a standards-compliant plug-in WYSIWYG editor that generates XHTML (Strict or 1.1) code. XHTML Strict/1.1 complies with Web Content Accessibility Guidelines (WCAG) developed by W3C.

XStandard uses only cascading style sheets (CSS) for formatting, so it ensures the clean separation of content from presentation. The Pro version automatically replaces noncompliant proprietary markup while at the same time it preserves essential structural elements like paragraphs, tables, bullet points, headings, hyperlinks, and images.

XStandard stores its localization data in an independent XML file, so modifying XStandard's 13 standard interface languages (including Hebrew and Arabic) is straightforward, and you can create new language versions as well.

XStandard's Screen Reader Preview displays your content as it is "read" by screen readers. This means content is laid out in a linear fashion, together with information that the author would normally not see (alt text, table summaries, tool tips, and so on).

XStandard also gives you alerts when you are missing markup that is necessary for accessibility, or when you have used expressions that are meaningless when processed by a screen reader.

XStandard Pro allows you to upload files through the editor with Web services that handle file uploads, building and using image libraries, and spell checking your content. These services also support communicating with third-party systems and importing third-party content, including preformatted content such as entire chunks from product catalogs. These Web services are used to supply your content management system with information such as the name of the document with which the file is associated, the Session ID, and the user ID of the person who uploaded the file. This feature can be customized for different content management systems.

Price: A 10 user/1 Web Service URL license is $179.

www.xstandard.com

Protecting Your Web Apps

BrowserHawk 8.0 by cyScape Inc. helps you ensure the best Web experience for site visitors, regardless of their browser and its settings, by detecting everything about the client so that you are able to react to client-side limitations proactively.

BrowserHawk 8.0 lets you automatically alert users who have blocked pop-ups that they should enable pop-ups or your site won't work as expected. In addition to detecting the features you would expect like browser type and version, service packs, and build information, BrowserHawk detects screen size, available browser window size, color depth, text size, installed fonts, and stylesheets.

The BrowserHawk Reports Service, a tracking database on the server, gives you an important trouble-shooting tool, which allows you to trace back to a particular user's actual session. What a boon if customer service can actually see what the customer did! The Reports Web Service will also give you the information that will let you answer questions such as, "Does it make sense to implement Flash 7?"

BrowserHawk is available as a native COM, .NET, or Java language component which integrates seamlessly in just minutes with classic ASP, ASP.NET, ColdFusion MX, JSP, and servlet-based Web sites. BrowserHawk is installed on the server side, and does not require any software to be installed on the visitor's machine. BrowserHawk supports Microsoft Internet Explorer 6.0 Service Pack 2, Firefox, and Safari browsers.

Price: BrowserHawk 8.0 Standard Edition with one year support and maintenance is $319.

www.cyscape.com

The Bookshelf

Even though Microsoft Excel 2003 still looks pretty much the same as it did in 2002, there is a lot of new stuff under the hood, such as SharePoint® integration, XML support, better Web services support, and integrated support for the .NET Framework. This new stuff is primarily devoted to collecting and sharing data, programming across applications, and maintaining security to support teamwork and collaboration.

Excel 2003 Programming: A Developer's Notebook by Jeff Webb (O'Reilly, 2004) focuses on the newest programming features of Excel 2003. The book shows users and programmers alike how to share spreadsheets with SharePoint, work with lists and XML data, create secure Excel applications, use Visual Studio Tools for Office, and consume Web Services.

Each chapter is organized into a collection of labs, each of which addresses a specific programming problem. Readers can follow along to complete the lab on their own, or jump ahead and use the samples the author has included. Webb sets out each feature, tells you what it does and why, then shows you how to do it and how it works. There are lots of screen shots and code samples.

Price: $29.95, 293 pages.

www.oreilly.com

If you think that smaller is better, and that complex and bloated Web sites are a thing of the past because they waste precious time and resources and are often ineffective, then you will want to read Small Websites, Great Results by Doug Addison (Paraglyph Press, 2004).

This book shows you what makes small sites really work, how to select winning themes and formats for a site, and design techniques that make sites look professional, even on a small scale. It also shares tips on how to get business from the Web and strategies for getting, keeping, and satisfying customers.

The book also includes examples of small sites that work for different types of businesses including artists, entertainers, writers, speakers, consultants, and professional organizations.

Price: $29.99, 352 pages.

www.paraglyphpress.com

All prices were confirmed at press time and are subject to change.

Send your New Stuff to newstuff@microsoft.com.

Marnie Hutcheson is president of Internet Development Associates (Ideva), a firm in Ocala, Florida, that specializes in Internet and intranet Web application design and development. She has published technical papers and books on various computing topics. You can reach her at marnie@ideva.com.


Web Q&amp;A: Get Authentication Type, Get Screen Resolution, and More

Web Q&A
Get Authentication Type, Get Screen Resolution, and More
Edited by Nancy Michell

Q I have an authentication question: I'm using Windows® authentication for a Web site in one environment, and Passport authentication in another. If I check the following value to get the authentication type in Windows

HttpContext.Current.User.Identity.AuthenticationType
sometimes an empty string is returned and sometimes "NTLM" is returned. Shouldn't it always return "NTLM"?

A This is by design. Once the client has an authenticated connec-tion, it does not need to send any credentials for subsequent requests on that connection. So if ASP.NET is pulling the data from the authentication header, then this header is not there. You can control this using the AuthPersistence setting in IIS. The AuthPersistence property specifies authentication persistence across requests on a connection using NTLM authentication.

In IIS 6.0, when the AuthPersistSingleRequest flag is set to true and you are using NTLM authentication, IIS automatically reauthenticates every request, even requests that are coming over the same connection. Reauthenticating every request also occurs on all other authentication schemes. In IIS 5.0 and IIS 5.1, there are two other flags for the AuthPersistence property specifying persistence based on the existence of a proxy server. (See "AuthPersistence" in the MSDN Library for more information on this.)

Q On my Web site I need to have different menus for different user groups and hide or show menu options based on the user's group. I know that you can base a menu on a site map. Is there a way to do this programmatically for a session so that when a user logs in, the app determines the user's group and chooses an appropriate site map?

A The site map providers will follow the security of the Web site if SecurityTrimmingEnabled="true" is set on the provider. The SecurityTrimmingEnabled setting indicates whether a site map provider filters site map nodes based on a user's role. This means that if you partition your Web site access based on roles, then the sitemap node visibility will respect the restrictions.

Q I want to adapt my Web content (layout and picture resolution, for instance) based on the end user's screen resolution. How can I get a user's display settings?

A You can use the following script to get the screen resolution. Once you have the pixel count, you have the screen resolution:

<script language='javascript'>
function sendScreenInfo()
{
  FormID.scrWidth.value = window.screen.width;
  FormID.scrHeight.value = window.screen.height;
}
</script>

Q I have an ASP.NET page that needs to access a repository of files located on another server within the same network. I tested my code using a Win32® application. However, when running the same code I get a "permission denied" exception because the IUSR_machinename account that ASP.NET uses does not have access to the server or folder where the repository is located. What is the best way to give read-only security access to the IUSR account so that it is able to read the data in the repository files?

A You can try the following: let's assume your Web server name is websrv1 and the computer with files is filesrv1. On filesrv1 create a folder, share it, and grant both share and access permissions to "NT AUTHORITY\Network Service" and "YOURNETWORK\webserv1$" (the computer account).

To grant access to the computer account you will have to click on the Object Types button and put a checkmark next to Computer. This is assuming your websrv1 is running Windows Server™ 2003. If it's running Windows XP you will have to change your Machine.config to run ASP.NET as the Network Service account instead of from an ASPNET account.

Note that your ASP.NET code runs under the Network Service account's permissions, not IUSR_machinename. IUSR_machinename is used to determine if the user has access to that page or not, but the page's code runs as Network Service under Windows Server 2003 or ASPNET under Windows XP. An application will run as Network Service only unless you have <identity impersonate="true"> in your Web.config file. If you do, the application runs under the user that is requesting the page (or IUSR_machinename if user is anonymous).

Q My app is downloading large Zip files from IIS to a client computer and is attempting to show a download progress indicator while this is running. I attempted to determine the data size by reading the Content-Length HTTP header using the HttpQueryInfo function. However, the server failed to report the file size. Is there a way to force IIS to always include the Content-Length header for this to work?

A Although many HTTP responses include the Content-Length header, it is not a required component of a response. Unless you have another mechanism for getting the data size, you will not be able to calculate progress if the Content-Length header is not included in the response. (See Knowledge Base article 234913, "How To Provide Download/Upload Progress Information when Using WinInet," for more information.)

However, you may have something running on your machine that's affecting this because, without outside influence, IIS will treat a Zip file as a static file, and IIS always adds a Content-Length header to a static file response. Check the machine for filters (and if it's IIS 6.0, check it for wildcard script-mapped extensions). Also, check to see if .zip is script-mapped to some handler.

If there is something other than the static file handler sending the response (and because there is no Content-Length, you can infer that there is), then it's up to that code to decide whether the Content-Length is appropriate. That said, it is perfectly normal, and a necessary part of HTTP, that the client has to be able to deal with an absent Content-Length header. Why do you want to make your client noncompliant? If you have control over the server, it is very possible to ensure that you always have Content-Length. If you think about it, there are two general ways to send things between server and client:

  1. Server tells the client to expect a specified number of bytes.
  2. Server tells the client to keep reading until special STOP sequence arrives.
Clearly in the first case, the client knows total size and what it has read, so a percentage is doable. In the second case, the client has no idea of the total size and thus cannot do a percentage.

As mentioned. if you are truly accessing a static file (such as http://server/file.htm), IIS will send Content-Length. The fact that you are not seeing Content-Length indicates that you need to look for another source of the problem.

Q I have a large DataGrid (60,000 items). There are two buttons that do other tasks not related to the DataGrid. After rendering the page and the DataGrid, if I click either button I get the following error:

Server Error in '/AdminTools' Application.
Maximum request length exceeded.
I set EnableViewState to False on the control, but that didn't help. In the Web.config I set <httpRuntime maxRequestLength="512000"/>, which fixes it, but that's rather clumsy and won't continue to work as the DataSet continues to grow. Is there a better solution?

A In a Web scenario, 60,000 items is too much data to display. Having so many items will cause a variety of problems including slow performance (even when the bandwidth is high), lots of data in view state (fortunately you have disabled it), and running up against the request limit. Of course, the request limit is a security feature and controlling it helps prevent denial of service (DoS) attacks in which large amounts of data are sent.

Possible solutions include the use of paging for the DataGrid and the use of stored procedures. Then you can return only the number of rows that you are going to display per page.

Q I have recently tried tweaking the Internet Explorer disk quota setting and I am seeing some weird behavior. I just tried setting the quota to 1MB (Tools | Internet Options | Settings button on general tab) and then downloaded several files that are each at or greater than 1MB. Each of these files is successfully written to the cache. I was expecting that when the quota was reached, the existing entries would be purged in some way. Is my assumption incorrect?

A Everything that Internet Explorer renders or downloads goes into the cache. If you set the boundary at 1MB it will exceed that temporarily rather than fail. However, after the session ends—when WININET receives the "INTERNET_OPTION_END_BROWSER_SESSION" message, which does not mean that the IEXPLORE.EXE process or the calling process has exited—the limit will be imposed.

Q I want to manage an org chart using a tree view. When the user creates a new group, she should be able to choose which parent group the new group would belong to and be able to switch the display order in the parent group. A Windows Explorer-style folder view lets you manage folders in the tree view but does not let you choose the display order. I don't need drag and drop, but I do want to be able to change the display order.

A Tree View controls can be problematic unless users either know the data hierarchy intimately or the data hierarchy uses commonly defined terms which the majority of your users can successfully use to navigate. Larger directories are easier to scan if they are in a logical order—you're granting users the right to create an illogical order, which could make retrieval of that information more difficult later (if they forgot where they put it).

However, if you proceed with a tree view, a good place to define the parent is in the interface where you invoke the "new group" command. This prevents the overhead of naming a new item and simultaneously choosing its position in the hierarchy. Only allow the repositioning of a group at the moment the new group command is issued or after the new group command is completed. Drag and drop seems like the best metaphor for positioning, but if you want an alternative, consider an interaction metaphor that gets away from direct manipulation of the object and makes the position a property. Perhaps you can display a property sheet for the object independent of the treeview control.

Q I want to convert a batch of HTML files into plain text using a regular expression. Any suggestions?

A If you're going to parse the text with a regular expression, you should use the Scripting.FileSystemObject to read the file, then manipulate the text with your regular expression. If you want to locate nodes in the doc and are sure the HTML is well formed, using the XML ActiveX® object is a good bet—load the document and use XPath expressions. Of course, when do you ever see a well-formed HTML document? The code in Figure 1 will give you access to the Document Object Model (DOM). Have the browser navigate to your files, not the paths you see here, then use Request/Response/Page, and so on.

Figure 1 Accessing the DOM (Close Figure 1)
var tagREADYSTATE =
{
   READYSTATE_UNINITIALIZED   : 0 ,
   READYSTATE_LOADING         : 1 ,
   READYSTATE_LOADED          : 2 ,
   READYSTATE_INTERACTIVE     : 3 ,
   READYSTATE_COMPLETE        : 4
}

var ieInst = null;

try {
   ieInst = WScript.CreateObject("InternetExplorer.Application");
   ieInst.navigate("http://www.money.msn.com");
   while(tagREADYSTATE.READYSTATE_COMPLETE != ieInst.ReadyState)
   {
       WScript.Sleep(10);
   }
   var htmDOMDocument = ieInst.document;
   WScript.Echo(htmDOMDocument.getElementsByTagName(
       "title")(0).innerText);
}
catch(e) {
   WScript.Echo(e.description);
}
finally {
   if(null != ieInst)
   {
      ieInst.Quit();
   }
}
WScript.Quit(0);

Got a question? Send questions and comments to webqa@microsoft.com.

Thanks to the following Microsoft developers for their technical expertise: Kashif Alam, Dmitriy Belenko, Joe Bergeron, Gaylon Blank, Mike Bolser, Jason Brown, Mike Burdick, Greg Bybee, Danny Chen, Jon Cole, Pierre Louis Coll, Brian Combs, Courtney Crawford, Jeff Davis, Sean Everhart, Christophe Fiessinger, Ravi Gopinath, Mick Hakobyan, Anand Hegde, Mike Hestness, Wade Hilmo, Drew Leaumont, Yunus Mohammed, Bernard O'Flynn, Dustin Rector, Miguel Filipe Santos, Stefan Schackow, Fyodor Soikin, Takeshi Tanaka, Abdallah Toutoungi, and David Wang.


Advanced Basics: Remembering User Information in Visual Basic .NET

Advanced Basics
Remembering User Information in Visual Basic .NET
Duncan Mackenzie

Code download available at: AdvancedBasics0504.exe (130KB)
Contents
Setting Up and Retrieving Application Settings
Why Shouldn't You Use App.Config?
Creating a Settings Class
Serializing Your Settings to Disk
Isolated Storage
Saving and Loading Within Your Application
What About Visual Basic 2005?
Conclusion

Many applications need to store user-specific settings to be persisted between sessions. But how do you go about saving and restoring these settings in your Microsoft® .NET Framework-based application? It's not all that easy to find the correct answer. You'll find a wide variety of solutions on newsgroups and forums, but only a few of the posted solutions illustrate the proper way to handle this requirement.

First, let's define the two main types of settings that an application is likely to have: application settings and user settings. Application settings are shipped with the app, affect every user, and are intended to be modified only as an administrative task when the overall behavior of the application needs to change.

Consider a commercial human resources application. When it is deployed within your particular company, the application settings are customized to specify the database server that your users are going to connect to, where to find your company's logo to add to the UI, and so on. These are application settings, not user settings, and can be installed right into the same directory as the program itself. These settings are likely installed in a directory where only an administrator has permission to edit or delete files (c:\Program Files\, for example), which helps to reinforce that these are definitely not user settings. The application should only need to read these settings, not to change them.

User settings are preferences that are user-specific and need to be editable from your code. For the mythical HR application, user settings would include the size/position of the application window, the startup state of the application, and even rapidly changing information such as the last five employees viewed (for that useful "Recently Viewed Items..." File menu option). All of these settings need to be retrieved at the start of the application session, edited as necessary while it is running, and persisted to disk to be available for the next time the user fires up your program.

Setting Up and Retrieving Application Settings

Creating an application configuration file is relatively well documented in the .NET Framework SDK, but I thought I would provide you with a brief overview. The first step is to create a new settings file inside your project by selecting Add New Item from the Project menu and choosing the Application Configuration File option. Leave the file with its default name of App.Config; it will be moved into the output directory and renamed appropriately (to yourappname.exe.config) when you build your project. Within the new file you can define custom configuration sections (with whatever structure you need for your data), or add your own settings just by using the appSettings block that can hold a list of name value pairs. Storing a couple of simple values produces a file that looks like this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <appSettings>
        <add key="Company Name" value="Java Jitters" />
        <add key="Database Server" value="BigSQLServer" />
    </appSettings>
</configuration>
It is easy to retrieve values stored in the appSettings section through the classes and methods of the System.Configuration namespace. You don't even need to load the file yourself; it is automatically discovered and loaded if it is named and located correctly (yourappname.exe.config in the application directory):
Dim companyName As String _
    = ConfigurationSettings.AppSettings("Company Name")

As I mentioned, creating and accessing application configuration files is well documented (search on "System.Configuration" in the .NET Framework SDK), so I won't go into any more detail on this topic here.

Why Shouldn't You Use App.Config?

I often see the question "How do I write to my app.config?" on a forum or newsgroup, and it is easy to understand how the developer has arrived at this point. After developers writing within .NET have used the application configuration file and its easy programmatic interface, they wonder why they can't just use the same file to store user settings. Why have two separate ways to configure an application? Shortly after, they discover that the System.Configuration namespace does not provide any method for writing to the app.config file, which should be a warning sign, but that is only a small barrier to the determined programmer. The Web is searched, newsgroup questions are posted, and then direct loading and editing using XmlDocument, or even regular file IO, usually follows.

This is not a good idea. Your application should not write to this file for at least the following three reasons:

Permissions Installing, removing, and editing application files is an administrative task, requiring different permissions from those of a normal user. In Windows® XP, one way that is indicated is through the default permissions on the Program Files directory: read-only to users, but read/write to members of the Administrators group. If your app is installed into the Program Files directory and uses its app.config file to store user preferences, then the act of saving changes to those settings will require administrative rights on the local machine. Although many developers (and users) run as administrators of their own machines, this will certainly be an issue in most corporate environments. Beware that you should not allow unfettered access to Program Files; there are good reasons to restrict write access.

No user isolation The application configuration file applies to every user on the machine, so if multiple users are sharing the same computer, everyone will be sharing the same set of options. The implications could range from the just annoying (one user likes to have their window maximized to start, and another wants it windowed) to an extreme functionality and security issue (the application needs your POP3 server information, and every user of the machine can read that information at will). Even if your current user base is all one user per machine, you can bet it won't stay that way; it is worth noting that even a Guest account on the system might have read access to Program Files directory (and therefore the application settings file), exposing any potentially sensitive information stored into that file.

The app.config file is part of the application's installation This file is part of the application itself and will be overwritten by new installations and likely removed if the application is ever uninstalled. Any user data stored into this file would be lost, which could be very serious depending on the type of data stored.

For these reasons, and probably a few more, you shouldn't use the app.config file to store user settings, but don't worry, saving the user's preferences is not as hard as you might believe.

Creating a Settings Class

The key to easily saving your user-specific settings is to define a class that represents all of that data. This class will be your interface with all of the user settings, and it is the first thing you should build once you determine that you need to store user data at all. Here, I am going to use a simple Windows Form that can open files from disk; the user settings file will store the user's choice of background color, the last window position, and a list of recently opened files. Describing all this data into a class, I came up with the definition shown in Figure 1.

Figure 1 Data Settings Class (Close Figure 1)
Imports System.Drawing
Imports System.IO
Imports System.IO.IsolatedStorage
Imports System.Environment
Imports System.Collections.Specialized

Public Class Settings

    Private m_BackgroundColor As String
    Private m_RecentFiles As New StringCollection
    Private m_LastWindowBounds As Rectangle _
        = New Rectangle(0, 0, 200, 200)
    Private m_LastWindowState As Windows.Forms.FormWindowState _
        = FormWindowState.Normal

    <Xml.Serialization.XmlIgnore()> _
    Public Property LastWindowBounds() As Rectangle
        Get
            Return m_LastWindowBounds
        End Get
        Set(ByVal Value As Rectangle)
            m_LastWindowBounds = Value
        End Set
    End Property

    Public Property LastWindowPos() As Point
        Get
            Return m_LastWindowBounds.Location
        End Get
        Set(ByVal Value As Point)
            m_LastWindowBounds.Location = Value
        End Set
    End Property

    Public Property LastWindowSize() As Size
        Get
            Return m_LastWindowBounds.Size
        End Get
        Set(ByVal Value As Size)
            m_LastWindowBounds.Size = Value
        End Set
    End Property

    Public Property LastWindowState() As Windows.Forms.FormWindowState
        Get
            Return m_LastWindowState
        End Get
        Set(ByVal Value As Windows.Forms.FormWindowState)
            m_LastWindowState = Value
        End Set
    End Property

    Public Property RecentFiles() As StringCollection
        Get
            If m_RecentFiles Is Nothing Then
                m_RecentFiles = New StringCollection
            End If
            Return m_RecentFiles
        End Get
        Set(ByVal Value As StringCollection)
            If Value Is Nothing Then
                m_RecentFiles = New StringCollection
            Else
                M_RecentFiles = Value
            End If
        End Set
    End Property

    Public Property BackgroundColor() As String
        Get
            Dim colorToReturn As String

            If m_BackgroundColor Is Nothing OrElse _
                m_BackgroundColor = String.Empty Then
                colorToReturn = ColorTranslator.ToHtml( _
                    Color.FromKnownColor(KnownColor.Control))
            Else
                colorToReturn = m_BackgroundColor
            End If
            Return colorToReturn
        End Get
        Set(ByVal Value As String)
            If Not ColorTranslator.FromHtml(Value).IsEmpty Then
                m_BackgroundColor = Value
            Else
                m_BackgroundColor = String.Empty
            End If

        End Set
    End Property

End Class

The BackgroundColor property is relatively complex because I wanted to save the color as the HTML representation, but otherwise the class is a simple set of properties. I implemented the list of Recent Files as a StringCollection instead of as an array because it is easier to add new files to a collection, but either approach works.

Serializing Your Settings to Disk

Once you have your class defined, the next step is to write code to read/write this information to a file on disk. It would be possible to generate and parse an XML file using the System.Xml.XmlReader and System.Xml.XmlWriter classes, but it is a bit simpler to just use XML Serialization. Serialization takes care of everything necessary to turn an instance of your Settings class into XML, and knows how to convert it back as well. The serialization approach is quite robust, capable of dealing with missing values, the addition (perhaps in future versions of your application) of new properties, and more, so it ends up being a great choice. You can save your settings class to disk through an instance of the System.Xml.Serialization.XmlSerializer class, and you can use the same class to deserialize your data back from disk as well. The basic code for both operations is shown in Figure 2.

Figure 2 Serializing and Deserializing Your Data (Close Figure 2)
Public Sub SaveSettings()
    Dim xs As New XmlSerializer(GetType(Settings))

    Dim sw As New IO.StreamWriter("C:\settings.xml")
    xs.Serialize(sw, currentSettings)
    sw.Close()

    Dim sr As New IO.StreamReader("C:\settings.xml")
    currentSettings = CType(xs.Deserialize(sr), Settings)
    sr.Close()

End Sub

Figure 2 is just a simple example; I like to put the real saving/loading code right into my settings classes, as a combination of Shared (static for you C# types) and Instance methods. I provide a few different options, a routine to load from a provided stream, one that takes care of getting a stream on its own, and the same options for saving the settings back to disk.

The act of serializing or deserializing requires only two things. First, you need an instance of the XmlSerializer class, created by passing your class's type to the constructor:

Dim xs As New XmlSerializer(GetType(Settings))
You also need a Stream, TextWriter, or XmlWriter instance representing the output for serialization, or a Stream, TextReader, or XmlReader for the input to be deserialized.
Dim sw As New IO.StreamWriter("C:\settings.xml")

Where you save your settings on disk is another important decision. I suggest that you choose either the user's application data directory or the application-specific area of Isolated Storage. The first option stores the settings into a location under "C:\Documents and Settings\userid\Application Data\" (the drive letter and exact location can vary from machine to machine), which is a perfect location for settings as it is user-writeable and consistent with the storage of most other application options. You can obtain a full path to this location using either Application.UserAppDataPath or System.Environment.GetFolderPath. The Application method will return a path like the one shown here that includes the AssemblyCompany, AssemblyProduct, and AssemblyVersion values from your AssemblyInfo.vb file:

C:\Documents and Settings\<userid>\Application Data\
company\product\version
That path is easy to get, but it is a version-specific location. As an alternative, the value returned from GetFolderPath doesn't have your company and product name information in it:
C:\Documents and Settings\userid\Application Data
If you go with the first location, any future release of your product (assuming you increment the version number) will start up as if it has no existing user preferences, which would be a problem for some applications; if you go with the second, your files might conflict with those of another application. I prefer to save my files up one level from the first location, so that it is a version-independent settings file. Getting my desired path could be accomplished a few different ways (taking the parent of the UserAppDataPath, for example), but the following code snippet works well for me:
Dim filePath As String
filePath = Path.Combine(GetFolderPath( _
        SpecialFolder.ApplicationData), Application.CompanyName)
filePath = Path.Combine(filePath, Application.ProductName)
filePath = Path.Combine(filePath, FILENAME)
Dim settingsFile As New FileStream(filePath, FileMode.Create)

You could perform standard string concatenations (or use a StringBuilder) instead, but I prefer to use Path.Combine whenever I'm working with path information.

Isolated Storage

The other ideal location to store user settings is isolated storage. Isolated storage is an abstraction over an area of the user's application data that provides a unique file storage area tied to the current user and some aspect of the running application. I normally use isolated storage tied to the user and the current assembly identity, using the classes in System.IO.IsolatedStorage to open a stream to a new or existing file within that storage area:

Dim ifs As IsolatedStorageFile
ifs = IsolatedStorageFile.GetUserStoreForAssembly()
Dim settingsFile As New IsolatedStorageFileStream( _
    FILENAME, FileMode.Create, ifs)
Return settingsFile

It is possible to find this file on disk if you are willing to browse through the various folders under "C:\Documents and Settings\userid\Local Settings\Application Data\IsolatedStorage" (location may vary). While it's certainly not hidden completely, it is less likely that a user will be able to easily discover this file, compared to one written directly to a normal file location.

For most of my applications, I use the application data location for my settings file. Isolated storage works better in some limited- trust situations where regular file access is restricted. In order to write to the application data location, your application requires FileIOPermission, but to open a file stream within isolated storage requires IsolatedStorageFilePermission. In the default security configuration in the .NET Framework 1.1, only apps with Full Trust have file IO permissions, while even applications launched from Internet sites have permission to write to isolated storage.

For the security-related reasons discussed here, there are applications that will need to use isolated storage and others where it doesn't really matter as much. To make my settings code more reusable, I added a Boolean constant that toggles between the two types of file storage:

Public Const USE_ISOLATED_STORAGE As Boolean = False

Saving and Loading Within Your Application

In your application's main form or function, you need to load the settings file from disk when your application is started, and then persist it (only really necessary if it has been changed) when your application ends. All of the functionality you need is attached right to your class, so all that is really required is that you call those functions and work with the class to edit the user's preferences as needed. In a simple (one main form) application, this can be handled very easily with just a few lines of code.

For loading your settings at startup, use code like that in Figure 3. Then, when saving, use code similar to that in Figure 4.

Figure 4 SaveSettings (Close Figure 4)
Protected Overrides Sub OnClosing(ByVal e As _
    System.ComponentModel.CancelEventArgs)
    SaveSettings()
End Sub

Private Sub SaveSettings()
    With Me.currentSettings
        .LastWindowBounds = Me.Bounds
        .BackgroundColor = ColorTranslator.ToHtml(Me.BackColor)
        .LastWindowState = Me.WindowState
        .PersistMe()
    End With
End Sub
Figure 3 LoadSettings (Close Figure 3)
Dim currentSettings As Settings

Public Sub LoadSettings()
    Try
        Me.currentSettings = Settings.Load()
        With Me.currentSettings
            Me.Bounds = .LastWindowBounds
            Me.BackColor = ColorTranslator.FromHtml(.BackgroundColor)
            Me.WindowState = .LastWindowState
        End With
        UpdateRecentFileMenu()
    Catch ex As Exception
        Me.currentSettings = New Settings
    End Try
End Sub

Public Sub New()
    MyBase.New()

    'This call is required by the Windows Form Designer.
    InitializeComponent()

    'Add any initialization after the InitializeComponent() call
    LoadSettings()

End Sub

Alternatively you could call SaveSettings only when your user's options are changed (after showing the user an option dialog, for example), or you could track its modified state using an additional property such as "dirty." If you decide to add an additional property, which you probably don't want serialized to and from disk, it should be marked with the XmlIgnore attribute. This attribute tells the XmlSerialization process to ignore this particular property; it won't be written into the settings file on disk. In my sample, I used XmlIgnore to indicate that the LastWindowBounds property shouldn't be serialized. That property represented a real value, the rectangle indicating the position and size of the main application Form, but the two parts of that value (the location and the size) are also exposed through their own individual properties, and there is no reason to put the same data into the settings file more than once:

<Xml.Serialization.XmlIgnore()> _
Public Property LastWindowBounds() As Rectangle
    Get
        Return m_LastWindowBounds
    End Get
    Set(ByVal Value As Rectangle)
        m_LastWindowBounds = Value
    End Set
End Property

Once you have your application working correctly, including editing, loading, and saving your settings, you may want to add some additional settings-related functionality. Two features that would be useful are the ability to reset your application's settings to a default (which could be loaded from the app.config file) and to save/load the settings to and from an arbitrary file location. I have added both options to my sample application for you to check out.

What About Visual Basic 2005?

This column has discussed where to store user settings, why you shouldn't use the same method as application settings, and how to create a custom settings class; many of these issues are handled for you in the upcoming release of Visual Basic®. In Visual Basic 2005, there is a settings designer (see Figure 5) built into Visual Studio® that allows you to set up your list of app and user-level settings.

Figure 5 Visual Basic 2005 Settings Designer
Figure 5 Visual Basic 2005 Settings Designer

Once you've configured this info for your project, you can easily set and retrieve these values through the My.Settings class:

Me.Text = My.Settings.CompanyName
If My.Settings.StartMaximized Then
    Me.WindowState = FormWindowState.Maximized
End If
I won't go into any more detail about this new feature, but if you are interested I suggest you check out the article "Using My.Settings in Visual Basic 2005" by Emad Ibrahim on MSDN® online.

Conclusion

When creating your application, divide any settings or preference information into values at the application level and at the user level. For application settings, .NET provides the app.config file and classes to handle reading those values, but for user settings, you need to take care of it yourself. I've presented one way to handle that using XML Serialization, which should get you started building your own settings class for your applications.

Send your questions and comments for Duncan to basics@microsoft.com.

Duncan Mackenzie is a developer for MSDN and the author of the Coding 4 Fun column on MSDN online. He can be reached through his personal site at www.duncanmackenzie.net.


Cutting Edge: A New Grid Control in Windows Forms

Cutting Edge
A New Grid Control in Windows Forms
Dino Esposito

Code download available at: CuttingEdge0504.exe (191KB)
Contents
Building a Simple Data Layer
Customizing the DataGridView Control
Cell Templates
The Data Connector
Sorting
Conclusion

Grid controls are essential in many of today's apps. Until now, though, most developers using Visual Basic® have had to buy third-party components to get an effective, easy to use grid component. The Windows® Forms DataGrid turned out to lack too many features for the average developer. Third-party grid controls are often more feature-rich than anything found in a system framework like the Microsoft® .NET Framework. But for in-house and personal applications that don't need a professional quality control, the new DataGridView in the forthcoming .NET Framework 2.0 is a viable alternative.

The DataGridView control is the successor to the Windows Forms DataGrid version 1.x. Once you have dropped the control onto a Windows Form, you'll see the popup window shown in Figure 1. You can declaratively set a number of features—add new rows, edit and delete the current row, reorder columns, bind the control to its data, and style it as needed.

Figure 1 DataGridView Smart Tag
Figure 1 DataGridView Smart Tag

The DataGridView class allows customization of cells, rows, columns, and borders through properties such as DefaultCellStyle, ColumnHeadersDefaultCellStyle, CellBorderStyle, and GridColor. This alone exceeds the capabilities of the Windows Forms DataGrid version 1.x.

You can use a DataGridView control to display data with or without an underlying data source. If you don't specify a data source, you can create columns and rows that contain data and add them directly to the DataGridView using the control's user interface. Alternatively, you can set the usual DataSource and DataMember properties, bind the control to a data source, and have it automatically populated with data.

The DataGridView eliminates the most commonly reported snags of the previous version, including data caching. In version 1.x, there are no special capabilities for working with a very large amount of data. You have to either bind the control to the whole DataSet or implement a hand-written caching system. In the .NET Framework 2.0, the DataGridView control can work in virtual mode. Setting the VirtualMode property to true enables the control to display a subset of the available data (you can also use VirtualMode to provide a mix of bound and unbound columns).

You create a DataGridView with a set number of rows and columns and then handle the CellValueNeeded event to populate the cells. Virtual mode requires the implementation of an underlying data cache to handle the population, editing, and deletion of DataGridView cells based on actions of the user. As you can see, the mechanism is not entirely automatic but is much simpler than in version 1.x.

Data binding takes advantage of some new IDE features available in Visual Studio® 2005. As you can see in Figure 2, any data-bound Windows control can be bound to a variety of data sources—server databases such as SQL Server, local databases like Microsoft Access or SQL Server™ 2005 files, Web service methods, or custom business objects. The object that gets bound to the control is always a collection; the preceding data sources are just the storage media. Visual Studio 2005 generates classes and code to contain bindable data. In doing so, it requires you to select the data entities to use from the selected source.

Figure 2 Data Source Config Wizard
Figure 2 Data Source Config Wizard

Figure 3 shows the dialog box for picking up a table from a Microsoft Access database. Around that table, Visual Studio 2005 would construct an XSD schema file to represent a typed DataSet. In addition, the typed DataSet would be empowered with additional methods to load data on demand, optionally in parameterized way.

Figure 3 Declaring the Data Source
Figure 3 Declaring the Data Source

The wizard and the options shown in Figure 2 and Figure 3 might change before the .NET Framework 2.0 ships. That's why I'll just give you the big picture here. To show you the new DataGridView control in action, I'll examine a handwritten data layer in which ADO.NET classes retrieve any data and custom collection objects convey data to the grid control.

Building a Simple Data Layer

A key enhancement in the .NET Framework 2.0, generic types are especially useful for creating custom collections in a snap. In .NET, generics are programming elements that can work with a variety of data types. You specify the data type when you declare the element. Among other things, generics can be used to create strongly typed collections. Here's an example:

Public Class EmployeeCollection
       Inherits Collection(Of Employee)
End Class

Suppose you want to create and bind an array of objects representing employee data. You load database records from an Employee table into an instance of the Employee class and group employees together within a custom collection (shown in Figure 4). To create a custom collection in .NET Framework 1.x, you derive a new class from CollectionBase and override a couple of members—at the very minimum, the indexer property Item and the method Add. This action requires you to write a great deal of code, and the actual collection is only apparently typed. In reality, the custom collection is an ArrayList object, and anything you add to it, or get from it, is cast to the proper type before use.

Figure 4 A Pretty Simple Data Layer (Close Figure 4)
Imports System.Collections.Generic
Imports System.Data
Imports System.Data.OleDb

' Class designed to hold employee data
Public Class Employee
    Private _id As Integer
    Private _name As String
    Private _title As Integer
    Private _country As Integer

    Public Property ID() As Integer
        Get
            Return _id
        End Get
        Set(ByVal value As Integer)
            _id = value
        End Set
    End Property
    ... // other properties similar
End Class

' Collection of customers
Public Class EmployeeCollection : Inherits BindingList(Of Employee)
End Class

Public Class DataHelper
    Public Function Load() As EmployeeCollection
        Dim ds As New DataSet
        Dim adapter As New OleDbDataAdapter("SELECT * FROM Employees", _
            MySettings.Value.Connection)
        adapter.Fill(ds)

        Dim rows As New EmployeeCollection
        For Each row As DataRow In ds.Tables(0).Rows
            Dim e As New Employee
            e.ID = Int32.Parse(row("employeeid"))
            e.Name = row("lastname").ToString()
            e.Title = Int32.Parse(row("title"))
            e.Country = Int32.Parse(row("country"))
            rows.Add(e)
        Next

        Return rows
    End Function
End Class

Generics provide a dual advantage when you're moving data across tiers. You write much less code, and the code you do write is easier to maintain. Performance is significantly improved because no boxing is required for value types and no cast is performed in order to access the object in a strongly typed fashion.

Figure 4 shows a simple data layer expressed as a custom collection and a load method. The data is captured using ADO.NET classes and then placed into individual Employee objects for binding to the grid. Should you go with the "manual" approach described here, or are you better off exploiting the design-time capabilities of Visual Studio 2005? Get the best of both worlds: compose all of your low-level ADO.NET code into a class with a few public methods to perform the usual database operations. Next, bind this intermediate class to the control using the Object option shown in Figure 2.

The following code snippet shows how to fill a EmployeeCollection object and bind it to an instance of the DataGridView control:

Dim loader As New DataHelper()
Dim data As EmployeeCollection = loader.Load()
DataGridView1.DataSource = data
Figure 5 shows the grid in action. It doesn't look much different from an old-style DataGrid control.

Figure 5 The DataGridView Control in Action
Figure 5 The DataGridView Control in Action

Customizing the DataGridView Control

Can you spot a drawback in the grid shown in Figure 5? The titles and countries are listed through numbers instead of more intelligible names. On the other hand, those numbers are the visible sign of a sensible data schema. Those numbers, in fact, represent foreign keys from cross-referenced tables. From the database design point of view, so far so good, but what about grid rendering? You need to transform those numbers into names—the country, the title, or whatever else they might stand for. In the .NET Framework 1.x, you first create a custom column type through an ad hoc class and then you add it to the grid's column collection. The DataGridView control has both enhanced capabilities and a richer set of built-in column types.

Figure 6 Editing the Columns of the DataGridView Control
Figure 6 Editing the Columns of the DataGridView Control

Figure 6 shows how to change the column type and all the choices available. By default, a grid column is rendered using a textbox column just as in version 1.x. To effectively render foreign keys, you need a combobox column bound to a distinct data source and a different pair of display/value members. The dialog in Figure 6 lets you change the column type to DataGridViewComboBoxColumn. Once you've done so, you can add a few extra properties to the column class to fill the combobox with both bound and unbound values. For example, you can set the DataSource property of the DataGridViewComboBoxColumn to the object that contains the bound items—all the titles and the countries to choose from. The DisplayMember property indicates the field to display through the combobox; ValueMember indicates the field to get and set the real value on the master table. Figure 7 illustrates the result.

Figure 7 Managing Foreign Keys in a DataGridView Control
Figure 7 Managing Foreign Keys in a DataGridView Control

In read-only mode, the combobox doesn't drop down and is limited to showing the bound value. (You can also make use of the combobox column's DisplayStyle property here to hide the drop-down arrow. This makes the UI look even better.) In edit mode, use the combobox to select the new value for the cell. If you have to change the country cell, you can pick up the new value from a list of country names rather than from a meaningless sequence of numbers.

In addition to combobox and classic textbox columns, you can also have image, link, and button columns. Link and button columns display their items as clickable elements. Each click on the contents of an item is bubbled up as a CellContentClick event.

Each column can have its own set of visual properties as shown in Figure 8. You can control padding, colors, fonts, and borders. Column settings can be set declaratively, programmatically on a per-column basis, or for all the columns at once through the DefaultCellStyle property.

Figure 8 Setting the Styles of a Column
Figure 8 Setting the Styles of a Column

The following code snippet sets the background color for alternating rows to override the default background set for all cells. In addition, it sets the Format property on the HireDate column to cause the bound date object to be formatted as "Month, Year."

grid.DefaultCellStyle.BackColor = Color.LightGray;
grid.AlternatingRowsDefaultCellStyle.ForeColor = Color.White;
grid.AlternatingRowsDefaultCellStyle.BackColor = Color.Black;
grid.Columns["HireDate"].DefaultCellStyle.Format = "y";

The width of each column can be set in a relative manner, unlike in Windows Forms 1.x grids. Aside from assigning a column a given number of pixels, you can require that the column determine its width automatically and adjust it based on the displayed data. The property AutoSizeCriteria can be set to any of the following values: None, HeaderOnly, Rows, HeaderAndRows, and HeaderAndDisplayedRows. HeaderOnly and Rows indicate that the column should be wide enough to display the entire header or the largest row. HeaderAndRows takes into account the width of the header and that of the largest row. Finally, HeaderAndDisplayedRows considers only the displayed rows and the header.

Cell Templates

If the column types supported out-of-the-box by the new Windows Forms DataGridView control don't meet your needs, you can create your own column types with cells that host controls of your choice. Creating custom column types is possible in version 1.x; it's as easy as deriving a class from a known base class. In the .NET Framework 2.0, things are even easier because of a new infrastructure for templating.

One thing you won't find in the Beta 1 list of column types—the set is subject to change by the time the platform ships—is a calendar column to display and edit dates in an easy fashion. To create a calendar column, you must define a class that derives from DataGridViewColumn and extend it with a custom cell template. The cell template is the area that is occupied by individual grid cells. The content of the cell template depends on the user interface of the column. Link columns contain a hyperlink, standard columns include a textbox, combo columns make up their user interface with a combobox. A custom column populates the cell template with any combination of Windows controls. The cells of this column display dates in ordinary text cells, but when the user edits a cell, a DateTimePicker control appears, as in Figure 9. Let's see how to create a calendar column.

Figure 9 The Custom Calendar Column in Action
Figure 9 The Custom Calendar Column in Action

In Figure 10, the CalendarColumn class inherits from DataGridViewColumn and overrides the CellTemplate property. The CellTemplate property returns a DataGridViewCell object that represents the content of the cell. As Figure 10 demonstrates, the CalendarColumn constructor uses a made-to-measure cell object—the CalendarCell class.

Figure 10 Custom Calendar Column Class (Close Figure 10)
Public Class CalendarColumn : Inherits DataGridViewColumn

    Public Sub New()
        MyBase.New(New CalendarCell())
    End Sub

    Public Overrides Property CellTemplate() As DataGridViewCell
        Get
            Return MyBase.CellTemplate
        End Get
        Set(ByVal value As DataGridViewCell)
            If Not (value Is Nothing) And _
              Not value.GetType().IsAssignableFrom( _
              GetType(CalendarCell)) Then
                Throw New InvalidCastException("Must be a CalendarCell")
            End If
            MyBase.CellTemplate = value
        End Set
    End Property
End Class

CalendarCell derives from the DataGridViewTextBoxCell class to avoid having to reimplement textbox display functionality. Alternatively, you can make it inherit from DataGridViewCell. The code for the CalendarCell class is split in two parts. One includes public properties such as the format of the date. The other portion of the code is the view editing control; that is, a class that derives from Control and implements the IDataGridViewEditingControl interface. This control provides the user interface when a particular cell enters into edit mode. For the CalendarCell, this user interface consists of a DateTimePicker control. For more details, take a look at the source code available for download from the link at the top of this article.

How can you use this new column with a DataGridView control? Once you bring the new column class into the project, it automatically becomes visible in the dropdown list of available column types. You can assign the calendar type to a column declaratively in the Visual Studio 2005 designer or proceed programmatically, as shown here:

Dim col As New CalendarColumn()
col.HeaderText = "Hire Date"
col.DataPropertyName = "HireDate"
Me.DataGridView1.Columns.Add(col)

The date picker control shows up only if the grid is working in edit mode. If the column is marked as read-only, the date picker remains hidden and the cell displays only static text.

The Data Connector

In the code download for this article, you'll see that the sample DataGridView is bound to a particular type of data source. A data source can be an enumerable data object or a data connector. The DataGridView accepts any of the following interfaces: IList, IListSource, IBindingList, IBindingListView.

IList and IListSource are old acquaintances to developers experienced in .NET. They are implemented by collections and special ADO.NET objects such as DataTable and DataSet. IBindingList has also been around since the .NET Framework 1.0 and derives from IList, adding support for change notification, AddNew semantics, and sorting (the new BindingList<T> generic class implements IBindingList). IBindingListView, new to the .NET Framework 2.0, further extends the IBindingList interface to add advanced sorting and filtering capabilities. The DataConnector class implements the IBindingListView interface.

In Windows Forms 2.0, the DataConnector component, renamed to BindingSource in Beta 2, is the data source object of choice; through it you can bind to a variety of data sources. You can bind the control to a data connector and have the connector component, in turn, bind to another data source or populate with data using a business object. The DataConnector binds to a physical data source through the DataSource and DataMember properties, as shown in the following code:

Me.EmployeesDataConnector.DataMember = "Employees"
Me.EmployeesDataConnector.DataSource = Me.MyEmployeesDataSet

When you connect a data source to a DataGridView control, Visual Studio 2005 creates three elements—the data connector, a data adapter, and a typed DataSet. The data adapter executes any query that fills the typed DataSet. The typed DataSet, in turn, is wrapped by the connector and populates the bound control.

The data connector provides a layer of abstraction between a Windows form and its data. All further interaction between form and data, including navigating, sorting, filtering, and updating, is performed through calls to the DataConnector.

As mentioned, a DataConnector encapsulates data and provides members for accessing that data. The Current property retrieves the current item; the property List retrieves the entire list. Currency management is handled automatically, but a number of related events are exposed anyway to allow for further customization. These events include CurrentItemChanged, CurrentChanged, DataSourceChanged, ListChanged, and BindingComplete.

It is worth noting that data sources bound to a DataConnector component can also be managed through the DataNavigator class. DataNavigator offers a VCR-like user interface for navigating the items of a list and performing a few common operations (Insert new, delete, update) on the currently selected record.

Sorting

The DataGridView class provides advanced sorting capabilities. Each column is given a SortMode property that determines whether the column is sortable or not. By default, all columns are sortable. To change this, set the SortMode property of a given column to NotSortable. You can do that both programmatically and declaratively at design time. The SortMode property also accepts two other values: Automatic (the default) and Programmatic. In both cases, sorting takes place in the default way, alphabetically. If automatic sorting is enabled, the sort direction glyph is displayed; otherwise, you have to show it yourself.

If the grid is configured for programmatic sorting, you have to rely on the sorting capabilities of the data source object in a data-bound scenario. If you're using a grid not bound to any data source, then you need to take care of the sorting yourself through the Sort method, shown here:

Dim titleComp As New TitleComparer
DataGridView1.Sort(titleComp)
The Sort method performs its magic by taking an instance of a class that implements the IComparer interface and determines the new order by comparing pairs of values.

Conclusion

All data-driven Windows and Web applications require a grid component. Grids need a good deal of extra functionality that the Windows Forms DataGrid control you get from the .NET Framework 1.x lacks. In the .NET Framework 2.0, the DataGridView control makes up for some of the limitations of the previous version. You'll find an improved user interface with more column types, more customization options, and even a virtual working mode that gets data only when required.

In this column I've only just scratched the surface of the DataGridView control with a sneak preview of the features that will be available with the next version of the .NET Framework. Note that the code and the discussion is based on the Beta 1 release and therefore all of the details are subject to change before the final release. In addition, you can almost certainly expect more features by the time the new version of the Windows Forms framework ships. Stay tuned.

Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is a Wintellect instructor and consultant based in Italy. Author of Programming ASP.NET and the new book Introducing ASP.NET 2.0 (both from Microsoft Press), he spends most of his time teaching ASP.NET/ADO.NET classes and speaking at conferences. Get in touch with Dino at cutting@microsoft.com or join the blog at weblogs.asp.net/despos.


Test Run: Lightweight UI Test Automation for ASP.NET Web Apps

Test Run
Lightweight UI Test Automation for ASP.NET Web Apps
James McCaffrey

Code download available at: TestRun0504.exe (116KB)
Contents
The Web App Under Test
The Test Automation
Extending and Adapting the Automation
Wrap-Up

The release of ASP.NET revolutionized Web development and made it easy to create full-featured Web applications. Visual Studio® 2005 and ASP.NET 2.0 will let you add even more functionality to your applications, but the more features a Web application has, the more important testing becomes.

In this column I'll show you how to create lightweight test automation that can verify the functionality of your ASP.NET Web application through its user interface. To get a better idea of where I'm headed, take a look at the screen shots in Figure 1 and Figure 2. Figure 1 shows a simple but representative ASP.NET Web application. To use it, the user enters a color in a textbox control and clicks the Submit button. Information is sent to the Web server, some processing is done, and the Web application's state is updated.

Figure 1 The Web Application to Test
Figure 1 The Web Application to Test

Of course, your real Web application will be much more complex, but the techniques I'll show you can be employed to test complex applications as well. Having to test even this simple Web app manually through its user interface would be tedious and inefficient. And in a development environment you'd have to repeat the testing every time there was a change in your application's code base. A much better approach is to write test automation that simulates a user's actions. Figure 2 shows a screenshot of a test harness based in JScript® that does just that.

Figure 2 Sample Test Run
Figure 2 Sample Test Run

If you examine Figure 2 you'll see that the automation tests the Web application through its user interface. The harness loads the application under test into an HTML frame (on the right) and uses a second frame (on the left) to hold a JScript test script that manipulates the app, determines if the app's functionality is correct, and logs a pass or fail result.

As you'll see, the test automation is easy to create, is applicable to a wide range of Web applications, and works with both traditional ASP and ASP.NET Web applications. In the sections that follow I will briefly examine the Web application under test so you'll know exactly what is being tested and how, and I'll explain in detail the test harness that generated the image in Figure 2. Then I'll discuss how you can extend the techniques presented here to meet your own needs.

The Web App Under Test

Let's take a look at the Web application under test so you can understand the goal of the test automation. Figure 1 shows the initial state of the Web app. After the user types in a color like "red" into the textbox control and clicks the Submit button, an HTTP request is sent to the Web server. The server processes the request and generates an HTTP response that contains a comment of some sort and an additional textbox control for the user to enter a second color. After another round-trip to the Web server, a second comment is displayed and an "All done!" message is placed in the document body. After a user types "red," submits, types "blue," and then submits again, the Web application will look like the right-hand frame in Figure 2.

If you were to run this sample, you'd notice that the test harness simulates typing "red," and then typing "blue" in the newly visible textbox control. Then the harness checks if the comment reads, "The sea is blue" and if "All done!" is displayed in the page body to determine if the test scenario has passed or failed. It then logs the result to the results.txt text file.

In order to create my test automation I need to examine the Web app's code. The entire code is listed in Figure 3. I used C# for the Web application logic code but the technique discussed in this column does not depend on your language. To keep the code as short as possible for the column, I created the ASP.NET Web application using an ordinary text editor rather than using Visual Studio .NET as I normally would.

Figure 3 Web Application Under Test (Close Figure 3)
<script language="c#" runat="server">
  private void Button1_Click(object sender, System.EventArgs e)
  {
    if (TextBox1.Text == "red" && TextBox2.Visible == false)
    {
      TextBox3.Text = "Roses are red";
      Label2.Visible = true;
      TextBox2.Visible = true;
    }
    else if (TextBox1.Text == "blue" && TextBox2.Visible == false)
    {
      TextBox3.Text = "The sky is blue";
      Label2.Visible = true;
      TextBox2.Visible = true;
    }
    else if (TextBox2.Visible == false)
    {
      TextBox3.Text = "Unknown color";
      Label2.Visible = true;
      TextBox2.Visible = true;
    }
    else if (TextBox2.Text == "red")
    {
      TextBox3.Text = "Some apples are red";
      Label3.Visible = true;
    }
    else if (TextBox2.Text == "blue")
    {
      TextBox3.Text = "The sea is blue";
      Label3.Visible = true;
    }
    else
      TextBox3.Text = "Try again";
  } // Button1_Click()
</script>

<html>
  <head><title>color.aspx</title></head>
  <body bgColor="#ccffcc">
    <h3>ASP.NET Web Application under Test</h3>
    <form method="post" name="theForm" id="theForm" runat="server">
      <p><asp:Label id="Label1"
        runat="server">Enter a first color:&nbsp&nbsp</asp:Label>
      <asp:TextBox id="TextBox1" runat="server" />
      <p><asp:Label id="Label2" visible="false"
        runat="server">Enter a second color:&nbsp&nbsp</asp:Label>
      <asp:TextBox id="TextBox2" runat="server" visible="false"/>
      <p><asp:Button id="Button1" runat="server"
        text="Send" onclick="Button1_Click" />
      <hr>
      <p>My comment:&nbsp&nbsp<asp:TextBox id="TextBox3" runat="server" />
    </form>
    <p><asp:Label id="Label3" runat="server"
        visible="false">All done!</asp:Label>
  </body>
</html>

Let me emphasize that I am purposely ignoring good coding style here because I want to keep the application example short. Plus, it simulates the raw nature of a typical prerelease application. The Web application's initial state has a prompt for the user to enter a color and a textbox control to accept the input. After the user submits the HTTP request, the application logic displays a comment and dynamically generates a second prompt and textbox control by modifying the textbox's Visible property. This is obviously rather artificial but makes explicit the idea that your ASP.NET Web application's state changes with each HTTP request-response pair. In other words, even if the Web application you want to test accesses a SQL Server™ database or does very complex processing, it still just changes state and will be reflected in the resulting HTTP response and in the user interface.

The Test Automation

The test harness system consists of three files. Figure 4 shows a block diagram of the overall structure. The Web application under test (color.aspx) is loaded into an HTML frame, and an HTML page (left.html) that contains the test automation JScript code is loaded into another HTML frame. The third Web page (main.html) is a container for the two frames and holds a global variable called timesAppLoaded which tracks how many times the Web application under test has been loaded into its test frame. The test scenario script uses the timesAppLoaded value to determine exactly what actions to take on the Web app and then reloads the Web application. The cycle continues until the system reaches a final state that you specify in the test script, and a pass or fail result is determined.

Figure 4 Test Harness System
Figure 4 Test Harness System

You saw the Web application's code in the previous section, but to test it using the technique described in this column you do not have to modify or instrument the application code—a nice feature of this technique. The code for main.html is extremely short, as you can see in Figure 5.

Figure 5 Main.html (Close Figure 5)
<html>
  <head>
    <script language="JScript">
      var timesAppLoaded = 0;
      var scenarioID = "001";
    </script>
  </head>

  <frameset cols="40%,*">
    <frame src="left.html" name="left">
    <frame src="color.aspx" name="right" onload="left.updateState();">
  </frameset>
</html>

The key to my test automation technique is keeping track of the Web application's state so I can synchronize the test harness with application events. I also declare and assign a variable named scenarioID. In a production environment you can add other test harness metadata into this section of code.

In main.html I named the two frames "left" and "right" to reflect their position relative to each other in the test harness from a tester's point of view. Notice that every time the color.aspx Web application is loaded or reloaded into the test harness right frame, a function called updateState in the left frame is called. As you'll see later, updateState initiates the simulated user's actions on the Web application that is under test.

The heart of the test harness is the left.html file (see Figure 6). I will walk you through it in detail so that you'll be able to modify it to meet your own needs. Although my JScript code is organized into function blocks, it is relatively tightly coupled, which is usually not quite as good as a loosely coupled design. This design tradeoff keeps my code shorter, however. You may want to parameterize the code to make it a little more flexible.

Figure 6 Left.html (Close Figure 6)
<html>
  <head>
    <script language="JScript">

    function updateState()
    {
      parent.timesAppLoaded++;
      if (parent.timesAppLoaded > 1)
        runScenario();
    } // updateState()

    function runScenario()
    {
      try {
        if (parent.timesAppLoaded == 1)
        {
          addComment("Setting TextBox1 to 'red'");
          addComment("Clicking submit button");
          parent.right.document.theForm.TextBox1.value = "red";
          parent.right.document.theForm.Button1.click();
        }
        else if (parent.timesAppLoaded == 2)
        {
          addComment("Setting TextBox2 to 'blue'");
          addComment("Clicking submit button");
          parent.right.document.theForm.TextBox2.value = "blue";
          parent.right.document.theForm.Button1.click();
        }
        else if (parent.timesAppLoaded == 3)
        {
          addComment("Checking app state");
          checkFinalState();
        }
      }
      catch(e) {
        addComment("Unexpected fatal error: " + e);
      }
    } // runScenario()

    function checkFinalState()
    {
      var pass = true;

      if (parent.right.document.theForm.TextBox1.value != "red")
        pass = false;

      if (parent.right.document.theForm.TextBox2.value != "blue")
        pass = false;

      if (parent.right.document.theForm.TextBox3.value !=
           "The sea is blue")
        pass = false;

      var trange = parent.right.document.body.createTextRange();

      if (trange.findText("All done!") == false)
        pass = false;

      addComment("Determining pass / fail");
      if (pass == true)
        result.value = " Pass ";
      else
        result.value = " *FAIL* ";

      addComment("Saving to 'results.txt'");
      saveResults();

      addComment("End test scenario");
    } // checkFinalState()

    function addComment(comment)
    {
      var currComment = document.all["comments"].value;
      var newComment = currComment + "\n" + comment;
      document.all["comments"].value = newComment;
    } // addComment()

    function saveResults()
    {
      var fso = new ActiveXObject("Scripting.FileSystemObject");
      var f = fso.CreateTextFile("C:\\Results\\results.txt", 2, true);
      f.WriteLine("Scenario = " + parent.scenarioID);
      if (result.value == " Pass ")
        f.WriteLine("Result = Pass");
      else
        f.WriteLine("Result = FAIL");
      f.Close();
    } // saveResult()

    </script>
  </head>

  <body bgColor="#ee7755">
    <h3>Test Scenario Script</h3>
    <p><input type="button" value="Run Scenario"
      onclick="runScenario();"> </p>
    <p><textarea id="comments" rows="10" cols="27">Actions:</textarea></p>
    <p>Scenario Result = <input type="text" name="result"></p>
  </body>

</html>

Let's suppose that you launch Microsoft® Internet Explorer and request main.html. Main.html will load and in turn load its two frames containing the test script in file left.html and the colors.aspx Web application. Figure 7 shows what the test system will look like at this point.

Figure 7 Test Harness Initial State
Figure 7 Test Harness Initial State

The variable timesAppLoaded is initialized to 0 and control is immediately transferred to the updateState function in left.html:

function updateState()
{
  parent.timesAppLoaded++;
  if (parent.timesAppLoaded > 1)
    runScenario();
} // updateState()

The variable timesAppLoaded is next incremented by 1. Notice that because timesAppLoaded is declared in main.html I have to use the JScript keyword "parent" to access it. Next, the function updateState checks to see if the value of timesAppLoaded is greater than 1. Because this is the initial load, timesAppLoaded is not greater than 1 so the thread of execution ends.

The body portion of the test harness just consists of a button to start the automation, an HTML <textarea> to display comments describing the progress of the automation, and an HTML text field to display the final test scenario result:

<body bgColor="#ee7755">
  <h3>Test Scenario Script</h3>
  <p><input type="button" value="Run Scenario" onclick="runScenario();"> </p>
  <p><textarea id="comments" rows="10" cols="27">Actions:</textarea></p>
  <p>Scenario Result = <input type="text" name="result"></p>
</body>

Now suppose you click on the Run Scenario button to initiate the test automation. Control is transferred to the runScenario function, which immediately checks the value of variable timesAppLoaded and branches if that value is 1:

if (parent.timesAppLoaded == 1)
{
  addComment("Setting TextBox1 to 'red'");
  addComment("Clicking submit button");
  parent.right.document.theForm.TextBox1.value = "red";
  parent.right.document.theForm.Button1.click();
}

The harness code calls a helper function, addComment, that displays some comments into the harness <textarea> comments field. Then the harness sets the value of the Web app's TextBox1 control to "red" and invokes the Web app's Submit button. This will trigger an HTTP request to color.aspx where the comment "Roses are red" will be generated and a second prompt and textbox will become visible to the test harness. The Web app will accept the resulting HTTP response, reload into its frame, which in turn will fire an onload event, and call the updateState function again.

Function updateState increments the global variable timesAppLoaded to 2 and in turn calls the runScenario function again. This time through, the runScenario function will branch when timesAppLoaded equals 2:

else if (parent.timesAppLoaded == 2)
{
  addComment("Setting TextBox2 to 'blue'");
  addComment("Clicking submit button");
  parent.right.document.theForm.TextBox2.value
     = "blue";
  parent.right.document.theForm.Button1.click();
   }

Some new comments are added to the test harness comment area, the now-visible TextBox2 control is given a value of "blue," and the Submit button click event is simulated. This sends an HTTP request to the Web server, a new comment "The sea is blue" is generated, and an "All done!" message is written into the HTTP response page body. The HTTP response reloads into its frame in the test harness, which once again fires an onload event and calls the updateState function.

The updateState function increments timesAppLoaded to 3 and calls the runScenario function, which branches for the last time:

else if (parent.timesAppLoaded == 3)
{
  addComment("Checking app state");
  checkFinalState();
}
After adding a comment, the checkFinalState function is called. This function examines the Web application to see if the correct final state exists. I declare a variable named pass and initialize it to true, the idea being that I will assume the Web application's state is correct and scan to see if anything is wrong. First I check the three TextBox controls:
var pass = true;

if (parent.right.document.theForm.TextBox1.value != "red")
  pass = false;

if (parent.right.document.theForm.TextBox2.value != "blue")
  pass = false;

if (parent.right.document.theForm.TextBox3.value != "The sea is blue")
  pass = false;

Notice that because the harness code is in a different frame from the code for the app under test, I must use the keyword "parent" and the frame name "left" to access the Web app's data by element ID. I have to use a slightly tricky technique to examine the text in the body of Web application:

var trange = parent.right.document.body.createTextRange();

if (trange.findText("All done!") == false)
  pass = false;

The TextRange object is a very useful part of the Internet Explorer Document Object Model (DOM) and allows you to examine and manipulate those parts of a Web page outside of an HTML element. Now I can determine whether the test scenario passed or failed and save my results to an external file using a saveResults helper function, as shown here:

addComment("Determining pass / fail");
if (pass == true)
  result.value = " Pass ";
else
  result.value = " *FAIL* ";

addComment("Saving to 'results.txt'");
saveResults();

addComment("End test scenario");

The helper functions addComment and saveResults do not use any special tricks. The saveResults function uses ActiveX® technology to write results directly to an external file. The advantage of this technique is that it is very simple. A disadvantage is that by default Web pages cannot write to files for security reasons. So, I had to add the test harness page to the list of Internet Explorer Trusted Sites (if you examine Figure 2 you'll see the Trusted Sites icon in the Internet Explorer status bar) and I had to enable ActiveX objects not marked as safe to avoid getting a warning message when saveResults is called. I'll discuss two alternatives to this in the following section of this column.

Extending and Adapting the Automation

You can extend and adapt the lightweight ASP.NET Web application UI test automation technique I've presented here in several ways. Notice that the test harness is not truly automated because you have to manually click on the Run Scenario button to launch the automation. If you want to fully automate the harness, all you have to do is change the code in updateState function from

if (parent.timesAppLoaded > 1)
  runScenario();
to
if (parent.timesAppLoaded > 0)
  runScenario();
and create a batch file with the single statement:
iexplore.exe  http://localhost/WebUIAutomation/main.html

Recall that when main.html loads it will set the value of the global variable timesAppLoaded to 0 and then load the application under test (in this column colors.aspx) which will cause updateState to be invoked, incrementing timesAppLoaded to 1. If you specify timesAppLoaded > 0, this will be immediately true and cause runScenario to be called, initiating the test scenario. By doing this, you can schedule the BAT file and hence the test harness to run automatically (for example, using the Windows Task Scheduler or from an automated build process) and automatically send test-run result summaries via e-mail.

The code I've presented manipulates the Web application under test through several states and then checks the final state to determine if the scenario passes or fails. An alternative is to check the state of the app after each state change. I've used both techniques in the past and have found that the best method depends entirely on the testing situation you're facing.

Testing ASP.NET app functionality through the UI is just one type of testing you'll need to perform. Performance testing, stress testing, security testing, and testing at the HTTP request/response level (see Test Run: Test Automation for ASP.NET Web Apps with SSL), are also critical. You can modify this technique to handle many of these testing situations. Load-testing tools usually accept a list of Web pages to navigate to, so by adding a test scenario page that simulates a user navigating to multiple page states, you'll get a very realistic profile.

The technique presented here calls a saveResults function that uses methods in Scripting.FileSystemObject to write results to an external file. But this means you have to modify the Trusted Sites list in Internet Explorer and the ActiveX control scripting execution properties. If you do this, you'll invalidate many kinds of security testing scenarios. There are two alternatives. First, you can add an HTML form to the test-automation script, then use the form to send the test scenario results to an ASP.NET page that reads the form pass/fail data, and writes it to external storage (text file, XML file, SQL table, and so on). Second, you can write results to a cookie on the test client machine and then create a small JScript helper function that reads and parses the cookie and saves the test scenario result to external storage.

Wrap-Up

Because this column is primarily instructional, for clarity I used only minimal error checking. In a production system you'll want to properly implement error and exception handling in your test harness to make sure your automation does not stop in the middle of the run. I also hardcoded most of the information in the test harness. You may want to parameterize some of the data, such as the simulated user inputs, to make your test system more flexible.

With Web application systems growing in complexity and with security becoming increasingly important, testing your software is more vital than ever before. Lightweight ASP.NET Web application UI testing as I've described here is an important part of your product's testing effort and the tools provided by Microsoft make this kind of testing simple to implement.

Send your questions and comments for James to testrun@microsoft.com.

James McCaffrey works for Volt Information Sciences Inc., where he manages technical training for software engineers working at Microsoft. He has worked on several Microsoft products including Internet Explorer and MSN Search. James can be reached at jmccaffrey@volt.com or v-jammc@microsoft.com.


Service Station: Developing .NET Web Services with Beta 2

Service Station
Developing .NET Web Services with Beta 2
Aaron Skonnard

Contents
Framework Evolution
Visual Studio 2005
XML Editor and Code Snippets
Whitehorse Designers
Interface-Based Service Contracts
More Core XSD/WSDL Improvements
Improved Standards Support, Interoperability
Wrapping It Up

Version 2.0 of the Microsoft® .NET Framework makes numerous improvements at various levels in the Web services protocol stack. In addition, better tool support and an increased focus on interoperability make your life easier. In this column I'll walk you through several of the major Web services tool improvements in Visual Studio® 2005 Beta 2, and introduce you to a new interface-based approach to defining service contracts. Then I'll discuss improved interoperability results across frameworks. But first, let me briefly explain how these changes mesh with what's going on in the Web Services Enhancements (WSE) and "Indigo" spaces.

Indigo is the code name for the upcoming set of .NET technologies for building and running connected systems that will be part of the next version of Windows and available as an add-on to Windows® XP. Note that, as usual, since this column discusses products that have not yet been released, implementation details may change in the future.

Framework Evolution

You can think of WSE as its own stack because you install it separately from the .NET Framework and you can use it as a standalone (you don't need ASMX to use WSE, and vice versa). You can, however, use WSE to enhance the behavior of ASMX with support for the WS-* implementations, giving you a hybrid of the two stacks, which happens to be the sweet spot today. Service Pack 2 for WSE 2.0 was recently released and is downloadable from the MSDN® Web Services Developer Center.

The choice between these stacks can be confusing for many developers. The guidance from Microsoft is to use ASMX by default, and to enhance ASMX with WSE 2.0 when you need message-level security. When you take this approach, you're still using the ASMX programming model; you're simply introducing security functionality through the ASMX SoapExtension framework (think pre- and post-processing), and when using policy, without making changes to your code (see Figure 1).

Figure 1 Evolution of Web Services Frameworks
Figure 1 Evolution of Web Services Frameworks

Why does Microsoft offer this guidance? The ASMX programming model is more likely to migrate cleanly to Indigo than the alternative WSE 2.0 messaging API will be. You may still find it strategic and beneficial to leverage the WSE 2.0 messaging API in your services today (to use the TCP transport or to do custom hosting, for example), just as long as you're prepared to migrate the code when Indigo becomes available. The primary Indigo programming model is going to be very similar to the ASMX model.

WSE 2.0 will continue to be used to enhance ASMX 2.0 with security and other WS-* implementations it might offer. Unfortunately, WSE 2.0 is not currently compatible with the .NET Framework 2.0 Betas—they should be compatible, however, by the time version 2.0 ships. Also, it's likely that the next version of WSE (version 3.0) will have full ASMX integration, as illustrated in Figure 1, simplifying future migration even more.

Visual Studio 2005

Let's start by looking at some of the new tool features. One of the improvements that you'll notice as soon as you begin developing Web services with Visual Studio 2005 is the move away from the dreaded "Web project" to the new "Web site" model.

Web projects required you to have a virtual directory in place before you could even open the project. This complicated matters when moving projects around, changing directories, or modifying virtual directory names, and so on. With the Web site model, you can simply point Visual Studio to a physical directory on disk and it will use its own Web server (based on the Cassini Web server from Microsoft) to serve up files in that directory over HTTP. This lets you browse to and test ASMX files without having an IIS virtual directory. You can also open an IIS Web site by pointing Visual Studio 2005 to the IIS virtual directory (just like you could with earlier versions).

The new ASP.NET compilation model also simplifies ASMX development. Instead of having to compile your ASMX code into the /bin directory, you can now place your ASMX source files in the /Code directory and they'll be automatically compiled for you. In addition to compiling source files found in the /Code directory, ASP.NET 2.0 also uses custom build providers to process .xsd and .wsdl files placed there. It generates DataSet classes for the .xsd files, which is equivalent to running xsd.exe/DataSet, and proxy classes for each .wsdl file, which is also equivalent to running wsdl.exe.

One of the cool things about this architecture is that you can write custom build providers that plug into the build process, an encouraging environment for third-party tool support. Visual Studio 2005 has impressive support for the /Code directory. When you drop .xsd and .wsdl files in the /Code directory from within the IDE, it automatically makes the generated classes available to you via the Object Browser and IntelliSense®.

Visual Studio .2005 also gives you the option to embed code within the .asmx file or in a codebehind file. And with both techniques you still get IntelliSense and syntax highlighting unlike before where you had no choice but to use codebehind.

XML Editor and Code Snippets

In addition to these core IDE improvements, Visual Studio 2005 also comes with a greatly improved XML editor that fully supports XSD-driven IntelliSense. The new editor is truly amazing. It will automatically provide IntelliSense for the schemas found in C:\Program Files\Microsoft Visual Studio 8\Xml\Schemas or those found in your project. Numerous schemas ship with Visual Studio 2005 including the schemas for XML Schema Definition (XSD), Web Services Description Language (WSDL), SOAP 1.1, SOAP 1.2, XML Signature, XSLT, XHTML, .NET Framework 2.0 configuration files, MSBuild, and code snippets, to name a few. This means you now get IntelliSense when authoring any of these file types—and it actually works correctly, even with namespace scopes and XSD sequencing, and so on.

Visual Studio 2005 Beta 2 will also support XML code snippets and expansions just like the ones for C# or Visual Basic® that shipped in Beta 1. You can define your own XML code snippets or expansions that you commonly use (with the provided schema), manage them using the Code Snippet Manager, and then use them to automate authoring common XML fragments. This is especially interesting in the area of XSD and WSDL, both of which are hard languages to work with unassisted. Microsoft is planning to ship Visual Studio 2005 Beta 2 with numerous XSD and WSDL snippets and expansions for the most common authoring tasks.

For example, typing "oneOpPortType" followed by Tab expands into the following <wsdl:portType> fragment. You simply tab through the highlighted slots and enter the appropriate values, without having to worry about typing in all the angle brackets:

<wsdl:portType name="name" >
  <wsdl:operation name="opname" >
    <wsdl:input message="inputMsg" />
    <wsdl:output message="outputMsg" />
  </wsdl:operation>
  $selection$$end$
</wsdl:portType>

These snippets and expansions should make hand-authoring XSD and WSDL more tractable for the average XML developer. Although they probably won't do much for those of you who are angle-bracket impaired, these enhancements will likely do a great deal of good if you're interested in contract-first development techniques. Check out the Code Snippet Manager to begin playing with the provided snippets and expansions.

Visual Studio 2005 still comes with a graphical XSD designer, but it's still missing the WSDL equivalent. Developers looking for a more user-friendly WSDL authoring tool will have to resort to a third party. However, Visual Studio 2005 does introduce a new application designer as part of the Visual Studio Team System components (code-named "Whitehorse") that comes with some Web service design support.

Whitehorse Designers

The Whitehorse Application Connection Designer (ACD) is a design tool for modeling the components of an application and the connections between them, where the "connections" represent service contracts. You open the ACD in Visual Studio 2005 by selecting File | New Project | Other Project Types | Visual Studio Solutions | Distributed System. You can use this designer for multiple tasks to model new Web service endpoints, to implement existing Web service contracts, or to consume external Web services that you don't control.

In order to model services with the designer, first you drag a Web service onto the design service and begin modeling the endpoint contract before ever writing a line of code. Although you're essentially modeling WSDL by doing this, I don't consider it a WSDL editor because the designer is very limited and restrictive in its approach to WSDL design. It works fine, however, for simple contract definitions.

You then drag other application types (such as ASP.NET or Windows Applications) onto the design surface and connect them to services. This establishes a relationship between the two components based on the service contract definition (see Figure 2).

Figure 2 ACD Designer
Figure 2 ACD Designer

Once you've modeled the entire application, the ACD allows you to generate the skeleton projects for all of the applications, with all of the code needed to implement and consume the defined service contracts. For example, it generates the ASMX classes in the Web service projects (with all the required attributes) and it generates the client-side proxy classes in the consuming applications. If you go back and modify the contract, it will automatically keep all the code synchronized.

The ACD is the first Microsoft design tool to truly head in the direction of contract-first development, but it's still missing a few important features. It makes you model the Web service contract from a method perspective and doesn't let you drop down to design the message formats (XSD definitions) from within the same environment. You have to fight the tool a bit to use a true XSD-first approach. Future versions should improve on these deficiencies, but not until the next version of Visual Studio. Nevertheless, it's still a powerful design tool worth considering today. In a future column, I'll walk through a complete example of contract-first development using Whitehorse.

Interface-Based Service Contracts

Most of the improvements in the .NET Framework 2.0 Web services framework are related to XSD and WSDL support, improved runtime serialization, and related optimizations. One of the newest, and most compelling, features in this area is support for interface-based Web services contracts.

The motivation for this feature is clear. Most distributed systems developers are used to designing their communication contracts using interface definitions, perhaps in the Interface Definition Language (IDL), which can be shared among all parties that need to communicate. The interface definition defines the message formats that go on the wire. Developers can then implement the interfaces in class definitions, assured that their implementations will be able to speak with any implementation that supports the same interface definition.

Contracts in Web services are very much the same, but they're expressed using XSD and WSDL definitions between parties. ASMX 1.1 makes it possible to generate class definitions from a WSDL contract (using wsdl.exe /server), similar to what IDL developers used to do, but since Visual Studio .NET doesn't provide a WSDL contract designer, most developers just start by writing class definitions and derive the WSDL contract from the implementation at run time. This approach mixes the notions of contract and implementation, cluttering the implementation class with numerous WSDL-related attributes that most developers don't even understand. This is dangerous considering that they could innocently modify an attribute without understanding the repercussions to the contract.

When taking a code-first approach, it makes more sense to start by authoring the Web services contract in the form of an interface definition, in an attempt to factor out the contract details from the implementation code. Keeping the contract and implementation separate makes things much clearer to the developer. The interface definition can then be mapped directly to a WSDL portType/binding definition for sharing with other systems that need to use the same contract. Other developers working with the .NET Framework can simply use the interface definition in their code instead of going through the WSDL translation. When developers implement the interface, they're guaranteed to be able to speak with anyone else that implements either the same .NET interface or the translated WSDL portType/binding definition.

Let's look at an example. The following is a traditional .NET interface that models a few weather-related operations:

public interface IWeatherService
{
    string GetForecast(string zip);
    string GetForecastByDate(string zip, DateTime forecastDate);
}
This interface could be the starting point for the contract used by a weather service. In order to make it a Web services contract, you need to annotate the interface with the System.Web.Services attributes (such as [WebServiceBinding] or [WebMethod], for instance). These attributes are used to specify the WSDL portType and binding details (such as XML namespaces, actions values, SOAP styles, and so on), which are not expressible in an unattributed .NET class definition. Here's an example:
[WebServiceBinding(Namespace = "http://example.org/weather")]
public interface IWeatherService
{
    [WebMethod]
    string GetForecast(string zip);

    [WebMethod]
    string GetForecastByDate(string zip, DateTime forecastDate);
}

You must use [WebServiceBinding] on the interface and [WebMethod] on each method you want to expose. You can also use [SoapDocumentMethod] to specify additional details per method, but it's not required. Notice also that I didn't use the [WebService] attribute here. [WebService] must be used on the implementation class, not the interface.

With these attributes in place, there's enough information in the interface definition to generate a WSDL contract (portType/binding combination) that other parties can use. You can implement the WSDL contract by simply implementing the .NET interface in a class definition, as illustrated in Figure 3.

Figure 3 Implement WSDL Contract (Close Figure 3)
[WebService(Namespace = "http://example.org/weather")]
public class WeatherService : IWeatherService
{
    #region IWeatherService Members

    public string GetForecast(string zip)
    {
        ... // implementation goes here
    }

    public string GetForecastByDate(string zip,
       DateTime forecastDate)
    {
        ... // implementation goes here
    }

    #endregion
}

The only attribute you have to use on the implementation is [WebService], which fills in the <wsdl:service> details for the given endpoint. All of the WSDL portType/binding information is inherited directly from the base interface, making the implementation code much cleaner than before.

When using this approach, it's important to note that the interface completely controls the contract. You cannot mix and match attributes across the interface and class definitions. All contract-related attributes must be on the interface definition and you can't add attributes to the class that will result in a modification of the contract defined by the base interface. This means you shouldn't use the XML serialization attributes, [SoapDocumentMethod] or [WebMethod]'s MessageName property on the class when deriving from such an interface.

You can, however, use [WebMethod] on the class to specify runtime behaviors that don't affect the contract, such as BufferResponse, CacheDuration, Description, EnableSession, and TransactionOption. However, when doing so, you still need to use [WebMethod] on the interface definition.

If you don't meet these requirements, the ASMX behavior will be the same as in the .NET Framework 1.1 in which attributes defined on the interface will not show up in the service contract—instead they must all be defined on the class as before.

Support for interfaces is only available on the server, within the service implementation. When you generate proxies using wsdl.exe, it will always generate class definitions, not interfaces. However, wsdl.exe does come with a new command-line switch (/serverInterface) for generating server-side interface definitions (like the one shown in Figure 3) from a WSDL definition. It will generate one interface for each portType/binding combination. It just generates the .NET manifestations of the WSDL contract; it doesn't generate any class definitions (it leaves that part up to you).

Assuming you have an existing WSDL document named weather.wsdl, here's how you'd run the tool:

C:> wsdl.exe /serverInterface weather.wsdl

Then you simply add the generated interfaces to your project and implement them in class definitions. Although I believe /serverInterface will be the preferred method when starting with WSDL, wsdl.exe still supports the /server switch.

The /server switch generates an abstract base class instead of an interface, but as in the .NET Framework 1.1, the attributes are not inherited by derived classes. This approach doesn't provide you with a much better alternative since the attributes have to be manually copied to the derived class. The only difference with the version 2.0 implementation is that /server generates a partial class (another new feature), giving you a little more flexibility in designing your code. A partial class allows you to add helper code in a separate partial class file that will be merged during compilation, which is nice when using /server since it won't be overridden the next time you run wsdl.exe. Figure 4 shows an example of what the generated partial class looks like.

Figure 4 Generated Partial Class (Close Figure 4)
[WebService(Namespace="http://example.org/weather")]
[WebServiceBinding(Namespace="http://example.org/weather")]
public abstract partial class WeatherService2 : WebService
{
    /// <remarks/>
    [WebMethod()]
    [SoapDocumentMethod("http://example.org/weather/GetForecast",
     RequestNamespace="http://example.org/weather",
     ResponseNamespace="http://example.org/weather",
     Use=SoapBindingUse.Literal,
     ParameterStyle=SoapParameterStyle.Wrapped)]
    public abstract string GetForecast(string zip);

    /// <remarks/>
    [WebMethod()]
    [SoapDocumentMethod("http://example.org/weather/GetForecastByDate",
     RequestNamespace="http://example.org/weather",
     ResponseNamespace="http://example.org/weather",
     Use=SoapBindingUse.Literal,
     ParameterStyle=SoapParameterStyle.Wrapped)]
    public abstract string GetForecastByDate(string zip,
      DateTime forecastDate);
}

Starting with WSDL and using /server or /serverInterface is considered the full-fledged contract-first model, which typically increases interoperability success when working across frameworks. The improvements made in this area make contract-first a more realistic possibility for many developers. The interface-based approach to defining service contracts is also very similar to the forthcoming Indigo programming model.

More Core XSD/WSDL Improvements

Most of the remaining changes revolve around improvements to core XSD and WSDL support. The underlying XSD serialization engine is now capable of handling a much wider range of XSD definitions. Before, when the serialization engine didn't understand a certain XSD construct, xsd.exe /classes would fail during code generation. Now, it simply maps unknown constructs to XmlElement fields that can be processed through the DOM API. The serialization engine is also able to support XSD sequencing through the [XmlElement(SequenceId=X)] attribute and the xsd.exe /sequence switch.

The wsdl.exe code generation engine has also been improved to provide a better developer experience. In addition to the /serverInterface addition, it also generates proxy classes that are easier to use, especially when performing asynchronous invocations. And since the code generator now produces properties instead of fields (by default), proxy classes can now be databound for use with form controls. Wsdl.exe also makes it easier to deal with real-world situations in which different proxy classes need to share the same XSD-derived classes (via the /shareTypes switch).

The core serialization engine has also been updated to deal with generics and SqlTypes in order to improve flexibility in code-first experience. Support for Nullable<T>, which maps to XSD's nillable='true', makes it possible to express nullable value types in .NET. This is especially valuable when consuming non-.NET WSDLs where this is common. However, generics are not used by the code generator when generating code from XSD or WSDL.

On top of these improvements, the .NET Framework 2.0 introduces additional flexibility via IXmlSerialiazable for custom serialization and schema importer extensions for custom code generation. It also promises improved performance by supporting compression between proxy and service instances, and by allowing you to pre-generate serialization assemblies using the new sgen.exe XML serialization support utility.

All of these improvements promise to increase interoperability and overall flexibility since the .NET Framework 2.0 can now deal with a wider range of XSD and WSDL documents found in the wild. For more information on these core changes, check out the MSDN Library article "New Features for Web Service Developers in Beta 1 of the .NET Framework 2.0,".

Improved Standards Support, Interoperability

The interoperability commitment from Microsoft is increasingly evident. For example, MSDN recently launched a new Web Services Interoperability site within the MSDN Web Services Developer Center. In conjunction with the new site, MSDN hosted an "Interoperability Month" packed full of webcasts focused on Web services interoperability topics and guidance on specific implementation scenarios. The Web services improvements in the .NET Framework 2.0 also substantiate the Microsoft commitment in this area. They have included support for two major industry standards: SOAP 1.2 and the WS-I Basic Profile(BP) 1.1.

SOAP 1.2 became a W3C Recommendation over a year and a half ago, and it still hasn't made its way into most mainstream Web services frameworks. Since not all frameworks will be able to move at the same time, committing to backward compatibility with SOAP is a must. When you implement an ASMX service in the .NET Framework 2.0, it will automatically support both SOAP 1.1 and SOAP 1.2 by default. Figure 5 shows what the corresponding section of machine.config looks like after installation.

Figure 5 Machine.config (Close Figure 5)
<!-- machine.config -->
<configuration>
     <system.web>
        <webServices>
            <protocols>
              <add name="HttpSoap1.2" />
              <add name="HttpSoap" />
              <!-- <add name="HttpPost" /> -->
              <!-- <add name="HttpGet" /> -->
              <add name="HttpPostLocalhost" />
              <add name="Documentation" />
            </protocols>
        </webServices>
     </system.web>
     ...
</configuration>

You can control which protocols your services support by modifying the Web.config file for the given virtual directory (notice that the HttpPost and HttpGet are now commented out by default). Adding support for SOAP 1.2 in turn increases reach and interoperability since clients can now choose which protocol they want to use with your service.

Server-side support for SOAP 1.2 won't do much good unless it also exists on the client side. Hence, the .NET Framework 2.0 adds support on the client side to facilitate choosing the SOAP protocol version. You can use the /protocol switch to specify the default protocol to use when generating proxy classes using wsdl.exe (if you don't specify the protocol, it defaults to SOAP 1.1). For example, the following command line generates a proxy class that uses SOAP 1.2 by default:

C:> wsdl.exe /protocol:SOAP12 contract.wsdl
The generated proxy class contains an extra line in the constructor that sets the SoapVersion property of SoapHttpClientProtocol, as illustrated here:
[System.Web.Services.WebServiceBindingAttribute(
  Name="POServiceSoap12", Namespace="http://example.org/po/")]
public class POService : SoapHttpClientProtocol {
    public POService() {
      this.SoapVersion = SoapProtocolVersion.Soap12;
      this.Url = "http://localhost/purchasingsystem/poservice.asmx";
    }
    ...

You can set this property at any time when using the proxy class. You can even switch between protocol versions (across different invocations) using the same proxy instance.

The other major interoperability improvement is support for the WS-I BP 1.1. This support is provided in several ways. First, ASMX services conform to the BP 1.1 by default—you have to add attributes and modify the default behavior to violate BP 1.1. However, most developers don't know what changes they can make without breaking conformance. So ASMX also lets you add a "conformance claim" to your implementation class via the [WebServiceBinding] attribute, as shown here:

[WebServiceBinding(ConformsTo=WsiProfiles.BasicProfile1_1,
  EmitConformanceClaims = true)]
public class POService : System.Web.Services.WebService {
    [WebMethod]
    public string HelloWorld() {
        return "Hello World";
    }
}

This is actually the default template you get every time you add a new ASMX Web service to your project. The ConformsTo property states that this implementation is supposed to conform to BP 1.1. When you view the ASMX documentation page for this service, it will attempt to validate this claim. If it notices something in your implementation that violates BP 1.1, it will tell you about it in human-readable form. One way to violate BP 1.1 is to annotate HelloWorld with the [SoapRpcMethod] attribute, which enables SOAP's rpc/encoded style, as shown here:

[WebServiceBinding(
  ConformsTo=WsiProfiles.BasicProfile1_1,
  EmitConformanceClaims = true)]
public class POService : System.Web.Services.WebService {
    [WebMethod]
    [SoapRpcMethod]
    public string HelloWorld() {
        return "Hello World";
    }
}

Since using the SOAP encoding violates BP 1.1, you'd see the page in Figure 6 when browsing to the ASMX page.

Figure 6 Displaying WS-I BP 1.1 Violations
Figure 6 Displaying WS-I BP 1.1 Violations

When making conformance claims in your code, you may also want to emit those claims into the WSDL definition so that consumers know that you're BP 1.1 compliant. The EmitConformanceClaims property controls this. When set to "true", the conformance claim shows up in the WSDL document, as shown here:

<wsdl:binding name="POServiceSoap" type="tns:POServiceSoap">
  <wsdl:documentation>
    <wsi:Claim conformsTo="http://ws-i.org/profiles/basic/1.1"
     xmlns:wsi="http://ws-i.org/schemas/conformanceClaim/" />
  </wsdl:documentation>
  <soap:binding transport="http://schemas.xmlsoap.org/soap/http"
   style="document" />
    <wsdl:operation name="HelloWorld">
With this in place, consumers are able to do their own analysis of your WSDL definition to confirm conformance. Wsdl.exe has been upgraded to do just this. When it sees a conformance claim, it will check conformance and display any problems it finds right away, helping you avoid interoperability problems further down the road.

Wrapping It Up

I've tried to give you a sneak peek at some of the new Web services features coming in the .NET Framework 2.0 Beta 2. Most of the changes are focused on polishing the existing framework with better tool support and more interoperable results. The outcome is a more mature and efficient Web services platform prepared for migration to Indigo.

Send your questions and comments for Aaron to sstation@microsoft.com.

Aaron Skonnard is a co-founder of Pluralsight, an education and content creation company, where he focuses on XML and Web services technologies. He's a contributing editor to MSDN Magazine and the author of several books, including Essential XML Quick Reference (Addison-Wesley, 2001). Reach him at www.pluralsight.com/aaron.


.NET Matters: ICustomTypeDescriptor, Part 1

.NET Matters
ICustomTypeDescriptor, Part 1
Stephen Toub

Code download available at: NETMatters0504.exe (163KB)

Q I write a lot of one-off utilities for personal use, and since they don't require any sophisticated user interfaces, I often use a System.Windows.Forms.PropertyGrid bound to a settings class in order to allow the user to configure a tool's operations. Unfortunately, sometimes I don't write these settings classes, and often they've been constructed in a way that's incompatible with the PropertyGrid. For example, I often bind to client proxy classes created by wsdl.exe in order to make Web service requests from my tool. The problem is that these proxy classes expose fields rather than properties. Thus, none of these configuration settings shows up in the PropertyGrid. Short of manually going in and changing each field to a property each time these proxy classes are autogenerated, is there any way to get the behavior I need?

A It sounds like you're using the Microsoft® .NET Framework 1.x, given your description of wsdl.exe and the proxy classes it generates. In the .NET Framework 2.0, the default operation of wsdl.exe is to generate public properties that wrap private fields, rather than the only mode of operation in version 1.x, which is to generate public fields. So, using proxy classes as you describe will just start working for you when you start using the .NET Framework 2.0. But obviously that doesn't solve the larger problem, nor does it work for you in the .NET Framework 1.x.

If the PropertyGrid bound to your types used only reflection to examine a type's metadata, you'd be out of luck, and there'd be no way to accomplish what you're trying to do (short of some hackish and non-user-friendly workarounds, one of which might be dynamic code generation). Luckily, it doesn't. PropertyGrid relies on special types from the System.ComponentModel namespace to provide it with information about the objects to which it is bound.

One such type, System.ComponentModel.TypeDescriptor, is used to retrieve information about the properties and events a particular component exposes. In the most general case, TypeDescriptor simply uses reflection to return a System.ComponentModel.PropertyDescriptor for each public instance property found on the type. However, other factors influence exactly which PropertyDescriptors are returned. For example, if the object being described is a component and is associated with a Container that also contains extender provider components (such as System.Windows.Forms.ToolTip) to provide additional properties to this object, TypeDescriptor.GetProperties will return a System.ComponentModel.PropertyDescriptorCollection that contains descriptors for these extender properties in addition to the ones found through reflection and the type's metadata (for more information on provider controls, see the Cutting Edge column in the November 2003 issue of MSDN®Magazine).

Before going straight to the metadata to get property information, TypeDescriptor first checks to see if the type being examined implements the System.ComponentModel.ICustomTypeDescriptor interface. If it does, and if some other conditions are met (to be discussed shortly), rather than using the metadata to get property information, TypeDescriptor will simply ask the object (through its ICustomTypeDescriptor.GetProperties method) which properties it supports. This gives the object itself a chance to hand back a PropertyDescriptorCollection containing exactly the properties it wants displayed in the PropertyGrid. That's good news for you, because it means your type can fool the PropertyGrid into thinking your type has properties it doesn't really have. More specifically, you can convince the PropertyGrid that each of your fields is actually a property. To do so, you need to create a custom PropertyDescriptor for each field you want displayed—in this case, all public fields.

PropertyDescriptor is an abstract class that derives from another abstract class, MemberDescriptor. You can think of MemberDescriptor as providing the basic information for each property, specifically its name and any System.Attribute instances associated with that property. It also provides some useful helper functions that pull out information from those attributes (for example, MemberDescriptor.Category looks to see if one of the attributes is a CategoryAttribute, and if it finds one, returns that attribute's Category string value). PropertyDescriptor adds functionality related to changing a property's value and determining when that value has changed. So, for example, it exposes GetValue and SetValue abstract methods as well as an OnValueChanged method, which will notify any listeners that this property's value has changed. Tricky naming, huh?

To create a PropertyDescriptor for each field, you first need to create a custom class that derives from PropertyDescriptor. An example of one viable class for your purposes is shown in Figure 1. This FieldPropertyDescriptor class wraps a System.Reflection.FieldInfo class describing a field on a type. The constructor initializes the base MemberDescriptor with the name of the field as the property name and the field's attributes as the property's attributes. (The PropertyGrid uses the attributes from the PropertyDescriptor to determine important display settings, such as name, category, description, and even which UITypeEditor to use.)

Figure 1 Custom PropertyDescriptor for Fields (Close Figure 1)
public class FieldPropertyDescriptor : PropertyDescriptor
{
    private FieldInfo _field;

    public FieldPropertyDescriptor(FieldInfo field) : base(field.Name,
        (Attribute[])field.GetCustomAttributes(typeof(Attribute), true))
    {
        _field = field;
    }

    public FieldInfo Field { get { return _field; } }

    public override bool Equals(object obj)
    {
        FieldPropertyDescriptor other = obj as FieldPropertyDescriptor;
        return other != null && other._field.Equals(_field);
    }

    public override int GetHashCode() { return _field.GetHashCode(); }

    public override bool IsReadOnly { get { return false; } }

    public override void ResetValue(object component) {}

    public override bool CanResetValue(object component) { return false;}

    public override bool ShouldSerializeValue(object component)
    {
        return true;
    }

    public override Type ComponentType
    {
        get { return _field.DeclaringType; }
    }

    public override Type PropertyType { get { return _field.FieldType; }}

    public override object GetValue(object component)
    {
        return _field.GetValue(component);
    }

    public override void SetValue(object component, object value)
    {
        _field.SetValue(component, value);
        OnValueChanged(component, EventArgs.Empty);
    }
}

The rest of the class is fairly self-explanatory. GetValue and SetValue use reflection and the stored FieldInfo to access and modify the underlying field's value. PropertyType returns the field's type. And the Equals method compares the stored FieldInfo objects rather than the FieldPropertyDescriptors. Note that any type that overrides Object.Equals should also override Object.GetHashCode, as I've done here.

With FieldPropertyDescriptor in hand, you now need to provide instances of it back to the PropertyGrid. As noted earlier, this can be accomplished by having your type implement the ICustomTypeDescriptor interface, paying particular attention to the GetProperties method. Unfortunately, yes, this requires you to modify your type. However, I'll show you how to cut down on the number of changes you need to make to each type by putting all of this functionality into a common base class. Then, all you need to do to use this functionality with your field-exposing type is to derive from this new base class; no changes to your type's members will be necessary.

Figure 2 shows just such a base class. The most important method is GetProperties, which returns the PropertyDescriptionCollection that'll be used by callers to determine which properties this instance supports. The core of it, ignoring for a moment all of the extra boilerplate, is implemented as follows:

PropertyDescriptorCollection props =
    new PropertyDescriptorCollection(null);
foreach(PropertyDescriptor prop in
    TypeDescriptor.GetProperties(this, attributes, true))
        props.Add(prop);
foreach (FieldInfo field in GetType().GetFields())
    props.Add(new FieldPropertyDescriptor(field));
return props;

Figure 2 Implementing ICustomTypeDescriptor (Close Figure 2)
public abstract class FieldsToPropertiesTypeDescriptor :
    ICustomTypeDescriptor
{
    object ICustomTypeDescriptor.GetPropertyOwner(PropertyDescriptor pd)
    {
        return this;
    }

    AttributeCollection ICustomTypeDescriptor.GetAttributes()
    {
        return TypeDescriptor.GetAttributes(this, true);
    }

    string ICustomTypeDescriptor.GetClassName()
    {
        return TypeDescriptor.GetClassName(this, true);
    }

    ...

    private PropertyDescriptorCollection _propCache;
    private FilterCache _filterCache;

    PropertyDescriptorCollection ICustomTypeDescriptor.GetProperties()
    {
        return ((ICustomTypeDescriptor)this).GetProperties(null);
    }

    PropertyDescriptorCollection ICustomTypeDescriptor.GetProperties(
        Attribute[] attributes)
    {
        bool filtering = (attributes != null && attributes.Length > 0);
        PropertyDescriptorCollection props = _propCache;
        FilterCache cache = _filterCache;

        // Use a cached version if possible
        if (filtering && cache != null && cache.IsValid(attributes))
            return cache.FilteredProperties;
        else if (!filtering && props != null)
            return props;

        // Create the property collection and filter
        props = new PropertyDescriptorCollection(null);
        foreach(PropertyDescriptor prop in
            TypeDescriptor.GetProperties(
            this, attributes, true))
        {
            props.Add(prop);
        }
        foreach (FieldInfo field in this.GetType().GetFields())
        {
            FieldPropertyDescriptor fieldDesc =
                new FieldPropertyDescriptor(field);
            if (!filtering ||
                fieldDesc.Attributes.Contains(attributes))
                props.Add(fieldDesc);
        }

        // Store the computed properties
        if (filtering)
        {
            cache = new FilterCache();
            cache.Attributes = attributes;
            cache.FilteredProperties = props;
            _filterCache = cache;
        }
        else _propCache = props;

        return props;
    }

    private class FilterCache
    {
        public Attribute[] Attributes;
        public PropertyDescriptorCollection FilteredProperties;
        public bool IsValid(Attribute[] other)
        {
            if (other == null || Attributes == null) return false;

            if (Attributes.Length != other.Length) return false;

            for (int i = 0; i < other.Length; i++)
            {
                if (!Attributes[i].Match(other[i])) return false;
            }

            return true;
        }
    }
}

The code first gets the collection of descriptors for the actual properties defined on the type with a call to TypeDescriptor.GetProperties. The collection returned will also include any properties inserted by extender provider components. (You'll notice, too, that the third argument to this call, noCustomTypeDesc, is true, which tells TypeDescriptor not to use the instance's ICustomTypeDescriptor implementation. Passing in a value of false here would be fatal, as it would result in infinite recursion and eventually a StackOverflowException as TypeDescriptor proceeded to call into your GetProperties implementation again and again and again.) All of these base descriptors are copied into a new collection. The code then loops through each of the public instance fields on the type and creates a FieldPropertyDescriptor for each, which are promptly added to that new collection. When all of the fields have been wrapped, the collection is returned.

Of course, you'll notice that there's more to my implementation of GetProperties than what I've just described. What I've outlined will be correct functionally, but it won't perform as it otherwise could. Specifically, the resulting PropertyDescriptorCollection could be cached such that future calls to this method could simply return the cached collection rather than recomputing it. However, you have to be careful with how you cache, since TypeDescriptor.GetProperties filters what properties are returned based on the attributes that are passed to the method (only properties that are attributed with one of the specified attributes, either explicitly or by default, are returned). Since I'm augmenting the results of calling to TypeDescriptor.GetProperties in my implementation of ICustomTypeDescriptor.GetProperties, and since TypeDescriptor.GetProperties will filter results, the set to be returned from my implementation must be recomputed based on a different set of attribute filters than were used originally.

To compensate for this, I maintain multiple cached property sets. First, I cache the properties to be used when no attributes are specified as a filter (in other words, the full list of properties). Second, I cache the last property set computed based on the last attribute filter specified, which is also cached. This way, I only have to compute the full set of properties once, and if the same attribute filter is used multiple times in a row, I likewise only have to compute it once.

As an aside, note that in the .NET Framework 1.x filtering in this situation is strictly optional, as TypeDescriptor.GetProperties will filter the property collection before returning it to the caller. If the creation of a custom property descriptor is expensive, you'll want to filter as much as possible before creating the descriptor. In the .NET Framework 2.0 Beta 1, only in certain scenarios (including, for backward compatibility, the scenarios available in the .NET Framework 1.x) will TypeDescriptor perform this filtering for you. More on this next month.

The rest of the FieldsToPropertiesTypeDescriptor class is boilerplate. GetPropertyOwner returns a reference to "this", since the type to be described is deriving from this class (FieldsToPropertiesTypeDescriptor), and it "owns" all of the properties being returned. Almost every other method then delegates to the appropriate method on TypeDescriptor, letting that method do all the heavy lifting (again, with the noCustomTypeDesc parameter set to true).

You can now give this solution a try. Assuming you have a type that looks something like the following

public class Person
{
    public string Name;
    public int Age;
    public string Hobbies;
}
you can simply modify it to use this new base class as follows:
public class Person : FieldsToPropertiesTypeDescriptor
{
    public string Name;
    public int Age;
    public string Hobbies;
}
And, poof, PropertyGrid will show all of your fields in the grid. This is made possible by the process I've described up to this point, outlined in Figure 3.

Figure 3 Using ICustomTypeDescriptor in the .NET Framework 1.x
Figure 3 Using ICustomTypeDescriptor in the .NET Framework 1.x

Of course, this isn't quite what you asked for. While this requires minimal modifications to your type, you didn't want to have to change your type at all. Unfortunately, in the .NET Framework 1.x, in order for TypeDescriptor to give back custom properties for a type, that type must implement ICustomTypeDescriptor.

There is a slight variation to this solution that might work for you, however. You can modify the FieldsToPropertiesTypeDescriptor class to be a proxy class rather than an abstract base class for the type in question. This would result in a class that looks similar to the code shown in Figure 4. Here I've renamed FieldsToPropertiesTypeDescriptor to be FieldsToPropertiesProxyTypeDescriptor, removed the abstract modifier, and added a constructor and a private object member variable. The constructor accepts as a parameter the target object that you would have otherwise set as the SelectObject for the PropertyGrid. This object is stored internally to the _target member, which is then used everywhere that the "this" reference was used in Figure 2. Any requests for metadata information about this proxy custom type descriptor will then actually return information about the target object. This allows you to keep your original type unmodified, writing code like the following to show the object in the property grid:

propertyGrid.SelectedObject =
    new FieldsToPropertiesProxyTypeDescriptor(person);
The solution discussed in this column is useful for more than just working with the PropertyGrid. In fact, another application of descriptors is used much more frequently: ASP.NET data binding. The System.Web.UI.DataBinder class is used heavily in applications that implement data binding. According to the documentation, the DataBinder.Eval method "uses reflection to parse and evaluate a data binding expression against an object at run time." More specifically, however, it uses property descriptors rather than directly using the System.Reflection namespace. DataBinder.Eval calls DataBinder.GetPropertyValue, which is implemented approximately as follows:
public static object GetPropertyValue(object container, string propName)
{
    ...
    PropertyDescriptor propDesc = TypeDescriptor.GetProperties(
        container).Find(propName, true);
    if (propDesc == null) throw new HttpException(...);
    return propDesc.GetValue(container);
}
Thus, you can use the FieldsToPropertiesTypeDescriptor or FieldsToPropertiesProxyTypeDescriptor classes described in this column to allow ASP.NET data binding to work with fields in addition to with properties.

Figure 4 A Proxy ICustomTypeDescriptor (Close Figure 4)
public class FieldsToPropertiesProxyTypeDescriptor :
    ICustomTypeDescriptor
{
    private object _target; // object to be described

    public FieldsToPropertiesProxyTypeDescriptor(object target)
    {
        if (target == null) throw new ArgumentNullException("target");
        _target = target;
    }

    object ICustomTypeDescriptor.GetPropertyOwner(PropertyDescriptor pd)
    {
        return _target; // properties belong to the target object
    }

    AttributeCollection ICustomTypeDescriptor.GetAttributes()
    {
        // Gets the attributes of the target object
        return TypeDescriptor.GetAttributes(_target, true);
    }

    string ICustomTypeDescriptor.GetClassName()
    {
        // Gets the class name of the target object
        return TypeDescriptor.GetClassName(_target, true);
    }

    ... // the rest of the class is the same as
        // FieldsToPropertiesTypeDescriptor except using
        // "_target" instead of "this"
}

Next time in this column, I'll continue this discussion of property descriptors, focusing on the .NET Framework 2.0, specifically on how the new TypeDescriptionProvider infrastructure dramatically improves the situation.

Send your questions and comments to netqa@microsoft.com.

Stephen Toub is the Technical Editor for MSDN Magazine.


Pure C++: Generic Programming Under .NET

Pure C++
Generic Programming Under .NET
Stanley B. Lippman

Contents
What Are Parameterized Types?
The Type Parameter List
Type Instantiation

Visual Studio® 2005 brings the type parameter model of generic programming to the Microsoft® .NET Framework. Parameterized types are, of course, standard fare for C++ programmers. So, for those who are unfamiliar with them, I'll give a brief introduction to generic programming in my next few columns.

The primary idea behind generic programming is the delivery of an invariant code base that supports a potentially infinite set of types. There are two general models used for generic programming: the Universal Type Container Model (UTCM) and the Type Parameter Model (TPM).

Under the UTCM, the type information associated with the object is stripped. This is simple to support because it is reductive. All objects are stored in a uniform and opaque manner. In a monolithic type system such as the Common Type System (CTS), the universal type container is Object; all CTS types are either derived directly or derived indirectly from Object. In a language like C, void* is the universal type.

Under the TPM, the binding of type information associated with the object is factored out and delayed. Values that may vary from one invocation to the next are factored out into parameters. That's why this implementation model is called parameterized types—it's more complex, but also more powerful.

For example, let's say that you're implementing the IEnumerator interface of the System::Collections namespace. It seems pretty trivial, providing two methods and a property. But providing a single implementation available to all users is incredibly hard in a strongly typed language. The intractable aspect of our implementation is flagged by "???":

interface class IEnumerator
{
    property ??? Current { ??? get(); }
    bool MoveNext();
    void Reset();
};
The type system requires you to statically identify the type associated with the property's backing store and the get accessor return type, but of course that's not possible. There are an infinite number of potential types that users will need to enumerate over. So what do you do?

The simpler general solution is the UTCM, in which Object serves as container:

interface class IEnumerator
{
    property Object^ Current { Object^ get(); }
    bool MoveNext();
    void Reset();
};
This provides one degree of separation. It allows you to support a potentially infinite number of types with a single, invariant code base. And for the passive storage and retrieval of objects of reference types, it works quite elegantly.

Things become somewhat less elegant as soon as you need to retrieve and manipulate the objects as concrete types. This requires a downcast back into the original object type. Unfortunately, the compiler is without the necessary type information to guarantee cast correctness and so it falls to the programmer to intervene manually with an explicit downcast, as shown in the following example:

extern void f( Object^ anyTypeWorks );
Object^ o = "a string of all things";

// no downcast ... passive storage
f( o );

// downcast ... we need to manipulate
String^ s = safe_cast<String^>( o );

For implementing collections, things become even more problematic. There is no way to statically constrain a collection to only hold objects of a single type under the universal type container model. This can only be provided at the program level, and is somewhat complicated and error prone. Moreover, because it is a program solution, it is only able to be applied during runtime. Again, you are left without compiler support.

Apart from safety and complexity, there is also a performance issue with large-scale storage and retrieval of value types under the universal type container model. These three problems are solved under type parameterization.

What Are Parameterized Types?

The type parameter model provides a second degree of separation, eliminating both downcasting and boxing and allowing for compile-time flagging of element type violations of a homogeneous container. It is a two-step solution. In the first step, you substitute a type parameter as a placeholder for the actual type, analogous to what you do with a function:

interface class IEnumerator
{
    property typeParameter Current { typeParameter get(); }
    bool MoveNext();
    void Reset();
}
In the second step, you indicate to the compiler (the machine reader of the program) that typeParameter is a placeholder and not a real program entity. You do this through a generic signature referred to as the parameter list. In C++/CLI, this is introduced with either the generic keyword, if you choose the common language runtime (CLR) generic mechanism, or the template keyword, if you choose to use the C++ template type mechanism, as shown in Figure 1.

Figure 1 Indicating typeParameter (Close Figure 1)

Generic Keyword
generic <typename typeParameter>
interface class IEnumerator
{
    property typeParameter Current { typeParameter get(); }
    ...
};
Template Keyword
template <typename typeParameter>
interface class IEnumerator
{
    property typeParameter Current { typeParameter get(); }
    ...
};

This results in a type-independent interface definition. Later, when a class implements IEnumerator, it must provide an actual type to bind to the typeParameter placeholder. You do this by pairing the parameterized class name with an actual type enclosed in brackets, such as IEnumerator<int>.

C++/CLI supports two parameterized type mechanisms, templates and generics, for defining parameterized reference, value, and interface classes, functions, and delegates. On the surface, at least, the definition of a parameterized generic or template type is syntactically equivalent (other than the use of the template or generic keyword). In other ways, they are remarkably different.

Consider the two stack declarations in Figure 2, the template instance (tStack) making use of the dynamic vector container provided by a CLI implementation of the Standard Template Library (STL) and the generic instance (gStack) making use of the dynamic List<T> container provided by the System::Collections::Generic namespace. Both parameterized type collection libraries are new to Visual Studio 2005.

Figure 2 Template and Generic Stack Declaration (Close Figure 2)

C++/CLI Template Stack Declaration
#include <cliext/vector>
using namespace cliext;

template <class elemType>
ref class tStack
{
    // the CLI vector is a reference class ...
    vector<elemType> ^m_stack;
    int top;

public:
    tStack();
    elemType pop();
    void push( elemType et );
    ...
};
C++/CLI Generic Stack Declaration
using namespace System::Collections::Generic

generic <class elemType>
public ref class gStack {
    List<elemType> ^m_stack;
    int top;

public:
    gStack();
    elemType pop();
    void push( elemType et );
    ...
};

You create an instance of the parameterized type by specifying an actual type within angle brackets following the class name. For example, Figure 3 shows the template stack instantiated with an integer and a string type argument in turn. Figure 4 shows the two same type arguments that are used in order to create equivalent instances of the generic stack.

Figure 4 Instantiating the Generic Stack (Close Figure 4)
void demo_generic_Stack()
{
    // an int value type argument...

    gStack<int>^ is = gcnew gStack<int>( 10 );
    for ( int ix = 0; ix < 10; ix++ )
          is->push( ix*2 );

    int elem_cnt = is->size();
    for ( int ix = 0; ix < elem_cnt; ++ix )
      Console::WriteLine( "({0}) {1}", ix+1, is->pop());

    // a String^ reference type argument...

    gStack<String^> ^ss = gcnew gStack<String^>( 10 );
    ss->push( "Pooh" ); ss->push( "Piglet" );
    ss->push( "Rabbit" ); ss->push( "Eeyore" );

    elem_cnt = ss->size();
    for ( int ix = 0; ix < elem_cnt; ++ix )
      Console::WriteLine( "({0}) {1}", ix+1, ss->pop());
}
Figure 3 Instantiating the Template Stack (Close Figure 3)
void demo_template_Stack()
{
    // an int value type argument...

    tStack<int>^ is = gcnew tStack<int>( 10 );
    for ( int ix = 0; ix < 10; ix++ )
          is->push( ix*2 );

    int elem_cnt = is->size();
    for ( int ix = 0; ix < elem_cnt; ++ix )
      Console::WriteLine( "({0}) {1}", ix+1, is->pop());

    // a String^ reference type argument...

    tStack<String^> ^ss = gcnew tStack<String^>( 10 );
    ss->push( "Pooh" ); ss->push( "Piglet" );
    ss->push( "Rabbit" ); ss->push( "Eeyore" );

    elem_cnt = ss->size();
    for ( int ix = 0; ix < elem_cnt; ++ix )
      Console::WriteLine( "({0}) {1}", ix+1, ss->pop());
}

The actual manipulation of the objects of the parameterized types, such as is and ss, are exactly the same as if they were objects of a nonparameterized type. One benefit of a parameterized type is that a single-source definition can spawn potentially an infinite number of specific type instances. In the example, both the generic and template stack classes support a string and an integer class out of the same parameterized class source code. There are no real constraints on using them with any and every type known to the application. As you'll see in a subsequent column, this is not the case with all parameterized types.

The generic and template definition and specific type instances are nearly equivalent here, though that isn't the case with all parameterized types, or there would be little benefit in C++/CLI supporting both mechanisms. As I go through the details of the two mechanisms in subsequent columns, you'll see additional differences. Even with these differences, it seems the better approach to integrate their presentation, highlighting the differences as I encounter them, rather than say the same thing multiple times.

The Type Parameter List

Each type parameter is prefaced with either the class or typename keyword. These keywords do not hold any platform significance—for example, class is not meant to suggest a native type nor is typename meant to suggest a Common Language Infrastructure (CLI) type. Both indicate that the name following represents a parameterized type placeholder that will be replaced by a user-specified type argument.

The reason for the two keywords is historical. In the original template specification, Stroustrup reused the existing class keyword to specify a type parameter rather than introduce a new keyword that might break existing programs. And up until the ISO-C++ standard, the use of the class keyword was the only way to declare a type parameter.

Reuse of existing keywords seems to always sow confusion. Did the use of class to indicate a type parameter limit the available type arguments to be class types rather than, say, a built-in or a pointer type? No. Then isn't the use of class in this context misleading? Well, yes. So, there was some feeling that not having introduced a new keyword caused unnecessary confusion. But that was not why the keyword typename was added.

Rather, typename was added to C++ to allow the parsing of template definitions. This topic is somewhat advanced, and I'll only briefly touch on it here. A fuller description can be found in Stroustrup's Design and Evolution of C++ (Addison-Wesley, 1994).

It is not always possible for the compiler to distinguish between type declarations and expressions in some cases. If the compiler encounters the expression Parm::name in a template definition and Parm is a template type parameter representing a class, does name refer to a type member or to a data member of Parm?

template <class Parm, class U>
Parm minus( Parm* array, U value )
{
    Parm::name * p; // Is this a pointer declaration or
                    // a multiplication expression?
                    // By default treated as expression.
}
By default, this statement is treated as an expression multiplying the operands Parm::name and p. The keyword typename was introduced to allow the programmer to override this default interpretation. For example, to declare p as a pointer of type Parm::name, you would rewrite the template function as follows:
template <class Parm, class U>
Parm minus( Parm* array, U value )
{
    typename Parm::name * p; // ok: pointer declaration
}
Since the keyword was now on the payroll, it was pretty much a no-brainer to fix the confusion caused by the reuse of the class keyword. Given the extensive body of existing code, books, articles, talks, and postings using the class keyword, it couldn't just be sent packing. And that is why C++ supports both keywords.

Following the class or typename keyword is an identifier that serves as a placeholder within the template or generic definition. Each identifier within the parameter list must be unique. However, across declarations both the keyword and the identifier can vary:

template <class T>
public ref class tStack;

// ok: both the keyword and identifier can vary across
//     declarations of the same type
template <typename elemType>
public ref class tStack {};
The scope of the identifier persists for the extent of the type declaration. In the forward declaration of tStack, this is terminated by the semicolon, and the name is never referenced. In the actual definition, the identifier is visible both within the class definition and within each member function of the class definition defined out-of-line.

Type Instantiation

A template or generic definition specifies how individual classes or functions can be constructed given a set of one or more actual types. The time of the instantiation is one of the major differences between templates and generics. Template instantiation is done at compile time; generic instantiation is done by the CLR at run time. (I'll address these in more detail in my next column.)

The template definition serves as a diagram for the automatic generation of type-specific instances; the compiler literally plugs in the specific type arguments provided by the user. The generic definition is more like a blueprint; the runtime constructs the type-specific instances, modifying the general syntax based on whether the type argument is a reference or value type. For example, a stack class for objects of type int and a stack class for objects of type String are created automatically from the template or generic definition when you write the following:

tStack<int> ^si;
tStack<String^> ^ss;

This generation of a class from the template definition is called template instantiation—this is how it is discussed in the ISO-C++ standard. In the literature describing generics, the class generation is referred to as construction—this again follows from the distinction between a template and a blueprint. Here, I use "instantiation" to describe this process. When a stack class for objects of type String is instantiated, each occurrence of the template parameter within the generic or template definition is replaced with type String. The correctness of the definition is verified for that type.

The name of the instantiation is Stack<int> or Stack<String^>. The <int> or <String^> tokens that follow the name are called the template arguments in ISO-C++. They are referred to as type arguments in the generic literature, and in this case I will follow their example. The type arguments must be specified in a comma-separated list and bracketed by the less than and greater than tokens. The name of an instantiation must always specify the type arguments explicitly. Unlike type arguments for functions instantiations, the type arguments for class instantiations are never deduced from the context in which the class instantiation is used—what that means will be made clear in a future column on parameterized functions and function type deduction.

A class instantiation can be used by the general program wherever a nonparameterized class type can be used. Similarly, the objects of an instantiated class are declared and used exactly the same as objects of a nonparameterized class.

Finally, it is important to realize that there is no special relationship—as there is, for example, between a derived and base class—between two instances of a generic (or template) type bound to independent type arguments. In the example, the two stack instantiations represent objects of two independent classes. You could not, for example, initialize or assign one to the other without explicitly programming for that. Nor does the String instantiation of the stack have access permission to the nonpublic members of the integer instantiation of the stack.

That pretty much covers what is common between templates and generics. More interesting, of course, are the differences. I briefly mentioned one difference—templates are instantiated at compile time while generics are instantiated by the runtime. The consequences of this are actually far-reaching, and will be the topic of the next column. See you then!

Send your questions and comments to purecpp@microsoft.com.

Stanley B. Lippman is an Architect with the Visual C++ team at Microsoft. He began working on C++ with its inventor, Bjarne Stroustrup, in 1984 at Bell Laboratories. In between, he worked in feature animation at Disney and DreamWorks, was a Distinguished Consultant with JPL, and was a Software Technical Director on Fantasia 2000.


C++ at Work: Form Validation with Regular Expressions in MFC

New information has been added to this article since publication.
Refer to the Editor's Update below.

C++ At Work
Form Validation with Regular Expressions in MFC
Paul DiLascia

Code download available at: C.exe (248KB)

I thought I'd use this month's column to describe an interesting app I built using the RegexWrap library described in my article "Wrappers: Use Our ManWrap Library to Get the Best of .NET in Native C++ Code" in this issue. RegexForm is a form validation system for MFC based on regular expressions. This app was my main reason for implementing RegexWrap in the first place. But since many of the details don't relate to regex per se, it makes more sense to describe RegexForm here.

As I mentioned in the article, one of the most practical applications of regular expressions is validating user input. Regular expressions make short shrift of validating ZIP codes, phone numbers, credit card numbers—all sorts of real-world information. One regular expression can replace dozens or even hundreds of lines of procedural code. UNIX and Web programming languages like Perl have had regular expressions since Day One, but until the .NET Framework, regular expressions weren't available in Windows [ Editor's Update - 3/15/2005: The CAtlRegExp and CAtlREMatchContext classes in the ATL Server Library provide support for regular expressions.] So now that .NET provides a complete regular expressions library, why not use it in MFC apps? And with my RegexWrap library described in the main article, you don't even need the Managed Extensions or /clr.

MFC already has a mechanism for validating dialog input called Dialog Data Exchange (DDX) and Dialog Data Validation (DDV). Technically, DDX transfers data between the screen and your dialog object, whereas DDV validates the data. DDX begins when you call UpdateData from your dialog's OnOK handler:

// user pressed OK:
void CMyDialog::OnOK() {
  UpdateData(TRUE); // get dialog data
  ...
}

UpdateData is a virtual CWnd function you can override in your dialog. Its Boolean argument tells whether to copy information from the screen to your dialog object or vice versa. (You can call UpdateData(FALSE) from OnInitDialog to initialize your dialog.) The default CWnd implementation creates a CDataExchange object and passes it to another virtual function, DoDataExchange, which you're supposed to override to call specific DDX functions to transfer data for individual data members:

void CMyDialog::DoDataExchange(CDataExchange* pDX) {
  CDialog::DoDataExchange(pDX);
  DDX_Text(pDX, IDC_NAME, m_name);
  DDX_Text(pDX, IDC_AGE, m_age);
  ...
  // etc.
}

Here IDC_NAME and IDC_AGE are the IDs of edit controls and m_name and m_age are CString and int data members. DDX_Text copies what the user entered for Name and Age into m_name and m_age (with an overload that converts Age to int along the way). The DDX functions know which way to go because CDataExchange::m_bSaveAndValidate is TRUE when copying from screen to dialog and FALSE going the other way. MFC has loads of DDX functions for all sorts of data and control types. For example, DDX_Text comes in at least a dozen overloaded flavors to copy and convert text input to various types like CString, int, double, COleCurrency and more. There's DDX_Check to convert a checkbox state to integer value, and DDX_Radio does the same for radio buttons.

DDX functions transfer the data; DDV functions validate it. For example, to limit the user's name to 35 characters, you'd write:

// in CMyDialog::DoDataExchange
DDX_Text(pDX, IDC_NAME, m_sName); // get/set value
DDV_MaxChars(pDX, m_sName, 35);   // validate
And to restrict your user's age to an integer between 1 and 120, you'd write the following:
// m_age is int
DDX_Text(pDX, IDC_AGE, m_age);
DDV_MinMaxInt(pDX, m_age, 1, 120);

While DDX works well, DDV is a bit primitive. MFC has a limited repertoire of validations it can perform. You can limit the number of characters in a text field, and you can do min/max constraints on various types. Min/max is fine, but what if you want to validate zip codes or phone numbers? MFC has nothing for this. You have to write your own DDV functions. When I first implemented validation using regular expressions, all I did was write one function, like this:

void DDV_Regex(CDataExchange* pDX, CString& val,
  LPCTSTR pszRegex)
{
  if (pDX->m_bSaveAndValidate) {
    CMRegex r(pszRegex);
    if (!r.Match(val).Success()) {
      pDX->Fail(); // throws exception
    }
  }
}
This lets you easily validate input using regular expressions like so:
// in CMyDialog::DoDataExchange
DDX_Text(pDX, IDC_ZIP, m_zip);
DDV_Regex(pDX, m_zip,_T("^\\d{5}(-\\d{4})?$"));

Pretty cool for only four lines of code. (Of course, that's assuming you have RegexWrap—otherwise you have to call the Framework Regex class directly using the managed extensions.) DDV_Regex works perfectly well within MFC's DDX/DDV scheme, but as I started adding more fields, I soon discovered some major shortcomings with DDX/DDV. For one thing, each DDV function displays an error message box and throws an exception if the field is bad, so if there are five bad fields, the user gets five message boxes—ouch! Also, I didn't want to hardcode the regular expression in the call to DDV. But my main objection to DDX/DDV is it's so procedural. To validate a new field, you have to add another data member and more code in DoDataExchange, which pretty soon ends up rambling, like this:

DDX_Text(pDX, IDC_FOO,...);
DDV_Mumble(pDX, ...)
DDX_Text(pDX, IDC_BAR,...);
DDV_Bletch(...)
... // etc for 14 lines

Why should I have to write procedural instructions to describe validation rules that are inherently static? One of my top five programming mantras is: abhor procedural code. Another one is: one table is worth a thousand lines of code. No doubt you can guess where this is heading. In the end, I wrote my own dialog validation system, one that's rule-based, ergo table-driven. It rides on top of DDX, but avoids DDV and has a much nicer user interface. It's easy to use and, of course, uses regular expressions to do the validating. Everything is encapsulated in a class CRegexForm you can use in any MFC dialog.

Figure 1 TestForm Tooltip Hints
Figure 1 TestForm Tooltip Hints

Naturally, I wrote a test program to show how it works. At first glance TestForm looks like a vanilla MFC dialog-based app. Its main dialog has several edit fields—Zip Code, SSN (Social Security number), Phone Number, and so on. But when you test-drive TestForm, you quickly realize it has more under the hood than you thought. If you tab to an input field, TestForm displays a tooltip hint that describes what you can enter (see Figure 1). If you type an illegal character—for example, a letter in Phone Number—TestForm rejects the character and beeps. When you press Enter or OK, you'll get an error message like the one in Figure 2 describing all the bad fields. All the errors appear in one message box, as opposed to one message box for each error. Then when the user tabs to one of these bad fields, TestForm displays the error message again in an application-supplied "feedback" window inside the dialog itself (see Figure 3), so users don't have to remember what the error message said, the form reminds them as they correct each bad field. And if only one field is bad, TestForm forgoes the message box and instead displays the error in the feedback window.

Figure 2 Wrong Fields Aplenty
Figure 2 Wrong Fields Aplenty

CRegexForm performs all this magic by itself. All you have to do is use it, and that's straightforward. First, you have to define your form. Here's how TestForm does it, in MainDlg.cpp:

// form/field map
BEGIN_REGEX_FORM(MyRegexForm)
  RGXFIELD(IDC_ZIP,RGXF_REQUIRED,0)
  RGXFIELD(IDC_SSN,0,0)
  RGXFIELD(IDC_PHONE,0,0)
  RGXFIELD(IDC_TOKEN,0,0)
  RGXFIELD(IDC_PRIME,RGXF_CALLBACK,0)
  RGXFIELD(IDC_FAVCOL,0,CMRegex::IgnoreCase)
END_REGEX_FORM()

The macros define a static table that describes each edit field. In most cases, all you need is the control ID, but there's also room for flags and RegexOptions. For example, in TestForm, Zip Code is required (RGXF_REQUIRED), Prime Number uses a callback (more shortly) and Favorite Columnist (IDC_FAVCOL) specifies CMRegex::IgnoreCase, which makes it case-insensitive.

Figure 3 Pietrek? I Think Not!
Figure 3 Pietrek? I Think Not!

Looking at the table, you might wonder where the regular expressions are. Answer: in the resource file. For each field/control ID, CRegexForm expects a resource string with the same ID. The resource string comprises five substrings separated by newline characters ('\n'). The general format is "Name\nRegex\nLegalChars\nHint\nErrMsg". Here's the string for IDC_ZIP:

"Zip Code\n^\\d{5}(-\\d{4})?$\n[\\d-]\n##### or #####-####"

The first substring, "Zip Code," is the field name. The second, "^\d{5}(-\d{4})?$," is the regular expression used to validate the zip. (You have to type two backslashes in the resource string in order to escape a regular expression backslash.) The third substring is another regular expression describing the legal characters. For Zip, it's "[\d-]", which allows digits or hyphen. If your field has no character restrictions, you can omit LegalChars by typing two newlines ("\n\n") for an empty substring. The fourth substring with all the pound signs is the tooltip hint. Finally, you can provide a fifth substring, the error message to display if the field is bad. For Zip Code, there's no error message, so CRegexForm generates one of the form "Should be xxx", where xxx is replaced with the hint. "Should be" is itself another resource string (more on that shortly). Of all these substrings, only the first (field name) is required.

Why use resource strings to hold all this information instead of coding it directly in the field map? For one thing, putting it in the map would make the code ungainly. It's much more tidy to stash the strings out of the way. And since macros can't have optional arguments, you'd need several macros like RGXFIELD3, RGXFIELD4 and RGXFIELD5 depending how many arguments you want to use. How gauche is that? But the real reason for resource strings is to make localization easy. Translators can translate the strings and create resource DLLs for different locales. The regular expressions themselves might even require translation (ZIP codes look different in other countries like Britain or Botswana), so they go in the resource file too.

While I'm here, let me point out in passing how easy it is to parse these substrings with regular expressions. MFC has a 26-line function AfxExtractSubString to parse document substrings, but CRegexForm uses CMRegex to do it in one line!

CString str;
str.LoadString(nID);
vector<CString> substrs = CMRegex::Split(str, _T("\n"));

Now substrs[i] is the ith substring, and if you want to know how many there are, just call substrs.size(). I'm sure glad I wrapped the Split function to return an STL vector.

Once you've defined your field map using BEGIN/END_REGEX_FORM and written your resource strings, the next thing to do is to instantiate a CRegexForm in your dialog and initialize it:

// in OnInitDialog
m_rgxForm.Init(MyRegexForm, this,
  IDS_MYREGEXFORM,
  MYWM_RGXFORM_MESSAGE);

Naturally, CRegexForm needs the field map and pointer to your dialog; the second and third arguments are another resource string and callback message ID. Like the individual field strings, the initialization string comprises multiple substrings separated by new lines. For TestForm, IDS_MYREGEXFORM is "Error: %s\nRequired\nShould be: %s\nBad Value". The first substring "Error: %s" is the error prefix. CRegexForm uses this to display "Error: xxx," where xxx is the actual error message. The second substring, "Required," is the word/phrase to use when a field is required (RGXF_REQUIRED). The third substring, "Should be: %s," is the one I described earlier. CRegexForm uses it to generate an error message "Should be: xxx" where xxx is the field hint. The last substring, "Bad Value," is a catch-all error message CRegexForm uses if the field has no hint and no error message. Users should never see this string because of course you're going to write a hint or error message for every field. Right?

The last Init argument, MYWM_RGXFORM_MESSAGE, is an app-defined callback message ID that lets CRegexForm talk to your app and do things like custom validation that requires procedural code. If you need to use mathematical algorithms or check the phase of the moon to validate your input, you can set RGXF_CALLBACK in your field flags and CRegexForm will send your dialog the callback message with notification code RGXNM_VALIDATEFIELD when it's time to validate. TestForm uses a callback to validate its Prime Number field; Figure 4 shows the details.

Figure 4 Procedural Validation Using Callback Message (Close Figure 4)
//////////////////
// Handle notification from Regex Form Manager:
// Do custom validation for Prime Number.
//
LRESULT CMainDlg::OnRgxFormMessage(WPARAM wp, LPARAM lp)
{
   UINT nID = LOWORD(wp);
   UINT nCode = HIWORD(wp);
   if (nCode==RGXNM_VALIDATEFIELD) { // custom validation:
      if (nID==IDC_PRIME) {
         const CString& val = *(CString*)lp;
         if (val.IsEmpty())
            return RGXERR_OK;
         int p = _tstoi(val);
         return IsPrime(p) ?  RGXERR_OK : RGXERR_NOMATCH;
      }
      ASSERT(FALSE); // shouldn't happen
   }
   return 0;
}

CRegexForm does DDX using its own internal CStrings, so you don't have to define a dialog member for each text field. All you have to do is call CRegexForm to transfer the data.

void CMyDialog::DoDataExchange(CDataExchange* pDX)
{
  CDialog::DoDataExchange(pDX);
  m_rgxForm.DoDataExchange(pDX);
}

When you initialize CRegexForm, it allocates an array of protected FLDINFO structures, one for each field in your map. One of the FLDINFO members is FLDINFO::val, a CString that holds the current field value. Internally, CRegexForm uses DDX_Text with this CString. You can get or set the internal field values by calling CRegexForm::GetFieldValue or SetFieldValue, both of which use the control ID to identify the field.

m_rgxForm.SetFieldValue(IDC_ZIP,_T("10025"));

CRegexForm treats all values as text and stores them in CStrings, but provides GetFieldValInt and GetFieldValDouble methods to get a value as int or double. For other types, you have to do your own conversion—or you can still use MFC's DDX functions in DoDataExchange. TestForm has a Populate button that uses CRegexForm::SetFieldValue to fill the form with the sample data you see in Figure 3. In general, CRegexForm uses control IDs to identify fields. It has methods called GetFieldName, GetFieldHint, and GetFieldError to get a field's name, hint, and error code—all of which take the control ID as parameter.

So far I've shown you how to create the field map, write the resource strings, initialize your CRegexForm, and hook it up through DDX. All that remains is to actually validate the user's input. The time to do it is when the user presses OK:

void CMyDialog::OnOK()
{
  UpdateData(TRUE); // copy screen->dialog
  int nBad = m_rgxForm.Validate();
  if (nBad>0) {
    m_badFields = m_rgxForm.GetBadFields();
    ...
}

UpdateData invokes MFC's DDX mechanism, which calls your dialog's DoDataExchange. DoDataExchange then calls CRegexForm::DoDataExchange, which copies the user's input to its internal FLDINFO structures. Now CRegexForm::Validate iterates the fields, calling CMRegex::Match to validate each one against its regular expression. If the field is bad, CRegexForm sets an error code RGXERR_NOMATCH in its internal FLDINFO, or RGXERR_MISSING for required fields that are empty. Validate returns the number of bad fields. If there are any, you can call CRegexForm::GetBadFields to get an array (STL vector) of bad field IDs. You can then iterate the array to get each error code and error message. This is what CMainDlg in TestForm does to build its error message box, like the one in Figure 2. If only one field is bad, CMainDlg calls CRegexForm::ShowBadField to highlight the field and display an error message in the feedback window, as in Figure 3. If all the fields are OK, TestForm displays a message box showing the values you entered (see Figure 5). In a real app, you'd copy the values to their ultimate destinations. Figure 6 shows the full code for CMainDlg::OnOK. By decoupling data validation from data exchange, CRegexForm gives you greater control over your UI, and lets you avoid MFC's hardwired error messages.

Figure 6 CMainDlg::OnOK (Close Figure 6)
//////////////////
// User pressed OK: validate form and display results: error message
// or field values—but don't call base class to end dialog.
//
void CMainDlg::OnOK()
{
   UpdateData(TRUE);    // get dialog data

   int nBad = m_rgxForm.Validate(); // validate
   CString msg;
   if (nBad>0) {
      vector<UINT> badFields = m_rgxForm.GetBadFields();
      BOOL beep = TRUE;
      if (nBad>1) {
         // Multiple bad fields: show message box with bad fields.
         msg = _T("The following fields are bad:\n\n");
         vector<UINT>::iterator it;
         for (it = badFields.begin(); it!=badFields.end(); it++) {
            UINT nID = *it;
            CString s;
            s.Format(_T("%s: %s\n"), m_rgxForm.GetFieldName(nID),
               m_rgxForm.GetFieldErrorMsg(nID));
            msg += s;
         }
         MessageBox(msg,_T("Oops—Some fields are bad."),
            MB_ICONEXCLAMATION);
         beep = FALSE;  // message box already beeped; don't beep again
      }
      // to highlight first bad field whether one or many
      UINT nID = badFields[0];
      m_rgxForm.ShowBadField(nID, beep, TRUE);

   } else {
      // all fields OK: show feeback
      msg = _T("You Entered:\n\n");
      for (int i=0; MyRegexForm[i].id; i++) {
         CString name = m_rgxForm.GetFieldName(MyRegexForm[i].id);
         CString val  = m_rgxForm.GetFieldValue(MyRegexForm[i].id);
         if (val.IsEmpty())
            val = _T("(nothing)");
         CString temp;
         temp.Format(_T("%s = %s\n"), name, val);
         msg += temp;
      }
      MessageBox(msg,_T("Congratulations! All fields OK."),MB_OK);
      m_rgxForm.Feedback(_T(" All fields OK or empty!"));
   }
}
Figure 5 Data Entered
Figure 5 Data Entered

I mentioned the feedback window. CRegexForm manages it entirely; all you have to do is provide one by calling CRegexForm::SetFeedBackWindow. You can specify a color for error messages. CRegexForm also takes care of hints. By default, it displays the field hint whenever the user tabs to a new field (see Figure 1). You can call CRegexForm::SetShowHints(FALSE) to turn hints off. SetShowHints(TRUE, nDelay, nTimeout) turns them on again, where nDelay is the number of milliseconds to wait before showing the hint (default=250) and nTimeout is the number of milliseconds to show the hint (default=0, forever). CRegexForm automatically kills the hint when the user leaves the field (EN_KILLFOCUS). TestForm uses SetShowHints to implement a checkbox that turns hints on or off (see Figure 1).

There's one other feature I hesitate to mention. CRegexForm has an option to do immediate validation. I don't recommend it because I think it's bad GUI design, but there are times when you don't want to let the user tab to the next field without entering valid information. In that case, you can use RGXF_IMMED in your field's flags, or call SetValidateImmed to make all the fields validate immediately. TestForm has a checkbox to turn this immediate validation feature on or off. If you check it, you'll see why immediate validation is a bad idea.

Finally, what about min/max and character-limit constraints? Min/max is one thing regular expressions can't do. And while ".{0,35}" is a regex that describes all strings at most 35 characters long, you really want to use EM_LIMITTEXT to limit the edit control text length, so the control beeps when the user types too many characters. Since I couldn't bear to implement a form-validation system that lacks any feature already supported by MFC, I introduced the notion of "pseudo regular expressions." For example, the regex for IDC_AGE (the Age field) is "rgx:minmax:int:1:120,maxchars:3". Obviously, this isn't a real regular expression. It's a pseudo-expression that CRegexForm recognizes and interprets on its own. The general format is "rgx:expr,expr,..expr", where each expression describes a different constraint. Currently, only two are supported: "minmax:type:minval:maxval" (where type is int or double) and "maxchars:maxval". CRegexForm parses these specially, and uses EM_LIMITTEXT for maxchars, just like DDV_MinMaxInt. For details, download the source. As with resource substrings, regular expressions make parsing these expressions a piece o' cake.

I've shown you all the things CRegexForm can do. How does it work? There isn't enough space for a full explanation here, but I can sketch the big picture. CRegexForm uses my CSubclassWnd to subclass your dialog. For readers who haven't seen it before, CSubclassWnd is a class I wrote eons ago that uses Windows subclassing to trap messages sent to another window. What's so cool about CSubclassWnd is it lets you subclass an MFC window without inserting a new class into your hierarchy. I could've derived CRegexForm from CDialog, but then you'd have to derive your dialog from CRegexForm. And what if you've already implemented your own CBetterDialog derived from CDialog? In that case, you'd have to perform surgery to insert CRegexForm, with unpredictable results.

This is one of the big flaws in MFC: it uses derivation to implement subclassing, so the class hierarchy exactly mirrors the Windows subclassing. But there's no need for this, and lots of times it's preferable to write plug-in classes like CRegexForm that subclass your dialog without making you alter your class hierarchy. For me CSubclassWnd is so indispensable I can't program without it!

CRegexForm uses CSubclassWnd to intercept EN_KILLFOCUS and EN_SETFOCUS to hide and show hints, and EN_CHANGE to clear the field's error state as soon as the user types anything. CRegexForm implements the hints themselves using my CPopupText class, first described in my September 2000 column. To prevent users from typing disallowed characters, CRegexForm installs another CSubclassWnd-derived hook for each edit control that has a LegalChars regular expression. This nested class, CRegexForm::CEditHook, intercepts WM_CHAR messages sent to the edit control to eat-and-beep any characters that aren't allowed (eat the character and call MessageBeep). For details, you can download the source, which is in the RegexWrap download for my article "Wrappers: Use Our ManWrap Library to Get the Best of .NET in Native C++ Code" in this issue.

Send your questions and comments for Paul to cppqa@microsoft.com.

Paul DiLascia is a freelance writer, consultant, and Web/UI designer-at-large. He is the author of Windows++: Writing Reusable Windows Code in C++ (Addison-Wesley, 1992). Paul can be reached at www.dilascia.com.


{ End Bracket }: Creating a Custom Metrics Tool

{ End Bracket }
Creating a Custom Metrics Tool
Stephen Toub

Code download available at: EndBracket0504.exe (137KB)

Metrics play an important role in our lives. Even if we don't realize it or characterize it as such, many daily activities have the potential to be quantified to some degree. So it's not surprising that metrics play an even greater role in the workplace, where there are goals and a bottom line and where much of a day's activity can be summarized in numbers. The problem is often figuring out exactly how to quantify one's progress and success.

Editing MSDN®Magazine is no different. Some articles are well written and require little work on our part; others require days of detailed attention. While off-the-cuff estimates often suffice, it's important to be able to review the edits that an article has undergone and to somehow be able to quantify the changes. This helps us determine how much effort was put into a particular piece. That information can be used to critique past decisions and to plan for the future, so a tool that could analyze articles and entire issues to quantify the effort involved would be quite useful.

88% Unadulterated!

We use Visual SourceSafe® (VSS) as a document repository. When an editor does an edit pass on an article, she checks it out, edits it, and checks it back in. Thus, each edit pass results in a separate version of that document in VSS. As a result, it's trivial to retrieve a version of an article as it existed before and after a particular edit pass.

With this in mind, I created a tool that uses the VSS client COM libraries to extract all versions of an article in order to compare each to the original document. Then it's just a click away to determine the metamorphosis of a document over its lifetime as well as how much a particular edit pass influenced that evolution. The tricky part is coming up with a valid metric: what does it mean for two documents to be similar?

In order to compare Microsoft® Word documents, first I need to obtain the text from each. Using the primary interop assembly (PIA) for Word, I create an instance of the Word ApplicationClass and open each document in turn. The unformatted text from the resulting Document object can then be obtained quickly with code such as the following:

Document doc = ...;                           // get the document
doc.ActiveWindow.Selection.WholeStory();      // select all
string text = doc.ActiveWindow.Selection.Range.Text;

Given the text for each document, how do I determine their similarity? Document similarity is an emerging field in computer science. Incorporating some of the field's ideas on n-gram-based metrics (an n-gram is a phrase made up of n grams, or words), my algorithm for comparing two documents works in two phases.

I first normalize each document's string, which includes optionally removing punctuation, and optionally lowercasing all text. It's then broken up into an array of its constituent words. From this array, I create a Hashtable of every n-gram in the string, where the key into the table is the n-gram phrase and the value of that entry is the number of times that phrase occurs in the string:

Hashtable CreateTable(string [] words, int n) {
    Hashtable table = new Hashtable(words.Length);
    for(int i=0; i<words.Length – n + 1; ++i) {
        string phrase = string.Join(" ", words, i, n);
        object val = table[phrase];
        table[phrase] = (val==null) ? 1 : 1 + (int)val;
    }
    return table;
}

The comparison operation is straightforward. I iterate over all of the phrases in the first Hashtable and look for each in the second Hashtable. If the phrase exists, this particular phrase from the first document exists in the second document, and the counts stored in each entry are used to determine whether the phrase was used the same number of times in each document. In this fashion, I build up a number that represents how many of the phrases from the first document exist in the second. This process is run again, but with the documents swapped (after all, if an editor adds a new paragraph, that constitutes a major change to the piece). At this point, I have a tally of hits of phrases between the two documents and can simply divide by the total number of phrases in both documents, arriving at a percentage. This percentage is meant to approximate the similarity of the final version of the document to the original.

Integrating this code into my GUI front end that retrieves documents from VSS, I was able to provide a utility to my team to provide these statistics on demand for both individual articles and for entire issues. The code I use to compare two documents is available for download from the link at the top of this article. So, for all of you out there who are as obsessive about numbers as I am, enjoy!

Stephen Toub is the Technical Editor for MSDN Magazine.