 MSDN Magazine December 2003

MSDN Magazine December 2003

Threading: Break Free of Code Deadlocks in Critical Sections Under Windows

Critical sections, a mechanism that prohibits more than one thread at a time from executing a particular section of code, is a topic that has not received much attention and thus tends not to be well understood. A solid understanding of critical sections in Windows can really come in handy when you need to track down multithreading performance issues in your code. This articles delves under the hood of critical sections to reveal information useful in finding deadlocks and in pinpointing performance problems. It also includes a handy utility program that shows all of your critical sections and their current states.

Matt Pietrek and Russ Osterlund MSMQ and .NET: Send MSMQ Messages Securely Across the Internet with HTTP and SOAP

When creating a distributed system you frequently need to provide for communication between two entities that are not in sync. Microsoft Message Queue Server (MSMQ) provides the kind of store-and-forward messaging in a pre-built infrastructure that can help you address these kinds of messaging needs. In the past, MSMQ was accessed using a COM wrapper. Now there's a .NET wrapper that lets you accomplish your messaging goals easily from your Framework-based code. To illustrate the use of the wrapper, the author builds a messaging application, sends MSMQ messages over the Web, and discusses messaging security.

David S. Platt Remoting: Managing the Lifetime of Remote .NET Objects with Leasing and Sponsorship

Leasing and sponsorship is the solution for managing the lifecycle of a remote object in .NET. Each object has a lease that prevents the local garbage collector from destroying it, and most distributed applications rely upon leasing. There are several ways in which objects and clients can extend the lease, including dedicated sponsor objects. In this article, the author explains leasing, shows how to configure it, and how it relates to the various remoting activation models. He then discusses design guidelines and options, along with their impact on throughput and performance. Additionally, he introduces a helper class used to automate the management of lease sponsors.

Juval Lowy Beyond Macros: Create Word and Excel Smart Documents with C++ and XML

One of the coolest new parts of Office 2003 is a programmability feature called Smart Documents, which allows developers to augment Word and Excel documents with programmable content and behavior. Typically, examples illustrating Office programmability use Visual Basic or Visual Basic .NET. In this article, the author develops a Smart Document for Excel using C++. He describes the new ISmartDocument interface and shows how to use it to manage a simple task list such as an Excel spreadsheet.

Mike Kelly Office 2003: Host an Interactive Visio Drawing Surface in .NET Custom Clients

Microsoft Office Visio 2003 introduces a new drawing component that allows you to embed an interactive drawing surface into your application's user interface. You can drive the Visio drawing component from events in your host application or with data from a Web Service and an ADO.NET data adapter. The Visio drawing component supports the rich Visio application programming model, giving you control over how graphics are used and displayed on the drawing surface. This article explains how to embed the Visio drawing component into a C#-based Windows Forms client app that retrieves data from the Fabrikam 2.0 Web Service.

Mai-lan Tomsen Bukovec and Blair Shaw Editor's Note: A Look Back and a Look Ahead

New Stuff: Resources for Your Developer Toolbox

Theresa W. Carey Web Q&A: Virtual Directories, Releasing DB Connections, and More

Edited by Nancy Michell Data Points: Exploring SQL Server Triggers

John Papa Cutting Edge: Custom Design-time Control Features in Visual Studio .NET

Dino Esposito The XML Files: A Survey of Publicly Available Web Services at Microsoft

Aaron Skonnard Advanced Basics: Windows Forms Q&A

Ken Spencer Design Patterns: Asynchronous Wait State Pattern in ASP.NET

Lyn Robison C++ Q&A: Docking the Menu Bar, Abstract Classes vs. Interfaces, and More

Paul DiLascia Talking To…: Steve Lombardi Discusses the Highlights of Microsoft MapPoint 2004


Threading: Break Free of Code Deadlocks in Critical Sections Under Windows

[ Editor's Update - 7/23/2004: This article is intended for informational purposes only and its contents should not be used to create production code. The article discusses undocumented data structures and processes internal to the Windows operating system that are subject to change in future versions, such as Longhorn.]
Threading

Break Free of Code Deadlocks in Critical Sections Under Windows

Matt Pietrek and Russ Osterlund
This article assumes you're familiar with Win32, C++, and Multithreading
Level of Difficulty 1 2 3
Code download available at: CriticalSections.exe (415KB)
SUMMARY

Critical sections, a mechanism that prohibits more than one thread at a time from executing a particular section of code, is a topic that has not received much attention and thus tends not to be well understood. A solid understanding of critical sections in Windows can really come in handy when you need to track down multithreading performance issues in your code. This articles delves under the hood of critical sections to reveal information useful in finding deadlocks and in pinpointing performance problems. It also includes a handy utility program that shows all of your critical sections and their current states.

Contents
Critical Sections: a Brief Review
Digging In: the RTL_CRITICAL_SECTION Structure
The RTL_CRITICAL_SECTION_DEBUG Structure
Critical Section States
Building a Better Mousetrap
The MyCriticalSections Utility
Spelunking in Key Critical Section Routines
Conclusion

I
n our many combined years of programming, we're quite surprised that Win32® critical sections haven't received very much "under the hood" attention. Sure, you probably know the basics of initializing and using critical sections, but have you ever taken the time to dig into the CRITICAL_SECTION structure defined in WINNT.H? There are some interesting goodies in this structure that have long been ignored. We'll remedy that and show you some interesting tidbits that come in handy when tracking down elusive multithreading bugs. More importantly, using our MyCriticalSections utility, you'll see how a CRITICAL_SECTION can be trivially extended to provide very slick features that can be used for both debugging and performance tuning (for complete code download, see the link at the top of this article).

To be honest, part of the reason that writers have ignored the CRITICAL_SECTION structure is that it's implemented very differently under the two main Win32 code bases: Microsoft® Windows® 95 and Windows NT®. Everyone knows that both code bases have spawned numerous descendents (most recently, Windows Me and Windows XP, respectively), but it's not necessary to list them all here. The point is that now that Windows XP is well established, developers can soon drop support for the Windows 95 line of operating systems. We have done so in this article.

Admittedly, most of the interest these days is in the Microsoft .NET Framework, but good old-fashioned Win32 programming isn't going away anytime soon. If you have existing Win32 code that makes use of critical sections, you'll find our tools and descriptions of critical section states to be useful. However, it's important to note that we're only discussing Windows NT and its successors and won't be covering anything related to .NET.

Critical Sections: a Brief Review

If you're intimately familiar with critical sections and can use them without blinking, feel free to skip this section. Otherwise, read along for a quick refresher. The sections that follow this one won't make much sense unless you are familiar with the basics.

A critical section is a lightweight mechanism for allowing only one thread at a time to execute a given piece of code. Critical sections are typically used when modifying global data such as collection classes. Unlike events, mutexes, and semaphores, which are also used for multithreaded synchronization, critical sections don't always perform an expensive control transfer to kernel mode. As you'll see later, acquiring an unheld critical section requires, in effect, just a few memory modifications and is very quick. Only if you try to acquire an already-held critical section does it jump into kernel mode. The downside to this lightweight behavior is that critical sections can only be used to synchronize threads within the same process.

A critical section is represented by the RTL_CRITICAL_SECTION structure defined in WINNT.H. You might not be aware of this because your C++ code typically declares a variable of type CRITICAL_SECTION. Examine WINBASE.H and you'll find this:

typedef RTL_CRITICAL_SECTION CRITICAL_SECTION;

We'll dig through the RTL_CRITICAL_SECTION structure in short order. For the moment, the important thing is that a CRITICAL_SECTION (also known as an RTL_CRITICAL_SECTION) is just a structure with readily accessible fields that are manipulated by KERNEL32 APIs.

The life of a critical section begins when it's passed to InitializeCriticalSection (or more accurately, when its address is passed). Once initialized, your code passes the critical section to the EnterCriticalSection and LeaveCriticalSection APIs. Once a thread returns from EnterCriticalSection, all other threads that call EnterCriticalSection block until the first thread calls LeaveCriticalSection. Finally, when you no longer need the critical section, good coding practice dictates that you pass it to DeleteCriticalSection.

In the ideal case where a critical section is unowned, a call to EnterCriticalSection is very fast because it simply reads and modifies memory locations within user-mode memory. Otherwise (with one exception that we'll get to later), threads that block on a critical section do so efficiently, without burning up extra CPU cycles. Blocked threads wait in kernel mode and aren't schedulable until the critical section owner releases it. If multiple threads are blocked on a critical section, only one thread acquires the critical section when another thread releases it.

Digging In: the RTL_CRITICAL_SECTION Structure

Even if you've used critical sections in your everyday work, odds are you haven't really looked beyond the documentation. It turns out there's quite a lot that can be learned relatively easily. For instance, it's not well known that a process's critical sections are kept in a linked list and can be enumerated. In fact, WINDBG supports the !locks command, which lists all the critical sections of the target process. The utility that we'll come to shortly also uses this little-known aspect of critical sections. In order to really understand how our utility works, it's essential to truly grasp the internals of critical sections. With that in mind, let's begin by examining the RTL_CRITICAL_SECTION structure. For convenience, the structure is shown here:

struct RTL_CRITICAL_SECTION
{
    PRTL_CRITICAL_SECTION_DEBUG DebugInfo;
    LONG LockCount;
    LONG RecursionCount;
    HANDLE OwningThread;
    HANDLE LockSemaphore;
    ULONG_PTR SpinCount;
};

The following paragraphs describe each field.

DebugInfo This field contains a pointer to a system-allocated companion structure of type RTL_CRITICAL_SECTION_DEBUG. This structure contains more nuggets of information and is also defined in WINNT.H. We'll look at it in more depth shortly.

LockCount This is the most important field in a critical section. It is initialized to a value of -1; a value of 0 or greater indicates that the critical section is held or owned. When it's not equal to -1, the OwningThread field (this field is incorrectly defined in WINNT.H—it should be a DWORD instead of a HANDLE) contains the thread ID that owns this critical section. The delta between this field and the value of (RecursionCount -1) indicates how many additional threads are waiting to acquire the critical section.

RecursionCount This field contains the number of times that the owning thread has acquired this critical section. If this value is zero, the next thread that attempts to acquire the critical section will be successful.

OwningThread This field contains the thread identifier for the thread that currently holds the critical section. This is same thread ID that APIs like GetCurrentThreadId return.

LockSemaphore This field is misnamed; it's really an auto-reset event, not a semaphore. It's a kernel object handle and it's used to signal the operating system that the critical section is now free. The operating system silently creates one of these the first time a thread tries to acquire the critical section, but is blocked by another thread already owning it. You should call DeleteCriticalSection (which issues a CloseHandle call on the event and frees the debug structure if necessary) or a resource leak will occur.

SpinCount Only used on multiprocessor systems. The MSDN® documentation describes this field as follows: "On multiprocessor systems, if the critical section is unavailable, the calling thread will spin dwSpinCount times before performing a wait operation on a semaphore associated with the critical section. If the critical section becomes free during the spin operation, the calling thread avoids the wait operation." Spin counts can offer better performance on multiprocessor machines since it's often faster to spin in a loop than to enter a kernel-mode wait state. This field defaults to zero, but can be set to a different value with the InitializeCriticalSectionAndSpinCount API.

The RTL_CRITICAL_SECTION_DEBUG Structure

Earlier we noted that within the RTL_CRITICAL_SECTION structure, the DebugInfo field points to an RTL_CRITICAL_SECTION_DEBUG structure, which is shown here:

struct _RTL_CRITICAL_SECTION_DEBUG
{
    WORD   Type;
    WORD   CreatorBackTraceIndex;
    RTL_CRITICAL_SECTION *CriticalSection;
    LIST_ENTRY ProcessLocksList;
    DWORD EntryCount;
    DWORD ContentionCount;
    DWORD Spare[ 2 ];
}
This structure is allocated and initialized by InitializeCriticalSection. It is allocated either from a pre-allocated array inside NTDLL or from the process heap. This companion to the RTL_CRITICAL_SECTION contains a collection of matching fields with widely disparate roles: two are mysterious, the next two provide the keys for understanding the structure of the critical section chain, two are Siamese twins, and the final two are unused.

Here's a description of the RTL_CRITICAL_SECTION fields.

Type This field is not used and is initialized to a value of 0.

CreatorBackTraceIndex This field is only used for diagnostic situations. Under the registry key HKLM\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\YourProgram are the keyfield, GlobalFlag, and StackTraceDatabaseSizeInMb values. Note that these values only appear if you run the Gflags commands described later. When these registry values are set correctly, the CreatorBackTraceIndex field is filled with an index value used in the stack trace. More information on this can be found by performing a search on MSDN for the phrases "create user mode stack trace database" and "enlarging the user-mode stack trace database" in the GFlags documentation.

CriticalSection Points to the RTL_CRITICAL_SECTION associated with this structure. Figure 1 illustrates the underlying structure and relationships between RTL_CRITICAL_SECTION, RTL_CRITICAL_SECTION_DEBUG, and other participants in the chain of events.

Figure 1 Critical Section Process Flow
Figure 1 Critical Section Process Flow

ProcessLocksList A LIST_ENTRY is the standard Windows data structure used to represent nodes in a doubly linked list. The RTL_CRITICAL_SECTION_DEBUG contains one as part of a linked list, allowing the critical sections to be traversed forwards and backwards. The utility presented later in this article demonstrates how to use the Flink (forward link) and Blink (back link) fields to move from member to member in a list. Anyone who has worked with device drivers or done any spelunking in the Windows kernel will recognize this data structure as an old friend.

EntryCount/ContentionCount These fields are both incremented at the same time and for the same reason. It is the number of threads that have entered a wait state because they could not immediately acquire the critical section. Unlike the LockCount and RecursionCount fields, these fields are never decremented.

Spares These two fields aren't used or even initialized (although they are zeroed out when the critical section structures are deleted). As we'll show later, these unused fields can be put to work to hold useful diagnostic values.

Even though the RTL_CRITICAL_SECTION_DEBUG contains a hodgepodge of fields, it is a necessary component to the regular critical section structure. In fact, if the system happens to be unable to acquire storage for this structure from the process' heap, InitializeCriticalSection returns a LastError result of STATUS_NO_MEMORY and then returns a critical section structure in an incomplete state.

Critical Section States

As your program executes, and enters and leaves critical sections, the fields in RTL_CRITICAL_SECTION and RTL_CRITICAL_SECTION_DEBUG structures change according to the state the critical section is in. The fields are updated by bookkeeping code in the critical section APIs, which we'll see later. These states are more interesting if your program is multithreaded and its thread access is a common resource guarded by a critical section.

There are, however, two states that are present regardless of the thread usage of the code. First, if the LockCount field has a value other than -1, the critical section is held and the OwningThread field contains the thread identifier of the thread that owns the critical section. In multithreaded programs, the LockCount in conjunction with the RecursionCount will tell how many threads are currently blocked on the critical section. Second, if RecursionCount is a value greater than 1, this tells you how many times the owning thread has (perhaps unnecessarily) reacquired the critical section, either by calling EnterCriticalSection or TryEnterCriticalSection. Any value that's greater than 1 may be an indication of inefficiencies or future bugs in your code. For example, any of your C++ class methods that access a common resource may be reentering the critical section unnecessarily.

It is important to note that the vast majority of the time, the LockCount and RecursionCount fields contain their initialized values, -1 and 0, respectively. In fact, for single-threaded programs, you cannot tell by merely examining these fields if the critical section has ever been acquired. However, multithreaded programs leave behind a few indications of whether two or more threads have attempted to own the same critical section at the same time.

One of the indications you'll find is when the LockSemaphore field contains a nonzero value even when the critical section is not held. This indicates that, at one time, the critical section blocked one or more threads—the event handle is used to signal that the critical section was freed up and that one of the threads waiting on the critical section could now acquire it and continue execution. Because the OS silently allocates the event handle when a critical section blocks another thread, if you neglect to delete your critical sections when they are no longer needed, the LockSemaphore field can create resource leaks in your program.

Another state that you may see in a multithreaded program is one in which the EntryCount and ContentionCount fields contain a value greater than zero. These two fields hold the number of times that the critical section blocked a thread. They are incremented each time this happens, but are never decremented during the lifetime of the critical section. These fields can be used to indirectly determine the execution path and characteristics of your program. For instance, a very high EntryCount means that the critical section is undergoing a lot of contention and is a potential bottleneck in the execution of the code.

While examining a deadlocked program, we also found a state for which there seemed to be no logical explanation. The LockCount field for one heavily used critical section contained a value greater than -1; that is, it was owned, but the OwningThread field was zero (which destroyed evidence against the guilty thread). The test program was multithreaded, and the condition arose on both single and multiprocessor machines. Although LockCount and other values differed from run to run, the program always deadlocked on the same critical section. We'd definitely be curious to find out if any other developers out there have seen the API call sequence leading up to this state.

Building a Better Mousetrap

As we were learning about the workings of critical sections, we stumbled across a few key discoveries that make a nifty utility possible. The first was the presence of the ProcessLocksList LIST_ENTRY field which gave us the idea that a process's critical sections might be enumerable. An additional big discovery was that we learned how to find the head of the critical section list. Another key insight was that the Spare fields of the RTL_CRITICAL_SECTION could be written to with impunity (at least in all of our testing). We also found that we could easily override some of the system's critical section routines without requiring any modification to source files.

Initially, we started out with a simple program that walked all the critical sections of a process and listed their current states to see if they were owned or unowned. If owned, we'd find out which thread owned it, and how many threads were blocked on it? While doing this is mildly intriguing to the OS enthusiast, it's not enough to be extremely useful for the typical programmer who just wants a little help understanding their program.

In even the simplest console-mode "Hello World" program there are dozens of critical sections. The vast majority are created by system DLLs such as USER32 or GDI32, and are rarely responsible for deadlocks or performance issues. We wanted a way to filter out these critical sections and leave only the interesting ones in your code. The Spare fields in the RTL_CRITICAL_SECTION_DEBUG structure work quite nicely for this. You can use either or both of them to indicate that a critical section is from user-written code, rather than from the OS.

The next logical issue then becomes how to determine which critical sections are from your code. Some of you may remember LIBCTINY.LIB from Matt Pietrek's January 2001 Under The Hood column. One of the tricks LIBCTINY employs is a LIB file that overrides the standard implementation of key Visual C++ runtime routines. By putting the LIBCTINY.LIB file ahead of other LIBs on the linker line, the linker uses that implementation, rather than subsequent versions with the same name from the import libraries supplied by Microsoft.

To make similar magic happen for critical sections, we created a replacement version of InitializeCriticalSection and an associated import library. By placing this LIB file ahead of KERNEL32.LIB, the linker links against our version, rather than that of KERNEL32. Our implementation of InitializeCriticalSection is shown in Figure 2. The code is conceptually very simple. It first calls the real InitializeCriticalSection in KERNEL32.DLL. Next, it obtains the address of the code that called InitializeCriticalSection and sticks it in one of the spare fields of the RTL_CRITICAL_SECTION_DEBUG structure. How does our code determine the calling code's address? An x86 CALL instruction places the return address on the stack. The CriticalSectionHelper code knows that the return address will be at a known, fixed location within the stack frame.

Figure 2 CriticalSectionHelper.cpp (Close Figure 2)
//======================================================================
// Russ Osterlund / Matt Pietrek
// MSDN Magazine, 2003
//======================================================================
#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include "CriticalSectionHelper.h"

typedef void (__stdcall *PFNINITIALIZECRITICALSECTION)(
    LPCRITICAL_SECTION );

static PFNINITIALIZECRITICALSECTION s_pfnInitializeCriticalSection = 0;

#pragma optimize( "y", on ) // Ensure that we get standard stack frames
#pragma warning(disable :4273) // We're overloading a function
                               // from WinBase.H. The compiler
                               // will be unhappy, so quiet it.
extern "C"
void
__stdcall
InitializeCriticalSection(LPCRITICAL_SECTION lpCriticalSection )
{
    if ( !s_pfnInitializeCriticalSection )
    {
        HMODULE hModuleKERNEL32 =
            GetModuleHandle( "KERNEL32.DLL" );

        s_pfnInitializeCriticalSection =
            (PFNINITIALIZECRITICALSECTION)
            GetProcAddress( hModuleKERNEL32,
            "InitializeCriticalSection" );
    }

    s_pfnInitializeCriticalSection( lpCriticalSection );

    DWORD   pReturnAddress;
    __asm   mov     eax, dword ptr [ebp+4]
    __asm   mov     [pReturnAddress], eax

    // Set one of the "spare" fields in the critical section to
    // the return address.  We subtract 6, since the return
    // address is after the "CALL" instruction, which is
    // typically 6 bytes.  Being off by 6 bytes is usually
    // enough for the line # lookup to pick the wrong line of
    // code.
    lpCriticalSection->DebugInfo->Spare[0] = pReturnAddress 6;
    lpCriticalSection->DebugInfo->Spare[1] =
        MYCRITSECT_SIGNATURE;
}

#if 0
extern "C" __stdcall
BOOL
InitializeCriticalSectionAndSpinCount(LPCRITICAL_SECTION
    lpCriticalSection, DWORD dwSpinCount );
#endif

The net effect is that any EXE or DLL that's linked correctly with our CriticalSectionHelper.lib will import our DLL (CriticalSectionHelper.DLL), and will have critical sections with their spare fields put to use. This makes things a lot easier. Now our utility can simply walk all the critical sections in a process and only display information for the critical sections with a properly filled-in Spare field. Now how much would you pay for this utility? But wait just a minute, there's more!

Since all your critical sections will now contain the address where they were initialized, a utility can identify each individual critical section by providing its initialization address. Raw code addresses by themselves aren't that helpful, though. Luckily, DBGHELP.DLL makes it almost trivial to translate a code address into a source file, line number, and function name. Even if a critical section doesn't have your signature in it, its address can be given to DBGHELP.DLL. If it was declared as a global variable, and if symbols are available, odds are that you can determine the critical section's name in the original source code. As a side note, you'll get great results if you let DbgHelp work its magic by setting the _NT_SYMBOL_PATH environment variable, and setting up DbgHelp to use its Symbol Server download functionality.

The MyCriticalSections Utility

Putting all our ideas together, we came up with the MyCriticalSections program. MyCriticalSections is a command-line program with a few options that are seen by running it with no arguments:

Syntax: MyCriticalSections <PID> [options]
        Options:
        /a = all critical sections
        /e = show only entered critical sections
        /v = verbose

The only required argument is a Program ID or PID (in decimal form). PIDs can be obtained by many means, but the easiest way is probably through the Task Manager. With no other options, MyCriticalSections lists the state of all critical sections from your code modules that you've linked CriticalSectionHelper.DLL into. If there are symbols available for the module(s), the code attempts to provide the name of the critical section and where it was initialized.

To see MyCriticalSections in action, run the Demo.EXE program that's included with the download. Demo.EXE simply initializes two critical sections and enters them from a couple of threads. Figure 3 shows the results of running "MyCriticalSections 2040" (where 2040 is the PID of Demo.EXE).

Figure 3 MyCriticalSections Output (Close Figure 3)
Initializing symbol engine
Symbol engine initialized OK
-------------------------------------------------------------------------
Critical Section 70
Address: 0x00437B38 !csMain
Initialized in function DEMO!main+00000025
Initialized at c:\projects\mycriticalsections\demo\demo.cpp, line 19
    LockCount: 0
    Recursion: 1
    Held by:   2348 (92C)
    DebugInfo:
        Entry Count:      0
-------------------------------------------------------------------------
Critical Section 71
Address: 0x00437B20 DEMO!yetAnotherCriticalSection
Initialized in function DEMO!main+0000004D
Initialized at c:\projects\mycriticalsections\demo\demo.cpp, line 22
    LockCount: 3
    Recursion: 3
    Held by:   2348 (92C)
    Threads Waiting: 1
    DebugInfo:
        Entry Count:      1

125 Critical Sections examined, 2 initialized with signature

In the figure, there are two critical sections listed. In this example, they're named csMain and yetAnotherCriticalSection. Each Address: line shows the address of the CRITICAL_SECTION along with its name. The "Initialized in" lines contain the name of the function in which the CRITICAL_SECTION was initialized. The "Initialized at" lines of code show the source file and line number within the initializing function.

For the csMain critical section, you'll see a lock count of 0 and a recursion count of 1, indicating a critical section that's been acquired by one thread and with no other threads waiting on it. Since no thread has ever blocked on it, the Entry Count field is 0.

Turning to yetAnotherCriticalSection, you'll see a recursion count of three. A quick look at the Demo code shows that the main thread calls EnterCriticalSection three times, so things are working as expected. However, a second thread has also attempted to acquire it and has blocked. As such, the LockCount field is also 3. The output shows that there's one waiting thread.

MyCriticalSections has a few options that make it useful for the more intrepid explorer. The /v switch displays more information for each critical section. The spin count and lock semaphore fields are of particular interest. You'll often see that NTDLL and other DLLs have critical sections with nonzero spin counts. The lock semaphore field will be nonzero if a thread has ever blocked while in the process of acquiring the critical section. The /v switch also shows the contents of the spare fields in the RTL_CRITICAL_SECTION_DEBUG structure.

The /a switch displays all critical sections in the process, even if they don't have the CriticalSectionHelper.DLL signature in place. If you use /a, be prepared for lots of output. The hardcore hacker will want to use /a and /v together to show the maximum amount of detail for everything in the process. One of the side benefits of using /a is that you'll see the LdrpLoaderLock critical section in NTDLL. This critical section is held during DllMain calls and at other key times. LdrpLoaderLock has been the silent partner in many of our obscure, seemingly inexplicable deadlocks. (In order for MyCriticalSection to correctly label the LdrpLoaderLock instance, you'll need the PDB file for NTDLL to be available.)

The /e switch causes the program to display only critical sections that are currently held. Without the /a switch, it only shows held critical sections in your code (as indicated by our signature in the spare field). With the /a switch, it shows all owned critical sections in the process, regardless of where they're from.

So, when would you want to run MyCriticalSections? An obvious time is when a program has deadlocked. Examine the held critical sections to see if anything jumps out at you. You can use MyCriticalSections even if the deadlocked program is being run under the control of a debugger.

Another use for MyCriticalSections is when you're performance tuning highly multithreaded programs. When you're stopped in a heavily used, non-reentrant function in the debugger, run MyCriticalSections to see what critical sections are held at that moment. If you have many threads all performing the same work, it's easy to get into situations where the majority of a thread's time is spent waiting to acquire a heavily used critical section. If you have several heavily used critical sections, they're similar to kinks in a garden hose. Fixing one contention problem simply shifts the problem to the next bottlenecking critical section.

A good way to see what the most contentious critical sections are is to set a breakpoint near the end of your program. When you hit the breakpoint, run MyCriticalSections and look for critical sections with the largest Entry Count values. These are the critical sections that have caused the most blocking and resultant thread switches.

Although MyCriticalSections runs on Windows 2000 and later, you'll need a relatively new version of DbgHelp.DLL—version 5.1 or later. This is the version that comes with Windows XP. You can also get it from other tools that use DbgHelp. For example, the Debugging Tools For Windows download usually has the most recent DbgHelp.DLL.

Spelunking in Key Critical Section Routines

This final section is for the fearless reader who wants to understand the guts of the critical section implementation. Doing a careful examination of NTDLL, we were able to create pseudocode for these routines and their supporting subroutines (see NTDLL(CriticalSections).cpp in the download). The following KERNEL32 APIs make up the public interface to critical sections:

InitializeCriticalSection
InitializeCriticalSectionAndSpinCount
DeleteCriticalSection
TryEnterCriticalSection
EnterCriticalSection
LeaveCriticalSection

The first two APIs are merely thin wrappers around the NTDLL APIs RtlInitializeCriticalSection and RtlInitializeCriticalSectionAndSpinCount, respectively. All the remaining routines are forwarded to functions in NTDLL. What's more, a call to RtlInitializeCriticalSection is another thin wrapper around a call to RtlInitializeCriticalSectionAndSpinCount, with a spin-count value of 0. Behind the curtain, you're really using the following NTDLL APIs when using critical sections:

RtlInitializeCriticalSectionAndSpinCount
RtlEnterCriticalSection
RtlTryEnterCriticalSection
RtlLeaveCriticalSection
RtlDeleteCriticalSection
In this discussion, we'll use the Kernel32 names because they are more familiar to most Win32 programmers.

The initialization of a critical section by InitializeCriticalSectionAndSpinCount is straightforward. The fields in the RTL_CRITICAL_SECTION structure are given their starting values. Likewise, the RTL_CRITICAL_SECTION_DEBUG structure is allocated and initialized, the CreatorBackTraceIndex is given the return value from a call to RtlLogStackBackTrace, and links to the preceding critical section are established.

Incidentally, CreatorBackTraceIndex normally receives a value of 0. If you have the Gflags and Umdh utilities, however, you can enter the following commands:

Gflags /i MyProgram.exe +ust
Gflags /i MyProgram.exe /tracedb 24
These commands cause registry entries to be added under "Image File Execution Options" for MyProgram. You should now see this field receive a value other than 0 with the next execution of MyProgram. For more information, take a look at the Knowledge Base article Q268343, "Umdhtools.exe: How to Use Umdh.exe to Find Memory Leaks". One additional point to note with the initialization of critical sections is that the first 64 RTL_CRITICAL_SECTION_DEBUG structures are not allocated from the process's heap, but instead come from an array located inside NTDLL's .data section.

When you have finished using a critical section, the call to DeleteCriticalSection (badly named because only the RTL_CRITICAL_SECTION_ DEBUG is deleted) travels an equally understandable path. If an event was created because threads were blocked trying to acquire the critical section, the event is destroyed with a call to ZwClose. Next, after acquiring protection through the RtlCriticalSectionLock (NTDLL guards its own internal critical section list with—you guessed it—a critical section), the critical section linked-list is updated to reflect the destruction of the debug information by eliminating it from the chain. The memory is null-filled and, if its storage was acquired from the process' heap, a call to RtlFreeHeap causes its memory to be releasd. Finally, the RTL_CRITICAL_SECTION is zero filled.

There are two APIs to acquire a resource guarded by a critical section, TryEnterCriticalSection and EnterCriticalSection. If a thread needs to enter a critical section but can perform useful work while waiting for a blocked resource to become free, then TryEnterCriticalSection is your ticket. This routine tests if the critical section is available; if it isn't free, the code returns with a value of FALSE, giving the thread the opportunity to continue with another task. Otherwise, it acts just like EnterCriticalSection.

If the thread absolutely needs to own the resource before continuing, use EnterCriticalSection. For the moment, set aside the SpinCount test used on multiprocessor machines. This routine, like TryEnterCriticalSection, adjusts the bookkeeping for the critical section if it is free or already owned by the thread. It's important to note that the all-important LockCount increment is done using the x86 "lock" prefix. This ensures that only one CPU at a time can modify the LockCount field. (In fact, the Win32 InterlockedIncrement API is just an ADD instruction with the same lock prefix.)

If the calling thread can't immediately acquire the critical section, RtlpWaitForCriticalSection is called to place the thread into a wait state. On multiprocessor systems, EnterCriticalSection spins the number of times specified by SpinCount and tests for the availability of the critical section during each loop iteration. If the critical section becomes free during this looping, the thread acquires the critical section and continues executing.

RtlpWaitForCriticalSection is probably the most complicated and important of all the procedures presented here. This isn't too surprising because if you have a deadlock and critical sections are involved, breaking into the process with a debugger will likely show at least one thread in a ZwWaitForSingleObject call inside RtlpWaitForCriticalSection.

As the pseudocode shows, there's a bit of bookkeeping in RtlpWaitForCriticalSection, such as incrementing the EntryCount and ContentionCount fields. More important, however, is the issuance of a wait on the LockSemaphore and handling the wait result. The default case passes a null pointer as the third parameter to the ZwWaitForSingleObject call, requesting that the wait never times out. When timeouts are permitted, debug message strings are generated and the wait begins again. Anything other than a successful return from the wait results in an error that halts the process. Finally, when there is a successful return from the ZwWaitForSingleObject call, execution returns from RtlpWaitForCriticalSection and the thread now owns the critical section.

One edge condition that RtlpWaitForCriticalSection must remain cognizant of is when the process is shutting down and the wait is for the loader lock (LdrpLoaderLock) critical section. RtlpWaitForCriticalSection must not allow the thread to block, but must skip the wait and allow the shutdown to continue.

LeaveCriticalSection is not nearly as complicated as EnterCriticalSection. If after decrementing the RecursionCount, the result is not 0 (meaning the thread still owns the critical section), the routine returns with a status of ERROR_SUCCESS. This is why you need to balance Enter calls with the appropriate number of Leave calls. If the count is 0, the OwningThread field is zeroed out and the LockCount decremented. If there are other threads waiting, for example, the LockCount is greater than or equal to 0, RtlpUnWaitCriticalSection is called. This helper routine creates (if it does not already exist) the LockSemaphore and signals it to alert the operating system that the thread has released the critical section. As part of the signaling, one of the waiting threads is taken out of the waiting state and made ready to run.

As a final note, how does the MyCriticalSections program determine the beginning of the critical section chain? If you have access to the correct debug symbols for NTDLL, finding and walking the list is trivial. First, locate the symbol RtlCriticalSectionList, dump its contents (it points to the first RTL_CRITICAL_SECTION_DEBUG structure), and start the walk. Not all systems will have debug symbols, however, and the address for the RtlCriticalSectionList variable changes with each version of Windows. In order to provide a solution that works for all versions, we devised the following heuristic. By observing the steps taken when starting a process, you'll see that critical sections inside of NTDLL are initialized in this order (the names are taken from the debug symbols for NTDLL):

RtlCriticalSectionLock
DeferedCriticalSection (this is the actual spelling!)
LoaderLock
FastPebLock
RtlpCalloutEntryLock
PMCritSect
UMLogCritSect
RtlpProcessHeapsListLock

Since the loader lock can be found by examining the address at offset 0xA0 in the Process Environment Block (PEB), it becomes relatively simple to locate the beginning of the chain. We read the debug information for the loader lock and then walk the chain backwards two links, which places us at the RtlCriticalSectionLock item, at which point we have the first critical section in the chain. See Figure 4 for an illustration of their method.

Figure 4 Initialization Order
Figure 4 Initialization Order

Conclusion

Critical sections are used in almost all multithreaded programs. Sooner or later you'll run up against a critical section that deadlocks your code, and you'll be stymied trying to determine how you got into the current state. With a deeper knowledge of how critical sections work, this scenario doesn't have to be as bleak as it first appears. You can examine a supposedly opaque critical section and determine who owns it, along with other useful details. If you're willing to add our library to your linker line, it's trivial to get far more information about your program's critical section usage. By taking advantage of unused fields in the critical section structure, our code can isolate and name just the critical sections your modules use, and tell you their exact state as well.

The enterprising reader can easily extend our code to do even more exotic things. For instance, by intercepting EnterCriticalSection and LeaveCriticalSection in a similar manner to our InitializeCriticalSection hooking, you could store away where the critical section was last successfully acquired and released. Likewise, the CritSect DLL has an easily callable API for enumerating the critical sections in your own code. Using Windows Forms in the .NET Framework, you could create a GUI version of MyCriticalSections with relative ease. The possibilities for extending our code are wide open, and we'd like to see what innovative ideas other people discover and develop.

For related articles see:
Global Flag Reference: Create kernel mode stack trace database
GFlags Examples: Enlarging the User-Mode Stack Trace Database
Under the Hood: Reduce EXE and DLL Size with LIBCTINY.LIB

Matt Pietrek is a software architect and writer. He works for Compuware/NuMega Labs as a lead architect for the BoundsChecker and Distributed Analyzer products. He has authored three books on Windows system programming and is a contributing editor for MSDN Magazine. His Web site (http://www.wheaty.net) has a FAQ and info on previous articles and columns.

Russ Osterlundis a software engineer in the BoundsChecker group with Compuware/NuMega Labs. He, his wife, and their cat are new residents of the state of New Hampshire. He can be reached at RussOsterlund@adelphia.net or at his Web site, http://www.smidgeonsoft.com.

© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


MSMQ and .NET: Send MSMQ Messages Securely Across the Internet with HTTP and SOAP

MSMQ and .NET

Send MSMQ Messages Securely Across the Internet with HTTP and SOAP

David S. Platt
This article assumes you're familiar with .NET Framework and C#
Level of Difficulty 1 2 3
Code download available at: MSMQandNET.exe (375KB)
SUMMARY

When creating a distributed system you frequently need to provide for communication between two entities that are not in sync. Microsoft Message Queue Server (MSMQ) provides the kind of store-and-forward messaging in a pre-built infrastructure that can help you address these kinds of messaging needs. In the past, MSMQ was accessed using a COM wrapper. Now there's a .NET wrapper that lets you accomplish your messaging goals easily from your Framework-based code. To illustrate the use of the wrapper, the author builds a messaging application, sends MSMQ messages over the Web, and discusses messaging security.

Contents
Simplest Example
More Complex Example: Formatter Choices
MSMQ and .NET Framework Trust
Sending MSMQ Messages over the Internet
Conclusion

D
esigners of distributed systems often need to communicate between two systems that are not running simultaneously. Store-and-forward messaging systems such as Microsoft® Message Queue Server (MSMQ) provide the prefabricated infrastructure that applications need to accomplish this task. In this article, I will discuss how to use MSMQ from the Microsoft .NET Framework, as well as the latest improvements in MSMQ 3.0, particularly its operation over HTTP networks.

Simplest Example

Programmers originally accessed MSMQ either through a native API or a COM wrapper. Microsoft has since added a .NET wrapper layer that provides easy access to .NET Framework-based applications, as shown in Figure 1.

Figure 1 MSMQ Wrappers
Figure 1 MSMQ Wrappers

To get you started, I've written a very simple example that demonstrates the use of MSMQ in the .NET Framework. It is a .NET solution containing three projects: a sender, a recipient, and an administrator. You can download it and other samples from the link at the top of this article. For convenience, I've written them to work on a single box.

The .NET wrapper that provides access to all MSMQ operations lives in the system assembly named System.Messaging.dll. To access MSMQ from .NET, you must first add a reference to System.Messaging.dll to the project.

A queue is a persistent structure that exists on a particular machine. You need to create this persistent structure before you can write to it or read from it. In many sample programs, either the sender or the recipient creates the queue, but in production, an application's installation or configuration program usually does that. The sample program, SimplestQueuingAdministrator, simulates the production example. When the user clicks Create, I call the static method System.Messaging.MessageQueue.Create, which creates the persistent structure at the location specified by the textbox string. Since attempting to create a queue that already exists causes an exception, the method MessageQueue.Exists allows you to check beforehand whether the queue with the specified name exists. Figure 2 shows the code for these methods. (Note that Exists and Create don't work on remote private queues.)

Figure 2 Check for Queue and Create Queue (Close Figure 2)
// User clicked "Exists ?" button. Find out if specified queue
// does or doesn't exist, and report to user.

private void button1_Click(object sender, System.EventArgs e)
{
    bool bDoesQueueExist = System.Messaging.MessageQueue.Exists
        (textBox1.Text) ;

    MessageBox.Show (bDoesQueueExist.ToString()) ;
}

// User clicked the "Create" button. Attempt to create the
// queue with the name entered by the user.

private void button2_Click(object sender, System.EventArgs e)
{
    System.Messaging.MessageQueue.Create(textBox1.Text);
}

Queues can be either public or private. The term "private" as used in MSMQ is highly misleading. It doesn't mean that a program needs special permission to access it, but rather that the queue is not listed by name in the directory of the MSMQ domain controller. Any program that knows the name of the private queue can access it in the same manner as it would for a public queue. Unlisted would be a better name than private because a private queue behaves exactly like an unlisted phone number, no more and no less. Creating public queues requires a domain controller to maintain that public directory, so I've made the default queue names private. That way you can run the sample programs on a single machine without having a domain controller somewhere in the system. You specify a private queue by placing the string "\private$\" between the machine name and the queue name.

Figure 3 Simplest Queuing Sender
Figure 3 Simplest Queuing Sender

Now that I've created the queue, I can send messages to it with the sender application, called SimplestQueueingSender, shown in Figure 3. I find that the coolest part of the .NET wrapper is that it abstracts away almost all of the details of MSMQ. As Alan Cooper wrote in his book About Face 2.0: The Essentials of Interaction Design (John Wiley & Sons, 2003), "any good interface makes simple things simple and complex things possible." You don't have to think about message headers or priorities if you don't want to. Instead, you can just shove .NET objects into one end and they magically come out the other end in a reasonable default fashion. The sample program demonstrates this by sending objects of class System.Drawing.Point, as shown in the code in Figure 4.

Figure 4 Sample Sending Program Listing (Close Figure 4)
private void button1_Click(object sender, System.EventArgs e)
{
    try
    {

    // Create a connection to the queue
    // Note: If you're going to another machine in workgroup mode,
    // prepend the string "FormatName:Direct=OS:" to the machine name.

        System.Messaging.MessageQueue mq =
            new System.Messaging.MessageQueue (textBox1.Text);

        // Create a point object to send

        Point myPoint =
            new Point (Convert.ToInt32(textBox2.Text),
                Convert.ToInt32(textBox3.Text)) ;

        // Send object

        mq.Send (myPoint) ;

    }

    // Catch the exception that signals all types of errors from the
    // message queuing subsystem. Report error to the user.

    catch (System.Messaging.MessageQueueException mqx)
    {
        MessageBox.Show (mqx.Message) ;
    }
}

The sender first has to open the queue for writing. This is done by instantiating an object of class System.Messaging.MessageQueue. The nomenclature can be misleading. Instantiating the object does not create the persistent queue structure, even if it doesn't yet exist. Only the previously discussed static method, MessageQueue.Create, does that. Instantiating the object opens the queue for reading or writing, which is conceptually similar to opening a file rather than creating one. If you think of the object you instantiate as a message queue connection (as I've done in the program comments) you'll have the right mental model. Once you've made the connection to the queue, you can create an object of any .NET class that supports XML serialization and pass it to the Send method of the queuing connection. The .NET wrapper now serializes the object into an XML stream and transmits that stream as the body of an MSMQ message. Other types of serialization are supported with a little more work, which I'll discuss later in the article, but XML is the default. The object will be automatically reconstituted on the recipient side, as you shall see later in this section.

If I don't want the default options for my message, I can exercise more control over it by instantiating an object of class System.Messaging.Message, and placing the object I want to send into the property named Body (not shown). I can then set the other properties of the message to the values that I want. MSMQ provides many powerful and flexible options. For example, the property Message.TimeToReachQueue specifies the amount of time allowed between a message being sent and the message reaching its final destination queue. If it doesn't reach the queue by that time, it is destroyed. The property Message.TimeToBeReceived specifies the amount of time allowed between a message being sent and that message being retrieved from the queue by the recipient.

The message is now in the queue and ready for reading. If the sender's and recipient's machines are not the same (as they are here for ease of use), the message would be buffered in an outgoing queue on the sender's machine and sent to the recipient when a connection between the two machines became available. You can see the message in the queue by using the MMC Computer Manager snap-in. The administrative sample program in this article demonstrates how easy such administrative programs are to write. When you click Enumerate, I open the queue and call the method MessageQueue.GetAllMessages. This returns an array of objects of class System.Messaging.Message, providing a static snapshot of all messages in the queue. Each message contains MSMQ properties such as priority and time sent. If you double-click the message in the listbox, my code pops up a form showing some interesting properties of the message including the body of the message containing the serialized Point structure (see Figure 5). The code is shown in Figure 6.

Figure 6 Enumerate Messages (Close Figure 6)
// User clicked "Enumerate Messages" button. Get snapshot of messages in
// the queue and place each one in the listbox for the user to see.

private void button3_Click(object sender, System.EventArgs e)
{
    // Clear out listbox

    listBox1.Items.Clear()  ;

    // Create a connection to the queue

    System.Messaging.MessageQueue mq =
        new System.Messaging.MessageQueue (textBox1.Text);

    // Get snapshot of all msgs in queue

    System.Messaging.Message[] msgs = mq.GetAllMessages () ;

    // place messages in listbox

    foreach (System.Messaging.Message m in msgs)
    {
        byte[] bytes = new byte [256] ;
        m.BodyStream.Read (bytes, 0, 256) ;
        System.Text.ASCIIEncoding ascii = new System.Text.ASCIIEncoding();
        listBox1.Items.Add (ascii.GetString (bytes, 0, 256)) ;
    }
}

// User double-clicked on a line. Pop up box showing msg contents

private void listBox1_DoubleClick(object sender, System.EventArgs e)
{
    MessageBox.Show (listBox1.SelectedItem.ToString(),
        "Message Contents") ;
}
Figure 5 Properties Popup
Figure 5 Properties Popup

The .NET wrapper provides two ways of receiving an MSMQ message: synchronous and asynchronous. I'll describe the synchronous message reception first since it's easier to understand, although it's probably not quite as useful in production situations. The code for a synchronous receive operation is shown in Figure 7. When the user clicks the Do It button in the Synchronous Read group box, I first create a connection to the queue, as I did for sending. I now need to specify the formatter to be used for reception. When the client sent the Point object, it didn't have to specify a formatter, as the default was set to XML. However, on reception, I do need to specify a formatter because even the default XML formatter needs to know which types of object it might see in the incoming stream. For more information on XML serialization, see Chapter 8 of my book, Introducing Microsoft .NET, Third Edition (Microsoft Press®, 2003). When I create the formatter, I pass it an array containing System.Type objects describing all the types of .NET objects that the message might contain. I could also pass an array of type names, from which the wrapper could fetch the System.Type objects, if I prefer.

Figure 7 Synchronous Reception Operation (Close Figure 7)
private void button1_Click(object sender, System.EventArgs e)
{
    // Create a connection to the queue

    System.Messaging.MessageQueue mq =
        new System.Messaging.MessageQueue (textBox1.Text);

    // Set the queue's formatter to decode Point objects

    mq.Formatter =
        new System.Messaging.XmlMessageFormatter (new Type[]
            {typeof (Point)}) ;

    // Try to receive msg

    try
    {
        // Receive message with timeout interval specified by user

        System.Messaging.Message msg = mq.Receive ( new
            TimeSpan(0,0,Convert.ToInt32(textBox2.Text))) ;

        // Convert received message to object that you think was sent

        Point pt = (Point) msg.Body ;

        // Display it to the user

        MessageBox.Show (pt.ToString(), "Synchronous Read Complete") ;
    }

    // Report any exceptions to the user. A timeout would cause
    // such an exception

    catch (System.Messaging.MessageQueueException x)
    {
        MessageBox.Show (x.Message) ;
    }

}

Once I've specified the formatter, I receive the message synchronously by calling the method MessageQueue.Receive. This method blocks the calling thread until a message is received or until a specified timeout expires, in which case it throws an exception. If you click the button while there's no message waiting, the calling thread will block. You can verify this by trying to drag the window around on the screen while waiting for a synchronous receive operation to complete. You'll find that you can't do it. Since I make the call to Receive from the same thread that handles the Windows® Forms user interface, the client can't process its internal message loop to respond to the mouse clicks attempting to move it. If the function does return successfully, the Body field of the message contains an object of the deserialized type, in this case a Point.

If I don't want the calling thread to block while it waits for an incoming message, I can do it asynchronously (see Figure 8). I open a connection to the queue and set its formatter as before. This time, instead of calling Receive directly, I create an event handler delegate and add it to the queue's ReceiveCompletedEventHandler property. This sets a handler function that will be called when the queue fires that event, which it does when a message arrives. I then call MessageQueue.BeginReceive. This hands off the operation to a background thread from the system thread pool and returns immediately. When a message does come in, the background thread calls my handler function. In it, I call the method MessageQueue.EndReceive, which returns the received message. I fetch the body and use it as in the synchronous case. Finally, I call BeginReceive again to replant the asynchronous read operation.

Figure 8 Asynchronous Message Reception (Close Figure 8)
private System.Messaging.MessageQueue AsyncReadQueue ;

private void button2_Click(object sender, System.EventArgs e)
{

    // Create a connection to the queue

    AsyncReadQueue =
        new System.Messaging.MessageQueue (textBox1.Text);

    // Set the queue's formatter to decode Point objects

    AsyncReadQueue.Formatter = new System.Messaging.XmlMessageFormatter
        (new Type[] {typeof (Point)}) ;

    // Create event handler delegate for the ReceiveCompleted event.
    // Specify the function MyOwnReceiveCompleted as the handler
    // function. Add it to the ReceiveComplete event handler list.

    AsyncReadQueue.ReceiveCompleted +=     new
        System.Messaging.ReceiveCompletedEventHandler(
        MyOwnReceiveCompleted);

    // Begin an asynchronous read operation

    AsyncReadQueue.BeginReceive ( ) ;

}

// This method gets called when the read is complete.

private void MyOwnReceiveCompleted(Object source,
    System.Messaging.ReceiveCompletedEventArgs asyncResult)
{
    try
    {

        // End the asynchronous receive operation.

        System.Messaging.Message msg =
            AsyncReadQueue.EndReceive(asyncResult.AsyncResult);

        // Convert received message to object that you think was sent

        Point pt = (Point) msg.Body ;

        // Display it to the user

        MessageBox.Show (pt.ToString(), "Asynchronous Read Complete") ;

        // Restart the asynchronous receive operation.

        AsyncReadQueue.BeginReceive();
    }
    catch(System.Messaging.MessageQueueException mqx)
    {
        MessageBox.Show (mqx.Message, "EndReceive") ;
    }

}

As you would expect, the .NET wrapper reports all MSMQ errors by means of .NET exceptions. Unfortunately, there's only one queuing-related exception—MessageQueueException. You've seen handlers for it in several of the code listings shown so far. To differentiate one type of MSMQ error from another, you have to examine the property MessageQueueErrorCode. The human-readable messages are, as usual, somewhat cryptic. For example, running the sender program without first creating the queue with the administrator program returns an error saying, "Queue is not registered in DS." The queue in question is a private queue and is therefore never registered with any DS. The error really means that MSMQ can't find that queue, but it can't quite figure out how to explain that simply and correctly to a human.

More Complex Example: Formatter Choices

As I explained in the previous section, when you send a .NET object in an MSMQ message, a piece of code called a formatter serializes that object into the message's body just before sending it. In the recipient program, a formatter deserializes the message's body into an object just before returning it to the reader. The .NET Framework provides you with a choice of three formatters: XML, binary, and ActiveX®. You can also create your own formatter by writing a class that implements the IMessageFormatter interface.

Most programmers use the XML formatter, which is the system default for sending. The sender-side object gets serialized into an XML packet, as shown in Figure 5. When the object gets deserialized, the recipient program has to provide the System.Type object describing the class to which the recipient wants the message reconstituted. This strategy is called loose coupling because the message itself does not specify the type of object to which it maps on the recipient. The recipient can deserialize the message into any class that can handle an XML packet of that layout. The sender and recipient do not have to share any code; they need only agree on the XML schema of the serialized object.

Figure 9 Serialized XML
Figure 9 Serialized XML

I've written a sample program that demonstrates the formatters used for loose and tight coupling. You'll find it in the FormatterChoices subdirectory in the download. The sender program creates an object of class SharedSchemaSendersPoint and sends it to the recipient using the XML formatter. The code is similar to the simplest example, so I won't bother showing it. The serialized XML of the message body is shown in Figure 9. The recipient, however, deserializes it into an object of a different class, SharedSchemaRecipientsPoint, as shown here:

[System.Xml.Serialization.XmlRoot ("SharedSchemaSendersPoint")]
public class SharedSchemaRecipientsPoint
{
    public int X ;
    public int Y ;
}
I've used the .NET attribute Xml.Serialization.XmlRoot to control the XML serialization so that this class expects to be deserialized from exactly this type of XML packet. Neither sender nor recipient knows or cares which class of code object the other is using to generate the XML packet. Each knows only to send or receive a message using a particular XML schema. You are basically sending XML packets through MSMQ, and the .NET object on either end is merely a convenient representation of the XML schema. Hence, loose coupling.

But suppose this loosely coupled behavior isn't what you want. The sample program also demonstrates the use of a binary formatter. When the user clicks Send SharedCodePoint, I create an object of this class and specify the use of the BinaryFormatter for the message queue, as shown in Figure 10. This causes the .NET MSMQ wrapper class to serialize the object into a binary packet rather than an XML packet. The serialized message is shown in Figure 11. You see that the serialized packet contains the assembly name, class name, and version of the object which it contains. The recipient's binary formatter will attempt to create an object of this particular class in the recipient app. As you can see in the recipient's code in Figure 12, you don't have to specify the class as you did for the XML serializer. This information is in the incoming message. But this means that the exact assembly needed by the message has to exist on the recipient side. I call this tight coupling because the sender and recipient have to agree on code, not just on data, and make sure that the right code is in the right places.

Figure 12 Recipient Using Binary Formatter (Close Figure 12)
// Create a connection to the queue

System.Messaging.MessageQueue mq =
    new System.Messaging.MessageQueue (textBox1.Text);

// Set the queue's formatter to decode the recipient's class of
// Point-like objects

mq.Formatter = new System.Messaging.BinaryMessageFormatter () ;

    // Receive message with timeout interval specified by user

System.Messaging.Message msg = mq.Receive ( new
    TimeSpan(0,0,0)) ;

    // Convert received message to object that we think was sent

SharedCodePoint.SharedCodePoint pt = (
    SharedCodePoint.SharedCodePoint) msg.Body ;
Figure 10 Sending MSMQ Message with Binary Formatter (Close Figure 10)
// Create a connection to the queue

System.Messaging.MessageQueue mq =
    new System.Messaging.MessageQueue (textBox1.Text);

// Create object to send

SharedCodePoint.SharedCodePoint  myPoint =
    new SharedCodePoint.SharedCodePoint (
        Convert.ToInt32(textBox2.Text),
        Convert.ToInt32(textBox3.Text)) ;

// Create the formatter and place it into the queue

System.Messaging.BinaryMessageFormatter bf =
    new System.Messaging.BinaryMessageFormatter ( ) ;

mq.Formatter = bf ;

// Send object

mq.Send (myPoint) ;
Figure 11 Serialized Message
Figure 11 Serialized Message

The ActiveX formatter is used when sending COM objects, which is less useful when working with the .NET Framework, so I won't be covering the use of that object in this article.

MSMQ and .NET Framework Trust

The .NET Framework provides code access security that allows an administrator to specify which operations an assembly is and isn't allowed to perform. For a fuller explanation of this, see Don Box's article "Security in .NET: The Security Infrastructure of the CLR Provides Evidence, Policy, Permissions, and Enforcement Services" in the September 2002 issue of MSDN Magazine. This control is often quite finely grained. For example, the default settings that specify an assembly's access to system environment variables allow code that runs from your local machine to read or write any of them; code that comes from the local intranet to have read-only access to the Username variable; and code that comes from the Internet to make no use of environment variables at all. The administrator can also concoct any combination of access permissions that they desire in a custom permission set, and specify which assemblies have the rights granted in that set.

The code access security permissions for the use of MSMQ are much more coarsely grained. The immediate caller of any MSMQ method needs to have the Full Trust permission set or the .NET Framework will throw a security exception. There's no explicit "Message Queuing" permission that can be granted to one assembly over another. There's also no fine-grained control that would allow an administrator to construct a permission set that would allow, say, read/write access to queue A, read-only access to queue B, and no access to any other queues. This coarse-grainedness is the automatic behavior of any shared assembly compiled without the AllowPartiallyTrustedCallers attribute. If you use ILDASM to examine System.Messaging.dll, you'll find it absent, as opposed to, say, System.Windows.Forms.dll, on which it is present and which does not enforce such restrictions on its callers.

For logistical convenience, it's pretty common to distribute production code to users from a local intranet server or even an Internet server. The full trust requirement thus poses difficulties when you want an assembly that you obtain in this manner to use MSMQ. You probably don't want to grant full trust to any random assembly from the Internet, so you have to create a new code group to which your specific MSMQ-enabled assembly will belong, and to which you grant full trust. You will want the membership condition of this code group to be as explicit as possible—being signed with a strong name that ensures it came from a trusted company, or perhaps that it came from a specific Web site that you know your administrator has control over.

Sending MSMQ Messages over the Internet

One of the most exciting things about MSMQ 3.0 (the version included in Windows XP and Windows Server™ 2003) is that you can use HTTP and SOAP to send MSMQ messages over the Internet, instead of requiring a dedicated Microsoft network connection. The basics aren't difficult. In fact, you can configure the simplest example program, which I showed in the previous section, to do this just by changing the queue names.

I've written a sample program that demonstrates the use of MSMQ over the Internet. You'll find it in the InternetQueueing subdirectory of the code download. It uses the queue created by the simplest example program. The only difference between this sending application and the simplest sender is the name of the queue. You see in the figure that I've prepended the string "FormatName:Direct=http://localhost/msmq/" to the name of the queue, replacing the machine name. The substring "localhost" represents the name of the server, here configured to run on a single machine. That's all the sender has to do to tell MSMQ to send the message via SOAP and HTTP. (Note that you can't open a queue for receive using the HTTP format name in Windows Server 2003 or Windows XP SP2.)

Similarly, the sample recipient shown in Figure 13 executes the same code to receive MSMQ messages from the Internet, as did the simplest sample program. The only change is in the name of the queue. So how do I know that the message came over HTTP and SOAP, and what's that SOAP-looking thing in the sample program's window? The native MSMQ API supports a property called SoapEnvelope, which represents the outer SOAP packet containing the message and its routing information. That's what I show in the sample program's display window. However, the .NET MSMQ wrapper does not expose this property. To display it to you, I had to import the COM-based MSMQ wrapper which does expose it, remove the message out of the queue into a COM-based wrapper, and fetch the SoapEnvelope property from that. I don't know why this property has been omitted. In any case, I won't bother listing the code that does this as it's not an important feature for most users, but it's in the online sample if you're interested.

Figure 13 Sample Recipient (Close Figure 13)

I'll quickly discuss the SOAP envelope, but only as a matter of academic interest since it's unlikely you'll need to fiddle with it. The whole point of the .NET wrapper is to hide these details from you. But it's often nice to have at least some notion of what lower layers are doing on your behalf. You can see that the SOAP envelope contains a <Header> element, which is the standard SOAP place to put infrastructure-specific information (as opposed to the <Body> element, which is used for application-specific information). This in turn contains the <path> element, the namespace of which tells you that it's part of the WS-Routing protocol. The <action>, <to>, and <id> elements are interpreted according to it. In the <to> element you see the name of the destination queue. The <properties> element belongs to the SOAP Reliable Message Protocol (SRMP). The <sentAt> and <expiresAt> elements contain information used by MSMQ to support its properties, such as SentTime, TimeToBeReceived, and timeToReachQueue message properties. Finally, the <MSMQ> element contains such MSMQ-specific data as the <Priority> element. You'll note that the message contents aren't sent in the SOAP packet. They're sent as SOAP attachments, probably in order to save space.

Security is a top priority—arguably the top priority—for any computer connected to the Internet. Before the inclusion of HTTP messaging in version 3.0, MSMQ usually ran on dedicated Microsoft-only subnets over which the administrator had a great deal of control. Both sender and recipient were usually part of the same trust domain instead of random Internet surfers-by. When you want to face your MSMQ server outwards to the anarchic Web, you need to ensure that you close every possible security hole. That's why MSMQ now supports Hardened MSMQ mode. You turn it on through the user interface shown in Figure 14, which toggles a registry setting. When you turn this mode on, MSMQ stops listening on any incoming port, thereby avoiding any kind of direct attack. (I'm reminded of the Greek philosopher Socrates, asking his friends if he could persuade them not to drag him out drinking one night. They say that he can't because they simply won't listen to him.) Instead, it accepts messages only from IIS, which accepts them over the Web and places them into the incoming queues on the server machine. All outgoing queues other than those that use HTTP are locked. Messages can be placed in them, but they won't actually be sent until the hardened mode is cancelled (which may be never, so don't plan on this).

Figure 14 Security Modes
Figure 14 Security Modes

While I'm on the topic of security, I'll discuss how the privacy and encryption requirements have changed from the previous inward-facing case to the new outward-facing global Internet case. When all the communicating parties are members of the same trust domain, you can take advantage of the MSMQ built-in authentication mechanism that generates a certificate using the sending process's logged-on identity. However, when you're on the open Internet, it's less likely that a long-term intimate relationship exists between sender and recipient, and more likely that you're talking to random strangers. The sender may well not have a login ID on the recipient's domain and it's entirely reasonable that your systems designs would now not rely on the MSMQ built-in authentication mechanism, but instead would handle authentication as part of the business process. You could also send a user ID and password as part of the message contents.

Message privacy is always a large consideration, all the more so if your messages are now carrying authentication credentials. You need to encrypt everything all the time and the easiest way to do this when using MSMQ over the Internet is to use HTTPS. Simply prefix the queue name with HTTPS rather than HTTP, and configure the server with an identity certificate from a trust provider.

While it's beyond the scope of this article, MSMQ's support of HTTP is highly flexible, so you can tweak it as much as you need to. For example, you can specify the use of proxy servers and set up automatic redirections of messages from one queue to another. For more detail on this advanced messaging topic, read the Platform SDK article "Delivering Messages Sent over the Internet" in the MSDN Library.

Conclusion

MSMQ is a useful tool that slots well into .NET Framework-based applications. It provides easy access for programmers who want simple default functionality. It also provides a high degree of fine-grained control for programmers who need to further tune its operation. Its operation over HTTP networks provides a greater degree of freedom than has ever before been possible for loosely coupled systems.

For related articles see:
Programming Best Practices with Microsoft Message Queuing Services

For background information see:
Advanced Basics: Using MSMQ with Visual Basic .NET

David S. Platt teaches Programming .NET at Harvard University and at companies all over the world. He was selected by Microsoft as one of their Software Legends, which you can read about at http://www.softwarelegends.com. He is the author of eight programming books, most recently Introducing Microsoft .NET, 3rd Edition (Microsoft Press, 2003).


Remoting: Managing the Lifetime of Remote .NET Objects with Leasing and Sponsorship

New information has been added to this article since publication.
Refer to the Editor's Update below.

Remoting

Managing the Lifetime of Remote .NET Objects with Leasing and Sponsorship

Juval Lowy
This article assumes you're familiar with .NET Remoting and C#
Level of Difficulty 1 2 3
Code download available at: LeaseManager.exe (166KB)
SUMMARY

Leasing and sponsorship is the solution for managing the lifecycle of a remote object in .NET. Each object has a lease that prevents the local garbage collector from destroying it, and most distributed applications rely upon leasing. There are several ways in which objects and clients can extend the lease, including dedicated sponsor objects. In this article, the author explains leasing, shows how to configure it, and how it relates to the various remoting activation models. He then discusses design guidelines and options, along with their impact on throughput and performance. Additionally, he introduces a helper class used to automate the management of lease sponsors.

Contents
Lease Properties
Configuring and Renewing Leases
Sponsors
Leasing Specifics
Conclusion

T
he Microsoft® .NET Framework manages the lifecycle of objects using garbage collection. It keeps track of memory allocation and objects accessed by all the clients in an AppDomain. When an object becomes unreachable by its clients, it will eventually be collected by the garbage collector. If the objects are in the same AppDomain, garbage collection functions quite well. In fact, even in the case of a client in one AppDomain accessing an object in a different AppDomain (but in the same process), garbage collection still works because in the same process all AppDomains share the same managed heap. In the case of a remote object accessed across processes and machines, the strategy breaks—the object may not have local clients at all. If garbage collection were to take place, the collector would not find a reference to the object and would deem it garbage, even though there may be remote clients on other machines (or even in a separate process on the same machine) that want to use the object.

This article focuses on leasing and sponsorship, which is a core piece of the .NET Remoting architecture designed to address this problem. I'll describe the leasing mechanism and interfaces, and show you its effect on the programming models, both on the client and server side.

The idea behind leasing is simple: each server object accessed by remote clients is associated with a lease object, so named because it literally gives the server object a lease on life. When a client creates a remote server object, the .NET Remoting Framework creates a lease object and associates it with the server object. A special entity in .NET Remoting called the lease manager keeps track of the server objects and their lease objects. Each lease object has an initial lease time. The clock starts ticking as soon as the first reference to the server object is marshaled across the AppDomain boundary, and the lease time is decremented as time goes by. As long as the lease has not expired, the common language runtime (CLR) considers the server object to be in use by its clients. The lease manager keeps a reference on the server object, which prevents it from being collected when a garbage collection is triggered. When the lease expires, the CLR assumes that the server object has no remote clients. It then disconnects the server object from the remoting infrastructure. The server object becomes a candidate for garbage collection and is eventually destroyed.

After the object is disconnected, any attempt by a client to access it will result in an exception of type RemotingException, letting the client know the object was disconnected. This may appear strange at first because the object may still be alive. The .NET Framework behaves this way because otherwise the client's interaction with the remote object will be nondeterministic. In other words, if .NET were to allow remote clients to access objects when their lease is up, it might work some of the time, but would fail in cases when garbage collection has already occurred.

Figure 1 Leasing Architecture
Figure 1 Leasing Architecture

That said, it would be incorrect to disconnect the object every time a lease expires. What if there are still some remote clients who would like to keep the server object alive? The smart thing to do is to contact such clients and ask them if they want to extend or sponsor the lease. Clients who want to be contacted when the lease expires need to provide a special sponsor object. The .NET Framework will contact the sponsor object so it can extend the lease or refuse to do so. A server object can have many sponsors associated with its lease. The lease manager keeps a list of all the sponsors. When a lease expires, the lease manager starts traversing the sponsor list looking for one that's willing to extend the lease. If such a sponsor is found, the lease manager extends the lease. Otherwise, the lease manager disconnects the server object. This architecture is shown in Figure 1.

Lease Properties

Every lease has a number of properties associated with it that control the manner in which the lease manager interacts with the remote object's lease. The .NET Remoting Framework assigns global default values to these properties, but you can instruct it to use other values. You can even override some of the default lease properties for individual objects. One property, the expired time, reflects the time that has expired since the beginning of the lease. Another property, the initial lease time, is initially set to five minutes, but you can override this default with your own value. The .NET Remoting Framework could have simply disconnected the object when the expired time is equal to the initial lease time, but what should it do if the clients continue to call the object? This would clearly indicate that the object is needed. If the lease is about to expire, the .NET Remoting Framework will automatically extend the lease on every call by the value set in the RenewOnCallTime property. The default call time renewal is two minutes. The value of the current lease time (the time the object has to live unless the lease is extended) is a product of the lease time (the initial lease time for the first time) and renew on call time, according to the formula shown in the following line of code:

current lease time = MAX(lease time - expired time,renew on call time)

The renew on call time will have an effect only if it is greater than the lease time minus the expired time. In that case, the expired time property is reset, and the lease time is set to the renew on call time. The result is that even if an object is very busy, its lease time does not grow in proportion to the amount of traffic it has. Even if the object has a temporary spike in its load, its lease will expire after some quiet time.

There are two properties pertaining to the lease manager itself. The first, the lease manager's poll time, governs the rate at which the lease manager polls the leases. The default poll time is 10 seconds. The other property, SponsorshipTimeout, controls how long the lease manager should wait for a reply from a sponsor that might reside on a remote machine (the default is two minutes). This property is important for handling network failures or sponsor machine unavailability. If the lease manager tries to reach a sponsor and it does not reply within the specified sponsorship timeout, the lease manager removes that sponsor from the list associated with the lease.

If you do not like the default values of the various lease and lease manager properties, you can provide your own values either programmatically or administratively using the application configuration file. To configure global defaults programmatically, use the static properties of the LifetimeServices class defined in the System.Runtime.Remoting.Lifetime namespace:

   public sealed class LifetimeServices
   {
      public static TimeSpan
         LeaseManagerPollTime { get; set; }
      public static TimeSpan LeaseTime
         { get; set; }
      public static TimeSpan RenewOnCallTime
         { get; set; }
      public static TimeSpan
         SponsorshipTimeout { get; set; }
   }

You typically use these properties in the Main method of your host (the application assembly hosting the remote objects) as shown in the following code snippet:

static void Main()
{
   LifetimeServices.LeaseTime       = TimeSpan.FromMinutes(10);
   LifetimeServices.RenewOnCallTime = TimeSpan.FromMinutes(15);

   /* Register types or load configuration file */
}
Note that you must set the global leasing defaults before you register types for remote access (programmatically or using the configuration file) because immediately after registration the host may start servicing remote calls, and these calls will not be using the new defaults. You can also provide these new values in the host config file using the <lifetime> element, as shown in Figure 2.

Figure 2 New Global Defaults (Close Figure 2)
<configuration>
   <system.runtime.remoting>
      <application>
         <lifetime
          leaseTime = "10M"
          sponsorshipTimeOut = "1M"
          renewOnCallTime = "15M"
          LeaseManagePollTime = "8s"
         />
      </application>
   </system.runtime.remoting>
</configuration>

Configuring and Renewing Leases

Every lease object implements the ILease interface, defined in the System.Runtime.Remoting.Lifetime namespace (see Figure 3). ILease allows you to control and configure the lease properties for an individual object and manage sponsors for that lease. Both the object and its clients can obtain the ILease interface. An individual lease can be in one of a number of states, most important of which are initial, active, or expired. You can obtain the state of the lease by accessing the CurrentState read-only property of the ILease interface. CurrentState is of the enum type LeaseState:

public enum LeaseState
{
    Active,
    Expired,
    Initial,
    Null,
    Renewing
}

Figure 3 ILease Interface (Close Figure 3)
public interface ILease
{
   TimeSpan CurrentLeaseTime   {get;}
   LeaseState CurrentState     {get;}
   TimeSpan InitialLeaseTime   {get;set;}
   TimeSpan RenewOnCallTime    {get;set;}
   TimeSpan SponsorshipTimeout {get;set;}

   void Register(ISponsor obj);
   void Register(ISponsor obj,TimeSpan renewalTime);
   TimeSpan Renew(TimeSpan renewalTime);
   void Unregister(ISponsor obj);
}

A remote object can provide its own values to the lease's properties, giving the object control over its own lifetime. To accomplish this, override the InitializeLifetimeService method of MarshalByRefObject and return a lease object. InitializeLifetimeService is called by the .NET Remoting Framework immediately after the remote object's constructor, but before a reference to the object is marshaled back to the client. InitializeLifetimeService is never called if the object is created by a local client. Although you can return from InitializeLifetimeService any object that implements the ILease interface, in practice you need to obtain the lease already associated with your object and modify its properties. You do that by calling your base class's InitializeLifetimeService and modifying the lease properties, as shown in Figure 4. You can only set the lease properties if the lease is in the LeaseState.Initial state, as shown in the figure.

Figure 4 Providing New Lease Properties for an Object (Close Figure 4)
public class MyServer : MarshalByRefObject
{
   public override object InitializeLifetimeService()
   {
       ILease lease = (ILease)base.InitializeLifetimeService();
       Debug.Assert(lease.CurrentState == LeaseState.Initial);

       //Set lease properties
       lease.InitialLeaseTime    = TimeSpan.FromMinutes(30);
       lease.RenewOnCallTime     = TimeSpan.FromMinutes(10);
       lease.SponsorshipTimeout  = TimeSpan.FromMinutes(2);
       return lease;
    }
}

Both the object and its clients can extend the lease explicitly by obtaining the object's lease and calling the ILease.Renew method to provide a new lease time. Renewing a lease explicitly will affect the current lease time according to this formula:

current lease time = MAX(lease time - expired time, renewal time)
This means that the renewal time will have an effect only if its time is greater than the lease time minus the expired time. In that case, the expired time is reset, and the lease time becomes the renewal time. Consequently, if different clients all try to renew a lease explicitly, the lease will not grow to the value of their combined renewal sum to make sure the object only remains connected when clients require it. Both the client and the object obtain the lease associated with the object using the static method GetLifetimeService of the RemotingServices class:
public static object GetLifetimeService(MarshalByRefObject obj);

You can only renew a lease if it is in the LeaseState.Active state. For example, here is how a client would renew a lease:

MyClass obj;
obj = new MyClass();

ILease lease = (ILease)RemotingServices.GetLifetimeService(obj);
Debug.Assert(lease.CurrentState == LeaseState.Active);
lease.Renew(TimeSpan.FromMinutes(30));
If the object wants to renew its own lease, it can simply call GetLifetimeService, providing itself as the parameter:
public class MyServer : MarshalByRefObject
{
   public void SomeMethod()
    {
       ILease lease = (ILease)RemotingServices.GetLifetimeService(this);
       Debug.Assert(lease.CurrentState == LeaseState.Active);

      lease.Renew(TimeSpan.FromMinutes(30));
      •••
    }
}

Sponsors

As mentioned already, a sponsor is a third party that the .NET Remoting Framework consults when a lease expires, giving that party an opportunity to renew the lease. The sponsor must implement the ISponsor interface, defined as:

public interface ISponsor
{
   TimeSpan Renewal(ILease lease);
}

The lease manager calls ISponsor's single method, Renewal, when the lease expires, asking for new lease time. To add a sponsor to a lease object, simply obtain the lease object using GetLifetimeService, then call the ILease Register method, as shown in Figure 5. The client can detach a particular sponsor from the lease by calling the ILease Unregister method.

Figure 5 Implementing and Registering a Sponsor (Close Figure 5)
public class MySponsor : MarshalByRefObject,ISponsor
{
   public TimeSpan Renewal(ILease lease)
   {
      Debug.Assert(lease.CurrentState == LeaseState.Active);
      //Renew lease by 5 minutes
      return TimeSpan.FromMinutes(5);
   }
}
ISponsor sponsor = new MySponsor();
MyClass obj = new MyClass();

//Register the sponsor
ILease lease = (ILease)RemotingServices.GetLifetimeService(obj);
lease.Register(sponsor);

If the sponsor does not want to renew the lease, it can return TimeSpan.Zero:

public TimeSpan Renewal(ILease lease)
{
   Debug.Assert(lease.CurrentState == LeaseState.Active);
   //Refuse to renew lease:
   return TimeSpan.Zero;
}
However, it would probably make better sense to unregister the sponsor instead.

Because the sponsor is called across an AppDomain boundary, the sponsor must be a remotable object, meaning it must be marshaled either by value or by reference. If you derive the sponsor from MarshalByRefObject, the sponsor will reside on the client's side, and it can base its decision of the renewal time on client-side events or properties that it is monitoring. That raises an interesting question: if the lease keeps the remote server object alive, and if the sponsor keeps the lease alive, who keeps the sponsor alive? The answer is that somebody on the client side must keep a reference on the sponsor, typically as a class member variable. Doing so also allows the client to unregister the sponsor when the client shuts down. The client can also unregister the sponsor in its implementation of IDisposable.Dispose. Unregistering a sponsor improves overall performance because the lease manager will not spend time trying to reach a sponsor that is not available.

If you only mark the sponsor as serializable, when you register the sponsor it will be marshaled by value to the host's side, where the sponsor will reside. This will eliminate the marshaling overhead of contacting the sponsor, but it will also disconnect it from the client. A marshaled-by-value sponsor can only base its decision on information available on the host's side.

When the sponsor is a marshaled-by-reference object, the client application must register a port with its channels to allow the remote lease manager to call back to the sponsor. In general, the client does not care which port is used. The client can instruct the .NET Remoting Framework to automatically select an available port by registering port number 0. The channel, port number, and sponsor object location are captured when the reference to the sponsor object is marshaled to the remote host.

Because the sponsor is provided to the host in the form of a remote callback object, the host must set its channels-type filtering level to full. Note that sponsoring will not work with the default-type filtering level of low. This is a tightened security requirement that was introduced in the .NET Framework 1.1. The rationale is that callback objects (such as a sponsor) leave the host inherently vulnerable to attacks that lure it to perform malicious operations unknowingly by calling back to a malicious remote object. Setting the type filtering to full does not address the problem. What it does is make it a conscious decision to allow callbacks if you do not perceive the threat to be real. You can set the filtering level both programmatically and administratively in the configuration files, on both the host and the client side. You need to set the type-filtering level explicitly for each channel and for each formatter used with that channel. For example, the required entries in the host configuration file when using the TCP channel on port 8005 are shown in the following code:

<channel ref="tcp" port="8005">
   <serverProviders>
      <formatter ref="soap" typeFilterLevel="Full"/>
      <formatter ref="binary" typeFilterLevel="Full"/>
   </serverProviders>
</channel>
These are the matching entries in the client configuration file:
<channel ref="tcp" port="0">
   <serverProviders>
      <formatter ref="soap" typeFilterLevel="Full"/>
      <formatter ref="binary" typeFilterLevel="Full"/>
    </serverProviders>
</channel>
Note the use of port 0 on the client side to automatically select any available port for the callback to the sponsor.

Leasing Specifics

A single lease can have multiple sponsors, and multiple clients can all share the same sponsor. In addition, a single sponsor can be registered with multiple remote leases. Typically, a client application will have one sponsor for all of its remote objects. As a result, a remote object will typically have as many sponsors to its lease as it has distinct clients. When the client application shuts down, it will unregister the sponsor from all the remote leases it sponsors.

Server-activated single-call objects don't need leasing because such objects are disconnected immediately after each call. This is not the case for both client-activated objects and server-activated singleton objects.

The singleton design pattern semantics mandate that the singleton object lives forever once it is created. This can't happen if its default lease time is five minutes. If no client accesses a singleton object for more than five minutes after it is created, the .NET Remoting Framework will deactivate the singleton. Future calls from the clients will be silently routed to a new singleton object in violation of the pattern's semantics. Fortunately, infinite lease time is supported. When you design a singleton object, override InitializeLifetimeService and return a null object as the new lease, indicating that this lease never expires:

public class MySingleton : MarshalByRefObject
{
   public override object InitializeLifetimeService()
   {
       return null;
   }
}

Returning an infinite lease relieves the pain of managing global lease defaults or dealing with sponsors. In fact, you can have any object activation mode use an infinite lease, but it makes sense only in the case of a singleton.

The case of the client-activated object is the one affected most by the leasing mechanism. When using a client-activated model, every client that creates a new object gets a dedicated server object. That object's lifetime is governed by a lease, and the object will be disconnected from the clients once its lease has expired. The only safe way to manage client-activated objects is to use sponsors. All other options, such as global lease properties or configuring individual objects' leases, are speculative at best. Ultimately, only the client knows when it no longer needs the object. The sponsor should set the renewal to a value that balances network traffic and load on the lease manager, but which is also granular enough to manage the resources that the object may hold.

If the sponsor provides too short a renewal period, it will cause too many queries from the lease manager, resulting in unnecessary traffic. If the sponsored lease is too long, the lease manager will still keep the remote object alive longer than the client needs it. Every case is unique, and when throughput and performance are a concern, you will have to investigate and profile the system with various sponsorship renewal times.

I can offer the following rule of thumb: the sponsorship time should be the same as the initial lease time. The reason is that if the initial lease time is appropriate for your case, does not generate too much traffic, and is not too coarse, then it is probably going to be adequate for the sponsors on subsequent lease extension requests. Figure 6 shows a client using a client-activated object whose lifetime is controlled by a sponsor. In fact, you can even use the code in Figure 6 as a template or a starting point for your remote client-activated objects.

Figure 6 Sponsoring a Client-activated Object (Close Figure 6)
//////  RemoteServerHost.exe.config : the host configuration file  ////////
<?xml version="1.0"?>
<configuration>
   <system.runtime.remoting>
      <application>
         <service>
            <activated type="RemoteServer.MyCAO,ServerAssembly"/>
         </service>
         <channels>
            <channel ref="tcp" port="8005">
               <serverProviders>
                  <formatter ref="soap"
                    typeFilterLevel="Full"/>
                  <formatter ref="binary"
                    typeFilterLevel="Full"/>
               </serverProviders>
            </channel>
         </channels>
      </application>
   </system.runtime.remoting>
</configuration>

////////  RemoteServerHost.exe : the host application assembly  //////////
public class ServerHostDialog : Form
{
   public ServerHostDialog()
   {
      InitializeComponent();
   }
   void InitializeComponent()
   {...}
   static void Main()
   {
      RemotingConfiguration.Configure("RemoteServerHost.exe.config");
      Application.Run(new ServerHostDialog());
   }
}

//////////////   ServerAssembly class library  ////////////////////////////
namespace RemoteServer
{
   public class MyCAO : MarshalByRefObject
   {
      public void Count()
      {
         m_Counter++;
         string appName = AppDomain.CurrentDomain.FriendlyName;
         MessageBox.Show("Counter value is " +
           m_Counter.ToString(),appName);
      }
      int m_Counter = 0;
   }
}

/////////  Client.exe.config: the client configuration file  //////////////
<?xml version="1.0"?>
<configuration>
   <system.runtime.remoting>
      <application>
         <client url="tcp://localhost:8005">
            <activated type="RemoteServer.MyCAO,ServerAssembly"/>
         </client>
         <channels>
            <channel ref="tcp" port="0">
               <serverProviders>
                  <formatter ref="soap"
                    typeFilterLevel="Full"/>
                  <formatter ref="binary"
                    typeFilterLevel="Full"/>
               </serverProviders>
            </channel>
         </channels>
      </application>
   </system.runtime.remoting>
</configuration>

////////////////////  Client EXE assembly  ////////////////////////////////
using System.Runtime.Remoting.Lifetime;

public class MySponsor : MarshalByRefObject,ISponsor
{
   public TimeSpan Renewal(ILease lease)
   {
      Debug.Assert(lease.CurrentState == LeaseState.Active);

      return lease.InitialLeaseTime;
   }
}

using RemoteServer;

public class ClientForm : Form
{
   Button m_CallButton;
   ISponsor m_Sponsor;
   MyCAO m_Obj;

   public ClientForm()
   {
      InitializeComponent();

      m_Sponsor = new MySponsor();
      m_Obj = new MyCAO();

      //Register the sponsor
      ILease lease = (ILease)
        RemotingServices.GetLifetimeService(m_Obj);
      lease.Register(m_Sponsor);
   }
   void InitializeComponent()
   {...}

   static void Main()
   {
      RemotingConfiguration.Configure("Client.exe.config");

      Application.Run(new ClientForm());
   }
   //Called when the button is clicked
   void OnCall(object sender,EventArgs e)
    {
      m_Obj.Count();
   }
   //Called when the form is closed
   void OnClosed(object sender,EventArgs e)
   {
      //Unegister the sponsor
      ILease lease = (ILease)
        RemotingServices.GetLifetimeService(m_Obj);
      lease.Unregister(m_Sponsor);
   }
}

The source code accompanying this article contains Figure 6 in the Leasing solution (see the link at the top of this article). The solution has three projects: the RemoteServerHost application assembly, the ServerAssembly class library, and the Client application assembly. The host is a do-nothing Windows® Forms application implemented by adding a single line in Main that loads the configuration file. The host configuration file registers the type MyCAO as a client-activated object and sets type filtering on the TCP channel to full to allow calling back into the sponsor. The ServerAssembly class library contains the MyCAO definition, so that both the client and the host could access its metadata. The class MyCAO provides a single method called Count, which displays the value of a counter. The Client assembly contains the MySponsor class because there is no need to put the sponsor in a class library. All the host needs is the metadata for the ISponsor interface, which is part of the .NET Framework class library already.

The MySponsor implementation of Renewal simply extends the lease by the initial lease time. The client configuration file designates the MyCAO type as a remote client-activated object, and it registers port 0 with the channel as well as setting type filtering to full to allow calling the sponsor. The client's class (ClientForm) has as member variables both the remote client-activated object and the sponsor. In its constructor, the client creates new instances of the server and the sponsor and registers the sponsor with the lease of the client-activated object. The client is a Windows Forms dialog with a single button. When clicked, the button calls the client-activated object. When the user closes the dialog, the client unregisters the sponsors in the OnClose event handler.

The sponsor in Figure 6 is completely generic. As such, it simply returns the initial lease time (whatever that may be) on every renewal request. The problem with the code shown in Figure 6 is that the client has to keep track of its remote objects and manually unregister the single sponsor from every remote lease when the application shuts down. However, you can automate this by providing a client-side sponsorship manager. The code in Figure 7 shows the SponsorshipManager helper class, which is also included in the code download. [ Editor's Update - 6/9/2005: The code in Figure 7 has been updated.]

Figure 7 SponsorshipManager Helper Class (Close Figure 7)
public class SponsorshipManager : MarshalByRefObject,ISponsor
{
   IList  m_LeaseList;
   public SponsorshipManager()
   {
      m_LeaseList = new ArrayList();
   }
   ~SponsorshipManager()
   {
      UnregisterAll();
   }
   public void OnExit(object sender,EventArgs e)
   {
      UnregisterAll();
   }
   TimeSpan ISponsor.Renewal(ILease lease)
   {
      Debug.Assert(lease.CurrentState == LeaseState.Active);
      return lease.InitialLeaseTime;
   }
   public void Register(MarshalByRefObject obj)
   {
      ILease lease = (ILease)
        RemotingServices.GetLifetimeService(obj);
      Debug.Assert(lease.CurrentState == LeaseState.Active);

      lease.Register(this);
      lock(this)
      {
         m_LeaseList.Add(lease);
      }
   }
   public void Unregister(MarshalByRefObject obj)
   {
      ILease lease = (ILease)
        RemotingServices.GetLifetimeService(obj);
      Debug.Assert(lease.CurrentState == LeaseState.Active);

      lease.Unregister(this);
      lock(this)
      {
         m_LeaseList.Remove(lease);
      }
   }
   public void UnregisterAll()
   {
      lock(this)
      {
         while(m_LeaseList.Count > 0)
         {
            ILease lease = (ILease)m_LeaseList[m_LeaseList.Count-1];
            lease.Unregister(this);
            m_LeaseList.RemoveAt(m_LeaseList.Count-1);
         }
      }
   }
}

SponsorshipManager implements ISponsor by returning the initial lease time of the provided lease. In addition, it maintains a collection of all the leases it sponsors in a member variable called m_LeaseList. SponsorshipManager provides the Register method:

public void Register(MarshalByRefObject obj);
Register accepts a remote object, extracts the lease from it, registers SponsorshipManager as the sponsor, and adds the lease to the internal list. The Unregister method is defined as:
public void Unregister(MarshalByRefObject obj);
This method removes the lease associated with the specified object from the list and unregisters SponsorshipManager as a sponsor. SponsorshipManager also provides the UnregisterAll method:
public void UnregisterAll();
This unregisters SponsorshipManager from all of the lease objects in the list. Note that the list access is done in a thread-safe manner by locking the SponsorshipManager on every access. Thread safety is required because multiple clients on multiple threads could use the same SponsorshipManager object to manage their remote leases. However, the clients don't have to bother calling Unregister or UnregisterAll because SponsorshipManager provides an event-handling method called OnExit:
public void OnExit(object sender,EventArgs e);
OnExit calls UnregisterAll as a response to the client application shutting down. Figure 8 shows how to use SponsorshipManager within a Windows Forms client. The client in Figure 8 maintains a remote object of type MyCAO and a SponsorshipManager object as member variables. The client registers the remote object with SponsorshipManager and hooks up the application's OnExit event with the SponsorshipManager OnExit event-handling method. SponsorshipManager does the rest. It registers itself as the sponsor and unregisters all the remote leases it sponsors automatically when the client shuts down.

Figure 8 Using SponsorshipManager (Close Figure 8)
public class ClientForm : Form
{
   SponsorshipManager m_SponsorshipManager;
   MyCAO m_Obj; //The remote CAO

   public ClientForm()
   {
      InitializeComponent();

      m_SponsorshipManager = new SponsorshipManager();
      m_Obj = new MyCAO();

      m_SponsorshipManager.Register(m_Obj);
      Application.ApplicationExit += new
        EventHandler(m_SponsorshipManager.OnExit);
   }
   void InitializeComponent()
   {...}

   static void Main()
   {
      RemotingConfiguration.Configure("Client.exe.config");
      Application.Run(new ClientForm());
   }
   void OnCall(object sender,EventArgs e)
   {
      m_Obj.Count();
   }
}

Conclusion

In general, you cannot develop a distributed .NET Framework-based application without taking advantage of leasing and sponsorship. The mechanism is an integral part of your application architecture and choosing the appropriate leasing and sponsorship properties will have a significant impact on the performance, scalability, and throughput of your application—not to mention its behavior. I recommend that you follow a few rules when it comes to the leasing of your remote objects and clients. First of all, do not provide a lease for single-call objects. Second, provide a null lease for singleton objects, and finally, return the initial lease time for client-activated objects, or better yet, use the SponsorshipManager class. You may still have to profile your application's load and calling patterns before you decide the exact initial lease time, but helper classes such as SponsorshipManager can encapsulate the interaction with the leasing mechanism.

For related articles see:
.NET Remoting: Design and Develop Seamless Distributed Applications for the Common Language Runtime
Contexts in .NET: Decouple Components by Injecting Custom Services into Your Object's Interception Chain

For background information see:
Programming .NET Components by Juval Lowy (O'Reilly, 2003)
IDesign: .NET Design and Business Solutions

Juval Lowy is a software architect providing consultation and training on .NET Framework design and migration. He is also the Microsoft Regional Director for the Silicon Valley. This article contains excerpts from his latest book, Programming .NET Components (O'Reilly, 2003). Contact Juval at http://www.idesign.net.


Beyond Macros: Create Word and Excel Smart Documents with C++ and XML

Beyond Macros

Create Word and Excel Smart Documents with C++ and XML

Mike Kelly
This article assumes you're familiar with C++ and XML
Level of Difficulty 1 2 3
Code download available at: SmartDocuments.exe (159KB)
SUMMARY

One of the coolest new parts of Office 2003 is a programmability feature called Smart Documents, which allows developers to augment Word and Excel documents with programmable content and behavior. Typically, examples illustrating Office programmability use Visual Basic or Visual Basic .NET. In this article, the author develops a Smart Document for Excel using C++. He describes the new ISmartDocument interface and shows how to use it to manage a simple task list such as an Excel spreadsheet.

Contents
Smart Document Concepts
Creating the Smart Document
Let's Begin
ISmartDocument Interface Tour
Property Bags
Creating a Smart Document Action DLL
Installing Your Smart Document Solution
Accessing the Smart Document Content
Debugging Smart Document DLLs
Change Notifications
Security
Generating the Status Report
Conclusion

S
mart Documents are a new programmability feature in Microsoft® Word and Excel 2003. Developers can create Word or Excel Smart Documents augmented with programmable pieces that help users create or modify them. Reusing boilerplate text, implementing business rules for how a document should be formatted or completed, and routing a document through an approval process are all procedures that Office Smart Documents can simplify.

For years it has been possible to use Microsoft Office applications in custom solutions using ActiveX® and Visual Basic® for Applications (VBA). These have some limitations, however, particularly in deployment, user interface, and security. Smart Documents address these with a tighter integration between the popular "task pane" interface introduced in Office XP and the custom solution. With Smart Documents, you use XML to define custom task pane content that automatically synchronizes help, instructions, and programmable controls with the user's position in the document. Code for the custom task pane controls can be developed in several languages: Visual Basic 6.0, Visual Basic .NET, Visual C++®, or Visual C#®.

Smart Documents and associated controls are installed when the user simply opens the document obtained from a trusted server or an attachment in an e-mail message. They can automatically update themselves from a trusted server, greatly simplifying update deployment. Smart Documents follow the Office security settings, requiring them to be downloaded from a trusted server and digitally signed by a trusted source. Users can be given control to decide whether to open Smart Documents that do not meet these security standards.

In this article, the business problem to be solved is one familiar to readers of MSDN® Magazine: scheduling and tracking progress during a software project. While there are many good solutions (including Microsoft Project) for tracking schedules, I'm going to treat the schedule as a database stored in Excel because I want a solution that you can play with simply by installing Office 2003 and downloading the code for my article, without having to install any additional products.

In the spreadsheet there's a row for each schedule item containing the usual information: who's doing the task, the estimated amount of time to complete the task, and whether the task has been completed. I'll assume that, on my project team, developers update the schedule each week to reflect the previous week's work. In addition, since I like to know what's going on with my team, I'll ask them to identify which tasks they'll be working on next week. This is also the time when changes to schedule estimates will be made. The Smart Document solution I'll build will wrap up all these schedule changes into a nicely formatted Word status document—just the type of thing that managers love. As a bonus, I'm going to use the new XML features of Word for the status document. In fact, my Excel Smart Document solution will generate an XML status summary, which I'll combine with XSLT in Word to get the nice formatting.

Smart Document Concepts

There are five main parts to an Office 2003 Smart Document solution. First, there is a Word or Excel document or template. This is the Smart Document that the user opens to perform a business function—in this case generating a nicely formatted weekly status report for a software project. Please note that in Office 2003, only Word and Excel support Smart Documents.

Second, there is an XML schema definition (XSD). The Word document or Excel spreadsheet is marked up with elements from the XML schema to overlay an XML data model on all or part of the document or workbook. For example, in the project schedule workbook, there is a Project Name cell and a Developer Name cell. These correspond to the ProjectName and DeveloperName elements in the XML schema. "Marking up" the spreadsheet is how the developer associates these specific cells with the XML schema elements.

Third, you have the action handler DLL, which can be developed in Visual Basic, Visual Basic .NET, Visual C++, or Visual C#. It is then deployed to a Web or network server. This DLL implements the OLE interface ISmartDocument.

Fourth, there is an XML expansion pack manifest file that specifies the installation instructions for the solution. It provides the server location of the solution DLL and the solution's COM CLSID. It also mentions any other associated files needed for the solution and the solution ID. The solution ID uniquely identifies this Smart Document solution. Although you can use any unique identifier string as the solution ID, it's best to make it a GUID that you generate.

Finally, there may be other files needed by the solution (such as GIFs or JPEGs, HTML, or document fragments) that you want to install on the local machine. These would be specified in the manifest and installed or updated automatically by Office when the Smart Document is opened. They may then be used by your Smart Document solution code.

To clarify the roles of each document in my example, note that the Smart Document is the document to which the XML schema and custom logic are attached—in my case, it's the Excel spreadsheet. The fact that I'm generating a Word document with this is incidental; I could just as easily be putting information into a database or generating a PowerPoint® presentation.

You can think of the XML elements, which factor the spreadsheet or document into logical structural components, as hooks to which your Smart Document solution can attach controls that assist the user in working with these various parts of the document. The behavior of these controls is defined by your action handler DLL at run time. The controls effectively become a sophisticated context-sensitive help system and are displayed in the new document actions task pane as the user moves into particular sections (corresponding to XML elements) of the document or spreadsheet. Because Smart Document controls can be much more than just help text, your context-sensitive help can actually act as a user assistant in completing the business task system for which the document is used.

Creating the Smart Document

Much of the work involved in generating a Smart Document solution consists of creating the Word or Excel document and associating XML schema elements with it. In fact, you can often use existing Word or Excel documents as a basis for creating a Smart Document. Since many business documents already have a structure, you're merely formalizing and naming it when you define the XML schema.

From the developer's point of view, the document or spreadsheet is the principal UI for the solution. The Smart Document action handler DLL defines an additional UI that appears side by side with the document when the user is editing. It's sensible when creating Smart Documents to use the underlying power of Word and Excel whenever possible rather than writing custom code in your Smart Document DLL. A formula in my Excel schedule workbook illustrates this point. I wanted the status report to be dated the previous Monday, but because people often fill out a status report for the previous week on Monday or Tuesday, the calculation to get the report date right is a bit tricky. On Monday and Tuesday, it's the previous week's Monday; after that, it's this week's Monday. Rather than write a bunch of C++ code to do this, I wrote an Excel formula to encapsulate the calculation and put it in a cell in the worksheet:

=IF(WEEKDAY(TODAY(),3)<2,
    TODAY()-(7+WEEKDAY(TODAY(),3)),
    TODAY()-WEEKDAY(TODAY(),3))
I simply load that calculated date value from the worksheet cell using a function I wrote to get values from an Excel workbook.

Let's Begin

Now, let's start by creating an Excel spreadsheet to track a software schedule. Note that I am assuming you've downloaded the sample code, which includes the necessary support files.

Start Microsoft Excel 2003 and open Project Schedule (Original).xls. This spreadsheet, like most Excel spreadsheets, already has a structure. Now associate an XML schema with that structure using an XSD document, in this case ScheduleSmartDocument.xsd. In Excel, point to XML on the Data menu and select XML Source. When the XML Source task pane appears, click the Workbook Maps button and add the ScheduleSmartDocument.xsd file. You then click OK to dismiss the Browse dialog, and OK again to dismiss the XML Maps dialog. Figure 1 shows the Excel XML Source task pane after attaching the schema.

Figure 1 XML Source
Figure 1 XML Source

To create the mapping, you must drag elements from the XML schema onto the workbook. For the elements like Project Name and Developer, this is a simple matter of grabbing the XML element name in the XML Source task pane and dragging it onto the cell in the worksheet that contains the associated data (for example, dragging ProjectName onto cell A2). For the repeating schedule items, shown as a hierarchy, grab the group in the task pane by selecting Items and drag it onto the header row (cell A5). You can confirm that the mapping is correct by selecting a particular element (for example Description) in the XML Source task pane; the corresponding portion of the workbook will be highlighted. Save the resulting workbook as "Project Schedule (Mapped).xls." In this example, I'll create a single spreadsheet. Or, you could save the schedule as an Excel Workbook Template. There, you simplify the process of users creating new copies of the document, each of which will itself be an Excel Smart Document.

ISmartDocument Interface Tour

The next step is to create a Smart Document action DLL and install it. Before you start working on a Smart Document action handler DLL (which implements the ISmartDocument interface methods), it's good to learn about this interface.

Your action handler DLL is going to create and manage a set of controls which will appear in the document actions task pane as the user moves through the document, enabling you to provide a custom user interface based on the position within the document. The XML schema elements you mapped to your document define the user's position within the document. In turn, the XML elements specify which controls should be shown to the user. The control types you can create are listed in Figure 2. You'll create the controls using ISmartDocument interface methods that you'll implement in your action handler DLL. When these controls are activated, such as when text is entered or a button is clicked, Office invokes your custom action handler DLL code through ISmartDocument interface methods.

Figure 2 Smart Document Control Types (Close Figure 2)

Symbolic Name | Control Type
C_TYPE_LINK | Hyperlink
C_TYPE_HELP | Help text provided as HTML and displayed in the document actions task pane
C_TYPE_HELPURL | URL-to-HTML help text to be displayed in the document actions task pane
C_TYPE_SEPARATOR | Separator line for formatting use only
C_TYPE_BUTTON | Command button for triggering actions
C_TYPE_LABEL | Static text to be displayed
C_TYPE_IMAGE | Image to be displayed, provided as path name to a GIF, BMP, TIFF, JPEG, WMF, or other image file
C_TYPE_CHECKBOX | Checkbox
C_TYPE_TEXTBOX | Edit box for entering text
C_TYPE_LISTBOX | List of items as a scrollable listbox
C_TYPE_COMBO | Combobox (an edit control and dropdown list of choices as a unit)
C_TYPE_ACTIVEX | ActiveX control
C_TYPE_DOCUMENT- | Document fragment, provided as Word
FRAGMENT | XML text (Word only)
C_TYPE_DOCUMENT- | Document fragment, provided as URL to
FRAGMENTURL | a file (Word only)
C_TYPE_RADIOGROUP | Radio group of choices

I've divided the methods of the ISmartDocument interface into several categories: the configuration methods, which describe the controls created in the document actions task pane to Office (see Figure 3); control draw-time methods, which are invoked as controls and are drawn in the task pane (see Figure 4); and change notification methods, which Office invokes as the document actions task pane controls are manipulated by the user (see Figure 5).

Figure 5 ISmartDocument Interface Notification Methods (Close Figure 5)

Method | Parameters | Description
InvokeControl | int ControlID, BSTR ApplicationName, LPDISPATCH Target, BSTR Text, BSTR Xml, int LocaleID | Called for controls of type C_TYPE_LINK, C_TYPE_BUTTON, C_TYPE_DOCUMENTFRAGMENT, and C_TYPE_DOCUMENTFRAGMENTURL when the user presses the button or clicks on the link or document fragment.
OnCheckboxChange | int ControlID, LPDISPATCH Target, VARIANT_BOOL Checked | Called for controls of type C_TYPE_CHECKBOX when the checked state changes.
OnTextboxContentChange | int ControlID, LPDISPATCH Target, BSTR Value | Called for controls of type C_TYPE_TEXTBOX when the text content changes.
OnListOrComboSelectChange | int ControlID, LPDISPATCH Target, int Selected, BSTR Value | Called for controls of type C_TYPE_LISTBOX or C_TYPE_COMBO when the selected item changes.
ImageClick | int ControlID, BSTR ApplicationName, LPDISPATCH Target, BSTR Text, BSTR Xml, int LocaleID, int XCoordinate, int YCoordinate | Called for controls of type C_TYPE_IMAGE when the user clicks on the image. X and Y are zero-based pixel coordinates relative to the upper-left corner of the image.
OnRadioGroupSelectChange | int ControlID, LPDISPATCH Target, int Selected, BSTR Value | Called for controls of type C_TYPE_RADIOGROUP when the user changes the selected state of the radio buttons. The Value string is the text of the item that has been newly selected.
Figure 4 ISmartDocument Interface Control Drawing Methods (Close Figure 4)

Method | Parameters | Description
get_ControlCaptionFromID | int ControlID, BSTR ApplicationName, int LocaleID, BSTR Text, BSTR Xml, LPDISPATCH Target, BSTR * Caption | Provides the visible display name of the control (as the out BSTR) given the control ID.
PopulateHelpContent | int ControlID, BSTR ApplicationName, int LocaleID, BSTR Text, BSTR Xml, LPDISPATCH Target, struct ISmartDocProperties * Props, BSTR * Content | Called for controls of type C_TYPE_HELP or C_TYPE_HELPURL to provide content for the help text.
PopulateCheckbox | int ControlID, BSTR ApplicationName, int LocaleID, BSTR Text, BSTR Xml, LPDISPATCH Target, struct ISmartDocProperties * Props, VARIANT_BOOL * Checked | Called for controls of type C_TYPE_ CHECKBOX to provide the checked state of the checkbox (as a BOOLEAN variant in the Checked out parameter).
PopulateTextboxContent | int ControlID, BSTR ApplicationName, int LocaleID, BSTR Text, BSTR Xml, LPDISPATCH Target, struct ISmartDocProperties * Props, BSTR * Value | Called for controls of type C_TYPE_TEXTBOX to provide the content for the textbox (in the BSTR Value out parameter).
PopulateListOrComboContent | int ControlID, BSTR ApplicationName, int LocaleID, BSTR Text, BSTR Xml, LPDISPATCH Target, struct ISmartDocProperties * Props, SAFEARRAY * * List, int * Count, int * InitialSelected | Called for controls of type C_TYPE_LISTBOX or C_TYPE_COMBO to provide content for the listbox or combobox dropdown. The content is provided as a VARIANT array in the List out parameter; the number of items in the array is provided in the Count out parameter; and the (1-based) initially selected is provided in the InitialSelected out parameter (-1 to select nothing).
PopulateDocumentFragment | int ControlID, BSTR ApplicationName, int LocaleID, BSTR Text, BSTR Xml, LPDISPATCH Target, struct ISmartDocProperties * Props, BSTR * DocumentFragment | Used by Word only. Called for controls of type C_TYPE_DOCUMENTFRAGMENT to provide content for the document fragment (as HTML).
PopulateActiveXProps | int ControlID, BSTR ApplicationName, int LocaleID, BSTR Text, BSTR Xml, LPDISPATCH Target, struct ISmartDocProperties * Props, ISmartDocProperties * ActiveXPropBag | Called for controls of type C_TYPE_ACTIVEX to provide information needed to instantiate the ActiveX control in the task pane. This information is provided through an IPropertyBag.
PopulateImage | int ControlID, BSTR ApplicationName, int LocaleID, BSTR Text, BSTR Xml, LPDISPATCH Target, struct ISmartDocProperties * Props, BSTR * ImageSrc | Called for controls of type C_TYPE_IMAGE to provide the path (local file path, UNC, or URL) to the image. The image may be a GIF, JPEG, or WMF.
PopulateRadioGroup | int ControlID, BSTR ApplicationName, int LocaleID, BSTR Text, BSTR Xml, LPDISPATCH Target, struct ISmartDocProperties * Props, SAFEARRAY * * List, int * Count, int * InitialSelected | Called for controls of type C_TYPE_RADIOGROUP to provide the contents of the radio group buttons. These are provided as a set of BSTR strings in the VARIANT array, the number of items in the array, and the (1-based) initially selected item.
PopulateOther | int ControlID, BSTR ApplicationName, int LocaleID, BSTR Text, BSTR Xml, LPDISPATCH Target, struct ISmartDocProperties * Props | Called for controls of type C_TYPE_LINK and C_TYPE_BUTTON to provide the contents of the link. These are provided as a set of BSTR strings in the VARIANT array, the number of items in the array, and the (1-based) initially selected item.
OnPaneUpdateComplete | LPDISPATCH Target | Called when all the controls have been drawn.
Figure 3 ISmartDocument Interface Configuration Methods (Close Figure 3)

Method | Parameters | Description
SmartDocInitialize | BSTR ApplicationName, LPDISPATCH Document, BSTR SolutionPath, BSTR SolutionRegKeyRoot | Called by an Office application when the Smart Document is opened by a user. Can be used to do any per-session initialization. The IDispatch points to the Word or Excel document being opened. The SolutionPath is the local machine directory where the solution files are installed.
get_SmartDocXmlTypeCount | int * Count | Provides Office with the number of XML elements in the Smart Document which have content in the document actions pane. Each document element with controls is assigned a unique 1-based Smart Document type ID (called SmartDocID in the methods listed below).
get_SmartDocXmlTypeName | int SmartDocID, BSTR * bstrElementName | Provides Office with the XML element name corresponding to SmartDocID (1-based).
get_SmartDocXmlTypeCaption | int SmartDocID, int LocaleID, BSTR * Caption | Provides Office with the localized caption to appear in the document actions task pane for the element name corresponding to SmartDocID. You could load this from a localized string table in your DLL.
get_ControlCount | BSTR bstrElementName, int * Count | Given an XML element name from your Smart Document, tells Office how many controls will be displayed in the document actions task pane when the user is positioned within that element.
get_ControlID | BSTR bstrElementName, int ControlIndex, int * ControlID | Given an element name and a (1-based) control index, you assign a unique control ID to each control using this method. The rest of the methods use this control ID to refer to controls.
get_ControlNameFromID | int ControlID, BSTR * Name | Given a control ID, provides the (internal) name of the control.
get_ControlTypeFromID | int ControlID, BSTR ApplicationName, int LocaleID, C_TYPE * Type | Given a control ID, provides the type of the control (see Figure 2 for a list of types).

Note that for many action DLLs, you will not use all the control types or even implement (beyond stubs) all the ISmartDocument methods. For example, since I have no ActiveX or listbox controls in my task pane, I have only boilerplate implementations for those methods in my sample code. It is even possible to take advantage of some simple Smart Document features (such as providing hyperlinks or context-sensitive help text) without writing an action handler DLL at all. Instead, you simply provide an XML file that describes the actions you want to associate with various XML elements in the associated Word or Excel Smart Document.

See Paul Cornell's article "Smart Documents Beta 2 Development Overview". Many of the ISmartDocument methods have common parameters, which are summarized in Figure 6.

Figure 6 ISmartDocument Common Method Params (Close Figure 6)

Parameter | Description
BSTR ApplicationName | ProgID of the Office application hosting the Smart Document (for example "Excel.Application.11").
LPDISPATCH Document or Target | An IDispatch pointer to the underlying Word or Excel document (for SmartDocInitialize and OnPaneUpdateComplete). For other methods, an IDispatch to the Word or Excel range of the XML element affected.
int LocaleID | LCID of the user's language; for use by multilanguage Smart Documents to enable localized captions and other resources to be loaded from the DLL.
ISmartDocProperties * Props | An IPropertyBag that is used to provide control properties.
BSTR Text | Text of the current document element.
BSTR Xml | XML representation of the document element; can be NULL.

When Office loads your action DLL, it will first invoke the configuration methods shown in Figure 3. Office first calls SmartDocInitialize (enabling you to do any special, one-time initialization). It will then query you for the number of XML elements defined in the document that have associated controls by calling your implementation of get_SmartDocXmlTypeCount. It is also possible to have XML elements in the document without associated controls. In that case, the document actions pane will be empty, so you may want to define some standard help content for those elements. Once you've told Office how many XML document elements you're interested in, it will ask for the name of each element, which it obtains through multiple calls to get_SmartDocXmlTypeName. You need to return the qualified name for the XML element (for example, schema#element). Each element will have a caption that appears in the title of the document actions pane when the user is positioned within that element. This is returned by a call to get_SmartDocXmlTypeCaption. The calls to get_SmartDocXmlTypeName and get_SmartDocXmlTypeCaption occur in pairs, one per XML element. Next, Office wants to know information about the controls to display in the task pane for each element. This information includes:

  • get_ControlCount—gives the number of controls for the element.
  • get_ControlID—allows you to assign a unique ID number to each control based on the (1-based) index of the control for this element.
  • get_ControlNameFromID—returns the (non-localized) name of the control after being given the control ID. This name is not displayed, but is used to refer to the control in the Word or Excel object model.
  • get_ControlTypeFromID—given the control ID, returns the type of the control (from Figure 2).

As the user moves through the document, Office updates the document actions task pane with the appropriate controls, then invokes the control drawing methods (see Figure 4) as it draws the controls in the task pane. First, it invokes get_ControlCaptionFromID to allow you to provide a caption for the control. Note that the caption is not provided at configuration time, but at display time, and so can change dynamically (based, perhaps, on document content). Controls with a blank or NULL caption string are not drawn by Office. This may be used to dynamically hide and show controls. In order to hide a control, you should simply return a NULL or empty string from get_ControlCaptionFromID for that control; Office will not invoke the corresponding Populate methods for that control.

Office then invokes a control type-specific Populate method (such as PopulateHelpContent) for each control. You use these methods to provide content for the controls, such as the location of an image or the text of a hyperlink. Note that the XML and Text content of the document element that is associated with this control is passed to all the Populate methods. You should also note that controls of type C_TYPE_LABEL and C_TYPE_SEPARATOR do not have associated Populate methods. After all the controls have been drawn and the task pane is complete, Office will call OnPaneUpdateComplete.

As the user interacts with the controls in the task pane, Office invokes one of the notification methods that is shown in the list in Figure 5. Labels and separators have no notification methods associated with them. For others, the notification methods indicate either a change in state or a click, which could invoke an action (such as for a hyperlink or image).

Note that Office seems to err on the side of too much notification rather than too little. For example, as the cell selection changes in Excel (even though the XML element remains the same), Office redraws the document actions workpane and invokes your methods so you can have truly dynamic content in the task pane. You could, for example, query the value of the selected cell through the Excel object model and update your controls based on that value. I take advantage of this in my sample code to update information about the selected task in the task pane without needing to manage Excel VBA events in my document action DLL.

Property Bags

Most of the control drawing methods provide an ISmartDocProperties parameter. This is a property bag (inherited from IPropertyBag) of name/value pairs that provide additional information on the controls. This is where you specify size, font, alignment, and so on for your controls. Figure 7 provides the key names that apply to all controls. The property values, including values for the integer properties, are all supplied as BSTRs to ISmartDocProperties::Write. For example, the following line in PopulateTextboxContent will set the width of the textbox control to 50 pixels:

Props->Write(CComBSTR("W"), CComBSTR("50"));

Figure 7 ISmartDocProperties for All Controls (Close Figure 7)

Property | Description
X | A positive integer that specifies how far from the left position the control should be indented, in pixels.
Y | A positive integer that specifies how far below the normal top position the control should be positioned, in pixels.
H | A positive integer that specifies the height of the control, in pixels.
W | A positive integer that specifies the width of the control, in pixels.
Align | A string giving horizontal alignment of the control within the task pane. "left" (the default) aligns the control on the left of the pane; "center" centers the control within the pane; "right" aligns the control against the right edge of the task pane.

Controls that have labels associated with them (C_TYPE_TEXTBOX, C_TYPE_CHECKBOX) have the additional properties shown in Figure 8 to describe attributes of the control's label. Note that these only affect the control label, not the font in which the control itself is drawn. Finally, there are some miscellaneous properties which apply only to particular types of controls. These are shown in Figure 9.

Figure 9 Other ISmartDocProperties Name/Value Pairs (Close Figure 9)

Property | Applies to Controls of Type | Description
IsMultiLine | C_TYPE_TEXTBOX | A setting of True allows the textbox to accept multiple lines of text, each separated by a carriage return/line feed character.
NumberOfLines | C_TYPE_TEXTBOX, C_TYPE_COMBO, C_TYPE_LISTBOX | The number of lines to display in a textbox. For a listbox or combobox, the number of lines visible without scrolling.
PasswordCharacter | C_TYPE_TEXTBOX, C_TYPE_COMBO | A single character that is used to obscure typing into the textbox so that it may be used to enter passwords.
ExpandHelp | C_TYPE_HELP, C_TYPE_HELPURL | A setting of True expands the help control when originally displayed.
Border | C_TYPE_IMAGE | A setting of True displays a border around the image when the mouse is positioned within the image.
IsEditable | C_TYPE_COMBO | A setting of True allows editing of the text portion of a combobox.
ControlOnSameLine | C_TYPE_TEXTBOX, C_TYPE_COMBO, C_TYPE_LISTBOX | A setting of True (the default value) displays the control label to the side of the control. A setting of False displays the control label on the line above the control.
Expanded | C_TYPE_DOCUMENTFRAGMENT, C_TYPE_DOCUMENTFRAGMENTURL | A setting of True expands the document fragment content in the task pane.
Figure 8 ISmartDocProperties for Control Labels (Close Figure 8)

Property | Description
FontFace | The font in which to display the control label ("Arial," for example)
FontSize | The font size, in points, in which to display the control label
FontStyle | The value "none" (default) or one or more of "underline," "italic," "strikeout" (separated by the "|" character if there are multiple in the list)
FontWeight | The value "normal" (default) or "bold"

Now that you've had a fairly quick tour of the various interface methods, I'll build a Smart Document action handler DLL in Visual Studio® .NET.

Creating a Smart Document Action DLL

I decided to create my Smart Document action DLL in Visual C++ using the Active Template Library (ATL). ATL is a good choice because it's got a smaller footprint than MFC, but still provides plenty of convenient COM functionality you'll need in working with Smart Documents. However, you can certainly choose to use MFC if you're more familiar with it.

After downloading the Office 2003 Smart Documents SDK from MSDN Online, I started Visual Studio .NET and created a new project of type ATL Project, then used Add | Class to add an ATL Simple Object to the project. This is CScheduleSmartDoc in my sample code. From the class context menu for that object (use View | Class to bring up the class explorer), I used Add | Implement Interface to have this class implement the ISmartDocument interface. If you don't have Office 2003 installed on your development machine, you'll need to copy the type library from a machine with Office 2003 to do this. You can usually find it in C:\Program Files\Common Files\Microsoft Shared\Smart Tag\mstag.tlb, but make sure you grab version 1.2 (the Office 2003 version). Then you should implement the methods you need, using my sample code and interface descriptions as a guide.

Installing Your Smart Document Solution

Office uses an XML file (called the XML expansion pack manifest file) to describe your Smart Document solution, including directions for updating and deploying the solution. Typically, this file is called manifest.xml. I've added a skeleton manifest.xml to my sample code (see Figure 10) and highlighted in red the things you'll need to change.

  • The solution ID. You should use GuidGen to generate a unique GUID for your Smart Document solution. It's imperative that all solutions have a unique solution ID and, while it's not strictly required to be a GUID, that's the best way to ensure uniqueness.
  • The updateFrequency, which is how often (in minutes) Office should check the server for updates to the manifest. Updates are controlled by the version tag of each file specified in the XML expansion pack manifest. If the versions are different, and the update time has expired, Office will download the new version of the corresponding file from the server to the local machine.
  • The file path to your Smart Document action DLL. This can be a URL or network path if the DLL is deployed on a network or a path or file name relative to the location of the manifest file.
  • The CLSID of your Smart Document action DLL. Because it implements the ISmartDocument COM interface, it has a COM CLSID that Office will use to load it. You'll find this in the header file of the ATL class you generated to implement the ISmartDocument interface. It's the uuid property of the coclass (CScheduleSmartDoc in my sample code).
  • If there are additional files you want to install along with your solution (such as GIFs or HTML files that contain additional text) you can list them here, as separate <file> elements under the <solution> element. I have one additional GIF for the graphic that appears in my task pane.

Figure 10 XML Manifest File (Close Figure 10)
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<manifest xmlns="http://schemas.microsoft.com/office/xmlexpansionpacks/2003">
    <version>1.1</version>
    <updateFrequency>20160</updateFrequency>
    <uri>ScheduleSmartDocument</uri>
    <solution>
        <solutionID>{15960625-1612-46AB-877C-BBCB59503FCE}</solutionID>
        <type>smartDocument</type>
        <alias lcid="*">Simple Schedule Smart Document</alias>
        <targetApplication>Excel.Application.11</targetApplication>
        <file>
            <type>solutionActionHandler</type>
            <version>1.0</version>
            <filePath>ScheduleSmartDocument.dll</filePath>
            <CLSID>{0A9D54DE-12F8-4711-AF10-7D4ACA1D1924}</CLSID>
            <regsvr32/>
        </file>
        <file>
            <type>Other</type>
            <version>1.0</version>
            <filePath>ScheduleGraphic.GIF</filePath>
        </file>
    </solution>
    <solution>
        <solutionID>{443E624C-AE85-4b20-8522-1CFD5B4E7CBC}</solutionID>
        <alias>Schedule Smart Document XSL Transform</alias>
        <type>Transform</type>
        <context>http://schemas.microsoft.com/office/word/2003/wordml
            </context>
        <file>
            <type>primaryTransform</type>
            <version>1.0</version>
            <filePath>ScheduleReport.xsl</filePath>
        </file>
    </solution>
</manifest>

One other important thing which blocked me for a few hours is that the <uri> element in your manifest must have the same name as the XML schema you are using for mapping your document elements. In my case, this was ScheduleSmartDocument.

Once your Smart Document is marked up with an XML schema (see the earlier section on creating the Smart Document) you can attach an XML expansion pack to associate your action handler DLL with the document. To do this in Excel, go to the Data | XML | XML Expansion Packs dialog. You use this dialog to first add a new expansion pack (pointing to the manifest.xml file for the solution), then attach an expansion pack to the document. This process is similar to marking the document with XML; you only need to do this as the Smart Document developer—users won't need this dialog. In Word, you use the XML Expansion Pack tab of the Add-Ins dialog (Tools | Templates and Add-Ins) to attach an expansion pack to the document.

When users open a document with an associated expansion pack, Office will automatically install or update the expansion pack files on their local machine. Expansion packs are typically installed to the C:\Documents and Settings\All Users\Application Data\Microsoft\Schemas directory, but this is subject to change in future versions of Office. If the user doesn't have administrative rights, the expansion pack files are installed in the corresponding per-user application data directory instead (for example, C:\Documents and Settings\Mike\Local Settings\Application Data\Microsoft\Schemas). Files in the XML expansion pack marked <runFromServer>true</runFromServer> will not be downloaded and installed on the user's machine, but will instead be opened directly from the server location on each use.

If you've been developing a solution on your local machine, you may find that after copying it to the deployment server, Office still looks for the manifest file in the path you used on your development machine. I had this problem and eventually found that Office encodes the path to the solution in a custom document property called Solution URL. After deploying the files to the server, you need to either change this property to the server path or use the XML Expansion Packs dialog to attach the expansion pack from the deployment server (which will reset this property value). You then save your Excel workbook back to the server with the server path to the solution files.

To test this as a user, you should then close the workbook, go to Data | XML | XML Expansion Packs (under Tools | Templates and Add-Ins in Word) and remove the ScheduleSmartDocument expansion pack files from your machine. This makes your development machine look like a normal user machine, which knows nothing of your expansion pack. Next, open the spreadsheet from the server. A dialog will appear asking if you want to download the XML expansion pack, to which you should click Yes. Office will then download and install the files listed in the expansion pack manifest on the local machine.

XML expansion packs have more features than you need for this simple solution. They can be used to distribute virtually any type of file from a server to the client, and can even be chained together into "manifest collections" which will cause Word or Excel to reference several chained manifests as if they were one large manifest. A good summary of all the fields in the expansion pack manifest file is in the article by Paul Cornell that I referenced earlier.

Accessing the Smart Document Content

To be useful, your Smart Document action DLL is often going to need to access (and possibly modify) the underlying Office document. In my case, that is the Excel project schedule spreadsheet. Several of the ISmartDocument methods provide you with a pointer to the IDispatch COM interface for the underlying document for just this purpose. As anyone who's programmed COM against Office knows, IDispatch provides the tunnel into the Office object model. In Visual Basic, using IDispatch and type libraries happens fairly automatically, but it's a bit more involved in C++.

At first I tried to use the #import directive, which allows Visual C++ to generate ATL smart pointer wrappers for all the interfaces in a type library. However, it required quite a bit of gymnastics to get this to compile correctly, and that is often a hint that something is awry. Sure enough, I found Knowledge Base article, Q238987 "Office Application Remains in Memory After Program Finishes", which describes known problems using the #import directive with Office type libraries and advises against it.

In the end, I decided to develop my own class, CExcelWorkbook, to wrap the Excel methods I need. This class inherits from the ATL template CComDispatchDriver, making it relatively easy to call methods on the Excel object model through IDispatch. Using the CComDispatchDriver method GetIDOfName, you can obtain a DISPID for a given property or method (for example, Range or ActiveSheet in the Excel object model). Once you have the DISPID (which I cache in the class for greater efficiency), you can use one of the other CComDispatch methods (such as GetProperty or InvokeN, where N is the number of parameters) to access properties and methods in the object model. Notice that the Excel Range value is exposed as a VBA property (rather than a method), but it also requires a parameter (the range address). Since CComDispatchDriver doesn't provide a way to pass a parameter to a GetProperty call, I implemented a new method (GetProperty1) in my custom class (CExcelWorkbook) which takes care of this.

Debugging Smart Document DLLs

Nothing ever goes as planned and my first attempt at doing the Smart Document action DLL was no exception. I tried to attach the XML schema in Excel and received an error message: "XML Expansion Pack Logic is Missing or Invalid." To debug this, I first used ListDLLs (a very handy tool from http://www.sysinternals.com) to determine that Excel was in fact loading my library, so the manifest seemed at least good enough to let Excel find the DLL. I quit Excel and then modified the Debugging properties of my project (right-click the solution and select Properties) to tell Visual Studio to use Excel as the EXE container for the DLL. For the Command value, I browsed to Excel.exe and selected it.

I then set a breakpoint on DllMain by pressing Ctrl-B to bring up the New Breakpoints dialog, typing DllMain in the Function field, and when a "disambiguate symbol" window came up (apparently there are two overloaded DllMain functions), selecting both of them. My goal was to get control when Excel first loaded the DLL. I then pressed F5, and Excel started up. Visual Studio warned me that I didn't have symbols for Excel, but I already knew that. I opened my Smart Document and tried again to attach the XML schema using Data | XML | XML Expansion Packs. Sure enough, Visual Studio popped up with the breakpoint in my ATL-generated DllMain.

My goal at this point was to determine what other methods, if any, were being called in my DLL. I set breakpoints on all my methods in my ISmartDocument interface implementation to see if any were getting called. In fact, several were and stepping through I was able to find that I had a very common problem of the index being off by one. The control index passed to get_SmartDocXmlTypeName is one-based, but my C++ code was treating it as zero-based, so the call for the last element was returning E_INVALIDARG. I later added the ATLTRACE2 macro to the entry point to all my interface methods, making it easier to debug what was being called and when.

Change Notifications

For my solution, I needed to know when the user selected a row in the Items area of the spreadsheet, which meant he wanted to edit information about that task. I then grabbed information from the current row and populated my task pane, allowing him to enter information about the week's work and planned work for the upcoming week. When he changed something in the task pane, I would use the Excel object model to copy that new information back into the worksheet row. But how would I know when a new row is selected? There is a hard solution here, which involves subscribing to Excel events (easy in Visual Basic, not so easy in C++), but it turns out to be unnecessary. The ISmartDocument interface methods are called whenever the selection changes in Excel, so if anything, you get too much change notification. By updating my internal view of the schedule tasks (CScheduleTaskList) on each change, I was able to ignore row selection changes.

Security

As macro viruses and other malicious uses of scripting technology have become widespread, Office has been adapted in two ways: by increasing the default security settings to prevent running most unsigned, potentially malicious code, and by proliferating the number of security settings to allow more administrative control over which solutions can be run even if not digitally signed. This recognizes the reality that many Office solutions (macros embedded in Word and Excel documents) are already widely deployed in large organizations and making them fully secure will take time. If a new version of Office were suddenly to require all solutions to be digitally signed (the ideal situation), it would create a deployment barrier for many organizations. However, the proliferation of Office security settings makes security a complex topic that I'll merely touch upon in this article.

The same Office security settings that apply to macros and add-ins apply to Smart Documents. These include:

  • The user's macro security setting (High, Medium, or Low, set in Tools | Macro | Security; the default is High).
  • Whether the Smart Document is being loaded from a trusted location: a trusted file system directory (for example, the per-user or workgroup templates directory), a Web server on the corporate intranet, or a trusted Internet site.
  • Whether the Smart Document components are digitally signed and if so whether the publisher is set as a Trusted Publisher in Tools | Macro | Security | Trusted Publishers.
  • Whether "Trust All Installed Add-Ins and Templates" is selected in Tools | Macro | Security | Trusted Publishers.

By default, the security level in Tools | Macro | Security is set to High and Office prevents loading add-in DLLs (including Smart Document action DLLs) that are not digitally signed. Other combinations of the security settings just listed can allow Smart Documents to be loaded (possibly with a security prompt to the user), but the fact remains that the best way to guarantee security in your organization (and avoid potentially confusing security prompts to your users) is to digitally sign all your deployed solutions, including Smart Document components, with a digital certificate obtained from VeriSign or GTE CyberTrust. XMLSign.exe, a tool provided with the Microsoft Office 2003 Smart Document SDK, can be used to digitally sign the XML Smart Document manifest file.

In addition, if an Office Smart Document solution is being run from a Web server, both the Microsoft Internet Explorer and Office security settings will affect whether the solution will run. If an XML expansion pack manifest file is on a server that is neither on the Internet Explorer trusted sites list nor in the intranet zone, no attempt will be made to retrieve it, nor will users be prompted to add the site to the Internet Explorer trusted sites list. If an XML expansion pack manifest file is on a server that is trusted or on an intranet, whether the manifest will be loaded depends on whether it is signed. If it is signed and allowed to run, it is still subject to the user's Office security settings.

During Smart Document solution development, you can enable or disable XML expansion pack manifest file security checking by editing the REG_DWORD value "DisableManifestSecurityCheck" under the registry key:

HKEY_LOCAL_MACHINE\Software\Microsoft\Office\Common\Smart Tag
A value of 1 disables XML expansion pack manifest file security checking, while a value of 0 enables it. If the registry setting is set to 1 (security checking is disabled) when you try to reference an XML expansion pack manifest file, a dialog box appears warning the user that disabling security is dangerous. It also gives you the option to reenable the XML expansion pack security checking immediately by clicking Yes. You should only set this registry value to 1 on a development machine during development of the solution (where it may be inconvenient to digitally sign each component after every build), but be sure to reenable it for the final testing of signed components. It is highly inadvisable to disable XML expansion pack manifest file security checking on user machines.

While this article discusses the creation of Smart Document action DLLs in C++, it is possible to create managed action DLLs in Visual Basic .NET or C#. In that case, the .NET Framework security model applies, but it is beyond the scope of this article to describe it in detail.

Generating the Status Report

My sample code uses IDispatch against the Excel object model to access information from the Schedule Smart Document worksheet to generate the status report. I could have also developed some C++ code to drive the Word object model to build the status document, but I opted for another approach, which also has the advantage of demonstrating the new XML features in Word 2003. I generate the status report as an XML file and open it in Word using XSLT to provide the nice formatting. This approach has the advantage of making it easy to combine several XML files (say, status reports from an entire team) into one large report using XSLT, which would be more difficult to do if they all were persisted as Word documents.

After mulling a bit about how to implement XML persistence, I decided to define classes to manage the task list of items that would be on the status report. As the developer is updating information in the schedule, this task list is being created and updated by code in the Smart Document action DLL, responding to controls it has placed in the document actions work pane. Then, at the end, the developer clicks a button in the work pane to generate the XML status report. Each class is responsible for correctly persisting the information it stores into XML using the MSXML DOM. The classes that handle this are CScheduleTaskList and CScheduleTask. The Word XSL stylesheet I use to format this is ScheduleReport.xsl, available in the sample code. When you install the Smart Document solution, this XSLT file is installed and made known to Word, so that when you open the XML status report in Word, it automatically applies this stylesheet. You could also use the XML Data Views task pane in Word to select this stylesheet.

One XML processing directive worth mentioning is in the XML I generate (see CScheduleTaskList::SaveToXML for the code that adds this and generates the XML status report):

<?mso-application progid="Word.Document"?>
This directive tells Windows Explorer that the default application for this XML file is Word. You'll also notice that the files icon changes as a result of this.

Getting the Word stylesheet to display the status report nicely took a bit of time, so I won't go into the details here. I'll warn you, though, that if you look at the stylesheet I've provided, you may be astonished at how complex the Word XML schema appears. Much of this is boilerplate code, a fair amount of which I obtained by creating a sample status document in Word (setting the styles and fonts the way I wanted them) and then saving it in XML. I copied large parts of the generated XML into my XSLT stylesheet. However, I make no claim that this approach generates the minimal XML needed to correctly format the status report in Word.

I can offer a few suggestions for developing an XSLT based on my experience. Remember that XML is a very precise, case-sensitive language. It's not unlike C++ in that way, but the diagnostics you receive if you get something wrong (if in fact you receive any diagnostics) tend to be of the "can't apply stylesheet" variety rather than the detailed descriptions compilers provide. If you're making large-scale modifications to something that already works, do so incrementally and with good version control so you can roll back your changes. I encountered a problem where my stylesheet referenced the namespace ScheduleSmartDocument but due to an error in the action DLL, the namespace in the generated XML status report was ScheduleSmartDoc. As a result, Word simply displayed an empty document when I applied the XSLT to the XML status report because the XPATHs used in the stylesheet matched nothing in the XML.

Getting Word to automatically apply the correct stylesheet took some work. In the end, I had the manifest that installs the Smart Document action DLL also install the stylesheet (but as a different solution of type Transform which registers the XSLT in Word's library of known XSLTs). For Word to be aware of the stylesheet, the solution entry in the manifest must have a context attribute that references the Word XML namespace (http://schemas.microsoft.com/office/word/2003/wordml).

The final key step is to use the Smart Document URI (ScheduleSmartDocument in my example) as the XML namespace in the manifest, the stylesheet, and the generated XML status report document. In my case, I made up the name ScheduleSmartDocument for my namespace. You should probably use a namespace that is based on the unique URL for your organization.

Working backwards to the XML from a Word document that looks right is a good approach. However, be very careful about slicing and dicing the resulting Word output. For instance, I tried to reduce the complexity of the lists section (which describes list formats used in the document) because I needed just a simple, one-level bulleted list for my status report. After an hour of errors, I gave up and left a lot of the section intact as Word generated it, even though I'm sure there's a lot there I don't really need.

If you open a generated Word XML file in Visual Studio, you'll find that it's too complex for the Visual Studio .NET XML schema viewer. So I suggest that you first click the Data view when the XML file is open, forcing Visual Studio to parse the XML, then go back to the XML view to see a nicely formatted version (which, when opened in Notepad, is virtually unreadable).

My final suggestion is that you use a tool I found very useful while tuning the stylesheet. The MSXSL.EXE command-line tool allows you to apply a stylesheet to an XML file and see the generated XML output. You can download it from here.

Conclusion

I first set out to develop the action DLL in C++ because there are so many examples of Office coding in Visual Basic. However, doing even relatively simple tasks like referencing a cell value in Excel with C++ is significantly more complicated than it is using Visual Basic. See Figure 11 for my C++ equivalent of the ActiveSheet.Range("A1").Value statement in Visual Basic (note that the sample code has a different version of this method because I rewrote it to share code with the set value method). I've provided C++ code for doing a few of the things a Smart Document action handler might want to do with the Excel object model, but the example I've written really doesn't benefit at all from being written in C++. I'd encourage readers to seriously consider the costs of using C++ versus Visual Basic.

Figure 11 ActiveSheet.Range("A1").Value in C++ (Close Figure 11)
HRESULT CExcelWorkbook::HrCellValue(
    int iRow,
    int iColumn,
    CComVariant *pValue)
{
USES_CONVERSION;
HRESULT        hr;

    ATLASSERT(p);        // IDispatch must be set prior to
                         // calling this method.
    if (pValue == NULL)
        return E_INVALIDARG;

    pValue->Clear();

    AssureDispidRange();

    TCHAR        tzRangeReference[20];

    ATLASSERT(iColumn<26);     // Not too hard to support > 26,
                               // but not needed here

    // Handle special value for "CurrentRow"
    if (iRow == iCurrentRow && FAILED(hr=GetSelectedRow(iRow)))
        return hr;

    wsprintf(tzRangeReference, "%c%d", 'A'+(iColumn-1), iRow);

    CComVariant    varRangeName(T2COLE(tzRangeReference));
    CComVariant    varRange;

    if (SUCCEEDED(hr=GetProperty1(m_dispidRange, &varRangeName,
                  &varRange)))
    {
        ATLASSERT(varRange.vt == VT_DISPATCH);
        ATLASSERT(varRange.pdispVal);
        LPDISPATCH            lpRange = varRange.pdispVal;

        CComDispatchDriver    dispRange(lpRange);

        AssureDispidRangeValue(dispRange);

        hr = dispRange.GetProperty(m_dispidRangeValue, pValue);
    }

    return hr;
}

Another option with Office 2003 is C#, which is attractive, but be sure you are aware of the performance issues with having unmanaged code (Office) calling managed code (your action handler assembly written in C# or Visual Basic .NET). It may just be that Visual Basic is the best way to go.

Smart Documents provide a remarkably powerful user interface paradigm for developers. You can harness all the power of Microsoft Office applications while providing custom UI and behavior seamlessly integrated into the Office task pane user interface. Combined with the ease of updating and deploying Smart Documents, I expect they will soon be as widely used in large organizations to simplify business processes as spreadsheets with macros are today.

For related articles see:
Smart Documents Development Overview
Smart Documents in Microsoft Office 2003
Microsoft Office Word 2003 XML: Memo Styles Sample
Visual Studio Tools for the Microsoft Office System
Developing Microsoft Office Solutions Using C#

For background information see:
Office 2003 XML: Integrating Office with the Rest of the World by Simon St. Laurent (O'Reilly, 2003)
CodeNotes for XML by Gregory Brill (Random House, 2002)

Mike Kelly, freelance technical writer and consultant, has just returned from a year in France and is now based in Redmond, WA. He was a developer and development manager on the Microsoft Office team from 1993-2002. His article on Windows Installer was published in Microsoft Systems Journal (September 1998). He can be reached at mikekell@exmsft.com.


Office 2003: Host an Interactive Visio Drawing Surface in .NET Custom Clients

Office 2003

Host an Interactive Visio Drawing Surface in .NET Custom Clients

Mai-lan Tomsen Bukovec and Blair Shaw
This article assumes you're familiar with C#
Level of Difficulty 1 2 3
Code download available at: Visio2003.exe (325KB)
SUMMARY

Microsoft Office Visio 2003 introduces a new drawing component that allows you to embed an interactive drawing surface into your application's user interface. You can drive the Visio drawing component from events in your host application or with data from a Web Service and an ADO.NET data adapter. The Visio drawing component supports the rich Visio application programming model, giving you control over how graphics are used and displayed on the drawing surface. This article explains how to embed the Visio drawing component into a C#-based Windows Forms client app that retrieves data from the Fabrikam 2.0 Web Service.

Contents
Drawing Component Fundamentals
The Visio Component API
The Visio Component at Work
Building the Application
Adding Shapes to the Visio Surface Using Events
Using Visio Events with the Host Application
Conclusion

I
ntegrating programmable graphics into line-of-business applications makes them more user-friendly but requires considerable development time and graphics expertise. The Visio® drawing component, available in Microsoft® Office Visio 2003, lets you build upon the functionality of rich diagramming capabilities through Visio automation. The Visio component can be embedded in a number of containers, ranging from Office documents to custom applications built in Visual Studio® .NET, so it's quite versatile as well. In this article, we'll show you how to drive the Visio drawing component from a Microsoft .NET-based client application to give your users the ability to manipulate graphics.

Drawing Component Fundamentals

The Visio drawing component is essentially an ActiveX® wrapper for the main Visio library. Like any ActiveX component, it can be embedded in custom host applications developed using Visual Studio .NET 2003, Office 2003 containers (such as Microsoft Word), Microsoft Internet Explorer 5.5 and greater, and other ActiveX component containers. Although the component uses the ActiveX architecture, it is more of an embeddable application component than the typical ActiveX control found on a Web site. The Visio component provides the full functionality of the Visio application rather than a small subset of features. The Visio control differs from the Visio client application in two ways. First, the drawing control only supports single document interface (SDI) windows. The control architecture supports a single drawing in a single window. That means that all multiple document interface (MDI) windows in Visio are disabled in the control context. The Visio component allows access to neither the ShapeSheet window (although you can program against the ShapeSheet), nor the icon editor window. Second, the drawing control does not load Visual Basic® for Applications (VBA) or run VBA code. That means that any existing VBA code must be ported to host application code. It also means that users never see the Visio macro warning dialog when opening a Visio document in the control.

The Visio component in the Microsoft Office Visio 2003 release opens up a new variety of options for integrating programmable graphics in your application, such as:

  • "Live" Visio organization charts displayed in the Visio component embedded in a Word document, automatically updated from events in the Word 2002 or Word 2003 document itself.
  • The Visio component wrapped in a SharePoint™ Web Part, displaying timeline drawings driven from the Microsoft Project Server Web Service or another SharePoint Web Part.
  • Visio business process diagrams displayed in the Visio component embedded in a custom Visual Studio .NET container, or a Word Smart Document, updated with status changes in the workflow.

Because it essentially repackages the Visio application engine in a component, the Visio control is installed with the Visio application setup program and requires the Visio client application to initialize. If the user does not have the Visio client application installed on the machine, the control will fail to initialize in the host container. You can handle this event gracefully in your host application at initialization time by checking the registry to see if Microsoft Office Visio 2003 is installed. If the right version of Visio is present, you can provide the user with the full rich graphical interface. If not, you can downgrade to a less graphically rich user experience.

You have two options in deploying the component. You can either require Visio 2003 on the client machine (it can coexist with Visio 2002), or the user can install the Visio component through the minimal install option of the Visio setup program. You can chain the Visio installation before your app installation and set the INSTALLLEVEL to 40, which tells the Windows® Installer application in Visio to install Visio silently. That way, your users will not see the Visio setup UI while installing your application.

The Visio Component API

The drawing component has a thin API that contains a handful of properties, two of which, Document and Window, provide access to the full Visio programming model. See Figure 1 for a breakdown of the Visio component properties and their usage.

Figure 1 Visio Drawing Component Properties (Close Figure 1)

Property | Action | Description
Document | Returns the underlying Visio application Document object. | Gets access to the Visio application object model.
Window | Returns the underlying Visio application Window object. | Gets access to the Visio application object model.
HostID | Returns or sets the string that defines where to get registry settings. | Defaults to an empty string (uses the Visio setting). Char limit of 128. Set up a separate registry hive for the Visio component so that settings don't conflict with the Visio application.
SRC | Returns or sets the string for the file to load. | Loads an existing document, stencil or template, or a blank drawing.
PageSizingBehavior | Indicates if the control should resize the page when the drawing is loaded into the control. | Automatically resizes the drawing loaded in the control to fit the control's dimensions.
NegotiateMenus | Indicates if the control does menu merging with the host container. | Allows manipulation of the Visio menus through the Visio object model.
NegotiateToolbars | Indicates if the control does toolbar merging with the host container. | Allows manipulation of the Visio toolbars through the Visio object model.

If you embed the Visio component in your host container without any extra programming, you offer the Visio drawing surface to your users within the context of your application's user interface. Figure 2 shows what the control looks like embedded in a Windows Form with no code behind it.

Figure 2 Visio Control
Figure 2 Visio Control

With no additional programming, the Visio component gives you a wide range of features. There are visual effects such as page tabs, scrollbars, rulers, and background grid on the Visio page. You can also turn these off (using Visio properties like Window.ShowRuler and Window.ShowTab) to generate a simple white background if you prefer.

There are right-click menus for shapes and pages, but no other Visio application toolbars or menus. You can turn on the Visio user interface toolbars programmatically, but it's better to listen for events on the drawing component and present your own custom application user interface instead.

Additionally, you have access to the current Visio application settings. You'll see that the background behind the Visio page is green rather than the default blue. That's because we have set the background color of our application to green. If we change the drawing component's HostID property (see Figure 1 for a description of HostID), the control will create a separate registry entry for Visio running as a component. As a result, changing the HostID property to the value of "MyNamespace.SampleApp" in design-time mode switches the background color for the control back to the default color of blue. Thereafter, when we run the Visio-based application, our page background will be green. When we run our form with the Visio component, the page background will remain the default blue (unless, for some reason, we decide to change it programmatically).

Finally, there is no stencil pane. You can use Visio automation to make the green stencil pane visible in the control so that users can drag and drop shapes onto the drawing page. In the control context, it's more likely that the shapes are displayed in a toolbar on the Windows Form or dynamically dropped onto the Visio drawing surface from data or events in the Space Plan application.

The Visio component gets a lot more interesting when you start to drive it using the Visio object model and events in your host application or data from a Web Service or database. The Visio object model is both robust and extensive. In fact, all Visio solutions that are shipped in the box (such as Organization Chart and Database Wizard) use the Visio automation layer. The vast majority of Visio component programming automation occurs using the Visio object model, which is well documented on MSDN® and also ships with the product.

Since the Visio component exposes the entire Visio object model, component programmers can use existing developer material as reference, including the Visio Web site on MSDN, which provides sample code, tutorials, articles, a Visio Software Development Kit, and the Visio automation reference.

The Visio Component at Work

To illustrate some of the key elements of Visio component programming, we'll walk through some of the code in our Space Plan sample application. The full application can be downloaded from the link at the top of this article and requires the .NET Framework version 1.1 and Visio 2003 to run. The goal of the sample application is to demonstrate the use of the Visio component API to access the Visio object model, the firing of events in the Space Plan sample application to trigger drawing creation in the component, and the trapping of events on the drawing component to drive the Space Plan sample application. We'll also show how to save Visio documents in the control context. In addition, we'll discuss some other aspects of the control that aren't explicitly demonstrated in the sample application code.

Figure 3 Space Plan
Figure 3 Space Plan

As you can see in Figure 3, the Space Plan sample application architecture consists of a furniture costing sample form, product details form, and the Fabrikam Web Service 2.0 (a locally installed Web Service downloadable from MSDN) which is used to provide the furniture catalog for the sample application.

Let's spend a moment on the customer scenario for Space Plan. Administration staff typically handles the procurement of furniture or other building supplies. To complete the task of ordering furniture for a conference room, the administrative staff may switch contexts from print or online catalogs to Microsoft Excel for budget management, and then to Word or Excel to generate purchase orders. A tape measure would be very handy to measure the room and the furniture.

Figure 4 Space Plan Sample App
Figure 4 Space Plan Sample App

The Space Plan sample application that we built simplifies this process by consolidating all of these actions in a single C#-based Windows Forms application (Figure 4). It enables you to review furniture options, select and the arrange furniture, and itemize when creating the purchase order.

Building the Application

When the sample application runs, the first thing we do is make an asynchronous call to the Fabrikam Web Service. We can also work with a locally cached data set if the user is not able to connect to the Web Service. The DataGrid displays details such as the retail and wholesale price of furniture available in the furniture catalog.

For simplicity, we've only implemented three furniture buttons, each representing a different product in the DataGrid. The button faces are bitmaps of the shapes in Visio that represent each piece of furniture. When the user clicks the furniture buttons, several things happen. As you can see in Figure 5, the corresponding furniture shapes drop onto the blank Visio drawing surface, where the user can rearrange them as desired. Since the furniture is scaled to the room (as represented by the component's drawing), the user can accurately select the proper quantity of furniture for the space.

Figure 5 Furniture Costing Form
Figure 5 Furniture Costing Form

On the shape add and shape delete events, the running totals for retail and wholesale price increment accordingly. Also, the quantity column in the DataGrid updates to reflect the shapes that are in the Visio component.

To get started with programming the Visio component, add the Visio drawing control to the Visual Studio .NET Toolbox, which adds a reference to the Visio 11.0 Type Library and Visio 11.0 Drawing component Type Library. If you're developing a managed code application like our C# sample, you'll also need to declare the Visio Primary Interop Assemblies (PIA) namespaces. These PIAs allow managed code applications to access the Visio unmanaged type libraries:

using Visio =
    Microsoft.Office.Interop.Visio;
using VisOcx =
    Microsoft.Office.Interop.VisOcx;

To access Visio objects from our C# application, we use the Visio component Window and Document properties cast to Visio application objects since C# is strongly typed:

VisWindow = (Visio.Window)
    axDrawingControl1.Window;

Using the Visio Window object, we can set the ShowRulers, ShowPageTabs, ShowScrollBars, and ShowGrid properties to false, which presents the Visio component as a white rectangle. We also set the window's Zoom property value to 1.00 (which maps to 100 percent) so that the blank white page fills the window. The Visio component itself has one zooming property which, when set, allows the control to resize the drawing to fit the control's window. The Visio-based application, however, has finer-grained drawing zoom that gives you better control. It includes the ability to lock the zooming of the window against any user modification.

Adding Shapes to the Visio Surface Using Events

The Visio component gives you the ability to integrate the drawing surface with the host application using events. Let's take a close look at how the button click event in the Windows Form adds (or drops) shapes in the Visio component.

The first thing we do in our sample application is define where the shapes should appear on the Visio drawing surface. We use an array of structures called ShapeLocation to store the PinX and PinY values for each of the three shapes. The ShapeLocation structure is simple, containing just two doubles (X and Y). Visio, like many drawing applications, is based on two-dimensional geometry. A shape's geometry is essentially a collection of horizontal and vertical locations (vertices). A shape's pin, or center of rotation, determines its location. The pin consists of X and Y coordinates relative to the shape's parent, which can be a page or a group. These coordinates dictate exactly where the shape is positioned on the Visio page. Each ShapeLocation structure defines X and Y coordinates for the location of the shape. We'll need these structures to place the shapes appropriately on the window when the user clicks the shape button on the form.

Figure 6 Chair Coordinates on a Page in Visio
Figure 6 Chair Coordinates on a Page in Visio

When populating the values in the ShapeLocation structs in the array, we use inches, which is the default unit of measure for Visio. The point of origin (0, 0) lies in the bottom-left corner of the Visio page. Figure 6 populates the struct for the OfficeChair shape type with the X and Y coordinate values:

ShapeLocations[(int)ShapeType.OfficeChair].X = .6495;
ShapeLocations[(int)ShapeType.OfficeChair].Y = 2.4271;

These coordinates map to a location close to the point of origin (0,0) in the lower-left corner of the page. In Figure 6, I've added X and Y axes to illustrate how Visio uses the coordinates.

With the shape locations defined, we can obtain a reference to the master shape in Visio we want to drop as part of the button click event handler on the form. A master shape can be a single shape, multiple shapes, a group of shapes, or an object from another application (like a bitmap) that is stored on a stencil. When you drop a master shape onto a page, you create an instance of that shape, which inherits all of the properties and behaviors of the master shape.

For our sample application, we have a Visio stencil called Space Plan.vss that contains the three master shapes of furniture. We'll have to open the stencil first using the Visio OpenEx method to access the masters. Figure 5 illustrates how to drop a shape onto the Visio drawing surface using the Drop method on the Page object. The arguments of the following Drop method specify the master (visMaster) and the X and Y coordinates that we defined in the ShapeLocation structure.

visShape = VisPage.Drop( visMaster, ShapeLocations[shapetype].X,
   ShapeLocations[shapetype].Y);

If the user clicks one of the buttons more than once, we want to make sure that shapes do not overlap one another. In order to avoid this overlapping, we add subsequent shapes at a .1250-inch offset of the PinX and PinY values in the ShapeLocation struct.

Using Visio Events with the Host Application

Now that we've shown how a form event adds shapes to the Visio component, let's look at how to use Visio events with the host application. In our sample, Visio events trigger two actions in the host app: they update the Retail and Wholesale price counters when a shape is added to the Visio drawing surface, and they display a custom Windows Form on a shape's click event.

In our sample, we implement IVisEventProc, a Visio event procedure interface, in our event sink to handle notifications of event objects that we create using the Visio AddAdvise method. This method has better performance than IDispatch when handling Visio events. We'll create a new event list first and call the Visio AddAdvise method to add the ShapeAdded event, as shown in the code snippet in Figure 7.

Figure 7 eventList (Close Figure 7)
Visio.EventList eventList = VisApp.EventList;
Visio.EventList eventListDoc = VisioDoc.EventList;
// for each event you want to receive a notification add an event handler
// here
eventSinkHandler = new EventSinkHandler( frm);
// ShapeAdded visEvtAdd+visEvtShape &H8040. Note the use of the unchecked
// statement to handle numeric overflow with visEvtAdd and visEvtShape.
resultEvent = eventListDoc.AddAdvise((unchecked( (short) _
    Visio.VisEventCodes.visEvtAdd ) + (short)_
    Visio.VisEventCodes.visEvtShape), (Visio.IVisEventProc)_
    eventSinkHandler, "", "" );

Our application uses the Visio ShapeAdded and ShapeDeleted events to increment and decrement the Retail and Wholesale counters. The implementation uses each of these event codes (documented in the Visio automation reference) to determine how to handle the event in our IVisEventProc implementation. Figure 8 shows how we've implemented the IVisEventProc interface to handle the ShapeAdded event, the NoEventsPending event, and the ShapeDeleted event. Figure 9 describes how we are handling each one of these events upon receiving notification through the IVisEventProc interface.

Figure 9 Sample Events Using IVisEventProc (Close Figure 9)

Event | Handler Action
ShapeAdded | Adds the new Action shape to a queue, which we're using to store a first-in, first-out collection of shape objects. The x method takes the shapes from the queue and looks up the cost of that piece of furniture from the DataGrid to increment the Retail and Wholesale price counters.
ShapeDeleted | Removes the shape from the queue. The x method takes the shapes from the queue and looks up the cost of that piece of furniture from the DataGrid to decrement the Retail and Wholesale price counters.
NoEventsPending | Visio fires this event after the application is done processing events in the Visio queue. Once the sample application hears the NoEventsPending event, it processes the items from its own queue.
Figure 8 Implementing IVisEventProc (Close Figure 8)
object Visio.IVisEventProc.VisEventProc(
                short   eventCode,
                object  source,
                int     eventID,
                int     eventSeqNum,
                object  subject,
                object  moreInfo)
{
    Visio.IVApplication        app = null;
    Visio.IVDocument           doc = null;
    Visio.IVShape              shape;
    try
        {
            if ( source is Visio.IVApplication)
            {
                app = (Visio.Application) source;
            }
                else if (source is Visio.IVDocument )
                {
                    doc = (Visio.Document) source;
                }
                switch (eventCode)
                {
                  case unchecked( (short) Visio.VisEventCodes.visEvtAdd) +
                  (short) Visio.VisEventCodes.visEvtShape:
                  shape = (Visio.Shape) subject;
                  HandleShapeAdd( doc, shape );
                  break;

                  case (short) Visio.VisEventCodes.visEvtApp +
                  (short)Visio.VisEventCodes.visEvtNonePending:
                  HandleNonePending( app);
                  break;

                  case (short) Visio.VisEventCodes.visEvtDel + (short)
                  Visio.VisEventCodes.visEvtShape:
                  shape = (Visio.Shape) subject;
                  HandleShapeDelete( doc, shape );
                  break;

                  default:
                  break;
                }
            }
        catch (Exception err)
            {
                MessageBox.Show("Exception in IVisEventProc.VisEventProc: "
                        + err.Message);
                }

Once the furniture shapes are added to the Visio drawing surface, the user can move the furniture around using the mouse or keyboard for any preferred arrangement. To lock down the Visio drawing surface to create more of a runtime mode, simply intercept all Visio mouse and keystroke events and throw them away. If the Visio component doesn't hear any events from the user interface, the net effect to the user will be a read-only drawing that can only be driven through the host application user interface.

In our sample application, if the user right-clicks a shape on the drawing surface, we present a custom form (see Figure 10).

Figure 10 Custom Form
Figure 10 Custom Form

The mouse and keyboard events take place on the Visio Document or Window object, not on the shape itself. To find out the actual shape that the Click event occurred on, you have to use the Visio SpatialSearch property. As demonstrated in Figure 6, SpatialSearch takes the X and Y coordinates for the Click event on the document or page, a constant that indicates that a shape can be contained within another shape, tolerance (typically a very small number since it's the area that you want to search around the Click event), and a flag that indicates that the shapes should be stored front-to-back. The property returns a Selection object that contains the shape that corresponds to the Click event:

Selection visSelection = visioPage.get_SpatialSearch(x, y,
    (short) VisSpatialRelationCodes.visSpatialContainedIn,
    tolerance,(short) VisSpatialRelationFlags.visSpatialFrontToBack);

In order to get the custom Windows Form to display properly, we'll also have to call a simple helper routine that translates the Visio coordinates (lower-left corner point of origin) to the Windows coordinate system (upper-right corner point of origin). This lets us position the form to the right of the shape being clicked using the correct scale.

In our app, the custom properties shown on the Product Detail form are stored in the Fabrikam dataset in the host application. The shape itself only stores one custom property: FabrikamProductID. On the mouse click event, the Visio component passes the cached dataset, FabrikamProductID, and metafile image of the product to a custom constructor for the Product Detail form. The Product Detail form is then populated by the data from the Fabrikam records. (The form also displays a metafile of the Visio shape, which is generated using Visio automation.) You can store data with Visio shapes using either shape custom properties or SolutionXML, which creates a user-defined XML data island on a shape or document. However, when using the Visio component, it's much more elegant to consolidate the data manipulation in the host application and just use the drawing surface for reacting to and triggering events.

The end result is a custom properties form that integrates seamlessly with the host application user interface. We could have used the standard Visio custom properties dialog (accessible through the context menu), but we wouldn't have been able to modify the appearance of the Visio dialog. In our Space Plan application, we don't display any Visio user interface menus or toolbars. We've turned off the default context menu for the Visio shape and replaced it with our custom Product Detail form. (Note that the context menu for the page in Visio is still accessible, in case you want to see what the standard dialog looks like in the context of the sample application.)

The best practice when using the Visio component is to intercept Visio events and display your own custom user interface. This allows you to integrate the Visio component more deeply into your application. We do provide the NegotiateToolbars and NegotiateMenus properties on the drawing component. Set these two properties to True and you can expose the Visio toolbars and menus in the drawing component using the application's CommandBar object or UI object. However, negotiating menus and toolbars can be finicky depending on your host container.

Menu and toolbar merging will only work if the container supports OLE 2.0 menu merging. If your container does not support it (the Visual Studio .NET Windows Form editor is one example), you can use IOleCommandTarget to run Visio commands through the IOleCommandTarget interface. Menu and toolbar merging gets especially tricky when you're using multiple instances of the control in a single application. You can embed the Visio control multiple times in a single application, but the instances share a single underlying Visio application object. As you can imagine, this can result in unexpected behavior if you are trying to do menu merging with multiple active instances of the control.

If you absolutely need to use the existing Visio menus in your UI, check with the Visio Developer Center on MSDN for technical articles for Visio component programmers—you'll be able to get more details on sample code and implementation guidelines. Typically, it's a far better idea to replace the Visio user interface with your own. The simple user interface of our Space Plan app presents a cohesive appearance due to the consistency of user interface elements throughout the application.

Because you have full access to the Visio programming model, you can drive Visio functionality through the menus of your host application. For example, our host application's File menu provides three options: New, Open, and Save. The New item resets the existing drawing in the Visio component by explicitly setting the SRC property to load an existing blank Visio drawing. The Open method allows the user to open an existing Visio drawing in the Visio component. It's important to note that SRC loads a copy of the file, rather than the file itself. As a result, you can't save changes to the physical file unless you use the Visio SaveAs method.

When the control reinitializes, it will reload the file from the SRC property and wipe out any changes made to the drawing previously. If you want to persist the changed document in-stream, reset the SRC property to an empty string after you've initialized the SRC property with an existing file. The next time someone uses the control, they will see the changes in the in-stream version of the document. Note that this in-memory document will not be saved to disk unless you explicitly use the Visio SaveAs method.

In our sample application, we use the SaveAs method for the File | Save menu item and filter the options to save the file as a Visio drawing and template extension (see Figure 11) instead of the full range of options available through the Visio application.

Figure 11 Saving Component Doc to Filtered Extensions (Close Figure 11)
private void MenuFileSave_Click(object sender, System.EventArgs e)
    {
        SaveFileDialog sfd = new SaveFileDialog();
        sfd.Title = "Save Visio Document (*.vsd; *.vsx)";
        sfd.Filter = "Drawing (*.vsd;*.vdx)|*.vsd;*.vdx";
        sfd.FilterIndex = 1;
        sfd.InitialDirectory = Application.StartupPath;

        if (sfd.ShowDialog() == DialogResult.OK)
        {
            try
            {
                VisDocument.SaveAs( sfd.FileName );
            }
            catch ( Exception ComExption)
            {
                Debug.WriteLine( "Exception Saving Document " +
                                  exp.Message);

                throw exp;
            }
        }
    }

For the New menu item, we've set the SRC property to load a blank drawing as the new drawing. This is the only way to start with a newly initialized Document object. The following code demonstrates how we can select all the existing shapes in the window and delete them:

VisWindow.SelectAll();
VisWindow.Delete();
ZeroQty();

This allows us to preserve any custom styles or formatting for our drawing, but does mean that you continue to use the existing Document object. When the control is first initialized in our application, we set the SRC property to an empty string to load a blank document. Note that resetting the SRC property to an empty string will not wipe the contents of the drawing surface. It simply retains an in-memory version of the copy of the original SRC document, with any changes made in the component. To reset the Visio drawing surface, you can choose between the following options: delete all the shapes in the drawing, dynamically destroy and recreate the component, or load an existing blank drawing.

Conclusion

The Visio component, as demonstrated in our sample application, lets you easily integrate programmable graphics with your user interface. In the Space Plan application, we were able to use the drawing surface to graphically represent the shapes in a furniture purchase order, allow the user to rearrange furniture images on a scaled drawing surface, and view the price and quantity impact of furniture selection in the drawing surface.

Because the Visio component provides access to the full Visio automation model, there is extensive support for event handling from the host application as well as for firing events for the host application. You can drive the Visio drawing component from data as well as from events in custom and application containers. More importantly, you can use the Visio component in-process and in context of your host application's user interface.

The Visio drawing component gives you the ability to build upon Visio automation so you can get the benefits of an intuitive, graphical interface without implementing the complexities of a graphical application.

For related articles see:
Programming with the Microsoft Office Visio 2003 ActiveX Control
Integrating Microsoft Office Visio 2003 with Microsoft SharePoint Products and Technologies

Mai-lan Tomsen Bukovec is a Lead Program Manager for Visio at Microsoft. She has written articles for Microsoft Systems Journal and Microsoft Internet Developer and two books for technical business managers for Addison Wesley Longman and Cambridge University Press.

Blair Shaw is an Application Development Consultant working with Microsoft partners. For the past six years, he has worked closely with ISVs and a variety of Microsoft product groups including Visio. You can reach him at blairsh@microsoft.com.


Editor's Note: A Look Back and a Look Ahead

Editor's Note
A Look Back and a Look Ahead

One day you're sitting there, making plans for the summer. Maybe a weekend at the beach house, lazy afternoons by the pool, languid days spent picking wildflowers in the bountiful meadow surrounding our new offices in midtown Manhattan. And then the next day, you turn around and find out that there was no summer. It never got above 56 degrees, it rained every day, the pool was broken, whenever you flushed the beach house toilets the sink overflowed with sea water, and the wildflowers (which were admittedly quite nice because of all the rain) were off limits due to an infestation of West Nile virus-carrying mosquitos, which were there because it rained every day. Then we had to board up the meadow because of the hurricane that never made it this far north. And before we could even unpack our sweaters, we looked up and realized that we're putting the last issue of the year to bed.

Despite the Bummer Summer, 2003 has had many bright spots in the world of programming. We've seen product releases, including Visual Studio .NET 2003 and Microsoft Office 2003. We've seen announcements of some exciting new products as well. Microsoft Windows code-named "Longhorn" was announced, as was the next version of SQL Server code-named "Yukon." There was even an announcement about the next release of Visual Studio .NET and the .NET Framework code-named "Whidbey." In addition, this year has seen the wider adoption of key programming concepts (the .NET Framework and XML Web Services) and, most importantly, an upturn in programming and consulting.

We do have a lot of new products to cover in 2004. Look for special issues on Longhorn and Yukon early next year. But we're going to be covering a lot of other topics that go beyond mere product features. Some of these concepts have been discussed in Editor's Notes in 2003.

Security More than ever, security has been on our minds. We've increased our attention to detail in the magazine to make sure that the samples we bring you are clean and no little security gotchas (like using sa with a blank password) are hiding within.

Best practices As you probably know, there's usually more than one way to accomplish a programming task. The difference between a novice and an expert is often determined by how efficient and sensible their coding solutions are. Best practices represent a generally agreed-upon set of methods that is commonly considered to be the "best" available way to accomplish a programming task. We put our articles through a series of reviews in order to ensure that, as much as possible, each sample and discussion represents the best practices associated with the methods that these articles are designed to demonstrate.

Testing No matter how good a programmer you are, bugs can and will appear in your work from time to time. Expert developers must also be expert testers. We've talked about testing techniques in past issues, and we'll be devoting more space to modern testing and debugging techniques.

Scalability This topic hasn't received enough coverage because it's a tricky subject. But as you know, once something goes live on the Internet, it's difficult to control its load factor. Designing apps for scalability involves elements of both best practices and testing.

Of course, we'll also have the inside track on all of these new developer products upcoming in 2004 and beyond:

Yukon The next version of SQL Server will feature common language runtime (CLR) integration, enhanced XML functionality, an extended T-SQL language, and lots of productivity enhancements. Watch this space for more information soon or check out http://www.microsoft.com/sql/evaluation/yukon.asp.

Whidbey The next release of Visual Studio and the .NET Framework will add CLR and language innovations like generics, as well as deeper integration into Yukon. And edit and continue debugging will be reintroduced!

C++ Although this is part of the Visual Studio "Whidbey" release, it merits its own mention here. Visual C++ in Whidbey will provide C++-specific constructs to access capabilities like generics, as well as core library enhancements. Two notable features include support for Windows Fusion, which makes it easier to deploy DLLs for native applications, and Profile Guided Optimizations (POGO), which is a new technology for instrumenting applications so that Visual C++ can optimize code based on usage patterns.

Of course, we have more surprises up our sleeves. As soon as we figure out what they are, you'll hear all about them. Weather-wise, this summer might've been a crushing disappointment, but we're going to make up for it with a lot of hot topics in 2004.

—J.T.

ActiveX, MapPoint, Microsoft, Microsoft Press, MSDN, MSN, PowerPoint, Visio, Visual Basic, Visual C++, Visual C#, Visual Studio, Windows, Windows NT, and Win32 are registered trademarks of Microsoft Corporation. SharePoint and Windows Server are trademarks of Microsoft Corporation. Other trademarks or tradenames mentioned herein are the property of their respective owners.

MSDN Magazine does not make any representation or warranty, express or implied with respect to any code or other information herein. MSDN Magazine disclaims any liability whatsoever for any use of such code or other information.


New Stuff: Resources for Your Developer Toolbox

New Stuff
Resources for Your Developer Toolbox
Theresa W. Carey

Contents
Mobile Device Development Suite
Customize GUI Apps
Compare and Edit Binary Files
Copyright Watermarking Tool for .NET-based Code
Protect Sensitive App Settings
Migrate Unix Apps to Windows
Live Customer Support in .NET-based Applications
Visual Basic Code Review and Doc Tool
The Bookshelf

Mobile Device Development Suite

ComponentOne LLC has announced the availability of ComponentOne Studio for Mobile Devices, the company's latest subscription for the Microsoft® .NET Compact Framework. It includes ComponentOne FlexGrid for Mobile Devices, ComponentOne Chart for Mobile Devices, and ComponentOne Zip for Mobile Devices.

ComponentOne Studio for Mobile Devices allows you to create Microsoft .NET Compact Framework-based applications for mobile devices. It will be available as part of ComponentOne Studio Enterprise, a complete solution for any Microsoft development environment. ComponentOne Studio Enterprise will include the latest versions of .NET, ASP.NET, Mobile Devices, and ActiveX®.

http://www.componentone.com

Customize GUI Apps

The Xtreme Toolkit Professional Edition, recently launched by Codejock Software, combines two existing products—Xtreme Suite and Xtreme Toolkit. Features include Office XP and Office 2003-style menus and toolbars, true Office colorization technology, enhanced WYSIWYG docking technology, Visio® and .NET auto-hide dockable windows, detachable tabs from dockable windows, and a hierarchical properties grid.

Codejock added customization features for dialogs, toolbars, and menus, plus hundreds of MFC components to the new product. Developers can give their application a user interface that resembles Office or Visual Studio® .NET or create their own theme.

http://www.codejock.com

Compare and Edit Binary Files

HexCmp 1.2, which combines the features of a binary file comparison application with a hex editor, has been released by Fairdell. The program will help you compare files in binary format, simultaneously serving as a hex editor that will edit and modify your files. Comparisons can be made on the fly while scrolling through the files. Features include color highlighting, synchronous scrolling, and an Info panel, which shows the file information in different formats. You can compare full or partial files.

HexCmp 1.2 supports files up to 2GB and allows navigation among different blocks or files, set custom color schemes, and use the Undo, Redo, and Search functions. Besides the hex format, the program can also compare files in text format. You can perform string searching in both formats as well. Since all commands are assigned to hotkeys, you can compare and edit files with HexCmp 1.2 without reaching for a mouse.

http://www.hexcmp.com

Copyright Watermarking Tool for .NET-based Code

RIIS LLC has launched ThatsMyCode, a free online service allowing programmers working in the Microsoft .NET environment to embed a copyright notice into their source code. This copyright notice or watermark allows you to search for and identify instances where others have used your code in violation of applicable copyrights.

This online tool currently works for C# code, with a Visual Basic® .NET version in final testing. Decompiler software that can reduce executables to source code is widely available, and the ongoing arms race between obfuscators and decompilers means that you can never be sure that your code is safe from anonymous theft and reuse.

http://www.riis.com

Protect Sensitive App Settings

Obviex has announced the release of CipherSafe.NET 1.1, a software solution for managing sensitive .NET Framework-based application settings. CipherSafe.NET can be used to protect such application data as database connection strings, passwords, and encryption keys. When you define a profile value, CipherSafe.NET will encrypt and store it so that only the application for which this value was defined will be able to get its plain text.

The product supports a wide range of .NET Framework-based apps including executable files, DLLs, Web Forms, Web Services, and others. It comes with a GUI-based administrative tool, class library, and documentation. The CipherSafe.NET API can be called from any application written in managed code, including C# and Visual Basic .NET.

http://www.obviex.com/ciphersafe

Migrate Unix Apps to Windows

MKS Inc. has launched MKS Toolkit 8.6, which provides scripting and porting tools to automate tasks and help migrate Unix applications to Windows®. MKS Toolkit 8.6 promotes enhanced productivity, platform interoperability, and application portability allowing users to operate Unix and environments that mix Windows.

New features in the version include MKS KornShell performance improvements, addition of Bourne Again Shell (BASH), and a suite that allows better connectivity between Windows and Unix. Along with new features listed in the ksh93 specification and the ability to save and restore the KornShell environment, the new MKS KornShell loads faster at startup and when running scripts and script-based applications.

Customizing the command shell and executing Unix scripts without translation is easier with MKS Toolkit 8.6 as the BASH joins the existing Korn, C, and other shells within the product. The ability to connect to and from machines running Unix and Windows in an automated and secure way is enhanced by an up-to-date version of the Secure Shell service and other utilities. Also important is smartcard support for public key storage and a new rlogind service. MKS Toolkit 8.6 fully supports both Windows Server™ 2003 and the 64-bit Windows operating system on IA-64.

There's also large file support in all tool and APIs, sparse file support on NTFS, remote process control for ps and kill, new compression and Web content manipulation utilities, and enhanced build commands.

http://www.mkssoftware.com

Live Customer Support in .NET-based Applications

SupportChat.Net, published by Software Factory, allows companies to add live customer sales and support to their existing Web sites. You can use the SupportChat.Net smart button on any Web page, not just ASP.NET pages.

SupportChat.Net runs on your own domain and site. Chat text is saved as XML and formatted with XSLT. You can change the formatting and add your own XSLT elements. It uses HTML for the chat window, which means faster display time for your clients and multiple browser support. You can require visitors to identify themselves before engaging in a chat, or let them chat anonymously.

SupportChat.Net can be integrated with your site's shopping cart, e-mail announcements, and more. It also features automatic ranking, a color-coded warning system with visual or audible alarms, after-hours messages, chat invitations, and extensive reporting.

http://www.softwarefactoryinc.com/supportchatoverview.aspx

Visual Basic Code Review and Doc Tool

Aivosto has updated Project Analyzer to version 7, adding semi-dead code detection, duplicate code search, DLL analysis, class design rules, advanced metrics, and updated support for ASP.NET and Visual Basic .NET 2003 to their Visual Basic code review and documentation tool.

Project Analyzer helps to eliminate dead and repeated code. It also finds semi-dead code—which is in use but not required at run time, including uninstantiated classes and unimplemented interfaces. Removal of these unnecessary elements decreases application size and facilitates code maintenance.

The new class design rules enforce coding practices on object instantiation, destruction, inheritance, scopes, and naming. The advanced metrics features can be used to evaluate system complexity, coupling, depth of inheritance, method and attribute hiding, polymorphism, and call and decision density.

Project Analyzer also counts the number of lines and the amount of commenting and dead code and includes features to document a solution with source, comments, and graphs. It supports Visual Basic solutions up to millions of lines of code.

http://www.aivosto.com

The Bookshelf

Addison-Wesley Professional and Prentice Hall PTR have announced four new books for the Microsoft .NET community.

Windows Forms Programming in C#, by Chris Sells, was written to help experienced programmers who possess a basic understanding of Visual C#® .NET and the .NET Framework to master the Windows Forms toolkit. Content includes discussion of the core elements of Windows Forms (with concise examples and graphic depictions of user interface features in action), as well as pitfalls to avoid and solutions to common problems.

.NET—A Complete Development Cycle, by Gunther Lenz and Thomas Moeller, demonstrates how to use sound software development methodologies in developing an application using the .NET Framework. Its goal is to teach developers how to use state-of-the-art software engineering practices in the .NET environment. The book is structured like a real project, and the authors take the reader through the development of a working imaging application.

.NET Patterns: Architecture, Design, and Process by Christian Thilmany is a guidebook on how to apply proven solutions to recurring design problems when coding for the .NET Framework. It serves as both an introduction to patterns for software developers and architects working in or transitioning to the .NET Framework, as well as a library of patterns unique to .NET. Content also includes primers on XML and Web Services, as well as coverage of debugging, exceptions, error handling, and architecture.

In .NET Security and Cryptography, authors Peter Thorsteinson and G. Gnana Arun Ganesh offer an in-depth guide to the worlds of cryptography and security programming for C# and Visual Basic .NET. The book addresses ASP.NET and Web Services security and the new WS-Security Toolkit. It provides real code samples and explains how they work. The book also provides a mathematical and theoretical background on cryptographic concepts.

http://www.awprofessional.com, http://www.phptr.com

Send your New Stuff to Theresa at newstuff@microsoft.com.

Theresa W. Carey is a freelance writer who lives in California. Her byline has appeared in Barron's and PC World.


Web Q&amp;A: Virtual Directories, Releasing DB Connections, and More

Web Q&A
Virtual Directories, Releasing DB Connections, and More
Edited by Nancy Michell

Q I need to include files from a sibling directory under a virtual directory from two different sibling subdirectories, yet make sure parent paths stay disabled (see Figure 1). The virtual include is relative to the root, so I need to know the full path \vdirTwo\subInclude\MyInclude.asp. I want to be able to include just \subInclude\MyInclude.asp from either sibling directory so the include doesn't have to be duplicated in child directories, but I want to do it without parent paths turned on.

Figure 1 Folder Hierarchy
Figure 1 Folder Hierarchy

A To solve your problem, you have two options: turn ASPEnableParentPaths on or create a virtual directory, named inc, pointing to the physical directory that contains the include files. In IIS, turn off script/execute/read permissions for this virtual directory. From the applications in vdir1 and vdir2, you can now do this:

<!--#include virtual = '/inc/foo.asp' -->

Q I'm having a problem with an ASP.NET site I worked on that's not releasing a Microsoft® Access database file. The file remains locked hours after the last site visitor, and the site owner can't upload a new MDB file to update his catalog. If I'm always careful to close OleDbConnection explicitly, do I gain anything additional by calling Dispose instead of, or in addition to, the Close method?

A Calling Dispose also saves the runtime from having to send the object to the finalize queue and then calling Dispose on it. If quite a few objects are ending up in the finalize queue, this can have a negative impact on performance.

Using the Jet database engine from ASP.NET can be difficult. Jet works best if all threads in the same process use the same single instance of a connection (this is the exact opposite of SQL Server™).

As an alternative, you could write static functions called GetConnection and ReturnConnection. GetConnection would hand back to the caller a single instance of the OleDbConnection (stored in a static member variable) and bump up its reference count. GetConnection would take the calling ASP.NET page as a parameter as well, so you could track down which object is not releasing the connection. You should open the database exclusively as well. ReturnConnection would be called by the code when you are finished using the connection (Close would not be called), and this would bump down the reference count. You should also add locking code and a global flag to reject requests for connections. Then you could block callers from getting new connections while you compacted the database and performed other maintenance. Of course, to avoid all of this, you might consider using SQL Server.

Q Is it possible to securely perform SQL Merge replication between a workstation client running the Microsoft Data Engine (MSDE) and a machine running SQL Server across the Internet? If so, where can I find some documentation on how to do it? My main questions are how to enable authentication to occur properly, how to ensure that data is encrypted on the wire, and that the machine running SQL Server remain secure.

A Merge replication in SQL Server 2000 does not currently support Web-based synchronization of incremental changes without using a virtual private network (VPN).

SQL Server Books Online, in the topic called "Publishing Data Over the Internet Using Microsoft Proxy Server," says that integrating Microsoft SQL Server 2000 replication with Microsoft Proxy Server lets you perform replication over the Internet with security configured on Windows NT® 4.0 or Windows® 2000 Server, Proxy Server, and SQL Server 2000.

Using this approach, Proxy Server provides a connection between the Internet and the server running SQL Server 2000. The subscriber connects to Proxy Server over the Internet and uses a pull subscription to retrieve the data. Proxy Server is configured so that unauthorized Internet users are denied access to the internal network, and the subscriber must connect to a port on the Proxy Server that limits access to the services for which permissions have been granted.

For information about how to configure Microsoft Proxy Server for replication, read the white paper titled "Configuring Proxy Server for Replication Across the Internet".

Q How can I completely remove a node in my XML document using C#? For example, if I had some XML like the following

<Results>
    <Run id="1">
       <ID>
       <Name>
    </Run>
       <Run id="2">
          <ID>
          <Name>
       </Run>
</Results>
how would I remove id="2" so that I'd be left with:
<Results>
     <Run id="1">
       <ID>
       <Name>
     </Run>
</Results>

A The easiest way to get rid of a node from an XML doc is to first select the node using an XPath expression; then call RemoveChild on its parent node:

XmlNode oNode = xmlDoc.SelectSingleNode("/Results/Run[@ID=" +
    iRunIDs[i] + "]");
XmlNode oParentNode = oNode.ParentNode;
oParentNode.RemoveChild(oNode);

Q I have an XML hierarchy that ideally looks like the following:

<?xml version="1.0" encoding="utf-8"?>
<ServiceOfferings xmlns="http://tempuri.org/Data.xsd">
    <ServiceOffering>
        <OfferingId>1</OfferingId>
        <Offering>Blah</Offering>
        <Description>Description</Description>
        <Features>
            <Feature>feature one</Feature>
            <Feature>feature two</Feature>
            <Feature>feature three</Feature>
      </Features>
    </ServiceOffering>
</ServiceOfferings>
I wrote an XML schema definition (XSD) that I thought would give me what I was hoping for (see Figure 2). But when I add elements in data view in Visual Studio® .NET and switch back to XML mode, it looks like the code in Figure 3. Why does the content inside those features tags get broken down that way, and how can I correct it so that it's formatted as follows:
<Features>
    <Feature>feature one</Feature>
    <Feature>feature two</Feature>
    <Feature>feature three</Feature>
</Features>

Figure 3 XML Result (Close Figure 3)
<?xml version="1.0" encoding="utf-8"?>
<ServiceOfferings xmlns="http://tempuri.org/Data.xsd">
    <ServiceOffering>
        <OfferingId>1</OfferingId>
        <Offering>Blah</Offering>
        <Description>Description</Description>
        <Features>
            <Feature>feature one</Feature>
        </Features>
        <Features>
            <Feature>feature two</Feature>
        </Features>
        <Features>
            <Feature>feature three</Feature>
        </Features>
    </ServiceOffering>
</ServiceOfferings>
Figure 2 XSD (Close Figure 2)
<?xml version="1.0" encoding="utf-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
  elementFormDefault="qualified">
    <xsd:element name="ServiceOfferings"
      type="ServiceOfferingsList" />
    <xsd:complexType name="ServiceOfferingsList">
        <xsd:sequence maxOccurs="unbounded">
            <xsd:element name="ServiceOffering"
              type="OfferingType" />
        </xsd:sequence>
    </xsd:complexType>
    <xsd:complexType name="OfferingType">
        <xsd:sequence>
            <xsd:element name="OfferingId" type="xsd:int"
              minOccurs="1" maxOccurs="1" />
            <xsd:element name="Offering" type="xsd:string"
              minOccurs="1" maxOccurs="1" />
            <xsd:element name="Description" type="xsd:string"
              minOccurs="1" maxOccurs="1" />
            <xsd:element name="Features" maxOccurs="1">
                <xsd:complexType>
                    <xsd:sequence>
                       <xsd:element name="Feature"
                         minOccurs="0" maxOccurs="unbounded" />
                    </xsd:sequence>
                </xsd:complexType>
            </xsd:element>
        </xsd:sequence>
    </xsd:complexType>
</xsd:schema>

A When a DataSet returns XML, it considers that this

<Features>
    <Feature>feature one</Feature>
    <Feature>feature two</Feature>
    <Feature>feature three</Feature>
</Features>
is a table called "Features" and that it has three fields called "Features." The following format says that there is a table called "Feature," that it has three records, and each record has a field called "Feature":
<Features>
    <Feature>feature one</Feature>
</Features>
<Features>
    <Feature>feature two</Feature>
</Features>
<Features>
    <Feature>feature three</Feature>
</Features>

So, the way the DataSet sees data is different from how XML sees it. (Remember that you are trying to build a text file out of something relational). You can try using the XmlDataDocument to pull it instead. You may never get the IDE to look they way you want it to. I would suggest sticking with one or the other, either DataSet to database, or XML to database.

Q I have an XML file like that looks like this:

<foo xmlns="urn:schemas-microsoft-com:xml-msdatasource">
    •••
    <bar>  ??? </bar>
    •••
 </foo>
I need to write code that, given an XmlReader that is positioned on the node "bar," will extract the entire node (and any subnodes) as a string. This seems simple enough, but when I try to do XmlReader.ReadOuterXml, the string returned has the namespace stuck in it (that is, <bar xmlns="urn:schemas-microsoft-com:xml-msdatasource"> ...).

How can I get a string corresponding to this node exactly as it appears in the file using an XmlReader?

A According to the XmlReader spec, you cannot pull the text without it being associated with the appropriate namespace. Basically, you will have to go through and take those out. Figure 4 shows how you can do it with a little function.

Figure 4 GetNodeString (Close Figure 4)
internal string GetNodeString(XmlNode node)
{
  NameTable nt = new NameTable();
  XmlNamespaceManager nsm = new XmlNamespaceManager(nt);
  Hashtable removeNamespaces = new Hashtable();
  if (node.NodeType != XmlNodeType.XmlDeclaration &&
    node.NodeType != XmlNodeType.DocumentType &&
    node.NodeType != XmlNodeType.Comment && node.NodeType !=
      XmlNodeType.Whitespace && node.NodeType !=
      XmlNodeType.ProcessingInstruction &&
    node.NodeType != XmlNodeType.SignificantWhitespace)
  {
    XmlNode tNode = node;
    if (node.NodeType != XmlNodeType.XmlDeclaration &&
      node.NodeType != XmlNodeType.DocumentType &&
      node.NodeType != XmlNodeType.Comment && node.NodeType !=
        XmlNodeType.Whitespace && node.NodeType !=
        XmlNodeType.ProcessingInstruction &&
      node.NodeType != XmlNodeType.SignificantWhitespace)
    {
      XPathNodeIterator xni =
        tNode.CreateNavigator().Select("namespace::node()");
      while (xni.MoveNext())
      {
        if (xni.Current.Name != "xml") {
          if (nsm.HasNamespace(xni.Current.Name)) {
            removeNamespaces.Add(xni.Current.Name,xni.Current.Value);
          }
          else
          {
            nsm.AddNamespace(xni.Current.Name, xni.Current.Value);
          }
        }
      }
      tNode = tNode.ParentNode;
    }
    string outputText = node.OuterXml;
    IDictionaryEnumerator idenum = removeNamespaces.GetEnumerator();
    while (idenum.MoveNext())
    {
      if (((string)idenum.Key).Length == 0)
        outputText = System.Text.RegularExpressions.Regex.Replace(outputText,
          " xmlns=." + (string)idenum.Value + ".","");
      else
        outputText = System.Text.RegularExpressions.Regex.Replace(outputText,
         " xmlns:" + (string)idenum.Key + "=." +
         (string)idenum.Value + ".","");
    }
    return outputText;
  }
  return node.OuterXml;
}

Q I have an application that uses System.Net.Sockets.Socket to make HTTP requests against an IIS server and get responses. How can I perform authentication with secure pages (using NTLM)? I know it is easy to do with WebRequest/NetworkCredential, but I have to use Socket. Typically you would not need to control that option (the default behavior is that the socket will remain in time_wait state for four minutes), but in my case I have a stress application which generates a very large number of requests concurrently on different sockets in order to simulate multiple clients being connected to the server. So unless I set each socket to not linger, I run out of ports. I would use WebRequest but I need to control keepalive and linger options, which WebRequest doesn't expose.

A You should reconsider using the WebRequest class (see HttpWebRequest Members). If you don't, then you need to read/write the HTTP headers yourself. You also need a good understanding of HTTP. There is a KeepAlive property in HttpWebRequest as well as a Timeout property, although the linger option you mentioned is a socket option, and not the same as timeout. Linger controls how long the socket remains in the time_wait state, waiting for out-of-band data after it is closed. You should really use HttpWebRequest.UnsafeAuthenticatedConnectionSharing supported in .NET Framework version 1.1 and higher. The behavior is similar to ReuseAddress in that it will reuse previous connections so you don't run out of available ports.

Q I have an app built on Visual Basic® 6.0 that performs a screen print (using Screen.ActiveForm.PrintForm). This screen capture ends up being an important legal document for the user. I'm worried about printer jams or out-of-paper situations, and I don't want to hold up the app waiting for printer issues to be resolved. What's the easiest way to capture the current screen to a file so it can be reprinted later?

A You can use the code from Knowledge Base article 161299 "Capture and Print the Screen, a Form, or Any Window". You can then use the SavePicture method to save the picture in the PictureBox. With a little code modification, you could bypass the PictureBox control and just save the picture directly to a file.

Got a question? Send questions and comments to webqa@microsoft.com.

Thanks to the following Microsoft developers: Mike Abrahamson, Jeff Abrams, Ron Alberda, Mark Ashton, Bhavesh Doshi, AJ Giardullo, Mike Joe, Pranav Kandula, Larry Kuhn, Douglas Laudenschlager, Eric Lippert, Matt Neerincx, Chris Sano, Amber Sitko, Bruce Taimana, Alan Tchochiev, Kevin Williamson


Data Points: Exploring SQL Server Triggers

Data Points
Exploring SQL Server Triggers
John Papa

Contents
Trigger Types
Firing Sequence and Referential Integrity
Special Tables
Performance
Until Next Time

Triggers are one of the core tools available in relational databases such as SQL Server™ 2000. As one of the mainstays of SQL Server database programming, triggers also happen to be one of the topics that I get most of the questions about. In this month's installment of Data Points, I will explore the different trigger types that SQL Server 2000 makes available along with many of the features that they expose. When used properly, triggers can play a key role in a data model and in implementing enterprise-wide business rules. Triggers can be implemented to enforce business rules or referential data integrity in database applications. There are even types of triggers that open the doors to possibilities such as allowing data modifications to multiple base tables of a view.

It is very important to evaluate your options when choosing to employ a trigger. In deciding whether you'll use triggers, the key is to balance functionality, scalability, maintenance, and performance. I'll explore some of these factors and offer some insight on how to weigh them. In addition, I'll explain the foundation of SQL Server triggers and the features that they expose. I will examine the differences between the two types of triggers while demonstrating the places where each can be useful. Then I'll show examples of using AFTER triggers that serve an important role in SQL Server-based applications. I will also walk through the use of triggers to enforce referential integrity and to implement business rule validation at the database level. Before wrapping up, I will discuss performance considerations, features unique to triggers, and some limitations to keep in mind.

Trigger Types

The first ingredient in properly employing triggers is to understand the differences between AFTER and INSTEAD OF triggers. AFTER triggers are the same type of trigger that is available in previous versions of SQL Server. They are also known as "FOR triggers" or even simply as "triggers" since they were the only type of trigger available prior to SQL Server 2000.

Let's first look at FOR triggers. You'll notice that the following trigger is created using the FOR keyword:

CREATE TRIGGER tr_Employees_U on Employees FOR UPDATE AS
    IF UPDATE(lastname)
    BEGIN
        RAISERROR ('cannot change lastname', 16, 1)
        ROLLBACK TRAN
        RETURN
    END
GO
This trigger, tr_Employees_U will execute after an UPDATE statement is run against the Employees table. It will then check to see if the lastname field was modified and if so it will raise an error and undo the changes that the UPDATE statement made. To accomplish this, this code uses three very common features of triggers: UPDATE, RAISERROR, and ROLLBACK TRANS. I will explain these in more detail later.

That syntax is also acceptable in older versions of SQL Server. However, now that there are two types of triggers in SQL Server 2000, I prefer to refer to FOR triggers as AFTER triggers. Thus, for the remainder of this article I will refer to either AFTER or INSTEAD OF triggers.

AFTER triggers execute following the triggering action, such as an insert, update, or delete. The example trigger you just saw will fire after an UPDATE statement has been executed against the Employees table. Therefore, the trigger does not fire until the row or rows have been inserted and constraints have been checked and passed. Basically, AFTER triggers fire very late in the process.

INSTEAD OF triggers, introduced with SQL Server 2000, are intended to be employed in different situations. INSTEAD OF triggers fire in place of the triggering action. For example, if an INSTEAD OF UPDATE trigger exists on the Employees table and an UPDATE statement is executed against the Employees table, the UPDATE statement will not change a row in the Employees table. Instead, the UPDATE statement causes the INSTEAD OF UPDATE trigger to be executed, which may or may not modify data in the Employees table.

The following trigger will fire in place of any UPDATE statement made against the Employees table:

CREATE TRIGGER tr_Employees_U_insteadof ON Employees INSTEAD OF UPDATE AS
    IF UPDATE(lastname)
    BEGIN
        RAISERROR ('cannot change lastname (source = instead of)', 16, 1)
        ROLLBACK TRAN
        RETURN
    END
    ELSE
        -- Go ahead and do the update or some other business rules here

GO
Like the AFTER trigger you saw earlier, this trigger prevents changes from being made to the lastname field. However, it implements this business rule differently than the previous example. Because the INSTEAD OF trigger fires in place of the UPDATE statement, the INSTEAD OF trigger then evaluates if the business rule test passes or not. If the business rule test passes, in order for the update to occur the INSTEAD OF trigger must explicitly invoke the UPDATE statement again.

In this situation, the AFTER trigger would be a more efficient mechanism to enforce this business rule because there is no overwhelming benefit of the INSTEAD OF technique and the AFTER trigger requires less complex code logic. This is especially true if you wanted the INSTEAD OF trigger to perform the original update since it would have to reconstruct the statement. This is possible using the UPDATE function and the inserted and deleted tables, but it is much more complicated than necessary.

One question I get frequently is whether you can have multiple triggers hanging off of a single table. There can be several AFTER triggers associated with a single table, even multiple AFTER triggers on the same action query type (UPDATE, INSERT, or DELETE). For example, the Employees table could have two distinct AFTER UPDATE triggers associated with the table, each of which performs a different set of tasks. In fact, there could be three or four, or even more AFTER triggers associated with the same table; they will all fire following the UPDATE statement. (Keep in mind that even though there could be several triggers associated with the same table, they each must have a unique name.)

You can even tell SQL Server which order you want the AFTER triggers to fire by using the system stored procedure sp_settriggerorder. However, I have rarely wanted to have more than one trigger hanging from the same table upon the same action query, and even when I did, I didn't care about the sequence they fired in. Most often you can combine all of your logic into a single AFTER trigger. But in cases in which it is easier to break logic into separate code blocks, different triggers would do the trick.

Note, however, that there can be only one INSTEAD OF trigger on the same table and same action query type. Remember that INSTEAD OF triggers replace the originating action query, so if there was an INSTEAD OF UPDATE trigger on the Employees table it would fire in place of the UPDATE query statement. However, there cannot be two INSTEAD OF UPDATE triggers hanging from the Employees table. Therefore, there can be at most three INSTEAD OF triggers associated with a table; one for each action query type—INSERT, UPDATE, and DELETE.

Firing Sequence and Referential Integrity

One of the most important factors in understanding the differences between AFTER and INSTEAD OF triggers is to understand their firing sequences. An AFTER trigger does not fire until after the action query that invoked the AFTER trigger has made its data modification. In addition, it is important to know what other events occur surrounding the AFTER trigger. If the Employees table had a single AFTER UPDATE trigger associated with it and an UPDATE statement was executed on the table, the following sequence of events would unfold.

  1. The following statement is executed:
    UPDATE Employees SET lastname = 'papa' WHERE lastname = 'king'
  2. All constraints are enforced.
  3. All declarative referential integrity (DRI) constraints are enforced (via foreign keys).
  4. The inserted and deleted tables are created for use within the trigger.
  5. The triggering action (in this case the UPDATE statement) is executed.
  6. The AFTER UPDATE trigger executes.

These steps follow sequence as long as each step succeeds. For example, if a NOT NULL constraint is violated, then the DRI is not checked. After reviewing this firing sequence for AFTER triggers, it is clear why referential integrity cannot be enforced through an AFTER trigger if the same reference is being enforced via DRI with a foreign key constraint: the trigger would never fire. An AFTER trigger cannot be used to execute any logic that might prevent constraint violation from occurring since the trigger isn't fired until all constraints have been checked.

It is not generally good practice to enforce DRI through a trigger unless there needs to be some more involved business logic surrounding the referential integrity check. In those cases, the foreign key would have to be removed and the trigger would enforce the referential integrity. However, I recommend against using triggers to enforce referential integrity since this approach is less efficient than using a foreign key. It means the data is written to the table and then has to be rolled back.

Assuming that there is only a single INSTEAD OF UPDATE trigger on the Employees table, let's set up another scenario to demonstrate the firing sequence of INSTEAD OF triggers. In this case, the sequence would be as follows:

  1. The following statement is executed:
    UPDATE Employees SET lastname = 'papa' WHERE lastname = 'king'
  2. The inserted and deleted tables are created for use within the trigger.
  3. The INSTEAD OF trigger executes.
  4. Any subsequent actions executed may be kicked off by the logic within the INSTEAD OF trigger.

There can be an AFTER UPDATE trigger and an INSTEAD OF UPDATE trigger on the same table. In this case, the INSTEAD OF trigger would fire and if it executed an UPDATE statement on the same table, then the events would fire as noted previously in the AFTER trigger's sequence of events.

When planning how you'll enforce data integrity you should first perform any entity integrity checks with primary key and unique key constraints. Entity integrity checks ensure that unique rows exist within the entity (the table). Domain integrity checks should then be enforced through constraints such as CHECK constraints. Domain integrity ensures that a column has a valid value from a specific domain and makes sure it has a value at all if it is required to (are nulls allowed or not). For example, a column called sTrafficLightColor which is defined as a VARCHAR(6) might have a CHECK constraint on it that ensures that the value is either red, yellow, or green. Finally, referential integrity should be enforced via foreign key constraints if at all possible.

I'll reiterate that triggers are less efficient than foreign keys at enforcing referential integrity. However, there are some situations in which AFTER triggers can offer value in this area. One example is when a business rule exists that requires a complex referential integrity check where a value in a table must reference a value in one of two parent tables. For example, assume that there is a tblBankAccount table and tblBrokerageAccount table, both of which have a column called nAccountNumber. This nAccountNumber field is defined as a CHAR(10) in both tables and is each table's primary key.

Now, assume there is a third table called tblTransaction which contains a required field called nAccountNumber. Normally, a foreign key constraint would be placed on this field to its parent table, but in this case there are two parent tables. This type of complex referential integrity can be validated using an AFTER trigger that checks to make sure that any value inserted or updated into the tblTransaction.nAccountNumber column is either a valid account number from tblBankAccount or tblBrokerageAccount. A foreign key constraint can only match one or more columns between two tables, while an AFTER trigger, despite being less efficient than a foreign key, has more possibilities. In this case the slight performance reduction might be less important than the gain you realize by enforcing the business rule.

Another reason AFTER triggers can be useful for enforcing complex constraints is that they can reference several tables and columns whereas CHECK constraints can only access the current row in the current table. AFTER triggers can also cascade changes throughout other tables, but cascading referential integrity constraints are more efficient than triggers in this situation.

Special Tables

There are two very special tables that can exist only within triggers: the inserted and deleted tables. These tables are used to store the before and after state of the table that was affected by the INSERT, UPDATE, or DELETE statement that caused the trigger to fire in the first place. For example, if an AFTER UPDATE trigger is associated with the Products table it can be used to check if a product's price is being changed. If the price is being changed, the trigger could perform a variety of business rules such as making sure the price change is not exceeding a set percentage. If the price change exceeds a specific percentage, the change could be rolled back and an error message could be raised. Another scenario would be to allow the price change but to notify a supervisor when the change occurs. The trigger in Figure 1 demonstrates this technique.

Figure 1 Trigger that Notifies on Price Change (Close Figure 1)
CREATE TRIGGER tr_Products_U ON Products AFTER UPDATE
AS

    DECLARE @sMsg VARCHAR(200),
        @sProductName NVARCHAR(40),
        @mOldPrice MONEY,
        @mNewPrice MONEY

    IF UPDATE(UnitPrice)
    BEGIN
        SELECT     @sProductName = d.ProductName,
            @mOldPrice = d.UnitPrice,
            @mNewPrice = i.UnitPrice
        FROM    inserted i
            INNER JOIN deleted d ON i.ProductID = d.ProductID

        SET @sMsg = 'The price of ' + @sProductName
            + ' has changed from '
            + CONVERT(VARCHAR(10), @mOldPrice)
            + ' to ' + CONVERT(VARCHAR(10), @mNewPrice)
            + ' on ' + CONVERT(VARCHAR(30), getdate()) +  '.'

        EXECUTE prSendMail 'testX@microsoft.com', @sMsg,
            'price change notification', 'testY@microsoft.com'

    END

    RETURN

GO

The AFTER trigger, called tr_Products_U in Figure 1, shows the use of the UPDATE function as well as the inserted and deleted tables. The UPDATE function accepts a single parameter that represents a column in the table. The function returns a bit value to indicate whether the column's value has been modified; it's only available inside of triggers. The inserted and deleted tables are crucial elements of trigger logic and here they store the before and after picture of what happened to the Products table. For example, if the following UPDATE statement ran, the inserted and deleted tables would contain a single row representing ProductID #1.

UPDATE Products SET UnitPrice = UnitPrice - 1 WHERE ProductID = 1

Both the inserted and deleted tables would contain all of the columns that the Products table contains. As noted, both the inserted and deleted tables would contain a single row representing ProductID #1 but with one slight difference. The value for the UnitPrice column would be different in these tables, thus reflecting the price change. Fortunately, you do not have to compare these values to see if a column's value has changed; the UPDATE function does this for you.

In the scenario in which the price has changed, an e-mail message is formatted and sent using a custom stored procedure called prSendMail (see Figure 2). In the trigger tr_Products_U, the values for both the old UnitPrice and the new UnitPrice are gathered by joining the inserted and deleted tables and then retrieving the UnitPrice from each of the tables.

Figure 2 The prSendMail Stored Procedure (Close Figure 2)
CREATE PROCEDURE prSendMail
    @sTo VARCHAR(500), @sBody VARCHAR(8000), @sSubject VARCHAR(500),
    @sFrom VARCHAR(500)
AS

    DECLARE @nCDO INT,
        @nOLEResult INT, @nOutput INT, @sSource VARCHAR(255),
        @sDescription VARCHAR(255)

    EXECUTE @nOLEResult = sp_OACreate 'CDO.Message', @nCDO OUT
    IF @nOLEResult <> 0
    BEGIN
        EXEC sp_OAGetErrorInfo @nCDO, @sSource OUT, @sDescription OUT
        SELECT 'Error Creating CDO.Message Object',
            hResult=convert(varbinary(4),@nOLEResult),
            Source=@sSource,
            Description=@sDescription
        RETURN
    END

    EXECUTE @nOLEResult = sp_OASetProperty @nCDO, 'To', @sTo
    IF @nOLEResult <> 0
    BEGIN
        EXEC sp_OAGetErrorInfo @nCDO, @sSource OUT, @sDescription OUT
        SELECT 'Error Executing CDO.To Property',
            hResult=convert(varbinary(4),@nOLEResult),
            Source=@sSource,
            Description=@sDescription
        RETURN
    END

    EXECUTE @nOLEResult = sp_OASetProperty @nCDO, 'From', @sFrom
    IF @nOLEResult <> 0
    BEGIN
        EXEC sp_OAGetErrorInfo @nCDO, @sSource OUT, @sDescription OUT
        SELECT 'Error Executing CDO.From Property',
            hResult=convert(varbinary(4),@nOLEResult),
            Source=@sSource, Description=@sDescription
        RETURN
    END

    EXECUTE @nOLEResult = sp_OASetProperty @nCDO, 'Subject', @sSubject
    IF @nOLEResult <> 0
    BEGIN
        EXEC sp_OAGetErrorInfo @nCDO, @sSource OUT, @sDescription OUT
        SELECT 'Error Executing CDO.Subject Property',
            hResult=convert(varbinary(4),@nOLEResult),
            Source=@sSource,
            Description=@sDescription
        RETURN
    END

    EXECUTE @nOLEResult = sp_OASetProperty @nCDO, 'TextBody', @sBody
    IF @nOLEResult <> 0
    BEGIN
        EXEC sp_OAGetErrorInfo @nCDO, @sSource OUT, @sDescription OUT
        SELECT 'Error Executing CDO.TextBody Property',
            hResult=convert(varbinary(4),@nOLEResult),
            Source=@sSource,
            Description=@sDescription
        RETURN
    END

    --Call Send method of the object
    EXECUTE @nOLEResult = sp_OAMethod @nCDO, 'Send', Null
    IF @nOLEResult <> 0
    BEGIN
        EXEC sp_OAGetErrorInfo @nCDO, @sSource OUT, @sDescription OUT
        SELECT 'Error Executing CDO.Send Method',
            hResult=convert(varbinary(4),@nOLEResult),
            Source=@sSource, Description=@sDescription
        RETURN
    END

    --Destroy CDO
    EXECUTE @nOLEResult = sp_OADestroy @nCDO

    RETURN @nOLEResult

GO

In an AFTER UPDATE trigger where at least one row was updated, both the inserted and deleted tables will contain the same number of rows. In an AFTER DELETE trigger, only the deleted table will contain data since there are no new values. On the other hand, in an AFTER INSERT trigger, only the inserted table will contain data since no existing record was deleted or modified. These tables are often at the heart of many triggers. One drawback of the inserted and deleted tables is that columns defined as a binary datatype including text, ntext, and images will not be represented.

The stored procedure shown in Figure 2 uses the system stored procedures that allow a stored procedure to invoke COM objects including SQLDMO, ADODB, or the CDOSYS library. This allows the stored procedure to enhance its functionality by reusing existing COM objects. I created this stored procedure as an example of how to roll your own mail sending function from SQL Server without having to rely on the xp_sendmail stored procedure. That procedure has some drawbacks including that it is not available in a SQL Server clustered environment. (I will save this topic for a future column.)

Performance

I often get asked about the performance of triggers and how to tune them. Actually, triggers themselves do not carry much overhead and, in general, are quite responsive. Most performance issues can be primarily attributed to the logic contained within the triggers. For example, if a trigger creates a cursor and then loops through hundreds of rows, you could expect a slowdown. If a trigger runs several SQL statements against other tables outside of the inserted and deleted tables, you should also expect a slowdown proportionate to the speed of the SQL statements contained within the trigger. A good rule is to keep the logic contained within a trigger simple and avoid using cursors, executing SQL statements against other tables, and other tasks that normally cause performance hits. SQL is optimized for rowset logic and as such the use of cursors should be avoided when it is possible to replace that logic with a SQL statement (even if it appears to be more complex).

The code within a trigger is now compiled and stored, just like a stored procedure or standard SQL batch. Because the trigger's code is repeated every time the action query on its table is executed, the trigger runs faster on subsequent executions, just like a stored procedure. This is because SQL Server stores the source for the trigger, compiles the trigger into an execution plan, and stores it in memory. This memory is allocated by SQL Server and is managed using a first in, first out plan that helps keep the most frequently run stored procedures, triggers, and other SQL batches running smoothly.

It is generally not a good idea to return results from a trigger. This can cause performance hits and could cause any applications that execute queries against the table to be adversely affected if they are not expecting a resultset to be returned from a trigger. For example, if an AFTER UPDATE trigger executes a SELECT statement that returns 100 rows, any calling application that updates that table will get back a 100-row rowset that they most likely did not expect. Every once in a while, though, returning a rowset from a trigger is worth considering. One example is when you're returning the IDENTITY value from an AFTER INSERT trigger. For more information on this scenario, see the July 2002 installment of Data Points.

To prevent rowsets from being returned from triggers by accident, do not include SELECT statements or variable assignments. If variable assignments are made, you could first execute the SET NOCOUNT ON statement to eliminate the return of a resultset.

Until Next Time

This month's column walked through the foundation of triggers and explained several applications of AFTER triggers. I also discussed the firing sequence of the two trigger types since it is crucial to understanding how the types operate. However, there are many other features of triggers I will explore in upcoming Data Points columns such as nested transactions, the auditing of data, cascading referential integrity, preparing triggers to handle changes to multiple rows in a table, and the use of the COLUMNS_UPDATED function. INSTEAD OF triggers are a great feature that allow you to perform complex action queries in place of a single action query on a table or a view. This column showed triggers that were written to handle scenarios in which a single row was affected.

Send your questions and comments for John to mmdata@microsoft.com.

John Papa is a baseball fanatic who spends most of his summer nights rooting for the Yankees with his two little girls, wife, and faithful dog, Kadi. He has authored several books on ADO, XML, and SQL Server and can often be found speaking at industry conferences such as VSLive. You can reach him at mmdata@microsoft.com.


Cutting Edge: Custom Design-time Control Features in Visual Studio .NET

Cutting Edge
Custom Design-time Control Features in Visual Studio .NET
Dino Esposito

Code download available at: CuttingEdge0312.exe (165KB)
Contents
The Design-time Architecture
Writing a Sample Control
Choosing the Proper Set of Attributes
Type Editors
Type Converters
Designer Verbs and Markup
ASP.NET Control Designers
Conclusion

In the Microsoft® .NET Framework, Windows® Forms and Web Forms controls have two distinct sets of functionality. Run-time capabilities dictate the behavior of the control when it is employed in production code and design-time capabilities let the programmer decide how the control will appear in a WYSIWYG environment and expose its programming interface. Both built-in and custom Windows Forms and ASP.NET controls can be architected to integrate with the Visual Studio® .NET designer. With a proper set of design-time features, a control can provide a realistic preview of its UI, offer a more user-friendly programming style, group properties by categories, and supply tips about them. In addition, a special design-time feature lets you add custom functions to the context menu of the control when it's placed on a form or a Web page created with Visual Studio .NET.

Both Windows Forms and Web Forms controls inherit some basic design-time capabilities from their base class—Control. Note that they both inherit from a base class called Control but the namespace of each is different—System.Windows.Forms versus System.Web.UI. The Control base class gives derived controls drag and drop capabilities and lets them interact with the Properties window in Visual Studio .NET. As a result, a control can be dragged and dropped onto a form. Furthermore, all controls, when selected within the designer, display their properties in the Properties window, making them editable.

The design-time behavior of a control can be enhanced by setting predefined attributes, by defining custom editors for properties, and by creating custom designers. In this column, I'll show how these types of design-time extensions can significantly improve the user experience with a sample Windows Forms control. Note that while both Control classes (from which all controls inherit) already provide a base set of capabilities that make controls usable within a visual designer, they don't necessarily provide the kind of rich user experience you're looking for.

Let's start with a quick overview of the overall design-time architecture of .NET Framework controls.

The Design-time Architecture

A typical design-time environment such as Visual Studio .NET contains at least a form designer and a property browser. In addition, a design-time environment will usually provide services that components access to customize their behavior at design time. These services include a set of functions that allow you to plug into the designer's internal mechanism. Components that can connect to the services exposed by the design-time environment are said to be designable components.

Figure 1 Control Inheritance
Figure 1 Control Inheritance

In the .NET Framework, a designable component is simply a class that implements the IComponent interface. The hierarchy shown in Figure 1 shows that all controls—both Windows Forms and Web Forms—are designable components because all derive from classes that implement the IComponent interface. The IComponent interface has just one property, Site, which corresponds to an object that implements the ISite interface. The ISite interface defines four properties: Component, Container, DesignMode, and Name. The role of each property is fully described in Figure 2.

Figure 2 ISite Properties (Close Figure 2)

Property | Description
Component | Gets the component associated with the class that implements the ISite interface. In a design-time control scenario, it represents the instance of the control being designed.
Container | Gets the IContainer object associated with the class that implements the ISite interface. In a design-time control scenario, it represents the Visual Studio .NET component that hosts the designer. In Visual Studio .NET 2003, the class name is Microsoft.VisualStudio.Designer.Host. DesignerHost.
DesignMode | Indicates whether the component is in design mode. Returns true.
Name | Returns the name of the component associated with the site. In a design-time control scenario, it is the name of the control being designed. Name is a read/write property.

The Site object binds a component to a container and enables communication between them. In addition, it provides a way for the container to manage its components and acts as a repository for container and component-specific information, such as the component name and attributes. The following code snippet shows how to use the Site object to detect whether the current control is being used at run time or design time:

if (this.Site != null && this.Site.DesignMode)
{
    string buf = "'{0}' is hosted by {1}";
    MessageBox.Show(String.Format(buf, this.Site.Name,
    this.Site.Container.ToString()));
}
else
    MessageBox.Show("The control works in a run-time context.");

Windows Forms controls derive from the Component class (defined in the System.ComponentModel namespace), which implements the IComponent interface. Web Forms controls, in contrast, have the IComponent interface implemented directly by the Control class (defined in the System.Web.UI namespace). The IComponent interface extends the IDisposable interface and implements the Dispose method. The Dispose method is available to components to release external resources explicitly. You should note that the Dispose method represents a deterministic way to free resources, unlike the default nondeterministic cleanup that is performed by the garbage collector.

The Visual Studio .NET design-time architecture requires that the design-time capabilities of a control not be implemented within the control itself but in separate classes. Design-time classes and controls are then linked declaratively using attributes. Separating run-time and design-time code has two main advantages. First, the code that creates the control has a smaller footprint, making it more manageable at run time. Second, insulating design-time functionality in external classes results in a much more flexible and extensible architecture, as you'll see in a moment. The various types of design-time functionality are summarized in Figure 3.

Figure 3 Design-time Functionality (Close Figure 3)

Extension | Description
Attribute | A bunch of predefined attributes let you instruct the host environment about the description to show for a property, or the category under which the property should be displayed.
Designer | A class that takes care of the appearance and behavior of the control when hosted in Visual Studio .NET. You can use this class to add new context menu verbs and, for ASP.NET controls, to explicitly indicate the HTML code to show in the designer.
Type converter | A class that performs conversions to represent values of arbitrary types as strings. Used by the Properties window.
UI type editor | A class that provides a convenient way of editing the values of a certain type. For example, a UI type editor is used to select a color from a palette.

You can extend the base design-time support for controls in various ways, the simplest of which is mostly declarative and consists of setting a few metadata attributes such as Browsable, Category, and Description. Achieving more advanced design-time support requires additional work and new classes. To do this, you need to import a few namespaces, including System.ComponentModel.

Writing a Sample Control

To see the design-time functionality of the .NET Framework in action, let's begin by building a custom Windows Forms control that contains many of the features that justify the extensive use of design-time features. The user control I'm going to write will be named FileChooser and will consist of Label, Textbox, and Button controls. The purpose of the component is to let users select a file name. The file name can be typed directly in the Textbox or the button can be clicked to show the standard File Open dialog to navigate to a file. The Label control gives the Textbox a caption. When it runs, the sample app shown in the designer in Figure 4 launches an "Open" dialog.

Figure 4 Sample App in Visual Studio .NET Designer (Close Figure 4)

The control is built around an internal method named RenderControl (not to be confused with the similarly named method of ASP.NET controls). The method takes care of moving the constituent controls around to make them fit into the assigned area—the area reserved for the control on the hosting form. Page and form developers decide the location (site) of each embedded control by resizing the control through drag and drop or by setting the width and height properties in the Properties window.

RenderControl ensures that the button has a fixed width and text (the ellipsis). The Textbox is the size of the site minus the button width. The label can be hidden or displayed above or below the Textbox. A specific property indicates the correct location. Font, alignment, and text of the label are grouped in a custom class—ChooserCaption—exposed through a Caption property. The name of the selected file is returned through a string property named FileName. Finally, a Boolean ReadOnly property restricts the control to accepting the name of the selected file only from the File Open common dialog. In this case, the Textbox control is marked as read-only. Figure 5 lists the properties supported by the control. The characteristics of those properties require different types of design-time features.

Figure 5 Sample Control Properties (Close Figure 5)

Property | Description | Design-time Extension
Caption | Property of type ChooserCaption that represents graphical attributes such as the text, font, and alignment of the label. | Type converter
CaptionLocation | Custom enum type that indicates the location of the label with respect to the Textbox. | Attribute
FileName | Gets and sets the name of the selected file. | UI type editor
ReadOnly | Indicates whether the Textbox is read-only. In this case, the file can be selected only through the File dialog box. | Attribute
TextBackColor | Indicates the background color of the Textbox. | Attribute
TextForeColor | Indicates the foreground color of the Textbox. | Attribute

Choosing the Proper Set of Attributes

By default, all public control properties implemented with an explicit pair of get/set accessors show up in the Visual Studio .NET Properties window. The following example demonstrates the implementation of the TextForeColor property. As you can see, the property simply mirrors the ForeColor property of the constituent Textbox (see Figure 6).

Figure 6
public Color TextForeColor
{
    get {return theFileName.ForeColor;}
    set {theFileName.ForeColor = value;}
}

The way in which properties appear in the Properties window depends on the attributes declaratively assigned to the properties in the control's source code. In particular, the Browsable attribute determines whether or not a control is displayed. The most commonly used attributes are listed in Figure 7. The following attributes give the property a description and bind it to the Appearance category of the Properties window. The Browsable attribute is not strictly necessary as all properties with get/set accessors show up by default. Use the Browsable(false) attribute only to prevent a property from being displayed.

Figure 7 Commonly Used Attributes (Close Figure 7)

Attribute | Description
Bindable(bool) | Indicates whether a property can be bound to data.
Browsable(bool) | Indicates whether a property or an event should be displayed in the Properties window.
Category(name) | Provides the name of the category in which a property or an event belongs. This allows for logical grouping of properties and events.
DefaultValue(value) | Used to provide a default value for a control property. If the property doesn't have a simple type, a type converter is needed.
Description(text) | Provides a description of a property or an event. The text is displayed at the bottom of the Properties window when the user selects a property or event.
Editor(type) | Specifies the editor to use for modifying a property in a visual designer.
Localizable(bool) | Indicates that a property can be localized. Any properties that have this attribute are automatically persisted into the resources file when a user chooses to localize a form.
ParenthesizePropertyName(bool) | Indicates that a property should be displayed with parentheses around its name.
Figure 8 Properties Windows
Figure 8 Properties Windows

Figure 8 shows the effect of the following attributes.

[Browsable(true)]
[Category("Appearance")]
[Description("Gets and sets the foreground color of the textbox")]
The TextForeColor property falls under the Appearance category when the categorized view is selected in the Properties window. Any text that is associated with the Description attribute is displayed in the bottom-most area of the window when the property is selected.

The attributes in Figure 7 characterize control properties. A few specific attributes exist for the control class, too. They are DefaultProperty and DefaultEvent, as shown here:

[DefaultProperty("FileName")]
[DefaultEvent("FileSelected")]
public class FileChooser : System.Windows.Forms.UserControl
{ ... }

The DefaultProperty attribute indicates the default property of the control. When an instance of the control is selected in the designer, the specified property is automatically selected and becomes visible in the Properties window. The properties grid scrolls up enough to show the default property in the current view. The DefaultEvent attribute denotes the default event (if any) defined for the control. A handler for this event is automatically added to the code-behind class of the host form or page whenever you double-click on the control in the designer. For example, the default event for a form is Load; Click for a Button; and so on. In addition, Web server controls also feature another class-level attribute—ToolboxData.

[ToolboxData("<{0}:MyDiv runat=server>
   </{0}:MyDiv>")]
public class MyDiv : WebControl
{ ... }
The attribute indicates the name of the tag generated for a custom control when it is dragged from a toolbox in the Visual Studio .NET designer. Based on the previous code snippet, the following markup is added to the underlying host Web page:
<msdn:MyDiv runat=server></msdn:MyDiv>

The {0} placeholder in the code previous to this one references the value of the TagPrefix attribute in the @Register directive used to link the custom Web control to the page. While defining the ToolboxData attribute you can add as many HTML attributes as needed to the output markup.

In Figure 8, you can see a dialog box just below the name of the property in the Properties window. The content of the dialog box lets you pick a color to assign as the new property value. In other words, the dialog box acts as a property-specific user interface to edit the value. A few types in the .NET Framework have a custom user interface editor, notably the Color, Font, Image, Boolean, and enum types. Whenever a property of any of these types shows up in the Properties window, Visual Studio .NET automatically binds it to a default, built-in editor. For example, properties of type Color are edited through the services of the ColorEditor class whereas font values are edited using FontEditor.

Booleans and values that belong to an enumeration get special treatment from the property grid and are rendered using a combobox with all possible values. What each editor does depends on the characteristics of the type it represents. For example, ImageEditor provides a UI for selecting an image file. Starting the editor displays an open file dialog box initially filtered to select all image files. In all these cases, you don't need to do anything to get the aforementioned services. By setting the Editor attribute on the property, though, you can force an editor-to-property binding.

Type Editors

All type editors inherit from the base class UITypeEditor. A type editor provides a UI to edit a property in a more user-friendly way. The .NET Framework comes with a handful of UI type editors but others can be created using the UITypeEditor class. The FileChooser control has Color and Boolean properties for which Visual Studio .NET automatically selects an appropriate UI editor. The FileName property is a string that represents a file name; users can enter the value by either manually typing in the property grid or using the associated editor. The .NET Framework even provides an editor for properties that represent a file name—the FileNameEditor class in the Sys-tem.Windows.Forms.Design namespace.

[Browsable(true)]
[Description("...")]
[Editor(typeof(
        System.Windows.Forms.Design.FileNameEditor),
        typeof(UITypeEditor))]
public string FileName
{ ... }

However, I'll pretend I don't know about it and create a clone of the UI editor programmatically. I'll call it the MyControls.FileNameEditor class.

Insulating the design-time classes in a separate assembly is a good programming practice, but at a minimum write your design-time code in a distinct file within the project. If a single assembly is used, you might want to mark the design-time classes as internal so that they won't be callable from outside the control's assembly. On the other hand, if you use a separate assembly, design-time classes must be public.

A UI type editor class is a class that inherits from UITypeEditor and overrides at least the GetEditStyle and EditValue methods. Figure 9 illustrates the full source code of the custom FileNameEditor class. A UI type editor is basically a dialog box whose controls let you define the new value for the particular property. The GetEditStyle method indicates the style of the requested user interface—modal or dropdown. The values are grouped in an enum type named UITypeEditorEditStyle. Modal displays a modal dialog box; DropDown displays a dialog box that is dismissed as soon as the window loses focus (like the list of a combobox). If you opt for modal, the property is characterized by an ellipsis button in the property grid; otherwise, a combobox-like button appears. No matter what the button looks like, clicking it always invokes the EditValue method.

Figure 9 Custom FileNameEditor Class (Close Figure 9)
internal class FileNameEditor : UITypeEditor
{
    public override UITypeEditorEditStyle
GetEditStyle(ITypeDescriptorContext context)
    {
        return UITypeEditorEditStyle.Modal;
    }

    public override object EditValue(ITypeDescriptorContext
        context, IServiceProvider provider, object value)
    {
        // value is the value in the PropertyBox
        FileChooser parent = (FileChooser) context.Instance;

        // PickUpFile is a public method on the control that
        // invokes the File Open common dialog
        string fileName = parent.PickUpFile();

        if (fileName == String.Empty)
            return value;

        return fileName;
    }
}

The EditValue method is responsible for popping up the dialog box and for any operation necessary to get and store the new value. Typically, EditValue displays the dialog, captures the new value, and returns it. Should you need to invoke a method on the designable control, the context parameter provides a reference to it. The current value of the property is passed to EditValue through the value parameter (see Figure 9).

Another feature that UI type editors may support is manual painting of the value. The GetPaintValueSupported virtual method indicates whether this editor supports painting a representation of an object's value. Both the ImageEditor class and the ColorEditor class provide a graphical representation of the value—a little icon for ImageEditor and a rectangle filled with the chosen color for ColorEditor. If you intend to do the same in your control, start by overriding GetPaintValueSupported and make it return true. Next, override PaintValue and provide your own painting code.

Type Converters

What if you want your control to contain a property of a custom type? At first, this task doesn't look particularly difficult. You create a UI type editor, design the dialog box to fit all the properties, and return an instance of the type from EditValue. Type editors, though, have been introduced for simple types such as strings, Booleans, enum values, and colors. The values you assign to the various properties of a custom type are to be serialized when the form (or page) class is persisted to disk. This is not a problem for simple types, but what about custom types? Class serialization could be the answer but Visual Studio .NET supports a better, lightweight alternative—type converters.

A type converter is a class that converts a custom type to and from a string. To write a type converter, derive a class from TypeConverter and override the ConvertFrom and ConvertTo methods. ConvertFrom parses the specified string and converts it to an instance of the target type. The latter does the reverse and serializes an instance of a type to a string. Let's see how it works for the Caption property implemented through the ChooserCaption class. The class features three properties, as shown in Figure 10.

Figure 10 ChooserCaption Class (Close Figure 10)
[TypeConverter(typeof(ChooserCaptionConverter))]
public class ChooserCaption
{
   public ChooserCaption() {
      CaptionText = "File";
      FontBold = true;
      Alignment = ContentAlignment.TopLeft;
   }

   [Description("...")]
   public string CaptionText
   { ... }

   [Description("...")]
   public bool FontBold
   { ... }

   [Description("...")]
   public ContentAlignment Alignment
   { ... }
}

The string value that represents a custom type is a string of comma-separated values, each of which represents a property. The type converter decides the order, number, and text of the serialized values. Figure 11 shows the full source code of the sample type converter for the Caption property. The ConvertFrom method receives a string of comma-separated values and splits its contents. Each part is then used to set the corresponding property of the custom class. The following lines of code show how to set the CaptionText property.

ChooserCaption caption = new ChooserCaption();
string serializedData = (string) value;
string[] parts = serializedData.Split(',');
caption.CaptionText = parts[0];

Figure 11 Type Converter (Close Figure 11)
internal class ChooserCaptionConverter : ExpandableObjectConverter
{
    public ChooserCaptionConverter()
    {
    }

    // ****************************************************
    // Convert the specified text sequence to a ChooserCaption
    // instance
    public override object ConvertFrom(ITypeDescriptorContext
        context,
        System.Globalization.CultureInfo culture, object value)
    {
        // Default instance
        if (value == null)
            return new ChooserCaption();

        // Deserialize to ChooserCaption from a string
        // representation of the type
        if (value is string)
        {
            // Get the source string
            string serializedData = (string) value;

            if (serializedData.Length == 0)
                return new ChooserCaption();

            // *********************************************
            // The string representation is a collection of 3
            // strings:
            //     + text
            //     + bold (Boolean)
            //     + alignment (ContentAlignment)
            // *********************************************

            // Split the string into parts (must be 3 parts—
            // one per each property)
            string[] parts =
                serializedData.Split (
                    culture.TextInfo.ListSeparator[0]);
            if (parts.Length != 3)
                throw new ArgumentException(
                    "Invalid ChooserCaption object", "value");

            // Create a new ChooserCaption object
            ChooserCaption caption = new ChooserCaption();

            // Populate the newly created object with the
            // deserialized data

            // Part 1 is a string representing the Text
            // property
            caption.CaptionText = parts[0];

            // Part 2 is a Boolean value representing the bold
            // attribute
            TypeConverter boolConv =
                TypeDescriptor.GetConverter(typeof(bool));
            caption.FontBold = (bool)
                boolConv.ConvertFromString(context, culture,
                    parts[1]);

            // Part 3 is a string representing a
            // ContentAlignment value
            string align = parts[3];
                    caption.Alignment = (ContentAlignment)
                        Enum.Parse(typeof(ContentAlignment),
                            align);

            return caption;
        }

        return base.ConvertFrom (context, culture, value);
    }
    // *****************************************************

    // *****************************************************
    // Convert a ChooserCaption to a string
    public override object ConvertTo(ITypeDescriptorContext
        context, System.Globalization.CultureInfo culture,
        object value, Type destinationType)
    {
        // Ensure the value is a ChooserCaption object
        if (value != null)
        {
            if (!(value is ChooserCaption))
                throw new ArgumentException(
                    "Invalid ChooserCaption object", "value");
        }

        // If null, return the empty string
        if (destinationType == typeof(string))
        {
            if (value == null)
                return String.Empty;

            // Get the object to serialize and necessary
            // converters
            ChooserCaption caption = (ChooserCaption) value;

            // Create the string
            string[] parts = new string[3];
            parts[0] = caption.CaptionText;
            parts[1] = caption.FontBold.ToString();
            parts[2] = caption.Alignment.ToString();
            return String.Join(culture.TextInfo.ListSeparator,
                parts);
        }

        return base.ConvertTo(context, culture, value,
            destinationType);
    }
    // *****************************************************
}

The second segment in the serialized string represents a Boolean value. That string is first converted to a Boolean and then assigned to the FontBold property. Similarly, the text representation of the ContentAlignment value is converted to the proper type and then assigned. The final instance of the ChooserCaption class is then returned by the method.

The ConvertTo method takes the current instance of the ChooserCaption class and serializes its content to a string of comma-separated values:

ChooserCaption caption = (ChooserCaption) value;
string[] parts = new string[3];
parts[0] = caption.CaptionText;
parts[1] = caption.FontBold.ToString();
parts[2] = caption.Alignment.ToString();
return String.Join(culture.TextInfo.ListSeparator, parts);

If you inherit your class from TypeConverter, the property appears in the Properties window as an editable string of comma-separated values. If you use ExpandableObjectConverter as the base class, the Caption property becomes an expandable object (like the default Font property) and once expanded shows all the child attributes serialized to the type converter string. You can edit values either as a string of comma-separated values (not recommended) or individually by editing each single property.

When defining a class used for a design-time property, you normally use the TypeConverter attribute along with the DesignerSerializationVisibility serialization attribute:

[DesignerSerializationVisibility(DesignerSerializationVisibility.Content)]
[TypeConverter(typeof(ChooserCaptionConverter))]
public class ChooserCaption
{ ... }

This attribute is typically set to Content. It indicates the type of persistence to use when serializing the property at design time. Content indicates that each public, non-hidden sub-property of the object will be serialized.

Designer Verbs and Markup

A third type of extension is a custom designer. A designer is a class that provides all the logic necessary to interact with the host environment and lets users build the appearance and behavior of a component at design time. All designers implement the IDesigner interface. The .NET Framework supplies a set of designers for specific types of components and controls. By writing a custom designer, you can add custom initialization code for a component in design mode. You could also configure and create components within a project and, quite interestingly, add menu items to the context menu of a control. Have you ever noticed the AutoFormat link button that magically appears in the Properties window when a DataGrid control is selected in the designer? If you want to add a similar feature to your controls, be prepared to write a custom designer. A custom designer is declaratively bound to the control using the Designer attribute:

[Designer(typeof(FileChooserDesigner))]
public class FileChooser : System.Windows.Forms.UserControl
{ ... }

The FileChooserDesigner class inherits ControlDesigner, one of the built-in designer classes in the .NET Framework. It provides all the logic needed to manage controls in Visual Studio .NET. If you want to add new verbs to the control, you override the Verbs property. A verb is an action that can be performed at design time on an instance of the control. Verbs appear in two places—in the context menu and in the Properties window just below the property grid. In the property grid, verbs are rendered as link buttons and clicking them causes an event handler to run. The Verbs property is a collection of DesignerVerb objects and is empty by default. The override in Figure 12 adds a couple of verbs to the FileChooser control—"Auto Format" and "Something Else".

Figure 12 Override Verbs (Close Figure 12)
public override DesignerVerbCollection Verbs
{
   get {
      if (m_Verbs == null)
      {
         // Create and initialize the collection of verbs
         m_Verbs = new DesignerVerbCollection();
         m_Verbs.Add( new DesignerVerb("Auto Format",
             new EventHandler(OnAutoFormatSelected)) );
         m_Verbs.Add( new DesignerVerb("Something Else",
             new EventHandler(OnSomethingElseSelected)) );
      }
        return m_Verbs;
   }
}

A designer verb is characterized by text (the hyperlink) and an event handler. The event handler is invoked when the user clicks the verb on the context menu or the property grid. If you look back to Figure 4, you can see the new verbs and the dialog box displayed by the AutoFormat verb. The same two verbs are added to the context menu that is displayed when you right-click on the FileChooser control in Visual Studio .NET. The effect for the code is the same—the same event handler is always invoked.

The event handler displays a dialog box, captures a few new values, and reflects them to the control being designed. The code in Figure 13 shows the behavior of the event handler associated with the AutoFormat verb.

Figure 13 AutoFormat Event Handler (Close Figure 13)
void OnAutoFormatSelected(object sender, EventArgs args)
{
   // Display a message
   AutoFormatDialog dlg = new AutoFormatDialog();
   DialogResult result = dlg.ShowDialog();
   if (result == DialogResult.OK)
   {
      // Update the control being designed with new values
      FileChooser ctl = (FileChooser) this.Control;
      ctl.ReadOnly = dlg.ReadOnly;
      ctl.CaptionLocation = dlg.CaptionLocation;
      ctl.Caption.Alignment = dlg.TextAlign;
   }
}

As you can see, a few public properties have been defined on the AutoFormatDialog class—the class that represents the popup dialog—to expose the values to map on the FileChooser instance under design. The AutoFormat dialog box mimics the layout of the similar dialog defined for DataGrid controls. What's the easiest way of providing a preview of the control's appearance? That's easy. You just use another instance of the same FileChooser control and set its properties as suggested by the selected format.

For the icing on the cake, let's spend a few minutes looking at an interesting feature—designer transactions. A designer transaction groups a series of design-time actions to enable changes to be undone. In addition, when multiple actions have been performed, transactions improve performance by deferring updates to the display until the completion of the transaction. To perform a transaction within your designer, call CreateTransaction on the designer host object. You get a reference to the host by calling GetService—a protected member inherited from ComponentDesigner:

IDesignerHost host = (IDesignerHost) GetService(typeof(IDesignerHost));
DesignerTransaction t = host.CreateTransaction();

CreateTransaction returns a DesignerTransaction object that can be used to control the transaction, commit it, or roll it back. All the transacted actions should be executed within a try block. To complete the transaction, call Commit from within a finally block. In C#, you can use the using statement rather than try...finally. The using statement defines a scope at the end of which the transaction object will be disposed:

using (host.CreateTransaction())
{
   // actions go here
}

To cancel and attempt to roll back a transaction before it has been committed, call the Cancel method. To undo actions which occurred as part of earlier transactions, you must use the undo command provided by the Visual Studio .NET IDE.

ASP.NET Control Designers

What I've discussed so far (attributes, type editors and converters, and verbs) applies to Windows Forms and Web Forms controls as well. Web Forms controls have a different set of predefined type editors but that's all; there's no other significant difference between the two. ASP.NET designers, though, have a larger set of methods and features. In particular, you can use a different rendering algorithm for design time and run time, and even disable resizing for controls when they're hosted in a Web page designer. The System.Web.ControlDesigner class provides a few virtual methods that relate to the markup code shown in the designer. The list includes methods such as GetDesignTimeHtml, GetEmptyDesignTimeHtml, and GetErrorDesignTimeHtml. GetDesignTimeHtml gets the HTML that is used to represent the control at design time. The other two methods get the HTML that is used to represent an empty control and the exception that occurred at design time. Resorting to these methods makes sense, especially when you're designing a data-bound control and need to provide sample data. For example, when its AutoGenerateColumns is true, the DataGrid control displays a number of columns with sample data. For an example of custom design-time HTML for ASP.NET controls, see my book Programming Microsoft ASP.NET (Microsoft Press®, 2003). The source code for the control described in the book is available on the Microsoft Press Web site.

Conclusion

If Visual Studio .NET is your development tool of choice, then chances are good that you'll find yourself assembling forms or pages using a visual designer. Once you choose the controls from a palette, drop them onto a design surface, and instantiate them, they can be configured declaratively through the Properties window. Built-in controls provide advanced features when hosted in Visual Studio .NET that make for a rich user experience. This is not always the case for custom controls. Both Windows Forms and ASP.NET custom controls need additional code to integrate well with Visual Studio .NET. In this column, I've discussed the few things you must know in order to build effective design-time features. This month's download, which can be found on the MSDN® Magazine Web site, contains the code for the FileChooser Windows Forms control (see the link at the top of this article).

Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is an instructor and consultant based in Rome, Italy. He is the author of Building Web Solutions with ASP.NET and ADO.NET and Applied XML Programming for .NET, both from Microsoft Press. Dino recently published Programming ASP.NET for Microsoft Press. Get in touch with him at cutting@microsoft.com.


The XML Files: A Survey of Publicly Available Web Services at Microsoft

The XML Files
A Survey of Publicly Available Web Services at Microsoft
Aaron Skonnard

Code download available at: XMLFiles0312.exe (284KB)
Contents
Microsoft.com Web Service 1.0
TerraService
MapPoint
.NET Alerts
The State of Web Services

Now that the dust has settled on the XML Web Services front, it seems like the right time to survey some of the mainstream Web Services available, with a focus on those offered by Microsoft (see Live Web Services). Microsoft offers several Web Services to the general public including the Microsoft.com Web Service, TerraService, MapPoint® Web Service, and .NET Alerts. Developers can use all of these services to introduce valuable functionality and content into their applications. This column will provide an overview of these Web Services along with instructions on how to get started with each one.

Microsoft.com Web Service 1.0

Microsoft.com Web Service 1.0 is the newest addition to the Microsoft repertoire of XML Web Services. This service makes it possible to programmatically obtain information about downloads available on the various Microsoft.com Web sites including MSDN®, TechNet, Microsoft Support, and others. Microsoft.com Web Service 1.0 allows developers to retrieve a summary of the most recent or most popular downloads, as well as complete details for any particular download.

This version of the service is not a supported Microsoft product, but rather a vehicle for validating the underlying architecture of the Web Service. Future versions will build on this architecture to provide richer content and functionality. Microsoft hopes that developers experimenting with version 1.0 of the Microsoft.com Web service will provide plenty of feedback to help shape future releases. A newsgroup has been created for this purpose at microsoft.public.mscom.webservices.general.

To start using the Web Service, you must first get a developer's token by browsing to https://ws.microsoft.com/MsComWs/apply.aspx and filling out the form. Interestingly, you don't have to provide any personal information like name, e-mail, or phone to get a token. You simply provide a password (also referred to as a PIN), accept a user agreement, verify the text in an image, and you get a token (something like CqK7XYuJVc0CsXH3Qinukvwzus1KnZUP) that you supply every time you interact with the service.

It's important to note that requiring a token doesn't imply a secure service. The service does not implement security today, instead packaging the token in the SOAP Header according to the WS-Security specification. The token allows Microsoft to track usage and regulate access, and nothing more. For example, Microsoft limits each token to 10 requests per second. If you exceed this limit, your token will become temporarily disabled. Also, since the service was provided primarily for evaluation purposes, the tokens automatically expire 120 days after issuance. To check the status of your token, you can use the form found at https://ws.microsoft.com/MsComWs/account.aspx.

Once you have a developer's token, you can begin using the various operations exposed by the service (see Figure 1 for details). You can call GetCultures to access a list of cultures supported by the GetTopDownloads and GetDownloadDetail operations. GetTopDownloads returns a summary of the current top downloads, sorted either by most recent or most popular. Once you've identified a download of interest, you can use the GetDownloadDetail operation to retrieve the details of the particular download. The last operation, GetVersion, simply returns the descriptive name and the related version number of the current version of the Microsoft.com Web Service.

Figure 1 Microsoft.com Web Service 1.0 Operations (Close Figure 1)

Operation | Description
GetCultures | Returns a list of cultures supported by GetDownloadDetail and GetTopDownloads
GetDownloadDetail | Returns detail information for a selected download
GetTopDownloads | Returns summary information for the top downloads, either most recent or most popular
GetVersion | Returns the descriptive name and the related version number of the current version of the Microsoft.com Web Service

You can browse to the basic documentation page for the service at http://ws.microsoft.com/mscomservice/mscom.asmx and access the official Web Services Description Language (WSDL) document at http://ws.microsoft.com/mscomservice/mscom.asmx?wsdl. Microsoft provides an SDK that includes help files integrated with both Visual Studio® .NET 2002 and Visual Studio .NET 2003. After you install the SDK, you'll find the help files in the Visual Studio .NET Help | Contents window. The SDK also contains various sample applications (in C#, Visual Basic® .NET, J#, and C++) that illustrate how the Web Service may be used in both Web Forms and Windows® Forms applications. The SDK contains helpful figures including some forms that illustrate the typical flow of using the service.

To start using this Web Service, all you need to do is reference the WSDL file and generate classes for accessing the various operations. If you have the Microsoft® .NET Framework installed, you can do this by either running wsdl.exe on the command line or by using the Add Web Reference command in Visual Studio .NET. For example, this is how you use the command-line version:

c:> wsdl.exe http://ws.microsoft.com/mscomservice/mscom.asmx?wsdl
Writing file 'c:\temp\mscomservice.cs'

Once you reference the WSDL file, mscomservice.cs contains all the classes necessary to interact with the Web Service. There will be one class that contains all of the operations modeled as methods—in this case it's called MsComServices—along with a bunch of other classes that model the different types of information exposed by the service like Downloads, Download, DownloadSummary, DownloadRelease, Culture, Cultures, and so forth. The complete object model generated from the WSDL document can be seen at Microsoft.com Web Services Object Model.

The hardest part about using the Web Service is sending your PIN and developer's token along with each request. These bits of information are not sent as parameters to the operations. Instead you're expected to send them within the WS-Security header using a UsernameToken structure, as shown in Figure 2.

Figure 2 Sending Your PIN and Token (Close Figure 2)
<soap:Envelope xmlns:soap=...>
<soap:Header>
<wsse:Security soap:mustUnderstand="1"
  xmlns:wsse="http://schemas.xmlsoap.org/ws/2002/07/secext"
>
  <wsse:UsernameToken
    xmlns:wsu="http://schemas.xmlsoap.org/ws/2002/07/utility"
    wsu:Id="SecurityToken-9cda5cb3-f5c6-4ddf-b9fa-30ff0bcc97f5">
    <wsse:Username>PZh3EOz8/GWV+PQel2OaZtrNv2x3/dBk</wsse:Username>
    <wsse:Password Type="wsse:PasswordDigest"
>
    aDmrV1Ek7pHKnFGiuc/WLQJMoiY=</wsse:Password>
    <wsse:Nonce>rXl+0wUmcdMNbeMFmqGDiw==</wsse:Nonce>
    <wsu:Created>2003-07-23T19:51:24Z</wsu:Created>
  </wsse:UsernameToken>
</wsse:Security>
•••

You supply your developer's token in the Username element and a digest of your password in the Password element. You must send this header with each message sent to the service. The easiest way to generate this header is to use Web Services Enhancements (WSE), available for download from http://msdn.microsoft.com/webservices.

With WSE, you simply create a UsernameToken object using your token and password and then add it to the security headers collection before invoking any of the operations. If you're not using WSE, you'll have to manually construct the appropriate security header using the available APIs, which is much more tedious. The code in Figure 3 creates the header using WSE.

Figure 3 Security Header Using WSE (Close Figure 3)
// read token and password from config file
string Token = ConfigurationSettings.AppSettings["MsComServicesToken"];
string Pin = ConfigurationSettings.AppSettings["MsComServicesPin"];

// Initialize the Web Service and add the credentials to it.
MsCom_ws = new MsComServices();

UsernameToken ut = new UsernameToken(Token, Pin,
  PasswordOption.SendHashed);
MsCom_ws.RequestSoapContext.Security.Tokens.Add(ut);

// Get the TopDownloads
Downloads dls = MsCom_ws.GetTopDownloads(DesiredTopType,
  NumDownloads, CultureID);
•••

The Web application shown in Figure 4 shows the Microsoft.com Web Service in action. The Web page displays a list of the 25 most popular downloads on the left and displays the details of the selected download on the right.

Figure 4 Microsoft.com Web Service in Action (Close Figure 4)

TerraService

Back in 1998, a group of researchers at Microsoft set up a massive SQL database for storing aerial, satellite, and topographic images of the earth. Microsoft made this database available to the public over the Internet and called it TerraServer (see http://terraservice.net).

Users can easily access the images in TerraServer using any Web browser like Microsoft Internet Explorer or Netscape. The HTML-based interface of TerraServer, however, has one major limitation: it doesn't facilitate application integration without resorting to brittle screen-scraping techniques. With the advent of Web Services, the researchers realized that the new Web Services platform would provide the right framework for dealing with application integration scenarios. Hence, they implemented a Web Services interface to their imagery database and called it TerraService (see http://terraservice.net/webservices.aspx).

TerraService essentially provides the same functionality as TerraServer using XML Web Services technologies including XML 1.0, XML Schema, SOAP, and WSDL. The images found in TerraServer are in the public domain so there are no restrictions on how they're used as long as you place an acknowledgment next to the image referencing that it is a United States Geological Survey (USGS) image and that it comes from TerraServer. The bottom line is you don't have to register or get any type of license key to begin using this public service.

TerraServer contains a plethora of small 200×200 pixel images, each with a specific scale and resolution. A 1-meter resolution covers a 200×200 meter area, while a 2-meter resolution covers a 400×400 meter area, and so on. These images are referred to as tiles. TerraServer is interesting because there are enough tiles to stitch together a "seamless mosaic of the entire earth" as described by the TerraService team.

TerraService exposes several operations designed for searching geographic and image coordinates by place name, converting coordinates from one projection plane to another, and fetching tile metadata and the actual tile images. All of the operations are described in Figure 5 and grouped by category. The descriptions should help you get a feel for each operation. More information can be found in the online TerraService documentation.

Figure 5 TerraService Operations (Close Figure 5)

Search
GetPlaceFacts | Returns a PlaceFacts structure for a specific Place (City, State, and Country)
GetPlaceList | Returns an array of PlaceFacts data for all names that match the placeName parameter
GetPlaceListInRect | Retrieves TerraServer Gazetteer data for a specific Geographic rectangle
CountPlacesInRect | Similar to the GetPlaceListInRect method except that it returns the count of matching places rather than the list of matching places
Projection
ConvertLonLatPtToNearestPlace | Returns a string describing the Place name and direction to the closest place to the LonLatPt passed to the method
ConvertLonLatPtToUtmPt | Converts a Geographic, LonLatPt point to a UtmPt point
ConvertUtmPtToLonLatPt | Converts a UtmPt point to a Geographic, LonLatPt point
GetTheme | Returns attributes that apply to all tiles belonging to a specific theme
GetLatLonMetrics | Describes the extent of available imagery for the LonLatPt passed to the function
Tile Information
GetAreaFromPt | Returns the tile metadata for a Geographic rectangle
GetAreaFromRect | Similar to the GetAreaFromPt method except that the input parameter is a Geographic rectangle
GetAreaFromTileId | Similar to the GetAreaFromPt method except that the input parameter is a TileId structure
GetTileMetaFromLonLatPt | Returns a TileMeta structure containing the metadata for a single TerraServer image tile
GetTileMetaFromTileId | Similar to the GetTileMetaFromLonLatPt method except that the input parameter is a TileId structure
GetTile | Returns Byte array containing the compressed image data for the requested tile

To begin using TerraService, you simply need to generate code from the WSDL file and start using it. The service endpoint for imagery data is at http://terraserver-usa.com/TerraService.asmx. You'll notice the typical ASMX-generated documentation page illustrating that TerraService itself was implemented using the .NET Framework. You can browse to any of the operations to see the sample SOAP messages used by the operation. If you press the Service Description link, you'll see the WSDL file (you can browse directly to the WSDL file using http://terraserver-usa.com/TerraService.asmx?wsdl).

You can generate code from the WSDL file using the Visual Studio .NET Add Web Reference command or wsdl.exe on the command line. Once you've generated the classes, you can begin writing code to interact with TerraService. To illustrate this process, I've provided a small sample application that fetches tiles for a specified place (see Figure 6). The sample allows you to specify the type of image (photo versus topographical map) and the image scale you want. The code in Figure 7 runs when you press the GetTile button on the Windows Form.

Figure 7 Getting the Tile Image (Close Figure 7)
TerraService ts = new TerraService();

// specify place
Place place = new Place();
place.City = textBox1.Text;
place.State = textBox2.Text;
place.Country = textBox3.Text;

// get place facts
PlaceFacts pfacts = ts.GetPlaceFacts(place);

// get tile metadata for place, read them/scale from form
TileMeta tm = ts.GetTileMetaFromLonLatPt(pfacts.Center, theme, scale);

// get tile image
Byte[] imageBytes = ts.GetTile(tm.Id);
MemoryStream ms = new MemoryStream(imageBytes);
pictureBox1.Image = new Bitmap(ms);
Figure 6 Photo versus Topographical Map
Figure 6 Photo versus Topographical Map

TerraService is a powerful Web Service that offers valuable data and functionality to geography-oriented applications. TerraService is an excellent place to begin experimenting with Web Services in general if you're just starting down that path. The researchers involved with TerraService put together a thorough white paper that provides a good introduction to Web Services along with an in-depth description of the architecture and design considerations they followed (see TerraService.NET: An Introduction to Web Services).

MapPoint

The Microsoft MapPoint Web Service allows developers to integrate powerful mapping functionality including street maps, driving directions, and proximity searches into their applications. The MapPoint Web Service exposes locations in 19 countries throughout North America and Europe. It also provides business listings, points of interest, and other types of related data that can be used in conjunction with mapping.

The MapPoint Web Service was one of the first commercial Web Services available and Microsoft's first stab at making money with a service-oriented software model. According to the Web site, the MapPoint Web Service comes with two primary licensing models: per transaction and per user. The MapPoint Web Service currently serves more than 10 million transactions daily.

You can begin experimenting with the MapPoint Web Service by requesting an evaluation account from MapPoint Web Service Developer Account. You'll receive an account ID and a password to authenticate with the service. Unlike TerraService, you must have a user account and password in order to access any of the service's operations.

The MapPoint Web Service is logically organized into four subservices: one for common operations and utility functions and others for finding locations, rendering maps, and generating routes and directions. Figure 8 describes the functionality provided by each of these different services. You will typically use operations from multiple services to build a complete mapping-oriented application.

Figure 8 MapPoint Services (Close Figure 8)

Service | Description
Common | Contains operations and basic utility functions common to the find, route, and render services
Find | Locates addresses, geographic entities, latitude and longitude coordinates, and points of interest from MapPoint .NET data, parses addresses and return location information for a specified latitude and longitude coordinate
Render | Renders maps of routes and found locations, places pushpins (using the MapPoint .NET stock set of icons or your own), sets the map size and map view, selects points on a map, gets location information about points on a map, pans and zooms a rendered map, and creates clickable maps
Route | Generates routes, driving directions, and calculated route representations (used to render a highlighted route on the map) based on locations or waypoints, sets segment and route preferences, and generates map views of segments and directions

All of the operations exposed by these subservices are listed and defined in Figure 9. The brief descriptions should give you a good feel for the overall functionality exposed by the MapPoint Web Service. Using these operations you can find addresses and nonaddressable places such as cities, rivers, postal codes, or airports. There are operations for address parsing, reverse geocoding, and finding points of interest and nearby locations. And there are operations for generating maps, routes, and driving instructions.

Figure 9 MapPoint Web Service Operations (grouped by service) (Close Figure 9)

Common
GetCountryRegionInfo | Returns the country or region name, entity ID, latitude and longitude coordinates, codes, and language for a specified entity.
GetDataSourceInfo | Returns the functionality and name of a specified data source.
GetEntityTypes | Returns the entity types and their corresponding properties contained in a specified data source.
GetGreatCircleDistances | Returns an array of great circle distances, in decimal degrees, between specified points.
GetVersionInfo | Returns the descriptive name and the related version number of the current version of the MapPoint .NET service.
Find
Find | Returns a list of found geographic entities based on search options, in order of how well the results match the search criteria.
FindAddress | Returns a list of found addresses based on an input address, in order of how well the results match the search criteria.
FindNearby | Returns a list of found points of interest based on entity type, in order of proximity to a selected point.
GetLocationInfo | Returns a list of addresses and geographic entities at a specified latitude and longitude coordinate within a specified data source.
ParseAddress | Parses a specified address string and returns an Address object.
Render
GetMap | Returns map images, map views, and hot area definitions based on map options.
GetBestMapView | Returns the best map view for a selected location or set of locations. The best map view is the largest scale map that contains all the desired locations.
ConvertToLatLong | Converts pixel coordinates to latitude and longitude coordinates on a specific map, returning the latitude and longitude for a pixel on a rendered map.
ConvertToPoint | Converts latitude and longitude coordinates to pixel coordinates on a specific map, returning the pixel coordinates for a latitude and longitude on a rendered map.
Route
CalculateRoute | Returns a route based on route segments and specifications.
CalculateSimpleRoute | Returns a route based on specified latitude and longitude coordinates.

As with the other services I described, all you need to begin programming against the service is the WSDL file from which you can generate code. The MapPoint Web Service WSDL files are available online. Staging files can be found at http://staging.mappoint.net/standard-30/mappoint.wsdl and https://staging.mappoint.net/secure-30/mappoint.wsdl. Production files can be found at http://service.mappoint.net/standard-30/mappoint.wsdl and https://service.mappoint.net/secure-30/mappoint.wsdl. By choosing one of these WSDL files, you're also choosing between the staging and production environments provided by Microsoft.

Each WSDL file points to a different server endpoint, one designed for testing and another for production use. The staging environment is identical to the production environment, but it's not scaled for the same volume and throughput. You should use the staging environment during the development and testing phases and move to the production environment once your application is ready to go live. This is as easy as swapping out the WSDL files in your project.

I've provided a few sample applications that illustrate how to get the MapPoint Web Service up and running. The first sample illustrates how to retrieve a map for an address, using a red pushpin to illustrate the identified location (see Figure 10). Notice that the app allows the user to enter their address on a single line using the standard U.S. address syntax and the service takes care of parsing it into the constituent fields.

Figure 10 Retrieving a Map for an Address
Figure 10 Retrieving a Map for an Address

The code for this service uses a combination of the ParseAddress, FindAddress, and GetMap operations (from the Find and Render services). The source code in Figure 11 executes when the user presses the GetMap button on the form.

Figure 11 GetMap Sample (Close Figure 11)
NetworkCredential cred = new NetworkCredential("user", "password");

FindServiceSoap fs = new FindServiceSoap();
fs.Credentials = cred;

// parse address
Address address = fs.ParseAddress(textBox1.Text, comboBox1.Text);

// find address
FindAddressSpecification spec = new FindAddressSpecification();
spec.InputAddress = address;
spec.DataSourceName = "MapPoint.NA";
FindResults results;
results = fs.FindAddress(spec);

RenderServiceSoap rs = new RenderServiceSoap();
rs.Credentials = cred;

// define map views
ViewByHeightWidth[] views = new ViewByHeightWidth[1] = {
 results.Results[0].FoundLocation.BestMapView.ByHeightWidth};

// define map specification
MapSpecification mapSpec  = new MapSpecification();
mapSpec.DataSourceName = "MapPoint.NA";
mapSpec.Views = views;
mapSpec.Options = new MapOptions();
mapSpec.Options.Format = new ImageFormat();
mapSpec.Options.Format.Height = 350;
mapSpec.Options.Format.Width = 450;

// define pushpin
Pushpin pin = new Pushpin();
pin.IconDataSource = "MapPoint.Icons";
pin.IconName = "1";
pin.LatLong = results.Results[0].FoundLocation.LatLong;
mapSpec.Pushpins = new Pushpin[1] { pin };

// get map and display
MapImage[] mapImages = rs.GetMap(mapSpec);
if (mapImages.Length > 0)
{
    MemoryStream ms = new MemoryStream(mapImages[0].MimeData.Bits);
    pictureBox1.Image = new Bitmap(ms);
}

I've provided another sample that shows how to calculate a route between two locations (see Figure 12). The sample displays a map with start and end locations, identified with pushpins, and a highlighted route. The driving instructions are displayed in the textbox under the map.

Figure 12 Calculating a Route between Two Locations
Figure 12 Calculating a Route between Two Locations

Adding the route functionality involved using the CalculateSimpleRoute operation of the Route service along with the ParseAddress, FindAddress, and GetMap operations that I used in the previous example. If you supply the calculated route to the GetMap function, MapPoint automatically takes care of highlighting the map, as illustrated in Figure 12. You can then layer pushpins on top of the map to highlight the start and end locations. Figure 13 contains the complete source code for the GetMap button handler.

Figure 13 GetMap Button Handler (Close Figure 13)
NetworkCredential cred = new NetworkCredential("user", "password");

FindServiceSoap fs = new FindServiceSoap();
fs.Credentials = cred;

// parse addresses
Address address1 = fs.ParseAddress(textBox1.Text, comboBox1.Text);
Address address2 = fs.ParseAddress(textBox2.Text, comboBox2.Text);

// find addresses
FindAddressSpecification spec = new FindAddressSpecification();
spec.InputAddress = address1;
spec.DataSourceName = "MapPoint.NA";
FindResults results1;
results1 = fs.FindAddress(spec);
spec.InputAddress = address2;
FindResults results2;
results2 = fs.FindAddress(spec);

// calculate route using RouteService
RouteServiceSoap routesvc = new RouteServiceSoap();
routesvc.Credentials = cred;
LatLong[] points = new LatLong[2] {
    results1.Results[0].FoundLocation.LatLong,
    results2.Results[0].FoundLocation.LatLong
};
Route route = routesvc.CalculateSimpleRoute(
    points, "MapPoint.NA", SegmentPreference.Quickest);

// display driving directions
for(int i = 0; i < route.Itinerary.Segments[0].Directions.Length ; i++)
    textBox3.Text +=
       route.Itinerary.Segments[0].Directions[i].Instruction + "\r\n";

// render map with route
RenderServiceSoap rs = new RenderServiceSoap();
rs.Credentials = cred;

// define map spec
MapSpecification mapSpec  = new MapSpecification();
mapSpec.DataSourceName = "MapPoint.NA";
mapSpec.Route = route;
mapSpec.Options = new MapOptions();
mapSpec.Options.Format = new ImageFormat();
mapSpec.Options.Format.Height = 350;
mapSpec.Options.Format.Width = 450;

// define pushpins
Pushpin startpin = new Pushpin();
startpin.IconDataSource = "MapPoint.Icons";
startpin.IconName = "28";
startpin.LatLong = results1.Results[0].FoundLocation.LatLong;
Pushpin stoppin = new Pushpin();
stoppin.IconDataSource = "MapPoint.Icons";
stoppin.IconName = "29";
stoppin.LatLong = results2.Results[0].FoundLocation.LatLong;
mapSpec.Pushpins = new Pushpin[2] { startpin, stoppin };

// get map and display
MapImage[] mapImages = rs.GetMap(mapSpec);
if (mapImages.Length > 0)
{
    MemoryStream ms = new MemoryStream(mapImages[0].MimeData.Bits);
    pictureBox1.Image = new Bitmap(ms);
}

Although these samples illustrate how to use the MapPoint Web Service in a traditional Windows-based app, it's also possible (and probably more likely) that it will be used from within a Web app. MapPoint provides an image cache for this case so you don't have to worry about saving the retrieved images on your Web server's file system. MapPoint will save the image and provide a URL that can be referenced in the HTML page sent to the client.

There are several other MapPoint Web Service samples available at http://demo.mappoint.net. MSDN also offers a wealth of resources on the MapPoint Web Service at MapPoint Web Service Overview, both in the form of articles and sample code.

.NET Alerts

.NET Alerts is the fourth major Web Service that Microsoft offers today. It provides a powerful communication and notification system for reaching customers or members of an organization. Individuals use the service to subscribe to certain types of notifications and applications use the service to send notifications to all subscribed users.

An alert is any message that you might want to send to a user who has subscribed to receive it. Alerts can be received via e-mail, on mobile devices like PDAs and phones, and through MSN® Messenger. Alerts can be used for a variety of purposes that are valuable to both the individual user and organizations at large.

Like the MapPoint Web Service, .NET Alerts is a commercial Web Service that requires a service agreement before you can begin using it. The service is divided into three subservices including the Alerts Notification Service (accepts your alerts and delivers them to your subscribers), the Alerts Subscription Management Service (provides the ability to create, delete, and update users' subscriptions to your alerts), and the Alerts Topic Management Service (provides the ability to create, update, and enumerate topics).

You can download the .NET Alerts 6.0 SDK and other related resources from MSDN at Microsoft Alerts for Developers. The SDK comes with documentation, sample code, and tools that facilitate the development process. The SDK contains emulators for the .NET Alerts notification and subscription components. The emulators allow you to test your application before going live. Microsoft even requires an official compliance test before your site can go live using the service.

.NET Alerts is a bit more involved than the other services I've discussed in this column, and since I'm out of space, I won't be able to go into the details further. However, there are plenty of good .NET Alerts resources online if you're serious about using it. The Adventure Works 2.0 sample is a great starting point for mastering the details of .NET Alerts.

The State of Web Services

Microsoft provides several interesting Web Services that developers can use to integrate valuable data and functionality into their applications today. Other major organizations like Amazon.com and Google.com have also exposed some of their data and functionality through Web Services interfaces. To begin using any of these services, all you need is a WSDL file and a good Web Services toolkit like Visual Studio .NET 2003.

Send your questions and comments for Aaron to xmlfiles@microsoft.com.

Aaron Skonnard teaches at Northface University (http://www.northface.edu) in Salt Lake City, where he's helping establish one of the finest universities in the world for software developers. Aaron coauthored Essential XML Quick Reference (Addison Wesley, 2001) and Essential XML (Addison Wesley, 2000). Reach him at http://www.skonnard.com.


Advanced Basics: Windows Forms Q&amp;A

Advanced Basics
Windows Forms Q&A
Ken Spencer

Code download available at: AdvancedBasics0312.exe (131KB)

Q I have been playing with console applications in Visual Basic® .NET. I would like to know if it is possible to gather the variables upon calling the application, rather than asking for input when it is running (for example, TestApp.exe Variable1 Variable2)?

A There are at least two ways to obtain the command-line parameters from an application. It doesn't matter whether it's a GUI or console application, the code is the same. Let's try accessing the parameters in a Windows®-based application.

Figure 1 Debugging Options
Figure 1 Debugging Options

First, Visual Studio® .NET makes it easy to work with command-line parameters as you build and test your application. You can set parameters in the development environment and then test your application by running it from Visual Studio .NET. Then when you run the application outside of the development environment, the real parameters are used by your code. Figure 1 shows the debugging options dialog where you can set the parameters. Once the parameters are set in the options, you can write code to access them. Figure 2 shows code that accesses the parameters in Sub Main, the startup function. The parameters are passed into the application automatically, as you can see from the Sub Main definition shown here:

Sub Main(ByVal argv As String())
Since argv is an array, you can access the values as you would any other array element, as shown in Figure 2.

Figure 2 Accessing Params in Sub Main (Close Figure 2)
Module modMain
Sub Main(ByVal argv As String())
Dim oParam As String
Dim frmMain As New Form1
    For Each oParam In argv
        If frmMain.txtCommandLineParams.Text <> "" Then
            frmMain.txtCommandLineParams.Text &= vbCrLf
        End If
        frmMain.txtCommandLineParams.Text &= oParam.ToString()
Next
frmMain.ShowDialog()
End Sub
End Module

Figure 3 shows my favorite way to access the parameters. The Environment class makes it easy to access the command-line parameters from anywhere in your application. It is a rich class that lets you discover a tremendous amount of information, including an app's command-line parameters. You can gain access to these parameters by calling the GetCommandLineArgs method, as shown in Figure 3 and in the following console application code:

Sub Main()
Dim CommandLineArguments As String() = _
    Environment.GetCommandLineArgs()
Dim i As Integer

For i = 0 To CommandLineArguments.GetUpperBound(0)
    Console.WriteLine(CommandLineArguments(i) & vbCrLf)
Next
End Sub
This function returns an array which you can process to obtain the values. The first element in the array is the full path name of the application. The code shows the parameters in the console window.

Figure 3 Accessing Params with Environment (Close Figure 3)
Private Sub cmdGetParams_Click(ByVal sender As System.Object, _
    ByVal e As System.EventArgs) Handles cmdGetParams.Click
Dim CommandLineArguments As String() = _
    Environment.GetCommandLineArgs()
Dim i As Integer

txtCommandLineParams.ResetText()
For i = 0 To CommandLineArguments.GetUpperBound(0)
    If txtCommandLineParams.Text <> "" Then
        txtCommandLineParams.Text &= vbCrLf
    End If
    txtCommandLineParams.Text &= CommandLineArguments(i)
Next

End Sub

Q If I have a project which is only retrieving data from a database and displaying that data on a form, do I need to have the Connection.Adapter object on each form? I have many forms and lots of queries, so how do I execute these queries?

A There are many ways you can architect an application. I am a big fan of splitting code apart into discrete modules or classes where the code is easier to work with. Let's see how.

Figure 4 shows a simple module with one function. The function contains code that executes a SQL statement and returns a DataSet. You can drop it into a module in a project and then use it as shown here:

Dim ds As DataSet
Dim sSQL As String

sSQL = "Select * from customers"
'Get a dataset from the SQL above
ds = RunSQLWithDataSet(sSQL)

'Set the DataSource of the grid to table 0 from ds
DataGrid1.DataSource = ds.Tables(0)
This lets you to separate your data access code from your interface code. Then you can add other functions that perform other data-related tasks and call them as needed.

Figure 4 General Database Query (Close Figure 4)
Module modData
Const ConnectionString = "server=localhost;database=NorthWind"

Public Function RunSQLWithDataSet(ByVal sSQL As String) As DataSet

Dim oDataSet As New DataSet
Dim oDataAdapter As SqlClient.SqlDataAdapter

Try
    'Create New DataAdapter
    oDataAdapter = New SqlClient.SqlDataAdapter( sSQL, ConnectionString)

    'Fill DataSet from DataAdapter
    oDataAdapter.Fill(oDataSet, "Table1")

    'Set return value of function
    Return oDataSet
    Catch e As Exception
    Throw New Exception( "An exception occurred in RunSQLWithDataSet", e)
        Finally
        End Try
    End Function
End Module

The RunSQLWithDataSet function creates the data connection using a data adapter and simply returns a DataSet. This allows you to call the function over and over from your application without the need to create any data adapters or connections in a form or other code. Instead, you simply create the SQL you want to execute:

sSQL = "Select * from customers"
ds = RunSQLWithDataSet(sSQL)
Now your code can use the new ds DataSet in order to access the data that's returned.

For a more powerful and easier way to accomplish data access, Microsoft has created something called a data access application block for these tasks. This is a well-built and tested component that can perform just about all of your data needs. The component has features for executing stored procedures or SQL queries and allows you to easily specify parameter details for stored procedures. For more information, see Data Access Application Block.

Microsoft has also created a number of other application blocks and they are continually adding more, so check back often.

Q I want to design two forms using Visual Basic .NET. The first form has one checkbox and the second form has an Enter button. When the checkbox is checked and the Enter button is clicked, how can I get the information that's on the first form to show up on the second form?

A Windows Forms do not behave exactly like Visual Basic 6.0 forms. In Visual Basic 6.0, you could access properties on other forms automatically. Not so in Visual Basic .NET; now you must treat everything like an object.

Once you understand that forms are objects, then you can manipulate them in many different ways. Let's take a look at your question and see how to communicate between two forms. One approach is to pass the data into the new form when you display it. Figure 5 shows this approach.

Figure 5 PassingVarData_Load (Close Figure 5)
Dim NewData As String

Sub New(ByVal IncomingData As String)
    'This call is required by the Windows Forms Designer.
    InitializeComponent()

    NewData = IncomingData
End Sub

Private Sub PassingVarData_Load(ByVal sender As _
    System.Object, ByVal e As System.EventArgs) _
    Handles MyBase.Load

    txtTitle.Text = NewData
End Sub

The first step is to create a form-level variable for the new data:

Dim NewData As String
The second step is to create a new constructor for the form. This code simply creates a constructor that takes the new parameter. Then the code calls the InitializeComponent method to perform all the normal startup functions for the form and sets the NewData variable to the value that's passed in when the form is created:
Sub New(ByVal IncomingData As String)
    'This call is required by the Windows Forms Designer.
    InitializeComponent()
    NewData = IncomingData
End Sub

Then in the Form_Load event, the NewData variable is used to load the textbox. The first two lines of code show how to call the form and pass the data (Me.Text) to the constructor:

Dim frm As PassingVarData
frm = New PassingVarData(Me.Text)
frm.txtOutput.Text = txtCommandLineParams.Text

frm.ShowDialog()

This approach works well when you can pass data to the form when the form is created, but what if you must communicate between two forms once the forms are already created? This may mean that you pass data from one form to the other or that you must communicate back and forth between two or more forms.

This code also shows another approach that allows you to communicate with a form element. Once you have created the instance of the new form, you can access the form's controls directly, as shown in the following line:

frm.txtOutput.Text = txtCommandLineParams.Text

In this case, you simply prefix the control or property you want to access with a variable that points to the instance of the form. This, of course, means you must keep a variable in scope if you want to continue to communicate with the form. In this example, you could move the definition for the form to a form-level global variable, giving you continued access to the form's controls. If you move the definition to a friend or public variable, then you can access the form from anywhere in your application. You can also store references to forms in arrays, collections, and in a variety of other ways to allow you to store groups of forms for later reference.

While you can find a tremendous amount of information on the Internet about developing applications using the .NET Framework, it's not always organized so you can find it easily. So, just because you don't see something on the Internet, don't assume it's not possible or that it's not there somewhere. The MSDN® Help feature in Visual Studio is a great place to start for most questions because the index is pretty good. When I needed to create a queue to store information in a client's application, I fired up MSDN Help and entered "queue" in the index. Boy, was I surprised to find the Queue class, which of course saved lots of time.

Send your questions and comments for Ken to basics@microsoft.com.

Ken Spencer works for 32X Tech (http://www.32X.com), where he provides training, software development, and consulting services on Microsoft technologies.


Design Patterns: Asynchronous Wait State Pattern in ASP.NET

Design Patterns
Asynchronous Wait State Pattern in ASP.NET
Lyn Robison

Contents
The Long Run
A Long Process Begins with a Single Step
Please Wait...
Redirection
Making Excellent Progress
Cleaning Up the Code
A Pattern Emerges
Conclusion

Putting a browser UI on business applications makes them easily accessible to users in remote locations, but it presents some challenges for the developer. One of those challenges is how best to construct it for a long-running process.

For example, when a user enters data into a Web Form and clicks the submit button, the page does not change and the UI is unresponsive until the server sends a new page to the browser. If the UI is unresponsive for more than a few seconds, users are inclined to press the submit button repeatedly. These multiple submissions can cause problems for the application, forcing the developer to write code to handle the repeats.

A cleaner approach would be to keep the user informed about the server process with updated messages and a progress bar in the browser. In this column, you will learn how to create an asynchronous ASP.NET wait page that does exactly that. Let's begin by creating a page that initiates a long-running process. (The samples in this article are written in C# unless otherwise noted.)

The Long Run

To experience the need for a wait page, create a page named ProcessTheData.aspx. This page will contain two label controls: one for showing the results of a process, and another displaying the text of the exception, only visible if an exception has occurred.

Figure 1 shows the design view of the ProcessTheData.aspx page inside Visual Studio® .NET. As you can see, I applied a stylesheet to give the page a little polish.

Figure 1 Design View
Figure 1 Design View

The Page_Load event of ProcessTheData.aspx instantiates a business object named BusinessService and calls a method named ProcessMyData on that object. As you can see in Figure 2, the code in the Page_Load event takes the contents of the query string variable named Data, passes it to the ProcessMyData method, then sets the Text property of ResultLabel to show the results of that method call. If the ProcessMyData method encounters any exceptions, it places the exception message in the processedString out parameter and returns false. The code in the Page_Load event uses the ErrorLabel control to show the exception message on the page.

Figure 2 Page_Load Event (Close Figure 2)
private void Page_Load(object sender, System.EventArgs e)
{
   if (!IsPostBack)
   {
      if (null == Request.QueryString["Data"]
         || 0 == Request.QueryString["Data"].Length)
      {
         ErrorLabel.Text = "You must provide data for processing. "
            + "<BR> Press the Back button to enter some data.";
         ErrorLabel.Visible = true;
      }
      else
      {
         string inputString, processedString;
         inputString = Request.QueryString["Data"];
         if ((new BusinessService()).
            ProcessMyData(inputString, out processedString))
         {
            ResultLabel.Text = processedString;
         }
         else
         {
            ErrorLabel.Text = processedString;
            ErrorLabel.Visible = true;
         }
      }
   }
}

The code in ProcessMyData is very simple. It processes the data by converting it to uppercase. To turn this operation into a long-running process, I added a Thread.Sleep(30000) call to make ProcessMyData take 30 seconds (see Figure 3).

Figure 3 ProcessMyData (Close Figure 3)
public bool ProcessMyData(string DataIn, out string DataOut)
{
   try
   {
      DataOut = DataIn.ToUpper();
      System.Threading.Thread.Sleep(30000);
      return true;
   }
   catch (Exception except)
   {
      DataOut = except.Message;
      return false;
   }
}

It is easy to imagine calling a database query or performing some other real-world operation that might take seconds or even minutes to complete.

You can test the ProcessTheData.aspx page from the server with a URL such as http://localhost/MyWebApp/ProcessTheData.aspx?Data=test. When you test the ProcessTheData.aspx page, you will see that because of the Thread.Sleep(30000) call, the page will take at least 30 seconds to display.

A Long Process Begins with a Single Step

Next, create another page that opens this page and launches this process. Create a page named StartTheProcess.aspx and place a Hyperlink control on it with a Text property of

Process the Data
and a NavigateURL property of:
ProcessTheData.aspx?Data=test
Test the StartTheProcess.aspx page by clicking on the hyperlink. You will see that it takes at least 30 seconds for the ProcessTheData.aspx page to display.

Add a TextBox control and a Button control to the StartTheProcess.aspx page. The button control redirects to ProcessTheData.aspx, passing the contents of the TextBox to be processed, as shown in the following code:

private void ButtonData_Click(object sender, System.EventArgs e)
{
   Response.Redirect("ProcessTheData.aspx?Data="
      + TextBoxData.Text);
}
My StartTheProcess.aspx page looks like Figure 4 in the Visual Studio .NET design view.

Figure 4 The ASPX Page
Figure 4 The ASPX Page

When you click on the button, the browser will be redirected to the ProcessTheData.aspx page. If you have not entered any text in the textbox, the ProcessTheData.aspx page will say "You must provide data for processing." If you have entered text in the textbox, the ProcessTheData.aspx page will display that text in uppercase, after 30 or more seconds.

You will notice that when you click the button, the UI does not respond and the page does not change until the new page arrives from the server. Clicking the button repeatedly actually lengthens the time that you must wait for the ProcessTheData.aspx page to arrive. This is a problem for users because their wait gets longer, as well as a problem for the developer because repeated submissions can cause extraneous data to be submitted to the database. A wait page can solve the problem.

Please Wait...

One way to implement a wait page would be to use a client-side script that disables the submit button once it has been pressed. This script might also render a progress bar until the new page arrives from the server. However, there are a couple of problems with this approach.

First, client-side wait scripts embedded in your Web Forms might conflict with validator controls you might be using. Second, a client-side script could be impractical for a process that consists of multiple steps on the server. In other words, a server-side wait page could be updated more easily from the server at each step, providing a more responsive UI.

Let's take a look at how a server-side ASP.NET wait page might work. Instead of redirecting the submit button on the StartTheProcess.aspx page directly to ProcessTheData.aspx, it could redirect to the wait page and pass ProcessTheData.aspx as a query string, as shown in the following code:

private void ButtonData_Click(object sender, System.EventArgs e)
{
   Response.Redirect("Wait.aspx?redirectPage="
      + "ProcessTheData.aspx&Data="
      + TextBoxData.Text
      + "&secondsToWait=30");
}

The wait page redirects to the destination page that is specified in the redirectPage query string variable. Then it passes the rest of the query string variables to the destination page. The wait page displays a progress bar, and the secondsToWait query string variable specifies the ultimate length of that progress bar. At this point, the page goes away from the user's browser as soon as the destination page comes up.

How do you create this type of wait page? There are two major features that you must implement: the progress bar and the redirect functionality to load the destination page. Let's look at how to create the redirect first.

Redirection

I will talk about two ways to code an automated redirect. One option is to use an HTTP response header to cause the browser to automatically redirect itself after a certain amount of time. Here is an example of the HTML to do this:

<HTML>
<HEAD>
<META HTTP-EQUIV="Refresh" CONTENT="5; URL=ProcessTheData.aspx">
</HEAD>

As you can see, a META tag with the HTTP-EQUIV element, which has a value of "Refresh," is included in the <HEAD> section of the Web page. The CONTENT element contains the number of seconds that the page should wait, followed by the URL of the page to which the browser is to be redirected. In this example, after five seconds the browser will request the ProcessTheData.aspx page from the server.

To implement this type of wait page, create a C# Web Form named Wait.aspx. Add an ASP.NET Label control named ProcessingLabel that says "Processing...". Add the following META tag in the <HEAD> section of the Wait.aspx page:

   <META HTTP-EQUIV="Refresh" CONTENT="5;
   URL=ProcessTheData.aspx">
View the Wait.aspx page in your browser, and you will see that after five seconds, the ProcessTheData.aspx page will be displayed. (The page should indicate that you did not provide any data in order for it to process.)

Of course, because this Wait.aspx page is hardcoded to always redirect to the ProcessTheData.aspx page, it is of limited usefulness. Let's make the wait page a bit more flexible. Instead of hardcoding the URL in the CONTENT element of the HTTP-EQUIV META tag, change it so that the value of the URL is generated by C# code, like this:

<META HTTP-EQUIV="Refresh" CONTENT=
"5; URL=<%Response.Write(redirectPage+Request.Url.Query);%>">
This code sets the URL value of the CONTENT element to the contents of a member variable (of the Wait.aspx page's Wait class) named redirectPage, followed by whatever query string was passed to the wait page.

To make this work, add the following few lines of code to your Wait.aspx.cs file:

protected string redirectPage = "";
private void Page_Load(object sender, System.EventArgs e)
{
   if (null != Request.QueryString["redirectPage"])
   {
      redirectPage = Request.QueryString["redirectPage"];
   }
}
This code snippet declares the member variable named redirectPage in the Wait.aspx page's Wait class, and in the Page_Load event sets the redirectPage member variable to the contents of the redirectPage query string variable. So, when the page is loaded, the redirectPage member variable of the Wait.aspx page's Wait class is set to the destination page where the browser is to be redirected; when the HTML from the page renders, the URL value of the CONTENT element is set appropriately.

You can see that the META tag uses Request.Url.Query to append the entire query string that was passed to the wait page to the name of the destination page. You can test this procedure by making the button's click event on the StartTheProcess.aspx page match the code shown here:

private void ButtonData_Click(object sender, System.EventArgs e)
{
   Response.Redirect("Wait.aspx?redirectPage="
      + "ProcessTheData.aspx&Data="
      + TextBoxData.Text
      + "&secondsToWait=30");
}

You will notice that when you click the submit button, five seconds pass before the wait page requests the ProcessTheData.aspx page from the server. This is because after the CONTENT element, the META tag specified that the page should wait five seconds. It then takes 30 or more seconds for the ProcessTheData.aspx page to arrive from the server.

This five-second delay is not ideal, but it mitigates a problem. If you press the browser's Back button from the ProcessTheData.aspx page, you will notice that you go back to the wait page. You have to press the browser's Back button again to get all the way back to the StartTheProcess.aspx page. If you don't get off the wait page, it will request the ProcessTheData.aspx page again after five seconds have elapsed. If it were not for the five-second delay on the wait page—say it was set to zero—you would not be able to get back to the StartTheProcess.aspx page using the browser's Back button because the wait page would immediately request the ProcessTheData.aspx page once again.

I mentioned earlier that I would talk about two ways to code an automated redirect. This process of using the HTTP response header with a META tag and the HTTP-EQUIV element tends to work, but is not ideal because of the Back button problem. The second option is to use the replace method of the DHTML location object. The location.replace method avoids the dreaded Back button problem by only keeping the destination page in the browser's history. The location.replace method is supported in Microsoft® Internet Explorer 4.0 and above and Navigator 4.0 and above, which means that most browsers that are in use today support it.

To implement the location.replace method, first remove the HTTP-EQUIV META tag from the <HEAD> section of your wait page. Then change the opening BODY tag of your wait page so that it looks like this:

<body id="b1" onload="redirectTo(
   '<%Response.Write(redirectPage);%>',
   '<%Response.Write(Request.Url.Query);%>');"
MS_POSITIONING="GridLayout">
You should note that the setting of the MS_POSITIONING element is not at all relevant.

In the BODY tag of your wait page, in the body's onload event, you are calling a client-side script function named redirectTo. Add the code for the redirectTo function in the <HEAD> section of your wait page. The client-side script code should look like this:

<SCRIPT language="javascript">
function redirectTo(targetPage, querystring)
{
   if (0 < targetPage.length)
   {
      b1.style.cursor="wait";
      location.replace(targetPage + querystring);
   }
}
</SCRIPT>

Now test your wait page from your StartTheProcess.aspx page, and you will see that it redirects to the ProcessTheData.aspx page without that five-second delay. In other words, it takes 30 or more seconds instead of 35 or more seconds for the ProcessTheData.aspx page to come up. You will also see that you do not have to press the Back button twice from the ProcessTheData.aspx page to get back to the StartTheProcess.aspx page. The wait page does not get in the way of the Back button because the location.replace method replaces the wait page in the history with the ProcessTheData.aspx page and prevents that vicious cycle.

So, you have two ways to code an automatic redirect: the HTTP-EQUIV META tag and the DHTML location.replace method. The HTTP-EQUIV META tag seems like it might be a better choice when you want to display a "This page/site has been moved" page, which pauses for a set time and then redirects to the new location. The location.replace method tends to provide a cleaner implementation for the type of wait page that I'm creating.

Now let's tackle the progress bar.

Making Excellent Progress

To begin implementing a progress bar in your wait page, add these two members to your Wait class in Wait.aspx.cs:

protected string secondsToWait = "0";
protected string minutesToWait = "0";
Also add the code in Figure 5 to the Page_Load event of your wait page. The code in Figure 5 sets the secondsToWait and the minutesToWait values appropriately. When calling the wait page, you can specify the number of seconds or minutes to wait. If you specify minutesToWait, the code disregards secondsToWait because the seconds are generally not significant if you are waiting several minutes.

Figure 5 SecondsToWait and MinutesToWait (Close Figure 5)
if (null != Request.QueryString["secondsToWait"])
{
   secondsToWait = Request.QueryString["secondsToWait"];
   if ((Int32.Parse(secondsToWait) % 60 == 0)
      && ((Int32.Parse(secondsToWait)) / 60) > 1)
   {
      int minutes = Int32.Parse(secondsToWait) / 60;
      minutesToWait = minutes.ToString();
      secondsToWait = "0";
   }
}
if (null != Request.QueryString["minutesToWait"])
{
   secondsToWait = "0";
   minutesToWait = Request.QueryString["minutesToWait"];
}

Change your wait page's HTML body tag to look like this:

<body id=b1 onload="redirectTo('<%Response.Write(redirectPage);%>',
   '<%Response.Write(Request.Url.Query);%>',
   <%Response.Write(secondsToWait);%>,
   <%Response.Write(minutesToWait);%>);">
   <form id="Form1" method="post" runat="server">
      <asp:Label id="ProcessingLabel" runat="server"
      CssClass="DarkMediumSizeText">Processing...</asp:Label>
      <p></p>
      <DIV id="d1" class="DarkSmallSizeText"></DIV>
      <DIV id="d2" style="BACKGROUND-COLOR:red"></DIV>
   </form>
</body>
You can see that I added parameters to the redirectTo function for the secondsToWait and minutesToWait values. I also added two DIV tags with ids of d1 and d2. The d1 DIV will show the number of seconds that have elapsed since the wait page came up; the d2 DIV will contain a red progress bar. I used a couple of CSS classes named DarkMediumSizeText and DarkSmallSizeText, which you can implement as you like. The code for the completed redirectTo function is shown in Figure 6.

Figure 6 redirectTo and timedIterations (Close Figure 6)
<SCRIPT language="javascript">
var i = 0;
function redirectTo(targetPage, querystring, secondsForWaiting,
    minutesForWaiting)
{
   if (0 < targetPage.length)
   {
      location.replace(targetPage + querystring);
      b1.style.cursor="wait";
      if (secondsForWaiting.valueOf() > 0)
      {
         ProcessingLabel.innerText =
            "This process can take up to "
            + secondsForWaiting + " seconds...";
         timedIterations(secondsForWaiting);
      }
      else
      {
         if (minutesForWaiting.valueOf() > 0)
         {
            ProcessingLabel.innerText =
               "This process can take up to "
               + minutesForWaiting + " minutes...";
            timedIterations(minutesForWaiting * 60);
         }
      }
   }
   else
   {
      ProcessingLabel.innerText = "Page not found."
   }
}

function timedIterations(secondsForIterating)
{
   incrementalWidth = 800 / secondsForIterating;
   if (i <= secondsForIterating + 10)
   {
      d1.innerText="Elapsed time: " + i + " seconds.";
      d2.style.width=i*incrementalWidth;
      setTimeout(
         "timedIterations(" + secondsForIterating + ");",
         1000);
      i++;
   }
   else
   {
      b1.style.cursor="";
      d1.style.visibility = "hidden";
      d2.style.visibility = "hidden";
      ProcessingLabel.innerText =
         "The server is taking longer than "
         + "anticipated to process your request. "
         + "Thank you for your patience. "
         + "You can wait a few minutes longer for "
         + "the process to complete, or you can press "
         + "the back button and try again later...";
   }
}
</SCRIPT>

Here's a brief summary of what the code in Figure 6 accomplishes. After calling location.replace, the redirectTo function sets the Text property of the ProcessingLabel Label control to let the user know how long the process will take. Then it calls the timedIterations function, which calls itself recursively once each second, setting the length (width) of the progress bar and reporting to the user the number of seconds that have elapsed. The maximum width of the progress bar is set for 800 pixels. When the wait time plus 10 seconds has elapsed, the report of the elapsed seconds and the progress bar are hidden and the ProcessingLabel Label control's text property tells the user that the server is taking longer than expected.

Figure 7 The Wait Page
Figure 7 The Wait Page

You can see what the wait page looks like in Figure 7.

Cleaning Up the Code

One thing that is still not very clean about this wait page implementation is the fact that the StartTheProcess.aspx page has to know how much time the ProcessTheData.aspx page is going to take. Wouldn't it be nice if the StartTheProcess.aspx page could just point directly to the ProcessTheData.aspx page, and have the ProcessTheData.aspx page bring up the wait page by itself if it needs to? In other words, wouldn't it be better for a slow page to put up its own wait page without the pointing page having to know about it? It turns out that you can add wait page functionality to any page that needs it.

First, change the StartTheProcess.aspx page so that it redirects directly to the ProcessTheData.aspx page, like this:

private void ButtonData_Click(object sender, System.EventArgs e)
{
   Response.Redirect("ProcessTheData.aspx?Data="
      + TextBoxData.Text);
}
Then, change the Page_Load event code of the ProcessTheData.aspx page so that it looks like Figure 8.

Figure 8 Revised Page_Load (Close Figure 8)
private void Page_Load(object sender, System.EventArgs e)
{
   if (!IsPostBack)
   {
      if (null == Request.QueryString["Data"]
         || 0 == Request.QueryString["Data"].Length)
      {
         ErrorLabel.Text = "You must provide data for processing. "
            + "<BR> Press the Back button to enter some data.";
         ErrorLabel.Visible = true;
      }
      else
      {
         if (null == Request.QueryString["redirectPage"]
            || 0 == Request.QueryString["redirectPage"].Length)
         {
            Response.Redirect("Wait.aspx"
               + Request.Url.Query
               + "&redirectPage=ProcessTheData.aspx"
               + "&secondsToWait=30", true);
         }
         else
         {
            //string inputString, processedString;
            inputString = Request.QueryString["Data"];
            if ((new BusinessService()).
               ProcessMyData(inputString, out processedString))
            {
               ResultLabel.Text = processedString;
            }
            else
            {
               ErrorLabel.Text = processedString;
               ErrorLabel.Visible = true;
            }
         }
      }
   }
}

The purpose of this new Page_Load code for the ProcessTheData.aspx page is that if there is no query string variable named redirectPage, then you know that this request did not come from the wait page. In that case, you do a redirect to the wait page (passing true for the second parameter in order to stop any further processing). The wait page then redirects back to this ProcessTheData.aspx page (and includes the redirectPage query string variable). Now, secure in the knowledge that the user is viewing the wait page, you can go ahead and take your own sweet time to process the data and return the result.

With this code you have a page that displays its own wait page, and the pointing page does not need to know a thing about it.

A Pattern Emerges

You can see that I'm handling a simple state transition in the Page_Load event for the ProcessTheData.aspx page. I'm using a query string variable named redirectPage to tell me which of the two states to enter (either the wait page not sent or the wait page sent). Let's see if I can implement a pattern that would enable me to handle complex state transitions for any process that would take several steps to complete.

To implement such a pattern, add another Button control to the StartTheProcess.aspx page. The Button control redirects to the ProcessTheDataPattern.aspx page, passing the contents of the textbox, as shown here:

private void ButtonData_Click(object sender, System.EventArgs e)
{
   Response.Redirect("ProcessTheDataPattern.aspx?Data="
      + TextBoxData.Text);
}

Create a Web Form named ProcessTheDataPattern.aspx and add two Label controls as you did on the the ProcessTheData.aspx page. Make the code for the ProcessTheDataPattern.aspx page look like the code in Figure 9.

Figure 9 ProcessTheDataPattern (Close Figure 9)
public class ProcessTheDataPattern : System.Web.UI.Page
{
   protected System.Web.UI.WebControls.Label ResultLabel;
   protected System.Web.UI.WebControls.Label ErrorLabel;
   protected string processedString;

   private void Page_Load(object sender, System.EventArgs e)
   {
      if (!IsPostBack)
      {
         if (null == Request.QueryString["Data"]
            || 0 == Request.QueryString["Data"].Length)
         {
            ErrorLabel.Text = "You must provide data for processing. "
               + "<BR> Press the Back button to enter some data.";
            ErrorLabel.Visible = true;
         }
         else
         {
            IProcessObject myProcObj;
            if (null==Session["ProcObj"])
            {
               ProcessObject1 procObj = new ProcessObject1();
               myProcObj = (IProcessObject) procObj;
               Session["ProcObj"] = myProcObj;
            }
            else
            {
               myProcObj = (IProcessObject) Session["ProcObj"];
            }

            if (myProcObj.ProcessMyData(Request, Response,
               Session, out processedString))
            {
               ResultLabel.Text = processedString;
            }
            else
            {
               ErrorLabel.Text = processedString;
               ErrorLabel.Visible = true;
            }
         }
      }
   }

You can see in Figure 9 that the Page_Load event looks in a Session variable named ProcObj to get the business object that it will use. If that Session variable is empty, it creates an instance of ProcessObject1, saves it in the ProcObj Session variable, and then calls its ProcessMyData method. The idea is that if an instance of the business object does not already exist, then the page creates a business object that handles the first step of the process. If an instance of the business object already exists, then this is a subsequent step in the process and the Page_Load event calls that instance's ProcessMyData method to perform that step.

Each of the business objects that handle each step in the process implement an interface. So create an interface named IProcessObject and define it as shown here:

public interface IProcessObject
{
   bool ProcessMyData(System.Web.HttpRequest request,
      System.Web.HttpResponse response,
      System.Web.SessionState.HttpSessionState session,
      out string DataOut);
}
Now create a class named ProcessObject1 that implements this interface. Now you can implement the ProcessMyData method, as shown in Figure 10.

Figure 10 ProcessMyData in ProcessObject1 (Close Figure 10)
public bool ProcessMyData(System.Web.HttpRequest request,
    System.Web.HttpResponse response,
    System.Web.SessionState.HttpSessionState session,
    out string DataOut)
{
    DataOut = "";
    try
    {
        if (null == request.QueryString["redirectPage"]
            || 0 == request.QueryString["redirectPage"].Length)
        {
            response.Redirect("Wait.aspx"
                + request.Url.Query
                + "&redirectPage=ProcessTheDataPattern.aspx"
                + "&secondsToWait=30", true);
        }
        else
        {
            DataOut = "Test1";
            System.Threading.Thread.Sleep(10000);
            ProcessObject2 procObj = new ProcessObject2();
            IProcessObject myProcObj = (IProcessObject)
               procObj;
            session["ProcObj"] = myProcObj;
            response.Redirect("ProcessTheDataPattern.aspx?Data=" +
               DataOut);
        }

        return true;
    }
    catch (Exception except)
    {
        DataOut = except.Message;
        return false;
    }
}

As you can see in Figure 10, the ProcessMyData method of the ProcessObject1 business object redirects to the wait page and then sets the DataOut parameter to "Test1". It sleeps for 10 seconds to simulate a long-running process, and then creates a ProcessObject2 business object to handle the second step in the process, which it stores in the ProcObj Session variable. Finally, it redirects the browser back to the ProcessTheDataPattern.aspx page in order to kick off the second step.

To implement the second step, create another class named ProcessObject2 that implements the IProcessObejct interface. Implement its ProcessMyData method, as shown in Figure 11.

Figure 11 ProcessMyData in ProcessObject2 (Close Figure 11)
public bool ProcessMyData(System.Web.HttpRequest request,
   System.Web.HttpResponse response,
   System.Web.SessionState.HttpSessionState session,
   out string DataOut)
{
   DataOut = "";
   try
   {
      if (null == request.QueryString["redirectPage"]
         || 0 == request.QueryString["redirectPage"].Length)
      {
         response.Redirect("Wait.aspx"
            + request.Url.Query
            + "&redirectPage=ProcessTheDataPattern.aspx"
            + "&secondsToWait=20", true);
      }
      else
      {
         DataOut = "Test2";
         System.Threading.Thread.Sleep(10000);
      }

      return true;
   }
   catch (Exception except)
   {
      DataOut = except.Message;
      return false;
   }
}

As you can see in Figure 11, the ProcessMyData method of the ProcessObject2 business object redirects to the wait page and then sets the DataOut parameter to "Test2". It sleeps for 10 seconds to simulate a long-running process. Then it simply returns true because this is the last step.

If there were a third step, the ProcessMyData method of the ProcessObject2 would need to create an instance of a ProcessObject3 business object, store it in the ProcObj Session variable, and then redirect to the ProcessTheDataPattern.aspx page to execute the third step.

When you click on the Button control in the StartTheProcess.aspx page that points to the ProcessTheDataPattern.aspx page, you will notice that the data in the query string of the wait page changes. After step one, the data becomes "Test1". After step two, the data becomes "Test2". In this example, both the ProcessObject1 business object and the ProcessObject2 business object use the same wait page. For your own multi-step processes, you would probably want to create a unique wait page for each step, each of which tells the user what is happening.

You will also notice that the progress bar starts over at each step. You may want to enhance your wait pages so that the progress bar takes a parameter for its starting width, enabling you to show a progress bar that grows through all of the steps.

With this pattern, you have a ProcessTheDataPattern.aspx page that either starts the process by instantiating the first business object or simply executes whatever business object is there already. The code for each step in the process is encapsulated in its own business object. Each business object displays its own wait page (if needed) and then instantiates the appropriate business object to handle the next step. Adding new steps is a relatively clean matter of creating a new business object that implements the IProcessObject interface and calls the next step after it if necessary.

Conclusion

Long-running processes are a frequent feature of business applications. A wait page that keeps users informed and prevents them from making extraneous page submissions is helpful for both users and developers. Pages that have the ability to display their own wait page are ideal because they encapsulate their own logic, alleviating the need for other pages to know the details of their operations. Using the design pattern illustrated in this column, you can create a browser UI for a multi-step process in which the wait page is updated throughout the process.

Send questions and comments for Design Patterns to mmpatt@microsoft.com.

Lyn Robison is a developer at Tualatin Software, a maker of tools for IT developers who specialize in .NET (see http://www.tualatinsoftware.com). You can contact Lyn at lynr@tualatinsoftware.com.


C++ Q&amp;A: Docking the Menu Bar, Abstract Classes vs. Interfaces, and More

C++ Q&A
Docking the Menu Bar, Abstract Classes vs. Interfaces, and More
Paul DiLascia

Code download available at: CQA0312.exe (334KB)

Q I'm writing a Windows®-based app using C++ and MFC. My app has a standard toolbar implemented using CToolBar. I want to allow the toolbar to be docked on any side of the frame window (top, bottom, left, right), but not floating. I tried various combinations of the flags CBRS_ALIGN_XXX, CBRS_FLOATING, and CBRS_SIZE_DYNAMIC, but I couldn't seem to make it work. Is there some way to make the toolbar dockable but disallow floating?

Meg Donovan

A Indeed there is a way, but it's not as simple as using the right combination of style flags. You have to write some code. I know, life is tough.

Figure 1 Docking the Toolbar
Figure 1 Docking the Toolbar

Just so everyone knows what I'm talking about, MFC control bars (of which CToolBar is an example) support a feature called docking. This means the user can drag the tool/control bar to one of the edges of the frame, as in Figure 1. To enable docking, you have to set things up, usually in your frame's OnCreate handler:

int CMainFrame::OnCreate(LPCREATESTRUCT lpCreateStruct)
{
  // ... create toolbar
  VERIFY(m_wndToolBar.CreateEx(..., CBRS_GRIPPER));
  m_wndToolBar.EnableDocking(CBRS_ALIGN_ANY);
  // enable docking feature
  EnableDocking(CBRS_ALIGN_ANY);
  DockControlBar(&m_wndToolBar);
  return 0;
}

When you want docking, it's important to use the CBRS_GRIPPER style, which causes the toolbar to display the grip handle. When you enable your toolbar for docking, it becomes automatically enabled for floating, too. This means the user can drag the toolbar somewhere off the border and leave it "floating" in mid-air, as shown in Figure 2. But what if you want to let your toolbar dock, but not float? I was unable to find any combination of style flags to do it, but I was able to write a bit of code to make it happen. The details are a bit grody so I encapsulated (hid) them in a separate class, CNoFloatBar, which you can simply drop into your app to achieve the desired effect. Just instantiate and call CNoFloatBar::Install, as you can see in the following code:

   int CMainFrame::OnCreate(LPCREATESTRUCT lpCreateStruct)
   {
     •••
     m_noFloat.Install(&m_wndToolBar);
     return 0;
   }

Pretty easy. But how does it work? MFC uses several private classes—CDockBar, CMiniDockFrameWnd, and CDockContext, all declared in afxpriv.h—to implement docking. When I first set out to implement CNoFloatBar, I figured I'd have to dig into this code, derive new classes, and override key functions. These efforts quickly sank in a morass of details. The MFC docking code is complex, difficult to understand, and full of non-virtual functions and hardwired class names. Clearly, this code was never meant to be extended. When you find yourself contemplating whether to copy a half dozen MFC functions verbatim into your own class just so you can change one line in each, it's a sure sign there's got to be a better way. Always remember: good programmers are lazy. Who wants to write all that code?

Figure 2 Floating Toolbar
Figure 2 Floating Toolbar

So how can you prevent floating? Floating happens when the user drags the toolbar somewhere away from the edge of the frame. If you can prevent dragging, you can prevent floating. One way might be to override CToolBar::OnMouseMove. But it doesn't work because when the user clicks the mouse on the toolbar to begin dragging, CToolBar::OnLButtonDown passes control to an internal function that enters a new message loop. (This is neither necessary nor ideal—but hey, what can you do?) You can override OnMouseMove all day, but your code will never be called because your program has gone off on a second message loop. So how can you trap mouse messages?

With a Windows hook, of course! When the toolbar gets WM_LBUTTONDOWN, CNoFloatBar installs a WH_GETMESSAGE hook. The hook procedure intercepts WM_MOUSEMOVE and calls a helper function CNoFloatBar::ConstrainPoint to constrain the cursor to lie along the edges of the frame. If the user moves the cursor away from the edges of the frame, CNoFloatBar::ConstrainPoint uses advanced pixel arithmetic to move it back, actually modifying MSG::lParam (the mouse coordinates) on the fly before passing the message on to the toolbar. This doesn't affect the actual cursor position on the screen (you'd have to call SetCursorPos for that); it only changes what the toolbar sees. In other words, CNoFloatBar lies. The user has moved the cursor away from the frame edge, but CNoFloatBar doesn't tell the toolbar. CNoFloatBar passes false mouse coordinates, ones that always lie along the frame edge. So CNoFloatBar doesn't actually prevent the toolbar from floating; it merely prevents the toolbar from ever seeing a cursor position that would cause it to float. Pretty clever, eh? You really have to run the code to see how it works.

There are, of course, other details to fret. You may not have noticed, but while you're dragging the toolbar you can press the control key to make it float. CNoFloatBar traps WM_KEYUP/DOWN to prevent this. MFC also lets users double-click to toggle the toolbar from floating to docked state. CNoFloatBar prevents this, too. Finally, in order to encapsulate all this no-float behavior in a separate class which you can plug into your app without deriving from CToolBar, CNoFloatBar derives itself from my world-famous CSubclassWnd class. CSubclassWnd provides a way to trap messages sent to any CWnd-derived object.

To show how everything works in practice, I wrote a little program called TestNoFloat that uses CNoFloatBar to implement a command, View | Allow Floating Toolbar, which turns the no-float feature on or off. Figure 3 shows the code for CNoFloatBar. As always, you can download the full app from the link at the top of this article.

Figure 3 CNoFloatBar (Close Figure 3)

NoFloat.h
///////////////////////////////////////////////////////////////
// MSDN Magazine — December 2003
// If this code works, it was written by Paul DiLascia.
// If not, I don't know who wrote it.
// Compiles with Visual Studio .NET on Windows XP. Tab size=3.
//
#pragma once

#include "subclass.h"

//////////////////
// Class used to implement non-floating control bars.
// To use it, instantiate in your CMainFrame, then install like so:
//
//    m_noFloat.Install(&m_wndMyControlBar);
//
// The control bar can now dock but not float. Install one for each
// control bar.
//
class CNoFloatBar : public CSubclassWnd {
protected:
   static CNoFloatBar* g_pNoFloat;   // global: this
   static HHOOK g_hook;              // global: hook
   static LRESULT CALLBACK GetMessageHook(int code, WPARAM wp,
      LPARAM lp);

public:
   CNoFloatBar();
   virtual ~CNoFloatBar();

   BOOL Install(CWnd* pWnd) {
      return HookWindow(pWnd);
   }
   void Remove() {
      HookWindow((HWND)NULL);
   }

protected:
   // override for different mouse constraint
   virtual void ConstrainPoint(POINT& pt);

   // Rarely override:
   virtual BOOL OnGetMessageHook(int code, WPARAM wp, LPARAM
                                 lp);
   virtual LRESULT WindowProc(UINT msg, WPARAM wp, LPARAM lp);
   DECLARE_DYNAMIC(CNoFloatBar);
};
NoFloat.cpp
///////////////////////////////////////////////////////////////
// MSDN Magazine — December 2003
// If this code works, it was written by Paul DiLascia.
// If not, I don't know who wrote it.
// Compiles with Visual Studio .NET on Windows XP. Tab size=3.
//
#include "stdafx.h"
#include "NoFloat.h"
#include <afxpriv.h> // CDockBar

// Globals. Note: this implementation assumes only one non-
// floating toolbar in the application. If you have several,
// you must make some provision for obtaining the CNoFloatBar
// object somehow—for example from the window handle (HWND) by
// looking it up in a map/hashtable.
//
HHOOK CNoFloatBar::g_hook = NULL;             // windows hook handle
CNoFloatBar* CNoFloatBar::g_pNoFloat = NULL;  // ptr to global
                                              // object

IMPLEMENT_DYNAMIC(CNoFloatBar, CSubclassWnd);

CNoFloatBar::CNoFloatBar()
{
}

CNoFloatBar::~CNoFloatBar()
{
}

//////////////////
// Handle message sent to control bar.
// I'm interested in mouse down and double-click.
//
LRESULT CNoFloatBar::WindowProc(UINT msg, WPARAM wp, LPARAM lp)
{
   if (msg==WM_LBUTTONDOWN) {
      // mouse down: install get message hook
      ASSERT(g_hook==NULL); // can't be already hooked!
      g_pNoFloat = this;
      g_hook = SetWindowsHookEx(WH_GETMESSAGE,
         GetMessageHook, NULL, ::GetCurrentThreadId());

   } else if (msg==WM_LBUTTONDBLCLK) {
      // don't allow double-click switch to floating state
      return 0;
   }
   return CSubclassWnd::WindowProc(msg, wp, lp);
}

//////////////////
// Hook proc: pass to virtual fn.
//
LRESULT CALLBACK CNoFloatBar::GetMessageHook(int code, WPARAM
    wp, LPARAM lp)
{
   if (code==HC_ACTION) {
      ASSERT(g_pNoFloat);
      if (!g_pNoFloat->OnGetMessageHook(code, wp, lp)) {
         // object returned FALSE: unhook
         LRESULT lRet = ::CallNextHookEx(g_hook, code, wp, lp);
         UnhookWindowsHookEx(g_hook);
         g_hook = NULL;
         return lRet;
      }
   }
   return ::CallNextHookEx(g_hook, code, wp, lp);
}

//////////////////
// Virtual hook proc: constrain mouse movement, prevent
// Shift/Control to float.
//
BOOL CNoFloatBar::OnGetMessageHook(int code, WPARAM wp, LPARAM lp)
{
   MSG& msg = *((MSG*)lp);
   if (msg.message==WM_LBUTTONUP || ::GetCapture()!=m_hWnd) {
      // unhook if mouse up or if the dock context released
      // capture
      return FALSE;

   } else if (msg.message==WM_MOUSEMOVE) {
      // mouse move: alter coords to possible docking area
      ConstrainPoint(msg.pt);
      CWnd* pWnd = CWnd::FromHandle(msg.hwnd);
      CPoint pt = msg.pt;
      pWnd->ScreenToClient(&pt);
      msg.lParam = MAKELPARAM(pt.x, pt.y);

   } else if (msg.message==WM_KEYUP || msg.message==WM_KEYDOWN) {
      // special keys that can end up floating: disallow
      if (msg.wParam == VK_SHIFT || msg.wParam == VK_CONTROL)
         msg.wParam = 0; // harmless
   }
   return TRUE; // keep hooking
}

//////////////////
// Constrain point to possible dockbars
// (top/bottom/left/right).
// Algorithm: loop through all dockbars, compute point on
// dockbar that mouse would snap to if forced to dock to that
// dockbar, then take closest of these. Distance squared is as
// good as distance for our purposes.
//
void CNoFloatBar::ConstrainPoint(POINT& pt)
{
   CFrameWnd* pFrame = CWnd::FromHandle(m_hWnd)->GetParentFrame();
   ASSERT(pFrame);
   CRect rc;
   pFrame->GetWindowRect(&rc);

   CPoint ptSnap;
   UINT distance2 = (UINT)-1;
   POSITION pos = pFrame->m_listControlBars.GetHeadPosition();
   while (pos != NULL) {
      CDockBar* pBar = (CDockBar*)pFrame->m_listControlBars.GetNext(pos);
      if (pBar->IsDockBar()) { // always true for real CDockBar
         CRect rc;
         pBar->GetWindowRect(&rc);
         CPoint ptBarSnap(min(max(pt.x, rc.left), rc.right),
                          min(max(pt.y, rc.top),  rc.bottom));
         int dx = ptBarSnap.x - pt.x;
         int dy = ptBarSnap.y - pt.y;
         UINT d2 = dx*dx + dy*dy;
         if (d2 < distance2) {
            // this snap point is closer than previous ones:
            // use it
            distance2 = d2;
            ptSnap = ptBarSnap;
         }
      }
   }
   pt = ptSnap;
}

Q In the Microsoft® .NET Framework, what's the difference between an abstract class and an interface? When should I use one as opposed to the other?

Sam Keiko

A Good question! In the .NET Framework, an abstract class is any class that's declared with the abstract keyword. Usually—but this is not required—such a class has at least one abstract method. For example, take a look at the following code:

abstract class Shape
{
  // abstract method/property
  public abstract double Area { get; }
  // concrete method/property/data
  private Rectangle bounds;
  public Rectangle Bounds {
    get { return bounds; }
  }
}

Shape is a generic class that has a read-only property called Area. The Shape class doesn't implement this property itself. Area must be supplied by a derived class, as shown here:

// concrete class implements Area
public class Circle : Shape
{
  private double r;
  public double Area
  {
    get { return pi*r*r; }
  }
}

In C++, you make a class abstract by giving it at least one abstract (pure) virtual function. In C#, you do so by declaring the class abstract; in C++ use the "=0" syntax:

class CShape
{
  // pure (abstract) virtual fn
  double GetArea() = 0;
}
Unlike C#, C++ has no way to declare a class abstract; you must declare at least one pure virtual function using the initialize-to-zero syntax. But in both C++ and C#, an abstract class cannot be instantiated. It can be used only to derive new classes:
// C# — error!
Shape s = new Shape();
// C++ — error!
CShape* pShape = new CShape();

A good example of an abstract class in MFC is CView. This class implements basic functionality for a window that displays the contents of a document. It provides data and functions that implement basic view behavior such as command routing, window sizing, display update, and so on—but CView leaves out one function, CView::OnDraw:

// from afxwin.h
class CView : public CWnd
{
  •••
  virtual void OnDraw(CDC* pDC) = 0;
};
CView doesn't implement OnDraw because it doesn't know how. Despite aspirations of clairvoyance, the programmers in Redmond can't know what you intend to draw or how. It's up to you to supply the implementation. So OnDraw is abstract. (Here's an MFC trivia question: which other CView function is abstract?)

So much for abstract classes. What about interfaces? An interface is an abstract class that has no data members and whose methods are all abstract:

public interface IShape
{
  public abstract double Area { get; }
  // no data, no implementation
  public abstract Rectangle Bounds { get; }
}

While an abstract class may provide partial implementation in the form of concrete methods and data, an interface has neither. In C++/OOP terminology, such a class is sometimes called an abstract base class (ABC). You could say that an interface is purely abstract. Like abstract classes, interfaces can be used only to derive other classes; they cannot be instantiated directly. In C#, the concept of interface is built into the language (through the interface keyword). In C++, there's no explicit concept of interface. In C++, interface is a design pattern that you decide to adopt.

Formal definitions are fine, but what's really going on? What's the best way to think about abstract classes and interfaces? You should think of an interface as a list of methods (or properties) that a class must implement in order to "be" the class. A typical example from the .NET Framework is IDisposable. It has just one method—Dispose.

public interface IDisposable
{
  void Dispose();
}

What this means is that in order to "be" a disposable thing, a class must implement a method called Dispose. A single class may implement several interfaces. For example, in the .NET Framework, ArrayList implements IEnumerable as well as IList and ICollection. An array list "is" enumerable and "is" a list and "is" a collection. An array list can be all these things because it implements the required methods for each interface. Interfaces are the building blocks for systems of reusable code objects.

Whether in COM, .NET, Java, or SNOBOL (just kidding), interfaces provide a way to implement multiple inheritance, without the implementation problems associated with multiple inheritance in C++. One crucial difference between multiple inheritance using interfaces and multiple inheritance by inclusion (as in C++) is that with interfaces you can't implicitly treat any object as an instance of some interface it implements, the way you can with C++ multiple inheritance. You must explicitly obtain the interface before using it—either by casting, as in C# and Java, or by calling a special method like QueryInterface in C++/COM.

So to make a long story short, interfaces are purely descriptive, with no implementation; abstract classes are part description, part implementation. The chart shown in Figure 4 summarizes some of main similarities and differences between abstract classes and interfaces in the C# and C++ languages.

Figure 4 Abstract Class versus Interface (Close Figure 4)

Abstract Class | Interface
Cannot instantiate. | Cannot instantiate.
Can only be used to derive. | Can only be used to derive.
Can contain data and/or implementation in the form of concrete methods. | No data, no implementation. (All methods/virtual functions must be abstract/pure.)
In C#: declare both class and abstract methods with abstract keyword. | In C#: declare class with interface keyword. All methods become virtual.
In C++: declare at least one pure virtual function using "= 0" syntax. Can use destructor, too. | In C++: no concept of interface built into C++; interface is a design pattern you must follow.
Use when you want to provide a partial implementation with some functions omitted. | Use when you want to define the abstract qualities of a thing, the methods a class must implement to "be" that thing.

Q I'm trying to migrate a small test program from C++ and MFC to C# and .NET. It seems that some things are easier with the .NET Framework while other things are more difficult. I'm having trouble setting the default location and style of a window/control. In MFC I'm used to overriding PreCreateWindow to modify the CREATESTRUCT. Is there some equivalent in the .NET Framework?

Justin Grover

A Indeed, there is! When the .NET Framework needs to register or create a control, it first calls Control.CreateParams to get the create parameters. Control.CreateParams is a read-only property that returns an instance of class, also called CreateParams, that holds the .NET equivalent of CREATESTRUCT. Since CreateParams (the property) is virtual, you can override it. Figure 5 shows a code snippet that demonstrates how. In the snippet, MyControl is created with the WS_BORDER and WS_POPUP styles, as well as WS_EX_TOOLWINDOW. These symbols aren't defined in Windows.Forms, so you must define them yourself using the values in winuser.h. Make sure you call base.CreateParams to get the default CreateParams before modifying them.

Figure 5 CreateParams (Close Figure 5)
//////////////////
// Code snippet showing how to override Control.CreateParams to
// set Windows parameters used to create a window. CreateParams
// is the .NET equivalent of CREATESTRUCT in the Windows API.
//

public class MyControl : Control
{
   // these come from winuser.h
   private const int
      WS_EX_TOOLWINDOW = 0x00000080,
      WS_BORDER  = 0x00800000,
      WS_POPUP   = -2147483648;

   protected override CreateParams CreateParams
   {
      get
      {
         CreateParams cp = base.CreateParams;
         cp.Style = WS_BORDER|WS_POPUP;
         cp.ExStyle = WS_EX_TOOLWINDOW;
         return cp;
      }
   }
   •••
}

Figure 6 shows the various create parameters. Usually you should use CreateParams only for more obscure features such as WS_EX_TOOLWINDOW, since the Control and Form classes provide direct properties for common features. For example, if all you want to do is change the starting size or location, you can use Control.Size and Control.Location without overriding CreateParams. For Forms, Form.DesktopLocation is better (see my April 2003 column). Unlike MFC, the .NET Framework lets you modify these properties before you actually create the control/form. Whatever values you set will be used. Happy programming!

Figure 6 CreateParams Properties (Close Figure 6)

Property | Meaning
Caption | Gets or sets the control's initial text
ClassName | Gets or sets the name of the Windows class to derive the control from
ClassStyle | Gets or sets a bitwise combination of class style values
ExStyle | Gets or sets a bitwise combination of extended window style values
Height | Gets or sets the initial height of the control
Param | Gets or sets additional parameter information needed to create the control
Parent | Gets or sets the control's parent
Style | Gets or sets a bitwise combination of window style values
Width | Gets or sets the initial width of the control
X | Gets or sets the initial left position of the control
Y | Gets or sets the top position of the initial location of the control

Send your questions and comments for Paul to cppqa@microsoft.com.

Paul DiLascia is a freelance writer, consultant, and Web/UI designer-at-large. He is the author of Windows++: Writing Reusable Windows Code in C++ (Addison-Wesley, 1992). Paul can be reached at http://www.dilascia.com.


Talking To...: Steve Lombardi Discusses the Highlights of Microsoft MapPoint 2004

Talking To...
Steve Lombardi Discusses the Highlights of Microsoft MapPoint 2004

MapPoint is a business mapping software that integrates with Microsoft Office to help organizations track their business opportunities. On the heels of its latest release, MSDN Magazine spoke to Steve Lombardi from the MapPoint product group.

MSDN What's new with MapPoint 2004?

Lombardi The basemap, demographic, and point-of-interest data in the product is all new. A mapping tool is only as good as the data that ships with it. MapPoint 2004 (with versions for Europe and North America) includes the latest data available from our data suppliers. We've also added consumer profile data, ideal for market analysis where understanding buying patterns and consumer behavior is needed. For instance, with just a few lines of code you can include a shaded map in your application indicating where households have cable TV versus satellite, which is highly valuable for someone in the cable industry planning a marketing campaign. MapPoint 2004 also has support for more GPS devices, as well as real-time GPS positioning. The enhanced GPS support, along with existing features such as multipoint driving directions, "do not drive" zones, drive-time polygons, and Yellow Page listings make MapPoint a great tool in fleet management.

MSDN What can developers do with MapPoint 2004?

Lombardi There are two ways developers can utilize MapPoint 2004 in their applications. First, it is easy to use Visual Basic for Applications (VBA) to quickly customize the UI. This is a great option if the MapPoint UI is close to what you need but you want to create custom dialogs or automate tasks such as data import. Second, the included MapPoint ActiveX control allows you to build mapping into standalone applications. Developers and ISVs have been building some great vertical applications for industries such as crime analysis, advertising, and insurance to name a few. Whatever approach you take, you have programmatic access to all of the rich functionality in MapPoint. For instance, it's very straightforward to write a custom application that reads sales data from a SQL database or local data source such as an Excel spreadsheet, and have MapPoint generate a thematic map showing territories where sales are strong. Imagine an executive dashboard that could show underperforming markets in real time. MapPoint has even been integrated in point-of-sale systems for delivery optimization and customer management. However, if MapPoint 2004 doesn't meet your requirements, the MapPoint Web Service might.

MSDN What is the MapPoint Web Service?

Lombardi The MapPoint Web Service is a hosted XML Web Service that exposes its functionality via a SOAP API. It's loaded with features including interactive mapping, driving directions, geocoding, reverse-geocoding, and much more. All the functionality and geographic data needed in most location-based apps can be found here, without building this infrastructure yourself. The MapPoint WSDL can be added to any Visual Studio .NET project. The MapPoint Web Service is the perfect tool for any Web-based app such as a store locator, but since it's an XML Web Service, it works with any other device or platform. Integrating location into existing apps, regardless of their architecture, is now quite easy.

MSDN What about performance and scalability?

Lombardi We back the service with an aggressive Service Level Agreement that guarantees our uptime and performance. The service has been live for nearly two years and is currently handling over 15 million transactions per day. Some of our largest customers throw a billion requests per year at the service. The great thing about this model is that the same service that supports these mega Web properties can be used for applications with much more modest needs—and since you pay only for the transactions you use, it remains totally cost effective.

MSDN Tell us a bit about Microsoft Location Server.

Lombardi We anticipate shipping Microsoft Location Server (MLS) in early 2004. The initial release is focused on enterprise customers who want to track mobile assets to solve business problems. MLS exposes its functionality through a SOAP API. The API has methods for managing privacy and contacts, and proxies to all of the MapPoint Web Service functionality as well.

To illustrate, I built a simple Web app at http://www.whatididwaswrong.com/whereissteve. It displays an interactive map centered wherever my mobile phone happens to be. I created it with just five lines of Visual Basic.

MSDN What's planned for the future of MapPoint technologies?

Lombardi We're focused on delivering the same mapping and location technologies on any device or platform and to making it easy for third parties to do the same thing. To that end, you're going to see more user applications that integrate the MapPoint Web Service with different devices.