 MSDN Magazine October 2006

MSDN Magazine October 2006

WCF Essentials: What You Need To Know About One-Way Calls, Callbacks, And Events

Object and component-oriented programming have only one way for clients to call a method, but Windows® Communication Foundation introduces two more. In this article Juval Lowy explains how they work.

Juval Lowy Peer To Peer: Harness The Power Of P2P Communication In Windows Vista And WCF

P2P applications face a number of barriers preventing their wide adoption as a productivity solution. Fortunately Windows Vista improves the situation, as you’ll learn here.

Justin Smith App Fundamentals: Build A Great User Experience With Windows Presentation Foundation

The Windows Presentation Foundation application model distinguishes between standalone and browser applications and between menu driven and hyperlink driven navigation, resulting in a more satisfying experience for your users.

Michael Weinhardt Web Service Workflows: Deploy Distributed Business Processes With Windows Workflow And Web Services

Due to the distributed nature of a business process it makes sense for a workflow to be deployed as a distributed application. See how Windows Workflow and Web Services hold the key.

Israel Hilerio SQL Server 2005: Jazz Up Your Data Using Custom Report Items In SQL Server Reporting Services

Custom report items in SQL Server 2005 Reporting Services address your needs for custom reports without the pain of doing it from scratch.

Teo Lachev Editor's Note: Some Things Are Easier Said Than Done

Executive Editor Josh Trupin gets himself in trouble with live electrical current and a penchant for twiddling.

Joshua Trupin Toolbox: Protecting Code, Persisting Data, and More

This month obfuscating your code, persisting your data, and a good read on SQL Server 2005.

Scott Mitchell CLR Inside Out: IronPython

IronPython, the CLR implementation of the dynamic programming language Python is introduced this month.

James Schementi Inside MSDN: Consuming MSDN Web Services

Get the inside track on how the MSDN team uses Web Services to power MSDN2.

Craig Andera Test Run: Competitive Analysis Using MAGIQ.

The goal of competitive analysis is to compare the overall quality of your software against similar systems. But it’s not easy. Here James McCaffrey accomplishes the goal with the help of a little MAGIQ.

Dr. James McCaffrey and Nasa Koski Cutting Edge: Simplify Task Progress with ASP.NET "Atlas"

Dino Esposito rewrites his task progress bar with the help of ASP.NET “Atlas.”

Dino Esposito Extreme ASP.NET: Control Adapters

Control adapters let you provide alternate renderings of controls for mobile devices. But they can also be used to completely change the rendering of a con¬trol based on browser type, which can be useful in a number of situations.

Fritz Onion Security Briefs: CardSpace, SqlMembershipProvider, and More

This month Keith Brown fields some reader questions on InfoCard turned CardSpace and passwords for SqlMembershipProvider.

Keith Brown Wicked Code: Running ASMX Web Services on STA Threads

Jeff Prosise describes performance problems in an ASMX Web service that relied on legacy COM and Visual Basic 6.0 to perform key processing tasks and the approach he took to find a fix.

Jeff Prosise C++ at Work: IRegistrar, Finding Submenus, and More

This month DLL problems, context menus, MFC strings to managed C++, and more.

Paul DiLascia { End Bracket }: Is Programming an Art?

This month Stanley B. Lippman asks the question: Is programming an art? Find out what he concludes after some serious contemplation.

Stanley B. Lippman


WCF Essentials: What You Need To Know About One-Way Calls, Callbacks, And Events

WCF Essentials

What You Need To Know About One-Way Calls, Callbacks, And Events

Juval Lowy
This article is based on a prerelease version of the .NET Framework 3.0. All information herein is subject to change.
This article discusses:
  • Invocation models in Windows Communication Foundation
  • One-way and callback operations
  • Duplex callbacks and events
  • A publish-subscribe framework
This article uses the following technologies:
.NET Framework 3.0
Code download available at: WCF2006_10.exe (345KB)
Contents
One-Way Operations
Callback Operations
Client Callback Setup
Callback Reentrancy
Events
The Publish-Subscribe Framework
Managing Persistent Subscribers
Event Publishing
Administering Persistent Subscribers
Queued Publishers and Subscribers
Conclusion

T
he classic object and component-oriented programming models offer only a single way for clients to call a method: the client issues a call, blocks while the call is in progress, and then continues executing once the method returns. Windows® Communication Foundation (WCF) supports this classic invocation model, but it also provides built-in support for two additional operation types: one-way calls for fire-and-forget operations and duplex callbacks that let the service call back to the client.

By default, Windows Communication Foundation operations are of the type known as request-reply: the client issues a request in the form of a message and blocks until it gets the reply message. If the service does not respond within a default timeout of one minute, the client will get a TimeoutException. In addition, the proxy will throw an exception on the client side if there were any communication or service-side exceptions. With the exceptions of NetPeerTcpBinding and NetMsmqBinding, all bindings support request-reply operations.

I'll start here by presenting the ways of invoking and calling Windows Communication Foundation operations and the related design guidelines, then illustrate using these operation types as the building blocks of a custom framework for publishing and subscribing to events. In the process, I'll show you a variety of advanced Microsoft® .NET Framework and Windows Communication Foundation programming techniques.

One-Way Operations

There are cases when an operation has no returned values and the client does not care about the success or failure of the invocation. To support this sort of fire-and-forget invocation, Windows Communication Foundation offers one-way operations. After the client issues the call, Windows Communication Foundation generates a request message, but no correlated reply message will ever return to the client. As a result, one-way operations can't return values, and any exception thrown on the service side will not make its way to the client. One-way calls do not equate to asynchronous calls. When one-way calls reach the service, they may not be dispatched all at once and may be queued up on the service side to be dispatched one at a time, all according to the service configured concurrency mode behavior and session mode. How many messages (whether one-way or request-reply) the service is willing to queue up is a product of the configured channel and the reliability mode. If the number of queued messages has exceeded the queue's capacity, then the client will block, even when issuing a one-way call. However, once the call is queued, the client is unblocked and can continue executing while the service processes the operation in the background. This usually gives the appearance of asynchronous calls. All the Windows Communication Foundation bindings support one-way operations.

The OperationContract attribute offers the Boolean IsOneWay property, which defaults to false, meaning a request-reply operation. But setting IsOneWay to true configures the method as a one-way operation:

[ServiceContract]
interface IMyContract
{
   [OperationContract(IsOneWay = true)]
   void MyMethod()
}

Because there is no reply associated with a one-way operation, there's no point in having any returned values or results, and the operation must have a void return type without any outgoing parameters. Since Windows Communication Foundation does not have any hooks into the language compiler and can't verify proper usage at compile-time, it enforces this by verifying the method signature when loading up the host at run time and throwing an InvalidOperationException in case of a mismatch.

The fact that the client doesn't care about the result of the invocation does not mean the client doesn't care if the invocation took place at all. In general, you should turn on reliability for your services, even for one-way calls. This will ensure delivery of the requests to the service. However, with one-way calls, the client may or may not care about the invocation order of the one-way operations. This is one of the main reasons why Windows Communication Foundation allows you to separate enabling reliable delivery from enabling ordered delivery and execution.

Callback Operations

Windows Communication Foundation supports allowing the service to call back to its clients. During a callback, in many respects the tables are turned: the service is the client and the client becomes the service (see Figure 1).

Figure 1 A Callback Operation
Figure 1 A Callback Operation

The client also has to facilitate hosting the callback object. Not all bindings support callback operations. Because of its connectionless nature, HTTP can't be used for callbacks and therefore you can't use callbacks over BasicHttpBinding or WSHttpBinding. Windows Communication Foundation offers callback support for NetTcpBinding and NetNamedPipeBinding because the underlying transport is bidirectional. To support callbacks over HTTP, Windows Communication Foundation provides WSDualHttpBinding, which actually sets up two HTTP channels: one for the calls from the client to the service and one for the calls from the service to the client.

A service contract can have at most one callback contract. Once the callback contract is defined, the clients are required to support the callback and also to provide the callback endpoint to the service in every call. The ServiceContract attribute offers the CallbackContract property of the type Type. You need to set it to the callback contract type and provide the definition of the callback contract, as shown here:

interface IMyContractCallback
{
   [OperationContract]
   void OnCallback();
}

[ServiceContract(CallbackContract = typeof(IMyContractCallback))]
interface IMyContract
{
   [OperationContract]
   void DoSomething();
}
Note that the callback contract need not be marked with a ServiceContract attribute—it is implied.

The client-side imported callback interface will not necessarily have the same name as in the original service-side definition. Instead, it will be have the name of the service contract interface suffixed with the word Callback.

Client Callback Setup

It is up to the client to host the callback object and expose a callback endpoint. The innermost execution scope of the service instance is the instance context:

public sealed class InstanceContext : CommunicationObject, ...
{
   public InstanceContext(object implementation);
   ... // More members
}
All the client needs to do to host a callback object is to instantiate the callback object and construct a context around it:
class MyCallback : IMyContractCallback
{
   public void OnCallback() {...}
}
IMyContractCallback callback = new MyCallback();
InstanceContext context = new InstanceContext(callback);

Whenever interacting with a service endpoint whose contract defines a callback contract, the client must use a proxy that will set up the bidirectional communication and pass the callback endpoint reference to the service. The proxy the client uses must derive from the specialized proxy class DuplexClientBase<T> shown in Figure 2.

Figure 2 DuplexClientBase<T> (Close Figure 2)
public interface IDuplexContextChannel : IContextChannel
{
   InstanceContext CallbackInstance { get; set; }
   ... // More members
}

public class DuplexClientBase<T> : ClientBase<T> where T : class
{
   protected DuplexClientBase(InstanceContext callbackContext);
   protected DuplexClientBase(InstanceContext callbackContext,
      string endpointName);

   public IDuplexContextChannel InnerDuplexChannel { get; }
   ... // More members
}

The client provides the constructor of DuplexClientBase<T> with the instance context hosting the callback object (as well as the service endpoint information as with a regular proxy). The proxy will construct an endpoint around the callback context, while inferring the details of the callback endpoint from the service endpoint configuration. The callback endpoint contract is the one defined by the service contract callback type. The callback endpoint will use the same binding (or transport, actually) as the outgoing call. Windows Communication Foundation will use the client's machine name for the address and even select a port when using HTTP. Simply by passing the instance context to the duplex proxy and using the proxy to call the service will expose that client-side callback endpoint.

When using SvcUtil or Visual Studio® 2005 to generate a proxy class, the tools will generate a class derived from DuplexClientBase<T> as shown in Figure 3. The client constructs a callback instance, hosts it in a context, creates a proxy and call the service, thus passing the callback endpoint reference:

class MyCallback : IMyContractCallback
{
   public void OnCallback() {...}
}
IMyContractCallback callback = new MyCallback();
InstanceContext context = new InstanceContext(callback);

MyContractClient proxy = new MyContractClient(context);
proxy.DoSomething();

Figure 3 Tool-Generated Duplex Proxy (Close Figure 3)
partial class MyContractClient :
    DuplexClientBase<IMyContract>,IMyContract
{
   public MyContractClient(InstanceContext inputInstance) :
      base(inputInstance) {}

   public MyContractClient(InstanceContext inputInstance,
      string endpointName) :
      base(inputInstance,endpointName) {}
   ... // More constructors

   public void DoSomething()
   {
      Channel.DoSomething();
   }
}

Note that as long as the client is expecting callbacks, it can't close the proxy. Doing so would close the callback endpoint and cause an error on the service side when the service tried to call back. It is often the case that the client itself implements the callback contract, in which case the client will typically use a member variable for the proxy and close it when the client is disposed, as shown in Figure 4.

Figure 4 Client Implementing Callback Contract (Close Figure 4)
class MyClient : IMyContractCallback, IDisposable
{
   MyContractClient m_Proxy;

   public void CallService()
   {
      InstanceContext context = new InstanceContext(this);
      m_Proxy = new MyContractClient(context);
      m_Proxy.DoSomething();
   }
   public void OnCallback() {...}

   public void Dispose()
   {
      m_Proxy.Close();
   }
}

The client-side callback endpoint reference is passed along with every call the client makes to the service and is part of the incoming message. The OperationContext class provides the service with easy access to the callback reference via the generic method GetCallbackChannel<T>. What exactly the service does with the callback reference and when it decides to use the reference is completely at the discretion of the service. The service can extract the callback reference from the operation context and store it for later use or the service can use the reference during the service operation to call back to the client. Figure 5 demonstrates the first option.

Figure 5 Storing the Callback References for Later Use (Close Figure 5)
[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
class MyService : IMyContract
{
   static List<IMyContractCallback> m_Callbacks =
      new List<IMyContractCallback>();

   public void DoSomething()
   {
      IMyContractCallback callback =
         OperationContext.Current.GetCallbackChannel<
             IMyContractCallback>();

      if(!m_Callbacks.Contains(callback))
      {
         m_Callbacks.Add(callback);
      }
   }

   public static void CallClients()
   {
      m_Callbacks.ForEach(delegate(IMyContractCallback callback) {
          callback.OnCallback();
      });
   }
}

Using the same definition of the callback contract as shown earlier, the service uses a static generic list to store references to interfaces of the type IMyContractCallback. Because the service does not know which client is calling it and whether the client has called it already, the service checks in every call to see whether the list already contains the callback reference. If the list does not contain the reference, the service adds the callback to the list. The service class also offers the static method CallClients. Any party on the host side can simply use that to call back to the clients:

MyService.CallClients();
Invoked this way, the invoking party is using some host-side thread for the callback invocation. That thread is unrelated to any thread executing an incoming service call.

Callback Reentrancy

The service may also want to invoke the callback reference passed in or invoke the list of callbacks during the execution of a contract operation. However, such invocations are disallowed because by default the service class is configured for single-threaded access: the service instance is associated with a lock, and only one thread at a time can own the lock and access the service instance. Calling out to the clients during an operation requires blocking the service thread while invoking the callbacks. The problem is that processing the reply message from the client once the callback returns requires ownership of the same lock, and so a deadlock would occur.

To avoid a deadlock, if the single-threaded service instance tries to call back to its clients, Windows Communication Foundation will throw an InvalidOperationException. There are three possible solutions. The first is to configure the service for multiple-threaded access, which would not associate it with a lock and would therefore allow callbacks. However, this would also increase the burden on the service developer because of the need to provide synchronization for the service.

The second solution is to configure the service for reentrancy. When configured for reentrancy, the service instance is still associated with a lock, and only a single-threaded access is allowed. However, if the service is calling back to its clients, Windows Communication Foundation will silently release the lock first.

The following code demonstrates a service configured for reentrancy. During the operation execution, the service reaches into the operation context, grabs the callback reference, and invokes it. Control will only return to the service after the callback returns, and the service's own thread will need to reacquire the lock:

[ServiceBehavior(ConcurrencyMode = ConcurrencyMode.Reentrant)]
class MyService : IMyContract
{
   public void DoSomething()
   {
      IMyContractCa llback callback = OperationContext.Current.
         GetCallbackChannel<IMyContractCallback>();
      callback.OnCallback();
   }
}

The third solution that allows the service to safely call back to the client is to have the callback contract operations configured as one-way operations. Doing so enables the service to call back even when concurrency is set to single-threaded, because there will not be any reply message to contend for the lock.

Events

The canonical use for duplex callbacks is with events. Events allow clients to be notified about something that occurred on the service side. The event may result from a direct client call or it may be the result of something the service monitors. The service firing the event is called the publisher, and the client receiving the event is called the subscriber, as shown in Figure 6.

Figure 6 Publishers and Subscribers
Figure 6 Publishers and Subscribers

Although events in Windows Communication Foundation are nothing more than callback operations, by their very nature events usually imply a looser relationship between the publisher and the subscriber than between a client and a service. When dealing with events, the service typically publishes the same event to multiple subscribers. The publisher often does not care about the order of the invocation of the subscribers or about errors the subscribers might have while processing the events. In addition, the service does not care about returned results from the subscribers. Consequently, event-handling operations should have a void return type, should not have any out-going parameters and should be marked as one-way. I also recommend factoring the events to a separate callback contract and not mixing events with regular callbacks on the same contract:

interface IMyEvents
{
   [OperationContract(IsOneWay = true)]
   void OnEvent1();

   [OperationContract(IsOneWay = true)]
   void OnEvent2(int number);

   [OperationContract(IsOneWay = true)]
   void OnEvent3(int number, string text);
}

However, using raw duplex callbacks for events often introduces too much coupling between the publisher and the subscribers. The subscriber has to know where all the publishing services are in the application and connect to them. Any publisher that the subscriber is unaware of will not be able to notify the subscriber of events. This in turn makes adding new subscribers (or removing existing ones) difficult in an already-deployed application. There is no way for the subscriber to ask to be notified whenever anyone in the application raises a particular type of event. In addition, the subscriber must make multiple, potentially expensive calls to each publisher both to subscribe and to unsubscribe. Different publishers may fire the same event, but offer slightly different ways to subscribe and unsubscribe, which of course couples the subscribers to those methods.

Much the same way, the publisher can only notify subscribers it knows about. There is no way for the publisher to deliver an event to whoever wishes to receive it, nor is there an ability to broadcast an event. Moreover, all publishers must have the necessary code to manage the list of the subscribers and the publishing act itself. This code has almost nothing to do with the business problem the service is designed to solve and can get fairly complex if you want to employ advanced features such as concurrent publishing.

Further, the duplex-based callbacks introduce coupling between the lifeline of the publisher and the subscribers. The subscribers have to be up and running in order to subscribe and receive events. There is no way for a subscriber to ask that if an event is fired, the application should create an instance of the subscriber and let it handle the event.

Finally, setting up subscriptions has to be done programmatically. There is no easy administrative way to configure subscriptions in the application or to change the subscriber's preferences when the system is running.

The solution to these problems is to design around them using the publish-subscribe design pattern and decouple the publishers from the subscribers by introducing a dedicated subscription service and a dedicated publishing service in between, as shown in Figure 7.

Figure 7 Publish-Subscribe System
Figure 7 Publish-Subscribe System

Subscribers that want to subscribe to events register with the subscription service, which manages the lists of subscribers and also provides a similar ability to unsubscribe. Similarly, all publishers use the publisher service to fire their events and avoid delivering the events directly to the subscribers. The subscription and publishing services provide a layer of indirection that decouples your system. No longer do the subscribers have any knowledge about the identity of the publishers. They can subscribe to a type of an event and will receive the event from any publisher, since the subscription mechanism is uniform across all publishers. In fact, no publisher has to manage any subscription list and the publishers have no idea who the subscribers are. They deliver the event to the publishing service to be supplied to any interested subscriber.

You can even define two types of subscribers: transient subscribers are in-memory, running subscribers, and persistent subscribers are subscribers that persist on the disk, representing services to invoke when the event takes place. For transient subscribers you can use the duplex callback mechanism as a handy way of passing the callback reference to the running service. For the persistent subscribers, all you need to record is the subscriber address as reference. When the event is raised, the publishing service will call to the persistent subscriber address and deliver the event. Another important distinction between the two types of subscriptions is that you can store the persistent subscription on the disk or in a database. Doing so will persist the subscription across application shutdowns or machine crashes and restarts, thus enabling administrative configuration of the subscriptions. Obviously, you can't save the transient subscription across an application shutdown and you'll need to set up the transient subscription explicitly every time the application starts.

The Publish-Subscribe Framework

The source code available with this article contains a complete publish-subscribe example. In this code, I wanted to provide not just sample publish-subscribe services and clients but also a general-purpose framework that automates implementing such services and adding the support for any application. The first step in building the framework was to factor the publish-subscribe management interfaces and to provide separate contracts for publishing for transient and persistent subscriptions.

For managing transient subscriptions, I defined the ISubscriptionService interface:

[ServiceContract]
public interface ISubscriptionService
{
   [OperationContract]
   void Subscribe(string eventOperation);

   [OperationContract]
   void Unsubscribe(string eventOperation);
}
Note that ISubscriptionService does not identify the callback contract expected by its implementing endpoint. The callback interface is provided in the application by deriving from ISubscriptionService and specifying the desired callback contract:
[ServiceContract(CallbackContract = typeof(IMyEvents))]
interface IMySubscriptionService : ISubscriptionService {}

The sub interface of ISubscriptionService does not need to add operations. The transient subscription management functionality is provided by ISubscriptionService. In each call to Subscribe or Unsubscribe, the subscriber needs to provide the name of the operation (the event) it wants to subscribe to or unsubscribe from. If the caller wishes to subscribe to or unsubscribe from all events, it can pass an empty or null string.

My framework offers an implementation for the methods of ISubscriptionService in the generic abstract class SubscriptionManager<T>:

public abstract class SubscriptionManager<T> where T : class
{
   public void Subscribe(string eventOperation);
   public void Unsubscribe(string eventOperation);
   ... // More members
}

The generic type parameter for SubscriptionManager<T> is the events contract. Note that SubscriptionManager<T> does not derive from ISubscriptionService.

The application needs to expose its own transient subscription service in the form of an endpoint that supports its specific sub interface of ISubscriptionService. To do this, the application needs to provide a service class that derives from SubscriptionManager<T>, specify the callback contract as a type parameter and derive from the specific sub interface of ISubscriptionService:

[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
class MySubscriptionService :
   SubscriptionManager<IMyEvents>,IMySubscriptionService {}
There is no need for any code in MySubscriptionService because IMySubscriptionService does not add any new operations and SubscriptionManager<T> already implements the methods of ISubscriptionService.

Finally, the applications need to define an endpoint for IMySubscriptionService:

<services>
   <service name ="MySubscriptionService">
      <endpoint
         address = "..."
         binding = "..."
         contract= "IMySubscriptionService"
      />
   </service>
</services>

Figure 8 shows how SubscriptionManager<T> manages transient subscriptions. SubscriptionManager<T> stores the transient subscribers in a static generic dictionary called m_TransientStore:

static Dictionary<string,List<T>> m_TransientStore;
Each entry in the dictionary contains the name of the event operation and all its subscribers in the form of a linked list. The static constructor of SubscriptionManager<T> uses reflection to get all the operations of the callback interfaces (the type parameter for SubscriptionManager<T>) and initializes the dictionary to have all the operations with empty lists. The Subscribe method extracts from the operation call context the callback reference. If the caller specifies an operation name, Subscribe calls the helper method AddTransient, which retrieves from the store the list of subscribers for the event. If the list does not contain the subscriber, AddTransient adds it.

Figure 8 TransientSubscribersManagement (Close Figure 8)
public abstract class SubscriptionManager<T> where T : class
{
   static Dictionary<string,List<T>> m_TransientStore;

   static SubscriptionManager()
   {
      m_TransientStore = new Dictionary<string,List<T>>();
      string[] methods = GetOperations();
      Array.ForEach(methods, delegate(string methodName) {
         m_TransientStore.Add(methodName, new List<T>());
      });
   }

   static string[] GetOperations()
   {
      MethodInfo[] methods = typeof(T).GetMethods(
         BindingFlags.Public | BindingFlags.FlattenHierarchy|
         BindingFlags.Instance);
      List<string> operations = new List<string>(methods.Length);

      Array.ForEach(methods, delegate(MethodInfo method) {
         operations.Add(method.Name);
      });
      return operations.ToArray();
   }

   static void AddTransient(T subscriber,string eventOperation)
   {
      List<T> list = m_TransientStore[eventOperation];
      if(list.Contains(subscriber)) return;
      list.Add(subscriber);
   }

   static void RemoveTransient(T subscriber, string eventOperation)
   {
      List<T> list = m_TransientStore[eventOperation];
      list.Remove(subscriber);
   }

      public void Subscribe(string eventOperation)
   {
      lock(typeof(SubscriptionManager<T>))
      {
         T subscriber = OperationContext.Current.GetCallbackChannel<T>();
         if(!String.IsNullOrEmpty(eventOperation))
         {
            AddTransient(subscriber,eventOperation);
         }
         else
         {
            string[] methods = GetOperations();
            Array.ForEach(methods, delegate(string methodName) {
               AddTransient(subscriber,methodName);
            });
         }
      }
   }

      public void Unsubscribe(string eventOperation)
   {
      lock(typeof(SubscribtionManager<T>))
      {
         T subscriber = OperationContext.Current.GetCallbackChannel<T>();
         if(!String.IsNullOrEmpty(eventOperation))
         {
            RemoveTransient(subscriber,eventOperation);
         }
         else
         {
            string[] methods = GetOperations();
            Array.ForEach(methods, delegate(string methodName) {
               RemoveTransient(subscriber,methodName);
            });
         }
      }
   }
   ... // More members
}

If the caller specified an empty string or null for the operation name, Subscribe calls AddTransient for each operation in the callback contract. Unsubscribe operates in a similar manner. Note that the caller can subscribe to all events and then unsubscribe from a particular one.

Managing Persistent Subscribers

For managing persistent subscribers, I defined the IPersistentSubscriptionService interface shown in Figure 9.

Figure 9 IPersistentSubscriptionService (Close Figure 9)
[ServiceContract]
public interface IPersistentSubscriptionService
{
   [OperationContract]
   [TransactionFlow(TransactionFlowOption.Allowed)]
   void PersistSubscribe(
      string address, string eventsContract, string eventOperation);
   [OperationContract]
   [TransactionFlow(TransactionFlowOption.Allowed)]
   void PersistUnsubscribe(
      string address, string eventsContract, string eventOperation);
   ... // More members
}

To add a persistent subscriber, the caller needs to call PersistSubscribe and provide the address of the subscriber, the events contract name, and the specific event operation itself. To unsubscribe, the caller uses PersistUnsubscribe with the same information. Note that IPersistentSubscriptionService does not imply in any way where the subscribers persist on the service side—that is an implementation detail.

The class SubscriptionManager<T> presented previously also implements the methods of IPersistentSubscriptionService:

public abstract class SubscriptionManager<T> where T : class
{
   public void PersistUnsubscribe(
      string address, string eventsContract, string eventOperation);
   public void PersistSubscribe(
      string address, string eventsContract, string eventOperation);
   ... // More members
}
SubscriptionManager<T> stores the persistent subscribers in SQL Server™. Note that SubscriptionManager<T> does not derive from IPersistentSubscriptionService. The using application needs to expose its own persistent subscription service, but there is no need to derive a new contract from IPersistentSubscriptionService because no callback references are required. The application simply derives from SubscriptionManager<T>, specifying the events contract as a type parameter and adding a derivation from IPersistentSubscriptionService:
[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
class MySubscribtionService : SubscriptionManager<IMyEvents>,
   IPersistentSubscriptionService {}
There is no need for any code in MySubscriptionService because SubscriptionManager<T> already implements the methods of IPersistentSubscriptionService.

Finally, the application needs to define an endpoint for IPersistentSubscriptionService:

<services>
   <service name ="MySubscriptionService">
      <endpoint
         address = "..."
         binding = "..."
         contract= "IPersistentSubscriptionService"
      />
   </service>
</services>
The implementation of the methods of IPersistentSubscriptionService by SubscriptionManager<T> is shown in Figure 10, which is similar to Figure 8 except the subscribers are stored in SQL Server, not in-memory in a dictionary.

Figure 10 Persistent Subscribers Management (Close Figure 10)
public abstract class SubscriptionManager<T> where T : class
{
   static void AddPersistent(string address,string eventsContract,
      string eventOperation)
   {
      ... // Uses ADO.NET to store the subscription in SQL Server
   }

   static void RemovePersistent(string address,string eventsContract,
      string eventOperation)
   {
      ... // Uses ADO.NET to remove the subscription from SQL Server
   }

   [OperationBehavior(TransactionScopeRequired = true)]
   public void PersistUnsubscribe(string address,string eventsContract,
      string eventOperation)
   {
      if(!String.IsNullOrEmpty(eventOperation))
      {
         RemovePersistent(address,eventsContract,eventOperation);
      }
      else
      {
         string[] methods = GetOperations();
         Array.ForEach(methods, delegate(string methodName) {
            RemovePersistent(address,eventsContract,methodName);
         });
      }
   }
   [OperationBehavior(TransactionScopeRequired = true)]
   public void PersistSubscribe(string address,string eventsContract,
      string eventOperation)
   {
      if(!String.IsNullOrEmpty(eventOperation))
      {
         AddPersistent(address,eventsContract,eventOperation);
      }
      else
      {
         string[] methods = GetOperations();
         Array.ForEach(methods, delegate(string methodName) {
            AddPersistent(address,eventsContract,methodName);
         });
      }
   }
   // More members
}

If the application needs to support both transient and persistent subscribers for the same events contract, simply derive the subscription service class from both the specialized sub interface of ISubscriptionService and from IPersistentSubscriptionService and expose the two matching endpoints:

[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
class MySubscriptionService : SubscriptionManager<IMyEvents>,
   IMySubscriptionService,IPersistentSubscriptionService {}

Event Publishing

My framework also enables easy implementation of the publishing service, which should support the same events contract as the subscribers and should be the only point of contact known to the publishers in the application. Because the publishing service exposes the events contract in an endpoint, you need to mark the events contract as service contract, even if you only use it for duplex callbacks with transient subscribers:

[ServiceContract]
interface IMyEvents {...}

The publish-subscribe framework contains the helper class PublishService<T> defined as:

public abstract class PublishService<T> where T : class
{
   protected static void FireEvent(params object[] args);
}

PublishService<T> requires as a type parameter the type of the events contract. To provide your own publishing service, derive from PublishService<T> and use the FireEvent method to deliver the event to all subscribers, whether transient or persistent:

[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
class MyPublishService : PublishService<IMyEvents>,IMyEvents
{
   public void OnEvent1() { FireEvent(); }
   public void OnEvent2(int number) { FireEvent(number); }
   public void OnEvent3(int number,string text) {
      FireEvent(number,text);
   }
}
Note that you can use FireEvent to fire any type of event regardless of the parameters because of the use of the params object array.

Finally, the application needs to expose an endpoint for the publishing service with the events contract:

<services>
   <service name ="MyPublishService">
      <endpoint
         address = "..."
         binding = "..."
         contract="IMyEvents"
      />
   </service>
</services>
Figure 11 shows the implementation of PublishService<T>.

Figure 11 PublishService<T> (Close Figure 11)
public abstract class PublishService<T> where T : class
{
   protected static void FireEvent(params object[] args)
   {
      StackFrame stackFrame = new StackFrame(1);
      string methodName = stackFrame.GetMethod().Name;

      PublishPersistent(methodName,args);
      PublishTransient(methodName,args);
   }

   static void PublishPersistent(string methodName,params object[] args)
   {
      T[] subscribers =
         SubscriptionManager<T>.GetPersistentList(methodName);
      Publish(subscribers,true,methodName,args);
   }

   static void PublishTransient(string methodName,params object[] args)
   {
      T[] subscribers =
         SubscriptionManager<T>.GetTransientList(methodName);
      Publish(subscribers,false,methodName,args);
   }

   static void Publish(T[] subscribers,bool closeSubscribers,
      string methodName, params object[] args)
   {
      WaitCallback fire = delegate(object subscriber) {
         Invoke(subscriber as T,methodName,args);
         if(closeSubscribers)
         {
            using(subscriber as IDisposable)
            {}
         }
      };
      Action<T> queueUp = delegate(T subscriber) {
         ThreadPool.QueueUserWorkItem(fire,subscriber);
      };
      Array.ForEach(subscribers,queueUp);
   }

   static void Invoke(T subscriber,string methodName,object[] args)
   {
      Type type = typeof(T);
      MethodInfo methodInfo = type.GetMethod(methodName);
      try { methodInfo.Invoke(subscriber,args); } catch {}
   }
}

To simplify firing the event by the publishing service, the FireEvent method accepts the parameters to pass to the subscribers, yet its caller does not provide it with the name of the operation to invoke on the subscribers. For that, FireEvent accesses its stack frame and extracts the name of its calling method. It then uses the helper method PublishPersistent to publish to all persistent subscribers and the PublishTransient helper method to publish to all transient subscribers. Both publishing methods operate almost identically: they access SubscriptionManager<T> to retrieve their respective subscriber list, they then use the Publish method to fire the event. Next, the subscribers are returned in the form of an array of proxies to the subscribers. That array is passed to the Publish method.

Publish could have simply invoked the subscribers at this point. However, I wanted to support concurrent publishing of the events, so that if any subscriber is undisciplined and takes a long time to process the event, this will not preclude other subscribers from receiving the event in a timely manner. Note that having the event operations marked as one-way is no guarantee of asynchronous invocation, and besides, I wanted to support concurrent publishing even when the event operation is not marked as one-way.

The publish method defines two anonymous methods. The first anonymous method calls the Invoke helper method, which results in firing the event to the individual subscriber provided and then closes the proxy if so specified. Because Invoke was never compiled against the specific subscriber type, it uses reflection and late binding for the invocation. Invoke also suppresses any exceptions raised by the invocation, because they are of no interest to the publishing party. The second anonymous method queues up the first anonymous method to be executed by a thread from the thread pool. Finally, Publish invokes the second anonymous method on every subscriber in the provided array.

Note how uniformly PublishService<T> treats the subscribers—it almost does not matter if they are transient or persistent. The only difference is that after publishing to a persistent subscriber you need to close the proxy. This uniformity is achieved by the helper methods GetTransientList and GetPersistentList of SubscriptionManager<T>. Of these two, GetTransientList is the simpler one, as shown here:

public abstract class SubscriptionManager<T> where T : class
{
   internal static T[] GetTransientList(string eventOperation)
   {
      lock(typeof(SubscriptionManager<T>))
      {
         List<T> list = m_TransientStore[eventOperation];
         return list.ToArray();
      }
   }
   ... // More members
}

GetTransientList looks up in the transient store for all the subscribers to the specified operation and returns them as an array. GetPersistentList faces a bigger challenge. There is no ready-made list of proxies to persistent subscribers—all that is known about them is their address. GetPersistentList therefore needs to instantiate the persistent subscriber proxies as shown in Figure 12.

Figure 12 Creating Persisent Subscribers (Close Figure 12)
public abstract class SubscriptionManager<T> where T : class
{
   internal static T[] GetPersistentList(string eventOperation)
   {
      string[] addresses = GetSubscribersToContractEventOperation(
         typeof(T).ToString(),eventOperation);

      List<T> subscribers = new List<T>(addresses.Length);

      foreach(string address in addresses)
      {
         Binding binding = GetBindingFromAddress(address);
        T proxy = ChannelFactory<T>.CreateChannel(binding,
                                  new EndpointAddress(address));

         subscribers.Add(proxy);
      }
      return subscribers.ToArray();
   }

   static string[] GetSubscribersToContractEventOperation(
      string eventsContract, string eventOperation)
   {
      ... // Uses ADO.NET to query SQL Server for subscribers to the
          // event
   }

   static Binding GetBindingFromAddress(string address)
   {
      if(address.StartsWith("http:") || address.StartsWith("https:"))
      {
         WSHttpBinding binding =
            new WSHttpBinding(SecurityMode.Message,true);
         binding.ReliableSession.Enabled = true;
         binding.TransactionFlow = true;
         return binding;
      }
      // Similar code for the TCP, named pipes and MSMQ bindings

      Debug.Assert(false,"Unsupported protocol specified");
      return null;
   }

   ... // More members
}

To create a proxy for each subscriber, GetPersistentList needs its address, binding, and contract. The contract is, of course, the type parameter for SubscriptionManager<T>. To obtain the addresses, GetPersistentList calls GetSubscribersToContractEventOperation to query the database and return an array of all addresses of the persistent subscribers who subscribed to the specified event. All GetPersistentList needs now is the binding used by each subscriber. For that, GetPersistentList calls the helper method GetBindingFromAddress, which infers the binding to use from the address schema. GetBindingFromAddress treats all HTTP addresses as WSHttpBinding.

Administering Persistent Subscribers

While you can add and remove persistent subscriptions at run-time using the methods of IPersistentSubscriptionService shown in Figure 9, because of their persistent nature it is likely that managing the subscriptions will be done via some kind of administration tool. To that end, IPersistentSubscriptionService defines additional operations that answer various queries of the subscriber store, as shown in Figure 13.

Figure 13 Administration Query Operations (Close Figure 13)
[DataContract]
public struct PersistentSubscription
{
   [DataMember]
   public string Address;

   [DataMember]
   public string EventsContract;

   [DataMember]
   public string EventOperation;
}

[ServiceContract]
public interface IPersistentSubscriptionService
{
   //Administration operations
   [OperationContract]
   [TransactionFlow(TransactionFlowOption.Allowed)]
   PersistentSubscription[] GetAllSubscribers();

   [OperationContract]
   [TransactionFlow(TransactionFlowOption.Allowed)]
   PersistentSubscription[] GetSubscribersToContract(
      string eventsContract);

   [OperationContract]
   [TransactionFlow(TransactionFlowOption.Allowed)]
   string[] GetSubscribersToContractEventType(
      string eventsContract, string eventOperation);
   ... // More members
}

These administration operations utilize a simple data structure called PersistentSubscription that contains the address of the subscriber, the subscribed contract, and the event. My publish-subscribe framework includes a sample persistent subscription administration tool called Persistent Subscription Manager, shown in Figure 14.

Figure 14 Persistent Subscription Manager
Figure 14 Persistent Subscription Manager(Click the image for a smaller view)
Figure 14 Persistent Subscription Manager
Figure 14 Persistent Subscription Manager (Click the image for a larger view)

The tool uses IPersistentSubscriptionService to add or remove subscriptions. To add a new subscription, you need to provide the tool with the metadata exchange address of the events contract definition. You can use the metadata exchange address of the persistent subscribers themselves or the metadata exchange address of the publish service because they are polymorphic. Enter the metadata exchange base address in the MEX Address textbox and click the Lookup button. The tool will programmatically retrieve the metadata of the event service and populate the Contract and Event comboboxes.

To subscribe, provide the address of the persistent subscriber and click the Subscribe button. Persistent Subscription Manager then adds the subscription by calling to the subscription service (the MySubscriptionService service in the examples so far). The address for the subscription service is maintained in the Persistent Subscription Manager config file.

Queued Publishers and Subscribers

Instead of using synchronous binding for either publishing or subscribing to the events, you can use NetMsmqBinding. Queued publish-subscribe events combine the benefits of a loosely coupled system and the flexibility of disconnected execution. When using queued events, all events on the contract must, of course, be marked as one-way operations. As shown in Figure 15, you can use queuing at either end independently.

Figure 15 Queued Publish-Subscribe
Figure 15 Queued Publish-Subscribe

You can have a queued publisher and connected synchronous subscribers. You could have a connected publisher publishing to queued subscribers or you could have both queued publishers and queued subscribers. Note, however, that you can't have queued transient subscriptions because there is no support with the MSMQ binding for duplex callbacks. As before, you can use the administration tool to manage the subscribers and the administration operations are still connected and synchronous.

To utilize a queued publisher, the publishing service needs to expose a queued endpoint using the MSMQ binding. When firing events at a queued publisher, the publishing service can be offline or the publishing client itself can be disconnected. Note that when publishing two events to a queued publishing service there are no guarantees as to the order of delivery and the processing of these events by the end subscribers. You can only assume the order of publishing when the events contract is configured for a session and only when dealing with a single publishing service.

To deploy a queued subscriber, the persistent subscribing service needs to expose a queued endpoint. Doing so will enable it to be offline even when the publisher is online. When the subscriber is connected again, it will receive all its queued events. In addition, queued subscribers are ideal for when the publishing service itself is disconnected, because no events are lost. When multiple events are fired at a single queued subscriber, there are no guarantees as to the order of delivery of the events. The subscriber can only assume the order of publishing when the events contract is configured for a session. Of course, having both a queued publisher and subscriber allows them to work offline at the same time.

Conclusion

Windows Communication Foundation offers built-in support for powerful and useful programming models, using request-reply, one-way operations or duplex callbacks. Understanding the operations and choosing the right calling mode is one of the first and most important design decisions you will make, and yet applying each mode is surprisingly easy. You can use these operation types as-is in your service-oriented applications or compose your own higher-level abstractions such as the publish-subscribe pattern.

Juval Lowy is a software architect with IDesign providing WCF training and WCF architecture consulting. He is currently working on a comprehensive book on WCF. He is also the Microsoft Regional Director for the Silicon Valley. Contact Juval at www.idesign.net.

© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


Peer To Peer: Harness The Power Of P2P Communication In Windows Vista And WCF

Peer To Peer

Harness The Power Of P2P Communication In Windows Vista And WCF

Justin Smith
This is article based on a prerelease version of Windows Vista and the .NET Framework 3.0. All information is subject to change.
This article discusses:
  • Peer-to-peer networking basics
  • P2P support in Windows Vista and Windows Communication Foundation
  • Developing with PeerChannel
This article uses the following technologies:
Windows Vista, .NET Framework 3.0
Code download available at: P2P2006_10.exe (668KB)
Contents
P2P Basics: Mesh Networks
Peer Name Resolution Protocol
PeerChannel
People Near Me
Conclusion

W
hen most of us think about peer-to-peer (P2P) applications, we instinctively think of instant messaging applications, simple file sharing programs, and games. For the most part, we have been conditioned to default to the client/server model when considering distributed application designs and hardly give P2P models even passing thought, especially for business applications. The primary reason for this focus on the client/server model is simple: P2P application development used to be expensive and time-consuming.

Traditionally, challenges to P2P application development have included the need to develop proprietary protocols for message exchange, having to locate and connect with instances of an application that are hidden behind a Network Address Translation (NAT) or a firewall, and the need to support the inevitable infrastructure required to locate applications in a wide-area network (WAN). These challenges, while surmountable, have represented a substantial barrier, and as a result, many of us never consider the amazing collaborative functionality P2P applications provide.

These barriers will be dramatically reduced with Windows Vista™ and the accompanying .NET Framework 3.0 release. The combination of Windows Vista enhancements to Peer Name Resolution Protocol (PNRP), People Near Me (PNM), and the introduction of PeerChannel in Windows® Communication Foundation have made P2P applications much more approachable. I personally expect increased activity in the P2P arena following the release of Windows Vista.

P2P development in Windows Vista is a huge topic, and no single article can cover it fully. So instead of attempting the impossible, I'll introduce some of the different P2P technologies in Windows Vista, and provide background for your P2P development efforts.

In addition to assuming you have a basic understanding of Windows Forms, I also assume you are somewhat familiar with writing Windows Communication Foundation applications. If not, you may want to get started by reading some of the content either in the Windows SDK or at wcf.netfx3.com/content/resources.aspx.

P2P Basics: Mesh Networks

Before diving into specific P2P technologies, it is important to examine some basic principles of P2P applications. For starters, a P2P application is one that connects directly with other instances of the application. In P2P language, each instance of the application is called a node. A named and connected grouping of these nodes is often referred to as a mesh. As a result, technologies that facilitate P2P application development are frequently referred to as mesh technologies. PNRP, PeerChannel (in Windows Communication Foundation), and PNM are all examples of mesh technologies in Windows Vista.

Mesh Topologies All Windows Vista mesh technologies produce meshes that have roughly the same topology. A mesh topology is generally an abstraction of the connection pattern amongst the nodes in the mesh. To illustrate, imagine, in your mind's eye, a mesh. I'll bet you imagined a mesh somewhat similar to the one in Figure 1.

Figure 1 Fully Connected Mesh
Figure 1 Fully Connected Mesh

Every one of the four nodes in the mesh in Figure 1 is connected to all of the other nodes in the mesh. In other words, if there are N nodes in the mesh, each node maintains N-1 connections. A mesh that meets this criterion is considered to be fully connected. Fully connected meshes are seldom a good idea; to see why, let's consider the connection between nodes.

Nodes in a mesh typically communicate using existing and familiar transports. Like all modern operating systems, Windows Vista leverages TCP/IP and UDP for network communication. If TCP/IP is the chosen transport of a fully connected mesh, then each node in a fully connected mesh of N nodes must create or accept N-1 sockets. As N increases, this model clearly becomes infeasible. For example, if you consider the case of N=1000, then each node would need to maintain 999 sockets, and this simply won't work.

To solve the problems of scalability and WAN connectivity, you must look to a mesh that is partially connected, like the one in Figure 2. As the name implies, nodes in a partially connected mesh are connected only to a few of the other nodes in the mesh. In P2P terms, these adjacent nodes are called neighbors. In general, a partially connected mesh places fewer resource demands on each node, thereby dramatically increasing the scalability of a mesh. In theory, a partially connected mesh can grow to include all of the applications on all of the computers in the world.

Figure 2 Partially Connected Mesh
Figure 2 Partially Connected Mesh

Joining a Mesh The way a node joins a mesh depends on the mesh technology used, but generally speaking, a prospective node must use the mesh name to resolve the physical addresses of one or more nodes already in the mesh. If you assume a partially connected mesh, the result of the mesh name resolution is a subset of the physical addresses available in the mesh. Upon receipt of the physical addresses of one or more physical nodes in the mesh, the prospective node must then connect to one, some, or all of those addresses. After connecting to the mesh, the newly added node must then ready itself to respond to subsequent mesh name resolutions from other prospective nodes.

Mesh name resolution is a complex topic. Much of this complexity is due to the fact that, in many cases, the resolution of a mesh name depends on one or more additional meshes. To illustrate, consider the mesh used by the U.S. Postal Service. More specifically, let's assume that I need to send a package to my friend Rusty. To send the package, I may need to go to a post office. If I do not know the location of the post office nearest to me, I may go to the Internet and look up the address of the nearest one. In an abstract sense, "connecting" to the U.S. Postal Service mesh required me to first go to the biggest mesh of all, the Internet, to resolve the address of the nearest node. In other words, you may use a mesh to resolve addresses contained in another mesh. I will discuss this concept more in the PNRP section of this article.

Communicating with Other Nodes Once a node is connected to a mesh, it may communicate with other nodes through one of two means: mesh flooding (also called multiparty messaging), or directional messaging. As the name implies, mesh flooding is an attempt to send a message to all nodes in the mesh. In general, a node in a mesh can propagate a message to all other nodes by sending the message to all of its neighbors. Upon receiving this message, the neighbor of the sending node is responsible for forwarding the message to its neighbors, and so on. In contrast, directional messaging is an attempt to direct a message to a particular node in the mesh. In a partially connected mesh, the initial sending node may not be connected to the intended recipient. If this is the case, then the initial sending node must send the message to one or more of its neighbors. A neighbor may be connected to the intended recipient. If so, then the neighbor can forward the message to it. If not, then the neighbor makes a guess about which of its neighbors may itself be connected to the intended recipient.

Meshes are seldom static. In most P2P applications, nodes may frequently join and leave the mesh, either due to changes in network connectivity, or, in the case of an instant messaging application, due to a user starting and stopping the application. In addition to the natural changes in a mesh, most mesh technologies have some mechanism to maintain themselves. In general, the goal of mesh maintenance is to repair or tune the mesh so that it operates more efficiently or is more robust. It is important to note that each mesh technology implements mesh maintenance differently.

Peer Name Resolution Protocol

As implied by its name, PNRP is designed to resolve physical addresses based on, among other things, a mesh name. PNRP is available for Windows XP Service Pack 1 (SP1) with the Advanced Networking Pack, Windows XP SP2, and Windows XP Professional x64 Edition. Windows Vista will also ship with PNRP version 2. At the simplest level, PNRP is itself a P2P application that takes the form of a Windows service, and the mesh of PNRP nodes is used exclusively for discovering the physical addresses of nodes participating in other meshes. (See the sidebar "Installing PNRP on Windows XP" for more on this.)

PNRP and IPv6 PNRP is built on Internet Protocol version 6 (IPv6). Since IPv6 is fairly new to most developers, it is important that I mention at least one significant aspect of IPv6 before I discuss the mechanics of PNRP. In IPv6, an address is a 128-bit value (which allows for approximately 3.4×1038 address combinations). This size of the IPv6 address pool enables one of the most important features of IPv6, end-to-end addressing, even when these addresses are carved into multiple subnets, and tucked away behind a NAT. For more information about IPv6, and the technologies that allow its use in an IPv4 infrastructure, see microsoft.com/technet/itsolutions/network/ipv6/introipv6.mspx.

A PNRP Example The PNRP function prototypes, structures, and error codes are defined in the p2p.h header file of the Windows SDK. If an application wishes to register a mesh name with PNRP, it must do so through the Windows API via unmanaged code, or in managed code through the P/Invoke facilities of the common language runtime (CLR). Currently, there is no managed wrapper included in the .NET Framework for the PNRP part of the Windows API. You can, however, access PRNP by using the netsh command-line utility. Through netsh, you can register a new PNRP name with PNRP, as shown here:

c:\temp>netsh
netsh>p2p pnrp peer
netsh p2p pnrp peer>add 0.justinsmith
Ok.

The command-line parameter 0.justinsmith is the P2P name. When this command executes, the PNRP infrastructure generates a PNRP ID, associates that PNRP ID with the P2P name, and assigns that PNRP ID an IPv6 and IPv4 address. If you go to another machine that has PNRP installed and started on it, you can resolve the mesh name 0.justinsmith with the netsh command here:

netsh p2p pnrp peer>resolve 0.justinsmith
Resolve started...
Found:  Comment:    gonzo
        Addresses:  [0000:0000:0000:0000:0000:0000:0000:0001]:8350 udp
                    192.168.42.100:8350 tcp

The output of the resolve command requires some explanation. First, the Comment field represents the machine name that 0.justinsmith was registered on (I name my machines after the Muppets). This field is automatically populated from netsh, and cannot be used as part of the resolution process. Secondly, notice the IPv6 and IPv4 addresses assigned to the node. This is a feature of both netsh and the Teredo transition technology that allows IPv6 traffic over an IPv4 network. Admittedly, I have just begun to scratch the surface of PNRP, but I have shown that PNRP allows me to use a P2P name to resolve an IP address. For more information about PNRP, see microsoft.com/technet/prodtechnol/winxppro/deploy/p2pintro.mspx.

PeerChannel

One of the major benefits of Windows Communication Foundation is that it offers a universal programming model for many different flavors of distributed applications. For example, the code required to write a distributed application that communicates via binary encoded messages over TCP/IP is incredibly similar to the code required to write a distributed application that communicates via WS-* compliant, interoperable messages over HTTP. One of the lesser-known features of Windows Communication Foundation is its support for building P2P applications using this same universal programming model. Because of its support for P2P applications, one may look at Windows Communication Foundation as a mesh technology, but in reality, only the PeerChannel Windows Communication Foundation module is dedicated to building P2P applications. For this reason, the term PeerChannel is commonly used to refer to the P2P capabilities of Windows Communication Foundation. Regardless of the way you refer to it, the PeerChannel in Windows Communication Foundation hides virtually all of the complexity traditionally associated with developing a P2P application and is, in my view, a game changer in the P2P application development arena.

The PeerChannel Mesh The PeerChannel mesh is designed for message flooding. However, PeerChannel contains mechanisms that can propagate a message to part of the mesh, rather than to the entire mesh. For this reason, it is more accurate to say that a PeerChannel mesh is designed for multiparty messaging.Installing PNRP in Windows XP

By default, PNRP is not installed on Windows XP. To install it, start the Add/Remove Programs applet in Control Panel, select Add/Remove Windows Components, select Networking Services. Next, click the Details button, then check the Peer Networking checkbox, and select OK/Next. These steps install several P2P Windows Services (PNRP is among them), but do not configure them to start automatically. To start PNRP, go to the Service Control Manager, and start the Peer Networking service. This will start, among other things, the PNRP service.

A P2P application can register many nodes with the PNRP service, and each node in the PNRP mesh is composed of a P2P Name and a PRNP ID. Generally speaking, a P2P Name represents a mesh name (like a mesh name in a PeerChannel application), and is dictated by the P2P application, while the PNRP ID is assigned to a registered node by the PNRP infrastructure. The PNRP mesh is designed for directional messaging, and uses PNRP IDs to structure the mesh.

The structure of the PeerChannel mesh is dictated by the number of neighbors each node is connected to. To this end, a PeerChannel mesh actively maintains the structure of the mesh. The effect of this maintenance is a robust, evenly distributed mesh. To be more specific, a node in the mesh strives to maintain between two and seven connections to neighbors. These thresholds represent a balancing act between resource demands on the local node and maintaining a robust mesh.

If a node enters the mesh with three neighbors, and two of its neighbors leave the mesh, then that node will start a maintenance cycle in an attempt to acquire new connections to neighbors. Likewise, a node with less than seven neighbors will accept new connections until it has seven connections to neighbors. A PeerChannel node is considered ideally connected when it has three neighbors, but a node will accept up to seven neighbors so that a node that has fallen below the minimum threshold of neighbors can quickly obtain new neighbors. It is important to note that your application code cannot change these thresholds or exert any control over the maintenance of the mesh. These details are taken care of entirely by the PeerChannel infrastructure on a node-by-node basis.

PeerChannel offers both PNRP and custom resolvers as means for a prospective node to discover the addresses of nodes already in the mesh. Regardless of the chosen method of resolution, the general idea is the same: pass the name of the mesh to the resolver and receive a list of IP addresses of other nodes in the mesh. Once the resolution produces a list of addresses, the prospective PeerChannel node concurrently connects to each of the addresses. When a node already in a PeerChannel mesh receives one of the connection requests, it either accepts or refuses the connection. If the connection is accepted, the existing node sends the newly joined node a welcome message that contains, among other things, a list of the addresses of other nodes in the mesh. If the connection is refused, then the existing node sends the prospective node a refusal message that contains a reason for refusal and a list of the addresses of other nodes in the mesh.

The important point here is that mesh name resolution (via PNRP or a custom resolver) is not the only way a list of addresses is returned to a prospective node in PeerChannel. This feature allows nodes to become ideally connected more quickly than if mesh name resolution was the only means by which a prospective node could obtain an address. Also, this feature allows nodes in the mesh to exert control over how many neighbors a node has, which in turn, impacts the robustness of the mesh.

Communication within a PeerChannel mesh is tuned to keep repetitive message passing to a minimum. When a node in the mesh sends a message to the mesh, it is really sending a message to its neighbors. Upon receipt of a message, each neighbor inspects the message and then forwards the message to its neighbors. If a PeerChannel node receives a message from a neighbor, it will not forward that message back to that neighbor. Also, if a PeerChannel node frequently receives a message from a neighbor that it has previously received and processed, the connection to that neighbor may be terminated in the next maintenance cycle. These features are implemented by a local cache on each node. Internally, each node in a PeerChannel mesh caches the value of the WS-Addressing Message ID and an identifier of the neighbor that delivered the message. A node checks this cache when deciding which neighbors to deliver the message to. The combination of these features results in a mesh that is tuned to deliver messages to nodes in the mesh with a minimum of repetition and network bandwidth consumption.

As I mentioned previously, a PeerChannel node can also send a message to a subset of the nodes in the mesh. This is accomplished by assigning a hop count to a message, which is actually a way to track the number of nodes the message has been forwarded through. Do not confuse this mechanism with directional messaging, in which a message is targeted to a particular node. Rather, a hop count is a way to draw a fuzzy boundary around the nodes that will receive the message. For example, if a PeerChannel node (Node A) has three neighbors and sends a message to the mesh with a hop count of 1, then that message will be delivered to three nodes. Likewise, if each of Node A's neighbors has three unique neighbors, and Node A sends a message to the mesh with a hop count of 2, then the message will be delivered to nine nodes. However, if any of Node A's neighbors have common neighbors, then this number will decrease proportionately.

Physically, a hop count is expressed in a message as an integer in a header block. When a node receives a message that has a hop count, it inspects the value of the hop count. If the value is greater than zero, the node decrements the hop count monotonically, and forwards the message (with the decremented hop count value) on to the appropriate neighbors. If a received message contains a hop count of zero, then the message is not forwarded. It is also important to note that the hop count header block is excluded from message signing, so changing this value does not impact the integrity of the digital signature applied to the message, and prevents the overhead associated with repeatedly generating digital signatures and serializing them into the appropriate parts of the message.

PeerChannel Example Let's build a simple P2P application, called PictureViewer, using PeerChannel and Windows Forms. The purpose of the application, as you can tell from its name, is to allow all nodes in the mesh to view the same picture. At a high level, the steps required to build this application are as follows:

  1. Define the basic Windows Forms boilerplate code.
  2. Add controls to the form.
  3. Define the necessary Windows Communication Foundation service contract.
  4. Write the Windows Communication Foundation code required to connect to and receive messages from the mesh.
  5. Write the code required to send a message to other nodes in the mesh.
Figure 3 shows the finished app. Steps 1 and 2 are necessary when developing any Windows Forms application, so I won't show them here. As with any Windows Communication Foundation application, the first development step is to define the service contract. A service contract that will be used by PeerChannel is similar to other Windows Communication Foundation contracts except that PeerChannel requires that all OperationContractAttribute annotations set the IsOneWay instance property to true. This property states that the receiver of the message should not send a reply. If you want the receiver to send a reply, you may define the service contract as a duplex contract, but each OperationContractAttribute annotation must still set the IsOneWay instance property to true. For the purposes of this example, I won't create a duplex contract (there are several examples of duplex contracts in the Windows SDK). The contract I will use is shown here:
[ServiceContract]
interface IPictureViewer {
    [OperationContract(IsOneWay = true)]
    void SharePicture(Stream stream);
}

Figure 3 The PictureViewer P2P Application
Figure 3 The PictureViewer P2P Application(Click the image for a smaller view)
Figure 3 The PictureViewer P2P Application
Figure 3 The PictureViewer P2P Application (Click the image for a larger view)

Notice that the SharePicture interface method is annotated with the OperationContractAttribute attribute, and the IsOneWay instance property is set to true. The SharePicture operation accepts a System.IO.Stream as a parameter because this operation will be used to propagate the bytes of a picture to other nodes in the mesh.

With our service contract defined, it is now time to add the Windows Communication Foundation code that will connect our application to the PeerChannel mesh and passively wait for messages from the mesh. First, I implement my newly defined service contract in the Form. Then, I define a field of type ServiceHost. Received messages will be dispatched to a single instance of the frmPictureViewer type. To indicate this functionality, I must assign the proper ServiceBehavior to the frmPictureViewer type. These two steps are shown in Figure 4.

Figure 4 Implementing the Service Contract (Close Figure 4)
[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single)]
public partial class frmPictureViewer : Form, IPictureViewer
{
    // host the receiver
    ServiceHost host;

    // method to be invoked upon receipt of a message
    public void SharePicture(Stream stream)
    {
        // get the image
        Image image = Bitmap.FromStream(stream);

        // show the image in the picture box on the form
        pbView.SizeMode = PictureBoxSizeMode.StretchImage;
        pbView.Image = image;
    }

    ... // other members elided for clarity
}

Next, I must instantiate a ServiceHost, add an endpoint, and begin listening for incoming messages. Since I am building a Windows Forms application, the logical place to do this is the constructor of the form, as shown in Figure 5.

Figure 5 Attempt to Join Mesh and Listen for Messages (Close Figure 5)
public frmPictureViewer()
{
    InitializeComponent();
    StartReceiving();
}

private void StartReceiving()
{
    // define the meshname and set up the peer binding
    // with the PNRP resolver
    Uri meshAddress = new Uri("net.p2p://pictureView");
    NetPeerTcpBinding binding = new NetPeerTcpBinding();
    binding.Resolver.Mode = PeerResolverMode.Pnrp;
    binding.Security.Transport.CredentialType =
        PeerTransportCredentialType.Password;
    binding.MaxReceivedMessageSize = 700000L;

    host = new ServiceHost(this);
    host.AddServiceEndpoint(typeof(IPictureViewer), binding,
        meshAddress);

    // define the password and get a cert for digsig
    host.Credentials.Peer.MeshPassword = "JustinSmith";
    host.Credentials.Peer.Certificate = GetCertificate();

    // attempt to join and listen for messages
    host.Open();
}

At this point, I have done all I need to in order to connect to the mesh and listen for messages. When compared to standard Windows Communication Foundation code, the only things that are different are the scheme of the Uri (net.p2p), the binding (NetPeerTcpBinding) that I am using, and the addition of password-based security. It is important to note that I have opted to put the mesh password directly in the code. Don't do this in your live applications if you want the mesh password to be a secret.

As soon as I call ServiceHost.Open, our application will attempt to resolve the mesh name (pictureView) via PNRP. At this point, I can verify that our PeerChannel application is using PNRP by running a netsh command to list the registered peer names. If PNRP can resolve the mesh name to one or more IP addresses, then our application will attempt to connect to those nodes. If not, then that node will be the first node in the mesh. As stated previously, existing nodes will either accept or reject the connection by sending either a welcome message or a refused message. The important point here is that this may be happening after the call to ServiceHost.Open returns.

Sending a Message to Other Nodes Before I share a picture, I must first load the picture. The code required to do this is basic Windows Forms code: first, instantiate an OpenFileDialog, get a Stream, convert that Stream into an Image, then reference the Image through the PictureBox.Image property. Wait a minute, isn't that what the SharePicture method does? As a matter of fact, it is. In essence, all I need to do to load the image into the PictureBox is call the SharePicture method, passing the Stream returned from OpenFileDialog.OpenFile as a parameter.

To send a message that contains the picture to the other nodes in the mesh, I must write a few lines of code, but this code is nearly identical to code you would write in any other Windows Communication Foundation application. For starters, I need to define fields in my form of type ChannelFactory<IPictureViewer> and IPictureViewer. Next, I need to instantiate these variables in the constructor of the form. These steps are shown in Figure 6.

Figure 6 Creating the Sending Infrastructure (Close Figure 6)
ServiceBehavior(InstanceContextMode = InstanceContextMode.Single)]
public partial class frmPictureViewer : Form, IPictureViewer
{
    ChannelFactory<IPictureViewerClient> channelFactory;
    IPictureViewer channel;

    public frmPictureViewer()
    {
        InitializeComponent();
        StartReceiving();
    }

    private void StartReceiving()
    {
        // other code elided for clarity
        channelFactory = new ChannelFactory<IPictureViewerClient>(
            binding, new EndpointAddress(meshAddress));
        channelFactory.Credentials.Peer.MeshPassword = "JustinSmith";
        channelFactory.Credentials.Peer.Certificate = GetCertificate();
        channel = channelFactory.CreateChannel();
    }

    ... // other code elided for clarity
}

Notice that I must use the same mesh password and certificate (used to create a digital signature of the message) I used when setting up the ServiceHost. Other than that, this code is identical to code required in non-PeerChannel Windows Communication Foundation applications.

Now that I have built my sending infrastructure, I can use it to send a message to the other nodes in the mesh. To do this, I simply write an event handler for the share button, as shown here:

private void btnShare_Click(object sender, EventArgs e)
{
    using(MemoryStream stream = new MemoryStream())
    {
        Image image = pbView.Image;
        image.Save(stream, ImageFormat.Jpeg); // store image in stream
        stream.Position = 0; // reset the position
        channel.SharePicture(stream); // send a message to the mesh
    }
}

In a nutshell, PeerChannel greatly simplifies P2P application development. The fully functioning version of PictureViewer is about 150 lines of source code, and most of that is dedicated to Windows Forms infrastructure. The fully functional application includes a hop count implementation, and is available for download at the MSDN®Magazine Web site.

People Near Me

PNM is a mesh technology integrated into Windows Vista that allows groups of nearby devices and people to discover, connect, invite, and collaborate with each other. PNM is well suited for tasks such as playing a game in a coffee shop with several other nearby people, sharing your desktop with a co-worker, or even connecting to a projector in a conference room. The capabilities offered by PNM are vast, and it is reasonable to assume that once released, the developer community will find new and inventive ways to leverage this technology. It is important to note that PNM is a completely opt-in mesh technology and is turned off by default.

PNM architecture is composed of, among other things, a P2P application called p2phost.exe. When running, this process creates a mesh by connecting to instances of p2phost.exe on other machines. In general, the purpose of this mesh is directional messaging. More specifically, PNM is designed to resolve local nodes, and communicate with a subset of those local nodes. The PNM API is available as part of the Windows API, and for the most part, focuses on configuring the behavior of p2phost.exe.An Example of Real-World People Near Me

PNM, while it contains an executable, requires another application to leverage its capability. The following scenario illustrates how PictureViewer can use PNM to provide collaborative features:

  1. Tom installs PictureViewer on his new notebook computer running Windows Vista. The installer registers PictureViewer with PNM.
  2. Tom takes his notebook computer to a coffee shop and signs in to PNM. When he is signed in, p2phost.exe begins discovering other nodes in the PNM mesh.
  3. Harry, who also installed PictureViewer, is also at the coffee shop with his notebook computer. When Harry signs in to PNM, his instance of p2phost.exe begins discovering other nodes in the PNM mesh.
  4. Harry and Tom were high school classmates and have not seen each other in many years. They recognize each other from across the room, and begin a conversation. Harry offers to show Tom pictures of his family.
  5. When searching for the pictures, Harry realizes that he hasn’t downloaded the latest pictures of his family from his camera, and he offers to show Tom the pictures later.
  6. Tom decides to show Harry some photos of his family, and starts PictureViewer. Since PictureViewer is integrated with PNM, he invites Harry, through PictureViewer, to look at pictures of his family.
  7. After Tom sends the invitation to Harry, Harry receives a notification on his taskbar that Tom has invited him to start PictureViewer.
  8. Harry accepts the invitation, and PNM starts PictureViewer. After PictureViewer starts, pictures of Tom’s family appear on Harry’s instance of PictureViewer.
  9. Harry promises to show Tom the pictures of his family in the future through PictureViewer. To this end, Harry adds Tom to his contact list by selecting menu items in PictureViewer. Tom does the same.
  10. Some time in the future, Harry and Tom are both away on business trips in different cities. Harry signs in to PNM and opens PictureViewer. He immediately notices that Tom has signed in to PNM also.
  11. Since Harry has downloaded the latest pictures of his family, he invites Tom, through PictureViewer, to take a look at the pictures.
  12. 12.Tom receives and accepts Harry’s invitation. Upon the acceptance of the invitation, PNM launches the PictureViewer app. After PictureViewer starts, Tom sees the long-awaited pictures of Harry’s family.
  13. While PictureViewer may not be the most efficient means to share photos, this story sufficiently tells the PNM story. In each of these steps, PictureViewer is leveraging a different part of the PNM API.

Broadly speaking, the main categories of the PNM API include functions, structures, events, and error codes that provide the ability to register an application with PNM, invite others to join a collaboration session, start a registered application, create a persistent contact, and invite a contact that is no longer local. The sidebar "An Example of Real-World People Near Me" illustrates the process. Notice that there is no support for an application to communicate using PNM. In PictureViewer terms, this means that the message passing between Tom and Harry's instances of PictureViewer explained in the sidebar is still handled by PeerChannel.

Conclusion

P2P application development is a very broad topic, and for most developers, a fairly new one. With the release of Windows Vista and the .NET Framework 3.0, the traditional barriers to entry in P2P application development will be dramatically lowered. My guess is that the advances in technology, like PNRP, IPv6, and the advent of new, more productive platforms, like PeerChannel and PNM, will usher in a new era of P2P application development. The end result will be more collaborative applications that offer functionality we can only begin to imagine.

Justin SmithJustin Smith is a trainer and consultant at Wintellect. He is currently writing Applied Windows Communication Foundation Programming for Microsoft Press, scheduled for publication in January 2007, and is the author of Wintellect's Mastering Distributed Applications course. Justin can be reached at justins@wintellect.com.


App Fundamentals: Build A Great User Experience With Windows Presentation Foundation

App Fundamentals

Build A Great User Experience With Windows Presentation Foundation

Michael Weinhardt
This article is based on prerelease versions of WPF and the .NET Framework 3.0. All information herein is subject to change
This article discusses:
  • Elements of a user experience
  • The Windows Presentation Foundation Resource model
  • The Windows Presentation Foundation Application type
  • Designing an XBAP application
This article uses the following technologies:
.NET Framework 3.0, Visual Studio 2005, XAML, and C#
Code download available at: WPF2006_10.exe (214KB)
Contents
Application Type
The Window
The Page Class
Hyperlink Class
NavigationWindow
NavigationService
XAML Browser Applications
Frame
Windows Presentation Foundation Resources
PageFunction
Where Are We?

A
user experience is a sum of its content and the way its content is hosted. In Windows® Presentation Foundation, content is created using standard controls, 2D and 3D graphics, animation, data binding, layouts, styling, and templating. However, the content is fairly useless until it is hosted in a way that allows users to both see and interact with it. The content needs to be packaged within an application and displayed through a window. This is where an application model comes in handy.

The Windows Presentation Foundation application model distinguishes between two application types: standalone and browser. A standalone application displays content through its own windows, dialogs, and message boxes, while a browser application consists of its own pages that are hosted in a browser.

Similarly, Windows Presentation Foundation distinguishes between two styles of navigation: menu driven and hyperlink driven. Menu-driven applications allow users to navigate content and functionality using menu bars, tool bars, windows, and dialogs, just as with any traditional desktop Windows-based application. Hyperlink-driven uses hyperlinks to deliver a navigation experience that resembles Web applications.

It goes without saying that standalone applications naturally support menu-driven navigation and browser applications naturally support hyperlink-driven navigation. But the Windows Presentation Foundation application model lets you mix and match elements of both. In most cases, this involves either partially or completely integrating a hyperlink-driven experience into a standalone application. The combination you use should be based upon the type of experience that will most benefit your users. Once you've decided on the experience you want to deliver, you can use the Windows Presentation Foundation application model to build it.

Application Type

Consider the sample Box Application, shown in Figure 1. This is a standalone, menu-driven application that allows people who need boxes to list, order, view, and delete box orders. To provide this user experience, you need to begin with the most fundamental of all application model building blocks: creating an application.

Figure 1 The Box Application
Figure 1 The Box Application(Click the image for a smaller view)
Figure 1 The Box Application
Figure 1 The Box Application (Click the image for a larger view)

A Windows-based application consists of some standard plumbing, including both an entry point and a message loop, and possibly may also require one or more of the following common application services:

  • Processing command-line parameters
  • Returning an exit code
  • Application-scope state
  • Detecting and responding to unhandled exceptions
  • Managing application lifetime

Windows Presentation Foundation centralizes both plumbing and services within a single type, System.Windows.Application, which you can use from markup (XAML), code (C# or Visual Basic®), or a combination of both (known as markup and codebehind). Application turns out to be so useful that Visual Studio® 2005 automatically adds an instance of one to every new .NET Framework 3.0 (formerly known as WinFX®) Windows Application project:

<!--App.xaml (markup)-->
<Application
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  x:Class="BoxApplicationWindow.App"
/>

// App.xaml.cs (codebehind)
public partial class App : Application { ... }

If you've programmed with a Windows presentation technology before, such as Windows Forms and Win32®, you may be in for a surprise. There isn't a single piece of code that looks remotely like the code used to establish the standard Windows-based application plumbing, including the entry point. This is because the application plumbing is generated for you, which is a result of Visual Studio 2005 configuring your Application markup file as an ApplicationDefinition build action, as shown in Figure 2.

Figure 2 Set an Application XAML File
Figure 2 Set an Application XAML File

Under the covers, the equivalent of this code is generated:

// App.cs
using System;

public partial class App : Application
{
  [STAThread]
  public static void Main()
  {
    // Initialize and run the application
    App application = new App();
    application.Run();
  }
}
What exactly is created doesn't really matter, though, since you neither have to write it nor understand its intricacies. Instead, you are shielded by the most complete application abstraction in a Microsoft presentation technology to date, which you can use to create a running application with only a single piece of markup. All you need to do is use Application's services. For standalone applications, this involves showing a window when an application starts running.

The Window

In Windows Presentation Foundation, a window is a Window. A Window has always been the core unit of content hosting in standalone applications. You can add a Window definition to your project in Visual Studio 2005 by choosing Project | Add New Item | WinFX Window, which generates the following:

<!--MainWindow.xaml (markup)-->
<Window
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  x:Class="BoxApplication.MainWindow"
</Window>

// MainWindow.xaml.cs (codebehind)
using System.Windows;
public partial class MainWindow : Window { ... }

When the window definition is added, Visual Studio 2005 automatically configures the markup file's build type to Page. When built, the markup is turned into a special type of resource that can be identified uniquely by a Uniform Resource Identifier (URI). Essentially, this enables Windows Presentation Foundation to load a Window declaratively from a URI, and you can use this capability to specify a window to be automatically opened when an application starts. You do this by setting the Application.StartupUri attribute in markup, as shown here:

  <!--App.xaml (markup)-->
  <Application ... StartupUri="MainWindow.xaml"  />
This creates and shows a window like the one in Figure 3. Like all windows, Windows Presentation Foundation windows contain a client area (this houses Windows Presentation Foundation content and controls) and a non-client area (a border, a title bar, and the various adornments associated with these).

Figure 3 A Window and Its Parts
Figure 3 A Window and Its Parts

The window that results from Application.StartupUri is modeless, which means it doesn't prevent users from using other windows in the application. Since you haven't created any other windows yet, this isn't so exciting. However, if you need to show other modeless windows, as you'll probably need to do in any nontrivial application, simply call Window.Show:

// MainWindow.xaml.cs (codebehind)
public partial class MainWindow : Window
{
  void helpContentsMenuItem_Click(object sender, RoutedEventArgs e)
  {
    HelpWindow window = new HelpWindow();
    window.Owner = this; // Ensure window always appears above us
    window.Show();
  }
  ...
 }

Windows Presentation Foundation also supports showing a window modally, which means the window prevents other windows in an application from being used. Modal windows are typically (but not always) used as dialogs, to collect data needed for completing a task such as creating a new order. To show a window modally in Windows Presentation Foundation, call Window.ShowDialog (see Figure 4).

Figure 4 Calling Window.ShowDialog (Close Figure 4)
// MainWindow.xaml.cs (codebehind)
public partial class MainWindow : Window
{
  void CreateOrder()
  {
    // Place order
    OrderABoxDialog dlg = new OrderABoxDialog();
    dlg.Owner = this; // Ensure dialog box always appears above us
    bool? dialogResult = dlg.ShowDialog();

    // If order details are fine, add order to orders list
    if (dialogResult == true)
    {
      this.orders.Add(dlg.Order);
    }
  }
  ...
}

The Window class also supports typical dialog behavior, which allows users to accept or cancel a dialog and have that choice returned to the calling code for appropriate processing.

Message boxes are a special type of dialog for displaying information to users, or asking them questions, and is supported in Windows Presentation Foundation with the MessageBox type:

// MainWindow.xaml.cs (codebehind)
public partial class MainWindow : Window
{
  void aboutMenuItem_Click(object sender, RoutedEventArgs e)
  {
    MessageBox.Show("Box Application, Version 1.0");
  }
  ...
}

Message boxes, dialogs, windows, and applications form the core of the standalone, menu-driven application development model. And these have been supported by previous presentation technologies for a long time now. However, Windows Presentation Foundation extends these with hyperlink-driven navigation support that begins with the fundamental unit of navigation content—the page.

The Page Class

Page is the Windows Presentation Foundation analog of the HTML Web page, which helped popularize the Web. As I mentioned, Windows Presentation Foundation supports hyperlink-driven navigation in both standalone and browser applications. The content cornerstone of the hyperlink-driven navigation experience in Windows Presentation Foundation is the Page.

You add a markup and codebehind Page definition to your project in Visual Studio 2005 by choosing Project | Add New File | WinFX Page. This generates code resembling what's shown in Figure 5.

Figure 5 Adding Markup and Codebehind Page Definition (Close Figure 5)
<!--HomePage.xaml (markup)-->
<Page
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  x:Class="BoxApplicationNavigationWindow.HomePage" ... >
  ...
  <!--Order Content-->
  ...
</Page>

// HomePage.xaml.cs (codebehind)
using System.Windows.Controls; // Page
public partial class HomePage : Page { ... }

The Page markup file is configured as a Page build item. As with Window, this is done so it can be loaded from a URI, which means you can configure Application.StartupUri to automatically load a page when an application starts:

<!--App.xaml (markup)-->
<Application ... StartupUri="HomePage.xaml" />

Because the Page class is not a window, and doesn't derive from Window, it can't host itself. Fortunately, the Application class is smart enough to detect when a particular page is set as the StartupUri. If so, Application creates a window in which to host the page.

Hyperlink Class

Any nontrivial hyperlink-driven application will have more than one XAML page, and you'll need to give your users a way to navigate between these pages. You can probably guess that Windows Presentation Foundation enables hyperlink-driven navigation with hyperlinks. You can add hyperlinks to your pages like so:

<!--HomePage.xaml (markup)-->
<Page ... >
  ...
      <Hyperlink NavigateUri=          "OrderingGuidelinesPage.xaml">
    Ordering Guidelines
  </Hyperlink>
  ...
</Page>
This hyperlink is configured to navigate to a XAML page using the same fundamental programming model as HTML HREFs. You specify a URI to navigate to (in this case, OrderingGuidelinesPage.xaml) and the text that users will see and click on to initiate navigation (in this example, "Ordering Guidelines").

Since so much browsable content is located in HTML-based Web pages, it's nice that Windows Presentation Foundation and Hyperlink let you seamlessly navigate to Web-based content. For example, the ordering guidelines already exist on the Box Application's Web site so, rather than replicating them as XAML pages within the application, you can simply change the value of the NavigateUri property:

<!--HomePage.xaml (markup)-->
<Page ... >
  ...
  <Hyperlink NavigateUri="OrderingGuidelinesPage.html">
    Ordering Guidelines
  </Hyperlink>
  ...
</Page>

NavigationWindow

At this point, you might be wondering one of several things. Since Page is not a window, where did the window that's hosting it come from? When a hyperlink is clicked, what actually handles the navigation? And how is HTML Web page content displayed from a Windows Presentation Foundation application? All of these things are taken care of by NavigationWindow.

When you set Application.StartupUri to either a XAML or HTML page, Application (knowing that neither of these can provide its own window) creates an instance of NavigationWindow to host them. NavigationWindow derives from Window and extends its visual appearance to look a little like a browser, as you can see in Figure 6.

Figure 6 Created By NavigationWindow
Figure 6 Created By NavigationWindow

When a user clicks a hyperlink that appears on a XAML page, Hyperlink asks NavigationWindow to navigate to the specified URI. NavigationWindow proceeds to load the page that is located at the URI, thus hosting it. The URI location from which the page was loaded is stored in the NavigationWindow.Source property, while the loaded page content is available from the NavigationWindow.Content property.

When content changes, navigation is considered to have taken place and the previous content is added to the navigation history. This, too, is managed by NavigationWindow. The navigation UI provides two buttons and dropdown lists for navigation. Note that you are not limited to NavigationWindow's default chrome. Using Windows Presentation Foundation's support for styles, you can easily create your own navigation UI.

So far, I've shown you how to use markup for configuring the URIs that hyperlinks should navigate to. Sometimes, though, you can't determine your navigation declaratively. For example, if you want to view an order, you need to create an instance of Page and pass it the order you want to view. This can't be done declaratively. Instead, you need to use code, as shown in Figure 7.

Figure 7 Using Code for Navigation (Close Figure 7)
HomePage.xaml (markup)
<Page ... >
  ...
  <Hyperlink Click="viewHyperlink_Click">
    View
  </Hyperlink>
  ...
</Page>

HomePage.xaml.cs (codebehind)
public partial class HomePage : Page
{
  void viewHyperlink_Click(object sender, RoutedEventArgs e)
  {
    // View Order
    ViewOrderPage page = new ViewOrderPage(GetSelectedOrder());
    NavigationWindow window =
      (NavigationWindow)this.Parent; // Don’t do this!
    window.Navigate(page);
  }

  Order GetSelectedOrder()
  {
    return (Order)this.ordersListBox.SelectedItem;
  }

  ...
}

When the hyperlink is clicked, its Click event handler gets the order that is currently selected, passes it to ViewOrderPage during instantiation, and calls its host NavigationWindow's Navigate method, which then navigates to the page as an object, rather than as a URI.

You may find it odd to have to acquire a reference to the host NavigationWindow. This is necessary because Page does not have explicit knowledge about what is actually hosting it. Page can determine the host using its Parent property, but Parent returns a reference to a DependencyObject, rather than a strongly typed reference to a particular host type. Thus, casting Parent to a specific type implies that Page knows what can host it. But, as you'll discover, Page can have different types of hosts. Consequently, if you intend for your pages to be hosted by multiple types of hosts, you need a host-independent way of programmatically performing navigation.

NavigationService

In Windows Presentation Foundation, independence between a page and a page's host is established by NavigationService, which implements the basic machinery of a navigation engine including navigation, navigation history, navigation lifetime, content, and finding a navigation service for a piece of content. Figure 8 shows the essential members of the NavigationService type.NavigationWindow doesn't actually implement its own navigation engine; it uses its own instance of NavigationService. With that in mind, Page can actually get a reference to the NavigationService of the NavigationWindow that hosts it using GetNavigationService:

// HomePage.xaml.cs (codebehind)
public partial class HomePage : Page
{
  void viewHyperlink_Click(object sender, RoutedEventArgs e)
  {
    // View Order
    ViewOrderPage page = new ViewOrderPage(GetSelectedOrder());
    NavigationService ns =
      NavigationService.GetNavigationService(this);
    ns.Navigate(page);
  }

  Order GetSelectedOrder() { ... }
  ...
}
This lets Page do navigation without any specific knowledge of its host. This requirement is so common that Page provides a special helper property, NavigationService, which has the same effect:
// HomePage.xaml.cs (code-behind)
public partial class HomePage : Page
{
  void viewHyperlink_Click(object sender, RoutedEventArgs e)
  {
    // View Order
    ViewOrderPage page = new ViewOrderPage(GetSelectedOrder());
    this.NavigationService.Navigate(page);
  }

  Order GetSelectedOrder () { ... }
  ...
}

Figure 8 Essential Members of the NavigationService Type (Close Figure 8)
sealed class NavigationService : IContentContainer
{
  // Navigation
  public bool Navigate(Uri source); // Navigate to URI
  public void Refresh(); // Re-navigate to current content
  public void StopLoading(); // Stop current navigation

  // Navigation History
  public bool CanGoBack { get; } // Content in back nav. history?
  public bool CanGoForward { get; } // Content in forward nav. history?
  public void GoBack(); // Go to previous content in nav. history
  public void GoForward();  // Go to next content in nav. history

  // Navigation Lifetime
  // Navigation requested
  public event NavigatingCancelEventHandler Navigating;
  // Navigated to content
  public event NavigatedEventHandler Navigated;
  // Content loaded
  public event LoadCompletedEventHandler LoadCompleted;
  // Navigation error
  public event NavigationFailedEventHandler NavigationFailed;
  // Bytes downloaded
  public event NavigationProgressEventHandler NavigationProgress;
  // Navigation stopped
  public event NavigationStoppedEventHandler NavigationStopped;

  // Content
  public object Content { get; set; } // The currently loaded content
  public Uri CurrentSource { get; }  // The URI for the current content
  public Uri Source { get; set; }   // The URI for the current content
                                   // or, if navigating, the URI for the
                                  // content being navigated to

  // Find a navigation service
  public static NavigationService GetNavigationService(
    DependencyObject dependencyObject);
}

Figure 9 illustrates the relationship between NavigationWindow, NavigationService, and Page. As you can see, NavigationWindow re-implements its NavigationService's Content property. NavigationWindow re-implements the majority of NavigationService's members in this way, and even adds some more. For example, you can enumerate the content in back and forward navigation history via the BackStack and ForwardStack properties, respectively.

Figure 9 The Relationship
Figure 9 The Relationship

Unfortunately, you can't create your own custom type that aggregates NavigationService (even though it is a public type, it has an internal constructor that consequently prevents instantiation). Instead, you have to rely on one of three Windows Presentation Foundation NavigationService aggregators to host content. Collectively, these are known as navigators and include NavigationWindow, Frame, and browsers (only Internet Explorer® versions 6 and 7 for Windows Presentation Foundation version 1.0). When a page is coded to use its own NavigationService property, it can be hosted in all three without change, as Figure 10 illustrates.

Figure 10 Using Code for Navigation
Figure 10 Using Code for Navigation

Perhaps most exciting is that you can take a page hosted within a single standalone application and suddenly you're able to host it anywhere using Internet Explorer.

XAML Browser Applications

NavigationWindow, Page, and Hyperlink provide a nice way to offer browser-style user experiences in a standalone application. In one sense, NavigationWindow is a browser, though without all the frills built into today's browsers like favorites, tabbed browsing, and the like. Since most users are browser savvy, many will feel more comfortable with an application that provides the same level of capability or, indeed, integrates with browsers. If your application will benefit from a browser-hosted and hyperlink-driven environment, Windows Presentation Foundation XAML Browser Applications (XBAPs) are the way to go.

To create an XBAP version of the sample application, create a new .NET Framework 3.0 Web Browser Application in Visual Studio 2005, copy the files from the NavigationWindow version of the sample (excluding the application definition), and voilà. The resulting application will run hosted by Internet Explorer, as shown in Figure 11.

Figure 11 App Hosted by Internet Explorer
Figure 11 App Hosted by Internet Explorer

XBAPs can also be published to and launched from Web servers, on either the intranet or the Internet. This is made possible through ClickOnce Deployment, which is included with the .NET Framework 3.0. To use ClickOnce, MSBuild generates the executable that users ultimately run, as well as two manifest files that ClickOnce requires for downloading and launching the executable. One of these is known as the application manifest; it has the .xbap file extension and is what users actually browse to when they want to launch an XBAP. Note that the launch process is seamless to the user—the experience of browsing to an XBAP application is like browsing to any application running in a browser.

When an application can be launched from the Internet, security is a significant factor. For this reason, XBAPs are not installed. Furthermore, XBAPs leverage the .NET Framework's Code Access Security (CAS) to protect users from malicious code through the use of an enforced security sandbox—XBAPs can only do things that are allowed for apps launched from the Internet zone, a restricted set of operations. Furthermore, if an XBAP attempts to execute functionality that exceeds what is enabled by the Internet zone, an exception is thrown and the application stops executing.

Internet zone permissions prevent a variety of capabilities in Windows Presentation Foundation version 1.0, including showing windows, using the SaveFileDialog, registry access, and accessing the HTML Document Object Model (DOM) via script. While you need to sacrifice features like these to enjoy the benefits of CAS-secured XBAP applications, you still have most of what is really cool about Windows Presentation Foundation at your command.

Frame

Frame embeds browser-style user experiences into content that may be hosted by other navigators (standalone or browser-based, menu- or hyperlink-driven). Given its flexibility, you should be guided by your intended user experience when determining how to use it.

Standalone, menu-driven applications don't provide the best model for navigating through document-style content, such as a help file. A hyperlink-driven approach may be more suitable and links can be easily embedded into a standalone application's window, as shown in Figure 12. This is enabled with the following markup:

    <!--HelpDialog.xaml (markup)-->
<Window ... >
    <DockPanel>
        <TextBlock
          Padding="20,20,20,20"
          DockPanel.Dock="Top"
          TextWrapping="Wrap"
          FontSize="15"
          FontWeight="Bold" >
            Box Application Help
        </TextBlock>
        <Frame Padding="20,0,20,0"           Source="HelpPageContents.xaml" />
    </DockPanel>
</Window>
Alternatively, Frame can be used much like the HTML IFRAME element by being hosted within the content of a Windows Presentation Foundation page, as shown in Figure 13. The markup will look like this:
 <!--HelpPage.xaml (markup)-->
 <Page ... >
     <DockPanel>
         <TextBlock
           Padding="20,20,20,20"
           DockPanel.Dock="Top"
           TextWrapping="Wrap"
           FontSize="15"
           FontWeight="Bold" >
             Box Application Help
         </TextBlock>
         <Frame ... Source="HelpPageContents.xaml" />
     </DockPanel>
 </Page>

Figure 12 Browsable Content in a Standalone Window
Figure 12 Browsable Content in a Standalone Window(Click the image for a smaller view)
Figure 12 Browsable Content in a Standalone Window
Figure 12 Browsable Content in a Standalone Window (Click the image for a larger view)

By default, when a frame is hosted within content that is itself directly or indirectly hosted by another navigator, the frame uses the navigation service that is managed by the parent navigator. This means that the frame's navigation history is really stored in its parent navigator's navigation history, along with the parent navigator's own navigation history. Thus, the frame's navigation history must be navigated all the way back or forward before the parent navigator's navigation history can be navigated. However, this is not a bad thing if your parent navigator hosts content that can be shared across the one or more pages. This is somewhat analogous to ASP.NET's master/child content.

Figure 13 Frame Hosted within Content
Figure 13 Frame Hosted within Content(Click the image for a smaller view)
Figure 13 Frame Hosted within Content
Figure 13 Frame Hosted within Content (Click the image for a larger view)

On the other hand, such navigation can be painful for users when the pages in a frame make up a single logical set of content—for instance, a help file rather than one or more help pages. Once a user goes to Help and navigates to the appropriate help page, he is unlikely to want to navigate back through all the help pages he browsed; instead, he'll probably want to go straight back to the previous parent page. In this situation, you can instruct a frame to use its own navigation history by setting its JournalOwnership property like so:

<Page ... >
  ...
  <Frame ... JournalOwnership="OwnsJournal" />
  ...
</Page>
JournalOwnership is a property you set to determine which "journal" a frame will use (the internal Journal type encapsulates navigation history in Windows Presentation Foundation), and can be one of the JournalOwnership enumeration values:
enum JournalOwnership
{
  Automatic,        // "UsesParentJournal" if hosted by Frame or
                    // NavigationWindow, "OwnsJournal" otherwise.
                    // (default)
  OwnsJournal,      // Frame manages navigation history
  UsesParentJournal // Frame's host manages navigation history
}

When Frame has its own navigation history, it displays its own navigation user interface to visually reinforce the distinction, as shown in

Figure 14 Frame with Its Own Navigation History(Click the image for a smaller view)
Figure 14 Frame with Its Own Navigation History
Figure 14 Frame with Its Own Navigation History (Click the image for a larger view)

Windows Presentation Foundation Resources

So far, I've discussed pages that are embedded into an application assembly. However, content can be loaded from a variety of locations—it can be embedded in the same assembly as the code that uses it, it can be embedded in a referenced assembly, or it can consist of loose content files that are not embedded into any assembly at all. Loose content itself can be located on the local disk, a file share, or even a Web site. And, whether a piece of content is embedded or loose, it doesn't have to be a page; content can include a variety of media, such as images, video, and audio. Finally, content doesn't even have to belong to a particular application. HTML pages that belong to other Web applications are also fair game.

This flexibility allows developers to more easily deal with a variety of real-world scenarios. Sometimes content is specific enough to an application, and an application is so dependent on this content, that it makes the most sense to deploy the content with the application by embedding it right in the assembly. Some applications have content that changes regularly enough that constantly rebuilding assemblies to redeploy new content is impractical, hence the support for loose content (and since loose content can reside in a common location, Internet- and intranet-based XBAP applications can avoid downloading assemblies that aren't needed). Furthermore, sometimes content is shared by more than one application, but there still needs to be a guarantee that it is available.

To enable this flexibility, Windows Presentation Foundation incorporates a special mechanism for uniquely identifying and loading resources. It doesn't matter where content resides or whether the content is embedded or loose. The foundation of this mechanism is the Pack URI scheme, which is an extensible scheme for identifying application resources with unique URIs. Windows Presentation Foundation uses the Pack URI scheme to support several different, but common, scenarios for loading content.

Throughout this article, the sample code has used Pack URIs to identify and load windows and pages, whenever Application.StartupUri and Hyperlink.NavigateUri have been used:

<!--App.xaml (markup)-->
<Application ... StartupUri="HomePage.xaml" />
This example uses the relative version of the Pack URI, which is a nice simplification that allows you to type less. The absolute version of this Pack URI is shown here:
pack://application:,,,/HomePage.xaml

The absolute Pack URI is composed of three key parts: the scheme (pack://), the authority (application:), and the path (,/HomePage.xaml). The authority describes the type of container that has a resource, while the path describes the location of that resource relative to the container. An "application:" container is really an assembly, and the path is the location of a resource relative to the root of the assembly.

Whether an absolute or relative Pack URI is used, the content that it refers to can either be embedded within an assembly or exist as a loose XAML file that is stored in a location relative to the application executable. For a loose XAML page that is in the same folder as the application executable, the Pack URI looks like this:

pack://application:,,,/HomePage.xaml

Interestingly, the Pack URI for the loose XAML file is the same as the Pack URI for an embedded file of the same name. To distinguish between the two, Windows Presentation Foundation uses a basic resolution mechanism that first looks in the assembly for an embedded resource before looking for a loose resource.

A Pack URI can also be used to access Windows Presentation Foundation resources that are embedded in referenced assemblies, although with a slightly different syntax:

pack://application:,,,/BoxApplicationLibrary;component/HomePage.xaml
The relative Pack URI equivalent is:
/BoxApplicationLibrary;component/HomePage.xaml

The Pack URI allows you to both locate and load a resource from the application's site of origin, that is, the location from which it is launched. For XBAP applications launched from a Web server, keeping content current is as simple as dropping new content onto the application's publish location. To access loose resources located at the site of origin, you need to use another special type of Pack URI that can only be used absolutely:

pack://siteoforigin:,,,/HomePage.xaml

For any page (embedded or loose), you can enable navigation to a fragment on the page. This is just like Web-style fragment navigation. You define a fragment on a page by giving a XAML element a Name attribute, like so:

<Page ... >
   <TextBlock Name="Paragraph1" TextWrapping="Wrap">
     ...
   </TextBlock>
</Page>
To navigate to a page fragment, you use yet another special Pack URI, by suffixing the page URI with "#XAMLElementName", like this:
HelpPage3.xaml#Paragraph3

PageFunction

With content coming from a variety of locations, and hyperlink-driven applications allowing navigation to any number of other locations, completing a task can be difficult. This is because hyperlink-driven applications can't easily constrain users to navigating to specific pages. No matter how many hyperlinks an application provides, users can still go anywhere they want using the browser's address bar. As a result, a user can leave the page that initiated processing of a task without knowing whether or not the task was completed. In the Web world, there are plenty of tricks to create pseudo-dialog style behaviors for Web pages, relying on session state and the like. Unfortunately, this requires a lot of overhead. Dialogs do solve the problem, but for security reasons Window cannot be instantiated from applications, such as XBAPs, that run within Internet Zone partial trust.

Fortunately, Windows Presentation Foundation supports modal dialog-style mechanics with page functions. These are encapsulated as the generic PageFunction type, which derives indirectly from Page. Consequently, a PageFunction looks much like a Page, and is also built similarly, as you can see in Figure 15.

Figure 15 PageFunction (Close Figure 15)
<!--OrderABoxPageFunction.xaml (markup) -->
<PageFunction
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:local="clr-namespace:BoxApplicationXBAP"
    x:Class="BoxApplicationXBAP.OrderABoxPageFunction"
    x:TypeArguments="local:Order"
    WindowTitle="Box Application - Order a Box" >
  ...
  <!--Content-->
  ...
<PageFunction>

// OrderABoxPageFunction.cs (code-behind)
public partial class OrderABoxPageFunction:
  PageFunction<Order> { ... }

The purpose of this particular page function is to collect new order information, which is encapsulated by Order. Because tasks typically operate over data in this way, PageFunction is generic and is declared to operate over a specific piece of data, hence the special x:TypeArguments attribute for markup. If the value of x:TypeArguments and the generic PageFunction's type argument don't match, you will get a compilation error.

To call a PageFunction, the calling page needs to instantiate a PageFunction and manually navigate to it:

// HomePage.cs (codebehind)
public partial class HomePage : Page
{
  void orderHyperlink_Click(object sender, RoutedEventArgs e)
  {
    OrderABoxPageFunction pageFunction = new OrderABoxPageFunction();
    pageFunction.Return += new ReturnEventHandler<Order>(
      OrderABoxPageFunction_Returned);
    this.NavigationService.Navigate(pageFunction);
  }
  ...
}
Next, the PageFunction needs to allow a user to complete the page before returning the result to the calling page:
// OrderABoxPageFunction.cs (codebehind)
public partial class OrderABoxPageFunction:
  PageFunction<Order>
{
  void orderHyperlink_Click(object sender, RoutedEventArgs e)
  {
    // Return order
    this.OnReturn(new ReturnEventArgs<Order>(this.order));
  }
  void cancelHyperlink_Click(object sender, RoutedEventArgs e)
  {
    // Cancel order
    this.OnReturn(null);
  }
  ...
}

PageFunction.OnReturn is called to return, passing an instance of the generic ReturnEventArgs. If the task is accepted, this contains an instance of the type over which the PageFunction operates. Otherwise, it's null. To detect PageFunction return, and to acquire the ReturnEventArgs and its data, the calling page needs to handle PageFunction.Returned, as shown in Figure 16. The returned data is available from the Result property of the Returned event handler's ReturnEventArgs.

Figure 16 PageFunction.Returned (Close Figure 16)
// HomePage.cs (code-behind)
public partial class HomePage : Page
{
  // Launch the page function
  void orderHyperlink_Click(object sender, RoutedEventArgs e)
  {
    OrderABoxPageFunction pageFunction = new OrderABoxPageFunction();
    pageFunction.Return += new ReturnEventHandler<Order>(
        OrderABoxPageFunction_Returned);
    this.NavigationService.Navigate(pageFunction);
  }
  // Handle page function return
  void OrderABoxPageFunction_Returned(
    object sender, ReturnEventArgs<Order> e)
  {
    if (e != null) this.orders.Add(e.Result);
  }
  ...
}

You'll likely want to ensure a PageFunction is removed from navigation history after the task completes. This requires only a simple configuration:

<!--OrderABoxPageFunction.xaml (markup) -->
<PageFunction RemoveFromJournal="True" ... >
  ...
  <!--Content-->
  ...
<PageFunction>
By removing the page function from navigation history, you prevent users from navigating back to the page function. This is important because users may otherwise be able to edit data that has changed since the last time a page function was accessed, creating the potential for inconsistencies in the data.

Note that a task can be composed from more than one page of content, which is common for wizard-style user experiences. PageFunction is equally capable in this situation, since it is possible to use the same techniques for operating over one page function as for operating over multiple page functions.

Where Are We?

The Windows Presentation Foundation application model is extremely flexible. At its most basic, it enables both standalone and browser-hosted applications—both of which support menu-driven and hyperlink-driven navigation. Furthermore, application content can be packaged in either the application's assembly, in a referenced assembly, or as loose files located in a number of places. All in all, the type of user experience you create within the Windows Presentation Foundation application model is limited only by what you choose to do.

For more information, see the Windows Presentation Foundation SDK.

Michael Weinhardt is a programmer/writer at Microsoft, working on the Windows Client SDK. Michael has coauthored a variety of articles, contributed to the "Wonders of Windows Forms" column at MSDN Online, reviewed several Windows technology books, and coauthored Windows Forms 2.0 Programming (Addison-Wesley Professional, 2006).


Web Service Workflows: Deploy Distributed Business Processes With Windows Workflow And Web Services

Web Service Workflows

Deploy Distributed Business Processes With Windows Workflow And Web Services

Israel Hilerio
This article is based on a prerelease version of the .NET Framework 3.0. All information contained herein is subject to change.
This article discusses:
  • Using workflow activities to expose interaction
  • Publishing a workflow as a runtime service
  • Integrating Web services into workflows
  • Integration with Windows Communication Foundation
This article uses the following technologies:
.NET Framework 3.0, ASP.NET
Code download available at: WebServiceWorkflows2006_10.exe (521KB)
Contents
Workflow Activities
Runtime Services
Using a Workflow
Windows Communication Foundation
Conclusion

W
orkflows provide a mechanism for modeling and implementing a business process as a series of activities that manage the interaction between people and back-end systems to carry out a business function. Business processes (or business applications) that require intense human involvement normally are executed over long periods of time due to the complex coordination of multiple entities. Using workflows to implement such business processes provides a mechanism for coordinating, aggregating, and routing business data.

Due to the distributed nature of a business process—coordinating information coming from multiple sources across an organization—it makes sense for a workflow to be deployed as a distributed application. To support this functionality, the Windows® Workflow Foundation team chose Web services and IIS as the host platform for executing distributed workflow applications in early versions of the .NET Framework 3.0 prerelease. This will be extended in the near future to natively support Windows Communication Foundation.

Web service definitions are designed to abstract how applications communicate with each other. They are used to model the types of interactions a workflow requires in order to carry out a business function. Using Web services, you are able to implement these interactions as distributed interoperable services that support application-to-application communications in a standard fashion. These services allow the decoupling of business logic from client application code. You can use this mechanism to generalize and publish service implementations and have them consumed by multiple clients.

Combining these two technologies provides the ability to expose business processes as Web services. This allows the creation of macro Web services that define business interactions, utilize workflow hydration and dehydration policies to support stateful Web services, and provide tracking support to enable execution transparency. Another advantage is the ability to schedule work to be executed after the completion of a Web service request. This allows a Web service request to continue executing without a direct request from a Web service client at some predefined time. Scenarios like escalation and timeout durations are supported by this functionality. In addition, the declarative nature of the workflow definition allows for the implementation of fully declarative business logic.

This article will show how you can implement long-running and stateful business processes using Windows Workflow Foundation and expose them as ASP.NET Web services. The workflow runtime will be hosted by ASP.NET and the long-running workflow would be able to continue to execute past a Web service invocation by using job scheduling constructs that are built into Windows Workflow Foundation. With this capability, business processes are able to notify users of escalation events that are affecting the normal execution flow of a process.

In this article, I will first review the declarative modeling technology of Windows Workflow Foundation. Second, I will review Web services and show how business processes implemented using Windows Workflow Foundation workflows can be exposed as Web services. Finally, I will discuss an expense reporting example that is modeled as a workflow and hosted in a Web service. By the end, you should have a good understanding of how to expose business processes as Web services using declaratively modeled workflows.

Workflow Activities

Windows Workflow Foundation uses Web services to expose interaction points modeled inside the workflow as Web service methods. Interaction points are implemented using Web service activities, which represent the functionality of the interface definitions on which they are based. There are three Web service activities, as shown in Figure 1.

Figure 1 Three Web Service Activities (Close Figure 1)

Activity Name | Activity Designer | Functionality
WebServiceInputActivity |  | Provides a mechanism for receiving input from a Web service method call.
WebServiceOutputActivity |  | Provides a mechanism for sending output as the return value from a Web service method call.
WebServiceFaultActivity |  | Provides a mechanism for sending errors as the result of the invocation of a Web service method call.

These activities are mapped directly to methods defined on interfaces. They identify the input and output boundaries of a Web service inside the workflow model. When WebServiceInputActivity is executed, the input defined by the interface method it represents is available to the workflow for manipulation or it can be bound to properties in the workflow. A WebServiceOutputActivity is required to follow a WebServiceInputActivity for those interface methods that contain a return value. This activity doesn't terminate the execution of the workflow instance. Multiple Web service activities can be modeled inside a workflow definition. However, the very first activity needs to be defined as the request responsible for activating a workflow instance.

WebServiceFaultActivity is used to send an error condition back to the Web service method invocation. Once a WebServiceInputActivity is processed by a WebServiceOutputActivity or a WebServiceFaultActivity, it cannot be invoked again unless the workflow instance is repositioned back to that activity. This can be done using various loop constructs inside a Sequential Workflow or a SetStateActivity inside a State Machine Workflow.

Job scheduling inside a workflow can be accomplished using Delay activities. Delay activities work by registering a timer event with the Scheduler Service for the duration specified in the TimeSpan property of the DelayActivity. The execution of the Delay activity takes place inside the workflow instance using the Web service request before the request is answered. However, the processing of the timer event is done in a separate thread by the ManualWorkflowSchedulerService. This allows the workflow to execute independent of any Web service requests. After the timer event is consumed by the DelayActivity, the workflow continues to execute from that point forward.

Figure 2 shows the elements of a sample workflow. The Web service input request is processed by WebServiceInputActivity. PolicyActivity1 manipulates the Web service input and constructs a result value. The result value is then returned by WebServiceOutputActivity. However, before the request thread is released by the workflow instance, it will attempt to execute as many activities as possible until it achieves an idle state. Once the workflow achieves the idle state, it will release the Web service request thread. At a later time when the timer event is raised, DelayActivity will process the event in a separate thread and will continue to execute PolicyActivity2. The separate thread is a common language runtime (CLR) thread that is allocated and managed by ManualWorkflowSchedulerService. Using a PolicyActivity, the workflow is able to execute code while supporting a completely declarative solution. This allows you to express all the business logic of Web services in a declarative fashion.

Figure 2 Sample Web Service Workflow Model (Close Figure 2)

Workflow Definition
Interface Contract
using System;
using System.Collections.Generic;
using System.Text;

namespace WorkflowLibrary1
{
    interface Interface1
    {
        string Salutation(string str);
    }
}

Configuration
The WebServiceInputActivity receives the parameter str contained by the Salutation method and it is bound to the Hello property inside the workflow.

The return value of Salutation is modeled by theWebServiceOutputActivity and it is bound to the HelloReturn property inside the workflow.

Runtime Services

The last step after building the workflow with Web service activities is to publish the solution. This creates a virtual directory that is used by IIS to deploy the solution as a Web service. Behind the scenes, the workflow library is wrapped by a WorkflowWebService class. This class is responsible for transferring the Web service requests from the SOAP request to the internal workflow runtime communications mechanism. In addition, this class is responsible for instantiating a new WorkflowRuntime. The system only instantiates WorkflowRuntime on the very first Web service request. Once the WorkflowWebService class creates a workflow runtime, it is cached and reused by future Web service calls. Publishing the workflow as a Web service enables the IIS server to be used as an out-of-box application host for the workflow.

By default, when publishing a Web service, the workflow runtime is configured to use the ManualWorkflowSchedulerService, which uses the Web service request thread to execute the workflow instance. This configuration needs to be modified to allow the scheduler to create a thread that processes timer events, thus allowing Delay activities to work properly. This is done by adding the UseActiveTimers flag and setting it to true. Failure to do this would result in timer events not being processed until the next Web service request is consumed by the workflow instance. The configuration change needs to take place in the web.config file created by the publish Web service step:

<add type="System.Workflow.Runtime.Hosting.
               ManualWorkflowSchedulerService,
           System.Workflow.Runtime, Version=3.0.0.0,
           Culture=neutral, PublicKeyToken=31bf3856ad364e35"
     UseActiveTimers="true"/>

In order to support long-running workflows, an additional change needs to be made to the workflow runtime configuration to allow workflow instances to be persisted into the database. Persistence allows the state of the workflows to be stored in the database after the workflow is idle. If the IIS server were to crash after the workflow was persisted, the next incoming Web service request would automatically start the workflow runtime and reload the workflow instance from where it was last saved. To enable persistence, the following lines need to be added to the web.config file:

<add type="System.Workflow.Runtime.Hosting.
               SqlWorkflowPersistenceService, System.Workflow.Runtime,
           Version=3.0.0.0, Culture=neutral,
           PublicKeyToken=31bf3856ad364e35"
     ConnectionString="Initial Catalog=ASPPersistence;Data
                       Source=localhost;Integrated Security=SSPI;"
     UnloadOnIdle="true" />

This configuration uses the SQLWorkflowPersistenceService provided by Windows Workflow Foundation. The database tables associated with this service are part of the Windows Workflow Foundation runtime installation. For this example, the tables are contained inside a database called ASPPersistence.

More complex business processes require the use of more than one WebServiceInputActivity, as many interaction points are required to carry out the process. In order to allow Web service client applications the ability to communicate with the same workflow instance, an HTTP module is provided. This HTTP module is automatically added as part of the standard Web services publishing step and placed inside the web.config file. To enable the reentrance of a workflow instance, the HTTP module stores and retrieves the workflow instance ID to and from a client cookie called WF_WorkflowInstanceId. After the HTTP module retrieves the workflow instance ID from the cookie, it places the ID in the current HTTP Context using the __WorkflowInstanceId__ key. If the client application needs to make multiple Web service calls to the same workflow instance and cookies cannot be enabled, then an alternate approach using HTTP or SOAP header handlers needs to be implemented by a developer.

In addition, supporting multi-user scenarios requires the ability to restrict Web service interactions to specific roles. In order to support this functionality, the Windows Workflow Foundation runtime environment leverages the ASP.NET roles provider model. This allows the restriction of WebServiceInputActivities to receiving information from a user with a specific role. To configure the ASP.NET roles provider, the information shown in Figure 3 needs to be added to the web.config file. This information configures the Web service to use the SQL roles provider contained in the aspnetdb database.

Figure 3 Role Provider Aded to Web.config (Close Figure 3)
<connectionStrings>
  <add name="SqlServerConnection"
    connectionString="Integrated Security =
      SSPI;server=.;database=aspnetdb" />
</connectionStrings>
<system.web>
  <roleManager enabled="true" defaultProvider="SqlProvider">
    <providers>
      <add name="SqlProvider" connectionStringName="SqlServerConnection"
        applicationName="ConsoleAppSample"
        type="System.Web.Security.SqlRoleProvider, System.Web,
        Version=2.0.3600.0, Culture=neutral,
        PublicKeyToken=b03f5f7f11d50a3a" />
    </providers>
  </roleManager>
</system.web>

Using a Workflow

Now let's look at a scenario that uses a workflow accessed over Web services. Imagine an expense approval process that is going to be implemented using a Windows Workflow Foundation workflow and exposed as a Web service for external consumption. The business process needs to support users who want to create expense reports and submit them for approval. Managers need to be able to participate in this process and approve or reject expenses. If the managers don't resolve the submitted expenses within some predefined time, the system needs to send them e-mail reminders. E-mail reminders need to be sent to the managers without forcing them to interact with the system.

The first step is to define the business process that I want expense submissions to go through. The model will use input and output Web service activities to define the interaction points between my client application and my business process. Modeling the expense approval process resulted in the Windows Workflow Foundation workflow shown in Figure 4.

Figure 4 Expense Reporting Workflow Model
Figure 4 Expense Reporting Workflow Model(Click the image for a smaller view)
Figure 4 Expense Reporting Workflow Model
Figure 4 Expense Reporting Workflow Model (Click the image for a larger view)

After modeling the business process, I can see that there are five interaction points I need to define a Web service contract for. The first interaction returns a unique ID used by the system to track expense reports. This information is sent to the client and persisted in the server for future use. The method definition for this interaction point is defined in Figure 5 with its corresponding activity definitions.

Figure 5 CreateExpenseReportId Method Contract (Close Figure 5)

string CreateExpenseReportId();

Notice that there aren't any input parameters defined for CreateExpenseReportId, but there is a return value. That means the CreateExRInput WebServiceInputActivity doesn't have any parameters and the CreateExROutput WebServiceOutputActivity has a ReturnValue property that is mapped against the return string value. PolicyActivity1 is responsible for creating the expense report ID and for setting the value that will be returned as part of the CreateExROutput WebServiceOutputActivity.

To show how a workflow instance can be reentered via a Web service, I've chosen to submit the expense report on a separate interaction point. This interaction point is used to submit a complete expense report object to the business process for reviewing. There is no return value for this interface method definition (see Figure 6). Submitting the expense report to the workflow allows the workflow to decide who needs to review the submitted report based on the amount or enables it to persist the information to a back-end system. For simplicity, the scenario I've chosen doesn't interact with a separate data store. However, most real applications won't use the workflow instance as the final destination of the expense report data. The workflow instance would only serve as a cache.

Figure 6 SubmitExpenseReport Method Contract (Close Figure 6)

void SubmitExpenseReport(ExpenseReport expenseReport);

Figure 6a

Figure 6b

In this case, the SubmitExRInput WebServiceActivity takes the input parameter passed in the Web service method invocation and binds it to the Report property contained by the workflow instance. The populated report object is now available to the workflow for processing.

After the expense report is submitted, the workflow goes into an approval loop. The loop was implemented using a WhileActivity with a DeclarativeCondition, which checks for a flag that signals the reviewing of the expense report. This flag is set when a manager approves or rejects the expense report. ListenActivity is contained inside WhileActivity; ListenActivity contains an interaction point to query the information contained by the expense report, an escalation path, and a reject or approve interaction point. The escalation path sets a time limit, using a DelayActivity, defining the maximum time allowed for a manager to review an expense report. Failure to do so triggers an e-mail to the manager via PolicyActivity2. The Timer event is an internal message, so it doesn't have to be defined in the Web service's contract.

The querying interaction point is implemented by a pair of input and output Web services activities. The ReviewExRInput WebServiceInputActivity receives the Web service request and the ReviewExROutput WebServiceOutputActivity returns the submitted expense report to the client application. Since this is a straight- forward query of the workflow state, there is no need to place any type of processing between these two activities (see Figure 7).

Figure 7 RetrieveExpenseReport Method Contract (Close Figure 7)

ExpenseReport RetrieveExpenseReport();

The last two interaction points contained inside ListenActivity are the reject and approval actions. These two events do not require any type of client acknowledgment. They are invoked without expecting any type of return value. Their purpose is to set the expense report status and to set the reviewing flag checked by the WhileActivity to false. The setting of these variables is done inside PolicyActivity3 and PolicyActivity4 (see Figure 8).

Figure 8 RejectExpenseReport and ApproveExpenseReport Method Contracts (Close Figure 8)

void RejectExpenseReport();

void ApproveExpenseReport();

Once the expense report is approved or rejected, the workflow instance is terminated and the expense report information is stored in a back-end system via the Policy Activities. Web services can no longer execute against the same instance ID.

An interesting aspect of the services inside WhileActivity is that they can only be executed by a manager. In order to enforce this security restriction, they were configured with a manager role.

Windows Communication Foundation

I have discussed implementing Web services using the ASMX technologies supported by Windows Workflow Foundation activities. The first release of Windows Workflow Foundation won't directly provide any activities or host for deploying the newer Windows Communication Foundation services; this functionality will be provided in a future release of the framework. However, there are several ways in which Windows Communication Foundation can be used today by Windows Workflow Foundation:

  • Develop a Windows Communication Foundation service that hosts the workflow runtime and executes the workflow instance.
  • Access a Windows Communication Foundation service from a workflow definition via the InvokeWebServicesActivity.
  • Access a Windows Communication Foundation service via a custom Activity.

The first approach allows developers to use the Windows Communication Foundation service as a host process for the workflow runtime. This service is responsible for triggering the execution of the workflow when a predefined OperationContract is invoked. Using this approach allows the workflow to receive input from OperationContract as workflow parameters. Figure 9 shows an example of what the OperationContract could look like.

Figure 9 Instantiating a Workflow Instance (Close Figure 9)
[ServiceContract]
class WorkflowApplication
{
  WorkflowCommunicationHandler internalWFCommunications;
  Guid workflowInstanceId;
  WorkflowRuntime workflowRuntime;

  [OperationContract]
  public void BeginWorkflowApplication(string inputVariable)
  {
    workflowRuntime = new WorkflowRuntime();
    internalWFCommunications = new WorkflowCommunicationHandler();
    string connectionString = "Initial Catalog=WorkflowPersistence;" +
      "Data Source=localhost;Integrated Security=SSPI;";
    workflowRuntime.AddService(
      new SqlWorkflowPersistenceService(connectionString, true,
      new TimeSpan(0, 10, 0), new TimeSpan(0, 0, 5)));

    ExternalDataExchangeService externalDataService =
      new ExternalDataExchangeService();
    workflowRuntime.AddService(externalDataService);
    externalDataService.AddService(internalWFCommunications);

    workflowRuntime.StartRuntime();

    Dictionary<string, object> inputParams =
      new Dictionary<string, object>();
    Type type = typeof(Workflow1);
    workflowInstanceId = Guid.NewGuid();

    inputParams["inputVariable"] = inputVariable;
    WorkflowInstance workflowInstance =
      Program.workflowRuntime.CreateWorkflow(type, inputParams,
      workflowInstanceId);

    workflowInstance.Start();
  }
}

In order to allow the Windows Communication Foundation service to facilitate ongoing interactions with the workflow instance, other interaction points can be exposed as additional OperationContract instances. The parameters of OperationContract are passed to the workflow inside a workflow communication event. Here is an additional OperationContract for the ServiceContract shown in Figure 9:

[OperationContract]
public void SendDataToWorkflowApplication(Guid workflowInstanceId,
  string inputVariable)
{
  WorkflowInstance workflowInstance =
    workflowRuntime.GetWorkflow(workflowInstanceId);
  internalWFCommunications.SendRequest(
    workflowInstance, inputVariable);
}

The second approach of invoking a Windows Communication Foundation service using the InvokeWebServicesActivity is possible due to the Windows Communication Foundation support for SOAP. Unfortunately, this approach doesn't leverage the WS-* services supported in Windows Communication Foundation. A better approach is the third one—to create a Windows Workflow Foundation Activity whose internal execution method leverages the Windows Communication Foundation ServiceProxy to invoke a Windows Communication Foundation service:

protected override ActivityExecutionStatus Execute(
  ActivityExecutionContext executionContext)
{
  RentalReservationsProxy p = new RentalReservationsProxy();
  this.ResultValue = p.MakeReservation(this.InputValue);
  p.Close();
  return base.Execute(executionContext);
}

The execution of this activity passes the InputValue property to the RentalReservationsProxy class. Using the proxy, the activity calls the MakeReservation OperationContract contained by the RentalReservationProxy service and returns a proxy class that abstracts the WorkflowApplication Windows Communication Foundation service. The MakeReservation invocation takes the InputValue property on the activity as an input parameter and sets the ResultValue property on the activity.

Using Windows Communication Foundation together with Windows Workflow Foundation allows workflows to expose themselves to remote clients as Windows Communication Foundation services and invoke Windows Communication Foundation services via activities.

Conclusion

You can take advantage of the rich, declarative Windows Workflow Foundation model to expose your business processes, while leveraging Web services as a deployment technology for executing them. Furthermore, Windows Communication Foundation and Windows Workflow Foundation can be utilized to publish robust WS-* compliant business applications. To learn more about Windows Workflow Foundation and developing workflows with .NET Framework 3.0, see the Windows Workflow Foundation site at msdn.microsoft.com/winfx/technologies/workflow. Developer information about Windows Communication Foundation is available from msdn.microsoft.com/winfx/technologies/communication.

Israel Hilerio is a program manager at Microsoft in the Windows Workflow Foundation team. He has 15+ years of development experience doing business applications and has a PhD in Computer Science.


SQL Server 2005: Jazz Up Your Data Using Custom Report Items In SQL Server Reporting Services

SQL Server 2005

Jazz Up Your Data Using Custom Report Items In SQL Server Reporting Services

Teo Lachev
This article discusses:
  • Extensibility in SQL Server 2005 Reporting Services
  • Creating and debugging custom report items
  • The ProgressTracker CRI sample
This article uses the following technologies:
SQL Server 2005 Reporting Services, Visual Studio 2005
Code download available at: CRI2006_10.exe (235KB)
Contents
Introducing the ProgressTracker CRI
The ProgressTracker CRI in Action
Implementing the CRI Design-Time Component
Initialization
Rendering
Verbs
Custom Property Editor
Implementing the CRI Run-Time Component
Deploying CRI
Debugging CRI
Wrapping It Up

W
hen it comes to presenting information these days, a picture may be worth a lot more than a thousand words. And even though SQL Server™ Reporting Services does provide charting capabilities, some are not up to the task. For example, you may need to render the value or the trend of a key performance indicator (KPI) as an image or use a chart type that is not supported by the native Reporting Services chart region. But SQL Server 2005 Reporting Services has a number of new extensibility features. One of the most exciting is custom report items, which allow you to transcend the limitations of traditional text-based reporting.

For years, developers using the Microsoft®.NET Framework and COM have been relying on home-grown or third-party user controls to spice up their presentation layers. With the advent of custom report items in SQL Server 2005 Reporting Services, you can implement your own report "controls" to do the same. Then, your custom report item (CRI) can be used just like a native report item. You can, for example, bind it to a report dataset to display a field value graphically. As you can imagine, custom report items redefine the concept of a standard report and open a new world of possibilities. I'm sure it won't be long before report vendors develop cool report widgets to meet common data visualization needs. As you'll see, implementing a custom report item on your own is not that difficult.

A CRI is a report item written to extend the capabilities of the native report items provided by SQL Server 2005 Reporting Services. Anything that can be represented as an image (already existing or programmatically generated) can be converted to a CRI.CRI allows you to implement both regular report items and data regions. This article walks you through the process of building a regular custom report item you can bind to a dataset field inside a table, a matrix, or a list data region. Should your report requirements call for a full-blown custom data region instead, this won't be difficult to do once you grasp the concepts presented here.

Introducing the ProgressTracker CRI

Most requirements will call for generating the CRI image programmatically. When I was envisioning the ProgressTracker CRI sample presented here, I wanted to find what it would take to convert an existing .NET Windows® Forms control to a CRI. My hypothesis was that any custom .NET control that is drawn using GDI+ can be promoted to a CRI. It turned out that this was true, and it was simpler than I anticipated.

To test my hypothesis, I authored a simple .NET Windows Forms control called ProgressTracker. Like the ProgressBar Windows Forms standard control, ProgressTracker graphically displays a numeric value as a progress bar. As noted, such a CRI could help an end user visualize the details of a KPI.

I intentionally kept the control implementation simple. The drawing of both rectangles (outer bar and progress value) is encapsulated in a DrawBar method and takes only a few lines of GDI code. First, DrawBar paints the rectangle outline by calling the DrawRectangle GDI function. Next, the code fills the rectangle with the specified fill color:

public static void DrawBar(
    Graphics g, Rectangle front, Color fillColor,
    Color borderColor, bool outLine)
{
    using(Pen pen = new Pen(borderColor, 1))
        g.DrawRectangle(pen, front);

    using(SolidBrush brush = new SolidBrush(fillColor))
        g.FillRectangle(brush, front);
}

As Figure 1 shows, the control supports a subset of properties found under the ProgressTracker category. The user can set these properties in the Visual Studio® 2005 Properties window to configure the control at design time. Here's a tip: to minimize development and testing time, start by implementing your CRI as a standard Windows Forms control. This lets you test it and debug it inside a Windows Forms project. When the control is ready, convert it to a CRI.

Figure 1 ProgressTracker Design-Time Component
Figure 1 ProgressTracker Design-Time Component(Click the image for a smaller view)
Figure 1 ProgressTracker Design-Time Component
Figure 1 ProgressTracker Design-Time Component (Click the image for a larger view)

The ProgressTracker CRI in Action

At this point, you are probably eager to see the CRI version of the control in action. The KPIDemo report shown in Figure 2 uses the ProgressTracker CRI to graphically display the value of the Product Gross Profit Margin KPI which is defined in the Adventure Works cube (included in the SQL Server 2005 samples).

Figure 2 KPI Demo Report
Figure 2 KPI Demo Report(Click the image for a smaller view)
Figure 2 KPI Demo Report
Figure 2 KPI Demo Report (Click the image for a larger view)

For your convenience, the report shows both the actual KPI status value (Profit Margin Status column) and the graphical value (Status column). If you don't want to install the Adventure Works SSAS sample, you can use the ProgressTrackerDemo report for testing.

Now, let's see how to convert the ProgressTracker Windows Forms control to a CRI. As with most controls you've used or implemented, a CRI has a dual life. At design time, the user configures the CRI by setting its properties. Typically, the CRI would also draw a design-time image, so the developer can see the effect of setting up these properties. At run time, the CRI is responsible for rendering an image that will appear on the report. The design-time aspect of a component is, by far, more difficult to implement.

Implementing the CRI Design-Time Component

The CRI design-time component must inherit from the Reporting Services CustomReportItemDesigner class. This class participates in the underlying Windows Forms host designer infrastructure. (See the "Resources" sidebar if you want to learn more about implementing .NET custom designers.) The more sophisticated the CRI is, the more complex its design-time component is likely to be. Deciding which design services you need to implement is a trade-off between usability requirements and implementation time. Simple CRIs are likely to implement only control properties and design-time rendering. More complex implementations may include adornment services, verbs, and a custom property editor.

A well-designed CRI should allow developers to configure it by using the Visual Studio Properties window. If you have used the Windows Forms PropertyGrid control in your apps, you know how to implement CRI properties. If you are new to this control, you may find the article "Getting the Most Out of the .NET Framework PropertyGrid" useful (see the "Resources" sidebar).

Any CRI public property will be automatically exposed in the Visual Studio Properties window. Additional attributes can be used to change the UI property behavior. For example, Figure 3 shows what it takes to implement the Maximum property that specifies the Progress Tracker maximum value.

Figure 3 Implementing the Maximum Property (Close Figure 3)
[Category("ProgressTracker"),
 DefaultValue(1f),Description("The maximum value.")]
public float Maximum {
    get {
        string max = this.GetCustomProperty(Shared.PROP_MAXIMUM);
        return string.IsNullOrEmpty(max) ? 1f : float.Parse(max);
    }
    set {
        _progress.Maximum = value;
        SetCustomProperty(Shared.PROP_MAXIMUM, value.ToString());
        Invalidate();
    }
}

The Category attribute assigns the property to the ProgressTracker group in the Visual Studio Properties window. The DefaultValue attribute assigns a default value to the property. The Description property provides description text to be displayed at the bottom of the Visual Studio Properties window. In the case of the ProgressTracker CRI, the Value property is the most difficult to implement. That's because it allows the user to type in either a constant value or the field name in case the CRI is bound to a dataset field.

All ProgressTracker custom properties persist their values in the Report Definition Language (RDL) by calling the SetCustomProperty helper method. Figure 4 shows what the serialized instance of ProgressTracker may look like. To read the property value when the control is instantiated, the property calls GetCustomProperty. When the value of a custom property is changed, the code calls the Invalidate method of the CustomReportItemDesigner class to force the control to repaint itself.

Figure 4 Serialized Instance of Progress Tracker (Close Figure 4)
<CustomReportItem Name="ProgressTracker1">
    <Type>ProgressTracker</Type>
    <Style>
        <TextAlign>Right</TextAlign>
        <FontSize>7pt</FontSize>
        <Color>Gold</Color>
    </Style>
    <CustomProperties>
        <CustomProperty>
            <Name>Value</Name>
            <Value>0.25</Value>
        </CustomProperty>
    </CustomProperties>
</CustomReportItem>

The CustomReportItemDesigner class exposes standard properties, such as Font, Size, and so forth. For example, you can get the selected Font from the CustomReportItemDesigner.Style.Font property. As its stands, the CRI designer infrastructure provides no way of hiding the standard properties in case you don't need them. For this reason, before introducing a new custom property, check if you can reuse an existing standard property instead. For example, I decided to use the standard Color property for the progress color and the BorderColor property for the outline color.

Initialization

When the control is added to the Layout tab of the Report Designer, the CustomReportItemDesigner class fires the InitializeNewComponent event. ProgressTracker uses this event to initialize the design-time component with default values by calling the SetDefaults helper method. A common initialization task is setting the default component size. ProgressTracker overrides the CustomReportItemDesigner DefaultSize property to set the default size of the design-time component to a width of 150 pixels and a height of 30 pixels:

public override DesignSize DefaultSize
{
    get
    {
        return new DesignSize(
            new RSDrawing.Unit(150), new RSDrawing.Unit(30));
    }
}

Rendering

To help the user visualize the effect of setting the CRI properties, ProgressTracker draws an image at design time (see Figure 5). As with the standard RS data regions, more sophisticated CRIs have different design-time and run-time presentations. In our case, ProgressTracker uses the same code to draw its image in both modes. The rendering code remains practically unchanged from the original Windows Forms control.

Like Windows Forms control programming, CustomReportItemDesigner invokes the virtual OnPaint method each time the CRI needs to be repainted, as in Figure 6.

Figure 6 onPaint Event (Close Figure 6)
public override void OnPaint(PaintEventArgs e)
{
    float value = 0f;
   _progress.Alpha = this.Alpha;
   _progress.ProgressColor = this.Style.Color.ColorRgb;
   _progress.OutlineColor = this.Style.BorderColor.Default.ColorRgb;
   _progress.Maximum = this.Maximum;
   _progress.Minimum = this.Minimum;
   _progress.Value = float.TryParse(this.Value, out value) ?
        value :
        (progress.Maximum + _progress.Minimum) / 2;
   _progress.ShowValue = this.ShowValue;
   _progress.Font = Shared.GetDrawingFontFromReportFont(this.Style.Font);
   _progress.Height = this.Height.Pixels;
   _progress.Width = this.Width.Pixels;
   _progress.DrawControl(e.Graphics);
}
Figure 5 ProgressTracker Rendsers Itself at Both Design Time and Run Time
Figure 5 ProgressTracker Rendsers Itself at Both Design Time and Run Time(Click the image for a smaller view)
Figure 5 ProgressTracker Rendsers Itself at Both Design Time and Run Time
Figure 5 ProgressTracker Rendsers Itself at Both Design Time and Run Time (Click the image for a larger view)

The OnPaint event configures the internal ProgressTracker instance and calls its DrawControl method to let the CRI draws its image onto the provided Graphics canvas.

The Reporting Services design-time infrastructure provides additional services you can implement to facilitate the user interaction with your CRI at design time. Take, for example, the Reporting Services chart data region, which sponsors a full-blown design-time component (see Figure 7). When you click on the chart region at design time, the control draws an adornment area outside the main control window. This area has three frames that act as drop containers for dataset fields. The Reporting Services adornment infrastructure is essentially a pass-through implementation of the Windows Forms Adorner class.

Figure 7 Services for Configuring the CRI
Figure 7 Services for Configuring the CRI(Click the image for a smaller view)
Figure 7 Services for Configuring the CRI
Figure 7 Services for Configuring the CRI (Click the image for a larger view)

As you can imagine, rendering the adornment window and responding to user events could get rather complex. The good news is that the adorner infrastructure is entirely optional. For the sake of simplicity, the ProgressTracker CRI doesn't implement an adornment window. If you decide to adorn, the Chris Hays polygon sample demonstrates how you can implement adornment features in a CRI (see the "Resources" sidebar).

For convenience, besides allowing the user to type in the field name in the Value property, ProgressTracker supports dragging a dataset field onto the image to bind the Value property to that field. Behind the scenes, ProgressTracker uses the CustomReportItemDesigner drag and drop support. The drag and drop implementation is essentially the same as in Windows Forms so I won't spend much time discussing it. The only detail worth mentioning takes place in the DragEnter event:

public override void OnDragEnter(DragEventArgs e)
{
    IFieldsDataObject fieldsDataObject =
        e.Data.GetData(typeof(IReportItemDataObject)) as
            IFieldsDataObject;
    if (fieldsDataObject != null &&
        fieldsDataObject.Fields != null &&
        fieldsDataObject.Fields.Length > 0)
    BeginEdit();
}

In the absence of the adorner window, BeginEdit draws a selection border around the CRI. The net effect is the same as when you drag a dataset field over a table cell of the standard table region.

Verbs

Another nice feature you may consider implementing is verbs. Designer verbs let you implement custom actions that can be launched by right-clicking on the control design surface and choosing the action from the context menu. For example, suppose you need to allow end users to reset the CRI properties to their default values.

To implement this requirement, override the CustomReportItemDesigner Verbs property. Next, add a new verb to DesignerVerbCollection and specify the desired caption and the callback event handler (see Figure 8). When the user selects the action from the context menu, CustomReportItemDesigner will invoke the callback event handler where you can execute whatever code is needed. In this case, the SetDefaults helper method (not shown) resets the CRI properties.

Figure 8 Implementing Verbs (Close Figure 8)
public override DesignerVerbCollection Verbs
{
    get {
        if (_verbs == null) {
            _verbs = new DesignerVerbCollection();
            _verbs.Add(new DesignerVerb("Reset Defaults",
                 new EventHandler(OnCustomAction)));
        }
        return _verbs;
    }
}

private void OnCustomAction(object sender, EventArgs e)
{
    switch (((System.ComponentModel.Design.DesignerVerb)sender).Text){
        case "Reset Defaults": SetDefaults(); Invalidate(); break;
        default: MessageBox.Show("Not supported"); break;
    }
}

At the end, the SetDefaults method calls IComponentChangeService.OnComponentChanged to inform CustomReportItemDesigner that there are unsaved changes. This also causes a refresh of the Visual Studio Properties window to synchronize the CRI properties with their new values. You need to call OnComponentChanged when you set a property value programmatically. As noted before, when the user drags and drops a dataset field, the Value property calls OnComponentChanged to notify the host.

Custom Property Editor

Finally, more complex CRIs may need a custom property editor. There is a special verb (Properties) whose task is to launch the editor. For example, the standard chart data region comes with a custom property editor that gives the user more control over the region properties (see Figure 7). The ProgressTracker CRI includes just the stub code needed to implement this feature (see Figure 8).

Implementing a custom property editor is easy. First, you need to decorate the ProgressTrackerDesigner class with the Editor attribute that references a class that inherits from ComponentEditor. Next, you implement a Windows Forms dialog to collect the properties from the user. Finally, you instantiate the dialog inside the editor class, collect the new property values, and apply them to the CRI. Figure 9 shows how that looks.

Figure 9 Implementing a Custom Property Editor (Close Figure 9)
[LocalizedName("ProgressTracker")]
[ToolboxBitmap(typeof(ProgressTrackerDesigner), "ProgressTracker.ico")]
[CustomReportItem("ProgressTracker")]
[Editor(typeof(CustomEditor), typeof(ComponentEditor))]
class ProgressTrackerDesigner : CustomReportItemDesigner {
    ...
}

internal sealed class CustomEditor : ComponentEditor {
    public override bool EditComponent(ITypeDescriptorContext context,
        object component) {
        MessageBox.Show("Implement CRI Properties Window here!");
        return true;
    }
}

Implementing the CRI Run-Time Component

Implementing the CRI run-time component is much simpler than implementing its design-time counterpart. That's because you only need to worry about drawing the run-time image of the CRI—the main purpose of having a CRI after all. The CRI run-time component must implement the ICustomReportItem interface, as you see in Figure 10.

Figure 10 Initializing the Component (Close Figure 10)
public class ProgressTrackerRenderer : ICustomReportItem
{
    public CustomReportItem CustomItem { set { _cri = value; } }

    public ChangeType Process() {
        // initialize CRI from properies in RDL
        Initialize();

        // create a memory stream that stores the image
        using(MemoryStream stream = new MemoryStream()) {
            Bitmap bmp = new Bitmap(progress.Width, progress.Height,
                PixelFormat.Format32bppArgb);
            using(Graphics graphics = Graphics.FromImage(bmp)) {
                Color backgroundColor = ((ReportColor)
                    _cri.Style["BackgroundColor"]).ToColor();
                if (backgroundColor == Color.Transparent)
                    backgroundColor = Color.White;
                graphics.Clear(backgroundColor);
                _progress.DrawControl(graphics);
            }
            bmp.Save(stream, ImageFormat.Bmp);

            // Rewind image stream
            stream.Position = 0;

            // Create a new RS image object
            _image = new Image(_cri.Name, _cri.ID);
            _image.MIMEType =  "image/bmp";

            // serialize the image stream into the RS image
            _image.ImageData = stream.ToArray();
        }
        _image.Sizing = Image.Sizings.AutoSize;

        return ChangeType.None;
    }

    public ReportItem RenderItem {
        get {
            if (_image == null) Process();
            return _image;
        }
    }
}

When the report processor executes the report and discovers the presence of a custom report item, it instantiates the run-time component (ProgressTrackerRenderer in this case) each time the CRI needs to be rendered on the report. For each instance, the report processor passes a CustomReportItem object to the ICustomReportItem.CustomItem property. The CustomReportItem object contains a set of properties that were set at design time. It includes both the CRI custom properties and the standard properties. The properties are scoped at the custom report item level. Currently, it is not possible to reference other report items on the report or the report object itself.

It is important to note that if the property value is expression-based, it is evaluated and resolved by the report processor before the CustomReportItem object is passed to the run-time component. For example, if the ProgressTracker component is bound to a dataset field (its Value property is set to a Fields!<FieldName>.Value expression), the field value is retrieved by the report processor and made available under the CRI Value property. As you can imagine, this saves an enormous amount of plumbing work on your part.

At this point, the report processor executes the ICustomReportItem.Process method to ask the CRI to render the run-time image. The code starts by initializing the CRI from its properties. Next, the code creates a Graphics object from a bitmap image. Then it calls the same DrawItem method that we used to render the design-time image. Once the image is drawn, it is saved into a memory stream. The report processor expects back an instance of a Microsoft.ReportingServices.ReportRendering.Image class (the only CRI type that is currently supported). To comply with this requirement, the code instantiates the Reporting Services image class and loads it from the memory stream. It sets the image MIME type to "image/bmp" (you can choose another standard image format if needed).

Once the Process method executes successfully, the report processor retrieves the image from the RenderItem property and outputs it on the report.

Deploying CRI

The ProgressTrackerCRI project (ProgressTracker assembly) hosts both the design-time component (ProgressTrackerDesigner.cs) and run-time component (ProgressTrackerRenderer.cs) of the ProgressTracker CRI. Follow these steps to deploy and configure the ProgressTracker CRI (note that these steps may vary slightly based on which components of SQL Server are installed).

First, to use the CRI in the Report Designer, deploy the ProgressTracker assembly to \Program Files\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies. To use the CRI in a managed report (running under the Report Server), deploy the assembly to \Program Files\Microsoft SQL Server\MSSQL.3\Reporting Services\ReportServer\bin. For your convenience, I've created a post-build script (see the Build Events tab in the project properties) that copies the assembly after the ProgressTrackerCRI is built successfully.

Next, to let the Report Designer know about your CRI, register the ProgressTracker CRI in the \Program Files\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies\RSReportDesigner.config file, as follows:

<ReportItems>
    <ReportItem Name="ProgressTracker" Type=
        "MsdnMag.RS.Extensibility.ProgressTrackerCRI.
            ProgressTrackerRenderer, MsdnMag.ProgressTracker" />
</ReportItems>
<ReportItemDesigner>
    <ReportItem Name="ProgressTracker" Type=
        "MsdnMag.RS.Extensibility.ProgressTrackerCRI.
         ProgressTrackerDesigner, MsdnMag.ProgressTracker" />
</ReportItemDesigner>
Each CRI requires a separate ReportItemDesigner element in the RSReportDesigner.config file. For your convenience, I enclosed my versions of the configuration files in the \Code\Config folder. Use them for reference only. Do not overwrite your config files with mine, as there's machine-specific information encoded in the file.

Since at run time the report processor interacts with the run-time component only, it is sufficient to add only the ReportItem element to \Program Files\Microsoft SQL Server\MSSQL.3\Reporting Services\ReportServer\rsreportserver.config.

Next, elevate the CAS permissions of the CRI assembly for run-time execution by adding the following CodeGroup to the \Program Files\Microsoft SQL Server\MSSQL.3\Reporting Services\ReportServer\rssrvpolicy.config file, like so:

<CodeGroup class="UnionCodeGroup" version="1" Name="CRICodeGroup"
    Description="Code group for the CylinderBar CRI"
    PermissionSetName="FullTrust">
    <IMembershipCondition class="UrlMembershipCondition" version="1"
        Url="C:\Program Files\Microsoft SQL Server\MSSQL.3\
             Reporting Services\ReportServer\bin\
             MsdnMag.ProgressTracker.dll" />
</CodeGroup>

Finally, to add the ProgressTracker CRI to the Visual Studio toolbox, right-click in the Toolbox and select "Choose Items...". On the .NET Components tab, select "Browse..." and navigate to \Program Files\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies\MsdnMag.ProgressTracker.dll.

If the CRI doesn't show up in the Preview tab of the Report Designer and there are no errors, it is probably not registered properly. Double-check the first two steps. Remember that if the CRI has dependent assemblies, they also have to be deployed to the Report Server bin folder. To troubleshoot errors related to missing dependent assemblies, use a trace listener and watch for Reporting Services exceptions.

Debugging CRI

When the report is loaded in the Report Designer, it locks the CRI assembly in the process space of the Visual Studio IDE that hosts the report project. Unfortunately, the only way to redeploy a new version of the CRI is to shut down that Visual Studio instance. As you can imagine, this can be quite irritating. For this reason, I recommend you employ the following debugging practice. First, you should exclude the report project from your Visual Studio CRI solution. Second, right-click on the ProgressTrackerCRI project in the Visual Studio Solution Explorer and select "Set as Startup Project". Third, you should open the ProgressTrackerCRI project properties, switch to the Debug tab, and set it as shown in Figure 11. Finally, you can set breakpoints in the CRI design-time and/or run-time components.

Figure 11 Set Project Settings to Load the Hosting Report Project
Figure 11 Set Project Settings to Load the Hosting Report Project(Click the image for a smaller view)
Figure 11 Set Project Settings to Load the Hosting Report Project
Figure 11 Set Project Settings to Load the Hosting Report Project (Click the image for a larger view)

As a result, when you hit F5 to debug the ProgressTracker CRI, Visual Studio IDE will load the report project first. When you switch to the Report Designer Layout tab, you should be able to debug your design-time component. Similarly, when you switch to the Preview tab, the breakpoints in the run-time component will be hit. More importantly, when you stop your debug session, Visual Studio will shut down the IDE instance that hosts the report project. This allows you to recompile and redeploy the CRI assembly without locking issues.

Resources

See the following resources for tips on developing custom report items with SQL Server 2005 Reporting Services:

  • Getting the Most Out of the .NET Framework PropertyGrid Control
  • Chris Hays’s Polygon Sample
  • Tailor Your Application by Building a Custom Forms Designer with .NET
  • Create and Host Custom Designers with the .NET Framework 2.0

Wrapping It Up

You should consider implementing a CRI when your reporting requirements go beyond what the standard Reporting Services items and data regions can do. Custom report items help you convey information to your report users in the form of graphic elements and images. Anything that can be rendered as an image can be implemented as a CRI and rendered on reports.

Teo Lachev works as a technical architect for a leading financial institution where he designs and implements .NET-centric Business Intelligence solutions. He is an MVP for SQL Server. Teo is the author of the books Applied Microsoft Analysis Services 2005 and Microsoft Reporting Services in Action.


Editor's Note: Some Things Are Easier Said Than Done

Editor's Note
Some Things Are Easier Said Than Done
Joshua Trupin

We had been poking around the coding4fun site and decided to embark upon a project of our own. What would be cooler, we thought, than combining our love of programming with our abject failures in that college-level electrical engineering class, and wire something up? In this case, it was adapting a GPS device to a nine-pin serial port, which could then be attached to a camera. It would be cheaper than buying a complete solution (which run upwards of $400), and new devices only seem to come in Bluetooth or USB these days.

Well, it turns out that there are a few rules you have to follow when you delve into hardware, much to our chagrin. First, your laptop doesn’t have a DB9 port when you need one. You have to dig out an old model that runs so slowly you can’t believe it has a whole 256MB RAM in it.

Second, a good rule of thumb is that while software can destroy your data, tinkering with hardware can take down the power grid of a small community. Did you know that with just a readily available multimeter, you can make your laptop short out without even opening it? Well, we didn’t until last week.

Messing with hardware used to be even more dangerous, when the combination of rows of unlabeled jumper pins and the nonexistence of the Web where you could look up the instruction sheet you tossed out when opening the card’s box combined to produce acrid pink smoke when you turned the machine’s power on. You will probably find it harder to actually make a machine billow smoke these days, but working with hardware still creates that vague feeling of extreme risk that memory protection schemes and file-level security no longer affords you as a developer.

Three weeks into the simple nine-pin rewiring project, we got our second wind. This project started out with a great idea: turning $70 in parts into a $400 component. It soon ran up a three-digit tab at the local electronics store. What genius decided that 28-gauge wire should be smaller than 26 gauge? Why does the wire stripper do a great job on six wires, then pull 4" of the seventh wire out of the cable hose? Isn’t there a better name for the cable hose, like "casing"? Why do they only supply 15 or so of those crimp pins in the nine-pin serial kit when you need like 400 of them because you don’t know how to use a crimper properly? When you touch the soldering iron tip to the roll of metal, where does the solder go? It’s like it just all disappears or something. Who decided that DB9 to USB would need conversion circuitry?

Software is so much easier. You don’t have to make 20 trips to the computer store. You just go to a search engine if you’re missing a piece of connector software. Someone else has likely faced the same challenge, and if they haven’t you can always do it yourself and even share your experiences with others.

Hardware, it turns out, doesn’t work the same way. Are you really going to pull your own wire? Make your own heat-shrink tubing? Have you ever accidentally reversed the polarity in your sample RSS aggregator and started uploading stories to Web sites?

The flip side of this is that with software, you can build just about anything you can think of. It’s just a matter of imagination. If you want software that can read a GPS signal, you just sit down and hack away until you get it right. If you want to see a map of your neighborhood, you can connect to Windows Live, or you can even get down into the weeds and do it yourself with nothing but a raw data file from the Census Department. Electronics stores are full of advanced hardware projects, like MP3 players and cell phones, that require teams of engineers to design. Never in a million years would we sit down and prototype our own working digital camera with USB 2.0 outputs.

Sitting here with a bag full of unconnected parts, a third wind hit us. Maybe it was time to stick with software.

—J.T.

Thanks to Thanks to the following Microsoft technical experts for their help with this issue: Nicholas Allen, Robert Bruckner, Simon Calvert, Jonathan Caves, Karen Corby, Matt Gibbs, Chris Hays, Tom Hollander, Jim Johnson, John Justice, Lauren Lavoie, Eugenio Pace, Mahesh Prakriya, Ravi Rao, Daniel Roth, Chris Sells, Tina Tam, Maura Van Der Linden, Brian Welcker, Kimberly Wolk, and Dawn Wood.

IntelliSense, Internet Explorer, JScript, Microsoft Press, Microsoft, MSDN, MSN, Visual Basic, Visual C++, Visual Studio, Windows, Win32, WinFX, Windows Server, and Windows Vista are either registered trademarks or trademarks of Microsoft Corporation. Other trademarks or tradenames mentioned herein are the property of their respective owners. MSDN MAGAZINE DOES NOT MAKE ANY REPRESENTATION OR WARRANTY, EXPRESS OR IMPLIED WITH RESPECT TO ANY CODE OR OTHER INFORMATION HEREIN. MSDN MAGAZINE DISCLAIMS ANY LIABILITY WHATSOEVER FOR ANY USE OF SUCH CODE OR OTHER INFORMATION. MSDN and Microsoft are registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. MSDN Magazine, MSDN, and Microsoft logos are used by CMP under license from owner. MSDN Magazine is published by CMP Technology LLC. CMP Technology LLC is an independent company not affiliated with Microsoft Corporation. Microsoft Corporation is solely responsible for the editorial contents of this magazine.

MSDN Magazine does not make any representation or warranty, express or implied with respect to any code or other information herein. MSDN Magazine disclaims any liability whatsoever for any use of such code or other information.


Toolbox: Protecting Code, Persisting Data, and More

Toolbox
Protecting Code, Persisting Data, and More
Scott Mitchell

Protect Your App's Source Code

The greatest asset for companies that create and sell software is their source code. That intellectual property is typically the sum total of the thousands of hours spent on planning, designing, coding, and testing the application. Therefore, it's vital that such companies protect their intellectual property against those who would use it for ill-gotten gains.

When shipping an application to your customers you are, effectively, handing over your intellectual property. After all, an executable is just a translation from a high-level syntax into a lower-level one. Decompilation—the process of turning the compiled syntax back into a high-level syntax—is especially easy with technologies such as the Microsoft® .NET Framework and Java. Without taking any proactive measures to complicate decompilation, any user who has your application's assemblies can use a tool like Lutz Roeder's Reflector (www.aisto.com/roeder/dotnet) to decompile the assembly's intermediate language back to C# or Visual Basic source code.

An application's source code can be protected from decompilers by using a technique called obfuscation. Obfuscation is the process of modifying the intermediate language generated by the compiler into a form that is functionally identical to the original code, but is difficult if not impossible to decompile back into a high-level language.

There are a number of third-party products available for obfuscating .NET-based assemblies, one of which is Dotfuscator Professional Edition version 3.0 by PreEmptive Solutions. If you have Visual Studio® .NET or Visual Studio 2005 installed on your machine, you probably already have the free Community Edition of Dotfuscator installed as well.

When obfuscating an assembly, the intermediate language is adjusted in a number of ways. The most common technique renames the human-friendly names for methods, properties, events, variables, and so on with shortened, less intelligible names. For example, a method named DisplayMessage() might be renamed to a(). Such renaming leaves the code functionality intact, but makes it much more difficult for a human to parse and understand the decompiled code.

In addition to renaming, the Professional Edition of Dotfuscator includes many more obfuscation techniques, such as control flow obfuscation, string encryption, and overload induction. With control flow obfuscation, standard control flow semantics are rewritten such that they result in spaghetti-code when decompiled. String encryption helps thwart those attackers who search the code for a particular string, such as the prompt for a password or registration number. And overload induction detects nonconflicting method sets and assigns them the same obfuscated name.

With Dotfuscator's operator overloading you'll find that many of your methods are able to be renamed to the same method name (such as "a" or "b"). When decompiling such an assembly, the result is a class hierarchy with dozens of methods using the exact same name, making it very difficult for a human to make sense of the decompiled output.

Obfuscation can also strip out unneeded metadata from the assembly, remove unused members, and heavily reuse identifier names across the assembly. Together, these techniques can reduce the file size of an assembly significantly, providing for improved performance. On the other hand, control flow obfuscation can impact the performance of finely tuned algorithms, and using string encryption can increase the assembly's size and run time.

Obfuscating an assembly with Dotfuscator is a straightforward process that can be done entirely through Visual Studio or integrated directly into the build process through MSBuild or the command line. To use Dotfuscator within Visual Studio, simply add a Dotfuscator Project to your Visual Studio Solution and identify what assemblies should be obfuscated. From the Dotfuscator Project you can configure the obfuscation options, including options for generating HTML reports. Finally, to obfuscate the selected assemblies, simply build the Solution. In addition to compiling the other projects, Visual Studio will also run Dotfuscator.

It's essential to protect source code in order to prevent lost revenue, to keep it out of the hands of competitors, and to reduce the likelihood of attackers discovering an exploit. To make such attacks much more difficult, you should consider using a tool like Dotfuscator to obfuscate your assemblies.

Price: $1,890 per user for the Professional Edition.

www.preemptive.com

Dotfuscator Project (Click the image for a smaller view)

Dotfuscator Project (Click the image for a larger view)

Persist State from a Database

Every single application I've created was at its core a means for users to interact with data residing in some database. Such applications can typically be divided into three tiers: the data tier, the application tier, and the presentation tier.

Persisting and retrieving information between the application and data tiers commonly requires a decent amount of repetitive code. For example, in an online shopping site, the application tier might contain objects like Product, Customer, and ShoppingCart. In order to populate a Product object with information about a particular item, the application tier needs to include code that establishes a connection to the database, issues the appropriate query, and populates the properties of the Product object with the returned database record's fields.

Code that takes care of all of these tasks is a pain to write. It's bulky, repetitive, and ripe for typos that have a way of manifesting themselves as runtime exceptions. One of the goals of an application tier is to abstract away the data tier so that you no longer need to think about the logical entities in your system in terms of database tables and SQL queries. Manually writing code to handle the marshaling of data between the application and data tiers runs counter to this goal.

Fortunately there are many tools—referred to as object-relational mappers (O/R mappers)—that can help automate this process. In addition to both commercial and open-source O/R mappers, Microsoft provides its own O/R mapper in the .NET Framework through the use of the strongly typed DataSet.

Hibernate is a popular open-source O/R mapper for Java that has been ported to the .NET Framework and renamed NHibernate version 1.0. NHibernate is a framework designed to automate moving data between the application and data tiers. Given an existing data model in the data tier and a set of business objects in the application tier, NHibernate can transfer the data from the data tier to the application tier or vice versa with a few lines of code. This transformation is possible through XML mapping files, which inform NHibernate what objects map to what database entities, and what object properties map to what database fields.

In addition to persisting and retrieving data between the two tiers, NHibernate provides functionality for querying and retrieving data. The Hibernate Query Language provides a SQL-like syntax for querying data directly against the business objects in the application tier.

Getting started with NHibernate is a relatively straightforward process—you simply download the framework from nhibernate.org, add the NHibernate assemblies to your application, and create the XML files that provide the mapping between your application's data and application tiers. If you already have the business objects in the application tier defined, but have yet to implement the data tier, NHibernate can generate the database schema automatically based on the XML mapping files.

Unfortunately, if you already have the data model implemented, but have yet to create the business objects, NHibernate isn't much help. While some O/R mappers will automatically generate the business objects and mapping information from an existing database schema, with NHibernate you must either create these business objects and mappings by hand or rely on other tools, such as CodeSmith (see the January 2006 installment of Toolbox) to generate them. With the mapping files created, marshaling data between the data and application tiers is as simple as working directly with the business objects and NHibernate's straightforward API.

If you build data-driven applications, consider looking into NHibernate as a means to automate persisting and retrieving objects to and from the data tier.

Price: Free. Includes complete source code.

www.nhibernate.org

The Bookshelf

Anytime a technology is upgraded, companies have to decide whether the costs and time required in migrating to the new version are outweighed by the new version's features and enhancements. With new versions of the .NET Framework, Visual Studio, and SQL Server™ all released in late 2005, this decision process is being played out across the world by companies both large and small.

Most computer trade books focus on exploring a particular subset of features in great depth and are ideal for developers interested in mastering a new technology. If, however, you want a high-level overview of a technology suitable for evaluation purposes (or for just gaining a broad understanding of the technology) you'll instead need to turn to a book like SQL Server 2005 Distilled (Addison-Wesley, 2006). Authored by former SQL Server team member Eric Brown, the book provides an executive summary of key features, such as security, database management, developer tools, business intelligence, working with XML data, CLR integration, and so on.

Apart from Chapter 6, "The Code Chapter," which has a hodgepodge of examples for performing various queries and administrative tasks, you won't find much T-SQL syntax or DBA-level walkthroughs. Instead, the other chapters provide their high-level overviews using mostly prose mixed with architectural and use case diagrams. And at 336 pages, SQL Server 2005 Distilled provides a digestible overview that can be read cover-to-cover over the course of a weekend.

If you're a developer needing to master the new T-SQL additions, this book probably won't meet all of your needs, as it isn't intended to sharpen your development skills or to transform you into a SQL Server guru. If, however, you are in search of a summary of the key new features of SQL Server 2005, consider picking up a copy.

Price: $34.99.

www.aw-bc.com

(Click the image for a smaller view)

(Click the image for a larger view)

Send your questions and comments for Scott to toolsmm@microsoft.com.

Scott Mitchell, author of numerous books and founder of 4GuysFromRolla.com, is an MVP who has been working with Microsoft Web technologies since 1998. Scott is an independent consultant, trainer, and writer. Reach him at Mitchell@4guysfromrolla.com or via his blog.


CLR Inside Out: IronPython

CLR Inside Out
IronPython
James Schementi

Contents
Dynamic Languages and Iterative Development
Line Noise
How the .NET Framework Fits In
IronPython Architecture
Putting IronPython to Work
Conclusion

IronPython is the code name for an implementation of the Python programming language written by the CLR team at Microsoft. IronPython runs on the Microsoft® .NET Framework and supports an interactive console with fully dynamic compilation. It is well integrated with the rest of the .NET Framework and makes all .NET libraries easily available to Python programmers, while maintaining full compatibility with the Python language.

This column will give a brief overview of Python and what sets dynamic languages apart from other languages. I will discuss iterative development, describe how IronPython integrates with .NET while staying true to Python syntax, and show the advantages of using IronPython to utilize .NET.

Dynamic Languages and Iterative Development

Dynamic programming languages allow for a program's structure to be changed while it runs: functions may be introduced or removed, new classes of objects may be created, and new modules may appear. These languages are usually dynamically typed, allowing variable declarations to change type during execution. But just having dynamic types does not make a language dynamic.

Python is a dynamic language with support for many programming paradigms such as object-oriented programming, structured programming, functional programming, and, with some extensions, aspect-oriented programming. While offering these choices, Python's designers are in favor of a sparse syntax to increase readability. It uses English keywords frequently where other languages use punctuation, and has notably fewer syntactic constructions than many structured languages. For example, Python uses indentation rather than curly braces to delimit blocks.

Here's an example of the Python function fib, which computes the Fibonacci numbers up to the specified value of n:

def fib(n):
   a, b = 0, 1
   while b < n:
      print b
      # equivalent to temp = a; a = b; b = temp + b (aka swap pattern)
      a, b = b, a+b

The most common question I hear about dynamic languages is "why should I use them?" Trying to show a longtime C++ or C# developer that a dynamic language is useful for problems outside the scripting domain is difficult. One concrete benefit of dynamic languages is that they facilitate an iterative development cycle. The interactive capabilities of dynamic languages make them a better match for this type of cycle than compiled languages. Developers can focus on exploring and prototyping functionality, refactoring if necessary, and then repeating with new functionality. This allows for a more exploratory process of development than the code/build/run/debug loop that most other languages constrain the programmer to.

So how are dynamic languages able to achieve this? Most provide an interactive console, a high-level syntax, and extensive standard libraries.

The interactive console allows you to type in an expression and see it evaluated right away. Figure 1 shows the IronPython console in action.

Figure 1 Programming the IronPython Interactive Console
Figure 1 Programming the IronPython Interactive Console(Click the image for a smaller view)
Figure 1 Programming the IronPython Interactive Console
Figure 1 Programming the IronPython Interactive Console (Click the image for a larger view)

Line Noise

If you compare the amount of code that goes into simple C# or Python classes, you'll see that the concise syntax of dynamic languages makes it easier to write code more quickly. When creating a new class in a C# application, you have to switch from a creative mode into more of a bookkeeping mode, creating a new file and typing a chunk of boilerplate code before attacking what the class actually does. Extra syntax that does not provide any additional expressiveness to the program can be referred to as line noise. Python is much less "noisy."

The line noise in C# is there for a reason, whether for static typing or explicit declaration, but it can make the code difficult to read. Dynamic languages try to take the configuration responsibilities away from the programmer and put it on the compiler and interpreter. Dynamic typing is so popular in dynamic languages because the compiler/interpreter keeps track of types rather than requiring the programmer to explicitly state them. Figure 2 and Figure 3 show common programming tasks, swap and multi-value returns, with a line-noise comparison between C# and Python. Allowing the developer to write less code that accomplishes more is one of the pillars of dynamic languages.

Figure 3 Swapping Two Variables in C# and Python (Close Figure 3)

C#
void Swap(ref int x, ref int y) {
    int temp = x;
    x = y;
    y = temp;
}
int x = 4, y = 2;
Swap(ref x, ref y);
Python
x, y = 4, 2
x, y = y, x
Figure 2 Multiple Return Values in C# and Python (Close Figure 2)

C#
object[] GetValues() {
    object[] results = new object[2];
    object[0] = 42;
    object[1] = "Hello";
    return results;
}
object[] results = GetValues();
int x = (int)results[0];
string s = (string)results[1];
Python
def get_values() :
  return 42, "Hello"

x, s = get_values()

How the .NET Framework Fits In

Most dynamic languages ship with a large standard library. This "batteries included" philosophy means dynamic languages can be used for a broad range of tasks. These libraries usually include modules for writing Internet-facing applications, creating graphical user interfaces, connecting to relational databases, and even unit testing frameworks. Having these tools be part of the language, rather than an extra download module or other third-party library, allows you to write code faster. One of the drawbacks is that these standard libraries vary among languages. A common set of libraries among all languages would allow programmers to move from language to language easily.

The .NET Framework is intended to be a single runtime engine for many programming languages. It does this via a shared byte code intermediate language, just-in-time (JIT) and ahead-of-time compilers, a highly tuned garbage collector, and reflection and dynamic loading support. The academic world has envisioned this common runtime for decades, but .NET is the first runtime engine with many languages used in production today. Microsoft maintains .NET support for C#, Visual Basic®, C++, J#, and JScript®, while others have integrated languages like Eiffel, COBOL, FORTRAN, RPG, and Delphi with .NET.

The common language runtime (CLR) enables deep integration among languages; a single project could use many languages that work together seamlessly. This allows developers to choose the best language for the problems they are trying to solve, rather than settling on one language. A multi-language framework builds value from a larger developer base and allows the framework to provide a richer set of libraries to more languages.

This vision sounds a lot like the libraries each dynamic language provides, except that the libraries are different from language to language. The .NET Framework already supports many languages, but it could leverage the dynamic language space by providing common libraries for them.

IronPython Architecture

IronPython has come a long way very quickly. Our goals are to have dynamic language support for the CLR, finish IronPython 1.0 as a working example, and help other languages target the CLR. IronPython is released under the Microsoft shared source license, and releases are scheduled every few weeks until the version 1.0 release. The quick development cycle allows the team to quickly address customer needs.

IronPython uses a common type of compiler architecture; the noticeable difference is in code generation. The IronPython byte code is MSIL, intermediate language for the CLR, and will eventually be converted into native code (see Figure 4), whereas CPython (the standard Python implementation, done in the C language) has a runtime loop that interprets its byte code. The IronPython libraries are written in C# rather than C and called from the compiled IL, which is also how IronPython accesses .NET libraries. If you are interested in learning more about how IronPython works, take a look at the shared source release on the IronPython Web site.

Figure 4 IronPython Compilation Process
Figure 4 IronPython Compilation Process

IronPython has two important, often-conflicting goals: it needs to be seamlessly integrated with .NET and it needs to be a true implementation of the Python language. As a first-class .NET language, IronPython integrates simply and easily with standard .NET features. This includes subclassing classes and implementing interfaces from .NET. IronPython knows about common .NET libraries and namespaces like System, which can be imported and used in a straightforward manner.

>>> import System
>>> from System.Collections import *
>>> h = Hashtable()
>>> h["a"] = "IronPython"
>>> h["b"] = "Tutorial"
>>> for e in h: print e.Key, ":", e.Value
>>> ...
a : IronPython
b : Tutorial

To utilize additional .NET libraries, the clr module must be imported and then specific assemblies referenced. The additional library assemblies can then be imported and used seamlessly. Once the library is imported, it can be used just as before.

>>> import clr
>>> clr.AddReference("System.Xml")
>>> from System.Xml import *
>>> d = XmlDocument()

IronPython needs to support all the dynamic features of Python, including adding existing fields, changing methods, and even changing types of instances at runtime. Also, the existing Python regression test and existing programs should run relatively unchanged, allowing today's Python users to use all the same scripts with IronPython.

This overlap of worlds introduces some interesting conflicts. IronPython handles them in ways that make both .NET and Python developers feel at home. Take the following code example:

>>> s = "  py"
>>> s.Trim()
Traceback (most recent call last):
  File , line 0, in <stdin>##2
AttributeError: 'str' object has no attribute 'Trim'
>>> import clr
>>> s.Trim()
'py'

If you ask both sets of developers what the first two lines of code do, they will give you different answers. The .NET developer will say it removes the whitespace from the front of s. The Python developer will say it throws an exception since Trim is not defined on a Python string. Both are correct. The .NET developer should not have to learn a new library just to use the language and the Python developer needs code to work as expected. The solution is to default to the standard Python libraries until clr is imported. After this the Python types will turn into .NET types and act as such.

Exceptions are another language design challenge. For example, the statement 2/0 will produce different exceptions in .NET and Python. The IronPython solution is to automatically convert the exception to the one that the developer wants to catch. See the "Resources" sidebar for links to more information.

Putting IronPython to Work

Since IronPython has full use of the .NET Framework, it can utilize Windows® Forms to create GUI applications, use C# code and extend any existing C# code, or even be integrated into Visual Studio®! The following walkthroughs are adapted from the IronPython tutorial, which is located in the IronPython installation directory under Tutorial\Tutorial.htm; it is a fairly comprehensive overview, so running through the tutorial after reading this column is recommended.

The code to initialize IronPython for Windows Forms is provided as winforms.py. To learn why this code is needed, look at winforms.py or the Windows Forms section of the tutorial.

Begin by making sure your current directory is the Tutorial folder and starting the IronPython console by running ipy.bat. You should see the console appear. Next, initialize Windows Forms by loading the winforms module/script:

>>> import winforms
The Python modules are automatically initialized (executed) upon import so the Windows Forms initialization code has executed as part of the import statement.

Now you can import the contents of the System.Windows.Forms and System.Drawing namespaces into the global namespace:

>>> from System.Windows.Forms import *
>>> from System.Drawing import *
Next, create an instance of the Form class and display it:
>>> f = Form()
>>> f.Show()
Now set the form Text property:
>>> f.Text = "My First Interactive Application"
At this point you should see the window shown in Figure 5.

Figure 5 Windows Form Dialog
Figure 5 Windows Form Dialog

Define a method to create a label at the position of a mouse click:

>>> def click(f, a):
>>>    l = Label(Text = "Hello")
>>>    l.Location = a.Location
>>>    f.Controls.Add(l)
Register the event handler:
>>> f.Click += click
Now clicking on the form with the mouse will add "Hello" labels as shown in Figure 6. You can also access the controls via mouse clicks and change them (see Figure 7):
>>> for i in f.Controls: i.Font = Font("Verdana", 15)
...
>>> for i in f.Controls: i.Text = "World"
...

Figure 6 Fancy Labels
Figure 6 Fancy Labels
Figure 7 Changing the Labels
Figure 7 Changing the Labels

After a few moments of clicking, the form will get quite crowded, so clear it out:

   >>> f.Controls.Clear()

Extending IronPython with C# is just as easy. This walkthrough will show you how to use C# to build a class that can be used from inside IronPython.

First, create a new file called csextend.cs and put the code shown in Figure 8 inside it.Open the Visual Studio 2005 Command Prompt and build the code using csx.bat. You can load the DLL you just built (csextend.dll) into IronPython. Then explore the Simple class using the built-in dir function, as shown here:

   >>> import clr
   >>> clr.AddReferenceToFile("csextend.dll")
   >>> import Simple
   >>> dir(Simple)
   ['Equals', 'GetHashCode', 'GetType', 'ToString',
   '__new__', '__repr__']
Now create an instance of the Simple class:
   >>> s = Simple(10)
   >>> print s
   Simple<10>

Figure 8 csextend.cs (Close Figure 8)
using System;
using System.Collections;

public class Simple {
    private int data;

    public Simple(int data) {
        this.data = data;
    }

    public override string ToString() {
        return String.Format("Simple<{0}>", data);
    }
}

The Visual Studio SDK ships with a Visual Studio Integration sample of IronPython. This features IronPython projects to be created, syntax coloring, debugging, and IntelliSense® in Python files and the IronPython console. This is another example of the benefits of being a full-fledged .NET citizen: Visual Studio support!

You can learn more about using IronPython in Visual Studio by visiting Aaron Marten's blog about Visual Studio Integration and following the direction at the post blogs.msdn.com/533273.aspx.

Resources

  • IronPython CodePlex Homepage
  • Python Homepage
  • Python Tutorial

Conclusion

Dynamic languages are what make agile software development and iterative processes successful. Their exploratory nature makes them fun for developers, and people can enjoy them with just an interactive console.

IronPython has shown that the .NET Framework is a powerful platform for dynamic languages. IronPython's approach can be generalized, allowing languages to easily be made available on the .NET Framework and giving developers the flexibility of many languages with familiar libraries.

Now that you have a broad overview of what IronPython is, give this dynamic language a try (if you haven't already). The tutorial, source code, and Web site are valuable resources that will make your IronPython experience fun!

Send your questions and comments to clrinout@microsoft.com.

James Schementi is a Program Manager intern at Microsoft. He is working on the IronPython sample applications and Web stories. As a graduate student at Worcester Polytechnic Institute in Massachusetts, he focuses on language design and compilers, especially dynamic languages, Web design and usability, and Web frameworks. His school Web site is located at http://users.wpi.edu/~jimsch06 and he can be reached at jimsch06@wpi.edu.


Inside MSDN: Consuming MSDN Web Services

Inside MSDN
Consuming MSDN Web Services
Craig Andera

Code download available at: InsideMSDN2006_10.exe (161KB)
Contents
Anatomy of the MTPS Content Service
Documents
Basic Viewer Design
GetContent
Populating the TreeView
GetNavigationPaths
Handling Errors
Now Get Creative

About three years ago, Microsoft rewrote the MSDN site from the ground up, developing the MSDN/TechNet Publishing System (MTPS). The new site, called MSDN2 (msdn2.microsoft.com), boasts some important enhancements made possible by MTPS, including improved performance, scalability, and usability. You can read more about this in the February 2005 edition of this column (msdn.microsoft.com/msdnmag/issues/05/02/InsideMSDN).

But from the very beginning, we on the MSDN® team had more in mind for MTPS than simply powering the MSDN2 front-end. We designed MTPS to be flexible, allowing the hundreds of thousands of articles, documents, whitepapers, images, and other resources on the site to be used in new and creative ways. The MTPS Content Service is an important step towards this goal.

The MTPS Content Service is a set of XML Web services that expose the full breadth of the MTPS content to your custom applications. If you are writing development tools you can, for example, create a class viewer application that automatically displays documentation for any common language runtime (CLR) types. Or you can use the services to populate a cache for a lightning-fast offline help viewer containing the documentation for the classes you use most often. Even if you're not writing development tools, you might find the Web services useful for creating a different view of the MSDN content, using a format that you like better than the MSDN2 style. Anything within the MSDN terms of service (microsoft.com/info/cpyright.mspx) is fair game.

In this column, I'll demonstrate the MTPS Content Service by writing a simple Windows® Forms viewer. It will use a TreeView control for navigation and a WebBrowser control for display to replicate the basics provided by the MSDN2 site. I'll even show you how to add local caching to improve performance. Without much code, I'll create an app that does a fair job of providing the same functionality as the MSDN2 Web site!

You can find the full documentation and Web Services Description Language (WSDL) for the MTPS Content Service at services.msdn.microsoft.com/ContentServices/ContentService.asmx.

Anatomy of the MTPS Content Service

The MTPS Content Service currently consists of exactly two operations: GetContent and GetNavigationPaths. These give callers the ability to discover and retrieve any of the content in MTPS, as well as to discover relationships between different pieces of content. Currently, no search functionality is exposed, but I'll briefly discuss a workaround at the end of this column.

Both GetContent and GetNavigationPaths are built upon the concept of a content item. A content item is a collection of documents identified by a content key. Before I get into documents, let me describe content keys.

A content key consists of three parts: an identifier, a version, and a locale. Identifiers are case-insensitive and can take on any of several forms, described in Figure 1.

Figure 1 Forms of Content Identifiers (Close Figure 1)

Identifier Type | Example | Description
Short ID | b8a5e1s5 | Short, easy-to-type identifier for a content item. Sometimes referred to as a “content ID.” Assigned by MTPS and cannot be inferred. Content items always have a Short ID.
GUID | d0ab826d-93d4-4b49-85d1-87fdc21f2b94 | Unique GUID for a content item. A content item always has a GUID.
Alias | system.xml.xmlreader | Easy-to-read, easy-to-remember identifier for a content item. Not every content item has one.
URL | http://msdn2.microsoft.com/library/system.xml .xmlreader | Primarily intended to provide an easy transition between various Microsoft Web sites and the MTPS Content Service.
Asset ID | AssetID:t%3asystem.xml.xmlnodetype | Semi-opaque MTPS-internal identifier for a content item. Always starts with “AssetID:”. A content item always has this type of identifier.

The second part of a content key is a version. We intend for MTPS to be around for a while, so we allow for more than one version of a particular content item. Versions are case-insensitive and consist of a dotted pair indicating a product family and version. For example, the documentation for the Microsoft® .NET Framework 2.0 version of the System.Xml.XmlReader class has a version of VS.80. Right now, most items in the system have only a single version, typically the one that corresponds to the .NET Framework 2.0. When the next version of .NET is released, a new content item will be entered into the system with the same content identifier, but with a different version. It will presumably be something like VS.90, but applications should treat versions as opaque strings for compatibility. Generally, if a version is not specified, the latest version is assumed.

The final part of a content key, locale, indicates which culture- or language-specific translation of a content item is desired. Including the locale as part of the content key allows MTPS to retain content simultaneously in several languages. Locales are case-insensitive, but otherwise follow the form outlined by System.Globalization.CultureInfo (and therefore outlined by RFC 1766 from the Internet Engineering Task Force). Thus, if you want to retrieve the U.S. English translation of a content item, you'd specify a locale of "en-us."

Documents

The content key identifies a content item that contains zero or more documents. A document always has a type and a format, and may also have a name and some content. The content contained by a document can be either XML or binary, depending on the type and format of the document. Figure 2 lists some of the currently defined document types.

Figure 2 Document Types (Close Figure 2)

Type | Example Formats | Description
Primary | Mtps.Xhtml, Mtps.Toc, Mtps.Failsafe | Holds the main content of a content item.
Common | Mtps.Links, Mtps.Search | Contains information applicable to all content within the system.
Feature | None currently defined | Contains auxiliary information that complements the content item.
Image | .gif, .jpg, .png | Images referenced by a primary document.

Arguably the most important document in most content items is the Mtps.Xhtml primary document. This format is generally compliant with the W3C's Modular XHTML 1.1 standard. Because of the large amount of legacy content in the system, complete compliance was not possible. But content is always well-formed XML, and in most cases it is also standards-compliant.

It's worth pointing out that the XHTML in MTPS is presentation-free. That is, the XHTML contains no script, styles, or other artifacts that control the look of the rendered content. This allows the XHTML to be used in different contexts by having styling applied at run time, after the document has been retrieved. The Mtps.Xhtml document does, however, contain some extension elements to enable certain behaviors on the MSDN2 Web site. (These are easy to detect, as they are not in the XHTML namespace.) If processing these extra elements poses problems for your application, you can use the Mtps.Failsafe primary document instead. This document is an equivalent to the Mtps.Xhtml document, but contains only XHTML elements.

Basic Viewer Design

Now that you understand what a content item looks like, let's start building a documentation viewer. Figure 3 shows the layout for the app, which I'll call MtpsClient. It has a TreeView for navigation on the left and a WebBrowser that renders the content for the currently selected node on the right. To keep this example focused on MTPS concepts, I won't do much with the XHTML beyond simply displaying it. (I'll leave styling and behaviors as an exercise for you.) For the same reason, I'm not going to deal with images or locales other than U.S. English.

Figure 3 MtpsClient Design
Figure 3 MtpsClient Design(Click the image for a smaller view)
Figure 3 MtpsClient Design
Figure 3 MtpsClient Design (Click the image for a larger view)

To begin, assume that when the user has selected a node in the TreeView, I want to retrieve and render the content for that node. This is done by hooking the AfterSelect event of the TreeView and making a call to GetContent with the appropriate content key that is stored in the selected TreeNode's Tag property for a Toc object. (I'll talk later about how to set up the Tag property properly.)

MtpsClient defines a class called ServiceHelper, where I've encapsulated all the Web service calls. After extracting the content key, AfterSelect ultimately calls ServiceHelper.GetContent (shown in Figure 4). ServiceHelper.GetContent returns the requested document from the cache if present (I use a simple directory with files named the ID of the document), and otherwise calls the Web service. In this example, the Web service call is far more interesting, so let's look at what's required.

Figure 4 ServiceHelper.GetContent (Close Figure 4)
public static string GetContent(string id, string locale, string version)
{
    // If the data is in the cache, retrieve it from there.
    if (IsInCache(id)) return GetCacheFileName(id);

    // Otherwise, go to the Web service to get it.
    getContentRequest request = new getContentRequest();
    request.contentIdentifier = id;
    request.locale = locale;
    request.version = version;

    // Make sure to request that the Web service send back the body of
    // the Mtps.Xhtml primary document. Without this you just get
    // summary information.
    requestedDocument[] requestedDocuments = new requestedDocument[1];
    requestedDocuments[0] = new requestedDocument();
    requestedDocuments[0].type = documentTypes.primary;
    requestedDocuments[0].selector = "Mtps.Xhtml";

    request.requestedDocuments = requestedDocuments;
    ContentService proxy = new ContentService();
    getContentResponse response = null;
    try
    {
        response = proxy.GetContent(request);
    }
    catch (SoapException e)
    {
        // If the call contains any MTPS-specific error messages, format
        // them for display and rethrow a new exception.
        throw MapSoapException(e);
    }

    // Look through the returned documents and pull out the body of the
    // Mtps.Xhtml document.
    XmlElement xhtml = GetXhtml(response);

    if (xhtml == null) throw new MtpsClientException(
        "No XHTML was returned from the web service.");

    // Cache the XHTML document.
    string cachefile = WriteXhtmlToCache(id, xhtml);
    return cachefile;
}

GetContent

GetContent's job is to return a content item for a content key. So the first thing I do after creating the request object is to set up the content key:

getContentRequest request = new getContentRequest();
request.contentIdentifier = id;
request.locale = locale;
request.version = version;
At this point, I could invoke the GetContent operation. The response, however, wouldn't actually contain any XHTML. By default, GetContent doesn't return the body of the documents; it just returns summary information, such as document format. This is because some documents can be quite large and returning all of the documents every time would often waste resources on both the client and the server.

To retrieve the contents of a document, I must use the requestedDocuments property on the request object. This is an array of requestedDocument objects. Each requestedDocument object has two properties: selector and type. The type property is a documentTypes enum that indicates the document type I am requesting (see Figure 2). The selector property allows me to indicate which particular document I want.

For primary, common, and feature documents, the selector indicates the format of the document—as only one document of a particular type and format will exist. For images, the selector indicates both the format and name of the desired image document, since more than one image in a particular format may exist. Name and format are specified as name.format.

I'm interested in the XHTML content, so I set the selector to Mtps.Xhtml and the type to primary, like so:

    requestedDocument[] requestedDocuments = new
         requestedDocument[1];
    requestedDocuments[0] = new requestedDocument();
    requestedDocuments[0].type = documentTypes.primary;
    requestedDocuments[0].selector = "Mtps.Xhtml";

    request.requestedDocuments = requestedDocuments;
And that's all that's required. From here, I can make the GetContent Web service call. The response is represented by an object of type getContentResponse. The properties of getContentResponse are shown in Figure 5, and can be broadly grouped into three categories: identity information, alternates, and documents. Identity information includes the contentId, contentGuid, contentAlias (if one exists), version, and locale. So one thing that GetContent can be used to do is to resolve the unknown parts of a content key. As long as you specify one valid identifier and a locale, you'll receive these other bits of the content item's identity in the response.

Figure 5 getContentResponse Properties (Close Figure 5)

Property | Description
contentId | The short ID of the content item. Always returned.
contentGuid | The GUID of the content item. Always returned.
contentAlias | The alias of the content item if it has one.
locale | The locale of the returned content item.
version | The version of the returned content item.
primaryDocuments | All the primary documents for this content item.
availableVersionsAndLocales | An array of availableVersionAndLocale objects listing all the alternative versions and locales available for the specified content identifier.
imageDocuments | All the image documents for this content item.
commonDocuments | All the common documents for this content item.
featureDocuments | All the feature documents for this content item.

The availableVersionsAndLocales property is an array of avail-able VersionAndLocale objects. Each availableVersionAndLocale lists an alternate version and locale for the specified content identifier. If, for example, you request the content item for System.Xml.XmlReader in U.S. English for the .NET Framework 2.0, you might receive a list of availableVersionAndLocale objects that tell you that System.Xml.XmlReader is also available in a .NET Framework 2.0/Italian flavor, .NET Framework 3.0/Italian flavor, and a .NET Framework 2.0/German flavor. You'll always receive information about alternates, even when you specify a version/locale combination that doesn't exist in the system.

For the purposes of MtpsClient, I'm only interested in the documents—specifically, the Mtps.Xhtml primary document. I retrieve this via a call to the method ServiceHelper.GetXhtml, which is shown in Figure 6. Note that the method loops through all the primary documents looking for a match on primaryFormat. Order is not guaranteed in the returned documents, and summary information is returned for all documents including ones not specified in requestedDocuments.

Figure 6 ServiceHelper.getXhtml Method (Close Figure 6)
private static XmlElement GetXhtml(getContentResponse response)
{
    return GetPrimaryDocument(response, "Mtps.Xhtml");
}
private static XmlElement GetPrimaryDocument(
    getContentResponse response, string format)
{
    foreach (primary document in response.primaryDocuments)
    {
        if (string.Compare(document.primaryFormat, format, true) == 0)
        {
            return document.Any;
        }
    }
    return null;
}

If a document of format Mtps.Xhtml is found, its Any property (which is simply a System.Xml.XmlElement object containing the XHTML document) is returned. Feature and common documents are similar, although they may contain more than one root XML element. As such, they are modeled as an array of XmlElement objects. Image documents are binary data and therefore have a Value property that is a byte array.

One important thing to consider is that links in both the Mtps.Xhtml and Mtps.Failsafe primary documents may need special handling from your application. Links between content items in these documents are by asset ID. While you can use subsequent calls to GetContent to resolve these AssetIDs into something more useful, this is inefficient. Instead, you can simply use requestedDocuments to retrieve the Mtps.Links common document at the same time you retrieve the primary documents. The Mtps.Links document, which is available for every content item that has primary documents that contain links, uses a simple XML format (shown in Figure 7) to associate every asset ID in the document with a short ID, a GUID, and possibly an alias.

Figure 7 Mtps.Links Document (Close Figure 7)
<mtps:links xmlns:mtps="urn:msdn-com:public-content-syndication">
  <mtps:link>
    <mtps:assetId>assetid:t%3asystem.xml.xmlurlresolver</mtps:assetId>
    <k:contentId xmlns:k=
        "urn:mtpg-com:mtps/2004/1/key">008et5fc</k:contentId>
    <k:contentGuid xmlns:k="urn:mtpg-com:mtps/2004/1/key">
        725ceabb-b81d-4b7e-8e5c-4b3179bdae5d</k:contentGuid>
    <k:contentAlias xmlns:k=
        "urn:mtpg-com:mtps/2004/1/key">
            system.xml.xmlurlresolver</k:contentAlias>
  </mtps:link>
  <mtps:link>
    <mtps:assetId>
        assetid:e695047f-3c0f-4045-8708-5baea91cc380</mtps:assetId>
    <k:contentId xmlns:k=
        "urn:mtpg-com:mtps/2004/1/key">2bcctyt8</k:contentId>
    <k:contentGuid xmlns:k=
        "urn:mtpg-com:mtps/2004/1/key">
            e695047f-3c0f-4045-8708-5baea91cc380</k:contentGuid>
    <k:contentAlias xmlns:k="urn:mtpg-com:mtps/2004/1/key" />
  </mtps:link>
</mtps:links>

To keep the code small and easy to understand, I won't use the Mtps.Links document in my MtpsClient application. Instead, I will call GetContent to resolve asset IDs to short IDs (even though this causes an unnecessary round-trip). But when you code your own applications, use Mtps.Links for the sake of efficiency.

Once the XHTML has been acquired, displaying it is easy. MtpsClient merely needs to apply a simple XSLT to render the desired page to a temporary directory, and it can then use the Web browser control's Navigate method to display the file. Refer to the source code download for this column at msdn.microsoft.com/msdnmag/code06.aspx for the complete details.

Populating the TreeView

When I first began designing the MTPS Content Service operations, I assumed there would be a document of type Mtps.Toc in every content item and that it would contain information about the content item's position within the system. I was wrong! Since items can exist in more than one location in the tree, content is extensively reused. There is a completely separate set of content items that track relationships, with a primary document of format Mtps.Toc. Figure 8 shows an example of an Mtps.Toc document. Notice that the Title attribute reflects the familiar hierarchy of a .NET class, with child nodes for members, the constructor, and so on.

Figure 8 Mtps.Toc Document (Close Figure 8)
<toc:Node toc:Title="XmlReader Class"
    toc:SubTree="AssetId:VS%7ccpref19%7c%24%5ccpref.hxt%400%2c0%2c47"
    toc:SubTreeVersion="VS.80" toc:SubTreeLocale="en-US"
    toc:Description="" toc:Target="AssetId:T%3aSystem.Xml.XmlReader"
    toc:TargetLocale="en-US" toc:TargetVersion="VS.80"
    toc:IsPhantom="false" xmlns:toc="urn:mtpg-com:mtps/2004/1/toc">
    <toc:Node toc:Title="XmlReader Members"
        toc:Target="AssetId:AllMembers.T%3aSystem.Xml.XmlReader"
        toc:TargetLocale="en-US" toc:TargetVersion="VS.80"
        toc:IsPhantom="false" />
    <toc:Node toc:Title="XmlReader Constructor"
        toc:Target="AssetId:M%3aSystem.Xml.XmlReader.%23ctor"
        toc:TargetLocale="en-US" toc:TargetVersion="VS.80"
        toc:IsPhantom="false" />
    <toc:Node toc:Title="XmlReader Methods"
        toc:SubTree="AssetId:VS%7ccpref19%7c%24%5ccpref.hxt
                     %400%2c0%2c47%2c2"
        toc:SubTreeVersion="VS.80" toc:SubTreeLocale="en-US"
        toc:Target="AssetId:Methods.T%3aSystem.Xml.XmlReader"
        toc:TargetLocale="en-US" toc:TargetVersion="VS.80"
        toc:IsPhantom="false" />
    <toc:Node toc:Title="XmlReader Properties"
        toc:SubTree="AssetId:VS%7ccpref19%7c%24%5ccpref.hxt
                     %400%2c0%2c47%2c3"
        toc:SubTreeVersion="VS.80" toc:SubTreeLocale="en-US"
        toc:Target="AssetId:Properties.T%3aSystem.Xml.XmlReader"
        toc:TargetLocale="en-US" toc:TargetVersion="VS.80"
        toc:IsPhantom="false" />
</toc:Node>

Of the remaining attributes, six are of particular interest. SubTree, SubTreeVersion, and SubTreeLocale combine to identify a navigation key. This is a content key of another content item with an Mtps.Toc document. Target, TargetLocale, and TargetVersion combine to identify a content key. They identify the content item containing the Mtps.Xhtml primary document with the documentation, article, or whatever it is that this TOC node represents. Another way to think of this is that the navigation key models the left-hand pane of the MSDN2 window, and the content key models the right-hand pane.

MtpsClient makes extensive use of Mtps.Toc documents. When the program first starts, it uses my ServiceHelper class to retrieve the Mtps.Toc document for the root of the MSDN documentation tree (short ID ms310241, locale en-us, version MSDN.10). It then deserializes this document into an XmlSerializable class of type Toc, like so:

// Retrieve the Mtps.Toc document from the Web service.
XmlElement tocElement = ServiceHelper.GetToc(id, locale, version);

// Deserialize it into a Toc object.
XmlSerializer ser = new XmlSerializer(typeof(Toc));
XmlNodeReader reader = new XmlNodeReader(tocElement);
Toc toc = (Toc)ser.Deserialize(reader);
The Toc class simply provides a convenient object-oriented view over the Mtps.Toc document.

With the Mtps.Toc information in a convenient format, I can use it to populate the TreeView by looping over its children and creating a TreeNode for each. By setting the Text property to the Title of the corresponding Toc object, I get the familiar MSDN documentation hierarchy.

I can tell whether a particular child node has children by examining its SubTree property. If specified, it indicates the ID of a document that contains the Mtps.Toc document for that node. If it's present, a user will be able to drill further down into the navigation tree from that point. I use this information to add a dummy child node to any node that itself has children. It doesn't really matter what the node says—I retrieve the Mtps.Toc document and populate the children of that node when the node is expanded.

The only other thing I need to do is store the deserialized Toc object in the Tag property of the TreeNode it corresponds to. As mentioned previously, this lets me easily display the right content when a node is selected, by examining the Target, TargetLocale, and TargetVersion properties of the Toc object to get a content key. The complete code for this sequence is shown in Figure 9.

Figure 9 Populating the TOC Tree (Close Figure 9)
private void PopulateNode(
    string id, string locale, string version, TreeNode parent)
{
    // Let the TreeView control know that I’m going to be making a bunch
    // of changes. Calling this now prevents flickering.
    parent.TreeView.BeginUpdate();
    try
    {
        // Clear out the children of this TreeNode and re-add them based
        // on the information in the Mtps.Toc document.
        parent.Nodes.Clear();

        // If there’s no ID (which there may not be in cases where we’re
        // populating a leaf node) then we’ve done all we can.
        if (string.IsNullOrEmpty(id)) return;

        // Retrieve the Mtps.Toc document from the Web service.
        XmlElement tocElement = ServiceHelper.GetToc(id, locale, version);

        // If no TOC element is returned, that means this node is a leaf
        // node so there’s no more work to do.
        if (tocElement != null)
        {
            // Deserialize it into a Toc object.
            XmlSerializer ser = new XmlSerializer(typeof(Toc));
            XmlNodeReader reader = new XmlNodeReader(tocElement);
            Toc toc = (Toc) ser.Deserialize(reader);

            foreach (Toc child in toc.Children)
            {
                TreeNode treeNode = new TreeNode();
                treeNode.Text = child.Title ?? "[no title]";

                // I store the Toc object right in the TreeNode so I can
                // later look up things like the content key for this
                // node.
                treeNode.Tag = child;

                // I can tell from looking at the Toc data whether a node
                // has children or not—the SubTree property will be
                // missing for nodes that don’t. So I can add a dummy
                // node for any nodes that have children, which will cause
                // the little plus sign to appear in the tree.
                if (!string.IsNullOrEmpty(child.SubTree))
                {
                    TreeNode placeholder = new TreeNode("Loading...");
                    treeNode.Nodes.Add(placeholder);
                    treeNode.Collapse();
                }

                // Red nodes indicate data that’s in our local cache.
                if (ServiceHelper.IsInCache(child.Target))
                {
                    treeNode.ForeColor = Color.Red;
                }

                parent.Nodes.Add(treeNode);
            }
        }
    }
    finally
    {
        // Let the TreeView know I’m done making updates, and that it
        // can redraw itself now.
        parent.TreeView.EndUpdate();
    }
}

GetNavigationPaths

Clicking through the TreeView is not the only way to navigate through content in MtpsClient. The Mtps.Xhtml document is full of links, and I'd like the user to be able to follow the links. Even better, I'd like the TreeView to synchronize to the clicked link, showing where in the documentation tree the displayed content item falls. But GetContent doesn't really provide a good way to do that, short of performing a brute-force search of Mtps.Toc documents. That would potentially require many round-trips to the Web service, making it prohibitively expensive and slow. Enter GetNavigationPaths.

GetNavigationPaths returns information about how the content items in MTPS relate to each other, even if they are not immediately adjacent in the navigation tree. We thought about calling it GetTableOfContentsData, but it is somewhat broader than that. For example, the information can also be used to generate the eyebrow (or breadcrumb) trail of topics you see at the top of an MSDN2 page. In MtpsClient, I use it to synchronize the navigation tree.

The parameters to GetNavigationPaths are conceptually pretty simple: you give it two content keys, one identifying the "from" content item and one identifying the "to" content item. The response is a list of paths between those two items.

Examining the parameters in more detail, you can see that GetNavigationPaths takes a getNavigationPathsRequest object. This object has root and target properties. Both of these are of type navigationKey and consist of an identifier, a version, and a locale. The identifier must be a short ID—aliases, GUIDs, asset IDs, and URLs are not legal identifiers. If necessary, you can resolve one of these other forms to a short ID with a call to GetContent first.

The other thing to note about the navigation keys is that root should identify a navigation node, and target should identify a content node. That is, the ID for root should be for a content item with a primary document of type Mtps.Toc, and the ID for target should identify a content item with a primary document of type Mtps.Xhtml. Put simply, GetNavigationPaths tells you possible routes between a particular point in the left-hand tree and a particular right-hand document. Most of the time you'll simply request paths from the documentation root (short ID ms310241, version MSDN.10) to some content node.

The response to GetNavigationPaths is a getNavigationPathsResponse object, which contains an array of navigationPath objects. Each item in this array represents one possible route between the root and target passed in the request. Remember, an item can appear more than once in the tree, so there may be more than one way to get from the root to the target. A navigationPath is itself an array of navigationPathNode objects—one for each step in the path between the root and target. And navigationPathNode is essentially the same as the TOC XML shown in Figure 8—it has a title, two keys (one for the navigation node itself and one for the content node it corresponds to), and information about whether the node is a phantom. (Phantoms are outside the scope of this column. Consult the online documentation for more details.)

Figure 10 shows how MtpsClient uses GetNavigationPaths to synchronize the TreeView with a content item. GetNavigationKeyFromId is a simple helper method that uses GetContent to resolve an asset ID to a short ID.

Figure 10 Synchronizing the TOC (Close Figure 10)
private void SyncToc(string id, string locale, string version)
{
    // Retrieve the navigation path between the root of the TOC and the
    // specified node.
    navigationPathNode[] path = ServiceHelper.GetNavigationPath(
        id, locale, version);

    // Now walk down the tree, matching each path node if possible. I
    // match by title because the IDs in the TOC are asset IDs, but the
    // IDs we get back from GetNavigationPaths are short IDs.
    TreeNode currentNode = navigationTree.Nodes[0];
    for (int i = 1; i < path.Length; i++)
    {
        foreach (TreeNode node in currentNode.Nodes)
        {
            Toc nodeInfo = node.Tag as Toc;
            if (nodeInfo != null && nodeInfo.Title == path[i].title)
            {
                node.Expand();
                currentNode = node;
                break;
            }
        }
    }

    navigationTree.SelectedNode = currentNode;
}

public static navigationPathNode[] GetNavigationPath(
    string id, string locale, string version)
{
    // I start off with the root of the documentation tree, which has a
    // well-known content key.
    navigationKey root = new navigationKey();
    root.contentId = "ms310241";
    root.locale = "en-us";
    root.version = "msdn.10";

    // Since I don’t know whether the specified ID is an asset ID, a
    // short ID, or so on, I have to resolve it to a short ID first—
    // GetNavigationPaths only accepts short IDs. Also, I may not know
    // the version, so I grab that in the same round-trip.
    navigationKey target = GetNavigationKeyFromId(id);
    if (!string.IsNullOrEmpty(version))
    {
        target.version = version;
    }

    getNavigationPathsRequest request = new getNavigationPathsRequest();
    request.root = root;
    request.target = target;

    ContentService proxy = new ContentService();

    getNavigationPathsResponse response = null;
    try
    {
        response = proxy.GetNavigationPaths(request);
    }
    catch (SoapException e)
    {
        // If the call contains any MTPS-specific error messages, format
        // them for display and rethrow a new exception.
        throw MapSoapException(e);
    }

    if (response == null || response.navigationPaths.Length == 0)
    {
        throw new MtpsClientException("No navigation information " +
            "was returned from the web service.");
    }

    return response.navigationPaths[0].navigationPathNodes;
}

Handling Errors

What if you call GetContent or GetNavigationPaths with erroneous input? It is illegal to omit version or locale when calling GetNavigationPaths. Or say the content identifier you specified in a call to GetContent does not exist in MTPS. In these cases, the MTPS Content Service returns a SOAP Fault element that contains extra information about the problem. This information can be retrieved by catching the SoapException that is thrown on the client side of the Web service call and accessing its Detail property. This property is an XmlNode that contains three items: an eventId, a source, and a helpLink.

MtpsClient handles errors by catching any SoapExceptions and passing them to the MapSoapException helper method. This method checks to see if MTPS has provided any extra information and, if so, formats the information into a helpful error message and tucks the info into an MtpsClientException. MtpsClient attaches an unhandled exception handler to the Application.ThreadException event and watches for exceptions of type MtpsClientException, displaying a friendly error message when one is detected. The extra info provided by MTPS is described in Figure 11; the MapSoapException helper method is shown in Figure 12.

Figure 12 Extracting Error Details from an MTPS SOAP Fault (Close Figure 12)
private static Exception MapSoapException(SoapException e)
{
    string eventId = null;
    string helpLink = null;
    string source = null;
    XmlNode detailNode = e.Detail;

    // If the detail node contains the mtpsFaultDetail element, then I
    // try to extract the eventID, helpLink, and source elements from the
    // fault message.
    if (detailNode.NamespaceURI ==
            "urn:msdn-com:public-content-syndication" &&
        detailNode.LocalName == "mtpsFaultDetail")
    {
        XmlNamespaceManager nsMgr =
            new XmlNamespaceManager(detailNode.OwnerDocument.NameTable);
        nsMgr.AddNamespace(
            "mtps", "urn:msdn-com:public-content-syndication");
        if (detailNode.LocalName == "mtpsFaultDetail")
        {
            XmlNode helpLinkNode =
                detailNode.SelectSingleNode("mtps:helpLink");
            if (helpLinkNode != null)
            {
                helpLink = helpLinkNode.Value;
            }
            XmlNode eventIdNode =
                detailNode.SelectSingleNode("mtps:eventId");
            if (eventIdNode != null)
            {
                eventId = eventIdNode.Value;
            }
            XmlNode sourceNode =
                detailNode.SelectSingleNode("mtps:source");
            if (sourceNode != null)
            {
                source = sourceNode.Value;
            }
        }
    }

    string message = "ERROR: The MSDN web service reported the " +
        "following error: " + e.Message;

    // After extracting eventId, source, and helpLink information
    // from the fault, I append them to the message.

    if (eventId != null && eventId.Length > 0)
    {
        message += "\nThe event ID was " + eventId;
    }
    else
    {
        message += "\nNo event ID was reported.";
    }

    if (source != null && source.Length > 0)
    {
        message += "\nThe source was " + source;
    }

    if (helpLink != null && helpLink.Length > 0)
    {
        message += "\nFor more infomation, see " + helpLink;
    }

    return new MtpsClientException(message, e);
}
Figure 11 MTPS Fault Detail Properties (Close Figure 11)

Field | Description | Example
eventId | Code that identifies what went wrong. | mtpsContentIdentifierNotFound
Source | Source of the error. Typically the name of the Web service operation. | GetContent
helpLink | URL that contains helpful information on how to resolve the error. | services.msdn.microsoft.com/ContentServices/ContentService.asmx

Now Get Creative

Now that you know how to use the MTPS Content Service, you can start designing your own uses for the information. We look forward to seeing what the developer community will dream up. There's great satisfaction when someone comes up with a great idea for the system. So I challenge you to come up with the killer app for the MTPS Content Service!

Just so you don't think that I'm asking you to do something I wouldn't do myself, I've published msdnman, an open-source tool modeled after the UNIX man command. It uses the MTPS Content Service to provide command-line access to the documentation managed by MTPS. Aside from providing a real-world example of how to deal with some of the more subtle issues around using the MTPS Content Service, it also demonstrates how to use search functionality (which the MTPS Content Service currently lacks) via the MSN Search Web service.

Send your questions and comments to insdmsdn@microsoft.com.

s
Craig Andera is a consultant specializing in the design and implementation of large-scale .NET systems and was a member of the team that implemented both MTPS and the MTPS Content Service. He has his own consulting company, Wangdera Corporation, where he holds the title of Jedi Master. He can be contacted via his blog at pluralsight.com/craig.


Test Run: Competitive Analysis Using MAGIQ.

Test Run
Competitive Analysis Using MAGIQ.
Dr. James McCaffrey and Nasa Koski

Contents
A Little Bit of MAGIQ
Setting Up the Problem
Determining Weights of the Attributes
Compare Each System on Each Attribute
Final Evaluation
MAGIQ versus AHP

Competitive analysis, an important type of software testing, involves comparing your software system under test with competitor systems. Even if your system does not have explicit competitors, you can consider previous builds of your own system as implied competitors. The goal is to be able to compare the overall quality of your software system with similar systems. The key here is overall quality, which is not so easy to measure. You must take into account many different attributes such as functionality, security, and usability. You must determine how to rate each attribute of each system, and you must find a way to combine all the attribute ratings into a single quality metric for each system.

In this month's column, I'm joined by Nasa Koski, a test lead in the MSN® division at Microsoft, and together we'll attempt to tackle the problem, presenting a methodology that seeks to translate data obtained through both measurement and perception into more easily comparable numeric statistics or metrics.

We'll demonstrate and explain a powerful but very simple method called the Multi-Attribute Global Inference of Quality (MAGIQ). This technique will enable you to quickly calculate the overall quality of your software system under test, thus allowing you to perform competitive analysis.

A Little Bit of MAGIQ

The MAGIQ technique is very general and can be applied to virtually any type of system including traditional applications, Web applications, class libraries, and so forth. Suppose, for example, you're developing a software system called System A that performs intranet searches. Imagine you want to rate it against four competitors' systems: System W, System X, System Y, and System Z. Using the MAGIQ procedure you will be able to compute metrics like those shown in Figure 1.

Figure 1 MAGIQ Competitive Analysis Metrics (Close Figure 1)

Performance (0.2500) |  |  |  | Accuracy (0.7500) |  | Overall
 | Startup (0.2778) | Search (0.6111) | Save (0.1111) | Top Result (0.7500) | First Screen (0.2500) |
System A | 0.4567 | 0.2567 | 0.4567 | 0.2567 | 0.0900 | 0.2449
System W | 0.0900 | 0.0400 | 0.0900 | 0.0400 | 0.1567 | 0.0667
System X | 0.2567 | 0.1567 | 0.0400 | 0.4567 | 0.2567 | 0.3479
System Y | 0.1567 | 0.4567 | 0.2567 | 0.0900 | 0.0400 | 0.1459
System Z | 0.0400 | 0.0900 | 0.1567 | 0.1567 | 0.4567 | 0.1947

Figure 1 shows the comparison of the five software systems based on performance and accuracy attributes. Both of these attributes have subattributes. The overall quality of System X is best with a quality metric of 0.3479 and the quality of System W is worst at 0.0667. Based on these results we conclude that our product, System A, is currently the second best of the five systems. In the sections that follow we will set up the problem that leads to the results in Figure 1, detail how to use the MAGIQ process, describe how to interpret the results of the technique, and explain how you can adapt this procedure to meet your own needs. We think you'll find the ability to quickly conduct competitive analysis an important addition to your software testing and project management skill sets.

Setting Up the Problem

In practice we use the MAGIQ technique most often to compare relatively similar systems in order to gauge relative product quality, but we've also used it to compare different builds of a particular system in order to gauge quality progress. In order to compare the overall quality of a set of software systems, we have to take into account many attributes of the system. For the purposes of the hypothetical intranet search system example presented here, we are considering just performance and accuracy. In a realistic scenario, we would also consider attributes such as security, functionality, usability, and so forth. The MAGIQ technique is very adaptable and can handle any number of attributes.

Next we decide to split the performance attribute into three subattributes: start-up performance, core search speed, and save performance. We also decide to break things down into the accuracy of the top result (how good the number one search result is), and first-screen accuracy (how good are the first 10 or so search results that fill the initial results screen). As we'll demonstrate, MAGIQ can accommodate as many levels of attribute decomposition as you want.

Our goal is to produce a single numeric value that represents the overall quality for each software system. After deciding which attributes you are going to use as the quality criteria, you would then have to determine how to compare each system on each attribute, determine how to assign values for that information, and then finally figure out how to combine all that data into a meaningful, overall quality metric, and how to interpret the results.

Figure 2 shows the example expressed as a diagram. The diagram form might seem strange compared to, say, a table, but as we'll explain later this form makes some of the MAGIQ calculations easier to understand. The diagram form tends to be a bit awkward for software engineers to interpret because the hierarchical structure combines different elements (problem, attributes, systems) than the more usual structure which relates similar elements (directories and files). In the end, any form of problem setup is okay as long as it clearly conveys what the problem is and what attributes you are using as the basis for comparison in the MAGIQ analysis.

Figure 2 Hierarchical Diagram Representation of Problem
Figure 2 Hierarchical Diagram Representation of Problem

A problem diagram or table is a convenient way to clearly communicate exactly what your MAGIQ quality metrics are, especially to people external to your group. An additional advantage is that it forces you to thoroughly analyze exactly which factors you are going to use as comparison criteria. This process often uncovers issues you may have overlooked.

Determining Weights of the Attributes

The next step is to determine the relative weights of each of the comparison attributes. In most situations, certain attributes are more important than others. For example, overall system performance may be more important to us than overall accuracy or vice versa. The MAGIQ technique uses an interesting concept called rank order centroids (ROCs). ROCs are a way to convert ranks (such as first, second, third) into ratings or weights, which are numeric values (such as 0.6, 0.3, 0.1).

For this example, we begin by looking at our top-level attributes: performance and accuracy. We rank them from most important to least important. Here we determine that overall accuracy is most important and overall performance is second. Then we compute the ROC for each of these two high-level attributes:

Accuracy:      w1 = (1 + 1/2) / 2 = 0.7500
Performance:   w2 = (0 + 1/2) / 2 = 0.2500

We will explain the calculations in a moment but the result is that accuracy is assigned a weight of 0.7500 and performance is assigned a weight of 0.2500. Next we compute the ROCs for each subattribute group. For the three performance subattributes, we decide that core search performance is most important, startup performance is second, and save performance is third. So the weights are as follows:

Search performance:     w1 = (1 + 1/2 + 1/3) / 3 = 0.6111
Startup performance:    w2 = (0 + 1/2 + 1/3) / 3 = 0.2778
Save performance:       w3 = (0 +  0  + 1/3) / 3 = 0.1111

Then for the two accuracy subattributes, we decide that top-result accuracy is most important and that first-screen accuracy is second in importance. So those weights are calculated:

Top-result accuracy:    w1 = (1 + 1/2) / 2 = 0.7500
First-screen accuracy:  w2 = (0 + 1/2) / 2 = 0.2500

Now, let's see how ROCs are calculated. The calculation pattern is pretty easy to see, especially if we show you what it looks like for a set of four items:

w1 = (1 + 1/2 + 1/3 + 1/4) / 4 = 0.5208
w2 = (0 + 1/2 + 1/3 + 1/4) / 4 = 0.2708
w3 = (0 +  0  + 1/3 + 1/4) / 4 = 0.1458
w4 = (0 +  0  +  0  + 1/4) / 4 = 0.0625

Notice that the ROC values add up to 1.0 (subject to rounding error). Expressed in sigma notation, if N is the number of attributes then the weight of the kth attribute is:

[ [image]] / N

This is easy to compute. For example, the code for a simple C# implementation without any error-checking is shown in Figure 3.The Math Behind Rank Order Centroids

Although a deep explanation of the mathematics behind rank order centroids is outside the scope of this column, we’ll briefly describe the rationale. The idea is to convert ranks (first, second, third, fourth) into values that are normalized on a 0.0 to 1.0 interval scale. An obvious way to try this is to assume each rank is distributed evenly within the unit interval. So, first => 0.80, second => 0.60, third => 0.40, and fourth => 0.20. But ranks are really a form of rate data—1st/4, 2nd/4, and so forth. And you may remember from an elementary statistics class that rate data is best handled using harmonic techniques. For example the average of 30 mph and 60 mph over a fixed distance is not (30 + 60) / 2 = 45 mph, but rather 2 / (1/30 + 1/60) = 40 mph. Notice that calculations for ROCs are similar. The concept of a centroid of a feasible space (simplex) is not new. The term "rank order centroid" was coined by F. H. Barron and B. E. Barrett, who also argued for its use in multiattribute decision problems.

Figure 3 Generate Rank Order Centroids (Close Figure 3)
using System;

class Program
{
   static void Main(string[] args)
   {
      try
      {
         Console.WriteLine("\nGenerating rank order centroids\n");
         int N = 5;
         Console.WriteLine("N = " + N);
         Console.WriteLine("===========");
         for (int k = 1; k <= N; ++k)
         {
            Console.WriteLine("w" + k + " = " +
               roc(N, k).ToString("0.0000"));
         }
         Console.WriteLine("\nDone");
      }
      catch (Exception ex)
      {
         Console.WriteLine("Fatal error: " + ex.Message);
      }
      Console.ReadLine();
   }

   static double roc(int N, int k)
   {
      double result = 0.0;
      for (int i = k; i <= N; ++i)
      {
         result += (1.0 / i);
      }
      return result / N;
   }
}

Running the code in Figure 3 for N = 5 results in the following rank order centroids:

N = 5
----------
w1 = 0.4567
w2 = 0.2567
w3 – 0.1567
w4 = 0.0900
w5 = 0.0400

Note that for any given value of N, there will be N weights and that the values of these weights are always the same. Therefore, an alternative to computing the weights for a given N each time you perform a MAGIQ analysis is to simply construct a set of tables for various values of N. For example:

N = 2
-----------
w1 = 0.7500
w2 = 0.2500

N = 3
-----------
w1 = 0.6111
w2 = 0.2778
w3 = 0.1111

Once you have these tables, you can simply do a look-up when needed. This approach is particularly useful if you distribute the responsibility for doing competitive analysis using MAGIQ among several members of your team. For more on the calculations, see the sidebar "The Math Behind Rank Order Centroids."

Compare Each System on Each Attribute

The next step is to compare each system based on each of the lowest level comparison attributes. In this example that means we need to rank System A against System W against System X against System Y against System Z on each of the five comparison attributes: startup performance, core search performance, save performance, top-result accuracy, and first-screen accuracy. This process is exactly the same as the attribute comparison process. We rank each subattribute, then convert the ranks into priority weights using rank order centroids. For startup performance, suppose we determine that my System A is best, System X is ranked second, System Y third, System W fourth, and System Z fifth. Then the ROCs for startup performance are as follows:

System A: 1st: w1 = (1 + 1/2 + 1/3 + 1/4 + 1/5) / 5 = 0.4567
System X: 2nd: w2 = (0 + 1/2 + 1/3 + 1/4 + 1/5) / 5 = 0.2567
System Y: 3rd: w3 = (0 +  0  + 1/3 + 1/4 + 1/5) / 5 = 0.1567
System W: 4th: w4 = (0 +  0  +  0  + 1/4 + 1/5) / 5 = 0.0900
System Z: 5th: w5 = (0 +  0  +  0  +  0  + 1/5) / 5 = 0.0400

For search performance, we determine that the systems rank, from best to worst, as Y, A, X, Z, W. For save performance, the rank is A, Y, Z, W, X. For top-result accuracy we determine that the systems rank X, A, Z, Y, W. And for first-screen accuracy, we rank the systems as Z, X, W, A, Y. These rankings are summarized in the table in Figure 4. Using these ranks we compute the relative weights of each system using ROCs as already described.

Figure 4 System Rankings (Close Figure 4)

 | Startup | Search | Save | Top | Screen
1st | A | Y | A | X | Z
2nd | X | A | Y | A | X
3rd | Y | X | Z | Z | W
4th | W | Z | W | Y | A
5th | Z | W | X | W | Y

Final Evaluation

After setting up the problem, computing the attribute priority ratings, and computing the system comparison ratings, the last step is to aggregate all the intermediate data to produce the final evaluation metrics. It's easiest to explain by example. If we put the intermediate results into a table, the result is Figure 5.

Figure 5 Intermediate Results (Close Figure 5)

Performance (0.2500) |  |  |  | Accuracy (0.7500)
 | Startup (0.2778) | Search (0.6111) | Save (0.1111) | Top Result (0.7500) | First Screen (0.2500)
System A | 0.4567 | 0.2567 | 0.4567 | 0.2567 | 0.0900
System W | 0.0900 | 0.0400 | 0.0900 | 0.0400 | 0.1567
System X | 0.2567 | 0.1567 | 0.0400 | 0.4567 | 0.2567
System Y | 0.1567 | 0.4567 | 0.2567 | 0.0900 | 0.0400
System Z | 0.0400 | 0.0900 | 0.1567 | 0.1567 | 0.4567

The final overall quality value for System A is:

(.2500)(.2778)(.4567) +
(.2500)(.6111)(.2567) +
(.2500)(.1111)(.4567) +
(.7500)(.7500)(.2567) +
(.7500)(.2500)(.0900) = 0.2449

The final quality metric for each system is the weighted sum of its attribute rankings. We obtain this by following the attribute tree to the value for the system. For System A this can be expressed as:

(PerformanceRank * StartRank * SystemA_StartRank) +
(PerformanceRank * SearchRank * SystemA_SearchRank) +
(PerformanceRank * SaveRank * SystemA_SaveRank) +
(AccuracyRank * TopResultRank * SystemA_TopResultRank) +
(AccuracyRank * FirstScreenRank * SystemA_FirstScreenRank)
This expression looks more complicated than it really is. If you match the numbers in the calculation above with the attribute tree in Figure 2, you'll see exactly how the calculation works. In the same way, the final quality value for System W is:
(.2500)(.2778)(.0900) +
(.2500)(.6111)(.0400) +
(.2500)(.1111)(.0900) +
(.7500)(.7500)(.0400) +
(.7500)(.2500)(.1567) = 0.0667

The final quality metrics for Systems X, Y, and Z are calculated similarly, producing the results shown in Figure 1. Notice that the structure of the table in Figure 5 mirrors the diagram form of the problem statement shown in Figure 2.

At this point we can interpret the quality of the five systems. Because the MAGIQ quality values sum to 1.0 you can compare each system by looking at its overall quality metric. System X is clearly best at 0.3479. System W is clearly worst at 0.0667. System A is second best, followed by System Z and then System Y. The four-decimal precision of the data is somewhat deceptive. Because the original ranking input data is so crude, based on our experience you should only look at two decimals. Additionally, we suggest that you use MAGIQ data primarily to monitor trends. A single MAGIQ analysis provides valuable information, but if you perform a series of MAGIQ analyses over time, you will get additional information about the relative quality of your systems under test.

You should use caution when interpreting the magnitude of the MAGIQ quality metrics. In this example, System X has an overall quality metric of 0.3479 and System A is at 0.2449. It is mathematically correct to say that, relative to System X, System A is (0.3479–0.2449) / 0.3479 = 30 percent worse than System X. However, the magnitudes of the quality metrics depend in part upon the number of systems being compared. Suppose for example, we were comparing only four systems instead of five. We would likely get the same order of results, but the magnitudes would be different, and therefore the relative difference between the quality metrics would differ too. If you adopt MAGIQ as one of your competitive analysis techniques, you'll quickly gain an intuitive sense of what the magnitudes of your metrics mean. As a rule of thumb, when comparing four or five software systems, we have found that a difference of 0.10 between two systems is significant (in the normal use of the word rather than in the statistical sense).

There is another argument for using rank order centroids. Imagine that you have just two items, A and B, which are ranked first and second, respectively. Because you know A is better than B, A's rating must be somewhere between 0.5 and 1.0, and B's rating must be somewhere between 0.0 and 0.5. If you take the midpoint of each of these two intervals, you get 0.7500 and 0.2500 as with ROCs. With just two items, you are essentially working on a one-dimensional line. ROCs generalize this concept across an N-dimensional space.

One detail to address is what to do in the case of tied rankings for comparison attributes or systems under comparison. In this situation you compute ROCs for the tied attributes or systems, arbitrarily ranking one higher than the other, then take the average of the resulting rating values. A quick example will show you what we mean. Suppose you are comparing four systems, A, B, C, and D, on some attribute (say authentication security). You determine that system A is best, systems B and C are tied, and System D is fourth. Without the tie, the rank order centroid weights are w1 = 0.5208, w2 = 0.2708, w3 = 0.1458, and w4 = 0.0625. You take the values for the tied systems (w2 and w3), compute their average (0.2708 + 0.1458 / 2 = 0.2803), and use that average for each of the tied systems.

MAGIQ versus AHP

The MAGIQ technique is a close cousin to a multi-attribute technique called the analytic hierarchy process (AHP). In Test Run in the June 2005 issue of MSDN®Magazine I describe the use of AHP for build quality (see msdn.microsoft.com/msdnmag/issues/05/06/TestRun). We originally developed MAGIQ as a way of validating build quality metrics obtained using AHP. The analytic hierarchy process decomposes a problem very much like MAGIQ, but AHP uses a pair-wise comparison method instead of rank order centroids. Pair-wise comparisons produce more accurate attribute weights than ROCs, but they take a lot of time and effort. For example, with 7 systems and 10 comparison attributes, you would have 21 comparisons per system times 10 attributes equal to 210 comparisons to perform. We soon discovered that our MAGIQ results correlated almost perfectly with our AHP results, so we started using the quicker MAGIQ technique on a daily basis, and used AHP just once a week to validate our MAGIQ metrics.

Another good feature of the MAGIQ technique is that it is very easy to understand. With just a few minutes explanation (or this column), everyone on your team can learn how to perform a MAGIQ analysis. This allows you to either distribute responsibility for competitive analysis or to have multiple evaluators. Allowing several people on your team to perform competitive analysis using MAGIQ can have a large morale-building effect since team members will be making a significant contribution to the overall system under development.

Although we've used the MAGIQ technique on several significant software products, it has not been subjected to serious academic investigation or research. That said, however, we think you'll find that MAGIQ can be an extremely useful complement to other, traditional software quality techniques such as bug count metrics. Based on our experience you should not rely completely on any single software system quality metric or technique. As the software development environment continues to mature, techniques like MAGIQ will become increasingly important components of your software engineering skill set.

Send your questions and comments to testrun@microsoft.com.

Dr. James McCaffrey works for Volt Information Sciences Inc., where he manages technical training for software engineers working at Microsoft. He has worked on several Microsoft products including Internet Explorer and MSN Search. James can be reached at jmccaffrey@volt.com or v-jammc@microsoft.com.

Nasa Koski is a Software Test Lead at Microsoft. Currently, she leads a team of system engineers at MSN who provide systems expertise and design in an extensive lab environment. Nasa can be reached at nasak@microsoft.com.


Cutting Edge: Simplify Task Progress with ASP.NET &quot;Atlas&quot;

Cutting Edge
Simplify Task Progress with ASP.NET "Atlas"
Dino Esposito

Code download available at: CuttingEdge2006_10.exe (555KB)
Contents
Remote Call Feedback in Atlas
An Updateable Atlas Panel
Adding a Progress Bar
Adding Triggers
Context-Sensitive Feedback
Summary

Last month I discussed progress bars in the context of ASP.NET Web applications and provided an implementation based on the built-in Script Callback API (see msdn.microsoft.com/msdnmag/issues/06/09/cuttingedge). Providing simple feedback to the user while a potentially lengthy task continues is not complicated. You just attach a bit of JavaScript code to the button that posts back and make it display a relatively static message, such as "Please, wait...". However, if the page is engaged in a traditional postback, you can't display much more than a static message. For example, if you add an animated image, the animation won't start because the browser freezes the current page and makes itself ready for the next response and the next page. For a better user experience, you can resort to out-of-band calls. The ASP.NET Script Callback API provides the ability to issue asynchronous calls without leaving the current page; however, the overall programming model is a bit incomplete for any complicated tasks.

In last month's column, we squeezed every little bit of functionality out of the Script Callback API and provided both static and context-sensitive feedback. In particular, providing context-sensitive feedback is challenging because it means showing the user messages that vary as the remote task makes progress. All calls operate in an out-of-band manner and report data back to the client where some JavaScript code updates the current page.

In summary, with the Script Callback API you can code virtually any functionality that revolves around the theme of monitoring ongoing tasks. The resulting code, though, is not easy to read and maintain for novice developers. What about ASP.NET "Atlas" then? Atlas is a framework that brings AJAX-style functionality to the ASP.NET 2.0 platform. Unlike other similar frameworks, Atlas is designed to be part of ASP.NET and therefore seamlessly integrates with the existing platform and application model.

Atlas comes with a rich suite of components, controls, and behaviors to make AJAX coding easy and, more importantly, mostly transparent to ASP.NET developers. On the client side, Atlas relies on a client UI framework and a script component model to design rich and interactive functionality quickly and effectively. On the server side, Atlas provides a new family of controls that make implementing AJAX functionality a breeze. For an overview of the whole set of Atlas features, take a look at the Matt Gibbs article in the July 2006 issue of MSDN®Magazine (see msdn.microsoft.com/msdnmag/issues/06/07/AtlasAtLast).

In this column, I'll rebuild using Atlas all the examples discussed in the last installment of Cutting Edge. You'll see how to implement static and context-sensitive progress bars in a significantly simpler and neater way.

Remote Call Feedback in Atlas

Sometimes the examples aimed at demonstrating a given feature or technology have to be simple and canonical; sometimes not. This is one of the cases in which a more realistic example helps to reveal the potential of the underlying framework and determine the best way of using its features. Let's start with a little data access layer built around some of the tables of the SQL Server™ Northwind database. As shown in Figure 1, I created a CoreDataService class with a few methods that are thin wrappers around queries. As you can see, most of the methods return an ADO.NET DataTable object.

Figure 1 Simplify Data Access Layer (Close Figure 1)
Public Class CoreDataService

    Public Function GetOrdersByEmployee(ByVal empID As Integer,
            ByVal year As Integer) As DataTable
        Dim adapter As New SqlDataAdapter( _
            CoreDataServiceCommands.Cmd_OrdersByEmployee, _
            CoreDataServiceCommands.ConnectionString)
        adapter.SelectCommand.Parameters.AddWithValue("@empID", empID)
        adapter.SelectCommand.Parameters.AddWithValue("@year", year)

        Dim table As New DataTable
        adapter.Fill(table)
        Return table
    End Function

    Public Function GetSalesByEmployee(ByVal empID As Integer, _
            ByVal year As Integer) As Decimal
        Dim adapter As New SqlDataAdapter( _
            CoreDataServiceCommands.Cmd_SalesByEmployee, _
            CoreDataServiceCommands.ConnectionString)
        adapter.SelectCommand.Parameters.AddWithValue("@empID", empID)
        adapter.SelectCommand.Parameters.AddWithValue("@year", year)

        Dim table As New DataTable
        adapter.Fill(table)
        Return DirectCast(table.Rows(0)(0), Decimal)
    End Function

    Public Function GetSalesEmployees(ByVal year As Integer) As DataTable
        ...
    End Function

    Public Function GetEmployees() As DataTable
        ...
    End Function

    Public Function GetAvailableYears() As DataTable
        ...
    End Function

End Class

To build a sample Atlas application, I used the Atlas Visual Studio® 2005 template installed with the latest community technology preview (CTP). Figure 2 shows the markup of the page, not including script. The page contains a couple of dropdown list controls, one for choosing a year and one for selecting the employee name. An HTML button completes the user interface. Note that the button control doesn't have to be a standard <asp:Button> tag because, in this case, a submit input field is all that's required. Clicking on a submit button is handled by the browser and resolved through a traditional postback; in order to use out-of-band calls, a client-side button is required along with a bit of JavaScript.

Figure 2 Markup for the Sample Page (Close Figure 2)
<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
   <title>Atlas:: Sync Execution</title>
</head>
<body>
   <form id="form1" runat="server">
      <atlas:ScriptManager ID="ScriptManager1" runat="server" />

      <div>
         <asp:ObjectDataSource ID="AvailableYearsSource" runat="server"
            TypeName="CuttingEdge.Samples.CoreDataService"
            SelectMethod="GetAvailableYears">
         </asp:ObjectDataSource>
         <asp:DropDownList ID="AvailableYears" runat="server"
            DataSourceID="AvailableYearsSource"
            DataValueField="Year">
         </asp:DropDownList>

         <asp:ObjectDataSource ID="EmployeesSource" runat="server"
            TypeName="CuttingEdge.Samples.CoreDataService"
            SelectMethod="GetEmployees">
         </asp:ObjectDataSource>
         <asp:DropDownList ID="EmployeeList" runat="server"
            DataSourceID="EmployeesSource"
            DataTextField="lastname"
            DataValueField="employeeid">
         </asp:DropDownList>

         <input id="Button1" type="button"
            value="Load" onclick="findData()"/>

         <hr />

         <div id="ProgressBar" style="display:none;">
            <img alt="" src="images/indicator.gif" />
            <span id="Msg">Please, wait ... </span>
         </div>
      </div>
   </form>
</body>
</html>

In Atlas, you have two ways to define remote methods callable via script: Web services and page methods. You can either call Web methods defined on an ASP.NET Web service or invoke ad hoc methods defined on the same page class. Clearly, page methods are a faster solution to write and form a simple solution whose scope never exceeds the boundaries of the page. Web services, on the other hand, promote a broader form of reusability and interoperability. Today Atlas supports ASP.NET Web services expressed as ASMX resources, and in the future it will support Windows® Communication Foundation services expressed as SVC resources and hosted by IIS. For simplicity, let's opt for a page method. Note, though, that the CoreDataService class can easily be encapsulated in a wrapper Web service. All you have to do is create a proxy class and decorate methods on that proxy with the WebMethod attribute.

A set of page methods allows you to publish a given page as a Web service in the context of the application. You define the page methods available for Atlas remote calls through a script tag:

<script type="text/VB" runat="server">
   <WebMethod()> _
   Public Function GetSalesByEmployee( _
           ByVal id As Integer, _
           ByVal year As Integer) As Decimal
       Return GetSalesByEmployeeInternal(id, year)
   End Function
</script>

The contents of the script tag is expressed in Visual Basic® or C#. You use the canonical WebMethod attribute to decorate methods and define the endpoints that an external caller will see. You can include any required code inline, although I personally prefer to reference the body of the method through an internal protected method in the codebehind class. To be able to use the WebMethod attribute, you need to import the right namespace in your ASPX page as well as any namespace required for return value and input parameters:

<%@ Import Namespace="System.Web.Services" %>

The GetSalesByEmployeeInternal method invokes the helper data service to get the total amount sold by an employee in a given year and return the value:

Function GetSalesByEmployeeInternal( _
        ByVal id As Integer, ByVal year As Integer) As Decimal
    Dim helper As New FakeDataService
    Return helper.GetSalesByEmployee(id, year)
End Function

Why is the method using the FakeDataService class instead of CoreDataService as in Figure 1? The FakeDataService mirrors the CoreDataService class and adds a little delay to test the behavior of the progress bar more comfortably and reliably: (Obviously you wouldn't want to explicitly add such a delay in your own production applications.)

Public Function GetSalesByEmployee( _
        ByVal empID As Integer, ByVal year As Integer) As Decimal
    Thread.Sleep(5000)
    Return coreDataService.GetSalesByEmployee(empID, year)
End Function

How can you start the remote call and incorporate results in the page? A bit of JavaScript code is still required to trigger the call and process the results. Figure 3 lists the required code as it appears in the sample ASPX page.

Figure 3 Triggering a Page Method and Processing Results (Close Figure 3)
<script language="javascript" type="text/javascript">
    function findData()
    {
        var list1 = document.getElementById("AvailableYears")
        var year = list1.options[list1.selectedIndex].value;

        var list2 = document.getElementById("EmployeeList")
        var id = list2.options[list2.selectedIndex].value;

        // Turn on progress
        var progress = document.getElementById("ProgressBar");
        progress.style.display = "";

        // Invoke the remote method
        PageMethods.GetSalesByEmployee(id, year, onSearchComplete);
    }

    function onSearchComplete(results)
    {
       // Turn off progress
       var progress = document.getElementById("ProgressBar");
       progress.style.display = "none";

       // Update the UI
       alert(results);
    }
</script>

The findData method is bound to the click event of the input button. It collects the selected values in the dropdown lists and turns on the user interface for the progress bar. The script ends up executing the following code:

PageMethods.GetSalesByEmployee(id, year, onSearchComplete);

The PageMethods object is the JavaScript proxy created by the Atlas infrastructure based on the contents of the text/VB script block. The object lists as many methods as there are Web methods declared on the page. The signature of each method matches the signature of the corresponding Web method plus a callback parameter used to update the user interface. The onSearchComplete object is a JavaScript function that turns off the progress bar and displays the received data.

The overall Atlas code is not as twisted as it would be using ASP.NET Script Callback, but it certainly requires JavaScript skills. In the progress bar block you can put any text and animation; it will remain in place during the asynchronous call. Figure 4 shows the sample page in action.

Figure 4 Invoking a WebMethod Using Atlas
Figure 4 Invoking a WebMethod Using Atlas(Click the image for a smaller view)
Figure 4 Invoking a WebMethod Using Atlas
Figure 4 Invoking a WebMethod Using Atlas (Click the image for a larger view)

The Web page method executed in the figure returns a scalar value, the total number of orders processed by a given employee. If the Web page method returns a data table, a made-to-measure binding mechanism is required. Thanks to the Atlas serialization engine, an ADO.NET DataTable object is correctly serialized to a JavaScript object. The point is that mapping this object to a table-like HTML structure requires a good deal of script code that can be hard to read and maintain.

Atlas provides advanced and automatic capabilities for client data binding. This feature set comes at the cost of learning a new programming model and refreshing or improving your JavaScript skills. Thankfully, that's not the only possibility. In Atlas, you also find a server-based API that is equally powerful, propounds a familiar programming model and, more importantly, provides native support for progress bars. The server-side API is centered around a new control named UpdatePanel.

An Updateable Atlas Panel

The UpdatePanel server control extends the ASP.NET Panel control with automatic out-of-band capabilities. Atlas sees the contents of an UpdatePanel control as a sort of sub-form that is subject to postbacks and partial rendering. The postback occurs via script and in asynchronous fashion. Only the input fields and other hidden fields in the panel are sent and received. Also, only the markup generated by the controls in the panel is refreshed on the client at the end of the post. There are various ways to trigger the partial page postback. Let's examine the simplest scenario first.

Figure 5 shows an Atlas page using the UpdatePanel control. A similar Atlas page has two ways to post back: regular full postbacks as in classic ASP.NET and partial postbacks using the panel. When events occur that trigger the postback on the panel, all controls contained in the ContentTemplate tag of the UpdatePanel control post their current content back to the server, using an out-of-band call. The response of the call is the new server-generated markup to replace in the client page using DHTML. The great news is that you don't have to learn about the script necessary to accomplish the task, or care about the HTML replacement. The UpdatePanel control will emit all required script code.

Figure 5 Atlas Page Using the UpdatePanel Control (Close Figure 5)
<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title>Atlas:: Sync Execution (using UpdatePanel)</title>
</head>
<body>
    <form id="form1" runat="server">
        <atlas:ScriptManager ID="ScriptManager1" runat="server"
            EnablePartialRendering="True" />

        <div>
            <asp:ObjectDataSource ID="AvailableYearsSource" runat="server"
                TypeName="CuttingEdge.Samples.CoreDataService"
                SelectMethod="GetAvailableYears">
            </asp:ObjectDataSource>
            <asp:DropDownList ID="AvailableYears" runat="server"
                DataSourceID="AvailableYearsSource"
                DataValueField="Year">
            </asp:DropDownList>

            <asp:ObjectDataSource ID="EmployeesSource" runat="server"
                TypeName="CuttingEdge.Samples.CoreDataService"
                SelectMethod="GetEmployees">
            </asp:ObjectDataSource>
            <asp:DropDownList ID="EmployeeList" runat="server"
                DataSourceID="EmployeesSource"
                DataTextField="lastname"
                DataValueField="employeeid">
            </asp:DropDownList>

            <hr />

            <atlas:UpdatePanel runat="server" ID="UpdatePanel1">
                <ContentTemplate>
                    <asp:Button ID="Button1" runat="server" Text="Load"
                        OnClick="Button1_Click" />
                    <asp:GridView ID="GridView1" runat="server"
                        DataSourceID="DataServiceSource"
                        AllowPaging="True">
                    </asp:GridView>
                    <small><asp:Label runat="server"
                        ID="TimeServed" /></small>
                </ContentTemplate>
            </atlas:UpdatePanel>

            <asp:ObjectDataSource ID="DataServiceSource" runat="server"
                TypeName="CuttingEdge.Samples.FakeDataService"
                SelectMethod="GetOrdersByEmployee">
                <SelectParameters>
                    <asp:ControlParameter Name="year"
                        ControlID="AvailableYears"
                        PropertyName="SelectedValue" />
                    <asp:ControlParameter Name="empid"
                        ControlID="EmployeeList"
                        PropertyName="SelectedValue" />
                </SelectParameters>
            </asp:ObjectDataSource>

            <atlas:UpdateProgress id="progress1" runat="server">
                <ProgressTemplate>
                    <div>
                        <img alt="" src="images/indicator.gif" />
                        <span id="Msg">Please, wait ... </span>
                    </div>
                </ProgressTemplate>
            </atlas:UpdateProgress>

        </div>
    </form>
</body>
</html>

For the UpdatePanel control to work as expected, you need to set the EnablePartialRendering property on the ScriptManager control, as shown here:

<atlas:ScriptManager ID="ScriptManager1" runat="server"
    EnablePartialRendering="True" />

Any Atlas page is characterized by exactly one instance of the ScriptManager server control. Defined in the Microsoft.Web.UI namespace, ScriptManager is the brains behind most of the Atlas server infrastructure and also orchestrates a good number of client functionalities. For example, the ScriptManager control is responsible for emitting a bunch of script to reference Web services and to extend server controls. Likewise, the ScriptManager control is charged with the task of partially updating client pages through the UpdatePanel control. If you leave the EnablePartialRendering property set to its default value of false, the UpdatePanel control will work exactly like a classic Panel control.

If the content template of an UpdatePanel control contains any button controls, the postback always occurs when the user clicks any of these buttons. However, a variety of other events can trigger the refresh of an Atlas panel. I'll return to this point in a moment. For now, let's focus on the page illustrated in Figure 5, where the UpdatePanel control contains a grid, a label, and a submit button, as shown in the following:

<atlas:UpdatePanel runat="server" ID="UpdatePanel1">
    <ContentTemplate>
        <asp:Button ID="Button1" runat="server" Text="Load"
            OnClick="Button1_Click" />
        <asp:GridView ID="GridView1" runat="server"
            DataSourceID="DataServiceSource"
            AllowPaging="True" />
        <small><asp:Label runat="server" ID="TimeServed" /></small>
    </ContentTemplate>
</atlas:UpdatePanel>

Destined to contain all the orders placed by a given employee in a given year, the grid is bound to an ObjectDataSource control that, in turn, retrieves data through the FakeDataService class:

<asp:ObjectDataSource ID="DataServiceSource" runat="server"
    TypeName="CuttingEdge.Samples.FakeDataService"
    SelectMethod="GetOrdersByEmployee">
    <SelectParameters>
        <asp:ControlParameter Name="year"
            ControlID="AvailableYears" PropertyName="SelectedValue" />
        <asp:ControlParameter Name="empid"
            ControlID="EmployeeList" PropertyName="SelectedValue" />
    </SelectParameters>
</asp:ObjectDataSource>

Whenever a postback is triggered from within the updateable panel, any server-side code associated with the originating control is executed. As mentioned, though, the postback is developed using an Atlas out-of-band call regardless of the characteristics of the posting button. When partial rendering is enabled, the ScriptManager control modifies the markup of the page on the fly and adds a submit script code to all the potential triggers of the panel refresh. Potential triggers include all button controls in the panel plus all controls registered as triggers. (More on this in a moment.)

Furthermore, the ScriptManager control hooks up to the default ASP.NET rendering process and captures the modified markup for the controls in the panel. The markup is then sent to the client as the response for the out-of-band call.

When the user clicks the submit button as in Figure 5, the partial postback occurs and the Button1_Click handler in the codebehind is executed:

Sub Button1_Click(ByVal sender As Object, ByVal e As EventArgs)
    TimeServed.Text = "Served at " + DateTime.Now.ToString("hh:mm:ss")
End Sub

Bound to an object data source, the grid is automatically refreshed on postback in a codeless manner. If you opt for a manual form of binding, then you're responsible for binding the grid to its data on postback.

If the grid supports paging or sorting, each click to page or sort will refresh the panel as well. Any click generated outside the panel is treated as usual by ASP.NET and originates a full page refresh. As mentioned, an exception to this rule is represented by all controls and events registered as triggers of the panel updates. Before I get to this aspect of the Atlas programming model, let's take a quick look at the built-in support for progress bars.

Adding a Progress Bar

An UpdatePanel control can be associated with an Atlas progress bar control. To be precise, the progress bar control is not a gauge component but rather a user-defined panel that the ScriptManager control shows as the panel refresh begins and hides immediately after its completion. The Atlas progress bar control is named UpdateProgress:

<atlas:UpdateProgress id="progress1" runat="server">
    <ProgressTemplate>
        <div>
            <img alt="" src="images/indicator.gif" />
            <span id="Msg">Please, wait ... </span>
        </div>
    </ProgressTemplate>
</atlas:UpdateProgress>

As you can see, the UpdateProgress control renders a chunk of ASP.NET markup through its ProgressTemplate property. You can insert any text, marquee, and animated images in the template and it will stay up until the panel refresh is complete. For this to happen, no script or managed code is required; all you have to do is drop an UpdateProgress control onto an Atlas Web Form. Figure 6 shows an example.

Figure 6 An Update Progress Control
Figure 6 An Update Progress Control

When you look at this example, you might wonder: would it be possible to personalize the wait message to include some of the parameters being used in the query? For example, would it be possible to show the message "Please wait while data for employee Fuller is retrieved"? Any server event on the progress control is fired when it's too late to update the text on the client. The only option is adding some JavaScript code on the client that runs before the partial postback is started. Note, though, that you can't capture the client Click event on a submit button; the Submit client event is already hooked by the ScriptManager control. One thing you can try is handling the client Change event on input controls. For example, try adding the following attribute to the EmployeeList dropdown list of Figure 6:

onchange="UpdateMsg(this.options[this.selectedIndex].text);"

The UpdateMsg function is a Javascript function you write like this:

<script type="text/javascript">
    function UpdateMsg(empName)
    {
        var msg = document.getElementById("Msg");
        msg.innerHTML = "Please, wait while data for " + empName +
                        " is retrieved ...";
    }
</script>

You use the page object model to modify the text of the Msg span tag as the selection in the dropdown list changes. Whenever an input field that affects the text you want to display changes, you programmatically update the progress panel using JavaScript. The result is shown in Figure 7.

Figure 7 Custom Message in the Progress Control
Figure 7 Custom Message in the Progress Control(Click the image for a smaller view)
Figure 7 Custom Message in the Progress Control
Figure 7 Custom Message in the Progress Control (Click the image for a larger view)

If you only want to programmatically set some of the attributes of the child controls of UpdateProgress, you define an OnLoad handler for the UpdateProgress control and retrieve the controls in the template using the FindControl method:

Sub progress1_OnLoad(ByVal sender As Object, ByVal e As EventArgs) _
        Handles progress1.Load
    DirectCast(progress1.FindControl("Msg"), Label).Text = "..."
End Sub

Once again, note that this code will run on the server when it's too late to update the text of the progress monitor on the client.

If you look back at the code in Figure 5, you won't find any element that links together the UpdatePanel control and the UpdateProgress control. The idea is that one UpdateProgress control fits all UpdatePanel controls in the page. You can have many refresh panels in the Web Form and just one update progress control; the Atlas script manager guarantees that each refresh panel will be served by its own unique update progress. What if you have multiple progress controls? All of them will be displayed at the same time, regardless of the panel being updated.

Adding Triggers

Programming Atlas with the UpdatePanel control doesn't push either a new programming model or a new API. The update panel mechanism, though, is more sophisticated than it shows here. For example, you can add triggers to a panel:

<atlas:UpdatePanel runat="server" ID="Panel1">
    <ContentTemplate>
        ...
    </ContentTemplate>
    <Triggers>
        <atlas:ControlEventTrigger ControlID="Btn" EventName="Click" />
    </Triggers>
</atlas:UpdatePanel>

In the Triggers block of an UpdatePanel control you can include instances of ControlEventTrigger. With event triggers, when the specified event on the specified control occurs, the script manager refreshes the panel.

By default, all update panels in the form refresh when any of triggers in the page act, regardless of which panel control actually defined the trigger. To change this behavior and make each update panel refresh only when one of its own triggers acts, you change the Mode attribute on the UpdatePanel control to Conditional:

<atlas:UpdatePanel ID="Panel1" Mode="Conditional" runat="server" ... />

A very interesting Atlas control that can be used with triggers and progress bars is the timer control:

<atlas:TimerControl ID="timer1" runat="server" Interval="5000"
    OnTick="timer1_Tick" />

The preceding timer control will fire a Tick event every five seconds. By binding a panel to the Tick event of a timer, you can refresh an updateable panel periodically:

<atlas:UpdatePanel runat="server" id="Panel1">
    <ContentTemplate>
       ...
    </ContentTemplate>
    <Triggers>
        <atlas:ControlEventTrigger ControlId="timer1"
            EventName="Tick" />
    </Triggers>
</atlas:UpdatePanel>

Periodic updates are the key to providing context-sensitive feedback to users during lengthy tasks.

Context-Sensitive Feedback

Context-sensitive user feedback indicates any status information that is displayed to the user while waiting for a response from the server. Unlike the information displayed through the Atlas UpdateProgress control, an infrastructure designed for context-sensitive feedback will provide up-to-date information that refreshes as the task progresses and as the response is generated. To get the idea, think of a classic progress meter that moves ahead as the monitored task advances. Last month I discussed a pure ASP.NET 2.0 solution based on the following steps: start a task, have the task update a server-side cache with status information, have some UI periodically and asynchronously refreshed with the currently available status information.

The first approach that springs to mind when you port this model to Atlas is using a first UpdatePanel for the task and a second UpdatePanel—optionally embedded in an UpdateProgress control—to implement the progress meter. The progress UpdatePanel will be bound to a timer and updated at the proper rate.

This logical model clashes with the internal implementation of the UpdatePanel and ScriptManager controls. Atlas uses a client-side object to place any out-of-band request. This object is part of the Atlas client library and is named PageRequestManager. The point is that there will be just one of these objects to handle all UpdatePanel controls in the page. As a result, requests are necessarily serialized and the net effect is that while the main task proceeds, no concurrent request is sent to read the status. (However, Atlas can automatically batch requests and can send them to the server as a group.)

To implement context-sensitive feedback in Atlas, you build a manual mechanism and take full control over the calls made to the server. The page will contain a lot of script code to set up a JavaScript timer, start the remote task, and update the page with status information. The monitored task is started through a page method. A page method is also used to read status information:

<script type="text/VB" runat="server">
    <WebMethod()> _
    Public Function GetSalesEmployees(ByVal taskID As String, _
            ByVal year As Integer) As String
        Return GetSalesEmployeesInternal(taskID, year)
    End Function

    <WebMethod()> _
    Public Function UpdateStatus(ByVal taskID As String) As String
        Return UpdateStatusInternal(taskID)
    End Function
</script>

A button click will call into the GetSalesEmployees method which in turn works with the FakeDataService to run the query and get data. The button click looks like this:

function StartTask()
{
    var years = document.getElementById("AvailableYears");
    var year = years.options[years.selectedIndex].value;

    EnableUI(false);

    globalTaskID = GetGuid();
    ShowProgress();
    PageMethods.GetSalesEmployees(globalTaskID, year, onTaskCompleted);
}

It first retrieves the selected year from the dropdown, generates a GUID to uniquely identify the task, activates the timer to read status information, and finally starts the potentially lengthy task.

The code that implements the task and the code that reads status must agree on an API and a storage medium to read and write status information. In this example, status information is saved to the ASP.NET cache in a slot named after the task ID. The task ID must be known on the client and is generated using a bit of JavaScript. A unique task ID is necessary to allow for multiple sessions to use the same page without conflicts. Figure 8 lists the code for a data service method that updates the status as it makes progress.

Figure 8 Task That Updates the Status as it Proceeds (Close Figure 8)
Public Function GetSalesEmployees(ByVal year As Integer) As _
        SalesDataCollection
    ' Clear the status (_taskID is a class member set through the ctor)
    TaskHelpers.ClearStatus(_taskID)

    ' Get all employees
    Dim employees As DataTable = GetEmployees()
    Dim sales As New SalesDataCollection

    ' Start processing each employee and calculate sales
    For Each row As DataRow In employees.Rows
        Dim id As Integer = DirectCast(row("employeeid"), Integer)
        Dim empName As String = DirectCast(row("lastname"), String)

        ' Update progress
        TaskHelpers.UpdateStatus(_taskID, _
            String.Format("Processing employee: <b>{0}</b>", empName))

        ' Query for the amount sold by a particular employee
        Dim amount As Decimal = GetSalesByEmployee(id, year)

        ' Save data to return
        Dim salesRow As New SalesData
        salesRow.Employee = empName
        salesRow.Amount = amount
        sales.Add(salesRow)
    Next

    ' Clear the status
    TaskHelpers.ClearStatus(_taskID)
    Return sales
End Function

Status information is read through a page method call run periodically and controlled by a JavaScript timer:

function ShowProgress()
{
    PageMethods.UpdateStatus(globalTaskID, onUpdateProgress);
    EnableStatusBar(true);
    timerID = window.setTimeout("ShowProgress()", 1000);
}
function StopProgress()
{
    window.clearTimeout(timerID);
    EnableStatusBar(false);
}

In which way is the previous code different from using the following Atlas timer?

<atlas:TimerControl runat="server" ID="Timer1" Interval="1000"
    OnTick="..." />

The effect is exactly the same: a piece of server code is run on the server every second. However, the Atlas timer won't let you specify additional script code to run around the out-of-band call, such as the script to enable or disable the client user interface.

Summary

ASP.NET Atlas is a framework designed to enable developers to create rich, interactive, cross-browser Web user interfaces with limited effort and a relatively short learning curve. There are quite a few scenarios where the Atlas programming model is a mere extension to the classic ASP.NET model, making the upgrade a breeze. Atlas provides great ready-made tools for a good number of common situations. Where your needs match the Atlas toolset, you can create sophisticated applications quickly and efficiently and, more importantly, code that is easy to read and maintain.

Atlas does a great job in keeping close to standard ASP.NET. However, the more flexibility you want, the more you have to write JavaScript code and integrate with server code. This may not be easy to do in all cases. That's where Atlas-based frameworks fit in. Atlas is designed to be a low-level framework with some excellent native tools. Other commercial frameworks may provide you with a more vertical set of tools where built-in components save you from much of the advanced client and server coding that is required if you need to go beyond the basic Atlas equipment.

Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is a mentor at Solid Quality Learning and the author of Programming Microsoft ASP.NET 2.0 (Microsoft Press, 2005). Based in Italy, Dino is a frequent speaker at industry events worldwide. Get in touch with Dino atcutting@microsoft.com or join the blog at weblogs.asp.net/despos.


Extreme ASP.NET: Control Adapters

Extreme ASP.NET
Control Adapters
Fritz Onion

Code download available at: ExtremeASPNET2006_10.exe (187KB)
Contents
Building a Control Adapter
Browser Recognition
CSS-Friendly Adapters
Using the Control Adapter in Your App
Wrap Up

The release of ASP.NET 2.0 introduces an alternate way to specify renderings of controls through a technology called control adapters. If you have heard of control adapters at all, it is probably in the context of providing alternate renderings of controls for mobile devices. They actually serve a more general purpose of providing a way of completely changing the rendering of a control based on the browser type of the client, which turns out to be useful in a number of situations.

Mobile device rendering is still the most obvious application of control adapters. For example, for a mobile phone, the control contents should render as WML instead of HTML. Ideally, developers would still use familiar ASP.NET controls, like Calendar and GridView, but when they are accessed by a WML device, a control adapter would kick in and provide an alternate rendering that would work within the constraints of the client device. With an updated collection of control adapters released as new devices come onto the market, developers would only have to update their list of control adapters, and suddenly their Web content would be available to these new devices without changing any of their actual code or page content.

But, as mobile devices continue to proliferate at what feels like an exponential rate—each with different form factors and rendering capabilities—trying to create mappings from standard ASP.NET controls to all devices becomes overwhelming. Even more problematic is that developers building sites that target screens with 1024×768 resolution or higher make assumptions that just can't be adapted to mobile devices without significant intervention by the developer. If you want a site that works well with a number of different devices today, you need to build a separate UI for each distinct device to make users happy. The Mobile Control Toolkit introduced as an add-on to ASP.NET 1.x and carried forward to 2.0 can help, but the goal of building sites that render successfully to a multitude of devices using control adapters has not come to fruition.

The good news is that the ASP.NET team found another compelling use for this technology in the form of the CSS Control Adapter Toolkit, which is available for download at www.asp.net/cssadapters. Instead of creating alternate renderings of controls for specific devices, this toolkit defines a number of control adapters that change the default table-based renderings of several controls (like Menu, TreeView, and FormView) to eschew tables completely and render using <ul>, <div>, and <span> elements with associated styles defined in CSS stylesheets (a common requirement for site designs today). In this column I'll introduce the control adapter architecture and explain how it can be used to completely change control renderings based on the User Agent string. I'll then discuss the CSS Control Adapter Toolkit and how you can integrate its features into your sites today.

Building a Control Adapter

At their core, control adapters are simply a way of providing different renderings for controls without actually modifying the controls themselves. Because control adapters are designed to provide alternate renderings for different clients, you specify control adapter mappings in a .browser file, which is where associations between User Agent strings and browser capabilities are defined. The control adapter class itself must inherit from the System.Web.UI.Adapters.ControlAdapter, which is an abstract base class that looks much like the Control base class, with events for Init, Load, PreRender, and Unload, as well as a virtual Render method.

Internally, when a control is rendering, the Control base class will first check to see if there is a control adapter currently associated with the control. If there is, it will invoke the Render method of the adapter; if not, it calls the standard Render method of the control. Figure 1 shows the control adapter architecture.

Figure 1 Control Adapter Architecture
Figure 1 Control Adapter Architecture

To build your own control adapter, you start by creating a new class that inherits from the ControlAdapter base class, or more commonly from one of its derivatives—WebControlAdapter. However, the control adapter class you derive from depends on the control you're building the adapter for. If it is one of the built-in WebControls, you should select the WebControlAdapter as it adds virtual RenderBeginTag, RenderEndTag, and RenderContents methods to more closely mirror how WebControls render. There are also more specific control adapters defined for data-bound controls and for the Menu control, each of which provides more virtual methods to access and modify features of the controls with which it is associated.

To show you how to build a control adapter to change the rendering of a particular control, I'll create a control adapter for the BulletedList control named BulletedListControlAdapter. This simple control renders a collection of items as an ordered or unordered list (<ol> or <ul>). To illustrate how flexible the adapter model is, I'll build the control adapter to completely change the rendering to be table-based instead. Since the BulletedList is derived from WebControl, I'll start by creating the BulletedListControlAdapter class as a derivative of WebControlAdapter. Next, I'll override the RenderBeginTag and RenderEndTag methods to render the opening and closing tags of the table. Finally, I'll override RenderContents to iterate across the items of the BulletedList control and render each item as a single-cell row in the table prefixed with an asterisk. The full control adapter implementation is shown in Figure 2.

Figure 2 The Control Adapter (Close Figure 2)
namespace MsdnMagazine
{
  public class BulletedListControlAdapter : WebControlAdapter
  {
    protected override void RenderBeginTag(HtmlTextWriter writer)
    {
      writer.WriteLine();
      writer.WriteBeginTag("table");
      writer.Write(HtmlTextWriter.TagRightChar);
      writer.Indent++;
    }

    protected override void RenderEndTag(HtmlTextWriter writer)
    {
      writer.WriteEndTag("table");
      writer.Indent--;
      writer.WriteLine();
    }

    protected override void RenderContents(HtmlTextWriter writer)
    {
      writer.Indent++;

      BulletedList bl = Control as BulletedList;
      if (bl != null)
      {
        foreach (ListItem i in bl.Items)
        {
          writer.WriteLine();
          writer.WriteBeginTag("tr");
          writer.Write(HtmlTextWriter.TagRightChar);
          writer.WriteLine();
          writer.Indent++;
          writer.WriteBeginTag("td");
          writer.Write(HtmlTextWriter.TagRightChar);
          writer.WriteLine();
          writer.Indent++;
          writer.Write("*");
          writer.Write(HtmlTextWriter.SpaceChar);
          writer.Write(i.Text);
          writer.WriteLine();
          writer.Indent--;
          writer.WriteEndTag("td");
          writer.WriteLine();
          writer.Indent--;
          writer.WriteEndTag("tr");
          writer.WriteLine();
        }
      }

      writer.Indent--;
    }
  }
}

The next step is to associate the BulletedListControlAdapter class with the BulletedList control. As I mentioned earlier, this is done by specifying which subset of browsers you want to target. For now, I'll simply associate all browsers with this adapter by specifying a refID attribute value of Default in the browser element of my .browser file. (In the next section I'll discuss how you can create more granular associations.) In this sample, I created a new file named MyAdapters.browser and placed it in the App_Browsers directory of my application:

<!-- File: MyAdapters.browser -->
<browsers>
  <browser refID="Default">
    <controlAdapters>
      <adapter
        controlType="System.Web.UI.WebControls.BulletedList"
        adapterType="MsdnMagazine.BulletedListControlAdapter"
      />
    </controlAdapters>
  </browser>
</browsers>

Any additional browser definitions or control adapters defined in this local file will be added to the collection of browsers and adapters already defined in the system browser configuration files.

The last step is to actually place an instance of the BulletedList control with some items on a page, as shown in Figure 3.

Figure 3 BulletedList Control a Page (Close Figure 3)
<%-- File: Default.aspx --%>
<%@ Page Language="C#" %>

<html xmlns="http://www.w3.org/1999/xhtml" >
<body>
    <form id="form1" runat="server">
    <div>
        <asp:BulletedList ID="_bulletedList" runat="server">
          <asp:ListItem Value="1">Item 1</asp:ListItem>
          <asp:ListItem Value="2">Item 2</asp:ListItem>
          <asp:ListItem Value="3">Item 3</asp:ListItem>
          <asp:ListItem Value="4">Item 4</asp:ListItem>
        </asp:BulletedList>
    </div>
    </form>
</body>
</html>

If you first run the page without the control adapter in place (by removing the .browser file or commenting out the control adapter mapping in the .browser file using standard XML comment syntax), the BulletedList will render as expected, using a <ul> element with <li> subelements:

<ul id="_bulletedList">
  <li>Item 1</li>
  <li>Item 2</li>
  <li>Item 3</li>
  <li>Item 4</li>
</ul>

If you then enable the control adapter, the rendering process changes to the new table-based rendering defined in the custom adapter, with no changes at all to the actual .aspx page (see Figure 4).

Figure 4 Table-Based (Close Figure 4)
<table>
  <tr>
    <td>
      * Item 1
    </td>
  </tr>
  <tr>
    <td>
      * Item 2
    </td>
  </tr>
  <tr>
    <td>
      * Item 3
    </td>
  </tr>
  <tr>
    <td>
      * Item 4
    </td>
  </tr>
</table>

As you can see, it is reasonably simple to create an alternate rendering for a control using the control adapter architecture. There are, however, some issues that should be raised at this point. First, note that the simplified rendering of the BulletedList did not look at any of the attributes of the BulletedList control except its items collection. This includes the rather important BulletStyle attribute that determines what type of list to render (unordered, numeric, alphabetic, and so on). It is generally important to honor all of the behavioral attributes of a control for which you're writing a control adapter. On the other hand, it may or may not make sense to incorporate style attributes into your rendering, especially if the target device for which the control adapter is being built does not have any means of displaying a particular style (for example, BackColor on a black and white device). As you will see later, the CSS Control Adapters elect to ignore most style attributes of controls they render, and instead provide a set of CSS classes that can be modified to dictate appearance.

Second, the simplified example of the BulletedListControlAdapter is not very realistic because I didn't actually take advantage of the browser mapping to associate specific user agents with this adapter. Instead I mapped all browser types (by specifying Default in the refID attribute) to use this control adapter. This means that a site with this control adapter installed would never call the standard rendering of the BulletedListControl but would always display the rendering supplied by the control adapter. If the only purpose is to create a control that has a different rendering from an existing control, another approach would be to derive a new control class from the existing control and override the necessary methods to change the rendering. The potential advantage control adapters have over custom control derivatives is that they can be applied to a Web application with no modifications to any of the pages (or even the web.config file). Thus it may be compelling to use them even without browser-contingent rendering in situations where an application is already written and deployed, but a rendering change is needed for a particular control across the entire site.

Browser Recognition

As you've seen, the control adapters are mapped onto controls in a particular site by specifying a configuration element in a .browser file. In ASP.NET 2.0, the browser recognition capability was moved from the <browserCaps> element of machine.config and web.config to a collection of .browser files. The <browserCaps> element is still supported, but the preferred way to specify browser capabilities is in a .browser file located either in the local App_Browsers directory of your application or in the machine-wide %SYSTEM%\Microsoft.NET\Framework\v2.0.50727\Config\Browsers directory. The purpose of browser recognition files, in addition to associating control adapters, is to populate the HttpBrowserCapabilities class that a request can access through the Request.Browser property. Through this class you can find out the name of the requesting browser and whether it supports JavaScript, cookies, and other features. To populate this class, each .browser file contains one or more <browser> elements, each typically with a subelement called identification that contains a regular expression that is applied to the User Agent string or to a specific capability from a parent browser class for each request (with an option to specify an exclusion expression as well). If it matches, the capabilities listed for that browser are used to populate the HttpBrowserCapabilities class. For example, the browser definition for Internet Explorer (found in the machine-wide Browsers directory in the file ie.browser) looks like Figure 5.

Figure 5 Internet Explorer Browser Definition (Close Figure 5)
<browsers>
  <browser id="IE" parentID="Mozilla">
    <identification>
      <userAgent
        match="^Mozilla[^(]*\([C|c]ompatible;\s*MSIE
               (?’version’(?’major’\d+)(?’minor’\.\d+)(?’letters’\w*))
               (?’extra’[^)]*)" />
      <userAgent nonMatch="Opera|Go\.Web|Windows CE|EudoraWeb" />
    </identification>

    <capabilities>
      <capability name="browser" value="IE" />
      <capability name="extra"   value="${extra}" />
      <capability name="isColor" value="true" />
      <capability name="letters" value="${letters}" />
      ...
Figure 5 Browse Type Hierarchy
Figure 5 Browse Type Hierarchy

Browsers are also grouped hierarchically using the parentID attribute on the <browser> element. In order for a <browser> element that has a parentID referencing another <browser> element to be applied, the regular expression conditions for the parent element must have succeeded. The root of the hierarchy is "Default", which matches all user agent strings and assumes that the browser has minimal capabilities. All other browser types have either Default or one of its children as a parent. The predefined set of browser definitions located in the machine-wide Browsers directory contains a useful categorization of browser types, with associated names that can be used to identify subsets of browsers. For example, if you wanted to apply a control adapter to all requests made from Internet Explorer versions 6.0 and higher, you could use the "IE6to9" browser name in the refId attribute of your local browser element. Figure 6 shows the hierarchy of browser types defined in the system .browser files (excluding mobile devices, which constitute another hierarchy at least twice this size).

You can use any of these browser names in the refID attribute when assigning control adapters. As an example, the CSS Control Adapter Toolkit enables adapters for IE6to9, MozillaFirefox, Opera8to9, and Safari, which covers the subset of browsers that support the CSS features needed by the adapters defined in the toolkit. Other browsers will revert to the standard control renderings.

CSS-Friendly Adapters

In April of 2006, Microsoft released a set of samples entitled CSS-Friendly ASP.NET 2.0 Control Adapters Beta 1.1, a suite of control adapters that change the rendering of several common ASP.NET controls to take better advantage of CSS. The set was designed primarily because developers using ASP.NET needed to specify all of the style attributes, including layout, for their controls using CSS stylesheets instead of server-side attributes. The initial release includes adapters for the Menu, TreeView, DetailsView, FormView, and DataList controls.

Each control adapter included in this collection ships with two accompanying stylesheets, one with a core set of styles defined for the adapter, which should rarely need modification on your part, and a second containing a set of styles you'll almost certainly want to modify to manage the appearance of the control in your application. For example, the MenuAdapter sample comes with a Menu.css file and a MenuExample.css file; the first file contains core styles you would rarely change and the second contains style attributes like background colors, fonts, and offsets that change the appearance of menus rendered with the control adapter. All of the commonly changed style attributes are contained within a top-level CSS class so you can create multiple sets of styles, each under a different class. You specify which set of styles you want a particular control to use with the CssSelectorClass attribute, which is looked at by the control adapter and used to map CSS classes onto the rendered HTML elements for that control. For example, the MenuAdapter defines a collection of styles under the top-level PrettyMenu CSS class in MenuExample.css. To apply this set of styles to a menu when using the MenuAdapter, you would specify PrettyMenu as the CssSelectorClass, as shown here:

<asp:Menu ID="_mainMenu" runat="server"
          DataSourceID="_siteMapDataSource"
          CssSelectorClass="PrettyMenu" SkinID="MainNav"
          Orientation="Horizontal" />

Once the adapter for one of these controls is installed, all instances of the control whose rendering is modified by that adapter will render using the style attributes defined in your .css file and will ignore any style attributes defined directly for the server-side control in its markup. The Visual Studio® designer will not reflect the control adapter rendering, so once you install an adapter, you can no longer set style attributes locally on controls being rendered as adapters and expect them to be honored.

Using the Control Adapter in Your App

To give you an idea of how you would incorporate these control adapters into your ASP.NET application, the following steps describe how you could add the MenuAdapter to an existing Web application. Note that the details of some of these steps may change slightly with future releases of the toolkit samples, but the core elements will undoubtedly remain the same.

The current toolkit ships as a Visual Studio Web project template, so the first step is to create a new Web site using the CSS Friendly ASP.NET Control Adapters template. The resulting project will contain a working project with examples of each control adapter.

Next, copy the MenuAdapter.cs and Utility.cs source code files (or .vb files, if you selected a Visual Basic® project) from the generated project's App_Code directory to your own Web application's App_Code directory. Note that you could also compile these files into an assembly and deploy them globally in the Global Assembly Cache (GAC ) if you want to use them across multiple projects.

Now copy the CSSFriendlyAdapters.browser file from the App_Browsers directory of the generated project into your own Web application's App_Browsers directory. Modify the CSSFriendlyAdapters.browser file to remove references to adapters you aren't using (TreeView, DetailsView, FormView, and DataList in this case), then copy the Basic and Enhanced directories from the generated project's App_Themes directory to your Web application's App_Themes directory. These will contain the .css files needed by the MenuAdapter, as well as supporting graphics files. If you are already using themes in your project, you can also just copy the contents of one of these directories (choose which appearance you prefer) to an existing theme directory that will be applied to all pages containing a Menu control.

If you want, you can now remove .css files from the theme directories you will not be using (for example FormView.css and FormViewExample.css in this case).

Next you'll need to copy the JavaScript directory from the generated project to your Web applications root. The MenuAdapter class relies on the AdapterUtils.js and MenuAdapter.js files being in this directory. Then subscribe all pages with a Menu control to the Basic or Enhanced theme (you may want to do this globally in web.config, especially if you are using Master Pages). This is necessary so that each page with a Menu control has a link to the needed .css files in the theme directory, which themes take care of automatically.

Finally, annotate each of your Menu controls with a CssSelectorClass attribute referencing the top-level CSS class you'd like each control to use for its appearance and layout. As noted, the one that is provided as a sample is called PrettyMenu.

All of your Menu controls should now render using <ul> elements instead of tables, and their appearance will be defined by the CSS styles contained in MenuExample.css. At this point you may want to create a new stylesheet class modeled after MenuExample.css with your own style definitions and then modify the CssSelectorClass to point to the new set of styles.

Wrap Up

With the control adapter architecture, it is possible to completely change the rendering of individual controls based on the type of browser that is being used to make the request, without any modifications to the controls themselves. Although this infrastructure was put in place initially to support alternate renderings for mobile devices, the CSS Control Adapter Toolkit demonstrates a completely different use of the technology, which you as an ASP.NET developer should find quite useful when constructing sites that need to have all styles defined centrally in CSS files.

Send your questions and comments for Fritz to xtrmasp@microsoft.com.

Fritz Onion is a cofounder of Pluralsight, a Microsoft .NET training provider, where he heads the Web development curriculum. Fritz is the author of Essential ASP.NET (Addison Wesley, 2003) and the upcoming Essential ASP.NET 2.0 (Addison Wesley, 2006). Reach him at pluralsight.com/fritz.


Security Briefs: CardSpace, SqlMembershipProvider, and More

Security Briefs
CardSpace, SqlMembershipProvider, and More
Keith Brown

Code download available at: SecurityBriefs2006_10.exe (151KB)

In this month's column, I'd like to take time to answer a few questions that I frequently get from readers. I'll cover information cards, the SqlMembershipProvider in ASP.NET 2.0, and access control GUIs.

Q How can I discover the long-term key for a security token from a self-issued information card?

A In a recent column that covered InfoCard (which has since been renamed Windows® CardSpace), I talked about using a self-issued information card in place of a user name and password. At about the same time I wrapped that column, the February community technology preview (CTP) shipped. Unfortunately, that was not a good release for CardSpace and most CardSpace samples stopped working under that release. However, in late May, Microsoft released a new version that brought CardSpace back to life.

I've got a new and improved Notepad Web service example that demonstrates how to discover the user's long-term key. Before I show you how to do this, let me explain why this is important. Say you write a service that accepts self-issued security tokens. When Alice contacts your service, the client-side CardSpace plumbing creates a security token with the claims you've requested. These claims can be an e-mail address, a phone number, or any other personal information that can be stored in a self-issued card. Of course, this could also be an identifier that allows the user to remain anonymous. This is the personal private identifier (PPID) I've talked about in earlier columns. E-mail addresses are common, but PPIDs are handy when user anonymity is important.

Imagine that you use an e-mail address to identify the user. When Alice first visits your site and sends you a self-issued security token, you record her e-mail address as a new account. When she visits a second time, you can look up her user account data using the e-mail address you receive in the security token. But how do you know the person on the other end is really Alice and not a different user trying to impersonate her? There's no longer a password, so what secret information is being used to authenticate the user?

Remember, the security token that is being sent by the user will be signed with a private key that only the user's self-issued security token service knows. The corresponding public key will be included in the signature so that Windows Communication Foundation (or whatever type of plumbing your relying party uses) can verify the signature. This is your cryptographic assurance that the client does, in fact, know her private key. However, Windows Communication Foundation doesn't know what the correct public key is for any given user. If an attacker wants to impersonate alice@fabrikam.com, all he needs to do is generate a new public/private key pair, sign his token with the private key, and send the corresponding public key in the signature. It's easy to forget this little detail, but while Windows Communication Foundation is happy to do the heavy lifting by verifying the signature against the submitted public key, it's still your job to make sure that the submitted public key is the same public key the user initially used.

So how do you discover this public key? It's easy using AuthorizationContext in Windows Communication Foundation. Enumerating the claims sent by your user gives you the e-mail address you requested as well as the sender's RSA public key.

Before any privacy advocates get upset with me, let me clarify what this all really means. If there were only one public key used for all interactions, it would be impossible to maintain anonymity across relying parties. If those parties had no scruples, they could use your public key as an identifier and easily correlate all the claims you gave each of them to create a shared dossier of information about you. This is clearly a scenario that should be avoided!

To understand what's really going on, recall how the PPID is constructed. It's essentially a hash of various bits of information that comes from the relationship between a particular information card and a particular relying party. You can use the same information card with 10 different relying parties and each relying party will see a different PPID since the information they provide (such as their public key) will differ. Well, the self-issuing security token service generates RSA key pairs in a similar way. Each of those 10 relying parties will see a different public key, even if you use the same information card. This helps keep the user in control of her identity.

Figure 1 shows the code you can use to retrieve this public key. Note that the URIs for self-issued information card claims have changed slightly since my last column.

Figure 1 Retrieve the Public Key (Close Figure 1)
object getCallerPublicKey() {
    const string CLAIMTYPE_RSA =  http://schemas.microsoft.com/ws/
        "2005/05/identity/claims/Rsa";
    AuthorizationContext ctx =
        ServiceSecurityContext.Current.AuthorizationContext;
    foreach (ClaimSet cs in ctx.ClaimSets) {
        foreach (Claim c in cs) {
            if (Rights.Identity == c.Right &&
                CLAIMTYPE_RSA == c.ClaimType) {
                    return c.Resource;
            }
        }
    }
    throw new Exception("Expected an RSA claim");
}

An RSA public key isn't just a string of bytes. It actually comes in two parts: a modulus and an exponent. When you read the public key claim from AuthorizationContext, you actually get an object that extends the RSA abstract class, which itself extends AsymmetricAlgorithm in the System.Security.Cryptography namespace. To extract the public key parameters, you need to downcast and call RSA.ExportParameters to get access to the modulus and exponent. You hash them together to produce a unique identifier for the key that you can store in your database. The method shown in Figure 2 does just that. If you don't mind storing binary data, you can skip the Base64 encoding step; this will produce a 32-byte identifier for the key.

Figure 2 Produce a Unique Identifier (Close Figure 2)
string hashPublicKey(object rsaKey) {
    RSAParameters keyParams = ((RSA)rsaKey).ExportParameters(false);
    SHA256Managed hashAlg = new SHA256Managed();
    hashAlg.TransformBlock(keyParams.Exponent, 0,
        keyParams.Exponent.Length, null, 0);
    byte[] hash = hashAlg.TransformFinalBlock(keyParams.Modulus, 0,
        keyParams.Modulus.Length);
    return Convert.ToBase64String(hash);
}

The Notepad sample I discussed in my earlier column used a PPID to identify users while allowing them to remain anonymous. And it kept a little piece of state for each user—a note—which he could access by calling GetNote or SetNote and presenting a self-issued security token to identify himself. My updated sample differs in that it doesn't index the notes based only on PPID. Instead, it combines the PPID and the hash of the public key to form a user identifier. This way if an attacker submits Alice's PPID along with his own public key, he won't be able to see Alice's note. Figure 3 shows the new code for GetNote and SetNote, showing how to pair up the PPID with the public key hash to index into the dictionary where I keep state for each user.

Figure 3 Pairing the PPID with the Public Key Hash (Close Figure 3)
public class Notepad : INotepad {

    static Dictionary<string, string> map =
        new Dictionary<string, string>();

    public string GetNote() {
        string userId = getUserIdentifier();
        lock (map) {
            if (map.ContainsKey(userId))
                 return mapUserIdToNotes[userId];
            else return null; // new user
        }
    }

    public void SetNote(string data) {
        string userId = getUserIdentifier();
        lock (map) {
            map[userId] = data;
        }
    }

    string getUserIdentifier() {
        string ppid = getCallerPPID();
        string publicKeyHash = getCallerPublicKeyHash();
        return ppid + publicKeyHash;
    }
    // ...
}

In general, you should consider storing the user's public key as part of any user tracking mechanism you rely upon. A column in a Users database table, for instance, would serve this purpose well. When you enumerate the claims in the self-issued token, grab the RSA claim as well and verify that it's the same key the user presented in the past. Be sure to download my latest sample code from the MSDN®Magazine Web site, which works with Beta 2 of Windows CardSpace.

Q What password format should I choose when using the SqlMembershipProvider in ASP.NET 2.0?

A The SqlMembershipProvider ships with three password formats: Clear, Encrypted, and Hashed. These values are found in the MembershipPasswordFormat enumeration, and you can select which one to use via configuration. If you already require your users to supply a password in order to log in, your system probably has assets that require protection—otherwise you wouldn't inconvenience your users with a logon prompt.

When you require users to submit passwords, you owe them a certain amount of due diligence. Exposing passwords in cleartext so that anyone with SQL SELECT privileges to the membership table can casually browse them is simply negligent. So that rules out Clear in production scenarios.

What about Encrypted? Well, one consideration is that if you encrypt a user's password, you can always decrypt it and communicate it to her if she forgets it. While it might be tempting to simply e-mail the password to the user, e-mail messages are normally sent in the clear and are not at all a secure way to transmit secret information like passwords. Unless the password is basically a nuisance fee for using your Web site, you should probably use an SSL protected Web page. But as you'll see soon enough, the main problem with encrypted passwords is that you need a secret to use them.

The secret key that SqlMembershipProvider uses to encrypt passwords is the one specified in the <machineKey> element in the configuration file. If you don't specify this configuration element, it is set to the equivalent of AutoGenerate, IsolateApps. With the AutoGenerate option, ASP.NET uses a randomly generated master key that is stored on the machine. Meanwhile, the IsolateApps option tells ASP.NET to derive individual keys for each application by modifying the master key based on the hash of the application's virtual directory. This doesn't work very well for encrypting passwords because there's no easy way to back up this data. If you are developing on one machine and testing on another, you need to create new test user accounts for each system. And if you lose the encryption key in a production system, you lose the ability to encrypt or decrypt passwords and thus you can no longer authenticate your existing users. So the first problem you'll likely encounter is the exception that the ASP.NET team put in to warn you not to use auto-generated keys if you're going to encrypt passwords.

To solve this problem, you need to generate a strong, random key and add the <machineKey> element to your configuration file. (Don't let the name of this element fool you—you can put this in your web.config file if you want your application to have its own individual key). If you need a tool to help you generate the key, surf to www.pluralsight.com/tools.aspx and download the ASP.NET Administration Console. This tool will help you generate the strongest possible <machineKey> that ASP.NET 2.0 supports, and since the tool comes with source code, you can verify that it doesn't e-mail a copy of the key to me. All jokes aside, this is a serious consideration when using tools of this nature. In addition, you should avoid the IsolateApps feature so that your key doesn't depend on the name of your virtual directory, since the name could very well change over time.

You don't want that key sitting around visible for any sysadmin to see through casual browsing. You can use aspnet_regiis to encrypt the <machineKey> section once you've made a backup copy of the key. To learn how this is done, see the patterns & practices article "How To: Encrypt Configuration Sections in ASP.NET 2.0 using DPAPI" available at msdn.microsoft.com/library/en-us/dnpag2/html/PAGHT000005.asp. It's really quite easy.

Now here's the really bad news. Because all user accounts are encrypted using <machineKey>, if you change that key, you'll need to rekey your user database, and there's no easy way to do that with the providers as they stand today. So once you do this, your <machineKey> becomes very difficult to change. And a key that is hard to change is a liability. What happens when your system administrator quits and joins the competition? Things could get uncomfortable. And many other security features, such as login cookie protection and view state protection, all depend on that same key that you can no longer easily change. In order to change the <machineKey> you'll need to write code to rekey your membership user database and there's no built-in mechanism to make this an easy thing to do.

So my recommendation is to use the Hashed password format. When you add a new user account with SqlMembershipProvider, it generates a salted hash based on the password and stores the salt and the hash in the user's record as a password verifier. (I described this technique in detail in the August 2003 Security Briefs column at msdn.microsoft.com/msdnmag/issues/03/08/SecurityBriefs.) When a user tries to log in, the provider looks up the user's account based on the name she has provided, reads the salt from the record, and uses that to compute a hash of the password supplied by the user. If the hash matches the stored verifier, the user is authenticated and allowed to move on.

Some functionality of SqlMembershipProvider isn't available if you choose Hashed passwords. For example, if you try to retrieve the user's password, you'll be treated with an exception since there's no direct way to reverse the hash. If a user forgets her password, you can use the ResetPassword method to have the membership system generate a new random password, which you must communicate to the user. She can then change her password using the ChangePassword control. Of course, now you need to worry about having a secondary means of authenticating the user. Otherwise, you'll have bad guys resetting passwords for legitimate users in order to take over their accounts. The question and answer system built into the Membership feature can help with this.

By the way, just because you can't directly reverse the hash doesn't mean the passwords cannot be retrieved. If an attacker gets his hands on your user records in the membership table, he can mount an offline dictionary or brute force attack, guessing passwords and computing their hashes until he finds one that matches. This attack will quickly break weak passwords. A strong password policy is the best way to slow down this sort of attack. If this is a concern, use the attributes on the membership provider in your config file to require strong passwords. By default, the SqlMembershipProvider requires a minimum of 7 characters for a password, and one of those characters must be a non-alphanumeric symbol. You can go further if you like, specifying a regular expression that must be satisfied when a new user registers. You can even write code to validate new passwords by hooking up to the provider's ValidatingPassword event. If you end up writing your own password validation code, you can do things like checking for dictionary words in the password and so on. Figure 4 provides an example configuration that uses the Hashed password format, requiring an 8-character password with at least two non-alphanumeric characters.

Figure 4 Hashed Password Format in Action (Close Figure 4)
<connectionStrings>
  <clear/>
  <add name="asp"
       connectionString="integrated securiyt=sspi;database=myaspnetdb"
       providerName="System.Data.SqlClient" />
</connectionStrings>

<system.web>
  <!-- other entries omitted for brevity -->
  <membership defaultProvider="MyProvider">
    <providers>
      <clear/>
      <add name="MyProvider"
        type="System.Web.Security.SqlMembershipProvider, System.Web,
              Version=2.0.0.0, Culture=neutral,
              PublicKeyToken=b03f5f7f11d50a3a"
        connectionStringName="asp"
        applicationName="MyApplication"
        enablePasswordRetrieval="false"
        enablePasswordReset="true"
        requiresQuestionAndAnswer="true"
        requiresUniqueEmail="false"
        passwordFormat="Hashed"
        maxInvalidPasswordAttempts="5"
        minRequiredPasswordLength="8"
        minRequiredNonalphanumericCharacters="2"
        passwordAttemptWindow="10"
        passwordStrengthRegularExpression="" />
    </providers>
  </membership>
</system.web>

Q Should I use the AclUIAdapter to edit the access control list on a file?

A A reader recently asked how he should go about building a dialog box in managed code that is similar to the one in Windows Explorer that allows a user to edit the access control list (ACL) for a file. My first inclination was to suggest using the AclUIAdapter, which I presented in my March 2005 column.

But there’s a much easier solution. Let Windows Explorer handle the heavy lifting! There’s a function in Win32 called SHObjectProperties, which will display the Windows Explorer property sheet for files and printers. Here’s the Win32® definition:

BOOL SHObjectProperties(
    HWND hwndOwner, DWORD dwType,
    LPCWSTR szObject, LPCWSTR szPage
);

The first argument is the owner window handle for the property sheet, which can be NULL if you don’t have a window handy. The second indicates the type of object for which Windows Explorer should display a property sheet. These types are defined in shlobj.h:

#define SHOP_PRINTERNAME 0x00000001
#define SHOP_FILEPATH    0x00000002
#define SHOP_VOLUMEGUID  0x00000004
Given this, it’s quite easy to whip up some P/Invoke stubs and call this function to display a property sheet. Figure 5 shows the code for FSP.EXE, short for File System Properties. Figure 6 shows the property sheet that pops up when it is run from the command line:
fsp security c:\autoexec.bat
This program could easily be modified to show a printer or volume property page as well.

Figure 5 Using SHObject Properties (Close Figure 5)
using System;
using System.Runtime.InteropServices;

// fsp == file system properties
// usage: fsp propertyPageName filename
//        fsp security c:\autoexec.bat
class DisplayFileSystemPropertySheet {
    static void Main(string[] args) {
        if (2 != args.Length) {
            printUsage();
            return;
        }
        string page = args[0];
        string path = args[1];

        if (!SHObjectProperties(IntPtr.Zero,
            ShellObjectType.FilePath, path, page)) {
                Marshal.ThrowExceptionForHR(
                    Marshal.GetHRForLastWin32Error());
        }
        Console.WriteLine("Press any key to continue...");
        Console.ReadKey();
    }

    static void printUsage() {
        Console.WriteLine("Usage: fsp propertyPageName filename");
        Console.WriteLine("Example: fsp Security c:\\autoexec.bat");
    }

    enum ShellObjectType : int {
        PrinterName = 1,
        FilePath = 2,
        VolumeGuid = 4
    }

    [DllImport("shell32.dll", CharSet = CharSet.Unicode,
               ExactSpelling = true, SetLastError = true)]
    [return: MarshalAs(UnmanagedType.Bool)]
    static extern bool SHObjectProperties(IntPtr hwndParent,
        ShellObjectType type, string name, string page);
}
Figure 6 Properties Displayed
Figure 6 Properties Displayed

Send your questions and comments for Keith to briefs@microsoft.com.

Keith Brown is a cofounder of Pluralsight, a premier Microsoft .NET training provider. Keith is the author of Pluralsight's Applied .NET Security course, as well as several books, including The .NET Developer's Guide to Windows Security, which is available both in print and on the Web. Learn more at www.pluralsight.com/keith.


Wicked Code: Running ASMX Web Services on STA Threads

Wicked Code
Running ASMX Web Services on STA Threads
Jeff Prosise

Code download available at: WickedCode2006_10.exe (152KB)
Contents
The Problem
The Solution
Conclusion

Many of the gnarliest issues you read about in Wicked Code come from real problems experienced by real people trying to build software that works. Recently, I came across a problem that I had never encountered before, but that I'm certain other developers have or someday will experience. The problem was one of concurrency, and had to do with ASMX Web services and legacy COM components. The development team who brought the issue to my attention had built an ASMX Web service that relied on legacy COM components written in Visual Basic® 6.0 to perform key processing tasks.

The design called for a high degree of parallelism in processing requests, so each request submitted to the Web service created its own Visual Basic component instance, and then called it. Performance was abysmal. Investigation proved that despite the number of Web service threads making concurrent calls to an equal number of Visual Basic component instances (even if there were 20 or more), only one component instance was executing at a time.

In effect, calls were being queued up at the doorway to COM and serialized into the respective component instances. Because some of the calls required 20 seconds or more to complete, the problem wasn't one the dev team could ignore. Instead, it was a showstopper—the product could not ship until the problem was rectified.

After countless hours of trying to get a handle on the problem, the dev team realized they were up against a wall. Something was causing all those concurrent calls to be executed sequentially rather than in parallel. But what? Was it COM? Was it Visual Basic 6.0? Or was it a nuance of the ASMX architecture? Whatever the problem was, a solution had to be found fast. Otherwise, the schedule would slip and the entire project would be placed in jeopardy.

The Problem

In a perfect world, everyone writing code for Microsoft platforms would write managed code and would never again have to deal with legacy technologies, such as COM. But the reality is that millions of lines of critical business logic are encapsulated in COM components—Visual Basic 6.0 COM components, in particular—and ASP.NET developers frequently have no choice but to call those components from managed code.

The Microsoft® .NET Framework does a fine job of allowing managed code to call out to unmanaged COM components. For example, the Framework's Tlbimp.exe utility can import COM type libraries and generate Runtime Callable Wrappers (RCWs) that permit unmanaged components to be called as if they were managed. But neither Tlbimp.exe, nor any other .NET tool, can mitigate the concurrency issues that arise when calls go out from managed code to unmanaged COM components.

It was a concurrency issue like this that produced the problem the dev team had encountered. Figure 1 shows the threading configuration of the Web service and the components that it has created. In this example, the Web service is processing five concurrent requests. ASMX Web services use the same HTTP pipeline that ASPX pages use, so each request has been allocated a thread by ASP.NET. ASP.NET threads are, in reality, COM multithreaded apartment (MTA) threads—that is, they're threads that run in a COM MTA. Because Visual Basic 6.0 COM components are incompatible with COM MTAs, the component instances that are created by the Web service threads are not in the MTA with their creator threads. Instead, they live in a single-threaded apartment, or STA, created by COM. The STA, like all COM STAs, is driven by a single thread. And because all five component instances share an STA, they share that single thread.

Figure 1 Default Apartment Configuration
Figure 1 Default Apartment Configuration

The fact that all five component instances run on the same thread in the same apartment explains why calls are serialized. When one of the MTA threads calls a COM component, COM marshals the call from one apartment to the other, performing a thread switch in the process. If the STA thread is busy processing a call from one MTA thread, COM continues to queue calls from other MTA threads until the STA thread becomes available. All parallelism is lost when the call crosses the boundary from the MTA to the STA because the STA can only do one thing at a time. In effect, the STA is like a big mutual exclusion lock that causes caller B to wait until the call from caller A has completed.

You can prove that Web services run on MTA threads with the simple Web service, TestService.cs, shown in Figure 2. Its one and only Web method, Test, returns a string that indicates what type of COM apartment it's running in. Figure 3 shows what happens when you invoke the Test method from ASP.NET's autogenerated test harness. The string "MTA" in the results clearly shows that the request was processed by an MTA thread.

Figure 2 Testing for MTA Threads (Close Figure 2)
using System;
using System.Web;
using System.Collections;
using System.Web.Services;
using System.Threading;

[WebService(Namespace = "http://tempuri.org/")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
public class TestService : System.Web.Services.WebService
{
    [WebMethod]
    public string Test()
    {
        return Thread.CurrentThread.ApartmentState.ToString();
    }
}
Figure 3 Proof of MTA Threads
Figure 3 Proof of MTA Threads(Click the image for a smaller view)
Figure 3 Proof of MTA Threads
Figure 3 Proof of MTA Threads (Click the image for a larger view)

The scenario depicted in Figure 1 may seem contrived, but in fact it's extraordinarily common. Because ASP.NET requests run on MTA threads by default, callouts to STA-based COM objects (this includes all Visual Basic 6.0 COM components) are marshaled from the ASP.NET MTA into the COM STA. Developers often don't realize that the STA is a choke point and that seemingly concurrent calls to individual component instances are actually serialized by COM. This fact only becomes obvious when individual calls take a long time to complete.

This would be less of a problem if COM would create multiple STAs—one per MTA thread that creates an object instance. But COM doesn't do that, in part because COM is loath to spin up new threads in a process. (COM+ will do that, but COM+ is a different entity from COM.) When an MTA thread creates an STA-based object, COM is forced to create a new thread to drive that STA. When a second MTA thread creates an STA-based object, COM places that object instance in the STA it created for the first object instance. The same goes for the third object instance, fourth object instance, and so on.

Fortunately, there is a convenient solution to the COM STA bottleneck that page developers can use. While including an AspCompat="true" attribute in an ASPX's Page directive provides COM components access to ASP-style infrastructure such as Request and Response objects, it also has the very desirable effect of creating an STA thread pool and processing requests for that page with STA threads. Besides eliminating the marshaling overhead incurred by MTA-to-STA thread switches, AspCompat="true" allows STA COM objects (specifically, instances of COM classes registered ThreadingModel="Apartment") to be created in their creators' apartments provided the creators are running in STAs themselves. If there are five request processing threads running in STAs and each creates an STA COM object, then all five object instances share an apartment—and a thread—with their creator. COM is happy to place each object instance in a separate STA because it doesn't have to create the STAs (or threads to drive them). Calls to the objects are no longer queued to a single thread, and the objects can execute code concurrently.

Alas, there is no equivalent of AspCompat="true" for ASMX files. The WebService directive won't accept an AspCompat attribute. It's left to the ASMX developer to come up with a way to put ASMX requests on STA threads to avoid the STA bottleneck. Now let's look at the means for doing this.

The Solution

Before coding, I scanned a number of forums and blogs. I found a post at www.epocalipse.com/blog/category/general that proposed a solution that was both elegant and concise. It outlined an idea to write an HTTP handler that derives from System.Web.UI.Page, register it as the handler for ASMX files, and use the AspCompat infrastructure already in the Page class to lend AspCompat support to ASMX. The post was even accompanied by sample code (which was correct in principle, albeit flawed in implementation).

I built on this to produce the Page derivative shown in Figure 4. When it invokes the handler, ASP.NET calls AspCompatWebServiceHandler's BeginProcessRequest method, since AspCompatWebServiceHandler implements the IHttpAsyncHandler interface. (Without that interface, ASP.NET would call the handler's synchronous ProcessRequest method instead.) BeginProcessRequest delegates to the AspCompatBeginProcessRequest method inherited from Page. This method serves as the gateway to the AspCompat infrastructure that is built into the Page class. Shortly after AspCompat processing begins, AspCompatWebServiceHandler's OnInit method is called. This creates a normal ASMX HTTP handler (new WebServiceHandlerFactory.GetHandler) and calls its ProcessRequest method, passing in the HTTP context for the current request. Thus, the request undergoes the normal ASMX processing, but it does so within an AspCompat wrapper that processes the request with an STA thread instead of an MTA thread.

Figure 4 Using STA within an AspCompat Wrapper (Close Figure 4)
using System;
using System.Web;
using System.Web.UI;
using System.Web.Services.Protocols;
using System.Web.SessionState;

public class AspCompatWebServiceHandler :
    System.Web.UI.Page, IHttpAsyncHandler, IRequiresSessionState
{
    protected override void OnInit(EventArgs e)
    {
        IHttpHandler handler =
            new WebServiceHandlerFactory ().GetHandler(
                this.Context,
                this.Context.Request.HttpMethod,
                this.Context.Request.FilePath,
                this.Context.Request.PhysicalPath);
        handler.ProcessRequest(this.Context);
        this.Context.ApplicationInstance.CompleteRequest();
    }

    public IAsyncResult BeginProcessRequest(
        HttpContext context, AsyncCallback cb, object extraData)
    {
        return this.AspCompatBeginProcessRequest(
            context, cb, extraData);
    }

    public void EndProcessRequest(IAsyncResult result)
    {
        this.AspCompatEndProcessRequest(result);
    }
}

In order for AspCompatWebServiceHandler to work its magic, it must be registered as the HTTP handler for ASMX files. You can register it using this web.config file:

<configuration>
  <system.web>
    <httpHandlers>
      <add verb="*" path="*.asmx"
        type="AspCompatWebServiceHandler, __code" />
    </httpHandlers>
  </system.web>
</configuration>
Note the assembly name used in the type attribute: __code. This assumes that you have placed the source code for AspCompatWebServiceHandler in the application's App_Code directory and allowed ASP.NET to generate an assembly from it. If, instead, you compiled the assembly yourself and place it in the bin directory, simply replace __code with the assembly name. (Since ASP.NET 1.x doesn't support App_Code directories, you'll have to do this if you're not running ASP.NET 2.0.)

In order to confirm that ASMX requests are now processed with STA threads, run TestService again after installing AspCompatWebServiceHandler. Figure 5 shows the results. The thread that ASP.NET assigned to the request is clearly an STA thread. Mission accomplished!

Figure 5 Confirming Use of STA Threads
Figure 5 Confirming Use of STA Threads(Click the image for a smaller view)
Figure 5 Confirming Use of STA Threads
Figure 5 Confirming Use of STA Threads (Click the image for a larger view)

While it is a bit of a hack (there's often a very fine line between a hack and wicked code!), this solution worked flawlessly on the problem application. Figure 6 shows the threading and apartment configuration of the modified app. The five request processing threads are now STA threads living in separate STAs. Because COM places STA object instances in the STAs of their creators when the creators are STA threads, each object instance now has its own STA. Moreover, calls to the objects now execute in parallel because each object instance runs on a separate thread.

Figure 6 New Apartment Configuration
Figure 6 New Apartment Configuration

Conclusion

Besides eliminating MTA-to-STA marshaling overhead, including the AspCompat="true" attribute allows COM to place object instances in separate STAs rather than lumping them into one apartment (and running them on one thread). Although AspCompat isn't supported for Web services, you can add it to your ASMX code and enjoy the same efficiency in calling Visual Basic 6.0 COM components from Web services as you do from Web pages.

If you'd like to learn more about COM threading and apartments, check out a pair of articles I wrote back when COM was cool. The first is available at www.codeguru.com/cpp/com-tech/activex/apts/article.php/c5529 and the second at www.codeguru.com/cpp/com-tech/activex/apts/article.php/c5533. The articles are several years old, but the content is just as pertinent today as it was back then—especially if you're a programmer who's into .NET and faced with the challenge of calling old COM components from managed code.

Send your questions and comments for Jeff to wicked@microsoft.com.

Jeff Prosise is a contributing editor to MSDN Magazine and the author of several books, including Programming Microsoft .NET (Microsoft Press, 2002). He is also a cofounder of Wintellect, a software consulting and education firm.


C++ at Work: IRegistrar, Finding Submenus, and More

C++ At Work
IRegistrar, Finding Submenus, and More
Paul DiLascia

Code download available at: C++atWork2006_10.exe (174KB)

Q When I generate a DLL with automation support using Visual C++® 6.0, some functions for registration are generated, but not for unregistering my DLL. I wrote a DllUnregisterServer that looks like this:

STDAPI DllUnregisterServer(void)
{
   AFX_MANAGE_STATE(AfxGetStaticModuleState());
   if (!COleObjectFactory::UnregisterAll())
      return ResultFromScode(SELFREG_E_CLASS);
   return NOERROR;
}
COleObjectFactory::UnregisterAll returns TRUE, but my DLL is still registered. How should I write the code?

Ivan Pavlik

A

Alas, you stumbled onto a small wormhole in the MFC universe. UnregisterAll would indeed appear to be the function you'd call to unregister your DLL. If you peer inside olefact.cpp, where UnregisterAll is implemented, you'll discover it looks something like this:

for (/* pFactory = all factories */) {
  pFactory->Unregister();
}
That is, it loops over all your module's factories and calls Unregister for each one. So far, so good. What does COleObjectFactory::Unregister do? The code tells the story:
BOOL COleObjectFactory::Unregister()
{
   return TRUE;
}

Oops. As you can see, COleObjectFactory::Unregister doesn't do anything! It just returns TRUE. Which is most peculiar, since COleObjectFactory::Register does actually register your COM class by calling ::CoRegisterClassObject. On the other hand, there's another function, COleObjectFactory::Revoke, that calls ::CoRevokeClassObject to unregister your COM class. MFC calls Revoke automatically from your COleObjectFactory destructor. So what the heck is going on here?

The problem is an unfortunate confusion in terminology, which stems from the different ways of registering COM classes for DLLs and EXEs. For DLLs, you register your class by adding keys in the Windows registry (CLSID, ProgID, and so on). For EXEs, you must call CoRegisterClassObject to register your class with the COM system at runtime. The issue is further confused by the fact that for EXEs, the opposite of register is not unregister, but revoke (CoRevokeClassObject).

When the Redmondtonians added COM support to MFC, they did their best to make life as easy as possible, but they didn't always succeed perfectly. COleObjectFactory::Register looks like this:

// in olefact.cpp
BOOL COleObjectFactory::Register()
{
   if (!afxContextIsDLL) {
      ::CoRegisterClassObject(...,
         CLSCTX_LOCAL_SERVER, ..);
   }
}
You can see right away it does nothing for DLLs; it only registers EXEs using CLSCTX_LOCAL_SERVER (context = local server, EXE running on the local machine). Following the underlying C API, the Redmondtonians used the same name Revoke for the function that unregisters the EXE: COleObjectFactory::Revoke (which is called automatically from your class factory's destructor).

So what's COleObjectFactory::Unregister for, the function that doesn't do anything? Perhaps the Redmondtonians intended to one day make Register/Unregister work for DLLs, too. But as it stands today, they don't. To register your DLL, you need yet another completely different function: COleObjectFactory::UpdateRegistry. This function takes a Boolean argument that tells MFC whether you want to register or unregister your COM class. There's also UpdateRegistryAll, which loops over all the class factories, calling UpdateRegistry for each one. So here is the proper way to implement your DllUnregisterServer::

STDAPI DllUnregisterServer(void)
{
   AFX_MANAGE_STATE(AfxGetStaticModuleState());
   return COleObjectFactory::UpdateRegistryAll(FALSE)
      ? S_OK : SELFREG_E_CLASS;
}
DllRegisterServer should look the same, except you should pass TRUE instead of FALSE to UpdateRegistryAll.

The default implementation for UpdateRegistry does its best to add or delete the appropriate registry keys in HKCR/CLSID–InprocServer32, ProgID, Insertable, ThreadingModel, and so on, but sometimes you need additional keys specific to your COM class. For example, you may need to register categories, which are a kind of extension of the old "Insertable" key (which means your COM class can be dropped into a form in design mode). In that case, you need to override UpdateRegistry to add or remove your own keys.

Way back in the November and December 1999 issues of Microsoft Systems Journal (now known as MSDN®Magazine), I showed how to build a Band Object for Internet Explorer using the Active Template Library (ATL) IRegistrar interface. (Band Objects need to register a special category CATID_DeskBand.) IRegistrar is a really cool tool that lets you write a registration script (.RGS file) to add your registry entries, instead of calling registry functions like RegOpenKey, RegSetValue, and the rest. Figure 1 shows a typical script.

Figure 1 IRegistrar (Close Figure 1)
HKCR
{
    NoRemove CLSID
    {
        ForceRemove %CLSID% = s ‘%ClassName%’
        {
            InprocServer32 = s ‘%MODULE%’
            {
                val ThreadingModel = s ‘%ThreadingModel%’
            }
            ProgID = s ‘%ProgID%’
        }
    }
}

As you can see, IRegistrar lets you define variables like %ClassName% and %ThreadingModel%, then replace them with actual values at runtime. With IRegistrar, you never have to call the registry API again. You can write a script that more resembles the actual registry entries and you can even use the same script to register or unregister your DLL. That's right, IRegistrar is smart enough to unregister. IRegistrar isn't officially documented anywhere I can find, but the code is there (find it in atliface.h). If you're not using IRegistrar to register and unregister your COM DLLs, you really should check it out—it'll save you tons of work. For details see my November 1999 article.

Q My main app has a normal menu with an Edit submenu of commands (Cut, Copy, Paste, and so on.). I'd like to show the Edit submenu as a context menu when the user right-clicks in the main window. The problem is how to get this submenu from the main menu. There doesn't seem to be any command ID associated with a submenu. I can use the zero-based index, but because of customization the Edit menu may not always be the second menu. I can't search for the text "Edit" because we support multiple languages and the actual word may change. How can I find the Edit submenu in all these cases?

Brian Manlin

A Well, let me point out that the Edit menu should be the second submenu, if you have one. The Official Windows® GUI Guidelines call for the first three menu items to be File, Edit, View, in that order—if you support them. See The Windows Interface Guidelines for Software Design (Microsoft Press®, 1995). That said, I can help you find your Edit submenu or any other submenu for that matter.

You are indeed correct, there's no command ID for a submenu. That's because internally Windows uses the command ID field to hold the submenu's HMENU, if the menu item is a submenu. Not to worry, assuming the submenu always holds a specific command (for example Edit | Cut), and assuming the command always has the same ID (for example, ID_EDIT_CUT), it's easy enough to write a function that finds the submenu that contains a given command. Figure 2 shows the code. CSubmenuFinder really acts as a namespace to hold the static function FindCommandID, which calls itself recursively to deep-search the menu and all submenus for a menu item whose command ID matches the one sought.

Figure 2 Finding a Submenu (Close Figure 2)

SubmenuFinder.h
#pragma once

//////////////////
// Class to find a submenu that contains a given command ID.
// This class is really just a namespace for the function FindCommandID.
//
class CSubmenuFinder
{
public:
   CSubmenuFinder() { }
   ~CSubmenuFinder() { }
   static CMenu* FindCommandID(CMenu* pMenu, UINT nID);
};
SubmenuFinder.cpp
#include "StdAfx.h"
#include "SubmenuFinder.h"

#ifdef _DEBUG
#define new DEBUG_NEW
#undef THIS_FILE
static char THIS_FILE[] = __FILE__;
#endif

//////////////////
// Find the submenu within a given menu that contains a given command // ID.
// Iterate the menu items looking for the command ID. Recurse
// into submenus if necessary.
//
CMenu* CSubmenuFinder::FindCommandID(CMenu* pMenu, UINT nID)
{
   if (pMenu) {
      for (UINT i=0; i<pMenu->GetMenuItemCount(); i++) {
         if (pMenu->GetMenuItemID(i)==nID)
            return pMenu; // found command--return menu

         // Recurse! Note this returns NULL if item is not a submenu.
         CMenu* pSubMenu = CSubmenuFinder::FindCommandID(
             pMenu->GetSubMenu(i),nID);

         if (pSubMenu) return pSubMenu; // found submenu -- return it!
      }
   }
   return NULL;
}
You can use CSubmenuFinder to find your edit menu, like so:

void CMainFrame::OnContextMenu(..)
{
   CMenu* pMenu = CSubmenuFinder::FindCommandID(
      GetMenu(),ID_EDIT_CUT);
   if (pMenu) pMenu->TrackPopupMenu(...);
}

This snippet searches the main menu for a submenu containing a menu item whose command ID is ID_EDIT_CUT, and returns the submenu if found. I wrote a little program called EdMenu to test CSubmenuFinder. EdMenu uses the previous code to handle WM_CONTEXTMENU to display the very same edit submenu as the one in the main menu, when the user right-clicks in the main window. You'll have to believe me that CSubmenuFinder finds the edit menu wherever it appears within the main menu. I did in fact test it by temporarily moving the Edit menu ahead of the View menu in one of my builds. To try it yourself, download the source.

Q How do I convert MFC CString to String in managed C++? For example, I have the following code in C++:

void GetString(CString& msg)
{
   msg = // build a string
}
How do I rewrite this function using managed C++ and replacing the CString parameter with a managed String? The problem is that GetString changes the caller's CString, and I want to do the same thing using a managed String.

Sumit Prakash

A Well, the answer depends on whether you're using the new C++/CLI syntax or the old Managed Extensions. Both flavors of syntax can be confusing, but never fear, I'm here to clear your coding confusion.

Let's start with the new syntax, since it is 2006. The trick to getting the syntax right is to remember two things. First, in C++/CLI, managed objects wear hats. So CString becomes String^. Second, remember that the managed equivalent to reference (&) is tracking reference, which in C++/CLI is %. I know, you can barely wrap your head around ^ but with enough practice, % will soon seem normal too. So the new functions looks like this:

// using C++/CLI
void GetString(String^% msg)
{
   msg = // build a string
}

Does that make sense? To show that it actually works, I wrote a little program strnet.cpp (see Figure 3). It implements two classes, CFoo1 and CFoo2, each with a member GetName. The first uses CString& and the second uses String^%. If you compile and run this program (using /clr of course), you'll see that in both cases—the one using CString and the one using String—the object passed (native or managed) is modified by the member function.

Figure 3 Using a Tracking Reference (Close Figure 3)
#include "stdafx.h"
#include "resource.h"

using namespace System;

//////////////////
// Passing a reference to a native MFC CString
//
class CFoo1 {
public:
   void GetName(CString& name)
   {
      name = _T("Rumplestilskin");
   }
};

//////////////////
// Passing a reference to a managed String^
//
class CFoo2 {
public:
   void GetName(String^% name)
   {
      name = _T("Rumplestilskin");
   }
};

int _tmain(int argc, TCHAR* argv[], TCHAR* envp[])
{
   CFoo1 foo1;
   CFoo2 foo2;

   CString name;
   foo1.GetName(name);
   tprintf(_T("GetName returns %s\n"), name);

   String^ s = gcnew String("");
   foo2.GetName(s);
   Console::WriteLine("GetName returns {0}\n",s);
}

By the way, you always want to use a tracking reference (%) for managed objects, instead of utilizing a native reference (&, which will also compile) because the tracking reference will track the reference even if the garbage collector moves the object within its managed heap.

If you're using the old-style syntax (/clr:oldSyntax), how come you're still living in the dark ages? Never fear, you can use a reference even if you're retro. If you're using the Managed Extensions, just remember that managed objects are __gc pointers, so CString becomes String __gc *. And since the compiler already knows String is a managed type, you don't even need the __gc; a simple pointer (*) will do. The reference is the same. So the old-style syntax conversion looks like this:

// __gc omitted
void GetString(String*& msg)
{
   msg = // build a string
}

Got it? It all kind of makes sense. Whenever you have trouble figuring out C++ syntax, the thing to do is go back to basics.

Q I recently generated a CLR console app with Visual Studio® 2005. I noticed that Visual Studio created a main function that looks like this:

int main(array<System::String ^> ^args)
{
   ...
   return 0;
}
This seems like a change from the old argc/argv I'm familiar with from C/C++. When I tried to access args[0], thinking it would be the filename (as in C/C++), I discovered that args[0] is not the filename, but the first command-line parameter. What happened to the filename? And can you explain the reason for this change?

Jason Landrew

A Oh, the modern world in which we frolic. Nothing is the same any more, is it? We've lived with argc/argv since C was developed in 1972 and now the Redmondtonians have gone and changed it. Perhaps they wanted to make sure everyone knows how to use the new array syntax.

One reason for the new syntax is to generate a program that compiles with /clr:safe, which is the ultimate in code security. It's kind of like locking your program inside a clean room—you know, one of those sterile environments full of chrome equipment where you have to enter through a negative-pressure airlock and don a bonnet and booties. When you throw the /clr:safe switch, the compiler imposes all sorts of onerous restrictions. You can't use native types. You can't use globals. You can't call unmanaged functions. In short, you can't do anything fun. And why would you subject yourself to such misery? To be really, really, really sure your code is secure. In technical terms, /clr:safe generates "verifiable" code, which means the common language runtime (CLR) can verify that your code doesn't violate security settings, like trying to access a file when the user's security settings don't allow it. One of the many items on the /clr:safe taboo list is native pointers—which is to say, pointers. ("Native" is redundant since you can't have a pointer to a managed type.) Since pointers are not allowed with /clr:safe, the old argv declaration simply won't do. So the Redmondtonians changed the declaration to use a String^ array instead. And since arrays know their length, there's no need for argc. The runtime library provides the appropriate startup code to create and initialize the arg array before calling your main function. If you don't plan to use /clr:safe, you can always write a main that uses the old argc/argv signature.

That explains the reason for using a managed array, but what about args[0], why is it the first command parameter and not the file name? I can't speak for the Redmondtonians, but perhaps they figured it makes more sense to omit the file name, maybe they figured you don't need it. In any case, it doesn't much matter because getting the file name is easy. I'm sure there are many ways to do it, but the most obvious and CLR-centric (compatible with /clr:safe) way I can think of is with the System::Environment class. It exposes the following method:

static array<String^>^ GetCommandLineArgs ()
Unlike the args parameter supplied to the main method, the array returned from Environment::GetCommandLineArgs does in fact start with the executable file name.

Finally, why would you want to know the name of your own EXE file, anyway? After all, you're writing the program, you presumably know what it's called. One reason is to generate a help message that contains the program name, without hardcoding it. The help message might look something like this:

FooFile -- Turns every word in your file to "foo"
usage:
   FooFile [/n:<number>] filespec
   filespec = the files you want to change
   <number> = change only the first <number> occurrences
Here, FooFile is the name of the program. I could simply write "FooFile" in my help text, but I learned long ago that I often rename programs or copy code from one program to another. By getting the program name from the program itself (through argv or Environment::GetCommandLineArgs), my help messages always display the correct name, even if I rename the file or copy my code to another program.

Happy programming!

Send your questions and comments for Paul to cppqa@microsoft.com.

Paul DiLascia is a freelance software consultant and Web/UI designer-at-large. He is the author of Windows++: Writing Reusable Windows Code in C++ (Addison-Wesley, 1992). In his spare time, Paul develops PixieLib, an MFC class library available from his Web site, www.dilascia.com.


{ End Bracket }: Is Programming an Art?

{ End Bracket }
Is Programming an Art?
Stanley B. Lippman

People who are only users of software frequently fail to appreciate just how very hard it is to write. Oh, the design and initial development are the fun parts. The difficult part is getting it to run correctly (and fast enough). It’s as if for a programmer, a meal begins with a scrumptious double fudge mocha cheesecake and ends with creamed spinach.

The reason we say programming remains an art, not a science or an engineering discipline, is because we haven’t as yet been able to break it down into component steps and mechanize it. Once we succeed in that, a new scale of possibility emerges: programs to write programs from people-oriented design languages (PODL), programs to prove program correctness, and to analyze and support semantic query. Until then, however, programming remains an art, or so conventional wisdom has it.

But as with much conventional wisdom, this analogy does not hold up to closer inspection. Crafting a program (and I’ve done this for over 20 years) is, in fact, nothing at all like art—at least not the art of writing fiction (and I’ve done that for over 20 years as well). Let me illustrate my point with these two fragments of program and prose.

This first piece is part of a lexical analyzer for parsing Visual C++® 7.1. It looks like this:

if ( index >= TokenID::lastKeyword() ) {
    string token = tokenString( index );
    if (( isdigit( token[ 0 ] )) || ( token[0] == ‘-’ )) {
#ifdef TELL_ME
        display( "!! TokenSet::tokenIdentify: ",
                     "a literal digit: ", token );
#endif
        return TokenID::TK_Literal;
    }
}

This second piece is the opening of a story I wrote some 27 years ago while living in Tucson, Arizona, teaching and writing prose:

We were her pride of 10, Miss Steward named us: the Prodigal, Phoenix,
Benjamin, and perspicacious, pacific Suzanne. Hush, child. Benjamin, be
still, Miss Steward commanded him gently. People are never just.

Both pieces were extensively designed—that is, I spent a great deal of up-front time designing the architecture of the overall work, and the fragments reflect that design effort. The program has one performance flaw—I retrieve the string entry in the associated string table by value rather than pointer. If the code were to be moved to production, it would require an obvious revision. It has nothing to do with correctness but simply with performance. For a small source code program, the difference in speed is not measurable, and so I chose the cleaner but slower object syntax since I dislike the C/C++ pointer manipulation tokens.

While my prose has not attracted millions of readers, those who do read prose appreciate the craft with its emotional weave under a rigorous symbolic framework. Those who do not appreciate the prose tradition find it mystifying because there is nothing functional about it. The purpose of art is to project feelings and the reality of a subjective private life in the midst of public events into a particular plastic medium—words, in the case of prose. Of course, there is no value in that, not like the value intrinsic to a program.

I like the design process of a program and working out how the different parts of it interact. I like defining abstractions or families of abstractions. I consider them my cast of characters. And I like doing interfaces, because those are the things my characters do to one another. And that’s really how I write programs.

I don’t like writing expressions and statements. I wish we had a symbolic programming language I could think in better and keep in my head. I can’t, frankly, keep arithmetic expressions and parenthetical nestings clear. While I can keep eight separate sections of text in my mind and contrast and echo them, I can’t make sense of a program beyond the classes I design and their interfaces.

In John Markoff’s excellent history of the PC industry, What the Dormouse Said, he states that the goal of the initial visionaries—to put a computer at the disposal of every person—has been realized. But he’s wrong. It has only been half realized. What good is all that computer power for a person, if that person can’t themselves program it? We haven’t realized their dream yet because we haven’t yet figured out how to make programming an art so that everyone can share in it. Until we do that, programming remains neither a discipline of science nor a domain of art. For now, programming is little more than a new-fangled alchemy. Nobody really understands it and nobody as yet has turned a specification into real gold.

Stanley B. Lippman began working on C++ with its inventor, Bjarne Stroustrup, in 1984 at Bell Laboratories. Later, Stan worked in feature animation both at Disney and DreamWorks and served as a Software Technical Director on Fantasia 2000. He has since served as Distinguished Consultant with JPL, and an Architect with the Visual C++ team at Microsoft.