October 2008
Paradigm Shift: Design Considerations For Parallel Programming
This article presents an overview of the motivation behind new techniques that decompose problems into independent pieces for optimal use of parallel programming. David Callahan
Coding Tools: Improved Support For Parallelism In The Next Version Of Visual Studio
We take a look at planned support for parallel programming for both managed and native code in the next version of Visual Studio. Stephen Toub and Hazim Shafi
Concurrency Hazards: Solving 11 Likely Problems In Your Multithreaded Code
Here we describe some of the more common challenges to concurrent programming and present advice for coping with them in your software. Joe Duffy
ASP.NET AJAX 4.0: New AJAX Support For Data-Driven Web Apps
Here is an ASP.NET AJAX data-driven Web application that takes the best features from server- and client-side programming to deliver an efficient, user-friendly experience. Bertrand Le Roy
Easy Async: Build Concurrent Apps From Simple F# Expressions
In this article, the author explores how the F# language helps you create asynchronous function libraries that can be called seamlessly from any other .NET-compliant language. Chance Coble
Columns
Editor's Note: new Thread(ReadEditorsNote).Start(); yourAttention.WaitOne();: Introducing the issue, Howard Dierking points out that you can't simply parallelize your code blindly if you expect to truly reap the benefits that parallelism promises.Howard Dierking | Toolbox: Realistic Test Data, Dare Obasanjo’s Blog, Color-Coding Tools, and More: This month Scott shows how to generate realistic test data, visits Dare Obasanjo's blog, and demonstrates color coding code for better readability.Scott Mitchell | CLR Inside Out: Security In Silverlight 2: Andrew Dai of the CLR team discusses the Transparency model, which creates a strong isolation boundary between privileged and unprivileged code for Silverlight apps.Andrew Dai
Basic Instincts: Dynamic Data Entry With XML Literals: Learn how to use Windows Presentation Foundation (WPF), XAML, and the deep XML support in Visual Basic to generate user interfaces dynamically.Beth Massi | Cutting Edge: Code reuse in WPF and Silverlight 2.: There’s a strong similarity between Web-based Silverlight 2 applications and desktop WPF applications. Enabling easy code reuse between the two is Dino’s focus here.Dino Esposito | Patterns in Practice: Cohesion And Coupling: Here are some design patterns that allow you to achieve higher cohesion and looser coupling for more flexible, reusable applications.Jeremy Miller
Service Station: Authorization In WCF-Based Services: Windows Communication Foundation (WCF) provides an easy role-based system and a more powerful and complex claims-based API for implementing authorization in services.Dominick Baier and Christian Weyer | Foundations: Managing State With Durable Services: This month Juval Lowy tackles questions such as when to keep proxies and services in memory for better state management in long-running workflows.Juval Lowy | Windows with C++: Exploring High-Performance Algorithms: See how you can gain efficiency in surprising ways by looking closely at your algorithms, the data they operate on, and the hardware you’re designing for.Kenny Kerr
Going Places: Ink-Enabled Apps For Tablet PC: We show you how to create ink-enabled apps quickly with the Tablet PC SDK and the InkEdit and InkPicture ActiveX controls.Gus Class | .NET Matters: False Sharing: This month: memory access issues in multi-core systems and diagnosing and avoiding false sharing in your parallel computing applications.Stephen Toub, Igor Ostrovsky, and Huseyin Yildiz | { End Bracket }: Don’t Be Afraid To Throw Away Your Work: With today's processing power at your disposal, Josh Phillips recommends speculative computation—performing operations ahead of time even if you may never need the results.Josh Phillips


Paradigm Shift
Design Considerations For Parallel Programming
David Callahan

This article is based on a prerelease of Visual Studio tools. All information herein is subject to change.

This article discusses:
  • Parallel computing
  • Concurrent programming
  • Performance improvements
 This article uses the following technologies:
Multithreading
Contents
Concurrency and Parallelism
Structured Multithreading
Data Parallelism
Data Flow
Streaming Parallelism
Single-Program, Multiple-Data
Concurrent Data Structures
Wrap-Up
From about 1986 to 2002, the performance of microprocessors improved at the rate of 52% per year. This astounding technology advance was the result of a combination of the ever-shrinking cost of transistors, as per Moore's Law, and the engineering excellence of processor vendors. This combination has been referred to as "Moore's Dividend" by Microsoft Researcher Jim Larus, who reflects on how this dividend has paid for the creation of the modern software industry and allowed computers to become as pervasive as they are (see go.microsoft.com/fwlink/?LinkId=124628).
On the software side, this phenomenon is called the "free lunch"—application performance improving simply by upgrading the hardware on which it runs. (For more on this, see "A Fundamental Turn Toward Concurrency in Software" at ddj.com/184405990.)
But the model is changing; today, performance is improved by the addition of processors. So-called multicore systems are now ubiquitous. Of course, the multicore approach improves performance only when software can perform multiple activities at the same time. Functions that perform perfectly well using sequential techniques must be written to allow multiple processors to be used if they are to realize the performance gains promised by the multiprocessor machines.

Concurrency and Parallelism
For some time now, programmers have had to think about a programming challenge related to parallelism—concurrency. In order to maintain responsiveness, there needed to be some way to offload long-latency activities from the thread that reacts to input events. Most of the activities to be offloaded had, in the past, been related to file I/O, but more and more often now they are related to conversations with Web services.
The Microsoft .NET Framework provides the asynchronous programming model and notions of background workers to facilitate this common programming concern. While many of the complexities of parallel programming have analogues in concurrent programming, the basic patterns and goals are different. Multicore processors will not alleviate the need for concurrent programming but rather will be a technique used to optimize the background activities and other computations performed in the system with the goal of improved performance.
Another familiar form of concurrency applies to server applications. Applications such as Web servers are presented streams of independent requests. These programs attempt to improve system throughput by executing many requests at the same time, typically using a separate thread for each request but then sequentially processing each request. This overlapping increases requests served per second but does not improve the latency (seconds per request) of individual requests.
These applications have also been enjoying a free lunch, one that will continue a bit longer as multi-core processing provides additional cost reduction and throughput benefits. However, any request that is sensitive to latency, even in a server environment, will eventually need to use parallel programming techniques to achieve acceptable performance.
Concurrent programming is notoriously difficult, even for experts. When logically independent requests share various resources (dictionaries, buffer pools, database connections, and so forth), the programmer must orchestrate the sharing, introducing new problems. These problems—data races, deadlocks, livelocks, and so forth—generally derive from a variety of uncertainties that arise when concurrent tasks attempt to manipulate the same data objects in a program. These problems make the basic software development tasks of testing and debugging extremely difficult, as recently outlined in "Tools and Techniques to Identify Concurrency Issues" by Rahul Patil and Boby George (MSDN Magazine, June 2008, msdn.microsoft.com/magazine/cc546569).
Parallel programming differs from concurrent programming in that you must take what is logically a single task—expressible using familiar sequential constructs supported by all major languages—and introduce opportunities for concurrent execution. (Later in this article, I will describe broad approaches to such opportunities.) However, when concurrency opportunities are introduced with subtasks that share data objects, you have to worry about locking and races. Thus, parallel programming has all of the correctness and security challenges of sequential programs plus all of the difficulties of parallelism and concurrent access to shared resources.
The combination of extra concepts, new failure modes, and testing difficulty should give every developer pause. Is this something you really want to bite off? Clearly, the answer is no! However, many will be forced into this swamp in order to deliver the necessary performance. Microsoft is actively developing solutions to some of the core problems, but high-productivity solution stacks are not yet available.
Last year, Microsoft announced its Parallel Computing Initiative (go.microsoft.com/fwlink/?LinkId=124050) to look at not only how to build and efficiently execute parallel programs but also to encourage creation of a new generation of applications that turn Moore's Dividend into customer value. I will soon examine some approaches to thinking about parallelism. Stephen Toub and Hazim Shafi’s article in this issue, “Improved Support For Parallelism In The Next Version Of Visual Studio,” describes some of the libraries and tools that we are providing to support these approaches. Joe Duffy’s article, “Solving 11 Likely Problems in Your Multithreaded Code”, also in this issue, discusses techniques and approaches to improving the safety of concurrent applications.
References

The Future Evolution of High-Performance Microprocessors, a lecture by Norman Jouppi, Stanford University
stanford.edu/class/ee380/Abstracts/060927.html

Thousand Core Chips: A Technology Perspective, a lecture by Shekhar Borkar
videos.dac.com/44th/papers/42_1.pdf
Previously, I used the phrase "opportunities for concurrent execution," which draws a further distinction between parallel and concurrent programming. When a developer uses asynchronous programming patterns or background workers, or handles concurrent requests in a server, there is an expectation that all the various threads make forward progress. The operating system scheduler will make sure every thread gets a fair share of resources. This, however, is not useful for parallel programming. It is particularly not valuable when you are interested in writing applications that will scale to new hardware systems.
If you want a return of the free lunch where software just gets better with hardware upgrades, then you need to provide the opportunity for more parallelism today in ways that can be used tomorrow. Here I will use the term "tasks" rather than "threads" to emphasize this shift in the implementation of parallelism and how I think about it: developers that tackle parallel programming will be encouraged to decompose problems into many more tasks than are needed today.
The implementation of parallel programming systems will need to deal with mapping those tasks onto system threads and processors on an as-needed basis. Visible to only a few programmers, there are deep changes in the way in which system resources are acquired from the operating system and managed within a process to execute parallel programs efficiently. This is like a thread pool on steroids, but one that is focused on matching opportunities for parallelism in an application to the available resources in the current hardware rather than simply managing threads as specified by the parallel programmer.
In concurrent programming, especially for servers, much of the difficulty arises from coordinating accesses to shared variables by long-running threads using tools such as locks. As we shift to parallel programming with tasks, a new concept can be used. I can talk about a task B running after a task A and provide coordination primitives to express this. This allows a programmer to think about the schedule of work. Typically this schedule will fit naturally into the algorithmic structure of the program and emerge from structured use of parallel programming abstractions. Good fits between program abstractions and parallel algorithms will greatly reduce the need for traditional concurrency mechanisms, such as locks and events, and avoid, but not eliminate, many of the risks of concurrent programming.
Next, I'll describe some major approaches to parallel programming and illustrate their use through abstractions that are under development. In particular, I will illustrate both the C++ Parallel Pattern Library (PPL) and the Parallel Extensions to .NET (available at go.microsoft.com/fwlink/?LinkId=124621) using C#.

Structured Multithreading
The problem-solving pattern humans use most often is divide and conquer: take a big problem, divide it into smaller problems with well-defined interactions that can be separately tackled, and then combine the results to solve the original problem. This technique is used to solve problems ranging from large corporate ventures to neighborhood potlucks. Not surprisingly, the recursive application of this idea is also a cornerstone of parallel programming.
Structured multithreading refers to providing parallel forms of key block-structured sequential statements. For example, a compound statement { A; B; } with sequential semantics where A is evaluated and then B is evaluated is made into a parallel statement by allowing A and B to be evaluated concurrently. The whole construct, however, does not complete, and control continues to the next construct until both subtasks have finished. This is an old concept and historically taught as a cobegin statement. It is sometimes referred to as "fork-join parallelism" to emphasize the structure. The same basic idea can be applied to loops where each iteration defines a task that may be evaluated concurrently with all the other iterations. Such a parallel loop completes when all iteration tasks complete.
A familiar example of the divide-and-conquer concept is the famous QuickSort algorithm. Here I'll illustrate a straightforward parallelization of this algorithm using C++ constructs. The basic algorithm takes an array of data, uses the first element as a key, partitions the data into two pieces, and updates the array so that all values less than the key occur before all values that are greater than the key. This step is then applied recursively to sort the data. Typically there is some cutoff where a non-recursive algorithm, such as insertion sort, is used to reduce the overhead near the leaves.
Figure 1 illustrates two features that are in development at Microsoft. The first feature demonstrated is the new C++ lambda syntax that makes it extremely convenient to capture an expression or statement list as a function object. The following syntax creates a function object that, when it's invoked, will evaluate the code between the braces:
[=] { ParQuickSort(data, mid); }
Figure 1 Quick Sort
// C++ using the Parallel Pattern Library
template<class T>
void ParQuickSort(T * data, int length, T* scratch) {
  if(length < Threshold)  InsertionSort(data,length)
  else {
    int mid = ParPartition(data[0], data,
                length, scratch, /*inplace*/true);
    parallel_invoke(
                [=] { ParQuickSort(data, mid); },
                [=] { ParQuickSort(data+mid, length-mid); });
  }
}
The leading [=] marks the lambda and indicates that any variables in outer scopes referenced in the expression should be copied into the object and that references to those variables in the body of the lambda will refer to those copies.
The parallel_invoke is a template algorithm that, in this case, takes two such function objects and evaluates each one as a separate task so that those tasks may run concurrently. When both tasks complete and, in this case, both of the recursive sorts have been completed, the parallel_invoke returns and the sort is then complete.
Note that parallelism in this example comes from the recursive application of divide-and-conquer parallelism. While at each level you only get two child tasks, toward the leaves of the computation you end up with a number of tasks proportional to the size of the data you are sorting. I have expressed all of the concurrency, and such a program is ready to scale to bigger problems or more cores. For a fixed-sized problem, there are always limits to the scalability on any real machine that has overheads. This is a consequence of the famous Amdahl's Law. A key engineering goal for platform vendors then is to drive down these overheads continually, which means that choosing values like Threshold (in Figure 1) can be hard to do and even harder to predict over time as systems change. They need to be big enough so that you don't pay excessive overhead but not so big as to limit future scaling.
This template algorithm is part of our PPL described in this issue by Stephen Toub and Hazim Shafi. In addition to parallel_invoke, there is also a parallel_for_each similar to the Standard Template Library's (STL) for_each. In the case of the parallel_for_each, the semantics are that every iteration is a separate task that may run concurrently with the other iteration tasks, and the parallel_for_each returns when they have all completed. There are also less structured techniques to create individual tasks that are associated with a common task group and then wait for them all to complete. This provides the same basic functionality as the Cilk spawn primitive (supertech.csail.mit.edu/cilk) but is built on standard C++ features.
A use of parallel loops might be code to perform a ray-tracing problem. Such a problem is trivially parallel over each output pixel and can be captured in code that might look like Figure 2. This is expressed using the Parallel.For method from the Parallel Extensions to .NET, which includes the same basic patterns for managed developers as are supported by the PPL for C++ developers. Here I use simple, nested loops to describe the space of tasks corresponding to each pixel on a rectangular screen. This code assumes that the various methods invoked in the body of a loop are safe for concurrent execution.
Figure 2 Parallel Loops
// C# using Parallel Extensions to the .NET Framework
public void RenderParallel(Scene scene, Int32[] rgb)
{
    Parallel.For(0, screenHeight, y =>
    {
        Parallel.For (0, screenWidth, x =>
        {
            Color color = TraceRay(new Ray(camera.Pos,
                GetPoint(x, y, scene.Camera)), scene, 0);
            rgb[x + y*screenWidth] = color.ToInt32();
        });
    });
}
Again, see Joe Duffy's article on concurrency safety. In these early days of parallel programming, developers will bear the burden of these concerns. Caveat emptor.
Structured multithreading is ideal for working with parallelism where there is a natural, perhaps recursive, possibly irregular, data structure where the parallelism reflects that structure. This is true even if there is some data flow across the problem. The following example traverses a graph in a topological order so that I don't visit a node before I have seen all predecessors. I maintain a count field on each node, which is initialized to the number of predecessors. After visiting a node, I decrement the counts of successors (careful to make this a safe operation since multiple predecessor tasks may attempt it at once). When I see a count go to zero, I can then visit that node:
// C++
void topsort(Graph * g, void (*action)(Node*)) {
    g->forall_nodes([=] (Node *n) {
        n->count = n->num_predecessors();
        n->root = (n->count == 0);
    });
    g->forall_nodes([=] (Node *n) {
        if(n->root) visit(n, action);
    });
}
I assume the graph exports a method that is parameterized by a function object and traverses all nodes in the graph in parallel and applies the function. That function will use the PPL internally to implement that concurrency. You then have two phases: the first counts predecessors and identifies root nodes; the second starts a depth-first search from each root that decrements and ultimately visits successors. That function looks like Figure 3.
Figure 3 Depth-First Search
// C++ using the Parallel Pattern Library
// Assumes all predecessors have been visited.
void visit(Node *n, void (*action)(Node*)) {
    (*action)(n);
    // assume n->successors is some kind of STL container
    parallel_for_each(n->successors.begin(),
         n->successors.end(),
         [=](Node *s) {
           if(atomic_decrement(s->count) == 0) // safely does "-- s->count"
            visit(s, action);
    });
}
The parallel_for_each method traverses the list of successors, applies a function object to each, and allows those operations to be done in parallel. Not shown is the assumed atomic_decrement function that uses some strategy for arbitrating concurrent accesses. Note that here you start to see more traditional concurrency concerns sneak into our otherwise parallel algorithm, and these concerns get worse as the elemental operations become more complex, like in the ray-tracing example.
The structure of this algorithm guarantees that "action" is given exclusive access to its parameter so no additional locking is needed if action updates those fields. Further, there are guarantees that all predecessors have been updated and are not changing, and that no successor has been updated and will not change until this action completes. The ability to reason about what state is stable and what state might be concurrently updated are key issues in building correct parallel algorithms.
The strength of the structured multithreading is that the parallel opportunities, including those over partially ordered computations, are convenient to express without the need to obfuscate the basic algorithm with a lot of mechanisms for mapping the work to worker threads. This provides a stronger compositional mode, as illustrated here, where a data structure can provide some basic parallel traversal methods (such as Graph::forall_nodes) that can then be reused to build more sophisticated parallel algorithms. Furthermore, it is convenient to describe all of the parallelism rather than trying to find just enough for two or four processors. Not only is this easier, it provides natural scaling opportunities for next year's machines that may have eight processors—the return of the free lunch.

Data Parallelism
Data parallelism refers to the application of some common operation over an aggregate of data either to produce a new data aggregate or to reduce the aggregate to a scalar value. The parallelism comes from doing the same logical operation to each element independent of the surrounding elements. There have been many languages with various levels of support for aggregate operations, but by far the most successful has been the one used with databases—SQL. LINQ provides direct support in both C# and Visual Basic for SQL-style operators, and the queries expressed with LINQ can be handed off to a data provider, such as ADO.NET, or can be evaluated against in-memory collections of objects or even XML documents.
Part of Parallel Extensions to .NET is an implementation of LINQ to Objects and LINQ to XML that includes parallel evaluation of the query. This implementation is called PLINQ and can be used to work conveniently with data aggregates. The following example illustrates the kernel of the standard K-means algorithm for statistical clustering: at each step you have K points in space that are your candidate cluster centers. Map every point to the nearest cluster and then, for all points mapped to the same cluster, recompute the center of that cluster by averaging the location of points in the cluster. This continues until a convergence condition is met where the positions of the cluster centers become stable. The central loop of this algorithm description is translated fairly directly into PLINQ, as shown in Figure 4.
Figure 4 Find Center
// C# using PLINQ
var q = from p in points.AsParallel()
        let center = nearestCenter(p, clusters)
                     // "index" of nearest cluster to p
        group p by center into g
        select new
        {
             index = g.Key,
             count = g.Count(),
             position = g.Aggregate(new Vector(0, 0, 0),
                    (accumulated, element) => accumulated + element,
                    (accumulated1, accumulated2) =>
                                     accumulated1 + accumulated2,
                    (accumulated) => accumulated
                    ) / g.Count()
        };
var newclusters = q.ToList();
The difference between LINQ and PLINQ is the AsParallel method on the data-collection points. This example also illustrates that LINQ includes the core map-reduce pattern but with clean integration into mainstream languages. One subtle point in this example is the behavior of the Aggregate operator. The third parameter is a delegate that provides a mechanism to combine partial sums. When this method is provided, the implementation is done in a parallel style by blocking the input into chunks, reducing each chunk in parallel, and then combining the partial results.
The great strength of data parallelism is that suitable algorithms are typically expressed much more cleanly and readably than if I had applied a structured multithreading approach where data structure assumptions can become entangled. Furthermore, the more abstract description allows the system greater opportunity to optimize in ways that would completely obscure the algorithm if done by hand. Finally, this high-level representation allows greater flexibility with the execution target: multicore CPU, GPU, or scale out to cluster. And as long as the leaf functions, for example, nearestCenter has no side effects—you don't face any of the concerns of data races or deadlocks of thread-oriented programming.

A common technique for exploiting parallelism is through the use of pipelining. Using this model, a data item flows between various stages of the pipeline where it is examined and transformed before being passed on to the next stage. Data flow is the generalization of the idea where data values flow between nodes in a graph, and computation is triggered based on the availability of input data. Parallelism is exploited both by having distinct nodes executing concurrently and by having one node activated multiple times on different input data.
Parallel Extensions to .NET supports the ability to explicitly create individual tasks (of type Task, the underlying mechanism for implementing structured multitasking) and then to identify a second task that begins execution when the first completes. The concept of a future is used as a bridge between the worlds of imperative programming and data-flow programming. A future is the name of a value that will eventually be produced by a computation. This separation allows me to define what to do with a value before I know that value.
The continueWith method on a future is parameterized by a delegate that will be used to create a task that will be executed when the future value is available. The result of a call to continueWith is a new future that identifies the result of the delegate parameter. Often in imperative programming, tasks are evaluated for side effects, so continueWith is also available as a method on Task.
As an example of this style, consider the parallelism within the Strassen matrix multiplication algorithm. This is a block-oriented version of the basic matrix multiplication algorithm. The two input matrices are each divided into four sub-blocks that are then algebraically combined to form the sub-blocks of the output matrix. (For details, see the Wikipedia article on the Strassen algorithm at wikipedia.org/wiki/Strassen_algorithm.)
One of these tasks might look like this:
// C# using Parallel Extensions to the .NET Framework
var m1 = Future.StartNew(() => (A(1,1)+B(1,1))*(A(2,2)+B(2,2));
For clarity I have used mathematical notation inside the delegate rather than code to express the computation. A and B are the input matrices and A(1,1) is the upper-left subblock of the A matrix. Addition is standard for matrices, and multiplication is a recursive application of the Strassen algorithm. The output is a temporary matrix corresponding to the results of this expression.
The first seven of the subtasks are independent, but the last four depend on the first seven as inputs. The basic data-flow graph would look like Figure 5. Here, the task labeled c11 depends on the results of tasks m2 and m3. I want that task to become eligible for execution when its inputs are available. In C# this can be expressed as:
var c11 = Task.ContinueWhenAll(delegate { ... },  m2, m3);
Figure 5 Subtasks
This illustrates what might be called medium-grain data flow where the units of computation are a few hundred to a few thousand operations in contrast with fine-grain data flow where each operation might be a single arithmetic operation. A few convenience methods to handle the "when all" and "when any" concepts make this more readable, but they can be implemented on top of the basic mechanisms as discussed by Stephen Toub (see blogs.msdn.com/pfxteam/archive/2008/07/23/8768673.aspx).
As with structured multithreading, I have identified opportunities for parallelism, and if you have only a single processor, then it is fine to serially execute the subtasks in the order in which they are created. But with additional resources, you have the chance to benefit from them by having captured the execution order constraints among the subtasks. Unlike structured multithreading, it is convenient to allow one task basically to enumerate the data-flow graph and keep the individual subtasks ignorant of how their results are used and combined—a different surface to a common set of mechanisms to handling parallel algorithms.
Frequently, data is streamed into an application and I wish to process it as it flows by. In this case, the paths that the data traverse are fairly stable. And rather than associate a task with the completion of other tasks, I want to associate a task with the availability of the next data item. One domain where this is a particularly valuable model is in robotics where algorithmic decision making sits at the confluence of various streams of sensor data that may arrive at different rates.
The Microsoft Robotics SDK (go.microsoft.com/fwlink/?LinkId=124622) takes this approach where the central concepts involve streams of data (called ports) and the binding of tasks that, among other ways, are activated on the arrival of data (messages). Of course, I have now circled around to a class of problems that are not motivated by the shift to multicore but rather, like Web servers, have concurrency as core characteristics that must be addressed as part of the overall architecture of the application. Similar concerns apply to distributed applications outside of the robotics domain, but these are beyond the scope of this discussion.

Streaming Parallelism
Beyond multiple cores, a second important feature of computer architecture is the multiple layers of memory hierarchy: registers, one or more levels of on-chip cache, DRAM memory, and, finally, demand paging to disk. Most programmers are blissfully unfamiliar (or at least unconcerned) with this aspect of system architecture because their programs are modest in size and fit well enough in the cache to which most references to memory are quickly returned. However, if a data value is not in the on-chip cache, it can take hundreds of cycles to fetch it from DRAM. The latency of providing this data makes the program appear to run slower since the processor spends a large fraction of the time waiting for data.
Some processor architectures support multiple logical processors per physical processing core. This is usually called (hardware) multithreading, and modest amounts of this have been used in mainstream processors (for example, Intel has called this hyper-threading in some of its products). A motivation for multithreading is to tolerate the latency of memory access; when one logical hardware thread is waiting on memory, instructions can be issued from the other hardware threads. This technique is also used on modern GPUs, but more aggressively.
As the number of processing cores grows, the number of requests it can make on a memory system increases and a different problem emerges, one of bandwidth limitations. A processor will be able to support only so many transfers per second to or from DRAM memory. When this limit is reached, there is no chance to get any gains through further use of parallelism; additional threads will just generate additional memory requests that will simply queue up behind earlier requests and wait to be serviced by memory controllers. GPUs typically have memory subsystems that support higher aggregate bandwidth (measured in gigabytes per second) because they support (and expect to have) a lot more parallelism and can benefit from the additional bandwidth.
Current multicore chip architectures are able to increase the number of cores faster than memory bandwidth, so for most problems where the data set does not fit in memory, using the memory hierarchy is an important concern. This imbalance gives rise to a style of programming called stream processing where the focus is to stage blocks of data into the on-chip cache (or perhaps private memory) and then perform as many operations against that data as possible before displacing it with the next block. Those operations may be internally parallel to use the multiple cores or they may be pipelined in a data flow style, but the key issue is to do as much work on the data in the cache while it is there.
While there have been proposals for special purpose languages to allow streaming algorithms to be specified and their execution carefully planned, it is also possible to achieve this in many cases by careful scheduling. For instance, you can apply this technique to the QuickSort example. If the size of the data set you are sorting is so large that it does not fit in the cache, the straightforward work-stealing approach will tend to schedule the largest and coarsest subproblems onto different cores, which then work on independent data sets and lose the benefit of a shared on-chip cache.
If, however, you modify the algorithm to use only parallelism on data sets that fit in the cache, you gain the benefits of streaming, as Figure 6 illustrates. In this example, you still break large problems into smaller problems (and use parallelism in the partitioning step), but if both sub-problems won't fit in the cache at the same time, then we'll do them sequentially. This means that once you get a small data set in the cache, you finish sorting it completely using all available resources while it is in the cache without it being pushed out to memory before you start the next data set.
Figure 6 Use Parallelism on Smaller Data Sets
// C++ using the Parallel Pattern Library
template<class T>
void ParQuickSort(T * data, int length, T* scratch, int cache_size) {
    if(length < Threshold)  InsertionSort(data,length)
    else {
           int mid = ParPartition(data[0], data,
                length, scratch, /*inplace*/true);
           if(sizeof(*data)*length < cache_size)
                parallel_invoke(
                  [=]{ ParQuickSort(data, mid, cache_size); },
                  [=]{ ParQuickSort(data+mid, length-mid, cache_size);});
           else {
                  ParQuickSort(data, mid, cache_size);
                  ParQuickSort(data+mid, length-mid, cache_size);
           }
    }
}
Note that you have had to explicitly parameterize this code with the size of the cache, which is an implementation-specific feature. This is unfortunate and is also a problem if this computation is sharing the system with other jobs or other tasks within the same job. But it illustrates one of the difficulties in getting performance out of complex parallel systems. Not every problem will have this issue, but good performance on multicore chips requires developers to think carefully about the interaction of parallelism and the memory hierarchy in some cases.

Single-Program, Multiple-Data
In the high-performance computing arena, parallelism has been used in technical and scientific applications for some time, based on a lot of work done in the 1980s. The kinds of problems are dominated by parallel loops over arrays of data where the bodies of the loops typically have a fairly simple code structure.
The earlier ray-tracing fragment is an example. The dominant parallelism model that emerged was called single-program, multiple-data, frequently abbreviated as SPMD. This term alludes to the classic Michael Flynn taxonomy of computer architecture that includes single-instruction, multiple-data (SIMD) and multiple-instruction, multiple-data (MIMD). In this model, programmers think about the behavior of each processor (worker, thread) as a set of processors that logically participate in a single problem but share the work. Typically the work is separate iterations of a loop that work with different parts of arrays.
The notion of work sharing in an SPMD style is at the heart of the OpenMP (openmp.org) set of extensions to C, C++, and Fortran. The core concept here is a parallel region where a single thread of activity forks into a team of threads that then cooperatively execute shared loops. A barrier synchronization mechanism is used to coordinate this team so that the entire team moves as a group from one loop nest to the next to insure that data values are not read before they have been computed by teammates. At the end of the region, the team comes back together, and the single original thread continues on until the next parallel region.
This style of parallel programming can also be used in non-shared memory systems where inter-node communication built on top of message passing is used to copy data around to where it is needed at phase boundaries. This technique is used to apply thousands of processor nodes to achieve the extreme performance that is needed for problems such as climate modeling and pharmaceutical design.
As in structured multithreading, an OpenMP parallel region may be inside a function, and so the caller of that function need not be aware of the use of parallelism in the implementation. However, the SPMD model requires careful attention to how the work of the problem is mapped down onto the workers in a team.
A load imbalance where one worker has a longer task than its teammates implies those teammates may end up waiting, twiddling their silicon thumbs for the one worker to complete. Similarly, in an environment where system resources may be shared with other jobs, if a worker is interrupted by the OS to perform some other task, the rest of the workers will be impacted by this artificial imbalance.
Future hardware systems may have different kinds of processor cores—some big cores that use a lot of power but are fast for a single thread mixed with little cores that use less power and are optimized for parallel operations. Such an environment is very difficult for an SPMD model because it makes the strategy for mapping work to workers very complex. These are problems that are nicely avoided by the structured multithreading approach but at the cost of small increases in scheduling overhead and possibly loss of control over the memory hierarchy for code where OpenMP is well suited.

Concurrent Data Structures
The previous discussion has focused almost entirely on control parallelism—how to identify and describe separate tasks that can be mapped down to the multiple cores that may be available. There is also a data side related to parallelism. If the effect of a task is to update a data structure, say insert a value into a Hashtable, that operation may be logically independent of concurrently executing tasks.
Standard implementations of data structures do not support concurrent access and may break in a variety of ways that are cryptic, unpredictable, and difficult to reproduce. Simply putting a single lock around the whole data structure may create a bottleneck in the program where all of the tasks serialize, resulting in a loss of parallelism because too few data locations are concurrently in use.
It is thus important to create, in addition to the parallel control abstractions, new concurrent versions of common data structures—Hashtables, stacks, queues, various kinds of set representations. These versions have defined semantics for the supported methods that may be invoked concurrently, and they are engineered to avoid bottlenecks when accessed by multiple tasks.
As an example, consider a standard approach to finding the connected components of a graph in parallel. The basic strategy is to start a number of depth-first traversals of the graph, identify nodes where they collide, and form a reduced graph. The top-level function would have the basic structure illustrated in Figure 7.
Figure 7 Depth-First Traversals
// C++
// assign to each Node::component a representative node in
// the connected component containing that node
void components(Graph * g) {
  g->forall_nodes([=] (Node * n) {
    n->component = UNASSIGNED;
  });
  Roots roots;
  EdgeTable edges;
  g->forall_nodes([&roots, &edges] (Node * n) {
    if(atomic_claim(n, n)) {
      roots.add(n);
      component_search(n,n, &edges);
    }
  });
  // recusively combine reduced graph (roots, edges)
       ...
}
We will represent the components implicitly by having each node point to a representative element. The input is a graph, and I assume that graph supports a forall_nodes method that can be used to traverse the graph using structured multithreading techniques. It is parameterized by a function object to be applied to each node. This interface insulates the parallel algorithm from the structural details of the graph but retains the key property of structured multithreading, that the concurrency is internal to method and can be highly structured.
First, initialize the component fields to a distinguished value, and then start parallel depth-first searches, logically from every node. Each search begins by claiming the start node. Since the node might be reached by some other search, this claim is basically an atomic test to determine which search reaches the node first. That function might look like this:
// C++
// atomically set n->commponent to component if it is UNASSIGNED
// return true if and only this transition was made.
bool atomic_claim(Node * n, Node * component) {
    n->lock();
    Node * c = n->component;
    if(c == UNASSINGED) n->component = n;
    n->unlock();
    return c == UNASSIGNED;
}
I have assumed a lock per node, but in this simple example I might have used the Windows primitive to perform an interlocked compare and exchange on pointer values. The key issue is that a single global lock to protect the concurrent access from multiple tasks would foil our attempt at parallelism by eliminating the data parallelism even if I have lots of control parallelism.
You don't specifically need one lock per node. Instead, you could map the nodes down to a smaller set of locks, but you must be wary of introducing a scaling bottleneck by using a specific number related to today's processor counts. Rather than using explicit locks at all, research into the use of "transactional memory" by Jim Larus aims to allow developers to simply declare code intervals that should be isolated from concurrently executed code where the implementation introduces locking as needed in order to guarantee these semantics.
Once you have identified a new root component, add it to the shared data structure roots. This is logically a set of all the root components found in this first step of the algorithm. Again, while there is logically only one instance of this container, you need its implementation to be optimized for concurrent additions. As part of the PPL and Parallel Extensions to .NET, Microsoft will provide suitable implementations of vectors, queues, and Hashtables that can be used as building blocks.
The depth-first search from a node iterates over adjacent nodes and attempts to claim each into its component. If this is successful, it recurses into that node (see Figure 8).
Figure 8 Recursive Search
// C++ using the Parallel Pattern Library
// a depth first search from "n" through currently
// unassigned nodes which are
// set to refer to "component". Record inter-component edges in "edges"
void component_search(Node * n, Node * component, EdgeTable * edges) {

  parallel_for_each(n->adjacents.begin(),
    n->adjacents.end(),
    [=] (Node * adj) {
    if(atomic_claim(adj, component)
      component_search(adj, component, edges);
    else if(adj->component != component) {
      edges->insert(adj->component, component);
    }
  });
}
If the claim fails, meaning that this or some other task reached this node first, you should look to see if it was added to a different component. If so, record the two components in the shared data structure, EdgeTable. This table might be created using a concurrent Hashtable to avoid duplicate information. Again, this is a logically shared structure where you must insure you have adequate support for concurrent access in order to avoid contention and loss of effective parallelism.
The two structures, roots, and edges form a logical graph that records connectivity between the initial component estimates. To complete the algorithm, find connected components on this logical graph and then update that node level information with final representatives (not shown).

Wrap-Up
A loss of data concurrency is, of course, just one of many performance surprises associated with parallel programming. You should not be surprised if your first attempts at this new problem either break mysteriously (because you forgot to use locks, for example) or actually slow down compared to sequential execution (tasks too small, too much locking, loss of cache effectiveness, not enough memory bandwidth, data contention, and so on).
As Microsoft continues to invest in parallel programming, abstractions will steadily improve as well as tools to help diagnose and avoid the associated problems. Indeed, expect the hardware itself to improve and to incorporate features to reduce various costs, but that will take time.
The shift to parallelism is an inflection point for the software industry where new techniques must be adopted. Developers must embrace parallelism for those portions of their applications that are time sensitive today or that are expected to run on larger data sets tomorrow. I have described different ways to think about and use parallelism in applications, and I have illustrated them with new tools being developed at Microsoft as part of our broad Parallel Computing Initiative.

Insights: Toward the Next 100X
Since 1975, microprocessor performance has improved 100X/decade, powered by exponential growth in clock frequencies (3,000X) and transistor counts (300,000X). Instruction "potency" has grown 8X-100X—compare an 8-bit ADD to an SSE4.1 DPPS "dot-product of packed-4-SP-float-vectors"—and on-die caches are now as large as early hard disks. As an industry, we have spent each 100X to deliver wave after wave of compelling new experiences. It's the wind in our sails.
However, we will have to sail a different tack deliver the next 100X. We can still look forward to the Moore's Law Dividend of four more doublings in transistors per die (the 32, 22, 16, 11 nm nodes), at a two-year cadence. But we now find ourselves in the asymptotic diminishing returns corners of several growth curves, notably voltage scaling and power dissipation (power wall), instruction-level parallelism (complexity wall), and memory latency (memory wall).
The Power Wall The dynamic power of a microprocessor is proportional to NCV2f, the number of transistors switching × switching capacitance × voltage squared × frequency. With each lithography node shrink, transistors/die potentially doubles (↑N) and transistors get smaller (↓C) and use lower voltage switching (↓V). Supply voltage has fallen from 15V down to 1V today, reducing switching energy by more than 100X. Unfortunately, the CMOS minimum threshold voltage is 0.7 V, so we have, at most, an additional (1.0/0.7)2=2X savings to come. Despite these savings, as we have increased the complexity (↑N) and frequency (↑f) of microprocessors, die power dissipation has grown from 1 W to 100 W in a few square centimeters of silicon, now at the limit of practical cooling solutions. There is little headroom left; power is a zero-sum game going forward. Clock frequencies will never again scale up as fast as they used to. We won't find our next 100X here.
The Complexity Wall High-performance microprocessors use heroic out-of-order execution to exploit instruction-level parallelism within a thread. But there are practical limits to further big gains from this approach. The serial code itself and its data dependences limit the parallelism that can be mined. In the hardware you may need up to N2 more circuitry sometimes to complete N operations in parallel. The up-front design and verification costs grow proportionally. Most important, assuming parallel software arrives, the power wall dictates that more energy-efficient microarchitectures are better architectures—results/joule trumps results/nanosecond.
The Memory Wall DRAM access latency (delay) improves relatively slowly; therefore CPUs use caches to avoid DRAM entirely. But caches are an expensive proposition—today a full-cache miss can cost 300 clock cycles. A rule of thumb is you halve the miss rate by quadrupling the cache size. And much of the complexity in a CPU core is dedicated to tolerating long and unpredictable memory accesses. However, it is easier to scale memory bandwidth. Then you can apply memory-level parallelism—parallel software threads issuing many concurrent memory accesses simultaneously. Each thread may still wait a long time for its access, but the overall throughput of the parallel computation is very high.
Serial processor performance will continue to improve but much more slowly in the next decade. Clever CPU designers and compiler writers will still find ways to squeeze out 5% or 10% here and there. This is a good thing because a lot of valuable software is serial and governed by Amdahl's Law. But this will not deliver the next 100X we need. That will require software that exploits parallelism. If and when sufficiently compelling software appears, processor vendors stand ready to deliver parallel processors with dozens of cores and high bandwidth memory solutions. If we come, they will build it.
—Jan Gray, Software Architect, Microsoft Parallel Computing Platform Team

David Callahan is the Distinguished Engineer and lead architect of the Parallel Computing Platform Team within Visual Studio. He came to Microsoft from supercomputer manufacturer Cray Inc. with a background in parallelizing compilers and parallel algorithms, architectures, and languages.


Coding Tools
Improved Support For Parallelism In The Next Version Of Visual Studio
Stephen Toub and Hazim Shafi

This article is based on prerelease versions of Visual C++, Visual Studio, and the .NET Framework. All information herein is subject to change.

This article discusses:
  • Expressing parallelism in managed code
  • Expressing parallelism in native code
  • Debugging parallel applications
  • Profiling parallel applications
This article uses the following technologies:
Visual Studio, Multithreading
Contents
Expressing Parallelism in Managed Code
Expressing Parallelism in Native Code
Debugging Parallel Applications
Profiling Parallel Applications
Looking Forward
It wasn't too long ago that many software performance bottlenecks could be fixed simply by a trip to the local computer store. Consumers and companies with the means and motivation to buy new computers every 18 to 24 months would find with every new purchase their compute-intensive applications running twice as fast as before.
Alas, those good old days would appear to be behind us. Gone are the days of exponential increases in CPU speed, largely due to pesky laws of physics that are preventing hardware manufacturers from economically and ecologically scaling clock speeds at rates anywhere near to what we're accustomed to—and to what modern advances in software demand. Instead, the influx of transistors as outlined by Moore's law is enabling chip manufacturers to exponentially scale the number of cores available in commodity hardware. Today, it's rare to find single-core machines on sale in the United States, with dual-core being the norm. Next year, expect the typical consumer machine on sale to be a quad-core, with eight-core as the norm to follow shortly thereafter.
Unfortunately, the majority of the software today will not automatically take advantage of such exponential growth in processor count. To do so, software must be written in a manner that supports safe and effective decomposition of its constituent components such that these individual parts can be computed in parallel and distributed across all of the cores, thus realizing the potential that multicore and manycore systems promise.
Such expression of parallelism with today's preeminent programming languages, frameworks, and development environments is no cakewalk. Multithreaded programming, for anything but the most trivial of systems, is incredibly difficult and error prone today. Parallel patterns are not prevalent, well-known, or easy to implement. Even when an application can be constructed to take advantage of concurrency, functional correctness and performance problems frequently arise, from the most well-known race conditions and deadlocks to the more obscure cache coherency overheads, lock convoys, and priority inversion.
Moreover, even if developers were able to conquer such patterns, the time and effort involved in doing so places a burden on businesses, where the best (and the best-compensated) developers are expected to focus on business problems that affect the bottom line rather than on concurrency problems that are a mere cost of staying competitive in a performance-centric industry. Compounding these problems significantly, the tools available for developers to debug and analyze their parallel applications are rudimentary at best.
As a result, the big players in software development are taking strides to make parallelism much easier for all developers to employ, and Microsoft is helping to lead the way. Plans for the next version of Visual Studio include a solid foundation for parallelism, encompassing significant improvements for native and managed code alike, both in the core platform and in tooling support.

Expressing Parallelism in Managed Code
Consider the following problem statement: given a list of baby birth information, search for babies of a given name, born in a given state, in a specific range of years, and return the results. Here is what a typical solution in C# might look like:
IEnumerable<BabyInfo> Search(
    IEnumerable<BabyInfo> babies, QueryInfo qi) {
  foreach (var baby in babies) {
    if (baby.Name == qi.Name && baby.State == qi.State &&
        baby.Year >= qi.YearStart && baby.Year <= qi.YearEnd) {
      yield return baby;
    }
  }
}
This is largely an "embarrassingly parallel" problem. Each individual baby can be analyzed independently, in theory making it easy to parallelize their processing by partitioning the work across all available processors. Unfortunately, that which is theoretical doesn't always translate into practical.
Figure 1 shows one attempt at parallelizing this solution. As you can tell from the length of the code figure, the task is not as easy with today's Microsoft .NET Framework as the parallelization-friendly description of the problem would suggest. Even with all of this code, the solution is far from ideal. Manual partitioning is employed to distribute the processing to multiple cores. Detailed knowledge of synchronization primitives is necessary to understand how to coordinate between threads. The overheads of locking will likely dominate the implementation due to the contention on the enumerator and the ratio of work done inside versus outside of the critical regions. The list goes on.
Figure 1 Naive Manual Parallelization
IEnumerable<BabyInfo> Search(
    IEnumerable<BabyInfo> babies, QueryInfo qi) {
  var results = new List<BabyInfo>();
  int partitionsCount = Environment.ProcessorCount;
  int remainingCount = partitionsCount;
  var enumerator = babies.GetEnumerator();
  try {
    using (ManualResetEvent done = new ManualResetEvent(false)) {
      for (int i = 0; i < partitionsCount; i++) {
        ThreadPool.QueueUserWorkItem(delegate {
          while(true) {
            BabyInfo baby;
            lock (enumerator) {
              if (!enumerator.MoveNext()) break;
              baby = enumerator.Current;
            }
            if (baby.Name == qi.Name &&
                baby.State == qi.State &&
                baby.Year >= qi.YearStart &&
                baby.Year <= qi.YearEnd) {
              lock(results) results.Add(baby);
            }
          }
          if (Interlocked.Decrement(ref remainingCount) == 0)
            done.Set();
        });
      }
      done.WaitOne();
    }
  }
  finally {
    IDisposable d = enumerator as IDisposable;
    if (d != null) d.Dispose();
  }
  return results;
}
What's even more disheartening when looking at this solution is seeing the ratio of parallel boilerplate code to the code actually implementing the method's business logic. It would be much more useful if all of that boilerplate could be handled by the system, such that the developer could focus on the problem statement on hand:
IEnumerable<BabyInfo> Search(
    IEnumerable<BabyInfo> babies, QueryInfo qi) {
  var results = new ConcurrentStack<BabyInfo>();
  Parallel.ForEach(babies, baby => {
    if (baby.Name == qi.Name && baby.State == qi.State &&
        baby.Year >= qi.YearStart && baby.Year <= qi.YearEnd) {
      results.Push(baby);
    }
  });
  return results;
}
Astute readers may notice that the problem statement also lends itself quite nicely to an implementation using LINQ-to-Objects. Imagine if we augmented the original problem statement such that we also want the baby information back in order sorted by year. That additional requirement would add a quite a bit of code to the previous manual implementation, especially if we wanted the sort done in parallel. With LINQ, that solution can be coded as shown at the beginning of Figure 2. Now imagine that the LINQ query could be parallelized automatically, simply by appending AsParallel to the babies enumerable, as shown at the bottom of Figure 2.
Figure 2 Parallelization with PLINQ
Sequential
IEnumerable<BabyInfo> Search(
    IEnumerable<BabyInfo> babies, QueryInfo qi) {
  return from baby in babies
         where baby.Name == qi.Name && baby.State == qi.State &&
               baby.Year >= qi.YearStart && baby.Year <= qi.YearEnd
         orderby baby.Year ascending
         select baby;
}
Parallel
IEnumerable<BabyInfo> Search(
    IEnumerable<BabyInfo> babies, QueryInfo qi) {
  return from baby in babies.AsParallel()
         where baby.Name == qi.Name && baby.State == qi.State &&
               baby.Year >= qi.YearStart && baby.Year <= qi.YearEnd
         orderby baby.Year ascending
         select baby;
}
Luckily, these constructs are not imaginary at all. Some readers may recognize Parallel.ForEach as being a part of the Task Parallel Library (TPL), which has been available publicly as part of a Community Technology Preview (CTP) for Parallel Extensions to the .NET Framework since December 2007. Others may recognize AsParallel as being a part of Parallel LINQ (PLINQ), also available through the Parallel Extensions CTPs.
As of the next version of Visual Studio, these will become part of the core .NET Framework, available for all developers using .NET to consume from any .NET-compliant language: Parallel.ForEach as part of mscorlib.dll and AsParallel as part of System.Core.dll right alongside the sequential LINQ implementation.
TPL, exposed through the System.Threading and System.Threading.Tasks namespaces, has changed quite a bit since it was introduced to the world in the October 2007 issue of MSDN Magazine (see msdn.microsoft.com/magazine/cc163340), but its core principles remain the same. It delivers replacements for high-level constructs used in sequential applications today, such as Parallel.For to parallelize for loops, Parallel.ForEach as shown in a previous example to parallelize foreach loops, and Parallel.Invoke to parallelize a set of statements. These methods all accept delegates representing the code to execute in parallel, which means they work especially well with languages that support anonymous methods and lambda expressions.
For example, the code in Figure 3 is an excerpt from a ray tracer written in F# by Luke Hoban (Program Manager for F# at Microsoft) that utilizes the Parallel.For method to parallelize the rendering of a scene. You should note that the main difference between the two implementations falls to a single line. Here's that line from the original code:
for y = 0 to screenHeight - 1 do
Figure 3 Using Parallel.For from F#
Sequential
member this.Render(scene, rgb : int[]) =
  for y = 0 to  screenHeight - 1 do
    let stride = y * screenWidth
    for x = 0 to screenWidth - 1 do
      let color = TraceRay(
        {Start = scene.Camera.Pos;
         Dir = GetPoint x y scene.Camera },
        scene, 0)
      let intColor = color.ToInt ()
      rgb.[x + stride] <- intColor
Parallel
member this.Render(scene, rgb : int[]) =
  Parallel.For(0, screenHeight, fun y ->
    let stride = y * screenWidth
    for x = 0 to screenWidth - 1 do
      let color = TraceRay(
        {Start = scene.Camera.Pos;
         Dir = GetPoint x y scene.Camera },
        scene, 0)
      let intColor = color.ToInt ()
      rgb.[x + stride] <- intColor)
And here's the parallel version:
Parallel.For(0, screenHeight, fun y ->
The partitioning and execution of the iterations that make up the scene rendering is then handled automatically by TPL.
As another example, the following sequential code executes a delegate for each node in a tree:
static void WalkTree<T>(Tree<T> tree, Action<T> handler) {
  if (tree == null) return;
  handler(tree.Data);
  WalkTree(tree.Left, handler);
  WalkTree(tree.Right, handler);
}
We can use Parallel.Invoke to easily parallelize the execution of this tree walk:
static void WalkTree<T>(Tree<T> tree, Action<T> handler) {
  if (tree == null) return;
  Parallel.Invoke(
    () => handler(tree.Data),
    () => WalkTree(tree.Left, handler),
    () => WalkTree(tree.Right, handler));
}
These higher-level constructs in TPL are built on top of a lower-level set of constructs that form the foundation for task-based parallelism. This includes a Task class that represents an individual work item that can be scheduled for execution, a Future<T> class that represents the promise of a computed value at some point in the future, a TaskManager class that represents a scheduler used to execute tasks and computational futures, and a handful of supporting types for effectively constructing and managing tasks. With these solutions, incredibly complex parallel systems can be built with much less effort than is required today.
They can also be executed more efficiently on more powerful hardware, thanks to a new concurrency scheduler built into mscorlib. This scheduler is represented publicly through the aforementioned TaskManager class, which enables individual schedulers to be constructed that give isolated tasks discrete policies for how they should be executed. A shared resource manager coordinates amongst scheduler instances to ensure that the processing resources of the system are utilized effectively in a manner that limits oversubscription as much as possible. The scheduler is designed with optimizations for solutions that typically occur when using fine-grained parallelism, such as enabling recursive algorithms to take better advantage of the caches in the machine.
It's important to note that the .NET Framework has supported multithreaded development since its inception. The System.Threading namespace provides a plethora of low-level synchronization primitives, such as monitors, mutexes, semaphores, events, and thread-local storage, many of them since version 1.0. Languages like C# and Visual Basic expose support through constructs such as their lock and SyncLock keywords, respectively. However, authoring parallel systems effectively with libraries like TPL often requires higher-level coordination and synchronization types.
As a result, the next version of the .NET Framework will also see the System.Threading namespace rounded out with additional constructs that make multithreaded applications easier to write. As an example, lazy initialization is a common technique used in applications to load and initialize data only when it's first needed. This is typically done either to delay the processing to a point where the processing time will be less noticeable (such as to offload some work that might otherwise be done during start-up in order to improve start-up time), or to completely eliminate the processing if the lazily initialized data is used infrequently enough that some runs of the application may not need it at all. Either way, to implement this technique in a multithreaded application can be error-prone and repetitive to code manually, as it frequently involves multiple threads racing to initialize the data.
To solve this, the .NET Framework will include the LazyInit<T> type, which makes the most common patterns of thread-safe lazy initialization a breeze. Accessing the Value property of the LazyInit<T> forces the initialization if it hasn't already occurred, and returns the initialized data; the application will still maintain its correctness even if multiple threads race to retrieve the Value.
Here we have Visual Basic code that takes advantage of a LazyInit<T> in several ways:
Private _d1 As LazyInit(Of MyData)

Private _d2 As LazyInit(Of MyData) = New LazyInit(Of MyData)( _
  Function() MyData.Load())

Private Shared _d3 As LazyInit(Of MyData) = New LazyInit(Of MyData)( _
  Function() MyData.Load(), LazyInitMode.EnsureSingleExecution)

Private _d4 As LazyInit(Of MyData) = New LazyInit(Of MyData)( _
  Function() MyData.Load(), LazyInitMode.ThreadLocal)
The _d1 variable is an instance member to be lazily-initialized. This example assumes that the MyData type has a public, parameterless constructor, allowing LazyInit<T> to construct an instance of MyData using Activator.CreateInstance.
The _d2 variable is also an instance member. However, this example uses a developer-provided lambda expression as the code for initializing the variable when it's accessed.
Like _d2, the _d3 variable is also initialized with a custom delegate. However, this time the variable is static (Shared in Visual Basic). The developer has also opted to override the default initialization mode to ensure that the provided delegate is only executed once, even if there's a race with multiple threads accessing the Value property. (The default mode allows the delegate to be executed multiple times, but ensures that only one result is ever published publicly through Value.)
Finally, the _d4 variable uses the ThreadLocal mode. For those readers familiar with the ThreadStaticAttribute, note that LazyInitMode.ThreadLocal is similar in that it ensures that each accessing thread gets its own copy of the data. However, while ThreadStatic is likely more efficient due to optimizations for it in the runtime, LazyInit<T> trumps ThreadStaticAttribute in capabilities. For one, ThreadStaticAttribute can only be used with static variables, whereas LazyInit<T> can have thread-locals as instance members.
Additionally, the ThreadStaticAttribute can bite a lot of developers when trying to use it. Consider the following code:
[ThreadStatic]
private static MyData _d5 = new MyData();
If no other code in the application sets the _d5 variable, would you be surprised to access _d5 and find its value null? That situation is perfectly possible. The issue here is that the C# compiler hoists the initialization of _d5 into the containing type's static constructor. The static constructor is only executed just before the type is needed (a great example of lazy initialization), and it's only executed once. This means the first thread to access the type will find _d5 initialized, but other threads will see the value as null. LazyInit<T> addresses this situation with its ThreadLocal mode to ensure that all threads accessing the LazyInit<T>'s Value property will find it properly initialized.
This is just one example of the new threading constructs that will be available in the next version of the .NET Framework. Others include CountdownEvent, Barrier, and SpinWait.
Additionally, the new support also includes a key set of thread-safe collections that aid in the manipulation of shared data. These collections appear in the System.Collections.Concurrent namespace (note that previous CTPs had them in the System.Threading.Collections namespace), and address a variety of situations. Types like ConcurrentQueue<T> and ConcurrentStack<T> (which was shown in an earlier example) represent thread-safe and scalable alternatives to the generic Queue<T> and Stack<T>, and types like BlockingCollection<T> make it easy to implement blocking producer/consumer scenarios, where multiple threads are producing data for consumption by a host of consumer threads, as well as pipeline patterns, where data is flowing through a series of filters that can be executed in parallel.
As already mentioned, PLINQ is a core part of the next version of the .NET Framework. For read-only queries that process large amounts of data, PLINQ provides a simple, declarative path towards parallelization. By adding AsParallel to the data source as shown in the earlier baby names example, the developer is switching over to the PLINQ implementation of the .NET Standard Query Operators. (This switch is made explicit so that developers are forced to opt-in to the parallelization support. This is important given that, while most LINQ queries are read-only, some do perform mutations. If PLINQ is used with those mutations, the mutations would need to be thread-safe in order for the parallelization to be safe.)
Under the covers, PLINQ uses classic data parallelism techniques to partition the input data, process it on multiple threads, and merge the data back together in a manner that is appropriate to how the data is being produced and consumed. For more information on PLINQ, see the October 2007 issue of MSDN Magazine (msdn.microsoft.com/magazine/cc163329), where it was first discussed in detail.

Expressing Parallelism in Native Code
Over the last few years, we've heard several developers question the devotion shown at Microsoft to native C++. While we understand the concerns, we also see all of the effort being directed at native code in the Developer Division at Microsoft and we can say with a clear conscience that we see those fears being addressed. For example, the Visual C++ team recently released a fully supported implementation of Technical Report 1 (TR1) as well as significant updates to MFC, both of which are available as part of the Visual C++ 2008 Feature Pack (see go.microsoft.com/fwlink/?LinkId=124366). With regard to concurrency, support for parallelizing native C++ applications has been one of the most frequently requested features, and providing that support is a primary goal of both the Visual C++ and Parallel Computing Platform teams. The next version of Visual Studio and Visual C++ will reflect that.
As with managed code, enabling fine-grained parallelism and the migration of sequential applications to parallel applications are at the heart of the concurrency support in Visual C++. Consider the motivating example of a simple matrix multiplication:
void MatrixMult(int size, double** m1,
  double** m2, double** result) {
  for (int i = 0; i < size; i++) {
    for (int j = 0; j < size; j++) {
      result[i][j] = 0;
      for (int k = 0; k < size; k++) {
        result[i][j] += m1[i][k] * m2[k][j];
      }
    }
  }
}
An attempt to parallelize this method is shown in Figure 4.
Figure 4 Parallelizing MatrixMult
void MatrixMult(int size, double** m1, double** m2, double** result) {
  int N = size;
  int P = 2 * NUMPROCS;
  int Chunk = N / P;
  HANDLE hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
  long counter = P;
    for (int c = 0; c < P; c++) {
        std::thread t ([&,c] {
            for (int i = c * Chunk;
                 i < (c + 1 == P ? N : (c + 1) * Chunk);
                 i++) {
              for (int j = 0; j < size; j++) {
                result[i][j] = 0;
                for (int k = 0; k < size; k++) {
                    result[i][j] += m1[i][k] * m2[k][j];
                }
              }
            }
            if (InterlockedDecrement(&counter) == 0) SetEvent(hEvent);
        });
    }
    WaitForSingleObject(hEvent,INFINITE);
    CloseHandle(hEvent);
}
This code demonstrates some useful new functionality being introduced in the next version of Visual C++, including support for the C++0x std::thread type as well as lambda expressions, which greatly simplify the development of asynchronous patterns.
However, even with these additions, this solution still leaves a lot to be desired. Static partitioning is used to divide the work across all available processors, but if there's any unevenness in the computation, or if there's anything else happening on the machine that could cause partitions to be processed at different speeds, many threads could end up waiting for (and offering no assistance to) one or more threads that need to finish their portions of the work.
The solution still requires knowledge of synchronization primitives like events—and how many readers can say, without looking at the documentation, how those CreateEvent parameters NULL, TRUE, FALSE, and NULL affect the synchronization?
The solution is error prone, with great potential for difficult-to-detect off-by-one errors in the partitioning step. Explicit interlocked operations are necessary to track the number of outstanding threads. The solution isn't taking advantage of the thread that's calling MatrixMult to engage in the processing, preferring instead to block it and thus waste some potentially valuable system resources. And so on.
We need an easier solution, and the next version of Visual C++ provides one:
void MatrixMult(int size, double** m1, double** m2, double** result) {
  parallel_for(0, size, 1, [&](int i) {
    for (int j = 0; j < size; j++) {
      result[i][j] = 0;
      for (int k = 0; k < size; k++) {
        result[i][j] += m1[i][k] * m2[k][j];
      }
    }
  });
}
This example still takes advantage of lambda expressions, but it's also using the new parallel_for method, part of the Parallel Pattern Library (PPL) to be shipped in the Visual C++ runtime libraries. Note that parallel_for is one of several high-level parallelization functions that are being developed, along with others like parallel_for_each, parallel_accumulate, and parallel_invoke. Many of these methods should feel familiar, as they provide direct parallel counterparts to their Standard Template Library (STL) equivalents, and enable easy substitution of the parallel method for the sequential method with minimal code rewriting (assuming, of course, that the operation being performed in parallel is thread-safe).
For example, consider a for_each call that iterates over a std::vector of objects representing enemies in a game, and updates each of their positions based on their capabilities and locations:
for_each(enemies.begin(), enemies.end(), [&](AIEnemy& cur) {
  cur.updatePosition();
})
As long as the vector in question contains no duplicates, and as long as the updatePosition member function only modifies the local object, this method can be parallelized simply by prepending parallel_ to the method call:
parallel_for_each(enemies.begin(), enemies.end(), [&](AIEnemy& cur) {
  cur.updatePosition();
});
Just as how in managed code the System.Threading.Parallel class is built on top of lower-level types for handling task parallelism, so too in the native solution are the parallel_ methods. (Where appropriate, the native and managed solutions will feel similar, enabling developers to more easily take knowledge from one domain and apply it to the other.) The two core types that serve as the foundation of PPL are task_handle and task_group. Both of these are shown in Figure 5.
Figure 5 task_handle and task_group
template <class _Func>
class task_handle {
public:
  task_handle(const _Func& f);
};

  class task_group {
public:
  task_group ();
  task_group(tr1::function<void(exception_ptr)> Fn);
  ~task_group ();
  template <class _Func> void run(const _Func& Fn);
  template <class _Func> void run(task_handle<_Func>& t);
  void cancel();
  task_group_status wait();
  void copy_exception(exception_ptr);
};
As an example of how these constructs can be used, consider a quicksort implementation as shown in David Callahan's article in this issue of MSDN Magazine (available at msdn.microsoft.com/magazine/cc872852). David's solution follows the typical quicksort pattern, falling back to a sort with less overhead when the length of the data being sorted is small enough, partitioning the data into two halves, and recursively using quicksort to sort each of the halves. The solution he shows uses parallel_invoke, but parallel_invoke is built on top of task_group, and an implementation that uses it directly might look something like this:
template<class T>
void QuickSort(T * data, int length, T* scratch) {
  if(length < THRESHOLD) InsertionSort(data, length);
  else {
    int mid = Partition(data[0], data, length, scratch, true);
    task_group g;
    g.run([&]{QuickSort(data, mid);});
    g.run([&]{QuickSort(data+mid, length-mid);});
    g.wait();
  }
}
Note that very little boilerplate was necessary to enable the implementation, simply creating a task_group, running each recursive call as a separate task, and then waiting on both to complete. That wait operation may not be a true kernel wait at all, instead preferring to inline a scheduled task onto the current thread rather than wasting that thread's resources.
Other core types are also provided. For example, the task_group type previously shown is thread-safe in that it enables multiple, unrelated threads to schedule tasks to it concurrently. For structured parallelism, where only tasks in parent/child relationships will ever schedule to the same group concurrently, the structured_task_group type provides optimized support for the same functionality.
As on the managed side, tasks are executed on an underlying and efficient scheduler. The scheduler uses work-stealing algorithms to enable efficient execution of fine-grained operations, supports cooperative blocking to enable efficient reuse of resources, provides detailed control over the policies employed by the scheduler for its operation, and more. Additionally, the scheduler has built-in support for primitives focused more on intra-process messaging than on fine-grained task execution.
One of the great concurrency features provided in the next version of Visual C++ is support for messaging and asynchronous agents. This gives developers an approach to writing parallel applications where they define coarse-grained components and enable them to interact. This interaction is accomplished through an in-process messaging infrastructure, allowing applications to tolerate latencies, to support component composition, and to utilize isolation as a mechanism for thread safety.
The core support revolves around several messaging functions (send, receive, asend, and try_receive) as well as ISource and ITarget interfaces that support being sent to and received from. The library includes several messaging blocks that implement these interfaces, such as unbounded_buffer, overwrite_buffer, single_assignment, choice, call, transform, and timer.
The system was also designed to encourage custom messaging blocks to be implemented as are necessary for the systems being developed. Messaging blocks support the ability to store messages and the ability to forward messages on to other message blocks. Blocks can be linked together, such that when messages arrive to one block, the block processes them as is appropriate to its semantics and then forwards on any resulting messages to blocks that have been linked to it in a network.
For example, the code in Figure 6 shows two unbounded_buffers, both linked to a join block, which is in turn linked to a transform block, which is in turn linked to a call block. When data arrives at both of the unbounded_buffers, that data will be passed through the join block to the transform block, which can manipulate the data with a developer-provided function and send the results to the call block, which will provide that data to another developer-provided method. (The messaging code samples shown use the currently intended syntax, which may differ slightly from that available in initial preview releases.)
Figure 6 Passing Messaging Blocks
unbounded_buffer<int> unboundedBuf1;
unbounded_buffer<int> unboundedBuf2;
join<int,int> join1;

transform<tr1::tuple<int,int>,int>
  transform1([](tr1::tuple<int,int>& input){
  return tr1::get<0>(input) * tr1::get<1>(input);
};
call<int> call1([](int& input){
  cout << "result: " << input << endl;
};

unboundedBuf1.link_target(join1);
unboundedBuf2.link_target(join1);
join1.link_target(transform1);
transform1.link_target(call1);
Agent-based apps can also benefit from the new agent_task class, which is in effect a dedicated thread from the scheduler that is meant to interact with other tasks by sending and receiving messages. Agent_task can be quite useful in producer/consumer designs.
For example, consider the code at the top of Figure 7, a simple log file parser that continually reads lines of text from a log file and processes them. That processing can be overlapped with the I/O for reading the file by having one thread dedicated to reading the file and another dedicated to processing its contents, as shown at the bottom of the figure. An unbounded_buffer is created for storing the messages sent from the reader to the parser, and the parser agent is spun up to continually receive from the buffer and process the resulting message. Meanwhile, as in the sequential example, the ParseFile method opens the file and reads lines from it, but rather than directly parsing each line, it simply sends each to the unbounded_buffer for the parser agent to consume. When the file has been completely read, it sends an exit message to the parser agent and waits for the agent to complete.
Figure 7 Parsing a File with Agents
Sequential
HRESULT LogChunkFileParser::ParseFile() {
  HRESULT hr = S_OK;
  WCHAR wszLine[MAX_LINE] = {0};
  hr = reader->OpenFile(pFileName);
  if (SUCCEEDED(hr)){
    while(!reader->EndOfFile()){
      hr = reader->ReadLine(wszLine, MAX_LINE);
      if (SUCCEEDED(hr)){
        Parse(wszLine);
      }
    }
  }
  return S_OK;
}
Parallel
HRESULT LogChunkFileParser::ParseFile() {
  unbounded_buffer<AgentMessage> msgBuffer =
    new unbounded_buffer<AgentMessage>();
  agent_task* pParserAgent = agent_task::start([&] {
    AgentMessage msg;
    while((msg = receive(msgBuffer))->type != EXIT) {
      Parse(msg->pCurrentLine);
      delete msg->pCurrentLine;
    }
  });
  HRESULT hr = reader->OpenFile(pFileName);
  if (SUCCEEDED(hr)){
    while(!reader->EndOfFile()) {
      WCHAR* wszLine = new WCHAR[MAX_LINE];
      hr = reader->ReadLine(wszLine, MAX_LINE);
      if (SUCCEEDED(hr)) {
         send(msgBuffer, AgentMessage(wszLine));
      }
    }
    send(msgBuffer, AgentMessage(EXIT));
    hr = agent_task::wait_for_completion(pParserAgent);
  }
  return hr;
};
As you can see, there's a great deal of useful functionality on its way to the development platform, for both native and managed code developers. However, it's worth pointing out that all of these new constructs are not a panacea. Much of this support is focused on enabling developers to more easily express parallelism in their applications, to remove the boilerplate that was previously necessary in doing so, and to aid in the replacement of sequential programming constructs with parallel ones. Yet constructs like parallel_for and AsParallel don't automatically make your code safe to run in concurrent contexts; race conditions, deadlocks, and the like are all still possible, and by making it so much easier to develop parallel applications, such issues will likely become even more common.

Debugging Parallel Applications
The challenges in parallel programming are not limited to programming models. Tool support is a critical ingredient in a successful platform for parallel programming. This is due to the increased complexity that developers face in debugging and performance tuning parallel applications. Thus, Visual Studio is investing in improving tool support for parallel programming on both the legacy and new programming models.
This is not to say that the tools in Visual Studio 2008 don't already offer support for multithreading. In fact, Visual Studio 2008 sports several useful features to aid in the development of multithreaded applications.
For example, an interesting dilemma for a debugging tool for parallel applications is the nature of breakpoint semantics as they relate to multiple execution contexts. Specifically, the question is whether to stop all execution contexts or just the one that encounters a breakpoint. The default behavior is typically to stop all threads at a breakpoint; however, it is sometimes desirable to allow one or more other execution contexts, such as threads, to proceed even when another thread is at a breakpoint. You can imagine how such a capability might be very useful in situations where race conditions do not manifest themselves with an all-blocking debugger implementation. The debugger offers a feature to freeze and thaw threads as users see fit. Initially, when a breakpoint is hit, all threads stop. At this point, a user may choose to thaw a subset of threads before proceeding with the debugging session. The user is also able to freeze threads as she sees fit.
However, there is still much more that needs to be done to improve productivity while developing and debugging parallel applications. Programming models such as those provided by TPL and PPL reduce the complexity threshold for developing well-performing parallel applications. A common theme for these programming models is the existence of a runtime system that abstracts away the details of partitioning work, scheduling it, and adding any necessary synchronization to ensure proper ordering constraints are met. In fact, a primary goal of these models is that developers no longer need to think about parallelism at the thread level, instead thinking about it at the fine-grained task level and higher.
Additionally, while having a runtime that does automatic parallelization eases the burden on the developer by removing the need to write all of the boilerplate code, it also means that the runtime chooses how to partition work, potentially into many thousands of tasks that are divided among many virtual processors that are further mapped onto an execution abstraction such as a thread. From a developer's perspective, if there's a need to debug the execution of individual tasks, it may be difficult, if not impossible, to trace the path of execution that led to a certain piece of code. This creates a dichotomy between the programming model and the debugger, which today focuses on threads.
Two new features in the next version of Visual Studio, MultiStack and a Task List views, are intended to address this difficulty by reducing the burden of debugging applications that make use of parallel programming runtimes.
The MultiStack view, as shown in Figure 8, graphs the logical call stacks for all (or a selected set of) tasks, where a logical call stack is one in which a child task's stackframes are linked back through its parent. The view coalesces common stack prefixes into a single node and tags the node with a count of the tasks sharing the prefix. Thus the graph presents a global view of the execution state of the set of tasks. The user can then traverse the call stacks of any running task at a breakpoint and switch to show the particular instance of a method in the editor.
Figure 8 MultiStack
The Task List in Figure 9 shows all tasks known to the runtime system at a breakpoint, their respective execution states, as well as the method currently being executed, with a list of arguments and their values. The list may be sorted, grouped by, or searched on any column, and it allows a developer to quickly switch among tasks and use the various other debugger features to further examine execution state.
Figure 9 Task List
Another challenging area in parallel programming, especially in a shared memory platform, is the ability to examine memory contents within multiple execution contexts simultaneously, while providing the usual expression evaluation capability efficiently. The next version of Visual Studio includes a MultiWatch view that specifically addresses this need. This feature allows the developer to add variable names and expressions to a watch window, where the expressions are evaluated at each debugger step or breakpoint and for every thread or task in the system. This makes it much easier to spot inconsistencies in data across threads in order to find problem areas and causes of otherwise insidious bugs.

Profiling Parallel Applications
Parallelism is all about performance (whether responsiveness or throughput), so a good set of performance analysis tools is critical to a platform's success. Exploiting parallelism for compute-bound operations can be challenging, but it is definitely less daunting than the task of parallelizing complex client applications. The added complexity is due to the fact that rich client applications include phases of varying bottlenecks, including disk and network I/O.
From a developer's perspective, the key to parallelizing such applications is to understand enough of their behavior to identify opportunities for parallelism. Once that is done, the developer can apply many of the programming patterns and abstractions available to accomplish the job. A key point to remember during this phase is that concurrency can be a technique for hiding latency, not just for speeding up computation. Once the application is written, there are additional sources of inefficiency (such as synchronization) that need to be understood, measured, and tuned in order to arrive at a well-performing solution.
The performance analysis tools in Visual Studio 2008 offer a significant number of features to aid developers of managed and native applications in their tuning efforts. The existing profiler supports sampling and instrumentation based profiling. Sample profiling intercepts the application program at user-defined intervals, captures a call stack per thread, and continues execution. At analysis time, all the call stacks are collected and collated to identify the functions where most samples were taken, along with the application paths that lead there.
One drawback of this technique is that it does not account for time during which the application's threads are blocked due to synchronization or I/O calls. As a result, sample profiling is recommended only for CPU-bound applications.
Besides sampling on-time intervals, the Visual Studio profiler allows sampling on other important events such as page faults or microprocessor performance monitoring counters (for example, L2 cache misses). These capabilities can be extremely valuable tools for understanding important aspects of application behavior such as cache and/or memory page locality. (See the .NET Matters column in this issue of MSDN Magazine at msdn.microsoft.com/magazine/cc872851 for further exploration of this topic.)
Instrumentation-based profiling, as the name implies, works by augmenting the code of each function in the target application or module. Such instrumentation is typically inserted at the entry and exit points of the function, including any call sites to other methods. This level of instrumentation allows the profiler to account for all delays in the application's execution, albeit at the expense of a significantly higher execution time and trace logging overhead. These overheads can significantly perturb execution and increase the time needed to analyze data. To reduce such overheads, the Visual Studio profiler allows users to partially instrument the application at the granularity of a function or a module (DLL). In addition, the profiler provides an API that allows programmers to tailor performance data collection during execution time.
From a parallel programmer's perspective, the current profiler experience lacks several features that are desirable such as higher-level support for the new parallel programming runtimes, as well as specific scenarios that address common parallel performance issues, such as synchronization. In the next version of Visual Studio, the profiler is augmented with tools that specifically address the needs of multithreaded application developers.
A crucial step in parallel development is understanding the degree of concurrency or the potential for performance improvement using parallelism in an application. This is important both prior to implementing parallelization and during tuning phases.
When examining an application that is a target for parallelism, it is necessary to determine the phases of the application that are candidates for parallelization. Usually, those phases are CPU bound, but there are currently no tools to easily determine such phases of execution and map them to application code.
The next release of Visual Studio will include concurrency analysis support, as shown in Figure 10. With time on the x-axis and the number of cores in the system on the y-axis, the graph shows how core utilization varies with time for the application of interest (shown in green). This view allows users to determine the phases of an application that are CPU bound, their relative durations and the level of system core utilization, then to zoom in on them to determine what the application was doing during those phases to start considering the forms of parallelism that may exist and how they may be exploited. During the performance-tuning phase, this view can confirm whether the expected concurrency is indeed observed during execution time.
Figure 10 Concurrency Analysis
In addition, basic timing measurements can be made to estimate speed improvements achieved compared to a previous or sequential execution. Finally, the concurrency view also gives the user an idea of the interaction between the application in question and other processes in the system, or the kernel. In the view, idle time is shown in black, while other non-kernel processes are shown in yellow. Kernel activity (minimal in this example) is shown in red.
Once the developer has identified a phase of interest using the Concurrency View, she can then further analyze the behavior of the application using a Thread Blocking View, shown in Figure 11. This view provides a wealth of information about the behavior of each thread in the application.
Figure 11 Thread Execution and Blocking Analysis
First, it generates a bar graph with an execution breakdown of each thread, depicting the fraction of each thread's lifetime that was spent executing code or blocking. Blocking delays are further broken down into various categories, such as I/O or synchronization.
Second, the view provides a timeline view with time on the x-axis and threads and physical disk I/O as lanes on the y-axis. For disk I/O, the display shows reads and writes as they occur in time on disk, as well as the files being accessed using tooltips. For threads, it shows when they are executing, when they are blocked, and for what category of delay using colors.
Third, call stack analysis allows the developer to hover over any blocking segment in the timeline view to understand in detail what each thread was doing when it was blocked. The screenshot in Figure 11 shows an example where the mouse is hovering over a synchronization blocking segment that is about 83ms long due to a call to EnterCriticalSection. It also shows the call stack, including line numbers, that resulted in the blocking attempt to acquire the critical section. Such details are crucial to allow developers to examine the root cause of an inefficiency and address it. Other features include detailed execution, blocking, and disk I/O statistics and reports that are available in the lower tabs.
Another important aspect of parallel performance analysis is thread migration due to its interaction with cache working sets. In Figure 12 we show the Core Execution and Thread Migration view, in which we associate a unique color with each thread, and then we display how the threads are scheduled to execute on the system's processor cores. A thread migration event would be graphically depicted by a color moving across cores. For each thread migration event that we are interested in, we can identify it in an application phase, make a note of the thread ID by hovering over the thread involved, and then switch to the thread blocking view. The thread blocking view can then illustrate the nature of the blocking event preceding the thread migration and indicate where it occurred in the app. The developer can then decide whether to optimize this type of problem by either reducing that class of blocking events and/or enforcing affinity if that is deemed to be acceptable for the app's execution environment.
Figure 12 Core Execution and Thread Migration

Looking Forward
At the time of this writing, no dates have yet been announced for when the next version of Visual Studio, which contains all of the aforementioned concurrency support, will be released. Nevertheless, very soon managed and native developers alike will be given the ability to effectively develop applications that efficiently utilize all of the computing power multicore systems have to offer. With such systems quickly becoming the standard in the enterprise and in the home, the time to start learning about such support is now.
Thanks to Rick Molloy, David Callahan, and Paul Maybee for their assistance with this article.

Stephen Toub is a Senior Program Manager Lead on the Parallel Computing Platform team at Microsoft, where he focuses his efforts on programming models for concurrency. He is also a Contributing Editor for MSDN Magazine.

Hazim Shafi is a Principal Architect responsible for performance tools on the Parallel Computing Platform team at Microsoft. He also teaches courses on parallel programming at Microsoft.


Concurrency Hazards
Solving 11 Likely Problems In Your Multithreaded Code
Joe Duffy

This article discusses:
  • Fundamental concurrency concepts
  • Concurrency problems and mitigations
  • Patterns for achieving safety
  • Cross-cutting concepts
This article uses the following technologies:
Multithreading, .NET Framework
Contents
Data Races
Forgotten Synchronization
Incorrect Granularity
Read and Write Tearing
Lock-Free Reordering
Reentrancy
Deadlocks
Lock Convoys
Stampeding
Two-Step Dance
Priority Inversion
Patterns for Achieving Safety
Immutability
Purity
Isolation
Concurrency is everywhere. Server-side programs have long had to deal with a fundamentally concurrent programming model, and as multicore processors become more commonplace, client-side programs will have to as well. Along with the addition of concurrency comes the responsibility for ensuring safety. In other words, programs must continue to achieve the same level of robustness and reliability in the face of large amounts of logical concurrency and ever-changing degrees of physical hardware parallelism.
Correctly engineered concurrent code must live by an extra set of rules when compared to its sequential counterpart. Reads and writes from memory and access to shared resources need to be regulated using synchronization so that conflicts do not arise. Additionally, it often becomes necessary for threads to coordinate to get a certain job done.
As a direct result of these additional requirements, it becomes nontrivial to ensure that threads are continually making consistent and adequate forward progress. Synchronization and coordination deeply depend on timing, which is nondeterministic and hard to predict and test for.
What makes these attributes difficult is simply that it requires a mindset shift. There isn't a single API that you can learn or a snippet of code to copy and paste. There really is a fundamental set of concepts that you need to learn and become comfortable with. It's likely that certain languages and libraries can hide some concepts over time, but if you're doing concurrency today, you won't have that luxury. This article describes some of the more common challenges to be aware of and presents advice for coping with them in your software.
I'll start by looking at the kinds of things that often go wrong in concurrent programs. I call these "safety hazards" because they are so easy to stumble upon and because their consequences are typically quite undesirable. These hazards cause your programs to break by either crashing or corrupting memory.

A data race—or race condition—occurs when data is accessed concurrently from multiple threads. Specifically, it happens when one or more threads are writing a piece of data while one or more threads are also reading that piece of data. This problem arises because Windows programs (in C++ and the Microsoft .NET Framework alike) are fundamentally based on the concept of shared memory, where all threads in a process may access data residing in the same virtual address space. Static variables and heap allocation can be used to share.
Consider this canonical example:
static class Counter {
    internal static int s_curr = 0;
    internal static int GetNext() {
        return s_curr++;
    }
}
The goal of Counter is presumably to hand out a new unique number for each call to GetNext. If two threads in the program both call GetNext simultaneously, however, two threads might be given the same number. The reason is that s_curr++ compiles into three separate steps:
  1. Read the current value from the shared s_curr variable into a processor register.
  2. Increment that register.
  3. Write the register value back to the shared s_curr variable.
Two threads executing this same sequence can both read the same value from s_curr locally (say, 42), increment it (to, say, 43), and publish the same resulting value. GetNext thus returns the same number for both threads, breaking the algorithm. Although the simple statement s_curr++ appears to be atomic, this couldn't be farther from the truth.

Forgotten Synchronization
This is the simplest kind of data race: synchronization is forgotten altogether. Races of this kind can be, in rare situations, benign, meaning that they are correct, but most are fundamental problems of correctness.
Problems like this aren't always so obvious. For example, an object may be part of some large complicated object graph that happens to be reachable from a static variable or became shared by passing an object as part of a closure when creating a new thread or queuing work to a thread pool.
It is imperative to pay attention to when an object (graph) transitions from private to shared. This is called publication and will be discussed later in the context of isolation. The reverse is called privatization—that is, when an object (graph) goes from being shared to private again.
The solution is to add proper synchronization. In the example of the counter, I could use a simple interlocked:
static class Counter {
    internal static volatile int s_curr = 0;
    internal static int GetNext() {
        return Interlocked.Increment(ref s_curr);
    }
}
This works because the update is confined to a single memory location, and because (conveniently) there is a hardware instruction (LOCK INC) that is equivalent to the software statement I'm trying to make atomic.
Alternatively, I could use a full-fledged lock:
static class Counter {
    internal static int s_curr = 0;
    private static object s_currLock = new object();
    internal static int GetNext() {
        lock (s_currLock) {
            return s_curr++;
        }
    }
}
The lock statement ensures mutual exclusion among all threads trying to access GetNext, and uses the CLR System.Threading.Monitor class. C++ programs would use a CRITICAL_SECTION for the same purpose. There is no need to go with a lock for this particular example, but when multiple operations are involved it's seldom possible to fold them into a single interlocked operation.

Incorrect Granularity
Even if access to shared state occurs with proper synchronization, the resulting behavior may still be incorrect. The granularity must be sufficiently large that the operations that must be seen as atomic are encapsulated within the region. There is a tension between correctness and making the region smaller, because smaller regions reduce the time that other threads will have to wait to concurrently enter.
For example, take a look at the bank account abstraction shown in Figure 1. All is well, and the object's two methods, Deposit and Withdraw, appear to be concurrency-safe. Some banking application can use them without worry that balances will become corrupt due to concurrent access.
Figure 1 A Bank Account
class BankAccount {
    private decimal m_balance = 0.0M;
    private object m_balanceLock = new object();
    internal void Deposit(decimal delta) {
        lock (m_balanceLock) { m_balance += delta; }
    }
    internal void Withdraw(decimal delta) {
        lock (m_balanceLock) {
            if (m_balance < delta)
                throw new Exception("Insufficient funds");
            m_balance -= delta;
        }
    }
}
What if, however, you'd like to add a Transfer method? A naive (and incorrect) attempt would assume that, because Deposit and Withdraw are safe in isolation, they can be combined easily:
class BankAccount {
    internal static void Transfer(
      BankAccount a, BankAccount b, decimal delta) {
        Withdraw(a, delta);
        Deposit(b, delta);
    }
    // As before
}
This is incorrect. There is actually a period of time between the Withdraw and Deposit calls where the money is missing completely.
A correct implementation would need to acquire the locks on both a and b up front and then make the method calls:
class BankAccount {
    internal static void Transfer(
      BankAccount a, BankAccount b, decimal delta) {
        lock (a.m_balanceLock) {
            lock (b.m_balanceLock) {
                Withdraw(a, delta);
                Deposit(b, delta);
            }
        }
    }
    // As before
}
It turns out that this approach, while solving the granularity problem, is prone to deadlock. You'll see how to fix it later on.

Read and Write Tearing
As mentioned earlier, benign races allow you to access variables without synchronization. Reads and writes of aligned, naturally sized words—for example, pointer-sized things that are 32 bits (4 bytes) on 32-bit processors and 64 bits (8 bytes) on 64-bit processors—are atomic. If a thread is just reading a single variable that some other thread will write, and there aren't any complex invariants involved, you can sometimes skip the synchronization altogether thanks to this guarantee.
But beware. If you try this on a misaligned memory location, or a location that isn't naturally sized, you can encounter a read or write tearing. Tearing occurs because reading or writing such locations actually involves multiple physical memory operations. Concurrent updates can happen in between these, potentially causing the resultant value to be some blend of the before and after values.
As an example, imagine ThreadA sits in a loop and writes only 0x0L and 0xaaaabbbbccccddddL to a 64-bit variable s_x. ThreadB sits in a loop reading it (see Figure 2).
Figure 2 Tearing about to Happen
internal static volatile long s_x;
void ThreadA() {
    int i = 0;
    while (true) {
        s_x = (i & 1) == 0 ? 0x0L : 0xaaaabbbbccccddddL;
        i++;
    }
}
void ThreadB() {
    while (true) {
        long x = s_x;
        Debug.Assert(x == 0x0L || x == 0xaaaabbbbccccddddL);
    }
}
You may be surprised to discover that ThreadB's assert might fire. The reason is that ThreadA's write will consist of two parts, the high 32-bits and the low 32-bits, in some order depending on the compiler. The same goes for ThreadB's read. Thus ThreadB could witness values 0xaaaabbbb00000000L or 0x00000000aaaabbbbL.

Lock-Free Reordering
Sometimes it's tempting to write lock-free code as a way of achieving better scalability and reliability. Doing so requires a deep understanding of the memory model of your target platform (see Vance Morrison's article "Memory Models: Understand the Impact of Low-Lock Techniques in Multithreaded Apps" at msdn.microsoft.com/magazine/cc163715 for details). Failure to understand and heed these rules can lead to memory reordering bugs. These happen because compilers and processors are free to reorder memory operations during the process or making optimizations.
As an example, assume s_x and s_y are both initialized to the value 0, as you see here:
internal static volatile int s_x = 0;
internal static volatile int s_xa = 0;
internal static volatile int s_y = 0;
internal static volatile int s_ya = 0;

void ThreadA() {
    s_x = 1;
    s_ya = s_y;
}

void ThreadB() {
    s_y = 1;
    s_xa = s_x;
}
Is it possible that, after ThreadA and ThreadB both run to completion, both s_ya and s_xa will contain the value 0? This seems ridiculous on its face. Either s_x = 1 or s_y = 1 will happen first, in which case the other thread will witness the update when it gets around to its update. Or so the theory goes.
Unfortunately, processors are free to reorder this code so that the loads effectively happen before the writes. You can get around this with an explicit memory barrier:
void ThreadA() {
    s_x = 1;
    Thread.MemoryBarrier();
    s_ya = s_y;
}
The .NET Framework offers this particular API, and C++ offers _MemoryBarrier and similar macros. But the moral of the story is not that you should insert memory barriers all over the place. The moral is that you should steer clear of lock-free code until you've mastered memory models, and even then tread with care.

In Windows, including Win32 and the .NET Framework, most locks support recursive acquires. That just means that, if the current thread already holds a lock and tries to acquire it again, the request will be satisfied. This makes it easier to compose bigger atomic operations out of smaller ones. In fact, the BankAccount example shown earlier depended on recursive acquires: Transfer called both Withdraw and Deposit, each of which redundantly acquired a lock that Transfer had already acquired.
This can be the source of problems, however, if you end up in a situation where a recursive acquire occurs when you didn't expect it to. This can happen as a result of reentrancy, which is possible due to either explicit callouts to dynamic code (such as virtual methods and delegates) or because of implicitly reentered code (such as STA message pumping and asynchronous procedure calls). For this reason, among others, it's a good idea to never make a call to a dynamic method from within a lock region.
For example, imagine some method that breaks invariants temporarily and then calls a delegate:
class C {
    private int m_x = 0;
    private object m_xLock = new object();
    private Action m_action = ...;

    internal void M() {
        lock (m_xLock) {
            m_x++;
            try { m_action(); }
            finally {
                Debug.Assert(m_x == 1);
                m_x--;
            }
        }
    }
}
C's method M ensures that m_x does not change. But there is a brief period of time where m_x is incremented by one before being decremented again. The callout to m_action looks innocent enough. Unfortunately, if it was a delegate accepted from users of the C class, it represents arbitrary code that can do anything it pleases. That includes calling back into the same instance's M method. And if that happens, the assert within the finally is apt to fire; there could be multiple calls to M active on the same stack, even though you didn't do it directly, which clearly can lead to m_x holding a value greater than 1.

When multiple threads encounter a deadlock, the system simply stops responding. Several MSDN Magazine articles have described how deadlocks occur and some ways of tolerating them—including my own article, "No More Hangs: Advanced Techniques to Avoid and Detect Deadlocks in .NET Apps," at msdn.microsoft.com/magazine/cc163618, and Stephen Toub's October 2007 .NET Matters column at msdn.microsoft.com/magazine/cc163352—so the discussion here will be kept light. In summary, whenever a circular wait chain is created—such that some ThreadA is waiting for a resource held by ThreadB, which is also reflexively waiting for a resource held by ThreadA (perhaps indirectly by waiting for a third ThreadC or more)—it is possible for all forward progress to come to a grinding halt.
A common source of this problem is with mutually exclusive locks. In fact, the BankAccount example shown earlier suffers from this problem. If ThreadA tries to transfer $500 from account #1234 to account #5678 at the same time ThreadB tries to transfer $500 from #5678 to #1234, the code can deadlock.
Using a consistent acquisition order can avoid the deadlock, as shown in Figure 3. This logic can be generalized to something called simultaneous lock acquisition, whereby multiple lockable objects are sorted dynamically according to some ordering among locks, so that any place two locks must be held at the same time they are acquired in a consistent order. Another scheme called lock leveling can be used to reject lock acquisitions that are provably done in an inconsistent order.
Figure 3 Consistent Acquisition Order
class BankAccount {
    private int m_id; // Unique bank account ID.
    internal static void Transfer(
      BankAccount a, BankAccount b, decimal delta) {
        if (a.m_id < b.m_id) {
            Monitor.Enter(a.m_balanceLock); // A first
            Monitor.Enter(b.m_balanceLock); // ...and then B
        } else {
            Monitor.Enter(b.m_balanceLock); // B first
            Monitor.Enter(a.m_balanceLock); // ...and then A
        }
        try {
            Withdraw(a, delta);
            Deposit(b, delta);
        } finally {
            Monitor.Exit(a.m_balanceLock);
            Monitor.Exit(b.m_balanceLock);
        }
    }
    // As before ...
}
But locks are not the only source of deadlock. Missed wake-ups are another phenomenon, where some event is missed and a thread sleeps forever. This is common with synchronization events like Win32 auto-reset and manual-reset events, CONDITION_VARIABLEs, and CLR Monitor.Wait, Pulse, and PulseAll calls. A missed wake-up is usually a sign of improper synchronization, failure to retest a wait condition, or use of a wake-single primitive (WakeConditionVariable or Monitor.Pulse) when a wake-all (WakeAllConditionVariable or Monitor.PulseAll) would have been more appropriate.
Another common source of this problem is lost signals with auto- and manual-reset events. Because such an event can only be in one state—signaled or nonsignaled—redundant calls to set the event will effectively be ignored. If code assumes that two calls to set always translates to two threads awakened, the result can be a missed wake-up.

Lock Convoys
When the arrival rate at a lock is consistently high compared to its lock acquisition rate, a lock convoy may result. In the extreme, there are more threads waiting at a lock than can be serviced, leading to a catastrophe. This is more common on server-side programs where certain locks protecting data structures needed by most clients can get unusually hot.
For example, imagine this scenario: On average, eight requests arrive per 100 milliseconds. We use eight threads to service requests (because we're on an 8-CPU machine). Each of those eight threads must acquire a lock and hold it for 20 milliseconds before it can do meaningful work.
Unfortunately, access to this lock must be serialized, so it takes 160 milliseconds for all eight threads to enter and leave the lock. After the first exists, there will be 140 milliseconds of time before a ninth thread can access the lock. This scheme inherently will not scale, and there will be a continuously growing backup of requests. Over time, if the arrival rate does not decrease, client requests are apt to begin timing out, and a disaster will ensue.
Fairness in locks is known to contribute to convoys. The reason is that periods of time where the lock could have been made available are artificially blocked out, so that threads arriving must wait until the chosen lock owner thread can wake up, context switch, and acquire and release the lock. Windows has changed all of its internal locks to be unfair over time in order to combat this problem, and CLR monitors are also unfair.
The only real solution to the fundamental convoy problem is to reduce lock hold times and to factor your system so that there are very few (if any) hot locks. This is easier said than done, but is crucial for scalability.

A stampede is a situation where many threads are awakened, such that they all fight for attention simultaneously from the Windows thread scheduler. If you have 100 threads blocked on a single manual-reset event, for example, and you set that event … well, you're apt to have a real mess on your hands, particularly if a large portion of those threads will have to wait again.
One way to implement a blocking queue is to use a manual-reset event that gets unsignaled when the queue is empty, and becomes signaled when it is non-empty. Unfortunately, when there are a large number of waiting threads during the transition from zero elements to one element, a stampede can occur. That's because only one thread will take the single element, which transitions the queue back to empty, and necessarily involves resetting the event. If you had 100 threads waiting, then 99 of them will wake up, context switch (and incur all those cache misses), just to realize they have to wait again.

Two-Step Dance
Sometimes you need to signal an event while holding a lock. This can be unfortunate if the waking thread needs to acquire the lock held, because it will be awakened only to find out that it must wait again. This is wasteful and increases the number of overall context switches. This situation is called the two-step dance, and can extend far beyond just two steps if many locks and events are involved.
Both Win32 and the CLR's condition variable support inherently suffers from the two-step dance problem. It is often unavoidable, or prohibitively difficult to work around.
The two-step dance issue is even worse on a single-processor machine. When events are involved, the kernel will apply a priority boost to the awakening thread. This is nearly guaranteed to preempt the thread setting the event before it has a chance to release the lock. This is two-step dance in the extreme, where the setting ThreadA is context switched out so that the awakening ThreadB can attempt to acquire the lock; it of course cannot, and so it will context switch out so that ThreadA can run again; eventually, ThreadA will release the lock, which again will priority boost ThreadB, which preempts ThreadA so that it can run. As you can see, there's a lot of wasteful context switching going on.

Priority Inversion
Modifying thread priorities is usually asking for trouble. Priority inversion can arise when many threads at different priorities share access to the same locks and resources, whereby a lower-priority thread actually indefinitely holds up the progress of a higher-priority thread. The moral of this story is to simply avoid changing thread priorities wherever possible.
Here is an extreme example of priority inversion. Imagine ThreadA at Low priority acquires some lock L. Then comes along ThreadB, which is at High priority. It tries to acquire L, but cannot because ThreadA holds it. This is the "inversion" part: it's as if ThreadA is artificially and temporarily given a higher priority than ThreadB, simply because it holds a lock that ThreadB needs.
The situation will eventually resolve itself when ThreadA releases the lock. Unfortunately, imagine what happens if ThreadC at Medium priority comes into the picture. Although ThreadC doesn't need lock L, its mere presence may prevent ThreadA from running at all, which indirectly prevents the High-priority ThreadB from running.
Eventually the Windows Balance Set Manager thread will notice this situation. Even if ThreadC remains runnable forever, ThreadA will eventually (after four seconds) receive a temporary priority boost by the OS. Hopefully this is enough to allow it to run to completion and release the lock. But the delay here (four seconds) is huge, and if there is any user interface involved, the application's user is certainly going to notice the problem.

Patterns for Achieving Safety
Now that I've chased down issue after issue, the good news is that there are design patterns you can follow to make the occurrence of many of the above issues (particularly the correctness hazards) far less frequent. The crux of most issues is that state is shared among multiple threads. What's worse, this state is freely manipulated, and it transitions from a consistent state to an inconsistent one and then (hopefully) back again with surprising regularity.
As developers write code for single-threaded programs, all of this makes sense. It's easy to use shared memory as a sort of scratch pad as you make your way towards a valid and final destination. C-style imperative programming languages have worked this way for quite a few years.
But with the rise of more and more concurrency, you need to pay closer attention to these habits. And you can take a cue from functional programming languages like Haskell, LISP, Scheme, ML, and even F# (a new .NET-compliant language), by adopting immutability, purity, and isolation as first class design concepts.

Immutability
An immutable data structure is one that doesn't change after construction. This is a wonderful property for concurrent programs because if data isn't being mutated, then there is no risk of conflict if many threads access it at once. That means synchronization isn't a concern.
Immutability has some support in C++ by way of const, and in C# by way of the read-only modifier. For example, a .NET type that has only read-only fields is shallow immutable. F# creates shallow immutable types by default, unless you use the mutable modifier. Going a step further, if each of those fields itself refers to another type whose fields are all read-only (and only refers to deeply immutable types), then the type is deeply immutable. This results in an entire object graph that is guaranteed to not change out from underneath you, which is very useful indeed.
All of this describes immutability as a static property. It's also possible for objects to be immutable by convention, meaning that there is some guarantee that state won't change during certain periods of time. This is a dynamic property. The Windows Presentation Foundation (WPF) freezable feature implements precisely this, and it also allows concurrent access without synchronization (although it cannot be checked in the same way that static support can). Dynamic immutability is often useful for objects that transition between immutable and mutable throughout the duration of their lifetime.
Immutability does have some downsides. Namely, whenever something has to change, you will need to make a copy of the original object and apply the changes along the way. Additionally, cycles in an object graph are generally not possible (except with dynamic immutability).
For instance, imagine you have an ImmutableStack<T>, as shown in Figure 4. Instead of a set of mutating Push and Pop methods, you will need to return new ImmutableStack<T> objects from them that contain the applied changes. Clever tricks can be used (as with the stack) in some cases to share memory among instances.
Figure 4 Using ImmutableStack<T>
public class ImmutableStack<T> {
    private readonly T m_value;
    private readonly ImmutableStack<T> m_next;
    private readonly bool m_empty;
    public ImmutableStack() { m_empty = true; }
    internal ImmutableStack(T value, Node next) {
        m_value = value;
        m_next = next;
        m_empty = false;
    }
    public ImmutableStack<T> Push(T value) {
        return new ImmutableStack(value, this);
    }
    public ImmutableStack<T> Pop(out T value) {
        if (m_empty) throw new Exception("Empty.");
        return m_next;
    }
}
As nodes are pushed, you have to allocate a new object for each one. In a standard linked-list implementation of a stack, you'd have to do this anyway. Notice that as you pop elements from the stack, however, you can use the existing objects. That's because each node in the stack is immutable.
There are already immutable types running rampant in the wild. The CLR's System.String class is immutable, and there is a design guideline that all new value types should be immutable, too. The guidance being given here is to use immutability where is it possible and feels natural, and to resist the temptation to perform mutation simply because it is made convenient by the current generation of languages.

Purity
Even with immutable data types, most operations performed by a program are method calls. And method calls can have side-effects, which are problematic in concurrent code because a side-effect implies mutation of some form. Most often this will be a simple write to shared memory, but it can also be a physically mutating operation like a database transaction, Web service invocation, or file system operation. In many circumstances, I'd like to be able to invoke some method without fear that it will lead to concurrency hazards. Good examples of this are simple methods like GetHashCode and ToString on System.Object. Most people wouldn't expect them to entail side-effects.
A pure method can always be run in a concurrent setting without added synchronization. Although there is no common language support for purity, you can define a pure method very simply:
  1. It only reads from shared memory, and it only reads immutable or constant state.
  2. It can, of course, write to local variables.
  3. It may only call other pure methods.
The set of things possible within a pure method, thus, is extremely limited. But when combined with immutable types, purity can be made possible and convenient. Some functional languages assume purity by default, most notably Haskell where everything is pure. Anything that must perform a side-effect has to be wrapped in a special thing called a monad. Most of us don't use Haskell, however, so we'll have to get by with purity-by-convention.

Isolation
Publication and privatization were mentioned in passing earlier, but they strike at the heart of a very important issue. Synchronization is necessary—and immutability and purity are interesting—because state is ordinarily shared among multiple threads. But if state is confined within a single thread, then there is no need for synchronization. This leads to more naturally scalable software.
Indeed, if state is isolated, mutation can happen freely. This is convenient because mutation is a fundamental built-in aspect of most C-style languages. Programmers are accustomed to it. It takes discipline to program in a mostly functional style, and is quite difficult for most developers. So give it a try, but don't deceive yourself into thinking the world will become functional overnight.
Ownership is a difficult thing to track down. When does an object become shared? When it is initialized, that is being done by a single thread and the object itself is not reachable from other threads just yet. Once a reference to that object is stored in a static variable, someplace that has been shared at thread creation or queuing time, or a field of an object transitively reachable from one of these places, the object becomes shared. It is imperative that developers watch specifically for these transitions between private and shared, and treat all shared state with care.

Joe Duffy is the Development Lead for Parallel Extensions to .NET at Microsoft. He spends most of his time hacking code, overseeing the design of the library, and managing an amazing team of developers. His new book is Concurrent Programming on Windows.


ASP.NET AJAX 4.0
New AJAX Support For Data-Driven Web Apps
Bertrand Le Roy

Code download available at: ASPNETAJAX42008_10.exe (188 KB)
Browse the Code Online

This article is based on prerelease versions of ASP.NET. All information herein is subject to change.

This article discusses:
  • Server-side data manipulation
  • UpdatePanel and the client
  • Reducing postbacks and payloads
  • Client-side template rendering
This article uses the following technologies:
ASP.NET AJAX 4.0
Contents
A Postback-Based Master-Details Page
The UpdatePanel Version
The Pure AJAX Version
The AJAX Version Templates
Event Bubbling
Managing the Back Button
The Finished Product
AJAX is an exciting Web platform for many reasons. Using AJAX, many tasks that were traditionally performed on the server happen in the browser instead, resulting in fewer round-trips to the server, lower bandwidth consumption, and faster, more responsive Web UIs. While these outcomes are the result of offloading a good deal of work to the client, the browser still isn't the environment of choice for many developers who would rather have the full power and flexibility of server apps at their disposal.
The solution employed until now has involved the UpdatePanel control, which has allowed developers to build AJAX applications while still retaining the full array of server tools. But UpdatePanel carries a lot of weight from the traditional postback model—an UpdatePanel request is still a full postback. In fact, using UpdatePanel, the whole form (including ViewState) is posted to the server, almost the entire page lifecycle gets executed there, and the rendering still happens on the server. Obviously, this method defeats one of the main reasons for moving to AJAX. The only real savings here are that XmlHttpRequest is used instead of a regular HTTP POST request and only updated parts of the page and the ViewState are sent back to the client. Thus, the response is much smaller, but the request isn't.
A pure AJAX approach will almost always perform better than the UpdatePanel approach. In a purely AJAX solution, the rendering happens on the client and the server sends back only data, which is usually much smaller than the equivalent HTML. This approach can also substantially reduce the number of network requests: having the data on the client allows much of the application's UI logic to run in the browser.
The main problem with the pure AJAX approach, however, is that the browser lacks the tools to turn data into HTML. Out of the box, it has only two crude facilities for doing so: innerHTML, which replaces the full contents of an element with the HTML string you provide, and the somewhat slower Document Object Model (DOM) APIs that operate on tags and attributes (similar in terms of abstraction level to HtmlTextWriter).
In this article, I'll show three iterations of a page written with classic postback, then with UpdatePanel, and then using pure AJAX to illustrate how techniques employed on the server can sometimes perform better on the client. The first two examples can be built today with the publicly available ASP.NET 3.5 SP1, while the third version will use some of the new client features in ASP.NET 4.0.

A Postback-Based Master-Details Page
The page I'll be building will show products in a list which, when selected, will display the detailed description of that product in a panel to the right of the list. I'll use the AdventureWorks sample database, which you can download from go.microsoft.com/fwlink/?LinkId=124953. I'll create only a rudimentary data layer using LINQ to SQL because data layers are not the focus of this article.
I'll start by adding the AdventureWorks .mdf file into the App_Data folder of the application. Then, I'll simply add a new "LINQ to SQL classes" .dbml file and drop the Product, ProductPhoto, and ProductProductPhoto tables and the vProductModelCatalogDescription view from the server explorer onto the design surface. The resulting data layer can be seen in Figure 1.
Figure 1 The Data Layer Architecture
The page will consist of two view panes, one for the list of products and the other one for the product details. Figure 2 shows the rendered page. On the first request to the page, the product list is bound to data using this code:
private void BindProductList() {
    ProductList.DataSource = from p in AdventureWorksContext.Products
                             where p.ProductSubcategoryID == 1
                             //Mountain bikes
                             orderby p.Name
                             select p;
    ProductList.DataBind();
}
Figure 2 he List of Bikes and Details
The list itself and its template can be seen in Figure 3. The code that queries the database can be found in the downloadable project and is fairly straightforward; it just queries the database for products in the Mountain Bike category and binds the ListView control to it. Of course, I could have used a data source control to do that same job but I find the code approach more flexible and predictable. Your results may vary if you are more of a design-view person.
The actual work of creating HTML markup from the dataset is all taken care of by the ListView control, which is very convenient and effortless. All I have to do is provide the template for that HTML in the LayoutTemplate and ItemTemplate properties (see Figure 3). This template will render the list of products as links inside an unordered list (UL and LI tags).
Figure 3 Unordered List of Products
<asp:ListView ID="ProductList" runat="server"
    DataKeyNames="ProductId"
    OnSelectedIndexChanging="ProductList_SelectedIndexChanging"
    OnSelectedIndexChanged="ProductList_SelectedIndexChanged">
    <LayoutTemplate>
        <ul ID="itemPlaceholderContainer" runat="server">
            <asp:PlaceHolder ID="itemPlaceholder" runat="server" />
        </ul>
    </LayoutTemplate>
    <ItemTemplate>
        <li><asp:LinkButton runat="server" ID="Select" CommandName="Select"
                                         Text='<%# Eval("Name") %>' /></li>
    </ItemTemplate>
</asp:ListView>
Note that the links themselves are not plain links but rather LinkButton controls, which means that they will post back to the page rather than navigate to a different one. They are effectively links that have the semantics of buttons. Of course, one could easily improve the accessibility of the page in the absence of JavaScript by replacing those LinkButton controls with appropriately styled regular Button controls.
The key feature of the LinkButtons that I'm using here is that instead of attaching an event handler to each of the buttons, I'm setting the CommandName property to "Select." The result will be that when clicked, the button will bubble the command up the control tree until it is handled by a control that understands it. This is a very powerful feature that enables an arbitrary UI element to send commands to its parent controls without having to know much more than what command and arguments it may expect. That's what allows for powerful data controls such as ListView to still enable complete developer control over the markup. You will see how this translates to similar browser concepts when I build the pure AJAX version of this same page.
At this stage, I have a product list that supports selection without having written any code. One could continue down this no-code path using DataSource controls and ControlParameters to tie the selected data key of the list to the selected data key of the details view, but I chose to do it in code. Next, I'll handle the SelectedIndexChanged event of the list and call the BindProductDetails method with the relevant product ID:
protected void ProductList_SelectedIndexChanged(object sender,
                                                 EventArgs e) {
    var productId = (int)ProductList.SelectedDataKey.Value;
    BindProductDetails(productId);
}
BindProductDetails queries the database for the product information and photos and then binds these to the corresponding controls in the details view.
The photos are being served by a simple handler that queries the database for the image bytes and copies them onto the response's binary stream (see Figure 4). This handler will be used by each of the three versions of the page. I now have a data-driven master-details view of my products, written entirely with a mix of imperative and declarative server code, but there are several things that could be improved.
Figure 4 Get Product Photo
public void ProcessRequest (HttpContext context) {
    int id;
    if (int.TryParse(context.Request.QueryString["id"], out id)) {
        context.Response.ContentType = "image/gif";
        AdventureWorksDataContext dc = new AdventureWorksDataContext();
        var bytes = dc.ProductPhotos
            .Where(p => p.ProductPhotoID == id)
            .Single().LargePhoto;
        context.Response.OutputStream.Write(bytes.ToArray(), 0, bytes.Length);
    }
    else {
        throw new HttpException(404, "Image not found");
    }
}
This is a very typical Web Forms page in that it is stateful in terms of volume of the state. A quick check on the rendered source in the browser shows about 4KB of ViewState due to the fact that the different views remember all of their internal state and carry it around with every postback.
At the same time, the page does not present clues about what its current state is; no matter what you do on the page, the URL in the navigation bar of the browser remains "1_WebForm.aspx." If a user bookmarks the page, he will always initially see the page without any product detail.
This could be fixed in this simple example by changing the links in the product list to be regular links instead of LinkButton controls and replacing the selection postbacks with plain navigation to a details page (I could even turn ViewState off and save about 4KB twice per round-trip). If you've followed the recent development of the ASP.NET Model View Controller (MVC) library (see msdn.microsoft.com/magazine/cc337884), you've probably guessed that this is a typical case where the MVC approach makes a lot of sense.
Navigation-based approaches also greatly improve the searchability of the site (which could be the subject of an entirely different article). That said, typical data-driven applications will be much more complex than this simple example and plain navigation often isn't the right way to build the UI flow. Therefore, even with this simple application, I will spend time illustrating how postback and Web Forms concepts translate and are improved with AJAX.
Both postback and link navigation might also be more disruptive to the user experience than you would like: during both postback and navigation, the UI is frozen and no other user interaction is possible until the server responds with new content, which then needs to be rendered to replace the whole document, sometimes losing subtle pieces of state, such as the scroll position.
Another problem is that users have very specific expectations about how the Back button and the history should work. Unfortunately, in the postback model, you have little or no control over what gets into the history or what happens when the user presses Back, Forward, or Refresh. Ideally, what constitutes a change of state and creates a history point should be in the hands of the developer, but in a postback application, almost any user interaction will create an entry in browser history.

The UpdatePanel Version
An easy way to improve this page is to use an UpdatePanel. An UpdatePanel will allow you to delimit the parts of the page that change when user actions would normally cause a postback. In this very simple case, the region of the page that I'll want to update is the details view. To enable UpdatePanel partial updates, I just need to add a ScriptManager control to the page, right after the form tag, like so:
<asp:ScriptManager ID="ScriptManager1" runat="server"/>
I also must add the UpdatePanel itself, around the details view:
<asp:UpdatePanel ID="UpdatePanel1" runat="server" RenderMode="Inline">
    <Triggers>
        <asp:AsyncPostBackTrigger ControlID="ProductList"
            EventName="SelectedIndexChanged" />
    </Triggers>
    <ContentTemplate>
        <div class="float" id="productDetails">
            <fieldset>
            ...
            </fieldset>
        </div>
    </ContentTemplate>
</asp:UpdatePanel>
Notice that this UpdatePanel has a trigger that watches the SelectedIndexChanged event of the product list. This isn't necessary when all controls that might trigger a partial update are themselves inside the UpdatePanel, but here the product list should remain outside of the UpdatePanel because its rendering doesn't need to be updated when its SelectedIndexChanged event occurs. If I didn't provide the trigger, a regular postback would occur instead of the partial update. Also, when using UpdatePanel remember to always ID all postback controls that may trigger a partial update. Forgetting to do this can lead the page to use regular postbacks for no apparent reason.
That's all there is to transforming a classic postback Web Form that has the appearance of AJAX. But it doesn't seem like I really have all the features I want. One problem is that I just lost what little support there was for the Back button. Now if the user goes back after browsing half a dozen bikes, he's going to go back to whatever site he visited before yours. Pressing the Forward button when he realizes this will take him back to the default state of the application (in this case, a list of products where none is selected).
Fortunately, ASP.NET 3.5 SP1 provides a simple way to return Back button support to the page. ScriptManager now has a very convenient EnableHistory property, an AddHistoryPoint method, and a Navigate event that together enable the application developer to control browser history far beyond anything regular postbacks allowed. This feature not only brings back what was lost by using UpdatePanel, but does so in a much more powerful form.
The big difference from regular postbacks is that I can now decide exactly what constitutes a change in the state of the application and filter out any user interaction I deem less important. I can also gain "bookmarkability" and ensure readable and meaningful entries in the browser's history dropdown.
In order to add history management to the page, I need to determine what information one would expect to be preserved when using a bookmark. Here, there's only one relevant piece of information that should enter state: the currently selected product's ID.
I'll need to intercept any event that is going to change that state. Actually, the only event I need to handle is the same one I used as a trigger earlier: the SelectedItemChanged event on the product list. I'm already handling that event to re-bind the details view, so I'll just add some code to create a new history point every time the event is raised:
protected void ProductList_SelectedIndexChanged(object sender,
                                                       EventArgs e) {
    var productId = (int)ProductList.SelectedDataKey.Value;
    var product = BindProductDetails(productId);
    if (ScriptManager1.IsInAsyncPostBack
                                   && !ScriptManager1.IsNavigating) {
         ScriptManager1.AddHistoryPoint("product",
           productId.ToString(), "AdventureWorks - " + product.Name);
    }
}
To ensure that the event wasn't raised as a result of the user navigating back to a previous state of the application, the code checks that the request is a part of an asynchronous postback and that it is not part of a navigation operation. If I didn't perform this check, I would create a new history point that would overwrite any forward history that may exist on the browser.
Once this check has been done, it is safe to call the AddHistoryPoint method, passing in the single piece of state that I care about, the product ID, with the parameter name "product." This name is what's going to be used in the modified URL, as you'll see. The value itself needs to be transformed into a string. Think of the history state as another form of query string. The last piece of information I'm giving the method is the document title. This is a great opportunity to make the user experience better as the user will be able to see meaningful information in the history navigation dropdown that will help him navigate the application (see Figure 5).
Figure 5 The History Dropdown
The way this state gets persisted is through the browser URL's hash (the part after the # sign that was originally designed to enable intra-document navigation). The reason for using that as the storage medium is that it's the only thing that enables adding a history entry without actually navigating away from the page and its JavaScript and DOM states (because the browser will only allow the addition of another history item if the URL changed). With that comes the constraint that you're storing state in the URL, which has limited space available. Some browsers may reject URLs larger than 1KB. If you need more than that, it may mean that you didn't select the relevant pieces of information as the state and you'll need to do some refactoring.
Notice how I use the ID of the product, which is a relatively small piece of data, and not its full name, which may be more friendly but will usually be larger. Insufficient space in the URL may also indicate that regular Web Forms and ViewState may be more appropriate for your design than AJAX and history would be.
The second half of the puzzle is that the state that I just saved needs to be restored when history is navigated. I'll do that by handling the Navigate event on ScriptManager (see Figure 6). In the code, I first handle the case where there is no state. This is to return to the default state of the page when going all the way back to the GET request. This may seem a little unusual if you don't know that the state is actually restored by doing a new postback. In this case, the "before" state of the postback, the one that gets restored by the framework automatically, is the "after" state in the browser history chronology, so I have to erase that restored state and substitute it with the default state.
Figure 6 Handling the Navigate Event
protected void ScriptManager_Navigate(object sender, HistoryEventArgs e) {
    var productIdString = e.State["product"];
    if (productIdString == null) {
        ProductList.SelectedIndex = -1;
        ProductDetails.DataSource = null;
        ProductDetails.DataBind();
        ProductModelDetails.DataSource = null;
        ProductModelDetails.DataBind();
        ProductPhotoList.DataSource = null;
        ProductPhotoList.DataBind();
        Page.Title = "AdventureWorks";
    }
    else {
        var productId = int.Parse(productIdString);
        var product = BindProductDetails(productId);
        ProductList.SelectedIndex = (
            from p in AdventureWorksContext.Products
            where p.ProductSubcategoryID == 1 // Mountain bikes
            orderby p.Name
            select p).ToList().IndexOf(product);
        BindProductList();
        Page.Title = "AdventureWorks - " + product.Name;
    }
}
Second, the state itself should be considered user-provided and should thus be validated, which I'm doing by parsing it as an integer. And finally, I'm restoring the title of the page when restoring the state, in addition to resetting the list and details states.
If you use the page after those changes, the URL in the browser will change every time a product is selected and will look something like this:
http://MyServer/MSDNAjax/2_UpdatePanel.aspx#&&5YLQHC81D2
OEdJU/9ZBdHUip1qx3ooPKDhCLgKogupQ=
It's ugly and not very readable, right? This is because the framework considers user-provided data dangerous by default, so it hashes the state in order to prevent tampering. However, in many cases, the developer will prefer relatively readable, less scary URLs, even if that means validating the state from code and allowing the user to tamper with it. Tamperable URLs are even considered a plus in some situations (the MSDN library is a good example of this; it allows users to build their own URLs, such as msdn.microsoft.com/library/system.web.ui.scriptmanager.aspx, from a predictable scheme because it makes navigation much easier).
In order to enable this type of scenario, ScriptManager exposes the EnableSecureHistoryState Boolean property. Just by setting this to false, my URLs become much friendlier, like this one:
http://MyServer/MSDNAjax/2_UpdatePanel.aspx#&&product=776
The result is a much more fluid page that isn't just made to look like an AJAX version of the Web Form, but one that has many handy additional features such as bookmarkability and optimal handling of the Back button. And all this was accomplished without writing a single line of JavaScript.

The Pure AJAX Version
While there is a lot to like in the UpdatePanel version of the page, I'm still dragging around the weight of the ViewState. In order to trim it down, I need to transfer more logic to the client. To do so, I'll need to start writing some fun JavaScript.
You could very well write a pure AJAX version with ASP.NET AJAX 3.5 SP1, but it would be tedious to take the data and format it as HTML. There are two basic ways you can turn data into HTML.
The first, which is what most client template engines do, is to concatenate strings alternating static template contents with the dynamic data contents. This seems relatively straightforward and fast since it uses innerHTML as its sole method of interaction with the DOM. It has a few problems though.
One thing it doesn't do well is protect from injection attacks: if you're going to generate HTML by concatenating strings, you'll need to encode all data before using it. Otherwise, introducing a quote into an attribute or a script tag into a text node, maliciously or inadvertently, might result in arbitrary code execution (which is bad). Encoding is harder than it seems because you might need different encoding algorithms depending whether you're injecting a text attribute, a URL attribute, or a text node.
A template engine also needs an expression language; while it is easy to inject plain data fields without modifications, this is only the simplest scenario. Often, you'll need to apply a format string, combine multiple fields and more generally manipulate the data before displaying it. This could be achieved by transforming the data before feeding it into the template, but it is easier and more efficient if that ability is built into the template engine. Once you start adding features such as formatting, you very quickly find you need the flexibility of a full expression language.
Being able to intersperse code and markup (as you can with <% %> blocks in ASP) enables interesting scenarios such as repeating markup using a loop around an HTML fragment or conditional rendering using a simple if statement. Again, this scenario requires a full language to be really useful.
Finally, HTML is only half of the story. AJAX applications really are about active content, not just client-side updates to the DOM. Once you've generated HTML, you still have to hook events to elements and attach controls and behaviors. You can do that through code after the HTML generation, but this creates an unpleasant dissymmetry between the HTML, which becomes very easy, and the logic, which becomes more difficult and requires knowledge of the template's structure.
In other words, in order to attach behavior, you need to know where to attach it, which in turn means that any change in the structure of the HTML template markup will require changes in the code that activates it. There are ways to make that coupling looser but an even nicer solution would be to make content activation part of the template engine.
The other way you can generate HTML from data is to manipulate DOM APIs directly and create elements, attributes, and text nodes from code. At first, this looks like a bad option for several reasons. Sure, it has the advantage of being agreeable to standards zealots, but for some strange reason it's a lot slower than innerHTML. But the main reason this is not commonly used is that the DOM APIs are not very expressive, and the resulting code is hard to read and harder to maintain—at least without any help and additional abstractions. There are toolkits such as jQuery that provide excellent abstractions and make the whole process a lot more fun, but even with such tools, it's harder than it needs to be (which is why even jQuery has several template plug-ins).
Some of you may know that Microsoft already had a template engine in ASP.NET AJAX Futures, but it was too slow and complex in design and we wanted to do a lot better. The good thing about this failed first attempt is that we learned a lot about what we didn't want the new version to be (that is, slow and complex).
The dev team tested many different designs for a new template engine for ASP.NET AJAX, from string concatenation to full DOM manipulations, and we evaluated them on performance, simplicity, and flexibility. We also compared them in terms of what scenarios they prevented from happening. There is no ideal solution, but we chose the one that seemed like the better compromise.
The principle of the new engine is simple: we take your template code, which contains HTML, data fields, expressions, declarative component instantiation and imperative code, and we turn that auto-magically into JavaScript that creates the equivalent HTML. That looks simple enough and it is (well, until browser quirks appear). We do pay a performance penalty for using the DOM APIs, but if we're careful and build elements outside of the DOM, and add as few as possible as late as possible, the performance hit is not too big and the amazing flexibility it creates is well worth the trade-offs. All the problems in the string concatenation approach seem to go away naturally.
Injection attacks take care of themselves. As I'm using code to create text nodes and set attribute values, there is no need for encoding because the APIs I use are already safe. This is analogous to using SQL parameters versus building SQL by concatenating strings. Nobody in his right mind does the latter anymore so why would you want to take a similar risk here?
Now who needs a new expression language? We already have one: JavaScript. When you transform the template markup into JavaScript code, what could be easier than injecting JavaScript expressions into that code you're generating?
To enable the most common template development tasks, one-time, one-way injection of data fields (which on the server is expressed by "<%= expression %>" and in our system is "{{ expression }}"), I use a feature of JavaScript that is often despised—the "with" keyword. It saves me from having to resort to expressions such as "{{ dataItem.myField }}" to inject a field of the data item associated with the template instance. Thanks to the "with" keyword, you can surround the generated code for the template with something like "with(dataItem) {…}" so that any member of the data item is promoted to the top scope of the template function, making the injection of expressions as simple as "{{ myField }}".
You can inject behavior into the template in two ways. First, you can write $attachEvent and $create code from the itemCreated event, or inline in the template using a special $element variable that is available from within the template and references the latest created element. Or you can use the declarative syntax we're providing. For example, if you want to add an autocomplete and a watermark behavior to an input tag, you'd write something like this:
<body xmlns:sys="javascript:Sys"
 xmlns:autocomplete="javascript:AjaxControlToolkit.AutoCompleteBehavior"
 xmlns:watermark="javascript:AjaxControlToolkit.extBoxWatermarkBehavior">
...
<input id="search" sys:attach="autocomplete,watermark"
 autocomplete:servicepath="SearchAutoComplete.asmx"
 watermark:watermarktext="Type your search terms here" />
Here I register the prefix for each declarative behavior on the HTML or body tag (or on the parent tag for the template) using the xmlns XHTML namespace declaration. This will enable me to extend the XHTML markup in a standard way, and it is similar to @Register directives for server code. The part after "xmlns:" is the prefix that is going to be associated with each behavior or control. The URL for the namespace is using the "javascript:" protocol to map the prefix to a specific JavaScript type. The "sys" namespace is a special system namespace that should be mapped to the Sys namespace, which is the root namespace in AJAX.
The instantiation itself is done through the special attribute sys:attach, whose value is a comma-delimited list of the prefixes of the behaviors or controls to instantiate and attach to the element. Then I can set properties for all of these behaviors without risking a conflict with regular HTML attributes or other behaviors on the same element because they are nicely differentiated by namespace.
One of the most elegant features of the engine is that the compilation of the template into JavaScript code really is analogous to a real compilation step. This means that it only has to happen once per template and that provides the opportunity to execute a number of tasks ahead of time instead of doing them each time the template is instantiated. But that's enough theory for now. How does this apply to our master/details page?

The AJAX Version Templates
The template for the list of products is quite simple:
<ul id="productListTemplate" class="sys-template">
    <li>
        <a href="{{ String.format('3_Client.aspx?product={0}',
        ProductID) }}">{{ Name }}</a>
    </li>
</ul>
The item template is a list item with a simple link in it. The text of the link is just the name of the product ("{{ Name }}") and the href attribute is a formatted string built from the product ID using plain JavaScript:
"{{ String.format('5_Client.aspx?product={0}', ProductID) }}"
The class "sys-template" is defined in the CSS for the page to hide the template from the initial rendering of the page. The compiled code for this simple template is shown in Figure 7. The details view is a little more complex and actually contains some inline code (see Figure 8). I could have used a nested template to render the list of photos, but it is a little simpler to use a regular loop over the markup for one photo. A nested template would have been justified if I were dealing with dynamically changing data and automatic reflection of the changes into the markup (which is a supported scenario, but outside of the scope of this article), but seeing that I'm dealing with one-way, one-time bindings here, inline code is fine.
Figure 7 Template-Compiled Code
function(__containerElement, $dataItem, $parentContext, __instanceId) {
   var __context = {}, $component, __app = Sys.Application,
      __creatingComponents = __app.get_isCreatingComponents(),
      __components = [], __componentIndex, __e, __f, __topElements = [],
      __p = [__containerElement], $index = __instanceId,
      $id = Sys.Preview.UI.Template._getIdFunction(__instanceId),
      $element = __containerElement;
   Sys.Preview.UI.Template._contexts.push(__topElements);
   with(__context) { with($dataItem || {}) {
      $element=__p[1]=document.createElement('LI');
      __topElements.push($element);
      $element=__p[2]=document.createElement('A');
      $component = $element;
      __e = document.createAttribute('href');
      __e.nodeValue = String.format('5_Client.aspx?product={0}',
                                                       ProductID);
      $element.setAttributeNode(__e);
      __p[1].appendChild($element);
      __p[2].appendChild(document.createTextNode(Name));
      $element=__p[2];
      __p[1].appendChild(document.createTextNode(" "));
      $element=__p[1];
   }
}
   for (var __i = 0, __l = __topElements.length; __i < __l; __i++) {
      __containerElement.appendChild(__topElements[__i]);
   }
Sys.Preview.UI.Template._contexts.pop();
 return new Sys.Preview.UI.TemplateResult(this, __containerElement, __topElements, __components);
}
Figure 8 Displaying the Details
<div class="sys-template" id="productDetailsTemplate">
    <fieldset>
        <legend>{{ Name }} ({{ ProductNumber }})
            {{ String.format("{0:C}", ListPrice) }}</legend>
        <ul class="photoList">
            <!--* for (var i = 0; i < Photos.length; i++) { *-->
            <li><img src="{{ String.format('productphoto.ashx?id={0}',
                Photos[i]) }}" /></li>
            <!--* } *-->
        </ul>
        <table>
            <tr><td class="label">Summary:</td><td>{{ Summary }}</td></tr>
            <tr><td class="label">Experience:</td>
                <td>{{ RiderExperience }}</td></tr>
              ...
            <tr><td class="label">Style:</td><td>{{ Style }}</td></tr>
            <tr><td class="label">Wheel:</td><td>{{ Wheel }}</td></tr>
            <tr><td class="label">Maintenance:</td>
                <td>{{ MaintenanceDescription }}</td></tr>
        </table>
    </fieldset>
</div>
The templates are lazily compiled the first time they are instantiated but they are prepared by creating a "new Sys.Preview.UI.Template" using the parent element of the template markup as the parameter of the constructor. The templates themselves are instantiated from the callback from the network call that brings the data back from the Web service on the server:
AdventureWorks.GetProducts(1 /* Mountain bikes */,
  function(productArray) {
    renderProductList(productArray, productListTemplate);
    selectProduct(initialProductID, true);
});

function renderProductList(productArray) {
    var target = $get("productList");
    target.innerHTML = "";
    for (var i = 0, l = productArray.length; i < l; i++) {
        productListTemplate.createInstance(target, productArray[i]);
    }
}
This will not be necessary in the shipping version of ASP.NET 4.0; its DataView component will take care of the template parsing, compiling and instantiating. Most of the code in this app will eventually go away, but it is useful in showing how things happen under the hood. It also shows how a component developer who wishes to include template rendering might use the feature.

Event Bubbling
Where do I put the code that displays the right details view when a user clicks on one of the products? The code, like its server-side equivalent, uses event bubbling, so I was able to write one single- click event handler for all the links in the list (so I could add or remove links to the list if I wanted to without having to worry about creating new handlers or cleaning up the old ones). The following code shows that handler. All click events for the links in the list will bubble up to the list itself and be handled there. "e.target" is a reference to the element that got actually clicked; in other words, it's the link, which enables me to retrieve the product ID from the href attribute and select the relevant product:
$addHandler($get("productList"), "click", function(e) {
    var href = e.target.href;
    selectProduct(parseInt(href.substring(href.indexOf('=') + 1), 10));
    e.preventDefault();
    e.stopPropagation();
});
Once that is done, the event's default action (the link navigation) is canceled and the event is prevented from bubbling up further. This is done by calling the W3C standard stopPropagation and preventDefault methods on the event object, which the framework makes available on all browsers, including Internet Explorer.

Managing the Back Button
The only feature that remains to be reproduced from the server-side version is history. Enabling history on the ScriptManager control also enables client-side APIs that are exactly analogous to the server-side APIs I've used before and that can even be used at the same time (enabling mixed client-server state management).
Creating history points is done by calling Sys.Application.addHistoryPoint from the events that correspond to a state change, in this case, clicking a product in the list:
Sys.Application.addHistoryPoint({product: productDetails.ProductID},
    "AdventureWorks - " + productDetails.Name);
Correspondingly, state is restored from the "navigate" event on Sys.Application. The HistoryEventArgs arguments that the event handler receives has one property—state—that enables you to retrieve the product to restore:
Sys.Application.add_navigate(function(sender, e) {
    var ProductID = parseInt(e.get_state()["product"], 10);
    selectProduct(ProductID, true);
});

The Finished Product
The resulting page behaves very much like the UpdatePanel version, but there is no comparison in terms of network traffic. When a product is selected, the UpdatePanel version sends more than 4KB of data to the server and receives back about 8KB. The pure AJAX version, on the other hand, sends only "{"productId":771}" plus standard HTTP headers and receives back 2KB of pure JavaScript Object Notation (JSON) data. That's about 10KB saved each time a user clicks on a product.
This is just one of the exciting features planned for ASP.NET 4.0. Share your thoughts at go.microsoft.com/fwlink/?LinkId=126987.

Bertrand Le Roy, Ph.D., is Program Manager in charge of AJAX at Microsoft. He spent five years as a developer in the same area. He also represents Microsoft at the OpenAjax alliance.


Easy Async
Build Concurrent Apps From Simple F# Expressions
Chance Coble

This article is based on prerelease versions of the Visual Studio add-in for F# and related technologies. All information herein is subject to change.

This article discusses:
  • Traditional and functional notation
  • Asynchronous expressions
  • Building and using primitives
  • Working with other .NET languages
This article uses the following technologies:
F#
Contents
The Problem of Traditional Notation
Functional Programming
Asynchronous Expressions
Using let! and return!
Primitives
Create Your Own Primitives
Primitives and Supporting Functions
Extending Async Calls to the Web
Working with Other .NET Languages
Asynchronous programming is considered so difficult by many programmers that they create synchronous version of programs that could run more efficiently as asynchronous applications just to avoid the assumed complexity of asynchronous code.
Until recently those choices had little consequence. However, with new opportunities in distributed computing such as multicore processors and service-oriented architectures, developers are beginning to feel the burden. A classic example of this is in I/O operations, where processor time could be employed more profitably while an I/O-bound operation is taking place. Plenty of applications exist where many files are read and processed or where a complex orchestration of Web services is called upon to retrieve data that is processed and then sent to another service.
In reality, an asynchronous program is a simpler arrangement than a synchronous one. An asynchronous program purposefully says nothing about the sequence of the commands to be executed. It is a lazy specification and thus should be simpler to define.
One stumbling block that developers encounter with asynchronous programming is that they become so concerned with getting concurrency right that they forget the core simplicity of the program. Locks, semaphores, and other concurrent computing tools tend to focus the program's layout less around its core logic and more around the concurrent management of its resources.
That is where you will see the value of asynchronous expressions in F#. You can separate simple programs from the concurrent control flow and reveal the simplicity and readability of your core program. Additionally, one of the most powerful aspects of the F# language is its ability to create libraries that can be called seamlessly from any other language compliant with the Microsoft .NET Framework, allowing you to leverage the intuitive parallelism provided by F# asynchronous expressions from within existing imperative C# or Visual Basic code.
In this article, I will demonstrate the essence of asynchronous expressions in F#. I will provide implementation examples and even show you how to incorporate your own frameworks into simple asynchronous primitives. Using the techniques I cover here, you will be able to create modular and expressive programs for your most challenging concurrency problems.

The Problem of Traditional Notation
Here's a traditional example of calling methods or functions:
commandOne();
commandTwo();
commandThree();
There is no question in this traditional notation that commandTwo is preceded by commandOne, and that it will be succeeded by commandThree only after it completes.
Trying to specify that you don't care what order these execute in is more difficult in this notation. It should be no more complex than a simple delimiter between them:
commandOne() | commandTwo() | commandThree()
Something like this could have been a useful notation for the three commands when you don't care what order they execute in. Perhaps even more useful would be a construct that indicates how you want them to run instead of a delimiter, such as this:
async {commandOne()}; async {commandTwo()}; async {commandThree()}
And suppose you want to set a value to the result of the first one that completes:
x = first(
     async { commandOne() };
     async { commandTwo() };
     async { commandThree()}; )
This seemingly simple instruction becomes a major effort in most popular paradigms.
From this perspective, the semicolon begins to look less like statement termination and more like an operator that blocks the execution of the following statement until the current statement has completed. That a synchronous program is a more specific definition of a program should be intuitive from this perspective.
Many advances in programming language design are shedding new light on the problem of asynchronous program design. By making it a programming language problem you immediately see ways to construct a notation that is much more expressive, clear, and concise. Using asynchronous expressions, the F# team has not only moved closer toward that goal, but has also facilitated a level of interaction between the programmer and the programming language to which the .NET community has not yet had access.

Functional Programming
You may have already seen a lot of F# in Ted Neward's primer (msdn.microsoft.com/magazine/cc164244). For the benefit of those new to this language, I will review a couple of key points, but only those required for an introduction to asynchronous expressions in F#.
This article uses the #light compiler directive for all code examples, even though it may not be explicit. That directive will cause the compiler to parse each line of code based on its alignment with the other code. While this results in a clear layout, it also makes white space an important part of the code body. When you explore with the code in this article yourself, be sure to follow the alignment used in the examples with spaces rather than tabs. To use the #light directive, place it above the code it is to influence:
#light
System.Console.WriteLine "Hello World"
All of the code included with this article can be compiled, but for easy execution consider using the F# Interactive Console. To start this console, open Visual Studio. Click View | Other Windows | F# Interactive.
An important aspect of F# is that everything is a value, even a function. By default, everything is an immutable value. For example, consider the following line of code:
let x = 2
This creates a value 2 that you can refer to as x. The value x cannot change to become something else later. Perhaps in another scope x could refer to something else entirely, but it cannot change in this scope. Values can also be created as the result of expressions. The following line would bind the result 5 to x:
let x = 2 + 3
You can almost begin to interpret the equals sign less as assignment in this setting, and more as an assertion that the right-hand side equals the left-hand side. This value setting also works for functions. If I want to write to the console frequently I can create an abbreviated value emit that represents the Console class's WriteLine method for strings:
let emit:string->unit = System.Console.WriteLine
The value emit now refers to the console method. After the colon there is a type signature for the function. This type signature says the function accepts a string and returns unit (unit is like void in C#). Again, the value emit isn't mutable and cannot represent anything else in this scope. So you can see that in F#, methods and functions are values.
Now that I have set a value to a function, it would be nice to know how to invoke them. After all, that is how we actually run programs, isn't it?
In functional programming, we use the term application rather than invocation. This is simply the application of functions to arguments. In F#, a function is not invoked until it has been applied to all of its arguments.
Applying a function to its arguments has a slightly different look in a functional setting. You apply functions to values so frequently in functional programming that the least intrusive operator is used to indicate application: the space. For example, to apply emit, which represents the function Console.WriteLine, you would write:
emit "Hello World"
That code would result in the following output:
Hello World
Alternatively, you could have written this:
emit("Hello World")
C# developers will sometimes find this flexibility confusing because it looks like their familiar method invocation. The parentheses are only present to group the arguments, and that is why the compiler allows the exclusion of the space. In this article, the convention will be to use parentheses only when required to group arguments and terms together.
Finally, some functions take one argument of void. In F# this is referred to as unit and is represented by the empty parentheses:
let emitter () = emit "Hello World"
emitter ()
This code results in the following output:
"Hello World"

Asynchronous Expressions
Now you are ready for a peek at an asynchronous expression. Figure 1 shows a synchronous program that reads text files in a directory and counts the space characters in each file. The function countSpace takes a file name and size and opens a stream to read in the appropriate number of bytes. It then counts every byte that is a space in a list-folding operation. Below that, the value "files" is the array of text files in that directory and the value "counted" computes the function countSpace for each of the files using Array.map.
Figure 1 Counting Spaces Synchronously
#light
open System.IO
open System
let countSpace filename size =
    let space = Convert.ToByte ' '
    use stream = File.OpenRead filename
    let bytes = Array.create size space
    let nbytes = stream.Read (bytes,0,size)
    let count =
     bytes
     |> Array.fold_left (fun acc x -> if (x=space) then acc + 1 else acc) 0
    count
let files = (DirectoryInfo @"C:\Users\Chance\Documents").GetFiles "*.txt"
let counted =
   files |>
   Array.map (fun f -> countSpace (f.FullName) (int f.Length))
One would expect, at the end of this program, an array of integers indicating the number of spaces in each file. So how much work is it to create a concurrent version?
Creating a concurrent version using asynchronous expressions means creating an expression from the function by wrapping it in braces preceded by the identifier async (see Figure 2). This expression is now of the type Async<int>, and that is the type aCountSpace returns when applied to its two arguments. The only other substantial difference from the synchronous version is that the stream calls ReadAsync instead of just read. This method is made available via the Microsoft.FSharp.Control.CommonExtensions namespace.
Figure 2 Counting Spaces Asynchronously
open Microsoft.FSharp.Control.CommonExtensions
#nowarn "057"
let aCountSpace filename size =
  async {
     let space = Convert.ToByte ' '
     use stream = File.OpenRead (filename)
     let bytes = Array.create size space
     let! nbytes = stream.ReadAsync (bytes,0,size)
     let count =
       bytes
       |> Array.fold_left (fun acc x -> if (x=space) then acc + 1 else acc) 0
     return count
     }
Now the function has to be applied to arguments to obtain the expression, and the expression must be executed. That is the real difference in obtaining the final array of counts. Like the calculation of the counted array, the Array.map function applies aCountSpace to each of the files. However, applying the function only returns an asynchronous computation (because aCountSpace returns an Async<int>) that can be executed when needed. The resulting array of asynchronous computations, then, is sent to the Async.Parallel primitive, which makes the array of asynchronous computations into an asynchronous computation which returns an array of results. The expression is then passed to the Async.Run function:
let aCounted =
    files
    |> Array.map (fun f -> aCountSpace (f.FullName) (int f.Length))
    |> Async.Parallel
    |> Async.Run
Async.Run performs the final magic of running the asynchronous computation and returning the result.

Using let! and return!
You may have noticed on the other side of the stream.ReadAsync call there was an exclamation point. That is no accident. The typical let binding creates a name for a value. Asynchronous expressions offer a new technique for binding to the results of other asynchronous expressions. This approach is used to create asynchronous computations that can be composed from other expressions. As I will illustrate later, that is the key value to asynchronous expressions.
You can think of an asynchronous expression in terms of the value that is delivered. This code can be thought of and treated as the integer 1:
async { return 1}
However, you run into a little trouble when you want to deal with the value that an asynchronous expressions yields, as in the following example:
let x = async {return 1}
let y = async { let res = x
  return 5 + res } // TYPE ERROR
Special consideration has to be given when performing composition of expressions. So you alter the operator slightly as a sign that you want to run this asynchronous expression at the time that the current expression is being run. You don't want to create an expression x; rather, you want to run the expression asynchronously and bind x to its resulting value. The type of res should be int, not Async<int> as in the previous examples.
The operator used to pierce the expression and access the underlying type is let!. Of course, the expression won't actually be executed until Async.Run is applied to the entire composition:
let x = async { return 1}
let y = async {  let! res = x
  return 5 + res } // WORKS!
Async.Run y
This results, of course, in the integer 6.
Similar to let! is return!, which is used to compose another expression directly into the return statement:
let x = async { return 1}
let y = async { return! x}
Not a very useful example yet, but it illustrates the point. When you are dealing with nested expressions, let! and return! allow you to avoid wasting intellectual effort on inventing ways to bind the expressions together. Instead, you just treat the expressions as the values they return. In fact, the above could just as well have been written using let!, as in the following:
let x = async {return 1}
let y = async { let! temp = x
        return temp }
This focus on compositionality is the key to the value added by this notation. It allows the programmer to focus on the basic program logic and to keep it separated from the description of the parallel nature of the computation.

Async.Run is a way to block, waiting for the asynchronous computation to complete. But the Async.Parallel example in the code shown earlier still hasn't been fully explained. It is a function that can be applied to an array of asynchronous expressions, and it returns an asynchronous expression on an array. Async.Parallel is an asynchronous primitive, a family of functions that are designed to glue the new execution logic into asynchronous expressions while taking advantage of built-in exception handling and compositionality.
The idea of primitives is to control the way an expression is executed when it is finally applied. For example, an asynchronous expression such as the one shown here won't actually run asynchronously until the execution flow logic been plugged in to run concurrently with other processes:
let one = async { return 1 }
The key to great primitives is that they manage a computation asynchronously in a reusable pattern. You can apply Async.Parallel to an array of asynchronous expressions (Async<'a> array) and it returns an asynchronous expression that returns an array of the resulting type (Async<array<'a>>). When executed, each of the expressions in the array is executed in parallel. Async.Parallel executes each member on a separate thread and handles all of the details of managing each thread's result in the corresponding element of the returned array.
Async.Parallel is a great example of a reusable execution approach for asynchronous expressions. In particular it is useful when the number of elements in the array corresponds to the number of processor cores on the machine. (Though you don't need to worry about aligning these—the thread pool will create and manage the correct number of threads and schedule the asynchronous work as appropriate.) To generate an array quickly, you can use the [| … |] generation notation in F#:
let nums = [|async {return 1 + 1};
            async {return 1+ 2};
            async {return 1 + 3}|]
  Async.Run (Async.Parallel nums)
Another primitive you saw previously is the ReadAsync (an asynchronous version of Stream.Read) in the stream that reads in file data. That primitive (when the Microsoft.FSharp.Control.CommonExtensions namespace is open) is available on streams that read data. The function is applied to a buffer array, a starting point, and a length. It returns an Async<int> indicating the number of bytes read. Of course, all of this is only executed when an execution function such as Async.Run or Async.Spawn is applied to the entire expression.
These primitives are the building blocks of composition. And when they are structured appropriately, you can modularize and separate the control flow details of a parallel computation away from the actual program logic. The representation of the program is abstract enough that you can begin to use new functions as computational devices that execute those programs in different ways.
Both of these primitives are asynchronous expressions, just like the ones created earlier. But when Async.Run (or another execution function such as Async.Spawn) is applied to them, they will execute, handling the details of their asynchronous approach. The alignment of these computation types as asynchronous expressions creates an expressive and compositional architecture with which you can choose how to run your programs. And this is the way to build large complex systems: by gluing together programs with simple primitives.

Create Your Own Primitives
The primitives used to stitch different expressions together are already well thought out and handle most of the logic you will want to create. While powerful libraries are a sure way to add simplicity for many tasks, much value would be added by developing highly customized solutions to meet particular needs in specific cases. Asynchronous expressions deliver the opportunity to add custom primitives when the need arises. In this section I'll walk you through an example implementation of a new asynchronous primitive.
One of the things that is exceptionally difficult in most imperative languages is to short-circuit I/O requests as a result of another call's completion. Consider the C# if statement:
if ( a || b) {Console.WriteLine("A and B were true");}
  else { Console.WriteLine("Either A or B was false");}
If a is true, then b is not checked and the first statement is executed. That kind of short circuit in an I/O operation, where the goal is to get the first successful result, could be really helpful if redundant data sources are being called on. In such a case, only the first data source to return a value needs to be checked. It doesn't matter if that data source is a database, flat file, Web service, or external sensor.
This is not a trivial undertaking in an imperative language. Any attempt would run the risk of getting bogged down in the concurrency issues, which is a result of failing to separate the concurrency control from the simple expression of retrieving the first data available. Issues such as exception handling would further exacerbate the complexity.
As an example, let's send a request for data to a Microsoft TerraService database. I first reference the TerraService.dll file in the WebReferences namespace:
#light
#r "TerraService.dll"
#nowarn "57"
open System
open System.Xml
open System.Xml.Serialization
open System.IO
open Microsoft.FSharp.Control
open Microsoft.FSharp.Control.CommonExtensions
open WebReferences
As mentioned, all examples use the #light compiler directive. And, since asynchronous expressions are to date an experimental feature, I skip the warning by including #nowarn "57". I will also need to open several namespaces, including the WebReferences namespace to provide access to the TerraService Web service proxy.
The next step here is to set up a couple of types using discriminated unions:
type 'a Result =
   | Complete of 'a
   | Failure of 'a

type 'a RetrievedValue =
   | File of 'a
   | Web of 'a
   | Timeout
   | Error

let ts = new TerraService()
The Result type provides a description of the computation's termination, and the RetrievedValue describes the source, and possibly error, of the TerraService call. I will be caching results to the file system, hence both File and Web are included as patterns in the union. Finally, I instantiate a new TerraService type.

Primitives and Supporting Functions
Most F# code is best read from the inside out and from the bottom up. In Figure 3, the first thing to understand is the function first, which is the actual asynchronous primitive. The only job of first is to create an asynchronous primitive that executes on a list of asynchronous expressions that happen to return one of the Result types in the discriminated union shown earlier.
Figure 3 updateAndContinue
let updateAndContinue n i cont res =
     lock n (fun () -> if !n = -1 then n:=i)
     if (!n=i) then cont res

let aContinueFirstSuccess cont n i x =
    async {
      let! y=x in
      do match y with
         | Complete(res) -> updateAndContinue n i cont res
         | _ -> ()
     }

let continueIfFirst cont n i x =
   Async.Spawn (aContinueFirstSuccess cont n i x)

let first (ls:Async<'a Result> List) =
  Async.Primitive
    (fun (cont,exn) -> let n = ref (-1) in
List.iteri (continueIfFirst cont n ) ls)
Like all asynchronous primitives, the core section is a function that takes a continuation function (cont) and an exception function (exn). Depending on what happens in the expressions computation either one could be called, but it is up to the developer to define under what circumstances each branch is taken.
The function first creates a reference to an integer with an initial value of -1. Reference cells in F# are one way to create mutable values. They can be assigned to, much like variables in C# with the := operator. Their values are accessed by preceding the name with an exclamation point (as in the updateAndContinue function in Figure 3).
The negative one will be a flag that the reference remains unset. Next, the list of asynchronous expressions is iterated over using the List.iteri construct and continueIfFirst is called on every expression with cont and the integer reference n. List.iteri, unlike List.iter, will apply the integer index of the element of the list being iterated, which is used to identify each executed expression.
The function continueIfFirst, just above the function first, spawns a new thread and fires the asynchronous expression aContinueFirstSuccess with the continuation function cont, the integer reference, the index in the list of the asynchronous expressions (from ls in first) and the expression to be executed.
The asynchronous expression aContinueFirstSuccess, now executing in a separate thread for each element in the original list of asynchronous expressions, executes the asynchronous expression in the current thread and checks its result. Recall that, in the implementation of the primitive, each asynchronous expression returns one of the Result types from the discriminated union bound to a generic type (represented by 'a). If the result is Complete, then the code updates and continues by calling updateAndContinue, passing it the same arguments with the exception of the original asynchronous expression. The function updateAndContinue ensures that only the first expression that updates n is able to continue. The reference cell n is updated in a thread safe manner using lock, and the value is subsequently checked to be equal to i. If the reference cell n and i are equal then this thread must have been the first to update n. The expression has been computed now, so I pass the result as res.
Ultimately, the primitive first and its few supporting functions provide the execution mechanism to spawn a list of asynchronous computations, and return the first successful result.
One of the most obvious things that can be created from this code is a timeout mechanism. After all, any time the first of a set of computations should be returned, there will almost always be a maximum time limit for which the user should have to wait.
Here are the two functions necessary to create a timeout. The first is a function called timeout that takes the value it is to return when that time (milliseconds) has passed:
let timeout i x =
  async {
    do System.Threading.Thread.Sleep(i:int)
    return Complete(x)
    }
The second one is a function called terminateInSeconds. Note that it is intended to be composed on top of an existing asynchronous expression:
let terminateInSeconds time zero expr =
  let success =
    async {
      let! res = expr
      return Complete(res)
    }
  first [ success; timeout (1000*time) zero]
Using first, it executes the expression it is given and returns the zero argument if the expression does not execute successfully in the allotted time.
A very simple use of the execution of a list of asynchronous computations would be to create a countdown. The countdown could be an asynchronous expression defined as follows:
let rec countdown x =
   async {
     do Console.WriteLine(x:int)
     do System.Threading.Thread.Sleep(1000)
     if x = 0 then return Complete("Done")
     else
      let! res =
       first [ countdown (x-1);
         timeout ((x-1) * 1000) "Launch"]
      return Complete(res)
    }
If you run this with the Async.Run primitive and pass it the argument 5, then you will get a countdown in 1-second intervals:
> Async.Run (countdown 5);;
5
4
3
2
1
0
val it : string Result = Complete "Launch"
Notice that the result was "Launch", not "Done". While the timeout fired in each case, it only terminated the asynchronous computations when it beat the next countdown in the recursive chain. If it's run using Async.Spawn, the same thing happens, but the return of the expression is printed immediately and you can execute other operations while the countdown is going on.

Extending Async Calls to the Web
I would really like to involve this kind of shortcut in my TerraService Web service call. All I/O, whether from the file system, the database, the Web, or even complex telemetry can be included in this kind of orchestration. You may find it useful to create redundant sources of data for situations where connectivity is slow or unreliable. This is often the case when deploying global software to environments with a poor connectivity infrastructure.
I wrote two asynchronous expressions, neither of which does anything asynchronous on its own. The first is aGetPlaceWeb, which takes a triple of strings indicating the city, country, and state of a location. It then retrieves the location using TerraService and saves it to a local file using savePlace:
type TerraService with
  member ts.AsyncGetPlaceFacts(p) =
   Async.BuildPrimitive
    ((fun (callback,asyncState) ->
      ts.BeginGetPlaceFacts (p,callback,asyncState)),
    ts.EndGetPlaceFacts)

let aGetPlaceWeb (city,state,country) =
  async {
    let p = new Place(City=city, State=state, Country=country)
    let! facts = ts.AsyncGetPlaceFacts(p)
    do savePlace facts
    return Complete(Web(facts.Center ))
  }
The second is aGetPlaceFile, which also takes a triple of strings indicating the location:
let aGetPlaceFile placeVals =
  async {
    let name = nameFromTriple placeVals
    if File.Exists(name) then
      let coord = coordFromFile name
      return Complete(File(coord))
    else return Failure(Error)
  }
It calls on nameFromTriple to retrieve a file name and then checks to see whether that file exists (as it would if the Web call had already retrieved that location). So on the first call for any location aGetPlaceWeb is the only viable option for retrieving the information. After that initial visit, you can retrieve the information from aGetPlaceFile. You might suspect that aGetPlaceFile is much faster than a call to the Web, and you should take advantage of that if you can. Further, a timeout mechanism should be included so that if none of the calls returns a successful result in a reasonable amount of time, a suitable message is offered.
A list of places is created as greatPlaces. Then getWebPlaces maps the asynchronous expression aGetPlaceWeb over greatPlaces. That is, the asynchronous expression aGetPlaceWeb is executed on every element of the list greatPlaces and that returns a list of asynchronous expressions (one for each location). The same thing is done with getFilePlaces, except that it uses aGetPlaceFile:
let greatPlaces = [
  ("Austin","Texas","United States");
  ("Las Vegas","Nevada","United States");
  ("Asheville","North Carolina","United States");
  ("Arlington","Virginia","United States");
  ("Laguna Beach","California","United States")]
let getWebPlaces = List.map aGetPlaceWeb greatPlaces
let getFilePlaces = List.map aGetPlaceFile greatPlaces
With these places and their subsequent retrieving expressions defined, you need to mash them together to create one complete list. In each element of that list both the file retrieval and the Web retrieval expressions should be executed asynchronously. That is where the primitive first will be used.
Once these are combined using the List.combine function, each pair is then passed to the primitive first to create an asynchronous expression that computes each in parallel and returns only the first result to the finish line:
let combineAsyncFirst l1 l2 =
  List.map (fun (one,two) -> first [one;two]) (List.combine l1 l2)

let combined = (combineAsyncFirst getWebPlaces getFilePlaces)

let firstPlaces = List.map (terminateInSeconds 3 Timeout) combined

let e = Async.Parallel firstPlaces

let result = Async.Run e
The list firstPlaces is this combined list with the terminateInSeconds asynchronous expression, which terminates after the allotted seconds. In this case it returns the Timeout value from the discriminated union RetrievedValue.
The opportunities composition provides are really starting to become salient in this piece of code. Creating a list of pairs of computations in which the first one to terminate successfully (whether that is because of speed or an error) is returned is not trivial in an imperative language. Adding to that a timeout, which short circuits the entire thing, would be a considerable effort. However, using asynchronous expressions along with composition simplifies creating all of the little pieces needed to make a much more complex asynchronous computation.
This next line of code shows that all of these asynchronous computations are to be executed in parallel and, as a result, you will get a list of location information from the Web or, if already available, stored on the file system:
let e = Async.Parallel firstPlaces
Figure 4 represents the computation model this asynchronous expression created. Once a framework was in place to return the first of two concurrently executing I/O processes, a timeout mechanism was possible. The exact same mechanism could then be used again to retrieve data from two redundant sources concurrently. All of this was plugged into a larger framework that executed each I/O task in parallel.
Figure 4 Asynchronous Computation Model
One can certainly imagine expanding on this composition and creating large networks of logic by reusing frameworks between parallel and first mechanisms. The countdown mechanism shows how flexible this architecture is and how it can create a clock-like framework. The interoperation of parallel and first can create the pieces, and composition allows you to glue them together.
It might seem awfully greedy to call a Web service that you might very well discard the results of later, asking the Web service to do work that may well be unnecessary. However, there are a variety of circumstances where it makes perfect sense. (For further insights on this, please see the End Bracket column in this issue at msdn.microsoft.com/magazine/cc872850.aspx.) I have deployed software in regions of the world where the real complexity is an unreliable network infrastructure. That is, in fact, where the idea of this kind of asynchronous composition came to me. Often in those circumstances having reliable networked software means being able to call on information at multiple sources redundantly.

Working with Other .NET Languages
The .NET Framework has many languages that excel at different things. One of the most powerful aspects of F# is its ability to create libraries that can be called seamlessly from any other .NET-compliant language because F# is a hybrid functional and object-oriented language. Of course, this does not happen for free. An F# library still has to be carefully designed to be intuitive to developers accustomed to other languages, such as C#.
For example, to create a sensible structure for a C# programmer, I will often embed my functions into a class-style architecture that will feel intuitive to a C# developer even though the underlying code was written in a remarkably different language:
type GreatPlaces() =
  let places = List<(string * string * string)>()
  member self.AddPlace(city,state,country) =
        places.Add( (city,state,country) )
  member self.RetrievePlaces () =
        let lplaces = List.of_seq places
        let result = runPlaces lplaces
        Array.combine (Array.of_seq places) result
        |> Array.map compileResultToken
To set up a Web service call in F# you can use the command line and automatic code generation facilities. In a mature language like C# these tools are integrated into Visual Studio and will generate references seamlessly. In F#, you just take a couple of small steps to achieve the same result. First, open the .NET command-line tool and use wsdl.exe to parse the Web service wsdl and generate a proxy in C#. Second, compile that C# into a DLL that will be referenced in the project.
For example, here is the script that you would use to communicate with the TerraServer Web service discussed earlier:
> wsdl /namespace:WebService http://terraservice.net/terraservice.asmx
Microsoft ® Web Services Description Language Utility
Writing file 'c:\dev\TerraService.cs'
>csc /target:library TerraService.cs
Now you have generated the necessary library for calling on the Web service just as you would any class. These two steps are followed by a reference indicator in the F# source. In F#, references to external libraries can be added directly in the text, so you can put this in your source file before any calls to the TerraService library:
#r "TerraService.dll"
This is the well-known power of .NET, interoperating programs that allow you to solve each small problem using the language that makes the most sense for that problem.

Chance Coble, M.S., is a scientist at Ideal Innovations Inc. His interest in functional programming started with Haskell several years ago, and he has been developing F# solutions for global production solutions related to machine learning, data mining, and data integration for the past two years.


Editor's Note
Howard Dierking


Toolbox
Realistic Test Data, Dare Obasanjo’s Blog, Color-Coding Tools, and More
Scott Mitchell

All prices were confirmed August 21, 2008, and are subject to change. The opinions expressed in this column are solely those of the author and do not necessarily reflect the opinions at Microsoft.

Contents
Create Database Test Data
Blogs of Note
Displaying Color-Coded Syntax in a Web Page
The Bookshelf

Create Database Test Data
When building and testing a data-driven application, it is important that the data in the development database is an accurate reflection of the data that will be used in production. Having similar amounts and types of data during development and production helps designers to better evaluate user interfaces, developers to judge how well the system will scale, and testers to build test cases based on more realistic scenarios. It also helps to have realistic data for demos and sales presentations.
While you can write your own SQL scripts to generate test data, a much easier and faster approach is to use SQL Data Generator (version 1.1) from Red Gate Software. With a few points and clicks of the mouse, SQL Data Generator can generate millions of records with realistic values, and customizing the generated data is a cinch.
To get started, launch SQL Data Generator and create a new project, which displays the Project Configuration wizard. From here you select the server and database, indicate whether to have any custom SQL scripts execute before or after the data is generated, and choose an assortment of options, such as whether to fire INSERT and DELETE triggers and whether to enforce CHECK constraints.
After choosing the database and options, SQL Data Generator displays a three-pane interface. The left pane lists the tables in the database and enables you to select which tables will have data generated for them. Selecting a table from the left pane loads information about that table in the right two panes. The top-right pane summarizes the data-generation settings for the table. Here you can choose how many rows to add, whether to delete the table's data before adding the generated data, and what action to take if attempting to insert invalid data. The bottom-right pane shows a preview of the data that will be generated.
To customize the data for a particular table column, select that column from the left or bottom-right panes. This loads the column's settings in the upper-right pane. From here you can indicate whether the generated values for this column should be unique and what percentage of values should be NULL. You can also specify how the data is generated to ensure that it contains realistic values instead of just random strings, numbers, and dates. You can pull data from a .csv file, a SQL query, a weighted list of possible values, or from a regular expression.
Alternatively, you can pick a category, such as colors, e-mail addresses, first names, last names, addresses, cities, postal codes, credit card numbers, company names, or URLs, and SQL Data Generator will use pre-defined values that belong to that category. While you certainly can choose how each column in every table is generated, the good news is that SQL Data Generator makes an educated guess at the appropriate value based on the table and column names and data types. For example, when generating data for a string-based column named FirstName, SQL Data Generator will use values from the First Name category.
After reviewing the settings, click the Generate Data button to add the data to the selected tables. Generating voluminous amounts of database data is quick and easy with SQL Data Generator.
Price: $295.
go.microsoft.com/fwlink/?LinkId=124071
SQL Data Generator Generates and Customizes Data Records

Blogs of Note
It's amazing how popular social software has become in the past several years. Just about everyone has a blog, and sites like Facebook, LinkedIn, and Blogger.com are some of the most visited sites on the Internet. Most social software makes use of Web services, XML, and other open standards to syndicate content and enable integration with third-party applications. For insights into these technologies and related development topics, look no further than Dare Obasanjo's blog.
Dare has years of experience assessing and working with different social software standards. He currently works as a Program Manager for the Microsoft Contacts team, which focuses on back-end services for Windows Live Messenger and Windows Live Spaces, and has worked on the XML and XML Schemas APIs in the Microsoft .NET Framework. Dare is also the creator of RSS Bandit, a popular RSS aggregator. Given his background, Dare has a lot of interesting insights into the standards used by social software, as well as tips and suggestions for those programming against these standards.
Reading Dare's blog can, at times, be like taking a sip from a fire hose. Dare adds a new blog entry almost every day of the week, and most of them are lengthy and packed full of information. But if you want a better understanding of how social software works and a deeper appreciation of the standards involved, Dare's blog is a must-stop destination.
25hoursaday.com/weblog

Displaying Color-Coded Syntax in a Web Page
All modern programming environments use color coding to improve the readability of source code. Visual Studio displays keywords, types, strings, and comments using different colors, and you can customize the fonts, sizes, and styles for these and other tokens. Yet many print and online code samples lack any color coding.
The extra costs associated with color printing may explain the lack of syntax highlighting in books and computer trade magazines, but why don't more blogs and Web sites color code source code samples? The primary challenge of syntax highlighting in a Web page is generating the appropriate HTML and CSS; however, there are a number of free tools that make this process a walk in the park.
One option is to use an online syntax highlighting service, such as Quick Highlighter (go.microsoft.com/fwlink/?LinkId=124072). Quick Highlighter's interface consists of a large textbox where you enter the code to format, a drop-down list with 96 different programming languages to choose from, and a handful of formatting options. It generates the HTML and CSS to color code the syntax, which you can paste into your blog entry or Web page.
One of my favorite color-coding tools is CopySourceAsHtml, a Visual Studio add-in by Colin Coller (read more about it at go.microsoft.com/fwlink/?LinkId=126646). With CopySourceAsHtml there's no need to visit a Web page to color code your syntax; instead, you can generate the HTML and CSS from Visual Studio.
Simply select the code to format, right-click, and then choose the Copy As HTML menu option. This brings up a dialog box with various formatting options. Clicking OK generates the corresponding HTML and CSS and copies it to the Windows Clipboard. (Colin's last update to CopySourceAsHtml was before Visual Studio 2008 was released. Guy Burstein has created a Visual Studio 2008 compatible version, which you can download at go.microsoft.com/fwlink/?LinkId=124074.)
You can also highlight syntax from server-side code in your ASP.NET page. With the squishySyntaxHighlighter component from squishyWARE (go.microsoft.com/fwlink/?LinkId=124076) and two lines of code, you can turn a string of plain C#, Visual Basic, or XML syntax into highlighted HTML and CSS markup.
Price: All free.
Quick Highlighter Is an Online Syntax Highlighting Service

The Bookshelf
ADO.NET is a conglomeration of classes within the System.Data and System.Xml namespaces that enable developers to programmatically query, insert, update, and delete data from a variety of data stores. There was a time when every developer working on a data-driven application needed a solid grounding in ADO.NET. However, as .NET has matured, a number of tools and application architectures from Microsoft and the community have encapsulated many of ADO.NET's low-level details and complexity. But don't forget all about ADO.NET just yet—there's still quite a bit of application code out there that uses ADO.NET, and having a solid understanding of the technology is important if you ever need to extend an existing architecture's data access layer functionality.
When I write or maintain ADO.NET code, I like to keep a copy of Bill Hamilton's book ADO.NET 3.5 Cookbook (O'Reilly, 2008) within arm's reach. ADO.NET 3.5 Cookbook offers more than 200 recipes for common ADO.NET problems—everything from where to store database connection strings to how to execute a SQL statement asynchronously. When I get stuck trying to recall how to perform a certain action, I reach for this book, flip to the Table of Contents, find the recipe, and am off and running.
Each recipe starts with a problem statement, such as: "You want to access an output parameter returned by a stored procedure." Next is the solution, and then the discussion. This problem/solution/discussion format results in self-contained recipes that are easy to understand. Moreover, the author presents virtually all solutions as console applications, as opposed to a Windows Forms or a Web application. This streamlines the solution section because there's no need to step through creating a user interface.
If you need a book that provides deep insights into the architectural decisions behind ADO.NET, or if you are looking for a book that focuses on new ADO.NET 3.5 features , such as LINQ, this book is not for you. But if you are looking for a reference book that you can use to find solutions to common problems quickly, pick up a copy of ADO.NET 3.5 Cookbook.
Price: $54.99.
go.microsoft.com/fwlink/?LinkId=124077

Send your questions and comments for Scott to toolsmm@microsoft.com.

Scott Mitchell, author of numerous books and founder of 4GuysFromRolla.com, is an MVP who has been working with Microsoft Web technologies since 1998. Scott is an independent consultant, trainer, and writer. Reach him at Mitchell@4guysfromrolla.com or via his blog at ScottOnWriting.NET.


CLR Inside Out
Security In Silverlight 2
Andrew Dai

Contents
Permissions and Sandboxing Untrusted Code
The Transparency Model
Transparency in CoreCLR
Inheritance Rules
Exposure to Silverlight Transparency
In the August 2008 MSDN Magazine CLR Inside Out column, Andrew Pardoe gave a brief description of the CoreCLR security model. He described why we chose not to use code access security (CAS) policy in Silverlight and the difference between Transparent, SafeCritical, and Critical code. In this installment I'll go into more detail about how the Transparency model was born, how it works in Silverlight, and what you can expect to see from it in the future.

Permissions and Sandboxing Untrusted Code
Before I dive into the Transparency model, it helps to have some background on CLR security within the full Microsoft .NET Framework. The CLR provides mechanisms for determining what abilities or rights, represented by permissions, code has based on evidence from the code—the location from which the code is running, the signature of the code if one exists, and so forth. This ensures that code coming from an untrusted location, such as the Internet, doesn't have the same privileges as code coming from a trusted location, such as the local machine's Program Files folder.
The CLR runs each managed application in its own application domain. The preferred mechanism to run code safely from untrusted locations is to run it in a sandboxed application domain. The code running in these domains is restricted to the rights of the sandbox—an environment with limited permissions appropriate for partially trusted code. In general, sandboxed application domains separate code into two groups—full trust and partial trust. I'll go into more detail as to the purpose of these two groups and how the distinction is drawn later in this section.
To make this separation easier to implement and use, we introduced what we call the simple sandboxing APIs in the .NET Framework 2.0, which create each application domain with a given permission set for its sandbox and a list of fully trusted assemblies that are not in the Global Assembly Cache (GAC), as all assemblies in the GAC are already fully trusted. Anything loaded into this application domain that is not on the full-trust assemblies list would get the permissions of the sandbox's granted set. In the case of the application running from the Internet, this would be the built-in Internet permission set. Shawn Farkas, a Software Design Engineer on the CLR security team, has an excellent blog post on how to use this API at go.microsoft.com/fwlink/?LinkId=124952.
With this model, a host (for example, a browser plug-in) would create an application domain for each application running under it—these would load as partially trusted and get the sandbox's permission set. The host may also want to (safely) wrap some full-trust managed or native functions and provide them to its hosted applications in the form of libraries—these library assemblies would be on the full-trust assembly list. For example, file system access is a privileged operation, but saving information locally is useful and can be safe if done correctly (fixed location, never executed, and so on). Safely exposing this functionality, though, can be challenging.

The Transparency Model
The Transparency model is not actually new to CoreCLR (the Silverlight version of the CLR)—it's been around since the .NET Framework 2.0. It was initially introduced to address the issue of the growing number of framework libraries exposing services to partially trusted code and the internal security audits they were incurring within the .NET Framework teams. Libraries with the AllowPartiallyTrustedCallers (APTCA) attribute expose full-trust services to partially trusted code—something potentially dangerous that requires proper security checks on the part of the APTCA library.
Prior to the introduction of Transparency, entire APTCA assemblies were subject to code audits. This was extremely expensive, especially considering that much of the library could be calling a common privileged function but otherwise be completely safe.
We developed Transparency with the intention of creating a strong isolation boundary between privileged and unprivileged code. In this model, there are two kinds of code—Transparent and Critical. Transparent code is subject to the following restrictions:
  • Cannot assert permissions or elevate.
  • Should not contain unsafe or unverifiable code. This is considered to be a Critical operation and causes demands to be injected if they occur from Transparent code.
  • Cannot directly call Critical code. Exceptions to this are explained later in this section.
  • Cannot call native code or APIs with the SuppressUnmanagedCodeSecurity attribute.
  • Cannot satisfy LinkDemands. All LinkDemands become full, stack-walking demands.
All permission demands flow through Transparent code and go against the application domain; Transparent code cannot assert these demands away. Therefore, Transparent code is always restricted to the rights of the sandbox. Critical code is not restricted in this way and can do the things mentioned above, and therefore the security audit burden for APTCA assemblies is reduced to the Critical surface area of the assembly.
So how do the two types of code interact? If a developer deemed a Critical API as safe to be called from Transparent code (for example, an API that calls into the operating system to retrieve the date and time), he could mark it as TreatAsSafe (actual attribute in code: SecurityTreatAsSafe) to enable this. It is also important to note that in the 2.0 Transparency model, the Transparency rules are only enforced within an assembly and, as a result, public Criticals are implicitly TreatAsSafe. (Throughout this column, I'll be referring to this Transparency model as the 2.0 Transparency model. This Transparency model is, in fact, also part of the .NET Framework versions 3.0, 3.5, and 3.5 SP1.) Therefore, a public Critical method can be called by Transparent code.
The 2.0 Transparency model helped a lot in the later versions of the desktop version of the .NET Framework, making security easier to reason about and making it much easier to enable our teams to expose their services to partial trust applications. With CoreCLR, we had an opportunity not only to improve on the Transparency system, but also to make it the primary mechanism for enforcing security across the runtime and the code built against it.

Transparency in CoreCLR
Although it helps a great deal in assessing the security of an assembly, Transparency is not the only determinant of security in the .NET Framework. Different application domains may have different sandboxes with different sets of permissions—an application running from the Internet and an application running from a corporate intranet do not have the same privileges. Therefore, it is necessary to check for and assert these permissions in code.
CoreCLR allowed us to make some simplifying assumptions due to the fact that we know all Rich Internet Applications built against it run within a Web browser. The first assumption is that each application domain has a fixed sandbox, and we don't have to deal with individual permissions since we know exactly what rights each Silverlight application will have. There are only ever two permission sets—full trust and the Silverlight sandbox. The second assumption we made is that all Silverlight applications are partially trusted and, therefore, cannot elevate beyond the permissions of the Silverlight sandbox.
These are significant simplifications that provide the following results: all Silverlight applications are Transparent and receive the Silverlight sandbox permission set, Silverlight platform libraries (most of which are fully trusted) will only ever be called by Silverlight applications, so we don't need to demand specific permissions; marking dangerous APIs as Critical is sufficient. Finally, since there are no demands, we will never need to stop a permission stack walk and, therefore, we don't need asserts.
Thus, Transparency becomes the sole security enforcement mechanism in CoreCLR, and we are able to ignore much of CAS. To make it a stronger enforcement mechanism, we tightened the rules for Transparent code and made additional improvements to the Transparency system to make it more intuitive and effective.
You may have noticed that in the 2.0 Transparency model, unsafe or unverifiable code that executes in Transparent code causes an injected demand for UnmanagedCodePermission instead of an outright failure/exception. Generally, this permission should only be available in full trust, so this code would fail in partial trust but work otherwise.
With CoreCLR, however, we don't have the concept of permission demands, so an exception will be thrown when this happens (instead of a demand being injected in the new Transparency model). This type of code fails in both models, but is more performant now since we don't have to deal with the demand's stack walk before finally failing.
Also, in Silverlight, all code is Transparent by default. This includes Silverlight platform libraries; all Critical code must be explicitly annotated. This provides us with a secure default that reduces the chances of creating an implicitly Critical method accidentally. More rules were added with regard to type inheritance and method overrides, which I'll explain in the Inheritance Rules section.
I mentioned earlier that a Critical API marked TreatAsSafe was considered to be callable from Transparent code. I also pointed out that, since Transparency on the desktop framework only applies within an assembly, public Critical APIs are implicitly TreatAsSafe. In CoreCLR, we created a new attribute, SafeCritical (again, the actual annotation is SecuritySafeCritical), to describe code that was both Critical and callable from Transparent code. Transparency now applies within and across assemblies, so Critical code, whether public or not, cannot be called from Transparent code.
This change helps to clear up confusion around how to expose Critical code to Transparent code. First, all Critical code is restricted—there is no exception for public code, so adding the Critical annotation will always mean that Transparent code cannot call it. Furthermore, TreatAsSafe only ever makes sense if the Critical attribute is already there—requiring two attributes may lead to some confusion, as it is possible to put TreatAsSafe on a Transparent method even though it does nothing. Finally, having only one way to specify safe, Critical code simplifies the model further.
One thing to realize regarding this new attribute is that when we talk about Critical code, we often include SafeCritical code in the discussion. This is because SafeCritical code is Critical code—it is just as capable as Critical code, but it is also callable by Transparent code. Therefore, to truly make it safe and Critical, the library developer must be sure that the proper input validation or other security checks are being made before Critical code (not safe) is called and that the necessary output validation and side-effect checks are being made afterward. SafeCritical becomes a contract that says this method accesses Critical code but does all the appropriate checks beforehand and afterword. With the new Transparency model, the focus of security audits is the SafeCritical code.

Inheritance Rules
For the Transparency model to be effective, we also had to ensure that the closure of a type or method was secure as well. Deriving types or methods with resource access rights that were different from the rights of the base type or method raised some access protection issues, especially when these were being cast to more accessible parent types or interfaces. For example, you could imagine someone creating a Critical override of a Transparent virtual method, such as ToString, and wanting to prevent Transparent code from calling the override by casting to Object.ToString.
You can imagine that there is an access and capability hierarchy for Transparency levels whereby access becomes more restricted and/or capabilities are increased as you move from Transparent to SafeCritical to Critical. (Here access refers to the ability to contain as well as call this level of code). The inheritance rule for types is that derived types must be at least as restrictive as the base type. This prevents developers from accidentally creating overloads of Critical types callable by Transparent code. Figure 1 shows what's allowed according to this pattern.
Figure 1 Allowed Type Inheritance Patterns
Base Type | Allowed Derived Type
Transparent | Transparent, SafeCritical, Critical
SafeCritical | SafeCritical, Critical
Critical | Critical
The virtual method override rule is different—the derived method must have the same accessibility from Transparent code as the base method. Figure 2 shows what's allowed according to this pattern.
Figure 2 Allowed Method Inheritance Patterns
Base (Virtual/Interface) Method | Allowed Override Method
Transparent | Transparent, SafeCritical
SafeCritical | Transparent, SafeCritical
Critical | Critical
Note that we don't allow Critical overrides of Transparent virtual methods—one could simply cast to the base method and bypass the Critical check on the override when calling it. We allow SafeCritical overrides of Transparent functions because the cast would buy you nothing—Transparent code can access SafeCritical code anyway. We also allow Transparent overrides of SafeCritical virtual methods—this may seem surprising at first, but it's actually safe because, again, nothing dangerous can be exposed via casts.
To summarize the Silverlight Transparency model, remember that all code is Transparent by default. All Silverlight applications are completely Transparent. All markings indicating otherwise are ignored. In addition, a fixed permission set means Transparency is the only enforcement mechanism needed for CoreCLR. Figure 3 summarizes the three types of code and their abilities and traits.
Figure 3 Comparing Transparent, SafeCritical, and Critical Code
 | Transparent | SafeCritical | Critical
Access Restrictions | Cannot directly call Critical code. | Can do everything that Critical code can. Must make sure to perform the applicable checks before and after calling into Critical code. | Is privileged and able to call any code. Equivalent of full trust only. Can never be called directly by Transparent code, but may be exposed via SafeCritical code.
Unsafe Code | Cannot contain unsafe or unverifiable code. | May contain unsafe or unverifiable code. | May contain unsafe or unverifiable code.
Availability to Application or Platform Code | Present in both application and platform code. | Only available to Silverlight platform assemblies. | Only available to Silverlight platform assemblies.
Type Inheritance Characteristics | Types must derive from other Transparent types. | Types must derive from Transparent or SafeCritical types. | Types can derive from all types regardless of annotations.
Method Override Characteristics | Methods must override Transparent or SafeCritical virtual/interface methods. | Methods must derive from Transparent or SafeCritical virtual/interface methods. | Methods must derive from Critical virtual/interface methods.

Exposure to Silverlight Transparency
Much of the benefit of the Silverlight Transparency model manifested itself in the simplicity of security enforcement in the Silverlight platform. Since Silverlight is a closed platform, there is currently no notion of trusted third-party libraries. Therefore, a developer's interaction with the Silverlight Transparency model is limited to the APIs he is allowed to call—those that are Transparent or SafeCritical—as his applications are completely Transparent. On the .NET Framework, however, the platform is not closed, and so it is possible to develop additional libraries that could benefit from simpler security enforcement.
However, developers may see these benefits in the next major release of the desktop framework. The Transparency model we introduced in .NET Framework 2.0 made the tasks of analyzing and judging the security of APTCA libraries much simpler; the Silverlight Transparency model took that another step up, allowing us to make assumptions and guarantees that previously had to be confirmed through painstaking review. In the future, we plan to introduce the tightened rules around Transparent code, the SafeCritical attribute, and the Transparency inheritance rules to the full .NET Framework. We also plan to make them available to external developers as well.
One major difference between the desktop framework and Silverlight is that most applications on the desktop framework are built for full-trust environments, while all Silverlight applications are built for a partial-trust environment. Most desktop applications can do anything they want and would continue to be able to do so—they won't have to think about Transparency at all. Implementationwise, these applications will be considered Critical.
APTCA library developers will experience many of the benefits of our Silverlight platform assemblies. The auditable surface area of an APTCA library is now focused on the SafeCritical layer, as the Critical layer is not accessible from partially trusted, Transparent code. Furthermore, the introduction of the inheritance rules ensures that the library overrides and derivatives aren't unintentionally exposing any security holes. (Just keep in mind that, due to the fact that there's a variety of possible sandboxes on the desktop framework, APTCA libraries will have to handle individual permissions. Therefore, SafeCritical APIs may have to perform permission demands as well as other input validation checks.)
The stricter Transparent code rules also mean that partially trusted applications will be safer and more performant than before. In the .NET Framework 2.0 Transparency model, Transparent code that called or contained unsafe or unverifiable code faced injected demands for UnmanagedCodePermission and the expensive stack walk that resulted when the unsafe/unverifiable code was called. This typically results in a failure, as most sandboxes do not contain this permission in their permission grant sets.
In the new Transparency model, these will be hard failures—as in the CoreCLR, there would be no injected demand and, therefore, no stack walk. By bringing the new Transparency model to the next version of the desktop framework, we hope to make it easier to develop APTCA libraries as well as improve the experience around developing partially trusted applications.

Send your questions and comments to clrinout@microsoft.com.

Andrew Dai is a Program Manager for the CLR at Microsoft. He works on security for Silverlight and the desktop framework.


Basic Instincts
Dynamic Data Entry With XML Literals
Beth Massi
Contents
Dynamically Loading XAML
Obtaining the Metadata
Generating UI Based on Metadata
Dynamic Data Entry
Wouldn't it be nice to generate all your maintenance screens in your data-driven applications automatically? When I say "maintenance" I mean all those simple lookup data tables or contact tables. Instead of handing those screens to the junior developer on the team, why not just generate them all at run time based on your object model or database schema? This is a relatively easy task in Windows Presentation Foundation (WPF) using XAML, and with the deep XML support in Visual Basic it becomes a snap to generate user interfaces dynamically.

Dynamically Loading XAML
Dynamically creating WPF user interfaces at run time is easy using Visual Basic XML Literals and XML namespace imports. This is because WPF expresses its user interface elements and layout in XAML and Visual Basic does all the heavy lifting, resolving the XML namespaces in order to create the correct XAML.
Let me show you what I mean. You can load and save XAML at run time using the XamlReader and XamlWriter classes in the System.Windows.Markup namespace. Create a new WPF project and, in the Window1 codebehind, you can write this:
Imports <xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation">
Imports <xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
Imports System.Windows.Markup

Class Window1
  Private Sub Window1_Loaded() Handles MyBase.Loaded
    Dim UI = <Label Name="Label1">This is COOL!</Label>
    Me.Content = XamlReader.Load(UI.CreateReader())
  End Sub
End Class
You'll see the label populate the contents of the window, as shown in Figure 1.
Figure 1 XAMLReader Loads XAML at Run Time
When you include the WPF namespace imports at the top of the code file, Visual Basic automatically creates the proper XAML you need for it to load at run time. In this example I'm using the default namespace for the controls; however, you can also provide a namespace prefix and change the XAML to use it. This allows you to reserve the default namespace for other XML you may be manipulating in your code:
Imports <xmlns:controls=  "http://schemas.microsoft.com/winfx/2006/xaml/presentation">
Imports <xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
Imports System.Windows.Markup

Class Window1
  Private Sub Window1_Loaded() Handles MyBase.Loaded
    Dim UI = <controls:Label Name="Label1">      This is COOL!      </controls:Label>
    Me.Content = XamlReader.Load(UI.CreateReader())
  End Sub
End Class
One caveat here is that the XAML you load at run time cannot require compilation, so you cannot specify event handlers on dynamically loaded controls in the XAML directly. You can still add them at run time in the codebehind. For example, you can't do this dynamically, as it will cause a XamlParseException "Must compile XAML file that specifies events:"
Dim UI = <Label
    MouseDown="Label1_MouseDown"
    Name="Label1">
    This is COOL!
  </Label>
Me.Content = XamlReader.Load(UI.CreateReader())
Instead, you can get the reference to the label at run time and explicitly add the event handler in code:
Private Sub Window1_Loaded() Handles Window1.Loaded
  Dim UI = <Label Name="Label1">This is COOL!</Label>
  Dim myLabel As Label = XamlReader.Load(UI.CreateReader())
  Me.Content = myLabel
  AddHandler myLabel.MouseDown, AddressOf Label1_MouseDown
End Sub

Private Sub Label1_MouseDown()
  MsgBox("Mouse Down!")
End Sub
As you can see, there are many possibilities. Of course, dynamically generating a UI isn't unique to WPF. You can do this in Windows Forms as well, but it is an exercise in coding the layout by hand. WPF and Visual Basic make this a breeze because you can define one piece of XAML and construct it from a single LINQ query.

Obtaining the Metadata
Before you can construct the XAML for your UI you're going to need some metadata. You need an easy way to describe how your UI should look based on the table's schema. You can use reflection to look at your object model. But if the model is really simple and maps one-to-one to your database table anyway, you can just ask the database to return the metadata about its tables.
In order to generate a simple UI, you would probably want to obtain, at minimum, the following properties for each column in a table: ColumnName, DataType, MaxLength, and IsPrimaryKey.One way to do this is by creating a stored procedure called GetTableSchema (in this example, I'm using the Northwind database) that uses the information schema views to return this basic schema for any given table (see Figure 2).
Figure 2 GetTableSchema
CREATE PROCEDURE dbo.GetTableSchema
  (
  @table varchar(50)
  )
AS
SELECT
c.table_name As TableName,
c.column_name As ColumnName,
c.data_type As DataType,
c.character_maximum_length As MaxLength,
  COALESCE (
  ( SELECT
    CASE cu.column_name
      WHEN null THEN 0
      ELSE 1
    END
  FROM information_schema.constraint_column_usage cu
  INNER join information_schema.table_constraints ct
  ON ct.constraint_name = cu.constraint_name
  WHERE
  ct.constraint_type = 'PRIMARY KEY'
  AND ct.table_name = c.table_name
  AND cu.column_name = c.column_name
  ),0) AS IsPrimaryKey
FROM information_schema.columns c
INNER JOIN information_schema.tables t
ON c.table_name = t.table_name
WHERE @table = t.table_name and
  (t.table_type = 'BASE TABLE' and not
  (t.table_name = 'dtproperties') and not
  (t.table_name = 'sysdiagrams'))
ORDER BY c.table_name, c.ordinal_position
A simple and strongly typed way to access this stored procedure is to add a new LINQ to SQL classes item to your project, expand the Stored Procedures node in the Server Explorer for the database, and drag the above procedure onto the methods pane. Next, you need to create an object on the design surface called TableSchema that contains the same properties as the fields returned from the stored procedure. For this example, I plan to edit the Northwind customer table, so you can add that to the model as well. Once that's set up, in the properties for the GetTableSchema method, set the result type to the TableSchema class.

Generating UI Based on Metadata
Now that you have the metadata, you're almost ready to write a query to generate the XAML at run time. You can generate the entire Window content or just parts of it by using a ContentControl. This way you can easily lay out the static elements of the Window and indicate where the dynamic content should be placed.
For this example, using the grid panel to lay out the form, create a simple textbox and Find and Save buttons at a fixed area at the top of the Window. Under that, place a ContentControl onto the Window and name it something like DynamicContent (see Figure 3). You should now have XAML that looks similar to that illustrated in Figure 3.
Figure 3 Defining the Window
<Window x:Class="Window1"
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  Title="Window1"  Name="Window1" SizeToContent="WidthAndHeight" >
  <Grid Name="MainGrid"  >
    <Grid.RowDefinitions>
      <RowDefinition Height="10*" />
      <RowDefinition Height="60*" />
    </Grid.RowDefinitions>
    <StackPanel Name="StackPanel1" Orientation="Horizontal"
      Margin="3" VerticalAlignment="Top">
      <Label Height="28" Name="Label1" Width="84"
        HorizontalContentAlignment="Right" FontWeight="Bold">ID</Label>
      <TextBox Height="25" Name="txtSearch" Width="120"></TextBox>
      <Button Height="25" Name="btnFind" Width="75">Find</Button>
      <Button Height="25" Name="btnSave" Width="75">Save</Button>
    </StackPanel>
    <ContentControl Grid.Row="1" Name="DynamicContent" Margin="3" />
  </Grid>
</Window>
In the codebehind, add the appropriate XML namespace imports at the top of the file. Then you can generate the UI by writing a LINQ query in the Window Loaded event handler. You can also add button click handlers for the actual loading and saving of the customer data.
The entire Window class code is shown in Figure 4. It's important to note that this code embeds a couple of LINQ query expressions to create the labels for the field names, controls, and data bindings using a helper function called GetUIElement that returns the control's XAML as an XElement (see Figure 4).
Figure 4 Generating UI Dynamically
Imports <xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation">
Imports <xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
Imports System.Windows.Markup

Class Window1
  Dim db As New NorthwindDataContext
  Dim CustomerData As Customer

  Private Sub Window1_Loaded() Handles MyBase.Loaded
    Dim customerSchema = db.GetTableSchema("Customers").ToList()

    Dim UI = _
      <Grid xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns=          "http://schemas.microsoft.com/winfx/2006/xaml/presentation">
        <Grid.ColumnDefinitions>
          <ColumnDefinition Width="100*"/>
          <ColumnDefinition Width="200*"/>
        </Grid.ColumnDefinitions>
        <StackPanel Name="StackLabels" Margin="3">
          <%= From column In customerSchema _
            Where column.IsPrimaryKey = 0 _
            Select <Label
              Height="28"
              Name=<%= column.ColumnName & "Label" %>
              HorizontalContentAlignment="Right">
            <%= column.ColumnName %>:</Label> %>
          </StackPanel>
          <StackPanel Grid.Column="1" Name="StackFields" Margin="3">
            <%= From column In customerSchema _
              Where column.IsPrimaryKey = 0 _
              Select GetUIElement(column) %>
          </StackPanel>
      </Grid>

    Me.DynamicContent.Content = XamlReader.Load(UI.CreateReader)

  End Sub

  Private Function GetUIElement(ByVal column As TableSchema) As XElement
    Select Case column.DataType
      Case "datetime", "int", "smallint", "money"
        Return <TextBox
          Height="28"
          Name=<%= "txt" & column.ColumnName %>
          Text=<%= "{Binding Path=" & column.ColumnName & "}" %>/>
      Case "bit"
        Return <CheckBox
          HorizontalContentAlignment="Left"
          Name=<%= "chk" & column.ColumnName %>
          IsChecked=<%= "{Binding Path=" & column.ColumnName & "}" %>>
          <%= column.ColumnName %>
          </CheckBox>
      Case "image"
        Return <Image
           Height="150"
           Width="150"
           Stretch="Fill"
           HorizontalAlignment="Left"
           Name=<%= "img" & columnInfo.ColumnName %>
           Source=<%= "{Binding Path=" & columnInfo.ColumnName & "}" %>/>
      Case Else
        Return <TextBox
          Height="28"
          Name=<%= "txt" & column.ColumnName %>
          MaxLength=<%= column.MaxLength %>
          Text=<%= "{Binding Path=" & column.ColumnName & "}" %>/>
    End Select
  End Function

  Private Sub btnFind_Click() Handles btnFind.Click
    If Me.txtSearch.Text <> "" Then
      Me.CustomerData = (From cust In db.Customers _
        Where cust.CustomerID = Me.txtSearch.Text).FirstOrDefault()

      Me.DataContext = Me.CustomerData
    Else
      Me.DataContext = Nothing
    End If
  End Sub

  Private Sub btnSave_Click() Handles btnSave.Click
    If Me.DataContext IsNot Nothing Then
      Try
        db.SubmitChanges()
        MsgBox("Saved")

      Catch ex As Exception
        MsgBox(ex.ToString)
      End Try
    End If
  End Sub
End Class
When you run the application this time, you are going to see all the customer fields in the UI (see the example in Figure 5). Remember that the GetTableSchema stored procedure will work for any table contained in the database, so you could even abstract this further. For instance, what if you just want one single Window that can edit any maintenance table in the database, no matter what the schema?
Figure 5 Customer Fields in the UI

Dynamic Data Entry
The previous example is still tied to the object model of Customer to load and edit the data. Even though the UI is dynamic, the data entry is not. What you probably want is a more abstracted form so that when you modify the database schema of any of the maintenance tables, you don't have to update your object model and recompile all the code.
One way to achieve this is to load and edit a simple untyped DataTable at run time. This works well using XML literals again, this time to generate the SELECT and UPDATE statements for the SqlDataAdapter. One thing to note is that this technique relies on the database to enforce any validation rules. This is why I would only use this type of dynamic form to edit very simple tables (such as maintenance or lookup tables). However, I suggest you play with the WPF Binding.ValidationRules property to declaratively add validation rules to the controls.
You can create a typed DataTable for the TableSchema metadata as well, which eliminates the need for LINQ to SQL classes in the project. To do this, just right-click on the project, add a new item, and select DataSet. Name it TableSchemaDataSet and drag the GetTableSchema stored procedure onto the design surface from the Server Explorer. This automatically creates a typed DataTable for you with no fuss. Rename the DataTable to TableSchema and save it.
Now you need to load an untyped DataTable into the dynamically generated UI to edit the data. You don't want to make any assumptions about the schema of the table you're editing except that there is a primary key of some sort. DataTables and DataSets work well with WPF, but there are a couple of things you need to set up manually since you're loading this all at run time.
A simple thing to do is to set up a public property called TableName on the form to hold the name of the table you want to edit. This can be set easily from calling code. Then create some private class-level variables to reference the ADO.NET objects you need. The XAML markup of the Window for this example is exactly the same as before. Figure 6 shows the class-level variables you need and the TableName property set to a default value of Shippers.
Figure 6 Set Variables and Properties for Dynamic Data Entry
Imports <xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation">
Imports <xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml">
Imports System.Windows.Markup
Imports System.Data.SqlClient
Imports System.Data

Partial Public Class Window2
  'This is the metadata table we created in the DataSet Designer
  Private TableSchema As New TableSchemaDataSet.TableSchemaDataTable
  'ADO.NET objects used to load and save the table we're editing
  Private TableDataAdapter As New SqlDataAdapter
  Private TableConnection As New _
          SqlConnection(My.Settings.NorthwindConnectionString)
  Private Table As DataTable
  'This is the key field used in searching for a row in this example
  Private PKField As TableSchemaDataSet.TableSchemaRow

  'This property can be set before the Form.Show() to edit any table
  Private m_tableName As String = "Shippers"
  Public Property TableName() As String
    Get
      Return m_tableName
    End Get
    Set(ByVal value As String)
      m_tableName = value
    End Set
  End Property
  End Class
In the Loaded event handler you can now load the metadata, create and load the XAML to display the UI just like before, and then set the UpdateCommand on the TableDataAdapter. Figure 7 shows the Loaded event handler and the code that generates the XAML.
Figure 7 Generate the Dynamic Data Entry UI
Private Sub Window1_Loaded() Handles MyBase.Loaded
  Try
    'Get the schema of the database table we want to edit
    Dim taSchema As New _      TableSchemaDataSetTableAdapters. TableSchemaTableAdapter
    taSchema.Fill(Me.TableSchema, Me.TableName)

    'Create the DataTable that will hold the record we're editing
    Me.Table = New DataTable(Me.TableName)
    Me.Title = Me.TableName
    Me.LoadUI()
    Me.SetPrimaryKey()
    Me.SetUpdateCommand()

  Catch ex As Exception
    MsgBox(ex.ToString)
    Me.Close()
  End Try
End Sub

Private Sub LoadUI() _
  Dim UI = _
    <Grid xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
      xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
      Name="Grid1">
      <Grid.ColumnDefinitions>
        <ColumnDefinition Width="100*"/>
        <ColumnDefinition Width="200*"/>
      </Grid.ColumnDefinitions>
      <StackPanel Name="StackLabels" Margin="3">
        <%= From column In Me.TableSchema _
          Where column.IsPrimaryKey = 0 AndAlso _              column.DataType <> "timestamp" _          Select <Label
            Height="28"
            Name=<%= column.ColumnName & "Label" %>
            HorizontalContentAlignment="Right">
          <%= column.ColumnName %>:</Label> %>
        </StackPanel>
        <StackPanel Grid.Column="1" Name="StackFields" Margin="3">
          <%= From column In Me.TableSchema _
            Where column.IsPrimaryKey = 0 AndAlso _               column.DataType <> "timestamp" _
            Select GetUIElement(column) %>
        </StackPanel>
      </Grid>

  Me.DynamicContent.Content = XamlReader.Load(UI.CreateReader())
End Sub

Private Function GetUIElement(ByVal columnInfo As _  TableSchemaDataSet.TableSchemaRow) As XElement
  Select Case columnInfo.DataType.ToLower
    Case "datetime", "int", "smallint", "money"
      Return <TextBox
        Height="28"
        Name=<%= "txt" & columnInfo.ColumnName %>
        Text=<%= "{Binding Path=" & columnInfo.ColumnName & "}" %>/>
    Case "bit"
      Return <CheckBox
        HorizontalContentAlignment="Left"
        Name=<%= "chk" & columnInfo.ColumnName %>
        IsChecked=<%= "{Binding Path=" & columnInfo.ColumnName & "}" %>>
        <%= columnInfo.ColumnName %>
        </CheckBox>
    Case "image"
      Return <Image
         Height="150"
         Width="150"
         Stretch="Fill"
         HorizontalAlignment="Left"
         Name=<%= "img" & columnInfo.ColumnName %>
         Source=<%= "{Binding Path=" & columnInfo.ColumnName & "}" %>/>
    Case Else
      Return <TextBox
        Height="28"
        Name=<%= "txt" & columnInfo.ColumnName %>
        MaxLength=<%= columnInfo.MaxLength %>
        Text=<%= "{Binding Path=" & columnInfo.ColumnName & "}" %>/>
  End Select
End Function
Now that you have the XAML UI generated, set the primary key field (which is a TableSchemaDataRow object) so that you can use it in your UPDATE statement as well as in the SELECT query when the user clicks the Find button on the form. Generally primary keys are surrogate keys (like auto-incrementing integers) and mean nothing to the user, so instead you may want to create another public property that captures the search field name.
Assuming that every table has only one primary key field defined you can use the indexer 0 on the query, which returns the first of the sequence:
Private Sub SetPrimaryKey()
    'Grab the Primary Key column of the table we want to
    ' edit so we can use it in the search
    Me.PKField = (From column In Me.TableSchema _
                  Where column.IsPrimaryKey = 1)(0)
  End Sub
To create the SELECT statement, use XML literals again, but this time without creating XML. Instead you can create a string by calling the XElement's .Value property. You then can create the SqlCommand and fill the untyped DataTable with the results. Setting the window's DataContext sets up the data binding to the fields that were specified when the XAML was generated. Figure 8 shows the code that generates the SELECT statement and sets the DataContext to the results.
Figure 8 Generate the SELECT Statement
Private Sub btnFind_Click() Handles btnFind.Click
  If Me.txtSearch.Text <> "" Then
    Try
      'Create the SELECT command
      Dim cmdText = <s>
        SELECT * FROM <%= Me.TableName %>
        WHERE <%= Me.PKField.ColumnName %> =
          <%= If(Me.PKField.DataType.Contains("char"), _
            "'" & Me.txtSearch.Text & "'", _
            Me.txtSearch.Text) %>
        </s>.Value

      Dim cmd As New SqlCommand(cmdText, Me.TableConnection)
      Me.Table.Clear()
      Me.TableDataAdapter.SelectCommand = cmd
      Me.TableDataAdapter.Fill(Me.Table)

      Me.DataContext = Me.Table
      Dim view = CollectionViewSource.GetDefaultView(Me.Table)
      view.MoveCurrentToFirst()

    Catch ex As Exception
      MsgBox(ex.ToString)
      Me.DataContext = Nothing
    End Try
  Else
    Me.DataContext = Nothing
  End If
End Sub
You want to be able to edit and save this record, too, so you need to generate an UPDATE command. For this example, the code only sets up the UpdateCommand on the SqlDataAdapter because the form just edits records, but it's easy enough to create Delete and Insert commands as well using the same techniques. Figure 9 shows the code that constructs the UPDATE statement and sets up the UpdateCommand:
Figure 9 Generate the UPDATE Statement
Private Sub SetUpdateCommand()
  'Set the UpdateCommand so that we can save edited records in the table
  Dim cmdText = <s>
    UPDATE <%= Me.TableName %>
    SET <%= From column In Me.TableSchema _
      Where column.IsPrimaryKey = 0 AndAlso _        column.DataType <> "timestamp" _
      Select <c>
        <%= column.ColumnName %> = @<%= column.ColumnName %>
        <%= If(Me.TableSchema.Rows.IndexOf(column) < _
          Me.TableSchema.Rows.Count - 1, ", ", "") %>
        </c>.Value %>
    WHERE <%= Me.PKField.ColumnName %> = @<%= Me.PKField.ColumnName %>
      <%= From column In Me.TableSchema _
        Where column.IsPrimaryKey = 0 AndAlso _          column.DataType = "timestamp" _
        Select <c>
          AND <%= column.ColumnName %> = @<%= column.ColumnName %>
          </c>.Value %>
    </s>.Value

  Dim cmd As New SqlCommand(cmdText, Me.TableConnection)
  Dim p As SqlParameter

  For Each column In Me.TableSchema
    If column.IsPrimaryKey = 0 AndAlso column.DataType = "timestamp" Then
    'Note: It's recommended to use a TimeStamp column in your tables for     'concurrency checking
      p = New SqlParameter("@" & column.ColumnName, SqlDbType.Timestamp)
      p.SourceVersion = DataRowVersion.Original
      p.SourceColumn = column.ColumnName
      cmd.Parameters.Add(p)
    Else
      p = New SqlParameter("@" & column.ColumnName, _
        CType([Enum].Parse(GetType(SqlDbType), _          column.DataType, True), SqlDbType))
      p.SourceColumn = column.ColumnName
      p.SourceVersion = DataRowVersion.Current
      cmd.Parameters.Add(p)
    End If
  Next

  Me.TableDataAdapter.UpdateCommand = cmd
End Sub
One important note here: when the UpdateCommand is created it assumes that concurrency checking is being done with a TimeStamp field (which I've added to my copy of Northwind), but you can also create the long version where it checks original against current values as well.
So now you can simply set the TableName property of this form and it will dynamically generate the UI, load the data when the Find button is clicked, and save changes back to the database. Try modifying your database table's schema and running it again without recompiling the application at all. Slick.
I've uploaded this project onto Code Gallery with both examples, so have a look at code.msdn.microsoft.com/dynamicWPF.

Send your questions and comments to instinct@microsoft.com.

Beth Massi is an Online Content and Community Program Manager on the Visual Studio Community Team at Microsoft. She is responsible for producing content for business application developers and driving community features onto MSDN Developer Centers. Read more about her at blogs.msdn.com/bethmassi.


Cutting Edge
Code reuse in WPF and Silverlight 2.
Dino Esposito

Code download available at: CuttingEdge2008_10.exe (604 KB)
Browse the Code Online

This column is based on a prerelease version of Silverlight 2. All information herein is subject to change.

Contents
WPF and Silverlight 2 Compatibility
Introducing Visual State Manager
Sharing Code
Reasoning about WPF Applications
Writing Cross-Platform WPF Code
Analyzing Managed Code
Get a Strategy for Critical Code
Going through an Example
Final Considerations
In Silverlight 2, you use Extensible Application Markup Language (XAML) to design and render the user interface. At the same time, you leverage the built-in core CLR to process managed code within the browser. As a result, there's a strong similarity between Web-based Silverlight 2 applications and desktop Windows Presentation Foundation (WPF) applications. One goal of having a similar programming model is that it enables easy code reuse between the two. In this column, I'll discuss a few patterns to make sharing code and XAML markup between Silverlight 2 and WPF as easy as possible.
CoreCLR In Silverlight
Silverlight 2 does incorporate a CLR, but it's not the same CLR used by the other .NET applications and assemblies. The Silverlight CLR is known as CoreCLR and has been designed with different goals in mind. Designed for cross-platform interoperability, the CoreCLR can run side by side with the CLR, and it supports a different security model and a different version of the base class library. More is explained in the excellent CLR Inside Out column of August 2008 (see msdn.microsoft.com/magazine/cc721609).
Having different CLRs behind Silverlight and .NET applications means that you can't have the same assembly referenced in two projects targeting a .NET application and a Silverlight application. The main issue is with the mscorlib assembly. Silverlight needs a much smaller set of functionality for its purposes—just the core stuff. Any .NET assembly, instead, links to the standard version of mscorlib, and this is where the trouble starts.
In the sample application discussed in this column, I have an interface to share between the Windows Presentation Foundation application and the Silverlight application. The only workaround is copying the C# with the interface definition between the two projects because you can't have a commonly referenced assembly; therefore, it is desirable that in the version of the .NET Framework the functionalities in the standard mscorlib assembly are split into two parts—core stuff and desktop stuff—in order to lay the groundwork for binary compatibility between Silverlight and .NET assemblies.

WPF and Silverlight 2 Compatibility
With the introduction of Silverlight 2, XAML is poised to become the API for a new generation of UI modules. Silverlight 2 supports a subset of the full WPF framework including features for rich layout management, data binding, styles, media, animation, graphics, and templates.
However, full support for XAML in Silverlight 2 is constrained by the size of the downloadable plug-in. In just a few megabytes (less than 5MB in Beta 2), the Silverlight 2 plug-in must deliver the core CLR, a version of the Microsoft .NET Framework 3.5 that includes WPF and a subset of the Windows Communication Foundation (WCF) client platform, a XAML parser, and a number of Silverlight-specific controls.
In Silverlight 2, you won't find support for the WPF 3D graphics capabilities, and you may find that some attributes and elements have been dropped or trimmed down. At the end of the day, though, you have a compatible subset, so you can produce a Silverlight version of a moderately complex WPF user interface. Shortly, I'll return to discuss critical areas of compatibility.
Note, however, that Microsoft is also adding some new APIs in Silverlight that may not currently have a counterpart in the desktop version of WPF. The most relevant features include controls, such as the DataGrid, and classes, such as WebClient for simple GET-style network calls. These features will be added to WPF too.

Introducing Visual State Manager
Another feature that ships in Silverlight 2 and will be added to WPF is the extremely cool Visual State Manager (VSM) for controls. VSM simplifies the development of interactive control templates by introducing visual states and state transitions. Since the introduction of WPF, developers could customize the look, feel, and behavior of WPF controls through templates and styles. For instance, you could not only change the shape and appearance of a control but also define a new action or animation for when the control is clicked, gets or loses focus, and so on.
With VSM, visual states and state transitions do some of that work for you. Typical VSM visual states are normal, mouse-over, disabled, and focused. For each of these states you can define a style or a shape. State transitions define how the control moves from one state to another. You typically define transitions through animations. At run time, Silverlight will play the appropriate animation and apply the specified style to elegantly and smoothly move the control from state to state.
To define a visual state, insert a fragment of markup in the control template, as shown here:
<vsm:VisualStateManager.VisualStateGroups>
  <vsm:VisualStateGroup x:Name="CommonStates">
    <vsm:VisualState x:Name="MouseOver">
      <Storyboard>
        ...
      </Storyboard>
    </vsm:VisualState>
  </vsm:VisualStateGroup>
</vsm:VisualStateManager.VisualStateGroups>
Each control defines one or more groups of states such as CommonStates and FocusStates. Each group, in turn, defines specific visual states such as MouseOver, Pressed, and Checked. For each visual state and transition between states, you can define a storyboard that Silverlight will automatically play when appropriate.
In a nutshell, there are features in WPF that Silverlight 2 doesn't support and features in Silverlight 2 that WPF lacks. These differences affect mostly the compatibility at the XAML level. For complete compatibility, you can use the common subset, which, thankfully, is large enough to still let you do virtually anything.

Sharing Code
When VSM becomes available for the desktop version of WPF, you'll be able to use the same approach for both Silverlight and Windows desktop applications and share control templates between WPF and Silverlight projects. Until then, let's see how you can reuse code between WPF desktop and Web projects today.
A WPF app is made of XAML and managed code. The managed code targets the classes of the supported version of .NET. It is your responsibility to use a common subset of XAML that both desktop WPF and Silverlight 2 can understand. Likewise, it is your responsibility to organize your codebehind classes such that differences in the back-end framework are handled properly.
There are basically two scenarios in which you'll want to reuse WPF code. One is where you have an existing WPF desktop application and want to offer it over the Web to simplify maintenance and deployment. The other is when you want to develop a front end for an existing system and offer it for both Windows and Web clients.
I'll approach the question of reusing code from the WPF-to-Silverlight perspective. In terms of patterns to refactor code and markup, there's very little difference.

Reasoning about WPF Applications
A typical WPF application is built from a tree of objects where Window is the root of the tree. In turn, the Window element contains a number of child elements laid out or stacked in a variety of ways. Elements refer to basic shapes, layout managers, storyboards, and controls (including custom third-party and user controls). Here's a basic example:
<Window x:Class="Samples.MyTestWindow"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="Test App" Height="300" Width="350">
    <StackPanel Margin="10">
        ...
    </StackPanel>
</Window>
This code cannot be incorporated into a Silverlight 2 application as is. First of all, the Window element is not supported in Silverlight. In the Silverlight assemblies, there is simply no such class in the System.Windows namespace. The root tag of any Silverlight application is UserControl. The element is mapped to the UserControl class defined in the System.Windows.Controls namespace.
Here's the modified heading of a Silverlight application:
<UserControl x:Class="Samples.MyTestWindow"
    xmlns="http://schemas.microsoft.com/client/2007"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="Test App" Height="300" Width="350">
    <StackPanel Margin="10">
        ...
    </StackPanel>
</UserControl>
All the markup code you have within the boundaries of the <Window> and <UserControl> elements should continue to match. It is up to you to modify the original desktop XAML to make it match the requirements of the Silverlight XAML parser. This task largely benefits from the high level of compatibility between the two syntaxes. Most of the time, adjusting XAML is a matter of fixing a few elements and attributes. Let me provide a couple of examples.
Desktop WPF features a <label> element that Silverlight doesn't recognize. When porting this markup to Silverlight, you have to find a workaround for the label. A rectangle with a text block inside may be a workable solution.
In WPF, you can associate a tooltip with controls using the ToolTip attribute, as you see in the following:
<Button Tooltip="Click me" ... />
In Silverlight 2, the ToolTip attribute is not supported. You have to resort to the ToolTipService, as in the following code snippet:
<Button ToolTipService.ToolTip="Click me" ... />
It should be noted that both solutions work in desktop WPF. In addition, the ToolTipService offers a number of additional tooltip properties in desktop WPF such as placement, offset, initial delay, and duration. However, none of these extra properties are supported by Silverlight.
Are these all of the compatibility issues between WPF and Silverlight? It depends on how you use WPF. In general, porting a nontrivial WPF application to Silverlight is difficult and not really even a primary goal. To begin with, in Silverlight 2 you do not have triggers everywhere you might need them. For example, there is a Triggers collection on UI elements—descendants of FrameworkElement—but not in styles, data, and control templates.
Likewise, data binding is supported in Silverlight but not in the same manner as it is in WPF. For example, you have the Binding element, you have data context, data templates, and observable collections. As mentioned, you have no triggers and a simplified XAML markup that requires you to write code more often than in WPF. Plus, the internal implementation is quite different. The Binding object in Silverlight has much fewer properties than in WPF.
Globalization is another area that might cause you headaches. For performance reasons, the core CLR doesn't include its own globalization data for all supported cultures. Instead, the CultureInfo class in Silverlight relies on the globalization functionality provided by the underlying operating system. This means that there's no way for you to give applications the same globalization settings across different operating systems.
Finally, WPF has a richer set of controls that are not available in Silverlight. A good example is the RichTextBox control.
In summary, porting a Silverlight application to WPF is relatively trivial, although as a developer you should be concerned with possible performance problems that could be caused by the richer object model. The key fact to remember is that Silverlight supports a subset of the possibilities offered by desktop WPF; it is up to you to devise a cross-platform solution by choosing the appropriate features.

Writing Cross-Platform WPF Code
The easiest way to port code from WPF projects to Silverlight projects is through user controls. Aside from a distinct XML namespace and markup differences, a user control is the only way to share markup and code between desktop and Web-based WPF.
If your WPF application is natively organized, or can be refactored, to make extensive use of user controls, porting code to Silverlight will be much easier than the other, less appealing, option of cutting and pasting.
So can assemblies be shared between WPF and Silverlight projects? In Silverlight 2, you can certainly create custom class libraries packed into assemblies. However, you should be aware that these libraries target the Silverlight version of the .NET Framework and use a different security model (for more see this month’s installment of CLR Inside Out).
Any assemblies you use in a desktop WPF application must be recompiled as Silverlight class libraries to ensure that they refer to the correct assemblies and make legal calls to classes. Needless to say, this process of recompiling may generate additional work to fix calls to unsupported classes.
In summary, with a bit of work you can certainly take a WPF application and expose it over the Web through Silverlight. And when you do so, you are actually exposing your code over a number of non-Windows platforms without forcing clients to have the full .NET Framework installed. Figure 1 shows a simple WPF standalone desktop application. Figure 2 shows the same application hosted in Internet Explorer through Silverlight.
Figure 1 A Sample WPF Application
Figure 2 A WPF Application Adapted for Silverlight

Analyzing Managed Code
For some reason, code reuse between WPF and Silverlight is perceived to be a XAML problem. As mentioned, you do have some XAML issues when adapting your code, but the biggest challenge is the codebehind class.
In both Windows and Silverlight, a XAML file is paired with a codebehind class written in C# or another managed language. Unfortunately, these codebehind classes target different versions of the .NET Framework. The desktop version of WPF relies on the full base class library (BCL) of the .NET Framework 3.5, while Silverlight 2 uses a lightweight version of the BCL.
The Silverlight version of the BCL is significantly smaller, but it still supports fundamental capabilities such as collections, reflection, regular expressions, string manipulation, threading, and timers. You also have tools to call into a variety of services such as XML Web services, WCF services, and ADO.NET Data Services. In addition, you have a rich networking support to communicate over HTTP—with Plain Old XML (POX) and Representational State Transfer (REST) services—and, in general, reach any public HTTP endpoint. Networking support also includes (cross-domain) sockets and duplex communication.
Finally, the Silverlight BCL provides good support for working with XML data, including ad hoc versions of the XmlReader and XmlWriter classes. These classes are quite similar to the analogous classes in the desktop version of the .NET Framework.
Building on these core capabilities, in Silverlight 2, you find full support for LINQ to Objects, LINQ to XML, and expression trees. Starting with Beta 2, Microsoft also added a brand new LINQ to JavaScript Object Notation (JSON) provider to run LINQ queries directly on JSON data.
Another point to mention is that networking in Silverlight can only happen asynchronously. To make a synchronous call you have to resort to a trick that essentially consists of invoking the browser interoperability layer and reaching the browser's implementation of XMLHttpRequest, whenever this is possible (for details on this, see go.microsoft.com/fwlink/?LinkId=124048).
The bottom line is that WPF and Silverlight codebehind classes leverage significantly different class libraries. This fact, much more than any XAML differences, hinders reusability. Let's tackle this problem next.

Get a Strategy for Critical Code
When you write code to be shared by Silverlight and Windows runtimes, you should know very well what each platform supports. Ideally, you'll have a list of requested tasks that require special handling on each platform. Next, you isolate these pieces of code in an object and extract an interface out of them.
As a result, you'll have a fully reusable block of code that calls into separate components for critical functionalities. Porting such an application to Silverlight (or WPF) would then be as easy as replacing critical components with others that are platform specific. Because all these components still expose a common interface, their implementation is transparent to the main block of code.
The pattern behind this approach is the "Strategy" pattern. For a formal definition of this, you should see go.microsoft.com/fwlink/?LinkId=124047. In a nutshell, the Strategy pattern is helpful whenever you need to dynamically change the algorithm used to perform a certain task in an application.
Once you have identified a region of your code that has the potential to vary based on runtime conditions, you define an interface that specifies the abstract behavior of your code. Next, you create one or more strategy classes that implement the interface and thereby represent different ways in which the abstract behavior can be accomplished. By then changing the strategy class, you are simply changing your approach for solving the algorithm defined by the interface. In doing so, you decouple the behavior's actual implementation (the strategy) from the code that is using it.
In ASP.NET, for example, the Strategy pattern is used in the implementation of the provider model for membership, roles, user profiles, and so forth. The ASP.NET runtime knows that there's a particular interface to deal with, say, membership and users. It also knows how to find and instantiate the concrete classes for these interface types. However, the runtime components depend only on the interface, making the details of the concrete class irrelevant to ASP.NET.

Going through an Example
Let's consider the sample application shown inFigures 1 and 2. In both Silverlight and Windows, the application presents a form where users can type a stock symbol to get a current quote. Figure 3 shows the diagram of the application when the user clicks the button. The user interface employs a Model-View-Presenter (MVP) pattern to convey in a single presenter class all the logic behind the UI. The presenter, in turn, invokes an internal QuoteService class that ultimately responds by providing a StockInfo object with any information to be incorporated in the user interface (see Figure 4).
Figure 3 Diagram of the Application’s Behavior
Figure 4 The Presenter Class
namespace Samples
{
    class SymbolFinderPresenter
    {
        // Internal reference to the view to update
        private ISymbolFinderView _view;

        public SymbolFinderPresenter(ISymbolFinderView view) {
            this._view = view;
        }

        public void Initialize() { }

        // Triggered by the user's clicking on button Find
        public void FindSymbol() {
            // Clear the view before operations start
            ClearView();

            // Get the symbol to retrieve
            string symbol = this._view.SymbolName;
            if (String.IsNullOrEmpty(symbol)) {
                _view.QuickInfoErrorMessage = "Symbol not found.";
                return;
            }

            QuoteService service = new QuoteService();
            StockInfo stock = service.GetQuote(symbol);

            // Update the view
            UpdateView(stock);
        }

         private void ClearView() {
            _view.SymbolDisplayName = String.Empty;
            _view.SymbolValue = String.Empty;
            _view.SymbolChange = String.Empty;
            _view.ServiceProviderName = String.Empty;
        }

        private void UpdateView(StockInfo stock) {
             // Update the view
            _view.QuickInfoErrorMessage = String.Empty;
            _view.SymbolDisplayName = stock.Symbol;
            _view.SymbolValue = stock.Quote;
            _view.SymbolChange = stock.Change;
            _view.ServiceProviderName = stock.ProviderName;
        }
     }
}
The QuoteService class doesn't attempt to retrieve quotes itself. It first creates a provider component and then yields to it. The QuoteService class implements a very simple algorithm. If there's an Internet connection, it uses a provider class that uses a public Web service to get financial data; otherwise, it switches to a fake provider that just returns random numbers. So at some point, the QuoteService class needs to check for an Internet connection, as shown in Figure 5.
Figure 5 QuoteService Checks for Internet Connection
public StockInfo GetQuote(string symbol)
{
    // Get the provider of the service
    IQuoteServiceProvider provider = ResolveProvider();
    return provider.GetQuote(symbol);
}
private IQuoteServiceProvider ResolveProvider()
{
    bool isOnline = IsConnectedToInternet();
    if (isOnline)
        return new FinanceInfoProvider();

    return new RandomProvider();
}
So far, there's no difference between Silverlight and WPF, and all the code is fully reusable. To check for an Internet connection in .NET, you can use some of the static methods on the NetworkInterface object. The object is defined in the System.Net.NetworkInformation namespace. In particular, the GetIsNetworkAvailable method returns a Boolean value that denotes whether any network connection is available. Unfortunately, this doesn't say much about Internet connectivity. To make sure you also have Internet access, the only reliable method is to attempt to ping a host (see Figure 6).
Figure 6 Ping for Internet Connection
private bool IsConnectedToInternet()
{
    string host = "...";
    bool result = false;
    Ping p = new Ping();
    try
    {
        PingReply reply = p.Send(host, 3000);
        if (reply.Status == IPStatus.Success)
            return true;
    }
    catch { }

    return result;
}
The only problem in Figure 6 is that the code isn't supported in Silverlight 2. (And likewise, it isn't supported by the NetworkInterface object.) You need to isolate this code (and any other code you spot with potential compatibility problems) in a replaceable class. (For more on this restriction, see the CoreCLR In Silverlight sidebar accompanying this column.) In the companion source code, I create a utility interface for these kinds of potentially problematic methods and then create strategy classes that implement the interface for each platform, as shown in Figure 7.
Figure 7 Component for Checking Internet Connection
public partial class SilverCompatLayer : ICompatLib
{
    public bool IsConnectedToInternet()
    {
        string host = "...";
        bool result = false;
        Ping p = new Ping();
        try
        {
            PingReply reply = p.Send(host, 3000);
            if (reply.Status == IPStatus.Success)
                return true;
        }
        catch { }

        return result;
    }

    public string GetRawQuoteInfo(string symbol)
    {
        string output = String.Empty;
        StreamReader reader = null;

        // Set the URL to invoke
        string url = String.Format(UrlBase, symbol);

        // Connect and get response
        WebRequest request = WebRequest.Create(url);
        WebResponse response = request.GetResponse();

        // Read the response
        using (reader = new StreamReader(response.GetResponseStream()))
        {
           output = reader.ReadToEnd();
           reader.Close();
        }

        return output;
    }

    // A few other methods that require a different implementation
    // (See source code)
    ...
}
The interface here decouples the platform-specific strategy class from the hosting application. By then hiding the code that instantiates the concrete strategy class behind a factory method, the following code can be used safely in both WPF and Silverlight:
private bool IsConnectedToInternet()
{
    ICompatLib layer = ServiceResolver.ResolveCompatLayer();
    return layer.IsConnectedToInternet();
}
The SilverCompatLayer class lives in a separate assembly. This assembly is all that you have to change when porting your WPF code to Silverlight, or vice versa.
After creating the necessary platform-specific strategy classes, all that remains is to create a Silverlight version of the application. The Silverlight project that is derived from the original WPF application contains exact copies of all files except for the compatibility assembly.
You'll notice in the code download associated with this article that I determined which strategy implementation to use by instantiating the concrete class explicitly in a factory method. You can instantiate classes directly like this or read their names from a configuration file and use reflection to get instances. These are implementation details that mostly depend on the type of system you're building. As far as the WPF-to-Silverlight compatibility is concerned, strategies and layering are the key concepts to understand.

Final Considerations
In the sample application, I make a network call. In the original WPF application, the call is synchronous. In Silverlight 2, however, there's no support at all for synchronous network calls. The only way you can make synchronous calls is by using the aforementioned trick based on calling the browser's implementation of XMLHttpRequest. (See the source code for details.)
In the sample code, I've successfully ported my original WPF application to the Web. When porting your code, you might also want to consider the native features of the Silverlight environment and modify the structure of your application when fitting. Note that in my simple example, rewriting the application using the Silverlight programming model would have taken less code and less effort than adapting it from a WPF application.
So WPF and Silverlight have a lot in common when it comes to purely visual things such as XAML. The underlying programming model, however, is a bit different, and solutions that work in WPF cannot always be applied as is to Silverlight and vice versa. This said, sharing XAML and code between WPF and Silverlight applications can definitely be done.

Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is currently an architect at IDesign and the author of Programming ASP.NET 3.5 Core References. Based in Italy, Dino is a frequent speaker at industry events worldwide. You can join his blog at weblogs.asp.net/despos.


Patterns in Practice
Cohesion And Coupling
Jeremy Miller
Contents
Decrease Coupling
Increase Cohesion
Eliminate Inappropriate Intimacy
The Law of Demeter
Tell, Don't Ask
Say It Once and Only Once
Wrapping Up
Much of software design involves the ongoing question, where should this code go? I'm constantly looking for the best way to organize my code to make it easier to write, easier to understand, and easier to change later. If I structure my code well, I'll go on to fame and glory. Structure it badly, and the developers who follow me will curse my name for eternity.
In particular, I would like to achieve three specific things with my code structure:
  1. Keep things that have to change together as close together in the code as possible.
  2. Allow unrelated things in the code to change independently (also known as orthogonality).
  3. Minimize duplication in the code.
To achieve these three goals, I need some tools to help me know where new code should go and some other tools to help me recognize when I've put code in the wrong place.
By and large, these goals are closely related to the classic code qualities of cohesion and coupling. I achieve these goals by moving toward higher cohesion and looser coupling. Of course, first we need to understand what these qualities mean and why coupling and cohesion are helpful concepts. Then I'd like to discuss some of what I'm going to call "design vectors" that help us move toward better structures and recognize when we need to move away from bad structures that have crept into our code.
Just as a note, I'm the primary developer of an open source inversion of control (IOC) tool called StructureMap. I'm going to be using some real-life examples from StructureMap to illustrate the design problems that these vectors address. In other words, don't make the same mistakes that I made.

Decrease Coupling
You can't help but hear the terms "loose coupling" or "tight coupling" in almost any discussion on software design. Coupling among classes or subsystems is a measure of how interconnected those classes or subsystems are. Tight coupling means that related classes have to know internal details of each other, changes ripple through the system, and the system is potentially harder to understand. Figure 1 shows a contrived sample of a business-processing module that is very tightly coupled to various other concerns besides the business logic.
Figure 1 Tightly Coupled Code
public class BusinessLogicClass {
  public void DoSomething()  {
    // Go get some configuration
    int threshold =
      int.Parse(ConfigurationManager.AppSettings["threshold"]);

    string connectionString =
      ConfigurationManager.AppSettings["connectionString"];

    string sql =
      @"select * from things
        size > ";

    sql += threshold;

    using (SqlConnection connection =
      new SqlConnection(connectionString)) {

      connection.Open();

      SqlCommand command = new SqlCommand(sql, connection);
      using (SqlDataReader reader = command.ExecuteReader()) {
        while (reader.Read()) {
          string name = reader["Name"].ToString();
          string destination = reader["destination"].ToString();

          // do some business logic in here
          doSomeBusinessLogic(name, destination, connection);
        }
      }
    }
  }
}
Let's say that what we really care about is the actual business processing, but our business logic code is intertwined with data-access concerns and configuration settings. So what's potentially wrong with this style of code?
The first problem is that the code is somewhat hard to understand because of the way the different concerns are intertwined. I'll discuss this further in the next section on cohesion.
The second problem is that any changes in data-access strategy, database structure, or configuration strategies will ripple through the business logic code as well because it's all in one code file. This business logic knows too much about the underlying infrastructure.
Third, we can't reuse the business logic code independent of the specific database structure or without the existence of the AppSettings keys. We also can't reuse the data-access functionality embedded in the BusinessLogicClass. That coupling between data access and business logic might not be an issue, but what if we want to repurpose this business logic for usage against data entered directly into an Excel spreadsheet by analysts? What if we want to test or debug the business logic by itself? We can't do any of that because the business logic is tightly coupled to the data-access code. The business logic would be a lot easier to change if we could isolate it from the other concerns.
To summarize, the goals behind achieving loose coupling between classes and modules are to:
  1. Make the code easier to read.
  2. Make our classes easier to consume by other developers by hiding the ugly inner workings of our classes behind well-designed APIs.
  3. Isolate potential changes to a small area of code.
  4. Reuse classes in completely new contexts.
Code Smells
It's obviously good to know how to do the right things when designing new code, but it might be even more important to recognize when your existing code or design has developed problems. Like inappropriate intimacy, "code smell" (defined by Martin Fowler in the book Refactoring: Improving the Design of Existing Code) is a tool that you can utilize to spot potential problems in code.
A code smell is a sign that something may be wrong in your code. It doesn't mean that you need to rip out your existing code and throw it away on the spot, but you definitely need to take a closer look at the code that gives off the offending "smell." A code smell can be spotted by simple inspection. Learning about code smells is a powerful tool to remove little problems in your code and class design before those problems become bigger issues.
Many, if not most, of the commonly described code smells are signs of poor cohesion or harmful tight coupling. Here are some other examples:
Divergent Changes A single class that has to be changed in different ways for different reasons. This smell is a sign that the class is not cohesive. You might refactor this class to extract distinct responsibilities into new classes.
Feature Envy A method in ClassA seems way too interested in the workings and data fields of ClassB. The feature envy from ClassA to ClassB is an indication of tight coupling from ClassA to ClassB. The usual fix is to try moving the functionality of the interested method in ClassA to ClassB, which is already closer to most of the data involved in the task.
Shotgun Surgery A certain type of change in the system repeatedly leads to making lots of small changes to a group of classes. Shotgun surgery generally implies that a single logical idea or function is spread out over multiple classes. Try to fix this by pulling all the parts of the code that have to change together into a single cohesive class.

Increase Cohesion
The academic definition of cohesion is that it is a measure of how closely related all the responsibilities, data, and methods of a class are to each other. I like to think of cohesion as a measure of whether a class has a well-defined role within the system. We generally consider high cohesion to be a good thing and repeat the words "highly cohesive" like a mantra. But why?
Let's think of coding as having a conversation with the computer. More accurately, we're having several simultaneous conversations with the computer. We're having conversations about how security is carried out, how the infrastructure concerns are supposed to behave, and what the business rules are.
When you're at a loud party with a lot of different conversations going on at once, it can be difficult to focus on the one conversation you're trying to have. It's much easier to have a conversation in a quiet setting where there's only one conversation going on.
An easy test for cohesion is to look at a class and decide whether all the contents of the class are directly related to and described by the name of the class—vague class names, such as InvoiceManager, do not count. If the class has responsibilities that don't relate to its name, those responsibilities probably belong to a different class. If you find a subset of methods and fields that could easily be grouped separately under another class name, then maybe you should extract these methods and fields to a new class.
To illustrate, if you find methods and data in your TrainStation, Train, and Conductor classes that seem to most accurately fit the theme of the TrainSchedule class, move those methods and data into the TrainSchedule. One of my favorite sayings about design is applicable here: put code where you'd expect to find it. It's most logical to me that functionality involving a train schedule would be in the TrainSchedule class.
To stretch my earlier conversation analogy, a system consisting of cohesive classes and subsystems is like a well-designed online discussion group. Each area in the online group is narrowly focused on one specific topic so the discussion is easy to follow, and if you're looking for a dialog on a certain subject, there's only one room you have to visit.

Eliminate Inappropriate Intimacy
Inappropriate intimacy refers to a method in a class that has too much intimate knowledge of another class. Inappropriate intimacy is a sign of harmful, tight coupling between classes. Let's say that we have a business logic class that calls an instance of class DataServer1 to get the data it needs for its business-logic processing. Figure 2 shows an example. In this case, the Process method has to know a lot of the inner workings of DataServer1 and a bit about the SqlDataReader class.
Figure 2 Inappropriate Intimacy
public void Process() {
  string connectionString = getConnectionString();
  SqlConnection connection = new SqlConnection(connectionString);
  DataServer1 server = new DataServer1(connection);

  int daysOld = 5;
  using (SqlDataReader reader = server.GetWorkItemData(daysOld)) {
    while (reader.Read()) {
      string name = reader.GetString(0);
      string location = reader.GetString(1);

      processItem(name, location);
    }
  }
}
Now, let's rewrite the code in Figure 2 to remove the inappropriate intimacy:
public void Process() {
  DataServer2 server = new DataServer2();
  foreach (DataItem item in server.GetWorkItemData(5)) {
    processItem(item);
  }
}
As you can see in this version of the code, I've encapsulated all the SqlConnection and SqlDataReader object manipulation inside the DataServer2 class. DataServer2 is also assumed to be taking care of its own configuration, so the new Process method doesn't have to know anything about setting up DataServer2. The DataItem objects returned from GetWorkItemData are also strongly typed objects.
Now, let's analyze the two versions of Process method against some of the goals of loose coupling. First, what about making the code easier to read? The first and second versions of Process perform the same basic task, but which is easier to read and understand? Personally, I can more easily read and understand the business logic processing without wading through data-access code.
How about making our classes easier to consume? Consumers of DataServer1 have to know how to create a SqlConnection object, understand the structure of the DataReader returned, and iterate through and clean up the DataReader. A consumer of DataServer2 only has to call a no-argument constructor and then call a single method that returns an array of strongly typed objects. DataServer2 is taking care of its own ADO.NET connection setup and cleaning up open DataReaders.
And as for isolating potential changes to a small area of code? In the first version of the code, almost any change in the way DataServer works would impact the Process method. In the second, more encapsulated version of DataServer, you could switch the data store to an Oracle database or to an XML file without any effect on the Process method.

The Law of Demeter
The Law of Demeter is a design rule of thumb. The terse definition of the law is: only talk to your immediate friends. The Law of Demeter is a warning about the potential dangers of code, as in Figure 3.
Figure 3 Breaking the Law
public interface DataService {
  InsuranceClaim[] FindClaims(Customer customer);
}

public class Repository {
  public DataService InnerService { get; set; }
}

public class ClassThatNeedsInsuranceClaim {
  private Repository _repository;

  public ClassThatNeedsInsuranceClaim(Repository repository) {
    _repository = repository;
  }

  public void TallyAllTheOutstandingClaims(Customer customer) {
    // This line of code violates the Law of Demeter
    InsuranceClaim[] claims =
      _repository.InnerService.FindClaims(customer);
  }
}
The ClassThatNeedsInsuranceClaim class needs to get InsuranceClaim data. It has a reference to the Repository class, which itself has a DataService object. ClassThatNeedsInsuranceClaim reaches inside Repository to grab the inner DataService object, then calls Repository.FindClaims to get its data. Note that the call to _repository.InnerService.FindClaims(customer) is a clear violation of the Law of Demeter because ClassThatNeedsInsuranceClaim is directly calling a method on a property of its Repository field. Now please turn your attention to Figure 4, which shows another sample of the same code, but this time it is following the Law of Demeter.
Figure 4 Better Decoupling
public class Repository2 {
  private DataService _service;

  public Repository2(DataService service) {
    _service = service;
  }

  public InsuranceClaim[] FindClaims(Customer customer) {
    // we're simply going to delegate to the inner
    // DataService for now, but who knows what
    // we want to do in the future?
    return _service.FindClaims(customer);
  }
}

public class ClassThatNeedsInsuranceClaim2 {
  private Repository2 _repository;

  public ClassThatNeedsInsuranceClaim2(
    Repository2 repository) {
    _repository = repository;
  }

  public void TallyAllTheOutstandingClaims(
    Customer customer) {
    // This line of code now follows the Law of Demeter
    InsuranceClaim[] claims = _repository.FindClaims(customer);
  }
}
What did we accomplish? Repository2 is easier to use than Repository because you have a direct method to call for the InsuranceClaim information. When we were violating the Law of Demeter, the consumers of Repository were tightly coupled to the implementation of Repository. With the revised code, I have more ability to change the Repository implementation to add more caching or swap out the underlying DataService with something completely different.
The Law of Demeter is a powerful tool to help you spot potential coupling problems, but don't follow the Law of Demeter blindly. Violating the Law of Demeter does add tighter coupling to your system, but in some cases you may judge that the potential cost of coupling to a stable element of your code is less expensive than writing a lot of delegation code to eliminate the Law of Demeter violations.

Tell, Don't Ask
The Tell, Don't Ask design principle urges you to tell objects what to do. What you don't want to do is ask an object about its internal state, make some decisions about that state, then tell that object what to do. Following the Tell, Don't Ask style of object interaction is a good way to ensure that responsibilities are put in the right places.
Figure 5 illustrates a violation of Tell, Don't Ask. The code is taking a purchase of some sort, checking for the possibility of a discount for purchases of more than $10,000, and finally examining the account data to decide whether there are sufficient funds. The previous DumbPurchase and DumbAccount classes are, well, dumb. The account and purchase business rules are both encoded in ClassThatUsesDumbEntities.
Figure 5 Asking Too Much
public class DumbPurchase {
  public double SubTotal { get; set; }
  public double Discount { get; set; }
  public double Total { get; set; }
}

public class DumbAccount {
  public double Balance { get; set;}
}

public class ClassThatUsesDumbEntities {
  public void MakePurchase(
    DumbPurchase purchase, DumbAccount account) {
    purchase.Discount = purchase.SubTotal > 10000 ? .10 : 0;
    purchase.Total =
      purchase.SubTotal*(1 - purchase.Discount);

    if (purchase.Total < account.Balance) {
      account.Balance -= purchase.Total;
    }
    else {
      rejectPurchase(purchase,
        "You don't have enough money.");
    }
  }
}
This type of code can be problematic in a couple of ways. In a system like this, you potentially have duplication because the business rules for an entity are scattered in procedural code outside of the entities. You might duplicate logic unknowingly because it's not obvious where the previously written business logic would be.
Figure 6 shows the same code, but this time following the Tell, Don't Ask pattern. In this code, I moved the business rules for purchasing and accounts into the Purchase and Account classes themselves. When we go to make a purchase, we just tell the Account class to deduct the purchase from itself. Account and Purchase know about themselves and their internal rules. All a consumer of Account has to know is to call the Account.Deduct(Purchase, PurchaseMessenger) method.
Figure 6 Telling Your App What to Do
public class Purchase {
  private readonly double _subTotal;

  public Purchase(double subTotal) {
    _subTotal = subTotal;
  }

  public double Total {
    get {
      double discount = _subTotal > 10000 ? .10 : 0;
      return _subTotal*(1 - discount);
    }
  }
}

public class Account {
  private double _balance;

  public void Deduct(
    Purchase purchase, PurchaseMessenger messenger) {
    if (purchase.Total < _balance) {
      _balance -= purchase.Total;
    }
    else {
      messenger.RejectPurchase(purchase, this);
    }
  }
}

public class ClassThatObeysTellDontAsk {
  public void MakePurchase(
    Purchase purchase, Account account) {
    PurchaseMessenger messenger = new PurchaseMessenger();
    account.Deduct(purchase, messenger);
  }
}
The Account and Purchase objects are easier to use because you don't have to know quite so much about these classes to execute our business logic. We've also potentially reduced duplication in the system. The business rules for Accounts and Purchases can be effortlessly reused throughout the system because those rules are inside the Account and Purchase classes instead of being buried inside the code that uses those classes. In addition, it should be easier to change the business rules for Purchases and Accounts as those rules are found in just one place in the system.
Closely related to Tell, Don't Ask is the Information Expert pattern. If you have a new responsibility for your system, in what class should the new responsibility go? The Information Expert pattern asks, who knows the information necessary to fulfill this responsibility? In other words, your first candidate for any new responsibility is the class that already has the data fields affected by that responsibility. In the purchasing sample, the Purchase class knows the information about a purchase that you would use to determine any possible discount rates, so the Purchase class itself is the immediate candidate for calculating discount rates.

Say It Once and Only Once
As an industry we've learned that writing reusable code deliberately is very expensive, yet we still try for reuse because of the obvious benefits. It may behoove us to look for duplication in our system and find ways to eliminate or centralize that duplication.
One of the best ways to improve cohesion in your system is to eliminate duplication wherever you spot it. It's probably best if you assume that you don't know exactly how your system is going to change in the future, but you can improve your code's ability to accept change by maintaining good cohesion and coupling in class structures.
I was peripherally involved years ago with a large shipping application that managed the flow of boxes on a factory floor. In its original incarnation, the system polled a message queue for incoming messages, then responded to those messages by applying a large set of business rules to determine where the boxes went next.
The next year the business needed to start the box-routing logic from a desktop client. Unfortunately, the business logic code was far too tightly coupled with the mechanisms for reading and writing to MQ Series queues. It was judged too risky to untangle the original business logic code from the MQ Series infrastructure, so the entire corpus of business rules were duplicated in a parallel library for the new desktop client. That decision made the new desktop client feasible, but it also made all future efforts more difficult because each change to the box-routing logic required a parallel change to two very different libraries—and those kinds of business rules change frequently.
This real scenario presents us with a couple of lessons. Duplication in the code had a real cost to the organization that built the system, and that duplication was largely caused by poor coupling and cohesion qualities in their class structure. This stuff can directly affect a company's profit and loss.
One way of looking at duplication in your code is as an opportunity to improve your design later. If you find yourself with two or more classes that duplicate some piece of functionality, you can say that the duplicated functionality must be a completely different responsibility. One of the best ways to improve the cohesion quality of your codebase is to simply pull out duplication into separate classes that can be shared across the codebase.
I learned the hard way that even innocuous-looking duplication can cause headaches. With the advent of generics in the Microsoft .NET Framework 2.0, many people started creating parameterized Repository classes that looked something like this:
public interface IRepository<T> {
  void Save(T subject);
  void Delete(T subject);
}
In this interface, T would be a domain entity like Invoice or Order or Shipment. Users of StructureMap wanted to be able to call this code and get a fully formed repository object that could handle a specific domain entity, such as an Invoice object:
IRepository<Invoice> repository =
  ObjectFactory.GetInstance<IRepository<Invoice>>();
It sounded like a great feature to have, so I sat down to add support for these kinds of parameterized types. Making this change to StructureMap turned out to be very difficult because of code like this:
_PluginFamilies.Add(family.PluginType.FullName, family);
And code like this:
MementoSource source =
  this.getMementoSourceForFamily(pluginType.FullName);
And code like this as well:
private IInstanceFactory this[Type PluginType] {
  get {
    return this[PluginType.FullName];
  }
  set {
    this[PluginType.FullName] = value;
  }
}
Have you spotted the duplication yet? Not to get too involved with this sample, but I had an implicit rule that stated that objects related to a System.Type were stored by using the Type.FullName property as the key in a Hashtable. It's a tiny little bit of logic, but I had duplicated it all over the codebase. When I implemented generics, I determined that it would be better if I stored objects internally by the actual type instead of Type.FullName.
This seemingly minor change in behavior took me days instead of the hours I assumed up front because of how many times I had duplicated this little bit of data. The lesson learned here is that any rule in your system, no matter how seemingly trivial, should be expressed once, and only once.

Wrapping Up
Cohesion and coupling applies to every level of a design and architecture, but I've mostly focused on fine-grained details at the class and method levels. Sure, you better get the big architectural decisions right—technology selection, project structure, and physical deployment are all important, but usually these are fairly constrained in the number of choices and the trade-offs are generally well understood.
I've found that the cumulative effect of the hundreds and thousands of little decisions you make at the class and method levels add up to have a more profound impact on the success of a project—and you've got a far greater range of choices and alternatives in regard to the little things. While it may not be true of life in general, in software design, sweat the small things.
There's a common attitude with developers that worrying about all this cohesion and coupling stuff is ivory tower theory that detracts from getting the work done. My feeling is that good cohesion and coupling qualities in your code will sustain productivity in your code over time. My strong advice is to internalize your recognition of cohesion and coupling qualities to the point where you don't have to consciously think about these qualities. Moreover, one of the best exercises I can recommend for improving your design skills is to revisit your previous coding efforts, try to find ways you could have improved the old code, and try to remember the elements of your past design that either made changing the code easy or made adjusting the code difficult.

Send your questions and comments to mmpatt@microsoft.com.

Jeremy Miller, a Microsoft MVP for C#, is also the author of the open source StructureMap (structuremap.sourceforge.net) tool for Dependency Injection with .NET and the forthcoming StoryTeller (storyteller.tigris.org) tool for supercharged FIT testing in .NET. Visit his blog, "The Shade Tree Developer," at codebetter.com/blogs/jeremy.miller, part of the CodeBetter site.


Service Station
Authorization In WCF-Based Services
Dominick Baier and Christian Weyer

Code download available at: ServiceStation2008_10.exe (175 KB)
Browse the Code Online

Contents
Role-Based Authorization
ASP.NET Role Provider
Custom Principal
Centralizing Authorization Logic
Claims-Based Authorization
Claim
Claim Sets
AuthorizationContext
Claims Transformation
Security Token Services
Once you start adopting service-oriented principles for your distributed applications, the security challenges become slightly different. You suddenly realize that you are crossing a security boundary for every service call you make.
Typically—though not necessarily—there will be a network between the sender and the recipient of a service call. While authentication is usually handled automatically by the communication framework, you still have to come up with your own authorization strategy and infrastructure.
Windows Communication Foundation (WCF) provides powerful facilities for implementing authorization in services. You have the choice between an easy-to-use, role-based system as well as a more powerful, but more complex, claims-based API. The remainder of this article will compare both systems and show how to use them to implement robust service authorization.

Role-Based Authorization
The idea behind role-based authorization is that you associate a list of roles with a user. At run time, the service code queries the list to make security-related decisions. These roles can come from the Windows security system (where they are called groups) or from some custom store, such as a database.
The roles API in the Microsoft .NET Framework is based on two interfaces called IIdentity (identity information) and IPrincipal (role information). An IIdentity-derived class holds information such as the name of the user, whether he is authenticated, and how he was authenticated.
An IPrincipal-derived class must implement a single method called IsInRole where you can pass in a role name and get a Boolean response. In addition, the principal has a reference to the identity class it wraps.
The .NET Framework ships with several implementations of theses interfaces. WindowsIdentity and WindowsPrincipal wrap details about a Windows user. GenericIdentity and GenericPrincipal can be used to represent a custom, authenticated user.
There is also a location where you can store a principal to associate it with the currently executing thread in your application (for example, a WCF operation request). This is the CurrentPrincipal thread static property on the System.Threading.Thread class. The typical course of events is that one part of an application populates Thread.CurrentPrincipal so that another part of the application can reach into this property to grab the IPrincipal implementation stored there. Then the IsInRole method can be called on that class to query the role list of the current user to ensure that the caller is authorized for the operation.
This is exactly how it works in WCF. Based on the configured authentication and credential type, WCF creates a corresponding IIdentity implementation—a WindowsIdentity for Windows authentication, a GenericIdentity for most other cases. Then, based on the configuration of the WCF service behavior, ServiceAuthorization, an IPrincipal implementation is created that wraps this identity and provides role information. This IPrincipal is then set on Thread.CurrentPrincipal so that it is accessible to the service operations.
Aside from programmatically calling IsInRole to perform role-based security checks, there is also an attribute called PrincipalPermissionAttribute that allows you to annotate your service operations with role requirements. PrincipalPermissionAttribute will signal a negative authorization outcome to your client by throwing a SecurityException prior to executing the service operation. WCF will catch this exception and turn it into an Access Denied fault message that is returned. The WCF client plumbing will turn this special fault into a .NET exception of type SecurityAccessDeniedException. Figure 1 shows the various ways to do a role check and perform error handling.
Figure 1 Role Check with Error Handling

Service
class Service : IService {
  // only 'users' role member can call this method
  [PrincipalPermission(SecurityAction.Demand, Role = 'users')]
  public string[] GetRoles(string username) {
    // only administrators can retrieve the role information for other users
    if (ServiceSecurityContext.Current.PrimaryIdentity.Name != username) {
      if (Thread.CurrentPrincipal.IsInRole('administrators')) {
        ...
      }
      else {
        // access denied
        throw new SecurityException();
      }
    }
  }
}

Client
var factory = new ChannelFactory<IService>('*');
factory.Credentials.UserName.UserName = 'bob';
factory.Credentials.UserName.Password = 'bob';
var proxy = factory.CreateChannel();

try {
  Console.WriteLine('\nBob: roles for Bob --');
  proxy.GetRoles('bob').ToList().ForEach(i => Console.WriteLine(i));

  Console.WriteLine('\nBob: roles for Alice --');
  proxy.GetRoles('alice').ToList().ForEach(i => Console.WriteLine(i));
}

catch (SecurityAccessDeniedException) {
  Console.WriteLine('Access Denied\n');
}
The ServiceAuthorization service behavior controls the creation of the IPrincipal instance to be associated with the request thread. By default, WCF assumes Windows authentication and tries to populate Thread.CurrentPrincipal with a WindowsPrincipal.
When your clients are not Windows users, you have the choice between getting roles from an ASP.NET role provider or implementing a custom authorization policy in order to get the roles from a custom database. Another alternative is to implement a custom IPrincipal type and then take complete control over the IsInRole implementation.

ASP.NET Role Provider
WCF can use an ASP.NET role provider to retrieve roles for a user. You can either use one of the built-in providers (for SQL Server or Microsoft Authorization Manager) or write one of your own by deriving from System.Web.Security.RoleProvider.
To make the connection to the role provider, WCF attaches a RoleProviderPrincipal to the executing thread that forwards all calls to IsInRole to the role provider's IsUserInRole method. This method takes the user name and role in question and returns true if the user is a member of that role and false if not.
Figure 2 shows a sample custom role provider along with the necessary configuration entries. WCF will call the role provider for every role check that occurs in your service. If you have a lot of role checks, it would be beneficial to implement some sort of caching strategy to avoid excessive round-trips to the role store.
Figure 2 Sample Role Provider and Configuration Entries

Role Provider
class CustomRoleProvider : RoleProvider {
  public override string[] GetRolesForUser(string username) {
    if (username == 'administrator') {
      return new string[] { 'administrators', 'users' };
    }
    else {
      return new string[] { 'sales', 'marketing', 'users' };
    }
  }

  public override bool IsUserInRole(string username, string roleName)  {
    return GetRolesForUser(username).Contains(roleName);
  }
  ...
}

Configuration
<behaviors>
  <serviceBehaviors>
    <behavior name='security'>
      <serviceAuthorization
        principalPermissionMode='UseAspNetRoles'
        roleProviderName='CustomProvider'
      </serviceAuthorization>
    </behavior>
  </serviceBehaviors>
</behaviors>

<system.web>
  <roleManager
    enabled='true'
    defaultProvider='CustomProvider'>
    <providers>
      <add name='CustomProvider'
        type='Service.CustomRoleProvider, Service' />
    </providers>
  </roleManager>
</system.web>
Customizations like caching could be implemented in the role provider itself or via a custom IPrincipal implementation. That's the next option we will explore.

Custom Principal
Another option is to supply your custom IPrincipal implementation to WCF. This gives you the chance to implicitly run code after the authentication stage of each request. For this you have to create your own custom principal and return it to the WCF plumbing. The custom principal will then be available from Thread.CurrentPrincipal to the service code. Custom principals allow full customization of role-based security and expose specialized security logic for the service developer to use.
Writing a custom principal is straightforward. Simply implement the IPrincipal interface and add your own custom functionality. To integrate the principal with WCF, you have to set the PrincipalPermissionMode attribute in the ServiceAuthorization element to "custom" and provide an authorization policy that is responsible for creating the principal and giving it back to WCF.
An authorization policy is simply a class that implements the System.IdentityModel.Policy.IAuthorizationPolicy interface. Implementations of this interface must employ a method called Evaluate, which gets called on every request. Here you can reach into the WCF service security context and set the custom principal. Figure 3 shows a custom principal as well as the authorization policy and its corresponding configuration entries.
Figure 3 A Custom Principal and Authorization Policy

Principal
class CustomPrincipal : IPrincipal {
  IIdentity _identity;
  string[] _roles;
  Cache _cache = HttpRuntime.Cache;

  public CustomPrincipal(IIdentity identity) {
    _identity = identity;
  }

  // helper method for easy access (without casting)
  public static CustomPrincipal Current {
    get {
      return Thread.CurrentPrincipal as CustomPrincipal;
    }
  }

  public IIdentity Identity {
    get { return _identity; }
  }

  // return all roles (custom property)
  public string[] Roles {
    get {
      EnsureRoles();
      return _roles;
    }
  }

  // IPrincipal role check
  public bool IsInRole(string role) {
    EnsureRoles();

    return _roles.Contains(role);
  }

  // cache roles for subsequent requests
  protected virtual void EnsureRoles() {
    // caching logic omitted – see the sample download
  }
}

Authorization Policy
class AuthorizationPolicy : IAuthorizationPolicy {
  // called after the authentication stage
  public bool Evaluate(EvaluationContext evaluationContext,
     ref object state) {
    // get the authenticated client identity from the evaluation context
    IIdentity client = GetClientIdentity(evaluationContext);

    // set the custom principal
    evaluationContext.Properties['Principal'] =
       new CustomPrincipal(client);

    return true;
  }

  // rest omitted
}

Configuration
<behaviors>
  <serviceBehaviors>
    <behavior name='security'>
      <serviceAuthorization principalPermissionMode='Custom'>
        <authorizationPolicies>
          <add policyType='Service.AuthorizationPolicy, Service' />
        </authorizationPolicies>
      </serviceAuthorization>
    </behavior>
  </serviceBehaviors>
</behaviors>

Centralizing Authorization Logic
So far you have seen how you can access the role information that is provided by the WCF security system from within your service operations. Sometimes, however, it is also useful to have centralized logic that can inspect every incoming request and make authorization decisions without having to spread that logic over all your service operations. Enter the service authorization manager.
The service authorization manager is a class that derives from System.ServiceModel.ServiceAuthorizationManager. You can override the CheckAccessCore method to run custom authorization code for each request. In CheckAccess, you have access to the current security context as well as the incoming message, including the headers. When you return false from CheckAccess, WCF will create the Access Denied fault message and send that back to the client. A return value of true will grant access to the service operation.
You can find the unique identifier of the operation that the client tries to call in the WS-Addressing action header. This value can be obtained from IncomingMessageHeader.Action property on the operation context that is passed into CheckAccess. The format of the action value is:
ServiceNamespace/ContractName/OperationName
For example, you might see something like this:
urn:msdnmag/ServiceContract/GetRoles
WCF passes in the complete request message as a ref parameter. This allows inspecting and even changing the message before it reaches the service operation (or gets bounced back because of a negative authorization outcome).
Be aware that messages in WCF are always read-once. That means you first have to create a copy of the message before you inspect the body. This can be done by creating a message buffer using the following code (you need to be careful with large or streamed messages):
MessageBuffer buffer =
  operationContext.RequestContext.RequestMessage.CreateBufferedCopy(
  int.MaxValue);
Once you have created a copy of the message, you can use the standard APIs to get access to its content. In Figure 4, you can see a sample implementation of a service authorization manager that moves the authorization logic from the service operation to a central place.
Figure 4 A Service Authorization Manager

Code
class AuthorizationManager : ServiceAuthorizationManager {
  public override bool CheckAccess(
    OperationContext operationContext, ref Message message) {

    base.CheckAccess(operationContext, ref message);
    string action = operationContext.IncomingMessageHeaders.Action;

    if (action == 'urn:msdnmag/IService/GetRoles') {
      // messags in WCF are always read-once
      // we create one copy to work with, and one copy for WCF
      MessageBuffer buffer = operationContext.RequestContext.RequestMessage.
        CreateBufferedCopy(int.MaxValue);
      message = buffer.CreateMessage();

      // get the username value using XPath
      XPathNavigator nav = buffer.CreateNavigator();
      StandardNamespaceManager nsm = new
        StandardNamespaceManager(nav.NameTable);
      nsm.AddNamespace('msdn', 'urn:msdnmag');

      XPathNavigator node = nav.SelectSingleNode
        ('s:Envelope/s:Body/msdn:GetRoles/msdn:username', nsm);
      string parameter = node.InnerXml;

      // check authorization
      if (operationContext.ServiceSecurityContext.PrimaryIdentity.Name ==
        parameter) {
        return true;
      }
      else {
        return (GetPrincipal(operationContext).IsInRole(
          'administrators'));
      }
    }
  return true;
  }
  // rest omitted
}

Configuration
<serviceAuthorization
  serviceAuthorizationManagerType='Service.AuthorizationManager, Service'>

Claims-Based Authorization
Once you reach a critical mass of services and complexity, role-based security may not be powerful or flexible enough. It doesn't take much for an enterprise scale, service-oriented application to become extremely complicated. Various clients using various client frameworks and credential types talk to various back-end services. These back-end services, in turn, call other services. Sometimes this is done in a trusted subsystem fashion where only the direct caller is authorized. Alternately, sometimes the original caller identity has to be forwarded throughout the call chain. The situation starts to get really complex as soon as external clients (such as partners or customers) need access to some of your internal services.
It quickly becomes apparent that, in a more complex system, the capabilities of IIdentity and IPrincipal may not be sufficient to model identity and authorization data. Role-based security is limited to binary decisions and does not allow arbitrary data to be associated with a subject. A technology-neutral format is needed that allows describing the identities of entities participating in your distributed system.
This is why the .NET Framework, starting in version 3.0, contains a new identity and access-control API that can handle these requirements with a claims-based approach. You can find this new API in the System.IdentityModel assembly and namespace. Furthermore, Microsoft just recently released a preview of its new identity framework code-named "Zermatt." This new framework builds upon the classes and concepts found in System.IdentityModel and makes it easier to build claims-based security into services and applications. It is important to note that System.IdentityModel as well as Zermatt are not bound in any way to WCF and can be used to claims enable any .NET application. It just happens that these APIs are deeply integrated into WCF.
The most important structural classes from the System.IdentityModel namespace that you need to understand are called Claim, ClaimSet, AuthorizationContext, and AuthorizationPolicy. We will now have a closer look at each of them and explore how they are integrated into the WCF security system.

A claim is a piece of information that can be associated with an entity in your system. This is most commonly a user but could also be a service or some resource. A claim consists of three pieces of information: a claim type, the claim content, and whether the claim describes the identity of the subject or a capability of the subject. This data structure is represented by a class known as System.IdentityModel.Claims.Claim. Furthermore, Claim is a DataContract, which makes it serialization-friendly (important when you plan to transmit claims over service boundaries):
[DataContract(Namespace =
  'http://schemas.xmlsoap.org/ws/2005/05/identity')]
public class Claim {
  [DataMember(Name = 'ClaimType')]
  public string ClaimType;

  [DataMember(Name = 'Resource')]
  public object Resource;
  [DataMember(Name = 'Right')]
  public string Right;
}
ClaimType is a URI that identifies the claim type. While you can come up with your own type URI, several standard claim types are available from the ClaimTypes class. A claim representing a name could use the URI value held by ClaimTypes.Name:
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name
The Resource property contains the actual claim value. Note that the Resource is of type object—this means that you could associate arbitrarily complex information with that claim (from a simple string to a complete graph). Just keep in mind that this may become a problem when it comes to serialization, and if that is a concern, you should probably stick to primitive types. The purpose of the Right property will become clearer once we talk about claim sets.
You can create a new claim by either using the Claim constructor or taking advantage of one of the static methods on the Claim class (for standard claim types):
Claim purchaseLimitClaim = new Claim(
  'http://www.leastprivilege.com/claims/purchaselimit',
  5000,
  Rights.PossessProperty);
Claim nameClaim = Claim.CreateNameClaim('Alice');

A claim usually doesn't arrive by itself. Claims are typically grouped in claim sets. The class ClaimSet holds a list of claims as well as a reference to the issuer of those claims:
[DataContract(Namespace='http://schemas.xmlsoap.org/ws/2005/05/identity')]
public abstract class ClaimSet : IEnumerable<Claim>, IEnumerable {
  public abstract ClaimSet Issuer { get; }
  public abstract Claim this[int index] { get; }
  public static ClaimSet System { get; }
  public static ClaimSet Windows { get; }
}
The issuer is an important concept once your distributed system gets more complex. Claims could come from various sources such as the current service, the calling service, or a security token service. Having an issuer associated with the set allows the service to distinguish between claim sources (which may also influence trust decisions).
An issuer is also described as a claim set, and System.IdentityModel ships with two predefined issuer claim sets called System (for claims coming from the system) and Windows (for claims coming from the Windows security subsystem). Both are available as static properties on the ClaimSet class.
Another important concept is the claim set's identity. A claim set typically needs a single claim that uniquely identifies the subject it describes. This is where the Right property on the Claim class comes into play. A claim set should have a single claim that has a Right value of Identity (this is the unique identifier) and a number of claims with a Right value of PossessProperty (additional claims describing the subject).
You can create your own claim sets by creating a DefaultClaimSet instance or, when you need more control, by deriving from the abstract class ClaimSet:
DefaultClaimSet myClaimSet =
  new DefaultClaimSet(ClaimSet.System, new List<Claim> {
  new Claim(ClaimTypes.Name, 'Alice', Rights.Identity),
  new Claim(purchaseLimitClaimType, 5000, Rights.PossessProperty),
  Claim.CreateMailAddressClaim(new MailAddress('alice@leastprivilege.com'))
  });
System.IdentityModel ships with two claim sets for converting Windows tokens and X.509 certificates to claims—these are WindowsClaimSet and X509CertificateClaimSet, respectively. These two claim sets are used by WCF for the Windows and Certificate client credential types.
ClaimSet provides two processing primitives for querying the claims: FindClaims and ContainsClaim. FindClaims returns a collection of claims for the specified claim type, whereas ContainsClaim gives you information about the existence of a specific claim.
Due to the general-purpose nature of claim sets, you will typically end up writing your own domain-specific extensions that work against the claim set data structure. Extension methods in C# 3.0 are a convenient way to extend these types with custom functionality. (I have written a library of extension methods for Claim, ClaimSet, and related types that you can download from leastprivilege.com/IdentityModel.)

AuthorizationContext
The last missing piece in the IdentityModel puzzle that you need to know before doing some real work with claims is the authorization context, which acts as a container for claim sets as well as transformation policies (more on this later). WCF makes the authorization context available via the thread-static ServiceSecurityContext. You can use the code snippet shown in Figure 5 to dump the claim sets and claims that are associated with the current service operation request.
Figure 5 Finding Claims and Claim Sets
public void ShowAuthorizationContext() {
  AuthorizationContext context =
    ServiceSecurityContext.Current.AuthorizationContext;

  foreach (ClaimSet set in context.ClaimSets) {
    Console.WriteLine('\nIssuer:\n');
    Console.WriteLine(set.Issuer.GetType().Name);
    foreach (Claim claim in set.Issuer) {
      ShowClaim(claim);
    }

    Console.WriteLine('\nIssued:\n');
    Console.WriteLine(set.GetType().Name);
    foreach (Claim claim in set) {
      ShowClaim(claim);
    }
  }
}

private void ShowClaim(Claim claim) {
  Console.WriteLine('{0}\n{1}\n{2}\n',
    claim.ClaimType,
    claim.Resource,
    claim.Right);
}
When you place this code into a WCF service, you will see different output depending on the configured client credential type. If Windows authentication is enabled, the WCF-generated claim set will contain the user's SID (identity claim), the group's SIDs, and the user name. For a request that is authenticated using a client certificate, the claim set will contain claims that describe the subject name, public key, thumbprint (identity claim), expiration dates, and so on. For users that authenticate with a simple user name/password pair, there will be only a single user name identity claim.
So you can see that the WCF integrated claims layer converts technology-specific identity information, such as a Windows token or certificate, to a general-purpose data structure that can be queried using standard APIs. This makes writing services that have to support multiple credential types much easier—you are not hardwired to any technology-specific APIs anymore. If new credential types are added to WCF, the corresponding plumbing will take care of converting that proprietary format to claims as well.

Claims Transformation
Service operations typically don't care about the user's SID or certificate thumbprint, but they do care about domain-specific identity information such as a user identifier, e-mail address, or purchase limit. Claims transformation is the process of transforming the technology-specific identity details into application-specific ones.
Now that you have all identity information in a common format, it is easy to parse and analyze the claims information to create new claims that make more sense in the application context. Based on the identity claim of the WCF-generated claim set, you can map the request to an application user ID and add the relevant identity and authorization information to a new claim set. The service operation would then simply reach into the WCF authorization context to retrieve the information in which it is interested, as illustrated in Figure 6.
Figure 6 Claims Transformation
Claims transformation is accomplished in an authorization policy. We have already used an authorization policy to create a custom principal, but this time it has a different purpose. Authorization policies can take part in the claims-generation process and run after the WCF internal claims generation is completed. That means the claims associated with the caller's credentials are already accessible to you.
You can query the existing claims for identity information and, based on that, create a new claim set that models your domain-specific claims. Then you can add it to the list of claim sets. This list will be used to create the authorization context before the request reaches the service operation.
There are three members on the IAuthorizationPolicy interface that you have to implement. Id returns a unique identifier for the policy (usually a GUID), and Issuer returns a claim set describing the issuer of the claims that this policy creates. The most important method, however, is Evaluate. This method receives an evaluation context, which basically represents the authorization context while it is still in the process of being built.
You get access to all currently generated claims via EvaluationContext.ClaimSets. Don't try to touch the ServiceSecurityContext from within the Evaluate method, as this will trigger the authorization policies again and you'll end up in an infinite loop. You can add a new claim set to the evaluation context by calling EvaluationContext.AddClaimSet—this claim set will become part of the authorization context later on.
Figure 7 shows a sample authorization policy that implements a common pattern. It first retrieves the identity claim of the first claim set. This claim is sent to a mapping component, which inspects the claim and returns an application user ID. After that the policy hits a data store using the ID to retrieve the information that should become part of the claim set. This information is then packaged as a new claim set and added to the evaluation context.
Figure 7 An Authorization Policy
class CustomerAuthorizationPolicy : IAuthorizationPolicy {
  Guid _id = Guid.NewGuid();
  // custom issuer claim set
  ApplicationIssuerClaimSet _issuer = new ApplicationIssuerClaimSet();

  public bool Evaluate(EvaluationContext evaluationContext,
    ref object state) {
    Claim id = evaluationContext.ClaimSets.FindIdentityClaim();
    string userId = Map(id);

    evaluationContext.AddClaimSet(this, new CustomerClaimSet(userId));

    return true;
  }

  public ClaimSet Issuer {
    get { return _issuer; }
  }

  public string Id {
    get { return 'CustomerAuthorizationPolicy: ' + _id.ToString(); }
  }
}
The final step is to add the authorization policy to the serviceAuthorization behavior configuration. You can add multiple policies, and they are invoked in the order you add them. This means you can write multi-step authorization policies where one policy relies on values added by a previous one. This is pretty powerful for more complex scenarios:
<serviceAuthorization>
  <authorizationPolicies>
    <add policyType='LeastPrivilege.CustomerAuthorizationPolicy, Service' />
    <add policyType='some_other_policy' />
  </authorizationPolicies>
</serviceAuthorization>
If you ran the previous code again to inspect the authorization context, you will see the new claim set and claims. The service operation would use similar code to check the claims for authorization (see Figure 8).
Figure 8 Checking Claims Authorization
public void PlaceOrder(Order order) {
  int purchaseLimit = GetPurchaseLimit();

  if (Order.Total > purchaseLimit) {
    // do appropriate action
  }
}

private int GetPurchaseLimit() {
  AuthorizationContext context =
    ServiceSecurityContext.Current.AuthorizationContext;

  foreach (ClaimSet set in context.ClaimSets) {
    foreach (Claim claim in set.FindClaims(
      Constants.PurchaseLimitClaimType,
      Rights.PossessProperty)) {

      return int.Parse(claim.Resource.ToString());
    }
  }

  throw new Exception('Claim not found');
}
The service authorization manager also works with claims-based authorization. WCF passes the operation context into the CheckAccessCore method. From there you can reach into the service security context that will give you access to the authorization context. This allows you to centralize certain authorization decisions in a single place.

Security Token Services
A security token service (STS) is a tool that allows further consolidation of security logic. The typical task of an STS is to authenticate users and create a security token that, in turn, can contain claims. Clients must first authenticate with the STS and then forward the returned token to the service with which the client wants to communicate.
Since the STS knows about that service (this information is part of the token request), it can do central authorization as well as pre-generate the claims on which the service relies. This way the claims transformation does not need to happen on the service endpoint at all but can be done centrally by the STS. This can dramatically streamline your security infrastructure when your system reaches a certain level of complexity.
A security token service is also an important infrastructure component when it comes to federating multiple trust domains. By establishing trust between several token services, you can exchange security tokens over the trust boundary that can be used by services.
WCF has automatic client/service-side support for the previous scenario as well as all the base classes needed to write an STS. But correctly implementing all the related WS-* specs is a complicated task. Instead, you should either buy a commercial STS or use a higher-level toolkit, such as Zermatt, to write a custom one. The upcoming version of Microsoft Active Directory Federation Services is designed to be a full-featured STS for WCF.

Send your questions and comments to sstation@microsoft.com.

Dominick Baier is a security consultant at thinktecture (www.thinktecture.com) in Germany. In addition, he is the security and WCF curriculum lead at DevelopMentor (develop.com), a developer security MVP, and the author of Developing More-Secure Microsoft ASP.NET 2.0 Applications. You can find his blog at leastprivilege.com.

Christian Weyer is cofounder and principal architect of thinktecture and has been modeling and implementing distributed applications with Java, COM, DCOM, COM+, Web Services, WCF, and other technologies. Get in touch with him at www.thinktecture.com/staff/christian.


Foundations
Managing State With Durable Services
Juval Lowy

Code download available at: Foundations2008_10.exe (240 KB)
Browse the Code Online

Contents
Instance ID and Durable Storage
Explicit Instance ID
Context Binding for Instance ID
Standard Key for Context ID
Automatic Durable Behavior
Persistence Providers
Long-running processes are common in distributed computing. Some business processes are made up of multiple execution sequences which may last many days or even weeks. They may involve clients (or even end users) that connect to the application, perform a small amount of work, transition the workflow to a new state, and then disconnect for an indeterminate amount of time before connecting again and continuing the workflow. The clients may at any point decide to terminate the workflow and start a new one, or the back-end workflow service may end it. With this intermittent flow, do you keep proxies and services in memory waiting for the clients to call?
In this column I will discuss several techniques for supporting such long-running services and answer some of these common questions. I will also present the accompanying Windows Communication Foundation (WCF) facilities and a few helper classes I have developed.

Instance ID and Durable Storage
With long-running services, there is obviously little point in keeping proxies and services in memory while waiting around. Such an approach will not withstand the test of time; at the very least, timeout issues will eventually cause the connection to terminate, and there is no easy way to allow machines on either side to reboot or log off. The solution is to avoid keeping the service state in memory, and to handle each call on a new instance with its own temporary in-memory state. For every operation, the service should retrieve its state from some durable storage (such as a file or a database), perform the requested unit of work for that operation, and save the state back to the durable storage at the end of the call. A service that operates in this manner is called a durable service. Because the storage can be shared, durable services can also scale across multiple machines, be it for scalability, redundancy or maintenance purposes.
Before the actual work of the long-running process begins, the service must write its state to the durable storage so that subsequent operations will be able to find it. When the workflow ends, the service must remove its state; otherwise, over time the store will become bloated with unnecessary state information.
Since a new service instance is created for every operation, the instance must have a way of looking up and loading its state from the durable storage. The client must therefore provide some state identifier for the instance. That identifier is called the instance ID. To support clients that occasionally connect to the service and client applications or even machines that recycle between calls, the client will typically save the instance ID in its own durable storage and provide that ID for every call. When the workflow ends, the client can discard that ID. It is important that your instance ID is serializable and equatable. The ID must be serializable because the service will need to save the ID along with its state into the durable storage. It must be equatable so the service can look up the state from the storage.
The durable storage is usually some kind of a dictionary that pairs the instance ID with the instance state. The service that uses it typically uses a single data structure to store all of its state rather than storing individual instance members with unique IDs. The service often then uses a dedicated helper class or a structure to aggregate all its member variables, and stores and retrieves that type from the durable storage in one operation. Access to the durable storage itself must be thread-safe and synchronized because multiple instances may try to access and modify the store concurrently.
To help you implement and support simple durable services, I wrote the FileInstanceStore<ID,T> class that you can see in Figure 1. FileInstanceStore<ID,T> is a general-purpose, file-based instance store that takes two type parameters: the ID type parameter, which is constrained to be an equable type, and the T type parameter, which represents the instance state. Additionally both types must be serializable.
Figure 1 FileInstanceStore
public interface IInstanceStore<ID,T> where ID : IEquatable<ID>
{
   void RemoveInstance(ID instanceId);
   bool ContainsInstance(ID instanceId);
   T this[ID instanceId]
   {get;set;}
}

public class FileInstanceStore<ID,T> : IInstanceStore<ID,T>
                                       where ID : IEquatable<ID>
{

   public FileInstanceStore(string fileName);

   //Rest of the implementation
}
FileInstanceStore<ID,T> provides a simple indexer, allowing you to read and write the instance state to the file. You can also check for the presence of the instance state in the file or remove it.

Explicit Instance ID
The simplest way a client can provide the instance ID to the service is as an explicit parameter for every operation that will access the state. Figure 2 demonstrates such a client and service. The service is a per-call pocket calculator with durable memory stored in a file.
Figure 2 Calculator with Explicit Instance ID
[ServiceContract]
interface ICalculator
{
   [OperationContract]
   double Add(double number1,double number2);

   /* More arithmetic operations */

   //Memory management operations

   [OperationContract]
   void MemoryStore(string instanceId,double number);

   [OperationContract]
   void MemoryClear(string instanceId);

}

[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
class MyCalculator : ICalculator
{
   static IInstanceStore<string,double> Memory = new
     FileInstanceStore<string,double> (Settings.Default.MemoryFileName);

   public double Add(double number1,double number2)
   {
      return number1 + number2;
   }
   public void MemoryStore(string instanceId,double number)
   {
      lock(typeof(MyCalculator))
      {
         Memory[instanceId] = number;
      }
   }
   public void MemoryClear(string instanceId)
   {
      lock(typeof(MyCalculator))
      {
         Memory.RemoveInstance(instanceId);
      }
   }
   //Rest of the implementation
}
All instances of the calculator use the same static memory in the form of a FileInstanceStore<string,double>. The calculator synchronizes access to the memory in every operation across all instances by using a lock on the service type. Clearing the memory signals the end of the workflow, so the calculator purges its state from storage.

Context Binding for Instance ID
Instead of explicit parameters, you could use the dedicated context bindings introduced with the Microsoft .NET Framework 3.5. These bindings, BasicHttpContextBinding, NetTcpContextBinding, and WSHttpContextBinding, are to be found in the System.WorkflowServices.dll assembly. These context bindings each derive from their regular respective protocol-specific bindings, yet add support for a dedicated context management protocol. These bindings can be used with or without a context.
The context management protocol lets you pass (in the message headers) a dictionary of pairs of key/value strings, referred to as a context. This represents your application-specific execution context, which is analogous to the built-in security and operation contexts. With a context binding you can only set the context to pass to the service once, before opening the proxy (or using it for the first time). After that, the dictionary is cached, and any attempt to modify it will result in an error. The client sets the context to send to the service using the IContextManager interface, defined as:
public interface IContextManager
{
   IDictionary<string,string> GetContext();
   void SetContext(IDictionary<string,string> context);
   //More members
}
The client obtains the reference to the IContextManager interface by accessing the proxy's InnerChannel property, which supports the IChannel interface with the GetProperty<T>() method:
[ServiceContract]
interface IMyContract
{
   [OperationContract]
   void MyMethod();
}
class MyContractClient : ClientBase<IMyContract>,IMyContract
{...}

MyContractClient proxy = new MyContractClient();
IContextManager contextManager =
    proxy.InnerChannel.GetProperty<IContextManager>();
Once the client obtains IContextManager, it can clone the current context by calling the GetContext method. Since the dictionary returned from GetContext is a copy of the actual context, the client cannot use it to change the context. Instead, the client needs to call the SetContext method and provide the new context. The client can override the old context or just add values to it and then set the context again, as shown in Figure 3.
Figure 3 Setting the Context on the Proxy
MyContractClient proxy = new MyContractClient();
IContextManager contextManager =
    proxy.InnerChannel.GetProperty<IContextManager>();

IDictionary<string,string> context = contextManager.GetContext();
context["NumberContext"] = "123";
contextManager.SetContext(context);

proxy.MyMethod();

proxy.Close();
The service reads the context values from the incoming message properties, accessed via the operation context:
public sealed class OperationContext : ...
{
   public MessageProperties IncomingMessageProperties
   {
      get;
   }
   //More members
}
MessageProperties is a non-type safe dictionary that accepts a string key and returns the matching object value:
public sealed class MessageProperties : IDictionary<string,object>
{...}
To obtain the context property, the service uses the static string ContextMessageProperty.Name. This then returns an object of the type ContextMessageProperty, which is defined like so:
[Serializable]
public class ContextMessageProperty : IMessageProperty
{
   public IDictionary<string,string> Context
   {get;}
   public static string Name
   {get;}

   //More members
}
The Context property of ContextMessageProperty is the same dictionary of parameters passed by the client. Figure 4 shows the required service-side steps to read the number context passed in Figure 3. You can streamline the steps required by the client to write to the context using my ContextManager static helper class, which is shown in Figure 5.
Figure 4 Reading the Context by the Service
class MyService : IMyContract
{
   public void MyMethod()
   {
      ContextMessageProperty contextProperty =
          OperationContext.Current
          .IncomingMessageProperties[ContextMessageProperty.Name]
          as ContextMessageProperty;

      string number = contextProperty.Context["NumberContext"];

      Debug.Assert(number == 123);
   }
}
Figure 5 Client-Side Methods of ContextManager
public static class ContextManager
{
   public static void SetContext(IClientChannel innerChannel,
                                 string key, string value) {
      SetContext(innerChannel, UpdateContext(innerChannel, key, value));
   }

   public static void SetContext(IClientChannel innerChannel,
                                 IDictionary<string, string> context) {
      Debug.Assert((innerChannel as ICommunicationObject).State !=
                   CommunicationState.Opened);
      IContextManager contextManager =
         innerChannel.GetProperty<IContextManager>();
      contextManager.SetContext(context);
   }

   public static IDictionary<string, string> UpdateContext(
      IClientChannel innerChannel, string key, string value) {
      IContextManager contextManager =
         innerChannel.GetProperty<IContextManager>();
      IDictionary<string, string> context =
         new Dictionary<string, string>(contextManager.GetContext());
      context[key] = value;
      return context;
   }

   //Proxy extensions

   public static void SetContext<T>(this ClientBase<T> proxy,
             IDictionary<string, string> context) where T : class {
      SetContext(proxy.InnerChannel, context);
   }

   public static void SetContext<T>(this ClientBase<T> proxy,
                        string key, string value) where T : class {
      SetContext(proxy.InnerChannel, key, value);
   }

   //More helper methods
}
ContextManager offers overloaded versions of the SetContext methods, allowing the client to set a new context on a proxy's inner channel, using a single key/value pair or a collection of pairs using a dictionary. These methods are useful with both a proxy class and a channel factory. ContextManager also allows you to set the context as a proxy class extension. You can use the CreateContext method to create a new dictionary or add a key/value pair to an existing context. By using ContextManager, Figure 3 is reduced to:
MyContractClient proxy = new MyContractClient();
proxy.SetContext("NumberContext","123");
proxy.MyMethod();
proxy.Close();
However, relying on SetContext this way requires you to explicitly use it with every instantiation of the proxy. It is better to encapsulate ContextManager in a dedicated proxy class, such as my ContextClientBase<T> proxy class:
public abstract class ContextClientBase<T> : ClientBase<T>
where T : class
{
   public ContextClientBase();
   public ContextClientBase(string endpointName);
   public ContextClientBase(string key,string value);
   public ContextClientBase(IDictionary<string,string> context);
   public ContextClientBase(string key,string value,
                                           string endpointName);
   public ContextClientBase(IDictionary<string,string> context,
                                           string endpointName);
   //More constructors
}
The constructors of ContextClientBase<T> accept the usual proxy parameters such as endpoint name or binding and address, but in addition accept contextual parameters to send to the service: either a single key/value pair or a collection of them using a dictionary. Your proxy can derive directly from ContextClientBase<T>:
class MyContractClient : ContextClientBase<IMyContract>,IMyContract
{
   public MyContractClient(string key,string value) : base(key,value)
   {}
   /* More constructors */
   public void MyMethod()
   {
      Channel.MyMethod();
   }
}
Using ContextClientBase<T>, Figure 3 is reduced to:
MyContractClient proxy = new MyContractClient("NumberContext","123");
proxy.MyMethod();
proxy.Close();
Figure 6 shows the implementation of ContextClientBase<T>.
Figure 6 Implementing ContextClientBase<T>
public abstract class ContextClientBase<T> : ClientBase<T>
                                             where T : class
{
   public ContextClientBase(string key,string value,string endpointName):
              this(ContextManager.CreateContext(key,value),endpointName){}

   public ContextClientBase(IDictionary<string,string> context, string endpointName) : base(endpointName){
      SetContext(context);
   }

   /* More constructors */

   void SetContext(IDictionary<string,string> context){
      VerifyContextBinding();
      ContextManager.SetContext(InnerChannel,context);
   }

   void VerifyContextBinding(){
      BindingElementCollection elements = Endpoint.Binding.CreateBindingElements();

      if(elements.Contains(typeof(ContextBindingElement))){
         return;
      }
      throw new InvalidOperationException("Can only use context binding");
   }
}
A few of the constructors of ContextClientBase<T> use ContextManager to create a new context and pass it to another constructor, which calls the SetContext helper method. SetContext first verifies that the binding used is indeed a context binding, and then it uses ContextManager to set the context. Verifying that the binding supports the context protocol is done by searching for the ContextBindingElement in the collection of binding elements. This way of verifying is better than looking at the binding type, since it also works automatically with a custom context binding.
For the service, the ContextManager helper class can encapsulate the interaction between operation context and message properties. ContextManager provides the GetValue method:
public static class ContextManager
{
   public static string GetValue(string key);

   //More members
}
Using GetValue, the service code in Figure 4 is reduced to:
class MyService : IMyContract
{
   public void MyMethod()
   {
      string number = ContextManager.GetValue("NumberContext");

      Debug.Assert(number == "123");
   }
}
Figure 7 shows the implementation of GetValue with some of the error handling code removed.
Figure 7 Implementing GetValue
public static class ContextManager
{
   public static string GetValue(string key)
   {
      if(OperationContext.Current == null)
         return null;

      ContextMessageProperty contextProperty =
             OperationContext.Current.IncomingMessageProperties
                                  [ContextMessageProperty.Name]
                                     as ContextMessageProperty;

      if(contextProperty.Context.ContainsKey(key) == false)
         return null;

      return contextProperty.Context[key];
   }
}
GetValue is similar to the explicit steps taken in Figure 4, except it adds state and error management. If the context does not contain the request key (or no context at all was found) GetValue returns null.
Clients can use ContextClientBase<T> to pass the instance ID over the client context binding protocol. Since the context bindings require a key and a value for every contextual parameter, the client needs to provide both to the proxy. Because the value of the key must be known to the service in advance, the client might as well hard-code the key in the proxy itself. The service will retrieve the instance ID in the relevant operations using ContextManager.
Figure 8 shows the calculator service using the context binding. In the figure, the service encapsulates the interaction with ContextManager in a dedicated context class.
Figure 8 Calculator with Instance ID over Context Binding
//Client Side:
[ServiceContract]
interface ICalculator
{
   [OperationContract]
   double Add(double number1,double number2);

   [OperationContract]
   void MemoryStore(double number);

   [OperationContract]
   void MemoryClear();
}
class MyCalculatorClient : ContextClientBase<ICalculator>,ICalculator
{
  public MyCalculatorClient(string instanceId) :  base("CalculatorId",instanceId)
   {}
  public MyCalculatorClient(string instanceId,string endpointName):
                             base("CalculatorId",instanceId,endpointName)
   {}

   //More constructors

   public double Add(double number1,double number2)
   {
      return Channel.Add(number1,number2);
   }

   public double MemoryStore (double number)
   {
      return Channel.MemoryStore (number);
   }

   //Rest of the implementation
}

//Service Side:
class CalculatorContext
{
   public static string Id
   {
      get
      {
         return ContextManager.GetValue("CalculatorId") ?? String.Empty;
      }
   }
}
[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
class MyCalculator : ICalculator
{
   static IInstanceStore<string,double> Memory = new
      FileInstanceStore<string,double>(Settings.Default.MemoryFileName);

   public double Add(double number1,double number2)
   {
      return number1 + number2;
   }
   public void MemoryStore(double number)
   {
      lock(typeof(MyCalculator))
      {
         Memory[CalculatorContext.Id] = number;
      }
   }
   public void MemoryClear()
   {
      lock(typeof(MyCalculator))
      {
         Memory.RemoveInstance(CalculatorContext.Id);
      }
   }
   //Rest of the implementation
}

Standard Key for Context ID
The need to hard-code the key used for the instance ID and the need to know it in advance are liabilities. As such, the durable service dispatcher logic automatically creates an instance ID in the form of a Guid (in string format), accessible via the reserved key named instanceId. The client and the service will see the same value for the instance ID. The value of the instance ID is initialized once the first call on the proxy returns, after the binding has had the chance to correlate it between the client and the service. Like any other parameter passed over a context binding, the value of the instance ID is immutable throughout the life of the proxy. To streamline interaction with the standard instance ID, I extended ContextManager with ID management methods, properties, and proxy extensions, as you can see in Figure 9.
Figure 9 Standard Instance ID Management
public static class ContextManager
{
   public const string InstanceIdKey = "instanceId";

   public static Guid InstanceId
   {
      get
      {
         string id = GetValue(InstanceIdKey) ?? Guid.Empty.ToString();
         return new Guid(id);
      }
   }
   public static Guid GetInstanceId(IClientChannel innerChannel)
   {
      try
      {
        string instanceId = innerChannel.GetProperty<IContextManager>()
         .GetContext()[InstanceIdKey];
         return new Guid(instanceId);
      }
      catch(KeyNotFoundException)
      {
         return Guid.Empty;
      }
   }
   public static void SetInstanceId(IClientChannel innerChannel,
                                                Guid instanceId)
   {
      SetContext(innerChannel,InstanceIdKey,instanceId.ToString());
   }

   //More members
}
ContextManager offers the GetInstanceId and SetInstanceId methods for the client to read or write an instance ID to the context. The InstanceId read-only property is used by the service to obtain the ID. ContextManager adds type safety by treating the instance ID as a Guid rather than a string, and adds error handling as well.
While the client can use ContextClientBase<T> as in Figure 8 to pass the standard ID, it is better to tighten it and provide built-in support on ContextClientBase<T> for the standard instance ID, as shown in Figure 10. Figure 11 shows the durable calculator client and service using the standard ID.
Figure 10 ContextClientBase<T> Support for Standard ID
public abstract class ContextClientBase<T> : ClientBase<T>
                                             where T : class
{
   public Guid InstanceId
   {
      get
      {
         return ContextManager.GetInstanceId(InnerChannel);
      }
   }
   public ContextClientBase(Guid instanceId) :
     this(ContextManager.InstanceIdKey,instanceId.ToString())
   {}

   public ContextClientBase(Guid instanceId,string endpointName) :
     this(ContextManager.InstanceIdKey,instanceId.
                                       ToString(),endpointName)
   {}

   //More constructors
}
Figure 11 Calculator Using Standard ID
//Client Side:
class MyCalculatorClient : ContextClientBase<ICalculator>,ICalculator
{
   public MyCalculatorClient()
   {}
   public MyCalculatorClient(Guid instanceIdId) : base(instanceId)
   {}
   public MyCalculatorClient(Guid instanceId,string endpointName) :
       base(instanceId,endpointName)
   {}

   //Rest same as Figure 8
}

//Service Side:
[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
class MyCalculator : ICalculator
{
   static IInstanceStore<Guid,double> Memory = new
       FileInstanceStore<Guid,double>(Settings.Default.MemoryFileName);
   public double Add(double number1,double number2)
   {
      return number1 + number2;
   }
   public void MemoryStore(double number)
   {
      lock(typeof(MyCalculator))
      {
         Memory[ContextManager.InstanceId] = number;
      }
   }
   public void MemoryClear()
   {
      lock(typeof(MyCalculator))
      {
         Memory.RemoveInstance(ContextManager.InstanceId);
      }
   }
   //Rest of the implementation
}

Automatic Durable Behavior
All the techniques shown so far for durable services require a non-trivial amount of work by the service—mostly to provide a durable state storage and explicitly manage the instance state against it in every operation. Given the repetitive nature of this work, WCF can automate it for you, serializing and deserializing the service state on every operation from an indicated state store, using the standard context ID. When the client provides a valid ID, for every operation WCF will deserialize an instance out of the store, call the operation, and serialize the new state modified by the operation back to the store. If the client does not provide an ID, WCF will create a new service instance by calling its constructor, but immediately following will serialize the instance to the state store.
WCF will never call the instance constructor if the store already contains state matching the ID provided to the proxy by the client. Instead, the call will be serviced on a new instance deserialized out of the state store. If the client provides an ID not found in the state store, WCF will throw an exception.
WCF provides the DurableService behavior attribute, which is defined as:
public sealed class DurableServiceAttribute :
    Attribute,IServiceBehavior,...
{...}
You apply this attribute directly on the service class. Most importantly, the service class itself must be marked either as Serializable or as a data contract with respective DataMember attributes on all members requiring durable state management.
The instance can manage its state in member variables, just as if it were a regular instance, and it can trust WCF to manage its members for it, as shown in Figure 12.
Figure 12 Using DurableService Behavior
[Serializable]
[DurableService]
class MyCalculator : ICalculator
{
   double Memory
   {get;set;}

   public double Add(double number1,double number2)
   {
      return number1 + number2;
   }
   public void MemoryStore(double number)
   {
      Memory = number;
   }
   [DurableOperation(CompletesInstance = true)]
   public void MemoryClear()
   {
      Memory = 0;
  }
   //Rest of the implementation
}
Any service relying on automatic durable state management must be configured as per-session, yet will always behave as per-call (WCF uses context deactivation after every call). In addition, the service must use one of the context bindings with every endpoint to enable the standard instance ID and the contract must allow or require a session, but cannot disallow it. These two constraints are verified at service load time.
The service can optionally use the DurableOperation operation behavior attribute to instruct WCF to purge its state from the store at the end of the workflow:
[AttributeUsage(AttributeTargets.Method)]
public sealed class DurableOperationAttribute : Attribute,...
{
   public bool CanCreateInstance
   {get;set;}

   public bool CompletesInstance
   {get;set;}
}
By setting the CompletesInstance property to true, WCF will remove the instance ID from the store once the operation call returns. The default value of the CompletesInstance property is false. In case the client does not provide an instance ID, you can also prevent an operation from creating a new instance by setting the CanCreateInstance property to false. Figure 12 demonstrates the use of the CompletesInstance property on the calculator's MemoryClear operation.
The problem with relying on CompletesInstance is that the context ID is immutable. If the client tries to make another call on the proxy after calling an operation marked with CompletesInstance set to true, then the call will fail, since the store no longer contains the instance ID.
The client must be aware that it cannot continue to use the proxy. If the client wants to make further calls against the service, it must do so on a new proxy that does not have an instance ID yet, and by doing so, the client will start a new workflow. To enforce this you can simply close the client program after completing the workflow or create a new proxy reference.

Persistence Providers
While the DurableService attribute instructs WCF when to serialize and deserialize the instance, it says nothing about where to do so. Rather, WCF uses a bridge pattern in the form of a provider model, which lets you specify the state store separately from the attribute. By doing so the attribute is decoupled from the store, allowing you to use automatic durable behavior with any compatible storage medium.
If the service is configured with the DurableService attribute, then you must configure its host with a persistence provider factory. The factory derives from the abstract class PersistenceProviderFactory and creates a subclass of the abstract class PersistenceProvider, as you see in Figure 13.
Figure 13 PersistenceProviderFactory
public abstract class PersistenceProviderFactory : CommunicationObject
{
   protected PersistenceProviderFactory();
   public abstract PersistenceProvider CreateProvider(Guid id);
}

public abstract class PersistenceProvider : CommunicationObject
{
   protected PersistenceProvider(Guid id);

   public Guid Id
   {get;}

   public abstract object Create(object instance,TimeSpan timeout);
   public abstract void   Delete(object instance,TimeSpan timeout)
   public abstract object Load(TimeSpan timeout);
   public abstract object Update(object instance,TimeSpan timeout);

   //Additional members
}
The most common way to specify the persistence provider factory is to indicate it in the host .config file (as a service behavior), and also to reference that behavior in the service definition, like the following:
<behaviors>
   <serviceBehaviors>
      <behavior name = "DurableService">
         <persistenceProvider
            type = "...type...,...assembly ..."
            <!--  Provider-specific parameters  -->
         />
      </behavior>
   </serviceBehaviors>
</behaviors>
If no persistence provider factory is specified, WCF aborts creation of the service host. Once the host is configured with the persistence provider factory, WCF uses the created PersistenceProvider for every call to serialize and deserialize the instance. A simple persistence provider factory is my FilePersistenceProviderFactory:
public class FilePersistenceProviderFactory : PersistenceProviderFactory
{
   public FilePersistenceProviderFactory();
   public FilePersistenceProviderFactory(string fileName);
   public FilePersistenceProviderFactory(
                                        NameValueCollection parameters);
}
public class FilePersistenceProvider : PersistenceProvider
{
   public FilePersistenceProvider(Guid id,string fileName);
}
FilePersistenceProvider wraps my FileInstanceStore<ID,T>. The constructor of FilePersistenceProviderFactory requires you to specify the desired file name. If no file name is specified, FilePersistenceProviderFactory defaults to Instances.bin.
The key to using a custom persistence factory in a .config file is to define a constructor that takes a NameValueCollection of parameters. These parameters are simple text-formatted pairs of the keys and values that are specified in the provider factory behavior section in the .config file. Virtually any free-formed key and value will work. For example, in order to specify what the file name is, use the following:
<persistenceProvider type =
      "FilePersistenceProviderFactory,ServiceModelEx"
         fileName = "MyService.bin"
 />
The constructor can then use the parameters collection to access these parameters:
string fileName = parameters["fileName"];
WCF ships with a persistence provider that stores the instance state in a dedicated SQL Server table. After a default installation of Visual Studio 2008, the installation scripts for the database are found under C:\Windows\Microsoft.NET\Framework\v3.5\SQL\EN. Note that you can only use SQL Server 2005 or SQL Server 2008 for state storage.
The SQL provider comes in the form of SqlPersistenceProviderFactory and SqlPersistenceProvider found in the System.WorkflowServices assembly under the System.ServiceModel.Persistence namespace. All you need to do is specify the SQL provider factory and the connection string name from the application settings, as shown in Figure 14.
Figure 14 Specifying SQL Provider and Connection String
<connectionStrings>
   <add name = "DurableServices"
      connectionString = "..."
      providerName = "System.Data.SqlClient"
   />
</connectionStrings>

<persistenceProvider
   type = "System.ServiceModel.Persistence.SqlPersistenceProviderFactory,
                System.WorkflowServices,Version=3.5.0.0,Culture=neutral,
                 PublicKeyToken=31bf3856ad364e35"
                 connectionStringName = "DurableServices"
/>
You can also instruct WCF to serialize the instances as text (instead of using the default binary serialization), perhaps for diagnostics or analysis purposes, by setting the serializeAsText parameter to true:
<persistenceProvider
   serializeAsText = "True"
  ...
/>

Send your questions and comments to mmnet30@microsoft.com.

Juval Lowy is a software architect with IDesign, providing WCF training and architecture consulting. His recent book is Programming WCF Services, Second Edition. He is also the Microsoft Regional Director for the Silicon Valley. Contact Juval at www.idesign.net.


Windows With C++
Exploring High-Performance Algorithms
Kenny Kerr

In the concurrency space, issues such as coordination, asynchronous behavior, responsiveness, and scalability get the lion's share of the attention. These are the high-level topics that developers think about when designing applications. But, perhaps due to inexperience or the lack of good performance tools, some equally important topics are often overlooked. A case in point is high-performance algorithms.
At the enterprise level, developers dwell on issues such as distributed file systems and caches, clusters, message queues, and databases. But what good are all those things if, at the core, you have inefficient algorithms and data structures?
Algorithm efficiency is not as straightforward as you might think. A well-designed algorithm on a single processor can often outperform an inefficient implementation on multiple processors. But a well-designed algorithm today also needs to show measurable scalability and efficiency when multiple processors are available. To complicate matters further, algorithms optimized for a single processor are often difficult to parallelize, whereas slightly less-efficient algorithms can often provide better performance on multiple processors.
As an example, I'm going to use Visual C++ and walk through the development of a fairly simple algorithm, but it's not entirely trivial even if it may seem that way at first glance. Here's what we need to implement:
void MakeGrayscale(BYTE* bitmap,
                   const int width,
                   const int height,
                   const int stride);
The bitmap parameter points to an image consisting of 32 bits per pixel. Again, this is to keep this article focused. The absolute value of the stride indicates the number of bytes from one row of pixels in memory to the next. Padding may exist at the end of each row. The stride's sign indicates whether these rows are top-down with a positive stride or bottom-up with a negative stride in memory.
Let's get the basics out of the way first. We can use the following structure to represent a pixel in memory:
typedef unsigned char BYTE; // from windef.h

struct Pixel
{
    BYTE Blue;
    BYTE Green;
    BYTE Red;
    BYTE Alpha;
};
A quick Web search reveals that a reasonable gray scale value can be obtained by blending 30% red, 59% green, and 11% blue for a given color pixel. Here's a simple function to convert just one pixel to gray scale:
void MakeGrayscale(Pixel& pixel)
{
    const BYTE scale = static_cast<BYTE>(0.30 * pixel.Red +
                                         0.59 * pixel.Green +
                                         0.11 * pixel.Blue);

    pixel.Red = scale;
    pixel.Green = scale;
    pixel.Blue = scale;
}
The byte offset of a particular pixel within the bitmap can be obtained by calculating the product of its horizontal position and the size of a pixel, along with the product of its vertical position and the stride, and then adding these values together:
offset = x * sizeof(Pixel) + y * stride
How then might you implement the MakeGrayscale function? If you jumped right in without further consideration you might write something along the lines of the algorithm in Figure 1. At first glance this might look reasonable and, if passed, a small bitmap might even appear to work quite well. But what about larger bitmaps? What about a 20,000 pixel by 20,000 pixel bitmap?
Figure 1 Inefficient Single-Threaded Algorithm
void MakeGrayscale(BYTE* bitmap,
                   const int width,
                   const int height,
                   const int stride)
{
    for (int x = 0; x < width; ++x)
    for (int y = 0; y < height; ++y)
    {
        const int offset = x * sizeof(Pixel) + y * stride;

        Pixel& pixel = *reinterpret_cast<Pixel*>(bitmap + offset);

        MakeGrayscale(pixel);
    }
}
I happen to have a Dell PowerEdge with a Quad-Core Intel Xeon X3210 processor. This machine has a clock speed of 2.13GHz, a 1066MHz front-side bus, 8MB of L2 cache, as well as a variety of other cool features. Admittedly it's not the very latest Intel Xeon processor, but it is certainly nothing to scoff at. It's being driven by the 64-bit version of Windows Server 2008 and will do nicely for performance testing.
With all that power I ran the algorithm from Figure 1 on a bitmap 20,000 pixels in width and 20,000 pixels in height. On average it took about 46 seconds over 10 iterations. Admittedly that bitmap is rather large, weighing in at around 1.5GB. But should that matter? My server has 4GB of RAM, so it isn't paging to disk. But Figure 2 reveals the all too familiar view of processor usage.
Figure 2 Processor Usage of an Inefficient Single-Threaded Algorithm
The algorithm is furiously working away but only using a quarter of the computer's processing power. The solution is clear—or is it? The most common reaction is to throw more threads at the problem. This is a reasonable position to take. After all, the algorithm as it stands only uses one thread, so only one of the four cores on the test system is being used. The code in Figure 3 adds threads.
Figure 3 Inefficient Multithreaded Algorithm
void MakeGrayscale(BYTE* bitmap,
                   const int width,
                   const int height,
                   const int stride) {
    #pragma omp parallel for
    for (int x = 0; x < width; ++x)
    for (int y = 0; y < height; ++y) {
        const int offset = x * sizeof(Pixel) + y * stride;

        Pixel& pixel = *reinterpret_cast<Pixel*>(bitmap + offset);

        MakeGrayscale(pixel);
    }
}
The only difference is the addition of the OpenMP pragma. The outer for loop will now be divided among threads so they can run in parallel and convert different parts of the bitmap. OpenMP creates one thread per processor by default. (Don't get too hung up on the use of OpenMP. It just happens to be a very concise way of partitioning a for loop across threads. There are a variety of other techniques for achieving the same thing depending on your compiler and run time.) This small change makes a big difference, as you can see in Figure 4.
Figure 4 Processor Usage of Inefficient Multithreaded Algorithm
Performance-Improvement Resources
For more information on the topics covered here, see the following resources:

  • Maximize Locality, Minimize Contention
  • CPU Cache
  • OpenMP and C++: Reap the Benefits of Multithreading without All the Work
  • Quad-Core Intel Xeon X3210
  • Converting Color to Grayscale
But the results are less than ideal. The multithreaded algorithm takes about 30 seconds compared to 46 seconds for the single-threaded version. The 34% increase in performance hardly seems worth the 300% increase in processing cost. What if I told you that an efficient implementation can complete in about 2 seconds using only a single thread, and with the help of multiple threads it can do it in about 0.5 seconds (on the same test system)? That's quite a lot faster than 30 seconds.
It's easier than you think. It just takes some thought about the physical hardware on which your algorithm is running and how data is organized in memory. All too often developers are lulled into a state of indifference toward the hardware that ultimately runs their code. I'm going to reveal an efficient algorithm and then discuss why it is so much better.
Take a look at Figure 5. It might surprise you that the only difference here is in the order of the for loops. That is, I changed the algorithm from Figure 3 such that the outer loop iterates over the Y axis and the inner loop iterates over the X axis. Now, why would such a seemingly innocuous change cut the time down from 30 seconds to less than 1 second? The answer lies in appreciating the fact that an algorithm tends to be severely impacted by the shape and locality of the data on which it operates as well as the characteristics of the hardware.
Figure 5 Efficient Algorithm
void MakeGrayscale(BYTE* bitmap,
                   const int width,
                   const int height,
                   const int stride) {
    #pragma omp parallel for
    for (int y = 0; y < height; ++y)
    for (int x = 0; x < width; ++x) {
        const int offset = x * sizeof(Pixel) + y * stride;

        Pixel& pixel = *reinterpret_cast<Pixel*>(bitmap + offset);

        MakeGrayscale(pixel);
    }
}
To understand this we must reexamine the data structure on which the algorithm operates. The bitmap's memory buffer logically contains rows of pixels. Obviously memory is not tabular, but there needs to be a way to easily and efficiently address any pixel given its coordinates; thinking in terms of rows helps to conceptualize this. This abstraction does, however, reveal its physical implementation when you consider that these rows may contain padding, which is necessary to properly align the beginning of each row, particularly when using bitmaps with indexed color tables (see Figure 6).
Figure 6 Rows and Padding (Bottom-up)
The direction of the Y axis illustrates a bottom-up layout indicated by a negative stride. It is theoretically possible that this direction could also impact performance, but that would be hard to measure without very good tooling support.
A good concurrency rule of thumb is that memory that is farther apart should not be used together by a single processor, while memory that is close together should not be used by different processors. In other words, keep the data that is used together close together and the data that is used separately far enough apart. In this way, a hardware cache manager can cache data that will be used by different processors without introducing any contention.
To appreciate how the memory layout affects performance, you need to remember that memory is continuous, not tabular. The virtual address space is flat. Figure 7 illustrates how the bitmap is partitioned using the inefficient algorithm in Figure 3.
Figure 7 Inefficient Data Partitioning
It should now be obvious why this algorithm performs so poorly. Although there are no loop-carried dependencies, the algorithm is parallelized in such a way as to almost guarantee contention at the hardware cache level. Larger bitmaps may reduce contention, but the repeated cache misses will force the cache manager to constantly reload cache lines, thus affecting performance. Even on a single processor it will perform very poorly as it has to replace the data in its cache lines constantly. Figure 8 provides a clearer indication that locality is at fault. The pattern will repeat for as many rows as there are in the bitmap.
Figure 8 Inefficient Data Partitioning (Repeating)
As a contrast, the efficient algorithm in Figure 5 partitions the bitmap along the Y axis so that each processor will be able to process a stream of pixels that are close together but far apart from pixels processed by other processors, thus making it possible for the cache manager to share the bitmap across processors without contention (see Figures 9 and 10). In particular, Figure 10 shows that each process has a single continuous block of memory to process, improving locating and caching performance immensely.
Figure 9 Efficient Data Partitioning
Figure 10 Efficient Data Partitioning (Non-Repeating)
Reaching the 0.5-second target cited above would be tough given the static scheduling used by OpenMP by default. OpenMP also provides dynamic and guided scheduling that can be used to squeeze those last few milliseconds out of the algorithm. The fundamental problem is that dividing the work perfectly among the available processors isn't likely to provide the best results since the computer might be doing some minor tasks in the background. This could result in some of the threads completing before the others and then sitting idly while waiting for the stragglers to complete.
On the other hand, you may be able to make it even faster, possibly using a different algorithm or data structure. You could load bitmap partitions into the physical memory of individual nodes in a Non-uniform Memory Access (NUMA) system. You could even send them off to different nodes on a Windows Server High-Performance Computing (HPC) grid. None of this would be possible with the inefficient algorithm. The point is that you can dramatically improve the performance of your code by giving a little thought to your data structures and algorithms and how performance may be affected by the hardware on which you're running.

Send your questions and comments to mmwincpp@microsoft.com.

Kenny Kerr is a software craftsman specializing in software development for Windows. He has a passion for writing and teaching developers about programming and software design. Reach Kenny at weblogs.asp.net/kennykerr.


Going Places
Ink-Enabled Apps For Tablet PC
Gus Class
Contents
Tablet PC Writing and Recognition
InkEdit and InkPicture
Using the InkEdit Control
Adding Ink Controls to Visual Studio
Adding the InkEdit Control
Using the InkPicture Control
Changing Drawing Attributes
Wrap-Up
From a high-level perspective, the process for ink capture and analysis on the Tablet PC is straightforward. In managed code, where object creation and access is consistent and simple, it is easy to develop managed applications for Tablet PC. And it won't be much more difficult for seasoned Windows developers to develop such apps in unmanaged code. To the uninitiated developer, however, creating unmanaged Tablet PC applications can be rather daunting.
Taking input from a pen and transforming it into meaningful data is a common and potentially arduous task for pen-computing developers in general. In the case of Tablet PC, this process is simplified because we designed the pen-computing system with an abstracted architecture to ease your design overhead.
This column gives a quick introduction to the Tablet PC ink architecture and shows how to capture and recognize ink for Tablet PC applications. I will include example code that illustrates how to use the Tablet PC SDK to recognize handwriting as text and create a simple pen-based illustration input.

Tablet PC Writing and Recognition
In the Tablet PC world, input from the pen to the PC is called "ink." Ink is a useful form of input data, as it can be easily transformed to other types of data (numbers, text, and characters) or can be left as ink if the input is best left in that form. Scribbled notes with intermixed diagrams are one such example where it makes sense to save input as ink.
Ink is made up of packets that contain all sorts of nifty information such as the x position, y position, and state of the pen. Ink also includes stroke collections that share attributes such as width, color, and transparency values. Figure 1 illustrates the high-level design of pen input (ink) data for Tablet PCs.
Figure 1 Ink Data Components
Reading stroke data is just like reading any other input stream; the strokes are captured by OS hooks and can then be rendered or recognized. Microsoft has implemented interfaces, classes, and managed objects for representing, rendering, and recognizing each of the aforementioned data types. These objects were originally distributed to developers in the Tablet PC SDK in Windows XP, and they are now a part of the Windows Vista SDK.
Ink recognition—the process of transforming pen input into text—is built into the Tablet PC SDK and is accomplished by capturing ink and passing this ink to an object that can recognize the ink. Ink is captured by using InkCollector or a derived object such as InkOverlay. After InkCollector or InkOverlay has captured packets, which are formed into strokes, the strokes can be retrieved. By passing the strokes to a recognizer and calling a method to analyze the captured strokes, the ink data can be translated to recognized content. Figure 2 illustrates the ink-recognition process.
More information on ink analysis can be found in the Tablet PC SDK (available at msdn.microsoft.com/library/ms812486). Another excellent resource for Tablet PC development is the Mobile PC Developer Center (msdn.microsoft.com/mobilepc).

InkEdit and InkPicture
The simplest way to access the powerful functionality of the Tablet PC SDK in an unmanaged context is to use the InkEdit and InkPicture ActiveX controls. These controls simplify the various procedures that are typically performed with the Tablet PC objects. For example, the InkEdit control will automatically perform the InkRecognition procedure for you in an optimal manner. This will get you going with Tablet PC development faster than if you were to create such a control on your own.
Each of the controls has applications where it makes more sense to use one rather than the other, and the differences between them are shown in Figure 3. The InkEdit control enables handwriting recognition, but operates more like the RichEdit control. InkEdit can be integrated with an application that targets Tablet PCs to get access to handwriting recognition without affecting the user's experience by presenting the Tablet PC Input Panel. If you were creating an application that gets input that will ultimately be converted to text, you should generally use the InkEdit control.
Figure 3 InkEdit Versus InkPicture
Function | InkEdit | InkPicture
Ink Capture | Yes | Yes
Ink Recognition | Yes | No
Edit Ink Appearance | No | Yes
Erasing Ink | No | Yes
Edit Text Per-Character | Yes | No
Ink to Text Conversion | Yes | No
Gesture to Text | Yes | No
Scale to Font Size | Yes | No
Handles Images | No | Yes
The InkPicture control enables illustration input, but does not automatically perform ink recognition. If you were creating an application that received drawn input from the pen, you should generally use the InkPicture control. This control is useful if you want the user to input ink and keep it that way.

Using the InkEdit Control
To illustrate the functionality and customization of the InkEdit control, I will demonstrate how to add the InkEdit control to an application and trigger recognition using a button. The following instructions will walk you through the steps I took to create a bare-bones MFC application and dialog box. Start by creating a new MFC Application project in Visual Studio. Change the project name to Simple Recognizer (or whatever name you prefer).
Click OK and you will be presented with the MFC Application Wizard. The Wizard gives you a quick overview of project settings. You should not need to change anything.
On the Application Type page, you will need to set the application type to Dialog. Click Next to see the User Interface Features page. Clear all checkboxes on the User Interface Features page.
Click Next to see the Advanced Features page. You shouldn't need to change any of the advanced features on this page, so go ahead and click Next, and then verify your settings and click Finish to create your project.
Under the Resources tab for your project, you will find the .rc file that contains UI information for your dialog application. Double-click on this to see how the application will look when it runs. Delete all text and buttons on the Form.
From the toolbox, drag a button (our recognizer trigger), a group box, and some static text onto your new dialog box (see Figure 4). Change the ID for the group box to IDC_PLACEHOLDER because that box (and its related hWnd) will need to have the ink control on top of it. You're now ready to set up the tablet controls in Visual Studio and get the code working.
Figure 4 DESCRIPTION_GOES_HERE

Adding Ink Controls to Visual Studio
The existing ink controls make it very easy to add ink support to an application. The ActiveX controls (InkEdit and InkPicture) are included in Windows Vista (Home Premium version and later) in addition to Windows XP for Tablet PC Edition. Once you have installed the controls and enabled them in Visual Studio, you will be able to simply drop the controls onto your application.
First, download and install the Tablet PC SDK or Windows Vista SDK. Once the installation has finished, restart Visual Studio. Reopen the project you just created and return to the resource editor by double-clicking the .rc file. After you have opened that file, the toolbox area will change to show the various components that you can use for designing applications. Right-click on this area and select Choose Items.
You will be presented with the Choose Toolbox Items window. Select the COM Components tab, and then scroll down and select Microsoft InkEdit Control and Microsoft InkPicture Control. Once you have enabled these controls, they will appear in your toolbox and you can add them to applications.

Adding the InkEdit Control
To use the recognizer, drag the InkEdit control from the toolbox to your application and place it within the group control. Update the Caption property on the group control to indicate the InkEdit area is for writing and resize the InkEdit control to fit well into the area. In your application, make sure that your group control doesn't share space with the InkEdit control. Additionally, be careful not to inadvertently change the control order by removing and re-adding the group-box control; the InkEdit control should be the topmost control.
Build and run your application. Figure 5 shows text that hasn't been recognized yet and text that has been recognized in my version of the application.
Figure 5 InkEdit Does Text Recognition
Because the InkEdit control has an automatic timeout after which it recognizes the ink, all you have had to do so far in order to recognize ink is drag and drop a control onto your application's dialog box. Next, you will be adding some code to customize the behavior.
To have the text recognized when the Recognize button is clicked rather than automatically based on the InkEdit control's default timeout period, you will have to alter the InkEdit's refresh timeout when the application starts and then trigger ink recognition when the Recognize button is pressed. This is convenient because the InkEdit control is an ActiveX control, and you can just use the SendMessage method to specify this functionality to the window hosting the ActiveX control.
First, add the following code to the OnInitDialog function for your application:
// Controls are now initialized; set control settings.
CWnd* pInkWnd = NULL;
pInkWnd = GetDlgItem(IDC_INKEDIT3);
if (pInkWnd != NULL){
  // Set the timeout to a large number.
  // LPARAM specifies the timeout in millisecs.
  // WPARMA must be left at 0.
  LRESULT result  =
    pInkWnd->SendMessage(EM_SETRECOTIMEOUT, 1000000042, 0);
}else{
  // Indicate that an error occurred.
  ERROR_MSG(L'ERROR: Could not specify recognizer timeout');
}
Next, add the following code to the button-click handler for the Recognize button. The easiest way to add a button-click handler is to double-click the button in the resource view of the resource editor. This will bring you right to the code section for the button you have created:
void Csimplest_recognizerDlg::OnBnClickedButton1() {
  CWnd* pInkWnd = NULL;
  pInkWnd = this->GetDlgItem(IDC_INKEDIT3);
  if (pInkWnd != NULL){
    // Trigger recognition within the InkEdit control.
    LRESULT result  = pInkWnd->SendMessage(EM_RECOGNIZE, 0, 0);
  }
}
Now, if you build and run the application, you will notice that the InkEdit control does not automatically recognize the written text and requires clicking the Recognize button to trigger recognition.

Using the InkPicture Control
The InkPicture control allows application designers to create applications that receive input from the pen with the intention of treating it as a picture. In this way, a user can draw objects on the control area and the objects can be interpreted or handled by the control. For this example, I will walk through creation of a very simple application that allows the user to input strokes by using the pen, erase those strokes, and select those strokes (see Figure 6).
Figure 6 Using InkPicture
When the application starts, the user will be in inking mode and can write in the drawing area. If the user clicks the Eraser button, she will be able to erase the lines that are drawn. Clicking the selection button will enable selection mode, and drawn objects can be selected.
In the same way that you set up the recognition application, create the dialog for this application. The only difference between this application and the last one is that you will be using the InkPicture control instead of the InkEdit control, and you will be adding three buttons to the lower section of the application dialog. The white area is an InkPicture control. Note that clicking the Pen, Eraser, or Selection buttons does not yet work, nor will the application enable the eraser if you invert your pen.
Now you will need to make the buttons change the state of the InkPicture control. To do this, you need to know which properties to change; you then set the properties by grabbing the CWnd associated with the control and then use the SetProperty method to change the control state. To simplify this process, create a function called setEditingMode that will take in a value from the InkOverlayEditingMode enumeration (see Figure 7).
Figure 7 setEditingMode
void CInkPaintDlg::setEditingMode(short mode){
  CComPtr<IInkPicture> ip;

  // Retrieve the DISPID by using a COM Client for the InkPicture  // control.
  DISPID dispid = 0;
  OLECHAR FAR* szMember = OLESTR('EditingMode');
  ip.CoCreateInstance(CLSID_InkPicture);
  ip->GetIDsOfNames(IID_IInkPicture, &szMember, 1,
    LOCALE_SYSTEM_DEFAULT, &dispid);

  // Retrieve the CWnd for the InkPicture Control.
  CWnd* pInkWnd = NULL;
  pInkWnd = this->GetDlgItem(IDC_INKPICTURE1);

  pInkWnd->SetProperty(dispid, VT_I2, mode);
  ip.Release();
}
Once you have a function to set the editing mode, you just need to add buttons that change the editing mode to the various states. Double-click the buttons you created in the resource designer and add handlers to set the mode accordingly. The following code shows how my button-click handlers change the InkPicture control's editing mode to the correct states:
// Pen mode
void CInkPaintDlg::OnBnClickedButton1() {
  setEditingMode(InkOverlayEditingMode::IOEM_Ink);
}
// Eraser mode
void CInkPaintDlg::OnBnClickedButton2() {
  setEditingMode(InkOverlayEditingMode::IOEM_Delete);
}
// Selection mode
void CInkPaintDlg::OnBnClickedButton3() {
  setEditingMode(InkOverlayEditingMode::IOEM_Select);
}
Now that your program can erase and select, you can perform new operations within your program beyond drawing strokes with the pen tool. Selecting the eraser mode will allow users to erase lines, selecting the selection tool will enable stroke selection, and so on. Figure 8 shows selected content being resized.
Figure 8 Resizing Strokes in InkPicture
After running the program, you may decide that you prefer to use the eraser in point erase mode rather than stroke erase mode. I added a function that takes a Boolean value to enable stroke erase mode or point erase mode (see Figure 9).
Figure 9 Enabling Erase Modes
void CInkPaintDlg::setPointEraseMode(bool setToPointMode){
  CComPtr<IInkPicture> ip;

  // Retrieve dispid.
  DISPID dispid = 0;
  OLECHAR FAR* szMember = OLESTR('EraserMode');
  ip.CoCreateInstance(CLSID_InkPicture);
  ip->GetIDsOfNames(IID_IInkPicture, &szMember, 1,
    LOCALE_SYSTEM_DEFAULT, &dispid);

  // Retrieve the CWnd for the InkPicture control.
  CWnd* pInkWnd = NULL;
  pInkWnd = this->GetDlgItem(IDC_INKPICTURE1);

  // Toggle the property based on the point mode.
  if (setToPointMode){
    pInkWnd->SetProperty(dispid, VT_I2,
      InkOverlayEraserMode::IOERM_PointErase);
  }else{
    pInkWnd->SetProperty(dispid, VT_I2,
      InkOverlayEraserMode::IOERM_StrokeErase);
  }

  ip.Release();
}
This function is called in the extra initialization code for my dialog application:
BOOL CInkPaintDlg::OnInitDialog() {
  // TODO: Add extra initialization here.
  this->setPointEraseMode(true);
  ...
}
Now that the eraser mode is set to point erase, you can erase sections of the illustrations rather than whole lines. In a full-featured application, it would make sense to have all of these settings toggled through a context menu.

Changing Drawing Attributes
Now that your application is working and you can select, erase, and draw, you can extend it to change the width and color of the pen. First, add a few more buttons to your application for the various colors you want to support; for example, red, blue, green, and black. Next add a button or two for changing stroke width to thick and thin. To simplify this, I created two utility functions, setStrokeThickness and setColor. The code for setStrokeThickness is shown in Figure 10.
Figure 10 setStrokeThickness
void CInkPaintDlg::setStrokeThickness(bool isThick){

  CComPtr<IInkPicture> ip;
  CComPtr<IInkDrawingAttributes> spIInkDrawAttrs = NULL;

  // Set to thick or thin based on bool.
  // 50 is the default width.
  // 250 is a good thickness.
  float newWidth= isThick ? 250 : 50;

  DISPID dispid = 0;
  OLECHAR FAR* szMember = OLESTR('DefaultDrawingAttributes');

  ip.CoCreateInstance(CLSID_InkPicture);
  ip->GetIDsOfNames(IID_IInkPicture, &szMember, 1,     LOCALE_SYSTEM_DEFAULT, &dispid);

  CWnd* pInkWnd = NULL;
  pInkWnd = this->GetDlgItem(IDC_INKPICTURE1);

  // Retrieve the drawing attributes interface.
  pInkWnd->GetProperty(dispid, VT_UNKNOWN, &spIInkDrawAttrs);

  // Now use the interface to set the width.
  spIInkDrawAttrs->put_Width(newWidth);

  // Now pass the interface back to set the drawing attributes.
  pInkWnd->SetProperty(dispid, VT_DISPATCH, spIInkDrawAttrs);

  // Free.
  spIInkDrawAttrs.Release();
  ip.Release();
}
There is only one significant difference between the utility functions and the other functions that set properties in the ink control. In this example, you retrieve a child interface (which is itself a property), and then use that interface to make changes rather than making changes in the parent interface's properties. Now you can easily change the stroke color and thickness by using the utility functions:
// RED
void CInkPaintDlg::OnBnClickedButton4() {
  COLORREF color = RGB(255,0,0);
  setColor(color);
}

// Set thin strokes.
void CInkPaintDlg::OnBnClickedButton8() {
  this->setStrokeThickness(false);
}
Figure 11 shows the various colors and stroke thicknesses in my finished application.
Figure 11 Customized Color and Line Thickness

Wrap-Up
Creating Tablet PC-enabled applications in MFC with unmanaged C++ and the InkEdit and InkPicture controls is actually pretty easy! The functionality of the controls also enables you to enhance applications with the most typical tablet-based features. Out of the box, controls typically run the way you want. If you need to change the control behavior, you can send messages to the controls or get and set relevant properties on them. Additional information on the Tablet API can be found in the MSDN Library available at msdn.microsoft.com/library/ms950406.
Thanks to Jason Groce and Eliot Graff for all their technical help with the column.

Send your questions and comments to goplaces@microsoft.com.

Gus "gclassy" Class is a Programming Writer/Evangelist for Microsoft where he has worked on Tablet PC, Windows mail, and DRM systems. He discusses developer gotchas and offers programming examples on his blog at gclassy.com.


.NET Matters
False Sharing
Stephen Toub, Igor Ostrovsky, and Huseyin Yildiz
Contents
It All Comes Down to Hardware
Contrary to Kindergarten Teaching, Sharing Can Be Bad
Methods for Detecting False Sharing
Using the Visual Studio Profiler
Is False Sharing Just Theoretical?
Memory Locations at Risk of False Sharing
Avoid Contiguous Allocation of Memory
Unless you've been living under a rock, you've likely heard about the "manycore shift." Processor manufacturers such as Intel and AMD are increasing the power of their hardware by scaling out the number of cores on a processor rather than by attempting to continue to provide exponential increases in clock speed. This shift demands that software developers start to write all of their applications with concurrency in mind in order to benefit from these significant increases in computing power.
To address this issue, a multitude of concurrency libraries and languages are beginning to emerge, including Parallel Extensions to the Microsoft .NET Framework, the Parallel Pattern Library (PPL), the Concurrency & Coordination Runtime (CCR), Intel's Threading Building Blocks (TBB), and others. These libraries all aim to decrease the amount of boilerplate code necessary to write efficient parallel applications by providing constructs such as Parallel.For and AsParallel. Unfortunately, while these constructs represent a monumental step forward in expressing parallelism, they don't obviate the need for developers to be aware of what their code is doing, how it's structured, and how hardware can have a significant impact on the performance of the application.
While the software industry is making strides in developing new programming models for concurrency, there does not seem to be a programming model on the horizon that would magically eliminate all concurrency-related issues. At least in the near term, understanding how memory and caches work will be important in order to write efficient parallel programs.

It All Comes Down to Hardware
The concept of knowing what's happening at the lower levels of an application is, of course, not new. To achieve optimal performance, developers need to have a good understanding of how things like memory accesses affect the performance of an application.
When we talk about reading and writing from memory, we typically gloss over the fact that, in modern hardware, it's rare to read from and write to the machine's memory banks directly. Memory access is slow—orders of magnitude slower than mathematical calculations, though orders of magnitude faster than accessing hard disks and network resources.
To account for this slow memory access, most processors today use memory caches to improve application performance. Caches come in multiple levels, with most consumer machines today having at least two levels, referred to as L1 and L2, and some having more than that. L1 is the fastest, but it's also the most expensive, so machines will typically have a small amount of it. (The laptop on which we're writing this column has 128KB of L1 cache.) L2 is a bit slower, but is less expensive, so machines will have more of it. (The same laptop has 2MB of L2 cache.)
When data is read from memory, the requested data as well as data around it (referred to as a cache line) is loaded from memory into the caches, then the program is served from the caches. This loading of a whole cache line rather than individual bytes can dramatically improve application performance. On our laptop the cache line size for both L1 and L2 is 64 bytes. Since applications frequently read bytes sequentially in memory (common when accessing arrays and the like), applications can avoid hitting main memory on every request by loading a series of data in a cache line, since it's likely that the data about to be read has already been loaded into the cache. However, this does mean that a developer needs to be cognizant of how the application accesses memory in order to take the greatest advantage of the cache.
Consider the C# program in Figure 1. We've created a two-dimensional array to which the application then writes in two different ways. In the first code segment (labeled Faster in the comments), the application loops over each row, and within each row it loops over each column. In the second code segment (labeled Slower), the application loops over each column, and within each column it loops over each row. The labels in this case give away the punch line, but on the laptop we tested this on, the Faster version runs almost twice as fast as the Slower version, even though the only difference in code between the two is that the order of the for loops has been swapped.
Figure 1 Memory Access Patterns Are Important
using System;
using System.Diagnostics;

class Program {
  public static void Main() {
    const int SIZE = 10000;
    int[,] matrix = new int[SIZE, SIZE];

    while (true) {
      // Faster
      Stopwatch sw = Stopwatch.StartNew();
      for (int row = 0; row < SIZE; row++) {
        for (int column = 0; column < SIZE; column++) {
          matrix[row, column] = (row * SIZE) + column;
        }
      }
      Console.WriteLine(sw.Elapsed);

      // Slower
      sw = Stopwatch.StartNew();
      for (int column = 0; column < SIZE; column++) {
        for (int row = 0; row < SIZE; row++) {
          matrix[row, column] = (row * SIZE) + column;
        }
      }
      Console.WriteLine(sw.Elapsed);

      Console.WriteLine("=================");
      Console.ReadLine();
    }
  }
}
This performance differential occurs because of the way the memory for the matrix array is organized. The array is stored contiguously in memory in row-major order, meaning that you can envision each of the rows from the matrix laid out in memory linearly one after the other. When the application accesses a value from the array, the relevant cache line will contain the value as well as other values directly surrounding the one requested, which are likely to be the values immediately before or after in the same row. Thus, accessing the next value from the row should be significantly faster than accessing a value in a different row. In the slower version, we're always accessing values in different rows by jumping from row to row, accessing the same column location in each row. For a real-world example of this phenomenon, see Kenny Kerr's column in this issue of MSDN Magazine at msdn.microsoft.com/magazine/cc850829.

Contrary to Kindergarten Teaching, Sharing Can Be Bad
Parallelism makes this issue even more relevant. Not only do we need to be aware of how one processor is accessing data in memory and taking advantage of its caches, but we need to be aware of how multiple threads/processors are accessing the data.
In many systems today, some of the caches are not shared between cores. Since a cache maintains copies of data from main memory, processors need to implement a mechanism for keeping cache contents in sync with both main memory and with other caches; such a mechanism is known as a "cache coherency" or "memory coherency" protocol.
The protocol used by most Intel processors today is known as MESI, named as such to represent the four states that a particular cache line can be in: Modified, Exclusive, Shared, and Invalid. The protocol ensures that a cache monitors what's happening with other caches in the system and takes appropriate action to maintain consistency, such as flushing modified data to main memory when another cache requires that data for a read operation. At a minimum these operations require memory reads/writes that, as we've already stated and demonstrated, can be expensive.
Unfortunately, the problem can be trickier than it might at first seem, as it isn't always obvious when data is being shared. True sharing occurs when two threads specifically require access to a given memory location which is properly kept consistent by the cache coherency protocol. However, unbeknownst to the developer, other sharing can occur where two threads access distinct sets of data, but due to the hardware architecture details such as the size of a cache line, those sets of data still look like sharing as far as the cache coherency protocol is concerned. This is known as false sharing (illustrated in Figure 2), and it can lead to significant performance problems in real-world parallel applications.
Figure o2 Cache Coherency and Cache Lines
Consider the code shown in Figure 3. In this code snippet, we're allocating instances of System.Random, one per processor. Then in parallel (using the Parallel.For construct from the Task Parallel Library that's part of Parallel Extensions), we run that many iterations. Each iteration uses one of the Random instances and does some work on it (in this case, continually calling Next on the Random instance).
Figure 3 False Sharing Example
static void WithFalseSharing() {
  int iterations = Environment.ProcessorCount;

  Random[] rands = new Random[iterations];
  for (int i = 0; i < iterations; i++) rands[i] = new Random();

  Parallel.For(0, iterations, i => {
    Random r = rands[i];
    DoWork(r);
  });
}

static void DoWork(Random rand) {
  for (int i = 0; i < WORKAMOUNT; i++) rand.Next();
}
It may not be clear in this example where the sharing exists. The application is only ever reading from the rands array, and it's perfectly valid for the same cache line to be in two different caches for reading (with the MESI protocol, the line would likely be marked as Shared with both). The problem is actually in the Random instances themselves. Each instance of Random contains some state in order to produce the next value in its sequence of pseudo-random values, and every call to Next computes the next value, updating that internal state. Due to the way we're allocating these Random instances, they can end up close enough to each other in memory to cause false sharing between instances, even though all of the threads involved in the loop are independent from a source code perspective.
There are several ways to address the false sharing issue, all of which involve allocating the Random instances far enough apart from each other in memory that they won't show up on the same cache line. In .NET, where memory allocation details are left to the runtime, this can require some workarounds.
As one example, the code in Figure 4 shows a version of this code that doesn't suffer from the same problem. Instead, it allocates a bunch of extra Random instances between those that we care about, thus ensuring that the ones we do care about are far enough apart so as to not be subject to false sharing (at least on this machine). For our tests, this updated version produced significantly better results, running up to six times faster than the code from Figure 3 on our dual-core test machine.
Figure 4 Fix for False Sharing Example
static void WithoutFalseSharing() {
  const int BUFFER = 64;
  int iterations = Environment.ProcessorCount * BUFFER;

  Random[] rands = new Random[iterations];
  for (int i = 0; i < iterations; i++) rands[i] = new Random();

  Parallel.For(0, iterations, BUFFER, i => {
    Random r = rands[i];
    DoWork(r);
  });
}

static void DoWork(Random rand) {
  for (int i = 0; i < WORKAMOUNT; i++) rand.Next();
}

Methods for Detecting False Sharing
Unfortunately, it's fairly easy for false sharing to go undetected because there aren't well-isolated indicators that singly point to it as a performance problem. Fortunately, there are some profiling metrics that can be used comparatively to confirm suspicions of false sharing and to help narrow it down in code. Above all, you need to know when to suspect effects of false sharing and you need to be aware of the memory access patterns in concurrent code when hunting it down.
Let's assume that you've identified a concurrent code segment as your performance bottleneck. You may want to focus on shared cache line effects if the following are true of that piece of code:
  • It is relatively CPU and memory bound, such that there are few I/O or blocking OS calls.
  • It doesn't scale well when the concurrency level is increased by running on more powerful hardware.
  • It occasionally performs significantly slower for different input data that requires the exact same amount of processing and number of memory accesses, but with a different pattern of traversal.
The best hints we've found for identifying false sharing effects come from CPU performance counters. These are hardware-level statistics made available by the CPU itself, and they provide information about how different subsystems of the CPU are performing. Examples include the number of L2 cache misses and the number of branch predictions misses. The variations of these counters can be different for each processor model, although there is a common subset that is exposed by most profilers. We are going to demonstrate with a few of these, namely L2 cache misses, instructions retired, and non-halted cycles.
L2 cache misses indicates the number of times a read from the L2 cache resulted in a load from main memory. An L2 miss can happen if the requested data isn't present in the L2 cache, or if the corresponding cache line has been marked as invalid due to an update from another processor. The latter occurs frequently in the case of false sharing, therefore in diagnosing this problem we need to watch for an undue spike in the L2 Cache Misses counter.
Cycles Per Instruction (CPI) is one of the most widely used statistics for characterizing the overall performance of algorithmic benchmarks. It denotes how many clock cycles were spent on average by the CPU to execute each instruction during a given period of the algorithm. While this is not exposed as a single CPU performance counter, it can be derived from the Instructions Retired and Non Halted Cycles counters.
Clearly this metric is highly dependent on hardware (CPU model and clock speed, cache architecture, memory speed, and so on), and CPI numbers are normally quoted alongside these hardware characteristics in order to provide insight into how efficiently an algorithm performs on particular hardware.
In diagnosing false sharing, we can rely on the fact that a CPI measurement includes the impact of memory access latencies and the overhead of cross-processor cache synchronization (both of which cause the CPU to sit idle, wasting valuable clock cycles) in addition to the actual clock cycles that the CPU spends executing instructions. This seems to fit the task at hand nicely, because everything else being equal, false sharing effectively causes an increase in both cross-processor communication overheads and accesses to the main memory. Therefore, the same operation being completed in the presence of false sharing will lead to an increased CPI number.
It's also interesting to note that, unlike the L2 Cache Misses counter, the CPI metric is independent from the execution time. Therefore it can be used to make comparisons between runs with different durations.

Using the Visual Studio Profiler
In Visual Studio, L2 misses can be tracked in both sampling and instrumentation-based profiling modes, while CPI requires instrumentation-based profiling because it involves using two CPU counters.
To profile for L2 misses in sampling mode, first create a sampling-based performance session in Visual Studio for your project. Then use the session's properties to change the sampling event to Performance Counter, and select either L2 Misses from the Portable Events > Memory Events category or L2 Lines In from the Platform Events > L2 Cache category. (On some processors, L2 Misses will be marked as unsupported.)
Now you can profile your application. The reported numbers will be based on profiling samples taken every 1M L2 miss events. This means that when sorted by exclusive samples, the functions causing most L2 misses will be on top.
To profile for L2 misses and CPI in instrumentation mode, start with an instrumentation-based performance session. In the properties of this session, enable collection of CPU counters and select the CPU counters of interest: the L2 Cache Misses event as described earlier, along with the Instructions Retired and Non Halted Cycles from the Portable Events > General category.
The reports generated by the profiler will include a number of new columns showing the inclusive and exclusive measurements for each of these CPU counters, as well as the regular elapsed time-related metrics. The most useful information for our purposes is to be found in the per function exclusive values for each of these counters.
It is natural to receive a lot of L2 misses in functions that access a wide range of memory, especially if it is larger than or comparable to the L2 cache size. But if a particular function causes a significant number of L2 misses even though it is known to access only a small memory range, there's good reason to suspect that those misses are triggered by shared cache lines. This simple test can be applied to both instrumentation and sampling-based profiling data. However, since there is no easy way to tell what kind of results to expect for a good case, this is best applied comparatively, such as by cross-referencing runs of the same algorithm under different settings and concurrency levels.
You can also try more advanced approaches. For the instrumented profiling mode, the exact number of L2 misses incurred for a given function will be reported under the L2 Misses Exclusive column. If you have a rough idea of how much memory is referenced from this function you can also use these numbers to estimate how L2 misses compare against the total expected memory reads. However, this technique requires careful attention to the architecture details such as cache line size and the actual behavior of the CPU counter being used.
To illustrate, the L2 miss statistics collected for a test profiling session are shown in Figure 5. The application under test runs four threads that repeatedly traverse and update an array of structs whose total size in memory is about 100KB. The test machine had four CPUs (dual-socket, dual-core), with a total of 4MB L2 cache.
Figure 5 L2 Miss Statistics for a Test Session
Test Scenario | Total Amount of Memory Reads | Size of Unique Memory Addresses Referenced | "Exclusive L2 Lines In" for the Hot Spot Function
With False Sharing (accesses 4 bytes apart) | ~1GB | 100KB | 310,000,000
Reduced False Sharing (accesses 16 bytes apart) | ~1GB | 100KB | 95,000,000
Without False Sharing (well separated) | ~1GB | 100KB | 240,000
In this example, we can identify the false sharing cases from the L2 statistics without even having to consider architectural details for a precise calculation. It's clear that a significant fraction of reads have resulted in L2 misses. While the application is referencing only 100KB of unique memory, which would fit into the L2 cache with ample room to spare, the reported number of L2 Lines In is approaching the order of total memory references.
Sometimes frequent, short function calls can cause instrumented profiling to introduce enough overhead to perturb runtime behavior and, therefore, profiling results. In those cases, it is a better idea to rely on sampling-based profiling to collect L2 statistics. The example shown inFigures 3 and 4 was such a case due to the large number of Random.Next calls involved. When we profiled this example with sampling, we got the results shown in Figure 6. Although not as dramatic as the results shown in Figure 5, our hint for false sharing here is that the L2 miss counts clearly indicate a considerable fraction of the Random.Next calls (3.5M / 40M) leading to cache reloads while the referenced data is known to remain completely static in memory from each thread's point of view. The sampling results for the corrected case, also shown in Figure 6, support this claim.
Figure 6 L2 Miss Statistics for the Random.Next Example
Test Scenario | Total Calls to Random.Next | "Exclusive L2 Lines In" Top Hits (configured for one sampling hit per 1000 L2 events)
With False Sharing (Figure 3) | 40M (4x10M) | System.Random.InternalSample -- 3,525 Program.DoWork -- 1,828 System.Random.Next -- 271 [mscorwks.dll] -- 161 [ntdll.dll] -- 78
Without False Sharing (Figure 4) | 40M (4x10M) | [mscorwks.dll] -- 101 [ntdll.dll] -- 76 System.Random.InternalSample -- 53
The CPI metric for a specific function can be calculated as the ratio of two performance counters: Exclusive Non Halted Cycles and Exclusive Instructions Retired. This requires the use of two separate sampling-based profiling sessions, since only one of these counters can be profiled in a given run.
As we mentioned earlier, CPI for a given algorithm will be highly dependent on the hardware it's running on. Therefore, it's important to make comparisons against similar hardware only. One way of using this metric is to compare different runs of the same algorithm with various concurrency levels. This can be helpful in cases where it's difficult to compare L2 statistics as a result of execution time differences when concurrency levels are lowered, because CPI is independent of the running time.
If the same algorithm yields a noticeably smaller CPI value when run on a smaller number of CPUs with the same input data, it implies the algorithm is using the hardware less efficiently for high concurrency levels. If the CPI metric was focused on a mostly computational piece of the code, such that software synchronization effects are excluded, then this is a good indication that the reduction in hardware efficiency is potentially due to cache effects.

Is False Sharing Just Theoretical?
The description of the false sharing problem sounds rather esoteric, something that you will never encounter in practice. However, this problem is real and occurs surprisingly often in real-world scenarios.
While it is possible to write your parallel application without thinking about false sharing, depending on performance testing to detect such problems, it is generally a good idea to keep the false sharing issue in mind throughout the design and implementation phases of your application development. Since you are bothering to write a parallel application, presumably you care about performance. It would be a shame to go though the effort of designing your application so that it scales nicely across multiple cores, and then see much of the parallel speed-up wiped away due to false sharing.
Another thing to keep in mind is that the false sharing problem is not always easy to catch via testing. The behavior is somewhat nondeterministic, may vary across different hardware (due to differences such as cache line sizes), and can change drastically after innocent changes to the code. In fact, many optimizations you might be used to implementing to improve application performance when writing single-threaded applications can actually lead to worse false sharing problems in multithreaded applications. If you are developing an application that demands high performance, thinking about false sharing early in the process will likely pay off in the end.

Memory Locations at Risk of False Sharing
False sharing only becomes a problem if the inefficiencies it introduces dominate the computation time. If a cached value is unnecessarily invalidated only once in a while, the effect on the overall execution time likely will be negligible. But if a thread spends much of its time reading from and writing to a particular memory location, and the cached copy of that memory location keeps getting invalidated by another thread due to false sharing, then resolving the false sharing issue is likely to significantly improve the performance of the application.
Due to the principle of locality, programs often spend much of their time accessing memory locations that are close to each other. For example, consider a common parallel-computing scenario: several threads are cooperatively computing a result. Each thread computes an intermediate result, and intermediate results from different threads are combined to produce the final result. If each thread spends much of the time reading and writing its intermediate result, and the intermediate results from different threads are close in memory and fit on the same cache line, false sharing may significantly impact the performance of the algorithm.
A simple example fitting this pattern is computing the sum of integers in an array in parallel. Each thread processes a section of the array, adding up the elements in that section into an accumulator. If the accumulators from different threads are located on the same cache line, performance is likely going to suffer as threads spend much of their time stomping over each other's cache, in effect serializing the execution of the application at the cache level.
As an example of a more complex real-world situation, we hit the false sharing issue when working on the Parallel LINQ (PLINQ) project. PLINQ is a LINQ provider similar to LINQ-to-Objects, except that it distributes the work of evaluating the query across all cores available on the machine. Without getting into too much detail, PLINQ can represent a parallel computation using a set of enumerators (classes implementing the IEnumerator<T> interface). When iterating, each enumerator walks over a partition of the input and computes corresponding outputs. Different enumerators are moved forward by different threads and, as a result, the computation is distributed across multiple threads. If multiple cores are available on which to schedule the threads, parallel speedup will likely be achieved.
As it turns out, this scenario is susceptible to false sharing. We have several threads, each of which will be reading and modifying fields on its own enumerator. If the enumerators end up close in memory—and in a naive implementation where the enumerators are created and then handed out to threads (as in the Random example earlier), they are likely to—false sharing will significantly eat into the parallel speed-up.

Avoid Contiguous Allocation of Memory
While it is true that the .NET memory allocator hides many of the details of memory layout, in many cases we can make good guesses about certain properties of the layout, or even know some of them for sure. We can assume the following:
  • Two instance fields in the same class instance are close in memory location.
  • Two static fields in the same type are close in memory.
  • Two elements with adjacent indices in an array are adjacent in memory.
  • Objects allocated consecutively are likely to be close in memory, and may get closer over time.
  • Local variables used together in a closure are likely captured in instance fields, and thus, from the first comment above, are likely close in memory.
In other cases, we know that two memory locations will likely be far apart. For example, objects allocated by different threads are likely to be far apart in memory.
If a thread allocates one object, then a number of medium or large objects, and then another object, the first object and the last object are likely to be separated by the distance equal to the sum of the objects allocated between them. This behavior is not guaranteed, but it often occurs in practice. (However, if the objects in the middle are too large, they may end up in completely different heaps, in which case the first and last object will likely be close in memory.)
If two objects are padded with unused fields whose total size is at least the size of the cache line, then the two objects will not end up on the same cache line. The padding fields should come last in memory. If we know the order in which objects appear in memory, it is sufficient to pad just the one that comes first.
Based on these assumptions, we can design a parallel application to minimize the number of false sharing issues that we would otherwise have to debug later.
There is hope that future programming models will hide some of this complexity from the user. In the near term, however, understanding how memory and caches work will continue to be important for writing efficient parallel programs.

Send your questions and comments to netqa@microsoft.com.

Stephen Toub is a Senior Program Manager Lead on the Parallel Computing Platform team at Microsoft. He is also a Contributing Editor for MSDN Magazine.

Igor Ostrovsky is a Software Development Engineer on the Parallel Computing Platform team at Microsoft. He is the primary developer for Parallel LINQ (PLINQ).

Huseyin Yildiz is a Software Development Engineer on the Parallel Computing Platform team at Microsoft. He works on the Task Parallel Library (TPL).


{ End Bracket }
Don’t Be Afraid To Throw Away Your Work
Josh Phillips

We talk often about the "manycore shift"—how it affects us and what we're doing about it—but we frequently gloss over how we can harness it to develop richer client experiences. Exactly what do you do with 4 cores, 64 cores, or 1,000 cores?
For certain domains, it just makes sense: their algorithms are more compelling when they're faster. For others, it takes a bit more thinking. Yes, intuitive user interfacing, such as natural language queries and gestures, can contribute significantly to richer client experiences. But these technologies are research-heavy and difficult to implement. So how else can you make your client apps richer today? Aside from increasing UI responsiveness in general, my radical suggestion is to start throwing away work.
What? Am I actually suggesting that you toss out your hard-earned CPU time as if it were yesterday's news? Yes, I am. In the days of faster and faster single-cores, CPU time was coveted—wasting it was getting in the way of doing something more important. With multicore and especially manycore machines, abundant processing resources afford the opportunity to make more compelling apps by doing both the important stuff and the not-so-important-but-still-useful stuff. You can now compute speculatively: predict what your user will do next, pre-compute the results, and have them ready when they're needed, if they're needed.
Consider a typical photo-editing suite bundled with filters for making your picture look like a painting or a pencil sketch. When you decide to apply a filter, you get a window with some canned picture that's already had each of the filters applied to it. Scroll through this box and search for the filter you want; after you've clicked it, the app chugs along, revealing the transformation to you piecemeal. After the wait, you decide that it doesn't look quite right. You undo, return to the filter box, and repeat.
Let's take this same scenario again and throw some speculative computation on it, sprinkled with 128 cores. Now as soon as the filter window pops up, it has already applied all of the visible filters to a thumbnail of your image. When you scroll to the right, all of those filters have previews as well. What's more, it's also started transforming the original-size image with all of the filters that are currently visible. Now when you click on a filter, the image is already transformed. Once a filter has been selected, the application could also speculatively process the image with different parameters. Take it further and imagine plugging in your camera: as you're browsing through your vacation photos, your computer is already cropping, removing red-eye, and auto adjusting. If you don't like what the computer has done, just say no, throw away the work, and revert to the unedited photo.
Speculative computing isn't limited to photo apps. When you hop into your car, your 4-core navigation system could determine which destination you're most likely to choose, based on the day of the week, time, trip history, and a Bluetooth connection to your phone's calendar and address book. Every time it determines a likely destination, it will pre-compute the route, incorporating traffic data, and, detecting that you're low on fuel, keep the nearest and cheapest gas station in mind. If it turns out you're just making a quick run to the local deli, it can simply throw the work away.
The takeaway here is that we now can stop acting like each cycle of CPU time is worth pure gold. We'll soon have such an abundance of processing power that we'll be more than willing to waste a little for a more compelling app.
And there's no need to wait for the 64-core processors. With quad-cores on shelves now and 8-cores just around the corner, you can start throwing away work today. The Parallel Computing Initiative at Microsoft is working to make multicore and manycore programming easier by abstracting away the difficult threading model. A Community Technology Preview (CTP) of the Parallel Extensions to the Microsoft .NET Framework is now available and is well suited for speculative computing, including task parallelism, data parallelism, coordination, and cancellation APIs. The Parallel Extensions (as well as similar technologies for the native stack) and tools are coming in the next version of Visual Studio.
So speculate. Make suggestions for your users and eliminate the bulk of their mundane tasks. Make your UIs snappy, immersive, and innovative. And don't be afraid to toss out some processing time if you've done it in the name of a great experience.

Josh Phillips is a Program Manager on the Microsoft Parallel Computing Platform Team. He is currently working on libraries, languages, and tools to enable developers to productively utilize concurrency and build the next generation of rich and compelling applications.