May 2009
.NET RIA Services: Building A Data-Driven Expense App with Silverlight 3
.NET RIA Services provides a set of server components and ASP.NET extensions such as authentication, roles, and profile management. We’ll show you how they work. Jonathan Carter
Cloud Computing: Patterns For High Availability, Scalability, And Computing Power With Windows Azure
Here we examine the typical cloud platform architecture and some common architectural patterns, along with their implementation on the Windows Azure offering from Microsoft. Joshy Joseph
Data Services: Access Your Data On Premise Or In The Cloud With ADO.NET Data Services
In this article the author looks at two versions of the same application--one consuming an on-premise Data Service, and one consuming an Azure Table Data Service to illustrate data access in the cloud. Elisa Flasko
SOA Simplified: Service Virtualization With The Managed Services Engine
By using Managed Services Engine service virtualization technology, you can begin participating in the cloud by simply configuring virtual service endpoints and operations that integrate with the .NET Service Bus. Aaron Skonnard
Columns
Editor's Note: Head in the Clouds: This month Editor-in-Chief Howard Dierking comes down from Cloud 9 when he realizes that employing cloud services is less about magic and more about planning. Take a few tips from his recent experiences.Howard Dierking | Toolbox: Easy File Backup, Exploring Files And Folders Inside Visual Studio, Multiple Monitor Software, And More: If the responsibility for creating, managing, and executing routine backups is yours, these tools will make it easier. Also see how you can browse folders and files from inside Visual Studio.Scott Mitchell | Basic Instincts: Documenting Your Code With XML Comments: XML comments provide an easy and effective way to document your code. We’ll show you how to use and customize XML comments in your Visual Basic projects.Lisa Feigenbaum
CLR Inside Out: Understanding The CLR Binder: In this article, we cover some of the best practices for assembly binding and loading using the CLR.Aarthi Ramamurthy and Mark Miller | Cutting Edge: Build Rich User Interfaces with jQuery: In this month's installment we build modal and modeless dialog boxes in jQuery and explain how to post data from them to the Web server.Dino Esposito | Extreme ASP.NET: The Life And Times of an ASP.NET MVC Controller: Here the author dissects the ASP.NET MVC framework and looks at how controllers work. He then explains how the framework interacts with your controllers and how you can influence those interactions.Scott Allen
Foundations: Versioning Workflows: Developers often struggle with versioning workflows and their related classes. Matt Milner discusses the core issues related to workflow versioning and provides recommendations for making changes to workflow definitions, activities, and workflow services.Matthew Milner | Going Places: Gaming In The Key Of Zune: Mike Calligaro shows you the basics of using XNA Game Studio 3.0 to write games for Zune.Mike Calligaro | Office Space: Custom Field Types for SharePoint 2007: Custom field types provide a powerful way to define a reusable SharePoint column definitions.Ted Pattison
Security Briefs: A Conversation About Threat Modeling: Listen in on a chat between a developer and security pro that delves into some of the major Security Development Lifecycle (SDL) requirements we impose on product teams here at MicrosoftMichael Howard | Under the Table: Programming with FileStreams in SQL Server 2008: There's always been disagreement about whether large blobs, such as document and multimedia items, should be stored in the database or file system. In SQL Server 2008 you don't have to choose; filestream storage provides the best of both approaches.Bob Beauchemin | Patterns and Practices: Simplifying Patterns and Practices: This month we in introduce a new column that takes you inside the Microsoft patterns & practices group. This first installment focuses on ways groups inside Microsoft have implemented the Dependency Injection pattern in Microsoft products and tools.Alex Homer
Usability in Practice: Useful, Usable and Desirable: Usability as a Core Development Competence: In this month's installment, learn how to achieve the most important outcome of all UI design: ensuring that your software is useful, useable, and desirable.Dr. Charles B. Kreitzberg and Ambrose Little | {End Bracket}: The Unconference: Where Geeks JIT Together: This month the author takes us on a tour of the unconference and other unconventional get-togethers. Learn how these unique approaches to conferences are being widely adopted.Steven M List


.NET RIA Services
Building A Data-Driven Expense App with Silverlight 3
Jonathan Carter
Code download available from the MSDN Code Gallery
Browse the Code Online

This article is based on a prerelease version of .NET RIA Services. All information is subject to change.

This article discusses:
  • Getting started with .NET RIA Services
  • Data services and domain operations
  • Code projection
  • Flexible data controls
This article uses the following technologies:
Silverlight 3, .NET RIA Services, WPF
Contents
Getting Started
Data Services Library
Domain Operations
Code Projection
Data Controls
ObjectDataSource
DataPager
DataForm
Metadata
Validation
Shared Code
Wrap Up
Within software development, there are many different styles of applications, each of which comes with its own purpose and requirements, strengths and weaknesses. When choosing a development platform or framework, part of your decision process will most likely rely on whether the framework can easily enable the type of application you are looking to build. No developer wants to spend hours writing plumbing or infrastructure code, and no customer wants to pay for that type of work to be done. Being able to primarily focus on business needs is important, so having a development platform that better enables that focus and increases productivity is an absolute must.
Silverlight is a great technology for creating compelling Web applications. While Silverlight 2.0 includes a subset of Windows Presentation Foundation (WPF), a fair amount of the functionality in WPF that made it easier to create data-driven applications wasn't inherited. This has caused many developers to ultimately pass up on using Silverlight because it would require too much work to implement the data-centric infrastructure of more sophisticated Web solutions. As part of the Silverlight 3 release, new controls and features are being introduced that specifically make data-driven applications easier to develop. This includes new data controls, navigation, validation, and embedded dialog windows.
While these client-side enhancements provide a substantial amount of value on their own, Silverlight is a multitiered environment, and as such requires knowledge of how to deal with client-server communication. To solve this problem, .NET RIA Services (code-named "Alexandria") provides a set of server components and ASP.NET extensions that ease the n-tier development process, making your applications almost as easy to develop as if they were running on a single tier. In addition, services such as authentication, roles, and profile management are provided. The combination of client and server enhancements to Silverlight 3 and ASP.NET, along with the addition of .NET RIA Services, streamline the end-to-end experience of developing data-driven Web applications—also known as Rich Internet Applications, or RIAs.
Figure 1 Default .NET RIA Services Project
In this article, I will show you how the enhancements made to Silverlight 3, in concert with the functionality introduced by .NET RIA Services, give you a rich environment for developing data-centric Web apps. To best illustrate the functionality that Silverlight 3 and .NET RIA Services provide, I am going to build out an expense report tracking application that will take advantage of the new feature sets. The application will allow employees to log in and manage and create their personal expense reports. Once an expense report is completed, it can then be submitted, awaiting the approval of a manager.

Getting Started
Once you've installed .NET RIA Services and Silverlight 3, Visual Studio includes a new project template called Silverlight Data Application that enables you to get up and running quickly. When your solution is created, you're presented with two projects: a Silverlight project and an ASP.NET project (see Figure 1). The Silverlight project represents your client application, and the ASP.NET project will contain your back-end logic.
What is the advantage of this default project structure? As far as the provided Silverlight project goes, it is identical to what you'd get if you created a regular Silverlight project. You have a standard App.xaml file and a default page. There are, however, subtle differences to this Silverlight project that will aid your development goals.
The ASP.NET project is straightforward as well, but if you look within the default.aspx page, you'll notice that it makes use of a new server control called SilverlightApplication.
<ria:SilverlightApplication
  ID="Xaml1" runat="server"
  Source="~/ClientBin/ExpenseReports.xap"
  MinimumVersion="1.0"
  Width="100%" Height="100%" />
This control is meant to make the act of hosting a Silverlight application within an ASP.NET WebForms application easier. The SilverlightApplication control is already configured to target the XAP that will be created from the Silverlight project.
At this point, you have a fully functional and hosted Silverlight application. Now you just need to begin adding custom business logic and UI.
Let's assume you've already got a database in place for this application, so the next thing to do is add an ADO.NET Entity Data Model to the server project. I'll be using the Entity Framework as the object-relational mapping (O/RM) in this article, but you could also use LINQ to SQL, NHibernate, traditional ADO.NET, or any other data access method.
The data model is very simple, containing only three entities: ExpenseReport, ExpenseReportDetail, and Employee (see Figure 2). For more information on the ADO.NET Entity Framework on MSDN.
Figure 2 Data Mode
With the data model in place and the O/RM selection made, you can now begin implementing the back-end logic. Since this project employs Silverlight as the client and ASP.NET as the host, you need to decide on an approach for communicating between the two tiers. You could use any of the currently available communication frameworks such as Windows Communication Foundation (WCF) or ADO.NET Data Services, depending on your requirements.

Data Services Library
.NET RIA Services introduces a library of server components that make building data-driven RIAs easier. The library is not reliant on any specific UI framework, and while this article focuses on its consumption by Silverlight, that is only one of its possible client options. In future releases, .NET RIA Services will also work with ADO.NET Data Services.
.NET RIA Services primarily revolves around a new class called DomainService that acts as a server endpoint for business logic and data model interaction. Creating one is as simple as taking advantage of the Business Logic Class templates installed by .NET RIA Services in Visual Studio.
A DomainService can exist in two forms: data model–specific or generic. The data model–specific implementation effectively wraps a data model, allowing you the option of writing a combination of business logic and data access code. This makes it more convenient to get up and running quickly in scenarios where you already have a data model that you want to work with.
There are two data model–specific DomainService implementations provided with .NET RIA Services: ADO.NET Entity Framework and LINQ to SQL. If your application makes use of either of those, then you can select the appropriate option in the wizard when creating a DomainService. If you're using another type of data model or O/RM, you could create your own specific implementation of a DomainService as well.
For this article, since I already have an Entity Data Model in place, I'll go ahead and create an Entity Framework–specific DomainService. I receive a new class that inherits from LinqToEntitiesDomainService<T>:
[EnableClientAccess]
public class ExpenseService :
  LinqToEntitiesDomainService<ObjectContext> {

  //public IEnumerable<Employee> GetEmployees()
  //{
  //    return this.Context.Employee;
  //}
}
The generic parameter in this case represents the type of ObjectContext instance that represents the connection to the Entity Data Model. The first step is to heed the advice of the TODO comment and replace the type parameter placeholder with the actual ObjectContext, in this case ExpenseReportContext.
In addition to inheriting from DomainService (or a derivative), a domain service can have an EnableClientAccessAttribute attached to it. This attribute is what really signifies your class as being a domain service and allows you to specify whether it should be publically exposed. By publically exposed, I mean accessible to your client application. This gives you the choice to determine whether some logic is solely required on the server, or if it should also be available on the client.

Domain Operations
A domain service isn't very useful on its own unless you add some functionality to it in the form of domain operations. A domain operation represents an endpoint of your domain service and can perform create, read, update, and delete (CRUD) operations against your data model, your arbitrary business logic, or both. Each domain operation must map to a specific operation type, including query, insert, update, delete, service operation, and custom. The operation type mapping can occur either by convention or by configuration.
Depending on which operation type your domain operation is going to perform, it has to follow a specific signature. In addition, some operations have to either follow a specific naming convention or have an attribute that determines what type of operation it is. For instance, if it is a query operation, then its return type has to be either IEnumerable<T> or IQueryable<T> where T is an entity type it works with. It can accept any number of parameters, which can act as filters, but none are required.
Creating a domain operation for retrieving all expense reports might look like this:
public IEnumerable<ExpenseReport>
  GetExpenseReports() {

  return Context.ExpenseReports;
}
ExpenseReport is an entity in the underlying data model and is, therefore, a valid return type. The DomainService class contains a property called Context that provides access to an instance of your data model (the type you provided as a generic parameter), which in this case allows you to query for a list of all expense reports. This operation is mapped to a type by convention.
Creating a data method for retrieving a specific expense report might look like this:
[Query(IsComposable = false)]
public IEnumerable<ExpenseReport>
  GetExpenseReport(int id) {

  var expenseReport =
    from rep in Context.ExpenseReports
    where rep.Id == id
    select rep;

  return expenseReport;
}
Notice that this data method takes a parameter for retrieving a specific expense report by ID. Because you have access to the underlying ObjectContext, you can also write the query using LINQ. This method is mapped by configuration, using the QueryAttribute, which also allows you to specify additional properties that can't be achieved conventionally. I'll touch on what the IsComposable property of the QueryAttribute does later in the article.
A DomainService can contain as many query methods as needed. In addition to retrieving data, you can create operations for persisting data back into the data model. Implementing basic persistence operations (insert/update/delete) for expense reports might look like this:
public void InsertExpenseReport(
  ExpenseReport expenseReport) {
  Context.AddToExpenseReports(expenseReport);
}

public void UpdateExpenseReport(
  ExpenseReport current, ExpenseReport original) {
  Context.AttachAsModified(current, original);
}

public void DeleteExpenseReport(
  ExpenseReport expenseReport) {
  Context.DeleteObject(expenseReport);
}
You could define the persistence operations for expense report details in much the same way. You can have only a single insert, update, and delete method per entity type (ExpenseReport in this scenario) because, when a request comes in to the DomainService to save changes back to the data model, the DomainService needs to know which method to call.
One thing to notice about these operations is their signatures. When you create an insert or delete method, it must take a single parameter that conforms to the entity type the method is responsible for inserting or deleting. When you create an update method, it must take two parameters: the modified (or current) entity instance and the original entity instance. The method signature in conjunction with the method name is what signifies to the DomainService which method is responsible for which operation for which entity type.
The naming convention that can be used when defining persistence operations is to prefix your method names with the type of operation. For instance, because the method is called DeleteExpenseReport, the Delete prefix signifies that is it a delete operation by convention. The signature then defines what entity type it is associated with (ExpenseReport). As you would expect, the conventional method name prefix for update operations is Update and insert operations is Insert. This is why I didn't have to do any additional configuration for these operations to work.
If an operation doesn't follow the expected naming convention or you need to specify additional metadata for your operation (like I did with the GetExpenseReport method), then you can apply configuration attributes such as QueryAttribute, InsertAttribute, DeleteAttribute, and UpdateAttribute as I did with the GetExpenseReport method.
If you need to define business logic that isn't necessarily tied to a CRUD operation, but is associated with an entity type, you can create a service operation. In this case, I want operations for approving and rejecting expense reports. This simply means modifying the expense report's status value (see Figure 3). The signature for a service operation has to accept an instance of the entity type it is associated with and must return void. Note that there is no convention for defining service operations, so you must apply the ServiceOperationAttribute to configure any appropriate domain operation.
Figure 3 Service Operations
[ServiceOperation]
public void ApproveExpenseReport(
  ExpenseReport expenseReport) {

  if (expenseReport.Status == 1) {
    if (expenseReport.EntityState ==
      System.Data.EntityState.Detached) {
      Context.Attach(expenseReport);
    }
    expenseReport.Status = 2;
  }
}

[ServiceOperation]
public void RejectExpenseReport(ExpenseReport er) {
  if (er.Status == 1) {
    if (er.EntityState ==
      System.Data.EntityState.Detached) {
       Context.Attach(er);
    }
    er.Status = 0;
  }
}
Now that you've got the domain service and operations created, how do you go about interacting with the service from the ASP.NET server host to the Silverlight client application? If you were using WCF or ADO.NET Data Services, you would create a reference in the Silverlight project that pointed to the service, which would generate a proxy. .NET RIA Services provides a slightly richer experience.

Code Projection
When you create the .NET RIA Services solution, the Silverlight project file had some special MSBuild tasks added to it that handle projecting specific server components to the client. When you create a domain service in the ASP.NET project that has an EnableClientAccessAttribute applied to it, the MSBuild task will project that domain service to the Silverlight application. (If you didn't create your solution using the new Silverlight Data Application project template, you can manually link a Silverlight project and ASP.NET project within Visual Studio to achieve the same effect.)
If you look in the code-behind of the Main.xaml file in the Silverlight project, you'll find both ExpenseContext and ExpenseReport types. When a data provider is projected to a Silverlight client application, it is no longer a DomainService (or subtype), but rather a DomainContext. In fact, if your domain service is suffixed with the word Service, it will be replaced with Context upon projection (which is why the ExpenseService class is reflected as Expense-Context in the Silverlight project). The DomainContext class acts as a client-side proxy for a DomainService and contains the logic necessary for communicating requests between the two tiers. It represents a unit of work and can build up a series of change sets made to any entity instance it is currently tracking.
In addition, certain methods (public methods that are deemed domain operations either by convention or configuration) will be projected along with their parent domain services. Of the six data method types, only three are projected: custom, service, and query. If a query method's name is prefixed with Get, upon projection Get will be replaced with Load. For example, because I defined a query method with the name GetExpenseReports, it will surface on the client as LoadExpenseReports. The two service operations were projected as well and are reflected as instance methods on DomainContext.
Finally, any entity types that are returned from a domain operation will also be projected. In my example, because the GetExpense-Reports method's return type is IEnumerable<ExpenseReport>, the ExpenseReport entity class will be projected to the Silverlight client. The projected entity classes inherit from a special class called Entity that provides behavior such as change tracking, validation checking, and WPF/SL-compatible editability. From an API perspective, your client-side entity class will look just like the entity on the server, but will contain additional functionality that is required for interacting with Silverlight data controls.
There are a lot more details surrounding the code projection behavior than are discussed here. This article doesn't touch on many additional scenarios that are possible, including the ability to customize the logic that determines how a specific type's code is shaped before projecting it to the client application.
I'm using the term code projection because I think it appropriately describes what is going on. The ExpenseService class isn't simply being copied from one project to another (there is actually an exception to this that I'll discuss later). It is being examined, shaped, and projected into the client application as an easy-to-use proxy. Hence, you can leverage the type from the client tier as if it were a local object.
Where is the projected domain service hiding? The Silverlight project looks untouched from its created state. If you turn on the option to show all files for the Silverlight project, you'll see the culprit (see Figure 4). A folder is created called Generated_Code that contains all of the code that has been projected from the partner server project. There is currently only a single file in there, ExpenseReportsServer.g.cs (the g stands for generated), which contains the code for the entire ExpenseReportsServer project and is the home of the ExpenseContext class.
Figure 4 Code Generated For The Silverlight Project
If you create a new data provider or modify an existing one, the changes will be silently updated within the Silverlight project, keeping the client and server constantly in sync. In scenarios where services are created for the sole purpose of supporting an application's client, having to continually refresh service proxies during development can be a pain. This subtle behavior, in addition to the reshaping during projection, turns out to be a pretty useful feature.

Data Controls
So I've got a database, data model, services, and client-side proxies in place. Now I need to build out the UI for displaying and editing expense report data within the Silverlight client. When dealing with data-driven applications, there are typically two types of ways of presenting data: tabular and form-based. When dealing with data-driven RIAs, tabular and form-based data needs to be presented in a manner that makes the user experience highly informative and productive.
Silverlight 2 didn't have very strong support for tabular data presentation out of the box. Not even the ListView control (which provided basic grid presentation in WPF) is present in Silverlight, which made it somewhat difficult to build business software. Silverlight later received a DataGrid control, which definitely helped, but lots of necessary features were still missing. Intrinsic controls were available, such as TextBox, ComboBox, Button, ListBox, and RadioButton, which made developing forms possible, but there wasn't strong support for other behaviors that are necessary, such as data validation and error reporting.
Silverlight 3 introduces a set of new controls that are specifically aimed at making the creation of data-centric RIAs easier. These controls include DataGrid, DataForm, DataPager, FieldLabel, DescriptionViewer, ErrorSummary, and ChildWindow. I took advantage of DataGrid, DataForm, and DataPager to enable data presentation in both tabular and form-based styles for my sample app. FieldLabel, DescriptionViewer, and ErrorSummary provide the dynamic UI, data validation, and error reporting needed to develop responsive data entry. Finally, ChildWindow enables rich modal dialog boxes.
The DataGrid control that comes with Silverlight 3 is robust. Among other things, it includes reorderable and resizable columns, row grouping, inline editing, and validation. It allows you to define the columns displayed in the grid in two ways: generated and explicitly.
A DataGrid that has its AutoGenerateColumns property set to True will automatically create a column for every public property on the type to which it is bound—the exception being that if a property has a BindableAttribute attached to it that specifies it isn't bindable, then the DataGrid won't create a column for it. This is one example of how the new data controls take advantage of entity metadata.
When you explicitly define the columns in a DataGrid, you are scoping the displayed data to just the columns that you want to show. This gives you control over which column types to use as well as header text and numerous other column properties. A DataGrid that explicitly defines its columns for displaying expense report data might look like Figure 5.
Figure 5 DataGrid For Expense Report
<dataGrid:DataGrid
  x:Name="ExpenseReportDataGrid"
  AutoGenerateColumns="False">
  <dataGrid:DataGrid.Columns>
    <dataGrid:DataGridTextColumn
      Binding="{Binding Company}"
      Header="Company" />
    <dataGrid:DataGridTextColumn
      Binding="{Binding Department}"
      Header="Department" />
    <dataGrid:DataGridTextColumn
      Binding="{Binding Description}"
      Header="Description" />
    <dataGrid:DataGridCheckBoxColumn
      Binding="{Binding Status}"
      Header="Approved" />
  </dataGrid:DataGrid.Columns>
</dataGrid:DataGrid>
The properties that are specified in the column bindings refer to properties on the ExpenseReport entity. Remember that the class was projected to the Silverlight application, making it usable in the client. The DataGridTextColumn class will display the data as text in read-only mode and as a text box in edit mode. The DataGridCheckBoxColumn will display a field as a checkbox in all modes and accommodate three states.
To fill a DataGrid you simply set its ItemSource property to any IEnumerable object. Remember that when I defined the Expense-Service and the GetExpenseReports data method, its return type was IEnumerable<ExpenseReport>. Now I just need to figure out how to consume the client-side proxy for the data provider that was generated.
When the ExpenseService class was projected to the client application, its GetExpenseReports method was projected as well (after being renamed to LoadExpenseReports). This means you should be able to just create an instance of ExpenseService and call its LoadExpenseReports method. While that is the correct approach to take, it wouldn't produce any results on its own. Interestingly enough, the LoadExpenseReports method doesn't return anything. How do you get the expense report data?
Silverlight requires any blocking calls to be performed asynchronously so that the UI thread doesn't lock up. Because of this, when .NET RIA Services projects your domain services to a Silverlight client, it refactors all of its query operations so that they no longer return anything. This explains why .NET RIA Services will rename your query methods to be prefixed with Load instead of Get (or Fetch, Find, Query, Retrieve, or Select), because it reflects its behavior more appropriately.
Instead of employing a callback approach, the classes generated by .NET RIA Services use an event model for notifying you after an asynchronous call has completed. Hence, to respond to the loading of a query operation, you simply subscribe to the DomainContext's Loaded event, which will be triggered as soon as any query operation completes. Because the event handler is called on the UI thread, you can carry out any data binding within it.
Figure 6 Expense Report Data Loading
public partial class Main : Page {
  ExpenseContext _dataContext;

  public Main() {
    InitializeComponent();

    this.Loaded += Main_Loaded;

    _dataContext = new ExpenseContext();
    _dataContext.Loaded += dataContext_Loaded;
  }

  void dataContext_Loaded(object sender, LoadedDataEventArgs e) {
    ExpenseReportDataGrid.ItemsSource = e.LoadedEntities;
  }

  void Main_Loaded(object sender, RoutedEventArgs e)  {
    _dataContext.LoadExpenseReports();
  }
}
Figure 7 Expense Reports Grid UI
Once the Loaded event handler has been triggered, there are two ways to retrieve your requested data: through the LoadedDataEventArgs.LoadedEntities property or through one of the strongly typed entity properties on your domain context (such as ExpenseContext). Accessing the data through the event arguments is the preferred approach since this will guarantee that you get only the data you want. Every time a query operation is called, the returned entity instances will be added to the domain context, therefore whenever you access its contents, you will be getting the accumulation of every query, not just the one you most recently performed.
Implementing the logic for retrieving all expense reports and filling the DataGrid with the returned data might look like Figure 6. At this point, our data-driven RIA looks like Figure 7.
In addition to being able to load data, the DomainContext class (and your generated subtypes) contains methods for managing change tracking and data persistence. Every change you make to an entity that was retrieved (or added or deleted) through a DomainContext will be tracked. When you want to save changes, you can simply call the SubmitChanges method:
private void SaveChangesButton_Click(
  object sender, RoutedEventArgs e) {
  if (_dataContext.HasChanges) {
    _dataContext.SubmitChanges();
  }
}
Since the DataGrid enables inline editing by default, you can modify any of the expense report records and click the Save Changes button to persist them to the server. When the SubmitChanges method is called, the DomainContext assembles a change set for all of its tracked entities and sends it to the corresponding DomainService. The DomainService then decomposes the change set and calls the respective domain operation for each entity that was inserted, updated, or deleted.
When you are editing data within the DataGrid control, it provides validation and error reporting automatically. If you enter invalid data, you'll be visually notified with an explanation of what is wrong (see Figure 8). This feature comes without any configuration or work on your part. As you'll see later on in the article, you can also define custom validation rules on your data model that will be picked up and ensured by the DataGrid.
Figure 8 DataGrid Validation
This view of the expense reports is fine, but it would be a lot more useful to see expenses grouped by status. That way I could immediately figure out which reports are pending approval by a manager. Luckily, this can be achieved easily in Silverlight 3 by appending this code to the existing DataGrid definition:
<dataGrid:DataGrid.GroupDescriptions>
  <dataGrid:PropertyGroupDescription
    PropertyName="Status" />
</dataGrid:DataGrid.GroupDescriptions>
The result looks like Figure 9.
Figure 9 DataGrid Grouping

ObjectDataSource
While some developers will prefer the imperative data binding approach used thus far, others might like being able to perform data binding purely declaratively, much like the way they might work with the ObjectDataProvider in WPF. For this, Silverlight 3 introduces the ObjectDataSource control.
ObjectDataSource is a nonvisual control that knows how to work specifically with DomainContext types. It can be thought of as providing all of the functionality that the previous imperative approach did but by means entirely declarative (plus more). You simply tell it which DomainContext type you're working with and which of its query operations you want called. ObjectDataSource will handle the rest.
You could remove all of the code from the previous section and replace it with an ObjectDataSource, and it will automatically call the specified load method:
<ria:ObjectDataSource
  x:Name="ExpenseReportsObjectDataSource"
  DataContextType="ExpenseReports.ExpenseContext"
  LoadMethodName="LoadExpenseReports"
  PageSize="20">
  <ria:ObjectDataSource.Filter>
    <data:FilterDescriptor Member="Department"
      Operator="IsEqualTo" Value="IT" />
  </ria:ObjectDataSource.Filter>
  <ria:ObjectDataSource.Sort>
    <data:SortDescriptor Member="Status"
      Direction="Descending" />
  </ria:ObjectDataSource.Sort>
</ria:ObjectDataSource>
Using the ObjectDataSource control doesn't just provide a declarative means of data binding, it also makes queries composable. You can add sort, group, and filter expressions to an ObjectDataSource, as well as page size, that will be applied to the call to its query operation. The best part is that the ObjectDataSource will modify the request made to the domain service so that any specified sorting or filtering is applied on the server side, meaning no unnecessary data will be passed over the wire.
Remember the IsComposable property of the DomainOperationAttribute? That is what determines whether a domain operation allows additional query parameters to be passed to it, making its returned data composable. My GetExpenseReports method didn't add any sorting or filtering code, but because of the composability of the DomainService and the fact that the ObjectDataSource knows how to make composed queries, I can get that functionality automatically.
ObjectDataSource is actually a wrapper for a DomainContext instance and therefore benefits from the same change-tracking behavior. It contains the same Load and SubmitChanges methods that DataContext does, allowing you to programmatically control it like you would a DomainContext.

DataPager
Since the amount of data currently being displayed in the grid is a little unwieldy, it would probably make sense to page it for presentation purposes. While the DataGrid control itself doesn't include paging behavior, Silverlight 3 introduces a new DataPager control that works seamlessly with other data controls to easily provide paging functionality alongside them.
The DataPager control simply presents the necessary UI for paging through a provided data source. If you bind a DataPager to the same data source as another data control (such as DataGrid), paging through the data using the DataPager will also page the displayed data in the other bound controls. The code to add a DataPager to the list of expense reports might look like this:
<data:DataPager
  Source="{Binding Mode=TwoWay, Source={StaticResource ExpenseReportDataSource}, Path=Data}"/>
Figure 10 DataPager Placed At The Bottom Of A DataGrid
Notice that all I have to do is define the control and bind it to the correct source and the control will handle the rest (see Figure 10).
The DataPager has numerous modes you can select from that vary how it presents the available pages to the user. In addition, you can completely re-skin the DataPager to look however you'd like while still maintaining its existing functionality.
While the DataGrid and DataPager provide a great inline editing experience, what if you wanted to display data in a form-based layout? For creating or modifying expense reports, you want to provide the user with an intuitive form to use instead of having to rely on the grid. For that you can employ the new DataForm control.

DataForm
The DataForm control allows you to define a set of fields that will be displayed in a form-based layout and can be bound to either a single entity instance or a collection. It allows working with your data in read-only, insert, and edit modes, with the ability to customize the appearance of each. You can optionally show controls for switching between modes and, when bound to a collection, the DataForm can also show a pager for navigation. Just like the DataGrid, DataForm also comes in various forms: generated, explicit, and template.
The generated mode works just like DataGrid. It will create a field and label pair for every public property on the type it is bound to. DataForm respects the BindableAttribute as well, which allows you to define your bindable field list at the entity level. Defining an edit form for expense reports could be as simple as this:
<dataControls:DataForm
  x:Name="ExpenseReportDataForm" Header="Expense Report"
  ItemsSource="{Binding Source={StaticResource ExpenseReportsObjectDataSource}, Path=Data}" />
By using the generated mode, you're allowing the DataForm to make all UI assumptions based on the entity's metadata. The resulting form is shown in Figure 11.
Figure 11 DataForm-Derived Expense Entry Form
Labels for any required fields (properties marked with Required-Attribute) are displayed in bold, indicating the requirement to the user. Also, to the right of the input controls, the information glyph provides a mouse-over tooltip description of the expected input. A description is optional and is retrieved by examining whether the field's corresponding property has a DescriptionAttribute attached to it. These are two more examples of how the new Silverlight 3 data controls enable data-driven scenarios, by adding to your UI in response to metadata on your data models.
Just like the DataGrid, the DataForm also provides data validation and error reporting. The two controls have a consistent appearance and functionality, which provides for an overall good user experience regardless of which data presentation you need.
Using the explicit form, you can declare which fields you want to display, what type of fields to use, and what label text to display (among other things). This form is useful when you don't want to leave the UI creation up to the DataForm and entity metadata. Declaring a DataForm explicitly might look like Figure 12.
Figure 12 Explicit Creation Of A DataForm
<dataControls:DataForm
  x:Name="ExpenseReportDataForm"
  ItemsSource="{Binding Source={StaticResource ExpenseReportsObjectDataSource}, Path=Data}"
  AutoGenerateFields="False">
  <dataControls:DataForm.Fields>
    <dataControls:DataFormTextField
      Binding="{Binding Company}" Label="Company" />
    <dataControls:DataFormTextField
      Binding="{Binding Department}" Label="Department" />
    <dataControls:DataFormTextField
      Binding="{Binding Description}" Label="Description" />
    <dataControls:DataFormCheckBoxField
      Binding="{Binding Status}" Label="Approved" />
  </dataControls:DataForm.Fields>
</dataControls:DataForm>
In addition to fields for text and checkboxes, there are fields available for date, combo box, template, separator, header, and field group. With these you can explicitly define the fields you want to show and give basic instruction to the DataForm for displaying them. Even with this flexibility, you're still restricted to a top-down form, where each field is a traditional label and input control pair. While the DataFormTemplateField allows you to define a template for all modes (display and edit), it is limited to the field level. What if you wanted to template the entire form?
When complete control over your UI is needed (or desired), the DataForm allows you to define custom data templates for each of its modes (display, insert, and edit). With this ability, you can break out of the default top-down form style and create whatever look makes sense for your situation.
Certain behavior is global to all three forms of the DataForm, such as navigation, validation, and error reporting. When you choose to redefine the data templates though, you lose the automatic field label and description viewer, which worked with your model's metadata. When developing a data-driven application, it would be a shame to lose this useful behavior just because you needed to customize your layout. Fortunately, the controls that are internally used by the Dataform and that provide this behavior are also usable manually.

Metadata
When you allow the DataForm to generate your list of fields for you, it automatically makes use of two controls to provide the label and description behavior: FieldLabel and DescriptionViewer. Both of these controls are easy to use and can be leveraged in any data-bound scenario, including custom DataForm templates.
FieldLabel is useful when you want to display a label for a control that is determined from the metadata of its associated bound property. The text used for the label is derived from the Name property of the DisplayAttribute attached to the property it is bound to. In addition, if the property is required (signified by having a RequiredAttribute marked as true attached to it), the field label text will be in bold.
In addition to being able to specify a custom name with the DisplayAttribute, you can specify a description for a property. If you'd like to display a field's description, you can use the DescriptionViewer control, which handles this for you automatically. It will display an information glyph that provides a tooltip containing the description of the property it is associated with.
With the FieldLabel and DescriptionViewer controls, you can develop custom data forms that take advantage of metadata from your data model without having to replicate information (such as field names and descriptions). If you use these controls throughout your application, any time a change is made to a property's name, description, or required status (at the model level), your UI automatically reflects the change due to its dependence on the data model. This is the type of behavior you'd expect when developing data-driven applications.

Validation
Since we're focusing on the development of data-driven applications, we'd like to keep our business logic and validation close to the data model. When using .NET RIA Services you can express validation logic in two ways: data annotations and custom/shared logic.
The Microsoft .NET Framework 3.5 SP1 release introduced a set of attributes called data annotations that are meant to attach metadata and validation rules to a data model. These annotations were initially used by ASP.NET Dynamic Data and are understood and respected by .NET RIA Services and the new Silverlight 3 data controls. With them you can express such validation aspects as string length, range, data type, and regular expression constraints:
[Bindable(true, BindingDirection.TwoWay)]
[Display(Name = "Expense Amount",
  Description = "The amount of the incurred expense.")]
[Range(0.0, 1000000.00)]
public object Amount;

[Bindable(true, BindingDirection.TwoWay)]
[Display(Name = "Category",
  Description = "The category of expense, i.e., mileage.")]
[StringLength(10)]
public object Category;
When the .NET RIA Services projection process runs, it makes a point to flow any server-side data annotations to the client. As long as you take advantage of the generic data annotations to represent validation rules on your server-side data model, that validation will carry over to your Silverlight application and be fully usable by aware controls (DataForm, DataGrid, FieldLabel, and so on). This effectively gives you client and server validation.
Using the data annotations is easy, but they can't express every possible validation requirement. In fact, they can really represent only the most common scenarios. For other situations you'll most likely need to define your validation imperatively. While this is simple to do, the .NET RIA Services projection process can't just flow your custom logic to the client as that process is limited to the creation of your DataContext and entity proxy classes.
You can keep the custom logic on the server and make a service call to it from your Silverlight client, but that would kill the application's responsiveness and remove the ability for the data controls to automatically determine data validity. You can copy and paste the logic to the client, and manually perform validation on both tiers, but code replication is never a good thing, and the automatic validation problem would still exist. This is a scenario that warrants the use of the .NET RIA Services shared code feature.

Shared Code
For the expense report application, I need to ensure two custom validation rules: any export report over $1,000 must include a description outlining their purpose, and no expense reports can be filed for future purchases. Neither of these conditions can be met using data annotations, but I can easily express them imperatively.
.NET RIA Services includes a feature called shared code that allows you to define logic in your server project that will be synchronized and available in your client application as well. During the code projection process, any code marked as shared will be copied between projects instead of translated and proxied. To take advantage of this feature, you first create a new code file in your server project with the suffix of .shared.[language extension] (for example, ExpenseData.shared.cs). When the code projection process runs, it will specifically look for files within the project with that suffix and treat it as shared code.
There is a new data annotation in the .NET Framework 4.0 called CustomValidationAttribute that allows you to associate a custom validation rule with a data model at either the entity or the property level. Specifying my two custom validation rules might look like this:
[MetadataType(typeof(ExpenseReportDetailsMetadata))]
[CustomValidation(typeof(ExpenseReportValidation),
  "ValidateDescription")]
public partial class ExpenseReportDetails { }

public partial class ExpenseReportDetailsMetadata {
  [Bindable(true, BindingDirection.TwoWay)]
  [CustomValidation(typeof(ExpenseReportValidation),
    "ValidateDateIncurred")]
  [Display(Name = "Date",
    Description = "The date of when this expense was incurred.")]
  public object DateIncurred;
The .NET RIA Services projection process is aware of the CustomValidationAttribute and will flow it to your client proxy. Because the custom validation is contained within a validation type (that you most likely want to define on the server), you can take advantage of shared code to handle its synchronization.
The signature for a custom validation method must follow a specific pattern:
[Shared]
public static class ExpenseReportValidation {
  public static bool ValidateDateIncurred(object property,
    ValidationContext context, out ValidationResult validationResult) {

    validationResult = null;
    bool result =       DateTime.Compare((DateTime)property, DateTime.Now) < 0;

    if (!result)
      validationResult = new ValidationResult(context.DisplayName +
        " must be today or in the past.");
      return result;
  }
}
Notice the use of the SharedAttribute on the ExpenseReportValidation class. This signifies to the projection process that it doesn't need to be translated to the client because it will also be covered by being a part of shared code.

Wrap Up
In the old days you would develop expense report applications by wrapping CRUD operations around the expense report data. The new Silverlight 3 DataGrid, DataForm, DataPager, and ObjectDataSource let you create the UI rapidly without having to invest in infrastructure development or sacrifice functionality to employ built-in controls. In addition, using .NET RIA Services you can define server-side business logic complete with validation rules and data access, and have it be easily consumable thanks to the projection process.
My sample expense report still needs a report details section as well as navigation and authentication. To achieve this, I'll need to use some additional controls introduced in Silverlight 3 as well as a set of application services provided by .NET RIA Services. In a future article, I'll demonstrate how this works.

Jonathan Carter is a Technical Evangelist at Microsoft.


Cloud Computing
Patterns For High Availability, Scalability, And Computing Power With Windows Azure
Joshy Joseph
Code download available from the MSDN Code Gallery
Browse the Code Online

This article is based on a prerelease version of Windows Azure. Details are subject to change.

This article discusses:
  • Cloud computing architecture
  • Windows Azure
  • Cloud patterns and best practices
This article uses the following technologies:
Windows Azure
Contents
Cloud Computing
Windows Azure Implementation
Cloud Application Types and Patterns
A Real-World Sample
Application Design and Development
Deploying the Service
Administration
Conclusion
During the last decade, the decoupling of interfaces from implementation, scalable hosting models, service orientation, subscription-based computing, and increased social collaboration became the goals of distributed systems. Now, Internet-hosted distributed applications with connectivity to internal applications—often referred as Software plus Services (S+S)—are gaining popularity. Organizations are leveraging datacenters hosted by third parties to alleviate concerns about hardware, software, reliability, and scalability. These are just some of the new architecture trends that help you build interoperable applications that scale, reduce capital expenditure, and improve reliability. Cloud computing offers many of these benefits.
A cloud computing platform enables applications to be hosted in an Internet-accessible virtual environment that supplies the necessary hardware, software, network, and storage capacities and provides for security and reliability, removing much of the burden of purchasing and maintaining hardware and software in-house. In the cloud, you can develop, deploy, and manage applications as you have in the past and integrate these services to your on-premise applications. You pay only for the time, resources, and capacity you use while scaling up to accommodate the changing business needs.
In this article, I will examine the typical cloud platform architecture and some common architectural patterns, along with their implementation on the Windows Azure offering from Microsoft.

Cloud Computing
Figure 1 illustrates the typical cloud computing platform architecture.
Figure 1 Layered Architecture of a Cloud Platform
In this model, each layer abstracts the layer below it, exposing interfaces that layers above build upon. There is no hard dependency between layers and each layer provides composable or Plug and Play architecture with services from other layers. Each layer provides horizontal scalability as needed.
As you can see, a cloud platform is composed of a number of subsystems. Let's look at each one next.
A hosting platform The hosting platform provides the physical, virtual, and software assets. These assets include physical machines, operating systems, network systems, storage systems, power management, and virtualization software. Bare metal and other operational resources are abstracted as virtual resources to the layers above.
Cloud infrastructure services The most important function of this layer is to abstract the hosting platform as a set of virtual resources and manage those resources based on scalability and availability needs. Fundamentally, this layer provides three kinds of abstract resources: compute, storage, and network, and exposes a set of APIs to access and manage these resource abstractions. Thus you gain access to the underlying physical resources without knowing the details of the underlying hardware and software and can control these systems efficiently through configuration. Services offered by this subsystem are often known as Infrastructure as a Service (IaaS).
Cloud platform services Developing and managing software for cloud computing is complex. It becomes really complex when you integrate on-premise software with hosted services. Platform services provide a set of capabilities exposed as services to help with such integration. For example, in the Azure Services Platform, Microsoft .NET Service Bus helps with discovery and access while the Microsoft .NET Access Control Service helps role- and rule-based claims transformation and mapping. Availability of platform services may differentiate one cloud provider from another. Services provided by this layer are referred to as Platform as a Service (PaaS).
Cloud applications This layer houses applications that are built for cloud computing. These applications expose Web interfaces and Web Services for end users, enabling multitenant hosting models. Some functions include connecting disparate systems and leveraging cloud storage infrastructure to store documents. These services fall under the umbrella of Software as a Service (SaaS).
Security services Security services ensure token provisioning, identity federation, and claims transformation. These services are built on the open standards, WS-Security, WS-Trust, WS-Federation, SAML protocols, and OpenID, for greater interoperability.
Management services Management interfaces cut across all the layers described above. The hosting platform leverages management interfaces and agents for automated scalability and availability administration. Even though the cloud is hosted and managed in a datacenter, customers may need functions that allow them to easily control their application and post deployment configurations, get analytics about service usage, and connect their enterprise management systems.
Tools Tools help you build, test, and deploy applications into the cloud. These tools may be extensions to existing tools (Visual Studio Tools for Windows Azure, for instance) or hosted tools from a specific cloud provider.
Users and providers of cloud computing There are three categories of users in cloud computing: cloud platform providers, cloud consumers, and end users. Cloud platform providers provide the hosting platform and cloud infrastructure services. Cloud consumers utilize the cloud platform and develop applications and services to be consumed by end users. Cloud consumers configure applications for scalability, availability, and security needs. End users leverage the services offered by cloud consumers. These users could be human, organization, or machine and may be hosted anywhere.
In this context, Azure provides a cloud platform while cloud consumers (or ISVs building integrated cloud solutions or enterprises) leverage this platform to build applications. For example the Live Mesh data synchronization platform leverages the Azure Services Platform and Windows Azure to develop and host S+S services for end users.
Figure 2 maps the Azure Services platform to the layered architecture in Figure 1. This platform provides a set of services to application developers. These services can be used both by applications running in the cloud and by applications running on local systems. Windows Azure, operating system for the cloud, is the foundation of Microsoft's cloud platform offering. As shown in Figure 2, Azure Services Platform provides a set of shared services: SQL Data Services, .NET Services, and Live Services, which can be used individually or collectively. In addition, Microsoft offers various finished cloud applications including Exchange Online, SharePoint Online, and CRM Online. Here, however, I'll focus on only the Windows Azure operating system and related patterns.
Figure 2 Azure Services Platform Mapped to Layered Architecture

Windows Azure Implementation
Windows Azure provides on-demand compute and storage capabilities to host, scale, and manage Web applications and services on the Internet hosted in Microsoft datacenters.
Windows Azure provides features that consumers of cloud services require. For example, physical hardware resources are abstracted away and exposed as compute resources ready to be consumed by cloud applications. Physical storage is abstracted with storage resources and exposed through well-defined storage interfaces. A common Windows fabric abstracts the physical hardware and software platform and exposes virtualized compute and storage resources. In addition, each instance of the application is monitored for availability and scalability, and automatically managed. For instance, if an application in an instance goes down, the Fabric controller (shown in Figure 3) will be notified and another instance in another virtual machine (VM) will be instantiated with limited impact to end users. Because of the amount of virtualization, when writing your code, you need to make sure that you don't make any assumptions about the state of the machine hosting your application. In Windows Azure your service could easily be moved to a new VM. Windows Azure follows a model-driven service management design with Azure Fabric Controller responsible for mapping declarative service specifications to available resources and managing the lifecycle of the services.
Figure 3 Windows Azure and Roles
By default most of the applications developed in .NET could be hosted in Azure with some specific restrictions on partial trust models, data storage/access, and inter-application communication. A rich developer SDK and programming tools help you get oriented toward this platform.
Right now you can create .NET applications, ASP.NET applications, and WCF-based Web services, using tools supported in Visual Studio.
Windows Azure currently offers two processing methods, Web role and Worker role (see Figure 3). Each role executes on a separate VM and communicates with Azure Fabric through an agent. The agent collects resource metrics and node metrics including VM usage, application status, logs, resource usage, exceptions, and failure conditions. It should be noted that each VM may be executed on a single physical host or on a Windows 2008 hypervisor VM. The specific runtime host configuration is dictated by Windows Azure, depending on the service-level agreement and other business/technical needs. A Web role hosts interactive Web applications and provides in-bound and out-bound connections (request-response patterns). In-bound calls are routed through Azure-provided load balancers to provide high availability. As you have noticed, this mandates that each Web role instance be stateless so that fabric can route the requests to any Web role in the cluster.
The worker role is a specialized application executing a .NET application in the background. These applications don't have in-bound connectivity from external applications. However they can send messages to external services. While interacting with Web roles, Worker roles send/receive messages using the Windows Azure Queue storage service. Another interesting aspect is the scale-out capability of the roles. An application deployment administrator can state how many instances of any role may be needed in the configuration and the Fabric will decide on running these instances depending on the system scale-out needs. In short, Windows Azure provides always-on, on-demand deployment and failure handling.
So far I've covered general concepts around Cloud platforms and specific features of Windows Azure. Next I will introduce some core cloud application types, architecture patterns, and best practices that you should be familiar when developing applications for cloud platforms.

Cloud Application Types and Patterns
There are a number of architecture and design patterns and best practices that help you select a cloud platform and implement cloud services and applications. In general, these patterns fall into four categories: compute, storage, communication, and management.
Compute patterns Once you know which application type you are dealing with, you should select the appropriate compute pattern. As I mentioned, the Web role is used for building interactive application patterns, while the Worker role is suitable for building background and scheduler tasks. In some cases you may need both features. One important consideration when planning your compute tasks is to remember to execute those tasks in such a way as to avoid moving large amounts of data around.
Storage patterns Cloud storage provides remote storage and abstracts the storage medium away from the users. The design is sufficiently flexible to support a wide range of application requirements. Azure addresses two patterns of cloud storage: table storage and blob storage. The table storage pattern allows the applications to store key/value pairs following a table structure while the blob storage pattern can be used to store any data.
Communication patterns These patterns address message exchange. Azure technology leverages Windows Communication Foundation (WCF) and REST APIs for Web service communication. You must consider partial trust models and the stateless nature of the application while implementing communication patterns.
Administration patterns Administration patterns differentiate two core aspects of service management: service deployment and service-level-management. Deployment patterns organize service definition, configuration, and monitoring, while other patterns address service-level management and regular operational maintenance.
Now let's look at cloud application types in detail. I will classify cloud application types into three categories based on the types of scenarios each addresses. The first category is Web applications. These include traditional hosted Web applications, emerging composed applications that may utilize two or more data sources and services. These applications need automatic scale-out and scale-down capabilities. An application like Facebook is a good example. In such scenarios the organization may be a startup that wants to spend little capital on infrastructure while being able to handle increasing demand.
Next there are the analytical applications whose main function is to run processor-intensive operations and data mining, often over the same data many times and thus they require access to a great deal of storage capacity and processor availability all at once. There is no need to pay for such huge capacity twenty-four hours a day, seven days a week, however, so cloud services are appealing.
Finally, there are the parallel computing applications that need to perform multiple tasks in parallel so that a huge project can be executed in a short period of time. Again, paying for the one-time-only large capacity that cloud computing can provide is a cost-effective solution.
Not all applications are suitable for running on the cloud platform. The obvious limiting factors include data security, potential lock-in with a cloud provider, open interfaces for communications, trust model limitations, efficiency of moving data in and out of the cloud, integration with existing services outside of the cloud platform, and legal/privacy concerns. Figure 4 lists some scenarios that are suitable for cloud computing, and Figure 5 lists suitable patterns, scenarios, and features.
Figure 4 Application Types Suitable for Cloud Computing
Cloud Application Type | Context | Scenario Example
Web Applications | Host traditional Web apps and interactive apps that compose two or more data sources and services. | A start-up company creating a Web collaboration application expecting to spend little capital on infrastructure.
Parallel Computing | Large-scale parallel execution of compute tasks. Normally these tasks execute for a short period of time utilizing more compute and storage resources. | A newspaper decides on digitization of reports for better Web consumption.
Analytical Applications | Analytical processing executes various analytical and data mining algorithms over the same data again and again. | A financial company executing Monte Carlo simulations on financial data to assess risk periodically.
Figure 5 Patterns, Features, and Their Corresponding Scenarios
System Considerations | Pattern Categories | Context | Windows Azure Feature | Scenario Example
Compute | On-demand Application Instance | Applications that need scale-out and scale-down capabilities. | Automated management of Web and Worker Roles | Let retail store sites be available during special events.
 | Worker | Executing parallel batch jobs or background applications. | Leveraging multiple instances of Worker Roles to execute background tasks | Using schedulers for analytics processing by executing background tasks in parallel.
Storage | Blob Storage | Storing large amount of unstructured data. | Leveraging Azure Blob Storage | Company storing legal compliance reports in backup store.
 | Structured Storage | Storing data in a table structure while not demanding full relational semantics. | Azure Table Storage | A structured storage to maintain a Web application's state—i.e. storing purchase order or shopping basket information.
Communication | Service Interface (Web and Web service API ) | Exposing application capabilities through UI and Web services. | Azure support for Building applications using ASP .NET, Silverlight, WCF Web Services | Company building digital asset management solution exposing APIs to other services.
 | Service-oriented integration | Invoking external Web services using Web-standard protocols. | Azure platform supports WCF clients and REST APIs | Web applications leveraging Microsoft-hosted live meeting services for collaboration.
 | Messaging | Share messages between applications in a scalable, reliable, and asynchronous way. | Leveraging Azure Queue storage service for Web to Worker role communication | Web application informing a scheduler to execute a specific task.
Administration | Cloud Deployment | Deploying applications with desired configurations such as scale-out and high-availability requirements. | Separation of service definition, services configuration, and packaging to help appropriate roles | A retail store configuring Web portal to automatically scale-out when usage exceeds threshold and scale-down as needed.
 | Design for Operations | How to make my application operations-ready by providing health status and logging. | Using Windows Azure RoleManger.WriteToLog API and overriding RoleEntrypoint.GetHealthStatus() in the Worker Role | Developers designing cloud applications to be operations friendly through instrumentation.
 | Service Instance Management | Start, stop, and suspend cloud apps. Manage service configurations. | Automatic handling of dynamic configuration changes and failure conditions | A Web application administrator using the Azure portal to manage application state.
 | Management Alerts | Sending Instant Messages, e-mail, or alerts about resource and billing info. | Provided through Live integration | Enabling applications to send IMs. Default notification on resource usage.
 | Service Level Management | Get info on app resource consumption such as processor time, bandwidth. | Automated service management via model-based approach | ISV looking for billing and resource usage info about the application deployed.
In Windows Azure, Web roles are suitable for executing interactive Web applications while Worker roles are suitable for background applications. Data is stored in Azure blob and table storage services. Inter-role communication is enabled through a Queue service, which provides an asynchronous message exchange. The platform supports WCF-based Web services and clients with Basic HTTP and WS* binding. Storage services are exposed through REST-style interfaces for easier consumption by other applications and cloud providers.
As you know, patterns offer flexible solutions to common problems in software development. Next I will identify common pattern categories corresponding to the above application types.
Rather than trying to describe each individual pattern in depth, I'll focus on broad problem areas and associated pattern categories and show how these patterns are implemented in the context of Azure development.

A Real-World Sample
Let me introduce a scenario that will help illustrate some of these concepts. Fabrikam, an ISV, would like to build a hosted digital asset management Web application. This application enables end users to store digital images in the cloud. Users must be able to preview the pictures submitted, tag them, and annotate them with location information. The application must allow users to use their own desktop applications to access these pictures directly from the storage. Figure 6 represents a high-level architectural view of the services exposed by Fabrikam. As shown in the diagram, Fabrikam exposes a Web application and Web service to the end users for managing their digital assets. These services store and retrieve digital assets from the asset store implemented on top of Windows Azure blob services. Fabrikam implements a background processor to execute tasks such as tagging, thumbnail generation, and so on.
Figure 6 Fabrikam Hosted in Windows Azure
Let's see how to design, develop, and deploy such an application to the Azure Services Platform. The discussions that follow demonstrate core concepts and features available in Windows Azure. You can download the code and documentation to see how the entire scenario is implemented and hosted in Windows Azure.

Application Design and Development
The first step is to understand the applications scenarios, select appropriate patterns, best practices, and programming models. During this process you will identify the type of the application and workloads targeting Windows Azure. In addition, you will figure out the inter-application messaging and data storage patterns. Here I'll assume that Fabrikam is building an interactive Web application with scale-out needs and a background processer to generate metadata from digital assets. The above scenario needs Azure Web and Worker roles, respectively. In addition, Fabrikam needs Blob storage to store digital assets and metadata, service interfaces to expose assets, and messaging patterns for communication.
To facilitate development, the Windows Azure SDK comes with a development environment that simulates Windows Azure on the local machine, thereby enabling offline development and debugging. The SDK supports Web and Worker roles, storage, Visual Studio templates, tools for packaging, viewing and managing role instances, and a Visual Studio add-in that enables the F5 experience. You can download the SDK from MS Download center.
To begin, use the Windows Azure Tools for Microsoft Visual Studio to create a new project using the Cloud Service project templates. You can add necessary user interface, application logic, and input processing to the Web application. Alternatively, you could start from an existing ASP.NET Web application. Because the application will execute under partial trust, avoid calling APIs that violate this restriction. For details about this restriction, please, see the Windows Azure SDK Trust Policy Reference.
Next, import Microsoft.ServiceHosting.ServiceRuntime assembly and use the respective APIs to enable logging, reading local environment configuration, and reading application configuration sections from the service definition file, as shown below.
// Writing log information
 RoleManager.WriteLog ("Information", "message to show in the log file");
 // Getting service configuration information
 RoleManager.GetConfigurationSetting("MyConfigSection");

// Getting local environment; code snippet below shows getting a pointer
//to a local file
ILocalResource resource = RoleManager.GetLocalResource(    "ComputeLocalStorage")
By the way, it is a programming best practice to replace the above logging and resource access code with reusable application blocks (Logging and Configuration blocks) such as Microsoft Enterprise Library application blocks. You can implement specific providers for Windows Azure logging and configuration management and plug them into the runtime. These best practices decouple applications from platform-specific APIs and help increased reusability between on-premise and cloud environment.
Storage needs for your application will need to be rewritten to take advantage of the Azure storage patterns supported. I will address this shortly.
Next, add a Worker role project into your project and then open WorkRole.cs. This file includes a class called WorkerRole that derives from the RoleEntryPoint class. This class provides methods to manage the initialization, starting, and stopping of a Worker role service, as well as for monitoring the health of the service.
In this step you normally override two methods, as shown in Figure 7.
Figure 7 WorkerRole
public class WorkerRole : RoleEntryPoint
{
  public override void Start()
  {
      RoleManager.WriteToLog("Information", "Generating metadata from digital assets");

      while (true)
      {
            RoleManager.WriteToLog("Information", "background task is executing thumbnail extraction");
            Thread.Sleep (1000);
      }
  }

  public override RoleStatus GetHealthStatus()
  {
            return RoleStatus.Healthy;
  }
}
Now run the application in the local development fabric by hitting F5. You will see logs written in the Web and Worker Roles console instances.
Now let's configure a blob store and access it from the Web role.
First you need to set up the blob storage account information.You'll need to specify the following parameters:
AccountName: Specify the name of your Windows Azure account.
AccountSharedKey: Specify the key used to authenticate a request made against Windows Azure storage. To authenticate a request, you must sign the request with the key for the account that is making the request.
BlobStorageEndpoint: Specifies the base URI of the blob storage service.
ContainerName: Specifies the name of the blob container used to store images for this application.
For example, take a look at the following configuration settings:
<ConfigurationSettings>
   <Setting name="AccountName" value="fabrikamAccount" />
   <Setting name="AccountSharedKey" value="Eby111M02xNOcqFlqUwJPLlmEtlCD
   XJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBekso45Gw==" />
   <Setting name="BlobStorageEndpoint" value="http://127.0.0.1:10000"/>
   <Setting name="ContainerName" value="fabrikamImagegallery"/>
  </ConfigurationSettings>
The REST API for Blob Storage exposes two resources: containers and blobs. A container is a set of blobs; every blob must belong to a container. Blobs are written using PUT operations as defined by the REST API. Using these APIs you can create a hierarchical namespace to organize assets such as fabrikamImagegallery/images and fabrikamImagegallery/documents, where fabrikamImagegallery is the name of the container and images refers to the blob name. It should be noted that each blob could be constructed from blocks of a maximum of 4MB in size for efficient management.
Working with the REST API is rather cumbersome and hence the SDK ships with a sample library that abstracts the REST calls and exposes well known access patterns through the BlobContainer and StorageAccountInfo classes.
Figure 8 shows how to work with Blob storage.
Figure 8 Using Blob Storage
BlobStorage blobStorage = BlobStorage.Create(StorageAccountInfo.GetDefaultBlobStorageAccountFromConfiguration());

BlobContainer newContainer = blobStorage.GetBlobContainer("ContainerName");

newContainer.CreateContainer(null, ContainerAccessControl.Public);

this.container.ListBlobs(String.Empty, false);

BlobProperties properties = new
 BlobProperties(string.Format(CultureInfo.InvariantCulture, "image_{0}", "idenitifier1"));

NameValueCollection metadata = new NameValueCollection();
metadata["Id"] = id;
metadata["Filename"] = "filename";
metadata["Tags"] = "Image";

properties.Metadata = metadata;

this.container.CreateBlob(properties, imageBlob, true);
Now, let's create a Queue storage service to send message from a Web role to a Worker role. The Windows Azure Queue service provides reliable, persistent messaging within and between services. The REST API for the Queue service exposes two resources: queues and messages. It is possible to create an unlimited number of queues each identified using a unique name. The maximum size of the message is restricted to 8KB. You are responsible for deleting the message after reading; otherwise the message will remain in the queue for the specified time.
Figure 9 shows how to access Queue storage and messages in a Worker Role.
Figure 9 The Queue Storage Operations
public class WorkerRole : RoleEntryPoint
{
        public override void Start()
        {
            Uri baseUri = new
                 Uri(RoleManager.GetConfigurationSetting("QueueEndpoint"));
            string accountName = RoleManager.GetConfigurationSetting("AccountName");
            string accountKey =
                RoleManager.GetConfigurationSetting("AccountSharedKey");

           StorageAccountInfo account = new StorageAccountInfo(
           baseUri,
                null,
                accountName,
                accountKey);

            QueueStorage service = QueueStorage.Create(account);
            MessageQueue queue = service.GetQueue("fabrikamimageprocessingqueue");

            while (true)
            {
                Thread.Sleep(10000);
                if (queue.DoesQueueExist())
                {
                   Message msg = queue.GetMessage();
                   if (msg != null)
                   {

                      queue.DeleteMessage(msg);
                   }
                }
            }
        }
….
}
Now you are ready to try out this application in the local development fabric by hitting F5. You will see Web Roles receiving inputs and sending messages to Worker Roles through Queues. By now you have completed all the necessary development activities needed and are ready to publish the application to Azure.

Deploying the Service
Deploying services to Windows Azure is relatively simple. However, you first need to decide what information to include in the service definition file (application configuration information) versus service configuration file (environment requirements), how to package the application, whether you can leverage existing tools to create and upload packages, and how to store your packages in the cloud storage and later deploy to Azure.
Here's how I will address these requirements for the sample application.
Fabrikam ISV has developer, deployment, and operational roles to manage its cloud applications. Developers create a service definition file that contains information about the application. The service definition file (.csdef) defines the roles available to a service, specifies the service endpoints, and establishes configuration parameters for the service. The service definition settings cannot be changed after a service is deployed. This file becomes part of the Service package file during the packaging process.
Developers also create a service configuration file (specifying the number of instances needed and Azure account information) for local debugging and Azure staging deployment. The service configuration file (.cscfg) specifies values for the configuration settings for one or more role instances within the running service. Operational staff can dynamically modify these service configuration settings without redeploying the service. The service configuration file is not packaged with the service, but is uploaded to the Windows Azure Fabric as a separate file.
The deployment staff updates the services configuration file before production deployment with Azure account-specific information and operational requirements (number of production instances needed).
Developers or the deployment staff create Azure deployment packages using Visual Studio tools or the CSPack tool that comes with the Azure SDK.
It is possible to store deployment packages in the Azure blob storage and retrieve while deploying to the Azure cloud platform
The service package file (.cspkg) is a zipped file that contains role definitions and all role related files (binaries, images and config) packaged into one artifact. You could generate these packages by using Visual Studio or using the CSPack tool. When you use Visual Studio to run on the development fabric the package created will have a .csx extension. However when you use the "Publish" feature, it creates a .cspkg file, which is a zipped and encrypted version of the .csx file. This encrypted and zipped version with .cspkg extension is uploaded to the cloud.
Now you can upload your service configuration file (.cscfg) and package file (.cspkg) to Windows Azure. For this, first you need to obtain Windows Azure compute and storage accounts from the Azure Services Developer Portal. After obtaining these accounts you can deploy the package to the cloud though the Azure developer portal. Once files are uploaded, you'll be provided with an internal staging URL that you can use to test your service in the Windows Azure Fabric. When you move into production you will get the production URL.

Administration
Administration functions cover the management, governance, and operational aspects of the applications deployed in a cloud environment. Leveraging the cloud platforms for application hosting assumes that the services deployed are in a controlled environment with appropriate service-level management, resource management, service provisioning, security/trust models, and monitoring. You should be familiar with the key functional capabilities needed for service management. They include:
  • Developing services with operations-friendly implementation best practices by including proper instrumentation.
  • Monitoring the health and availability of cloud applications and services.
  • Collecting metrics and reporting on service usage, performance, and billing.
  • Enabling automated provisioning of services and updating service configurations.
In Windows Azure, agents monitor each VM instance for failure conditions and collect metrics regarding failures, performance measures, and usage metrics.
Azure Fabric automates the provisioning and management aspect with limited human input. People and process are error prone. This automation helps avoid human errors. In Windows Azure, applications run in a partial-trust sandbox, requests are load balanced, and failure conditions are automatically managed.
As a developer you should be familiar with the operational behaviors of your service. These behaviors are documented as part of service definition and configuration models and communicated to the cloud platform to facilitate automation.
In addition, follow these best practices:
Adding logging information in the code
RoleManager.WriteLog ("Information", "message to show in the log file");
  • Overiding GetHeatlthStatus method
public override RoleStatus GetHealthStatus()
{
      // return the health status of worker role.
      return RoleStatus.Healthy;
 }
  • Getting Log files for analyzing application states
Windows Azure Service Management
In Windows Azure, service management follows a model-driven approach where models are used to collect desired configuration information from developers and deployers. Azure Fabric uses these models to manage the service lifecycle, including dynamic configuration updates, automated failure handling, and service monitoring. Windows Azure avoids single point of failures (hardware or software) by introducing fault domains where each role is isolated across multiple compute nodes or machine racks. In addition, Windows Azure ensures services are up and running while updating (rolling forward or backward a subset of service) by requesting upgrade domains though configurations.
You should use the "configure" option and "Copy Log" option in the Azure portal to copy the log messages to the blob storage.
Once the copy is completed you could access the logs from blob storage using the programming model described earlier.
In the future Windows Azure will expose management interfaces to the access and control management and operational activities. This helps on-premises service management tools to monitor and manage cloud applications.

Conclusion
Understanding the cloud computing architecture models, finding relevant patterns, deciding on the programming model, and enabling automated management are critical to the success of cloud computing evolution. Windows Azure is designed with developers in mind by enabling developers to quickly and easily create, deploy, manage, and distribute Web applications and services.
As a developer, you should accumulate a core set of patterns that will help you develop cloud applications most effectively. In addition, you should remember to capture and share recurring solutions to problems and other emerging patterns such that our solutions and products can provide better out-of-the-box support for core cloud computing patterns.
I would like to thank Steve Marx, Jason Hogg, David Hill, Fred Chong, Eugenio Pace, and Nataraja Koduru for their valuable feedback on this article.

Joshy Joseph is a principal architect with Microsoft Services Managed Solutions Group. His primary skills and expertise include distributed computing, grid computing, and Web services. He is the author of the book Grid Computing (Prentice Hall, 2004) and a prolific inventor, with more than 35 patents on file. In addition, he has written numerous technical articles on distributed computing and business process development. Joshy can be reached at jojoseph@microsoft.com.


Data Services
Access Your Data On Premise Or In The Cloud With ADO.NET Data Services
Elisa Flasko
Code download available from the MSDN Code Gallery
Browse the Code Online

This article is based on prerelease versions of Windows Azure and ASP.NET MVC.

This article discusses:
  • Windows Azure
  • ASP.NET MVC
  • Windows Azure Table
  • On-Premise vs Cloud app development
This article uses the following technologies:
Windows Azure, ASP.NET MVC
Contents
The Application
Consuming an On-Premises Data Service with the ADO.NET Data Services Client Library
Querying Data
Create, Update, and Delete
Consuming an Azure Table Data Service
The Azure Table Data Model
The ADO.NET Data Services Client Library
Defining the Model on the Client
Creating a Table
Querying Data
Create, Update, and Delete
Summing Up
Web Services has been a hot topic for a number of years now, but what, you may wonder, are these Data Services everyone's talking about lately? As the architecture of Web applications has changed and matured (with the popularity of Rich Internet Applications [RIAs], for example), there has been an increased awareness of the value of exposing raw data, minus any interface or formatting, to any service or application that wants to consume it. Data Services are simple data-centric services with a RESTful interface, that do exactly that—expose pure data for consumption by applications. Such services expose data using a uniform interface to be consumed by Web clients across a corporate intranet or the Internet. Data Services can be built and hosted on your premises using the ADO.NET Data Services Server Library to expose data securely as a service, or may be made available by various hosted services such as Windows Azure Table. Windows Azure is the foundation of the Microsoft Cloud Platform. As the "operating system for the cloud" it provides the essential building blocks you need to write scalable and highly available services. A major part of Azure, Windows Azure Table, allows users to persist data in the cloud and expose the data it stores as a Data Service.
Regardless of where the data service is hosted, the ADO.NET Data Services framework provides a set of Client Libraries built for a range of Microsoft Development platforms (.NET, Silverlight, ASP.NET AJAX, and so on) that allow you to easily consume these services in your applications.

Figure 1 Entity Data Model Exposed by the Micro-Blogging Data Service
The Application
Throughout this article, I will be building a simple Micro-Blogging application using ASP.NET Model View Controller (MVC) (release candidate) and will detail some of the typical scenarios that arise when building such services-based applications. I'll assume a prior understanding of MVC and will therefore not discuss the specifics of this pattern. If you need more information about ASP.NET MVC, see the ASP.NET MCV site. For the purpose of comparison, I will look at two versions of the same basic application—one consuming an on-premises Data Service, and one consuming an Azure Table Data Service. The code for both versions is included in the accompanying code download.
The Micro-Blogging application allows users to post messages to specific channels, create new channels, and view messages on the channels of interest to them. The model that has been exposed by the Data Service for this application consists of three types of entities: Messages, Channels, and Users, with a largely one-to-one mapping to the database. It is worth noting at this point, that with an on-premises data source, this model does contain a first-class notion of relationships between entities. Messages are posted to a particular Channel by an individual User (see Figure 1).

Consuming an On-Premises Data Service with the ADO.NET Data Services Client Library
Let's begin by creating a new ASP.NET MVC Web Application using the Visual Studio template installed with ASP.NET MVC.
There are two primary ways that you could go about defining the classes to represent the model on the client. You could simply define the model by defining the POCO (Plain Old CLR Objects) classes that will be used on the client to represent the entities exposed by the service, which I will do when I develop this application against Windows Azure Table, or you could use the Add Service Reference Wizard in Visual Studio to generate the necessary classes. As when you use Add Service Reference for a Windows Communication Foundation (WCF) service, you simply right-click on the Project and choose Add Service Reference. In the Add Service Reference dialog box, enter the URI for the entry point of the service as the Address, and then click OK. This will generate the associated classes, based on the data service definition, and add them to the project.
As seen in Figure 2, the Add Service Reference Wizard has generated a class for each of my Entities: Channel, Message, and User, as well as a class representing the service as a whole—microblogEntities. The class microblogEntities inherits from DataServiceContext and will be used as the starting point for access to the data service.
Figure 2 Classes Representing Micro-Blogging Model

Querying Data
With the app now set up to access the data service, I can start developing the remainder of my application. On the homepage, the application will provide a listing of the available Channels and allow the user to view today's messages by selecting a Channel. The user will also be able to filter within that Channel based on Author. To create the homepage, I will need to create an ActionResult in the Home controller that will map to the Index view (the Home controller and Index view are created by default in the MVC template). On the homepage, I will need to retrieve a listing of all available Channels from the data service, as well query the service for today's messages, filtering based on user input (see Figure 3).
Figure 3 Site HomePage
The ADO.NET Data Service Client Library allows you to easily query data with the use of language integrated query (LINQ), as you see in Figure 4.
The Index action of the homepage controller is called by the MVC application when the homepage is accessed by the browser and returns the data that will be used to render the Index.aspx view. In Figure 4, inside the Index action, I instantiate a new instance of the DataServiceContext microblogEntities, like so:
microblogEntities svc = new microblogEntities(new Uri("http://localhost:50396/MicroData.svc"));
Figure 4 Accessing Data as .NET Objects Using LINQ
public ActionResult Index(string channel, string author)
    {
        microblogEntities svc = new microblogEntities(new
        Uri("http://localhost:50396/MicroData.svc"));
        ViewData["channels"] = svc.Channels;

        int y = DateTime.Now.Year;
        int mm = DateTime.Now.Month;
        int d = DateTime.Now.Day;

        var q = from m in svc.Messages.Expand("Author").Expand("Channel")
                where m.Channel.ChannelName == channel && (m.DatePosted.Year == y &&
                m.DatePosted.Month == mm && m.DatePosted.Day == d)
                select m;

        if (!string.IsNullOrEmpty(author))
        {
            q = from m in q
                where m.Author.UserName == author
                select m;
        }

        ViewData["msg"] = q;

        return View();
    }
This is the object that I will use every time I want to hit the service. The first query that will be executed against the Data Service accesses all of the Channels available. The second, a LINQ query, retrieves all of the Messages in the channel that was requested when the user hit the page and filters by author if a name was provided. Note that in the provided sample code, messages are viewed by entering the Channel Name and, if desired, the Author name into the text box and then clicking View. It is also worth noting that the queries require a ChannelName; and a search cannot be executed by author name alone. I place both of the aforementioned queries into ViewData to be executed as the page is rendered and then return.
Now, in the view for the homepage, Index.aspx, I will actually enumerate over the queries, effectively executing the query against the store at this point, and will then print the results to the page (see Figure 5).
Figure 5 Homepage View (On-Premises Service)
<h2>Today's Messages</h2>
    <form action="./">
        Channel:<span style="color: #FF0000">*</span> <input type="text"
            name="channel"
            />&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
            Author: <input type="text" name="author" /><br />
        <input type="submit" value="Search" />
    </form>
    <ul>
    <!-- Execute query placed in ViewData["msg"] - enumerates over
        results and prints resulting message data to the screen -->
    <% foreach (var m in (Ienumerable
        <MicroBloggingOnPremise.Models.Message>)
    ViewData["msg"])
       { %>
    <li><%=m.Author.UserName%>@<%=m.DatePosted %>: <%=m.Text%>
        [<%=m.Channel.ChannelName %>]</li>
    <%} %>
    </ul>

    <h2>Channels</h2>
    <ul>
    <!-- Execute query placed in ViewData["channels"] - enumerates over
         results and prints resulting channel data to the screen -->
        <% foreach (var channel in
        (IEnumerable<MicroBloggingOnPremise.Models.Channel>)
            ViewData["channels"])
           { %>

          <li><%=channel.ChannelName %> (<%=channel.DateCreated %>)</li>

        <% } %>
    </ul>

Create, Update, and Delete
To create a new entity instance in the data service using the ADO.NET Data Service Client Library, I create a new instance of the .NET object that represents the entity set and call the AddTo… method for the Entity Set on the DataServiceContext instance that I'm using, passing the new object and the entity set that I am adding it to.
For example, let's look at the PostMessage action of the homepage controller, in Figure 6, which is called when I attempt to post a message by clicking the Send button.
Figure 6 (On-Premises) Create a New Message and Push to Data Store
[AcceptVerbs("POST")]
        public ActionResult PostMessage(string channel, string author,
            string msg)
        {
            microblogEntities svc = new microblogEntities(new
            Uri("http://localhost:50396/MicroData.svc"));

            Channel chan = (from c in svc.Channels
                    where c.ChannelName == channel
                    select c).FirstOrDefault();

            if (chan == null) throw new ArgumentException("Invalid
                channel");

            User u = (from auth in svc.Users
                     where auth.UserName == author
                     select auth).FirstOrDefault();

            if (u == null) //throw new ArgumentException("Invalid
                Author");
            {
                //To simplify this example we will create a new user,
                //if the user was null when we queried.
                //It is possible that another client creates
                //the same UserName
                //before we call SaveChanges()
                //and we end up with 2 Users with the same name
                var user = new User();
                user.UserName = author;
                user.DateCreated = DateTime.UtcNow;
                svc.AddToUsers(user);
                u = user;

            }

            var m = new Message();

            m.DatePosted = DateTime.UtcNow;
            m.Text = msg;
            svc.AddToMessages(m);
            svc.SetLink(m, "Channel", chan);
            svc.SetLink(m, "Author", u);

            try
            {
                svc.SaveChanges();
            }
            catch (Exception e)
            {
                throw (e);
            }
            return this.RedirectToAction("Index");
        }
Notice that the method is decorated with the AcceptVerbs("POST") attribute, since we're using it for a form post. Again I start by instantiating a new DataServiceContext as the entry point to my service and then I query for the specified channel to check that it exists. If the channel does not exist, I throw an exception; otherwise, I create a new Message passing in the data entered by the user. Once the Message has been created, I need to tell the context that I have a new object to be added by calling svc.AddToMessages(m); passing in the new Message object, m. I also set links representing the relationships attached to this new message, in this case svc.SetLink(m, "Channel", chan); associating the message with the specified Channel and svc.SetLink(m, "Author", u); associating the message with the specified Author. Finally, I want to push the changes to the service and into the database by calling svc.SaveChanges();.
If I were modifying an existing instance rather than creating a new one, I would first query for the object that I want to update, make the desired changes, and rather than AddToMessages, I would call the UpdateObject method to indicate to the context that it needs to send an update for that object the next time I call SaveChanges(). Similarly, if I were deleting an instance, once I had the object, I would call the Delete method to indicate to the context that I want to delete that object.
In this example, and so far in this article, you have seen a single operation on the client map to a single HTTP request being sent to the service. In many scenarios, it is also beneficial for the client to be able to batch a group of operations and send them to the service in a single HTTP request. This reduces the number of round trips to the service and enables a logical scope of atomicity for sets of operations. The ADO.NET Data Service Client Library supports sending a group of Insert, Update, and/or Delete operations to a service in a single HTTP request via a call to the SaveChanges() method, passing SaveChangesOptions.Batch as the only parameter. This parameter value instructs the client to batch all pending changes into a single HTTP request sent to the service. By sending changes as a batch, you ensure that either all the changes will complete successfully or none of the changes will be applied.

Consuming an Azure Table Data Service
When you are developing a .NET application using Windows Azure Table, the data is accessed using the ADO.NET Data Services client libraries. In this section, I will walk through the development of the same simple application used above, but this time I will be accessing and manipulating data stored in Azure Table. For the remainder of this article, I assume that a Windows Azure Storage account has already been set up and is available for your use. For more information on getting started with Windows Azure Table, see the Azure Services Platform Developer Center.

The Azure Table Data Model
Before getting started on an application, there are a couple things that you must know about the Azure Table Data Model for best performance. An application may choose to create many tables within a storage account where each table contains a set of Entities (rows) and each Entity contains a set of Properties (columns). Each Entity is able to have up to 255 properties, stored as <Name, TypedValue> pairs and is identified using two key properties that together become the unique ID. This first key property is the PartitionKey (type string), enabling partitioning for scalability, and the other is the RowKey (also type string), uniquely identifying the entity within the partition. Beyond the two key properties, there is no fixed schema for the remaining properties, meaning that the application is responsible for enforcing any schema that must be maintained.
One of the important differences when developing against Windows Azure Table compared to local database storage surrounds partitioning. With Azure Table, the application controls the granularity of partitions, and partitioning becomes very important to both the performance and scalability of the application. This means that the choice of the PartitionKey property is very important, and impacts performance because the table clusters entities (Entity Locality) with the same partition key value. To choose the best PartitionKey possible, you should choose the key such that it is common to the majority (or all) of the queries used in the application. Performance and scalability can also be impacted similarly by the number of partitions used by the application (number of different values in the PartitionKey); having more partitions allows the server to better balance the load across nodes.

The ADO.NET Data Services Client Library
Now I will build the same Micro-Blogging application, using ASP.NET Mode View Controller (MVC), and this time I will detail some of the questions that typically arise when building applications against data stored in Windows Azure Table.
Similar to my on-premises database, this will require three Entities in my model, which I will translate into three Tables: a Messages table, a Channels table, and a Users table, all set up in my storage account.
Throughout the application I will make use of the StorageClient sample that ships within the Windows Azure SDK. The StorageClient sample implements a library you can use to access the Windows Azure blob storage service, queue service, and the table storage service. I will use the library to help me access the table storage service. The Windows Azure SDK can be downloaded from the Azure Services Platform Developer Center cited earlier.
To begin, I will create a new ASP.NET MVC Web Application using the Visual Studio template installed with ASP.NET MVC, and add the StorageClient project from the Windows Azure SDK to the solution.

Defining the Model on the Client
The first step in building the application is to define the classes that will be used on the client to represent the entities exposed by the service. Since Azure Table does not have a fixed schema, I cannot use the Add Service Reference Wizard as I did with the on-premises service. However, as I mentioned above, I can create my own POCO classes. These classes will effectively be used to define the schema for the application, as Azure Table does not enforce any schema on the server side. I will define a Message class, a Channel class, and a User class to align with the EntityTypes exposed by the service. To begin, I add a new code file, MicroData.cs, to the Models folder where I will define my classes. These classes will be defined in the MicroBloggingAzure.Models namespace.
If you take a look at the Messages class shown in Figure 7, you can see that I have defined both a PartitionKey and a RowKey, as required by Azure Table. This is a departure from what you saw in the on-premises service because they are specific to Windows Azure Table. It is worth noting that the best practices for key determination are not recommended across Relational and Cloud storage, and in my sample applications I have used the most appropriate key determination for each data source rather than trying to align the two in any way. The PartitionKey and RowKey are simply defined as type string and identified using naming conventions (that is, calling them PartitionKey and RowKey). In the case of this application, the PartitionKey is made up of the name of the Channel to which the message is posted, concatenated with an "@" and the date the message was posted in the format "yyyy-mm-dd". The RowKey is a string representation of the date that the message was posted. This PartitionKey was chosen such that the application can efficiently retrieve messages that were recently posted to a particular channel. To begin with, I have chosen to show only today's posts when a channel is accessed; however, if my application becomes very popular and receives thousands of posts a day, this key can be easily modified to partition more granularly— for example, by adding in a time component (hour perhaps). Similarly, if my application is not as busy as anticipated, I could modify the PartitionKey to partition by month rather than by day. I have then identified the two key properties as a uniquely identifying DataServiceKey, which is done by adding the DataServiceKey attribute to the class. The Messages class also defines a number of other properties containing information about the message, a part of the required keys. All of the properties associated with the Message entity will be stored in Azure Table. The User and Channel classes are created following the same principles as the Message class.
Figure 7 Message Entity Definition
[DataServiceKey("PartitionKey", "RowKey")]
    public class Message
    {
        // ParitionKey is channel name [+ day]
        public string PartitionKey { get; set; }
        // Date/Time when the message was created
        public string RowKey { get; set; }

        public string Text { get; set; }
        public string Author { get; set; }
        public string Channel { get; set; }
        public DateTime DatePosted { get; set; }
        public int Rating { get; set; }

        public Message()
        {
        }

        public Message(string channel, string author, string text)
        {
            this.Text = text;
            this.Author = author;
            this.Channel = channel;
            this.DatePosted = DateTime.UtcNow;

            this.PartitionKey = MakeKey(this.Channel, this.DatePosted);
            this.RowKey = System.Xml.XmlConvert.ToString(
               this.DatePosted);
        }
        public static string MakeKey(string channel, DateTime dt)
        {
            return channel + "@" + dt.ToString("yyyy-MM-dd");
        }
    }

Creating a Table
Once my entity classes have been defined, I can begin really developing the application. But first, I need to add my Azure Table Account access information to Web.config, as seen here:
<appSettings>
  <add key="TableStorageEndpoint"        value="http://table.core.windows.net/"/>
  <add key="AccountName" value="<MyAccountName>"/>
  <add key="AccountSharedKey" value="<MyAccountSharedKey-Provided when        you register for Azure Tables>"/>
</appSettings>
Although this is not the only way of passing account information when developing against Azure Table, this will allow me to use the Azure SDK to automatically pick up my account settings and deal with connections to the data source.
Often in applications written against Windows Azure Table, the creation of tables required by the app is done programmatically within the application. This is a little different from how one would normally do things with an on-premises data store, where the tables are created separately in the database and the application simply interacts with them.
One way of creating the required tables programmatically would be to simply create a new Entity in the master table called Tables. Every table that you create in your storage account must be represented as an entity in the master table, which is predefined when you register your account. For this application, however, I will use the StorageClient, which provides functionality to check, upon application initialization, that the required Tables exist in the storage account, and if they do not exist, will create them.
By creating the tables programmatically within the application, I simplify the process by which I can reflect updates to the model in the data store. For example, if down the road I add the ability for a user to subscribe to a channel, and add a subscription entity to the model, the associated table will automatically be added to the data store the next time the application starts. This choice also ensures that if the app is distributed for use with different Azure Tables accounts, all the user needs to do is to enter his or her account access information and the tables are correctly set up and ready for use when the application is run.
If you have existing data already in Azure Table, or if you do not have control over the tables in Azure, you may skip the step of creating tables altogether and simply program against your existing tables.
To check and create the tables in my application, I will use the TableStorage.CreateTablesFromModel() method that is a part of the StorageClient library, as seen here:
protected void Application_Start()
{
       RegisterRoutes(RouteTable.Routes); //MVC Routing Rules

       TableStorage.CreateTablesFromModel (typeof(Models.MicroData));
}

Querying Data
With my app set up to access my account in Azure Tables, the remainder of it works just as if I were accessing an on-premises service. Just as with my on-premises service, since I'm using the ADO.NET Data Service Client Library, I am able to easily work with my data as .NET objects and to query it using LINQ (see Figure 8).
Figure 8 Accessing Data as .NET Objects Using LINQ Again
public ActionResult Index(string channel, string author)
        {
            var svc = new Models.MicroData();
            ViewData["channels"] = svc.Channels;

            var q = from m in svc.Messages
                    where m.PartitionKey == Models.Message.
                       MakeKey(channel, DateTime.UtcNow)
                    select m;
            if (!string.IsNullOrEmpty(author))
            {
                q = from m in q
                    where m.Author == author
                    select m;
            }

            ViewData["msg"] = q;

            return View();
        }
In Figure 8, you see the Index action of the homepage controller. Inside this action, I instantiate a new instance of my DataServiceContext (MicroData inherits from TableStorageDataServiceContext), the object I use to access the service, and write the queries to be executed against the Data Service. I then retrieve a listing of all available Channels from the data service and write the LINQ query to retrieve all of Today's Messages, filtering based on user input (Channel and possibly Author if provided). I place both queries into ViewData to be executed as the page is rendered and then return. Notice that the pattern here is the same as when I accessed the on-premises data source; I instantiate a new DataServiceContext instance and query using simple LINQ queries against my model.
In the view for my homepage (Index.aspx in Figure 9), I enumerate over my queries, executing the queries against the store, and print the results to the page.
Figure 9 Homepage View
<h2>Today's Messages</h2>
    <form action="./">
        Channel:<span style="color: #FF0000">*</span> <input type="text"
        name="channel" /><br />
        Author: <input type="text" name="author" /><br />
        <input type="submit" value="View" />
    </form>
    <ul>
    <!-- Execute query placed in ViewData["msg"] - enumerates over
         results and prints resulting message data to the screen -->
    <% foreach (var m in (IEnumerable<MicroBloggingAzure.Models.Message>)
       ViewData["msg"])
       { %>
    <li><%=m.Author%>@<%=m.DatePosted %>: <%=m.Text%> [<%=m.Channel %>]
    </li>
    <%} %>
    </ul>

   <h2>Channels</h2>

    <ul>
    <!-- Execute query placed in ViewData["channels"] - enumerates over
    Results and prints resulting channel data to the screen -->
        <% foreach (var channel in
        (IEnumerable<MicroBloggingAzure.Models.Channel>)
        ViewData["channels"])
        { %>

          <li><%=channel.PartitionKey %> (<%=channel.DateCreated %>)</li>

        <% } %>
       </ul>

Create, Update, and Delete
Creating a new entity in the table service using the ADO.NET Data Service Client Library, just as if I were working against an on-premises store, is done by creating a new instance of the .NET object that represents the entity set and calling AddObject().
Let's look at the PostMessage action of my homepage controller, in Figure 10.
Figure 10 Create New Message and Push to Data Store
[AcceptVerbs("POST")]
        public ActionResult PostMessage(string channel, string author, string msg)
        {
            var svc = new Models.MicroData();

            var q = from c in svc.Channels
                    where c.PartitionKey == channel
                    select c;

            if (q.FirstOrDefault() == null) throw new
            ArgumentException("Invalid channel");

            User u = (from auth in svc.Users
                     where auth.UserName == author
                     select auth).FirstOrDefault();

            if (u == null)
            //throw new ArgumentException("Invalid Author");
            {
                var user = new User();
                user.UserName = author;
                user.DateCreated = DateTime.UtcNow;
                user.PartitionKey = user.UserName;
                user.RowKey = string.Empty;
                svc.AddObject("Users", user);
                u = user;
            }

            var m = new Models.Message(channel, author, msg);
            svc.AddObject("Messages", m);

            svc.SaveChanges();

            return this.RedirectToAction("Index");
        }
Just as I did to query the data service, I instantiate a new DataServiceContext as the entry point and query for the specified channel to check that it exists, throwing an exception if it is not found. I create a new Message, passing in the user input. Once the Message has been created, I tell the context that I have a new object to be added by calling svc.AddObject("Messages", m); passing in the EntitySet name that I am adding to, "Messages", and new Message object, m. Finally, I want to push the changes into the Table service by calling svc.SaveChanges();.
There are a couple of differences that you may notice here. The first is due to the lack of a fixed schema in Azure Table. This lack of schema means that I do not have strongly typed data and must call the untyped AddObject() on the DataServiceContext instance, passing the new object and the entity set that I am adding it to, rather than the typed AddTo…() methods that I used against my on-premises store. It is worth noting here that the more generic AddObject() method is still available in the on-premises scenario; however, in my example I use the strongly typed methods to take advantage of compile-time checking and so forth.
The second difference, also somewhat attributed to the lack of fixed schema, is the lack of a first-class relationship concept. You first notice this in the differences between how I structure the data in Azure Table compared to my on-premises data source. In the PostMessage action you'll notice that I am not setting links on my new object; rather, any relationship information is stored directly as properties in my Entities.
If I were modifying an existing instance rather than creating a new one, I would first query for the object that I want to update, make the desired changes and, finally, rather than AddObject, I would call the UpdateObject method to indicate to the context that it needs to send an update for that object the next time I call SaveChanges(), just as I would with an on-premises data service. Similarly, deleting an instance by calling the Delete method to indicate to the context that I want to delete that object, is also the same whether I am accessing an Azure Table service or my own on-premises service.

Summing Up
Looking at these two examples together, you can see how simple the move between developing against an on-premise service and an Azure Table service is made by the use of the same simple ADO.NET Data Services Client Library. The learning curve involved is rather flat if you are also responsible for managing each data source and mostly consists of a shift in thinking between the relational storage model and the more unstructured Azure Table model, as well as an awareness of the Azure Table PartitionKey and RowKey model. If you are focused primarily on the application development against these services, the skills you build up developing against one service are transferable to development against any other data service.

Elisa Flasko is a Program Manager in the Data Programmability team at Microsoft, including the ADO.NET technologies, XML technologies, and SQL Server Connectivity technologies. She can be reached at blogs.msdn.com/elisaj.


SOA Simplified
Service Virtualization With The Managed Services Engine
Aaron Skonnard

This article is based on prerelease versions of the Managed Services Engine and .NET Services. All information is subject to change.

This article discusses:
  • Service virtualization basics
  • Microsoft Services SOA Infrastructure
  • Getting started with the Managed Services Engine
  • Importing and configuring service resources
This article uses the following technologies:
Managed Services Engine, .NET Services, Windows Server, SQL Server, WCF
Contents
Service Virtualization
A Service Virtualization Implementation Pattern
Microsoft Services SOA Infrastructure
Understanding the Managed Services Engine
Getting Started with the MSE
Importing Service Resources from WSDL
Importing a RESTful Service Resource
Defining Virtual Service Endpoints
Versioning, Policies, and WCF Behaviors
Defining Virtual Cloud Endpoints
As businesses begin to adopt service-oriented architectures (SOA), they initially focus on service-enabling core business functions and processes. It's often accomplished by assigning ownership of individual services to autonomous domains within the business that largely operate in isolation from one another. This paradigm shift promotes loose-coupling, reuse, and the overall alignment of software assets with how the business actually works. When done right, the business ends up with a federated library of services that can be composed into solutions with greater agility. And as the overall service ecosystem grows over time, so do the potential benefits to the business.
While SOA can definitely provide benefits to the business, there's a common misperception that SOA makes things easier. This is simply not true. SOA embraces the fact that building distributed systems is inherently complex, and it outlines architectural principles that can help you manage some of that complexity. Large SOA initiatives are still inherently complex, but with a new set of challenges and issues. One of the greatest challenges in succeeding with SOA is establishing and maintaining a service governance solution that addresses the complexities inherent in a growing service ecosystem, thereby allowing the business to realize more benefits with less pain.
Ask yourself these questions to see if you might be struggling with some of the most common SOA challenges:
  • Can you trust your services to support mission-critical business processes?
  • Can you make changes to existing services without wondering what might break?
  • Do you have design-time and runtime visibility across all your systems and services?
  • Can business, operational, and policy changes be made without requiring developer involvement?
Today, most SOA initiatives cannot answer these questions in the affirmative because they lack a sound strategy for service governance within their SOA. Lacking a focus on service governance can prevent companies from realizing a net gain through their SOA initiatives. Without a governance solution, most SOA initiatives devolve over time into an unwieldy spaghetti mess of services without any sense of management, visibility, or versioning control, and it's only going to be worse as companies move more towards cloud computing over time.

Service Virtualization
One way to deal with all this complexity is through a governance solution that provides service virtualization. Service virtualization is an emerging trend in the SOA landscape that focuses on providing a common infrastructure for building and managing a complex service ecosystem while addressing the challenges of SOA initiatives. Microsoft Services has been providing leadership in this area through their Microsoft Services SOA Infrastructure offerings and a technical solution referred to as the Managed Services Engine (MSE). First I'll describe what they provide at a very high level before diving into the technical details of how it works.
The Microsoft approach to service virtualization is based on a common architecture and a centralized runtime that provides the service plumbing required by all services and their consumers throughout the entire heterogeneous service ecosystem. This lets developers focus on building new functionality without worrying about how the functionality will be exposed, consumed, and managed over time. The runtime provides the core capabilities all services need such as versioning, protocol-mapping, monitoring, routing, and run-time policy enforcement.
New services are added to the runtime to take advantage of these common runtime capabilities. Services are added to the runtime by simply importing the service metadata through Web Services Description Language (WSDL) definitions, WS-MetadataExchange endpoints, or custom importers for legacy line-of-business apps that describes the service. Once the metadata is imported, administrators can manage the physical service resources by defining models for different types of virtual services that the runtime can automatically host and expose to the various service consumers. The idea is consumers will interact only with the virtual services, which are significantly easier to manage than the real ones.
You define virtual services using models, which are essentially just expressions on top of the original service metadata. Microsoft provides a management tool for defining these models that can be customized for the different SOA stakeholders. The service virtualization runtime translates these models into service behaviors that provide the aforementioned capabilities without requiring any changes to the deployed and running service code. This approach allows you to easily configure virtual service endpoints to accommodate different types of consumers, to version operations, and to modify service policies as business requirements change on the fly.
The service model becomes the communication rendezvous point between all parties involved in the SOA ecosystem—architects, developers, IT professionals, and the business owners—all of whom can interact with the models through role-specific views in the management tool. It's important to note that the service models are actually executed by the service virtualization engine to provide different service behaviors at run time, which helps provide full fidelity with the original intent. Microsoft is moving from a world where models describe the application to a world where models are the application. The new "Oslo" modeling initiative is one example of that, as are these service virtualization techniques used by Microsoft Services.
From a consumer perspective, a virtual service looks just like any other service it might consume. It's a feature that consumers cannot tell the difference between a virtual service and the real service sitting behind the scenes.

A Service Virtualization Implementation Pattern
Service virtualization architectures usually revolve around the idea of a service intermediary sitting between the service client and the service implementation (see Figure 1).
A service intermediary is necessary to decouple the client from the service implementation. In this type of architecture, clients never interact with service implementations directly; they always communicate through the service intermediary. The service intermediary is where the virtual services are hosted and exposed. It can be located with the client, with the implementation, or somewhere in between to provide topology flexibility.
Figure 1 A Common Pattern for Service Virtualization
The service intermediary determines exactly how to host the virtual services by reading the service models from the service catalog. This allows you to expose multiple virtual services for a single service implementation so you can easily accommodate different customer scenarios. You can even define a model for exposing a virtual service in the cloud that relays communication back to your local service implementation. Stakeholders interact with the service catalog through the management tool that provides a role-specific user experience.
Because all communication travels through the service intermediary, it's possible for this central node to intercept each service invocation and introduce additional service behaviors (for example, operation versioning, protocol mapping, monitoring, routing, run-time policy enforcement, and custom behaviors) without having any impact on the client or service code. This service interception pattern is what enables the Microsoft Services SOA Infrastructure solution.

Microsoft Services SOA Infrastructure
Microsoft Services SOA Infrastructure provides an implementation of the service virtualization pattern through a solution called the Managed Services Engine (MSE). The MSE is built on the Microsoft server platform including Microsoft Windows Server 2003 or Microsoft Windows Server 2008, Microsoft SQL Server 2005 or SQL Server 2008, and Microsoft .NET Framework 3.5. There is a heavy focus on Windows Communication Foundation (WCF) for the underlying communication and the implementation of the service interception pattern described earlier. Microsoft Services has made the MSE solution (including source code) available for download via CodePlex. Microsoft Consulting Services (MCS) are available to assist for complex installations and customizations.
Although it's not required, the MSE can also integrate with Microsoft BizTalk Server 2006 R2 to provide additional capabilities such as business activity monitoring for end-to-end visibility, dynamic business rules processing through the Business Rules Engine (BRE), host system integration for service assisting legacy systems, and enterprise service bus (ESB) guidance for durable itinerary-based message routing. Microsoft Services combines the MSE service virtualization technology with these BizTalk Server technologies to offer a compelling SOA infrastructure solution.
The MSE can also make it easier to move toward cloud computing on the Azure Services Platform through integration with .NET Services. As part of .NET Services, the .NET Service Bus makes it possible to publish service endpoints in the cloud that relay communication back to your local on-premises services. You accomplish this by simply using a new set of relay bindings in your endpoint definitions (they ship as part of the .NET Services SDK). The MSE runtime is also built on WCF, which means it can support virtual service endpoints that use the .NET Service Bus relay bindings, thereby allowing you to integrate with virtual service endpoints in the cloud.

Understanding the Managed Services Engine
The MSE comes with a service runtime engine, a service catalog (repository), and a management tool for implementing a real-world service management solution based on service virtualization. It's built on WCF from the ground up, using common techniques and taking advantage of its various extensibility points when necessary.
The MSE runtime engine is implemented as a Windows service that manages a set of WCF service host instances that are automatically configured from the information found in the service catalog at run time. The MSE runtime consists of three logical components internally: the messenger, the broker, and the dispatcher (see Figure 2). The messenger is primarily responsible for message normalization. The broker receives the normalized message and is primarily responsible for operation rationalization (choosing a specific version of a specific operation). The dispatcher is primarily responsible for invoking the target implementation.
Figure 2 The Managed Services Engine Architecture
These three components are completely decoupled from one another since all communication happens via WCF channels. Hence, you can distribute them across multiple servers in order to accommodate a wide variety of system topologies that your environment might require.
The service catalog is often referred to as the metadata repository—you can think of it as the source of truth across an MSE environment and the participating systems. The catalog contains the models that drive your virtual services hosted by the MSE runtime. You import services (as well as other artifacts) into the repository using a management tool. Once you've imported the service metadata, you can begin defining what your virtual services will look like to the rest of the world.
When defining a virtual service, you start by choosing and configuring a WCF binding. You can choose what protocol to use, what security features to enable, or any other setting supported by the binding. You can also specify other service behaviors like message inspection, message transformation, version information, and what policies to enforce (including custom policies) using the same technique.
The metadata repository is implemented as a traditional SQL Server database. Consumers don't interact directly with the repository, although information from the repository can be published to an external registry (such as UDDI) for discovery purposes. The MSE maintains a clear distinction between the notion of a repository and public registry. The repository is the source of truth for data that drives the runtime, whereas the registry is a separate store of service metadata that you decide to make available for external consumption and discovery.
The MSE allows for independent ownership of each layer within the MSE environment. This is important because in most SOA environments, managing the overall ecosystem is a team effort with numerous roles involved. For example, developers usually define the data entities, operation contracts, and implementation logic, whereas the IT operations staff dictates service instance configuration, endpoint policies, and messaging standards. Architects can influence service policies and define execution patterns while business owners dictate business policy requirements. The MSE makes it possible for all stakeholders to collaborate with one another through the MSE catalog, which acts as a central rendezvous point for the system. And the MSE runtime dynamically executes those models to influence the runtime behavior of each layer within the configured MSE environment.

Getting Started with the MSE
In order to get started with the MSE, you'll first need to download and install it on one of your development machines. The installation will set up and configure the various components described in the previous section.
The installer will create and populate a SQL Server database called MSE_REPOSITORY. (The MSE supports both SQL Server 2008 and SQL Server 2005.) The repository database is where the service catalog information is stored that drives the runtime and that ultimately shows up in the management tool. If you poke around, you'll find various tables for holding data entities, operations, endpoints, policies, and runtime servers.
The installer also sets up Windows services that host the MSE runtime and catalog logic. After installation, you'll find a Windows service called MSE Runtime Server and another called MSE Catalog Server. The MSE Runtime Server service is the process that ultimately hosts your virtual service endpoints and provides the runtime messenger/broker/dispatcher logic. The MSE Catalog Server provides an interface to underlying service catalog.
You'll also find a new program group in your Start menu labeled Microsoft Managed Services Engine—Management Tool. If you expand the group, you'll find two programs: the MSE Administration Tool and the MSE Service Tester. The former is the management tool I've been referring to up to this point while the latter is a generic service testing tool that makes it easy to test the virtual service endpoints hosted in the MSE runtime.
Figure 3 shows what the MSE Administration Tool looks like the first time you launch it. This is showing what's referred to as the end-to-end view of the MSE system. The six areas spanning the top row show the end-to-end execution path starting from the virtual MSE runtime servers (left) to the real service systems (right). Figure 4 provides a brief description of each section shown in the MSE Administration Tool's end-to-end view.
Figure 3 The MSE Administration Tool
Figure 4 MSE End-to-End View Elements
Section | Description
Systems | Describes the servers hosting the real services (that you wish to virtualize)
Instances | Describes the individual service endpoints running on a particular system
Resources | Describes the service operations supported by a particular service "instance"
Operations | Describes virtual service operations that map to specific resources
Endpoints | Describes virtual service endpoints that support a set of specific virtual operations
Servers | Describes the MSE runtime servers hosting the virtual endpoints
Bindings | Describes a set of binding configurations used by instances and endpoints
Policies | Describes a set of policies that can be used by most artifacts
Data Entities | Describes a set of data entities that are used to define operation contracts
The Layout menu allows you to select different views of the system that simplify the experience for different types of SOA stakeholders. Each system artifact within the MSE models can be assigned an owner where you can specify name, contact, and location information. This owner information can be viewed and modified by the different stakeholders to figure out who to talk to when issues arise throughout the SOA lifecycle.
We don't currently have any virtual services defined, but the catalog does come prepopulated with some default configurations. In order to get started, you need to import your service resources into the catalog.

Importing Service Resources from WSDL
Before you can start creating virtual services in the MSE, you must first populate the repository with metadata describing the real service resources that you intend to virtualize. Since the MSE will be invoking the real service instance on behalf of consumers, it needs to know what address it's listening on, the specific WCF binding configuration to use, and what the request/response messages will contain (basically the information you'll find in a service's WSDL definition). You can enter this information manually for a specific service or you can import it from the service's WSDL definition using one of the wizards provided by the MSE Administration Tool. Since the latter approach is more common, let's walk through an example of importing a new service resource.
You import new service resources through the MSE Import Wizard, which you can launch from the Tools menu by selecting Load Resource | Load Web Service Metadata. The wizard will first ask for you to specify the location of the WSDL definition describing the service by either specifying a URL or a path to a local file.
Then the wizard will ask you what type of service metadata you'd like to extract from the WSDL definition to import into the repository. You can choose to import only the schema definitions, and they will be imported as schemas and data entities in the MSE repository. You can also choose to import all of the service resources found in the WSDL definition including the schemas, operations, bindings, and endpoints, and they will be imported into the MSE repository as data entities, resources, bindings, and instances, respectively.
There's yet another option that imports all service resources (like the second option) but it also generates a virtual service operation for each operation found in the service metadata. This option can be useful to speed things up when your service contains a lot of operations, but it's not necessary since there's also a separate wizard for defining virtual service operations. I'll show you how to create new virtual operations a bit later.
At this point, the wizard will show you everything it plans to import, giving you an opportunity to exclude certain items if you so choose. It will also show you if the item already happens to exist in the MSE catalog, indicating that you don't need to import a particular item again. If you selected the option to generate virtual operations, you'll also see a summary of the virtual operations it plans to import into the catalog.
You can modify the names of your virtual service operations during this step—I've prefixed each operation name with "Virtual" in my example (VirtualGetPizzaMenu, VirtualGetPizza, and so on). Figure 5 shows the summary of the virtual operations it's going to define and map to the real service operations. Notice how the operation name is extended with an operation version. In this case, the initial versions are 1.0.0.0. As newer versions of operations are incorporated, the operation name remains the same but the version is incremented as needed.
Figure 5 Specifying Virtual Operations in the MSE Import Wizard
It's interesting to note that the MSE repository doesn't contain the definition of a service contract as you will find in a WSDL definition. Instead the model revolves around service operations. The model allows you to arbitrarily compose service operations onto endpoints however you see fit, dynamically creating different contract configurations at run time. This also means you have to worry about versioning only at the operation and data entity levels, which is possible because the service contract itself is never represented in the messages at run time.
Once you confirm the import, the wizard will bring the selected items into the MSE repository and now you're ready to start defining some virtual service endpoints to expose your virtual operations.

Importing a RESTful Service Resource
The MSE also makes it possible to virtualize RESTful services. However, since RESTful services don't typically come with a WSDL definition, you'll have to use a different wizard for defining and importing metadata that describes the RESTful service in terms of operations. You can begin this process by selecting Load Resources | REST Service Metadata from the Tools menu. Doing so launches the RESTful Service Virtualization Wizard.
This wizard starts by asking you to specify the name of the resource exposed by the RESTful service along with the base URI for the resource. Then it asks you to specify the HTTP verbs (GET, POST, PUT, or DELETE) and content type (for example, text/xml, application/json, or application/atom+xml) supported by the resource. It will also ask you to specify a set of resource attributes that can be used in conjunction with the different verbs (see Figure 6). The resource attributes are basically the parameters required by the different HTTP verbs.
Figure 6 Specifying Resource Metadata for a RESTful Service
The next step in the wizard allows you to define the URL contract for each of the HTTP verbs supported on the resource. This is where you specify the UriTemplate pattern, including any URI variables—the URI variables should map to the resource attributes you defined in the previous step (see Figure 6).
Once you finish specifying all of these details, you can complete the wizard to import the metadata for that particular resource. You'll need to repeat the wizard for each resource exposed by the RESTful service. The MSE runtime will be able to use this information to figure out how to interact with each of the different REST resources at run time.
After you've finished importing all of your REST resources using the wizard, you can define virtual operations for them, making it possible for consumers to interact with them through the MSE.
You can define a virtual operation by running the Virtual Operation Creation Wizard (right-click in the Operations area and select Add New Item). This wizard allows you to specify the name and version of the virtual operation, along with a human readable description. You can also specify if the virtual operation is public, active, or testable (see Figure 7). A public operation is discoverable via the virtual service endpoint or MSE-associated registry. An active operation can be invoked through the MSE (for example, you might have five different versions of an operation that are all active, but only one that's publicly discoverable). A testable operation can be called even if it's not backed by a real service implementation (the MSE represents a sample implementation, making it possible to test consumers before the service has been deployed).
Figure 7 Virtual Operation Creation Wizard
The next step of the wizard will ask you to specify what data entities to use for the request/response. Then you'll have a chance to map the virtual operation to a real service resource—this is where you'll map it to the REST resource defined using the RESTful Service Virtualization Wizard. This essentially tells the MSE runtime which service instance to execute in response to calls targeting this virtual operation.
It's important to note that when you import the SOAP service resource using WSDL, the wizard takes care of creating default virtual operations for you (if you select the Virtual Service Operations option in the wizard). Hence, you don't have to do this step manually. You'll always have to do this manually for RESTful services because they don't provide WSDL metadata that can be used to automate things.
Figure 8 shows what the MSE Administration Tool looks like after importing both the SOAP and REST operations. Notice the four virtual operations, each of which maps to a real service resource that will be executed by the MSE runtime. Each service resource maps to a service instance hosted on a backend system. At this point, you're ready to expose some virtual service endpoints through an MSE runtime server.
Figure 8 MSE Administration Tool after Importing Services

Defining Virtual Service Endpoints
The MSE Administration Tool makes it easy to define virtual service endpoints. Simply right-click on the Endpoints area and select Add New Item. This will give you an interface for defining a WCF endpoint definition. You specify the port and path for the endpoint address (but not the server, since that's controlled separately by the server configuration), along with the binding configuration you want to use. You specify the binding configuration by dragging one of the items from the binding configurations area into the form. Then you can associate virtual operations with the virtual endpoint by dragging the virtual operations onto the endpoint form (see Figure 9). The operation associations are what ultimately define the service contract for the virtual endpoint.
Figure 9 Defi ning a Virtual Endpoint
You can create as many virtual endpoints as you need this way, making it possible to accommodate a wide variety of different consumers with differing communication requirements. As an example, I created three different virtual endpoints: one using the WSHttpBinding (SOAP 1.2), another using the NetTcpBinding, and yet another using a POX binding (no SOAP).
Once you have the different virtual endpoints defined, you can choose which virtual operations you'd like to associate with each one. For this example, I can associate all of the virtual operations with each of my virtual endpoint definitions. This means I can invoke all four virtual operations through each endpoint.
After you've defined your virtual endpoints, the last thing you have to do is associate them with a configured MSE runtime server. You configure the runtime server instances within the Servers area in the MSE Administration Tool. They provide a default runtime server configuration called LocalRS when you install the MSE. You can publish virtual endpoints on this default runtime server by simply dragging them onto the LocalRS icon.
These virtual endpoints are now active and ready to be called by consumers. At this point, you can browse to any of the virtual endpoint addresses and retrieve its metadata (for example, http://localhost:10002/vpizza/ws?wsdl). Consumers can then download the metadata and build client applications against the MSE virtual service. You can also bring up the MSE Universal Service Tester (from the Start menu) and enter the WSDL address of the virtual service. This tool makes it easy to test your virtual service endpoints without writing a client (see Figure 10).
Figure 10 Using the MSE Universal Service Tester
When a consumer invokes one of the virtual operations, the MSE runtime will dispatch the call to the corresponding service resource behind the scenes. During this process, the MSE maps the virtual operation invocation to the real service resource behind the scenes.
During the invocation process, the MSE runtime provides support for operation versioning and runtime policy enforcement to enable things like message transformation, message validation, custom service behaviors, and business activity tracking and monitoring. All of these things are easy to configure via the MSE Administration Tool.

Versioning, Policies, and WCF Behaviors
One of the most difficult things to manage in real-world SOA environments is service versioning. Through service virtualization, the MSE is able to provide a versioning solution that is intuitive and easy to manage. When you need to version a service operation, you simply define a new virtual operation with an incremented version number. Then you make whatever adjustments you need to make to the new virtual operation version. The virtual operation will show up only once in the virtual operation list, but it will contain two version configurations.
When you drag the versioned operation onto a virtual endpoint, it will ask you which of the versions available you'd like to make active and which version you'd like to publish. Multiple versions can be active at the same time, but only a single version can be published to the registry—typically the latest version—due to the inherent restrictions of the WSDL specification. Similar versioning support is also provided at the data entity level in the MSE. In general, having a built-in versioning solution is critical for managing large-scale SOA environments as they grow over time.
In addition to service versioning, the other key feature provided by the MSE is dynamic policy configuration. The MSE formalizes the concept of policies, policy assertions, and policy assertion types, which behind the scenes end up mapping to WCF behaviors that can extend your virtual services in a variety of different ways. If it can be done with a WCF behavior, you can map it to a declarative policy for use within the MSE environment.
You can import existing WCF behaviors into the MSE catalog through the Policy Import Wizard (select Tools | Load Policy Components | WCF Behavior Assertion Types). This wizard walks you through the process of creating a new policy assertion that maps to a WCF behavior and creates the necessary policy assertion types within the MSE catalog. Once you've done this, you can easily apply policies to systems, instances, resources, virtual operations, and virtual endpoints by simply dragging the policies onto those items within the MSE Administration Tool.
For example, you can define a policy that performs message transformations to enable a more intelligent mechanism for mapping between versioned operations. You can also define a policy that performs custom schema validation against the incoming messages in a standard way. Or you can define polices that introduce custom security, tracking, and logging behaviors. The Microsoft Services SOA Infrastructure solution comes with built-in polices for enabling business activity monitoring across your virtual services hosted by the MSE along with reports for viewing the data.

Defining Virtual Cloud Endpoints
Since both the MSE and the .NET Service Bus are built on the WCF binding/channel model, it's easy to bring these two worlds together. Doing so makes it possible to enable two key cloud scenarios illustrated in Figure 11.
Figure 11 Service Virtualization in the Cloud
First, you can virtualize .NET Service Bus endpoints through the MSE. This means you can create different virtual endpoints in the MSE that act as facades for a .NET Service Bus endpoint. When consumers invoke one of the virtual service endpoints, the MSE will invoke the corresponding .NET Service Bus endpoint in the cloud, which relays the message to an on-premises service somewhere in the world. (It relays the message to any listeners registered with that address on the .NET Service Bus.)
Second, you can create virtual service endpoints hosted by the MSE that actually listen on the .NET Service Bus. In this case, the .NET Service Bus endpoints relay to the MSE runtime server, which in turn relays to a real service instance managed by MSE.
To begin integrating your MSE installation with .NET Services, you'll need to install the .NET Services SDK on the MSE server. Then you'll need to define a custom binding within the MSE Administration Tool for each of the .NET Service Bus relay bindings you wish to use within the MSE catalog. You can do this by switching to the Binding Management layout and adding a new binding configuration. Here you can enter a name for your new binding configuration (such as NetTcpRelay) along with the actual XML binding configuration (the same format you'd use in a WCF configuration file). This is the XML for the default NetTcpRelayBinding configuration:
<bindings >
  <netTcpRelayBinding>
    <binding name="default">
    </binding>
  </netTcpRelayBinding>
</bindings>
Once you've defined all of the relay bindings you plan to use, you can begin using them in conjunction with the other MSE artifacts. For example, to configure the MSE to call the .NET Service Bus, you'd first create a new MSE system using the address to the .NET Service Bus (). Then you'll define a new instance for the .NET Service Bus, specifying the solution name to use in the URL, and you'll map that to the .NET Service Bus system you just defined. Then you'll need to leverage the .NET Service Bus policy that supplies the .NET Service Bus credentials at run time, and you'll associate the policy with the instance.
Once you're finished, you can begin defining MSE resources that route to the .NET Service Bus instance. Then you can define virtual operations and endpoints that map to the .NET Service Bus resources.
You can also define MSE virtual endpoints that use the .NET Service Bus relay bindings that you've added to the catalog. For the virtual endpoints, you'll need to specify the appropriate .NET Service Bus path (/services/[solution-name]/[path]) and define a new server instance that refers to the .NET Service Bus host. The MSE runtime will register a listener on the .NET Service Bus for each virtual endpoint, allowing the MSE runtime to receive relayed messages from the cloud.
As you can see, by using the MSE service virtualization technology, you can begin participating in the cloud by simply configuring virtual service endpoints and operations that integrate with the .NET Service Bus.
Service virtualization will help reduce your time-to-market for new services on-premises or in the cloud, and it will provide a more flexible and realistic approach for managing your service ecosystem as it grows and matures over time. The bottom line: service virtualization can help you realize the full benefits SOA has to offer your business.

Aaron Skonnard is a Microsoft MVP and 10-year columnist for MSDN Magazine as well as co-founder of Pluralsight, an industry think-tank where he stays busy recording Pluralsight On-Demand! courses on Windows Azure, WCF, and REST. Reach him at http://pluralsight.com/aaron and follow him on http://twitter.com/skonnard.


Editor's Note
Head in the Clouds
Howard Dierking

A few weeks ago, I had quite the technical scare at home. A hard drive on my network file server failed and I nearly lost several months worth of family photos (like this one). The ironic thing about the whole event was that I had my network server configured to back up to a service running in the cloud. All I had to do, I thought, was store all files on that server and I would never have to worry about losing data again. What I failed to consider, though, was that my backup service had a limit of 150GB, and once I started adding family videos to my list of data that required backup, I quickly surpassed this limit.
Fortunateley, I was able to work some magic on the drive and get it back to a workable state long enough to replicate the data onto another drive. However, the experience left me quite motivated to ensure that I had not just a good backup strategy in place, but a good overall data management strategy. What this meant for me was going through all of the data that I store on my network and making a deliberate decision about what needed to happen with it. For example, I left music and photos on the network server because those are shared between multiple members of my family. However, I took all of my programming projects off of my server—in fact, off of my network entirely—and moved them to a hosted version control system running in the cloud.
Forwarding to the present day, I've been spending a fair amount of time researching the current releases of Microsoft's cloud computing technologies, and my early experiences in working with the technology samples remind me of one of the main takeaways from my recent almost data loss. Simply put, moving to the cloud is not something that should be done blindly. Sure, you could probably take your existing Web applications and services and deploy them directly to Windows Azure, and they would likely run fine—just as they likely run fine on your current hardware. However, to look at the cloud like this—as simply the further commoditization of hardware—is, I believe, to miss out on some real opportunities to look critically at our existing application architectures and find the places where we can make changes to make better use of the platform.
For example, I was listening to a talk recently on how to leverage Windows Azure queues and how this technology should impact the overall application architecture. The result is something that should not look foreign in principle—asynchronous services receive a message and put it into a worker queue, where it will later be read and processed by a different service. However, in the reality of the elastic-scale cloud platform, taking advantage of this platform component can enable one to create more independently scalable units of work and by doing so, scale much more efficiently than an application partitioned in a more "all-or-nothing" style. From the business perspective, when your cost is based on the resources your application requires, architectural shifts like this are more than just an academic exercise.
Another major theme of mine that has reemerged from my recent research foray into the cloud computing technologies is that if you're a developer these days, there is simply an overwhelming volume of information to absorb and apply. Additionally, these kinds of technologies are meaningless unless you can apply them in your current projects. Therefore, I'm happy to announce we are launching the first of what I hope will be many "brownfield development" series. The first series will start in May and will run weekly at MSDN Magazine online, focusing on taking an existing ASP.NET application and making it into a Rich Internet Application (RIA). I hope to take this same model and use it to create further series on a host of topics, one being the refactoring of applications to efficiently leverage the cloud.

Visit us at msdn.microsoft.com/magazine. Questions, comments, or suggestions for MSDN Magazine? Send them to the editor: mmeditor@microsoft.com.

Thanks to the following Microsoft technical experts for their help with this issue: Paul Andrew, Arkadi Brjazovski, Pablo Castro, Sandu Chirica, Fred Chong, Andy Conrad, Mike Flasko, Phil Haak, David Hill, Jason Hogg, Nataraja Koduru, Bertrand LeRoy, Steve Marx, Eugenio Pace, and Chester Smith.


Toolbox
Easy File Backup, Exploring Files And Folders Inside Visual Studio, Multiple Monitor Software, And More
Scott Mitchell

All prices confirmed at press time and are subject to change. The opinions expressed in this column are solely those of the author and do not necessarily reflect the opinions at Microsoft.

Contents
A Blog of Note
Explore Files and Folders from within the Visual Studio IDE
Tools for Multiple Monitors

File Backup and Synchronization Made Easy
One of the benefits of working in a large company with an entrenched IT department is that the myriad of computer maintenance tasks and core developer tools are already in place. You don't have to worry about backing up your workstation's data or which source control system to use, or spend time installing software and applying security updates. But if you work for a smaller company with a limited IT staff, or if you make your living as an independent software consultant or run a micro-ISV, these important decisions are likely to fall on your shoulders.
If the responsibility for creating, managing, and executing routine backups is yours, check out SyncBackPro (version 5.3) from 2BrightSparks. Getting started with SyncBackPro involves creating one or more profiles, where each profile spells out the source files, the destination location, options as to whether the files are compressed or encrypted when copied to the destination, whether to overwrite previous backup content or to version the data, and so forth. Once defined, a profile can be executed immediately, according to a schedule, every specified number of minutes or hours, whenever any of the source files or folders changes, or whenever Windows is shut down. SyncBackPro can also synchronize the files and folders between a source and a destination or mirror the contents of the source on the destination site.
SyncBackPro works with a variety of backup sources and destinations. In addition to using folders on the workstation or network, you can also perform backups using FTP or SFTP, e-mail, or removable media such as DVD burners. And the FTP and e-mail options can be used as either a backup source or destination. This is especially useful if you have a Web site managed by a Web host provider and you want to maintain a copy of the contents on your local computer—just create a SyncBackPro profile that periodically backs up your important data from the Web site's FTP server to your workstation.
What impressed me most about SyncBackPro is how it balances its many features with ease of use. With each profile there are countless settings and configuration options at your disposal. Take logging, for example. There are about a dozen choices as for which information is logged, who the log file should be e-mailed to (if anyone), and whether the log should only be e-mailed if certain circumstances unfold. And if the wealth of configuration options isn't sufficient, you can write your own scripts against SyncBackPro's runtime API to perform custom logic. Despite this level of customization, SyncBackPro remains easy to use. Creating a new profile launches a wizard that walks through the basic settings. The options screen offers an "Easy" mode that streamlines the number of configuration options to the most essential ones. And both the online and offline documentation is well written and rife with screen shots, videos, and step-by-step instructions.
A freeware edition of SyncBack is available as well. The freeware version provides basic backup and synchronization, but lacks many of the more advanced features found in SyncBackPro, such as backups to CDs/DVDs, backup via SFTP, and file versioning, among others.
Price: $49.95 per user license
2brightsparks.com/syncback
SyncBack

A Blog of Note
When designing a software application it's tempting to plan on adding features to address every potential use case. While the majority of customers have similar requirements, there will always be someone with a demand for a feature that falls outside of your project's initial scope. Therefore, adding more and more features to cover these one-off needs increases the application's market size. While this approach sounds reasonable, it has several drawbacks. For starters, more features means more complexity, more bugs, and more time spent coding, debugging, and testing. It also results in an application with more buttons and menu options and screens and dialog boxes, making for a more challenging user experience.
In many cases, simpler is better. Focus on the core features and make sure that the most common use cases are easy to accomplish. And above all, don't waste your time or clutter your user interface with edge case features. One company that has embraced this sentiment is 37signals, a company that sells Web-based project management, contact management, and team collaboration software. These products all share a common design goal: offer just the features that customers need and nothing they don't.
The staff at 37signals maintains a blog with daily posts on Web design and programming topics, and is a great resource for anecdotes and lessons learned from developers building software that embraces the "less is more" philosophy. The 37signals blog also includes a number of insights on marketing and running a small business. If you currently sell a product or service, or are considering starting your own business, be sure to check out "The benefits of a monthly recurring revenue in tough economic times," where the pros and cons of recurring revenues and one-off sales are examined. "Beware of future creep" is another interesting read, which warns about adding code or modifying the architecture of an application to accommodate features that are not needed now, but may be useful at some point in the future.
In addition to the blog, 37signals published a book in 2006 titled Getting Real, which details their design and programming philosophies over the course of 91 essays. You can buy a paperback or PDF version of the book, or read it entirely online for free. And a new, upcoming book is in the works.
37signals.com/svn

Explore Files and Folders from within the Visual Studio IDE
Visual Studio and Windows Explorer are two programs that are always running on my workstation, and it's not uncommon for me to use these two programs in an interrelated manner. I may be debugging code that logs diagnostic data to an XML file, or working on a program that generates PDFs, ZIP files, or other types of files as output. What's more, certain source control tools, like Subversion and Tortoise SVN, use Windows Explorer as the interface for checking files in and out of the repository. All of these cases require frequent toggling between Visual Studio and Windows Explorer.
If you find yourself routinely switching between these two programs, check out LogicNP Software's VS Explorer (version 2009), a Visual Studio Add-In that provides a Windows Explorer–like interface directly within Visual Studio. Once VS Explorer is installed, go to the View menu and select the VS Explorer option to display the VS Explorer window, which can be positioned, pinned, and resized like any other window in Visual Studio.
Much like Windows Explorer, the VS Explorer window offers a variety of ways to view files and folders. You can view the file system as a tree, as a list of items, or in a dual-pane configuration that includes both views; the items can be displayed as icons, as a simple or detailed list, or as thumbnails. Right-clicking on a file or folder brings up the same context menu shown in Windows Explorer. What's more, the files displayed in VS Explorer appear with the same overlay icons as they do in Windows Explorer, which is great news if you use TortoiseSVN or other software that adds such overlays.
VS Explorer offers a number of features that are not found in Windows Explorer. For example, with the click of a button you can immediately browse to the Solution Folder of the currently loaded application, or to the active Project folder, output folder, or the selected item in the Solution Explorer window. There is also a button that drops you to the command line in the same directory that's currently selected in the VS Explorer window. You can add an item from the file system into your project by dragging a file or folder from the VS Explorer window to the Solution Explorer window. Copy the full path of a file or folder by right-clicking it in the VS Explorer window and choosing "Copy Path to Clipboard." And adding a reference to an assembly has never been easier. Simply browse to the .exe or .dll file, right-click on the file, and choose the "Add as Reference" option from the context menu.
VS Explorer works with the full versions of Visual Studio 2005 and Visual Studio 2008.
Price: $35 per user license
ssware.com
VS Explorer

Tools for Multiple Monitors
Several independent studies have found a high correlation between developer productivity and multiple monitors. Developers are commonly working on tasks that involve simultaneously processing information from two (or more) windows, such as a running application and the debugger. While Microsoft Windows has long supported multiple monitors, there is a noticeable lack of built-in tools for managing windows in multiple-monitor environments. Instead, developers who use multiple monitors must turn to third-party and open-source tools.
One must-have tool for multiple monitor setups is UltraMon by Realtime Soft, which extends the taskbar across all monitors and adds buttons for moving windows between monitors, along with other enhancements. UltraMon was reviewed in the December 2007 issue ("Live Chat, Code Conversion, Multiple Monitors, and More").
Another indispensible tool is WinSplit Revolution (version 9.02), a free utility created by Raphael Lencrerot that adds a number of keyboard shortcuts for moving, resizing, and positioning windows. By default, the WinSplit Revolution keyboard shortcuts take the form Ctrl+Alt+key, where different keys result in different behavior. For example, Ctrl+Alt+Left Arrow moves the current window one monitor to the left, while Ctrl+Alt+Right Arrow moves it one monitor to the right. Hitting Ctrl+Alt plus a number from the numeric keypad repositions the window in the current screen. Ctrl+Alt+1 resizes and repositions the current window so that it occupies the bottom left quadrant of the current screen. Ctrl+Alt+5 resizes the window so that it fills the entire screen, whereas Ctrl+Alt+9 repositions the window in the upper right quadrant.
With multiple monitors it's also fun to decorate your desktop. Do you want a single image stretched across monitors, or a separate image per screen? To manage your desktop backgrounds in a multi-monitor environment, check out John's Background Switcher (version 3.6), created by John Conners. With John's Background Switcher you can choose a specific background image for each monitor or define a collection of images from which random images are chosen. This collection of images can come from pictures on your computer; from picture sharing and social network Web sites like Flickr, Facebook, or Picasa, among others; and even from RSS feeds. What's more, the images selected for display can be scaled to fit the screen or combined in a variety of montage styles.
WinSplit Revolution: Free
winsplit-revolution.com
John's Background Switcher: Free
johnsadventures.com/software/backgroundswitcher
Send your questions and comments for Scott to toolsmm@microsoft.com.

Scott Mitchell, author of numerous books and founder of 4GuysFromRolla.com, is an MVP who has been working with Microsoft Web technologies since 1998. Scott is an independent consultant, trainer, and writer. Reach him at Mitchell@4guysfromrolla.com or via his blog at ScottOnWriting.NET.


Basic Instincts
Documenting Your Code With XML Comments
Lisa Feigenbaum

This article is partly based on a prerelease version of Visual Studio. All information is subject to change.

Contents
XML Comment Basics
Customizing XML Comments
Generating the XML Documentation File
Enhancing Visual Studio with XML Comments
The Advantages of Commenting Someone Else's Code
Tricks from the .NET Framework
Generating Nicer Documentation
XML Comments in Visual Studio 2010
Looking for an easy yet effective way to document your code? XML comments provide a great solution. XML comments for Visual Basic were first introduced in Visual Studio 2005. They can be used to create a documentation file for the project, and provide a rich development environment experience for yourself, your teammates, or other people using your code.
In this article, I will introduce you to XML comments and explain how to use them. I will explore ways in which XML comments can be used to customize your coding environment and to create documentation files from the comments in your code. I will also show you some future Visual Studio features that will create an even better experience for working with XML comments in your code.

XML Comment Basics
XML comments can be used to document almost all definitions except namespaces. This includes types (Class, Module, Interface, Structure, Enum, Delegate), fields (Dim), properties (Property), events (Event), and methods (Sub, Function, Declare, Operator).
XML comments are inserted inline, directly in your source code. This makes it easier to keep them up to date as the code evolves. To insert an XML comment, type three single-quotation marks ('") immediately above the definition, like this:
'''
Function RegKeyExists(ByVal regKey As String) As Boolean
  Dim exists As Boolean = False
  Try
...
Alternatively, you can also type '" at the start of the definition line and press Enter. Visual Studio will insert a skeleton of the XML comment for you to fill in.
''' <summary>
'''
''' </summary>
''' <param name="regKey"></praram>
''' <returns></returns>
''' <remarks></remarks>
Function RegKeyExists(ByVal regKey As String) As Boolean
  Dim exists As Boolean = False
  Try
...
For the purpose of this article, I used a simple function with one parameter to demonstrate the XML comments feature. The XML skeleton varies according to the definition. For example, the XML skeleton for a Function includes a returns element, whereas the skeleton for a Sub does not. The number of param tags for methods will also vary based on the number of parameters.
Please note that although Visual Studio will insert the appropriate XML skeleton for the definition, and will provide some warnings if the comment gets out of sync, it will not automatically correct the comment for you. Therefore, make sure to update the comment as you make changes to the definition.
Once the XML skeleton is inserted, you can tab through the content wells to insert your comments. Visual Studio colorizes the XML content distinctly from the tags. You can also delete elements if you don't need them, such as the remarks element.
Finally, you can add elements that were not in the original XML skeleton. As you start typing a left angle bracket (<), a list of common tags will appear in an IntelliSense pop-up menu as shown in Figure 1.
Figure 1 XML Comment in IntelliSense
You can add any valid XML to the XML comment. A list of commonly used tags can be found in the article "Recommended XML Tags for Documentation Comments (Visual Basic)." If the comment takes up too much space, you can collapse it to its summary using the +/- controls on the left-hand side of the code editor, as shown in Figure 2.
Figure 2 Collapsed XML Comment

Customizing XML Comments
In the previous example, I made a number of changes to the original XML skeleton. Developers working in an enterprise environment might need to change the default XML comments to match their particular corporate guidelines. To help with these scenarios, Visual Basic provides a way to customize the default XML skeleton.
First, create a new XML file called VBXMLDoc.xml that includes your default comment templates. A sample file is included in the code download for this article. Save VBXMLDoc.xml to the appropriate location based on your version of Windows and Visual Studio as shown in Figure 3. In each case, replace <user> in the path with your own user name.
Figure 3 Where to Save VBXMLDoc.xml
Visual Studio 2005 | Visual Studio 2008 | Windows XP | Windows Vista | Path
X |  | X |  | C:\Documents and Settings\<user>\Application Data\Microsoft\Visual Studio\8.0
X |  |  | X | C:\Users\<user>\AppData\Roaming\Microsoft\VisualStudio\8.0
 | X | X |  | C:\Documents and Settings\<user>\Application Data\Microsoft\Visual Studio\9.0
 | X |  | X | C:\Users\<user>\AppData\Roaming\Microsoft\VisualStudio\9.0
Visual Studio has built-in defaults for the XML skeletons that get inserted, but when VBXMLDoc.xml is present upon startup, Visual Studio will use the XML definitions from that file instead. The version of VBXMLDoc.xml provided in the code download contains the default tags that would otherwise be inserted by Visual Studio. To change the defaults, find the code element type you are interested in and edit the XML elements in the file.
As an example, let's change the XML skeleton that gets inserted for Function. Figure 4 shows the default and customized entries for a Function. The Template element's children represent the XML elements that will be inserted in the XML comment skeleton. The CompletionList element's children represent the suggestions that will appear in IntelliSense upon typing a left angle bracket (<) above a Function.
Figure 4 XML Tags for Function
Default XML
<CodeElement type="Function">
  <Template>
    <summary/>
    <param/>
    <returns/>
    <remarks/>
  </Template>
  <CompletionList>
    <exception cref=""/>
    <include file="" path=""/>
    <param name=""/>
    <remarks/>
    <returns/>
    <summary/>
  </CompletionList>
</CodeElement>
Customized XML
<CodeElement type="Function">
  <Template>
    <summary/>
    <param/>
    <returns/>
    <author/>
  </Template>
  <CompletionList>
    <exception cref=""/>
    <include file="" path=""/>
    <param name=""/>
    <permission cref=""/>
    <returns/>
    <summary/>
    <author/>
    <history/>
  </CompletionList>
</CodeElement>
As you can see in the second part of Figure 4, I made a few simple customizations. First, I removed the remarks element from both the default skeleton and IntelliSense. Also, I added an author element to both the default skeleton and IntelliSense, and I added a history element to IntelliSense.
At this point you will need to close and re-open Visual Studio for the changes in VBXMLDoc.xml to get picked up. After restarting, the XML comments automatically inserted above Function will include an author element instead of remarks:
''' <summary>
'''
''' </summary>
''' <param name="regKey"></praram>
''' <returns></returns>
''' <author></author>
Function RegKeyExists(ByVal regKey As String) As Boolean
  Dim exists As Boolean = False
  Try
...
While it is possible to add XML elements to the template, it is not possible to add XML values. Thus, you can make the IDE insert <author></author> by default, but it is not possible to make it insert <author>Microsoft Corporation</author>.

Generating the XML Documentation File
The Visual Basic compiler generates an XML document for your assembly with all the XML comments defined in the code. The compiler will also resolve symbols used in cref, permission, and name attributes, as well as file references in include elements.
The generated file doesn't show your commented members hierarchically. Rather, it is a flat list. It includes a unique ID string for each definition that allows the comments to be mapped back to their definitions in code (see Figure 5). In this case, the string is M:RegLib.Helpers.RegKeyExists(System.String). M stands for method, RegLib.Helpers specifies the path, RegKeyExists is the method name, and System.String the parameter type.
Figure 5 Generated XML Document Excerpt
<?xml version="1.0" ?>
<doc>
<assembly>
<name>RegLib</name>
</assembly>
<members>
...
<member name="M:RegLib.Helpers.RegKeyExists(System.String)">
  <summary>Determines whether a specific registry key exists.</summary>
  <param name="regKey">Name or path of the registry key.</param>
  <returns>True if the registry key exists; otherwise, False.</returns>
  <author>Microsoft Corporation</author>
</member>
...
</members>
</doc>
You can generate the XML documentation file using either the command-line compiler or through the Visual Studio interface. If you are compiling with the command-line compiler, use options /doc or /doc+. That will generate an XML file by the same name and in the same path as the assembly. To specify a different file name, use /doc:file.
If you are using the Visual Studio interface, there's a setting that controls whether the XML documentation file is generated. To set it, double-click My Project in Solution Explorer to open the Project Designer. Navigate to the Compile tab. Find "Generate XML documentation file" at the bottom of the window, and make sure it is checked. By default this setting is on. It generates an XML file using the same name and path as the assembly.
My example is a class library project called RegLib, so the assembly and XML documentation file will be RegLib.dll and RegLib.xml, respectively. The path where they will be created is listed in the "Build output path" textbox. There must be an explicit build for these files to be produced.
Microsoft uses XML comments to generate documentation for all the Microsoft .NET Framework assemblies. The XML documentation files are placed beside the DLLs they document, and they match in name.

Enhancing Visual Studio with XML Comments
Many Visual Studio features use XML comments to provide a better experience for working with members. Because the Visual Basic compiler is always running in background, Visual Studio can consume XML comments as they are authored, without requiring an explicit build.
The most prevalent place where XML comments are used is IntelliSense. The summary content from the XML comment appears in the tooltip. As the method is used, IntelliSense also displays the param content in what is called a parameter help tooltip (see Figure 6).
Figure 6 Parameter Help Tooltip Using XML Comment
Exploring members in the Object Browser is a much improved experience with XML comments. Object Browser displays summary, param, return, remarks, typeparam, and exception comments when they exist (see Figure 7). Class Designer, Class View, and Object Test Bench also use XML comments when they are available.
Figure 7 Object Browser Using XML Comments

The Advantages of Commenting Someone Else's Code
Earlier I demonstrated how to customize the Visual Studio experience for a function by adding XML comments to its definition. For members defined in referenced assemblies, however, it is not practical to follow the same process since you don't necessarily have access to the source. Fortunately, there is a different process (also involving XML comments) that you can use for referenced members!
There is one key difference. For members in the current solution, Visual Studio features work against an in-memory representation of the XML comments based on the source. In that case, the XML documentation file is purely an output and doesn't even need to be generated for the Visual Studio features to work. On the other hand, in the case of referenced assemblies, the XML documentation files are read as inputs, and do influence behavior within the coding environment.
Let's look at an example. I'll create a new project and reference the assembly that I built earlier (RegLib.dll). The generated XML documentation file (RegLib.xml) must be placed beside the assembly and must have the same name. If I access the RegKeyExists method from the current project, it will appear in IntelliSense. However, I can alter its appearance.
I open RegLib.xml and change the summary content to "Determines whether the registry key exists." The update happens immediately, and the next time I access the member in IntelliSense, I see the new text in the tooltip (see Figure 8).
Figure 8 IntelliSense Tooltip

Tricks from the .NET Framework
The .NET Framework is another example of assemblies that you reference from your projects. Consider Microsoft.VisualBasic.dll, whose XML documentation file can be found here:
C:\Windows\Microsoft.NET\Framework\v2.0.50727\en\Microsoft.VisualBasic.xml
Open the XML file and examine the XML comments. There are some interesting XML elements you'll find in there. For example, Filterpriority determines how a member appears in Visual Basic IntelliSense. A value of 1 means that it should appear in the Common tab, 2 means it should appear in the All tab, and 3 means it should be hidden from IntelliSense completely. This member's filterpriority is set to 1, so it will appear in Common. You could easily change the filterpriority value to 2 and save the XML file so the member appears in the All tab.
Note that, before editing any .NET Framework XML documentation files, it's a good idea to make a copy beforehand. You will also need to open the files in an application that has elevated privileges. Visual Studio is a good option since it will preserve the file format.
You can use the methodology described in this section to influence members in a referenced assembly, as long as you have access to its XML documentation file or you can create one if it doesn't exist. To influence the filterpriority of a member defined in the current assembly, use the System.ComponentModel.EditorBrowsable attribute.
Another interesting element is PermissionSet. This element specifies under what conditions the member is accessible. The content of the element refers to the System.Security.Permissions.SecurityPermission type.
Let's lower our current permission level and see what effect this has on our access to FileWidth. Create a Console or Windows Application. Click on My Program in Solution Explorer to open the Project Designer. Select the Security tab, then check "Enable ClickOnce Security Settings" and "This is a partial trust application."
At this point the required permissions are not available, so FileWidth and neighboring members become grayed out and inaccessible in IntelliSense. This Visual Basic feature is called IntelliSense-In-Zone. The tooltip indicates which permission type is required to use the member.
You can add PermissionSet elements to member XML documentation files and specify the appropriate permission.

Generating Nicer Documentation
The XML generated by the Visual Basic compiler is readable, but not the most user-friendly. A number of tools have been created to create nicer looking documentation that is easier to navigate.
The first tool is called Sandcastle, developed by Microsoft. After installing Sandcastle, collect the assemblies that you will be documenting, their dependencies, and their XML documentation files. Copy all of those materials to the vSandcastle installation folder (typically C:\Program Files\Sandcastle\Examples\sandcastle).
Open an elevated command prompt and navigate to the same directory, then run this command:
build_Sandcastle.bat prototype <your assembly name without the file extension>
Directories and files will be generated by this operation. Open the .chm directory, and then open the .chm file inside. It should look something like the help in Figure 9.
Figure 9 Sandcastle .chm Output
The tool offers a number of other output formats other than .chm. For more information and further announcements on Sandcastle, please watch the blog at blogs.msdn.com/sandcastle.
NDoc is another popular tool you might consider using, or you could use the powerful XML features in Visual Basic 9 to write your own custom tool!

XML Comments in Visual Studio 2010
Looking forward, there are new possibilities for the way that you might interact with XML comments in code. The Visual Studio 2010 editor is rewritten using Windows Presentation Foundation (WPF) and allows for rich visualizations. The editor updates also involved moving to a new component system, the Managed Extensibility Framework (MEF), which makes it much easier to register add-ins with Visual Studio. Figure 10 shows an example of an XML comment viewer that was built on top of the new editor and component model.
Figure 10 XML Comment Viewer in Visual Studio 2010
Clicking on the button in the top-right corner switches the display between the WPF control and the original XML. The WPF control certainly provides a much clearer view. It also takes advantage of WPF capabilities such as gradients and round edges. WPF makes it possible to include images or videos, too! This will certainly be a rich area for third-party add-ins to take advantage of in the Visual Studio 2010 release.

Send your questions and comments to instinct@microsoft.com.

Lisa Feigenbaum is the Microsoft Program Manager for the Visual Basic Community. She has been a member of the Visual Basic team since 2004. Before her current role, Lisa was the Program Manager for the Visual Basic Editor and Debugger, working on features such as IntelliSense, edit-and-continue, and code snippets. You can find Lisa at various U.S. and international conferences and user groups, watch her Channel 9 webcasts, or read her posts on the VB team blog at http://blogs.msdn.com/vbteam.


CLR Inside Out
Understanding The CLR Binder
Aarthi Ramamurthy and Mark Miller
Contents
Always Use Fully Specified Assembly Names
Avoid Partial Binds
Use Fusion Log Viewer
Understand the Context
When All Else Fails, AssemblyResolve!
Know When to Use the GAC
The CLR Binder is responsible for locating necessary assemblies at run time and binding to them, so it's an important piece of .NET code. To ensure that binding is working efficiently and correctly, there are a few best practices you should follow, and we'll present them here. Some practices are simple, yet crucial. Proper assembly naming is one such task, and we'll tackle it first.

Always Use Fully Specified Assembly Names
Since the distinction between assembly and file names is often a source of confusion, let's take a closer look. A filename is the name of a file in the filesystem (such as System.dll). An assembly name, on the other hand, is a name given to an assembly to establish its unique identity. In managed code, assemblies provide identity to the code that resides in it. Two assemblies representing the same identity can have the same name (and different versions, signatures, and so forth). A filesystem is simply one of the locations from which to load assemblies—assemblies can also, for example, be loaded from byte arrays. It's best to keep the file name the same as the assembly name. While the most obvious reason is convenience, it is also because assemblies are mostly loaded by assembly names and keeping the names consistent makes it easier for the loader to find the assembly. A fully qualified assembly identity consists of four fields: the simple name of the assembly, the version, the culture, and the public key token.
One way to make effective use of the Binder and its various features is to avoid partial binds (unless you really know what you're doing). A partial bind occurs when the user specified only part of the assembly identity. For example, let's assume that the user tries to load an assembly whose simple name is MySampleAssembly as follows:
Assembly.Load("MySampleAssembly");
In this case, the user failed to specify the other three fields that are a part of the assembly's identity. This is a partial bind.
Another instance is when the user loads the assembly using Assembly.LoadWithPartialName(). Assembly.LoadWithPartialName also uses partial binding. It exists for an extremely specific scenario and should not be used for normal binding. This is why it is marked obsolete in current versions of the framework.

Avoid Partial Binds
Partial binds are a problem because they can lead to nondeterministic Binder behavior, since the Binder does not have the complete information to load the correct assembly. In the case of LoadWithPartialName(), the Binder simply tries Load() and if that fails, it then picks up the highest version of the assembly in the GAC. This may not be the version that is compatible with the current application. If a servicing update of a different application installs a higher version of this assembly in the GAC, one with which the current application is not compatible, LoadWithPartialName() will choose this newer assembly and load it, potentially breaking the current application. Also, not specifying important attributes such as the public key token can lead to binding and loading the incorrect assembly since there is no guarantee that the assembly was provided by the expected publisher.
So, if partial binds are bad, you're probably wondering why they are supported in the CLR in the first place. Well, while LoadWithPartialName() is deprecated in CLR 2.0, loading an assembly with partially specified reference is still supported. Partial binds can be advantageous if their purpose is well understood, and they are used judiciously. At the very least, you should provide the public key token of the assembly to load. Otherwise, this process may return an assembly from a completely different publisher.
Also, if you need to load a particular assembly multiple times or do not wish to hard-code assembly versions into your strings, you can specify the fully qualified assembly name as a part of the <qualifyAssembly> element in the application configuration file and then specify only a partial reference for the assembly in Assembly.Load(). This keeps the code simple, and at the same time ensures that the desired assembly gets loaded. It is to be noted that in such cases each application should have its own application configuration file containing the <qualifyAssembly> element.
In general, it is always safer to specify the fully specified reference of the desired assembly to ensure that binds are predictable.

Use Fusion Log Viewer
There are times when assembly binding fails, typically with an exception (usually, FileNotFoundException, FileLoadException, or BadImageFormatException). Often, a little insight into the internals of the Binder can help you debug the issue at hand.
The Microsoft .NET Framework SDK includes a tool called Assembly Binding Log Viewer (fuslogvw.exe), often referred to as Fusion Log Viewer. This tool logs specific Binding steps in .html files that can be viewed using the Fusion Log Viewer's user interface.
Figure 1 contains a code snippet that demonstrates loading an assembly that does not actually exist. When this code is executed, a FileNotFoundException is thrown. If the user has turned on the ability to log binding failures, the Fusion Log Viewer logs the failure.
Figure 1 Some Misbehaving Code
using System;
using System.Reflection;

class FusLogSample
{
    public static void Main()
    {
        try
        {
            Assembly.Load("TheNonExistentAssembly");
        }
        catch (Exception e)
        {
            Console.WriteLine(e.ToString());
        }
    }
}
By default, logging to disk is turned off, since logging is generally expensive and causes the running application to take a performance hit. To debug the bind failure, click on the Settings button and then turn on "Log bind failures to disk." To turn on logging for all binds, select "Log all binds to disk."
Clicking on the highlighted text in the resulting dialog box, leads you to the actual log itself (as shown in Figure 2). The first two lines tell you that the attempt to load the assembly failed and provides the HRESULT. The next two lines indicate where the CLR was loaded from (the specific directory) and the name of the executable that caused the assembly load to be initiated.
Figure 2 Fusion Log Generated While Executing Code in Figure 1
*** Assembly Binder Log Entry  (2/23/2009 @ 12:29:19 PM) ***

The operation failed.
Bind result: hr = 0x80070002. The system cannot find the file specified.

Assembly manager loaded from:  C:\Windows\Microsoft.NET\Framework\v2.0.50727\mscorwks.dll
Running under executable  \\TKZAW-PRO-15\MYDOCS5\aarthir\My Documents\Visual Studio 2008\
 Projects\Sample\Sample\bin\Debug\Sample.vshost.exe
--- A detailed error log follows.

=== Pre-bind state information ===
LOG: User = REDMOND\aarthir
LOG: DisplayName = TheNonExistantAssembly
 (Partial)
LOG: Appbase = file://TKZAW-PRO-15/MYDOCS5/aarthir/My Documents/Visual Studio
 2008/Projects/Sample/Sample/bin/Debug/
LOG: Initial PrivatePath = NULL
LOG: Dynamic Base = NULL
LOG: Cache Base = NULL
LOG: AppName = NULL
Calling assembly : Sample, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null.
===
LOG: This bind starts in default load context.
LOG: No application configuration file found.
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v2.0.50727\config\machine.config.
LOG: Policy not being applied to reference at this time (private, custom, partial, or location-based assembly bind).
LOG: Attempting download of new URL file://TKZAW-PRO-15/MYDOCS5/aarthir/My Documents/Visual Studio
2008/Projects/Sample/Sample/bin/Debug/TheNonExistantAssembly.DLL.
LOG: Attempting download of new URL file://TKZAW-PRO-15/MYDOCS5/aarthir/My Documents/Visual Studio
2008/Projects/Sample/Sample/bin/Debug/TheNonExistantAssembly/TheNonExistantAssembly.DLL.
LOG: Attempting download of new URL file://TKZAW-PRO-15/MYDOCS5/aarthir/My Documents/Visual Studio
2008/Projects/Sample/Sample/bin/Debug/TheNonExistantAssembly.EXE.
LOG: Attempting download of new URL file://TKZAW-PRO-15/MYDOCS5/aarthir/My Documents/Visual Studio
2008/Projects/Sample/Sample/bin/Debug/TheNonExistantAssembly/TheNonExistantAssembly.EXE.
LOG: All probing URLs attempted and failed.
Under the "Pre-bind state information" section, the first line indicates the user name under which the code was executed. The next line provides the identity of the assembly. Note that the log indicates "partial" because the reference was only partially specified.
The next line shows the ApplicationBase directory (the directory under which the executable is present) for the application domain.
The next four fields (Private Path, Dynamic Base, App Name, and Cache Base) are application domain-specific properties that affect binding in different ways. For example, the privatePath attribute specifies subdirectories of the ApplicationBase directory to search while loading assemblies.
The next line gives you the name of the parent assembly from which Assembly.Load() was initiated. The following line specifies the loader context where this bind began (you'll read more on loader contexts in the next section).
The next three lines deal with policy files—policy files such as application configuration, publisher policy files, and machine configuration files are different means to configure assembly binding behavior (you can read more about these configuration files on MSDN, "Step 1: Examining the Configuration Files)."
The final lines show that the Binder looks for the desired assembly in different subdirectories (within the AppBase). In the end, the Binder declares that the assembly could not be found. Thus, the Fusion log provides information useful for debugging bind failures.
In the .NET Framework 4.0, the Binder explicitly warns about partial binds. Figure 3 shows a snippet from a Fusion log covering partial binds.
Figure 3 Partial Binds in the Log
*** Assembly Binder Log Entry  (1/7/2009 @ 10:17:05 PM) ***

The operation was successful.
Bind result: hr = 0x0. The operation completed successfully.

Assembly manager loaded from:  C:\Windows\Microsoft.NET\Framework64\v4.0.AMD64chk\clr.dll
Running under executable E:\tests\LoggingDCR\PartialNames\LoadPartial.exe
--- A detailed error log follows.

=== Pre-bind state information ===
LOG: User = SampleUser
LOG: DisplayName = Lib
 (Partial)

WRN: Partial binding information was supplied for an assembly.
WRN: A partial bind occurs when only part of the assembly display name is provided.
WRN: This might result in the binder loading an incorrect assembly.
WRN: It is recommended to provide a fully specified textual identity for the assembly,
WRN: that consists of the simple name, version, culture, and public key token.
WRN: See whitepaper http://go.microsoft.com/fwlink/?LinkId=109270 for more information and common solutions to this issue.

WRN: Detected case of partial bind:
WRN: Assembly Name: Lib | Domain ID: 1
The warnings here indicate that partial bind information was specified for the assembly. The Binder also warns about loading the same assembly into multiple contexts, as shown below. We'll cover loader contexts later.
WRN: The same assembly was loaded into multiple contexts of an application domain.
WRN: This might lead to runtime failures.
WRN: It is recommended to inspect your application on whether this is intentional or not.
Overall, fusion logs are a very valuable resource, both to debug bind failures and to understand how the runtime locates and loads assemblies.

Understand the Context
No article on the Binder is complete without addressing loader contexts and the reason for their existence. Loader contexts are often the source of confusion. Think of loader contexts as logical buckets within an application domain that hold assemblies. Depending on how the assemblies were being loaded, they fall into one of three loader contexts.
Load context To put it simply, all assemblies that are present either in the GAC, or in the ApplicationBase, or in the PrivateBinPath under the ApplicationBase, that are loaded using Assembly.Load will be loaded in the Load context. Assemblies resolved using the AssemblyResolve event also fall in this category.
LoadFrom context If you are attempting to load an assembly by providing a specific path that is outside the ApplicationBase, and the assembly would not have been found in the Load context, then the assembly is loaded in the LoadFrom context.
Neither context If you are attempting to load an assembly using Assembly.LoadFile(), Assembly.Load(byte[]), or Reflection.Emit, those assemblies are loaded into the Neither context.
In the case of assemblies loaded into the LoadFrom context, the Binder first checks to see if the exact assembly (same identity and location) is already present in the Load context. If it is, it discards the assembly information in the LoadFrom context and uses the assembly information from the Load context. In determining whether it is the same assembly, the location information is important, and we'll cover this shortly. In .NET Framework 1.1, this was known as LoadFrom's second bind, since the Binder used to perform two steps—first to place the assembly in the LoadFrom context, and then promote it over to the Load context if it found a matching assembly identity and location in the Load context.
Make sure that the assembly is loaded into the Load context as much as possible. For this, the assembly should be locatable from the GAC, the ApplicationBase, or the PrivateBinPath of the AppDomain. Assemblies loaded into this context automatically get benefits of NGen and the assembly's dependencies present in this context are automatically picked up.
Loading assemblies into the LoadFrom context has its own advantages—it allows multiple assemblies outside the ApplicationBase to be loaded by specifying their paths.
Now, let's talk about the location of the assembly, while identifying if the assembly loaded via LoadFrom() is the same as the Assembly loaded via Load(). Even if the types in two assemblies are identical, if the two assemblies are loaded from different paths, they are not considered identical as far as loader contexts are concerned. This leads to situations where the same assembly is loaded repeatedly in the same application domain, but into different contexts (Load and LoadFrom) and a type in the assembly in the Load context will not be allowed to be the same type in the LoadFrom context (even if they are the same assemblies as far as the assembly identities are concerned). This is one of the disadvantages of LoadFrom. Also, assemblies in the LoadFrom context do not automatically reap the benefits of NGen.
As for the Neither context, assemblies in this context cannot be bound to, unless the application subscribes to the AssemblyResolve event. This context should generally be avoided.
So why does the CLR have loader contexts in the first place? Loader contexts help ensure load-order independence while loading assemblies. In addition, they provide a measure of isolation to assemblies and their dependencies when they are loaded into different contexts.

When All Else Fails, AssemblyResolve!
The CLR follows a series of steps (described in the article "How the Runtime Locates Assemblies") to locate and bind to the desired assembly. When the assembly cannot be located at the end of all of these steps, the Binder raises an AssemblyResolve event. In order to load an assembly that could not be resolved earlier, it is possible to subscribe to the AssemblyResolve event.
In .NET Framework 4.0, the CLR is extending the AssemblyResolve event to indicate which parent assembly or RequestingAssembly was causing the load of the dependent assembly. This is very useful in cases where an assembly has a reference to another assembly, and an AssemblyResolve Event occurs for the referenced assembly. As of .NET Framework 3.5, there was no means to determine the identity of the parent assembly (or the referencing assembly).
This means that when the AssemblyResolve event is fired, apart from the current assembly (which could not be located by the Binder by normal probing means), the parent assembly is also provided so that the user knows which assembly caused the load event and the event handler can now make use of the parent assembly that is passed. This makes it easier to leverage the Neither context to provide binding isolation when needed.
The RequestingAssembly field is now another member exposed by ResolveEventArgs.

Know When to Use the GAC
The Global Assembly Cache (known as the GAC) is a machine-wide repository of managed assemblies. The primary goal of the GAC is to enable sharing of assemblies across several managed applications installed on a machine. For example, when writing add-ins you can simply place the common code of the add-ins into one assembly and place this assembly in the GAC. The add-in developer can now ensure that all the add-ins (written by him) share this assembly, instead of having to redeploy the shared component every time a new add-in is installed. This also provides the benefit of central servicing—the add-in developer can now service the common assembly present in the GAC without having to re-deploy servicing updates individually for all add-ins. This is not to say that only add-in developers can make use of the GAC—all managed developers can install assemblies to the GAC and share them across applications.
So when should an assembly be installed in the GAC as opposed to leaving the assembly as a part of the application (within the ApplicationBase)? If you have any assembly that must be shared across multiple applications and hence need to be centrally serviced, you should consider placing them in the GAC. Shared frameworks and components typically fall under this category.
When shouldn't an assembly be placed in the GAC? If you need the application to be xcopy deployable (deployed using the xcopy command to copy the directory containing the application) on different machines, placing the assembly in the GAC is probably not the best idea. In such scenarios, assemblies in the GAC will also need to be moved across machines.
In any case, the Binder follows the "GAC always wins" policy. This might play a role in deciding whether or not an assembly needs to be placed in the GAC.
A short digression: If you aren't familiar already, GACUtil is a tool used to install, uninstall, and enumerate assemblies to and from the GAC. This tool ships as a part of the .NET Framework SDK. You can read more about GACUtil online.
In .NET Framework 4.0, the GAC went through a few changes. The concept of placing assemblies into a global directory began in CLR v1.1. In case of .NET Framework 1.1 (which had CLR v1.1) and .NET Framework 2.0 (which had CLR 2.0), the GAC was split into two, one for each CLR. This avoided the leaking of assemblies across CLR versions. For example, if both .NET 1.1 and .NET 2.0 shared the same GAC, then a .NET 1.1 application, loading an assembly from this shared GAC, could get .NET 2.0 assemblies, thereby breaking the .NET 1.1 application.
The CLR version used for both .NET Framework 2.0 and .NET Framework 3.5 is CLR 2.0. As a result of this, there was no need in the previous two framework releases to split the GAC. The problem of breaking older (in this case, .NET 2.0) applications resurfaces in Net Framework 4.0 at which point CLR 4.0 released. Hence, to avoid interference issues between CLR 2.0 and CLR 4.0, the GAC is now split into private GACs for each runtime.
Tools such as GACUtil and Shfusion will behave exactly the same as they did in pre–.NET Framework 4.0 scenarios. Also, the behavior of publisher policy (explicitly mentioned, since it is required to install these policy files to the GAC) will not change. The main change is that CLR v2.0 applications now cannot see CLR v4.0 assemblies in the GAC.
To conclude, we covered some of the ways in which you can get the best out of the CLR Binder, while also looking at some of the improvements made in the Binder for .NET Framework 4.0.
Send your questions and comments to clrinout@microsoft.com.

Aarthi Ramamurthy is a Program Manager for CLR at Microsoft and primarily works on the the assembly binding and loading aspects of the runtime. She can be reached at aarthi@microsoft.com.

Mark Miller is a Software Development Engineer in Test for the CLR team and works on many of the unmanaged areas of the framework. He can be reached at markmil@microsoft.com.


Cutting Edge
Build Rich User Interfaces with jQuery
Dino Esposito
Contents
The jQuery UI Library
The Sample Scenario
Preparing a Tabbed Pane
Preparing a Dialog Box
Filling the Dialog Box
Initialization Before Display
Posting Data to the Controller
Upgrading the Web
A dialog box is a fundamental element of a graphical user interface. As you probably know, it can be modal or modeless. It is modal if its presence prevents further interaction with the application until the user addresses the question in the dialog box. If the user simply dismisses the dialog box, then no further action will occur that could be triggered by the dialog box.
In Windows development, dialog boxes are a common way of building a user-to-application interaction. In particular, dialog boxes are frequently used to drill down into the content of some displayed items or to edit the content. A familiar pattern in many Windows applications requires the user to click to edit some content and the application displays a modal dialog box with input fields for you to enter updated data.
This pattern has never been popular in Web applications. Browsers have supported pop-up windows forever, but functionally speaking they're not exactly the same as Windows dialog boxes. A pop-up window is simply a child Web browser window lacking some window features such as the address bar or the toolbar. The window displays a Web page, which is great for showing additional content on demand. However, posting data from a pop-up to the server has never been easy for a number of reasons. Add to this that pop-up blockers are usually running on users' machines, preventing the pop-ups from opening. As a result, if you want dialog boxes in your Web app, pop-up windows are not the answer.
In this article, I'll build modal and modeless dialog boxes in jQuery and explain how to post data from them to the Web server.
Like many other libraries and frameworks, the jQuery library doesn't use the browser's pop-up windows to implement dialog boxes. Instead, jQuery relies on script code to pop up a fragment of the Document Object Model (DOM) and dismiss it when the user confirms the content or the abort operation is performed. Because the displayed content is part of the DOM of the current page, there are no postback problems and, more importantly, AJAX capabilities can be implemented.

The jQuery UI Library
Support for dialog boxes is not part of the core jQuery library, whose version 1.3 is now available. To get dialog boxes as well as other nice UI widgets, such as accordion, tabs, and a date picker, you need to resort to an extension of jQuery—the jQuery UI library. More information and download instructions can be found at ui.jquery.com. Some documentation is available at docs.jquery.com/UI.
The jQuery UI library consists of the components listed in Figure 1 and Figure 2.
Figure 1 Components in the jQuery UI Library
Component | Description
Accordion | Renders a collapsible content panel filled with any HTML content you wish
Datepicker | Renders a pop-up date picker to select a date into an input field
Dialog | Renders a modeless or modal floating confirmation dialog box with features that make it look like a Windows dialog box
Progressbar | Renders a progress bar
Slider | Renders an input element that works and looks like a slider
Tabs | Renders a tabbed content pane
Figure 2 Interactions
Interaction | Description
Draggable | Enables you to drag items around the browser window using the mouse
Droppable | Enables you to drop draggable items onto the element that becomes a drop target
Sortable | Enables you to reorder a list of items using the mouse
Selectable | Enables you to select items in a list using either the mouse or keyboard
Resizable | Enables you to resize a displayed item
An interaction is essentially an aspect that can be applied to DOM elements selected in a wrapped set. An interaction represents a cross-cutting behavior and is designed to make target elements more interactive. For example, by combining the Dialog widget with the Draggable and Resizable interactions, you can build a Windows-like dialog box that can be moved around and resized at will.
As the name suggests, widgets are individual components that represent UI components, such as tabs or dialog boxes that are ready to use in your pages.
As mentioned, the jQuery UI library must be downloaded separately from the core library and has dependencies on the version of the core library. This article is based on version 1.5.3 of jQuery UI and version 1.2.6 of the core library. By the time you read this, version 1.7 of jQuery UI, which is based on latest core library version 1.3.2 will be available.
Now, without further ado, let's see an example that combines tabs and dialog boxes. I'll use an ASP.NET MVC sample application. Although the ASP.NET MVC framework ships with the core jQuery library, there's no special dependency between jQuery and ASP.NET MVC. You can achieve the same performance and functionality by using jQuery with classic ASP.NET Web Forms.

The Sample Scenario
Let's consider a sample page with a list of customers. Each listed customer is associated with a couple of buttons through which the user can delete or edit customer information (see Figure 3).
Figure 3 A List of Customers to Edit in Place
The idea is that users click on the Edit button to make changes in place. However, the table in Figure 3 won't swap labels with text boxes as classic server controls like DataGrid and GridView do in in-place editing mode. In this case, a dialog box will pop up showing a prefilled form for you to edit and save.
For simplicity, all of the UI elements are stuffed into the home page of the sample ASP.NET MVC application. To edit the markup of this page, you need to resort to the associated view file. By default, it is index.aspx located under views\home.
At least with the default view factory (based on the Web Forms rendering engine), creating views for an ASP.NET MVC page results in a mix of plain HTML literals and C# or Visual Basic .NET code blocks. HTML helpers exist to make the HTML generation a smoother and quicker process. However, no official HTML helper exists for producing a classic DataGrid without your writing every single line of markup. A few sample HTML helpers for a grid exist. If you're interested, have a look at MVCContrib.
In this example, for old times' sake I used a GridView control to generate the table in Figure 3. (Having written so much about DataGrids in the recent past, let me point out that ASP.NET server controls are mostly gone once you opt for ASP.NET MVC.) The traditional ASP.NET server controls work with ASP.NET MVC, but they should be viewed only as simple HTML factories with no postback capabilities. In ASP.NET MVC, any interaction between the client and the Web server occurs over HTTP calls that target a controller action—a radically different model from Web Forms postback. Figure 4 shows the source code of the home page of the sample application.
Figure 4 The View for the Home Page
<%@ Page Language="C#"
         MasterPageFile="~/Views/Shared/Site.Master"
         AutoEventWireup="true"
         CodeBehind="Index.aspx.cs"
         Inherits="MvcApplication1.Views.Home.Index" %>
<%@ Import Namespace="DataModel" %>

<asp:Content ID="indexContent" runat="server"
             ContentPlaceHolderID="MainContent" >
    <h2><%= Html.Encode(ViewData["Message"]) %></h2>

    <script type="text/javascript">
        $(document).ready(function() {
            $("#AppConsole > ul").tabs();
        });
    </script>

    <div id="AppConsole">
        <ul>
            <li><a href="#tabCustomers"><b>Customers</b></a></li>
            <li><a href="#tabOrders"><b>Orders</b></a></li>
            <li><a href="#tabOther"><b>Other</b></a></li>
        </ul>
        <div id="tabCustomers">
            <span style="font-size:125%"><b>Customers</b></span>
            <hr style="height: 2px;border-top:solid 1px black;" />

            <%
                // Tells a user control to render the grid
                Html.RenderPartial("uc_ListCustomers", ViewData.Model);
            %>

        </div>
        <div id="tabOrders">
            <span style="font-size:125%"><b>Orders here</b></span>
            <hr style="height: 2px;border-top:solid 1px black;" />
        </div>
        <div id="tabOther">
            <span style="font-size:125%"><b>Other content here</b></span>
            <hr style="height: 2px;border-top:solid 1px black;" />
        </div>
    </div>

    <%-- This is the fragment of markup that identifies the
         content of the dialog box --%>
    <% Html.RenderPartial("uc_dlgEditCustomer"); %>

</asp:Content>
The sample page is made of a series of DIV tags, each of which represents a tab. All tabs are linked from a sequence of list-item tags. This markup is transformed into a tab pane by a piece of script code. In particular, the jQuery's tabs() method takes all elements in the wrapped set and renders them as tab panes. The resulting user interface contains three panes—Customers, Orders, and Other.
The Customers tab pane is mostly based on the content generated by the uc_ListCustomers MVC user control. The user control receives data to display from the parent view. Data to display is referenced by the ViewData.Model expression and consists of a List<Customer> object. Needless to say, in this context Customer is an entity class defined in your domain model created with the ADO.NET Entity Framework, LINQ to SQL, or whatever else. The controller action responsible for the home page is in charge of retrieving this data and passing it along, as you see here:
public ActionResult Index()
{
    ViewData["Title"] = "Home Page";
    ViewData["Message"] = "In-place Editing with jQuery UI";

    // Get data to fill the grid
    List<Customer> customers = DataModel.Repository.GetAllCustomers();

    return View("index", customers);
}
To receive the list of customers, the user control must declare its base class accordingly, as you see in Figure 5.
Figure 5 Declare the Base Class
public class uc_ListCustomer : ViewUserControl<List<Customer>>
{
    void Page_Load(object sender, EventArgs e)
    {
        if (!IsPostBack)
        {
           GridView1.DataSource = ViewData.Model;
           GridView1.DataBind();
        }
    }

    void GridView1_RowDataBound(object sender, GridViewRowEventArgs e)
    {
        // Further customize the grid here if required
    }
}
If you use a data-bound server control for generating the markup, then you bind the data to it in Page_Load. Figure 6 shows the markup for the grid with links to JavaScript functions using jQuery.
Figure 6 The Markup for the Grid
<asp:GridView ID="GridView1" runat="server"
     AutoGenerateColumns="false"
     HeaderStyle-BackColor="LightBlue"
     OnRowDataBound="GridView1_RowDataBound">
  <Columns>
    <asp:BoundField DataField="CompanyName" HeaderText="Company" />
    <asp:BoundField DataField="Country" HeaderText="Country" />
    <asp:TemplateField>
         <ItemTemplate>
            <a onclick="return confirm('[<%#Eval("CompanyName") %>]\n
                                    Are you sure want to delete this customer?');"
               href="/home/deletecustomer/<%#Eval("CustomerID") %>">
               <img style="border:0" src="delete.jpg" />
            </a>
          </ItemTemplate>
     </asp:TemplateField>
     <asp:TemplateField>
          <ItemTemplate>
             <img style="border:0"
                  onclick="fnEditCustomer('<%#Eval("CustomerID")%>',
                                          '<%#Eval("CompanyName")%>',
                                          '<%#Eval("ContactName")%>',
                                          '<%#Eval("Address")%>',
                                          '<%#Eval("City")%>')"
                  src="edit.jpg" />
          </ItemTemplate>
     </asp:TemplateField>
   </Columns>
</asp:GridView>
As you can see, each displayed customer is bound to a JavaScript function named fnEditCustomer. This function is responsible for displaying the dialog box and governs further updates. I'll return to this in a moment.

Preparing a Tabbed Pane
To create a tabbed content pane, you call the tabs() method on any element in a wrapped set. All widgets in the jQuery UI library need a DOM subtree to use as a template plus some optional configuration. Typically, you provide the default configuration settings upon page loading and wait to apply other settings later based on the context. The following code shows how to prepare a tabbed pane.
<script type="text/javascript">
    $(document).ready(function() {
        $("#AppConsole > ul").tabs();
    });
</script>
The tabs method works on all <ul> child elements of the element with the AppConsole ID. Each <li> within the <ul> becomes a tab. The <li> must then be linked to a <div> within the page that provides the content.
The tabs method has settings to indicate which tabs are disabled, what happens when the user clicks, and if any tab must be selected upon display. The following code shows how to configure the tab widget to disable the third tab, switch tabs on mouse hovering, and use some animation when switching between tabs.
<script type="text/javascript">
    $(document).ready(function() {
        $("#AppConsole > ul").tabs(
                {
                    disabled: [2],
                    event: 'mouseover',
                    fx: { opacity: 'toggle'}
                }
        );
    });
</script>
In order to successfully use the tab widget in your pages, you need to link the following script files in addition to the core jQuery library file.
<script type="text/javascript" src="ui.core.js"></script>
<script type="text/javascript" src="ui.tabs.js"></script>
The tabs method features a richer API for you to add, remove, or select tabs and download content dynamically via AJAX.

Preparing a Dialog Box
The dialog box has a similar API, based on a method to invoke on a wrapped set and some options to configure it at will.
Unlike tabbed panes, a dialog box needs more work before it can be displayed. Only dialog boxes expected to show some static content can be fully configured in the ready event on the document. Most of the time, though, you want to display a dialog box whose content reflects the current state of the page and perhaps the currently selected item. Let's start with the definition of a jQuery dialog box.
The content of the dialog box is defined in a <div> tag located somewhere in the page. In Figure 4, the content of the dialog box is inserted as a user control in the DOM of the page that will call it.
<!-- This is the fragment of markup that identifies the
         content of the dialog box -->
    <% Html.RenderPartial("uc_dlgEditCustomer"); %>
The dialog template is rendered along with the view when the host page is first loaded. At this time, you may not know the data to display within the dialog. You have basically two options to fill the dialog box with context-sensitive data. First, you pass actual data to the JavaScript function that will actually pop up the dialog. Second, you make an AJAX call to the Web server that retrieves (and ideally caches) any required data just before displaying the dialog box. In this article, I opted for the former approach.
The following JavaScript code runs within the ready event handler for the page DOM and creates and initializes the dialog box, but doesn't display it.
$("#__dlgEditCustomer").dialog(
    {
      autoOpen: false,
      modal: true,
      overlay: { opacity: 0.2, background: "cyan" },
      width: 550,
      height: 400,
      position: [600, 250]
    }
);
As the code shows, you'll get a modal dialog box with a given width and height that displays at a given position with a semi-transparent overlay. The autoOpen=false setting prevents the dialog box from showing up as the page loads.
Some script files are necessary in order to use dialog boxes. At the very minimum, you need the following two script files:
<script type="text/javascript" src="ui.core.js"></script>
<script type="text/javascript" src="ui.dialog.js"></script>
In addition, you need to link the JavaScript files necessary for two common interactions, such as draggability and resizability:
<script type="text/javascript" src="ui.draggable.js"></script>
<script type="text/javascript" src="ui.resizable.js"></script>
Note, though, that the draggable and resizable aspects can be disabled through settings. In this case, you don't need the preceding script files any longer.
The dialog() method creates a dynamic IFRAME and copies the content of the dialog template in it. If the content exceeds the maximum area, scrollbars will automatically appear. The resulting dialog box has a title bar and an X close button in the top-right corner. In addition, a bottom button bar can be configured so that the dialog can offer the usual combination of buttons such as OK/Cancel and Yes/No/Cancel in the context of an overall user experience that is very similar to the desktop. Figure 7 shows the typical style of a standard jQuery dialog box.
Figure 7 A jQuery Draggable and Resizable Dialog Box
The cyan background results from the overlay settings defined in the dialog constructor.
overlay: { opacity: 0.2, background: "cyan" }
The dialog box is placed on top of every other element in the page and a layer separates the dialog box from other clickable elements in the page. Any user clicks outside the dialog box are simply lost. The dialog can be resized by dragging handles on the sides.
The appearance of the jQuery widgets depends on the selected theme. In this example, I'm simply using the default theme for jQuery UI 1.5. In any case, a jQuery theme consists of a bunch of files to put in the project and a single CSS file to reference in the master page. You can get free jQuery themes, and even create your own, from the ui.jquery.com/themeroller Web site.

Filling the Dialog Box
As mentioned, the dialog box is primarily a DOM subtree. Subsequently, to show users an input form with text boxes, drop-down lists, and checkboxes, you start by creating an HTML form and a table-based layout to present input fields with descriptive labels. Figure 8 shows an excerpt of the dialog box's internal markup.
Figure 8 Markup for a Dialog Box
<% Html.BeginForm("UpdateCustomer", "Home"); %>
   <table>
     <tr>
       <td><b>ID</b></td>
       <td><input id="ID" name="ID" type="text" disabled="disabled" /></
          td>
     </tr>
     <tr>
       <td><b>Company</b></td>
       <td><input id="CompanyName" name="CompanyName" type="text" /></td>
     </tr>
     <tr>
       <td><b>Country</b></td>
       <td>
          <%= Html.DropDownList("Country",
                                new SelectList(ViewData.Model) %>
       </td>
     </tr>
     :
   </table>
<% Html.EndForm(); %>
You can use ASP.NET MVC helpers to generate the markup for the <form> tag. The BeginForm helper takes at least two parameters—the controller action to fire on submit and the controller name. It goes without saying that if you want a dialog box to post to the server, then you need to have an HTML form within the dialog.
The form contains input fields laid out the way you like and with the features and styles you prefer. When defining the layout of the dialog box, you don't pay much attention to the actual data to display. The only explicit data you put in the dialog at this time is constant data. For example, each customer has a country name in his record so you might want to display a drop-down list with all countries. In this example, though, I'm making it a bit more restrictive. You can change the country of a customer but you can't specify a new country where you have no other customers. This means that the list of customers must be passed to the user control by its parent.
The parent of the dialog box user control is the Index.aspx view. As you saw earlier, this view receives a list of customers from the controller action. If you want users to pick up a country from a list, then you need to pass the list of available countries along with the list customers. The Index.aspx view class is shown in Figure 9.
Figure 9 The Index.aspx View Class
public partial class Index : ViewPage<HomeIndexViewData>
{
}

public class HomeIndexViewData
{
    public HomeIndexViewData()
    {
        Customers = new List<Customer>();
        Countries = new List<string>();
    }

    public List<Customer> Customers { get; set; }
    public List<string> Countries { get; set; }
}
The HomeIndexViewData is a custom class that groups together all the information to be passed along to the Index.aspx view. Using an ad hoc class results in code that is neater than it would be using the untyped ViewData general-purpose container.
The dialog box will receive the list of countries via the following code:
<% Html.RenderPartial("uc_dlgEditCustomer", ViewData.Model.Countries); %>
To create a drop-down list, you can use the built-in DropDownList helper. In this case, though, you must wrap the enumerable object with the item list into a view specific SelectList object.
<%= Html.DropDownList("Country", new SelectList(ViewData.Model) %>
Figure 10 shows the final dialog box.
Figure 10 The Dialog Box in Action

Initialization Before Display
All that remains to do is explain how you could fit customer data into a dialog box. As you can see in Figure 6, the dialog box gets displayed as the user clicks on an image button in the same row of the grid. The onclick event of the image button is linked to the fnEditCustomer JavaScript function. This function is ultimately responsible for initializing and displaying the dialog box (see Figure 11).
Figure 11 JavaScript Edit Customer
function fnEditCustomer(custID, custName, custContact, custAddress, custCity, custCountry)
{
  $("#__dlgEditCustomer").data('title.dialog', "Customer: [" + custName + "]");

  $("#LastUpdate").datepicker();
  $("#__dlgEditCustomer input[@id=CustomerID]").val(custID);
  $("#__dlgEditCustomer input[@id=CompanyName]").val(custName);
  $("#__dlgEditCustomer input[@id=ContactName]").val(custContact);
  $("#__dlgEditCustomer input[@id=Address]").val(custAddress);
  $("#__dlgEditCustomer input[@id=City]").val(custCity);
  $("#__dlgEditCustomer select > option[@text=" +
             custCountry + "]").attr("selected", "selected");

  $("#__dlgEditCustomer").dialog("open");
}
The first line of code in the body of the function sets the caption of the dialog box to a context-sensitive information. Next, a date-picker widget is attached to the textbox destined to receive a date. Other instructions serve to initialize the various input fields to values passed to the function. You see the true power of jQuery CSS selectors in those lines. Consider the following line:
$("#__dlgEditCustomer input[@id=CompanyName]").val(custName);
The line reads like this: select all input fields within an element named _dlgEditCustomer whose ID property equals "CompanyName" and set their value to the specified content. Needless to say, in this context the use of jQuery is arbitrary. Classic DOM code enriched with facilities from the Microsoft AJAX client library would work as well.
The power of jQuery selectors also becomes evident when selecting an element in a drop-down list when the index is not known in advance. When it comes to editing a customer record, you know the country she lives in; but all you have in your hands is a string. At the DOM level, to make a selection in a SELECT element you need to specify the index. How can you match, say, the string "Italy" to the corresponding index in the list? You can write a loop yourself and find a matching element, or you can rely on jQuery selectors:
$("#__dlgEditCustomer select > option[@text=" + custCountry + "]").attr("selected", "selected");
The expression reads like this: within an element named _dlgEditCustomer, find all OPTION element, child of a SELECT, where the text property equals the value of the specified variable. For any matching element, you set the selected attribute.
Finally, once all initialization work has been done, you display the dialog box. A call to the dialog method with the "open" command string will suffice.

Posting Data to the Controller
The final step is posting data to the Web server and precisely to the specified controller method—UpdateCustomer on the Home controller.
In ASP.NET MVC, you might want to make each URL significant and fully representative of the target resource. To update customer FRANS, you should use a URL such as /Home/UpdateCustomer/FRANS rather than a common /Home/UpdateCustomer URL with some code working out the details from the request body. This means that you should update the form's action URL in the dialog box each time you pop up the dialog. Here's how to do it:
$("#__dlgEditCustomer form").attr("action", "/home/updatecustomer/" + custID);
As you can see, jQuery selectors strike back again. You select the form within an element named __dlgEditCustomer and for that form you change the action attribute to the URL of choice. Figure 12 shows that fresh data is really submitted to the Web server and handled by the right controller method. (Look at Figure 12 and Figure 7 to spot any differences.)
Figure 12 The Dialog Posted Updated Data to the Controller Method

Upgrading the Web
If you want more interaction over the Web, then you need effects, widgets, and aspects; and all rigorously written in JavaScript. The jQuery library has become an extremely popular choice for Web developers all around the world and Microsoft itself ships the latest version 1.3 with the just-released ASP.NET MVC framework.
The jQuery library, though, is in no way bound to ASP.NET MVC. It is simply JavaScript, so wherever JavaScript is accepted, the library is OK. In the past two installments of this column at "Explore Rich Client Scripting With jQuery, Part 2" and "Explore Rich Client Scripting With jQuery, Part 1," I covered the basics of jQuery, including selectors, wrapped sets, events, and methods. This month, I focused on a separate leg of the library—the jQuery UI library—that specializes in widgets and interactions. This month you saw how jQuery makes building modal dialog boxes in Web pages much easier.

Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is an architect at IDesign and the coauthor of Microsoft .NET: Architecting Applications for the Enterprise (Microsoft Press, 2008). Based in Italy, Dino is a frequent speaker at industry events worldwide. You can join his blog at weblogs.asp.net/despos.


Extreme ASP.NET
The Life And Times of an ASP.NET MVC Controller
Scott Allen
Code download available from the MSDN Code Gallery
Browse the Code Online

This article is based on a prerelease version of the ASP.NET MVC Framework. Details are subject to change.

Contents
All Routes Lead to a Factory
Factory Extensibility
Execution Is Just the Beginning
Selector Attributes
Filter Attributes
Custom Action Filters
Getting Results
The Action Is Over
Controllers are the lynchpins of the Model View Controller (MVC) design pattern. They are on the front line, first receiving a client's requests, and then translating the requests into instructions for the model where your application's domain logic and data reside. Controllers are also responsible for selecting a view for presenting information to the user.
In this article, I will dissect the ASP.NET MVC framework and look at how controllers work. I'll explain how the framework interacts with your controllers and how you can influence those interactions. I'll look at controller factories, controller actions, action filters, and action results as well.
I'll dig in fairly deeply, so if you are looking for a more general introduction to the ASP.NET MVC framework, see Chris Tavares' article "Building Web Apps without Web Forms."

All Routes Lead to a Factory
It's difficult to talk about the life of a controller without talking about routes. The routing table in an ASP.NET application contains the information necessary for the ASP.NET routing module to extract information from an incoming URL and direct the request to the proper software component. In January's column, I looked at using the ASP.NET routing module with Web Forms ("Routing with ASP.NET Web Forms".) In that column, I built my own routing handler in order to execute a Web form, but the ASP.NET MVC framework provides a routing handler that will eventually direct a request to one of our controllers.
For this MVC routing handler to process requests, you need to configure the routing table during application startup. The default routing configuration provided by the MVC project template lives in the global.asax file and is shown in Figure 1.
Figure 1 The Default Routing Configuration
public static void RegisterRoutes(RouteCollection routes)
{
    routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

    routes.MapRoute(
        "Default",
        "{controller}/{action}/{id}",
        new { controller = "Home", action = "Index", id = "" }
    );

}

protected void Application_Start()
{
    RegisterRoutes(RouteTable.Routes);
}
One of the routing configuration entries in Figure 1 is a route named Default with an URL template of "{controller}/{action}/{id}". This URL template is a pattern the routing engine will use to first see if an incoming URL is a match for this route. A URL that would match such a route would be http://localhost/home/index/. When the routing engine finds a match, it will again use the URL template as a pattern to lift parameters from the incoming URL. In this example, the string "home" becomes the controller parameter because it is in the {controller} position, and the string "index" becomes the action parameter.
The anonymously typed object constructed as the third parameter to MapRoute represents the default values for the routing engine to use if it cannot find a given parameter in the URL. In the case of http://localhost/home/index/, the routing engine doesn't find "id" information in the URL, but it will still pass along an id parameter with the default value of an empty string. The routing engine passes all these parameters to a routing handler via a RouteData object.
It's important to note that the routing engine knows nothing about ASP.NET MVC. The engine's only job is to analyze URLs and pass control to a route handler. The MapRoute method being invoked inside of the RegisterRoutes method of Figure 1 is an extension method provided by the MVC framework. Every route registered with MapRoute is configured to use an MvcRouteHandler object—this is the route handler provided by the MVC framework. As you saw in the January column, it is the route handler's job to find an HTTP handler for a request—that is an object implementing the IHttpHandler interface. In MVC applications, this object will be an object of type MvcHandler, and inside MvcHandler is where processing becomes interesting.
Figure 2 shows the processing flow for a typical MVC request. When control reaches the MvcHandler, the MvcHandler is able to extract the controller parameter from the RouteData produced by the routing module earlier in the processing. The handler will ultimately send this controller parameter, which is a string value, to a controller factory. It is then the factory's responsibility to construct and return a controller. All controllers in an MVC application implement the IController interface.
Figure 2 Control Flow for a Typical MVC Request
The MVC framework provides a default controller factory (aptly named DefaultControllerFactory) that will search through all the assemblies in an appdomain looking for all types that implement IController and whose name ends with "Controller." Thus, if you tell the factory to look for a "Home" controller, the factory can return a newly instantiated instance of a HomeController class regardless of the namespace or assembly it lives in—as long as it implements IController. This behavior is part of the MVC's "convention over configuration" style. There is more to the factory story, but let's first finish with the MvcHandler processing.
Once the MvcHandler has an IController reference from the factory, it invokes Execute on the controller and waits for the controller to work its magic. When execution is complete, the MvcHandler will check if the controller implements the IDisposable interface, and if so, will invoke Dispose on the controller to clean up unmanaged resources.

Factory Extensibility
The controller factory is a key extensibility point in the ASP.NET MVC framework. Although the default factory provided by the framework can find a HomeController anywhere in your solution, it can only instantiate the HomeController if you provide a parameterless constructor. This limitation is a problem for teams who follow the dependency inversion principle and inject a controller's dependencies via its constructor. As an example, consider the EmployeeController (shown in Figure 3), which requires someone to pass a logging component into its only constructor.
Figure 3 EmployeeController
public class EmployeeController : IController
{
    public EmployeeController(ILogger logger)
    {
        _logger = logger;
    }

    public void Execute(RequestContext requestContext)
    {
        // ...
    }

    ILogger _logger;
}
Fortunately, you can create a custom factory. Any class implementing the IControllerFactory interface is a candidate, and you only need to implement CreateController and ReleaseController methods. However, inversion of control containers such as StructureMap, Unity, Ninject, and the Castle project's Windsor are readily available and are a perfect fit for this scenario. In fact, the MVC Contrib project on CodePlex includes IControllerFactory implementations for all of the containers listed above.
If you want to use StructureMap as your inversion of control container, you can reference the StructureMap and MvcContrib.StructureMap assemblies, then write the code shown in Figure 4. The InitializeContainer method in this listing first tells StructureMap to use a SqlServerLogger type whenever an ILogger is required. The code then sets the controller factory for the entire application using the SetControllerFactory method of the current ControllerBuilder. During request processing, the MvcHandler will ask this same ControllerBuilder for the currently configured factory and work with a StructureMapControllerFactory instead of the default framework factory.
Figure 4 Initialize Container
protected void Application_Start()
{
    InitializeContainer();
    RegisterRoutes(RouteTable.Routes);
}

private void InitializeContainer()
{
    StructureMapConfiguration
        .ForRequestedType<ILogger>()
        .TheDefaultIsConcreteType<SqlServerLogger>();

    ControllerBuilder.Current.SetControllerFactory(
        new StructureMapControllerFactory());
}
The StructureMapControllerFactory from the MVC Contrib project does inherit from the MVC framework's default controller factory and will still use the convention I described earlier when locating controller types to instantiate. However, the factory will use StructureMap to instantiate the controller, and StructureMap knows how to work with parameterized constructors. The configuration shown in Figure 4 would be everything you need to process a request for http://localhost/Employee/. StructureMap will instantiate the EmployeeController from Figure 3 by passing in a SqlServerLogger reference.

Figure 5 Class Hierarchy
Execution Is Just the Beginning
Earlier I stated that the MvcHandler invokes the controller's Execute method, waits, and then cleans up. This is because the MvcHandler only knows about controllers through an IController interface. If you wanted to write your application at this level, you could simply derive all your controllers from the IController interface and provide an implementation of the Execute method. However, the ASP.NET MVC framework offers a richer execution model for controllers through the hierarchy of classes shown in Figure 5.
By default, the controllers you add to an ASP.NET MVC project derive from the System.Web.Mvc.Controller class. One way to add a new controller is to right-click on the Controllers folder in Solution Explorer and select Add | Controller, which gives you the dialog box shown in Figure 6. Remember, for the controller factory to find your controller the name must end with "Controller."
The Controller base class introduces the concept of an action. Actions are methods on the controller that serve as the ultimate request destination in an MVC application. Earlier, I pointed out that the ASP.NET routing module will lift out "home" as the controller parameter from a URL of http://localhost/home/index/. This is enough information to route the request to the proper controller. The routing module also picked out "index" as an action parameter. When the MvcHandler tells the HomeController to execute, it is up to the logic composed inside the base Controller class to inspect this action parameter and invoke the proper controller method. The majority of this action routing logic resides inside the public ControllerActionInvoker class.
Figure 6 Add Controller
Figure 7 is the HomeController provided by the ASP.NET MVC project template. The public instance methods, Index and About, represent actions the framework will invoke when the client requests home/index/ and home/about/, respectively. Any public instance method can serve as a controller action as long as the framework can determine for certain that the action is the correct action to invoke (in other words, be careful with method overloading). There are additional rules in play when the framework is searching for an action to invoke. You can influence the framework's choice of actions, establish additional rules for action selection, and manipulate the behavior of your actions using attributes.
Figure 7 HomeController
[HandleError]
public class HomeController : Controller
{
    public ActionResult Index()
    {
        ViewData["Message"] = "Welcome to ASP.NET MVC!";

        return View();
    }

    public ActionResult About()
    {
        return View();
    }
}

Selector Attributes
You can decorate a controller action with selector attributes to give the framework additional information to consider when it is selecting an action to invoke. For example, adding a [NonAction] attribute to a controller method will cause the method to be excluded from the list of available actions. You can also give a method a specific action name. By default, the name of a method is also its action name, but if you put [ActionName("help")] on your HomeController's About method, then "about" is no longer a valid action for your controller. Instead, the About method's action name is "help" and the framework will invoke the About method for requests such as /home/help/.
One particularly important selector attribute is the AcceptVerbs attribute. This attribute only allows the framework to select an action when one of the verbs listed in the attribute matches the verb of the current HTTP request. For example, decorating a method with[AcceptVerbs(HttpVerbs.Post)] means that the method can only be invoked as an action for an HTTP POST operation. It is important to select the proper verb for your controller actions, particularly if the actions modify state on the server. For more information, see Stephen Walther's ASP.NET MVC Tip #46.

Filter Attributes
Action filters are another type of attribute you can place on an action. With action filters you can add caching, validation, and error handling behaviors to your actions using a declarative programming model. An example of a filter attribute is the [HandleError] attribute on the HomeController in Figure 7. You can apply this attribute to individual actions, or add the attribute to the controller class to apply the behavior to all actions in a controller.
When a HandleError attribute is present on an action and the action throws an exception, the MVC framework will look for a view with the name of "Error"—first in the controller's view folder, and then also in the shared view folder. The error view allows you to present the user with a friendly error page. You can also map exceptions to specific views using a more explicit HandleError attribute. For example, [HandleError(ExceptionType=typeof(SqlException), View="DatabaseError")] would map an unhandled SqlException to a view named "DatabaseError." The rest of the action filters provided by the MVC framework are outlined in Figure 8.
Figure 8 Action Filters
Name | Description
OutputCacheAttribute | Similar to the OutputCache directive in ASP.NET Web forms. The OutputCache attribute allows the MVC framework to cache the output of a controller.
ValidateInputAttribute | Similar to the ValidateRequest attribute in a Web form. By default, the MVC framework will inspect an incoming HTTP request for HTML or other dangerous input. If detected, an exception will be raised. You can use this attribute to disable the request validation.
AuthorizeAttribute | The Authorize attribute allows you to place declarative authorization checks on controller actions. The attribute can restrict an action to users in specific roles. You can use this attribute when you create an action that should only be available to users in an Administrator's role.
ValidateAntiForgeryTokenAttribute | This attribute is one half of the solution to help prevent cross-site request forgeries (CSRF). It lets the framework verify presence of a user-specific token in HTTP POSTs. For more on CSRFs, see "Prevent Cross-Site REquest Forgery (CSFR) using ASP.NET MVC's AntiForgeryToken() helper."

Custom Action Filters
You can create your own action filters to surround an action with custom logic. The code in Figure 9 is the code for a simple logging action filter that can write to the Output Window of Visual Studio during debugging. We can apply this attribute to an individual action, or we can place this attribute on a controller class to have logging for all the actions in a controller.
Figure 9 A Logging Action Filter
public class LogAttribute : ActionFilterAttribute
{
    public override void OnActionExecuting(ActionExecutingContext filterContext)
    {
        Log("Action Executing", filterContext.RouteData);
    }

    public override void OnActionExecuted(ActionExecutedContext filterContext)
    {
        Log("Action Executed", filterContext.RouteData);
    }

    public override void OnResultExecuting(ResultExecutingContext filterContext)
    {
        Log("Result Executing", filterContext.RouteData);
    }

    public override void OnResultExecuted(ResultExecutedContext filterContext)
    {
        Log("Result Executed", filterContext.RouteData);
    }

    void Log(string stageName, RouteData routeData)
    {
        Debug.WriteLine(
            String.Format("{0}::{1} - {2}",
                routeData.Values["controller"],
                routeData.Values["action"],
                stageName));
    }
}
As you can see, there are four virtual methods provided by the base ActionFilter class. You can override one or more of these methods to not only pre- and post-process controller actions, but also pre- and post-process the controller action's result. Before an action executes, the OnActionExecuting method will fire, and when the action is complete the OnActionExecuted method will fire (and will fire even if the action throws an unhandled exception). Likewise, the OnResultExecuting method will fire before the result executes and the OnResultExecuted method will fire after.
The context parameter passed into the action filter methods will allow you to inspect the HTTP request, the HTTP context, the route data, and more. Throwing an exception from one of these methods will halt the request processing flow. An exception is a useful tool if you are writing an ActionFilter to check preconditions in the environment.

Getting Results
The successful execution of an MVC controller action will produce an object derived from ActionResult. Rendering a view and redirecting the browser to a new URL are both valid types of results you might want from your controller. The complete list of ActionResult derived types is shown in Figure 10.
Figure 10 ActionResult Derived Types
Name | Framework Behavior | Producing Method
ContentResult | Writes a string value directly into the HTTP response. | Content
EmptyResult | Does not write to the HTTP response. |
FileContentResult | Takes the contents of a file (represented as an array of bytes) and write the contents into the HTTP response. | File
FilePathResult | Takes the contents of a file at the given location and writes the contents into the HTTP response. | File
FileStreamResult | Takes a file stream produced by the controller and writes the stream into the HTTP response. | File
HttpUnauthorizedResult | A special result used by authorization filters when authorization checks fail. |
JavaScriptResult | Responds to the client with a script for the client to execute. | JavaScript
JsonResult | Responds to the client with data in JavaScript Object Notation (JSON). | Json
RedirectResult | Redirects the client to a new URL. | Redirect
RedirectToRouteResult | Renders the specified view to respond with an HTML fragment (typically used in AJAX scenarios). | RedirectToRoute / RedirectToAction
PartialViewResult | Renders the specified view to respond with an HTML fragment (typically used in AJAX scenarios). | PartialView
ViewResult | Renders the specified view and responds to the client with HTML. | View
Note that a controller action never needs to instantiate one of these types directly. Instead, the controller action can invoke the method name shown in Figure 10 to produce the result. These methods are inherited from the MVC controller's base class. It's also worth noting that a controller action doesn't need to return an ActionResult object. If the controller returns something other than an ActionResult, the framework will convert the object to a string and wrap the string in a ContentResult (which simply writes the string into the HTTP response). A controller that returns void will produce an EmptyResult.
The ActionResult class defines an ExecuteResult method that each of the types in Figure 10 will override. This method is invoked by the ControllerActionInvoker—the same object that called the controller's action. When invoked, each result will take care of all the little details needed to successfully deliver the result to the client. For example, the JavaScript result will set the content type header of the response to "application/x-javascript", while the HttpUnauthorizedResult will set the HTTP status code of the response to a 401 (unauthorized).
A common response from a controller action will be a ViewResult. You've seen this result in previous code listings as all of your actions have invoked the View method of the controller and returned the result. This is another example of "convention over configuration," because the ViewResult will, when used in this default fashion, look for a view in the controller's view folder and with a file name matching the action. For example, views\home\about.aspx is the conventional view for the About action of the Home controller. Overloaded versions of the View method allow you to explicitly state the name of the view.

The Action Is Over
This month I've taken a deep look at the abstractions and behavior surrounding ASP.NET MVC controllers. Yo u should now have a good grasp of how the MVC framework finds, creates, and uses controllers, as well as how to hook into the extensibility points of the MVC framework that revolve around controllers. In the next edition of this column, I'll take a look at guidelines and best practices to put these controllers to work in real applications.
Insights: Helper Methods
If you felt strange using the helper methods for action results when they first appeared in the framework (View(), Content()), you might be wondering how it all came about and what was behind that particular design decision.
Well, the story behind the helper methods for returning action results is that 99% of an MVC developer's time writing an MVC application is going to be spent writing controller actions. We wanted to be sure that the typical method is clean, readable, and as declarative as possible.
For example, you can still write an action method like so:
public ActionResult Foo {
    // ... do stuff ...
    ViewData.Model = myModel;
    return new ViewResult {ViewName = "Foo", ViewData = this.ViewData};
}
We wanted to clean this up a bit, so we made some adjustments, as you can see:
public ActionResult Foo {
    // ... do stuff ...
    return new View(myModel);
}
This is a more declarative approach (despite the use of a very imperative language). When you read that action method, it reflects your intention. "I want to return a view containing this model."
--Phil Haak, Senior Program Manager, Microsoft

Send your questions and comments to xtrmasp@microsoft.com.

K. Scott Allen is a member of the Pluralsight technical staff and the founder of OdeToCode. You can reach Scott at scott@OdeToCode.com or read his blog at odetocode.com/blogs/scott.


Foundations
Versioning Workflows
Matthew Milner
Code download available from the MSDN Code Gallery
Browse the Code Online
Contents
The Issues
.NET Versioning
Versioning With XOML Workflow Definitions
Activity Versioning
Versioning Workflow Services
Dynamic Update
In any application, there is one thing you can be certain of: change happens. One of the most common issues I find developers struggling with is how to deal with versioning workflows and their related classes. In this month's column, I will discuss the core issues related to workflow versioning and provide my recommendations for making changes to workflow definitions, activities, and workflow services.

Figure 1 Simple Workflow That May Persist
If you build a .NET application today—say, a Windows Presentation Foundation (WPF) or ASP.NET application—you might think about versioning, or you might not. When you want to deploy updates to an ASP.NET application, you typically have a process in place to simply copy over all or some of the components including pages, binaries, and configuration. In some scenarios, you may have concerns about this deployment, such as making sure no requests are currently executing, but this process is generally manageable. So why is deploying new versions of workflow applications so hard?
Workflows let you model long-running business process or business logic. In addition, Windows Workflow Foundation (WF) provides persistence services that allow for the state of individual instances of a business process to be saved to a durable store, such as a Microsoft SQL Server database. That saved state is made up of serialized .NET objects; and therein lies the problem.
Consider a workflow that uses some .NET types as data—say, an Order class. The workflow is started with an Order object passed in as a parameter, then persisted to the database, its state containing the serialized Order object. Now change the Order object class and rebuild your application library. When that persisted workflow needs to resume because a delay has expired or some event is raised to the workflow, the Order object needs to be deserialized. Unfortunately, because that type has changed, the deserialization fails, throwing an exception.
Some changes cause problems with deserialization because the standard .NET binary serialization fails and other problems occur because of the WF implementation of serialization. Regardless of the cause, the end result is the same: an exception when you try to load a workflow that references types that have changed. These types might be classes or interfaces you create or they might be custom activities you have written.
To provide a concrete example, consider the workflow in Figure 1. It simply writes some data to the console using a custom WriteLine activity, delays (which also allows it to persist), and then writes more data to the console.
Now, if I start an instance of this workflow and let it run to the idle point—the delay—it will persist in the database. I can shut down the runtime and begin work on my version 1.1 of the workflow. I'll simply add an activity to the end of the workflow definition to write more data to the console, as shown in Figure 2.
Starting up the runtime without starting another workflow instance will cause the persistence service to poll the database looking for expired timers. When it finds the existing instance, the persistence service will attempt to load the object and will get an exception. I can detect this error by registering an event handler for the ServicesExceptionNotHandled event on WorkflowRuntime. What I'll see is that the persistence service will raise an event for the exception with the message "Index was outside the bounds of the array."
Figure 2 Modified Workflow Definition
The problem here is that I've changed the definition of my workflow type and it no longer matches the definition that existed when the workflow was persisted. When the persistence service tries to deserialize the workflow, I get exceptions because the new type definition contains fields that did not exist when the instance was serialized. The deserialization code is reading the fields on the new type and attempting to load those values from the serialized data. Because some fields did not exist when the instance was serialized, an exception is raised when attempting to access that data.

.NET Versioning
One way to work around the versioning issues is to take advantage of the versioning system built into the .NET Framework. Since version 1.0 of the .NET Framework, versioning of assemblies has been supported and was the only way to enable side-by-side support for multiple versions of assemblies. In short, each assembly can have a version number attached to it and when assemblies are signed with a key, multiple versions of the assembly can be deployed to the Global Assembly Cache (GAC). For more on assembly versions and strong naming assemblies, see the MSDN documentation on assemblies.
This enables you to build your workflows and activities into projects with a version of 1.0, sign them, and deploy them to the GAC. Now, when you start a workflow and it persists, the version information is part of the serialized state. If you change the workflow, it is important to change the version of the assembly and any other assemblies the workflow depends on if those have changed, too. For example, if you had the workflow from the previous example and you made a change to the WriteLine activity and the workflow definition, then you would need to increment the version number for each project and deploy the updates to the GAC.
At this point, you would have both version 1.0 and version 1.1 of the workflow deployed into the GAC. When a version 1.0 workflow is loaded from the persistence store, the runtime will be able to resolve the type definition for version 1.0 and use that type information when deserializing the workflow. Any version 1.1 workflows that were also persisted would work since the runtime would be able to find the definition for the version 1.1 types, too. Essentially, as long as the assemblies containing the types for the workflow and its related classes can be resolved and match the original definition, the workflow will deserialize correctly and processing can continue. You can see how this process works in Figure 3 where the different versions are loaded from the database, and their types resolve correctly when found in the GAC.
Figure 3 Side by Side .NET Versioning
One important point to notice is that version 1.0 workflows will continue working and processing, but they are still based on the version 1.0 definition of the workflow. This means any changes you introduced in version 1.1 of the workflow will not be present in the workflow definition. In other words, any activities you added, removed, or otherwise changed will not be present in the workflow definition and you will not see the impact of those changes in the version 1.0 workflows. Shortly, I will discuss a feature known as dynamic update that provides the ability to change the existing, in-process, version 1.0 workflows.
Simply having side-by-side versions of the workflows and related assemblies does not mean that things roll along nicely, however. Your host application is likely built against a particular, and probably the most recent, version of the workflows so that it can interact with them. This can cause problems as a host built against version 1.1 of the workflow will encounter problems when attempting to interact with persisted version 1.0 workflows that were built against version 1.0 of the related classes.
One area where this trouble often comes up for developers is when using ExternalDataExchangeService in conjunction with HandleExternalEvent and CallExternalMethod activities for local communications. Many people, thinking they are doing the right thing for versioning, follow the previous steps so that both the 1.0 and 1.1 versions are available. However, when the host attempts to send data to the workflow through an event, the type of the interface being used no longer matches. The host is using version 1.1 of the interface, while the workflow instance was built against version 1.0.
To fully understand why this is a problem, recall how communications really work in WF. (For more on workflow communications see the September 2007 installment of Foundations.) The HandleExternalEvent activity creates a queue to receive data and the name of that queue includes the type information for the interface. When data is sent from the host, it must be sent to the correct queue. ExternalDataExchangeService (EDS) uses the type information and the event name to create what it believes is the appropriate queue name. Unfortunately, since the queue was created based on version 1.0 of the interface and the EDS is creating the queue name based on version 1.1, the two IComparable objects will not match and the queue will not be found.
To fully work around this problem, your host application needs to raise events on the version 1.0 interface. Two possible ways to accomplish this include reflection and inheritance. The first option involves using reflection to load the original version of your local service and add it to the EDS, after which you raise an event on that interface, again using reflection.
The second option involves building a new assembly where you create version 1.1 of your interface and have it derive from the version 1.0 interface, allowing you to add events to the derived type while not changing the base type. In this way, if you raise an event that was defined on version 1.0 of the interface, the correct queue name will get created and the message delivered even if you invoke that operation through the derived type. The key is that version 1.0 of the interface must remain deployed and unchanged, and your new interface and service must derive from the version 1.0 interface. Figure 4 shows this approach.
Figure 4 Derived Interfaces for Local Communications
Notice that only the derived service and interface are added to the EDS. This enables the host to interact with a single service but to raise events based on multiple interfaces. The goal is to keep the version 1.0 interface around and in use, so the EDS creates queue names that match what the workflows will be creating. Also notice that this involves creating a whole new project rather than just creating a new version of the communication library. Untenable is the word that comes to mind when thinking about using this approach long term.
Generally, my recommendation around communications is to avoid the local communications activities for this very reason and use custom activities that create queues based on simpler names, such as strings, removing much of the type dependence between the workflow and the host. The other suggestion is to use workflow services that provide additional capabilities around receiving requests and routing them to the correct version of the workflow, as I will discuss shortly.

Versioning With XOML Workflow Definitions
The first step to making life easier when it comes to versioning is to use XOML-based workflows without compiling them. This truly declarative model provides many benefits in addition to versioning. See the May 2008 issue of MSDN Magazine for details ("Loading Workflow Models in WF"). I will focus on the benefits for versioning here.
When building workflows via a code approach, you are defining a new type. For example, if you create a new code workflow using the Visual Studio project or item template, you are defining a new class that derives from a type in the WF assemblies. This is the primary reason for the issues described previously because when you change the workflow by adding or removing activities, you are changing the type. This does not happen when you build XOML workflows.
A XOML workflow that is not compiled is simply an XML document that describes a collection of existing types. This may seem like a subtle difference, but describing existing types instead of creating a new type makes a big difference.
Here is a XOML workflow definition:
<SequentialWorkflowActivity x:Name="Workflow2"
    xmlns:ns0="clr-namespace:SimpleWorkflows"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/workflow">
  <ns0:WriteLineActivity x:Name="Hello" OutputText="Hello" />
  <DelayActivity TimeoutDuration="00:00:03" x:Name="SmallDelay" />
  <ns0:WriteLineActivity x:Name="World" OutputText="World" />
</SequentialWorkflowActivity>
You can see that the root element is SequentialWorkflowActivity, which is defined in the System.Workflow.Activities assembly. This type is both versioned, as 3.0, and nonchanging. If you build a workflow using this type as your root activity and the workflow is serialized and persisted, you can be assured that, when the workflow is loaded and deserialized, the runtime will be able to find version 3.0 of the SequentialWorkflowActivity in the GAC.
What about changing the workflow definition by adding or removing activities? In a XOML workflow definition, you are simply changing some XML, which does not change any types. Even if you change the root element in the workflow definition to some other type, you are not changing a type on which the existing workflow instances depend. In fact, once a workflow is created from the XOML, that XOML would not be referenced again by the runtime for the life of that instance.
Using XOML-based workflows means that you do not have to version your workflows because you are not defining types. Because you do not have to strongly name and version your workflows, you also do not have to strongly name and version the other types, such as local communications interfaces and business entities. It is much easier to make nonbreaking changes, such as adding members or methods to types, without much work in your application.
The downside to working with XOML workflows is that you have little to no access to code, other than what is encapsulated in your activities, so you end up writing more activities. But since activities are simply classes, this is not a major blocker and allows you to capture methods that can be used to provide event handling logic or condition checking for activities in a XOML workflow.
Additionally, since the SequentialWorkflowActivity class does not have any properties of use to your business case defined on it, you won't be able to pass parameters to the CreateWorkflow method. Fortunately, you can either create your own root activity type, deriving from one of the included workflow types, and add your required properties there, or begin your workflow with a receive activity or custom activity so you can send in initial data right after starting the workflow.
Finally, looking forward, fully declarative workflows based on XAML are the primary modeling option coming in WF 4.0. Many improvements are being made to the capabilities, including replacing events with what are currently being called an ActivityAction. Essentially, these will provide you with a way to model delegates using activities and have the activity execute those activities rather than invoke a delegate.

Activity Versioning
Workflows are activities, so it follows that many of the issues discussed so far with workflow types apply to activity types as well. In fact, just as a workflow type causes problems if you add activities, if you add properties to an activity type and do not version that activity assembly, you will get exceptions upon deserialization of the workflow. Unfortunately, activities need to be classes and must be compiled types, so there is no easy XOML solution as with workflows.
The short answer for activities is that you should version them using .NET versioning techniques when you change the interface for the activity in any way. This enables existing workflows and new workflows to reference the correct version and execute correctly. Keep in mind, however, that if you are fixing a defect in a method within the activity that does not change the interface, you can make the fix and deploy the assembly with the same version number. When that activity is loaded and executed by existing or new workflows, the correct code will run.
The long answer involves a deep understanding of the serialization mechanics in the .NET Framework, which is beyond the scope of this article. However there are two keys things I can share that may provide the most benefit with the least amount of investment.
First, if you are adding properties or fields to your activity, you can mark the fields with the NonSerialized attribute to avoid the Index Out of Bounds exception upon deserialization of workflows built against the previous version of your activity.
Second, you can override the OnActivityExecutionContextLoad method to initialize any state that was not automatically deserialized. This method gets called when the activity is recreated from persisted state or when a new execution context is created. Both of these techniques together can help you make changes to activities without requiring that you strongly name them and change version numbers.

Versioning Workflow Services
Workflow services provide similar challenges for local communications as well as additional opportunities for decoupling messages from workflows. Because a workflow service is built using a service contract, any versioning of the service or data contracts means the workflow will need to be rebuilt against the new versions of the contracts. When a message arrives at a service endpoint, it will be received by a service using a contract that has a particular version. If the workflow for which the message is intended was built against the same version of the contract, then things work well. However, if the workflow was built against the earlier version of the contract and then persisted, then the message cannot be delivered. To support two versions of the contracts, you will need to deploy two versions of the service.
While it is unfortunate that you cannot take full advantage of the versioning capabilities inherent in Windows Communication Foundation (WCF) that would enable a new version of the service to process messages from clients built against the old version, WCF does provide a mechanism to manage this scenario. At the level of WCF, clients and services are simply exchanging messages and .NET types do not cross the wire. The versioning issues I am speaking about are purely at the .NET level and can be hidden from clients using a messaging router.
The benefit with workflow services is that it is possible to publish two services side by side, with different endpoint addresses, yet publish a façade that makes both services accessible at the same address. Using this approach, when you deploy version 1.0 of your service, you can deploy it behind a pass-through WCF router that simply takes in messages and passes them on to the service. When you are ready to deploy version 1.1 of the service, you can update the routing information so that messages get sent to either the version 1.0 service or the version 1.1 service, depending on the message or the context in which it was received. Figure 5 shows this router concept with the two versions of a service behind a router and a client interacting with them.
Figure 5 Using a Router for Versioning Services
The key to ensuring that a solution with a router will work is to deploy it with version 1.0 of your service. Then when you move to the next version, the router is not something new you are adding to the deployment and testing process. The .NET Framework 3.5 SDK provides a sample router, and you can also follow a hands-on article about using WF rules to make routing decisions.

Dynamic Update
After all the discussion about how to best manage changing the definition of workflows and dealing with new and old versions running side-by-side, there is often a question about how to handle those old workflows. You made a change to your workflow for a reason. Perhaps it was to fix a defect in logic, or maybe you had compliance regulations that needed to be followed. Whatever the reason, it is often the case that allowing all of the existing workflows to complete as they were defined is not a tenable outcome. In those cases, you might consider taking advantage of a powerful feature in WF: dynamic update.
In short, dynamic update allows you to make changes to a running workflow instance. Those changes can consist of adding or removing activities from the workflow structure. That is all you can do, but it can completely change the definition of your workflow. For example, in a state machine, you have the ability to add entire states, along with the events they listen to and the transitions defined. You can change state transitions by removing and re-adding SetState activities. In a sequential workflow, you can add or remove steps as your business process changes.
To make changes to a workflow, you first need to create a WorkflowChanges object based on the workflow. At that point, the WorkflowChanges object provides a clone of the workflow structure that you can manipulate through its TransientWorkflow property. You make changes to the clone by adding and removing activities. When your changes are complete, you call the ApplyChanges method on the WorkflowInstance class, passing the WorkflowChanges to represent the changes you want to apply to the actual instance.
Here is the example workflow shown earlier, which contained two WriteLine activities separated by a delay activity, with an additional WriteLine activity added to the end of the sequence. This code runs when the workflow goes idle, or when the delay begins execution.
WorkflowChanges changes =
  new WorkflowChanges(instance.GetWorkflowDefinition());
CompositeActivity root = changes.TransientWorkflow;
WriteLineActivity wl = new WriteLineActivity {
  Name = "newWriteLine",
  OutputText = "dynamically added"
};
root.Activities.Add(wl);
instance.ApplyWorkflowChanges(changes);
This simple example shows what is possible, but much more advanced scenarios can be built to apply many different changes to a running workflow. This allows you to take a version 1.0 workflow that has already been started and make changes to the instance so that it works more like a version1.1 workflow.

Send your questions and comments to mmnet30@microsoft.com.

Matt Milner is a member of the technical staff at Pluralsight, where he focuses on connected systems technologies (WCF, Windows WF, BizTalk, "Dublin" and the Azure Services Platform). Matt is also an independent consultant specializing in Microsoft .NET application design and development. Matt regularly shares his love of technology by speaking at local, regional, and international conferences such as Tech Ed. Microsoft has recognized Matt as an MVP for his community contributions around connected systems technology. Contact Matt via his blog at pluralsight.com/community/blogs/matt.


Going Places
Gaming In The Key Of Zune
Mike Calligaro
Contents
Man, That C Is Sharp
Anatomy of a Game
Sprites on Pixie Dust
Of Asset Names and Color Keys
Ya Gotta Move To Play
Sprite in a Box
Bing Bang Boom
Advanced Drawing
The Rest Is Up To You
My son, who's been playing Xbox since he was three, has started showing an interest in programming—specifically writing games. This prompted me to download XNA Game Studio and dive into it myself, you know, so that I could teach him. That I've always wanted to write my own games had nothing to do with it. Really.
XNA Game Studio 3.0 added the ability to write games for Zune, which means the Zune is now a mobile development platform. So it's fair, er, game for a Going Places article. If you're already an XNA developer, I may be reviewing some concepts you already understand. But, if you're like me, a programmer who is enthusiastic about but fairly new to game development, read on.

Man, That C Is Sharp
For the majority of my fifteen year career at Microsoft, I've been a systems and drivers developer. My language of choice and necessity has been a fairly bare bones C++. I rarely get to use runtimes like MFC and the Microsoft .NET Framework. Until recently, I couldn't have even spelled STL much less made use of it.
That's my day job. But in my off time I love playing around with C#. Mostly I write little apps for my Windows Mobile devices and occasionally for my PC. C++ doesn't treat me as badly as it does many native developers, but I still I get a little giddy when I write in C#. You can get a lot done with very few lines of code. I swear, C# is so much fun they should make it a controlled substance.
In the fun category, XNA Game Studio is like C# on steroids. The team over there has done an amazing job of making game development easy. The framework is straightforward, the hard stuff is largely handled for you, and they've released a ton of samples aimed at teaching you the various aspects of game development.
Start at creators.xna.com. From there you can download the free XNA Game Studio 3.0. If you already use one of the various incarnations of Visual Studio 2008, XNA Game Studio will integrate with it. If you don't have Visual Studio 2008, don't fret. XNA Game Studio also works with the free Visual C# Express Edition. (In other words, although I mention Visual Studio, you can substitute Visual C# Express Edition if that's what you're using.)
The creators.xna.com Web site is also full of great information to get you going. Click the Education link at the top of the page to find beginner's guides, samples, and how-to's. The "Beginner's Guide to 2D Games" is especially good, as is the documentation that gets installed with XNA Game Studio. In some cases, the installation documentation has information that's not on the Web. In Visual studio, you can get to that documentation by selecting Help | Contents and setting the filter to XNA Game Studio 3.0.
XNA Game Studio lets you write one code base and deploy it to Xbox, PC, and Zune. Everything I do here will work on all three platforms. The free downloads are all you need to develop for PC or Zune, but Xbox development requires a Premium membership that costs a yearly fee.

Anatomy of a Game
Once you've got XNA Game Studio 3.0 installed, you can create a Hello World game for Zune by selecting File | New | Project. Then select Visual C# | XNA Game Studio 3.0 and click Zune Game (3.0). While the program this generates doesn't actually say Hello World, it does watch for input, paint the background blue, and exit on demand.
If you're expecting something really complicated, you're in for a pleasant surprise. Your entire game infrastructure amounts to around twenty lines of code spread over six functions. Here's a description of those functions.
The constructor is just a standard, garden variety C# object constructor. Treat it the same as you would any other one.
Initialize is called after your constructor finishes. It lets you do further initialization, especially for things involving the graphics system.
LoadContent is called after Initialize and is where you load your images and sounds.
UnloadContent is called on game exit and lets you unload your content. But, since the unloading of most images and sounds is handled for you, you generally don't need to do anything here.
Update is probably the place where game development differs most from normal app development. Instead of a message loop, games regularly poll their input, calculate what to do with it, and then draw the result of those calculations. Most of that happens in Update, so this is where most of your code will live.
Draw is called after each Update and is where you draw your images.
That's it. Even though the program is very short, it starts, paints the screen blue, reads input, and exits when you hit the back button. What's that? Did you think game development would be hard? Not with XNA Game Studio.

Sprites on Pixie Dust
That was easy, but a blue background that exits does not make for a compelling game. You need actual images if you're going to have a game. Thankfully, putting pictures on the screen isn't difficult.
In a minute I'm going to show you the code, but first you need to load an image into your project. In Visual Studio, you should have a window called Solution Explorer (if you don't, you can find it under the View menu). In the Solution Explorer, right-click on Content and select Add | Existing Item. From here you can point it at any .bmp, .jpg, .png, .tga, or .dds image file.
Let's assume you have a picture of a person called player.jpg and that you just added it to Content. Figure 1 shows the code to display that image on the screen.
Figure 1 Displaying an Image
// Add these member variables
Texture2D texture;
Vector2 position;

// Add this line to the constructor
position = new Vector2(100, 100);

// Add this line to LoadContent()
texture = Content.Load<Texture2D>("player");

// And add these lines to Draw();
spriteBatch.Begin();
spriteBatch.Draw(texture, position, Color.White);
// (Draw the rest of your sprites)
spriteBatch.End();
Before I explain those lines, let's talk lingo. XNA Game Studio started as a development system for the Xbox. As such, it's all based on 3D graphics. 2D games (and, since the Zune doesn't have a 3D renderer, you need to do 2D games) are really 3D games where everything is really thin. So you'll see 3D terms sprinkled around your code. For instance, while sprite is the normal term for an image in a 2D game, the word texture comes from 3D gaming. As far as XNA Game Studio is concerned, sprites are really thin 3D rectangles with textures mapped to them.
The Content.Load function takes an Asset Name, which by default is the file name without the extension. Since I loaded an image called player.jpg, I passed the string "player" to Content.Load. In the LoadContent function, you'll find that there's already a line that allocates a SpriteBatch. All drawing is done with this SpriteBatch. In the Draw function, you tell the SpriteBatch you're about to begin, draw all the sprites, and then tell it you're finished.
The position member variable is a vector structure that holds the x and y location where you want the sprite to be drawn. In the example, I used pixel location (100, 100). The color is a tint you can add to your sprites. Using white leaves the image unchanged.
So, to get an image on the screen, you just needed to load the texture and tell the sprite batch to draw it. The first image basically takes four lines of code, and each subsequent one takes two more.

Of Asset Names and Color Keys
Hopefully, at this point you're looking at your player image on the screen and thinking that this isn't so difficult. But maybe you're not fully satisfied. What if you don't like that your code needs to use the file name of the image? And, more importantly, what if the picture you loaded is rectangular, but the player isn't? How can you draw just the player and not the background?
To answer both of these questions, you need to look at the properties for the image you loaded. Go back to Solution Explorer and right-click on the file (in my example, player.jpg). Select Properties. A properties pane will appear below the Solution Explorer, giving you a lot of useful information. First look at Asset Name. This is the name you passed to Content.Load. If you want to make it something other than the file name, change it here.
Now expand Content Processor by clicking the little + to the left of the name. The property you care about here is the Color Key Color. Any pixel whose color matches the color key will be transparent. That's how you take a rectangular image and strip out the background. So, to get rid of your sprite's background, either change its color or change the color key setting so that they match. The setting defaults to Red, Green, Blue, and Alpha values of 255, 0, 255, 255, which is magenta.

Ya Gotta Move To Play
Certainly a single sprite on a blue background isn't going to be the next big thing in gaming. Minimally, you need the user to be able to move that sprite around somehow. And, to do that, you need to know what the user wants to do. If you've looked at the Update function in your program, you probably already have an idea how to do this. The default code reads the input and exits if the back key is pressed. Unfortunately, they kind of monopolized the input there. Let's change things around a bit to let you use the input as well.
Here's what you currently see:
if (GamePad.GetState(PlayerIndex.One).Buttons.Back ==
    ButtonState.Pressed)
    this.Exit();
I suggest changing it to look like this:
GamePadState input =
    GamePad.GetState(PlayerIndex.One);
if (input.Buttons.Back == ButtonState.Pressed)
    this.Exit();
Now you can use the input variable to look at user input. Note that input is a GamePadState. Think of the Zune's input as an Xbox gamepad that's missing some of the buttons. Figure 2 provides the full list of mappings for the Zune's buttons.
Figure 2 Zune Button Mappings
Button | Mapping
Back | GamePadState.Buttons.Back
Play/pause | GamePadState.Buttons.B
DPad center | GamePadState.Buttons.A
DPad edges | GamePadState.DPad
Sliding on the ZunePad | GamePadState.Thumbsticks.Left
Clicking on the ZunePad | GamePadState.Buttons.LeftShoulder
Okay, so let's make the player move. The code in Figure 3 covers both DPad and ZunePad input, so it will work on any Zune. (The original Zunes didn't have the ZunePad functionality.)
Figure 3 Supporting Movement
// add these lines to Update()
const int c_speedScalar = 100;
Vector2 speed = Vector2.Zero;

speed.X = input.ThumbSticks.Left.X * c_speedScalar;
speed.Y = input.ThumbSticks.Left.Y * c_speedScalar * -1;

if (input.DPad.Left == ButtonState.Pressed)
    speed.X -= c_speedScalar;
else if (input.DPad.Right == ButtonState.Pressed)
    speed.Y += c_speedScalar;

if (input.DPad.Up == ButtonState.Pressed)
    speed.Y -= c_speedScalar;
else if (input.DPad.Down == ButtonState.Pressed)
    speed.Y += c_speedScalar;

position += speed * (float)gameTime.ElapsedGameTime.TotalSeconds;
Remember that the Draw routine is drawing the player at the location specified by the position member variable. So, to move the sprite, all you need to do is update that location.
The really fancy part here is the last line where you change the position. The Xbox has stronger graphics capabilities than the Zune, so its Update routine gets called more frequently. In other words, it has a higher framerate. This means you need to make movement be independent of the framerate. To accomplish this, use the input to calculate a speed, not a position. Then convert the speed to a position by multiplying by the amount of time that has passed since the last call to Update.

Sprite in a Box
Anyone can move. The trick is knowing when to stop.
If your game is going to be interesting, then you need to know when one sprite collides with another. The fancy game developer's term for this is collision detection, and the simplest way to do collision detection in XNA Game Studio is to use an object called a BoundingBox. Basically, you wrap invisible boxes around your objects and then check to see if any of those boxes intersect with each other.
Aside from the current game being less than exciting, the biggest problem is that the sprite can move off the screen. Let's fix that. Whenever the sprite hits an edge, we'll make it jump back to location (100, 100).
First, you'll need a bounding box for the screen:
// Add this member variable
BoundingBox bbScreen;

// Add these lines to Initialize()
Vector3 min = new Vector3(0, 0, 0);
Vector3 max = new Vector3(graphics.GraphicsDevice.Viewport.Width,
    graphics.GraphicsDevice.Viewport.Height, 0);
bbScreen = new BoundingBox(min, max);
There's a bit of magic here, but it's not very complicated magic. The Hello World framework gave you a member variable called graphics that contains the width and height of the game screen. You can set the upper-left corner of the bounding box to the upper-left corner of the screen and the lower-right corner of the bounding box to the lower-right corner of the screen. BoundingBoxes require 3D vectors, so set the z coordinates to 0.
Now let's create a box for the sprite, like so:
// Add these lines to Update() after
// the position has been calculated.
Vector3 min =
    new Vector3((int)position.X, (int)position.Y, 0);
Vector3 max =
    new Vector3((int)position.X + texture.Width - 1,
    (int)position.Y + texture.Height - 1, 0);
BoundingBox bb = new BoundingBox(min, max);
Finally, let's see if the sprite is fully contained in the screen. If not, it must have hit an edge. Add these lines directly below the ones added previously:
ContainmentType ct = bbScreen.Contains(bb);
if (ct != ContainmentType.Contains)
    position = new Vector2(100, 100);
Now the sprite will jump back to (100, 100) whenever it hits the edge of the screen. You, of course, should probably do something more intelligent, like stop or warp to the other side.
Colliding with the edge of the screen is a little different from colliding with another object. The normal situation is that you're inside the screen, but you're outside of other objects. So, to check for a collision with the edge of the screen, you want to see if the screen's box contains you. But if you want to check for a collision with another object, you want to see if that object's box intersects with you. To that end, the BoundingBox object has an Intersects member function. Intersects returns a bool that is true when the two boxes intersect. Use Intersects if you want to check whether two sprites hit each other.

Bing Bang Boom
You're now one technology away from writing your own games. You can draw images, move them in response to user input, and act on them running into each other. Since this is a game, you probably want a sprite collision to end in a huge explosion. You already know how to draw the explosion, but it won't be really satisfying unless you rattle the player's speakers, too. Fortunately, playing sounds is even easier than drawing sprites.
First, you'll need to add a sound to your project. The process here is similar to adding an image. Go to the Solution Explorer, right-click on Content, and choose Add | Existing Item. From there you can choose a .wav, .wma, .mp3, or .xap file. Also, like the image file, if you want a different Asset Name than the file name, you can change it in the properties.
Now let's play that sound. This code assumes that the file you loaded was explode.wav:
// Add this member variable
SoundEffect sndBoom;

// Add these lines to LoadContent()
ContentManager contentManager =
    new ContentManager(Services, "Content");
sndBoom =
    contentManager.Load<SoundEffect>("explode");

// Add these lines to Update()
if (input.Buttons.B == ButtonState.Pressed)
    sndBoom.Play();
The contentManager.Load call is similar to the function you used to load your sprite's texture. Then, as you see, when you want to hear the sound, you just Play it. Note that, as written, this code will try to play a new sound every frame while the B button is pressed. Depending on the length of the sound, this could result in too many copies of it playing and your game throwing an exception. You should guard against that by playing only once per button press. The classic way to do that is to store the previous state of the input and act only when the state changes.

Advanced Drawing
You've now got the basic tools you need to build your own games. I suggest you take some time to play with those tools and get comfortable with them. When you're ready for more, you can read this section for a brief introduction to advanced ideas. The help documents for these functions will give you everything you need to know to use them.
First, let's talk about the Z order. As the code currently stands, if two sprites overlap, the one that's drawn last will end up on top. That's kind of a pain. A much better method is to give each sprite a variable that states the order in which it should be drawn. To do this, you need to use more complicated versions of spriteBatch.Begin and spriteBatch.Draw, like so:
spriteBatch.Begin(
    SpriteBlendMode.AlphaBlend,
    SpriteSortMode.BackToFront,
    SaveStateMode.None);
spriteBatch.Draw(texture, position,
    null, Color.White, 0, Vector2.Zero,
    1.0F, SpriteEffects.None, zOrder);
Spend some time with the SpriteBatch documentation and experiment. These functions are a bit complicated, but they're also very powerful. For instance, if you want to change the sizes of your sprites, you can use a rectangle instead of a vector for the position. If the rectangle bounds are different than the texture's size, XNA Game Studio will scale it for you automatically. If your texture has multiple images that you want to cycle between, you can pass in a source rectangle that specifies which part of the texture to draw.
If you want a translucent sprite, change the alpha value of the color you pass to draw:
Color trans = new Color(
    Color.White.R, Color.White.G,
    Color.White.B, 128);
Pass trans to draw instead of Color.White, and the sprite would be half translucent.
Finally, here's some neat code you can use to rotate the screen from portrait to landscape mode:
// Add this member variable
Matrix matTransform;

// Add these lines to the constructor
matTransform = Matrix.Identity;
matTransform *=
    Matrix.CreateRotationZ(MathHelper.ToRadians(90));
matTransform *=
    Matrix.CreateTranslation(240, 0, 0);

// In Draw() change spriteBatch.Begin to this
spriteBatch.Begin(SpriteBlendMode.AlphaBlend,
    SpriteSortMode.BackToFront, SaveStateMode.None,
    matTransform);

The Rest Is Up To You
The full code for my example is shown in Figure 4, and it should get you moving. If you run aground, the creators.xna.com Web site is full of samples that show you how to do all sorts of arcane game development things, from per-pixel bounding boxes to physics models.
Figure 4 A Simple Zune Game
public class Game1 : Microsoft.Xna.Framework.Game {
    GraphicsDeviceManager graphics;
    SpriteBatch spriteBatch;
    Matrix matTransform;
    Texture2D texture;
    SoundEffect sndBoom;
    BoundingBox bbScreen;
    Vector2 position;

    public Game1() {
        graphics = new GraphicsDeviceManager(this);
        Content.RootDirectory = "Content";
        position = new Vector2(100, 100);
        matTransform = Matrix.Identity;
        // Uncomment these lines to do Landscape Mode
        //matTransform *= Matrix.CreateRotationZ(          MathHelper. ToRadians(90));
        //matTransform *= Matrix.CreateTranslation(240, 0, 0);
    }

    protected override void Initialize() {
        Vector3 min = new Vector3(0, 0, 0);
        Vector3 max =
            new Vector3(
            graphics.GraphicsDevice.Viewport.Width,
            graphics.GraphicsDevice.Viewport.Height, 0);
        bbScreen = new BoundingBox(min, max);
        base.Initialize();
    }

    protected override void LoadContent() {
        spriteBatch = new SpriteBatch(GraphicsDevice);
        texture = Content.Load<Texture2D>("player");
        ContentManager contentManager =
            new ContentManager(Services, @"Content\Audio");
        sndBoom = contentManager.Load<SoundEffect>("explode");
    }

    protected override void UnloadContent() {
    }

    protected override void Update(GameTime gameTime) {
        GamePadState input = GamePad.GetState(PlayerIndex.One);
        if (input.Buttons.Back == ButtonState.Pressed)
            this.Exit();

        if (input.Buttons.B == ButtonState.Pressed)
            sndBoom.Play();

        const int c_speedScalar = 100;
        Vector2 speed = Vector2.Zero;

        speed.X = input.ThumbSticks.Left.X * c_speedScalar;
        speed.Y = input.ThumbSticks.Left.Y * c_speedScalar * -1;

        if (input.DPad.Left == ButtonState.Pressed)
            speed.X -= c_speedScalar;
        else if (input.DPad.Right == ButtonState.Pressed)
            speed.X += c_speedScalar;

        if (input.DPad.Up == ButtonState.Pressed)
            speed.Y -= c_speedScalar;
        else if (input.DPad.Down == ButtonState.Pressed)
            speed.Y += c_speedScalar;

        position += speed * (float)gameTime.ElapsedGameTime.TotalSeconds;

        Vector3 min = new Vector3((int)position.X, (int)position.Y, 0);
        Vector3 max = new Vector3((int)position.X +
            texture.Width - 1, (int)position.Y + texture.Height - 1, 0);
        BoundingBox bb = new BoundingBox(min, max);

        ContainmentType ct = bbScreen.Contains(bb);
        if (ct != ContainmentType.Contains)
            position = new Vector2(100, 100);

        base.Update(gameTime);
    }

    protected override void Draw(GameTime gameTime) {
        GraphicsDevice.Clear(Color.CornflowerBlue);
        spriteBatch.Begin(SpriteBlendMode.AlphaBlend,
            SpriteSortMode.BackToFront, SaveStateMode.None, matTransform);
        spriteBatch.Draw(texture, position, null, Color.White,
            0, Vector2.Zero, 1.0F, SpriteEffects.None, 0.5F);
        spriteBatch.End();

        base.Draw(gameTime);
    }
}
I hope you find XNA game development in C# to be as much fun as I do. Maybe something you write will become the next big thing in gaming... if my son doesn't beat you to it. Happy coding.

Send your questions and comments to goplaces@microsoft.com.

Mike Calligaro is a Senior Development Lead with the Windows Mobile team at Microsoft and a contributor to the Windows Mobile team blog, which you can see at blogs.msdn.com/windowsmobile.


Office Space
Custom Field Types for SharePoint 2007
Ted Pattison
Code download available from the MSDN Code Gallery
Browse the Code Online
Contents
What Exactly Is a WSS Field Type?
Data Validation
Multicolumn values
Custom Property Fields
As the adoption of Microsoft SharePoint 2007 continues to accelerate with both business users and developers, one has to attribute much of its success to how quickly users can create lists within their sites. Windows SharePoint Services (WSS) 3.0 provides many useful built-in list types that are used to track common forms of business data such as tasks, scheduled events, and announcements. WSS also provides users with the flexibility to add and remove columns from a list to handle scenarios where the built-in WSS list types do not provide an exact match to the business problem at hand.
When you create a new column inside a particular list instance and configure it for a specific business scenario, it can be hard or impossible to reuse your efforts across multiple lists. Fortunately, the latest version of WSS introduced reusable column types known as site columns. Advanced users can create a site column in the site column gallery of a top-level site. This site column can be configured a single time and then added to many different lists across the scope of the current site collection. Developers can extend this scope of reuse by defining a site column inside a feature, which can then be reused across site collections and across SharePoint farms.
While site columns provide users and developers with new capabilities in reuse, there is a way to define a reusable column definition that is even more powerful. WSS allows developers to drop down to a lower level by creating custom field types. These will be the focus of this month's column. Once you learn how to create a custom field type, you will have the greatest amount of control that WSS offers when it comes to creating a polished user interface for your users to display and edit column values. Developing custom field types also provides a powerful new means of performing data validation before allowing user input values to be written into the content database.

What Exactly Is a WSS Field Type?
When users want to create a new column and add it to a list or document library, they must select an underlying field type on which to base this new column. Likewise, a user or developer must also select an underlying field type when creating a new site column. Figure 1 shows an example of the field types that are available out-of-the-box when you are creating a new list column within a farm running Microsoft Office SharePoint Server (MOSS) 2007.
Figure 1 Base Field Types Available in SharePoint
As you can see, the list of field types made available to users can be extended through the development of custom field types. In this month's column, I will show you how to do this as well as how to create a solution package to distribute and deploy your custom field types that follows best practices in SharePoint development.
To create a custom field type, you can create a project in Visual Studio that contains the following items:
  • An ASP.NET user control that defines an editing surface known as a RenderingTemplate
  • A public class to initialize and manage the RenderingTemplate
  • A public class that defines the actual custom field type
  • An XML file that is used to deploy the custom field type
Figure 2 shows a screenshot of the sample project named OfficeSpaceFieldTypes created with Visual Studio. This project is included as a download to accompany this month's column. This project contains four custom field types, including FieldHelloWorld, FieldSocialSecurityNumber, FieldUnitedStatesAddress, and FieldEmployeeStatus.
Figure 2 Visual Studio Project to Develop Custom Field Types
I am going to begin by stepping through the custom field type named FieldHelloWorld to show you how all the pieces fit together. Then I will examine a few more interesting aspects of the other three custom field types to illustrate some of the possibilities that might motivate you to develop custom field types of your own.
Note that the OfficeSpaceFieldTypes project was built for Visual Studio using the STSDEV utility. You can open it using either Visual Studio 2005 or Visual Studio 2008. For more background information on developing with the STSDEV utility, you can read my Office Space column in the March 2008 issue of MSDN Magazine titled "Simplify SharePoint Development with STSDEV."
The user control file named OfficeSpace.HelloWorld.ascx contains a RenderingTemplate control tag that acts as the editing surface for a custom field type:
<%@ Control Language="C#" Debug="true" %>
<%@ Assembly Name="Microsoft.SharePoint, [full 4-part name]" %>
<%@ Register TagPrefix="SharePoint"
  Assembly=" Microsoft.SharePoint, [full 4-part name]"
  Namespace="Microsoft.SharePoint.WebControls" %>

<SharePoint:RenderingTemplate ID="HelloWorldRendingTemplate" runat="server">
  <Template>
    <asp:TextBox ID="txtUserInput" runat="server" CssClass="ms-long" />
  </Template>
</SharePoint:RenderingTemplate>
The top-level control tag is based on the RenderingTemplate control, which is defined inside the Microsoft.SharePoint assembly in the Microsoft.SharePoint.WebControls namespace. I have given the RenderingTemplate control tag an ID value of HelloWorldRendingTemplate. Later you will see how to use this string-based ID value to load and initialize the RenderingTemplate control from the .ascx file.
Inside the control tag for the RenderingTemplate control, there is an inner Template element. You create the editing surface for a custom field type by adding ASP.NET control tags and HTML inside this Template tag. You have the flexibility to create the editing surface for a custom field type using a composite of ASP.NET controls and to lay them out using div elements or an HTML table. I have defined our first sample using a simple ASP.NET TextBox control:
<asp:TextBox ID="txtUserInput" runat="server" CssClass="ms-long" />
If you look back at the Visual Studio project structure in Figure 2, you can see that all four .ascx files have been deployed inside the WSS RootFiles directory within the directory structure of TEMPLATE/CONTROLTEMPLATES. This is the directory where WSS requires you to deploy any .ascx file that defines a RenderTemplate control that is to be used by a custom field type.
As a general best practice in SharePoint 2007 development, it is best to avoid deploying your files directly inside one of the WSS system directories, such as CONTROLTEMPLATES, IMAGES, or LAYOUTS. Instead, you should create a solution-specific directory inside one of these WSS system directories for deploying your files so that you can avoid potential naming conflicts with files deployed by Microsoft and by other third-party developers.
However, the technique of creating a solution-specific directory inside the CONTROLTEMPLATES directory does not work correctly when dealing with the .ascx file that defines a RenderingTemplate control. This .ascx file must be deployed directly inside the CONTROLTEMPLATES directory because that is the only directory in which the WSS runtime looks, when it loads .ascx files, to discover the list of available RenderingTemplate controls during its initialization of the hosting worker process.
In just a moment, I will show you how to create the class that will be used to define a custom field type. However, I want to first discuss the other class required to create a custom field type, which is known as the field control class. The field control class will be used to load and initialize the RenderingTemplate control from the .ascx file.
You create a field control class by creating a public class that inherits from a base class from the WSS object model named BaseFieldControl. When you create a field control class, you must override a read-only property named DefaultTemplateName that allows you to pass the ID of the RenderingTemplate control to the WSS runtime:
public class HelloWorldFieldControl : BaseFieldControl {
  // pass ID of RenderingTemplate control to WSS runtime
  protected override string DefaultTemplateName {
    get { return "HelloWorldRendingTemplate"; }
  }
}
Once you have implemented the DefaultTemplateName property, you should then add a protected field to track a reference to each control inside the RenderingTemplate that you want to access programmatically. Next, you should override the CreateChildControls method to properly initialize these fields:
public class HelloWorldFieldControl : BaseFieldControl {
  // add field to reference control
  protected TextBox txtUserInput;

  // initialize field with reference to control
  protected override void CreateChildControls() {
    base.CreateChildControls();
    txtUserInput =
      (TextBox)TemplateContainer.FindControl("txtUserInput");
  }
}
Note that you will not be instantiating control instances in the CreateChildControls method, but rather going through a protected property of the base class named TemplateContainer, which exposes a FindControl method. This technique allows you to obtain references to existing control instances.
Now it's time to add the code to the field control class that is responsible for reading and writing the field's underlying value to and from the content database. You do this by overriding a property of the BaseFieldControl class named Value. The Value property is based on the type System.Object, which allows quite a bit of flexibility. You can work with any type of object that you'd like as long as it supports .NET serialization. The HelloWorldFieldControl class illustrates a simple example of implementing the Value property:
public class HelloWorldBaseFieldControl : BaseFieldControl {
  public override object Value {
    get {
      this.EnsureChildControls();
      return txtUserInput.Text;
    }
    set {
      this.EnsureChildControls();
      txtUserInput.Text = (string)this.ItemFieldValue;
    }
  }
}
As you can see, both the get and set methods of the Value property begin their implementation with a call to the EnsureChildControls. A call to EnsureChildControls ensures that the CreateChildControls method has already executed. This is required to ensure that the txtUserInput field contains a valid reference so that you can program against the control without a null reference exception.
You can see that the get method of the Value property simply returns the string value from the TextBox control. The WSS runtime will call the get method when a user updates an item that contains a column based on this custom field type. The WSS runtime writes your return value directly to the content database.
The WSS runtime calls the set method when a user opens the item in edit view just before the controls in your RenderingTemplate control are shown to the user. The key point to understand about implementing the set method is that the ItemFieldValue property provides you with access to the current column value and you must use this value to initialize the control (or controls) in your RenderingTemplate.
Now that I have walked through the implementation of the field control class, let's move on to the class that defines the actual custom field type itself. Begin by creating a public class that inherits from one of the WSS built-in field types such as SPFieldText, SPFieldNumber, SPFieldDateTime, or SPFieldMultiColumn. The class created for the first example named FieldHelloWorld inherits from SPFieldText:
public class FieldHelloWorld : SPFieldText {
  // custom field type implementation goes here
}
The first thing you must do when implementing the class for a custom field type is add two standard public constructors. These nondefault constructors are required because the WSS runtime will use them to create instances from your custom field type. You don't need to supply any code inside the curly braces. You just need to define parameters and pass them on to the public constructor of the base class with a matching parameter list:
public class FieldHelloWorld : SPFieldText {
  // add public constructors to call base class constructors
  public FieldHelloWorld(
    SPFieldCollection fields, string fieldName)
    : base(fields, fieldName) { }

  public FieldHelloWorld(SPFieldCollection fields,
    string typeName, string displayName)
    : base(fields, typeName, displayName) { }
}
Once you have added the two standard public constructors, you next override a read-only property named FieldRenderingControl. You implement the get method of this property by creating an instance of the field control class and initializing its FieldName property using the InternalName property of the custom field type class. The return value of the get method should be a reference to this newly created instance:
public override BaseFieldControl FieldRenderingControl {
  get {
    BaseFieldControl ctr = new HelloWorldFieldControl();
    ctr.FieldName = this.InternalName;
    return ctr;
  }
}
When creating the class for a custom field type, you also have the option of overriding the get method of the DefaultValue property so that you assign a new value to a column when a user creates a new item:
public override string DefaultValue {
  get {
    return "Hello World (default value)";
  }
}
So far, I created an ASP.NET User Control (an .ascx file) that defines a RenderingTemplate control, and I created two public classes for the field control and the custom field type. Note that these two public classes must be compiled into an assembly with a strong name and this assembly must be installed in the Global Assembly Cache (GAC). The final step in deploying and testing your development efforts is to create an XML file using a special naming scheme and place it in a well-known location where the WSS runtime can find it.
When you create the XML file used to deploy a custom field type, you must start its name with the string "fldtypes" and you must give the file an extension of .xml. The XML file used in the OfficeSpaceFieldTypes project is fldtypes_OfficeSpace.xml. As you can see in Figure 2, the file must also be deployed within the WSS RootFile directory structure at a location of TEMPLATE/XML.
A key point to understand is that the WSS runtime searches the TEMPLATE/XML directory when it initializes the hosting worker process to find any file that matches the pattern of fldtypes*.xml. The WSS runtime then reads XML data from each file that matches this pattern and uses this data to determine what field types are currently active and in use on a farmwide basis.
If you look in the TEMPLATE/XML directory, you will find a system file named fldtypes.xml that defines all the core field types supplied by WSS. The installation of MOSS deploys several more files into the TEMPLATE/XML directory, such as fldtypes_publishing.xml and fldtypes_spsbizdata.xml, to supply additional custom field types of its own. By inspecting these files that are part of the standard SharePoint 2007 installation process, you can see how the WSS team and the MOSS team have structured the XML data for built-in field types, which might teach you a few tricks you can use when creating your own.
An XML file used to deploy custom field types such as fldtypes_OfficeSpace.xml requires a top-level FieldTypes element. You will add one inner FieldType element for each of custom field type you would like to deploy:
<FieldTypes>
  <FieldType>
    <!—add field type data here -->
  </FieldType>
<FieldTypes>
The FieldType element will contain several Field elements with a Name attribute that defines the type of value inside. Here is the XML used to define the FieldType element for the custom field type named FieldHelloWorld:
<FieldType>
  <Field Name="TypeName">HelloWorld</Field>
  <Field Name="ParentType">Text</Field>
  <Field Name="TypeDisplayName">
    Hello World (Office Space Demo)</Field>
  <Field Name="TypeShortDescription">
    Hello World (Office Space Demo)</Field>
  <Field Name="UserCreatable">TRUE</Field>
  <Field Name="FieldTypeClass">
    [field type class name],[4-part assembly name]</Field>
</FieldType>
Now let's examine selected implementation details from the three other custom field types to show you how to get a little more involved.

Data Validation
The three other custom field types defined in the OfficeSpaceFieldTypes project have been designed for a scenario in which you need to collect data about employees for a SharePoint list. Figure 3 shows the edit view for an item in a list that uses the custom field types named FieldSocialSecurityNumber, FieldUnitedStatesAddress, and FieldEmployeeStatus. These three field types demonstrate various techniques, such as adding validation logic, working with multicolumn values, and adding custom properties.
Figure 3 OfficeSpaceFieldTypes captures employee data in a SharePoint list
The custom field type named FieldSocialSecurityNumber provides an example of how to perform custom data validation. This is done by overriding a method named GetValidatedString within the class that defines the custom field type. The implementation of GetValidatedString in FieldSocialSecurityNumber has been designed to ensure the user input matches the pattern for a social security number using regular expressions:
public override string GetValidatedString(object value) {
  string UserInput = value.ToString();
  string SSN_RegularExpression = @"^\d{3}-\d{2}-\d{4}$";
  if (
     (this.Required || !string.IsNullOrEmpty(UserInput) ) &
     (!Regex.IsMatch(UserInput, SSN_RegularExpression) ) ) {
      throw new SPFieldValidationException(
        "SSN must be form of 123-45-6789");
    }
  return base.GetValidatedString(value);
}
As you can see, the implementation tests for a regular expression match using the Regex class that is supplied as part of the Microsoft .NET Framework. The implementation also contains logic to test whether a column based on FieldSocialSecurityNumber has been defined to require the user to add a social security number or the field value can be left blank. However, if the field value is not required, the validation still requires that the user either leave the field value empty or provide a valid social security number.
If your validation logic determines that the user input is invalid, you simply throw an exception of type SPFieldValidationException. WSS responds by canceling the users request to save the current item and displays your error message directly to the user as shown in Figure 4.
Figure 4 Custom validation logic prevents invalid data

Multicolumn values
The custom field type named FieldUnitedStatesAddress demonstrates the technique of creating a field type with multicolumn values. This is useful in the case of capturing an address from the user where there are several pieces of related data that must be captured and validated as a whole. You can refer back to Figure 3 to see what this custom field looks like when a user is in edit view for an item.
One requirement of multicolumn field types such as FieldUnitedStatesAddress is that they must inherit from the SPFieldMultiColumn. In most cases, there will also be a requirement for your RenderingTemplate to display more than one input control to the user. Figure 5 shows an example of the RenderingTemplate used by the custom field type named FieldUnitedStatesAddress.
Figure 5 RenderingTemplate for Multicolumn Values
<SharePoint:RenderingTemplate  runat="server"
  ID="UnitedStatesAddressRenderingTemplate" >
  <Template>
    <table class="ms-authoringcontrols" >
      <tr>
        <td>Street:</td>
        <td><asp:TextBox ID="txtStreet" runat="server" /></td>
      </tr>
      <tr>
        <td>City:</td>
        <td><asp:TextBox ID="txtCity" runat="server" /></td>
      </tr>
      <tr>
        <td>State:</td>
        <td><asp:TextBox ID="txtState" runat="server" /></td>
      </tr>
      <tr>
        <td>Zipcode:</td>
        <td><asp:TextBox ID="txtZipcode" runat="server" /></td>
      </tr>
    </table>
  </Template>
</SharePoint:RenderingTemplate>
When you create a multicolumn field type, you typically create the RenderingTemplate as a composite of several input controls like the example you have just seen. Next, you need to learn the trick of moving the values from these controls back and forth to and from the content database as a single multicolumn value. The WSS object model supplies a creatable class type named SPFieldMultiColumnValue that makes this possible using programming syntax similar to dealing with a string array. The Value property implementation of UnitedStatesAddressFieldControl shows the common pattern of reading and writing multicolumn values to and from the content database (see Figure 6).
Figure 6 Reading and Writing Multicolumn Values
public override object Value {
  get {
    this.EnsureChildControls();
    SPFieldMultiColumnValue mcv = new SPFieldMultiColumnValue(4);
    mcv[0] = txtStreet.Text;
    mcv[1] = txtCity.Text;
    mcv[2] = txtState.Text;
    mcv[3] = txtZipcode.Text;
    return mcv;
  }
  set {
    this.EnsureChildControls();
    SPFieldMultiColumnValue mcv =
              (SPFieldMultiColumnValue)this.ItemFieldValue;
    txtStreet.Text = mcv[0];
    txtCity.Text = mcv[1];
    txtState.Text = mcv[2]; ;
    txtZipcode.Text = mcv[3];
  }
}
It would be possible for you to extend this simple example further to exploit the potential of multicolumn field types. For example, what if you were able to call to a Web service and pass a ZIP code that would return the associated city and state? That would allow you to add extra functionality to auto-populate the textboxes for city and state and to perform validation to ensure that the address is correct.

Custom Property Fields
The custom field type named FieldEmployeeStatus demonstrates how you can extend a custom field type with one or more custom property fields. You can accomplish this by adding a PropertySchema element to the bottom of the FieldType element for a specific custom field type in the fldtype.xml file. The custom field type named FieldEmployeeStatus has been defined with two custom property fields named AllowContactors and AllowInterns (see Figure 7).
Once you have added custom property fields in this manner, WSS will automatically add input controls to the page that allows a user to add or update columns based on your custom field type. Figure 8 shows what the user sees when adding or updating a column based on the custom field type named FieldEmployeeStatus. The user has the option of choosing to include or exclude choice values for adding list items that are contractors or interns.
Figure 7 Custom Property Fields
<FieldType>
  <Field Name="TypeName">EmployeeStatus</Field>
  <Field Name="ParentType">Text</Field>
  <!—other Field elements omitted for clarity -->

  <PropertySchema>
    <Fields>
      <Field Name="AllowContractors"
             DisplayName="Allow for Contractors"
             Type="Boolean">
        <Default>0</Default>
      </Field>
      <Field Name="AllowInterns"
                   DisplayName="Allow for Interns"
                   Type="Boolean">
        <Default>0</Default>
      </Field>
    </Fields>
  </PropertySchema>

</FieldType>
Figure 8 Custom Field Types with Custom Property Fields
Once you have extended a custom field type with one or more custom property fields, you then must write code through the WSS object model to see what values the user has assigned to them. In the case of the custom field type named FieldEmployeeStatus, there is code in the CreateChildControls method of the field control class that initializes the RadioButtonList control from the RendingTemplate that adds items for contractors and interns only when it sees that the user has set the values for the associated custom property fields to true (see Figure 9).
Figure 9 Conditional Display of Property Fields
protected override void CreateChildControls() {
  base.CreateChildControls();
  lstEmployeeStatus =
    (RadioButtonList)TemplateContainer.FindControl("lstEmployeeStatus");
  if (lstEmployeeStatus != null) {
    lstEmployeeStatus.Items.Clear();
    lstEmployeeStatus.Items.Add("Full-time Employee");
    lstEmployeeStatus.Items.Add("Part-time Employee");
    // check to see if contractors are allowed
    bool AllowContactors =
         (bool)this.Field.GetCustomProperty("AllowContractors");
    if (AllowContactors) {
      lstEmployeeStatus.Items.Add("Contractor");
    }
    // check to see if interns are allowed
    bool AllowInterns =
         (bool)this.Field.GetCustomProperty("AllowInterns");
    if (AllowInterns) {
      lstEmployeeStatus.Items.Add("Intern");
    }
  }
}
When a user adds a new column to a list based on the custom field type named FieldEmployeeStatus and assigns a value of true, a user adding and updating items inside the list will now see additional choices in the RadioButtonList control.
You can see that this style of development can be quite powerful when it comes to handling how users input data and data validation. It might also give you new ideas about storing types of data inside standard SharePoint lists that you might not have thought were practical.

Send your questions and comments to mmoffice@microsoft.com.

Ted Pattison is an author, trainer, and SharePoint MVP who lives in Tampa, Florida, and whose latest book is Inside Windows SharePoint Services 3.0 from Microsoft Press. He also delivers advanced SharePoint training to professional developers through his company, Ted Pattison Group.


Security Briefs
A Conversation About Threat Modeling
Michael Howard
Contents
The Thespians
Scene I
Scene II
Scene III
Scene IV
Many years ago, I read a paper by Bill Bryant entitled "Designing an Authentication System: A Dialogue in Four Scenes" that explains the Kerberos authentication system. The paper is a reflection of how grumpy and jaded security people think and react to questions from nonsecurity people. Often, when nonsecurity people ask security people questions about security, it can be like extracting teeth to arrive at an appropriate and secure solution because there is a communication mismatch between the domain-expertise of the developer and of the security professional. So I thought I would document an amalgam of conversations I have had over many years into one long security chat. I do not pretend that this chat will do justice to the original Kerberos paper, however, and I apologize profusely in advance.
A major goal of this chat is to tie in some of the major Security Development Lifecycle (SDL) requirements we impose on product teams here at Microsoft.

The Thespians
Paige: a young, bright software developer.
Michael: a simple security guy at Microsoft.

Scene I
A small hallway between two sets of cubicles, supposedly designed to enhance agile software development and communication.
Paige: Hey, grumps. I need your help building some software.
Michael: So?
Paige: Seriously, I want your help building this system I'm working on.
Michael: You mean you're going to design something, build it, pretend to test it, and then ask me to find the security vulnerabilities?
Paige: Don't be so grumpy. No, I want your help up front.
Pause: Michael wipes his hand down his face and forces a smile.
Michael: Wow! That's a first. OK, I'll help. What do you want?
Paige: We have the skeleton design laid out. It's for a Web application, but I want to make sure we're doing the right security things.
Michael: You mean you want to pass your Final Security Review?
Paige looks perplexed and a little agitated.
Paige: Don't be so GRUMPY! No, I want to do the right security things, I really do. I don't want this product in the headlines for the wrong reasons. I want to make sure any sensitive data is secured and I want to keep any emergency patches to a minimum. Our customers have told us they want to reduce patch deployment incidents.
Michael attempts a really bad Yoda impersonation.
Michael: Wise you are.
Paige: Where do we start?
Michael: Where's the threat model? Without a threat model we're just guessing.
Paige: Here it is. We've identified the threats, but I guess we need to make sure the mitigations are correct.
Michael leans over Paige's desk and looks over the threat model rapidly.
Michael: Good. Very Good. I'm impressed. It's a good threat model. If you squint, the application looks a lot like many Web-based applications: Web front end, Web back end, database server, application logic, storing customer data at the back end. Hmmm… pretty standard stuff.
Paige: OK, now what?
Michael: I see you've identified the first-order threats.
Paige: Actually, the threat modeling tool did that.
Michael: I know, I was being nice! Let's look at some of the biggies. At this point, I'm assuming the application diagram is accurate.
Paige: Yes, the design has been pretty stable for a couple of weeks.
Michael: Excellent. A good diagram is critical to the rest of the process, as it is the most important bridge between your application domain and my security domain.

Scene II
Michael pulls up a chair at Paige's desk to look at the threat modeling tool analysis in more detail.
Michael: The first thing you should know is that attacks happen and there is really nothing you can do to stop them. As long as attackers are drawing breath, they will attack. Web applications are a big target because they often provide the front end to sensitive data and they are often full of really bad security bugs.
Paige: Why are Web apps so full of security bugs?
Michael: I don't have empirical data, but I've seen thousands of bugs, reviewed more code than I care to remember, and heard every plausible and implausible excuse for not fixing bugs.
I think we see so many security bugs because of two things. First, there is a lot of Web code written today, and the law of big numbers says that a very small number of security bugs in a lot of code is still a big number of bugs.
Second, most developers just don't know the issues. It's sad, but true. Unfortunately, there are a large number of attackers out there who do know the issues.
Third, a lot of code is rushed to market. I hear "Get it out there" all too often. Don't get me wrong; your Web app has to provide real business value, and if it's stuck in the development labs, it's doing no one any good. But if you're building something that is highly exposed, such as a Web app, then you need to perform security due diligence so you don't put customers at risk.
Paige: You said there were two reasons, but gave three.
Michael: I know. I lied.
Michael furrows his brow.
Michael: When did you last go to security training?
Paige: Three weeks ago; it was the Web security class.
Michael: Excellent. Who taught the class?
Paige: Bryan.
Michael has a worried look on his face. Paige responds to the worried look.
Paige: WHAT?
Michael: Just kidding, you had one of our best guys. Did he tell you about his AJAX security book?
Paige: Of course.
Both Paige and Michael smirk.
Michael: OK, let's get back on course.
Short pause as Michael looks over the threat model.
Michael: You've noted that the database server stores potentially sensitive user data.
Paige: Yeah. There's a small client application that runs in the browser, written in C#. This code can upload files from the user to our back end via the Web server, and the files are stored in the file system along with various file metadata stored in SQL Server for rapid lookup. We might store billions of files eventually.
Michael: This raises the security and privacy bar substantially because the data is potentially sensitive; and because it is data, it is subject to information disclosure threats at the back-end data store and while it's in transit from the client to the server, right?
Paige: Er, yeah.
Michael: OK, that gives me a feel for the risk of your application. So let's start at the beginning.
I see you have a user—an external entity—on your system. You clearly need to authenticate users because they can be spoofed. How do you authenticate them?
Paige: We use a user name and password or Windows authentication.
Michael: Why both?
Paige: We have two distinct usage scenarios.
Paige shows the core scenarios in the threat modeling tool. The first is the Internet. The second is an intranet scenario.
Michael: What do you mean by intranet?
Paige: From the client code we detect whether the computer is domain joined, and if it is, we change a few of our internal settings to take full advantage of the Windows platform.
Paige shows the following code:
bool _fIsDomainJoined = true;
try {
    Domain d = Domain.GetComputerDomain();
}
catch (ActiveDirectoryObjectNotFoundException e) {
    // not in a domain
    _fIsDomainJoined = false;
}
Michael grimaces.
Michael: There are two bugs in the code. The first is you're checking whether the machine, not the user account, is joined to the domain. It's possible to log on using a local account even if the machine is domain joined, so change the code to use Domain.GetCurrentDomain.
Next, you're failing open. If the code raises any other kind of exception and it's handled elsewhere in the code, then the code will use the domain-joined logic because the domain-joined flag is true. You need to change the code to set the domain-joined flag to true only within a successful try block.
Kudos, by the way, on only catching the exception you can handle. Many people catch all exceptions, even the exceptions they cannot handle, and that simply masks real code errors.
Paige: OK, I get it.
Michael: Also, you should be specific in the threat model: change Intranet to Domain Joined.
Paige updates the threat modeling tool.
Michael: Tell me about the Windows authentication you use when domain joined.
Paige: We use NTLM or Kerberos.
Michael grins.
Michael: You mean you use Kerberos?
Paige: What do you mean?
Michael: You said this is a domain-joined scenario, so you should enforce Kerberos. You should move away from NTLM where possible and use only Kerberos. All currently supported domain-joined versions of Windows support Kerberos.
Paige: How do we enforce that?
Michael: Just make sure you are not requesting NTLM directly; look for "NTLM" in your code, and if you see it, come see me and we can set things right. Do not violate the SDL and require NTLM. This sample code is bad because it won't take advantage of Kerberos:
Michael shows Paige some C++ code that calls:
AcquireCredentialsHandle(NULL, NTLMSP_NAME ,...)
Paige makes a note about using Kerberos.
Paige: What's so special about Kerberos?
Michael: I could simply say, don't use NTLM because it says so in the SDL. But here's why you should use Kerberos: it is more scalable than NTLM and it authenticates the client and the server. NTLM authenticates only the client.
Now tell me about the Internet-based authentication system. Is this your own system or something else?
Paige: We use Windows Live ID.
Michael: I'll be honest. As a security guy, I'm not a fan of user ID/password authentication systems.
Paige: But we need usability.
Michael: Exactly. I totally understand. Just make sure you're using the latest version of the Windows Live ID SDK.
We've addressed one threat: spoofing the user; mitigated with authentication, in this case Windows Live ID or Kerberos. Make sure you capture that in the threat model.
Paige: Done.
Michael: This application you have running in the browser is subject to all the STRIDE risk elements because it's a process. I want to focus on two of the big threats. We can focus on the others tomorrow.
Let's look at spoofing. Are you sure it's the real application and not some other application pretending to be your code? Let's also look at elevation of privilege. The last thing you want is to allow an attacker to get this code to perform tasks beyond its supposed capability. Many security pros claim that a secure program is a program that does what it's supposed to do, and no more.
Paige: We have this nailed. I think! First, the CAB file that's downloaded to the browser is digitally signed by a certificate authority that is subordinate to a root CA certificate installed by default in Windows and Linux, our two target platforms for this first release. Second, the code is C#, so we can restrict the permission set to only those permissions required by the code, which basically is only FileDialogPermission, UIPermission, DnsPermission, and SocketPermission. At least, that's what we think we need right now.
Michael smiles.
Michael: Wow. Good stuff! You're doing something I think is awesome: you're offloading the security work to either the runtime or the operating system rather than creating your own security infrastructure. If you do your own security stuff, you'll just get it wrong! You know what's cool about using digital signatures on the managed code application?
Michael: You just addressed the T threat in STRIDE: tampering. So make sure the threat model incorporates the fact that your digital signature is the T mitigation.

Scene III
Paige and Michael grab a coffee and head back to Paige's desk. Michael is wearing a white T-shirt, so he drinks his coffee with an outstretched neck!
Michael: Next is the server. Again, it's a process in the threat model, so it's subject to all the STRIDE threats. Let's only look at spoofing for a moment. How do you know the Web site is the site you really want and not a rogue?
Paige: Ha! We have this one sewn up. In fact, now that I think about what you said earlier, we might have two solutions. The first is we require SSL/TLS, which gives us server authentication, right?
Michael: Very good. As long as the common name, or CN, in the server certificate matches the domain name in the URL.
Paige: But wait. There's more. If we're using the intranet... I mean domain-joined scenario, we get server authentication by using Kerberos! Cool.
Michael: I'm impressed. OK, client code spoofing addressed. Server spoofing addressed. Client spoofing addressed. How about threats on-the-wire: the data flow from the client to the server, and the data flow from the server to the client. Clearly (no pun intended), all are subject to information disclosure and tampering—I and T in STRIDE—because they are data flows. The good news is they're already mitigated.
Paige: They are?
Michael: Yes.
Michael pauses for effect and to see if Paige gets the solution.
Paige: Ohhh, that's right. SSL/TLS! We get on-the-wire encryption and tamper detection for free!
Michael: Correctammundo.
Paige: You just said, "Correctammundo!"
Michael: I'm sorry. You are correct. For encryption, SSL/TLS uses symmetric-key encryption such as AES, and it uses message authentication codes for tamper detection. Oh, and make sure your Web server is configured to use 128-bit or better symmetric encryption—it's an SDL requirement.
Paige makes a note about encryption key length.
Michael: But I still think there's an elephant in the room. What about the data that flows from the client to the server and is then stored in the file system and in SQL Server? That data is potentially confidential, so how do you mitigate the information disclosure and tampering threats?
Paige: What do you mean? Is the data unprotected at the server?
Michael: Sure it is. Once the data pops out of that secure SSL/TLS channel it's back in the clear. So the question is, how do you protect the data from tampering and information disclosure by your administrators or, dare I say it, attackers if they compromise the back-end server?
Paige: Wow. I never thought of that.
A boyish smile spreads rapidly across Michael's face.
Michael: That's why I get paid the mediocre bucks!
Paige: OK, simple. We encrypt the data using, say, DES.
Michael makes a really annoying FAIL sound.
Paige: Explain.
Michael: Encryption can mitigate the information disclosure threat, but encryption does not mitigate tampering threats, at least not without using sophisticated cipher modes. Also, you can't use DES because it's not secure enough and is a violation of SDL policy. You can use AES for symmetric encryption.
Paige: How do we fix this, then?
Michael: I'll be honest, it's not trivial. Encryption is pretty easy. You just call some library code, pass in a key, the plaintext, initialization vector, cipher-type, key-size, cipher-mode, padding mode, and you're done.
Paige: You're being cynical.
Michael: I'm not. That's the easy part!
Paige: You've utterly lost me.
Michael: The hard part is the key management. How do you generate the encryption keys? Where do you store them? How do you revoke them? How do you transport them between computers or users? What if a key is lost?
Paige: We only allow the user to access his or her own data.
Michael: That makes things a little easier. But not much! Where possible, rely on the operating system.
Paige: So what can we use?

Scene IV
Michael takes a deep breath and swivels his chair toward Paige.
Michael: All right, here's what I would do. You have two distinct scenarios: Internet and domain-based. Domain-based is Windows-specific, so leverage the Windows operating system security primitives as much as possible.
Paige: OK, go on.
Michael: The SDL mandates that private data be protected using the Data Protection API (DPAPI), so just use that. Your client code can detect when it is executed by a domain-joined user, so just call CryptProtectData and CryptUnprotectData and leave it at that. Windows takes care of the key management as well as transferring keys between domain-joined machines when needed. Another advantage is that this solution imposes very little on the user. The user does not know the data is protected. It all happens transparently, and it's "psychologically acceptable" to the user.
Paige looks puzzled.
Paige: What?
Michael: Schedule me next week, and we'll discuss a couple of important security principles you should know: the Bell-LaPadula Disclosure Model and the Saltzer and Schroeder Secure Design principles.
Paige opens up Outlook and adds an appointment for the following week.
Paige: Back to our application: what about the integrity of the data and tamper detection?
Michael: DPAPI addresses that too!
Paige: Seems like a no-brainer to me.
Michael: Yes, and you're SDL compliant!
Both Michael and Paige grin as Paige updates the threat model.
Paige: So what about the Internet scenario?
Michael: This is where it can get very tricky. To be honest, I don't have a solution for you right now, and designing one is difficult as I need to know some things about your application. What operating systems do you support? Should the server administrators really have access to the data? How do you generate and store the encryption keys? What are your auditing requirements? Assuming the encryption uses data derived from the user, what if the user forgets their password? I could keep going!
Paige: Do you want to discuss this tomorrow?
Michael: Sure, we should dedicate a whole hour—actually, make it two hours!
But before we wrap up this chat, I need to ask a couple of code-level questions. First, you're using a Web application, so you're using the SDL-mandated ValidateRequest and ViewStateUserKey, right?
Paige: Absolutely.
Michael: And you're using an HTML encoding library where appropriate?
Paige: Yes. All input that comes from outside the server, and is then echoed back, including metadata from the database, is encoded using the Microsoft AntiXSS library.
Michael: Good. And cookies?
Paige: Marked as HttpOnly.
Paige opens Visual Studio and shows the following line of code from the Web server:
cookie.HttpOnly = true;
Michael: Excellent! SQL statements?
Paige: Using stored procedures and parameterized queries, and denying access to underlying database objects.
Michael: Bryan taught you well!
Paige: Thanks. Well, that was a very useful chat. I learned a lot today.
Michael: Good to hear. We still have a great deal to cover, but I think we're off to a good start. We can look at the rest of your threat model once we've addressed the Internet back-end data security issues. But for the moment, I think we've addressed some big issues.
Michael gets up to go back to his cubical. Then stops and turns around.
Michael: Oh, hang on. You will have a privacy statement on the Web site, right?
Paige: Yes. Our lawyers and privacy folks crafted one for us.
Michael: Good, that's another SDL requirement you've addressed.

Send your questions and comments for Security Briefs to briefs@microsoft.com.

Michael Howard is a Principal Security Program Manager at Microsoft focusing on secure process improvement and best practice. He is the coauthor of many security books including Writing Secure Code for Windows Vista, The Security Development Lifecycle, Writing Secure Code, and 19 Deadly Sins of Software Security.


Under the Table
Programming with FileStreams in SQL Server 2008
Bob Beauchemin
Code download available from the MSDN Code Gallery
Browse the Code Online
Contents
Programming Filestreams with Transact SQL
Programming Filestreams with File I/O
The SqlFileStream .NET Data Type
FileStreams and Transactions
Feature Synergy with Filestreams
There has always been lots of discussion about whether large blobs (binary large objects), such as document and multimedia items, should be stored in the database or in the file system. On one hand, the database is a specialized data repository that provides built-in integrated backup and restore, point-in-time recovery, transactions, indexing, and much more. On the other hand, having large data in a database can cause database fragmentation, and the nice recovery features also mean that the large object data is always written twice, once to the database itself and once to the transaction log. Even reading large data with SQL means using precious database buffers and flushing data that would otherwise be cached in memory out to disk as a side-effect.
To settle this controversy once and for all, Microsoft Research did a study on the subject and published their conclusions in a whitepaper "To BLOB or Not To BLOB: Large Object Storage in a Database or a Filesystem?" (See research.microsoft.com/apps/pubs/default.aspx?id=64525). Their conclusions were a setback for the "let's store everything in a database" opinion, as they tested a SQL Server 2005 database against the NTFS file system and recommended "BLOBs smaller than 256KB are more efficiently handled by SQL Server, while NTFS is more efficient for BLOBS larger than 1MB. This break-even point will vary among different database systems, file systems, and workloads." Some programmers attempt to have the best of both worlds by storing the location of an NTFS file in a database row, but this sacrifices transactional consistency, integrated query, and the other database features. In SQL Server 2008, you don't have to choose; a new feature known as the filestream storage attribute allows you to define columns in SQL Server tables with a specification that the data is actually stored in the file system. You can query the data with Transact SQL or streaming APIs using C++ and .NET-compliant languages, and retain all of the other database management features. In this column, I'll describe how you'd program this hybrid storage model using both types of APIs.
First, you may have noticed that I referred to the feature as the filestream storage attribute. The filestream is not a new data type; it's a new storage mechanism. It's available only with the varbinary(max) data type introduced in SQL Server 2005. To use filestream storage requires that the system administrator enable it at an operating system level using SQL Server Configuration Manager and the database administrator enable it on a SQL Server instance level using sp_configure. Filestream can be disabled, enabled for local access, or enabled for local and remote access. In addition, the DBA must define a database filegroup that ties an NTFS file system location to a SQL Server database. Note that the filegroup needs to point to a local file system location; filestreams can't live on a remote server or a network addressable storage (NAS) devices unless the NAS device is presented as a local NFS volume via iSCSI. Accessing the filestream uses the server message block (SMB) protocol, so if you're going to allow file I/O-style access from outside the machine where SQL Server is installed, you must allow access to the SMB port (usually port 445, with port 139 as a fallback) through the firewall. Once the administrative prerequisites are in place, you simply define a varbinary (max) column as part of a table definition with the FILESTREAM property and your data for this column is automatically stored in files. In addition, each table that uses a FILESTREAM column requires a column defined using the UNIQUEIDENTIFIER data type that has the ROWGUIDCOL property. A version of the Northwind database's Employees table that uses filestreams would look like Figure 1.
Figure 1 Employees Table Using Filestreams
CREATE TABLE [dbo].[Employees2](
  [EmployeeID] [int] IDENTITY NOT NULL PRIMARY KEY,
  [LastName] [nvarchar](20) NOT NULL,
  [FirstName] [nvarchar](10) NOT NULL,
  [Title] [nvarchar](30) NULL,
  -- filestream storage for photo column
  [Photo] [varbinary](max) FILESTREAM NULL,
  [ReportsTo] [int] NULL,
  -- identifier column
  [RowGuid] [UNIQUEIDENTIFIER]  NOT NULL
        ROWGUIDCOL UNIQUE DEFAULT NEWID()
);
Filestream uses both the traditional database log and a specific logging mechanism as an extension of traditional transaction log that records file changes. The extension is separate from the regular SQL Server database transaction log but integrated with the backup and restore utilities. For more information on filestream from an administrative and database internals point of view, refer to Paul Randal's excellent whitepaper "Filestream Storage in SQL Server 2008" at msdn.microsoft.com/library/cc949109.
From a SQL Server programmer's point of view, the filestream object can behave almost exactly like the varbinary (max) data type would with a few caveats. You can use Transact SQL to query the data without changing application programs at all. The simple Windows Forms application included in the code for this article needs no changes to display the Employees2 table I defined above in a DataGrid control. However, the value added from a performance perspective comes when using the filestream-specific API enhancements to read and write the data through stream-based APIs. An additional win is that the maximum size of the data in a particular column is no longer limited to 2GB for the filestream-based varbinary(max) column. The maximum size of these particular columns is unlimited. That means that you can store, for example, 7GB medical images, like X-Ray images, in a column in an ordinary SQL Server table. Filestream data does not count toward the 4GB maximum size limit in a SQL Server Express Edition database, so it can even be used with blobs in SQL Server 2008 Express Edition.
Before I go into the APIs, I'd like to stress that, once you've stored file data using SQL Server's filestream storage, you must not access or change the data outside of SQL Server's control. The directories where filestream's files live are protected by a discretionary access control list (DACL) that allows access only by the Windows group that contains the SQL Server service account and the system administrator account. And the system administrator can remove her access by changing the DACL. When a user tries to open a file using the streaming APIs, an access check is performed against the traditional SQL permissions on the database, schema, table, and column levels and NTFS permissions are ignored. Editing the filestream data directly using notepad.exe (or more likely, a specialized photo editor program) will usually result in a corrupt database. However, using the transactional streaming APIs that SQL Server provides, you could write your own SQL Server-based "transactional Notepad" program.

Programming Filestreams with Transact SQL
Although programming filestreams with T-SQL is just like ordinary T-SQL programming, there are a few caveats. First, partial updates to the filestream column using the varbinary(max) WRITE method are not allowed. In addition, because the streaming APIs require a file path name supplied by SQL Server (that is, the streaming APIs can only be used when the filestream column value is non-NULL),, T-SQL is the only way to obtain the file name. Although inserting a NULL value into a filestream column will not create a file, INSERTing any non-NULL value will cause a file to be created. An UPDATE to a filestream column will cause the old file to be deleted and a new file to be created in its place. A DELETE operation on a row will cause the corresponding file to be deleted. Note that the file deleted with an UPDATE or DELETE operation will not disappear from the file system immediately; a garbage collector thread will be used to physically delete the file and reclaim the space.
All of the T-SQL built-in functions that work with varbinary(max) work with a filestream column, including SUBSTRING, REPLACE, LEN, and CHARINDEX. Filestream storage is permitted only for columns in database tables; variables, parameters (including table-valued parameters), columns in temporary table structures, and table-valued function return values may not use the FILESTREAM attribute. Also note that the FILESTREAM attribute is available only on varbinary(max) columns; other large data types like varchar(max) and XML may not specify the filestream attribute. If you want to store character or XML data in filestream storage, you must specify the column as varbinary(max) and use CAST or CONVERT to process the data as characters or XML.
In order to access the filestream data using the streaming APIs, you must first use T-SQL to obtain a path name; this is accomplished with the (case-sensitive) PathName() method on the filestream column. Filestream's logical paths are versioned, and in the version 1 format that's used by SQL Server 2008, the path name is tied to (but not equal to) the value of the ROWGUIDCOL column in the same row. Although you can use filestream columns in views and derived tables, be sure to keep the ROWGUIDCOL around in case you need to use the PathName() method. For example, for table T that contains a filestream and a rowguid column, write the code in Figure 2.
Figure 2 Create a View
CREATE VIEW View1
AS
SELECT RowGuidColumn , FileStreamColumn
FROM T;

SELECT FileStreamColumn.PathName() FROM View1;  -- works
GO

CREATE VIEW View2
AS
SELECT FileStreamColumn FROM T;
GO

-- Fails because it is missing the RowGuidColumn
SELECT FileStreamColumn.PathName() FROM View2;
GO
Finally, data encryption is not supported on filestream columns. This is not only true for the data encryption APIs like EncryptByKey, but also for the SQL Server 2008 Transparent Data Encryption feature. Although databases with filestream storage may specify Transparent Data Encryption, the filestream files will not be encrypted.

Programming Filestreams with File I/O
Before using streaming I/O with the filestream storage column, there are some requirements on the client side. You must use an Integrated Security account when using streaming I/O as there is no way to pass SQL credentials when opening the handle.Because filestream storage was implemented using a special file system filter driver, programming with a column that uses filestream storage involves working with a file handle that does not support all Win32 operations. The nativeWin32 function that retrieves the file handle is OpenSqlFilestream. This function is part of the SQL Native Client 10 DLL that also contains the SQL Server ODBC driver, OLE DB provider, and network libraries. The function returns a Win32 handle that supports most of the streaming functionality of a Win32 file handle. While obtaining a Win32 file handle requires a path name and some additional options, the OpenSqlFilestream function requires two pieces of information that can be provided only by SQL Server. These are the name for the file returned by calling the PathName() method on the column that uses filestream storage and a transaction token. This means that when you're programming with OpenSqlFilestream, you're splitting the INSERT, UPDATE, or SELECT statement into the equivalent of two "SQL" statements, the T-SQL statement and the streaming statement. These statements must be tied together with a transaction. Although I'll mention more about the transaction and the supported isolation levels later, for now, it's enough to know that you must call a T-SQL function, GET_FILESTREAM_TRANSACTION_CONTEXT() to obtain the transaction token. This token must be obtained in the context of same windows user as the one opening the file. Notice that the original SQL statement to get the transaction token must be part of a transaction, otherwise the transaction token will be NULL and OpenSqlFilestream will fail. You may not commit the transaction until the HANDLE is closed. The file handle APIs that are supported using the special file handle are ReadFile, WriteFile, TransmitFile, SetFilePointer, SetEndOfFile, or FlushFileBuffers. Attempting to call any other file API returns ERROR_ACCESS_DENIED. Memory-mapped files are specifically not supported with the special HANDLE.
A complete example of using ODBC and C++ to insert a new row and filestream data using OpenSqlFilestream is provided in the SQL Server 2008 Books Online.
With a SELECT statement, you'll make only a single round-trip to the database for your PathNames and transaction token; with an INSERT or UPDATE statement, you'll want to make only one database round-trip as well. With these statements, the T-SQL OUTPUT clause, introduced in SQL Server 2005, comes to your assistance. Here's an UPDATE statement that processes a single row and gets all of the information you need in one round-trip:
UPDATE dbo.Employees2
SET name = 'NewName' WHERE id = 8
OUTPUT inserted.photo.PathName(),
       GET_FILESTREAM_TRANSACTION_CONTEXT()
I haven't mentioned using the DELETE statement because deleting a row will also delete the file; you cannot use streaming I/O to perform a DELETE. However, there are some things you need to know when using INSERT or UPDATE to read and write the blob. In addition, I'll divide the blob update into two use cases: complete replacement (rewrite) and partial update.
Inserting a row that contains a blob using INSERT is one of the best reasons for using streaming I/O. SQL Server APIs do not support streaming input using a varbinary(max) column, although you can write the data in chunks by using the T-SQL STUFF function or the WRITE method. (Remember that the WRITE method is prohibited when using filestream storage.) The interesting twist with INSERT is that inserting a NULL value creates no file. The PathName() method would also return NULL and you'd have no file for streaming the data. The way to approach an INSERT is to insert an empty string rather than a NULL value, retrieve the path name and stream the content into the empty file. The T-SQL statement would look something like this:
INSERT dbo.Employees2 (id, ... photo)
VALUES(1, ... CAST('' as VARBINARY(MAX))
OUTPUT inserted.photo.PathName(),GET_FILESTREAM_TRANSACTION_CONTEXT()
You can then open the file for writing and stream in the data. A complete replacement UPDATE would work like an INSERT. You'd update the non-filestream columns, return the path name and transaction token, and stream data in through the file handle. Unless you know that your filestream storage column already contains data, it's a good idea to set the data to the empty string in the T-SQL update statement.
A partial update of a filestream column is a little more complicated because you may want to read the information in the file before doing an update. To accomplish this, you can use the DeviceIoControl function with the handle and the FSCTL_SQL_FILESTREAM_FETCH_OLD_CONTENT parameter. This causes a server-side copy of the contents of the file. After performing this call, ReadFile will return the appropriate data before the call ReadFile will return End Of File. If you're reading and updating this way, it's a good idea to use overlapped I/O for high performance apps and especially those that may access filestream from the same machine as Sql Server. Bear in mind that if you need to do a lot of partial updates, using filestream storage is much slower than using SQL Server's varbinary(max) without filestream storage.

The SqlFileStream .NET Data Type
.NET programmers don't usually want to deal with Win32 file handles. Although it's possible to use OpenSqlFilestream in .NET programs using a technique called PInvoke (platform code invoke), it would be more convenient to have the OpenSqlFilestream function and file handle access encapsulated in the .NET class. In .NET Framework 3.5 SP1, the class System.Data.Types.SqlFileStream fills the bill nicely. The usage is mostly the same; the SqlFileStream's constructor requires a path name and a transaction token, as well as some options. Using SqlFileStream to INSERT a row is illustrated in the code download that accompanies this column.
The SqlFileStream object derives from System.IO.Stream, and you use it as you would use an "ordinary" .NET Stream. Some important differences between using the SqlFileStream API and the OpenSqlFilestream Win32 function is that SqlFileStream uses a default buffer size of 4K; OpenSqlFilestream's handle does not. In addition, using SqlFileStream for ReadWrite mode will automatically perform the DeviceIoControl calls as a convenience; the Native API does not support this and "partial update" patterns will involved explicit DeviceIoCtrl calls. To use the SqlFileStream data type, you must have .NET 3.5 SP1 must be installed on the client or web server.

FileStreams and Transactions
As mentioned previously, one of the attractions of using filestream storage instead of simply storing file names in SQL Server and files on the file system is that the data is transactionally consistent. If you insert a row in Transact SQL and insert a file using the streaming APIs, the entire operation either succeeds or fails. There is no possibility of having file names of files that do not exist or orphaned file system items that do not have a corresponding entry in SQL Server. The OpenSqlFilestream API ensures this by requiring you to have a valid, open transaction at all times when using the streaming APIs. But SQL Server supports a variety of transaction isolation levels and two different transaction semantics: locking and versioning. How well does the streaming APIs work with all of the isolation levels, given that SQL Server's lock manager does not manage file system locks?
When using streaming, it's helpful to think of your atomic operation as consisting of two distinct SQL statements--one for the T-SQL portion and a second statement for the streaming portion. In order to obtain the transaction token, you need to perform the T-SQL portion first. Before executing the T-SQL command you must begin an explicit transaction by using the ADO.NET SqlConnection.BeginTransaction method or System.Transactions' TransactionScope; analogous methods for manual and automatic transaction management are surfaced on other APIs like ODBC. The transaction must be kept open until the file handle is closed. Issuing a SqlTransaction.Save call is disallowed while the file handle is open and will make the transaction uncommittable. A SqlTransaction.Rollback call issued will roll back the transaction, even if the file is opened. Triggers fire when the file handle is closed.
Using the streaming APIs and filestream storage is permitted under all four locking transaction isolation levels, although the semantics of streaming access is always similar to read committed. Using read-committed isolation level (the default), if the file is opened for reading, other readers will be able to read the file—writers will be blocked. If the file is opened for reading, it remains blocked until the end of the transaction. Read-uncommitted isolation level is allowed with the caveat that if an update of the file is in progress, the read-uncommitted transaction will read the old version of the file, rather than the new, proposed version as is the usual read-uncommitted behavior. Repeatable read and serializable behavior is the same when using file streaming as when rows are locked in the database.
Versioning isolation levels—that is, read-committed snapshot isolation and snapshot transactions—are not allowed in a database that uses filestreams. This is true for the entire database, not just the table that contains a filestream column. The ALTER DATABASE statement that enables read-committed snapshot isolation and that enables snapshot transactions will fail in the presence of filestream storage.

Feature Synergy with Filestreams
Filestream is useful for storing large data in a database without incurring the transaction log overhead and database fragmentation behavior, but on occasion it will be useful to hoist a portion of the data to a persisted computed column. This allows you to do queries on properties of the blob without reading the file at all. As an example, say that you were storing photos in JPEG format and would like to store portions of the color table, such as background color or photo height and width, separately. You can simply define a persisted computed column for that part of the varbinary(max) column; the data is stored in-row. An example, using the Employees2 table defined above, would look like this:
ALTER TABLE dbo.Employees2
  ADD PhotoWidth AS dbo.ExtractWidth(Photo) PERSISTED;
A SQL query to obtain the photo width does not need to access the file at all:
SELECT Id, Photo
FROM dbo.Employees2
WHERE PhotoWidth > 1200;
Be aware, however, that indexes on persisted computed columns of type varbinary are disallowed.
A great example of feature synergy is the combination of the SQL Server fulltext indexes and fulltext searching with filestream storage. Fulltext search filter components exist for document types, such as Microsoft Office documents, PDF files, and text files, and filestream storage allows these files to be stored in the file system rather than in the database. As a concrete example, you can store copies of error logs or Web log files in a filestream storage column and fulltext index the column. In SQL Server 2008, fulltext searching runs in-process, so it's not necessary to make out-of-process calls to the search service. A query like the one below works in-process and can be quite fast:
SELECT id, Abstract
FROM error_logs
WHERE CONTAINS(ErrorText, 'unhandled');
I began this article by suggesting filestream storage provided not only integrated backup and restore, but transactional consistency between SQL data and file data, although filestream storage could use only local disk storage. If you're concerned only with transactional consistency and want the files to reside on a remote server or content-addressable storage, SQL Server 2008 includes a companion feature, Remote Blob Storage (RBS). This feature is available as a free download as part of the SQL Server 2008 Feature Pack. RBS consists of an API that permits transaction manipulation and a series of stored procedures for handling blob pointers in database tables, blob garbage collection, and transaction participation. Content-addressable storage vendors provide drivers for their specific hardware. A sample RBS driver for the NTFS file system is available as a download on CodePlex and EMC has release a beta RBS provider for their Centera content-addressed storage system. The RBS API is not similar to the filestream API, but these APIs may be aligned in future.
In conclusion, filestream storage gives you the ability to store large blobs as files in the file system and access them with either T-SQL statements or transactional streaming APIs. With this feature, you can gain the performance of streaming as well as full database integration for large blobs.

Send your questions and comments for Bob to mmdbdev@microsoft.com

Bob Beauchemin is a database-centric application practitioner and architect, course author and instructor, writer, and Developer Skills Partner at SQLskills . He's written books and articles on SQL Server, data access and integration technologies, and database security. You can reach him at bobb@sqlskills.com.


Patterns and Practices
You Can Depend On Patterns and Practices
Alex Homer
Contents
Patterns and Practices
A Guiding Hand
Coping with Abstraction
A Brief History of Dependency at Microsoft
Extending Yourself
Summary
Writing computer code used to be extremely difficult. When I began programming home computers more years ago than I care to remember, the only way to make a program run at more than a desultory crawl was to write it using machine code. Even a simple word processing utility could take several weeks to create as you meticulously planned the memory allocations for variables, wrote routines to perform simple tasks such as drawing characters on the screen and decoding keyboard input, and then typed every individual processor instruction into an assembler.
Today, by comparison, creating powerful code is easy. We take for granted the wealth of tools, high-level languages, runtime frameworks, code libraries, and operating system services that make it possible to build applications quickly and efficiently. However, while creating the code has become much easier, the task of creating applications has not. We expect much more from our modern applications than ever before. They must communicate over networks, interact with other applications and services, expose highly interactive and responsive user interfaces, and support rich media and graphics. And, of course, they must be robust, secure, reliable, and manageable.
In fact, it would be almost impossible to meet the requirements demanded of our modern applications without the continuing growth and evolution of programming languages, operating systems, platform services, and frameworks. The huge increase in complexity of these applications has also forced us to discover ways to simplify and organize the code. Over the years, a number of tried and tested programming and design techniques have evolved, such as componentization, object-orientation, and—more recently—service orientation.

Patterns and Practices
While improvements in tools and frameworks make it easier to write code more quickly, and modern languages and programming styles make it easier to write better code, the one area that has had the most impact on our ability to create applications over the last 10 to 15 years has been the evolution and increasing acceptance of software design patterns.
Design patterns describe common issues that occur repeatedly in application design and development and provide techniques for handling these issues. Patterns also describe the current industry practice for resolving architectural issues and for handling the design complexity demanded of applications. The arrangement of code and solution elements is at the heart of software design today, and patterns provide us with ways to simplify and organize these elements—providing the best opportunity to maximize performance, flexibility, and maintainability for applications.
Since the release of the ground-breaking book by the Gang of Four, Erich Gamma, Richard Helm, Ralph Johnson, and John M. Vlissides, Design Patterns: Elements of Reusable Object-Oriented Software (Addison-Wesley, 1996), which documented many of the basic design patterns we take for granted today, the number of design patterns available to developers and designers has grown at a phenomenal rate. Almost every aspect of the creation of software has associated design and implementation patterns, and simply documenting them all has become an impossible task. Instead, designers and developers tend to fragment into specialties and learn the patterns most applicable to their own area of expertise.

A Guiding Hand
In 2002, the patterns & practices group at Microsoft Corporation published the Application Architecture for .NET: Designing Applications and Services guide, bringing together the basic design guidance and specialist advice to help architects build applications on the Microsoft .NET Framework. However, technologies change over time, and while the fundamentals of design and development described in the guide are equally valid today, the original guide did not cover some of the new capabilities of the .NET Framework or address the design issues arising for new types of applications.
For the last several months, a broad team of industry specialists and subject matter experts, both within and outside of Microsoft, has been collaborating to create a new edition of the guide that provides a comprehensive introduction to architecting and designing solutions on the Microsoft .NET Framework. The Microsoft Application Architecture Guide (2nd Edition) describes high-level architectural principles and patterns and current thinking on accepted design and development practices. It also includes guidance on applying these factors to specific types of applications, and it provides a broad overview of the many platform services and code libraries available to make implementation easier.
Of course, to borrow an often-used phrase, software and system architecture is something that embraces the question of life, the universe, and everything. No single publication can answer every possible question or hope to provide completely comprehensive coverage of every possible scenario. However, the new guide aims to provide the information you need whether you are just starting out designing.NET Framework-based applications, an experienced designer moving to the .NET Framework from another platform, an accomplished architect looking for specific information and advice, or simply interested in learning more about the broad scenarios and opportunities offered by the .NET Framework.

Coping with Abstraction
To illustrate industry practices, the guide lists general principles that you should apply when designing almost any type of application. These principles include maintaining separation of concerns, using abstraction to implement loose coupling between layers and components, implementing service location capabilities, and managing crosscutting concerns such as logging and security. While these may seem to be desirable but unrelated aims, one technique can help you to apply several design principles easily. The Dependency Inversion principle implies separation of concerns through abstractions rather than concrete implementations. In terms of design patterns, you can achieve this by applying the Inversion of Control (IoC) pattern and its related pattern, Dependency Injection (DI).
The theory is simple enough. Instead of specifying at design time the actual concrete type that each class or component will use to perform some activity or process, you arrange for these classes or components to retrieve the appropriate object from a container that you previously configured with type maps and registered types. For example, as illustrated in the simple application in Figure 1, your data access component may require the services of a logging component. As you have correctly abstracted your crosscutting code from the task-specific and application-specific code, you may have several logging components from which to choose. Perhaps one is designed for use when debugging the application, another for use when running the application internally on your own network, and a third most suited to running in an enterprise-level system that makes use of a monitoring environment, such as Microsoft System Center.
Figure 1 A Simple Application That Makes Use of Layering and a Separate Logging Component
It could even be the case that you have multiple data access components, each designed for use in a specific environment or with a different type of data store. Therefore, your business layer will need to choose the appropriate data layer component depending on the current deployment or runtime environment. In a similar way, you may have services that your application uses repeatedly, such as an e-mail message sending service or a data transformation service. Dependency injection can act as a service location facility to help your application retrieve an instance (either a new instance or an existing instance) of the service at run time.
Each of these examples effectively describes a dependency of one part of the application on another, and resolving these dependencies in a way that does not tightly couple the objects is the aim of the Dependency Inversion principle.

A Brief History of Dependency at Microsoft
Although the principles of Dependency Inversion have been around for a long time, features to help developers implement it in applications running on the Microsoft platform are relatively recent. In fact, there is a story that a renowned developer in the Java world, when visiting the Microsoft campus, remarked that the general belief was that nobody at Microsoft could spell "Dependency Injection." While that is, without doubt, an urban myth, it is the case that tools to help developers implement many of the common patterns have not been a priority in most areas of the company.
However, the patterns & practices group at Microsoft stands out through our unique position of being inside the company but outside of the main product development teams and divisions. The goal of p&p, as illustrated by our tagline "proven practices for predictable results," is to provide developers with guidance, tools, libraries, frameworks, and a myriad of other facilities to help them design and build better applications on the Microsoft platform. A quick glance at our home page on MSDN will illustrate the broad range of assets we provide.
Among these assets are several products that make use of the Dependency Injection pattern, including Enterprise Library, composite application frameworks, and software factories. During development of these assets, in particular the original Composite Application Block (CAB), it became clear that a reusable and highly configurable dependency injection mechanism was required—and so the team built the original version of Object Builder.
Object Builder is almost completely configurable and is now used in a wide range of products throughout p&p and elsewhere within Microsoft. However, it is quite difficult to use. It requires a great many parameters that take complex objects, and it exposes a range of events that you must handle to apply the configuration you require. Initial attempts to document Object Builder as part of the CAB project soon revealed that this was going to be an uphill task. In addition, Object Builder was rather more than a dependency injection container, and it seemed overkill in terms of the common requirements for implementing the DI and IoC patterns.
During the development of Enterprise Library 4.0, Object Builder was updated—not to simplify it, but to make it faster and more efficient. It was also fine-tuned for use in the first major dependency injection mechanism from Microsoft that is aimed squarely at developers who want to implement the DI and IoC patterns. Object Builder is the foundation for Unity, a lightweight, extensible dependency injection container that supports constructor injection, property injection, and method call injection.
Unity provides capabilities for simplified object creation, especially for hierarchical object structures and dependencies; abstraction of requirements at run time or through configuration; simplified management of crosscutting concerns; and increased flexibility by deferring component configuration to the container. It has a service location capability, and allows clients to store or cache the container—even in ASP.NET Web applications.
Since the release of the first version Unity in early 2008, it has found a home in many p&p assets as the default mechanism for implementing Dependency Inversion. Unity has also continued to evolve while remaining backward compatible; you can use it to enable features within Enterprise Library, as well as use it as a stand-alone DI container. In the most recent release, it offers facilities to implement instance and type interception (through a plug-in extension) that allow implementations of Aspect Oriented Programming techniques such as policy injection.
Unity has also spawned other DI container implementations aimed at specific tasks and requirements, such as an extremely lightweight implementation designed for use in mobile devices and smart phones. Meanwhile, planned future developments in the Unity and Enterprise Library arena include features to open up Enterprise Library to other third party container mechanisms, while providing additional extensions that enable new capabilities for Unity. Applying Dependency Inversion
Leaving this historical distraction and returning to the hypothetical application, how can you apply the Dependency Inversion principle to achieve the aims, discussed earlier, of separation of concerns, abstraction, and loose coupling? The answer is to configure a dependency injection container, such as Unity, with the appropriate types and type mappings and allow the application to retrieve and inject instances of the appropriate objects at run time. Figure 2 illustrates how you can use the Unity application block to implement this container. In this case, you populate the container with type mappings between interface definitions for the data components and logging components, and the specific concrete implementations of these interfaces that you want the application to use.
Figure 2 Dependency injection can select the appropriate components at run time based on configuration of the container.
At run time, the business layer will query the container to retrieve an instance of the correct data layer component, depending on its current mapping. The data layer will then query the container to obtain an instance of the appropriate logging component, depending on the mapping stored for that interface type. As an alternative, the data and logging components may inherit from respective base classes, and registrations in the container can map between these base types and the inheriting concrete types.
This container-driven approach to resolving types and instances means that the developer is free to change the implementations for the data and logging components, as long as these implementations provide the required functionality and expose the appropriate interface (for example, by implementing the mapped interface or inheriting from the mapped base class). The container configuration may be set in code at run time using methods of the container that register types, type mappings, or existing instances of objects. Alternatively, you can populate the container by loading the registrations from a configuration source or a file, such as the web.config or app.config file.
When you want to register more than one instance of a type, you can use a name to define each one and then resolve the different types by specifying the name. The registration can also specify the lifetime of the object, making it easy to achieve service location-style capabilities by registering the service object as a singleton or with a specific lifetime, such as per-thread. The following code example shows some examples of registering type mappings with the container:
C#
// Register a mapping for the CustomerService class to the IMyService interface.
myContainer.RegisterType<IMyService, CustomerService>();

// Register the same mapping using a mapping name.
myContainer.RegisterType<IMyService, CustomerService>("Data");

// Register the first mapping, but as a singleton.
myContainer.RegisterType<IMyService, CustomerService>(
                         new ContainerControlledLifetimeManager());
Note: The code examples reference classes and types using just the class name. You can use type alias definitions within the configuration file to alias the fully qualified type names of classes, which simplifies container registration when you're using a configuration file.
To retrieve instance of an object, you simply query the container by specifying the type, the interface type, or the base class type (and the name, if you registered the type using a name), as shown in the next example. The container resolves the type, if it is registered, and creates or returns an instance of the appropriate object. If it is not registered, the container simply creates a new instance of that type and hands it back. Why would you resolve items through the container when there is no registration for that type? The idea is to take advantage of the additional and very useful features that Unity, and many other DI container mechanisms, provides—the ability to inject objects using constructor, property setter, and method call injection.
C#
// Retrieve an instance of the mapped IMyService concrete class.
IMyService result = myContainer.Resolve<IMyService>();
// Retrieve an instance by specifying the mapping name.
IMyService result = myContainer.Resolve<IMyService>("Data");
For example, when you create an instance of an object through the container, Unity examines the constructors and will automatically inject instances of the appropriate type into the constructor parameters. Going back to our earlier simple application example, the data access component may have a constructor that takes a reference to a logging component as a parameter. If this parameter type is the interface or base class for logging components registered with the container, Unity will resolve the mapped type, create an instance, and pass it to the constructor of the data component (as shown in Figure 3). You do nothing except register the mappings.
Figure 3 Injecting Objects into Constructor Parameters
C#
// In main or startup code:
// Register a mapping for a logging component to the ILogger interface.
// Alternatively, you can specify this mapping in a configuration file.
myContainer.RegisterType<ILogger, MyLogger>();

...

// In data access component:
// Variable to hold reference to logger.
private ILogger _logger = null;

// Class constructor. Unity will populate the ILogger type parameter.
public DataAccessRoutines(ILogger myLogger)
{
  // store reference to logger
  _logger = myLogger;
  _logger.WriteToLog("Instantiated DataAccessRoutines component");
}
This means that you can change the actual concrete type that the application uses simply by changing the configuration of the container—either at design time, at run time by editing the configuration, or dynamically based on some value that your code gathers from the environment and uses to create or update the mapping in the container. You can plug in your debug logging component when required, or plug in a new "super fast" logging component when you find that the old one is too slow. Meanwhile, the system administrator can update the configuration as required to monitor, manage, and adapt the application behavior at run time to suit changing environments and operational issues.
Likewise, if you have a dependency between two objects, such as the dependency of a view on its presenter when you implement the Model-View-Presenter (MVP) pattern, you can use dependency injection to loosen the coupling between these classes. Simply define a property of the view class as the presenter type or base class type, and mark the property with a Dependency attribute, as shown in the next example:
C#
// Variable to hold reference to controller.
private IPresenter _presenter;

// Property that exposes the presenter in the view class. Unity will inject
// this automatically because it carries the Dependency attribute.
 [Dependency]
public IPresenter MyViewPresenter
{
  get { return _presenter; }
  set { _presenter = value; }
}
Note: Attributes are the quickest way to specify properties to inject. If you do not want to use attributes (to avoid coupling your classes to the container) you can instead use a configuration file or the Unity API to specify which properties should be injected.
When you create the view by resolving it through the container, Unity will detect the Dependency attribute, automatically resolve an instance of the appropriate concrete presenter class, and set it as the value of that property of the view class. The Quick Start example included with Unity demonstrates this approach in a Windows Forms application. It actually resolves the main Form for the application through the container, which causes Unity to create and populate several dependencies throughout the entire application, using both constructor and property setter injection.

Extending Yourself
Unity provides a wealth of DI-related functionality, but there is always something extra you want to achieve. The challenge with Unity was to keep it generic enough to satisfy the maximum number of requirements, while being extensible so that you can adapt it to your own specific requirements. This is achieved by using container extensions, which allow you to do almost anything in terms of managing object creation and retrieval.
As an example, the Quick Starts included with Unity demonstrate a container extension that implements a simple attribute-driven Publish\Subscribe mechanism. As Unity creates instances of objects, it wires up event handlers for them based on attributes in the class files. Another example generates detailed logging information as it creates or retrieves each type through the container, which helps when you're debugging complex applications.
This huge flexibility comes about because Unity allows you to interact with the underlying Object Builder mechanism through your container extension. Ah, you may say, but Object Builder is extremely difficult to use and is not fully documented. In fact, the Unity documentation does contain information about Object Builder in terms of how you interact with it from a container extension, and the Quick Start examples provide plenty of sample code you can use and modify.

Summary
There are many views on application architecture and design. The ISO/IEC standard 42010:2007/IEEE 1471 "Recommended Practice for Architecture Description of Software-Intensive Systems" describes software architecture as "the fundamental organization of a system embodied in its components, their relationships to each other and to the environment, and the principles guiding its design and evolution". However, in his book Patterns of Enterprise Application Architecture (Addison-Wesley, 2002), Martin Fowler says that "...in the end, architecture boils down to whatever the important stuff is," which is a much simpler way to capture the spirit of software architecture!
The Microsoft Application Architecture Guide (2nd Edition) will help you understand what the important stuff is so that you can build better, higher quality applications more quickly and more efficiently. As you have seen in this article, one specific area—taking advantage of the Dependency Injection and Inversion of Control patterns—can help you achieve many of the design goals promoted by the guide. This includes separation of concerns, the use of abstraction to implement loose coupling between layers, service location, and implementation of features for improved management of crosscutting concerns.

Alex Homer is a documentation engineer working with the Microsoft patterns & practices team. His random ravings about life, technology, and the world in general can be found at http://blogs.msdn.com/alexhomer/.


Usability in Practice
Useful, Usable and Desirable: Usability as a Core Development Competence
Dr. Charles B. Kreitzberg and Ambrose Little
Contents
THE COGNITIVE VIEW
Useful—The Foundation of User Satisfaction
Usable
Desirable
Usability is Much More than Skin Deep
Usability: Do it Early and Often
Wireframes
It Doesn't Slow You Down
Your Personal Journey
THE SOFTWARE VIEW
Useful
Desirable
What About Developers?

THE COGNITIVE VIEW
Dr. Charles B. Kreitzberg
Every now and then I need an "elevator speech" to explain why design matters. An elevator speech, in case you haven't come across the term, is an explanation that can be delivered in the time it takes to get from the lobby of an office building to the 10th floor. I usually tell people that as a user experience designer my job is to design interactive products that are useful, usable and desirable. The phrase useful, usable and desirable resonates with me because it captures the outcome we hope for with every development project.

Most every interactive product we develop—whether it's an application, a game, a Web site or a mobile device—is some form of tool.
And no tool can be successful without being useful. Creating useful products has been a fundamental goal of the development community from the beginning and a lot of good thinking has gone into figuring out how best to understand and document requirements. Getting the functionality right, however, is only part of what makes a tool useful.
Figure 1 The E3T Model
Useful—The Foundation of User Satisfaction

Recently I came across the phrase "usability exposes usefulness" and I thought it was an interesting comment. Finding functionality and being able to use it in the way you want requires a usable design. Obviously, the more usable a tool is, the better it works and the more useful it is. So if we want to build truly excellent products, they must be usable; you can't really make a truly useful tool if it's not really usable.

Desirability refers to a person's affective (emotional) feelings about a product. Many factors affect the desirability of a product. Four of the most useful are shown in a model I developed which I call the "E3T model." E3T refers to the four dimensions shown in Figure 1.
How important is desirability? It's obvious that products we develop for sale be highly desirable, to attract and retain users. But even within the business world, people are demanding more from software products, and they know good design when they see it. (Figure 2 explains the elements of the E3T model).
Figure 2 Characteristics of the E3T Elements
Dimension | Desirability
Engagement | Is the visual design appealing? Do people find the product attractive and interesting? Is there a "wow" factor?
Empowerment | Does the product make the person feel smart and competent? Does the tool make the person feel more powerful?
Ease of Use | How much effort does it take to learn and use the product? Is it flexible and supple and does it allow people to use it the way they want to?
Trust | Is the product accurate? Is it reliable and predictable? If it makes recommendations, do people feel they are in their best interests?
So if your goal is to create useful, usable and desirable products, you will need to integrate usability thinking into the development process. In another installment, we will talk about how to fit user experience design into an agile development process. In this article, I want to focus on creating a high-level design.

Usability is Much More than Skin Deep
Often developers think of usability as the product's skin. While some aspects of usability are related to the visual design, much of it is below the surface.
Fourteen years ago, in his book About Face, Alan Cooper made a distinction between two software models: the technical model and the manifest model through which the software "represents its functioning to the user." I prefer the term UI Model to Manifest Model, but whatever term you use, it's an important distinction.
While the technical model is built of such items as objects, methods, algorithms, and data structures, the UI model is built around tasks, people, and business objects. Obviously there needs to be alignment between the technical and UI models. Developers are, of course, most comfortable with the technical model and tend to focus on it. As a result, in most development environments, a formal UI model is never constructed. Instead, the developers agree on a basic framework and the UI is constructed in an ad hoc manner when the screens are coded.
Bringing user-centric design into the development process means that you (or someone on the team) will gather information about the users and produce a UI model of the product.

Usability: Do it Early and Often
Developing a high-level design early helps ensure that everyone working on the project shares a clear idea of what the end result should be.
I think about the UI design in two stages: a high-level conceptual design and a detailed design. This approach is analogous to the way that functional requirements are developed: first a high-level functional spec is created and then detailed (field-level) requirements are defined.
Breaking the design into high-level and detailed designs is particularly important in an agile environment where some details may not be defined until late in the development process. The high-level design serves as a framework that ensures consistency of the UI. The detailed design can then be spread across the iterations.

Wireframes
The high-level design is usually presented as wireframes and a style guide. Wireframes are drawings that show how screens would look and how the user would interact with them. A style guide is a document that provides the rules for constructing screens. It covers such issues as control placement, standards for labels, colors, and fonts. It should also provide rules for data display. The goal of a style guide is to ensure that all screens use consistent elements.
A variety of tools are used to create wireframes. A popular tool is Visio which produces mockups quickly. But Visio suffers from a number of limitations. It's a generalized drawing program rather than one designed to for creating screen mockups. And it has limited ability to apply behaviors to the screens which limits its ability to show interaction. Recently, more specialized products have emerged. Axure and iRise are two examples that offer more sophisticated screen layout capabilities.
But it seems to me that in a .NET environment, it makes sense to create wireframes in Visual Studio, using the same controls that will be used in the product. You can use master pages to create a consistent framework and build in limited functionality to show how interactions would work and end up with a UI prototype that looks and acts like the real thing.
It's important not to go too far with your prototypes. Like almost everything in UX design, wireframes are refined iteratively. At early stages they can be schematic and you can add detail as you learn more about the users and their requirements. Many developers tend to put too much sophistication into wireframe prototyping. Remember that in the early stages you are building a high-level design. You probably don't need a database, or data validation. The UI prototype is intended to create the illusion of a working product; it's not an early iteration of the actual code.
I call the UI prototype I create at this stage a key screen prototype because its purpose is to show the main (or key) screens, the navigational flow, and the information architecture.
There are three ways you can use the key screen prototype. A primary use is as a communications tool to make certain that everyone involved in the project is on the same page. Many problems that plague software projects have their origins in poor communication and misunderstandings between the development team and the business clients. Developers often overestimate their customers' ability to understand what they are agreeing to. The result is that problems are not discovered until late in the project when it's harder and more expensive to deal with them. If you can show a model of the product early in the development cycle you can clearly show stakeholders what the product will look like and how it will work.
When demonstrating wireframes to stakeholders in a review session, I find that it is best to show them in terms of task flows. Showing a screen and inviting the participants to criticize it tends to elicit a lot of off-the-cuff opinion which can be counterproductive. Instead, I demonstrate flow by showing how a user would perform major tasks. Later, I'll work one on one with trusted partners to discuss details.
It's important to decide how to handle the visual design. Some designers prefer to keep the wireframes schematic so that stakeholders don't respond to a visual design they dislike by criticizing other aspects of the design. In the early stages, I typically use schematic wireframes and create separate screens to display the visual design. However, once we agree on a visual design, I integrate it into the wireframes to make the UI prototype as realistic as possible.
A second use of the wireframes is to support usability testing. The goal is to make certain that that the UI is intuitive and makes sense to the users. While it is certainly possible to get useful information from a paper prototype or by displaying drawings on screen, I prefer to make the interaction as natural as possible—after all, that's what you are testing. It is useful to make menus, buttons, and links work so the user can browse from screen to screen. If controls link to screens not yet designed or that I cannot support (such as search results) I'll take the user to a placeholder screen. I design the sample content so it makes reasonable sense as the user goes from screen to screen but limit the work by not making it 100% realistic or consistent.
A third use of the wireframes is to provide a mock-up of the product to facilitate the creation of content such as sales and marketing collateral and user support content. The key screen prototype can help content authors get started earlier and make their jobs more efficient. Often my prototypes are taken to trade shows where they are shown to prospective customers to get their reactions.
The benefits of creating a wireframe UI prototype are so overwhelming that I cannot understand why it has not become a universal practice.

It Doesn't Slow You Down
It's common for developers to be concerned that the UX design processes will slow the project down. There is no question that if you start the UX design too late there is a high probability of conflict. It is far better to start the UX design process well before the technical design. Done this way, UX design will speed up the development project. Having a set of wireframes and customers who understand what they are getting can eliminate missteps and rework. If you are working in an agile environment, having a great set of wireframes for iteration zero will jumpstart the entire process. And the UX design is not extra work. You would still need to talk with users, develop screens and review them. UX design almost always brings the cost of projects down, sometimes a lot and it improves customer satisfaction a great deal.

Your Personal Journey
The decision to make user experience design a core competence is both an institutional and personal decision. Unless you are a sole practitioner, the company you work for needs to decide that useful, usable and desirable products are a goal worth striving for and it must reward the people who help achieve it so that the organizational culture begins to integrate UX design.
But there is also a personal journey you can take, as you become increasingly aware of UX. You may want to do UX design or you may ultimately prefer that others take on the task. What is important is that you understand how to recognize good design and are clear about the practices that produce it.
Here are a few ways you can increase your awareness and understanding of user experience engineering:
  • Look for examples of poor usability in everyday life. Road signs and airports are rife with them. Instructions that make no sense, a latch you can't figure out how to undo, confusing menus on your cell phone are all everyday examples of poor usability. Once you start looking you will be amazed at how much you are inconvenienced by poor design and how much time and energy it wastes. As you identify usability problems think of solutions—how could the design be improved.
  • Also look for examples of usability issues and improvements on the software tools you use every day.
  • Watch non-technical people using computers or cell phones. Your relatives may offer a fertile population to observe. Don't just correct them or show them how to do something but try to understand their misconceptions. Again, think about how you could support them through better design.
  • Observe interactive products like cell phones and websites, from a user perspective. Note the design problems and think about how to solve them.
Sadly, you will not have much problem finding design problems. They abound. But making the decision to understand usability and UX design in a deep way will make you a better developer and help you bring the products you create to the next level.

THE SOFTWARE VIEW
Ambrose Little
The truth is that we're nowhere near a place where we can have a UX designer on every software project. I mean, even if every team working on software were convinced of the importance of a focus on UX, there just aren't enough folks with that expertise and probably won't ever be. So we're left with the reality we have today—developers, developers, developers! And as a once-upon-a-time developer, I don't think that's a bad thing.
Developers are smart. We like to build good stuff, and by and large we're always learning and trying to improve ourselves and our trade. As I see it, a focus on UX, a focus on useful, usable, and desirable is part and parcel of our continued self-improvement. Charlie focused in on usability, so I won't belabor that too much myself. I'll just quote my boss, Dr. Komischke, who likes to quote the International Standards Organization (ISO) definition:
Usability is the "effectiveness, efficiency and satisfaction with which a specified set of users can achieve a specified set of tasks in a particular environment."
Actually, that's a pretty broad definition that seems to encompass what I think most people call "user experience" these days. I think there's a maturing in the UX community as well and plenty of discussions about what UX professionals do and what it should be called. So as the field matures, definitions get refined, and so today usability is more refined. I like the way Dr. Larry Constantine talks about it—designing for user performance.
The thing about usability is that it is focused on minimizing and eliminating barriers and errors that keep people from achieving what they need to achieve with software. The boon for us is that there is a lot of science in the discipline of usability, which is probably the most studied area of UX, thanks to the military, aviation, health care, and so forth where errors can mean life or death. There is a wealth of knowledge derived from these studies that can inform how we build software and the assessment techniques we use (such as usability testing, which we'll cover in July) to help us better ensure the usability of products we build.

Userful
Useful is an easy one because we already get it. It's about the functionality, but even so, there's more to making software useful than making it functional. Software could be completely functional, bug free, and designed to spec and yet totally not be useful to the people that it was made for. It's a change in thinking from requirements gathering and specification (and building to specifications) to understanding what the people who will be using the software want to achieve—what their goals are and how they go about achieving them.
I think this area is where there is most overlap between user-centered/UX approach to software and Agile. Agile recommends regular, iterative input from stakeholders. A UX approach may be more direct—you (ideally) have regular, iterative feedback from users. It helps keep you on the straight and narrow, but it goes farther than that. It urges you to do some more work up front to get a better understanding of the people and what they want to achieve. You can't just ask people what they need the software to do. You ask them what they do. You watch what they do (if you can). You try to understand their motivations--what makes them tick, what they like and don't like about what they do, and more. We call it user research, and there are techniques like user interviews, surveys, and ethnography, which is the science of observing people in their natural habitats, and taking the data collecting and condensing and refining into usable tools such as personas.
I think that this aspect and the body of knowledge and techniques that the UX profession brings to the table has more to add to improving software than it is given credit for. When most people think of UX, they tend to think of either usability or desirability. But if you get the usefulness wrong, the rest doesn't matter. It can be the most functional, the most usable, and the most aesthetically pleasing software ever built, but if it doesn't help people do what they want to do, it'll be nothing more than a splash in the pond. Putting the focus on what people want and need to accomplish and endeavoring to see how software fits in that puzzle will yield unexpected insights; it'll help ensure the software is useful, and it could even keep you from building software that isn't really needed.

Desirable
Ah, eye candy. As humans, we innately understand aesthetic appeal, and it certainly does sell well. It's kind of a no-brainer that you want to invest in this aspect of UX if you need to convince people to use your software. But what if you don't need to sell something? What if you have a captive audience or your software is mandated internally?
Well, in recent years the academics in the field of human-computer interaction have become more and more interested in aesthetics as a way to enhance both the usability and usefulness of software. I was at the 2008 annual meeting of the Human Factors and Ergonomics Society last September, and there were several papers presented around this topic. It's an early area of inquiry, but it makes sense.
There's another aspect of aesthetics that shouldn't be overlooked, even for internal software. Think about your own experiences. If something is aesthetically pleasing, it can make up for other deficiencies in that thing or even the overall experience of a company.
Further, if you actually enjoy using a piece of software, you're less reluctant to use it—you won't try to work around it like so many people do with poorly designed software that they are required to use. It contributes to job satisfaction and, ergo, employee retention. So even for internal software, there are good reasons to focus on aesthetics.
And it's worth noting that there's more to desirability than just aesthetics. The desirability of software is intertwined in usability and usefulness, and aesthetics applied to software at its best will only enhance those aspects of software to increase the overall desirability.
Stay tuned, because we're looking to dive into more on aesthetics late this year.

What About Developers?
So what does all this mean for developers? Essentially, I think it means that developers can and should embrace and learn about UX, about usability, usefulness, and desirability, just as much as they embrace Agile, Domain-driven design, test-driven development, and other approaches and techniques that enhance their overall effectiveness. Incorporating best practices from UX development will only help to make you more effective in building software, regardless of your environment. Usability matters to software developers. That's why we're here each month.

Send your questions and comments for Charlie and Ambrose to magux@microsoft.com.

Dr. Charles Kreitzberg is CEO of Cognetics Corporation (www.cognetics.com), which offers usability consulting and user experience design services. His passion is creating intuitive interfaces that engage and delight users while supporting the product's business goals. Charles lives in Central New Jersey where he moonlights as a performing musician.

Ambrose Little lives with his wife and four children in central New Jersey. He's been designing and developing software for more than 10 years and is honored to be an INETA speaker and Microsoft MVP. Lately, he's shifted from technical design to designing for people and is now a user experience designer for Infragistics.


{End Bracket}
The Unconference: Where Geeks JIT Together
Steven M List

More collaboration and less imposed structure. That's where technology is leading us. Whether it's Wikipedia's collaborative bottom-up organization or the Unconference's on-the-fly topic and presentation planning, the trend is clearly about less prescription and more participation.
Just a few years ago, there were no Unconferences, Open Spaces, BarCamp, FOO Camp, or DemoCamp. So why does it seem like today there's another Unconference or Open Space every other week? What's so compelling about geeky, nerdy, tech folks getting together just to talk about whatever's on their minds?
Well, I'll borrow the answer from the Agile community. What makes these events so compelling is that they're self-organizing.
Traditional events—such as conventions and conferences—are organized, planned, scheduled, and have content chosen by one or a few organizers. Then the content is prepared and presented by the speakers they select. What do you, the attendee, have to say about it? Nothing. How engaged does that make you feel? It depends.
Believe it or not—and this may be hard for some readers to swallow—Unconferences and Open Spaces are really about spirit. The spirit of self-organization. The spirit of the participants. The spirit of the community.
The Unconference Web Site
Self-organization: The idea is that the participants (note, not attendees), decide what to talk about and when to talk about it, and select topics based on their interests. These events truly involve the people who come, and sessions are generally dialogues and conversations, not presentations and passive reception.
I'm amazed to see various Unconference approaches used in such a wide variety of settings today. At ThoughtWorks, we've used Open Space Technology to kick off projects. The ALT.Net global community has embraced Open Space Technology for many of their events around the world. Of course, the Agile community uses Open Space actively, both at the big events (Agile2009) and smaller regional ones such as the Agile Open series of events. BarCamp, DemoCamp, and ProductCamp all exist as instances of Unconference technologies. Universities use these technologies, and user communities and professional communities use them as well.
There's something quite appealing about feeling like it's your own event. I understand that some of you may be skeptical of an event that has no content or agenda until the participants arrive, but trust me, you get over it quickly.
Now don't get me wrong. I'll admit that I believe that the facilitator makes a difference. Classic facilitation (which I do), involves helping to clarify issues and bringing out information, making it easier for people to communicate with each other. Open Space facilitation involves introducing the participants to the event, guiding them through creating the agenda, holding space and time (more on this another time), and guiding the participants through a closing. That's it.
It would seem that because running an Open Space event requires so much less "planning," it would be easy. And yet, when I facilitate an Open Space, I find myself exhausted. Perhaps it's because I bring my spirit and my passion, and I put them all out there. Just like the participants.

Steven M. List (known as "Doc") is a career technologist, leader, coach, and facilitator and currently works for ThoughtWorks as an organizational transformation consultant. He is a recognized professional Open Space facilitator, including having been embraced by Microsoft and the ALT.Net community. He explores the topics of facilitation, communication, team building, and personal growth in his blog at www.stevenlist.com/blog.