 MSDN Magazine August 2006

MSDN Magazine August 2006

Business Apps: What You Need To Know About Using Office As A Development Platform

Andrew Whitechapel and John Peltonen Gathering MOSS: New Dev-Centric Features In Office SharePoint Server Keep Your Apps Rolling

Microsoft Office SharePoint Server (MOSS) 2007 provides great portal and search features and much more, and Ted Pattison puts them to good use here.

Ted Pattison Add-In Power: Let Users Customize Your Apps With Visual Studio Tools For Applications

If you're looking to increase the usefulness of your applications by making them customizable, you'll want to read about these three technologies available from Microsoft.

Paul Stubbs InfoPath 2007: Designing Form Templates With The New Features Of InfoPath

InfoPath 2007 is an XML forms designer and editor, and a fully featured and integrated member of the Office family. This article takes a sneak peek at some of its new features.

Scott Roberts and Hagen Green Use The Source, Luke: Source Server Helps You Kill Bugs Dead In Visual Studio 2005

The latest releases of WinDBG and Visual Studio know exactly how to use source server, so its benefits are available to both .NET and native C++ developers. See why this is so important in tracking down bugs.

John Robbins Editor's Note: The Office-Driven Life

Josh Trupin introduces this month's issue.

Joshua Trupin Toolbox: Synchronize Data, Review Runtime Errors, and More

This month we bring you products that let you synchronize data, correct runtime errors, and more.

Scott Mitchell Basic Instincts: Resources and Localization in ASP.NET 2.0

In this installment of Advanced Basics Ted Pattison discusses the localization of Web sites in ASP.NET 2.0.

Ted Pattison CLR Inside Out: CLR Hosting APIs

By using the CLR 2.0 hosting APIs, developers of native hosts can execute managed code in-process with complete knowledge and control over how the CLR behavior can affect their application.

Alessandro Catorcini and Piotr Puszkiewicz Cutting Edge: All About Enums

Dino Esposito reviews the basics of enumeration types and their implementation in the Microsoft .NET Framework.

Dino Esposito Design Patterns: Model View Presenter

The MVP pattern helps you separate your logic and keep your UI layer free of clutter. This month learn how.

Jean-Paul Boodhoo Service Station: Serialization in Windows Communication Foundation

Windows Communication Foundation supports several serialization mechanisms and provides a simple, interoperable foundation for future service-oriented applications. Here Aaron Skonnard explains it all.

Aaron Skonnard Security Briefs: Security in Windows Communication Foundation

Windows Communication Foundation provides three major protections— confidentiality, integrity, and authentication. This month Keith Brown explains what they can do for you.

Keith Brown Netting C++: Resource Cleanup

Last time Stan Lippman did not consider the visibility of types and nondeterministic finalization in writing his TQL wrapper code. This month he rewrites the code.

Stanley B. Lippman C++ at Work: Addin a Combobox Cancel Feature

This month Paul DiLascia codes some Microsoft Office-style dialog box features.

Paul DiLascia { End Bracket }: Deliver Beautiful Apps

Kevin Moore ruminates on the future of UI design in Windows Vista.

Kevin Moore


Business Apps: What You Need To Know About Using Office As A Development Platform

Business Apps

What You Need To Know About Using Office As A Development Platform

Andrew Whitechapel and John Peltonen
This article discusses:
  • Fundamental elements of the Office platform
  • A brief look at the object models of Office
  • Office extensibility points
  • The Office development toolset
This article uses the following technologies:
Microsoft Office system, Visual Studio Tools for Office, .NET Framework
Contents
Office Platform Fundamentals
The Office Object Model
Extensibility Points
The Office Development Toolset
More Benefits of Office
Serious Development Platform

T
here's little doubt that the 2007 Microsoft Office system provides the basis for some pretty sophisticated custom solutions, and there are thousands of such solutions running happily on Office today. But can Office also serve as a robust development platform? Does Office provide the building blocks for creating entirely new applications?

Office has traditionally been seen as a suite of discrete client-side applications—Word, Excel®, and Outlook®—each designed to provide a rich set of functionality. Since this functionality was directed toward a specific range of user tasks for each application, there was minimal explicit interoperability between the different applications. But Office has evolved to become a framework of applications and technologies that provides services across both clients and servers and embraces Microsoft InfoPath®, SharePoint®Services, Content Management Server, and Exchange Server.

Word, Excel, Outlook, and the rest of the Office apps were intentionally designed to explicitly expose most of the functionality for external consumption as COM servers. Clearly, the Office architects had custom solutions in mind. As a result, you can easily build a solution that seamlessly hooks one or even multiple Office applications. Moreover, an Office component like SharePoint brings portal services, team services, workflow functionality, and content management. You might argue that SharePoint is a pretty comprehensive development platform all on its own.

Figure 1 Platform Characteristics
Figure 1 Platform Characteristics

A true development platform must satisfy some basic requirements. We all have our own ideas on this, but a minimal set would probably include generic baseline business services, extensibility points for solutions to hook into the generic services, and a consistent toolset for building against the baseline services. You also need a base level of non-functional attributes that allow the functional services to be useful in building realistic enterprise solutions (see Figure 1). Not only does Office meet these requirements, it has two other things going for it: developers who've worked with Windows® already have the knowledge and experience they need to take advantage of the Office platform, and Microsoft has released a number of tools that will help developers build new applications based on the platform.

Office Platform Fundamentals

We can map the requirements of a development platform to the Office system, and start to fill in some of the specific features, as shown in Figure 2. As a development platform, Office must address a range of fundamental requirements. These include reliability, scalability, security, deployment and updating, reusability through componentization, a consistent object model, and version compatibility. Let's take a look at these requirements and see how the 2007 Office release addresses them.

Figure 2 Office Development Platform Landscape
Figure 2 Office Development Platform Landscape

Office provides reliability in three main ways. First, the Office applications themselves are built with a high degree of robustness. Second, Office takes a number of steps to protect itself from rogue customizations. For example, if an add-in crashes its host application, Office detects this and blacklists that add-in so that it's not loaded in the future. Most exceptions that might be thrown by an Office customization or add-in are handled silently to avoid destabilizing the host. In some cases, the application will throw up a dialog box with a suitable error message so the user knows that Office is functioning correctly but the add-in might be in an undefined state. Third, Microsoft® Office SharePoint Server (MOSS) 2007 is built to the same exacting standards as other Microsoft server software, so services based on SharePoint get this high level of reliability, too.

There are two dimensions to scalability and performance: client-side and server-side. On the client, each Office application imposes certain limits, such as the maximum number of rows and columns in an Excel worksheet. This makes scalability predictable—you simply can't build a custom solution that exceeds these limits. However, it is possible to deploy solutions that exceed practical limits in a context where there is no defined hard limit. For example, there are no specific limits to the number of add-ins you can register for a single application, but clearly there would be practical limits—think of the performance penalty of loading 10,000 add-ins when Excel starts up. This is a good example of the type of decision a development platform should not make on behalf of consumers.

You probably shouldn't automate Office client applications on the server for the same reason you wouldn't load 10,000 add-ins: it's just not practical, even though it's technically possible. However, the 2007 Office release does expose the Excel calculation engine and other features on the server through SharePoint and Excel Services (see Figure 3). Windows SharePoint Services (WSS) 3.0 provides integrated workflow support, native support for ASP.NET 2.0, support for blogs, wikis, and RSS. MOSS 2007 builds on that foundation, introducing content management, business intelligence (BI), Web-based Excel Services spreadsheet capabilities, InfoPath forms support, document and list-centered workflows, and search enhancements. In addition, the MOSS Business Data Catalog (BDC) services allow retrieval of information from back-end systems such as customer relationship management (CRM) and enterprise resource planning (ERP) applications. (See Ted Pattison's article in this issue for more details about MOSS 2007.)

Figure 3 Excel Services Components
Figure 3 Excel Services Components

In the past, Office security has been somewhat difficult for users to understand and for administrators to manage. The 2007 Office release has completely revamped its security model to be more understandable to users, more transparent, and easier to administer, without losing any of its strength. Visual Studio Tools for Office has always employed a stricter security model, but that strength made it difficult to administer.

Traditional Office-based solutions either automated Office externally, registered add-ins of some kind (COM add-ins, smart tags, or real-time data), or provided document-based solutions where the automation was handled in-process by using Visual Basic for Applications (VBA) embedded in the document. Deployment and updates for external automation solutions are pretty straightforward; any solution that requires registration in the Windows registry typically has a very well-defined set of deployment processes around it, and updating simply involves installing a new version.

Deployment is really a problem only with document-based solutions. Most documents are useless unless users can modify them, but changing documents opens the possibility of changing the VBA. Even if you prevent this, the VBA code can still (and typically will) behave differently depending on the data in the document. You can rapidly end up with multiple versions of the solution, and tracking them becomes a nightmare, VBA doesn't lend itself to source control, and auditing and regulatory compliance become all but impossible.

Visual Studio Tools for Office addresses these issues by providing a very flexible deployment model. Visual Studio Tools for Office solutions—both application-level add-ins and document-level customizations—allow the code to be deployed locally or remotely, and both incorporate automatic updates. If you deploy remotely, you have a simplified administrative task since the assemblies are on a server rather than propagated across every user's desktop. When Office loads a solution, the Visual Studio Tools for Office runtime performs a check for an updated version and, if it finds any, automatically downloads the updates to the local machine.

The Office Object Model

As noted earlier, Office applications expose COM interfaces. For example, while you can focus on the Word Document object and logically instantiate it independently, in practice the Word process will be executed to serve up this object. But if you really want a development platform then, though you might only use one component in a specific scenario you still want the rest of the platform to be available. You want that component to use whatever other platform services it requires without having to instantiate those services yourself manually.

Where is the right place for componentization in an Office solutions context? Many customers build their own domain-specific reusable components layered on top of Office. For example, a component that is specific to some investment banking functionality might use the Excel calculation engine. The solution-side interface of this component is domain specific, while the platform-side interface is Excel specific. This abstraction allows the component to work against multiple versions of Excel, which is a typical customer requirement.

Alternatively, you could build a generic reusable component that is host-application neutral. Consider, for example, a UserControl that uses Web services to display data in a custom task pane. That same control could be reused in multiple Office applications.

Office applications have evolved over time and they have very different sets of functionality, so they will inevitably expose different object models. That said, there is some consistency across the apps. The object model is mostly hierarchical, for example, and there is generally an Application object at the root. In Word you can start from the Application object, find the Documents collection, drill into individual Document objects, and then into individual Range objects. Similarly, in Excel you can start from the Application object, walk down to the Workbooks collection, find the Workbook objects within the collection, and then get individual Range objects within a workbook.

In addition, Visual Studio Tools for Office layers on a consistent, strongly typed object model. You only have to look at two very different Visual Studio Tools for Office solutions—say an Excel document-based solution and an Outlook application-based solution—to see the consistency. Both solutions present simple Startup and Shutdown methods so you can focus on the business requirements without worrying about the individual traits of the host application. While you still have complete access to the underlying object models, you can also choose to work at a higher level of abstraction.

With each new release, Office maintains a very high bar for backward compatibility. Office applications are COM servers, and COM imposes a number of rules such as the immutability of interfaces, which protects against incompatibility across versions. Office mostly plays by the rules. All Office interfaces are either dual interfaces or pure dispatch interfaces. Pure vtable interfaces (and by extension, dual interfaces) are immutable, but dispatch interfaces are not because the set of available functions and their signatures is discoverable at run time (therefore late binding). The Office team routinely takes advantage of this by adding additional methods and properties to the end of an interface when a new version is released. They also typically add new optional parameters to existing methods. Using these techniques, Office can preserve backward compatibility.

Another technique for maintaining version resilience is to use loose typing. The classic example is the IDTExtensibility2 interface that COM add-ins must implement. When an Office application loads an add-in, the application calls IDTExtensibility2::OnConnection and passes in a loosely typed object that represents the host application. In .NET code this parameter is typed as a System.Object while in C++ it is a generic IDispatch pointer. At run time, this will effectively be a pointer to the Application object exposed by the host application. When you build a traditional shared add-in, the code generated by the wizard is completely host-neutral. This confers two benefits: it allows the add-in to run in any application that supports COM add-ins, and it allows the add-in to run in any version of any of these applications.

The price you pay for this host-neutral model is the standard price for loose typing and late binding: you get no design-time or compile-time support for the specific types involved because they are unknown until run time. That makes it a lot easier to write code that will fail at run time. Also, late binding carries a performance penalty because each method invocation has to go through the discovery process to see if a matching method can actually be found at run time.

When you develop add-ins with the .NET Framework, you can continue to use this loose typing, but the advantages are marginal unless the add-in itself provides only host-neutral functionality. As soon as the add-in needs to provide host-specific functionality, you're likely to use the host's Primary Interop Assemblies (PIAs), which are version specific. Also, you can use Visual Studio Tools for Office to build add-ins with .NET code. Visual Studio Tools for Office mandates strong typing, giving you the benefits of design-time IntelliSense® and autocomplete in addition to compile-time type checking. Strong typing avoids the performance overhead of late binding.

Does strong typing mean you lose version resilience? Not necessarily. You can build a loosely typed add-in against some version of Office and the add-in will almost certainly work completely in later versions of Office. With the .NET Framework you can achieve the same results although the behavior is slightly more complicated. A .NET add-in built against a particular version of the Office PIAs should also work in later versions. There are actually two ways this can happen: either the add-in uses the version of the PIAs it was built against or you deploy a binding redirect for the PIAs so that at run time the add-in uses a later version of the PIAs that corresponds to the installed version of Office.

Extensibility Points

From the early days of Office it has been possible to extend its functionality virtue of the fact that Office client applications are COM servers and therefore expose their functionality for external automation. This automation can even be embedded in-process with the host application, traditionally by using VBA. Office offers a wide range of extensibility points, and we can drill down on our landscape diagram to provide the list in Figure 4. You could slice this list in a number of ways. Figure 5 describes the techniques that typically involve automation of the Office object models, while Figure 6 shows techniques that rely on other protocols.

Figure 6 Extensibility Points Not Reliant on Office Object Models (Close Figure 6)

Extensibility Point | Description | Office Applications
User-defined functions for Excel Services | Custom worksheet functions built using .NET DLLs. | Excel 2007.
Accessibility | Applications implement IAccessible to extend greater support for users with vision, hearing, or motion disabilities. | Office 2000 and later.
XML | Applications read and/or write XML documents. | Increasing support in Office 2000 and later.
OLE DB | Access Office documents as an OLE DB data source. | Office 97 and later, to varying degrees.
Research services | Search local and remote data sources from within Office. | Excel, Word, PowerPoint in Office 2003 and later.
SharePoint Workflow | Define workflow actions and publish to SharePoint Designer. Users can employ these actions in custom workflows. | Word, Excel, and PowerPoint documents, plus SharePoint.
Server-side BDC APIs | Business-specific Web services that provide access to BDC data. | Word, Excel, and PowerPoint documents, plus SharePoint.
BDC Actions | Parameterized URLs that follow BDC data across the portal. These can send parameters into Form Server or custom .aspx pages. | SharePoint and InfoPath Form Server.
ASP.NET 2.0 Web Parts | Web parts can now be used across the enterprise, including in SharePoint sites. | SharePoint.
Figure 5 Extensibility Techniques Using the Office Object Models (Close Figure 5)

Extensibility Point | Description | Office Applications
External automation | Any out-of-process use of the COM object models exposed by Office applications, including Windows Forms and console applications. | All client applications.
In-process automation (VBA) | In-process use of the COM object models exposed by Office applications through VBA. | All client applications.
In-process automation (Visual Studio Tools for Office) | In-process use of the COM object models exposed by Office applications, using .NET code. | All client applications.
COM add-ins | In-process DLLs that implement IDTExtensibility2. | Most client applications starting with Office 2000, plus InfoPath 2007, Publisher, Project, SharePoint Designer (formerly FrontPage), and Visio.
Smart tags | Elements of text in an Office document that are recognized as having associated custom actions. | Excel and Word in Office XP; Access, Excel, PowerPoint, and Word in Office 2003 and later.
Smart documents | DLLs that implement custom code associated with an XML schema attached to a document. | Excel and Word 2003 and later.
Excel real-time data (RTD) | In-process or out-of-process automation server components that bring data into Excel in real time. | Excel 2000 and later.
User-defined functions for Excel client | Custom worksheet functions built using automation add-ins. | Excel 2000 and later.
Web services | Connect Web services to Office using either the SOAP toolkit (VBA) or .NET Web service proxies. | Office 97 and later.
Custom task panes | Application-level task panes that support any ActiveX control or Windows Forms UserControl. | Access, Excel, InfoPath, Outlook, PowerPoint, and Word in the 2007 Office release.
Custom Ribbons | Document-level and application-level custom Ribbons. | Access, Excel, Outlook, PowerPoint, and Word in the 2007 Office release.
Custom form regions | Custom regions within Outlook forms. | Outlook 2007.
Custom digital signatures | Custom digital signature and encryption providers. | Office 2007.
Custom blogs | Custom blog extensions for Word. | Word 2007.
Figure 4 Office System Extensibility Points
Figure 4 Office System Extensibility Points

This is a wide range of extensibility points. Of course, if you evolve a suite of applications over time, inevitably you'll end up with a varying range of behaviors—and this is true of extensibility points also. If you design a development platform from the ground up, you would obviously design the extensibility points to be as consistent as possible. We all know the history of Office, but are the extensibility points consistent across the breadth of the application suite? That is, can you hook into the same functional piece in the same way across different applications?

At one level, the techniques that involve some kind of automation of the Office object models are consistent by definition. That is, the way you use the object models is scoped by standard COM and automation rules. The individual Office application object models have a degree of consistency but ultimately different sets of functionality. Dropping down a level, the way you code a COM add-in is different from the way you code a smart tag, for example. This is because an add-in needs to implement IDTExtensibility2, while a smart tag needs to implement ISmartTagRecognizer or ISmartTagAction. Obviously, the only way these two sets of functionality could be made more consistent is if the interfaces were combined. While the idea of a single extensibility interface for all types of extensible functionality might seem attractive, it would be either very limited or very loose in scope.

Rather than combining interfaces, the 2007 Office release introduces five new extensibility interfaces covering four functional areas (see Figure 7). What's interesting about these new interfaces is that while they are very different in functionality, they can all be implemented through a standard COM add-in. This was a conscious design decision and is also an indicator for the future roadmap for Office extensibility. The COM add-in technology is tried and tested, well understood, and well supported by developer tools. The processes for deploying and maintaining add-ins are also well defined and familiar to administrators. It makes a lot of sense to provide a consistent extensibility point in this way—not by designing some single extensibility interface but instead by allowing a single implementation to employ one or more of the growing number of extensibility interfaces. Using this approach, developers and administrators would have only one model to worry about.

Figure 7 Office Extensibility Interfaces (Close Figure 7)

Interfaces | Description
IRibbonExtensibility | Used to build customized Ribbons.
FormRegionStartup | Used to build custom form regions for Outlook.
ISignatureProvider and IEncryptionProvider | Used to build custom digital signature and encryption providers.

This convergence of extensibility techniques into the add-in model looks to be the design for all future extensibility interfaces. Could it also be applied to existing interfaces? For example, could you implement a smart tag via an add-in? Could you implement an Excel function via an add-in? Technically, this might actually be feasible, but it is probably not the best thing to do. For new interfaces it makes sense, but the drawback of trying to apply this to old interfaces is that there are already many solutions that use the old approach. If you could implement Excel functions via regular add-ins as well as via automation add-ins, you'd then have two different ways to achieve the same results. So, the ideal of convergent design brings with it divergent deployment. This may be acceptable for a transition phase during which developers migrate their solutions from the old model to the new one. Right now, add-ins do not support any of the other old extensibility interfaces, but this is a possible direction for the future—perhaps depending on customer demand.

While we're looking at extensibility points, we should also consider the document file formats. One way Office makes itself extensible is by providing programmatic access to its documents. Over the last few releases, Office has introduced some level of support for XML. For example, Excel 2000 had minimal support for reading and writing XML. This support has increased incrementally and also spread to other applications, specifically Access, InfoPath, PowerPoint®, and Word. The 2007Office release provides support for completely open XML file formats for Excel, PowerPoint, and Word.

The Open XML formats create new opportunities for developers because you can extend solutions through deep integration with document contents. This follows the Open XML formats submission to ECMA International as a proposed open standard. Microsoft is working to support the opportunities for independent development in several promising areas. There's even a new open technical community of developers, known as the Open XML Formats Developer Group that includes 40 industry leaders such as Intel, Apple, BP, and Toshiba.

The use of XML offers the benefits of greater transparency and openness than possible with the previous binary file formats. The new formats allow Office documents to integrate easily with existing and future line-of-business systems, as the contents are now open and accessible. This clearly addresses server and scalability issues: for many solutions, you don't need to automate Office on the server because you can manipulate the files directly. The new formats are also designed with long-term robustness and accessibility in mind, so that file corruption will be easily repairable, and there is no reliance on any particular software application to provide access to the document contents. The 2007 release files are also much more efficient, taking up far less space than the previous formats and allowing for quicker transmission times and a smaller impact on storage.

The new file formats represent a major step forward and will be made available not only to customers who adopt the 2007 release but also to customers using previous versions of Office. Free conversion tools enable users of Office 2000, Office XP, and Office 2003 to open and save to the new formats, so that everyone can benefit from this innovation.

The Office Development Toolset

One perspective argues that a development platform merely needs to expose its extensibility points and doesn't also have to provide a toolset. Up to a point, the question is academic since Microsoft does, in fact, provide tools for development against Office. The relevant question thus becomes, does the provided toolset adequately support the platform?

Focusing on tools, you can build Office solutions with a variety of frameworks and languages. You can use any COM-aware language for externally automating Office. You can use VBA for internal (document-based) automation of Office. When working with the .NET Framework, developers can use Visual Studio Tools for Office if they want a structured, consistent toolset and runtime. A mapping between the extensibility points and the available tools is shown in Figure 8.

Figure 8 Mapping Extensibility Points to Toolsets (Close Figure 8)

Extensibility Point | VBA | Shared Add-Ins | VSTO | One-Off Extensions
Generic Techniques |  |  |  |
Web services |  |  |  |
XML |  |  |  |
IAccessible |  |  |  |
OLE DB |  |  |  |
Specific Use of Office
Object Models |  |  |  |
Automation |  |  |  |
Server-side BDC APIs |  |  |  |
SharePoint Workflow |  |  |  |
Extensibility Interfaces
IDTExtensibility2 |  |  |  |
ISmartTagRecognizer |  |  |  |
ISmartTagAction |  |  |  |
ISmartDocument |  |  |  |
IRtdServer |  |  |  |
ICustomTaskPaneConsumer |  |  |  |
IRibbonExtensibility |  |  |  |
FormRegionStartup |  |  |  |
ISignatureProvider |  |  |  |
IEncryptionProvider |  |  |  |
IDispatch (Excel UDFs) |  |  |  |

VBA covers a small subset of the available extensibility points, mainly because VBA is restricted to document-based solutions. Both traditional shared add-ins and Visual Studio Tools for Office solutions cover most of the extensibility points. Visual Studio Tools for Office covers slightly more because it also folds in some of the one-off extension types—for example, you cannot build a smart tag as a traditional add-in, but you can build it either as a one-off DLL or as a Visual Studio Tools for Office add-in.

The Visual Studio Tools for Office runtime provides essential services optimized for a wide range of solutions. Alternatively, you have the flexibility to build solutions without Visual Studio Tools for Office if you need to build a custom runtime infrastructure. Consider, for example, COM add-ins. Right now you have two main toolset choices for building add-ins: the traditional shared add-in project types and Visual Studio Tools for Office add-ins.

This situation is less than ideal, though. A better solution would be a single project type with all the capabilities of both shared add-ins and Visual Studio Tools for Office add-ins. This is clearly difficult to achieve, given that shared add-ins are fundamentally loosely typed while Visual Studio Tools for Office add-ins are fundamentally strongly typed. This difference gives the shared add-ins the ability to work across multiple versions of Office, and the Visual Studio Tools for Office add-ins, in contrast, a better design-time experience and significantly increased runtime robustness.

Failing the ideal, the optimal toolset would rationalize these two project types to make them as consistent as possible and to make the developer's choice simple. For example, Visual Studio Tools for Office add-ins are organized like all other project types in Visual Studio, with root nodes that correspond to the programming language. In contrast, the shared add-ins offer the developer the language choice as a step in the project wizard. Another example is that Visual Studio Tools for Office add-ins don't offer the developer a choice about the text string to use for the Description registry value because you can set or change this at any time after the wizard generates the project. This value is actually never used by Office. The shared add-ins, on the other hand, prompt you to supply this string and in almost all cases it is redundant.

Where does VBA fit in the Office developer toolset? VBA was originally introduced to support non-professional development—that is, power users (professionals in their own area, but not necessarily in software development) who needed to expand the base set of Office functionality to provide more domain-specific solutions. It was never intended to be used for highly sophisticated professional solutions, but has been used in that role for lack of a good alternative. Visual Studio Tools for Office is positioned at the other end of the spectrum. It is a professional development tool that brings the full power of Visual Studio and managed development to Office.

Of course, VBA and Visual Studio Tools for Office can coexist, and there's considerable overlap in the types of solutions built with these technologies. For many solutions you could use either or even both (though this approach is likely to be limited to transitional projects that are being migrated over time from VBA).

Visual Studio Tools for Office certainly covers the high end of the scale and extends down towards the low end, but it doesn't offer macro recording. This is one area where VBA is still the only tool for non-professional solutions based on user interaction. Again, the ideal would be one toolset to cover the entire spectrum. With this in mind, Microsoft is looking hard at the problem and considering ways in which Visual Studio Tools for Office might be used to cover the low-end macro recording capabilities of VBA.

More Benefits of Office

Thus far we've made a strong case for Office as a development platform, but in fact it offers much more. Consider that the primary information-worker user interface is Office. This isn't a requirement for a development platform, but it is clearly a huge bonus. Information workers demand applications that are built around Office. The Office UI is familiar, the Office application workflow is familiar, and many tasks require specific Office functionality. More than that, these workers need applications that let them interact seamlessly between the Office host and the custom solution functionality.

Can you build solutions with Office where your custom functionality is seamless with the native application's functionality? Do the custom solutions look as if they're part of the host? Consider the command UI—that is, the Ribbon in the 2007 Office release and the equivalent command bars in earlier versions. If you provide customization of the Ribbon or command bars, these custom controls and menu items behave in the same way as built-in controls. If you build an Excel user-defined function, it can be used in exactly the same way as a built-in function, although there is a limitation in the level of context-sensitive help support you can provide. Outlook custom form regions are designed to be seamlessly integrated with built-in forms and, in fact, you have several choices as to how this integration is represented.

The Office programmability team has taken great pains to ensure that extensibility features implemented by custom solutions are integrated within the host application as seamlessly as possible. The net result is that the end user sees the custom solution as a natural extension of Office.

One point worth mentioning here is that Office exposes extension points in an open-ended way, leading to potential abuse. For example, the Outlook folder view allows you to place HTML within the Outlook window, and the HTML could include ActiveX® controls. You can see how this could be taken to extremes, as Outlook has no knowledge of the controls you place here and can't impose any restraints on their behavior. Overall, the extensibility features of Office give you all the flexibility you need, but you should use the platform sensibility.

Serious Development Platform

We've seen that Office has all the attributes of a serious enterprise development platform. Office offers an extensive baseline set of functionality; a range of extensibility points and a choice of developer toolsets. It is not perfect and there are curious anomalies and inconsistencies, but considering the history of Office it is truly remarkable how far the suite of applications has converged into a cohesive platform. The baseline functionality is extremely comprehensive. The extensibility points are a little uneven and there could be more of them. The developer toolsets are still evolving and could do with some rationalization. So, there's still some way to go, and the supporting toolset needs to evolve in lock-step with the platform itself, but the indications are that the strategic direction for the future of Office is to evolve it into a world-class development platform.

Andrew Whitechapel spent many years as an architect consultant building enterprise solutions for a wide range of customers, and is now Program Manager Technical Lead for the VSTO team at Microsoft.

John Peltonen and his company, 3Sharp, are passionate Office system solutions architects and developers. Read more about them at blogs.3sharp.com

© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


Gathering MOSS: New Dev-Centric Features In Office SharePoint Server Keep Your Apps Rolling

Gathering MOSS

New Dev-Centric Features In Office SharePoint Server Keep Your Apps Rolling

Ted Pattison
This article is based on a prerelease version of the 2007 Microsoft Office system. All information herein is subject to change.
This article discusses:
  • Building portal sites with MOSS 2007 and WSS 3.0
  • Shared Service Providers, user profiles, and portal search
  • Integration with the BDC and back-end systems
  • Managing Web sites, business intelligence, and enterprise data
This article uses the following technologies:
Office SharePoint Server (MOSS) 2007, ASP.NET 2.0, WSS 3.0
Contents
Building MOSS 2007 Portal Sites
Shared Service Providers
User Profiles
MOSS 2007 Search
Business Data Catalog Integration
Web Content Management
Business Intelligence Features
Managing Documents and Business Processes
InfoPath Forms Services
ECM and Single Sign-On
Wrapping It Up

M
icrosoft Office SharePoint Server (MOSS) 2007 provides portal and search features that represent the next generation of SharePoint® Portal Server (SPS) 2003. However, Office SharePoint Server 2007 is much more than that. The Office team made significant investments in many other areas to give MOSS 2007 extra dimensions in Web content management, document and records management, business process integration and forms, and business intelligence. In this article I'll describe the basic architecture of MOSS 2007 and explore the various opportunities for developers to build portal sites and business solutions.

One of the most appealing aspects of MOSS for developers is that it is built on top of Windows® SharePoint Services (WSS) 3.0 and ASP.NET 2.0 (see Figure 1). MOSS 2007 portal sites are created using WSS features and site definitions along with familiar ASP.NET building blocks such as Master Pages and Web Parts. This means that MOSS solutions can easily be extended with ASP.NET components such as server-side controls and custom Web Parts as well as custom components that utilize WSS features like custom list definitions, document libraries, event handlers, and workflows.

Figure 1 MOSS 2007 within the SharePoint Ecosystem
Figure 1 MOSS 2007 within the SharePoint Ecosystem

It's important to understand the extent of the synergy that exists between MOSS 2007 and WSS 3.0. Many of the new features in MOSS are made possible by new WSS innovations such as Master Page integration, content types, versioning, and workflows.

As a developer, it's also important to set your expectations accordingly about what you'll need to do to get a MOSS solution up and running. Unlike many other development projects, you don't usually start by firing up Microsoft® Visual Studio®. In some scenarios you can get an entire business solution up and running without ever writing any custom code. This is quite different from developing directly on the ASP.NET platform by itself where you need Visual Studio to put together the simplest site.

The Microsoft Office team promotes a philosophy in which you begin building a MOSS business solution with an assemble-and-configure mentality. Once you determine the MOSS features and services you want to utilize, you can create a new portal site and do as much "development" as possible simply by configuring services through browser-based administrative pages. If you are going to succeed at building MOSS-based solutions, it's important to embrace this assemble-and-configure mindset.

However, there will inevitably be times when this mindset doesn't cut it for a particular solution. Then you can put your developer's hat back on and crank up Visual Studio 2005 to create many of the custom component types that are discussed here.

Building MOSS 2007 Portal Sites

You can expect to find all the portal features in MOSS 2007 that you have become accustomed to in SPS 2003, such as user profiles, audience targeting, personal sites (that is, My Sites), search, and single sign-on (SSO). MOSS 2007 also includes several new portal features like the Business Data Catalog.

If you have a background in building portal solutions on top of SPS 2003, you will find that MOSS 2007 is different in many ways. For example, SPS provides its own separate administrative Web application for provisioning and configuring portal sites. MOSS, on the other hand, doesn't require its own separate administrative application. Instead, MOSS integrates its administrative features and configuration links into the WSS Central Administration application using the "light-up" features of WSS 3.0.

MOSS 2007 is also quite different from SPS under the hood. SPS builds its portal site infrastructure around the concepts of areas and listings. However, areas in SPS 2003 represent a strange and undocumented layer on top of WSS 2.0 that developers have found difficult to extend using standard WSS techniques. So areas and listings have been eliminated in MOSS 2007 and replaced with a portal infrastructure that was designed and implemented much more in line with WSS best practices.

A MOSS 2007 portal site is a WSS site collection containing a top-level site along with several child sites below it. Unlike SPS 2003, a portal site does not have to be created at the root of an IIS Web site. This provides more flexibility because you can host hundreds of portal sites inside a single IIS Web site. (Remember that in WSS 3.0 terminology, an IIS Web site extended with WSS functionality is known as a Web application.)

You create a new MOSS portal site just as you would create any other new site collection through the WSS Central Administration application, the stsadm.exe command-line utility, or through custom code. When creating a new MOSS portal site, you typically use one of the portal site templates created by the Office team. For example, there is the Corporate Intranet Portal site template that has an internal name of SPSPORTAL#0. If you want to create a new portal site of this type, you can use the code shown in Figure 2, which works against the object model provided by WSS 3.0.

Figure 2 Creating a New Template-Based Portal (Close Figure 2)
    // Create the GlobalAdmin object
using(SPGlobalAdmin admin = new SPGlobalAdmin())
{
    // Open the target Web Application
    Uri path = new Uri("http://LitwareServer1");
    SPVirtualServer webApp = admin.OpenVirtualServer(path);

    // Create new site collection for portal site
    webApp.Sites.Add(@"http://LitwareServer1/sites/Portal1",
        "My new portal site", "My portal site description",
        1033, "SPSPORTAL#0", "LITWAREINC\\Administrator",
        "Litware Administrator", "administrator@litware.com");
}
Figure 3 Configuring MOSS 2007 through Shared Service Providers
Figure 3 Configuring MOSS 2007 through Shared Service Providers

In addition to this template, MOSS also supplies top-level portal site templates titled Internet Presence Web Site and Publishing Site. The Office team has also provided new site templates to create child sites within a portal site collection, such as Report Center and Search Center. In addition, the team evolved some of the older SPS 2003 site templates for use within MOSS 2007 portal site collections such as News, Topics, and the Sites Directory.

Shared Service Providers

The architecture of MOSS 2007 is based on shared service providers (SSPs). An SSP represents a set of services that can be configured a single time and shared across many different MOSS portal sites and WSS sites. Understanding SSPs is critical to being able to take advantage of MOSS features and services.

After you have finished installing MOSS 2007 in a farm, you must explicitly create and configure one or more SSPs to take advantage of MOSS features such as user profiles, audiences, personal sites, Excel® Services, the Business Data Catalog, and search. After creating an SSP through the Central Administration pages, you can then configure the individual MOSS services you need from the main SSP administration page, shown in Figure 3.

This new SSP architecture was designed to replace the shared services infrastructure of SPS 2003 to provide greater flexibility in deployment and configuration. For example, it's possible to create two different SSPs within the same server farm, each with its own configuration, as shown in Figure 4.

Figure 4 Each Web App Has Its Own Shared Service Provider
Figure 4 Each Web App Has Its Own Shared Service Provider

Within a MOSS 2007 farm, each Web application is associated with exactly one SSP. One Web application can be associated with one SSP while a different Web application can be associated with a second SSP. The MOSS portal sites and WSS team sites will use the SSP configuration for their parent Web application. As a result, the search results from within one portal site might be very different from the search results within another portal site if they are associated with different SSPs that have been configured to have different content sources.

User Profiles

User profiles are a MOSS 2007 feature that has been carried over from SPS 2003 to track information about the users within an organization. Profiles allow users to learn more about the people they work with and see how everyone fits into their company's organization chart. User profiles also provide the foundation for other MOSS features such as audience targeting and personal sites.

Once you have configured the user profile service through an SSP, MOSS stores the data for user profiles in Microsoft SQL Server. MOSS 2007 makes it possible to import and synchronize user profile data from external sources such as an Active Directory® domain as well as other LDAP-based identity management systems.

MOSS 2007 provides several standard pages and Web Parts to display the information tracked within user profiles. For example, Figure 5 shows how MOSS displays an organization chart based on user data imported from Active Directory. The MOSS object model provides an API to read and modify user profile data from components such as Web Parts, event handlers, and custom workflows.

Figure 5 Org Chart
Figure 5 Org Chart

As with SPS 2003, user profiles can be extended with custom properties to track user data for domain-specific business solutions. However, MOSS 2007 enhances custom properties by allowing for multivalued properties and properties defined with open or closed vocabularies.

When it's time to write custom code against a user profile, you can program against the UserProfileManager to load and inspect the property values for a specific user. Figure 6 shows a simple example of code written in the RenderContents method for a custom Web Part to display all property values within the user profile of the current user.

Figure 6 Displaying User Profile Values (Close Figure 6)
protected override void RenderContents(HtmlTextWriter writer)
{
  UserProfileManager mgr = new UserProfileManager();
  string UserName = Context.User.Identity.Name;

  if (!mgr.UserExists(UserName)) {
    writer.Write("User profile does not exist for " + UserName);
  }
  else {
    UserProfile prof = ProfileLoader.GetProfileLoader().GetUserProfile();
    writer.AddAttribute(HtmlTextWriterAttribute.Width, "100%");
    writer.AddAttribute(HtmlTextWriterAttribute.Border, "1");
    writer.RenderBeginTag(HtmlTextWriterTag.Table);
    writer.RenderBeginTag(HtmlTextWriterTag.Tr);
    writer.RenderBeginTag(HtmlTextWriterTag.Td);
    writer.Write("<b>Property</b>");
    writer.RenderEndTag(); // </Td>
    writer.RenderBeginTag(HtmlTextWriterTag.Td);
    writer.Write("<b>Value</b>");
    writer.RenderEndTag(); // </Td>
    writer.RenderEndTag(); // </Tr>

    PropertyCollection pc = prof.ProfileManager.Properties;
    foreach (Property p in pc) {
      writer.RenderBeginTag(HtmlTextWriterTag.Tr);
      writer.RenderBeginTag(HtmlTextWriterTag.Td);
      writer.Write(p.Name);
      writer.RenderEndTag(); // </Td>
      writer.RenderBeginTag(HtmlTextWriterTag.Td);
      writer.Write(prof[p.Name].Value + "&nbsp;");
      writer.RenderEndTag(); // </Td>
      writer.RenderEndTag(); // </Tr>
    }
    writer.RenderEndTag(); // </Table>
  }

}

Audience targeting is another valuable feature that has been carried over from SPS 2003. You can create an audience by specifying criteria to define a subset of users. For example, you can create a Sales audience that is defined as all the users who are members of an Active Directory group named Sales. You could define another audience named Carpool as the set of users with the custom Carpool property's value set to True.

Once you have defined an audience, you can then configure a Web Part to conditionally display its content only when the current user is a member of that audience. Audience targeting is a great way to show privileged users links to secured pages while hiding these links from non-privileged users who would receive Access Denied errors when attempting to follow them. This makes it straightforward to target content to those users who need it while hiding that content from users who either don't want to or shouldn't see it.

One more feature carried over from SPS 2003 that is built on user profiles is personal sites. When personal sites are enabled within an SSP, MOSS 2007 will provide each user with his own personal site—a My Site—that is provisioned on demand the first time it is used. Personal sites enable users to edit certain aspects of their user profiles to better describe themselves to their coworkers. A personal site has a public aspect, which makes it easy for a user to share data and documents with other users within the organization. A personal site also has a private aspect, allowing a user to store data and documents that are not meant to be shared.

MOSS 2007 Search

SPS 2003 search makes it possible to search through, not only content and documents within SPS portal sites and WSS team sites, but also through external content such as Windows file shares, public Microsoft Exchange Server folders, and standard Web sites. MOSS 2007 search was designed to give you these same features in a manner that is more performance oriented and easier to configure.

With the previous version of SharePoint, WSS 2.0 and SPS 2003 each used a different underlying infrastructure to support indexing and search. This created problems for companies that upgraded from WSS to SPS. These upgrade problems have been addressed, as the WSS and MOSS search services are now based on the same underlying indexing and search infrastructure, which is an evolved version of what was provided by SPS 2003.

MOSS 2007 search makes it possible to search external content across the network such as Windows file shares and Business Data Catalog (BDC) data, while WSS search is limited to searching within the current site collection. MOSS 2007 search can be configured to run the indexing service and search service on different servers within a farm to increase scalability and throughput. WSS search is limited to running the indexing service and search service on the same physical server.

Configuring MOSS 2007 search is an administrative exercise that entails creating and configuring content sources within the scope of a particular SSP. A content source defines a set of searchable content. When you create a new SSP, MOSS 2007 automatically creates a content source to search through user profile data as well as the content and documents within MOSS and WSS sites in the Web applications associated with the current SSP. However, the SSP administrator must explicitly create and configure additional content sources to support building indexes and searching through external content such as documents in a Windows file share or content from a public Web site.

WSS 3.0 and MOSS 2007 provide a rich user interface for searching by adding search boxes and result pages to each WSS site and MOSS portal site. MOSS 2007 goes even further to supply a dedicated child site named Search Center within a portal site collection that provides a specialized user interface for searching, as shown in Figure 7.

Figure 7 Search Center
Figure 7 Search Center

You might conclude that companies can take full advantage of MOSS 2007 search facilities without ever writing any custom code. However, MOSS makes it possible for you customize how search results look by modifying the XSLT transforms it uses to display search results. MOSS 2007 also exposes its search engine through a programmable API that enables you to extend the WSS 3.0 or the MOSS 2007 search service using custom code. For example, you can write a server-side component such as a Web Part or a custom workflow that queries the MOSS search service programmatically and deals with the returned search results in a customized fashion. Likewise, you can create a Windows Forms application that queries the search service from across the network through built-in MOSS Web services.

Business Data Catalog Integration

The BDC is a new framework that provides MOSS 2007 portal sites and standard WSS 3.0 sites with integration into back-end line-of-business systems such as those created by SAP, Seibel, and PeopleSoft. The BDC additionally provides the means to integrate data directly from database systems such as SQL Server and Oracle. While SPS 2003 made it possible to integrate portal sites with back-end systems, it required you to write custom code to manage connections and retrieve the data you needed to display. Furthermore, the required code changed significantly if you switched between back-end systems from vendors such as SAP and PeopleSoft. The Office team designed the BDC to make things much easier.

The BDC enables you to integrate data from back-end systems without requiring custom code for managing connections and retrieving data. The BDC design is based on standardized metadata that describes the location and format of a back-end system and data entities defined within it. The BDC also provides an execution component that is capable of reading BDC metadata and that is able to retrieve external data from back-end systems and return that data to MOSS in a standardized format.

Figure 8 shows the high-level architecture of the BDC. As you can see, connectivity between the BDC and traditional line-of-business systems is achieved by using standard Web services. Connectivity between the BDC and database systems is achieved by using ADO.NET providers.

Figure 8 Accessing Data in Back-End Systems
Figure 8 Accessing Data in Back-End Systems

The first step in using the BDC is to author an XML file containing the metadata to connect to a back-end system. When you author metadata for the BDC, you define the data you want to retrieve in terms of entities. For example, you might define a customer as one entity and an invoice as another entity. The BDC metadata format also lets you define associations between entities in scenarios when there is a one-to-many relationship such as one that might exist between customers and invoices.

The definition of a BDC entity contains identifiers, properties, and methods. The methods define how the BDC interacts with entry points exposed by the back-end system. For a back-end system accessible through Web services, methods define the names of the Web service operations and the parameters required to call them. For a back-end system that is a database such as SQL Server or Oracle, methods define the names of stored procedure and SQL statements.

Entities can also define actions. A BDC action is used to dynamically parse together the URL behind a hyperlink that allows a user to navigate from a page in a MOSS 2007 portal site to another location. For example, an action defined on a BDC customer entity could be written to redirect users to a Web page in an SAP application that supports updates to customer information. Actions were designed to support scenarios where the BDC is used to display read-only data and to bootstrap the user into another application when updating or some other type of external operation is required.

Once you have the XML file with the required BDC metadata for a back-end system, you must import it into the BDC within the scope of a particular SSP to create what is known as a BDC application. You can accomplish this by using the SSP administrative Web pages. You can alternatively import an XML file with BDC metadata using custom code written against the BDC administrative object model.

Once you have imported the required metadata, there are several out-of-the-box techniques to display the BDC application's data within a portal site. MOSS 2007 ships with a set of Business Web Parts that can be quickly added to pages to query and display BDC data. You can also add new columns to lists and document libraries based on an entity defined in a BDC application. A user editing a column based on a BDC entity is automatically presented with a user interface making it possible to query the back-end system.

The BDC integrates with the MOSS 2007 search service. For example, a back-end system and its entities can be defined as a content source so that the MOSS indexing service will crawl through its data and build indexes for the search engine. This becomes a powerful feature because it allows users to discover data from back-end systems about things like customers and invoices when running standard search queries.

The BDC provides convenient features to map data from a BDC entity to properties in a user profile and to synchronize this data at periodic intervals. For example, if your company has an SAP system that contains employee data you like to include in user profiles, such as phone numbers or social security numbers, you can configure this type of data importing without writing any custom code.

Finally, BDC entities can also be accessed programmatically using custom code written against the BDC object model. This makes it possible to write custom Web Parts as well as other server-side components and services that run their own BDC queries. One nice aspect of writing code to query BDC entities is that you don't have to worry about managing connections or whether you are accessing the back-end system through Web services or ADO.NET. All those details are abstracted away by BDC metadata and the BDC execution engine.

Web Content Management

The Microsoft strategy for publishing content on the Web has been, until recently, based on Microsoft Content Management Server (CMS) 2002. CMS provided a structured way for content authors to add content to a company's public Web site using professionally formatted layout pages. CMS also provided a formalized scheme where a privileged user must approve any page modification before it can be seen by the Web site's visitors. While there is a connector that provides a certain degree of integration between CMS 2002 and SPS 2003, these two products are built on very different architectures, and you cannot build a site that fully benefits from both the CMS Web content management features and the SPS portal features.

With the introduction of MOSS 2007, Microsoft recommends migrating CMS-based Web content management to MOSS. This will obviously have a significant impact on customers who have already become familiar with CMS development. If you have worked with CMS in the past, it's important to note that the CMS concepts of channels and postings are not used in the MOSS 2007 Web content management infrastructure. Instead, the infrastructure has been designed using basic WSS 3.0 building blocks such as child sites, page templates, content types, document libraries, and security groups. This approach lends itself to building custom solutions that extend the basic Web content management infrastructure by using standard WSS components such as custom event handlers and workflows.

When you need to brand a MOSS 2007 portal site, you can modify a single ASP.NET Master Page to customize the appearance of the entire Web site just as you would in a standard WSS 3.0 site collection. However, MOSS 2007 extends this functionality by introducing a publishing scheme based on page layouts. A page layout provides a structured approach to collecting content from content authors and displaying it on a page within a portal site. Some of the page layouts provided by MOSS 2007 include welcome pages, articles, and news items. An example of a page layout in editing mode as seen by a content author is shown in Figure 9. As you can see, page layouts are designed to make it fairly straightforward to add and modify content from within the browser. You might also notice that MOSS 2007 provides a toolbar within the browser to give content authors and approvers a convenient way to move content pages through the editing and approval processes.

Figure 9 Editing Mode
Figure 9 Editing Mode

Each page layout is based on a WSS content type and an associated .aspx page template. By layering page layouts on top of content types, MOSS makes it possible to add custom fields for storing different types of structured content such as HTML, links, and images. Once a custom field is defined inside the content type associated with a page layout, it can be data bound to the associated .aspx page template using another new MOSS 2007 component known as a field control. MOSS 2007 ships with several field controls such as a rich HTML editor as well as others for editing custom fields based on images and links. Many field controls support adding extra declarative constraints to keep portal content within a highly structured format.

Note that the page layout infrastructure is extensible because WSS content types support inheritance. It's fairly straightforward to take one of the built-in page layouts and customize it by extending its underlying content type or .aspx page template.

In addition to field controls, an .aspx page template for a page layout can also contain ASP.NET server controls and Web Part zones. A page layout with Web Part zones provides the content author with the flexibility to add Web Parts displaying content outside the schema of the current page layout. MOSS 2007 provides several Web Parts that have been designed for use in portal pages including the Table of Contents Web Part and the Content Query Web Part.

The .aspx page templates associated with page layouts are stored along with the portal site's Master Page in the master page gallery. The master page gallery also contains a metadata column to associate each .aspx page template with a content type. Note that it is possible to have multiple page layouts, each with its own .aspx page template, that are all associated with the same content type. This is useful when you want to create several different views for the same set of structured content.

Whenever a content author creates a new content page from a page layout, MOSS 2007 creates a new instance of the associated content type and stores it in a document library named Pages. When a content author updates content for custom fields within a page layout, WSS stores the data within a structure defined by the underlying content type. The fact that content page instances are stored in a WSS document library means that the MOSS Web content management infrastructure can take advantage of basic document library features such as versioning, auditing, approval, workflows, per-document security configuration and security UI trimming.

By default, MOSS uses the basic document approval features of a WSS document library to control when the updated content is shown to the site's visitors. However, the infrastructure was designed to make it straightforward to associate custom workflows with the Pages document library for scenarios where you need something more sophisticated than the default content approval functionality.

Note that an instance of a content page stored in the document library does not represent a copy of the page template. Instead, it contains redirection logic to link it with the .aspx page template at run time. That means updating the .aspx page template will always affect content pages that have already been created from the associated page layout.

A number of other MOSS 2007 features focus on Web content management. MOSS provides a framework for document converters, components designed to read content from an external format such as a Word document and convert it into a format that can be displayed within a content page. Several document converters are scheduled to ship with MOSS 2007, as well as a framework for building and integrating custom document converters.

MOSS provides content deployment features that allow you to transfer content from one site collection to another. This is valuable for companies that prefer to author content in a staging environment before moving it into their production environment. You take advantage of MOSS 2007 content deployment features by configuring paths and jobs. A path defines one site collection as a content source and another site collection as a content destination. Once you have defined a path, you can define one or more jobs to move content from the source to the destination. Jobs can be run on demand or they can be scheduled to run at a future time or on a periodic basis.

MOSS 2007 also supports a feature known as site variations for companies that need to duplicate a site's content for translation into multiple spoken languages or for targeting different types of rendering devices. For example, imagine you have configured variation support for German, French, and Spanish. MOSS 2007 maintains a parallel structure across these three different sites with respect to pages and child sites. When a content author adds a new page to the master variation site maintained in Spanish, MOSS automatically adds the same page into the structure of the other sites as well. MOSS can also be configured to create a WSS task marking the required translation as a to-do item for a language translator. While MOSS 2007 will not actually convert your content from one language to another, it does keep multiple sites in sync with respect to their content structure, which provides a good deal of value.

MOSS 2007 provides several caching options. While MOSS doesn't allow you to use ASP.NET output caching directives the same way you do in a standard ASP.NET page, it provides a more sophisticated framework to reach the same end. You can enable MOSS output caching at the site collection scope. When using these caching features, you configure caching profiles to control caching page items and complete pages in memory. Developers should take note that MOSS supplies dedicated caches for navigation nodes and content returned from potentially expensive retrieval operations such as standard WSS queries run using an SPQuery object and cross-site queries run using a SPS SPSiteDataQuery object.

MOSS 2007 also supports Web front-end disk caching. If you enable this cache, MOSS will begin writing the large files it retrieves from the SQL Server database into a special cache on the local file system of the front-end Web server. This procedure eliminates the need to continually move .jpg, .png, gif, .css, and .js files from the SQL Server database server to front-end Web servers on a per-request basis.

Finally, it's important to note that MOSS 2007 publishing sites benefit from WSS 3.0 security advancements. In particular, WSS 3.0 is built on top of the ASP.NET 2.0 authentication provider infrastructure. Unlike SPS 2003, which is tightly coupled to Active Directory, you can configure a MOSS 2007 portal site to use forms-based authentication. That means you can store the user credentials in a SQL Server database or another LDAP identity management system of your choosing.

Business Intelligence Features

SPS 2003, the Office 2003 Web Parts and Components Add-in, and Business Scorecard Manager all provided customers with important business intelligence (BI) features. Developers have used these BI technologies as a platform for building dashboard-style applications that provide upper-level management with up-to-date data that reflects the health of a business and flags potential problems in a timely matter.

Based on their experience with these earlier BI components, the Office team designed MOSS 2007 to include a next-generation platform for building dashboards and integrating with other technologies such as Microsoft Excel 2007, SQL Server Reporting Services, and SQL Server Analysis Services. As with all other aspects of MOSS 2007, its BI platform builds on top of ASP.NET and WSS 3.0 and provides many opportunities for extending the default functionality.

Consistent customer feedback indicated that a large percentage of corporations maintain a significant amount of business logic in Excel workbooks, and that this business logic has been hard to reuse across a large organization. This feedback led the Office team to create MOSS 2007 Excel Services, a server-side version of the traditional Excel calculation engine that has been rewritten from the ground up on top of WSS 3.0. Excel Services doesn't suffer from the same types of scalability problems that occur when you run the desktop version of Excel on the server.

Excel Services also provides a server-side rendering engine that can display worksheets in the browser as HTML. That means a company can store its Excel workbooks in a centralized document library and make them viewable by users who don't even have Excel installed on their desktops. Furthermore, users can see the numbers displayed by a worksheet within the browser without having any access to the business logic behind it, thereby protecting a company's intellectual property.

The 2007 Microsoft Office system introduces a new paradigm, recognizing that companies maintain business logic within Excel just as they maintain business logic within compiled assemblies or SQL stored procedures. To support this new paradigm, the Office team added many new features to the 2007 Office system products designed to expose and update this business logic as well as to protect the intellectual property represented by this logic.

The desktop version of Excel 2007 has been enhanced to allow information workers with Excel expertise to publish and update their workbooks in a document library within an MOSS 2007 portal site or a WSS team site. Users running Excel can view these workbooks through the familiar client experience while other users can rely on Excel Services to view the same workbook inside the browser. Note that this new workbook publishing metaphor allows a company to maintain a single master copy of its critical workbooks. It also allows the workbook author to post updates without the need to involve the development or IT staff.

It's important to note that the use of Excel Services isn't restricted to the browser. You can create a Windows Forms application that uses the server-side Excel calculation engine but that doesn't use the rendering engine. For example, a Windows Forms application can use Web services from Excel Services to load a workbook on the server, enter input data, perform calculations, and return a result as shown in Figure 10.

Figure 10 Calculating with Excel Services (Close Figure 10)
// Create instance of proxy from ExcelService.asmx
ExcelService ws = new ExcelService();

// Set up authentication
ws.Credentials = System.Net.CredentialCache.DefaultCredentials;

// Open the Excel workbook on server
Status[] status = null;
string sessionID = null;
sessionID = ws.OpenWorkbook(
    textBoxURL.Text, "en-US", "en-US", out status);

// Assign the cell values for input data
status = ws.SetCellA1(sessionID, "Mortgage Calculator",
                      "CustomerName", textBoxCustomerName.Text);
status = ws.SetCellA1(sessionID, "Mortgage Calculator",
                      "MortgageAmount", textBoxAmount.Text);
status = ws.SetCellA1(sessionID, "Mortgage Calculator",
                      "InterestRate", textBoxInterestRate.Text);
status = ws.SetCellA1(sessionID, "Mortgage Calculator",
                      "MortgageLength", textBoxLength.Text);

// Perform server-side calculation
status = ws.CalculateWorkbook(sessionID, CalculateType.Recalculate);

// Show the result
object result = null;
result = ws.GetCellA1(sessionID, "Mortgage Calculator",
                      "Payment", true, out status);
if (result != null) MessageBox.Show("You pay " + result.ToString());

// Close workbook
status = ws.CloseWorkbook(sessionID);

You can see it's not that hard to take advantage of formulas in Excel spreadsheets from across the network in a desktop application. This example furthers the analogy that Excel Services exposes the business logic defined in a workbook just as SQL Server exposes the business logic defined in a stored procedure.

MOSS 2007 provides a special site template named Report Center for companies that want to build dashboard-style applications. Report Center was designed to make the new BI features of MOSS 2007 easy to discover and use. A Report Center site contains a document library named Reports Library that is tuned for storing and displaying BI reports such as Excel workbooks and reports built for SQL Reporting Services.

Another important aspect of Report Center is the built-in support for creating and importing key performance indicators (KPIs), visual BI indicators that tell a manager how some aspect of the business is doing. For example, the KPI for a product inventory level might display a green light when there is enough inventory to supply all the orders for the coming week. However, the light might turn from green to yellow when the inventory level drops below some predefined threshold such as the amount of inventory required to supply orders for the next four days. The light then might change from yellow to red when the inventory level drops to a point where it will run out within the next 48 hours. The main idea is that a KPI flags business problems that require immediate attention.

MOSS 2007 provides out-of-the-box support for several different KPI types. For example, you can create a KPI whose indicator changes automatically depending on data it reads dynamically from a WSS list or an Excel workbook. MOSS also provides integration support for KPIs in SQL Server 2005. That is, if you have already created KPIs with SQL Server Analysis Services, you can import and display them on a Report Center site alongside other supported KPIs.

The last aspect of Report Center I want to discuss is the built-in framework for filtering data before it's shown to the user. This is a key component of the MOSS 2007 dashboard framework because it makes dashboard pages more relevant to the user. When a manager visits a Report Center site, the experience is enriched if the dashboard views have been customized with data that is relevant for that particular manager. For example, a sales manager for the Eastern region can be presented with a different view of sales figures than the sales manager for the Western region. Furthermore, managers like to be able to see high-level data at first and then drill down into more specific categories on demand.

Filtering support is built into MOSS 2007 dashboards at the page level using Web Part connections. MOSS supplies Web Parts that allow page designers and users alike to specify criteria such as the name of the current user, a date range, or a product category. There are also many supplied Web Parts that can be configured to consume the filtering criteria supplied by other Web Parts such as the standard WSS List View Web Part as well as the Web Parts designed for use with the BDC, Excel Services, SQL Reporting Service, and SQL Analysis Services.

Managing Documents and Business Processes

In past versions of Word, Excel, and PowerPoint®, Microsoft relied on a default file structure based on binary files written in a proprietary format. These formats have been very hard to read and modify unless you go through the object model of the hosting Office application, such as Word and Excel. As a result, companies have tried to run Office desktop applications on the server, which poses problems with scalability and robustness.

Office 2000 and Office 2003 added some modest capabilities for creating Excel workbooks and Word documents using XML. In the 2007 Office release, Microsoft takes this idea much further by adopting the Office Open XML File Formats for Word, Excel, and PowerPoint documents. These formats use a new file standard for creating composite documents containing multiple inner XML files that factor out content from other aspects of the document such as formatting instructions, data, and code.

The top-level file in the Office Open XML File Formats is known as a package and it is structured using standard XML and ZIP technologies. The internal files contained within a package are known as parts. Many parts within Word, Excel, and PowerPoint files contain XML structured in accordance with published XML schemas. Other parts within a package can consist of binary files for items such as graphics, audio clips, and video.

A major goal of the Office Open XML File Formats is to provide a standard approach for reading, manipulating, and generating documents in server-side scenarios where using the object model of a desktop application such as Word or Excel isn't a viable option. Think about a scenario in a MOSS portal site when you have created and configured an event handler to fire whenever someone uploads a new Word document. The new Office Open XML File Formats make it significantly easier to extract data or to perform hygiene on the document such as removing comments and personal information. You can also use Office Open XML File Formats to develop server-side components that generate Office documents on the fly using data pulled from content sources such as a WSS list or the BDC.

To get started working with the Office Open XML File Formats, you need to learn how to program against the new WinFX® packaging API. You must also learn the specific package structure and XML schemas for the type of Office document you are working with. Some of these details will change as you move between Word, Excel, and PowerPoint documents. Microsoft has started a developer community effort around the Office Open XML File Formats at openxmldeveloper.org.

InfoPath Forms Services

Since the introduction of Office 2003, many companies have found that InfoPath® provides a quick and efficient solution for creating electronic input forms to collect XML-based data from users. The InfoPath forms designer provides high levels of productivity because it significantly reduces or eliminates the need to write custom code when developing forms. InfoPath also creates a very reliable way to collect data because InfoPath forms are built on top of XML schemas, allowing them to automatically validate the user's input.

InfoPath 2003 provided a convenient integration point with WSS 2.0 where a form designer could publish an InfoPath form on a WSS site, creating what is known as a forms library. A forms library is a hybrid WSS document library that uses an InfoPath form template as its underlying document template and acts as a repository for XML documents containing form data entered by users.

One common complaint that Microsoft received about this forms-based technology is that it required a full installation of InfoPath on the user's computer. This was true not only for those designing forms, but also for any users who needed to read or modify data in an InfoPath form. The problem was that the features of InfoPath 2003 did not extend to users for whom deploying Office 2003 on their desktop wasn't an available option.

MOSS 2007 introduces InfoPath Forms Services to solve this problem. Forms Services renders InfoPath forms within the browser to reach users who are not running InfoPath. In fact, Forms Services doesn't require Internet Explorer® or even Windows on the user's system. The Office team is actively testing Forms Services for compatibility with browsers such as Firefox, Safari, and Netscape as well as several HTML-enabled mobile devices.

You can use the InfoPath 2007 form designer to create Web-enabled forms, which can then be deployed to Forms Services. The form designer provides a compatibility checker to ensure that your forms contain only controls and elements that are compatible with what Forms Services can render to the browser. It's also possible to create and deploy InfoPath forms intended for dual use. Such a form is downloaded to the desktop and loaded into the client environment when InfoPath is available or rendered through the browser when InfoPath is not installed.

ECM and Single Sign-On

Government and other regulatory requirements, such as the Sarbanes-Oxley Act and emerging privacy laws, make compliance ever more complicated for companies that generate and store large numbers of documents. MOSS 2007 provides several Enterprise Content Management (ECM) features that can help address these problems. Many of these ECM features are built on top of the new information policy feature, which is a WSS component that can be enabled and configured within the scope of a list or document library. Default examples that ship with MOSS 2007 include information policies for document expiration, auditing, and the automatic generation of bar code labels to identify physical documents and associate them with electronic copies maintained in a document library.

The WSS framework for information policies was designed with extensibility in mind. For example, you can create a custom policy that checks the integrity of a digital signature on every document within a document library. You can create another policy that promotes document hygiene and privacy by removing all the comments and personal information from documents as they are uploaded to a document library.

MOSS 2007 provides a dedicated site template named the Records Repository. A Records Repository site provides archiving support for companies that are required to keep certain types of business documents as official records of the company's activities. While requirements vary across different regulated industries, keeping records is required to provide evidence of a company's activities in the event of litigation or an audit.

The final topic I would like to cover is the SSO service, an enhanced version of the credential-mapping service that ships with SPS 2003. SSO is used to map the identity of a user who has logged on to a MOSS 2007 portal site with another identity for the same user in a back-end system. For example, imagine a user named Bob has two user accounts. First, he has a Windows account in Active Directory that he uses to log on to the local network and authenticate against the MOSS 2007 Web server. Bob also has a second user account with different credentials that needs to be used when accessing the company's back-end SAP application.

SSO solves the problem of allowing server-side code running on behalf of Bob to seamlessly access the SAP system using his secondary account once he has logged on to the portal site using his primary Active Directory account. SSO accomplishes this by providing a credential-mapping database to store his SAP user name and password in an encrypted format. SSO also provides the means for custom Web Parts and other MOSS 2007 services such as the BDC, Excel Services, and Forms Services to retrieve user credentials that are required when accessing various back-end systems.

Wrapping It Up

The article has provided a broad survey of the most significant services and features provided by MOSS 2007. You have seen the investment in functionality for areas such as portal and search, Web content management, business intelligence, business process management, and enterprise content management. You have also seen that MOSS 2007 promotes an "assemble and configure" philosophy that enables you to construct rich business solutions without ever starting Visual Studio or writing a line of managed code. However, remember that MOSS 2007 is built on top of ASP.NET 2.0 and WSS 3.0, providing ample opportunities for extending standard functionality with custom components such as Web Parts, event handlers, workflows, page layouts, document converters, and server-side forms. For more information about MOSS 2007 and the availability of beta software, see microsoft.com/office/preview/servers/sharepointserver/highlights.mspx.

Ted Pattison is an author, trainer and SharePoint MVP who lives in Tampa, Florida. Ted is writing a book titled Inside Windows SharePoint Services 3.0 for Microsoft Press and he delivers advanced SharePoint 2007 Training to professional developers through his company, Ted Pattison Group (http://www.TedPattison.net).


Add-In Power: Let Users Customize Your Apps With Visual Studio Tools For Applications

Add-In Power

Let Users Customize Your Apps With Visual Studio Tools For Applications

Paul Stubbs
This article is based on a prerelease version of the 2007 Microsoft Office system. All information herein is subject to change.
This article discusses:
  • The System.AddIn namespace
  • Add-in basics and the ShapeApp sample
  • Creating an add-in for InfoPath 2007
  • Getting Visual Studio Tools for Applications
This article uses the following technologies:
Visual Studio 2005, InfoPath 2007, C#, Visual Basic
Contents
About Add-Ins
Learning with ShapeApp
More Add-Ins: InfoPath 2007
Getting Visual Studio Tools for Applications

M
any software developers have increased the usefulness of their applications by making them customizable, usually through the addition of a technology like Microsoft® Visual Basic® for Applications (VBA). But these days VBA doesn't go far enough. You need a way to provide customers with better security, better development tools, 64-bit support, and generally more robust solutions—solutions that can be easily taken over by professional developers after a user's macro turns into an enterprise-critical tool.

Three application-customizing technologies are currently available from Microsoft: Visual Studio® Tools for Applications, Visual Studio Tools for Office, and VBA. Developers often ask how these similar-yet-diverse technologies address their specific needs. VBA is the oldest and most familiar. You can think of Visual Studio Tools for Applications as the .NET Framework alternative to VBA. Visual Studio Tools for Applications and VBA can run side-by-side within the same app. This enables apps that use VBA to add Visual Studio Tools for Applications support, allowing users to begin using Visual Basic or Visual C#® when they are ready.

The same team that created Visual Studio Tools for Applications also developed Visual Studio Tools for Office and the two technologies share a common architecture. However, Visual Studio Tools for Office is designed for developers who want to include Microsoft Office in custom solutions, while Visual Studio Tools for Applications is designed for end users of any application that includes it. For example, an accountant could add a new capability to accounting software by writing a function, without having to involve the company's development department. End users add new functionality by creating add-ins.

Visual Studio 2005 Tools for Applications is designed to meet developers' needs by offering application customization through .NET Framework-based code, which the end user can develop in a simplified Visual Studio 2005 development environment.

The list of requested features for every application is normally greater than the development resources available, so how do you satisfy the customers within your constraints? With Visual Studio Tools for Applications, your app users can create add-ins that meet their unique needs. Integrating Visual Studio Tools for Applications allows you to spend less time developing features for a minority of customers, makes it easier to empower users of your application as developers, and cuts the cost of bringing new developers up to speed on your application.

There are really two aspects to using Visual Studio Tools for Applications. The first involves integrating it into your application. This means deciding what the experience will be for those who will develop solutions for your application. The second aspect is what the end user developer (the person who is using your application and wants to customize it) will see. You design the development experience for the user through the decisions you make during the integration phase. For illustration purposes, I'll show you an example of the choices made by the developers of Microsoft Office InfoPath® 2007 when they integrated Visual Studio Tools for Applications into InfoPath.

About Add-Ins

As mentioned, the vehicle for new functionality is an add-in. The term add-in has different meanings depending on context, but Visual Studio Tools for Applications uses the term "add-in" or "customization" to mean any component that is dynamically loaded by an application. The application that loads the add-in is called the host application. Application-level add-ins are familiar to users of Office in the form of COM add-ins.

Add-ins always have a lifetime. For example, an add-in can have an application-level lifetime, meaning that it loads when the application starts and unloads when the application stops. Add-ins can have a narrower lifetime as is the case with Visual Studio Tools for Office-based Word and Excel® solutions, whose lifetime is tied to a document. Add-ins can also be scoped to apply to only a portion of a host application's object model. In an application like Word, for example, application-level add-ins would be scoped to the Application object and document-level add-ins would be scoped to the Document object. An easy way to think of the scope is that it is the object returned when you reference the "Me" keyword in Visual Basic or the "this" keyword in C#. These are just two examples of lifetime and scope.

The host application is free to include add-ins with any lifetime or scope. For instance, you can implement add-ins scoped to forms or even dialog boxes. Visual Studio Tools for Applications makes no restrictions on the host application and is flexible enough to handle a wide-variety of scenarios.

Visual Studio Tools for Applications provides classes that let the host application load and unload add-ins dynamically. The default behavior loads each add-in into its own application domain, which provides isolation between the host and the add-in and between other add-ins. Application domain isolation prevents the add-in from interfering with the host application and enables the add-in to be unloaded from memory. Other add-in loading models enable various levels of isolation by allowing multiple add-ins to be loaded into the same application domain or even the host's application domain. Add-ins may even be loaded into an application domain in a process outside of the host's application process, which provides the greatest isolation.

As more applications move to 64-bit processors, the ability to load 64-bit add-ins becomes increasingly important. Visual Studio Tools for Application supports creating and loading 32-bit, 64-bit, or bit-neutral add-ins. Bit-neutral is the default behavior and allows the add-in to run in either a 32-bit or a 64-bit process.

The add-in loader is defined in the System.AddIn namespace and is contained in the System.AddIn.dll assembly. This namespace will ship first with Visual Studio Tools for Applications and then as part of WinFX®. The components that will ship as part of WinFX are collectively known as the Managed Add-In Framework (MAF). MAF allows you to create new versions of add-ins independent of the host application's version. MAF also lets add-ins dynamically load and unload in a secure manner. MAF uses contracts as the immutable piece that allows independent versions. The contracts are defined in the System.AddIn.Contract namespace and are contained in the System.AddIn.Contract.dll.

Contracts enable proxies and adapters to communicate across application domains. Proxies and adapters are the pieces that enable add-ins to communicate with the host application (see Figure 1). Visual Studio Tools for Applications includes a command-line tool called Proxygen.exe that takes your app's object model as input and creates the proxies as C# source code files. You can then modify the proxies to suit your needs. Proxygen.exe can also generate an XML-based descriptor file that represents your object model. The proxy descriptor file can be the sole input to Proxygen.exe, letting you get the proxy you need by manipulating the input descriptor file or the generated C# file or both.

Figure 1 Add-Ins Communicate with the Host App
Figure 1 Add-Ins Communicate with the Host App

Adapters implement the System.AddIn.Contract interfaces that the proxies call into. Proxies and adaptors hide the implementation and complexity of using contracts from the host application and the add-ins. The proxy provides a friendly view of the host application's object model as well as a good place to optimize cross-boundary calls by implementing techniques like caching. Each part is packaged in a separate assembly. The proxy is used as the base class for the add-in assembly, which is created using Visual Studio Tools for Applications.

Visual Studio Tools for Applications contains a subset of features normally contained in the Visual Studio 2005 development environment, but otherwise it looks and operates much like the familiar Visual Studio IDE (see Figure 2). You can embed the Visual Studio Tools for Applications environment into your application to give your users a seamless add-in development environment of their own.

Figure 2 Coding a Macro in Visual Studio Tools for Applications
Figure 2 Coding a Macro in Visual Studio Tools for Applications

Like VBA, the Visual Studio Tools for Applications environment can be launched from within your host application using a key combination like Alt+F11. Unlike VBA, Visual Studio Tools for Applications can also be launched from the Start menu, giving users a development model more like Visual Studio. You are free to choose the level of integration appropriate for your application and your end users.

A benefit of having a common architecture with Visual Studio is that you can open Visual Studio Tools for Applications projects in other versions of Visual Studio. This solves a problem often encountered when using VBA in the enterprise: someone creates a VBA-based solution for personal or limited workgroup use. The solution becomes so successful that it begins to see wider distribution, but it isn't able to scale or may not comply with corporate security or other standards. Another possible complication is that the team responsible for corporate-wide solutions may not be familiar with VBA. In such cases, the common response is to rewrite the customization entirely.

These problems disappear if the host application employs Visual Studio Tools for Applications projects. Visual Studio Tools for Application projects are true Visual Studio projects. Developers can open a Visual Studio Tools for Applications project using the full version of Visual Studio to add enterprise features like unit testing and performance testing.

Learning with ShapeApp

ShapeApp is a sample app included in the Visual Studio Tools for Applications SDK. This drawing program allows you to drag shapes onto a drawing surface (see Figure 3).

Figure 3 ShapeApp Sample App
Figure 3 ShapeApp Sample App

There are two versions of ShapeApp, one written in C++ and one written in C#, and there are three samples for each version. ShapeAppCSharp.zip and ShapeAppMFC.zip contain the source code projects for the ShapeApp application with no Visual Studio Tools for Applications modifications. Walkthroughs in the documentation use these projects as a starting point to take you through integrating Visual Studio Tools for Applications.

ShapeAppBasicCSharp.zip and ShapeAppBasicMFC.zip contain the source code projects for ShapeApp with a basic implementation of Visual Studio Tools for Applications. This is the end result of the walkthroughs, so if you just want to see a running sample, skip the walkthrough and look at this instead. The basic implementations focus on creating an application level add-in. The walkthrough shows you how to use the add-in manager to load and unload add-ins and how to create MAF-enabled proxies for your object model.

This sample also takes you through the steps to create Visual Studio Tools for Applications project templates for your add-in. These templates are regular Visual Studio project templates. They're the ones you see when you open the New Project dialog box in Visual Studio. Project templates are used to create a project instance.

ShapeAppAdvancedCSharp.zip and ShapeAppAdvancedMFC.zip contain the source code projects for the ShapeApp application, integrating some of the advanced features of Visual Studio Tools for Applications.

More Add-Ins: InfoPath 2007

You can also try out an example of Visual Studio Tools for Applications integration by looking at InfoPath 2007. I'll walk you through the process of creating an add-in so you can see the type of experience that Office designed—but keep in mind that this is not the only way that the user experience can be designed. All you need to follow along is a copy of InfoPath 2007.

InfoPath lets you design and fill out structured data forms. Unlike most Office applications, InfoPath does not support VBA. Instead it supports JScript® and VBScript out of the box and uses Microsoft Script Editor. Starting with InfoPath 2003 Service Pack 1 (SP1), you could download the InfoPath 2003 Toolkit for Visual Studio .NET 2003, which lets you create solutions in Visual Basic or C#. The toolkit was updated for Visual Studio 2005 and was included with Visual Studio 2005 Tools for Office. InfoPath 2007 includes Visual Studio Tools for Applications, so you can create applications in Visual Basic or C# using the embedded Visual Studio Tools for Applications environment. Figure 4 shows what the development environment and code look like when you press Alt+Shift+F12 while in design mode.

Figure 4 Using Visual Studio Tools for Applications with InfoPath 2007
Figure 4 Using Visual Studio Tools for Applications with InfoPath 2007

Let's create a simple InfoPath form and customize it using Visual Studio Tools for Applications. InfoPath includes a number of sample forms, so we'll start with one of these. On the File menu choose Fill Out a Form, and then choose Customize a Sample. Select SampleForm, then the Meeting Agenda form. Click Design this Form from the tasks on the right. Now you can see the sample form in design mode.

Press Shift+Alt+F12 or click Tools | Programming | Microsoft Visual Studio Tools for Applications to open the development environment. You'll be prompted to save your form first. Click OK and choose a name and location to save your form. The development environment opens with Visual Basic as the default language. You can switch between C# and Visual Basic as the default language in InfoPath by selecting Tools | Options | Design.

At this point you'll see that InfoPath has generated some code, which is shown in Figure 4. It has added a few Import statements to reference the InfoPath object model and the XML classes in the .NET Framework. The namespace matches the name of your form, and a class called FormCode contains an InternalStartup method. InternalStartup is where InfoPath will add the event handlers. Pressing F5, or clicking Debug | Start Debugging, will compile the code and start debugging your InfoPath form in Preview mode. At this point the preview form will open, but nothing else happens because you haven't written any code yet.

So now let's go ahead and add some code to the form. Switch focus back to Visual Studio Tools for Applications and press Ctrl+Alt+Break or Debug | Stop Debugging. When the debugger stops it will close the Preview mode of the form. The first thing we'll do is display a simple message box when the form is loaded (see Figure 5). You can get InfoPath to create the form-loading event by clicking Tools | Programming | Loading Event from the design view of the form. Add the code to show the message box to the FormEvents_Loading method. This method will be called when the form opens.

Figure 5 Simple Message Box (Close Figure 5)

C#
public void InternalStartup()
{
    EventManager.FormEvents.Loading +=
        new LoadingEventHandler(FormEvents_Loading);
}

public void FormEvents_Loading(object sender, LoadingEventArgs e)
{
    MessageBox.Show("The form has loaded");
}
Visual Basic
Private Sub InternalStartup( _
        ByVal sender As Object, ByVal e As EventArgs) Handles Me.Startup
    AddHandler EventManager.FormEvents.Loading, _
               AddressOf FormEvents_Loading
End Sub

Public Sub FormEvents_Loading(ByVal sender As Object, _
ByVal e As LoadingEventArgs)

    MessageBox.Show("The form has loaded")
End Sub

Run the form now by pressing F5 and you'll see that the message box is displayed before the preview form loads. Click OK to close the message box and allow the form to finish loading. At this point your code is complete and attached to the form. You could close Visual Studio Tools for Applications and the code would still run whenever you switch to preview mode.

That's how easy it is to get started. Now let's try adding code that's a bit more complicated to this scenario. The sample form has a Prepared By field. On the loading event I want to populate the project name, manager name, and department name. As you can see in Figure 6, I have created a helper function called SetFieldValue to make it easy to set the value to multiple fields. I can call SetFieldValue on the loading event, passing in the new value and the XPath to the field. So how do you know what the XPath value should be? You can find the XPath value by selecting the field in the design view of InfoPath and showing the Data Source task pane. The Data Source task pane displays a tree view of the data fields. Right-click on the field you're interested in and choose Copy XPath from the context menu. Paste this XPath expression into code where you need to reference a field.

Figure 6 Using SetFieldValue to Populate Multiple Fields (Close Figure 6)

C#
public void FormEvents_Loading(object sender, LoadingEventArgs e)
{
    MessageBox.Show("The form has loaded");

    SetFieldValue("VSTA Overview",
        "/my:meetingAgenda/my:meeting/my:meetingTitle");
    SetFieldValue("Paul Stubbs",
        "/my:meetingAgenda/my:meeting/my:meetingOrganizer");
}

private void SetFieldValue(string value, string xpath)
{
    XPathNavigator root = this.MainDataSource.CreateNavigator();
    XPathNavigator field = root.SelectSingleNode(xpath,
        this.NamespaceManager);
    //set the value
    field.SetValue(value);
}
Visual Basic
Public Sub FormEvents_Loading( _
        ByVal sender As Object, ByVal e As LoadingEventArgs)
    MessageBox.Show("The form has loaded")

    SetFieldValue("VSTA Overview", _
        "/my:meetingAgenda/my:meeting/my:meetingTitle")
    SetFieldValue("Paul Stubbs", _
        "/my:meetingAgenda/my:meeting/my:meetingOrganizer")
End Sub

Private Sub SetFieldValue(ByVal Value As String, ByVal xpath As String)
    Dim root As Xml.XPath.XPathNavigator = _
        Me.MainDataSource().CreateNavigator()
    Dim field As Xml.XPath.XPathNavigator = _
        root.SelectSingleNode(xpath, Me.NamespaceManager())

    field.SetValue(Value)
End Sub

This is just one way to design the development experience. You can provide basic functionality or you can add advanced features such as macro recording to help your users. Visual Studio Tools for Applications gives you the tools, but what you create is completely up to you.

Getting Visual Studio Tools for Applications

Visual Studio Tools for Applications is included in the Visual Studio 2005 SDK, which you can download from the MSDN® Extensibility Web site at (msdn.microsoft.com/vstudio/extend) though you'll need to register for free as a VSIP affiliate member). The SDK contains the Visual Studio Tools for Applications setup package. Running the package installs the Visual Studio Tools for Applications development environment and runtime. The SDK also contains the help documentation and sample applications. The help file walks you through the process of integrating Visual Studio Tools for Applications into your application. You'll also find tools to help with the integration process: Proxygen.exe creates the MAF proxies from your object model; Projectgen.exe generates Visual Studio Tools for Applications project templates to create add-ins.

The Visual Studio SDK contains help and samples for customizing Visual Studio and many of these topics relate directly to Visual Studio Tools for Applications as well. You'll find information about creating custom help, project templates, project wizards, and profiles, all of which will be needed to create a seamless development experience for your end users.

Visual Studio Tools for Applications is the way to provide end users with a powerful and flexible customization experience that helps them get what they want from your application. The Visual Studio Tools for Applications development environment and the .NET Framework give your users tools for rapid development in an environment that helps keep applications secure. The result is that your app is tailored for each user in a way that can be expanded to be enterprise ready at any time—and you didn't have to add any new, unplanned features to your app to let them do it.

Paul Stubbs is a program manager with the Visual Studio Tools for Applications team at Microsoft and also works with the Visual Studio Tools for Office team developing Office solutions. He has spoken at Tech·Ed and TechReady and participates in the developer community on the Microsoft forums. Read Paul's blog at blogs.msdn.com/pstubbs.


InfoPath 2007: Designing Form Templates With The New Features Of InfoPath

InfoPath 2007

Designing Form Templates With The New Features Of InfoPath

Scott Roberts and Hagen Green
This article is based on a prerelease version of the 2007 Microsoft Office system. All information herein is subject to change.
This article discusses:
  • Design once for InfoPath and the Web browser
  • Template Parts—building reusable components in InfoPath
  • Hosting the InfoPath form control
  • The InfoPath managed code object model
This article uses the following technologies:
InfoPath 2007, Office SharePoint Server 2007
Contents
InfoPath Forms Services
Design Once
Template Parts
Document Information Panel
Hosting the InfoPath Form Control
Importing and Exporting Forms
Managed Object Model
Visual Studio Integration

I
nfoPath 2007 is an XML forms designer and editor package in the 2007 Microsoft® Office system. Now in its third iteration (if you include InfoPath® 2003 Service Pack 1), InfoPath has matured into a fully featured and integrated member of the Office family. In this article, we give you a sneak peek at some of its highlights and new features.

InfoPath 2007 includes a new Design a Form dialog (see Figure 1) that simplifies the process of designing form templates. You can also build Template Parts (reusable components you can insert into multiple form templates) as well as browser-enable your form templates and Template Parts. Browser-enabled forms are designed in InfoPath and can be filled out either in InfoPath or in a Web browser. Browser-based forms are hosted using InfoPath Forms Services from Microsoft Office SharePoint® Server 2007.

Figure 1 The Design a Form Dialog
Figure 1 The Design a Form Dialog

You'll find a number of new controls in InfoPath 2007, such as the Combo Box and Multiple-Selection List Box, both of which function like their Windows® control counterparts. To accommodate multiple selections, the Multiple-Selection List Box control binds to a repeating field in the data source and every selection adds another item to the repetition. As with the Combo Box, users can enter custom values into the Multiple-Selection List Box; this is configurable on the control during the design. Other new controls include Horizontal Repeating Table and Horizontal Region.

Integration with various Office applications is much improved. For example, InfoPath brings native form-filling capabilities directly to Outlook® 2007 and e-mail messages. Other Office system applications such as Word, Excel®, and PowerPoint® use InfoPath in their Document Information Panels, which contain a mini-form that lets users fill in metadata properties on the document.

InfoPath 2007 has many other enhancements: better form merging, exporting data, built-in form template importers for Word and Excel, the addition of Information Rights Management (IRM) to form templates, improvements for Tablet PC, and Visual Studio® 2005 integration. And there's InfoPath Forms Services, which lets your InfoPath forms reach out to anyone with or without InfoPath, at any time, on almost any device.

InfoPath Forms Services

InfoPath is great for building forms, but what about deployment? Not all of your customers will have access to the InfoPath client application—and they won't have to. InfoPath Forms Services, which is built on top of SharePoint Server 2007, uses SharePoint document libraries, content types, permissions, and administration to deliver an integrated forms-management package. Moreover, Forms Services is designed to extend the reach of your forms. Not only does it support Microsoft Internet Explorer® on Windows, it can also serve forms to Netscape and Firefox for UNIX and Windows, and even Firefox and Safari for the Macintosh, and it can render content for mobile devices such as Smartphones and PDAs.

To get a form template into InfoPath Forms Services for filling out in the browser, you design a form with the "Enable browser-compatible features only" setting checked from the Design a Form dialog shown in Figure 1. This is not an up-front requirement; the setting can actually be changed at any time during the design. While you're designing the form, notice that not all InfoPath features are available and some will be limited in their functionality in the Web browser. As we'll see, the few limitations that exist allow the same form to be used in InfoPath and Forms Services without any modifications. We call this "design once," and we'll review the concept in more detail in the next section.

Let's design the Status Report sample form under the Customize a Sample link in the Design a Form dialog shown in Figure 1. This form is already browser enabled, so all we need to do is publish it to a server running InfoPath Forms Services. To do so, click on the Publish Form Template link in the Design Tasks task pane. This brings up the Publishing Wizard, with various options for publishing forms. In this case, we're publishing to a SharePoint Server with InfoPath Forms Services. Subsequent wizard steps ask which server to use for publishing, whether you want to create a SharePoint document library or content type, and what form data (if any) to promote as columns in the library or content type. The last page offers a link to the library where we've published our browser-enabled form template. From there we can fill out our Status Report form.

Simply clicking the New button in the document library creates a new Status Report form. By default, the form will open in the InfoPath client application if it's available. That's because InfoPath is the preferred client for form filling (though there's a setting in the library to default to the browser). If InfoPath is not available, the form will render in the browser automatically. Figure 2 shows a filled-out form in InfoPath Forms Services. Notice that browser forms support rich text editing (as seen in the Summary field) with the Rich Text Box control. For comparison, Figure 3 shows the exact same Status Report filled out in the InfoPath client application.

Figure 3 Filling Out the Same Status Report Using InforPath
Figure 3 Filling Out the Same Status Report Using InforPath
Figure 2 Filling Out the Status Report Form in a Web Browser
Figure 2 Filling Out the Status Report Form in a Web Browser

Design Once

Design once means you can create a form template that will work properly within both the InfoPath application and a Web browser. There's no need to decide ahead of time whether the form should target rich client or browser-based users. If you do choose to design a form specifically for the InfoPath client (and therefore not browser enabled), InfoPath exposes an extended set of features. Controls such as the Combo Box, User Roles, showing message boxes through a rule action, and the custom task pane are some of the extra features you'll see.

There are classes of features that don't work with Forms Services: unsupported and incompatible. If you include unsupported features (such as message boxes and custom task panes), you can still use the form template in Forms Services, but those features are nonfunctional. Incompatible features, in contrast, will fail when published to Forms Services.

How do we know which features are compatible and which are not? The Design Checker, shown in Figure 4, is a new task pane that communicates potential design problems to the developer. For this example, we designed a complex form that is not browser enabled. To make it compatible with Forms Services, we clicked on the Change Compatibility Settings link and modified the setting that enables the form to work in either a browser or in InfoPath. The Design Checker displays errors because our initial design included InfoPath client-only features, such as two Master/Detail controls that are incompatible with browser-enabled forms. When designing a browser-compatible form template, such features are not even available in the user interface. To begin fixing the errors, you simply click on the error to jump to the invalid control. If an error is not on a control, InfoPath gives instructions on how to proceed.

Figure 4 Design Checker Task Pane
Figure 4 Design Checker Task Pane

In addition to browser enabling the form, we also specified our server's name in the compatibility settings so we can enable the Verify on server option in the Design Checker. This option actually uses Forms Services to verify the integrity of the form template being designed. In this case, the server validation found an unsupported expression that InfoPath verification didn't catch. The Design Checker will also list browser optimizations when it detects form templates that may induce performance lags. Unlike for the InfoPath client application, performance optimization for InfoPath Forms Services is critical when tens of thousands of users could be filling out your forms simultaneously.

The Design Checker also serves other purposes. When you're using a form template importer like Word or Excel, the Design Checker will show importer warnings when, for example, unsupported features are removed. Additionally, the Design Checker reveals incompatibilities in form templates that must be backward-compatible with InfoPath 2003.

Template Parts

Custom controls were introduced with InfoPath 2003 Service Pack 1 (SP1), but support was limited to ActiveX® controls. Since ActiveX controls have been around for over 10 years, there are plenty to choose from. But only controls safe for scripting and initialization can be used in InfoPath form templates. And even those that are safe may not work correctly in InfoPath. If you don't find an existing ActiveX control that fits your needs, you can always create your own. This is time consuming, though, and requires in-depth knowledge of ActiveX technologies.

Luckily, InfoPath 2007 introduces Template Parts, which let you build custom components without code. As the name implies, Template Parts allow you to reuse parts of one form template in other form templates without having to rebuild each one from scratch. Let's say you have an XML schema that includes customer information such as name, address, phone number, and so on. You want to build a set of forms all based on the customer section of the schema without having to recreate the controls and schema that pertain to the customer information in each form template. With InfoPath 2007 you don't have to recreate all those controls.

When you design a form template in InfoPath 2007, you now have the option to create a Template Part instead of a form template, as the new Design a Form dialog in Figure 1 shows. You can create a blank Template Part just as you'd create a blank form template, or you can base your Template Part on an XML schema that includes customer information.

Once you specify Template Part in the Design a Form dialog, InfoPath opens in design mode. From here, the process of creating your Template Part is almost exactly the same as creating a form template. The only difference, as when creating a browser-enabled form template, is that not all features are available. Those features not supported in Template Parts simply aren't available in the user interface in design mode. For example, you can't add custom code to Template Parts so the Programming menu item is not available on the Tools menu.

Once you've created a Template Part based on the XML schema that contains customer information, you can insert controls that are bound to the customer information in the schema. And just as you normally would when designing a form template, you can add data validation, rules, calculated default values, and data connections. When you're finished, you save it as you normally would. This creates a file with an .xtp file name extension. (The .xtp file is similar to the .xsn file created when you save a form template.)

Before you can use your Template Part in form templates, you must install it into the InfoPath Controls task pane in design mode using the Add or Remove Custom Controls wizard. After the Template Part is installed, it is available in your Controls task pane just like any of the built-in InfoPath controls, and you can insert it into as many form templates as you wish.

As with the built-in InfoPath controls, when inserting a Template Part into a form template, the schema structure that pertains to your new control will be created automatically in the form template (if you have selected the Automatically create data source option in the Controls task pane) or you can choose to bind your control to an existing schema structure. Any of the features you added to your Template Part earlier, such as data validation or rules, will be automatically added to your form template. Template Parts not only enable you to create your own custom controls without writing code, they also give you a way to share parts of a form template across multiple form templates, thus saving much time and effort.

Document Information Panel

One of the main benefits of using InfoPath to fill out forms is its ease of use. As a form designer, you can quickly create form templates that provide a rich user experience and are based on common XML technologies. Users get the familiar experience of using an Office application along with the straightforward form-filling experience provided by InfoPath.

With InfoPath 2007 you can take advantage of this form-filling experience in your own applications. You might want to convert an existing application to use InfoPath for filling out forms without having to retrain your users. Or maybe you want to create a form repository tool that enables users to fill out forms and quickly find existing forms within the same interface. In either case, you probably don't want to create a new form editing tool from scratch.

Let's look at how InfoPath is hosted in other Office applications. The 2007 Office system introduces a new XML-based file format that is used in the core Office applications—Word, Excel, and PowerPoint. Since InfoPath can be used to edit XML-based forms, it makes sense that InfoPath can also be used to edit some of the XML in this new Office file format.

Word, Excel, and PowerPoint 2007 all include a new Document Information Panel, which is used to edit document properties in much the same way as you'd use a properties dialog in Office 2003. However, since the document properties are stored as XML, it makes more sense to use InfoPath for editing these properties. Figure 5 shows the standard Document Information Panel in Word 2007. As you may be able to tell, this is simply an InfoPath form hosted inside a Word document. At first, you may think this is no different from using the old properties dialog. But not only do you get certain InfoPath features such as data validation, you also have the ability to customize the user experience for editing document properties. Since the Document Information Panel is hosting InfoPath, you can now create your own form templates to edit the document property data.

Figure 5 Word Document Information Panel and InfoPath Form
Figure 5 Word Document Information Panel and InfoPath Form

Hosting the InfoPath Form Control

Now let's talk about how you can put InfoPath to use in your own applications. Doing so can be quite simple. InfoPath 2007 packages its form-filling component as an ActiveX control you can host in your app. Support for hosting InfoPath in both managed and unmanaged code is provided so you can choose the technology you want to use to build your application. Obviously, hosting any ActiveX control in an unmanaged application is more complicated than in a managed app. However, if you already know how to implement an ActiveX control host application, doing so with the InfoPath ActiveX control is no different.

Let's take a brief look at how to host InfoPath in a managed Windows-based application using C#. (Any InfoPath-specific concepts we discuss can just as easily be applied to unmanaged applications and to other managed languages.)

After stepping through the New Project wizard in Visual Studio 2005 to create your Windows-based application in C#, you'll need to add the InfoPath form control to the Visual Studio toolbox before you can use it. To do so, click on Choose Toolbox Items from the Tools menu, click on the .NET Framework Components tab, and then locate the component named FormControl in the Microsoft.Office.InfoPath namespace. (In unmanaged code, the FormControl class is equivalent to the IInfoPathEditor interface.) Once the InfoPath form control is in your toolbox, you can add it to your Windows form just as you would with any other control.

If you build and run your application at this point, the InfoPath form control will be instantiated but it won't do anything useful. Most likely you'll want the InfoPath form control in your application to load a form. If you look at the FormControl object in the Object Browser in Visual Studio 2005, you'll see that the InfoPath form control provides a number of useful methods and properties. Probably the most important methods available are the ones that let you load a form into the InfoPath control—Open and NewFromFormTemplate.For the purposes of this simple example we'll use one of the two NewFromFormTemplate methods—the one that takes the URI of the InfoPath form template you want to load as a string parameter. For example, if you want to load a form template containing customer information, you could do so by adding the following code:

formControl1.NewFromFormTemplate(@"\\mysharelocation\customerInformation.xsn");

Alternatively, if you want to load an existing form, you could use the open method, like so:

formControl1.Open(@"\\mysharelocation\myForm.xml");

Note that the XML file you pass to the open method must be a valid InfoPath form.

The XmlForm property is equally useful. (In unmanaged code it is equivalent to the get_XDocument method on the IInfoPathEditor interface.) This property gives you access to the entire InfoPath object model for the form. Certain objects in the InfoPath object model, such as the Application and Window objects, aren't available when hosting the InfoPath control. These objects pertain to the InfoPath client application, so have no meaning when hosting the InfoPath form control.

The methods and properties available on the FormControl object give you a few ways to manage and interact with the hosted InfoPath form control. But this object obviously doesn't provide full control over the hosted form. For example, how do you make text bold or italic in the form? How do you insert a table into a RichText Box control? These and other features that are available in the InfoPath client application through the user interface aren't available through the FormControl object. However, you can access virtually all of these features from your own applications, using the IOleCommandTarget interface.

In unmanaged code, using IOleCommandTarget is straightforward. You simply obtain a pointer to the interface from the hosted control and then pass a command to the Exec method (Likewise, you can call the QueryStatus method to see if a command is currently supported.) In managed code, you must use COM interop to access the interface. To find a list of all the available commands simply look at the FormControlCommandIds.CommandIds static helper class in the Microsoft.Office.InfoPath namespace. (In the Object Browser in Visual Studio, the FormControlCommandIds.CommandIds is listed under the Microsoft.Office.InfoPath.FormControl namespace. There are literally hundreds of commands that enable you to control hosted forms from within your application.

With the support for hosting InfoPath in managed applications, you can create a simple hosting application in literally just a few minutes. You can also host the InfoPath Forms Services component. A form in the browser can be rendered by hosting the XmlFormView ASP.NET control in an .aspx page. (The XmlFormView class is defined by the Microsoft.Office.InfoPath.Server.dll assembly in the Controls namespace.) In fact, when you fill out a form in the Web browser that isn't hosted, you're really looking at a mostly empty .aspx page that hosts the XmlFormView control.

To use this control in your own ASP.NET page, your page must live on a SharePoint Server 2007 server that is running Forms Services. Then you can use Visual Studio 2005 to add the control to the Toolbox (as described earlier for the InfoPath Form Control) and insert it into the Visual Studio design surface of your ASP.NET Web site project. As you code against the XmlFormView control, you might notice that the entire InfoPath object model is available to the host page. This is a façade, though, because only a strict subset of the object model is actually allowed to be called by the hosted page. For example, the data source is accessible through the MainDataSource property, but accessing the form's Errors collection is restricted. An exception is thrown when calling any restricted properties and methods at run time.

Hosting a Web Form in the browser is another exciting prospect for your InfoPath forms. Not only can you reach out to users without InfoPath, you can also integrate the InfoPath form-filling experience into your own Web pages while maintaining the integrity of your Web site's appearance.

Importing and Exporting Forms

InfoPath 2007 provides more support for exporting data to other applications. InfoPath 2003 let you export forms to a Web page or to an Excel spreadsheet. InfoPath 2003 did allow you to export a form to an Excel spreadsheet but now there is more extensive support for exporting your data.

And InfoPath allows you to import forms as well. The framework for building form importers was available in InfoPath 2003 SP1, but only now does InfoPath ship with built-in importers. With InfoPath 2007, you can import forms from either Word or Excel. When you import a form, an InfoPath form template is created which you can then customize to meet your needs. You may be thinking that you can already transform a Word or Excel form into an InfoPath form by copying and pasting the form into InfoPath. However, importing has a huge advantage over copy and paste: the built-in importers will create controls and schema for you automatically. For example, suppose your Word form contains text, as shown in the following:

Enter your name: _________________________
In this case, users know they are supposed to enter their name in the underlined area of the document. InfoPath will detect this construct and create a Textbox control in the view. That Textbox control will be bound to a field element in the schema. The fact that the built-in InfoPath importers create controls and schema will certainly save you lots of time when you're migrating forms from Word and Excel to InfoPath.

InfoPath 2007 also provides tight integration with Outlook 2007. You can publish a form template in InfoPath 2007 to a list of e-mail recipients. This gives you a quick way to distribute a form template to accomplish simple tasks such as gathering survey data. When Outlook 2007 users receive the form, they can fill it out from within Outlook without having to open the InfoPath client application. Figure 6 shows an InfoPath e-mail form that will collect survey data.

Figure 6 InfoPath E-Mail Form
Figure 6 InfoPath E-Mail Form

Outlook 2007 supports creating InfoPath form folders to store submitted forms and includes rules for moving specific InfoPath forms to those folders. Using InfoPath form folders along with InfoPath property promotion, the data from the submitted forms are visible as columns in each form folder. Therefore, you can quickly see the data contained within the submitted forms without having to open each individual form. Since these folders are typical Outlook folders, you can sort and group information, which lets you make decisions in less time.

Managed Object Model

If you wrote managed code behind a form in InfoPath 2003 SP1, you're accustomed to hooking up events using attributed methods and the thisXDocument and thisApplication objects to access the bulk of the object model. In InfoPath 2007, the API is much improved. To hook up an event, simply use the .NET-style event assignment in the InternalStartup method of your form code class (this code and the associated event handler can also be auto-generated through the Tools | Programming menu flyout):

EventManager.FormEvents.Loading +=
    new LoadingEventHandler(FormEvents_Loading);

There have been some name changes: the old OnLoad event is now named Loading, and all XML events have been renamed to Changing, Validating, and Changed. Also, the InfoPath 2007 data source is exposed in code as an XPathNavigator instead of MSXML's XML document and node interfaces. This change provides a performance-oriented cursor-based approach to working with the underlying form data. (To get the main data source navigator, use this.MainDataSource.CreateNavigator.)

Many objects that were grouped in thisXDocument are now inherited members, and some even sport updated names (such as the MainDataSource member). A few others include CurrentView, DataConnections, DataSources, Errors, and SignedDataBlocks. One of the more popular methods to use in form code was to display a dialog using:

thisXDocument.UI.Alert("message")

Now you'd use the System.Windows.Forms namespace's MessageBox class. By calling the static method Show off of the MessageBox class, you can show the same dialog as the old Alert method. Better yet, Show also has overrides to use message box captions, buttons, and even a custom icon.

Best of all, the InfoPath object model abides by the design-once philosophy we talked about earlier: the code you write behind your form can be used on both the client and server.

Visual Studio Integration

To help rapid development of sophisticated form templates, InfoPath takes advantage of the Visual Studio 2005 environment. Since Visual Studio is the platform of choice for Windows-based development, it is a smooth transition to InfoPath.

For InfoPath 2003, Microsoft Script Editor (MSE) was the included development environment. But its usefulness was limited to writing JScript® and VBScript code. Without Visual Studio .NET 2003 and the associated InfoPath Toolkit, you could not easily write managed code behind an InfoPath form. For InfoPath 2007, Visual Studio Tools for Applications supplements MSE as the environment for writing your managed form code. It will be familiar if you've used Visual Studio and it includes many of the core features you need: IntelliSense®, an object browser, and a full-featured debugger. Simply run the project in Visual Studio Tools for Applications and you'll be debugging your form in InfoPath's preview mode within seconds.

To meld the development of the form code with the designing of the form template, you can use Visual Studio 2005 Tools for the 2007 Microsoft Office system, which takes the InfoPath design mode and integrates it directly into the Visual Studio IDE. The full lifecycle of designing a new (or existing) form, inserting controls, modifying data sources, and sprinkling features such as conditional formatting, data validation, and rules all take place in Visual Studio. Previewing and actually filling out the form, however, are left to InfoPath. Designing a form with Visual Studio Tools for Office is shown in Figure 7.

Figure 7 Designing an InfoPath Form with Visual Studio Tools for Office
Figure 7 Designing an InfoPath Form with Visual Studio Tools for Office

Besides the integrated design environment, there are other benefits to using Visual Studio Tools for Office. They include being able to use multiple simultaneously visible task panes, the Visual Studio Find and Replace functionality, and taking advantage of plug-ins such as Visual SourceSafe for source code control when collaborating in a team setting.

The topics we've discussed are just a few of the highlights of InfoPath 2007. We hope this brief introduction will spark your interest and will prompt you to explore InfoPath 2007 further. If you are new to InfoPath or would like more detailed information about designing forms, see the InfoPath Web site at microsoft.com/office/preview/programs/infopath. Tune in to the InfoPath Team Blog at blogs.msdn.com/infopath for info from the product team.

Scott Roberts is a Senior Development Lead in the InfoPath group at Microsoft. Scott is the coauthor of the upcoming book Designing Forms for Microsoft Office InfoPath and Forms Services 2007 (Addison-Wesley). Scott is also the author of Programming Microsoft Internet Explorer 5 (Microsoft Press,1999) as well as many technical articles and publications.

Hagen Green is a Software Design Engineer in Test on the InfoPath team at Microsoft. Hagen has worked on InfoPath features such as the designer platform, XSL support, Web services and DataSets, and most recently on InfoPath Forms Services. He is the coauthor of the upcoming book Designing Forms for Microsoft Office InfoPath and Forms Services 2007 (Addison-Wesley). Scott and Hagen can be reached at DesigningIPForms@hotmail.com.


Use The Source, Luke: Source Server Helps You Kill Bugs Dead In Visual Studio 2005

Use The Source, Luke

Source Server Helps You Kill Bugs Dead In Visual Studio 2005

John Robbins
This article discusses:
  • Source Server
  • Symbol Servers
  • WinDBG and Visual Studio debugging
This article uses the following technologies:
Visual Studio 2005, Visual SourceSafe
Code download available at: Bugslayer2006_08.exe (181KB)
Contents
Working with Source Server
What You'll Need
A Simple Scenario
Getting Started with Indexing
Working with Subprojects
Debugging with WinDBG
Debugging in Visual Studio 2005
Better and Easier Indexing
Wrap Up

W
hen you look at a minidump or live debugging session while trying to rout out an elusive bug, you need the right contributors to those bits. Access to the correct symbols and source code is critical, whether you're doing Microsoft® .NET or native C++ development. Without them, you'll be debugging at the assembly language level. Sure, if you're paid by the hour, this can be a fantastic way to earn yourself a new car! But there are better ways to spend your time.

In past columns (msdn.microsoft.com/msdnmag/issues/02/06/Bugslayer), I've discussed what it takes to get the symbols you need into the debugger. Obviously, you'll need a symbol server. I can't stress enough how important this is. If you don't already have one set up in your development shop, get one now.

Once you have access to the right symbols set up, you need to get the right source. If your team has been vigilant about version control, you'll know exactly which version of the source code is in a particular binary. But wouldn't it be fantastic if the debugger would automatically grab the exact source code as it's loading the exact PDB file? Then you wouldn't have to guess and debugging would be much simpler.

In this article, I'll show how you can always get your source code with Source Server. Because the latest releases of WinDBG and Visual Studio® know exactly how to use Source Server, its benefits are available to both .NET and native C++ developers.

Working with Source Server

A symbol server is a database that uses the file system. When you load FOO.PDB, the debugger builds a directory path that starts with your cache directory, appends the file to open, and finishes with the GUID uniquely identifying the version of the file. In that directory path is the actual PDB file the debugger needs (see Figure 1).

Figure 1 Building the Path
Figure 1 Building the Path

The source server doesn't store source code as the symbol server does with PDBs. A PDB file created with any of the recent compilers has in it the full path for each of the source files (compilands) that were used to create the binary. When you run Source Server on your binaries, it adds a section to the PDB file. Here it embeds the exact version of the file reported by the version control system and the actual commands used to retrieve the file from the version control system. This process is referred to as indexing the source code.

Once you've indexed the source code, and assuming you have access to the version control system, the debuggers will ask the version control system for the correct version of the file. I'll show you how you can see all of this information in the PDB files a bit later in this article.

Source Server must be told how to use your version control system. With the latest source server binaries, you get support for Perforce, Subversion, Concurrent Versions System (CVS), and Microsoft Visual SourceSafe®. Sadly, there is no support for the new Visual Studio Team Foundation System version control system out of the box. If you'd like to add support for Visual Studio Team Foundation System or a different version control product, the source server tools are extensible. As long as the version control system you use supports a command-line tool for getting version information and doing pulls, you can write a module to operate with your system. The SRCSRV.DOC file in the Source Server installation directory explains how you can create and integrate your own module.

The Source Server workflow is straightforward. First, make the debug and release builds of your app. After you've run your automated smoke tests to ensure you have a good build, use the Source Server programs to index the PDB files with version control information. The final step is to store those indexed PDB files in your symbol server. Figure 2 illustrates the process.

Figure 2 Source Server Workflow
Figure 2 Source Server Workflow

What You'll Need

Let's take a look at what you'll need to get your system up and running. As an example, I'll tell you about the configuration I used at the time of writing this article.

I used Visual SourceSafe for version control as it is the most common version control system for Windows® development, but the concepts I discuss here apply to most common version control solutions. For debugging I used WinDBG 6.6.6.0 for both x86 and x64. Versions back to 6.4.7.2 will work with Source Server.

Only Visual Studio 2005 Professional and Team Editions know about Source Server. If you're a native C++ developer using Visual Studio .NET 2003 or Visual C++® 6.0, you can use Visual Studio 2005 to debug your binaries. (But be aware that once you see the symbol and source server support in Visual Studio 2005, and a slew of other enhancements, you'll be thinking seriously about upgrading permanently.)

The Source Server binaries, which do the actual indexing, are part of the Debugging Tools for Windows package, which you can download for free from the Microsoft Web site (microsoft.com/whdc/devtools/debugging/default.mspx). If you will be running on a 64-bit system, you can use the 64-bit versions, but the 32-bit versions also work fine on x64. Visual Studio 2005 comes with a version of SRCSRV.DLL older than the one with the 6.6.6.0 version of the Debugging Tools for Windows, which fixes some bugs in the Visual SourceSafe usage. You will want to copy the x86 version of SRCSRV.DLL from the Debugging Tools for Windows directory to Visual Studio .NET Installation Directory\Common7\IDE and overwrite the old version.

The necessary Source Server binaries for doing the source indexing are not installed by default. You'll need to custom-install the SDK portion of the Debugging Tools for Windows. This will add an SDK directory under the Debugging Tools for Windows directory. Inside this directory, you'll find a SRCSRV directory, which contains all the goodies. You can leave it and point to it or copy it somewhere else, but the directory with the tools has to be in your path.

After installing the Debugging Tools for Windows, you'll need to install Perl. I use Perl 5.8.7 from www.cpan.org/src/5.0. I built my binaries from the sources with Visual Studio .NET 2003, as the source code does not yet build with the Visual Studio 2005 compiler.

Notice in the Source Server directory that most of the heavy lifting is done in Perl. The SSINDEX.CMD file is just a batch file that calls PERL.EXE and passes the Perl code in through standard input. The .PM files are the Perl modules that can talk to the version control systems; the SSINDEX.CMD loads these modules and calls through a standard interface to isolate the system differences. The other CMD files, such as VSSINDEX.CMD, are wrappers for working with specific version control systems, as SSINDEX.CMD needs the version control system on the command line. The PDBSTR.EXE and SRCTOOL.EXE files in the SRCSRV directory read and write to PDB files. The VSSDUMP.EXE program is a helper for the Visual SourceSafe integration; it's only installed with the x86 version of the Debugging Tools for Windows package.

When you run SSINDEX.CMD, after it validates parameters it asks the version control system for the list of all source files (and their version numbers) in the target project and below. PDB files are then searched for recursively. For each PDB file found, the Perl code calls SRCTOOL.EXE to extract the list of source files. The code looks through the source files in the PDB file and sees if they match any of the files stored earlier from the version control search: if a match is found, the code saves that file info. Once the code has looked at all the source files in the PDB file, it's time to call PDBSTR.EXE to write the index stream, called SRCSRV, to the PDB file. When debugging, the debugger looks for this stream. If it is found, the debugger knows there's a source server involved and calls into SRCSRV.DLL to execute the version control system to ensure the right file is accessed.

That's an overview of indexing source code. For more on working with SSINDEX.CMD and its friends, look at the code or SRCSRV.DOC. And remember, if your version control system is not supported, you'll need to write the Perl module that conforms to the interface specified in SRCSRV.DOC.

A Simple Scenario

Now let's put Source Server to work. I'll discuss a simple source tree that comprises two console applications, CppApp and NetApp. CppApp is a native C++ application and NetApp is written in C#. The projects reside in C:\Code\CppApp and C:\Code\NetApp. All output is built to C:\Code\Debug. The Visual SourceSafe version control is on the share \\Critias\BugslayerSource, with CppApp and NetApp as projects of $/ (which is the root in Visual SourceSafe). The $/ project is set to using C:\Code as its working directory, making the $/CppApp and $/NetApp working directories C:\Code\CppApp and C:\Code\NetApp, respectively. Note that to index source files with Visual SourceSafe you are required to set the working directories on the project. I've also applied the version control label "First Version" to the project. Finally, I built both projects. Now they're ready for indexing.

First I need to ensure that the Visual SourceSafe program, SS.EXE, Perl, and the Source Server programs are all in the path for my command shell. If the version control program is not in the path, the source file fetch will silently fail and you won't know why you're looking at the Disassembly window.

Be aware that while the Source Server help says you can specify the server to use on the command line, that doesn't actually work with Visual SourceSafe. You need to set the SSDIR environment variable to the Visual SourceSafe database. In my example I issue the following in the command prompt:

set SSDIR=\\Critias\BugslayerSource\

Another problem with Visual SourceSafe is the concept of current project, which is the project selected when you end the SourceSafe Explorer GUI. The source indexing for Visual SourceSafe "respects" the current project, which means it assumes that the current project is the one you want to work on. If you've got a single project in your version control system, that's fine, but no one actually does. In order to get the Visual SourceSafe indexing to work, you'll need to set the Visual SourceSafe current project to the project you are about to index. For indexing at the command line, you can set it with the SS.EXE cp option. In my example, I'd run the following command because I'm working with the root project:

ss cp $/

Next, in C:\Code, I create a new SRCSRV.INI for indexing by defining a logical name for the server and the server path. The example SRCSRV.INI in the Debugging Tools for Windows\SDK\SRCSRV directory has more info on this. In the case of my Visual SourceSafe database, the SRCSRV.INI looks like this:

[variables]
BSSRC=\\critias\BugslayerSource\
(With Visual SourceSafe, the value to the right of the equal sign must exactly match the value in the SSDIR environment variable—even the trailing backslash.)

In almost all instances, you'll only need the SRCSRV.INI file during indexing. It will store the BSSRC logical name and location for the server in the processed PDB files. If the machine CRITIAS dies and you need to move the version control, you can put a SRCSRV.INI in the same directory as the debugger's SRCSRV.DLL and the settings in that file will override the indexing embedded in the PDB files. Thus, you wouldn't need to re-index the PDB files.

You can also use the SRCSRV.INI file during debugging to point the debugger to a mirrored version control system that provides read-only access. It's even acceptable to include all your company's version control systems in the SRCSRV.INI under the [variables] section.

SRCSRV.INI can reside in several places. If you've copied the source server code directory to its own spot on the machine, you can put your version there and the indexing tools will pick it up automatically. Or you can put it in the current directory where you execute the indexing commands. You can also tell the tools where to find SRCSRV.INI with a command-line option or set the SRCSRV_INI environment variable to point to it. In my example, I put SRCSRV.INI in the root directory for my source code (C:\Code) and run all my indexing commands from there.

Getting Started with Indexing

To begin indexing, many of you will set command-line options to tell the tools where to find sources, symbols, and the like. Since I'm using Visual SourceSafe, I run VSSINDEX.CMD to do my indexing. I could also use SSINDEX.CMD /System=VSS. There are two help options for the command-line options, -? and -??, where more question marks show more help. This help provides all sorts of switches and environment variables.

There are four command line options of real importance for all version control systems and a fifth necessary for Visual SourceSafe. The first is /Ini, which specifies the SRCSRV.INI file to use. As I mentioned, the file will be automatically found if it's placed in certain locations. But if you want to be specific, use /Ini to set the location manually.

The /Source switch tells the indexing tools where the source code root starts. The default is the current directory. The indexing script will attempt to correlate any source file found in the directory specified and below with files in the version control system.

The /Symbols switch indicates the root directory that will be recursed when looking for PDB files. If a PDB file contains one or more source code files from the version control system, then the PDB file is indexed with the SRCSRV stream, which I'll show later.

For Visual SourceSafe only, the /Label command is required. Because of limitations in Visual SourceSafe, the source indexing cannot figure out the version of a source file in a given directory. Thus, you need to first set a label on the root of the project with Visual SourceSafe and pass that same name to the source indexing tools. As you should always be labeling your builds for good version control hygiene, that should not be too onerous.

The last command-line option, /debug, is the most important. There is a major drawback of the source indexing tools: if any problems are encountered and no PDB files are indexed, there is no notification that anything went wrong. Everything looks to be in tiptop shape until you start debugging and nothing works. This is where /debug comes in. Always run the indexing commands with /debug so you can see what worked and what didn't.

Now let's do some indexing. I have a command shell open and my current directory is C:\Code. Since that's the root of both my source code and symbols, I don't have to set those command-line options. My SRCSRV.INI file is also in that directory, so there's no need to set that command-line value either. I've applied the label "First Version" in SourceSafe already. Finally, I've set the SSDIR environment variable to specify the server. Executing VSSINDEX /Label="First Version" /debug produces the output shown in Figure 3.

Figure 3 Initial Source Indexing Output (Close Figure 3)
[C:\Code]vssindex /Label="First Version" /debug
--------------------------------------------------------------------------------

SSIndex.cmd [STATUS] : Server ini file: C:\Code\srcsrv.ini
SSIndex.cmd [STATUS] : Source root    : C:\Code
SSIndex.cmd [STATUS] : Symbols root   : C:\Code
SSIndex.cmd [STATUS] : Control system : VSS
SSIndex.cmd [STATUS] : VSS Server     : \\Critias\BugslayerSource
SSIndex.cmd [STATUS] : VSS Client Root: C:\Code
SSIndex.cmd [STATUS] : VSS Project    : $/
SSIndex.cmd [STATUS] : VSS Label      : First Version
--------------------------------------------------------------------------------

SSIndex.cmd [STATUS] : Running... this will take some time...
SSIndex.cmd [STATUS] : Processing vssdump.exe output ...
SSIndex.cmd [INFO  ] : ... indexing C:\Code\CppApp\Debug\vc80.pdb
SSIndex.cmd [INFO  ] : ... zero source files found ...
SSIndex.cmd [INFO  ] : ... indexing C:\Code\debug\CppApp.pdb
SSIndex.cmd [INFO  ] : ... wrote C:\DOCUME~1\John\LOCALS~1\Temp
\indexAEBF.stream to C:\Code\debug\CppApp.pdb ...
SSIndex.cmd [INFO  ] : ... indexing C:\Code\debug\NetApp.pdb
SSIndex.cmd [INFO  ] : ... wrote C:\DOCUME~1\John\LOCALS~1\Temp
\index14DA6.stream to C:\Code\debug\NetApp.pdb ...
SSIndex.cmd [INFO  ] : ... indexing C:\Code\NetApp\obj\Debug\NetApp.pdb
SSIndex.cmd [INFO  ] : ... wrote C:\DOCUME~1\John\LOCALS~1\Temp
\indexD8BE.stream to C:\Code\NetApp\obj\Debug\NetApp.pdb ...

As you can see, VSSINDEX.CMD found some PDB files and indexed them. Technically, this all worked. But if you scan down to the line containing vc80.pdb, you'll see that the next line says zero source files found. That's normal for any VC?0.PDB files since those are disposable files. If, however, one of your necessary PDB files ever reports zero source files found, that's a serious problem. However, the indexing tools don't stop on indexing errors. That means you must specify the /debug option and carefully check that your PDB files are all properly indexed.

The output in Figure 3 shows another problem that can slow down indexing performance. I left the /Symbols switch set to the default, which treats the current directory as the root for all PDB files. The last PDB file indexed with the file C:\Code\NetApp\obj\Debug\NetApp.pdb, which is where .NET applications are built and then copied to the output directory you specify. If you index these files, the indexing time will double.

It's best to build applications to a central directory or directory tree for easier maintenance. In my example, everything builds to the C:\Code\Debug directory. Therefore, when I use the command line VSSINDEX /Label= "First Version" /symbols=.\Debug /debug, VSSINDEX only indexes the PDB files I will put in my symbol server.

Note that if all your source directories are directly off the drive root directory, source indexing will not work. The first time I used the Source Server tools, my setup looked like this and I was unable to index any of my PDB files. In my current example, I've purposely put my source code in the C:\Code directory to work around this problem. If all your source directories are off the root, you'll need to change your build system to put the source code at least one level down.

Working with Subprojects

The next surprise will be if the project you want to index is a subproject in your version control system. For example, imagine that my sample has one root project, $/Happy, with two subprojects, $/Happy/Foo and $Happy/Bar. Under these are all the subprojects necessary for those pieces of code. Now imagine that I am only responsible for source indexing $/Happy/Foo and the working directory is D:\Dev\Happy\Foo. If I run the indexing command vssindex /debug /Symbols=.\Debug, I'll find that zero source code files were indexed.

Why? I made a bad assumption—that the source indexing commands consider the current directory and match the version control projects appropriately—but they don't. The default project for source indexing is the root of the version control system. (Remember that all my testing is with Visual SourceSafe. For other version control systems, you'll need to do some testing.)

I need to use the command-line vssindex /debug /Symbols=.\Debug /Project=Happy/Foo. The Visual SourceSafe source indexing code puts in the leading $/ necessary to SS.EXE. It's a good idea to always specify the /Project command line option, even if it is the root. (Again, you need to manually test various scenarios to see what works in your environment.)

Seeing that a .STREAM file was written to the PDB file is a good sign. To verify the stream, I use PDBSTR.EXE to look inside it. PDBSTR.EXE is a little persnickety, so I have to get the command-line options just right so that the output looks like no stream is in the PDB. The first parameter must be –r (read the PDB). The second parameter is –p:file.PDB (the PDB to view). And the last parameter is the stream that I want to see: this is always –s:srcsrv. Figure 4 shows the output that I get from running PDBSTR.EXE on CPPAPP.PDB.

Figure 4 PDBSTR.EXE Output (Close Figure 4)
[C:\Code\debug]pdbstr -r -p:CppApp.pdb -s:srcsrv
SRCSRV: ini ------------------------------------------------
VERSION=1
INDEXVERSION=2
VERCTRL=Visual Source Safe
DATETIME=Fri May 19 00:29:22 2006
SRCSRV: variables ------------------------------------------
SSDIR=\\Critias\BugslayerSource
SRCSRVENV=SSDIR=%BSSRC%
VSSTRGDIR=%targ%\%var2%\%fnbksl%(%var3%)\%var4%
VSS_EXTRACT_CMD=ss.exe get -GL"%vsstrgdir%" -GF- -I-Y -W "$/%var3%" -V"%var4%"
VSS_EXTRACT_TARGET=%targ%\%var2%\%fnbksl%(%var3%)\%var4%\%fnfile%(%var1%)
BSSRC=\\Critias\BugslayerSource
SRCSRVTRG=%VSS_extract_target%
SRCSRVCMD=%VSS_extract_cmd%
SRCSRV: source files ---------------------------------------
c:\code\cppapp\cppapp.cpp*BSSRC*CppApp/CppApp.cpp*First Version
c:\code\cppapp\stdafx.h*BSSRC*CppApp/stdafx.h*First Version
c:\code\cppapp\stdafx.cpp*BSSRC*CppApp/stdafx.cpp*First Version
SRCSRV: end ------------------------------------------------

The ini section specifies the source-indexing version, the version control system used, and the time at which the indexing was performed. The debuggers look for the SRCSRVCMD value in the variables section and treat values surrounded by percent signs as variables to build up. In Figure 4, VSS_EXTRACT_CMD contains the command line that will be built up to call the version control system with the appropriate file and directory to extract to. The source files section shows the values used to fill the variables. Each line contains the data for a single source file, delineated by asterisks. The first value is the path to the source file as it came from the PDB. The second value, in this case BSSRC, indicates the version control database to use. The third value is the version control project, and the last is the file version in version control. SRCSRV.DOC illustrates all this.

At this point, I'm ready to start debugging, but given all the fiddly steps necessary to get source indexing working, I wanted to give the checklist in Figure 5 to follow to achieve Source Server nirvana. I hope that this will save you some time getting started.

Figure 5 Source Server Indexing Checklist (Close Figure 5)

Is your source tree in a directory below root?
Have you built both debug and release builds?
Have you built all binaries and PDBs to a logical directory structure?
Did you add your version control logical name(s) to SRCSRV.INI?
Is SRCSRV.INI in the tools directory, the SRCSRV_INI environment variable, the current directory, or specified with the /Ini switch?
Is your version control system command-line tool in the path?
For Visual SourceSafe, have you set the SSDIR environment variable?
For Visual SourceSafe, is the working directory set for each project?
For Visual SourceSafe, have you applied a label to the project?
Is Perl 5.0 or greater installed and in the path?
Are the Source Server tools and scripts in the path?

Debugging with WinDBG

Now the moment you've been waiting for! Let's look at how you can use the source server in the debuggers. Turning on source server support in WinDBG, NTSD.EXE, or CDB.EXE is similar to turning on symbol server support. To turn it on globally for all WinDBG sessions, start WinDBG with no debuggee open and then select Source File Path from the File menu to get to the Source Search Path dialog. Alternatively, you can set the _NT_SOURCE_PATH environment variable. In both cases, the value should be SRV* Cache Dir;.

SRV tells the debugger that you're expecting to use a source server and the bit after the asterisk specifies the directory where you want the local cache for all the files. I strongly recommend setting a download cache. If you don't, the sources will be pulled to a Src directory in the Debugging Tools for Windows installation directory. And since the default directory is in Program Files, the pull will abort if you're correctly running in a Least User Access environment. In addition to setting source paths globally, you can always set the path during a debugging session with the .srcpath command.

Once you start debugging in WinDBG and do any operation where you need source code, the dialog shown in Figure 6 will pop up. If you select "Perform this action each time from now on" that will write the decision to the Default value in HKCU\Software\Microsoft\Source Server\Warning so you never see the security dialog again.

Figure 6 Turning Off the Security Alert
Figure 6 Turning Off the Security Alert

You'll probably want to turn off the security warning and have WinDBG jump directly to executing the version control command to get the source. By using the source server, that's the first place the debugger will look for source, no matter what. If you have a fast version control system, that won't be a problem, but not everyone has a fast system.

An alternative way of disabling the security dialog is to put a copy of SRCSRV.INI in the Debugging Tools for Windows program directory. That's where SRCSRV.DLL resides, and it knows to look for the file there. If SRCSRV.INI has the version control system in the [trusted commands] section, it will skip the security dialog. For each trusted command, you specify the base name of your version control system command-line program on the left. On the right, you specify the full path to the version control system command-line program. In my scenario, the [trusted commands] section looks like this:

[trusted commands]
ss.exe=C:\VSNET8\Microsoft Visual SourceSafe\ss.exe

I used a source path of SRV*C:\Code\Source. In Figure 6, you can see the entire path to which version control will extract the file: C:\Code\Source\BSSRC\CppApp\CppApp.cpp\First Version. The C:\Code\Source\ portion is obvious, but the rest of the string identifies the particular file. The BSSRC directory matches the tag value I specified in the SRCSRV.INI file I used to index the PDB files. The CppApp directory identifies the version control project and CppApp.cpp identifies the source file. In this example, I indexed the source by specifying the /Label= "First Version" command-line option. For version control systems other than Visual SourceSafe, if you don't specify the label, you would see the raw file version number. Like a symbol server, Source Server uses the file system as a database in order to uniquely identify the version of the file I need. If the source server code in SRCSRV.DLL sees that the file already exists in the extract directory, it will load the source file from there without querying version control.

I like to set the Source Server cache directory to a directory under my symbol server cache directory. If my symbol server directory is C:\SYMBOLS, my Source Server cache directory is C:\SYMBOLS\SOURCE. In this fashion, all my caches are in the same place, and to reclaim disk space I can simply wipe out everything and start over.

Once I click the Yes button in the Microsoft Source Server Security Alert dialog, provided everything is OK, WinDBG pops up with the source code. If there's a problem, use the .srcnoisy command to have WinDBG report the results of running the version control command-line tools.

Debugging in Visual Studio 2005

Setting up Source Server debugging in Visual Studio 2005 is nearly as simple as it is in WinDBG. Instead of using an environment variable, the magic is in the Options dialog. Once you get to the Options dialog, navigate to Debugging | General (see Figure 7). Check the "Enable source server support" and "Print source server diagnostic messages to the Output window" options. Then click OK and you'll be notified that using Source Server is a potential security problem. Click Yes and you're on your way.

When you start debugging with Visual Studio, a security dialog like the one shown in Figure 8 is displayed. This lets you choose whether to execute the program. Unfortunately, it offers no option to say you trust the symbol server and don't want to see this dialog again, so you'll see it for every source file. There is the option, however, to place a SRCSRV.INI file that defines your [trusted commands] section with your version control system in the Visual Studio installation directory\Common7\IDE directory and you'll never see this security dialog again.

Figure 7 Turning on Source Server in Visual Studio
Figure 7 Turning on Source Server in Visual Studio
Figure 8 Debug Security Check
Figure 8 Debug Security Check

You may be wondering where Visual Studio puts the source server download cache. The default location is C:\Documents and Settings\Username\Local Settings\Application Data\SourceServer. This is great because with proper security in place only logged-in users can see that directory. Though my login account is never an admin account, I do want to share the source server cache directory with WinDBG, so I share the source between the debuggers. While I can change the WinDBG source path to SRV* C:\Documents and Settings\Username\Local Settings\Application Data\SourceServer, I'd like to keep the source files under my symbol server directory.

This problem began my quest to figure out where Visual Studio sets the cache directory. Looking at the registry, I eventually found the SourceServerExtractToDirectory under HKCU\Software\Microsoft\VisualStudio\8.0\Debugger—this is where the cache directory setting resides.

Since I don't relish changing Registry keys manually, I wrote a JavaScript program, VsSourceServerCache.JS (which is part of the source code download for this article). If I pass no command-line options, the program simply shows me the current settings. If I pass a valid directory on the command line, the program changes the Source Server cache directory.

Fortunately, the Import and Export Settings Wizard in Visual Studio will properly export and import the settings between machines. Note, if you import the settings to a new machine without the cache directory, the Source Server file pull will silently fail. Always make sure the cache directory exists.

If you are having trouble using Source Server with Visual Studio, make sure to look at the Output window. There you'll see the reports of any problems with Source Server, and it's the first place you should check when you encounter any odd behavior. In addition, you can verify where the source file is located by moving the mouse cursor over the source file tab in Visual Studio; the tool tip will display the full path to the open file. Finally, like WinDBG, Visual Studio will grab the source code through the Source Server before it will search the raw paths in the PDB file.

My last tip about debugging with Visual Studio 2005 is that WinDBG generally has two public releases a year, while Visual Studio has fewer. As I mentioned in the beginning of the article, the 6.6.6.0 version of WinDBG comes with a newer version of SRCSRV.DLL than the one with Visual Studio. I always copy the latest x86 version of WinDBG to the Visual Studio installation directory\Common7\IDE directory so I'm always using the latest version. And you'll need the 32-bit version even if you're using 64-bit Windows since Visual Studio 2005 is a 32-bit x86 program.

Better and Easier Indexing

I've covered all the work you'll need to do to use Source Server. But if you're like me, you quickly realized that while you include a CMD file in your build, you've got better things to do than look through the output of source indexing. Once you hit about, say, five binaries in your project, your eyes will glaze over and you'll miss important stuff. It would be great if there was an automated way to search for possible errors. Not to mention, the source indexing portion should be part of your build, so that's screaming to be yet another MSBUILD task. In my most recent column (msdn.microsoft.com/msdnmag/issues/06/03/bugslayer), I presented some MSBUILD tasks for automating unit testing and code coverage. I want to build on that code with tasks to handle source indexing.

My requirements are simple. If any problem is encountered when indexing source files, the process should generate an error so MSBUILD stops. These problems can manifest in a few ways: any kind of [ERROR] output from SSINDEX.CMD, any PDB file indexing that reports zero source files found, and if there are no source files indexed at all (an insidious problem, for sure).

My plan was to build a usable base task, SourceIndexTask, from which all version control product-specific tasks could inherit. I wanted to do the VssSourceIndexTask too. If you do write a task for a Source-Server-supported version control system, please send it to me and I'll include it in Bugslayer.Build.Tasks.DLL and post the code for all MSDN®Magazine readers.

To keep things simple, I'll assume you have the Source Indexing tools in the path. The base SourceIndexTask will find the tools in the path as MSBUILD expects the full path in order to run. My second assumption is that your directory setup matches your version control project names. While the product-specific indexing scripts let you set project and client parameters, it's a better idea to have the directory and project names align so you can run the indexing script at the top directory. If there are mismatches, you'll have to run the source indexing separately on each directory.

SourceIndexTask has three properties that you must set in your PROJ files: symbols, sources, and VersionControlSystem. Symbols defines the directory where your PDB files are located (this can be a relative path from your PROJ file). I made sources a required property because the indexing commands work better if the task switch to the sources directory is known before execution. The VersionControlSystem parameter is not marked with RequiredAttribute because I didn't want derived classes, such as VssSourceIndexTask, to have to specify the version control system redundantly. However, if VersionControlSystem is not defined, the task will error out as though it were required.

The first of the optional properties for SourceIndexTask is IniFile. This lets you explicitly specify the SRCSRV.INI file. The AdditionalCommandLineOptions property lets you add any commands not covered by derived types. This is also useful if you're using a version control system that does not have its own task class and you want to start indexing immediately. You can add the version control-specific files to the AdditionalCommandLineOptions and be up and running in no time. Obviously, there's no error checking on the values inside AdditionalCommandLineOptions. The last of the optional properties is IgnorePdbFiles. This ITaskItem array is where you can put PDB files when you want to have the tasks ignore the dreaded "zero source files found" error message. This can happen on autogenerated binaries and items like the VC?0.PDB type files I described earlier. The SourceIndexTask already knows to ignore VC?0.PDB, but this property gives you the place to add any additional PDBs you don't want error checking performed on.

The VssSourceIndexTask, which is the task that handles Visual SourceSafe, defines two required parameters, Server and Label. Server holds the version control system. The task sets the SSDIR environment variable before executing the indexing commands to work around the bug in the Visual SourceSafe processing.

The Label property sets the indexing commands to use the specific Visual SourceSafe label. If you are indexing source code from a project that's below the top level in Visual SourceSafe, the indexing will not work. If you do not set the Project property, the VssSourceIndexTask will take the Source property, expand it to the full path, and convert that path to the Visual SourceSafe project. If your Visual SourceSafe projects map to your file directories, which you should always do anyway, this will automatically get the /Project option set with no effort on your part. If you do set the Project property, that value is sacred and not changed by the task, so VssSourceIndexTask passes it directly to the source indexing tools.

One issue with the Project property is if you are working with projects directly off $/ (the Visual SourceSafe root) you will need to manually set the Project property to / in your MSBUILD project. I've racked my brain on how best to handle subproject indexing and this is the best I've come up with. If you have a better idea, please feel free to contact me!

As I mentioned earlier, the source indexing tools for Visual SourceSafe default to the current project, which will cause source indexing to fail when you run on a different project. To work around this problem, the VssSourceIndexTask calls SS.EXE with the cp command line to set the current project to the Project value. Now you don't even have to think about the current project, and you'll have much better luck indexing source code!

To see all the magic in action, look at the MSBUILD project, Index.Proj, in the root directory of the source code. This project indexes the source code from this article.

From an implementation standpoint, there's not much excitement going on. The most interesting challenge was figuring out how I could look for errors in the source indexing output. I originally thought I'd redirect the output to a file and use my ninja regular expression skills to parse up all the problems. However, as I was deriving the SourceIndexTask from Microsoft.Build.Utilities.ToolTask, which is the killer class for command-line tools, I observed that the class sets up a standard I/O reader. Whenever a command-line tool writes a line of output, the ToolTask calls the overridable LogEventsFromTextOutput method. This meant that all I needed to do was set up a tiny state machine to look for errors, warnings, the PDB file being processed, and the processing result. The big work was going on to find all the issues with the source indexing tools so I could account for them.

Wrap Up

I'm sure that, after having read this, you understand the importance of Source Server. Along with the symbol server, you're armed to tackle any debugging challenge with the best information possible. There are a few rough edges to work around, but you'll notice improved productivity immediately. It won't take long before you wonder how you ever debugged without it!

John Robbins is a cofounder of Wintellect, a software consulting, education, and development firm that specializes in the .NET and Windows platforms. His latest book is Debugging Applications for Microsoft .NET and Microsoft Windows (Microsoft Press, 2003). You can contact John at www.wintellect.com.


Editor's Note: The Office-Driven Life

Editor's Note
The Office-Driven Life
Joshua Trupin

Spring is quickly turning into summer up here in the Northern Hemisphere. And like the explosion of new summer growth all around us now, we are seeing all sorts of new software blooming. For instance, this is the first MSDN Magazine Editor's Note written with 2007 Microsoft Office system Beta 2 on Windows Vista Beta 2. The research is being conducted with Internet Explorer 7.0 Beta 2—and on a Tablet PC, no less. We even had the Treo 700 fired up there for a while (and running Windows Mobile 5.0—alas, released software, not beta), but the wonderful program (PDANet) that lets it double as a laptop modem doesn't yet have Vista drivers.

If you give the magazine's cover a scan before opening it up each month, you'll see that this is our first deep dive into the 2007 Office system. Beta 2 has been released, and you don't even need to be an MSDN or TechNet member to download it. There has been a lot of coverage of the new UI in the 2007 release. While usability is a key element of this release, the real excitement (for us) is what's under the covers.

The 2007 Office release is expanding the limits of what applications can be created in conjunction with its familiar apps. There are a few key areas where the Office package has become more of a development platform.

The default document file format for Office uses the Open XML standard. If you are developing an app that reads or writes an Office file, you won't have to take stabs at a binary file format. This makes your life a lot easier because the .NET Framework has all sorts of XML readers and writers—you just need to know which tags and elements you want.

Although Office is thought of as a desktop-only suite, the upcoming version will extend to the server in several places (SharePoint, Project, and lightweight Excel Services, for instance) in such a way as to improve collaboration.

The RibbonX API is your way into the new Ribbons that are the default UI experience in the 2007 Office system. Menus are on their way out, replaced with context-sensitive choices that change as you work. You'll be hearing a lot about the Ribbon in the next few months. A LOT. Everyone sitting nearby is hearing a lot about it from us, and we're only in Beta 2.

In this month's issue, we will take a broad look at Office as a development platform. From there, we will drill down into three additional areas of interest. Ted Pattison looks at MOSS, Microsoft Office SharePoint Services 2007. We also look at InfoPath 2007, the forms engine that will give programmers the ability to improve the user experience of their Office-based apps.

We also take an introductory tour of VSTA, which is not the OS we are beta testing, but Visual Studio Tools for Applications. VSTA lets you add extensibility into your own applications. Don't confuse it with Vista; as our high-school math club coach once told us, "There's no I in VSTA."

Behind the scenes, we also have some good news to report. Kerry Gates, who has been our publisher for years now, is moving on at CMP Media. She has taken on a new role as Vice President at the company! At the same time we welcome our new publisher, Jill Thiry. Jill has had many roles in the world of tech media, including a stint as publisher of Network Computing magazine. Good luck, Kerry, and welcome to the family, Jill!

Finally, we were hoping to have coverage of a special event this month. Stephen Toub is the technical editor responsible for the content you see in MSDN Magazine each issue. After putting together this blockbuster August issue, he sped off to get married to Tamara, his college sweetheart. We were hoping we'd have photos back in time to print them this month, but the wedding WAS just yesterday and Steve might be thinking about other things besides his coworkers back home. We hope to bring you more hard-hitting coverage of this event in a future issue. Meanwhile, you can send Steve your best wishes at wedding@microsoft.com.

—J.T.

Thanks to Thanks to the following Microsoft technical experts for their help with this issue: Simon Calvert, Nikola Dudar, Joe Duffy, John Gossman, Martin Gudgin, Jim Johnson, Doug Purdy, Andy Rich, Arpan Shah, Pat Styles, and Maura Van Der Linden.

Joshua Trupin

Active Directory, ActiveX, Excel, InfoPath, IntelliSense, Internet Explorer, JScript, Microsoft, Microsoft Press, MSDN, Outlook, PowerPoint, SharePoint, Visual Basic, Visual C#, Visual C++, Visual SourceSafe, Visual Studio, Windows, Win32, WinFX and Windows Server are either registered trademarks or trademarks of Microsoft Corporation. Other trademarks or tradenames mentioned herein are the property of their respective owners. MSDN MAGAZINE DOES NOT MAKE ANY REPRESENTATION OR WARRANTY, EXPRESS OR IMPLIED WITH RESPECT TO ANY CODE OR OTHER INFORMATION HEREIN.MSDN MAGAZINE DISCLAIMS ANY LIABILITY WHATSOEVER FOR ANY USE OF SUCH CODE OR OTHER INFORMATION. MSDN and Microsoft are registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. MSDN Magazine, MSDN, and Microsoft logos are used by CMP under license from owner. MSDN Magazine is published by CMP Media LLC. CMP Media LLC is an independent company not affiliated with Microsoft Corporation. Microsoft Corporation is solely responsible for the editorial contents of this magazine.

MSDN Magazine does not make any representation or warranty, express or implied with respect to any code or other information herein. MSDN Magazine disclaims any liability whatsoever for any use of such code or other information.


Toolbox: Synchronize Data, Review Runtime Errors, and More

Toolbox
Synchronize Data, Review Runtime Errors, and More
Scott Mitchell

Synchronize SQL Server Data

For data-driven applications, developers frequently use multiple databases: one for development, perhaps one for staging, and one for production. For apps whose scope and feature sets continue to evolve, these databases start to differ from one another. For example, the production server's schema is designed from the feature set of the last stable release, whereas the development server's schema has been modified to incorporate the functionality currently being designed. Even if the schemas are identical, the production database has live, accurate data, but the testing server contains an approximation. Often, these databases need to be compared and synchronized.

Databases can differ from one another both in their schemas and in their underlying data, and tools typically focus on comparing and synchronizing either schema differences or data differences. The April 2006 Toolbox column introduced you to SQL Examiner, a product for comparing and synchronizing SQL Server™ database schemas. For comparing the actual data in two schema-identical databases, however, you'll need to use a different product, such as SQL Data Compare version 5 from Red Gate Software. (Red Gate Software also offers SQL Compare, a tool for comparing and synchronizing database schemas.)

SQL Data Compare prompts you to select the two schema-identical databases to compare, along with their related tables, columns, and records. The two databases are examined and SQL Data Compare provides the results in an intuitive, split-screen interface. The top pane lists a summary of the results—what tables were compared, whether the data in the tables are identical or different, and how many records are identical, different, missing, or additional. Clicking on a table in the top pane lists the details of the comparison in the bottom pane. For the differing records, you can see precisely which columns vary.

Quickly determining how the data differs is quite useful, but SQL Data Compare's real power shines in its synchronization capabilities. Let's say that the testing and production servers' schemas are identical, but using SQL Data Compare you've found that the production database has several new records, a handful of deleted records, and many column-level changes. To synchronize these two database tables by hand, you'd need to craft your own T-SQL script that had the correct combination of INSERT, UPDATE, and DELETE statements—a time-consuming effort, to be sure. With SQL Data Compare, simply click the Synchronize button and the T-SQL script is automatically generated. You can execute immediately against the database to synchronize, launch the script in SQL Server Management Studio, or save it as a text file.

Price: $295 for a single-user license.

www.red-gate.com

Review Your Web App's Errors

In my day job I work as an independent consultant building ASP.NET Web applications for small- to medium-sized businesses. These applications are typically intranet-based and designed to move an existing paper-based business process to the Web. As such, when something goes awry I am quickly notified of the problem. Many times I am able to identify the problem and fix it before I actually receive their e-mails, which has convinced my clients that I am clairvoyant.

My little secret is a set of free, open-source HTTP modules and handlers designed by Atif Aziz (www.raboof.com) and referred to as Error Logging Modules and Handlers (ELMAH). Whenever a runtime error occurs in one of my Web applications, ELMAH logs the error and e-mails me its details. This provides me with a few precious minutes to fix the problem before the user has a chance to report it.

In a perfect world there would be no runtime errors. But we program in reality, where deadlines and never-ending feature requests are a given and where databases and Web servers can and do go offline. It's vital, then, that when such errors occur that the pertinent information be recorded and the developers of the project promptly notified. Without such error logging and notification you are at the mercy of your users to notify and provide you with the error's details; in short, you are making your users become testers.

When an unhandled exception is encountered in an ASP.NET application, the application's Error event is raised, which can be handled in the application's Global.asax file or by an HTTP module. ELMAH includes an HTTP module for recording the error's details in a database and another that provides notification via e-mail. For unhandled exceptions these details include the exception's message, stack trace, and request-specific details, such as the URL, query string, cookie values, post headers, and so on. In addition to its HTTP modules, ELMAH also includes HTTP handlers for displaying the error log in a Web page and as an RSS feed.

ELMAH is a cinch to set up and can be added to an existing Web site in under a minute without requiring any code changes or recompilation. Start by adding the ELMAH assembly to the application's /bin directory. Next, run the provided SQL scripts to create the database table and stored procedure used by the HTTP module for recording error details in the database. Finally, register the HTTP handlers and modules in web.config along with the ELMAH-specific settings. That's all there is to it!

Furthermore, ELMAH can be automatically deployed across all the sites on a Web server by adding the assembly to the Global Assembly Cache (GAC) and properly configuring machine.config.

ELMAH is a tool I could not live without; I use it religiously in all of my projects. The 30 seconds or so that it takes to set up ELMAH has saved me hours in debugging time and has greatly increased my turnaround time for identifying and resolving runtime errors. Just ask my clients, who are still baffled at how I can know the details about an error they just encountered before they tell me about it!

Price: Free. Includes source code.

workspaces.gotdotnet.com/elmah

Bring Windows Shell UI to Desktop Apps

Working with the file system is a task most computer users can handle regardless of their expertise level because of an unvarying user experience. Microsoft® programs use a consistent Windows® shell user interface for interacting with files and folders. Folders all have the same icon while files are represented by icons based on their file type. Moving files from one folder to another is as simple as dragging and dropping. To encourage the adoption of this UI, Microsoft provides Windows shell dialog boxes as part of the .NET Framework. The OpenFileDialog and SaveFileDialog controls display a dialog box for opening and saving files. The FolderBrowserDialog control provides a standard interface for selecting a folder from the file system.

While these Microsoft-supplied dialog box controls are helpful when designing Windows-based applications, there are times when their interfaces need to be incorporated directly within an existing form. Sure, you can use the TreeView control or ListView control for selecting a folder or file, but you're still on the hook for associating the correct icons with each folder or file type and adding the other standard Windows shell features.

Rather than building this UI from the ground up, consider using Shell MegaPack 7.1 controls from Sky Software. Shell MegaPack is a collection of four .NET-based controls designed to mimic all features of the Windows shell UI, including thumbnails, context menus, drag and drop, renaming, and so on. Want to add a tree of folders from which the user can select a particular one? Just drag the FolderView control onto your form and root the tree to a particular folder by setting the CurrentFolder property. You now have a folder selection UI that your users will be familiar with.

The Shell MegaPack also includes the FileView, ShComboBox, and ShellObjects controls. The FileView control provides a list of files, much like the file list in the OpenFileDialog and SaveFileDialog controls. If you need to list the folders in the file system in a dropdown list, use the ShComboBox control. The ShellObjects controls can be used to display Microsoft Office-style popups with slide and fade animations and system tray notifications. It also contains an API to create, enumerate, and dissect Shortcut (.lnk) and Internet Shortcut (.url) files.

Available both as ActiveX® controls and .NET libraries, Shell MegaPack 7.1 is an easy way for you to replicate the Windows shell user experience in your application's forms.

Price: $295 per developer license.

www.ssware.com

Send your questions and comments for Scott to toolsmm@microsoft.com.

Scott Mitchell, author of numerous books and founder of 4GuysFromRolla.com, is an MVP who has been working with Microsoft Web technologies since 1998. Scott is an independent consultant, trainer, and writer. Reach him at Mitchell@4guysfromrolla.com or via his blog atScottOnWriting.NET.

Send your questions and comments for Scott to toolsmm@microsoft.com.

Scott Mitchell, author of numerous books and founder of 4GuysFromRolla.com, is an MVP who has been working with Microsoft Web technologies since 1998. Scott is an independent consultant, trainer, and writer. Reach him at Mitchell@4guysfromrolla.com or via his blog.


Basic Instincts: Resources and Localization in ASP.NET 2.0

Basic Instincts
Resources and Localization in ASP.NET 2.0
Ted Pattison

Code download available at: BasicInstincts2006_08.exe (878KB)
Contents
Controlling Page-Level Culture Settings
Tracking Language Preferences Using Profiles
Resource Files in ASP.NET 2.0
Working with Local Resources
Embedding Resources in a DLL Project
Displaying Localized Images
Summing Up

In my last column I discussed the use of resource files in .NET development. I internationalized a Windows® Forms application and localized it for users who speak and read different languages. This month I'll discuss the localization of Web sites in ASP.NET 2.0.

I'll assume you're familiar with culture UI and culture settings, CultureInfo objects, resources files, the ResourceManager class, and automatically generated strongly typed resource classes. I'll also assume you have a basic understanding of the Microsoft® .NET Framework mechanisms for determining whether to load resources from the default assembly or a localized satellite assembly. If you need more background on these topics, read my May 2006 Basic Instincts column before continuing.

Controlling Page-Level Culture Settings

ASP.NET 2.0 makes it easy to change the culture settings on a page-by-page basis. You simply add the UICulture and the Culture attributes to the Page directive within an .aspx page like so.

<%@ Page UICulture="fr" Culture="fr-BE" %>
Early in the page lifecycle, an .aspx page running with these attribute settings will initialize the current thread's CurrentUICulture property and CurrentCulture property with the appropriate CultureInfo objects. If you add the attributes shown above to an .aspx file during your testing, then add a built-in ASP.NET Web control such as the Calendar control to the page, you'll see immediately that things are working properly, as Figure 1 illustrates. The Calendar control on the left has been rendered on a page with a culture setting of en-US while the Calendar control on the right has been rendered on a page with a culture setting of fr-BE.

Figure 1 Localized Calendar Controls
Figure 1 Localized Calendar Controls

In most production Web sites, however, it doesn't make sense to hardcode specific culture settings into your pages like this. Instead, users who speak and read different languages should see different localized renderings of the same page. ASP.NET can automatically initialize culture settings for you on a per-request basis if you assign a value of "auto" to both the UICulture and the Culture attributes.

<%@ Page UICulture="auto" Culture="auto" %>

ASP.NET initializes the settings by examining HTTP headers sent by the browsers. You can test different localized versions of your pages in Internet Explorer® by changing the language preference settings in the Internet Options dialog.

Typically, you'll want all of the pages in your site to conform to the same culture settings. You can assign a site-wide value of "auto" for the UICulture and Culture attributes so you don't have to touch every page individually. Just add the following element into the web.config file located at the root of a site:

<globalization uiCulture="auto" culture="auto" />

You can also specify a default culture (in addition to the auto setting) that ASP.NET will use if it can't find an HTTP header to determine the user's preferred culture:

<globalization uiCulture="auto:en" culture="auto:en-US" />

Tracking Language Preferences Using Profiles

While the auto setting certainly makes things easy, it isn't always convenient for the user. Suppose, for example, a user prefers to read technical Web sites in English and business-related Web sites in French. He'd find it really annoying to change browser settings as he moved back and forth between sites. This user would really appreciate a Web site that let him select a language preference.

To see how to build a UI and a supporting implementation that makes it easy for users to switch back and forth between different languages download this month's code sample, an ASP.NET 2.0 Web site named LitwareWebApp. Its UI is shown in Figure 2.

Figure 2a English Version
Figure 2a English Version
Figure 2b Localized Version
Figure 2b Localized Version

The LitwareWebApp site tracks user language preferences using the new profile feature introduced in ASP.NET 2.0. You can add a string-based profile property named LanguagePreference that supports anonymous identification by adding the following elements to the site's web.config file:

<configuration>
    <system.web>
        <anonymousIdentification enabled="true"/>
        <profile>
            <properties>
                <add name="LanguagePreference" type="string"
                    defaultValue="Auto" allowAnonymous="true" />
            </properties>
        </profile>
    </system.web>
</configuration>

The LitwareWebApp site is designed with a standard layout in a Master Page containing a RadioButtonList control named lstLanguage. Note that while this control displays friendly language names such as U.S. English and Belgian French, it also tracks the real culture names such as en-US and fr-BE using the SelectedValue property. When a user changes the language preference, the SelectedIndexChanged event of the lstLanguage control fires and executes the following code to update the LanguagePreference profile property:

Profile.LanguagePreference = lstLanguage.SelectedValue
Response.Redirect(Me.Request.Url.AbsolutePath)

The call to Response.Redirect forces a new round-trip from the browser to the Web server, which causes the lifecycle of the page to start again after the profile property has been set up with the desired language preference.

The next thing you have to deal with is programmatically adjusting the culture setting at the appropriate time in the page lifecycle. The proper way to do this in ASP.NET 2.0 is to override a method of the Page class named InitializeCulture. The page lifecycle has been designed to always call InitializeCulture before the page itself or any of its child controls have done any work with localized resources.

The design of the sample site requires that you add an overridden implementation of InitializeCulture to every page in the site. Unfortunately, you can't override the InitializeCulture method at the level of a Master Page because the MasterPage class does not inherit from the Page class. Furthermore, it would be tedious to override the InitializeCulture method separately for every page in a Web site. This would lead to many redundant implementations that would pose a serious maintenance issue.

A more efficient site-wide approach to initializing culture settings is to create a common Page-derived base class and then have all your .aspx page files inherit from that, and that's what I did in the LitwareWebApp sample site. My Page-derived base class named LitwarePage (see Figure 3) was defined in a source file named LitwarePage, which was added to the App_Code directory so that it is automatically compiled by ASP.NET and made available to other code in the current Web site.

Figure 3 Site-Wide Override of the InitializeCulture Method (Close Figure 3)
Imports System.Globalization
Imports System.Threading

Public Class LitwarePage : Inherits Page

    Protected Overrides Sub InitializeCulture()

        ‘*** make sure to call base class implementation
        MyBase.InitializeCulture()

        ‘*** pull language preference from profile
        Dim LanguagePreference As String = _
            CType(Context.Profile, ProfileCommon).LanguagePreference

        ‘*** set the cultures
        If LanguagePreference IsNot Nothing Then
            Me.UICulture = LanguagePreference
            Me.Culture = LanguagePreference
        End If

    End Sub
End Class

Once you've created a Page-derived base class, you can update your .aspx page definitions to derive from it instead of from the standard Page class. For example, you can modify the partial class in default.aspx.vb to look like this.

Partial Class _Default : Inherits LitwarePage
    '*** page class definition goes here
End Class

At this point, I have a Web site capable of tracking user language preferences and initializing culture settings on a per-request basis. Now I have to localize the string literals for an ASP.NET 2.0 Web site using resource files so I can accommodate users who speak and read different languages.

Resource Files in ASP.NET 2.0

Since, by default, Visual Studio® 2005 doesn't use projects to manage ASP.NET 2.0 Web sites, there's no project-level resource file as there is for a Windows Forms application or a class library DLL. Instead, you must explicitly create resource files and add them to your Web site. Furthermore, you must use special folders that were introduced with ASP.NET 2.0: resource files containing global resources should be added to the App_GlobalResources folder, and local resources specific to a single file should be added to the App_LocalResources folder. Global resources are those that can be used on a site-wide basis from within pages as well as from within other files such as a site map. ASP.NET files types that support local resources include pages (.aspx files), Master Pages (.master files) and user controls (.ascx files).

Another difference with ASP.NET 2.0 is that you don't have to compile resource files ahead of time, as you do when developing an internationalized Windows Forms application. Instead, the ASP.NET runtime compiles global and local resource files into DLLs in a just-in-time fashion just as it does with .aspx files. This is a powerful feature because a company can add localization support for a new language simply by XCOPYing .resx files to a production Web server.

Let's work through an example of creating and using a global resource file in an ASP.NET 2.0 Web site with Visual Studio 2005. You can create a new global resource file by choosing the Add New Item command and then selecting Resource File.

When you click the Add button to create a new global resource file, Visual Studio 2005 prompts you with a dialog box recommending that you place the new resource file in the App_GlobalResources directory. Click Yes. If you put it elsewhere, ASP.NET will not automatically compile the resource file into a DLL.

Working with resource files in ASP.NET is similar to working with them in a Windows Forms application. You begin by creating a resource file with string literals localized for the default culture. In our sample Web site, there's a global resource file for this purpose named Litware.resx as shown in Figure 4. Once you've added all the named strings for the default culture, you can copy the resource file and rename it to something such as Litware.fr.resx to provide strings localized in French. You can also copy that French resource file and rename it to Litware.fr-BE.resx to maintain strings that have been regionally localized for Belgian French.

Figure 4 Localized Resources
Figure 4 Localized Resources

Adding and maintaining named strings within a resource file is easy because Visual Studio 2005 provides the convenient resource editor shown in Figure 5. Remember that resource files do not limit you to localizing strings. You can add in other types of resources such as image files, cascading style sheets and client-side JavaScript files.

Figure 5 The Visual Studio 2005 Resouce Editor
Figure 5 The Visual Studio 2005 Resouce Editor

Now let's create pages that retrieve named strings from a global resource file. It's very easy to do. Just as when you're developing an internationalized Windows Forms application, there's no need to program directly against the ResourceManager class supplied by the .NET Framework. That's because ASP.NET and Visual Studio 2005 build a strongly typed resource class behind the scenes for each global resource file and make it available through IntelliSense®.

The named strings you add to a global resource file are accessible through a strongly typed class nested within a top-level namespace named Resources. It takes a single line of code to assign a localized string to a control's property value:

lblApplicationName.Text = Resources.Litware.ApplicationName

In addition to programmatic access, ASP.NET 2.0 also introduces declarative syntax you can use to bind a named string to a property of a page or control. The syntax involves using the dollar sign ($) followed by the Resources namespace, the name of the resource file and the name of the string:

<%$ Resources:Litware, ApplicationName %>

For example, if you want to bind the string named ApplicationName to the Text property of a label within an .aspx page, you can write the tag to look like this:

<asp:Label ID="lblApplicationName" runat="server"
    Text="<%$ Resources:Litware, ApplicationName %>" />

Visual Studio 2005 also provides a convenient tool named the Expression Builder, shown in Figure 6. This utility can help you generate the required syntax for binding named strings in a resource file to the properties of controls and pages. Once you've added one or more global resource files with named strings, you can access the Expression Builder by putting an .aspx page into design view and accessing the Expressions property through the Property sheet.

Figure 6 Expression Builder
Figure 6 Expression Builder

Note that declarative resource binding expressions are not limited to .aspx files, .ascx files and .master files. They can also be used to localize string literals from a site map defined in a Web.sitemap file. Figure 7 shows the XML from the site map used in the LitwareWebApp Web site to localize the link titles shown in the site's navigation menu.

Figure 7 Retrieving Literal Strings from a Global Resource File (Close Figure 7)
<siteMap xmlns="http://schemas.microsoft.com/AspNet/SiteMap-File-1.0"
        enableLocalization="true" >
    <siteMapNode url="default.aspx"
        title="$resources: Litware, HomePageLink"  >
        <siteMapNode url="AddCustomer.aspx"
            title="$resources: Litware, AddCustomerPageLink"  />
        <siteMapNode url="Controls.aspx"
             title="$resources: Litware, ControlsPageLink"  />
        <siteMapNode url="About.aspx"
             title="$resources: Litware, AboutPageLink"  />
    </siteMapNode>
</siteMap>

Working with Local Resources

A local resource file contains resources for a single file-based item within a site such as a page, a Master Page or a User Control. Each local resource file must be properly named and added to the App_LocalResources folder to be compiled by ASP.NET.

A local resource file should be named in accordance with the file-based item to which it is supplying its resources. For example, the local resource file containing default culture resources for the AddCustomer.aspx page should be named AddCustomer.aspx.resx. The local resource file containing French resources should be named AddCustomer.aspx.fr.resx.

Once you've added named strings to a local resource file, you can access them from within a page or a user control in three different ways. First, you can access them programmatically. Second, you can bind to them declaratively using explicit syntax. Third, you can bind to them declaratively using implicit syntax. Let's examine each of these approaches.

Imagine you've created local resource files to localize all the control captions shown on the page AddCustomer.aspx. To localize the caption shown on the page's submit button, you can create a localized string named btnSubmit.Text. Once you've added this named string to the local resource file, you can access it through code by calling the GetLocalResourceObject method and converting the return value to a string:

'*** code inside AddCustomer.aspx.vb
btnSubmit.Text = _
    Me.GetLocalResourceObject("btnSubmit.Text").ToString()

This code isn't as nice as the code shown earlier to access a named string from with a global resource using a strongly typed class. Local resource files do not have associated strongly typed classes, so you won't benefit from IntelliSense and you must explicitly convert the Object-based return value when calling GetLocalResourceObject.

If you want to use an explicit declarative binding syntax, it works almost identically as with global resources. The only difference is that you omit the name of the resource file when you're working with local resources:

<asp:Button ID="btnSubmit" runat="server"
    Text="<%$ Resources:btnSubmit.Text %>" />

Implicit declarative binding syntax is the most powerful option. You begin by adding a special attribute named meta:resourcekey to a control tag or to an ASP.NET directive such as Page, Master or Control. For example, if you want to use implicit declarative binding syntax with a Button control in an .aspx file, you would write its tag to look like this:

<asp:Button ID="btnSubmit" runat="server"
    meta:resourcekey="btnSubmit" />

Once you've added the meta:resourcekey attribute, the only other thing to worry about is making sure there are strings in the local resource file with the proper name. In my example, ASP.NET will automatically load the localized string named btnSubmit.Text and assign it to the Text property of the control named btnSubmit.

The key point is that implicit binding is based on creating strings that have names matching targets defined by the meta:resourcekey attribute plus the name of a property. Since the meta:resourcekey in this example is targeted at btnSubmit, you can bind to several other property values in addition to Text simply by adding more named strings into local resource files as shown in Figure 8.

Figure 8 Adding Named Strings
Figure 8 Adding Named Strings

Note that Visual Studio 2005 provides a convenient command called Generate Local Resource in the Tools menu when a Page, User Control, or Master Page is open in design view editor. This command automates the creation of a local resource file for the default culture. It also adds meta:resourcekey attributes throughout the page and creates corresponding string values in the local resource file to act as targets for meta:resourcekey attribute entries.

Finally, note that there's a new ASP.NET 2.0 component called the Localize control, which lets you localize any element on an .aspx page. It provides a design time feature not offered by its base class, the Literal control; in particular, the Localize control provides design time editing of static content so you can see a default value while working in page design mode.

Embedding Resources in a DLL Project

I'm going to depart from the topics of internationalization and localization for a moment to discuss a new ASP.NET technique for using embedded resources in a class library DLL. This technique allows you to embed resources such as images files, cascading style sheet files, and JavaScript files inside a DLL and expose them through a URL on the hosting Web server.

Note that this technique requires the use of a class library DLL targeting ASP.NET 2.0 Web sites. The new functionality was added by the ASP.NET team specifically to give server-side control authors a better way to distribute the resource files that accompany their custom controls and Web Parts. Instead of having to distribute resource files along with a DLL and ensure they are copied into an accessible path on the hosting Web server, resource files can be distributed inside the DLL itself and exposed through URLs generated by ASP.NET at run time.

The LitwareWebApp Web site contains a class library DLL project named LitwareWebComponents that demonstrates this technique. Inside this project, an image file named LitwareSlogan.png has been embedded as a resource. You can embed a resource into an assembly by changing the file's Build Action to Embedded Resource, as shown in Figure 9.

Figure 9 Embed a Resource
Figure 9 Embed a Resource

To provide Web-based access to an embedded resource file inside a DLL, you must add an assembly-level attribute named WebResource. When you add the WebResource attribute, you must include the qualified name of the resource file along with its MIME type. Within a Visual Basic® class library DLL project, the qualified resource file name includes the project name.

'*** inside AssemblyInfo.vb
Imports System.Web.UI
<Assembly: WebResource( _
    "LitwareWebComponents.LitwareSlogan.png", "image/png")>

The WebResource attribute is what allows to you to provide the ASP.NET runtime with the metadata it needs to retrieve the resource file from the DLL using a URL that can be generated at run time. To generate the URL to the resource file from code within a server-side control you call a method named GetWebResourceUrl as shown in Figure 10.

Figure 10 Accessing an Embedded Resource File (Close Figure 10)
Public Class LitwareCustomControl
    Inherits WebControl

    Protected imgSlogan As Image

    Protected Overrides Sub CreateChildControls()
        ‘*** determine Url to Web resource
        Dim ResourceName, ResourceUrl As String
        ResourceName = "LitwareWebComponents.LitwareSlogan.png"
        ResourceUrl = Me.Page.ClientScript.GetWebResourceUrl( _
            Me.GetType(), ResourceName)

        ‘*** Point Image control to Web Resource Url
        imgSlogan = New Image
        imgSlogan.ImageUrl = ResourceUrl
        Controls.Add(imgSlogan)
    End Sub

End Class

Here's what happens behind the scenes to make this technique work. A call to GetWebResourceUrl generates a URL pointing to a built-in HTTP handler named WebResource.axd. The dynamically generated URL also includes a query string identifying the name of the target DLL and the embedded resource file. The ASP.NET runtime responds to requests for WebResource.axd by loading a custom HttpHandler class named the AssemblyResourceLoader.

When the AssemblyResourceLoader class is called upon to load a resource file from a DLL, it reads the metadata supplied by the WebResource attribute. The AssemblyResourceLoader class has been implemented to extract the request resource file from the image of the DLL and stream it back to the caller. The AssemblyResourceLoader class even supplies a caching algorithm which can reuse the same resource file across multiple requests after it has been loaded into memory on the front-end Web server host.

Displaying Localized Images

Though using embedded resource files and the WebResource attribute is powerful, there are some notable limitations. First, you can only use this technique inside a DLL project that's targeted for ASP.NET 2.0 Web sites. You can't use it directly inside an ASP.NET 2.0 Web site. Second, this technique doesn't really support any form of localization. You'll have to resort to a different approach if your Web site has resource files such as graphic images and cascading style sheets that have been localized.

The LitwareWebApp Web site displays a graphic image named LitwareSlogan.png. The site has been designed to display a different version of the image depending whether the current user prefers English or French. While ASP.NET 2.0 doesn't directly support localizing image files, it doesn't require too much custom code to achieve the desired effect.

You can start by adding the localized versions of an image file to localized versions of a global resource file. For example, the English version of LitwareSlogan.png has been added to the global resource file named Litware.resx while the French version of LitwareSlogan.fr.png has been added to Litware.fr.resx. The resources in both resource files have been given the same name of LitwareSlogan.

When you have localized versions of the image file in different localized versions of a global resource file, you can load them conditionally based on the user's language preference using the custom handler file named LitwareSlogan.ashx shown in Figure 11.

Figure 11 Loading Localized Images (Close Figure 11)
<%@ WebHandler Language="VB" Class="LitwareSlogan" %>

Imports System
Imports System.Web
Imports System.Drawing
Imports System.Drawing.Imaging
Imports System.Threading
Imports System.Globalization

Public Class LitwareSlogan : Implements IHttpHandler

    Public Sub ProcessRequest(ByVal context As HttpContext) _
               Implements IHttpHandler.ProcessRequest
        ‘*** switch to correct MIME type
        context.Response.ContentType = "image/png"

        ‘*** change UICulture before loading image
        Dim LanguagePreference As String = CType(context.Profile, _
            ProfileCommon).LanguagePreference
        If LanguagePreference IsNot Nothing AndAlso _
           LanguagePreference <> "Auto" Then
            ‘*** change culture UI setting fro current thread
            Thread.CurrentThread.CurrentUICulture = _
                New CultureInfo(LanguagePreference)
        End If

        ‘*** load in image from resource file and send to browser
        Resources.Litware.LitwareSlogan.Save( _
            context.Response.OutputStream, ImageFormat.Png)
    End Sub

    Public ReadOnly Property IsReusable() As Boolean _
            Implements IHttpHandler.IsReusable
        Get
            Return False
        End Get
    End Property

End Class

The custom handler class defined in LitwareSlogan.ashx initializes the current thread's CurrentUICulture setting before retrieving the image file from the global resource file using logic similar to what you saw earlier in the custom InitializeCulture method. You might notice that to load the proper resource file, you must initialize the current thread's CurrentUICulture property, but it is not necessary to initialize CurrentCulture property.

Once this custom handler has properly initialized the CurrentUICulture setting, it can then access the image file through the strongly typed resource class for Litware.resx. Then it is simply a matter of writing the bits of the image file to the HTTP response stream. The very last step to display the localized image is to assign the LitwareSlogan.ashx URL to the ImageUrl property of an Image control on any page within the site.

Summing Up

ASP.NET 2.0 makes internationalizing Web sites and resources easier. It's trivial to initialize the culture settings for pages within a Web site by examining HTTP headers sent by the browser. Plus it's easy to design a more elaborate scheme that allows users to personalize their experience by configuring their desired language preferences.

Send you questions and comments for Ted to instinct@microsoft.com.

Ted Pattison is an author, trainer and SharePoint MVP who lives in Tampa, Florida. Ted is writing a book titled Inside Windows SharePoint Services 3.0 for Microsoft Press and he delivers advanced SharePoint 2007 Training to professional developers through his company, Ted Pattison Group (http://www.TedPattison.net).


CLR Inside Out: CLR Hosting APIs

CLR Inside Out
CLR Hosting APIs
Alessandro Catorcini and Piotr Puszkiewicz

Contents
Hosting Managers
Memory Manager
Threading Manager
Synchronization Manager
I/O Completion Manager
Assembly Load Manager
CLR Configuration Manager
Using the Hosting APIs
Conclusion

Suppose you are developing a large application in native C++ and you want to allow your customers to extend this app so they can mold it to their needs. Allowing your customers to write the extensions in managed code inside the Microsoft® .NET Framework would make their development experiences much smoother than if they had to work with native code, but loading the common language runtime (CLR) into your application's process can be worrisome. What if an uncaught CLR exception ends up killing the application's process? What if the managed extensions used more memory than your application was willing to give up? You could solve these problems by launching the runtime in another process, but moving large amounts of data across process boundaries could soon become prohibitively expensive. Is there any way to take advantage of the benefits of managed code without such expenses?

These issues quite frequently arise in the development of extensible applications. The CLR provides a variety of functionality, resources, and libraries to reduce the cost of building extensions. However, developers of such extensible applications are understandably wary of allowing potentially unreliable and resource-hungry third-party code to execute inside of their applications. Fortunately, the CLR 2.0 hosting APIs allow application developers to control the CLR's resource consumption and to guarantee a specific level of reliability for code running within the CLR. By using the hosting APIs, developers of native hosts can execute managed code in-process with complete knowledge and control over how the CLR behavior can affect their application.

The CLR has always allowed a level of integration between itself and a host. In the .NET Framework 1.x, native applications were able to load a specific version of the CLR, to start and stop its execution, and to configure a limited number of settings. In version 2.0 of the Framework, the CLR allows for a much deeper integration. The updated hosting APIs provide layers of abstraction that let the host manage many of the resources currently provided by Win32®. Furthermore, the updated APIs extend the set of CLR functionality that is configurable by the host.

Hosting Managers

The CLR 2.0 hosting APIs are split into two primary sets: host managers and CLR managers. Host manager interfaces have names prefixed with IHost, and follow with a function description (as in IHostMemoryManager and IHostSecurityManager). The implementations of the host manager interfaces are provided by the developer of the host and are registered with the CLR, which uses these interfaces to make callbacks against the host for the duration of the process lifetime.

The CLR manager interfaces are prefixed with ICLR, and what follows ICLR in the name describes the function of the manager (as in ICLRTaskManager and ICLRSyncManager). CLR managers are handed from the CLR to the host and are implemented by the CLR. The host uses these interfaces to request that the CLR perform some specific action (for example, a host can use the provided ICLRGCManager to force a garbage collection).

For the purposes of this article, we'll group the CLR managers and host managers according to their intended uses, which will help clarify the key benefits of using the hosting APIs. We will include a list of the associated ICLR* and IHost* interfaces and their descriptions when covering an area of functionality.

Memory Manager

The memory manager lets the host provide an interface through which the CLR will request all memory allocations. It replaces both the Windows® memory APIs and the standard C CLR allocation routines. Moreover, the interface allows the CLR to inform the host of the consequences of failing a particular allocation (for example, failing a memory allocation from a thread holding a lock may have certain reliability consequences). It also permits the host to customize the CLR's response to a failed allocation, ranging from an OutOfMemoryException being thrown all the way up through the process being torn down. The host can also use this manager to recapture memory from the CLR by unloading unused app domains and forcing garbage collection. The memory manager interfaces are listed in Figure 1.

Figure 1 Memory Manager Interfaces (Close Figure 1)

Interface | Description
ICLRGCManager | Provides methods that allow a host to interact with the CLR’s garbage collection system.
ICLRMemoryNotificationCallback | Notifies the CLR of the memory load on the computer.
IHostGCManager | Notifies the host that the thread from which the method call was made is about to block for a garbage collection.
IHostMAlloc | Provides methods that allow the CLR to request fine-grained allocations from the heap through the host.
IHostMemoryManager | Provides methods that allow the CLR to make virtual memory requests through the host instead of using the standard Windows platform virtual memory APIs.

SQL Server™ 2005 demonstrates the usefulness of the memory manager hosting APIs. SQL Server 2005 operates within a configurable amount of memory and is often set to use nearly all of the physical memory on the machine. To maximize performance, SQL Server 2005 tracks all memory allocations in an attempt to ensure that paging never occurs. The server would rather fail a memory allocation than page to disk. To accurately track all allocations, SQL Server uses the memory manager hosting APIs such that any memory allocation requests by the CLR are made through SQL Server rather than directly to Windows. This gives SQL Server the ability to fail CLR allocation requests before paging occurs.

Threading Manager

The new hosting APIs abstract the notion of a Win32 thread and essentially let the host define the unit of scheduling and execution. The hosting APIs use the term "Task" to define this abstraction. As part of abstracting Win32 threads, tasks provide a variety of methods for modifying the CLR.

The threading manager can:

  • Allow the host to provide an interface that the CLR will use to create and start new tasks.
  • Provide the host with a mechanism to "reuse" or pool the CLR-implemented portion of a task.
  • Support standard operations like start, abort, join, and alert.
  • Implement a callback to notify the CLR when a task has been moved to or from a runnable state. When a call is moved from a runnable state, the CLR must be able to specify that the task should be rescheduled as soon as possible.
  • Provide a way for the CLR to notify the host that a given task cannot be moved to a different physical OS thread and cannot have its execution blocked during a specified window.
  • Allow the host to provide an implementation of the thread pool. The CLR must be able to queue work items, set and query the size of the thread pool, and so on.
  • Provide notifications on both the host and CLR sides that the locale has been changed on a given task.
  • Provide a means for the CLR (and user code) to adjust the priority of a task.
Figure 2 shows the threading manager interfaces.

Figure 2 Threading Manager Interfaces (Close Figure 2)

Interface | Description
ICLRTask | Provides methods that allow the host to make requests of the CLR or to provide notification to the runtime about the associated task.
ICLRTaskManager | Provides methods that allow the host to request explicitly that the CLR create a new task, to get the currently executing task, and to set the geographic language and culture for the task.
IHostSecurityContext | Allows the CLR to maintain security context information implemented by the host.
IHostSecurityManager | Provides methods that allow access to and control over the security context of the currently executing thread.
IHostTask | Provides methods that allow the CLR to start, awaken or abort tasks, and to associate an ICLRTask instance with a corresponding IHostTask instance.
IHostTaskManager | Provides methods that allow the CLR to work with tasks through the host instead of using the standard operating system threading or fiber APIs.
IHostThreadPoolManager | Provides methods that allow the CLR to configure the thread pool and to queue work items to it.

Synchronization Manager

While letting the host create tasks is sufficient for many multithreaded applications, some hosts also require the ability to override the CLR's synchronization primitives. This ensures that locks are not taken on an operating system thread without the host's knowledge, and it allows CLR tasks to further integrate with the host's scheduling mechanism as well as permitting the host to perform deadlock detection.

Using the synchronization manager allows the host to provide implementations for several synchronization primitives to the CLR, including critical sections, events (both manual and auto-reset), semaphores, monitors, and reader/writer locks. See the synchronization manager interfaces in Figure 3.

Figure 3 Synchronization Manager Interfaces (Close Figure 3)

Interface | Description
ICLRSyncManager | Defines methods that allow the host to get information about requested tasks and to detect deadlocks in its synchronization implementation.
IHostCrst | Serves as the host’s representation of a critical section for threading.
IHostManualEvent | Provides the host’s implementation of a representation of a manual-reset event.
IHostAutoEvent | Provides a representation of the host’s implementation of an auto-reset event.
IHostSemaphore | Represents the host’s implementation of a semaphore for threading.
IHostSyncManager | Provides methods that allow the CLR to create synchronization primitives by calling the host rather than through the standard Windows platform APIs.

I/O Completion Manager

The I/O completion manager lets the host provide a custom port implementation for the CLR to use in place of the default Windows I/O completion port functionality. The host provides a way for the CLR to bind a handle to an I/O completion port. In return, the CLR supplies a callback to be invoked by the host when an asynchronous I/O operation completes. In addition, this manager also allows the host to insert custom data at the end of the OVERLAPPED structure passed to the I/O routines. Figure 4 shows the I/O completion manager interfaces.

Figure 4 I/O Completion Manager Interfaces (Close Figure 4)

Interface | Description
ICLRIoCompletionManager | Implements a callback method that allows the host to notify the CLR of the status of specified I/O requests.
IHostIoCompletionManager | Provides methods that allow the CLR to interact with I/O completion ports provided by the host.

Assembly Load Manager

Hosts relying on the old hosting APIs customized assembly-loading behavior by catching events on System.AppDomain that are thrown when an assembly, resource, or type cannot be found (AssemblyResolveEvent, ResourceResolveEvent, and TypeResolveEvent). The requested assembly was then loaded by the host using the overload of Assembly.Load that accepts a byte array and is passed back from the event. This approach has proven insufficient for a few reasons, including performance. Loading an assembly by specifying a managed byte array involves copying the bytes multiple times between managed and unmanaged memory before the assembly can be run by the host.

In version 2.0 of the hosting APIs, the host specifies which assemblies are to be loaded from the Global Assembly Cache (GAC) and fulfills all other requests directly. When binding to an assembly, the host can return the assembly as a pointer to unmanaged memory (an unmanaged IStream *). In effect, this allows a host to implement a custom assembly store. In fact, this is the primary reason that SQL Server 2005 implements its own custom assembly load manager. The assemblies that make up a database application in SQL Server 2005 are physically stored in the SQL Server database rather than as separate files on disk. SQL Server 2005 uses the assembly load manager APIs to efficiently load application assemblies out of the database while continuing to rely on the usual .NET Framework mechanisms to load the assemblies. Figure 5 shows the assembly load manager interfaces.

Figure 5 Assembly Load Manager Interfaces (Close Figure 5)

Interfaces | Description
ICLRAssemblyIdentityManager | Provides methods that support communication about assemblies between the host and the CLR.
ICLRAssemblyReferenceList | Manages a list of assemblies that are loaded by the CLR and not by the host.
ICLRHostBindingPolicyManager | Provides methods for the host to evaluate current binding policy and communicate policy changes for a specified assembly.
ICLRProbingAssemblyEnum | Provides methods that allow the host to get the probing identities of an assembly using the identity information of that assembly that is internal to the CLR, without needing to create or understand that identity.
ICLRReferenceAssemblyEnum | Provides methods that allow the host to manipulate the set of assemblies referenced by a file or stream using assembly identity data internal to the CLR, without needing to create or understand those identities.
ICLRValidator | Provides methods for validation of portable executable images and detailed reporting of validation errors.
IHostAssemblyManager | Provides methods that allow a host to specify sets of assemblies that should be loaded by the CLR or by the host.
IHostAssemblyStore | Provides methods that allow a host to bind to assemblies and modules independently of the CLR.

CLR Configuration Manager

The CLR configuration manager gives access to interfaces (see Figure 6) that allow the host to configure several aspects of the CLR, including logically grouping tasks to simplify debugging, setting a custom heap dump configuration, registering callbacks for important CLR events, blocking certain functionality from being loaded into the CLR, and setting a critical failure escalation policy.

Figure 6 CLR Configuration Manager Interfaces (Close Figure 6)

Interface | Description
IActionOnCLREvent | Provides the OnEvent method, which performs callbacks on events that have been registered using a call to ICLROnEventManager::RegisterActionOnEvent.
ICLRControl | Provides methods that allow a host to get references to and configure aspects of the CLR.
ICLRDebugManager | Provides methods that allow a host to associate a set of tasks with an identifier and friendly name.
ICLRErrorReportingManager | Provides methods that allow the host to configure custom heap dumps for error reporting.
ICLRHostProtectionManager | Provides the host with a way to block managed classes, methods, properties, and fields that offer specified capabilities from running in partially trusted code.
ICLROnEventManager | Provides methods that allow the host to register and unregister callbacks for CLR runtime events.
ICLRPolicyManager | Provides methods that allow the host to specify policy actions to be taken in the event of failures and timeouts.

The concept of an escalation policy deserves some explanation. Critical failures include resource allocation failures (such as out-of-memory conditions) and asynchronous failures (such as stack overflows), and every type of failure has a default behavior associated with it, ranging from throwing an exception through process termination. In version 2.0, the CLR enables a host to override the default behavior when a critical failure occurs. Advanced hosts can define a chain of events that will be tried in a sequence determined by the host to respond to failures. For example, a host may decide that every out-of-memory failure will translate into aborting the thread or unloading the AppDomain.

SQL Server 2005 uses escalation policy in part to ensure process stability by escalating thread aborts to AppDomain unloads when resource failures occur in areas of code that could affect multiple tasks. For example, suppose a task that's holding a lock receives a failure when trying to allocate memory. In this scenario, aborting just the current task is not sufficient to ensure stability of the AppDomain because there may be other tasks in the domain waiting on the same lock or expecting the data that was being manipulated to be in a consistent state. For more information on escalation policies and reliability, see Stephen Toub's article "High Availability: Keep Your Code Running with the Reliability Features of the .NET Framework" in the October 2005 issue of MSDN®Magazine.

Using the Hosting APIs

In his article "No More Hangs: Advanced Techniques to Avoid and Detect Deadlocks in .NET Apps" in the April 2006 issue of MSDN Magazine, Joe Duffy developed a thin host that intercepted thread and synchronization primitive operations to perform deadlock detection (the source code is available at msdn.microsoft.com/msdnmag/issues/06/04/Deadlocks). The code shows how to use the hosting APIs.

The code in Figure 7 is similar to what you'll need in every host you write. It loads the CLR into the process, starts it, and loads an assembly into the default AppDomain, invoking a method on it. (Error checking code is omitted for clarity.)

Figure 7 Typical Host Main Method (Close Figure 7)
int main(int argc, _TCHAR* argv[])
{
    // Bind to the runtime.
    ICLRRuntimeHost *pClrHost = NULL;
    HRESULT hrCorBind = CorBindToRuntimeEx(
        NULL,   // Load the latest CLR version available
        L"wks", // Workstation GC ("wks" or "svr" overrides)
        0,      // No flags needed
        CLSID_CLRRuntimeHost,
        IID_ICLRRuntimeHost,
        (PVOID*)&pClrHost);

    // Construct our host control object.
    DHHostControl *pHostControl = new DHHostControl(pClrHost);

    // Notify the CLR that this host implements hosting managers.
    pClrHost->SetHostControl(pHostControl);

    // Now, start the CLR.
    HRESULT hrStart = pClrHost->Start();

    // Load an assembly and execute a method in it.
    HRESULT hrExecute = pClrHost->ExecuteInDefaultAppDomain(
        pwzAssemblyPath, pwzAssemblyName,
        pwzMethodName, pwzMethodArgs,
        &retVal);
}

The first task in hosting the CLR is getting a pointer to the ICLRRuntimeHost interface from the CLR:

ICLRRuntimeHost *pClrHost = NULL;
HRESULT hrCorBind = CorBindToRuntimeEx(
    NULL, // Load the latest CLR version available
    L"wks", // Workstation GC ("wks" or "svr" overrides)
    0, // No flags needed
    CLSID_CLRCLRHost,
    IID_ICLRRuntimeHost,
    (PVOID*)&pClrHost);

In the .NET Framework 1.x you'd use ICorCLRHost to communicate between the CLR and the host; in version 2.0, ICLRRuntimeHost is the recommended interface, as it allows you to access all the new opt-in functionality described here. Among other functions, ICLRRuntimeHost facilitates the exchange of hosting interface implementations between the CLR and the host. ICLRRuntimeHost::SetCLRControl allows the host to notify the CLR that some host-defined services need to be hooked up to override the CLR's default behavior. Our example uses the DHHostControl class, which implements IHostControl, to override some of the CLR's default managers:

DHHostControl *pHostControl = new DHHostControl(pClrHost);
pClrHost->SetHostControl(pHostControl);

Once the host successfully calls SetHostControl, the CLR will use any interfaces defined in the host's IHostControl implementation in place of the default actions. In this example, DHHostControl overrides IHostTaskManager and IHostSyncManager by providing two classes in DHHostControl that inherit from these interfaces.

ICLRRuntimeHost also allows the host to get pointers to the CLR's implementation of the ICLR* interfaces. This is done by calling the ICLRRuntimeHost::GetCLRControl method to get a pointer to an ICLRControl object. ICLRControl lets the host access all other ICLR* interfaces through which the host can request specific actions from the CLR. Since the sample application is monitoring the CLR, it does not use the ICLR* interfaces.

Once you've loaded and started the CLR, the easiest way to load and run your managed code is to use the ExecuteInDefaultApp-Domain method on ICLRRuntimeHost. This will simply load a managed assembly and execute a method on it:

HRESULT hrExecute = pClrHost->ExecuteInDefaultAppDomain(
    pwzAssemblyPath, pwzAssemblyName,
    pwzMethodName, pwzMethodArgs,
    &retVal);

Conclusion

Unmanaged hosts are able to control very fine-grained details of the internal workings of the CLR. Using the new .NET Framework 2.0 hosting APIs, a host can, in fact, place itself between the CLR and the operating system and broker any request from the CLR. You can find more information in the online documentation of the hosting APIs and in the book Customizing the Microsoft .NET Framework Common Language Runtime by Steven Pratschner (Microsoft Press®, 2005).

Send your questions and comments to clrinout@microsoft.com.

Alessandro Catorcini is a lead program manager in the CLR team. In Visual Studio 2005 he was responsible for the hosting API layer and CLR integration into SQL Server 2005.

Piotr Puszkiewiczis a program manager in the CLR team. He is currently responsible for the hosting API, the CLR exception system, and Managed Debugging Assistants.


Cutting Edge: All About Enums

Cutting Edge
All About Enums
Dino Esposito

Code download available at: CuttingEdge2006_08.exe (170KB)
Contents
Enumeration Essentials
Under the Hood
Working with Enum Types
Binding Enum Values to the UI
A Custom Dropdown List Control
Formatting Enum Values
The Flags Attribute
Enum Values in a Data Access Layer

Among the many benefits of writing magazine articles, I particularly enjoy reader feedback, especially when readers are inspired to try new things and report back with useful development tips. Recently I've had some interesting mail about enumeration types, so I've decided to talk about them here. I'll review the basics of enumeration types and their implementation in the Microsoft® .NET Framework, then look at a reader's real-world use of them.

Enumeration Essentials

An enum type, or enumeration, is a homogeneous collection of named constants. The following is the declaration of the Enum class—the root of all enumerated types:

Public MustInherit Class Enum
    Inherits ValueType
    Implements IComparable, IFormattable, IConvertible

Recall that .NET data types are separated into value and reference types. Whereas reference types are always allocated on the common language runtime (CLR) heap, value types are allocated on the stack or as part of a reference type. Both types ultimately derive from the base class Object. Integers, numbers, dates, Booleans, characters, and bytes are all value types, as is the System.Enum type. Only integral types can be used as the underlying type for the constants in an enum type. The full list of valid types includes Byte, SByte, Int32, UInt32, Int16, UInt16, Int64, and UInt64. Note that each programming language provides special syntax to let you declare an enumeration as a set of name/value pairs. Here's how to declare an enum in Visual Basic®:

Public Enum Categories
    Speaking
    Training
    Consulting
    Mentoring
End Enum

If no underlying type is explicitly declared, Int32 (Integer in Visual Basic) is used implicitly. If you want to use constants of a particular type, then you add the type name in the declaration line:

Public Enum Categories As Int64
    Speaking
    Training
    Consulting
    Mentoring
End Enum

Each entry in an enum represents a value of the specified (or inferred) type. By default, values are assigned consecutively starting from 0. Values can also be set explicitly, both individually and for a group of elements. The following code defines 100 as the starting point for the enumeration. The Training entry will be 101, Consulting will be 102, and so on:

Public Enum Categories As Int32
    Speaking = 100
    Training
    Consulting
    Mentoring
End Enum

You could also use the following:

Public Enum Categories As Int32
    Speaking
    Training = 100
    Consulting
    Mentoring = 200
End Enum

In this case, Speaking is 0, Training is 100, Consulting 101, and Mentoring is 200. It is perfectly legal that each element is assigned its own value without any form of regular succession. Moreover, values associated with enum entries don't have to be unique:

Public Enum Categories As Int32
    Speaking
    Training = 100
    Consulting
    Mentoring = 100
End Enum

Although an enum is declared using syntax that does not look at all object oriented, the fact is that in the .NET Framework an enum type is a regular type with its own well-defined inheritance hierarchy. Enum types are not allowed to have an object model exposing their own set of methods, properties, and events. An object model exists, however, on the base class of all enums—System.Enum. Before I examine methods on System.Enum and their usage, let's take a look at enum types under the covers.

Under the Hood

In most .NET-targeted languages, an enum is declared as a collection of named constants. It is interesting to take a look at the intermediate language (IL) code generated for a sample enum as shown in the following:

Public Enum Categories As Int32
    Speaking
    Training
    Consulting
    Mentoring
End Enum

The disassembled IL for this snippet looks like the code in Figure 1. As you can see, this class extends System.Enum with a sequence of static public fields marked as literals. In particular, this means that compilers will transform any enum values referenced by code into raw values. For example, consider the following piece of Visual Basic code from an ASP.NET sample page:

Sub DropDownList1_SelectedIndexChanged( _
    ByVal sender As Object, ByVal e As EventArgs) _
        Handles DropDownList1.SelectedIndexChanged
    Response.Write(Categories.Mentoring)
End Sub

Figure 1 IL for the Categories Enum (Close Figure 1)
class public auto ansi sealed Categories
    extends [mscorlib]System.Enum
{
    .field public static literal Categories Consulting =
        int32(0x00000002)
    .field public static literal Categories Mentoring = int32(0x00000003)
    .field public static literal Categories Speaking = int32(0x00000000)
    .field public static literal Categories Training = int32(0x00000001)
    .field public specialname rtspecialname int32 value__
}

The corresponding IL is shown in Figure 2. Look carefully at line L_0006, which pushes the argument for the Response.Write value onto the evaluation stack. As you can see, the argument is an Int32 explicit value (ldc.i4 is used to load a 4-byte integer). Next, in the L_000b statement the box operation pops the argument value from the stack and replaces it with a new object that boxes the numeric value type. This is a necessary step because Response.Write requires an object as its argument.

Figure 2 IL for the Visual Basic Subroutine (Close Figure 2)
.method family instance void DropDownList1_SelectedIndexChanged(
    object sender, [mscorlib]System.EventArgs e) cil managed
{
    .maxstack 8
    L_0000: ldarg.0
    L_0001: callvirt instance [System.Web]System.Web.HttpResponse
                [System.Web]System.Web.UI.Page::get_Response()
    L_0006: ldc.i4 1000
    L_000b: box Categories
    L_0010: callvirt instance void
                [System.Web]System.Web.HttpResponse::Write(object)
    L_0015: ret
}

It is also interesting to dig out some internals of the System.Enum class—the parent of all enum types.

As shown earlier, System.Enum implements three interfaces that add specific functionalities, in this case, comparison, formatting, and conversion. The IComparable interface provides a CompareTo method whereas the IFormattable interface's contribution is all in one of the overloads for the ToString method. The IConvertible interface provides a number of ToXXX methods to convert enum values into more specific, mostly numeric, types.

Working with Enum Types

The System.Enum provides a number of methods to list, parse, convert, and compare values, as you can see in Figure 3. Let's take a look at the GetValues method. It is defined as follows:

Public Shared Function GetValues(ByVal enumType As Type) As Array
Simply put, the method returns an array of objects filled with all values associated with the enum. Here's some sample code that shows how to use the GetValues method to grab the values in the Categories enum type:
Dim values As String() = [Enum].GetValues(GetType(Categories))

Figure 3 Important System.Enum Methods (Close Figure 3)

Method | Description
CompareTo | Instance method; compares two enum values and returns an indication of how they relate.
Format | Static method; converts the specified value to its equivalent string representation using the specified format.
GetName | Static method; retrieves the name of the constant that has the specified value.
GetNames | Static method; retrieves an array of the names of the constants.
GetUnderlyingType | Static method; retrieves an array of the names of the constants.
GetValues | Static method; retrieves an array of the values of the constants.
IsDefined | Static method; returns an indication whether a constant with a specified value exists in the enumeration.
Parse | Static method; converts the string representation of the name or numeric value of one or more enumerated constants to an equivalent enumerated object.
ToObject | Static method; returns an instance of the enum for a specific value.
ToString | Instance method; converts the value of a named constant to an equivalent string representation.

How can this work if all the compiler emits is the code in Figure 1? In other words, the compiler creates a new class that extends System.Enum and adds constant fields. Where does the GetValues method (and similarly the GetNames method) get content for the array it returns?

If you look under the hood of the System.Enum class using a decompiler or disassembler, you'll see that the class internally makes use of a Hashtable synchronized for access. The Hashtable is initialized in the static constructor of the Enum class. Also known as a type initializer, a static constructor is a method that the CLR invokes automatically (if any is defined on a type) to initialize a type. A type initializer in Microsoft intermediate language (MSIL) is a static method named .cctor. Type initializers are useful to ensure that some static code executes before a class is instantiated. The following is Visual Basic pseudocode for the relevant portion of the .cctor method of System.Enum:

Shared Sub New()
   ...
       Enum.fieldInfoHash = _
           Hashtable.Synchronized(New Hashtable)
End Sub

When will the fieldInfoHash internal member be accessed? It is used by most of the methods on System.Enum, that is Parse, GetNames, GetValues, and IsDefined, plus a few other internal and private methods. GetValues, for example, instantiates an array to return to callers and populates it with data from fieldInfoHash.

The Hashtable is populated on-demand through reflection with the values of all the named constants that form the enum. It is worth noting that the GetHashEntry method takes a type parameter; the Hashtable contains information for all enum types being used in the application. The cost of the reflection to populate the Hashtable for a particular enumeration is paid only once per application; any successive access is to cached data. All static methods on the System.Enum class consume cached data with no significant performance costs.

Binding Enum Values to the UI

The main purpose of GetValues is to provide the contents of the enum type in a bindable form. In both ASP.NET and Windows® Forms, data can be bound to user interface controls if it's supplied in collection containers. There might be circumstances in which showing the contents of an enum through a data-bound control is desirable. This is the case, for example, when you're building middle-tier objects and need to strongly identify a particular category of values whose natural type—be it Integer, Byte, or Long—is too generic for the role the data plays in the context of the application. Enumerations make code easier to read and write and easier to understand and debug. A named constant is far more readable than a number. In addition, enums promote strongly typed programming. If the parameter of a method is defined as being of type Categories, for instance, you have to either pass a value from Categories or explicitly throw away some type safety by casting a valid integral value to the Categories type before supplying it to the method.

A set of options and settings for a number of components can be effectively expressed using enum values. Where users can configure the behavior of these components you might want to show options through data-bound controls such as a dropdown list or a group of checkboxes.

The following ASP.NET code shows how to bind the names of the values in an enum to a dropdown list:

DropDownList1.DataSource = _
    [Enum].GetNames(GetType(Categories))
DropDownList1.DataBind()

The dropdown list displays the names of the elements and returns the same names when users make their selection. Take a look at the following code:

Response.Write(DropDownList1.SelectedValue)

If the dropdown list is bound to an enum type (see Figure 4), the preceding code fragment returns the displayed string regardless of the value of each entry. It would be nice to have the ability to display symbolic names in a data-bound control and return real numeric values when a selection is made. For this to happen, some changes must be made on the control to specialize it for an enum.

Figure 4 Enum in Dropdown
Figure 4 Enum in Dropdown

To map a symbolic name to an enumerated value, you use the Parse method, as in the following:

Dim c As Categories = _
    [Enum].Parse(GetType(Categories), "Mentoring")

To bind the contents of an enum type to a control, you need to transform those contents into a collection which is then bound. Likewise, to retrieve a selection from the same control, you need an additional parse step. Both actions can be incorporated into a new control. Let's see how to specialize a data-bound control (a dropdown list, for example) to work with enums declaratively.

A Custom Dropdown List Control

Figure 5 lists the code for a new dropdown list control that is built to work with enums. The idea is that the new control takes an enum type name and automatically sets the data source of the control to the contents of the enum type. The new control introduces two new properties to express the type of the bound enum: DataSourceType and DataSourceTypeName. The two properties refer to the same piece of information—the enum type name—in two distinct forms—object and string.

Figure 5 EnumDropDownList (Close Figure 5)
Namespace MsdnMag.CuttingEdge

    Public Class EnumDropDownList : Inherits DropDownList
        Public Property DataSourceType() As Type
            Get
                Dim o As Object = ViewState("DataSourceType")
                If o Is Nothing Then
                    Return Type.Missing
                Else
                    Return DirectCast(o, Type)
                End If
            End Get
            Set(ByVal value As Type)
                ViewState("DataSourceType") = value
                If value.BaseType Is GetType(System.Enum) Then
                    DataSource = [Enum].GetNames(value)
                End If
            End Set
        End Property

        Public Property DataSourceTypeName() As String
            Get
                Dim o As Object = ViewState("DataSourceTypeName")
                If o Is Nothing Then
                    Return String.Empty
                Else
                    Return DirectCast(o, String)
                End If
            End Get
            Set(ByVal value As String)
                ViewState("DataSourceTypeName") = value
                Dim t As Type = Type.GetType(value)
                If t.BaseType Is GetType(System.Enum) Then
                    DataSource = [Enum].GetNames(t)
                End If
            End Set
        End Property

        Public ReadOnly Property SelectedEnumValue() As Object
            Get
                Dim t As Type
                If String.IsNullOrEmpty(DataSourceTypeName) Then
                    t = DataSourceType
                Else
                    t = Type.GetType(DataSourceTypeName)
                End If
                If t Is Nothing Then
                    Return Nothing
                End If
                If t.BaseType Is GetType(System.Enum) Then
                    Dim o As Object = [Enum].Parse(t, SelectedValue)
                    Return [Enum].ToObject(t, o)
                Else
                    Return Nothing
                End If
            End Get
        End Property
    End Class

End Namespace

Let's examine the set method of the DataSourceTypeName string property:

Set(ByVal value As String)
    ViewState("DataSourceTypeName") = value
    Dim t As Type = Type.GetType(value)
    If t.BaseType Is GetType(System.Enum) Then
        DataSource = [Enum].GetNames(t)
    End If
End Set
If the Type object that corresponds to the specified string derives from System.Enum, the DataSource property of the control is automatically set to the array returned by the GetNames method. The DataBind method is not invoked, so you still need to trigger the data-binding operation programmatically when it is more convenient to your code.

As the code demonstrates, by associating a type name (or an equivalent Type object) with the dropdown control you are laying the groundwork for flowing the named constants of the enum into the control via binding. To illustrate the retrieving process, let's examine the code for the SelectedEnumValue property by taking a look at Figure 6.

Figure 6 SelectedEnumValue Property (Close Figure 6)
Public ReadOnly Property SelectedEnumValue() As Object
   Get
       Dim t As Type
       If String.IsNullOrEmpty(DataSourceTypeName) Then
           t = DataSourceType
       Else
           t = Type.GetType(DataSourceTypeName)
       End If

       If t Is Nothing Then Return Nothing

       If t.BaseType Is GetType(System.Enum) Then
          Dim o As Object = [Enum].Parse(t, SelectedValue)
          Return [Enum].ToObject(t, o)
       Else
          Return Nothing
       End If
   End Get
End Property

The get accessor of the property SelectedEnumValue is a read-only property. First it ensures that either DataSourceType or DataSourceTypeName is defined and can provide type information. Next, it parses the string returned by the SelectedValue property of the base dropdown list class and obtains an enumerated value. Finally, the value returned by the property is the result of the ToObject method on the enumerated value.

Putting it all together, with the new dropdown list control in Figure 5 you can write ASP.NET pages like so:

<%@ Register Namespace="MsdnMag.CuttingEdge" TagPrefix="msdn" %>
...
<msdn:EnumDropDownList ID="DropDownList1" runat="server"
   AutoPostBack="true"
   DataSourceTypeName="Categories">
</x:EnumDropDownList>
The SelectedIndexChanged event on the dropdown list can be used to capture the value of the new SelectedEnumValue property, as shown here:
Protected Sub DropDownList1_SelectedIndexChanged( _
    ByVal sender As Object, ByVal e As EventArgs) _
        Handles DropDownList1.SelectedIndexChanged
    Response.Write(DirectCast(DropDownList1.SelectedEnumValue, Int32))
End Sub
The sample EnumDropDownList control provides the same capabilities as the parent dropdown plus the ability to bind directly to enums. A similar mechanism can be easily built for Windows Forms applications extending the combobox control.

Formatting Enum Values

The ToObject method takes type information and a symbolic constant and returns the corresponding value. The Format method does something similar except that it converts the specified constant of the enum into an equivalent string or numeric representation:

Dim cat As Categories = Categories.Training
Response.Write([Enum].Format(GetType(Categories), cat, outputFormat))
The Format method supports a few predefined output formats, each identified with a letter. The outputFormat variable in the preceding code snippet can take any of the following values: F, D, X, or G. The X format represents the value as a hexadecimal value without the leading 0x. The D format renders the value as a decimal value. The F format toggles between the symbolic and numeric representation of an enum value. For example, if you call Format and specify the value as, say, Training, it will return the numeric value of the element. If you call Format on a numeric value, the output string contains the symbolic name. The G format works like F except that it also fully supports enum types with the Flags attribute.

The Flags Attribute

The Flags attribute indicates that the elements of an enumeration can be composed in a bitwise manner:

<Flags> _
Public Enum Categories As Int32
    Speaking = 1
    Training = 2
    Consulting = 4
    Mentoring = 8
    All = Speaking Or Training Or Consulting Or Mentoring
End Enum
When the Flags attribute is used, you can use the Or operator in Visual Basic (and the | operator in C#) to combine values to form new valid values of the same enum type. In this case, it is essential that you give explicit values to enum values as unique powers of two. It is interesting to note that when Flags is specified, Parse and Format methods feature advanced capabilities. For example, the following code returns a comma-separated string of constants: "Speaking, Training, Consulting".
Response.Write([Enum].Format(GetType(Categories), 7, "g"))
Likewise, the Parse method can successfully parse a comma-separated string like the one just shown into the proper numeric value. This is a very powerful feature, especially when you're reading options from the configuration file of an application:
<appSettings>
   <add key="Categories" value="Speaking, Training" />
</appSettings>

You use the ConfigurationManager class to retrieve the string from the configuration file and then convert it into the format that is more convenient:

Dim o As Object = [Enum].Parse(GetType(MyCategories), _
    ConfigurationManager.AppSettings("Categories"))
Dim n As Int32 = DirectCast(o, Int32)
Response.Write(n)
The preceding code reads back the configuration section and converts to an integer.

Enum Values in a Data Access Layer

Now that I've recapped the basics of enum types and investigated some of the internals, I'd like to turn to the set of questions asked by one of my readers. The reader built a relatively simple data access layer; for the most part, he relied on table adapters. However, he extended the set of classes generated by the Visual Studio® 2005 data designer with some helper classes to better describe some of the logical entities and relationships in the problem domain. One of the entities—I'll call it the Document—required a member describing the source of the contained data. In the example, the member can take a value from a short list of fixed values. The reader defined an enum like this:

Public Enum SourceType As Short
    Unknown = -1
    Xml = 1
    SqlDatabase
    JetDatabase
    RawText
End Enum

He wrote ADO.NET code to save and restore these values to a SQL Server™ database and was looking for an effective way to list available options from within ASP.NET pages and Windows Forms. The GetValues and GetNames methods, along with the Parse method, were obvious choices. I suggested the creation of a couple of custom controls (like the aforementioned EnumDropDownList) to make the writing of the user interface simpler. All in all, is using an enum in a similar situation a good choice?

I see two main disadvantages to using enum types: maintainability and localization. Incorporating the values of the enum types into the user interface was child's play for the reader and made the involved component neater, easier to read, even more elegant, and extremely cheap to implement. The rub here is that an enum type bound to UI controls exposes internal information—specifically, type information—to the presentation layer. A dropdown list like the one in Figure 4 shows off type information. If you modify the type by renaming one of the members, for example, you have to recompile the assembly in which the type is defined.

As an alternative you could create an entity class for the information and supply a data provider component that wraps the behavior of the entity. This provider component would take care of exposing the contents of the entity through a set of contracted methods. Where does the provider get its data? Typically, entity information is obtained from database tables. Changes to the information involve the database administrator and don't require a compile step. In the database table, you could provide for an ID and DisplayName pair of columns for a neat separation between UI, presentation data, and type information, and still maintain a strongly typed programming style. If you're familiar with the Northwind database, you know that it features a Categories and Products table. Each product is bound to a category via a classic foreign-key relationship. Admittedly, there's a bit of abstraction in this solution that many real programmers tend to skip as an unneeded layer of complexity. The reader decided that in his own solution with a lot of table adapters and DataSet, a data-mapper layer would have been too much even if limited to a couple of entities. So he opted for enum types and actually delivered the application.

A few months later, his boss asked him to provide a localized version of the application. How can you localize an enum type? Well, normally you won't need to. Enum types are numeric values and numbers look the same in many cultures. But this is not the case if you use the symbolic name of the constant in the user interface. In that case, you'd have to redefine the type!

To make a long story short, a choice made to simplify and speed up coding proved later to be patently wrong. There is a lesson to be learned here: enum types are a powerful tool, but they are not appropriate for every situation.

Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is a mentor at Solid Quality Learning and the author of Programming Microsoft ASP.NET 2.0 (Microsoft Press, 2005). Based in Italy, Dino is a frequent speaker at industry events worldwide. Get in touch with Dino atcutting@microsoft.com or join the blog at weblogs.asp.net/despos.


Design Patterns: Model View Presenter

Design Patterns
Model View Presenter
Jean-Paul Boodhoo

Code download available at: DesignPatterns2006_08.exe (4423KB)
Contents
Following the MVP
Making the First Test Pass
Filling the DropDownList
Implementing the View Interface
What's Next?

As UI-creation technologies such as ASP.NET and Windows® Forms become more and more powerful, it's common practice to let the UI layer do more than it should. Without a clear separation of responsibilities, the UI layer can often become a catch-all for logic that really belongs in other layers of the application. One design pattern, the Model View Presenter (MVP) pattern, is especially well suited to solving this problem. In order to illustrate my point, I will build a display screen that follows the MVP pattern for customers in the Northwind database.

Why is it bad to have lots of logic in the UI layer? The code in the UI layer of an application is very difficult to test without either running the application manually or maintaining ugly UI runner scripts that automate the execution of UI components. While this is a big problem in itself, an even bigger problem is the reams of code that are duplicated between common views in an application. It can often be hard to see good candidates for refactoring when the logic to perform a specific business function is copied among different pieces in the UI layer. The MVP design pattern makes it much easier to factor logic and code out of the UI layer for more streamlined, reusable code that's easier to test.

Figure 1 shows the main layers that make up the sample application. Notice that there are separate packages for UI and presentation. You might have expected them to be the same, but actually the UI layer of a project should consist only of the various UI elements—forms and controls. In a Web Forms project this is typically a collection of ASP.NET Web Forms, user controls, and server controls. In Windows Forms, it is a collection of Windows Forms, user controls, and third-party libraries. This extra layer is what keeps the display and the logic separate. In the presentation layer you have the objects that actually implement the behavior for the UI—things like validation display, collection input from the UI, and so forth.

Figure 1 Application Architecture
Figure 1 Application Architecture

Following the MVP

As you can see in Figure 2, the UI for this project is pretty standard. When the page loads, the screen will display a dropdown box filled with all of the customers in the Northwind database. If you select a customer from the dropdown list, the page will update to display the information for that customer. By following the MVP design pattern you can factor behaviors out of the UI and into their own classes. Figure 3 shows a class diagram that indicates the association between the different classes that are involved.

Figure 2 Customer Information
Figure 2 Customer Information

It's important to note that the presenter has no knowledge of the actual UI layer of the application. It knows it can talk to an interface, but it does not know or care what the implementation of that interface is. This promotes reuse of presenters between disparate UI technologies.

I am going to use Test Driven Development (TDD) to build the functionality of the customer screen. Figure 4 shows the details for the first test I will use to describe the behavior I expect to observe on page load. TDD lets me focus on one problem at a time, write just enough code to make the test pass, and then carry on. In this test I am making use of a mock object framework called NMock2 that allows me to build mock implementations of interfaces.

Figure 4 The First Test (Close Figure 4)
[Test]
public void ShouldLoadListOfCustomersOnInitialize()
{
    mockery = new Mockery();
    ICustomerTask  mockCustomerTask = mockery.NewMock<ICustomerTask>();
    IViewCustomerView  mockViewCustomerView =
        mockery.NewMock<IViewCustomerView>();
    ILookupList  mockCustomerLookupList = mockery.NewMock<ILookupList>();

    ViewCustomerPresenter presenter =
        new ViewCustomerPresenter(mockViewCustomerView,
          mockCustomerTask);

    ILookupCollection mockLookupCollection =
        mockery.NewMock<ILookupCollection>();

    Expect.Once.On(mockCustomerTask).Method(
        "GetCustomerList").Will(Return.Value(mockLookupCollection));
    Expect.Once.On(mockViewCustomerView).GetProperty(
        "CustomerList").Will(Return.Value(mockCustomerLookupList));
    Expect.Once.On(mockLookupCollection).Method(
        "BindTo").With(mockCustomerLookupList);

    presenter.Initialize();
}
Figure 3 MVP Class Diagram
Figure 3 MVP Class Diagram

In my MVP implementation, I have decided that the presenter is going to take as a dependency the view it is going to work with. It is always good to create objects in a state that enables them to do their work immediately. In this application, the presentation layer is dependent on the service layer to actually invoke the domain functionality. Because of this requirement it also makes sense to construct a presenter with an interface to a service class that it can talk to. This ensures that once a presenter is constructed, it is ready to do all of the work it needs to do. I start off by creating two specific mocks: one for the service layer and one for the view that the presenter will work with.

Why mocks? A rule of unit testing is to isolate the test as much as possible to focus on one specific object. In this test I am only interested in the expected behavior of the presenter. At this point I don't care about the actual implementation of the view interface or service interface; I trust the contracts defined by those interfaces and set the mocks to behave accordingly. This ensures that I focus my test solely around the behavior I expect of the presenter, not of any of its dependencies. The behavior I expect the presenter to exhibit after its initialize method is invoked is as follows.

First, the presenter should make one call to the GetCustomerList method on the ICustomerTask service layer object (mocked in the test). Notice that with the use of NMock I can simulate behavior of the mock. In the case of the service layer, I want it to return a mock ILookupCollection to the presenter. Then, after the presenter retrieves the ILookupCollection from the service layer, it should invoke the collection's BindTo method and pass the method an implementation of an ILookupList. By using the NMockExpect.Once method I can be sure that the test will fail if the presenter does not call the method once and only once.

After writing that test I am in a completely non-compilable state. I'm going to do the simplest thing possible to get the test to pass.

Making the First Test Pass

One of the advantages of writing the test first is that I now have a blueprint (the test) I can follow to get the test to compile and eventually pass. The first test has two interfaces that do not yet exist. These interfaces are the first prerequisites for getting the code to compile correctly. I'll start with the code for IViewCustomerView:

public interface IViewCustomerView
{
    ILookupList CustomerList { get; }
}
This interface exposes one property that returns an ILookupList interface implementation. I don't yet have an ILookupList interface or even an implementer, for that matter. For the purpose of getting this test to pass I don't need an explicit implementor, so I can proceed to create the ILookupList interface:
public interface ILookupList { }

At this point the ILookupList interface looks pretty useless. My goal is to get the test to compile and pass, and these interfaces satisfy the test's requirements. It's time to shift focus to the object that I'm actually testing—the ViewCustomerPresenter. This class does not exist yet, but looking at the test you can glean two important facts about it: it has a constructor that requires both a view and service implementation as dependencies, and it has a void Initialize method. The code in Figure 5 shows how to compile the test.

Figure 5 Compiling the Test (Close Figure 5)
public class ViewCustomerPresenter
{
    private readonly IViewCustomerView view;
    private readonly ICustomerTask task;

    public ViewCustomerPresenter(
        IViewCustomerView view, ICustomerTask task)
    {
        this.view = view;
        this.task = task;
    }

    public void Initialize()
    {
        throw new NotImplementedException();
    }
}

Remember that a presenter requires all of its dependencies in order to do its work meaningfully; this is why the view and service are passed in. I have not implemented the initialize method so if I run the test I get a NotImplementedException.

As mentioned already, I am not coding the presenter blindly; I already know, from looking at the test, what behavior the presenter should exhibit once the initialize method is called. The implementation of that behavior is as follows:

public void Initialize()
{
    task.GetCustomerList().BindTo(view.CustomerList);
}

In the source code accompanying this article there is a complete implementation of the GetCustomerList method in the CustomerTask class (which implements the ICustomerTask interface). From the perspective of implementing and testing the presenter, though, I don't need to know whether there is a working implementation yet. It is this level of abstraction that allows me to plow through with the testing of the presenter class. The first test is now in a state that will compile and run. This proves that when the Initialize method on the presenter is called, it will interact with its dependencies in the manner that I specified in the test, and ultimately when the concrete implementations of those dependencies are injected into the presenter, I can be sure that the resulting view (the ASPX page) will be filled with a list of Customers.

Filling the DropDownList

So far I have been dealing mainly with interfaces to abstract the actual implementation details away, allowing the focus to be on the presenter. It is now time to build some of the plumbing that will ultimately allow the presenter to populate a list on a Web page in a way that can be tested. The key to making this work is the interaction that will occur in the BindTo method of the LookupCollection class. If you take a look at the implementation of the LookupCollection class in Figure 6, you will notice that it implements the ILookupCollection interface. The source code for the article has the accompanying tests that were used to build up the functionality of the LookupCollection class.

Figure 6 The LookupCollection Class (Close Figure 6)
public class LookupCollection : ILookupCollection
{
    private IList<ILookupDTO> items;

    public LookupCollection(IEnumerable<ILookupDTO> items)
    {
        this.items = new List<ILookupDTO>(items);
    }

    public int Count { get { return items.Count; } }

    public void BindTo(ILookupList list)
    {
        list.Clear();
        foreach (ILookupDTO dto in items) list.Add(dto);
    }
}

The implementation of the BindTo method is of particular interest. Notice that in this method the collection iterates through its own private list of ILookupDTO implementations. An ILookupDTO is an interface that caters well to binding to comboboxes in the UI layer:

public interface ILookupDTO
{
    string Value { get; }
    string Text { get; }
}

Figure 7 shows the code that tests the lookup collection's BindTo method, which will help explain the expected interaction between a LookupCollection and an ILookupList. The last line is of particular interest. In this test I expect that before attempting to add items to the list, the LookupCollection will invoke the Clear method on the ILookupList implementation. I then expect Add to be called on an ILookupList 10 times, and as an argument to the Add method, the LookupCollection will pass in an object that implements the ILookupDTO interface. To make this actually work with a control that lives in a Web project (such as a dropdown listbox) you will need to create an implementation of ILookupList that knows how to work with controls in a Web project.

Figure 7 A Test that Describes Behavior (Close Figure 7)
[Test]
public void ShouldBeAbleToBindToLookupList()
{
    IList<ILookupDTO> dtos = new IList;
    ILookupList mockLookupList = mockery.NewMock<ILookupList>();

    Expect.Once.On(mockLookupList).Method("Clear");

    for (int i = 0; i < 10; i++)
    {
        SimpleLookupDTO dto =
            new SimpleLookupDTO(i.ToString(),i.ToString());
        dtos.Add(dto);
        Expect.Once.On(mockLookupList).Method("Add").With(dto);
    }

    new LookupCollection(dtos).BindTo(mockLookupList);
}

The source code that accompanies this article contains a project named MVP.Web.Controls. This project contains any Web-specific controls or classes that I chose to create to complete the solution. Why did I place the code in this project and not in the APP_CODE directory or the Web project itself? Testability. Anything that lives in the Web project is difficult to test directly without either running the application manually or automating the UI using some sort of testing robot. The MVP pattern allows me to think at a higher level of abstraction and test implementations of the core interfaces (ILookupList and ILookupCollection) without manually running the application. I am going to add a new class, a WebLookupList control, to the Web.Controls project. Figure 8 shows the first test for this class.

Figure 8 First Test for WebLookupList Control (Close Figure 8)
[Test]
public void ShouldAddItemToUnderlyingList()
{
    ListControl webList = new DropDownList();
    ILookupList list = new WebLookupList(webList);

    SimpleLookupDTO dto = new SimpleLookupDTO("1","1");
    list.Add(dto);

    Assert.AreEqual(1, webList.Items.Count);
    Assert.AreEqual(dto.Value, webList.Items[0].Value);
    Assert.AreEqual(dto.Text, webList.Items[0].Text);
}

Some things stand out in the test that is shown in Figure 8. The test project clearly needs a reference to the System.Web library so that it can instantiate DropDownList Web controls. Looking at the test, you should see that the WebLookupList class will implement the ILookupList interface. It is also going to take a ListControl as a dependency. Two of the most common ListControl implementations in the System.Web.UI.WebControls namespace are the DropDownList and the ListBox classes. A key feature of the test in Figure 8 is the fact that I am ensuring that a WebLookupList correctly updates the state of an actual Web ListControl to which it is delegating responsibility. Figure 9 shows the class diagram for the classes involved in the WebLookupList implementation. I can satisfy the requirements of the first test for the WebLookupList control with the code in Figure 10.

Figure 10 WebLookupList Control (Close Figure 10)
public class WebLookupList : ILookupList
{
    private ListControl underlyingList;

    public WebLookupList(ListControl underlyingList)
    {
        this.underlyingList = underlyingList;
    }

    public void Add(ILookupDTO dto)
    {
        underlyingList.Items.Add(new ListItem(dto.Text, dto.Value));
    }
}
Figure 9 WebLookupList Class
Figure 9 WebLookupList Class

Remember, one of the keys to MVP is the separation of layers introduced by the creation of a view interface. The presenter doesn't know what implementation of a view, and respectively an ILookupList, it will be talking to; it just knows that it will be able to call any of the methods defined by those interfaces. Ultimately, the WebLookupList class is a class that wraps and delegates to an underlying ListControl (base class for some of the ListControls defined in the System.Web.UI.WebControls project). With that code now in place, I can compile and run the WebLookupList control test which should pass. I can add one more test for the WebLookupList that tests the actual behavior of the clear method:

[Test]
public void ShouldClearUnderlyingList()
{
    ListControl webList = new DropDownList();
    ILookupList list = new WebLookupList(webList);

    webList.Items.Add(new ListItem("1", "1"));

    list.Clear();

    Assert.AreEqual(0, webList.Items.Count);
}
Again I am testing that the WebLookupList class will actually change the state of the underlying ListControl (DropDownList) when its own methods are invoked. The WebLookupList is now feature complete for the purposes of populating a DropDownList in a Web Form. It is now time for me to wire everything together and get the Web page's dropdown filled with a list of customers.

Implementing the View Interface

Because I am building a Web Forms front end, it makes sense that the implementer for the IViewCustomerView interface would be a Web Form or a user control. For the purpose of this column, I'll make it a Web Form. The general appearance of the page has already been created, as you saw in Figure 2. Now I need only to implement the view interface. Switching to the codebehind for the ViewCustomers.aspx page, I can add the following code, indicating that the page is required to implement the IViewCustomersView interface:

public partial class ViewCustomers : Page,IViewCustomerView

If you look at the code sample, you'll notice that the Web project and the Presentation are two completely different assemblies. Also, the Presentation project has no reference whatsoever to the Web.UI project, further maintaining the layer of separation. On the other hand, the Web.UI project has to have a reference to the Presentation project, as that is where the View interface and the presenter live.

By choosing to implement the IViewCustomerView interface, our Web page now has a responsibility to implement any methods or properties defined by that interface. Currently there is only one property on the IViewCustomerView interface and that is a getter that returns any implementation of an ILookupList interface. I added a reference to the Web.Controls project so that I can instantiate a WebLookupListControl. I did this because the WebLookupListControl implements the ILookupList interface and it knows how to delegate to actual WebControls that live in ASP.NET. Taking a look at the ASPX for the ViewCustomer page, you will see that the list of customers is simply an asp:DropDownList control:

<td>Customers:</td>
<td><asp:DropDownList id="customerDropDownList" AutoPostBack="true"
        runat="server" Width="308px"></asp:DropDownList></td>
</tr>
With this already in place, I can quickly continue to implement the code required to satisfy the implementation of the IViewCustomerView interface:
public ILookupList CustomerList
{
    get { return new WebLookupList(this.customerDropDownList);}
}

I now need to invoke the Initialize method on the presenter that will trigger it to actually do some work. To do that, the view needs to be able to instantiate the presenter so that methods on it can be invoked. If you look back to the presenter, you'll remember that it requires both a view and a service that it will work with. The ICustomerTask interface represents an interface that lives in the service layer of the application. Service layers are typically responsible for orchestrating the interaction between domain objects and converting the results of those interactions into Data Transfer Objects (DTOs) that are then passed from the service layer to the presentation layer and then to the UI layer. There is a problem, however—I have stipulated that the presenter needs to be constructed with both the view and service implementations.

The actual instantiation of the presenter is going to take place in the codebehind for the Web page. This is a problem, because the UI project has no reference to the service layer project. The presentation project does, however, have a reference to the service layer project. This allows me to solve the problem by adding an overloaded constructor to the ViewCustomerPresenterClass:

public ViewCustomerPresenter(IViewCustomerView view) :
    this(view, new CustomerTask()) {}

This new constructor satisfies the presenter's requirement for implementations of both the view and the service, while also maintaining the separation of the UI layer from the service layer. It is now fairly trivial to finish off the code for the codebehind:

protected override void OnInit(EventArgs e)
{
    base.OnInit(e);
    presenter = new ViewCustomerPresenter(this);
}

protected void Page_Load(object sender, EventArgs e)
{
    if (!IsPostBack) presenter.Initialize();
}
Notice the key to the instantiation of the presenter is the fact that I am making use of the newly created overload for the constructor, and the Web Form passes itself in as an object that implements the view interface!

With the code for the codebehind implemented, I can now build and run the application. The DropDownList on the Web page is now filled with a list of customer names without the need for any databinding code in the codebehind. Plus, scores of tests have been run on all the pieces that ultimately work together, ensuring that the presentation layer architecture will behave as expected.

I'm going to wrap up my discussion of MVP by showing what is required to display customer information for a customer selected in the DropDownList. Once again, I start by writing a test that describes the behavior I hope to observe (see Figure 11).

Figure 11 One Last Test (Close Figure 11)
[Test]
public void ShouldDisplayCustomerDetails()
{
    SimpleLookupDTO lookupDTO = new SimpleLookupDTO("1","JPBOO");

    CustomerDTO dto = new CustomerDTO("BLAH", "BLAHCOMPNAME",
        "BLAHCONTACTNAME", "BLAHCONTACTTILE", "ADDRESS", "CITY",
        "REGION", "POSTALCODE", Country.CANADA, "4444444", "4444444");

    Expect.Once.On(mockViewCustomerView).GetProperty(
        "CustomerList").Will(Return.Value(mockCustomerLookupList));
    Expect.Once.On(mockCustomerLookupList).GetProperty(
        "SelectedItem").Will(Return.Value(lookupDTO));
    Expect.Once.On(mockCustomerTask).Method(
        "GetDetailsForCustomer").With(1).Will(Return.Value(dto));
    Expect.Once.On(mockViewCustomerView).SetProperty(
        "CompanyName").To(dto.CompanyName);
    Expect.Once.On(mockViewCustomerView).SetProperty(
        "ContactName").To(dto.ContactName);
    Expect.Once.On(mockViewCustomerView).SetProperty(
        "ContactTitle").To(dto.ContactTitle);
    Expect.Once.On(mockViewCustomerView).SetProperty(
        "Address").To(dto.Address);
    Expect.Once.On(mockViewCustomerView).SetProperty(
        "City").To(dto.City);
    Expect.Once.On(mockViewCustomerView).SetProperty(
        "Region").To(dto.Region);
    Expect.Once.On(mockViewCustomerView).SetProperty(
        "PostalCode").To(dto.PostalCode);
    Expect.Once.On(mockViewCustomerView).SetProperty(
        "Country").To(dto.CountryOfResidence.Name);
    Expect.Once.On(mockViewCustomerView).SetProperty(
        "Phone").To(dto.Phone);
    Expect.Once.On(mockViewCustomerView).SetProperty("Fax").To(dto.Fax);

    presenter.DisplayCustomerDetails();
}

As before, I am taking advantage of the NMock library to create mocks of the task and view interfaces. This particular test verifies the behavior of the presenter by asking the service layer for a DTO representing a particular customer. Once the presenter retrieves the DTO from the service layer, it will update properties on the view directly, thus eliminating the need for the view to have any knowledge of how to correctly display the information from the object. For brevity I am not going to discuss the implementation of the SelectedItem property on the WebLookupList control; instead I will leave it to you to examine the source code to see the implementation details. What this test really demonstrates is the interaction that occurs between the presenter and the view once the presenter retrieves a CustomerDTO from the service layer. If I attempt to run the test now, I will be in a big failure state as a lot of the properties don't yet exist on the view interface. So I'll go ahead and add the necessary members to the IViewCustomerView interface, as you see in Figure 12.

Figure 12 Completing the IVewCustomerView Interface (Close Figure 12)
public interface IViewCustomerView
{
    ILookupList CustomerList{get;}
    string CompanyName{set;}
    string ContactName{set;}
    string ContactTitle{set;}
    string Address{set;}
    string City{set;}
    string Region{set;}
    string PostalCode{set;}
    string Country{set;}
    string Phone{set;}
    string Fax{set;}
}

Immediately after adding these interface members, my Web Form complains because it is no longer fulfilling the contract of the interface, so I have to go back to the codebehind for my Web Form and implement the remaining members. As stated before, the entire markup for the Web page has already been created, as have the table cells which have been marked with the "runat=server" attribute and are named according to the information that should be displayed in them. This makes the resulting code to implement the interface members very trivial:

public string CompanyName
{
    set { this.companyNameLabel.InnerText = value; }
}
public string ContactName
{
    set { this.contactNameLabel.InnerText = value; }
}
...

With the setter properties implemented, there is just one thing left to do. I need a way to tell the presenter to display the information for the selected customer. Looking back at the test, you can see that the implementation of this behavior will live in the DisplayCustomerDetails method on the presenter. This method will not, however, take any arguments. When invoked, the presenter will turn back around to the view, pull from it any information it needs (which it will retrieve by using the ILookupList), and then use that information to retrieve the details about the customer in question. All that I need to do from a UI perspective is set the AutoPostBack property of the DropDownList to true, and I also need to add the following event handler hookup code to the OnInit method of the page:

protected override void OnInit(EventArgs e)
{
    base.OnInit(e);
    presenter = new ViewCustomerPresenter(this);
    this.customerDropDownList.SelectedIndexChanged += delegate
    {
        presenter.DisplayCustomerDetails();
    };
}
This event handler ensures that whenever a new customer is selected in the dropdown, the view will ask the presenter to display the details for that customer.

It is important to note that this is typical behavior. When a view asks a presenter to do something, it asks without giving any specific details, and it is up to the presenter to return to the view and get any information it needs using the view interface. Figure 13 shows the code required to implement the required behavior in the presenter.

Figure 13 Completing the Presenter (Close Figure 13)
public void DisplayCustomerDetails()
{
    int? customerId = SelectedCustomerId;
    if (customerId.HasValue)
    {
        CustomerDTO customer =
            task.GetDetailsForCustomer(customerId.Value);
        UpdateViewFrom(customer);
    }
}

private int? SelectedCustomerId
{
    get
    {
        string selectedId = view.CustomerList.SelectedItem.Value;

        if (String.IsNullOrEmpty(selectedId)) return null;

        int? id = null;

        try
        {
            id = int.Parse(selectedId.Trim());
        }
        catch (FormatException) {}

        return id;
    }
}

private void UpdateViewFrom(CustomerDTO customer)
{
    view.CompanyName = customer.CompanyName;
    view.ContactName = customer.ContactName;
    view.ContactTitle = customer.ContactTitle;
    view.Address = customer.Address;
    view.City = customer.City;
    view.Region = customer.Region;
    view.Country = customer.CountryOfResidence.Name;
    view.Phone = customer.Phone;
    view.Fax = customer.Fax;
    view.PostalCode = customer.PostalCode;
}

Hopefully you now see the value of adding the presenter layer. It is the presenter's responsibility to attempt to retrieve an ID for a customer for whom it needs to display details. This is code that would normally have been performed in the codebehind, but is now inside a class that I can fully test and exercise outside of any presentation-layer technology.

In the event that the presenter can retrieve a valid customer ID from the view, it turns to the service layer and asks for a DTO that represents the details for the customer. Once the presenter has the DTO in hand, it updates the view with the information contained in the DTO. A key point to note is the simplicity of the view interface; aside from the ILookupList interface, the view interface consists entirely of string DataTypes. It is ultimately the responsibility of the presenter to correctly convert and format the information retrieved from the DTO so that it can actually be handed to the view as a string. Although not demonstrated in this example, the presenter would also be responsible for reading information from the view and converting it to the necessary types that the service layer would expect.

With all the pieces in place I can now run the application. When the page first loads, I get a list of customers and the first customer appears (not selected) in the DropDownList. If I select a customer, a postback occurs, the interaction between the view and the presenter takes place, and the Web page is updated with the related customer information.

What's Next?

The Model View Presenter design pattern is really just a fresh take on the Model View Controller pattern that many developers are already familiar with; the key distinction is that MVP truly separates the UI from the domain/service layer of the application. Although this example was fairly simple from a requirements perspective, it should help you abstract the interaction between a UI and the other layers of your applications. Also, you should now see ways that you can use these layers of indirection to make your application more testable with automation. As you delve deeper into the MVP pattern, I hope you'll find other ways to pull as much formatting and conditional logic out of your codebehinds and place them into testable view/presenter interaction models.

Send your questions and comments to mmpatt@microsoft.com.

Jean-Paul Boodhoo is a senior .NET delivery expert for ThoughtWorks where he is involved in the delivery of enterprise-scale applications that utilize the .NET Framework and agile methods. He frequently delivers presentations on harnessing the power of .NET utilizing test-driven development. Jean-Paul can be reached at bitwisejp@gmail.com or at www.jpboodhoo.com/blog.


Service Station: Serialization in Windows Communication Foundation

Service Station
Serialization in Windows Communication Foundation
Aaron Skonnard

Code download available at: ServiceStation2006_08.exe (154KB)
Contents
Serializing and Encoding
Behind the Scenes
Defining Serialization Mapping
Working with XmlSerializer
Working with DataContractSerializer
Working with NetDataContractSerializer
Advanced Serialization Concepts
Conclusion

Windows Communication Foundation has been built from the ground up around the tenets of service orientation. It supports several serialization mechanisms that make it easy to bring existing types forward and provides a simple, interoperable foundation for future service-oriented applications. Windows® Communication Foundation (which is still in beta) embraces XML as a key enabling technology. While you can use it to build services that process XML directly, most developers prefer to leverage serialization mechanisms that automate moving between objects in the Microsoft® .NET Framework and XML Infosets.

There are two general approaches you can take when implementing a Web service. One is to embrace XML and program directly against the messages. This offers a high degree of flexibility, especially when tackling tough challenges like versioning, where core XML technologies like XPath, XSLT, and XQuery are indispensable. But many developers may find this technique tedious and overwhelming. The other approach is to predefine a mapping between .NET and XML, and then rely on automated serialization mechanisms. This hides the various XML details in order to simplify the developer experience. Windows Communication Foundation supports both approaches, with equal depth.

Internally, Windows Communication Foundation represents all messages with the Message class, which is found in System.ServiceModel.Channels. The Message class models a SOAP message, commonly referred to as a SOAP envelope, which has a header section and a body that carries the payload. The Message class provides an interface for interacting with the header section and body section using either System.Xml classes or type-based serialization.

When working at the message level, you can explicitly choose which technique to use. However, the most common way to make use of serialization in Windows Communication Foundation is to author service contracts in terms of serializable types, as shown here:

[ServiceContract]
public interface IEchoService
{
    [OperationContract]
    Person EchoPerson(Person person);
}

By annotating the .NET interface with [ServiceContract], you indicate that the .NET type definition will also serve as a service contract. Think of Web Services Description Language (WSDL). Annotating the method signature with [OperationContract] indicates that you want the method to be included in the service contract. At run time, Windows Communication Foundation automatically maps the method signature to a pair of messages behind the scenes, each containing a Person in the SOAP body. It then uses a serializer to map the Person object into the message. (For complete control over what goes where in the SOAP envelope, you can use [MessageContract] types in the signature.)

Serializing and Encoding

Windows Communication Foundation supports three serializers: XmlSerializer, DataContractSerializer, and NetDataContractSerializer. Each of these comes with different mapping algorithms and customization techniques. (See Figure 1 for a comparison.) Nevertheless, each performs the same fundamental task—mapping between .NET objects and XML Infosets.

Figure 1 Comparing Serializers (Close Figure 1)

Feature | XmlSerializer | DataContractSerializer NetDataContractSerializer
Explicitness | Opt-out | Opt-in * Opt-out **
Default mapping | Public fields/props | All [DataMember]s * All fields **
Attribute required | No | Yes
Default order | Same as class | Alphabetical
XML Schema | Extensive | Constrained
Code generator | Xsd.exe | SvcUtil.exe
Override | IXmlSerializable | ISerializable
Type fidelity | No | NetDataContractSerializer
Versioning support | No | Yes
Initialization | Constructor | Callbacks
Compatibility | ASMX | .NET Remoting
* Using [DataContract] ** Using [Serializable]

DataContractSerializer is the default and is always used unless specified otherwise. You can choose a different serializer by annotating the service contract with an attribute, as shown here:

[XmlSerializerFormat]
[ServiceContract]
public interface IEchoService
{
    [DataContractFormat]
    [OperationContract]
    Person EchoPerson(Person person);

    [OperationContract]
    Address EchoAddress(Address address);

    [OperationContract]
    Phone EchoPhone(Phone phone);
}

In this example, [XmlSerializerFormat] specifies XmlSerializer as the default serializer for all methods on the contract. However, I've overridden the serializer on EchoPerson by annotating the method with [DataContractFormat]. There isn't an attribute for NetDataContractSerializer, but you can write a custom attribute to apply it in the same way as the others if needed. The serializer is considered part of the service contract because it directly impacts your code.

Windows Communication Foundation also lets you specify the encoding to be used. Whereas serialization defines how .NET objects map to XML Infosets, the encoding defines how the XML Infoset is written out to a stream of bytes. Windows Communication Foundation currently supports the following encodings: text, binary, and Message Transmission Optimization Mechanism (MTOM). However, more encodings, including your own custom encodings, could be added down the road.

If you use each of the three encodings to write out the same XML Infoset, you'll get three very different byte streams, but they'll all represent the same logical data. The encoding isn't considered part of the service contract, but rather a configuration detail since it doesn't impact your code—you control the encoding by configuring the endpoint's binding.

The separation of serialization from encoding makes it possible to build your applications on a consistent data model (the XML Infoset) while providing flexibility in representation (the encoding). This is a key feature when applying Windows Communication Foundation to a variety of real-world scenarios. If you care about interoperability, you can choose to use the text encoding. When performance is more of a concern, you can choose to use the binary encoding. In either case, the encoding choice is decoupled from the serialization mechanism.

At first, Windows Communication Foundation serialization might remind you of ASMX. Once you start to peel off the layers, however, you'll see an amazing world of new possibilities.

Behind the Scenes

When Windows Communication Foundation creates a message to send at run time, it chooses the serializer based on the service contract and the encoding based on the binding configuration. Let's look at how this is accomplished. Figure 2 shows how to create and write a message containing a Person object.

Figure 2 Message Containing a Person Object (Close Figure 2)
static void CreateAndWritePersonBinary()
{
    Person person = new Person("Bob", 34);

    using (Message msg = Message.CreateMessage(
        MessageVersion.Soap11WSAddressing10,
        "http://example.org/person",
        person, new DataContractSerializer(typeof(Person))))
    using (FileStream fs = new FileStream("msg.bin", FileMode.Create))
    using (XmlDictionaryWriter writer =
        XmlDictionaryWriter.CreateBinaryWriter(fs))
    {
        // serialize Person object to XML
        msg.WriteMessage(writer);
    }
}

The last parameter to CreateMessage specifies the serializer to use when creating the message. The serializer controls how the Person object is added to the message body as an XML Infoset. I then create a message writer and pass it to WriteMessage. The writer determines which encoding will be used to write the message out to a byte stream. The new XmlDictionaryReader and XmlDictionaryWriter classes allow you to create readers and writers based on any of the supported encodings (text, binary, or MTOM).

For the example shown in Figure 3, I used a binary XmlDictionaryWriter. If you open the generated file in Visual Studio®, you'll see that it's binary; you won't find the angle brackets or XML tags you're used to seeing in traditional text encoding.

Figure 3 Binary Encoding
Figure 3 Binary Encoding

The code in Figure 4 shows how to accomplish the reverse–deserialize a Person from the binary file on disk. The reader supplied to CreateMessage determines which encoding to use when reading bytes from disk; in this case a binary XmlDictionaryReader. Then the call to GetBody<Person> indicates that I want to use DataContractSerializer to deserialize the XML Infoset back into a Person object.

Figure 4 Deserializing a Person (Close Figure 4)
static void ReadAndProcessPersonBinary()
{
    using (FileStream fs = File.OpenRead("msg.bin"))
    using (XmlDictionaryReader reader =
       XmlDictionaryReader.CreateBinaryReader(fs))
    {
        using (Message msg = Message.CreateMessage(
            reader, 1024, MessageVersion.Soap11WSAddressing10))
        {
            // deserialize Person object from XML
            Person person = msg.GetBody<Person>(
                new DataContractSerializer(typeof(Person)));
            Console.WriteLine("name: {0}", person.Name);
            Console.WriteLine("age: {0}", person.Age);
        }
    }
}

In cases where you prefer to avoid serialization, the Message class lets you work directly with the XML Infoset. Instead of calling GetBody<T> to deserialize the body into a .NET object, you can call GetReaderAtBodyContents and use the returned XmlReader to process the body (see Figure 5). By working directly with the XML Infoset, you can make use of your favorite XML processing technique. In Figure 5, I simply advance to the name and age elements and print their values to the console.

Figure 5 Working Directly with the XML Infoset (Close Figure 5)
static void ReadAndProcessMessage()
{
    using (XmlReader reader = XmlReader.Create("msg.xml"))
    using (Message msg = Message.CreateMessage(
        reader, 1024, MessageVersion.Soap11WSAddressing10))
    {
        XmlReader bodyReader = msg.GetReaderAtBodyContents();
        // use your XML powers here (XPath, XSLT, etc)
        if (reader.ReadToDescendant("name"))
        {
             Console.WriteLine("name: {0}",
                 reader.ReadElementContentAsString());
             Console.WriteLine("age: {0}",
                 reader.ReadElementContentAsString());
        }
    }
}

Defining Serialization Mapping

Although you can access the XML Infoset directly, a great deal of engineering effort has gone into making the serialization concepts easy and approachable throughout the programming model. The main thing you must focus on when using serialization is how the mapping works between your .NET objects and XML Infosets.

When you use .NET types in the context of serialization, there are two contracts you must consider. There's the local .NET contract, which defines the data structure along with corresponding behavior (you'll have constructors, properties, and other helpers that make the type easier to use). And then there's the external data contract, which identifies precisely what data to serialize and how to do this. Windows Communication Foundation refers to this as the data contract of a .NET type.

Since a data contract ultimately defines the structure of an XML Infoset, it's natural to represent the data contract using an XML Schema definition. In fact, XML Schema is always used to share data contracts with non-Windows Communication Foundation applications. However, data contracts are also defined in .NET type definitions. Each serializer comes with a default algorithm that defines most of the mapping details, but includes attributes that let you customize the mapping. This information is stored along with the .NET type as metadata which serializers can access at run time to determine specific mapping details.

Windows Communication Foundation provides a new command-line tool named svcutil.exe for moving between these different data contract representations (you can also use xsd.exe when working with XmlSerializer types). If you pass an XML Schema definition to svcutil.exe, it will automatically produce the corresponding .NET serializable types annotated with all the right attributes. If you pass a .NET assembly to svcutil.exe, it will automatically generate the corresponding XML Schema definitions for all serializable types (use the /dataContractOnly switch).

The result is great flexibility. You're free to define new data contracts in either XML Schema or .NET code and you can easily convert to the other. If you're working in a situation where the XML Schema definitions already exist and you need to support them, you can simply start with svcutil.exe. If you're defining new contracts and need to quickly, start by writing class definitions.

Working with XmlSerializer

As most of you know, XmlSerializer is the serializer currently used in ASMX (found in System.Xml.Serialization). The fact that Windows Communication Foundation supports XmlSerializer is good news for anyone who has made significant .NET Web services investments over the years and plans to eventually migrate them forward. You can easily use your XmlSerializer-based types in new service contracts by applying [XmlSerialzerFormat] to the service contract as shown earlier.

Let's review the basics of how XmlSerializer works. First, you instantiate XmlSerializer and specify the type you intend to serialize. Then you call the Serialize and Deserialize methods to move between instances of the .NET type and the corresponding XML Infoset. XmlSerializer defines a default data contract mapping algorithm for moving between these representations.

XmlSerializer can operate on any public type without any special attributes. With XmlSerializer, a type's public data interface maps directly to what goes in the XML Infoset. Hence, XmlSerializer automatically includes all public read/write fields and properties in the mapping, and it ignores anything that's private, protected, or so on. The .NET class name maps to the root element while the public field and property names map to local element names. The element order is the same as the order of the members in the class, with fields grouped first followed by properties. XmlSerializer does not use XML namespaces by default.

Figure 6 shows a simple class definition. Since this type is public, it can be serialized using XmlSerializer without any special attributes. But only the public read/write fields and properties will be included during the mapping, which includes the sensitiveData and spouse fields, as well as the Name and Age properties.

Figure 6 A Simple Class Definition (Close Figure 6)
public class Person
{
    private string name;
    private double age;
    public string sensitiveData;
    public Person spouse;

    public Person() { }
    public Person(string name, double age, string data)
    {
        this.name = name;
        this.age = age;
        this.sensitiveData = data;
    }

    public string Name
    {
        get { return name; } set { name = value; }
    {

    public double Age
    {
        get { return age; } set { age = value; }
    {
}

Figure 7 shows how to serialize a Person with XmlSerializer. Notice the call to Serialize also accepts an XmlWriter, which means I can specify any XmlDictionaryWriter implementation and choose the desired encoding. When this code executes, it produces the following person.xml document:

<Person>
  <sensitiveData>secret</sensitiveData>
  <spouse>
    <sensitiveData>secret</sensitiveData>
    <Name>Jane</Name>
    <Age>33</Age>
  </spouse>
  <Name>Bob</Name>
  <Age>34</Age>
</Person>

Figure 7 Serializing a Person with XmlSerializer (Close Figure 7)
static void WritePersonXmlSerializer()
{
    Person p = new Person("Bob", 34, "secret");
    p.spouse = new Person("Jane", 33, "secret");

    using (FileStream fs = new FileStream("person.xml", FileMode.Create))
    using (XmlDictionaryWriter writer =
        XmlDictionaryWriter.CreateTextWriter(fs))
    {
        XmlSerializer serializer = new XmlSerializer(typeof(Person));
        serializer.Serialize(writer, p);
    }
}

Note that the name of the type (Person) became the name of the root element while the field and property names became the local element names. Also note that the private fields, Name and Age, were not included except via the properties whose get methods are called. Fields are grouped first, followed by properties (each group is in the same order as they were defined in the class definition). The elements are not namespace qualified.

In this case, the data contract defining the actual mapping can be published as an XML Schema definition using the xsd.exe tool from the .NET Framework. The schema shown in Figure 8 accurately describes the XML instance that I produced with the serialization code shown in Figure 7 and can be shared with other parties. You can supply this same schema to xsd.exe /classes and it will generate a class definition similar to the one we started with, and it will have an equivalent data contract.

Figure 8 Schema from xsd.exe (Close Figure 8)
<xs:schema elementFormDefault="qualified"
   xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="Person" nillable="true" type="Person" />
  <xs:complexType name="Person">
    <xs:sequence>
      <xs:element minOccurs="0" name="sensitiveData"
         type="xs:string" />
      <xs:element minOccurs="0" name="spouse" type="Person" />
      <xs:element minOccurs="0" name="Name" type="xs:string" />
      <xs:element name="Age" type="xs:double" />
    </xs:sequence>
  </xs:complexType>
</xs:schema>

Figure 9 illustrates how to deserialize a Person object from person.xml. The serializer knows how to read the XML according to the data contract. During deserialization, the default constructor is called (providing an opportunity to do any necessary initialization) and the set methods are called for properties.

Figure 9 Deserializing a Person Object (Close Figure 9)
static void ReadPersonXmlSerializer()
{
    using (FileStream fs = new FileStream("person.xml", FileMode.Open))
    using (XmlDictionaryReader reader =
        XmlDictionaryReader.CreateTextReader(fs))
    {
        XmlSerializer serializer = new XmlSerializer(typeof(Person));
        Person person = (Person)serializer.Deserialize(reader);
        Console.WriteLine("Person: {0} is married to {1}",
            person.Name, person.spouse.Name);
    }
}

I didn't have to do anything special to define this two-way mapping; it was automatically derived from the type's public data interface. XmlSerializer provides a suite of customization attributes (also found in System.Xml.Serialization) that you can use to influence the mapping for a particular type. Figure 10 shows how to use some of these attributes. I've made several changes to the default mapping in this example. I specified an XML namespace to qualify the root element, modified the order of the elements, mapped the Age property to an attribute instead of an element, and told the serializer to ignore the sensitiveData field. Now, when an object of this type is serialized you'll get the following XML:

<Person Age="34" xmlns="http://example.org/person">
  <Name>Bob</Name>
  <spouse Age="33">
    <Name>Jane</Name>
  </spouse>
</Person>

Figure 10 XmlSerializer Customization Attributes (Close Figure 10)
[XmlRoot(Namespace="http://example.org/person")]
public class Person
{
    private string name;
    private double age;
    [XmlIgnore]
    public string sensitiveData;
    [XmlElement(Order=2)]
    public Person spouse;

    public Person() { }
    public Person(string name, double age)
    {
        this.name = name;
        this.age = age;
    }

    [XmlElement(Order=1)]
    public string Name
    {
        get { return name; } set { name = value; }
    }

    [XmlAttribute]
    public double Age
    {
        get { return age; } set { age = value; }
    }
}

If I generate the XML Schema again using xsd.exe, it will reflect all of these customizations. In addition, you can completely override the default mapping by implementing IXmlSerializable on the type, in which case you're defining your own mapping algorithm with custom reader/writer code.

There are a few things that make XmlSerializer unique. First, the data contract isn't explicit; it's implicitly derived from the public data interface, which may give you more than you're bargaining for (like the sensitiveData field). You have to use opt-out techniques (such as [XmlIgnore]) to address those situations. Second, XmlSerializer gives you a great deal of flexibility when it comes to XML Schema. We used an attribute instead of an element, for example. There are several more sophisticated XML Schema concepts you can apply when using XmlSerializer, but using these more advanced features can often lead to interoperability problems across frameworks.

Working with DataContractSerializer

When the Windows Communication Foundation architects contemplated the type of serializer that would make the most sense, they decided that it should use a very explicit data contract model ("boundaries are explicit") and constrain developers to a subset of XML Schema to improve interoperability results. Since XmlSerializer didn't provide either of these characteristics, the architects had to provide a new serializer. That's where DataContractSerializer comes in.

DataContractSerializer has been designed from the ground up for Windows Communication Foundation. It benefits from the lessons learned from working with XmlSerializer, and should be just as easy to use. The class is found in the System.Runtime.Serialization namespace because the implementation is quite similar to the IFormatter approach used in .NET remoting. In fact, this is meant to replace that mechanism moving forward.

The mechanics for using DataContractSerializer are similar to those for XmlSerializer. First, you instantiate DataContractSerializer and specify the type you intend to serialize. Then you call the WriteObject and ReadObject methods to move between instances of the .NET type and the corresponding XML Infoset. DataContractSerializer defines a default mapping algorithm for moving between these representations.

DataContractSerializer can operate on any .NET type annotated with either the [DataContract] or [Serializable] attributes. It supports backwards compatibility with [Serializable] types to simplify moving .NET remoting code to Windows Communication Foundation. The [Serializable] mechanism was designed to serialize the entire object by value so you can reconstitute the same object on the other side of a .NET remoting call.

In this sample, I use the same Person class as before, but without the XmlSerializer customization attributes, and I annotate the class with [Serializable]:

[Serializable]
public class Person
{
    private string name;
    private double age;
    private string sensitiveData;
    public Person spouse;
    ... // constructors and properties
}
The default mapping for [Serializable] is different from the one used with XmlSerializer. Here, all fields are included in the mapping, whether public or private, and properties are never included. The .NET class name maps to the root element, while the field names map to local element names. The elements are ordered alphabetically in the data contract and a namespace is derived from the .NET namespace in use.

The code in Figure 11 shows how to serialize a Person object with DataContractSerializer. The generated XML will now look a bit different given the differences in the [Serializable] mapping:

<Person xmlns=
  "http://schemas.datacontract.org/2004/07/DataContractSamples"
   xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
  <age>34</age>
  <name>Bob</name>
  <sensitiveData>secret</sensitiveData>
  <spouse>
    <age>33</age>
    <name>Jane</name>
    <sensitiveData>secret</sensitiveData>
    <spouse i:nil="true"/>
  </spouse>
</Person>
Notice that only fields are included, the elements are ordered alphabetically, and the element is automatically qualified with an XML namespace (this namespace is a combination of "http://schemas.datacontract.org/2004/07/" and the .NET namespace containing the type, which in this case was "DataContractSamples"). You can produce the XML Schema that describes this format by using svcutil.exe /dconly. Figure 12 shows the code used for deserializing a Person object.

Figure 12 Deserializing a Person Object (Close Figure 12)
static void ReadPersonSerializable()
{
    using (FileStream fs = File.OpenRead("person.xml"))
    using (XmlDictionaryReader reader =
       XmlDictionaryReader.CreateTextReader(fs))
    {
        DataContractSerializer serializer =
            new DataContractSerializer(typeof(Person));
        Person person = (Person)serializer.ReadObject(reader);
        Console.WriteLine("{0} is married to {1}",
            person.Name, person.spouse.Name);
    }
}
Figure 11 Serializing a Person Object (Close Figure 11)
static void WriterPersonSerializable()
{
    Person p = new Person("Bob", 34, "secret");
    p.spouse = new Person("Jane", 33, "secret");

    using (FileStream fs = new FileStream("person.xml", FileMode.Create))
    using (XmlDictionaryWriter writer =
        XmlDictionaryWriter.CreateTextWriter(fs))
    {
        DataContractSerializer serializer =
            new DataContractSerializer(typeof(Person));
        serializer.WriteObject(writer, p);
    }
}

Another difference from XmlSerializer is that constructors are not called during deserialization. However, if you need to perform initialization, DataContractSerializer supports the [OnDeserializing], [OnDeserialized], [OnSerializing], and [OnSerialized] callbacks that were also supported on the Binary/SoapFormatter classes.

The only customization you can make to the default [Serializable] mapping is to exclude a field from the data contract using the [NonSerialized] attribute, as shown here:

[Serializable]
public class Person
{
    private string name;
    private double age;
    [NonSerialized]
    private string sensitiveData;
    public Person spouse;
    ... // constructors and properties
}
However, you can completely override the mapping algorithm by implementing ISerializable on the type. In this case, DataContractSerializer will call your code to perform the mapping, giving you complete control.

DataContractSerializer also supports a more explicit mapping mechanism through the [DataContract], [DataMember], and [EnumMember] attributes. When using this approach, you annotate the type with [DataContract] to make it serializable and then you annotate any fields or properties you wish to map with [DataMember]. Similarly, you annotate the enum values you wish to map with [EnumMember]. In this scenario, you are explicitly defining the data contract—nothing maps by default.

The code that is shown in Figure 13 presents a Person class that uses the [DataContract] mapping mechanism. When I serialize an instance of this type using the same code shown in the previous example, I get the following XML:

<Person xmlns=
  "http://schemas.datacontract.org/2004/07/DataContractSamples"
   xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
  <Age>34</Age>
  <name>Bob</name>
  <spouse>
    <Age>33</Age>
    <name>Jane</name>
    <spouse i:nil="true"/>
  </spouse>
</Person>
The mapping details are identical to [Serializable]. The only difference is how I indicated what should be included in the data contract. With [Serializable], all fields become part of the data contract (unless they are marked with [NonSerialized]). With [DataContract], only members marked with [DataMember] are included. Note that if a type has both [DataContract] and [Serializable] attributes on it, it will use the [DataContract] mapping.

Figure 13 Using [DataContract] for Mapping (Close Figure 13)
[DataContract]
public class Person
{
    [DataMember]
    private string name;
    private double age;
    private string sensitiveData;
    [DataMember]
    public Person spouse;

    ... // constructors and properties
    [DataMember]
    public double Age
    {
        get { return age; } set { age = value; }
    }
}

The [DataContract] mapping is more customizable than that offered by [Serializable], but it's also more constrained than XmlSerializer. The code in Figure 14 shows how to use a few of the customization properties found on [DataContract] and [DataMember]. In this example, I've customized the XML namespace, some of the element names, and the element order. Setting IsRequired=true causes the serializer to verify that the element is present during deserialization (and it shows up as minOccurs="1" in the schema). Here's what the resulting XML looks like:

<Person xmlns="http://example.org/person"
    xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
  <Name>Bob</Name>
  <Age>34</Age>
  <Spouse>
    <Name>Jane</Name>
    <Age>33</Age>
    <Spouse i:nil="true"/>
  </Spouse>
</Person>

Figure 14 Using Customization Properties (Close Figure 14)
[DataContract(Namespace="http://example.org/person")]
public class Person
{
    [DataMember(Name = "Name", Order = 1, IsRequired = true)]
    private string name;
    private double age;
    private string sensitiveData;
    [DataMember(Name="Spouse", Order = 3)]
    public Person spouse;

    ... // constructors and properties
    [DataMember(Order=2, IsRequired=true)]
    public double Age
    {
        get { return age; }
        set { age = value; }
    }
}

More sophisticated schema customizations are not possible via attribution. You cannot, for example, change elements into attributes or change from sequence to choice compositors. If you want complete control over the XML mapping, you can avoid the [DataContract] attributes altogether and implement IXmlSerializable or ISerializable (the former takes precedence over the latter). This approach, however, requires more work.

When you are using DataContractSerializer, you can only work with XML Schema definitions that meet the [DataContract] mapping constraints. If you pass a more complex XML Schema to svcutil.exe, it will warn you when it doesn't meet the [DataContract] constraints and encourage you to use xsd.exe /XmlSerializer.

Despite its constraints, [DataMember] makes it possible to implement some interesting data versioning heuristics via both the Order parameter and the IsRequired parameter. (Basically, doing this allows you to safely add new fields in future versions of the contract.) You can also implement IExtensibleData on your data contracts to enable the serializer to round-trip unrecognized data during serialization—when, for instance, you need to pass a new version of a message to an old implementation.

You always have to provide DataContractSerializer with type information before it can serialize/deserialize instances. This is different from .NET remoting, where type information is identified at run time and serialized into the message in order to provide type fidelity across the wire. When you need support for this type of scenario, you should use NetDataContractSerializer.

Working with NetDataContractSerializer

The major difference between DataContractSerializer and NetDataContractSerializer is that the latter serializes .NET type information into the XML. NetDataContractSerializer supports [Serializable] and [DataContract], the same mapping algorithms, and the same customization attributes. However, when you use NetDataContractSerializer, you no longer have to supply type information ahead of time, as illustrated here:

NetDataContractSerializer serializer =
    new NetDataContractSerializer();  // no type specified
serializer.WriteObject(writer, p);
The generated XML contains the .NET type information in attributes placed on the root element (z:Type and z:Assembly) as illustrated here:
<Person z:Id="1" z:Type="DataContractSamples.Person"
    z:Assembly="DataContractSamples, Version=1.0.0.0, Culture=neutral,
                PublicKeyToken=null" xmlns="http://example.org/person"
    xmlns:i="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:z="http://schemas.microsoft.com/2003/10/Serialization/">
  <Name z:Id="2">Bob</Name>
  <Age>34</Age>
  <Spouse z:Id="3">
    <Name z:Id="4">Jane</Name>
    <Age>33</Age>
    <Spouse i:nil="true"/>
  </Spouse>
</Person>
Since the .NET type information is found in the XML, you don't need to specify the .NET type when deserializing either.

Microsoft has provided this ability specifically to address situations where you need type fidelity across the wire. But for the most part, Windows Communication Foundation encourages developers to embrace the more explicit DataContractSerializer approach. This is why you can't enable NetDataContractSerializer on a service contract without writing your own behavior/attribute.

Advanced Serialization Concepts

When using DataContractSerializer or NetDataContractSerializer, there are a few concepts to consider. First, when serializing types that use inheritance, make sure all types in the hierarchy are serializable (either via [DataContract] or [Serializable]). If they aren't, you'll get exceptions at run time. Likewise, you must ensure that any contained types are also serializable.

If you find yourself stuck in a situation where you cannot make some of the types serializable, you can use what is known as a serialization surrogate. This is basically a serializable type that can take the place of the non-serializable type without requiring you to modify the original class definition. You do this by implementing the IDataContractSurrogate and handing it to the serializer. The serializer will call your surrogate during the serialization process, at which point you can replace the non-serializable object with a different object.

The Windows Communication Foundation serializers also provide a way to specify known type substitutions that you want the serializers to recognize at run time. This lets you type a serializable field as some base type and actually serialize different derived types at run time. This is done by annotating base types with the [KnownType] attribute (similar to how [XmlInclude] works in XmlSerializer). There is also [ServiceKnownType] for use on service contract definitions.

These new serializers can also maintain object references and deal with issues like circular references (see the constructor with a preserveObjectReferences flag). This is a nice improvement over XmlSerializer, which choked on such object graphs. I can, for example, serialize a circular spouse reference in the previous examples. I've included such an example in the downloadable sample code.

Conclusion

Windows Communication Foundation is built on a consistent data model (XML Infoset) and flexible representations (encodings). When working with it you can choose to work with the XML directly or you can use the built-in serialization techniques to automate mappings. You'll rarely need to program against the serializer classes directly since you can simply annotate your service contracts to choose a serializer. It's important, however, to understand how each serializer works.

It's also important to understand which serializer is best for a given scenario. You should use DataContractSerializer whenever possible in Windows Communication Foundation. Its constrained mapping improves interoperability when starting with code. NetDataContractSerializer should only be used when you absolutely need type fidelity across the wire. Alternatively, you should use XmlSerializer when you need backwards compatibility with ASMX types, you need more flexibility in the XML representations, or you're starting with existing schemas that don't meet the [DataContract] mapping constraints.

Send your questions and comments for Aaron to sstation@microsoft.com.

Aaron Skonnard is a cofounder of Pluralsight, a Microsoft .NET training provider. Aaron is the author of Pluralsight's Applied Web Services 2.0, Applied BizTalk Server 2006, and Introducing Windows Communication Foundation courses. Aaron has spent years developing courses, speaking at conferences, and teaching professional developers. Reach him at pluralsight.com/aaron.


Security Briefs: Security in Windows Communication Foundation

Security Briefs
Security in Windows Communication Foundation
Keith Brown

Contents
What CIA and Authorization Can Do for You
Choices in Windows Communication Foundation
Transport Security
Message Security
Credentials
Default Security in Standard Bindings
Discovering Client Identity
Installing a Certificate with HTTPCFG
Winding Up

W indows Communication Foundation performs a lot of the heavy lifting to make it easier for your service to provide the basic security features that most distributed systems need. The big three protections— confidentiality, integrity, and authentication (or CIA as I like to think of them) are provided by most standard Windows® Communication Foundation bindings. If you don't want these protections, you'll have to turn them off, because they are on by default.

What CIA and Authorization Can Do for You

So what's the role of each of these protections? Well, confidentiality makes sure that messages are encrypted so that an eavesdropper cannot read the contents of the messages. Integrity ensures that you use a keyed hash to perform a checksum on the contents of each message so you know that the message hasn't been tampered with since it was produced, or injected wholesale into the message stream by an attacker. And of course it makes no sense at all to encrypt a connection if the guy on the other end is the enemy! So in order to exchange the keys used for providing confidentiality and integrity, Windows Communication Foundation uses an authentication handshake, which not only helps the client and service discover these keys, but also helps them discover each other's identities.

Once the service knows the client's identity, it can then authorize the client to perform various operations. Authorization is performed by checking claims that the client has provided. Windows Communication Foundation provides a number of extensibility hooks that allow you to wire in the authorization policy of your choice and check the validity of the claims. You can use existing components like the ASP.NET membership and role providers and Authorization Manager (AzMan), which you can learn more about at msdn.microsoft.com/msdnmag/issues/03/11/AuthorizationManager. You can also use the standard .NET role-based security infrastructure with Thread.CurrentPrincipal, IPrincipal.IsInRole, and PrincipalPermissionAttribute. Or you can use the authorization policy framework in Windows Communication Foundation, centered around the IAuthorizationPolicy interface that provides fine grained control over the claims. You can also audit the actions of the client. You'll find that Windows Communication Foundation already audits many security events for you right out of the box.

Transparency is important when you're building a secure system. If the security features you build into the system aren't relatively transparent, users will be put off and will generally try to avoid using them. It's sad but true that as a security guy, my best work is always invisible! Windows Communication Foundation helps you achieve transparency in many ways, but one of the most important is through single sign-on. By tightly wiring into Windows domain authentication, you can leverage the user's domain login and reduce the number of credentials he needs to keep track of. Windows Communication Foundation also makes it easy to use federated identity, which allows partner organizations to have the same single sign-on experience even when they use Web services that your company exposes.

Another important aspect of security is robustness in the face of malformed input. Windows Communication Foundation can't really help with this, so it's still your responsibility to assume all input is evil until proven otherwise. And don't assume that an authenticated user won't send you malformed input, either! Even if you only service internal clients, you must remain robust. What I'm talking about here is resilience to attacks such as SQL injection, improperly formed resource names such as file paths, and, in some cases, even cross site scripting. When you hear people say, "Windows Communication Foundation is secure by default," understand that they are probably talking about CIA.

Choices in Windows Communication Foundation

Windows Communication Foundation pulls together the best features of the numerous communication stacks we used individually in the past. This means you have many choices to make when you expose a service, and some of these choices impact security. The biggest choice involves what binding to use. Do you want messages to be queued? Do you want interoperability or raw performance?

Always keep in mind that you don't need to make just one choice. You can expose a single service contract over as many bindings as you like; just expose a different endpoint for each binding. For example, you might expose a very simple WS-I basic profile binding over HTTPS to interoperate with existing partners in a supply chain. At the same time, you could expose a TCP endpoint that uses Windows authentication for local clients in your domain. This only works, however, if the user does not specify the ProtectionLevel property in the contract (using ServiceContract, OperationContract, MessageHeader, and so forth).

A typical standard binding has a lot of different knobs for adjusting security settings, but each binding has been carefully tuned to provide secure default settings for most of the scenarios you'll encounter. The two knobs you'll most often need to toggle are the security mode (transport versus message security) and the client credential type. These knobs are really important, so let's look at the right way to turn them.

Transport Security

When you're configuring security for a Windows Communication Foundation binding, the first choice you'll need to make is whether you want CIA to be provided at the transport level or at the message level. There are pros and cons to each choice.

If you configure wsHttpBinding for transport security, Windows Communication Foundation will not supply CIA for your messages. You're expected to run over HTTPS so that secure sockets layer (SSL) can provide these guarantees. In fact, Windows Communication Foundation will refuse to set up the connection unless an HTTPS URI is provided. If you're hosting inside IIS, you'll need to install an SSL certificate for the Web site. On the other hand, if you're hosting in your own process and using HTTP.SYS directly, you'll need to register a certificate with HTTP.SYS, either programmatically or via the command-line tool, HTTPCFG.EXE.

There are good reasons to use SSL. Because it's been around for so many years, SSL has been the subject of much analysis by cryptographers, and administrators have already figured out how to deploy it. There is hardware out there to accelerate it, reducing the load on your server's CPU. Best of all, if broad reach across platforms is your goal, there's no better choice today than using the WS-I basic profile over SSL (which of course would also mean using basicHttpBinding instead of wsHttpBinding).

For internal communication in a domain environment where both client and service use Windows Communication Foundation, netTcpBinding with transport security might be a better choice. It certainly performs a lot better given its binary encoding, and since it uses Kerberos exclusively by default, there's no need for certificates, although you will want to configure a service principal name (see the "Configuring a Service Principal Name" sidebar).

Message Security

Instead of providing CIA at the transport level, you can choose to push these details down into the SOAP messages themselves by using message security. The main benefit here is flexibility. The WS-Security and related specs for securing SOAP messages are very flexible; you can use any type of security credentials you want, largely independent of transport, as long as both client and service agree. In fact, you could even send multiple credentials, which becomes interesting in delegation scenarios where intermediaries are involved.

Take wsHttpBinding as an example. By default it uses message-level security, assuming the service and the client will identify and authenticate themselves using Windows credentials. By default, the body and most headers are signed to maintain the integrity of the message (specifically, it signs all WS-Addressing headers, and it encrypts and signs all application headers), and the body is encrypted. An attacker looking at a message trace would see the SOAP envelope in the clear, but the SOAP body would contain an <EncryptedData> element full of base64-encoded ciphertext.

Windows Communication Foundation also allows you to mix transport and message security. In this mode, confidentiality and integrity and the server's authentication are provided at the transport level (therefore the entire bytestream is encrypted, not just the message body), whereas client authentication is performed at the message level. The client can then use WS-Security to send any shape of credential that she wants. This mode gives you the performance and maturity of transport security with increased flexibility for choosing credentials.

Credentials

Once you've chosen either transport or message security (or mixed mode), and assuming you've not chosen a security mode of None, the next major decision is the shape of credential that the client and service will use. Windows Communication Foundation simplifies this by giving you only one knob, the client credential type, which also determines the service credential type that makes sense in each scenario. There are at least five options for client credentials, although some options may not be available in certain contexts.

The first option is None, where the client is anonymous. In this case the service uses a certificate to assure the client of the service's identity. This is similar to what you get when you visit a typical Web site over HTTPS.

The second option is UserName, where the client supplies a user name and password. Again, in this case the service uses a certificate to identify itself to the client. This certificate will also be used to encrypt the client's credentials for transmission to the service.

The next option is Windows, where both the client and service use a Windows account to authenticate. Windows Communication Foundation will negotiate Kerberos or NTLM, preferring Kerberos if a domain is present (NTLM does not actually authenticate the service to the client, only the client to the service). If you want to use Kerberos it's important to have the client identify the service with a service principal name (SPN) in config. If you're building a service for clients in a domain environment, you should definitely give them the option of sending a Windows credential. This allows them to use their existing Windows logon, giving you single sign-on for free!

Certificate is another option, where the service has a certificate and the client also has a certificate of her own. This is typical in many business-to-business scenarios today.Configuring a Service Principal Name

Configuring a Service Principal Name (SPN) for a service is easy. You can do it programmatically, or you can use a tool called SETSPN.EXE, which, like HTTPCFG.EXE, is also part of the support tools included with server versions of the operating system. The idea behind an SPN is straightforward: it simply decouples your client code from having to know exactly which account the service will be deployed under. By giving the client an SPN (which is just a string), when the client requests a ticket for the service account, she can request the ticket based on the SPN rather than the service account name.

Then an administrator can map that SPN onto the actual account name being used for the service using a tool like SETSPN. Here’s how it’s done:

setspn -A MyService/MyMachine MyDomain\ServiceAccount

This adds an SPN to the MyDomain\ServiceAccount user account in Active Directory. In this example the SPN is MyService/MyMachine. Now clients wanting to talk to the service called MyService on a machine called MyMachine can ask for a Kerberos ticket using an SPN formed from this information. It’s all about decoupling the client from the details of how the service is deployed.

If the administrator wants to run the service under a different account later, he’ll remove the old SPN mapping and add a new one, and clients will continue to function properly:

setspn -D MyService/MyMachine MyDomain\ServiceAccount
setspn -A MyService/MyMachine MyDomain\OtherAccount

Finally, IssuedToken allows your service to accept a signed set of claims from a security token service (STS). While this might feel alien to you at first, this option is going to be important going forward as it enables federated identity scenarios and InfoCard. When you federate with a partner organization, you allow the partner to authenticate its own users via whatever technique makes sense for them. In the best case, this will allow users in the partner organization to use your service with single sign-on, even though they don't share an Active Directory domain or trust with you. The users in the partner organization are required to authenticate with an STS, which issues a signed Security Assertion Markup Language (SAML) token. You can either accept that token directly, or require that the token be presented to an STS in your own organization so it can evaluate the partner's claims and issue a second SAML token that you can use.

In either case, your service will receive a security token issued by an STS that you trust. Windows Communication Foundation will verify the signature and this trust and present the claims in the token to your service via an instance of ServiceSecurityContext. Your service can access this object while processing a request by simply reading the ServiceSecurityContext.Current static property. You can then use this information in order to perform both authorization and auditing.

The beauty of federation is that it automates existing trust relationships between partners: there's no need to issue shadow Windows accounts or certificates to each user in the partner organization when you federate. There are a lot of benefits to automating these relationships, too. Automation tends to reduce mistakes and latency, which means you're getting more accurate and up-to-date information about users in the partner organization. It also reduces cost for your IT staff because they don't need to provision user accounts for partners. Federated identity is an important topic that I'll spend more time on in a future column.

Default Security in Standard Bindings

The three most popular standard bindings are basicHttpBinding, wsHttpBinding, and netTcpBinding. The simplest and most interoperable is basicHttpBinding, which supports the WS-I basic profile. It's important to note that this particular binding doesn't provide CIA by default like most of the others. The most popular way to secure this binding is by simply running over HTTPS, due to the ubiquity of SSL. To do this, you'll need to tweak the binding to let Windows Communication Foundation know you'll be using transport security:

<bindings>
  <basicHttpBinding>
    <binding name="MyBindingTweaks">
      <security mode="Transport">
        <transport clientCredentialType="None"/>
      </security>
    </binding>
  </basicHttpBinding>
</bindings>

Once you've done this, the easiest way to deploy is to host in IIS and turn on SSL support for your virtual directory (which will mean installing a certificate for the Web site if you've not already done so). Note that in this particular binding, I've allowed anonymous clients. If you're deploying in IIS and plan to require client certificates, change the clientCredentialType to Certificate.

The next binding, wsHttpBinding, uses message security by default, with WS-Security and WS-SecureConversation. The default client credential type is Windows, which makes it really easy to get started with; it simply uses the Windows logon of the client and service as credentials. One of the most common security tweaks to use on this binding is to switch it to use TransportWithMessageCredential. Here you'll use an HTTPS endpoint to provide accelerated server authentication, integrity, and confidentiality, while the client credential remains in the SOAP Security header for flexibility. One benefit of this technique is that the entire SOAP envelope is encrypted by the transport, including all the headers. It also lets you use hardware acceleration to offload Secure Sockets Layer/Transport Layer Security (SSL/TLS) from your server's CPU. There are some disadvantages, however, such as the lack of end-to-end security at the message level. The following shows how you'd apply this binding tweak to accept a SAML token at the message level, for example:

<binding name="MyBindingTweaks">
  <security mode="TransportWithMessageCredential">
    <transport clientCredentialType="None"/>
    <message clientCredentialType="IssuedToken"/>
  </security>
</binding>

If you want raw speed for Web services on a Windows-based intranet, you should seriously consider using netTcpBinding, which encodes each SOAP envelope using a proprietary binary encoding of the XML Infoset instead of the traditional angle bracket encoding. By default this binding uses transport security with Windows credentials, and is very efficient. This is probably the closest you'll come to matching the performance and security model of DCOM. The default binding uses transport security with negotiated authentication. This negotiation attempts to use Kerberos, but if that doesn't work, it'll fall back and use the older NTLM protocol. Kerberos is a great choice if you're in a domain environment; in order to use it, you'll need both your service and clients to be running under domain accounts. You'll also want to configure a service principal name (SPN) for your service, as shown in the sidebar on this page. Clients will need to tweak their endpoint descriptions to include the identity of the service, which is the SPN the service registers. (This is not always required. If the service is hosted in IIS and is running under the NetworkService account, the SPN is provided automatically. Windows Communication Foundation will infer it correctly from the URL on the client side.) Here's the syntax for doing that in config. Note the servicePrincipalName element:

<client>
  <endpoint name="MyEndpoint" address="net.tcp://..."
    binding="netTcpBinding" contract="IFoo" >
    <identity>
      <servicePrincipalName value='MyService/MyMachine' />
    </identity>
  </endpoint>
</client>

When the client's Windows Communication Foundation plumbing asks the domain controller for a Kerberos ticket for the service, it'll use the name MyService/MyMachine to request the ticket, and as long as this SPN has been registered in Active Directory for the service's domain account, the client will be issued a ticket and Kerberos will be used. Otherwise, if Kerberos is available but the SPN is incorrect, an exception will be thrown. And if Kerberos is not available for the authentication to the service, the client will fail to get a ticket (which simply wastes a round-trip to the domain controller) and you'll head into NTLM fallback mode, which means even more round-trips for the fallback negotiation. Also, you'll miss out on features of Kerberos such as the potential to delegate client credentials.

Discovering Client Identity

By far the simplest way to discover a client's identity is to leverage Thread.CurrentPrincipal, which is the standard way in .NET that plumbing such as ASP.NET and Windows Communication Foundation communicate the identity of an authenticated client to your code. This rather magical static property will return different values depending on the thread that requests it. This allows you to have multiple threads servicing different clients, where each will see the identity of the client they are servicing without stepping on one another.

If you've specified Windows as your clientCredentialType, then Thread.CurrentPrincipal will point to a WindowsPrincipal, and you can see which groups the user belongs to by calling WindowsPrincipal.IsInRole. The only glitch here is that for security reasons you really should specify the fully qualified group name, which includes the domain or machine on which the group is defined, as shown in the following:

IPrincipal client = Thread.CurrentPrincipal;
if (client.IsInRole(@"MyMachine\Managers")) {
    // allow client to do manager stuff
}

When I'm building systems that authorize based on groups like this, I'll usually add some level of indirection that supplies the machine or domain name for the groups I'm using by looking in config. That simplifies deployment quite a bit.

I'll speak more about authorization in a later column—there's a lot to say about that subject. Note that Windows Communication Foundation doesn't always set Thread.CurrentPrincipal. It does that only if PrincipalPermissionAttribute is used or if the configuration says it should. Instead of relying on Thread.CurrentPrincipal, you should use ServiceSecurityContext.Current.WindowsIdentity to get the client's identity if one is available. Here's how you get the client's name in case you want to log what the client is doing:

string clientAccountName =
    ServiceSecurityContext.Current.WindowsIdentity.Name;

If the client is using an issued token credential to authenticate, you'll need to use ServiceSecurityContext.AuthorizationContext to pick up those details. If the client is authenticating with a certificate not mapped to a Windows account, ServiceSecurityContext.Current.PrimaryIdentity can be used. I plan to drill into these topics further in the future.

Installing a Certificate with HTTPCFG

If you want to use SSL but you're not going to host your service in IIS, you can use HTTP.SYS to provide SSL support. In order to use the SSL support in HTTP.SYS, you'll need to tell HTTP.SYS where to find its SSL certificate, and you can use the HTTP API to do this, or use HTTPCFG.EXE to call those APIs from the command line. HTTPCFG.EXE isn't installed by default, but you can get it by installing the support tools found in the SUPPORT\TOOLS directory on your operating system installation disk.

If you want to use a certificate with HTTP.SYS, you'll need to install it in the local computer store, not a user store. You can do this by running the Certificates Microsoft® Management Console (MMC), snap-in; when you add the snap-in just be sure to choose Computer Account when asked, then choose the computer where the service is deployed, or Local Computer if you're already logged into the computer where the service is deployed. Then import the certificate into the Personal store for that machine. The Personal store is where you normally put certificates for which you have a private key.

View the properties of the certificate and get the SHA-1 thumbprint of the certificate (see Figure 1). This is how HTTP.SYS looks up certificates, so you'll need this value. Now, from a command prompt running on the machine where the service is deployed, here's how you register the certificate with HTTP.SYS:

HTTPCFG add ssl -i 0.0.0.0:4242 31883a61892d8c...

Figure 1 Certificate Thumbprint
Figure 1 Certificate Thumbprint

The first number you see is an IP address of 0.0.0.0, which represents all IP addresses that the machine can potentially have. If you're binding to a particular IP address you should use that value instead. Next is the port number on which the service will listen. And last is the thumbprint (you'll need to remove the spaces between the bytes if you copy this from the certificate viewer). I've only shown the first part of the hash for brevity, as it is 20 bytes long.

Winding Up

This is only the first of many pieces that I intend to write about security in Windows Communication Foundation. Here I introduced the security features and illustrated some of the basic choices you'll need to make when deciding how to secure services that you write. As you've seen, Windows Communication Foundation provides message confidentiality, integrity, and authentication over many different bindings, with the vast majority of the bindings being secured by default. The tips I've presented here on tweaking your bindings will help you get the most out of these features. In the future I'll explore other areas of Windows Communication Foundation security, including claims-based authorization and other fun topics. Stay tuned!

Send your questions and comments for Keith to briefs@microsoft.com.

Keith Brown is a cofounder of Pluralsight, a premier Microsoft .NET training provider. Keith is the author of Pluralsight's Applied .NET Security course, as well as several books, including The .NET Developer's Guide to Windows Security, which is available both in print and on the Web. Learn more at www.pluralsight.com/keith.


Netting C++: Resource Cleanup

Netting C++
Resource Cleanup
Stanley B. Lippman

Code download available at: NettingC++2006_08.exe (166KB)

In my last column, we successfully wrapped my native Text Query Language (TQL) application for processing natural language texts. That is, the application executed correctly, but I did make a number of subtle program design errors that I’d like to correct in this column. For those who haven’t incised the code in memory, Figure 1 shows how we left it.

Figure 1 Initial TQL Reference Class (Close Figure 1)
#include "TextQuery.h"

ref class TQL {
public:
    // constructor creates a TextQuery object on native heap
    // destructor frees it ...
    TQL()  { pTQuery = new TextQuery; }
    ~TQL() { delete pTQuery;          }

    // the operations we wish to publish within .NET
    // these will generally be inlined by the compiler

    void build_up_text(){ pTQuery->build_up_text(); }
    void query_text()   { pTQuery->query_text();    }

private:
      // our native object through which to invoke
      // our application compiled in Section 1 ...
    TextQuery *pTQuery;
};

// Our Managed TQL Wrapper Put to Work ...
int main()
{
    TQL ^tq = gcnew TQL;

    tq->build_up_text();
    tq->query_text();

      return 0;
}

The first problem is pretty trivial, but it would be embarrassing should the class be made generally available because all types within a module default to internal scope. That is, they can’t be seen from outside the module. This is a good default because it prevents the global namespace of the application from being polluted by module-only types. You must explicitly declare as public all classes you want to make visible across modules:

public ref class TQL {
public:
    // no change ...
};

This also holds for native classes you compile under C++/CLI, which is the opposite of C++ visibility of native types. So if you need your native class to be public across modules, you’ll also need to label that class public. This is an extension to ISO-C++ in Visual C++® 2005.

The second problem involves cleaning up resources other than memory from the common language runtime (CLR) heap, which is garbage collected. A garbage-collected heap has deep ramifications for destructors, and that’s where we’ll start. Under the Microsoft® .NET Framework, there is no notion of a destructor, so we’ve artificially mapped destruction to .NET, as I’ll explain shortly.

First, remember that destruction is a two-phase operation. When you delete a pointer, for example, internally the compiler first checks to see that the pointer is not null. If it is, nothing else happens. Otherwise, the compiler invokes the destructor associated with the object being addressed. If that concludes without an exception, the actual heap memory is freed. For CLR types, this two-phase operation is carried out by two different components. The invocation of the destructor is carried out by the compiler. The actual freeing of the memory is always handled by the garbage collector. When you invoke a destructor in .NET, the actual memory associated with the reference object is not yet reclaimed and won’t be until the garbage collector kicks in.

So, the problem with my initial implementation is that while each TQL object will eventually be reclaimed by the garbage collector, the second phase of destruction, it will not have the associated destructor invoked, the first and (from my point of view) more important phase of destruction. Before the memory associated with an object is reclaimed by the garbage collector, an associated Finalize method, if present, is invoked. Think of this method as a kind of super-destructor since it is not tied to the program lifetime of the object. This is referred to as finalization. The timing of when or whether a Finalize method is invoked remains undefined, meaning that garbage collection exhibits nondeterministic finalization.

Nondeterministic finalization works well with dynamic memory management. When available memory becomes sufficiently scarce, the garbage collector kicks in and things pretty much just work. Under a garbage-collected environment, destructors to free memory are unnecessary.

Nondeterministic finalization does not work well, however, when an object maintains a critical resource such as a database connection, native heap memory, or a lock of some sort. In such cases, you need to release that resource as soon as possible. In the native world, that is done through the pairing of a constructor and a destructor. As soon as the lifetime of the object ends, either through the completion of the local block within which it is declared or through the unraveling of the stack because of a thrown exception, the destructor kicks in and the resource is automatically released. This approach works very well, but was unfortunately missing in the CLR world.

It soon became clear that programmers using .NET needed a canonical way of indicating that the potentially scarce resources held by objects of a type need to be quickly disposed of, and the design solution is the System::IDisposable interface with a single Dispose method that contains the clean-up code. The primary drawback of this solution is that Dispose requires an explicit invocation by the user. This is potentially error-prone and therefore a step backwards. The C# language provides a modest form of automation through a special using statement which, when followed correctly, generates the call to Dispose for the objects within the statement.

Beginning with Visual Studio® 2005, Visual C++ instead translates the class destructor into the class Dispose method. The destructor is renamed internally to the Dispose method and the reference class is automatically extended to implement the IDispose interface:

// internal transformation of destructor under C++/CLI
public ref class TQL : IDisposable
{
...
void Dispose() {
    // suppress finalize method for this object
    // then generate the user code ...
    System::GC::SuppressFinalize(this);
    delete pTQuery;
}
};

When either a destructor is invoked explicitly under C++/CLI or when delete is applied to a tracking handle, the underlying Dispose method is invoked automatically. If it is a derived class, a call of the Dispose method of the base class is inserted at the close of the synthesized method.

While this transformation is a good thing, by itself it’s not good enough. First, reference objects don’t have scope constraints, so without explicitly having the programmer delete the reference object, the destructor doesn’t get invoked. Second, since the destructor now goes to Dispose rather than Finalize, the garbage collector is without any method to invoke. So, at first glance, this design change seems to have been a mistake!

It is certainly the mistake I made in the implementation. As a native programmer, I didn’t realize that a destructor under .NET isn’t a complete solution to managing objects, due to the nature of garbage collection. Therefore, I didn’t think to explicitly provide a finalization function for the garbage collector to invoke. For just one object, probably nobody would notice. In a running system spawning a new TQL object for every query session, it would be quite an embarrassment. For your convenience, I redisplay the declaration of the TQL object:

// the tq object never invokes the destructor ...
// and we have not provided a finalize method ...
// so the native memory held by tq is never freed ...
int main()
{
    TQL ^tq = gcnew TQL;

    tq->build_up_text();
    tq->query_text();

    return 0;
}

You need to provide a finalizer, and I’ll show you how in a minute. First let’s walk through how the CLR features in Visual C++ simulate deterministic finalization—by syntactically binding a reference object to a local or class scope; each represents a deterministic lifetime. The tricky part is that .NET itself doesn’t support this, and so we had to be clever.

Visual C++ supports the declaration of an object of a reference class on the local stack or as a member of a class by declaring the object using the type name but without requiring its formal top hat (^). All uses of the object, such as invoking a member function, are done through the member selection dot (.) rather than arrow (->). At the end of the block, the associated destructor, transformed into Dispose, is invoked automatically:

// OK, this invokes our destructor ...
int main()
{
    TQL tq;

    tq.build_up_text();
    tq.query_text();

    // destructor is invoked here ...

    return 0;
}
For those with more of a library bent to your syntax there is an auto-handle<> template that functions equivalently. (I prefer going hatless in my designs.) As with the using statement within C#, this is syntactic sugar rather than defiance of the underlying .NET constraint that all reference types must be allocated on the CLR heap. The underlying semantics remain unchanged, except that the invocation of the destructor is automatic. In effect, the destructor is again paired with constructors as an automated acquisition/release mechanism tied to an object’s lifetime.

The problem with this solution is that you can’t force programmers to use it and, therefore, you can’t safely provide a destructor without providing a finalizer as well. Here’s how to do that:

public ref class TQL {
public:
    // constructor creates a TextQuery object on native heap
    TQL()  { pTQuery = new TextQuery; }

    // destructor frees it
    ~TQL() { delete pTQuery;          }

    // finalizer frees it, called by garbage collector if
    // destructor is not invoked ...
    !TQL() { delete pTQuery;          }
};
The ! prefix is meant to suggest the analogous tilde (~) that introduces a class destructor—that is, both post-lifetime methods have a token prefixing the name of the class. If the synthesized Finalize method occurs within a derived class, an invocation of the base class Finalize method is inserted at its end. If the destructor is explicitly invoked, the finalizer is suppressed.

There are some remaining issues, as always, with these sorts of language designs. A finalizer, in general, is inefficient. (For a great discussion of this issue, see the book CLR via C#, Second Edition, by Jeffrey Richter.) When possible, it’s preferable not to define one. But currently there is no way to restrict a class containing a destructor from always having its objects defined such that deterministic finalization is guaranteed. I can’t force users of my class to always declare local reference objects:

void f()
{
    TQL t;     // ok, guaranteed to be disposed
    TQL ^ht;   // oops. No guarantee. Need a finalizer, then ...
    ...
}

One possibility would be to play with the accessibility of the finalizer to indicate whether you want to allow users to invoke it. That is, placing a finalizer in a private section would indicate that you are disallowing the use of objects outside of bound scope. However, in the current extensions design, all finalizers are public, regardless of the accessibility the user specifies.

Therefore, the finalizer always seems to be necessary as a failsafe companion definition to a reference class just as, canonically, you generally need to pair a copy constructor with a copy assignment operator or an operator new with an operator delete.

The other remaining issue, admittedly trivial, is that the code for the destructor and finalizer tends to be the same and, therefore, it is vexing to some of us to have to either duplicate the code or fret over how it should canonically be designed. Michael Vanier, a programming language professor at CalTech, suggested a ~! syntax to indicate to the compiler that it should use this code to support both destructor and finalization. I really like that idea, and perhaps the ECMA C++/CLI committee will too in some future revision!

So, for a small piece of code, there were quite a few mistakes. Why? I think it’s because these issues—visibility of types, nondeterministic finalization—don’t exist in native programming and, therefore, represent leaks in our thinking about .NET. Next time we’ll look at regular expressions—in the quest to turn verbose C++/CLI into pithy Perl. Until then, may your code compile without error and execute to completion swiftly and correctly.

Send your questions and comments for Stanley to purecpp@microsoft.com.

Stanley B. Lippman began working on C++ with its inventor, Bjarne Stroustrup, in 1984 at Bell Laboratories. Later, Stan worked in feature animation both at Disney and DreamWorks and served as a Software Technical Director on Fantasia 2000. He has since served as Distinguished Consultant with JPL, and an Architect with the Visual C++ team at Microsoft. Thanks to Jim Hogg and Michael Vanier for their help with this column.


C++ at Work: Addin a Combobox Cancel Feature

C++ At Work
Addin a Combobox Cancel Feature
Paul DiLascia

Code download available at: C++atWork2006_08.exe (193KB)

Q I have a dialog with a combobox. If I press the Cancel button the dialog closes immediately, although the focus is on the combobox. However, if I start auto-completion by typing a letter or opening the dropdown list, and then press the Cancel button, the dropdown closes but the dialog box does not. This is somewhat nasty for the user. In Microsoft® Word, the file open menu manages to close the dropdown list and cancel the dialog. If I open the dropdown list and press Cancel, the file dialog closes. How can I make Cancel close my dialog immediately?

Tahir Helvaci

A I wish I had ten dollars for every question I get like this: "Microsoft Office has feature X, but not my app—How can I add it?" The Redmondtonians have so many programmers in the Office division they have time to practically reinvent Windows®. The Office products have their own toolbars, dialogs, and other UI components. All you need is Spy++ to discover this. For example, the file open dialog in Word is not the standard dialog class you and I get, it's a window with class name "bosa_sdm_Microsoft Office Word 11.0". It kind of makes Windows look bad, as if it's not good enough for Office. And it's frustrating to programmers who expect the same features for their own apps.

Well, I can't change the world, but I can show you how to make your dropdown list close without losing the click that closed it. I've seen solutions for this on the Web, but they all involve changing the dialog box to simulate a Cancel or OK the appropriate time. This band-aid approach doesn't really solve the problem at its root, and therefore doesn't work in all situations—for example, when you click the close or minimize/restore buttons in the window's caption bar. And it requires modifying each dialog on an ad hoc basis. A better and more easily reusable solution would have the following characteristics:

  • Clicking outside the combobox dropdown should feed the mouse click to the underlying window, wherever the mouse happens to be—whether it's the Cancel button, OK button or some other button, control, or window.
  • It should be easy (for programmers) to use: just include some code and set a flag or call a function or something to get the feature. It shouldn't require writing message handlers or modifying dialog procs.
  • It should be universal for the entire app. That is, you should be able to turn on the fix-the-combobox feature for your entire app in one fell swoop. You shouldn't need to remember to add code every time you implement a new dialog. Either the whole app gets the feature or it doesn't.

Lucky you, I implemented a little class that does all this. It's called CComboClickFix for lack of a clever name. (I'm all in favor of long descriptive names, but CPassComboDropdownOutsideClicksToUnderlyingWindow is too much, even for me.) As promised, all you have to do to use CComboClickFix is add the source files to your project and a couple of lines in your application module:

// one-and-only global instance
CComboClickFix combofix;
...
BOOL CMyApp::InitInstance()
{
   ...
   combofix.Install(); // init
}

Now any time the user clicks the mouse outside a combobox dropdown anywhere in your app, the system closes the dropdown and passes the click through without eating it. How does CComboClickFix perform this magic?

As Windows wonks have no doubt already guessed, CComboClickFix uses window hooks to peer into the message stream. But which messages should I hook? To find out, I wrote a little class called CComboTrace that spies on combo notifications. CComboTrace uses CSubclassWnd, a class I frequently use to subclass a window in the Windows sense of subclassing its window procedure. CComboTrace displays TRACE diagnostics any time the dialog gets a combo notification—that is, a WM_COMMAND message from a combobox. To use CComboTrace, just instantiate the class and call HookWindow:

   class CMyOpenDlg : public CFileDialog {
   protected:
      CComboTrace m_cbTrace;
      ...
   };

   BOOL CMyOpenDlg::OnInitDialog()
   {
      m_cbTrace.HookWindow(GetParent());
   }

CComboTrace displays CBN_XXX codes in human-readable format; Figure 1 shows a typical stream. After two minutes of testing with CComboTrace, it appeared that the key notification to trap was, not surprisingly, CBN_CLOSEUP. But further investigation revealed that by the time the combo sends CBN_CLOSEUP, it has already eaten the mouse message. So the notification to look for is actually CBN_SELENDCANCEL, which comes before CBN_CLOSEUP. There's just one snag: comboboxes also send CBN_SELENDCANCEL when the user first tabs to the combobox. The upshot is that you can't look for just one message; you have to look for the sequence CBN_SELENDCANCEL followed immediately by CBN_CLOSEUP.

Figure 1 Typical ComboTrace Stream
Figure 1 Typical ComboTrace Stream

To see messages, CComboClickFix installs two Windows hooks: a WH_MOUSE hook to trap mouse messages and a WH_CALLWNDPROC hook to trap combo notifications. Hooks are always a delicate affair, you don't want to do much processing in your hook procedure or you'll bog the system down. You also don't want to send messages from your hook proc because the system is in a funky state, in the middle of processing a message. I've learned from experience that the best way to handle hooks is to save the information you want, then process it later when the message queue reaches a settled state. So my mouse hook simply records clicks. Any time it sees WM_LBUTTONDOWN, it saves the message code (WM_LBUTTONDOWN) in m_state and the cursor/mouse position in m_point. At any given time, CComboClickFix::m_point holds the coordinates of the most recent WM_LBUTTONDOWN message.

The call-window-proc hook is slightly more complex. It spies on combo notifications. If the notification is anything but CBN_SELENDCANCEL or CBN_CLOSEUP, it clears the state by setting m_state equal to zero. The purpose of this is to ignore the case of CBN_SELENDCANCEL followed by some other combo notification like CBN_KILLFOCUS or CBN_SETFOCUS. If the combo notification is CBN_CLOSEUP, and m_state is nonzero, the CBN_CLOSEUP can only have come from clicking the mouse outside the dropdown. In that case, CComboClickFix posts itself a private message, MYWM_COMBOTRAP, with LPARAM equal to the screen coordinates of the WM_LBUTTONDOWN message that caused the close. It's important to post, but not send this message, so it can be processed later, after the app finishes processing the current message (CBN_CLOSEUP).

But where does CComboClickFix post MYWM_COMBOTRAP? It needs a window. I could create an invisible window to talk to myself, but that's way overkill for a puny combo feature, so CComboClickFix uses the application's main window instead. If for some reason your app doesn't have a main window (in which case you probably don't have a UI, so why do you need CComboClickFix?), you can tell CComboClickFix which window to use when you call CComboClickFix::Install. To ensure MYWM_COMBOTRAP doesn't collide with other application message IDs, I call RegisterWindowMessage to create a unique ID.

Now whenever the user clicks outside the combo dropdown, CComboClickFix posts MYWM_COMBOTRAP to the main window, with lp equal to the screen coordinates where the mouse was when the user clicked. Does that mean you have to handle MYWM_COMBOTRAP in your main window? No, CComboClickFix uses my ubiquitous CSubclassWnd to subclass the main window and handle MYWM_COMBOTRAP all by itself. The handler calls ::SendInput to feed the WM_LBUTTONDOWN back into the message queue, in effect reposting the mouse click that caused the combo dropdown to close. Since the dropdown is now closed, Windows processes the posted click the normal way—in whatever fashion it would've been processed had the combo not eaten it.

To test CComboClickFix, I wrote a little program CComboTest that runs the standard file open dialog (CFileDialog). Figure 2 shows the dialog running with a combo dropdown open and the mouse poised over the Cancel button. It's a little hard to show the interaction in a screen capture; you'll have to download and build the project yourself to see that clicking outside the dropdown closes it without losing the click.

Figure 2 ComboTest in Action
Figure 2 ComboTest in Action

Before closing the box on comboboxes, I want to underscore one minor detail. In Windows, there are two flavors of comboboxes: ComboBox and ComboBoxEx32. CComboClickFix uses a function—IsComboBox—to see if the WM_COMMAND is coming from a combobox. By comparing only the first eight characters of the class name, IsComboBox catches both "ComboBox" and "ComboBoxEx32"—or any other variations the Redmondtonians might add some future day. Figure 3 shows the code highlights. Download the full source from the MSDN®Magazine Web site.

Figure 3 Two Types of ComboBoxes (Close Figure 3)

ComboFix.h

#pragma once
#include "subclass.h"

//////////////////
// Test for combo notification code (CBN_XXX)
//
inline BOOL IsComboCode(UINT code)
{
    return (CBN_SELCHANGE<=code && code<=CBN_SELENDCANCEL);
}

//////////////////
// Test for hwnd = a combobox.
// Note class name can be ComboBox or ComboBoxEx32!
//
inline BOOL IsComboBox(HWND hwnd)
{
    TCHAR buf[32];
    GetClassName(hwnd,buf,32);
    return _tcsncmp(buf,_T("ComboBox"),8)==0;
}

//////////////////
// Class to implement ability to click-outside-the-combo-dropdown.
//
class CComboClickFix {
protected:
    static CComboClickFix* g_trap;    // one-and-only global closer

    // main window hook to handle private messages
    class CMainWndHook : public CSubclassWnd {
    public:
        virtual LRESULT WindowProc(UINT msg, WPARAM wp, LPARAM lp);
    } m_mainWndHook;
    CWnd*     m_pMainWnd;        // main app window
    HHOOK     m_hhMouse;         // mouse hook
    HHOOK     m_hhCallWnd;       // CallWndProc hook
    UINT      m_state;            // current state (mouse message)
    CPoint    m_point;              // ..and cursor pos

    // hook procedures
    static LRESULT WINAPI MouseProc(int nCode, WPARAM wp, LPARAM lp);
    static LRESULT WINAPI CallWndProc(int nCode, WPARAM wp, LPARAM lp);

public:
    CComboClickFix();
    ~CComboClickFix();

    void Install(CWnd* pMainWnd=NULL);
    void Remove();
};
ComboFix.cpp
#include "stdafx.h"
#include "ComboFix.h"

// private message posted to myself
const UINT MYWM_COMBOTRAP = RegisterWindowMessage(_T("MYWM_COMBOTRAP"));

// Ptr to THE combo trap
CComboClickFix* CComboClickFix::g_trap=NULL;

CComboClickFix::CComboClickFix()
{
    ASSERT(g_trap==NULL); // singleton class, only one instance allowed!
    g_trap=this;          // set global
    m_hhMouse = m_hhCallWnd = NULL;
}

CComboClickFix::~CComboClickFix()
{
    Remove();
}

//////////////////
// Install combo trap. Set hooks and clear state.
// Arg is main window—or any window used to communicate.
//
void CComboClickFix::Install(CWnd* pMainWnd)
{
    if (pMainWnd==NULL)
        pMainWnd = AfxGetMainWnd();
    ASSERT(pMainWnd);                   // need main window!
    m_pMainWnd = pMainWnd;
    m_mainWndHook.HookWindow(m_pMainWnd);

    HMODULE hmod = GetModuleHandle(NULL);
    DWORD id = GetCurrentThreadId();
    m_hhMouse   = ::SetWindowsHookEx(WH_MOUSE, MouseProc, hmod, id);
    m_hhCallWnd = ::SetWindowsHookEx(WH_CALLWNDPROC, CallWndProc, hmod,
        id);
    m_state=0;
}

//////////////////
// Remove combo trap: Remove hooks.
//
void CComboClickFix::Remove()
{
    m_mainWndHook.Unhook();             // unhook subclass
    if (m_hhMouse) {
        UnhookWindowsHookEx(m_hhMouse);
        m_hhMouse=NULL;
    }
    if (m_hhCallWnd) {
        UnhookWindowsHookEx(m_hhCallWnd);
        m_hhCallWnd=NULL;
    }
}

//////////////////
// Mouse hook: Every time I get WM_LBUTTONDOWN, remember it and pos too.
//
LRESULT CComboClickFix::MouseProc(int code, WPARAM wp, LPARAM lp)
{
    if (code==HC_ACTION && wp==WM_LBUTTONDOWN) {
        MOUSEHOOKSTRUCT* pmhs = (MOUSEHOOKSTRUCT*)lp;
        g_trap->m_state = wp;             // save message code
        g_trap->m_point = pmhs->pt;       // ..and point
    }
    return ::CallNextHookEx(g_trap->m_hhMouse, code, wp, lp);
}

//////////////////
// CallWndProc hook: if message is a combobox close message and close was
// due to left-button-down, post private message to resend the
// left-button-down. For some reason trying to catch the mouse message
// here doesn’t work. It never arrives.
//
LRESULT CComboClickFix::CallWndProc(int code, WPARAM wp, LPARAM lp)
{
    const CWPSTRUCT& cwp = *((CWPSTRUCT*)lp);
    if (code==HC_ACTION) {
        if (cwp.message==WM_COMMAND) {
            if (IsComboBox((HWND)cwp.lParam)) {
                UINT nCode = HIWORD(cwp.wParam);
                if (IsComboCode(nCode)) {
                    if (nCode==CBN_CLOSEUP && g_trap->m_state) {
                        // dropdown box was closed via left-button-down:
                        // post private message.
                        // I will handle it myself with my own
                        // CSubclassWnd hook below
                        CPoint& pt = g_trap->m_point;
                        g_trap->m_pMainWnd->PostMessage(MYWM_COMBOTRAP,
                            0, MAKELONG(pt.x,pt.y));
                    }
                    if (nCode!=CBN_SELENDCANCEL && nCode!=CBN_CLOSEUP)
                        g_trap->m_state=0;
                }
            }
        }
    }
    return ::CallNextHookEx(g_trap->m_hhCallWnd, code, wp, lp);
}

//////////////////
// Window proc for main window hook: handle private
// MYWM_COMBOTRAP messages.
//
LRESULT CComboClickFix::CMainWndHook::WindowProc(UINT msg, WPARAM wp,
    LPARAM lp)
{
    if (msg==MYWM_COMBOTRAP) {
        // got MYWM_COMBOTRAP: lp has the mouse coords.
        // Call SendInput to feed the lost mouse message back to the app.
        CPoint pt(LOWORD(lp),HIWORD(lp));
        INPUT input;
        memset(&input,1,sizeof(input));
        input.type=INPUT_MOUSE;
        input.mi.dx=pt.x;
        input.mi.dy=pt.y;
        input.mi.dwFlags=MOUSEEVENTF_ABSOLUTE|MOUSEEVENTF_LEFTDOWN;
        SendInput(1,&input,sizeof(input));
        return 0;
    }
    return CSubclassWnd::WindowProc(msg,wp,lp);

Q I am trying to make Ctrl+A select all of the items in a listbox. How can I accomplish this?

Mark Fowler

A If all you want to do is implement Ctrl+A for one listbox, you only need a WM_CHAR handler that looks for Ctrl+A (0x01). When you see it, you can select all the items by sending LB_SETSEL with nItem equal to -1:

void CMyListBox::OnChar(UINT nChar, ...)
{
   if (nChar==CONTROL_A) {
      SetSel(-1,TRUE);
   }
}
But if you were paying attention to the previous question, you might think to ask: Why not make Ctrl+A/Select All a global feature that works for any listbox throughout your whole app? Why not, indeed.

The same hook tricks I used to fix the combobox can be employed to implement your Ctrl+A feature for all listboxes in your app. I wrote a little class, CListBoxAccel, that does it. Just instantiate the class and call Install:

CListBoxAccelerator lba;
...
BOOL CMyApp::InitInstance()
{
   ...
   lba.Install(pMainWnd);
}

Install sets a WH_GETMESSAGE hook whose hook procedure looks for WM_CHAR messages sent to a listbox. If the listbox allows multiple selection—that is, if it has the LBS_MULTIPLESEL or LBS_EXTENDEDSEL style—the hook posts LB_SETSEL with LPARAM equal to-1 to select all items:

// in hook proc
const MSG& msg = *((MSG*)lp);
if (code==HC_ACTION && wp==PM_REMOVE &&
   msg.message==WM_CHAR && IsListBox(msg.hwnd)) {
   if (IsLBMultiSel(msg.hwnd)) {
      ::PostMessage(msg.hwnd, LB_SETSEL, TRUE, -1);
   }
}

With WH_GETMESSAGE hooks, it's very important to check for WPARAM equal to PM_REMOVE because otherwise you may end up processing the event twice. Windows calls your WH_GETMESSAGE hook whenever the app calls ::GetMessage or ::PeekMessage. Many apps call ::PeekMessage to check for messages before fetching them, in order to do idle processing between messages. When the app calls ::PeekMessage, Windows calls your hook with WPARAM set to PM_NOREMOVE. Most times you want to ignore this. As in the previous example for comboboxes, CListBoxAccelerator::HookProc posts LB_SETSEL, rather than sending it, so the listbox can process it after the app finishes the current message.

CListBoxAccelerator is even simpler than CComboClickFix. Since it sends no messages to itself, it doesn't need CSubclassWnd or registered messages. Everything happens in the hook proc. This code is shown in Figure 4. The test program populates a list with file names; you can Tab to the list and type Ctrl+A to select them all. If you decide to use CListBoxAccelerator in your own app, you could add more accelerator keys for other functions.

Figure 4 ListBox Accelerators (Close Figure 4)

ListBoxAccel.h
#pragma once

// Test for hwnd = a list box.
inline BOOL IsListBox(HWND hwnd)
{
   TCHAR buf[32];
   GetClassName(hwnd,buf,32);
   return _tcsncmp(buf,_T(«ListBox»),7)==0;
}

//////////////////
// Test for multi-select styles.
//
inline BOOL IsLBMultiSel(HWND hwnd)
{
   DWORD dwStyle = ::GetWindowLong(hwnd,GWL_STYLE);
   return dwStyle & (LBS_MULTIPLESEL|LBS_EXTENDEDSEL) ? TRUE : FALSE;
}

//////////////////
// Class to implement Ctrl+A = Select All in listboxes.
// You can add more accelerators if you like.
//
class CListBoxAccelerator {
   static CListBoxAccelerator* g_lba;  // one-and-only listbox accelerator
   HHOOK  m_hHook;                     // call wnd proc hook

   // hook procedure
   static LRESULT WINAPI HookProc(int nCode, WPARAM wp, LPARAM lp);

public:
   CListBoxAccelerator();
   ~CListBoxAccelerator();
   void Install();
   void Remove();
};
ListBoxAccel.cpp
#include "stdafx.h"
#include "ListBoxAccel.h"

// Ptr to THE listbox accelerator
CListBoxAccelerator* CListBoxAccelerator::g_lba=NULL;

CListBoxAccelerator::CListBoxAccelerator()
{
   ASSERT(g_lba==NULL);    // singleton class, only one instance allowed!
   g_lba=this;             // set global
   m_hHook = NULL;
}

CListBoxAccelerator::~CListBoxAccelerator()
{
   Remove();
}

//////////////////
// Install listbox accelerator. Set hooks and clear state.
//
void CListBoxAccelerator::Install()
{
   HMODULE hmod = GetModuleHandle(NULL);
   DWORD id = GetCurrentThreadId();
   m_hHook   = ::SetWindowsHookEx(WH_GETMESSAGE, HookProc, hmod, id);
}

//////////////////
// Remove listbox accelerator: Remove hooks.
//
void CListBoxAccelerator::Remove()
{
   if (m_hHook) {
      UnhookWindowsHookEx(m_hHook);
      m_hHook=NULL;
   }
}

//////////////////
// Mouse hook: Every time I get WM_LBUTTONDOWN, remember it and pos too.
//
LRESULT CListBoxAccelerator::HookProc(int code, WPARAM wp, LPARAM lp)
{
   const MSG& msg = *((MSG*)lp);
   if (code==HC_ACTION && wp==PM_REMOVE &&
          msg.message==WM_CHAR && IsListBox(msg.hwnd)) {
      if (IsLBMultiSel(msg.hwnd)) {
         ::PostMessage(msg.hwnd, LB_SETSEL, TRUE, -1);
      }
   }
   return ::CallNextHookEx(g_lba->m_hHook, code, wp, lp);
}

If you were paying close attention, you may have noticed CComboClickFix uses two kind of hooks, WH_MOUSE and WH_CALLWNDPROC; and CListBoxAccelerator uses yet another kind of hook, WH_GETMESSAGE. How do you know which hook to use? There's no simple answer. You have to know how Windows transmits each kind of message. I know from experience that Windows synthesizes WM_CHAR from WM_KEYDOWN and WM_KEYUP messages (::TranslateMessage is the function that does it), so I need a WH_GETMESSAGE hook.

WH_MOUSE works better for mouse messages. (I could've used WH_GETMESSAGE to trap WM_LBUTTONDOWN, but by the time it arrives via GetMessage, Windows has already converted the cursor to client coordinates and for CComboClickFix I wanted screen coordinates.) Until you become an experienced hookmaster, you may have to experiment with different kinds of hooks to find the right one. Fortunately, it's not hard to do since hook procs all have the same structure.

Happy programming!

Send your questions and comments for Paul to cppqa@microsoft.com.

Paul DiLascia is a freelance software consultant and Web/UI designer-at-large. He is the author of Windows++: Writing Reusable Windows Code in C++ (Addison-Wesley, 1992). In his spare time, Paul develops PixieLib, an MFC class library available from his Web site, www.dilascia.com.


{ End Bracket }: Deliver Beautiful Apps

{ End Bracket }
Deliver Beautiful Apps
Kevin Moore

I remember my first rich client application. I was a senior in high school and I wrote a "virtual lens lab" for a project in my physics class. My tool was Visual Basic® 6.0. I think it was the first time I ever stayed up all night on a coding binge, writing the infamously requisite logic to make sure the controls looked right when the window was resized. I remember feeling restricted by my tool set: using the few features I was given to change the appearance of my application made it look silly—you can’t do very much with only font styles and background colors to play with. Leaving things as-is left the application looking boring—a sea of battleship gray.

This was certainly a problem in the early days of the Web: too many blink tags, too many image maps, not enough elegance and style. The Web evolved, though. We got CSS and Flash, XHTML and AJAX. Look at the CSS Zen Garden. Look at Live.com. As any number of sites show, the available tools can enable beautiful Web experiences.

Unfortunately, most Windows®-based developers are stuck with the same limited features of Visual Basic 6.0. Going beyond the simple tweaks involves crossing the "Owner Draw Cliff," requiring code to make seemingly trivial tweaks to controls. You have to be quite experienced and quite patient to create a compelling and seamless application in Windows that doesn’t look like every other app in Windows. Thankfully, this is about to change. We’ve designed Windows Presentation Foundation to enable the kinds of new, breakthrough user experiences you’ve only seen in designer-fabricated demos.

Everything in the Windows Presentation Foundation supports XML markup. We call it Extensible Application Markup Language (XAML). It’s easy to tweak and easy to tool. Everything supports reusable styles. What CSS did for the Web, Windows Presentation Foundation will do for applications. Using a consistent XML-based format, styles can be defined to tweak a single control or all of the controls in an application.

It’s simple. Everything is usable in a consistent way. No more using ActiveX® controls, COM, and HWNDs to bring in 3D objects, video, and flowing text. It’s baked in. Everything is designed to work together. 3D objects in a text flow. Text flows on 3D objects. Videos in a button. Images in a listbox. It’s all supported.

A button can look like it’s made from chrome, plastic, or a blob of jelly. All without losing any of the functionality of the standard button. All without writing a single line of code. A listbox can scroll left-to-right. It can have rounded edges. It can show pictures that pop up when you drag your mouse over and fade when you unselect. Again, all with the functionality and familiarity of the out-of-the box control. Yep, still no code. Do you want white borders around your pictures? Just add another tag.

As a developer, I now have the tools to tweak controls in an iterative way. Increase the margin. Maybe a bit more on the left and right. Change the background color. Change the background to a gradient. Add another gradient stop. Make the border thicker. Change the border brush. Make it rounded. This can be done in a tool like XamlPad (which is part of the SDK) without the change-compile-run-debug dance. Make a change, see the result, repeat. It’s faster than editing HTML, with far more flexibility.

My first a-ha moment with this design method was last August. My coworker Robby and I were at the office working on our PDC presentations. I was building up a color picker control sample and I wanted to make it look pretty. Robby threw over the fence some canned styles he’d created for a slider. You have to understand that the slider, like all of the controls in Windows Presentation Foundation, is not simply a set of mouse-down handlers and drawing commands. It’s built out of smaller elements, such as panels and buttons. Each of these slider subelements is also designed to be customized with different visualizations and different behaviors—in most cases without a single line of code.

I started by rounding all of the edges. I next replaced solid colors with subtle gradients. I added some simple triggers for mouse over and mouse down. I then wrote a simple converter in C# to generate the background gradients for my sliders. There was a learning curve. I had to understand styles, templates, binding, and converters. But once I’d worked through these concepts, I was amazed. After a few minutes of tweaking I had a color picker that looked beautiful. And it was easy.

Our hope is that Windows Presentation Foundation will allow developers to cross the line from dreaming about beautiful applications to delivering them.

Kevin Moore is a Program Manager on the Windows Presentation Foundation team at Microsoft. He works on data binding, controls, and automation. Visit his blog at blogs.msdn.com/okoboji.