 MSDN Magazine April 2007

MSDN Magazine April 2007

Identity: Secure Your ASP.NET Apps And WCF Services With Windows CardSpace

Windows CardSpace replaces traditional authentication with a more consistent and streamlined login process and improves trust between end-users, applications and services. Michèle Leroux Bustamante explains.

Michèle Leroux Bustamante Aero Glass: Create Special Effects With The Desktop Window Manager

In this article we introduce the Desktop Window Manager, the new interface that manages how windows are rendered on the Windows Vista desktop.

Ron Fosner XmlLite: A Small And Fast XML Parser For Native C++

XmlLite provides a high-performance, low-overhead XML reader and writer geared for applications written in native C++. Learn more here.

Kenny Kerr Event Tracing: Improve Debugging And Performance Tuning With ETW

Event Tracing for Windows (ETW) provides general-purpose, high-speed tracing of events raised by both user-mode applications and kernel-mode device drivers. Learn how ETW can improve your development and debugging work.

Dr. Insung Park and Ricky Buch ASP.NET 2.0: Enforce Web Standards For Better Accessibility

Web standards are about much more than closing HTML tags. They are a critical factor in how well software components can be used in future contexts.

Ben Waldron Editor's Note: Winning Team

Josh Trupin discusses MSDN Magazine's awards from the Society for Technical Communication

Joshua Trupin Toolbox: WiX Installation, Creating PowerPoint Decks, and More

The WiX toolset’s use of a command-line environment allows for integration with automated build processes. The Toolbox column explains.

Scott Mitchell CLR Inside Out: New Library Classes in "Orcas"

The next version of Visual Studio currently code-named “Orcas”supports advanced encryption algorithms, Elliptic curve cryptography, big integers, and other security enhancements. The CLR team explains.

Mike Downen, Inbar Gazit, and Justin Van Patten Data Points: Disabling Constraints and Triggers

What to do when you need to disable triggers and constraints to perform data synchronization and other tasks with an offline database.

John Papa Team System: Work Item Tracking

In this column, Brian Randell explains how to build a simple Work Item explorer and demonstrates the core operations needed to add work item support when building your own add-in.

Brian A. Randell Cutting Edge: Subclassing and Overriding ASP.NET Pages—Part I

There are a number of techniques that allow you to modify a running ASP.NET page without touching its source code. Dino discusses some this month.

Dino Esposito Security Briefs: Improve Manageability through Event Logging

When something goes wrong, a manageable application will tell the administrator how to fix the problem. The Windows Event Log can provide the necessary information.

Keith Brown Service Station: WCF Messaging Fundamentals

This month Aaron Skonnard highlights some of the key messaging features behind the sophisticated XML-based messaging framework of Windows Communication Foundation.

Aaron Skonnard Extreme ASP.NET: Web Deployment Projects

ASP.NET 2.0 development is the easiest ASP development yet. Fritz Onion reveals why.

Fritz Onion Foundations: 3D Mesh Geometries

This month Charles Petzold focuses on a very important part of the Viewport3D assemblage—the MeshGeometry3D class that defines the geometry of a 3D object in Windows Communication Foundation.

Charles Petzold .NET Matters: Restart Manager and Generic Method Compilation

This month Stephen Toub answers questions pertaining to the Windows Vista Restart Manager API and generic method compilation.

Stephen Toub Netting C++: Mapping Templates to Generics

In this installment, Stanley Lippman continues to port his ISO-C++ Text Query Language application to the Microsoft .NET Framework and C++/CLI.

Stanley Lippman { End Bracket }: Geopegging

Josh Trupin introduces geopegging--a special technique for storing GPS location data in a JPG.

Joshua Trupin


Identity: Secure Your ASP.NET Apps And WCF Services With Windows CardSpace

Identity

Secure Your ASP.NET Apps And WCF Services With Windows CardSpace

Michèle Leroux Bustamante
This article discusses:
  • Windows CardSpace and information cards
  • Web browsers and identity selectors
  • Security tokens
  • Integration with WSFederationHttpBinding and ASP.NET membership
This article uses the following technologies:
.NET Framework 3.0
Code download available at: CardSpace2007_04.exe (462KB)
Contents
Windows CardSpace and Information Cards
Personal Cards
Managed Cards
Browsers and Identity Selectors
Object and XHTML Parameters
Processing the Security Token
Associating Cards with an Account
Integrating Windows CardSpace with WCF
Conclusion

I
n May 2005, Microsoft introduced its vision for an identity metasystem that would reduce the complexities and risks associated with managing and exchanging digital identities. This vision-discussed in the article "Microsoft's Vision for an Identity Metasystem" explains interactions based on interoperability standards such as WS-Security, WS-Trust, WS-MetadataExchange, WS-SecurityPolicy, and other WS-* protocols. These protocols enable applications and services to communicate their security requirements via security policy and to transfer digital identities safely and securely. The identity metasystem vision describes a simple and consistent experience for users to identify themselves to applications and services while also identifying the target site to the user in order to reduce risks associated with sending private information to unauthorized or malicious sites.

Windows CardSpace™ (formerly known as "InfoCard"), released with Windows Vista™ and the Microsoft® .NET Framework 3.0, plays an important role in this identity metasystem. Windows CardSpace replaces traditional username and password authentication with a tool that helps users better manage their digital identities and helps shield users from various forms of identity attack such as phishing.

In this article, I will start by reviewing the roles of each party in the identity metasystem, introducing Windows CardSpace, and discussing both personal and managed information cards. Then I'll dig into some scenarios involving both ASP.NET Web applications and Windows Communication Foundation services-explaining how to integrate Windows CardSpace as an authentication mechanism for each scenario.

This article assumes you have some experience with ASP.NET and the Windows Communication Foundation, as well as familiarity with Windows CardSpace features. For a detailed overview of Windows CardSpace, I recommend reading the article "Introducing Windows CardSpace" by David Chappell. You can get additional background from Keith Brown's two columns on the subject titled "A First Look at InfoCard" and "A Deeper Look at InfoCard".

Windows CardSpace and Information Cards

In the identity metasystem, there are three key participants: the relying party (RP), the subject, and the identity provider (IP). The RP can be anything (though typically it is an application or a service) that accepts tokens to execute a given task, such as signing-in and presenting membership details. For the purpose of this article, I will demonstrate how a subject can utilize the Windows CardSpace identity selector to choose and present the appropriate credentials to be used for authentication.

The subject is a person who wishes to safely present information (such as identification information) to the RP. This information is communicated in the form of a security token which is issued by an IP such as a bank, employer, government, and so forth.

The IP is responsible for issuing the security token on behalf of the subject, vouching for the subject's information described by the claims held in the token. The IP is normally implemented as a Security Token Service (STS), a Web service endpoint that implements WS-Trust protocol to issue, renew, validate, or cancel security tokens describing a subject.

Figure 1 Identity Metasystem Participants
Figure 1 Identity Metasystem Participants(Click the image for a smaller view)
Figure 1 Identity Metasystem Participants
Figure 1 Identity Metasystem Participants (Click the image for a larger view)

Figure 1 illustrates the relationship between the three parties. The RP relies on a particular type of security token to authorize calls. The IP authenticates the subject and issues to the subject a security token in the format required by the RP. The subject then relays that token, which contains claims on behalf of the subject, to the RP. The separation of these roles, and the interoperability protocols used to communicate between them, are part of the identity metasystem credo. Each participant can be implemented on a different technology or platform, so long as they comply with the appropriate Web and WS-* protocols to facilitate interoperable communications.

In the context of this identity metasystem, Windows CardSpace has two roles. First, it is a client technology used for creating, managing, and selecting digital identities in a secure and consistent manner. (In the latter capacity, Windows CardSpace serves as an identity selector.) Second, it has a local IP for personal digital identities and can generate security tokens for those identities.

A digital identity is represented as an information card. Windows CardSpace allows users to create personal cards or to import managed cards issued by other IPs. Users can later select a card when an RP requires that users authenticate themselves with one. The user can only select cards that satisfy the requirements of the RP-or, alternatively, select a personal card and add the necessary claims on the fly-and the associated IP issues the security token containing the appropriate subject claims.

Cards are issued by an IP to describe the claims that the IP is able to vouch for, on behalf of an authenticated subject. Each card represents important information about the IP that issued it, including the following:

  • The URL (or URLs) of the IP's STS, which issues security tokens. This way, when the user selects a card to authenticate to an RP, Windows CardSpace knows where to request the security token containing the appropriate claims.
  • The types of claim associated with the card. Since the IP issues the card, it gets to define what claims the card represents. The RP specifies the claims that it requires and only cards representing these claims will be enabled for a user to select.
  • The types of security token that the IP can issue. The RP indicates which tokens it supports, so only cards that support these tokens will be enabled for selection.

A card issued by an IP can be serialized as a signed XML document with a .crd file extension. The .crd file contains an <InformationCard> section with this and other pertinent information for Windows CardSpace. Figure 2 shows an example of a signed, managed card imported into Windows CardSpace.

Figure 2 Serialized Version of a Managed Information Card (Close Figure 2)
<dsig:Signature xmlns:dsig="http://www.w3.org/2000/09/xmldsig#">
  <dsig:SignedInfo>...</dsig:SignedInfo>
  <dsig:SignatureValue>...</dsig:SignatureValue>
  <dsig:KeyInfo>...</dsig:KeyInfo>
  <dsig:Object Id="_Object_InfoCard">
    <ic:InformationCard xmlns:ds="http://www.w3.org/2000/09/xmldsig#"
xmlns:ic="http://schemas.xmlsoap.org/ws/2005/05/identity"
xmlns:mex="http://schemas.xmlsoap.org/ws/2004/09/mex"
xmlns:wsa="http://www.w3.org/2005/08/addressing"
xmlns:wsid="http://schemas.xmlsoap.org/ws/2006/02/addressingidentity"
xmlns:wst="http://schemas.xmlsoap.org/ws/2005/02/trust" xml:lang="en-us">
      <ic:InformationCardReference>
        <ic:CardId>http://www.thatindigogirl.com/card/
            ED18504F-40B9-5A58-C200-761DC886DF29</ic:CardId>
        <ic:CardVersion>1</ic:CardVersion>
      </ic:InformationCardReference>
      <ic:CardName>thatindigogirl</ic:CardName>
      <ic:CardImage MimeType="image/png">[base64 encoded image]
          </ic:CardImage>
      <ic:Issuer>http://www.thatindigogirl.com/tokenissuer.svc</ic:Issuer>
      <ic:IssuerName>That Indigo Girl</ic:IssuerName>
      <ic:TimeIssued>2006-12-29T22:32:17Z</ic:TimeIssued>
      <ic:TimeExpires>9999-12-31T23:59:59.9999999Z</ic:TimeExpires>
      <ic:TokenServiceList>
        <ic:TokenService>
          <wsa:EndpointReference>
            <wsa:Address>http://www.thatindigogirl.com/sts/tokenissuer.svc

                </wsa:Address>
            <wsa:Metadata><mex:Metadata><mex:MetadataSection>
                  <mex:MetadataReference><wsa:Address>
                        http://www.thatindigogirl.com/sts/mex</wsa:Address>
                  </mex:MetadataReference>
            </mex:MetadataSection></mex:Metadata></wsa:Metadata>
            <wsid:Identity>
              <ds:KeyInfo><ds:X509Data>
                  <ds:X509Certificate>[base64 encoded
                       certificate of the IP]</ds:X509Certificate>
              </ds:X509Data></ds:KeyInfo>
            </wsid:Identity>
          </wsa:EndpointReference>
          <ic:UserCredential>
            <ic:UsernamePasswordCredential>
              <ic:DisplayCredentialHint>Please enter your
                   password</ic:DisplayCredentialHint>
              <ic:Username>thatindigogirl</ic:Username>
            </ic:UsernamePasswordCredential>
          </ic:UserCredential>
        </ic:TokenService>
      </ic:TokenServiceList>
      <ic:SupportedTokenTypeList>
        <wst:TokenType>urn:oasis:names:tc:SAML:1.0:assertion
            </wst:TokenType>
      </ic:SupportedTokenTypeList>
      <ic:SupportedClaimTypeList>
        <ic:SupportedClaimType Uri=
            "http://schemas.xmlsoap.org/ws/2005/05/identity/
                claims/personalprivateidentifier">
          <ic:DisplayTag>PPID</ic:DisplayTag>
          <ic:Description>Unique identifier for the card</ic:Description>
        </ic:SupportedClaimType>
        ...other claim types
      </ic:SupportedClaimTypeList>
      <ic:PrivacyNotice>http://www.thatindigogirl.com/PrivacyPolicy.txt
      </ic:PrivacyNotice>
    </ic:InformationCard>
  </dsig:Object>
</dsig:Signature>

There are two types of card: personal and managed. Personal cards (sometimes referred to as self-issued cards) are created by the user within Windows CardSpace and are issued by the local Windows CardSpace IP. Managed cards can be issued by any other IP and must be explicitly imported into Windows CardSpace.

The interaction between the subject, the RP, cards and associated IPs, and Windows CardSpace is shown in Figure 3. When a Web site or service (the RP) requires a token issued by Windows CardSpace, it specifies the claims it needs. The client application (a browser or a Web service client) invokes Windows CardSpace (1), passing this information along. Windows CardSpace uses this information to present the appropriate cards that can satisfy those claims to the user (2). When the user selects a card (3), Windows CardSpace authenticates the user using a method defined by the IP, calling the appropriate IP as referenced by the card (4) to have a security token issued containing the claims requested by the RP (5). However, it is up to the IP to determine which claims are included in the token (this could be a superset or subset of those in the RP's policy). Ultimately, a signed and encrypted security token is passed back to the application or Web browser (6), which forwards it to the RP (7).

Figure 3 How Cards Are Selected and Used to Issue Security Tokens
Figure 3 How Cards Are Selected and Used to Issue Security Tokens(Click the image for a smaller view)
Figure 3 How Cards Are Selected and Used to Issue Security Tokens
Figure 3 How Cards Are Selected and Used to Issue Security Tokens (Click the image for a larger view)

Cards are stored and secured on the local machine in a file that is encrypted twice, protected by a machine key, your Windows login, and optionally a pin number you supply to secure access to the card. If you use multiple computers, you can export your cards to an encrypted data file and then import them on another machine that you trust.

Personal Cards

Personal cards are created through the Windows CardSpace Control Panel. When you create a personal card, you are in effect creating two things: a set of claims representing some personal information, and a card that lists those claims. Essentially the local Windows CardSpace IP issues personal cards representing the claims you entered and safely stores your claims waiting for a request for a security token. Figure 4 illustrates how a personal card indicates to the Windows CardSpace IP which claims to put into a requested security token.

Figure 4 Generate a Security Token from a Personal Card
Figure 4 Generate a Security Token from a Personal Card(Click the image for a smaller view)
Figure 4 Generate a Security Token from a Personal Card
Figure 4 Generate a Security Token from a Personal Card (Click the image for a larger view)

When you create a personal card, you can only enter information for some or all of a predefined set of 15 claims, such as first and last name, e-mail address, and mailing address information. Claims are represented by a URI from the WS-Identity namespace, schemas.xmlsoap.org/ws/2005/05/identity. Figure 5 lists the URI for the claims that a personal card can store (the URIs are abbreviated for clarity).

Figure 5 Claims Supported by Personal Cards (Close Figure 5)
http://schemas.xmlsoap.org/.../identity/claims/privatepersonalidentifier
http://schemas.xmlsoap.org/.../identity/claims/name
http://schemas.xmlsoap.org/.../identity/claims/givenname
http://schemas.xmlsoap.org/.../identity/claims/surname
http://schemas.xmlsoap.org/.../identity/claims/emailaddress
http://schemas.xmlsoap.org/.../identity/claims/streetaddress
http://schemas.xmlsoap.org/.../identity/claims/locality
http://schemas.xmlsoap.org/.../identity/claims/stateorprovince
http://schemas.xmlsoap.org/.../identity/claims/postalcode
http://schemas.xmlsoap.org/.../identity/claims/country
http://schemas.xmlsoap.org/.../identity/claims/homephone
http://schemas.xmlsoap.org/.../identity/claims/otherphone
http://schemas.xmlsoap.org/.../identity/claims/mobilephone
http://schemas.xmlsoap.org/.../identity/claims/dateofbirth
http://schemas.xmlsoap.org/.../identity/claims/gender
http://schemas.xmlsoap.org/.../identity/claims/webpage

Creating personal cards is not much different from creating a username and password for a site. Web sites and Web services can associate personal cards with your user account so you can use Windows CardSpace to select your identity with a few mouse clicks, instead of typing in your username and password. A unique identifier, based on the card's personal private identifier (PPID), is used to associate a card with an account. This identifier is unique to each card and RP combination. That is, if the same card is used for multiple RPs, each RP will receive a distinct PPID to protect the privacy of the user's usage patterns.

Managed Cards

Unlike personal cards, managed cards can represent any claims the IP wishes to assert about a subject (the actual card data is stored within the IP's systems, not on the local machine). You can't create managed cards in the Windows CardSpace interface because the local Windows CardSpace IP can only issue cards for a fixed set of claims. Any IP can issue managed cards representing the claims for which they can generate security tokens. In fact, since a relationship must exist between the RP and the IP, it may be that the RP and IP are owned by the same enterprise.

For example, if you have an account with a bank that knows your account number and type, the IP might issue a card that indicates a security token can be issued with those claims. To avoid the risks associated with username and password login to the account, the managed card can be installed into Windows CardSpace so that the user can select it when authenticating to banking sites. This implies that the bank associated the card with your account when it issued the card, or that an additional step to associate the card once installed is required.

When the user selects this card to authenticate to the banking RP, only compatible managed cards that can satisfy these claims can be selected. Managed cards do not contain the actual claim values, but Windows CardSpace knows where to find the IP to issue the encrypted token containing these claims. Figure 6 illustrates how the managed card indicates to the managed IP what to put into the requested security token.

Figure 6 Generate a Security Token from a Managed Card
Figure 6 Generate a Security Token from a Managed Card(Click the image for a smaller view)
Figure 6 Generate a Security Token from a Managed Card
Figure 6 Generate a Security Token from a Managed Card (Click the image for a larger view)

Each RP and IP that have a relationship should also agree on the claims to be included in security tokens. These claims can be identified by unique URIs. Consider these Create/Read/Update/Delete (CRUD) claims for example:

http://www.thatindigogirl.com/2006/06/claims/create
http://www.thatindigogirl.com/2006/06/claims/read
http://www.thatindigogirl.com/2006/06/claims/update
http://www.thatindigogirl.com/2006/06/claims/delete

An application or service could potentially use these claims to do claims-based authorization against features and resources.

Though managed cards are secured by the local machine and can be exported to other machines, the actual claims are stored at the IP and never leave the IP without being wrapped in an encrypted security token. The IP encrypts the security token with the public key of the RP so that nothing-not even Windows CardSpace-can view the claims inside the token. (If the user wants to preview the claims that are about to be sent to the RP, an extra and optional display token can be requested. This is encrypted in such a way that Windows CardSpace can decrypt it and display it to the user.)

Browsers and Identity Selectors

Other identity selectors are being built to perform the same function that Windows CardSpace does in Windows. Windows CardSpace or any other identity selector can be used to select personal or managed cards to satisfy an authentication request.

For Web applications to support personal or managed card authentication, they must first provide a Web page with an object tag or an XHTML binary behavior describing their information card requirements. Browsers that support these tags and have an information card extension will be able to launch the appropriate identity selector on the client machine for users to select a card. As Figure 7 illustrates, there are no specific ties to Windows CardSpace as the identity selector, to Internet Explorer® as the browser, or to ASP.NET as the Web site technology. In fact, other browsers have their own extensions for handling these and other object tags and behaviors, and each operating system can have its own identity selector that supports information cards. All of these elements are platform-neutral.

Figure 7 Trigger an Identity Selector from an Object Tag or Binary Behavior
Figure 7 Trigger an Identity Selector from an Object Tag or Binary Behavior(Click the image for a smaller view)
Figure 7 Trigger an Identity Selector from an Object Tag or Binary Behavior
Figure 7 Trigger an Identity Selector from an Object Tag or Binary Behavior (Click the image for a larger view)

Any Web application platform, including ASP.NET, can expose pages that include the required object tag or XHTML binary behavior to trigger identity selection. The object tag uses the MIME type application/x-informationCard as shown here:

<object id="informationCard" name="informationCard"
        type="application/x-informationCard">
    ...
</object>

The information card binary behavior can be specified in XHTML by using a #default#informationCard behavior in the <informationCard> element.

  <ic:informationCard name="xmlToken"
      style="behavior:url(#default#informationCard)" ... >
      ...
  </ic:informationCard>

Within these <object> and <informationCard> elements are parameters that describe a Web site's requirements for information card authentication. I'll get to those details shortly.

Although earlier versions of Internet Explorer support object tags and XHTML, only Internet Explorer 7.0 supports information cards using a MIME handler extension called Microsoft Information Card IE Helper, available from icardie.dll. (An extension adding Windows CardSpace support to Mozilla Firefox is available.) The following registry key links the MIME type for information cards to the handler:

HKEY_CLASSES_ROOT\MIME\Database\Content Type\application/x-informationCard

This MIME handler is then linked to object tag and behavior configurations for the browser. This handler is responsible for calling the identity selector, passing parameters specified by the object tag or XHTML binary behavior. For Internet Explorer 7.0, the identity selector defaults to Windows CardSpace, indicated by the following registry entry:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\InformationCard Token Provider

For the object tag syntax, the MIME handler is triggered through the InformationCardSigninHelper Class, enabled by default in the Internet Explorer 7.0 add-ons dialog as shown in Figure 8. (You can find this dialog by opening Internet Options and from the Programs tab selecting "Manage add-ons.") If this add-on is disabled, the object tag will not trigger identity selection, but this setting has no effect on XHTML syntax. The XHTML information card behavior is configured in a string entry labeled INFORMATIONCARD under the registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\Default Behaviors

Figure 8 Information Card MIME Handler in Internet Explorer 7.0
Figure 8 Information Card MIME Handler in Internet Explorer 7.0(Click the image for a smaller view)
Figure 8 Information Card MIME Handler in Internet Explorer 7.0
Figure 8 Information Card MIME Handler in Internet Explorer 7.0 (Click the image for a larger view)

This behavior also triggers the MIME handler to activate the identity selector.

Unfortunately, consistently detecting browser support for information cards is not an exact science and differs across browsers. While not recommended, for Internet Explorer you can perform a preliminary check of the UserAgent looking for Internet Explorer 7.0 and .NET common language runtime (CLR) 3.0 support. Here's an example of a UserAgent string that indicates this support:

Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; SLCC1;
.NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506)

In ASP.NET, you can check possible support at the server using the UserAgent property of the Request object as follows:

string userAgent = Request.UserAgent;
if (userAgent.Contains("MSIE 7.0") &&
    userAgent.Contains(".NET CLR 3.0"))
    { Response.Write(
        "Information cards are supported, but might not be enabled."); }
else
    { Response.Write("Information cards may not be supported."); }

This approach has many shortcomings. Other browsers support Windows CardSpace besides Internet Explorer, and even in browsers that support Windows CardSpace, the MIME handler could be disabled. A better solution is to use client-side script to detect whether the browser supports Windows CardSpace and whether that support is enabled. For an example of this, see www.fearthecowboy.com/2006/12/detecting-cardspace-support-including.html.

Internet Explorer 7.0 and the appropriate MIME handler configurations are central to triggering Windows CardSpace from object tags and XHTML. In addition to this, the Web site must be SSL-enabled to secure communications. Extended validation (EV) certificates issued by a trusted root certification authority such as VeriSign or Thawte won't require any special configuration, but if you are using test certificates, you'll need to install the test SSL certificate as a trusted root and add the site to the trusted sites list in Internet Explorer 7.0. Instructions for this are included with the code samples for this article.

Object and XHTML Parameters

As I mentioned, you can use object tags or XHTML binary behaviors to trigger an identity selector from supporting browsers. With either syntax, you must provide a base set of properties to the information card browser extension, listed in Figure 9.

Figure 9 Information Card Activation Properties (Close Figure 9)

Property | Description
requiredClaims (mandatory) | You can indicate any of the personal claims listed in Figure 5 or custom claims from a managed IP. If any of the claims cannot be satisfied by a personal card, a managed card must be installed to satisfy the claims.
issuer (optional) | Indicates the IP from which to request a security token. This defaults to the URI for personal tokens: schemas.xmlsoap.org/ws/2005/05/identity/issuer/self. Leave this setting as the default to trigger Windows CardSpace.
issuerPolicy (optional) | Indicates the URL where the security policy for the IP can be found. This does not need to be set for the local Windows CardSpace IP.
tokenType (optional) | If omitted, this defaults to the URI for SAML 1.0 tokens, urn:oasis:names:tc:SAML:1.0:assertion. For personal cards, you can specify either SAML 1.0 or SAML 1.1 token format: http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.1#SAMLV1.1. Anything else implies a managed card.
optionalClaims (optional) | You can indicate any personal or custom claims as optional. This causes Windows CardSpace to give the user the option of sending the information to the site.
privacyVersion (optional) | Specifies the privacy policy version. If this changes, the user is notified and given a chance to review any changes to the privacy policy.

Using the object tag, you configure these properties as in Figure 10. Each <param> represents one of the information card properties by name. The values for requiredClaims and optionalClaims are space-delimited lists of claim URIs. You could optionally omit the issuer and tokenType parameters if the defaults are acceptable to your application.

Figure 10 Object Tag Syntax (Close Figure 10)
<object id="informationCards" name="informationCards"
  type="application/x-informationCard" >
  <param name="issuer"
    value="http://schemas.xmlsoap.org/ws/2005/05/identity/issuer/self"/>
  <param name="tokenType" value="urn:oasis:names:tc:SAML:1.0:assertion"/>
  <param name="requiredClaims" value="http://schemas.xmlsoap.org/ws/
        2005/05/identity/claims/privatepersonalidentifier
    http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress"/>
  <param name="optionalClaims" value="http://schemas.xmlsoap.org/ws/
        2005/05/identity/claims/dateofbirth"/>
</object>

Using XHTML you configure information card properties as shown in Figure 11. In this case, attributes are used to specify the behavior style, issuer, and tokenType. Each claim is specified with an <add> instruction indicating the claim URI and whether it is optional. At least one claim must be specified.

Figure 11 Binary Behavior Syntax (Close Figure 11)
<ic:informationCard name="informationCards"
  style="behavior:url(#default#informationCard)"
  issuer="http://schemas.xmlsoap.org/ws/2005/05/identity/issuer/self"
  tokenType="http://docs.oasis-open.org/wss/
      oasis-wss-saml-token-profile-1.1#SAMLV1.1">
  <ic:add claimType="http://schemas.xmlsoap.org/ws/2005/05/
       identity/claims/privatepersonalidentifier" optional="false" />
  <ic:add claimType="http://schemas.xmlsoap.org/ws/2005/05/
       identity/claims/emailaddress" optional="false" />
  <ic:add claimType="http://schemas.xmlsoap.org/ws/2005/05/
       identity/claims/dateofbirth" optional="true" />
</ic:informationCard>

In any case, when a Submit button is selected on a page that includes an information card trigger, the browser invokes the MIME handler and passes the parameters to the identity selector. If a card is selected, the signed and encrypted token is posted to the site in a form parameter named for the <object> or <informationCard> identifier. In Figures 10 and 11, the identifier is informationCards.

If, for some reason, information cards are disabled by the browser, or if the site is not SSL-enabled, the form parameter collection will not include an informationCards item. If Windows CardSpace is launched and the user cancels for any reason, including if the parameters are invalid and cause an error loading Windows CardSpace, the informationCards form parameter is posted as an empty string. In other words, there is nothing to do unless the informationCards form parameter includes an <encryptedData> element.

Using either syntax for <object> or <informationCard>, you can trigger Windows CardSpace from any ASP.NET page of your choice, but there are a few additional steps required to ensure that Windows CardSpace is only invoked for specific Submit buttons on the page. ASP.NET pages have a single active <form> object, so if you slap the <object> or <informationCard> tag inside the only <form> object, any button that posts back to the server will trigger Windows CardSpace, like it or not.

To avoid this, you can include the information card object in the HTML header section and explicitly invoke it using script when the appropriate button is clicked. Figure 12 shows a simple ASP.NET page that includes the <object> tag in the header, along with a SelectInformationCard function that activates the object and puts the return value into a hidden input field. Inside the <form> tag is a standard ASP.NET Login control and an ImageButton.

Figure 12 Explicitly Invoke the Object Tag (Close Figure 12)
<%@ Page Language="C#" AutoEventWireup="true"
    CodeFile="Login.aspx.cs" Inherits="Login" %>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" >
<head runat="server">
  <title>Login</title>
  <!--Declaration of information card object -->
  <object declare id="informationCards" name="informationCards"
    type="application/x-informationCard" > ... </object>

  <script type="text/javascript" language="javascript">
  function SelectInformationCard()
  {
    var infoCardObject =document.getElementById("informationCards");
    var hiddenToken = document.getElementById("hiddenXmlToken");
    hiddenToken.value = infoCardObject.value;
  }
  </script>
</head>
<body>
  <form id="standardLogin" name="standardLogin" method="post"
        runat="server">
    <asp:Login ID="Login1" runat="server" />
    <asp:ImageButton ID="cardSpaceSubmit" runat="server"
        ImageUrl="cardspacelogin.jpg"
        OnClientClick="SelectInformationCard()"
        OnClick="CardSpaceLogin" />
    <input id="hiddenXmlToken" type="hidden" name="hiddenXmlToken"
           value="empty"/>
  </form>
</body>
</html>

When the Login control submits, Windows CardSpace is not invoked since the object tag is in the header section. When the ImageButton submits, Windows CardSpace is invoked because the OnClientClick attribute calls SelectInformationCard. You could also create a custom ASP.NET control that emits the object tag and associated script. The important thing is to distinguish each Submit button so that Windows CardSpace is only activated at the appropriate time.

Processing the Security Token

As I mentioned earlier, if Windows CardSpace is invoked, a signed and encrypted token is posted. In the case of Figure 12, the hiddenXmlToken input field will contain this <encryptedData> section if a token is successfully issued. If Windows CardSpace was activated but the user canceled, the field will contain an empty string. If Windows CardSpace could not be activated, it will contain its initial value "empty". The Windows CardSpaceLogin method is invoked if the Windows CardSpace login option is selected on this page; thus you can check the value of the hidden field and proceed from there.

The first problem you'll encounter when an encrypted token is posted back to your Web site is that ASP.NET disallows posting XML and HTML data in order to reduce vulnerability to scripting attacks. You'll receive an HttpRequestValidationException like this one:

System.Web.HttpRequestValidationException: A potentially dangerous
Request.Form value was detected from the client
(informationCards="<enc:EncryptedData T...").

You can disable this feature for the entire site by adding the following to the <system.web> section of your web.config:

<pages validateRequest="false"/>

This is one of those cases where I would recommend you set this only for those pages that will receive a postback with a security token. Fortunately, you can set this per page as follows:

<%@ Page Language="C#" AutoEventWireup="true"
         CodeFile="Login.aspx.cs" Inherits="Login"
         ValidateRequest="false" %>

The next problem you'll run into is that token-processing code is not built into ASP.NET. You must manually process the <encryptedData> element, decrypt the token, validate the token signature, and pull out the claims before you can perform any authentication or authorization. Fortunately, Microsoft provided an example TokenProcessor class with the Windows CardSpace samples that ship with the Windows SDK for Windows Vista and the .NET Framework 3.0 (referenced at msdn2.microsoft.com/aa967562.aspx). I have included a slightly modified version of this TokenProcessor in the code sample for this article.

To use the TokenProcessor class with your ASP.NET applications, all you have to do is construct the type with the <encryptedData> element and access the claims that were passed in the security token. For example, this code looks for the PPID, emailaddress, and dateofbirth claims required by the object tag discussed earlier:

using Microsoft.IdentityModel.TokenProcessor;

string token = Request.Form["hiddenXmlToken"] as string;
if (!String.IsNullOrEmpty(token)) {
  Token tokenProcessor = new Token(token);
  string ppid = tokenProcessor.Claims[ClaimTypes.PPID];
  string emailaddress = tokenProcessor.Claims[ClaimTypes.Email];
  string dateofbirth = tokenProcessor.Claims[ClaimTypes.DateOfBirth];
}

From a very high level, the TokenProcessor works as follows. The token is encrypted using the Web site SSL certificate. The TokenProcessor uses the thumbprint of this certificate to find the associated private key, defaulting to the My/LocalMachine store. The Web application host identity (typically ASPNET or NETWORK SERVICE) must be granted access to this private key or the decryption will fail. The decrypted result is then converted into a SamlToken and validated to ensure that the contents of the token have not been tampered with since the token was created. Once validated, a reference to the token and its claims are available to your code as shown earlier.

For personal cards, the key used to sign the SAML token is not known to your application since it is generated on the fly when the card is used for the first time at your site. Thus, you can't check the signature for personal cards against a trusted public key. You can, however, rely on a hash of the PPID if you previously associated the card with a user account.

For managed cards, since a relationship exists between the RP and IP, you should add an additional step to the RP's token processing code to validate that the token has been signed by an IP you accept. You can do this by using the public key of the IP to validate the signature.

Associating Cards with an Account

Now that I've explained how to trigger Windows CardSpace from ASP.NET, and how to process the resulting security token, you might be wondering what you should do with the claims inside the token. This is influenced by the type of card you require at the Web site: personal or managed.

Sites that support personal cards largely do so to provide users with an improved login experience. The types of claims supported, as I've discussed, are limited-although the associated demographic information could be useful to the site. That said, these claims won't remove the need for role-based authorization. One way you can get the best of both worlds is to allow users to associate a personal card with their existing account and use the card to look up the user's roles when they log in with it.

Figure 13 illustrates the interaction between the user, the Web site, and Windows CardSpace to achieve this goal. The user can create the personal card ahead of time and then log in to an existing account to associate the card. It is also possible to create the card during the association step if a card that satisfies the site's requirements doesn't already exist. Once a personal card is selected to associate with the site, the resulting security token is posted to the site for processing. The claims contained in the security token must be sufficient to look up the user account when a user logs in with the card.

Figure 13 Associate a Personal Card with a Web Site Account
Figure 13 Associate a Personal Card with a Web Site Account(Click the image for a smaller view)
Figure 13 Associate a Personal Card with a Web Site Account
Figure 13 Associate a Personal Card with a Web Site Account (Click the image for a larger view)

The following code associates a card with the current user account only if the e-mail address claim supplied with the token matches the e-mail of the logged-in user:

MembershipUser user = Membership.GetUser(this.User.Identity.Name);
if (user.Email == emailaddressClaim) {
    user.Comment = ppidClaim;
    Membership.UpdateUser(user);
}

This code uses the Comment field of the user record to associate the card's PPID with the user account. The PPID claim is unique to the card and the RP it is sent to, thus uniquely identifying the user. (Note, though, that the PPID can only be relied on if the token's signature is first verified and validated cryptographically.)

When the user logs in using the personal card, the e-mail address claim can be used to look up the user account using the ASP.NET Membership API. A matching PPID in the comment field indicates the correct user has been found, as shown here:

MembershipUserCollection matchingUsers =
    Membership.FindUsersByEmail(emailaddressClaim);
if (matchingUsers.Count > 0) {
    foreach (MembershipUser user in matchingUsers) {
        if (user.Comment == ppid) {
            authenticatedUser = user;
            break;
        }
    }
}

Having found the user account, the usual steps to create an authentication ticket for the user can be completed. A simple call to RedirectFromLoginPage can facilitate this:

FormsAuthentication.RedirectFromLoginPage(
    authenticatedUser.UserName, true);

Interestingly, this also authenticates the user in the traditional way for the duration of the ASP.NET session, which means that traditional role-based security checks will also work across the application.

Managed cards introduce a slightly different workflow for associating cards to accounts. The distribution of managed cards to end users can be handled in a number of ways, one of which could be to allow users to request a card after logging in to the site. The card is then imported to Windows CardSpace for later use from the site's login page. Figure 14 illustrates this flow of communication.

Since managed cards can contain literally any set of claims deemed valuable to the IP and RP, they may even have more meaning to the authorization process. A set of claims received by an RP might indicate a list of rights to the application. In this case, the ASP.NET membership and role provider may be bypassed altogether since it relies on role-based security and has no notion of claims.

Figure 14 Request and Import a Managed Card to Log In
Figure 14 Request and Import a Managed Card to Log In(Click the image for a smaller view)
Figure 14 Request and Import a Managed Card to Log In
Figure 14 Request and Import a Managed Card to Log In (Click the image for a larger view)

Integrating Windows CardSpace with WCF

Browser-based applications are natural candidates for integrating with Windows CardSpace authentication. Web services can also require Windows CardSpace authentication such that client applications invoke Windows CardSpace to issue security tokens in a similar way.

In the case of Windows Communication Foundation, this is done by configuring service endpoints to use WSFederationHttpBinding. This generates a security policy for the service, included in the Web Service Description Language (WSDL) document that indicates it requires personal tokens. When SvcUtil generates a Windows Communication Foundation proxy from this WSDL, matching client endpoints are generated that use an equivalent WSFederationHttpBinding configuration. Client applications use this proxy to construct a channel to call the service, and the proxy handles calls to Windows CardSpace to gather a security token satisfying the claims of the target service. The resulting interaction looks much like that in Figure 3, where the RP is the Windows Communication Foundation service instead of a Web application, and the client application is the Windows Communication Foundation client and proxy instead of a browser.

WSFederationHttpBinding allows you to expose Windows Communication Foundation services over HTTP that rely on an issued token from an IP. The IP can be Windows CardSpace, which implies a personal or managed card must be supplied to find the appropriate IP to issue the actual token. The IP can also be explicitly configured, which means that the token request does not go through Windows CardSpace. For the purpose of this discussion I'll focus only on the Windows CardSpace configuration.

The parameters required of WSFederationHttpBinding will look very similar to the requirements of the object tag or XHTML configuration used to trigger Windows CardSpace from a Web page. You must specify an issuer, a token type, and a list of required or optional claims. The issuer must be set to schemas.xmlsoap.org/ws/2005/05/identity/issuer/self to trigger Windows CardSpace; the token format can be any format supported by a personal or managed card. The claims can likewise be personal or managed claims. These settings will determine whether a personal or managed card can satisfy the request for a security token.

Figure 15 shows a complete <system.serviceModel> section for a simple Windows Communication Foundation service that exposes a WSFederationHttpBinding endpoint. The <wsFederationHttpBinding> section describes the properties I just discussed for the binding configuration. In this case, SAML 1.1 tokens are required to contain a PPID and e-mail address claim. The service must also provide a certificate to facilitate secure exchanges with clients, and this certificate is specified in the <serviceBehavior> section within the <serviceCertificate> element.

Figure 15 Service Config to Trigger Windows CardSpace (Close Figure 15)
<system.serviceModel>
  <services>
    <service name="HelloIndigo.HelloIndigoService"
        behaviorConfiguration="serviceBehavior">
    <endpoint address="http://localhost:8000/HelloIndigo"
        contract="HelloIndigo.IHelloIndigoService"
        binding="wsFederationHttpBinding"
        bindingConfiguration="wsFed" />
    </service>
  </services>
  <bindings>
    <wsFederationHttpBinding>
      <binding name="wsFed">
        <security mode="Message">
          <message issuedTokenType="http://docs.oasis-open.org/wss/oasis-
wss-saml-token-profile-1.1#SAMLV1.1" negotiateServiceCredential="false" >
            <claimTypeRequirements>
              <add claimType="http://schemas.xmlsoap.org/
                    ws/2005/05/identity/claims/privatepersonalidentifier"
                    isOptional="false"/>
              <add claimType="http://schemas.xmlsoap.org/
            ws/2005/05/identity/claims/emailaddress" isOptional="false"/>
            </claimTypeRequirements>
            <issuer address=
          "http://schemas.xmlsoap.org/ws/2005/05/identity/issuer/self" />
          </message>
        </security>
      </binding>
    </wsFederationHttpBinding>
  </bindings>
  <behaviors>
    <serviceBehaviors>
      <behavior name="serviceBehavior" >
        <serviceCredentials>
          <issuedTokenAuthentication
            allowUntrustedRsaIssuers="true" />
          <serviceCertificate findValue="RPKey"
            storeLocation="LocalMachine" storeName="My"
            x509FindType="FindBySubjectName" />
        </serviceCredentials>
      </behavior>
    </serviceBehaviors>
  </behaviors>
</system.serviceModel>

The allowUntrustedRsaIssuers setting in the <issuedTokenAuthentication> element is important when relying on personal cards. By default this setting is false. However, it must be set to true for personal cards since the resulting token is signed with an unknown private key. For managed cards, this isn't required since the service will likely install the public key of any IP you accept into the Trusted People certificate store. In the case of managed cards, the signature of the issued token is checked against this list of trusted public keys. Custom authorization options also exist.

The generated client configuration used to initialize the proxy is equivalent to the service configuration with a few exceptions. The client requires access to the service public key to encrypt the token and its communications to the service. Fortunately, SvcUtil generates a base64-encoded version of the public key with the client endpoint configuration under <identity> to avoid the step of installing the public key into the client machine certificate store. This configuration is shown in Figure 16. The <wsFederationHttpBinding> section is collapsed in this illustration because it is equivalent to Figure 15. The client proxy is initialized with this information when constructed and uses it to determine how to acquire the security token to authenticate to the service. In fact, the actual code to invoke the service can be as simple as this:

HelloIndigoContractClient proxy = new HelloIndigoContractClient();
string s = proxy.HelloIndigo("Hello");

Figure 16 Client Configuration for WSFederationHttpBinding (Close Figure 16)
<system.serviceModel>
  <bindings>
    <wsFederationHttpBinding>...</wsFederationHttpBinding>
  </bindings>
  <client>
    <endpoint address="http://localhost:8000/HelloIndigo"
        binding="wsFederationHttpBinding"
        bindingConfiguration="wsFed"
        contract="Client.localhost.HelloIndigoContract"
        name="wsFed">
      <identity>
        <certificate encodedValue="base64 encoded public key" />
      </identity>
    </endpoint>
  </client>
</system.serviceModel>

Windows CardSpace is triggered by Windows Communication Foundation at the client because the issuer setting tells it to use the personal token provider. The user is presented with the identity selection interface with only cards satisfying the required claims in the binding presented. Managed cards are required when the token type or required claims cannot be satisfied by personal cards.

One area where Windows Communication Foundation significantly differs from ASP.NET in regards to Windows CardSpace authentication is in how claims are processed at the server. Unlike with ASP.NET, the security token issued through Windows CardSpace is packaged as part of the security headers in the SOAP message sent to the service. Before the service operation is invoked, this security token is decrypted and validated, and its claims extracted into the service security context. These extracted claims are added to an abstraction known as a ClaimSet, which contains the list of claims and information about the issuer of the claims. In the case of personal cards, the claims are self-issued, which means that you won't be authorizing the public key of the issuer. That's why it is important to associate the card's PPID claim (which is unique and kept secret) with an account in the system. You can check the claim sets attached to the AuthorizationContext and look for a PPID claim to find a match, and then check the values of other expected claims for further authorization, as shown in Figure 17.

Figure 17 Access Claims through the AuthorizationContext (Close Figure 17)
AuthorizationContext authContext =
    ServiceSecurityContext.Current.AuthorizationContext;

DateTime? birthDate = null;
string ppid = null;

foreach (ClaimSet cs in authContext.ClaimSets)
{
    IEnumerable<Claim> claims = cs.FindClaims(ClaimTypes.PPID,
        Rights.PossessProperty);

    ppid = ValidatePPID(claims);
    if (ppid!=null)
    {
        claims = cs.FindClaims(ClaimTypes.DateOfBirth,
            Rights.PossessProperty);
        foreach (Claim c in claims)
            birthDate = Convert.ToDateTime(c.Resource);
    }
}
if (birthDate == null)
    throw new FaultException("Missing date of birth claim.");
if (birthDate.Value.AddYears(13) > DateTime.Now)
    throw new FaultException(
        "User is too young to access this operation.");

The code in Figure 17 checks for PPID and date-of-birth claims from any ClaimSet-assuming that the token was issued by a personal card-then checks for a date-of-birth claim to verify the age of the caller. One or more sets of claims may be attached to the AuthorizationContext as different security tokens are extracted from the message and processed by service authorization policies. If the issuer is known and can be identified, it would be possible to select the correct ClaimSet from a particular issuer before processing claims.

Managed cards yield an extended set of possibilities since the claims have an identifiable issuer. The issuer is also described by a ClaimSet that often includes an identity claim with the public key of the IP you accept. You can search for an identity claim and check that it is a valid RSA claim as follows:

ClaimSet csIssuer = cs.Issuer;
IEnumerable<Claim> issuerClaims = csIssuer.FindClaims(
    ClaimTypes.Rsa, Rights.Identity);
foreach (Claim c in issuerClaims)
{
    System.Security.Cryptography.RSACryptoServiceProvider rsa =
        c.Resource as RSACryptoServiceProvider;
    if (rsa != null)
    {
        ... // claim exists; check the public key to identify the IP
    }
}

Once you're sure you have the right ClaimSet, you can perform claims-based security checks. For example, if the claims are based on the CRUD claims mentioned earlier, you can check for create, read, update, and delete claims for appropriate operations. In this way, managed cards not only make it easier for a user to select his identity, but they also facilitate the generation of a rich set of claims identifying the user's rights. Service developers can focus on claims-based authorization checks instead of traditional role-based security checks. A complete sample that illustrates claims-based security can be downloaded from msdn.microsoft.com/msdnmag/code07.aspx.

Conclusion

Integrating Windows CardSpace with your ASP.NET applications and Windows Communication Foundation services improves the end-user experience by providing a simple and consistent identity selection process for authentication to trusted applications and services. While personal and managed cards provide an equivalent login experience, managed cards have the advantage of enabling an IP to supply a specific set of claims that applications and services can use to better identify the user's rights in their system. The topics discussed in this article and implemented in the sample code for this article will help you on your way to triggering the Windows CardSpace experience from ASP.NET or Windows Communication Foundation. Enjoy!

Michèle Leroux Bustamante is a Chief Architect for IDesign Inc., a Microsoft Regional Director and MVP for Connected Systems, and an internationally known speaker and author. Michele contributed to the design and implementation of Windows CardSpace during its early beta stages. She recently completed the book Learning WCF (O'Reilly, 2007; book blog: www.thatindigogirl.com). Reach her at www.idesign.net or visit her main blog at www.dasblonde.net.

© 2007 Microsoft Corporation and CMP Media, LLC. All rights reserved; reproduction in part or in whole without permission is prohibited.


Aero Glass: Create Special Effects With The Desktop Window Manager

Aero Glass

Create Special Effects With The Desktop Window Manager

Ron Fosner
This article discusses:
  • Understanding the DWM
  • Programming the DWM
  • Drawing on glass and incorporating live thumbnails
This article uses the following technologies:
Windows Vista, .NET Framework
Code download available at: DWM2007_04.exe (166KB)
Contents
Technical Overview of the DWM
The Benefits of a Compositing Desktop
Some Common Compositing Functions
Getting Ready for Aero Glass
Programming the DWM
Drawing on Glass
Thumbnails
Wrap-Up

I
always look forward to new releases of Windows. It's fun to poke around MSDN® and the SDK documentation looking for the latest innovations to play with and build upon, to amaze friends and coworkers and-if you're lucky-your employer. Windows Vista™ contains a lot of goodies in this regard. I've been particularly looking forward to getting my hands on this release ever since I heard that it integrates a 3D/compositing layer into the desktop. I've written a ton of 3D applications over the years, and one thing I always found annoying was being able to provide cool user-interface eye-candy in a 3D application but not in a non-3D app. With Windows Vista and the Desktop Window Manager (DWM), this is starting to change (see Figure 1).

Figure 1 DWM Enables Features Like Flip 3D Task Switching
Figure 1 DWM Enables Features Like Flip 3D Task Switching(Click the image for a smaller view)
Figure 1 DWM Enables Features Like Flip 3D Task Switching
Figure 1 DWM Enables Features Like Flip 3D Task Switching (Click the image for a larger view)

The DWM is the new interface that manages how the various windows that are running and rendering are merged onto the Windows Vista desktop. Windows® Presentation Foundation (WPF) provides a higher-level layer that controls the rendering to the desktop layer, and the Windows Display Driver Model (WDDM) handles the actual low-level rendering to the display. I'm only going to talk about using the DWM interfaces. Read the MSDN article "Introducing Windows Presentation Foundation" by David Chappell for more on that subject (msdn2.microsoft.com/aa663364.aspx). WDDM is only of interest in the context of this article because of the new effects it makes possible through the DWM interfaces, and the problems it was designed to fix.

Technical Overview of the DWM

The DWM interface, available on all versions of Windows Vista except Windows Vista Home Basic, is housed in dwm.exe. All applications in the system benefit from the DWM without any modifications or recompilations.However, applications that choose to take advantage of specific DWM features can call to interfaces in dwmapi.dll (the public interface of DWM), which will then pass them along to dwm.exe. The interface declarations are found in dwmapi.h and you can get up-to-date API information online from windowssdk.msdn.microsoft.com.

Windows Vista was designed to use graphics accelerators for every window, not just for 3D DirectX® applications. In order to do this, DWM talks to WDDM, which is the ultimate owner of the graphics processor and video memory. (DWM relies on milcore.dll, a component shared with Windows Presentation Foundation, for output and rendering to DirectX.) Rendering is handled by a separate thread in DWM, inaccessible to the user, that owns the DirectX device. In addition, most applications have their own threads that handle rendering and UI (such as the USER message pump thread in a typical Win32® application), but which don't conflict with DWM's rendering thread. The DWM gets a list of windows and manages their bitmaps in a tree structure, which it then composites onto the final desktop.In other words, each application renders its own bitmaps that are then composited by the DWM.

An application's main window thread renders its scene, the scene is accessed by the DWM render thread, and the render thread updates the desktop through its DirectX interface. The information passed is compacted to just changes (deltas) from the previous render, and large data, like images, are placed in shared memory. This can potentially allow the scene generation to occur on one machine while the final rendering can be done on another machine. 3D programmers who have used OpenGL will be familiar with this architecture, which allows a server to manage a 3D scene and just send deltas to the client machines. You can have a distributed rendering of a 3D scene and have it running with full hardware acceleration on any number of client machines.This architecture allows the DWM to provide first-class support for Remote Desktop scenarios.

While Windows Vista will run with a legacy Windows XP-compatible driver, a WDDM video driver is required to get all the DWM features. Contrary to some speculation, DWM doesn't require DirectX 10, but does require some more video/texture memory and a video card that supports Shader Model 2.0 or better. The biggest change with using WDDM is that it introduces the Video Memory Manager (VidMM), which can swap video memory allocations between system memory and video memory. This means that WDDM can virtualize the video card's resources so that it's possible to do a better job of sharing and swapping video memory, and of context-switching the graphics processor among different threads in disparate applications. It used to be nearly impossible to run multiple 3D applications with any kind of stability, due to drivers that were unable to handle context switches. And prior to WDDM, there was no formal scheduling available, so often one DirectX application would starve others.With WDDM, it's much harder for that to happen. Drivers are also on a much tighter leash for Windows Vista and are forced to be much more robust than for previous versions of Windows.

As an aside, note that DirectX 10 is a Windows Vista-only API. Applications designed for previous version of DirectX will run on a legacy DirectX API implementation expected to be called DirectX 9 L.This will be the last version supported by pre-WDDM drivers. DirectX 9 L apps are expected to run on Windows XP with DirectX 9 L installed as well as on Windows Vista. DirectX 10 contains no legacy interfaces.

The Benefits of a Compositing Desktop

All these new subsystems get you the ability to render windows independently and perform a compositing step on them prior to rendering them onto the desktop. Some cool uses of this are visible in Windows Vista and some of the updated applications that ship with it. The two I'm going to show you how to use are the Aero™ glass effect and thumbnails. The glass effect is available only when running the Aero Glass scheme and compositing is turned on. It isn't available for Aero Basic.

Since each window is created in its own section of video memory, the DWM gets to do the final compositing of that window onto the desktop. This means the DWM has access to the image that's on the desktop and can blend it with your window's rendering, creating a rendering that's a composition of both. This is most noticeable in any areas of a window that are blended with the existing desktop image to create a frosted glass effect. Since each window is rendered to an intermediate off-screen surface, this means the DWM is the only program involved in updating the glass effect. When you move a window with a glass effect, no underlying window needs to be invalidated. The DWM handles updating the visible translucent image to the new coordinates. It's possible to tell the DWM to add some of the client area of a window to be rendered with glass-thus letting you create glass areas for your own use.

This off-screen compositing makes the desktop more responsive. Since each window is now rendered separately from the desktop, the problem you often see in slow-to-update applications like Web browsers is eliminated. In previous version of Windows, it wasn't uncommon to see a display like Figure 2. You could move a window around on top of another application and get this tearing effect because the window underneath took too long to update itself. With desktop composition, you'll no longer see this artifact.

Figure 2 Slow Rendering Causes Tearing
Figure 2 Slow Rendering Causes Tearing(Click the image for a smaller view)
Figure 2 Slow Rendering Causes Tearing
Figure 2 Slow Rendering Causes Tearing (Click the image for a larger view)

Some of the applications that ship with Windows Vista take advantage of the ability to render glass into the client area. One of the nicest examples is Windows Media® Player, which extends the frame into the bottom of the client area where it draws some custom controls. The minimal version looks like Figure 3.

Figure 3 Glass Effects in Windows Media Player
Figure 3 Glass Effects in Windows Media Player(Click the image for a smaller view)
Figure 3 Glass Effects in Windows Media Player
Figure 3 Glass Effects in Windows Media Player (Click the image for a larger view)

When I move the window around my desktop, the image I can see through the controls is made up of the part of the desktop that's under the player. If I move the window over some animation, I can see the animation through the window that's on top of it! This is the power of a composited desktop.

If you turn the composition effect off, you'll just get the "aurora" effect on the windows and the window will render opaquely with the default window color as in Figure 4.

Figure 4 Windows Media Player with Glass Effects Turned Off
Figure 4 Windows Media Player with Glass Effects Turned Off(Click the image for a smaller view)
Figure 4 Windows Media Player with Glass Effects Turned Off
Figure 4 Windows Media Player with Glass Effects Turned Off (Click the image for a larger view)

In essence, the Aero Basic and Aero Glass interfaces are the demarcation between the pre-DWM UI standards and the new. The Aero Basic interface presents the same API to programs to maintain backwards compatibility, but running the Basic interface means you are using the legacy window manager, and DWM is not active. The legacy interface means that the UI layout will behave as expected by programs written prior to Windows Vista. The DWM controls the Aero Glass interface and restricts access to it. If an application draws in the non-client area (the glass frame), the DWM detects this and will switch over to the Aero Basic frame.

Other new features in Windows Vista-available because of a compositing desktop-are the new Windows Flip (Alt+Tab) and Flip 3D (Alt+Windows Key) task switchers. The Flip 3D is of particular interest because its functionality depends upon some code in the DWM that takes each top-level window in the scene graph and renders it onto a series of skewed windows you can scroll through with your keyboard or mouse (see Figure 1).

The DWM controls how windows interact with the desktop composition engine. Being able to integrate your program into the DWM functionality requires that you gain an understanding of how the DWM works and how you can interact with it. While it does much more, the DWM has four primary functional areas in its public API:

  • Basic desktop composition settings
  • Rendering the glass effect in a client window
  • Rendering thumbnails
  • Tuning rendering for interaction with multimedia programs

I'll cover the first three sections in this article. The last one is provided for DirectX and video playback applications because the DWM is running asynchronously, which can lead to sampling artifacts if not tightly controlled.

Some Common Compositing Functions

If you want to use the desktop composition functionality in your program, you need to query and set various DWM parameters. For example, if some application switches to full-screen display and then DWM turns off composition and renders in the desktop background color with no transparency, your application should recognize this situation and disable composition-specific features. Here are the basic functions for integrating your program with the DWM:

DwmEnableComposition enables or disables DWM composition. The DWM will maintain this setting for either the duration of the current process or until it is reset. Changing the setting causes a WM_DWMCOMPOSITIONCHANGED notification. Most applications have no need to call this function, but you will likely want to monitor for the resulting Windows message.

DwmIsCompositionEnabled gets the DWM composition-enabled state for the desktop.

DwmSetWindowAttribute sets the value of the specified DWM attribute for a window, controlling how DWM transitions are handled, whether non-client rendering is allowed, and how Flip 3D will treat the window. For example, if non-client rendering is turned off for a window, then later calls to extend the frame or blur behind the window will fail.

DwmGetWindowAttribute retrieves the current value of the specified DWMWINDOWATTRIBUTE for the specified window.

DwmGetColorizationColor retrieves the current color that is being used for DWM glass composition. This value is based on the current color scheme. Changing the setting causes a WM_WMCOLORIZATIONCOLORCHANGED notification.

DwmDefWindowProc enables DWM hit-testing within the non-client area when called with the WM_NCHITTEST notification as well as in certain situations where you might need to process WM_NCCALCSIZE and the like because you have extended the client frame.

Rendering the glass effect in your program is pretty straightforward. The DWM provides two functions for this:

DwmExtendFrameIntoClientArea , a simple function, extends the non-client frame edge into your window.

DwmEnableBlurBehindWindow , a more complicated function, gives you a lot more control over how the glass effect is rendered.

Since all composited windows are rendered to an off-screen window by the DWM for later composition onto the desktop, it's a simple matter to take those images and provide a live thumbnail representation of the application. The DWM provides four functions to let you control how thumbnails get rendered:

DwmQueryThumbnailSourceSize returns the source size of the DWM thumbnail.

DwmRegisterThumbnail creates a thumbnail relationship between the destination and source windows.

DwmUnregisterThumbnail removes a DWM thumbnail relationship created by DwmRegisterThumbnail.

DwmUpdateThumbnailProperties updates the properties for a given thumbnail.

The DWM has five functions for fine-tuning how the DWM renders, but these functions are beyond the scope of this article.

Getting Ready for Aero Glass

To program the DWM interface, you need to be running a version of Windows Vista capable of displaying Aero Glass.While it's easiest to call these new functions from C++ code, I like to write user-interface code in C# if I can. All the code for this article is written in C#, but that does mean that you have to jump through a few hoops. To use the functions discussed in this article, you'll either need to use C++ and link in the correct library, or you'll have to write P/Invoke wrappers for the functions and structures in C#. In the download for this article, I've included a library that provides wrappers for the functions and structures required by the DWM so you can call it from your C# program. Basically, it's just a set of instructions to load the interface from dwmapi.dll. In order to use the DWM functions for the glass effect and thumbnails used in this article, you'll need to create C# declarations of the DWM functions and data structures.The ones I created for this article look like Figure 5.

Figure 5 C# Declarations for DWM (Close Figure 5)
internal class DwmApi
{
    [DllImport("dwmapi.dll", PreserveSig = false)]
    public static extern void DwmEnableBlurBehindWindow(
        IntPtr hWnd, DWM_BLURBEHIND pBlurBehind);

    [DllImport("dwmapi.dll", PreserveSig = false)]
    public static extern void DwmExtendFrameIntoClientArea(
        IntPtr hWnd, MARGINS pMargins);

    [DllImport("dwmapi.dll", PreserveSig = false)]
    public static extern bool DwmIsCompositionEnabled();

    [DllImport("dwmapi.dll", PreserveSig = false)]
    public static extern void DwmEnableComposition(bool bEnable);

    [DllImport("dwmapi.dll", PreserveSig = false)]
    public static extern void DwmGetColorizationColor(
        out int pcrColorization,
        [MarshalAs(UnmanagedType.Bool)]out bool pfOpaqueBlend);

    [DllImport("dwmapi.dll", PreserveSig = false)]
    public static extern IntPtr DwmRegisterThumbnail(
        IntPtr dest, IntPtr source);

    [DllImport("dwmapi.dll", PreserveSig = false)]
    public static extern void DwmUnregisterThumbnail(IntPtr hThumbnail);

    [DllImport("dwmapi.dll", PreserveSig = false)]
    public static extern void DwmUpdateThumbnailProperties(
        IntPtr hThumbnail, DWM_THUMBNAIL_PROPERTIES props);

    [DllImport("dwmapi.dll", PreserveSig = false)]
    public static extern void DwmQueryThumbnailSourceSize(
        IntPtr hThumbnail, out Size size);

    [StructLayout(LayoutKind.Sequential)]
    public class DWM_THUMBNAIL_PROPERTIES
    {
        public uint dwFlags;
        public RECT rcDestination;
        public RECT rcSource;
        public byte opacity;
        [MarshalAs(UnmanagedType.Bool)]
        public bool fVisible;
        [MarshalAs(UnmanagedType.Bool)]
        public bool fSourceClientAreaOnly;
        public const uint DWM_TNP_RECTDESTINATION = 0x00000001;
        public const uint DWM_TNP_RECTSOURCE = 0x00000002;
        public const uint DWM_TNP_OPACITY = 0x00000004;
        public const uint DWM_TNP_VISIBLE = 0x00000008;
        public const uint DWM_TNP_SOURCECLIENTAREAONLY = 0x00000010;
    }

    [StructLayout(LayoutKind.Sequential)]
    public class MARGINS
    {
        public int cxLeftWidth, cxRightWidth,
                   cyTopHeight, cyBottomHeight;

        public MARGINS(int left, int top, int right, int bottom)
        {
            cxLeftWidth = left; cyTopHeight = top;
            cxRightWidth = right; cyBottomHeight = bottom;
        }
    }

    [StructLayout(LayoutKind.Sequential)]
    public class DWM_BLURBEHIND
    {
        public uint dwFlags;
        [MarshalAs(UnmanagedType.Bool)]
        public bool fEnable;
        public IntPtr hRegionBlur;
        [MarshalAs(UnmanagedType.Bool)]
        public bool fTransitionOnMaximized;

        public const uint DWM_BB_ENABLE = 0x00000001;
        public const uint DWM_BB_BLURREGION = 0x00000002;
        public const uint DWM_BB_TRANSITIONONMAXIMIZED = 0x00000004;
    }

    [StructLayout(LayoutKind.Sequential)]
    public struct RECT
    {
        public int left, top, right, bottom;

        public RECT(int left, int top, int right, int bottom)
        {
            this.left = left; this.top = top;
            this.right = right; this.bottom = bottom;
        }
    }
}

You'll need to create something similar to this in your code if you're using C#. Then, assuming you're running Windows Vista, you'll be able to make DWM calls. Of course, your application shouldn't just assume it's running on Windows Vista.In order to be sure, you need to verify that Environment.OSVersion.Version.Major is at least 6.0. Alternatively, you can catch the exceptions that result from trying to call nonexistent functions through P/Invoke.

If you want to use the glass effects, the computer you're using will need to meet three requirements. First, you need to be running the Premium, Business, or Ultimate editions of Windows Vista. Second, you need hardware capable of running the Aero interface (see microsoft.com/windowsvista/getready/capable.mspx for details). Finally, you need to select the Windows Aero color scheme in Windows Vista.Be willing to use it sparingly, as the effect can be taxing on your computer's GPU and, if overdone, on your users.

Figure 6 Window Color and Appearance Options
Figure 6 Window Color and Appearance Options(Click the image for a smaller view)
Figure 6 Window Color and Appearance Options
Figure 6 Window Color and Appearance Options (Click the image for a larger view)

You enable the Aero scheme by opening the Personalization control panel and clicking the Window Color and Appearance option. On this screen (see Figure 6), make sure the Enable Transparency option is selected, then click the "Open classic appearance properties." link. In the Appearance Settings dialog box, under Color scheme, click Windows Aero (see Figure 7). When you click OK, the Aero interface and glass effect will be visible. If you like, you can also customize the window color and the opacity level.

Figure 7 Windows Aero Glass Effects
Figure 7 Windows Aero Glass Effects(Click the image for a smaller view)
Figure 7 Windows Aero Glass Effects
Figure 7 Windows Aero Glass Effects (Click the image for a larger view)

Programming the DWM

You can check to see if the Aero scheme is enabled in your program by calling DwmIsCompositionEnabled. Be aware, though, that not only can the user change the current Aero scheme at any time, it's also possible for other applications to programmatically enable or disable it.Thus, checking the result of this function once may not be robust enough.

The DwmEnableComposition function allows a program to turn the Aero scheme on or off. For example, if you're writing an application that may encounter compatibility problems, you might want to disable composition while you app is running (if you're writing a full-screen DirectX exclusive application, composition will be automatically disabled). This setting is only maintained for the duration of the process that sets it; when the process ends, the composition flag will be reset to its original value. In general, unless for application compatibility reasons, applications shouldn't use this and should let the system or user make the decision instead.

When the status of desktop composition is changed, a WM_DWMCOMPOSITIONCHANGED message is broadcast. There are no parameters telling you if it's being enabled or disabled, so it's up to you to call DwmIsCompositionEnabled if you're interested. The code to do the check is straightforward-the tricky part is deciding how you want your window to look if composition is disabled.

// Check to see if composition is Enabled
if (DwmIsCompositionEnabled())
{
    // enable glass rendering
}
else
{
    // fallback rendering
}

Finally, even if the Aero scheme is enabled, the user might have changed the glass color and made the composition opaque. I wrote a small app that creates a window that's entirely glass and then I changed the glass properties in the control panel (see Figure 8). The first image shows the default setting for the window color and transparency. Then I turned transparency off, leaving an opaque window. I changed to the red window color and the default transparency where you can still make out some of the underlying window image. On a large enough opaque window, you'll observe the highlighting placed in the glass rendering-it's that streaky diagonal "aurora" effect visible on the window.

Figure 8a Changing Color and Transparency
Figure 8a Changing Color and Transparency

Figure 8b

Figure 8c

You can check to see the composition color and opacity by calling the DwmGetColorizationColor function. If this function succeeds, it will set a GDI+ ARGB color value and a Boolean indicating whether the color is opaque. Just like changing the Aero scheme in the control panel, there's a message broadcast when the composition color has changed. WM_DWMCOLORIZATIONCOLORCHANGED is sent when this happens, but in this case the parameters tell you what the new color and opacity are.

protected override void WndProc(ref Message msg)
{
    switch (msg.Msg)
    {
        case WM_DWMCOLORIZATIONCOLORCHANGED:
            // The color format of currColor is 0xAARRGGBB.
            uint currColor = (uint)msg.WParam.ToInt32();
            bool opacityblend = (msg.LParam.ToInt32() != 0);
            ...
            break;
    }
}

The WM_DWMNCRENDERINGCHANGED message is sent when DWM rendering has changed for the non-client area. The wParam will be true if non-client rendering by the DWM is enabled. You'll also get notification when a DWM composited window is maximized or un-maximized when you get a WM_DWMWINDOWMAXIMIZEDCHANGE message. The wParam will be true if the window has been maximized.

Earlier we noted the two functions you can use to get the glass effect into your program. The first is DwmExtendFrameIntoClientArea. A window with the Aero scheme has glass in the title bar area and in a border around the edges of the window-essentially all of the non-client area of the window. This function allows you to extend each of the sides of the non-client area rendering into the client area, rendering it with the glass effect. In other words, you can extend the top, left, right and bottom edges of the glass window frame into your window seamlessly.

The second function is DwmEnableBlurBehindWindow, which lets you render an arbitrarily shaped region with the glass effect and specify more parameters with greater control over the effect, though I suspect that most users of the glass effect will simply extend the glass from the edge into the client area. With either function, you'll need to closely track the composition status to see if you should render with the glass effect enabled. This means tracking the four WM_DWM* messages or calling DwmIsCompositionEnabled to see if you should render with the glass effect turned on or off.

Let's look at the simpler call first. This function is intended for use on windows without a frame (such as the taskbar, Sidebar, tablet pen input window, and the Start menu); behavior on framed windows is undefined.

The DwmExtendFrameIntoClientArea function takes a window handle and a MARGINS structure. The window handle is the window for which the frame is to be extended from the edge into the client area. You'll need to set up a MARGINS structure that contains the number of pixels to extend the frame into the client area. A C# implementation of the MARGINS structure is shown in Figure 5.

It's a bit confusing at first since there are no other Win32 functions that work like this, but, basically, you control each side independently from the others. Pick the sides you want to extend and specify how far into the client area the effect should be rendered (see Figure 9). If you want more than one side to be extended, they can overlap. If you want the effect to dynamically track the window size, then you'll need to call the DwmExtendFrameIntoClientArea function every time the window size changes. A special case is to set one or more margins to -1, which will then extend the glass effect to your entire window. To reset the margins, just set all of the margin values to 0 and call DwmExtendFrameIntoClientArea again.

Figure 9 Glass Margins in Client Area
Figure 9 Glass Margins in Client Area(Click the image for a smaller view)
Figure 9 Glass Margins in Client Area
Figure 9 Glass Margins in Client Area (Click the image for a larger view)

So what do you do if you don't want to extend the glass effect from the frame into the client area? The DwmEnableBlurBehindWindow function gives you a bit more control over how the glass effect is added to your window. Again, it takes a window handle of the window you want to add the glass effect to, but it also takes a DWM_BLURRBEHIND structure that lets you set various parameters regarding how to use the blur effect on your window. The most important one of the parameters is a region, which is a GDI term that describes an arbitrarily shaped area that is usually constructed out of a series of lines and curves.

The DWM_BLURRBEHIND structure, shown in Figure 5, contains the parameters that control how the blur effect is presented.

If you want to turn on the glass effect in the client area, you set the fEnable flag to true. To turn it off you'd set the flag to false. The hRgnBlur parameter is a handle to a region you create that the glass effect appears on. Just like setting a margin value to -1 for Dwm-ExtendFrameIntoClientArea, setting the hRgnBlur parameter to null in the blur structure informs the DWM to apply the glass effect to the entire window.

The final parameter, fTransitionOnMaximized, is a bit misleading. Since the glass effect is turned off on windows that are maximized, you'd think this flag would have something to do with that. This flag actually controls whether or not the window transitions to the maximized color when there is a maximized window on the desktop. Unfortunately, if you set this parameter to true, when you render the window you'll get a region without the transparency, just the aurora effect.

The dwFlags parameter is how you tell the interface which parameters you are setting; when you want to set a parameter, you'll need to turn on the corresponding bit in the dwFlags parameter. This is consistent throughout the DWM interface.

Don't forget that you'll have to render the glass color into the region. Using the same black brush that was used in DwmExtendFrameIntoClientArea will work fine for creating a glass effect on the specified region.

It's easy to see that DWM has provided two interfaces for you-the more complicated one that lets you construct an arbitrarily shaped region, and the simple one that just lets you extend the window frames glass effect into your client area so you can draw additional controls and the like onto something that looks like it's part of the title bar. In both cases, you have to worry about what happens to the region's shape when the window is resized and update the area to render the glass effect in if it's not the entire client area. The source code that accompanies this article will allow you to set both an extended client frame or a region and to toggle the composition flag as well.

Drawing on Glass

Using glass as a background on your window is a bit tricky. If you render anything naturally opaque (such as GDI functions), you'll get your item rendered on glass, though sometimes with unexpected results. If you want to actually blend rendering into the glass surface, you'll need to take advantage of functionality that utilizes the alpha channel of colors, such as GDI+, Windows Presentation Foundation, or the Windows XP Theme API.

One particular gotcha is that rendering a GDI item in black uses the bit pattern 0x00000000-which also happens to be a completely transparent black if you are using an alpha channel. This means that if you draw with a black GDI brush or pen you'll get a transparent color, not a black one. The biggest problem this presents is when you try to use the default text color in a control of a text label that sits on the glass area. Since the default text color is usually black, the DWM will consider this to be transparent and the text will be written in the glass incorrectly. An example can be seen in Figure 10. The first line is written with GDI+, the second is a text label control using the default color. As you can see, it's nearly illegible because it's actually incorrectly rendered text that shows up as gray, not black.

Figure 10 Transparent Dialog Box
Figure 10 Transparent Dialog Box

Happily, there are a number of ways around this problem. Using owner-draw controls is one. Rendering to a bitmap that has an alpha channel is another. Fortunately, the easiest way to get text on controls is to let the .NET Framework 2.0 use GDI+ for you. This is easily accomplished by setting the UseCompatibleTextRendering property on your controls. By default, this property is set to false so that controls written for previous versions of the .NET Framework will render the same. But if you set it to true, your text will come out looking correct. You can set the property globally with the Application.SetUseCompatibleTextRenderingDefault method. If you're using Visual Studio® 2005, the template code will include a call to set compatible text-rendering to false in the main routine before your form is created. You can just edit this to set it to true as shown below and all your controls will look correct when written on a glass window.

static void Main()
{
    Application.EnableVisualStyles();
    Application.SetCompatibleTextRenderingDefault(true);
    Application.Run(new GlassForm());
}

You can find more information about this and using the TextRenderer class in the March 2006 MSDN Magazine article, "Build World-Ready Apps Using Complex Scripts In Windows Forms Controls", by Miguel A. Lacouture.

You should enable the glass effect just before you start rendering your window. The composition engine will look at the alpha values of your window and will apply the blur effect to those regions that are not opaque. This can be a problem when using some GDI functions because they don't preserve alpha values. You can use GDI+ when you have to, but you should be careful because GDI+ calls are rendered in software, not hardware, so a high window refresh frequency mixed with GDI+ calls can cause a significant hit to system resources.

Getting the glass effect in DirectX applications is done the same way. All you need to do is to control the alpha value of the render target in addition to using one of the two glass-enabling DWM functions. Wherever you've told the DWM to use glass, it'll use the alpha value of the render target. Anywhere else, the render target should be opaque or you'll get undefined behavior.

Thumbnails

Thumbnails are live display-only windows on open applications rendered by the DWM. Thumbnails are used by the Flip and Alt+Tab task switchers. You can essentially request a thumbnail of an application's window and have it rendered in your application. The thumbnail API will provide you with a live representation of an application's window.

Thumbnails are simple to use, as most of the hard work is done for you by Windows. The hard part is getting hold of an app's HWND. Once you've got the HWNDs you want, you simply register a thumbnail to associate that HWND with the HWND where you want the thumbnail rendered and the location in that window. The OS takes care of updating from then on. Whenever the source window is changed, the change is reflected in the target window.

To use a thumbnail, you must first register a thumbnail using the DwmRegisterThumbnail function. You provide two window handles, the source HWND (that is, the window you want the thumbnail view of) and the target HWND (the window where you want the thumbnail rendered). When you are finished using a thumbnail, you have to let the DWM know that the relationship is ending by calling DwmUnregisterThumbnail. After you have created a thumbnail, the DwmRegisterThumbnail function returns a thumbnail handle and all further thumbnail functions will take this handle as an argument. You'll have to call DwmUpdateThumbnailProperties after you register a thumbnail to get the thumbnail updating. Example code for a form that renders a live thumbnail of another window is shown in Figure 11.

Figure 11 Basic Thumbnail (Close Figure 11)
public partial class Thumbnail : Form
{
    private IntPtr m_hThumbnail;

    public Thumbnail() { InitializeComponent(); }

    public void CreateAndShow(IntPtr sourceWindow)
    {
        m_hThumbnail = DwmApi.DwmRegisterThumbnail(
            Handle, sourceWindow);

        DwmApi.DWM_THUMBNAIL_PROPERTIES m_ThumbnailProperties =
            new DwmApi.DWM_THUMBNAIL_PROPERTIES();
        m_ThumbnailProperties.dwFlags =
            DwmApi.DWM_THUMBNAIL_PROPERTIES.DWM_TNP_VISIBLE +
            DwmApi.DWM_THUMBNAIL_PROPERTIES.DWM_TNP_OPACITY +
            DwmApi.DWM_THUMBNAIL_PROPERTIES.DWM_TNP_RECTDESTINATION +
            DwmApi.DWM_THUMBNAIL_PROPERTIES.
                DWM_TNP_SOURCECLIENTAREAONLY;
        m_ThumbnailProperties.opacity = 255;
        m_ThumbnailProperties.fVisible = true;
        m_ThumbnailProperties.rcSource =
            m_ThumbnailProperties.rcDestination = new DwmApi.RECT(0, 0,
                ClientRectangle.Right, ClientRectangle.Bottom);
        m_ThumbnailProperties.fSourceClientAreaOnly = false;

        DwmApi.DwmUpdateThumbnailProperties(
            m_hThumbnail, m_ThumbnailProperties);

        Show();
    }

    protected override void Dispose(bool disposing)
    {
        if (disposing && (components != null)) components.Dispose();
        base.Dispose(disposing);

        if (m_hThumbnail != IntPtr.Zero)
        {
            if (DwmApi.DwmIsCompositionEnabled())
                DwmApi.DwmUnregisterThumbnail(m_hThumbnail);
            m_hThumbnail = IntPtr.Zero;
        }
    }
}

In addition to the two functions for registering and unregistering a thumbnail, there are two other functions that work with thumbnails. DwmQueryThumbnailSourceSize returns the source size of the specified thumbnail. DwmUpdateThumbnailProperties allows you to update the properties for a given DWM thumbnail. It takes a DWM_THUMBNAIL_PROPERTIES structure, for which a C# implementation is shown in Figure 5.

The DWM_THUMBNAIL_PROPERTIES structure allows you to specify a number of properties such as the destination rectangle in the target window (the rcDestination member) and the rectangular region of the source window to use (the rcSource member) in the event that you don't want to use the entire source window.

You can also specify the opacity of the thumbnail if you don't want it to be totally opaque by using the opacity member, where 0 is transparent and 255 is opaque. If you want the thumbnail to be invisible, you can set the fVisible flag to false. If you want to use just a windows' client area for the thumbnail instead of the entire source window (which will include the non-client areas such as the frame and title bar), you can set the fSourceClientAreaOnly Boolean to true. The dwFlags parameter is how you tell the interface which parameter(s) you are setting. When you set a parameter, you'll need to turn on the corresponding bit in the dwFlags parameter.

Finally, there's no restriction as to the size of the target window-it's perfectly legal to use the thumbnail interface to magnify the source window or to shrink it. There is a restriction about maintaining the aspect ratio. The aspect ratio of the source window is always maintained. If the source window changes size, the thumbnail will change size as well to maintain itself within the boundaries specified.

In the source code that accompanies this article, there's a button that will create a small live thumbnail of the main application window, as seen in Figure 12.

The thumbnail rendering is live, which you can easily see. If you change the main application window, you'll see the thumbnail update. You will find that with a little ingenuity, it's pretty easy to create your own task switcher using thumbnails and the FindWindow and GetWindow Win32 functions.

Figure 12a Create a Live Thumbnail
Figure 12a Create a Live Thumbnail

Figure 12b

Wrap-Up

That was a whirlwind tour of the DWM interfaces. I think you'll be able to find some nifty applications for these APIs. In particular, I think we'll be seeing some very slick application of the glass effect in the future as folks get used to applying it to their windows. For more information, I recommend Greg Schechter's blog at blogs.msdn.com/greg_schechter.

Ron Fosner has been writing 3D applications on Windows for 20 years and is starting to get the hang of it. He runs DirectX.com and loves to write fast OpenGL and Direct3D applications. You can reach him at Ron@directx.com. Thanks to Greg Schechter and Jevan Saks of Microsoft for their help with this article.


XmlLite: A Small And Fast XML Parser For Native C++

XmlLite

A Small And Fast XML Parser For Native C++

Kenny Kerr
This article discusses:
  • How XmlLite compares to other available XML parsers
  • The strengths and limitations of XmlLite
  • Reading and writing XML
  • XML security considerations
This article uses the following technologies:
XML, C++
Contents
Why a New XML Parser?
COM "Lite"
Reading XML
Writing XML
Working with Streams
Text Encoding when Reading
Text Encoding when Writing
Dealing with Large Data Values
Security Considerations
Conclusion

D
espite the ongoing success of the .NET Framework, Microsoft is still serious about native C++ development. This is demonstrated by the introduction of XmlLite, a high-performance, low-overhead XML reader and writer geared for applications written in native C++.

Managed code has broad support for XML through the System.Xml namespace, and traditional Visual Basic® and C++ applications that rely on COM have access to similar functionality from Microsoft® XML Core Services (MSXML). However, these don't offer very attractive options for native C++ developers who want a fast and lean XML parser. Enter XmlLite.

In this article, I'll explore what you can do with XmlLite. First, however, to set expectations, I want to quickly review what XmlLite does not provide-at least, not in this initial release. For starters, it doesn't provide a Document Object Model (DOM) implementation, nor does it provide XML Schema or Document Type Definition (DTD) validation. It also lacks support for high-level facilities like cursor-based navigation (such as XPath), style sheets, and serialization. Any gaps, however, can be filled as needed with functionality built on top of XmlLite in the same way that almost all of the XML functionality in the Microsoft .NET Framework is built on top of the XmlReader and XmlWriter classes.

So what does XmlLite provide? Briefly, it provides a non-cached, forward-only parser (that delivers a pull programming model) and a non-cached, forward-only XML generator. Both have proven to be very valuable functions.

Why a New XML Parser?

Developers grow accustomed to libraries they use every day, and with their extensive use of XML, they are certainly going to ask some tough questions about a brand new XML parser. To appreciate the value of this new parser, let's first consider what the XML parser landscape looks like today.

Naturally, if an application is already making use of the .NET Framework, then the decision is usually a simple one: just use System.Xml. As a testament to this, the design of XmlLite is based on the design of the XmlReader and XmlWriter classes in the .NET Framework. There are usually no advantages to using XmlLite from managed applications written in C++. After all, XmlLite's functionality is much lighter than that provided by the XmlReader and XmlWriter classes. (The table in Figure 1 outlines how the main types in XmlLite map to those in the .NET Framework.) If, on the other hand, an application makes use of native code exclusively, then MSXML has traditionally been the solution of choice as far as Microsoft technology is concerned.

Figure 1 XmlLite Classes Map to the .NET Framework (Close Figure 1)

XmlLite | .NET Framework
IXmlReader interface | XmlReader class
IXmlWriter interface | XmlWriter class
XmlReaderProperty enum | XmlReaderSettings class
XmlWriterProperty enum | XmlWriterSettings class
CreateXmlReaderInputWithEncodingCodePage CreateXmlReaderInputWithEncodingName CreateXmlWriterOutputWithEncodingCodePage CreateXmlWriterOutputWithEncodingName | Encoding class
XmlNodeType enum | XmlNodeType enum

MSXML offers two very different XML parsers. The first is a DOM implementation that has been available in various incarnations. If you are working with relatively small XML documents and require random access to the XML documents for in-memory reads and writes, then the DOM implementation is a reasonable choice. Later versions of MSXML introduced an implementation of Simple API for XML (SAX2). Whether it is actually "simple" is debatable. When using SAX2, before you even get off the ground, you need to implement at least two COM interfaces-one that receives notifications of the various nodes in the XML document and one that receives notifications of parsing errors.

The reason the SAX2 implementation was added to MSXML was because, unlike the DOM implementation, the SAX2 parser reads an XML document as a stream and notifies you when various nodes have been reached. This means that your application's memory consumption does not grow along with the size of the document you are parsing.

The problem with SAX2, and the reason why the .NET Framework does not provide an implementation of it, is the inherent complexity of the SAX2 model. It requires interfaces or events to be implemented and forces the developer to employ a more indirect programming model, requiring the developer to manage additional state that inevitably complicates the application. Instead, the XmlReader and XmlWriter classes in the .NET Framework, as well as XmlLite's IXmlReader and IXmlWriter interfaces, provide a straightforward parser that can be used directly within a function without having to manage any external state or notifications.

Due to the simplicity of its design, XmlLite manages to provide considerably better performance, even when compared to the MSXML SAX2 implementation. Although the SAX2 parser is better at handling large documents than the DOM implementation, it pales in comparison to XmlLite.

Simply put, XmlLite outperforms MSXML and it is much easier to use from native C++. MSXML will remain the most viable solution for Visual Basic and COM-based scripting languages, but now native Visual C++® finally has an XML parser designed specifically for it. Although XmlLite is included with Windows Vista™ and later versions, an update is also available for 32-bit and 64-bit versions of Windows® XP and Windows Server® 2003. Since COM registration is not involved, this update package should not cause the headaches that MSXML has typically posed with regard to installation and versioning.

COM "Lite"

XmlLite is not just a catchy name; it is, in fact, a light XML parser. XmlLite takes the best of COM, namely the programming discipline and conventions, and drops the complex and potentially unnecessary parts, such as the COM registration, runtime services, proxies, threading models, marshaling, and so on.

Functions exported from XmlLite.dll create the XML reader and writer. You gain access to them by linking to XmlLite.lib and including the XmlLite.h header file from the Windows SDK. The resulting COM-style interfaces use the familiar IUnknown interface methods for lifetime management. The COM IStream interface also plays a part and represents storage. Other than that, there are no dependencies on COM; you do not need to register any COM classes or even call the obligatory CoInitialize function. The Active Template Library (ATL) CComPtr class takes care of the small slice of COM that remains. You do, however, need to concern yourself with thread safety, as XmlLite is not thread-safe for the sake of performance in single-threaded scenarios.

I use the COM_VERIFY macro in the following samples to clearly identify where methods return an HRESULT that needs to be checked. You can replace this with appropriate error handling-whether that is throwing an exception or returning the HRESULT yourself.

Reading XML

XmlLite provides the CreateXmlReader function that returns an implementation of the IXmlReader interface:

CComPtr<IXmlReader> reader;
COM_VERIFY(::CreateXmlReader(__uuidof(IXmlReader),
                             reinterpret_cast<void**>(&reader),
                             0));

Although optional, the CComPtr class template ensures that the interface pointer is released promptly.

CreateXmlReader accepts an interface identifier (IID) as well as a pointer to a void pointer. This is a common pattern in COM programming, allowing the caller to specify the type of interface pointer to return. My example uses the __uuidof operator, which is a Microsoft-specific keyword that extracts the GUID associated with a type. In this case, it is used to retrieve the IID for the interface. The final argument to CreateXmlReader accepts an optional IMalloc implementation to allow the caller to control memory allocations.

Once you have created the reader, you need to indicate the storage that the reader will use as input. The IStream interface represents storage, letting you use XmlLite with any stream implementation you might devise:

CComPtr<IStream> stream;

// Create stream object here...

COM_VERIFY(reader->SetInput(stream));

(I'll discuss streams later in this article.)

Once you have set the input for the XML reader, you can read with repeated calls to the Read method. The Read method accepts an optional argument that returns the node type on each successful call. The Read method returns S_OK to indicate that the next node has been successfully read from the stream and S_FALSE to indicate that the end of the stream has been reached. Here's an example of how you can enumerate through the nodes:

HRESULT result = S_OK;
XmlNodeType nodeType = XmlNodeType_None;

while (S_OK == (result = reader->Read(&nodeType)))
{
    // Get node-specific info
}

To enumerate the attributes of the current node, you use the MoveToFirstAttribute and MoveToNextAttribute methods. Both return S_OK if the reader was successfully repositioned and S_FALSE if no more attributes are present. The following example illustrates how you can enumerate through the attributes for a given node:

for (HRESULT result = reader->MoveToFirstAttribute();
     S_OK == result;
     result = reader->MoveToNextAttribute())
{
    // Get attribute-specific info
}

When you call the IXmlReader's Read method, it automatically stores any node attributes in an internal collection. This allows you to move the reader to a specific attribute by name using the MoveToAttributeByName method. However, it is usually more efficient to enumerate the attributes and store them in an application-specific data structure. Note that you can also determine the number of attributes in the current node using the GetAttributeCount method.

Once you have settled on a node or attribute, getting its information is straightforward. This example demonstrates how you can get the namespace URI and local name for a given node:

PCWSTR namespaceUri = 0;
UINT namespaceUriLength = 0;

COM_VERIFY(reader->GetNamespaceUri(&namespaceUri,
                                   &namespaceUriLength));

PCWSTR localName = 0;
UINT localNameLength = 0;

COM_VERIFY(reader->GetLocalName(&localName,
                                &localNameLength));

All of the IXmlReader methods that return string values follow this pattern. The first argument accepts a pointer to a wide-character pointer constant. The second argument is optional and if it's not zero, it will return the length of the string measured in characters excluding the null terminator.

Here is another example of the emphasis on performance. The string pointers returned from IXmlReader methods are only valid until you move the reader to another node or invalidate the current node in some other way, such as by setting a new input stream or releasing the IXmlReader interface. In other words, the IXmlReader is not returning a copy of the string to the caller.

Unlike its counterpart in the .NET Framework, IXmlReader does not provide any methods for reading typed content. If a particular element or attribute contains a number or date, for example, you need to first get its string representation and then convert it yourself as necessary. Many of the other helper methods present in the XmlReader class of the .NET Framework are also not present in IXmlReader, but can be written as helper functions. XmlLite certainly conforms to the C++ philosophy of minimal interface design.

Figure 2 shows the objects and abstractions involved in reading an XML document with IXmlReader. Keep in mind, though, that IStream could abstract any storage, and the file shown here is just a common example.

Figure 2 Reader
Figure 2 Reader

Writing XML

XmlLite provides the CreateXmlWriter function to return an implementation of the IXmlWriter interface:

CComPtr<IXmlWriter> writer;

COM_VERIFY(::CreateXmlWriter(__uuidof(IXmlWriter),
                             reinterpret_cast<void**>(&writer),
                             0));

Once you have created the writer, you need to indicate the storage that the writer will use as output:

CComPtr<IStream> stream;

// Create stream object here

COM_VERIFY(writer->SetOutput(stream));

Before beginning to write, you can modify the writer properties. The XmlWriterProperty enum defines the properties that are available. You might, for example, want to specify whether the XML output is indented for human readability, as can be done with the SetProperty method:

COM_VERIFY(writer->SetProperty(XmlWriterProperty_Indent, TRUE));

You can then start writing to the underlying stream using IXmlWriter methods. XmlLite supports XML fragments. If you are planning to write a complete XML document, you should start by calling the WriteStartDocument method, which takes care of writing the XML declaration. The declaration is dependent on the encoding in use but the default is UTF-8, which should be suitable in most cases. (I'll cover text encoding in a moment.) A number of WriteXxx methods are provided for writing various node types, attributes, and values.

Consider this example:

COM_VERIFY(writer->WriteStartDocument(XmlStandalone_Omit));
COM_VERIFY(writer->WriteStartElement(0, L"html",
                                     L"http://www.w3.org/1999/xhtml"));

COM_VERIFY(writer->WriteStartElement(0, L"head", 0));
COM_VERIFY(writer->WriteElementString(0, L"title", 0, L"My Web Page"));
COM_VERIFY(writer->WriteEndElement()); // </head>

COM_VERIFY(writer->WriteStartElement(0, L"body", 0));
COM_VERIFY(writer->WriteElementString(0, L"p", 0, L"Hello world!"));

COM_VERIFY(writer->WriteEndDocument());

The WriteStartDocument method handles writing the XML declaration to the stream. Its single argument accepts a value from the XmlStandalone enum, indicating whether the standalone document declaration appears, and if so, what value it holds. When writing an XML fragment, you typically omit the call to WriteStartDocument.

The WriteStartElement method accepts three arguments: the first specifies an optional namespace prefix for the element, the second specifies the local name of the element, and the third specifies the optional namespace URI. WriteElementString is one of the rare convenience methods provided by XmlLite. The following code for writing the XHTML document's title is equivalent to the WriteElementString used in the previous example:

COM_VERIFY(writer->WriteStartElement(0, L"title", 0));
COM_VERIFY(writer->WriteString(L"My Web Page"));
COM_VERIFY(writer->WriteEndElement());

Clearly, the WriteElementString method is not strictly necessary, but it certainly is useful.

Finally, the WriteEndDocument method closes the document. You may have noticed that the body and html elements were not explicitly closed. WriteEndDocument automatically closes any open elements. For that matter, releasing the writer also closes any remaining elements. However, the practice of not explicitly closing such elements can lead to bugs if you're not careful, as the lifetime of the stream and the lifetime of the writer can often be different. Speaking of which, if you need to ensure that all writes have been written to the underlying stream, just call IXmlWriter's Flush method.

Figure 3 shows the flow of objects and abstractions involved in writing an XML document with IXmlWriter. Keep in mind that IStream could abstract any storage and the file here is just a common example.

Figure 3 Writer
Figure 3 Writer

Working with Streams

Thus far, I haven't talked much about streams. Unlike some of the more full-featured XML libraries, XmlLite does not provide any supporting functionality for reading from and writing to common storage locations (such as files or over network protocols). Because of this, you need to provide an IStream implementation for whatever storage you want to read from or write to. Implementing the IStream interface is not complicated, but in many cases, you won't need to do this since implementations may already exist.

The CreateStreamOnHGlobal function provides an IStream implementation backed by virtual memory. The first argument is an optional memory handle created using the GlobalAlloc function. You can, however, simply pass zero and CreateStreamOnHGlobal will create a memory object for you. The following example creates an IStream implementation that is backed by system memory and will dynamically grow as needed:

CComPtr<IStream> stream;
COM_VERIFY(::CreateStreamOnHGlobal(0, TRUE, &stream));

Freeing the stream will free the memory.

The SHCreateStreamOnFile function provides another useful IStream implementation. It creates an IStream backed by a file:

CComPtr<IStream> stream;
COM_VERIFY(::SHCreateStreamOnFile(L"D:\\Sample.xml",
                                  STGM_WRITE | STGM_SHARE_DENY_WRITE,
                                  &stream));

Text Encoding when Reading

Although XmlLite, by default, uses UTF-8 for writing and when attempting to detect the text encoding when reading, you can override this behavior. First, let's take a look at what you get automatically. Given a stream, IXmlReader will detect encoding hints via a byte order mark as a preamble to the XML. IXmlReader will also honor any encoding specified in the XML declaration. These characteristics are both expected of any XML parser. If you have an input stream that perhaps does not have any encoding information defined and XmlLite cannot heuristically determine the encoding in use, then you can direct IXmlReader to a specific encoding given a code page or encoding name.

Instead of passing the stream directly to the IXmlReader, you can create an XML reader input object under the guise of the IXmlReaderInput interface. Two functions are provided for creating the input object wrapping the input stream. The CreateXmlReaderInputWithEncodingCodePage function accepts an encoding in the form of a codepage number. The CreateXmlReaderInputWithEncodingName function accepts an encoding using its canonical name. Apart from this, the functions have identical signatures. To recap, ordinarily you would set the input stream for the XML reader as follows:

CComPtr<IStream> stream;

// Create stream object here

COM_VERIFY(reader->SetInput(stream));

To override the encoding, change the code to this:

CComPtr<IStream> stream;

// Create stream object here

CComPtr<IXmlReaderInput> input;

COM_VERIFY(::CreateXmlReaderInputWithEncodingName(stream,
                                                  0, // default allocator
                                                  L"ISO-8859-8",
                                                  TRUE, // hint
                                                  0, // base URI
                                                  &input));

COM_VERIFY(reader->SetInput(input));

The first argument indicates the stream that the XML reader will read from. The second argument accepts an optional IMalloc implementation. If provided, it overrides the XML reader's own implementation. The third argument specifies the encoding name. The documentation at msdn2.microsoft.com/ms752827.aspx lists the encodings natively supported; to support other encodings, you can provide an IMultiLanguage2 interface implementation. The next argument indicates whether the specified encoding must be used or whether it is merely a hint. Specifying TRUE instructs the parser to attempt to use the suggested encoding but if it fails it is free to attempt to determine the actual encoding heuristically. Specifying FALSE instructs the parser to attempt the suggested encoding and return an error if it does not match the input stream. The next argument accepts an optional base URI that may be used for resolving external entities. And the last argument returns an interface pointer representing the input object to pass to the SetInput method.

Text Encoding when Writing

The XML writer will determine the encoding to use based on the object passed to the SetOutput method. If the object implements the IStream interface, or even the limited ISequentialStream interface, the XML writer will employ UTF-8 encoding. You can create an XML writer output object to override this behavior. Two functions are provided for creating the output object wrapping the output stream. The CreateXmlWriterOutputWithEncodingCodePage function accepts an encoding in the form of a codepage number and the CreateXmlWriterOutputWithEncodingName function accepts an encoding using its canonical name. Apart from this, the functions have identical signatures. Ordinarily, you would set the output stream for the XML writer like this:

CComPtr<IStream> stream;

// Create stream object here

COM_VERIFY(writer->SetOutput(stream));

To override the default encoding, write this code:

CComPtr<IStream> stream;

// Create stream object here

CComPtr<IXmlWriterOutput> output;

COM_VERIFY(::CreateXmlWriterOutputWithEncodingName(stream,
                                                   0,
                                                   L"ISO-8859-8",
                                                   &output));

COM_VERIFY(writer->SetOutput(output));

The first argument indicates the stream that the XML writer will write to. The second argument accepts an optional IMalloc implementation. If provided, it overrides the XML writer's own implementation. The third argument specifies the encoding name. The last argument returns an interface pointer representing the output object to pass to the SetOutput method.

Dealing with Large Data Values

To limit memory usage when reading large data values, the XML reader provides a mechanism for reading values in chunks. The IXmlReader ReadValueChunk method will read up to a set maximum number of characters, moving the reader forward in anticipation of subsequent calls. This example illustrates how you might call ReadValueChunk repeatedly to read a large data value:

CString value;

WCHAR chunk[256] = { 0 };
HRESULT result = S_OK;
UINT charsRead = 0;

while (S_OK == (result = reader->ReadValueChunk(chunk,
                                                countof(chunk),
                                                &charsRead)))
{
    value.Append(chunk, charsRead);
}

ReadValueChunk returns S_FALSE when no more data is available. In this example, I am writing the chunks to a CString object. This is only to illustrate how chunk lengths are managed and would obviously defeat the benefits of chunking in practice.

Security Considerations

XML-centric apps invariably have to deal with XML from untrusted sources. XmlLite provides a number of facilities for protecting your applications from known and future vulnerabilities.

XML documents can contain references to external entities. Some XML parsers resolve these automatically. Although potentially useful, this approach can lead to exploits if the XML resolver is not carefully written to mitigate various threats. XmlLite does not automatically resolve external entities, nor does it provide an XML resolver. To provide your own implementation (if necessary), implement the IXmlResolver interface and use the XmlReaderProperty_XmlResolver property with the IXmlReader SetProperty method to instruct the reader to use your resolver.

XML documents may also contain DTD processing instructions. Although XmlLite does not support document validation-with either XML Schema or DTD-it does support DTD entity expansion and default attributes. Since these DTDs can contain references to external entities, they may open up your application to various attacks. XmlLite disables DTD processing by default. You can allow DTD processing by setting the XmlReaderProperty_DtdProcessing property to the DtdProcessing_Parse value. There is also built-in mitigation for the DTD entity expansion attack (also known as the billion laughs attack), controlled by XmlReaderProperty_MaxEntityExpansion. The default for this property is 100,000.

Another way attackers can exploit applications using XML is by creating documents with very long names. If not prevented, this may use up tremendous amounts of memory and allow denial of service attacks. I have already hinted at ways that can be done. One obvious way to mitigate such a threat is to read large data values in chunks as the previous section described. Another useful technique is to provide a custom IMalloc implementation with restrictions on memory allocations. Given an input stream supporting random access, you can also instruct the XML reader to avoid caching attributes using the XmlReaderProperty_RandomAccess property. This reduces the amount of memory used to read a start element tag, but may also slow down parsing as the parser must seek back and forth to retrieve the various attribute values upon request.

Excessively deep XML hierarchies can also use up system resources quickly. To stop attackers from providing XML documents that have excessively deep hierarchies, you can use the XmlReaderProperty_MaxElementDepth property to limit the depth that the parser will allow. This property defaults to 256.

Conclusion

XmlLite provides a powerful XML parser for your native C++ applications. It emphasizes performance, is aware of the system resources it uses, and supports a great deal of flexibility in controlling these characteristics. With support for all common text encoding, XmlLite is a very useful and practical tool that can simplify XML usage in native C++ applications. For more information, see the XmlLite documentation at msdn2.microsoft.com/ms752872.aspx.

Kenny Kerr is a software craftsman specializing in software development for Windows. He has a passion for writing and teaching developers about programming and software design. Reach Kenny at http://weblogs.asp.net/kennykerr.


Event Tracing: Improve Debugging And Performance Tuning With ETW

Event Tracing

Improve Debugging And Performance Tuning With ETW

Dr. Insung Park and Ricky Buch
This article discusses:
  • Understanding the ETW architecture
  • What's new for ETW in Windows Vista
  • Programming with the event provider APIs
This article uses the following technologies:
Windows Vista
Contents
Event Tracing for Windows
The Unified Event Provider Model and APIs
Design and Implementation Guidelines
Tools and Support
Conclusion

T
he increasing size and complexity of today's software systems make software development and management a tough challenge. Accounting for all execution states is nearly impossible, and applications often exhibit behaviors developers did not expect. Furthermore, an enormous number of hardware combinations and constantly varying workload characteristics add to the difficulty of diagnosing various software problems. It is not surprising then that reliability and manageability are becoming important features themselves. These features, in turn, create a need for instrumentation.

Clever instrumentation added for a few key error states in software execution can drastically reduce the time spent debugging problems. Instrumentation is also useful in other cases. Certain undesirable situations, such as software and hardware failures and low resource conditions, must be monitored and acted upon in a managed business environment for a large group of computers. Moreover, instrumentation can be very helpful for resolving performance problems, which can be hard to diagnose since they are sensitive to external workload, configuration parameters, and underlying hardware and software status. Traces from a production machine during a period of performance degradation may enable developers and administrators to pinpoint poorly performing components or services or identify bottlenecks that were not foreseen during development. Finally, IT professionals use various management tools to compute resource usage statistics from transaction traces for capacity planning and trend analysis.

Event Tracing for Windows® (ETW) is a general-purpose, high-speed tracing facility provided by the operating system. Using a buffering and logging mechanism implemented in the kernel, ETW provides a tracing mechanism for events raised by both user-mode applications and kernel-mode device drivers. Additionally, ETW gives you the ability to enable and disable logging dynamically, making it easy to perform detailed tracing in production environments without requiring reboots or application restarts. The logging mechanism uses per-processor buffers that are written to disk by an asynchronous writer thread. This allows large-scale server applications to write events with minimum disturbance.

ETW was first introduced on Windows 2000. Since then, various core OS and server components have adopted ETW to instrument their activities, and it's now one of the key instrumentation technologies on Windows platforms. A growing number of third-party applications are using ETW for instrumentation as well, and some take advantage of the events provided by Windows itself. ETW has also been abstracted into the Windows preprocessor (WPP) software tracing technology, which provides a set of easy-to-use macros for tracing "printf" style messages for debugging during development.

On Windows Vista™, ETW has gone through a major upgrade, and one of the most significant changes is the introduction of the unified event provider model and APIs. In short, the new unified APIs combine logging traces and writing to the Event Viewer into one consistent, easy-to-use mechanism for event providers. At the same time, several new features have been added to improve developer and end user experience. In this article, we will introduce the new ETW provider model and describe how developers can adopt the new model in their Windows Vista-based applications.

We start by presenting an overview of ETW architecture and usage model, then describe the new event model and the APIs. We'll follow with a short guide to event instrumentation design and implementation and, finally, look at in-box tools for controlling ETW sessions, processing logged events, and analyzing them for higher-level report generation.

Event Tracing for Windows

The core architecture of ETW is illustrated in Figure 1. As shown, there are four main types of components in ETW: event providers, controllers, consumers, and event trace sessions. Buffering and logging take place in event tracing sessions, which accept events and create a trace file. There are a number of logging modes available for ETW sessions. For instance, a session can be configured to deliver events directly to consumer applications or to overwrite old events in a file by wrapping around when a certain size is reached. A separate writer thread created for each session flushes them to a file or to real-time consumer applications. To enable high-performance, per-processor buffers are used to eliminate the need for a lock in the logging path.

Figure 1 ETW Architecture
Figure 1 ETW Architecture(Click the image for a smaller view)
Figure 1 ETW Architecture
Figure 1 ETW Architecture (Click the image for a larger view)

An event provider is a logical entity that writes events to ETW sessions. Any recordable activity of significance can be an event, and each is represented by an event logged to ETW. An event provider can be a user-mode application, a managed application, a driver, or any other software entity. The only requirement is that the event provider must register a provider ID with ETW through the registration API. A provider first registers with ETW and writes events from various points in the code by invoking the ETW logging API. When a provider is enabled dynamically by the ETW controller application, calls to the logging API send events to a specific trace session designated by the controller. Each event sent by the event provider to the trace session consists of a fixed header that includes event metadata and additional variable user-context data. Due to the growing event instrumentation in many OS components, even a simple application for Windows Vista will already contain several components that are event providers.

When an event is logged to a session, ETW adds a few extra data items along with the user-provided data. They include timestamp, process and thread ID, processor number, and CPU usage data of the logging thread. These data items are recorded in the ETW event header and passed on to event consumers along with the variable event content given by the provider. Many trace consumers find these data fields to be essential in their analysis.

A controller starts and stops ETW sessions and enables providers to them. In some scenarios, such as debugging and diagnosis, a controller tool is invoked as needed to collect in-depth traces. In contrast, for events such as admin-targeted events (we will define these in a later section) that need to flow to the Event Viewer at all times, providers are enabled automatically by the event log service when they register. A controller must have ETW permission on Windows Vista to control sessions, which is given only to a small group of privileged users by default.

Lastly, a consumer is an application that reads log files or listens to a session for real time events and processes them. Event consumption is callback-based; a consumer registers an event callback, which ETW calls with one event at a time. Events are delivered to the ETW consumer in chronological order. There are general-purpose event consumer tools that dump the events into various formats. Figure 2 shows an XML dump of a "Process" event logged by the kernel provider as generated by the tracerpt.exe tool on Windows Vista. This event indicates the start of a Notepad process. Since events contain custom user content logged by the provider, some type of metadata is needed to decode them correctly. The providers using the new APIs are expected to supply an event manifest-an XML file-that defines all events that providers write along with their layout information. A general-purpose consumer application uses Trace Data Helper (TDH) APIs to retrieve the event metadata, decode the events, and display them.

Figure 2 XML Dump of Process Start Event (Close Figure 2)
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
    <System>
        <Provider Guid="{9e814aad-3204-11d2-9a82-006008a86939}" />
        <EventID>0</EventID>
        <Version>2</Version>
        <Level>0</Level>
        <Task>0</Task>
        <Opcode>1</Opcode>
        <Keywords>0x0</Keywords>
        <TimeCreated SystemTime="2006-12-18T12:26:27.887309500Z" />
        <Correlation
            ActivityID="{00000000-0000-0000-0000-000000000000}" />
        <Execution ProcessID="3396" ThreadID="3260" ProcessorID="0"
            KernelTime="390" UserTime="195" />
        <Channel />
        <Computer />
    </System>
    <EventData>
        <Data Name="UniqueProcessKey">0xFFFFFA800143FA80</Data>
        <Data Name="ProcessId">0x10EC</Data>
        <Data Name="ParentId">0xD44</Data>
        <Data Name="SessionId">1</Data>
        <Data Name="ExitStatus">0</Data>
        <Data Name="UserSID">guest</Data>
        <Data Name="ImageFileName">notepad.exe</Data>
        <Data Name="CommandLine">notepad</Data>
    </EventData>
    <RenderingInfo Culture="en-US">
        <Opcode>Start</Opcode>
        <Provider>MSNT_SystemTrace</Provider>
        <EventName xmlns=
            "http://schemas.microsoft.com/win/2004/08/events/trace">
            Process</EventName>
        </RenderingInfo>
        <ExtendedTracingInfo xmlns="
            http://schemas.microsoft.com/win/2004/08/events/trace">
        <EventGuid>{3d6fa8d0-fe05-11d0-9dda-00c04fd7ba7c}</EventGuid>
        </ExtendedTracingInfo>
</Event>

To many, tracing means collecting events from certain providers of interest. In this way of thinking, an event trace session is tied to one or more providers in a conceptual collection as a whole, and a session itself (the logging engine) is often overlooked. The ETW architecture allows for more dynamic and flexible trace and event management. Here, sessions and providers exist in different spaces. A controller is the one that starts and stops ETW sessions and enables providers to sessions dynamically. Thus, a controller can choose to enable a group of providers to a session, disable some of them after a while, and enable another provider to that same session later. Sessions operate in the kernel and are not statically tied to providers. Likewise, providers typically are not aware of which sessions their events are being logged to. There are large scale applications and services that are providers, controllers, and consumers all at the same time. APIs are provided for all operations for providers, controllers, and consumers, and applications may assume any combination of roles. In general, however, developers implement only event providers and use in-the-box tools to collect traces and view them.

One of the advantages of the separation of providers and trace sessions is that tracing becomes immune to application problems such as crashes or hangs. Events logged by providers before a crash will be in the kernel memory, if not in a trace file already, which makes this particularly useful in debugging application anomalies.

As mentioned earlier, events are used by developers, IT administrators, and management tool developers for debugging, monitoring, diagnosis, and capacity planning. The usual analysis methodologies based on events can be categorized into the following techniques.

Scanning Users scan through the event dump to find a single important event or a small pattern of known events. This is typically done when events are used to debug for failure cases corresponding to end-user problems or to search for significant failures in the event log.

Delta Analysis Since ETW captures timestamp and CPU usage numbers for each event, such as simple delta analysis of the form

Property (Event B)-Property (Event A)
allows for response time and CPU usage statistics of application activities. If two events mark a beginning and end of an activity, a large set of events collected from production mode applications can be processed in this manner to produce a summary of response time and CPU usage statistics.

Statistical Analysis Sometimes, just counting certain events provides extra insight into software behavior.

State Machine and Resource Tracking A sufficient set of events enables the construction of a state machine and in turn, a simulation based on traces. For instance, since the majority of core OS activities are instrumented with ETW events, OS traces can be used to build a state machine that keeps track of scheduler, memory, I/O activities, and so forth.

End-to-End Tracing Large applications often consist of a number of distributed components integrated via complicated interconnections. They frequently encompass multiple machines, each serving a different role. Instrumentation geared toward request tracking is one of the approaches to overcoming the increasing difficulty of diagnosing problems in such environments. In this framework, instrumentation points are added throughout applications that record activities along with the unique ID for the request currently being served. After traces are collected, events that correspond to the same request are correlated during event consumption such that its activity and progress can be tracked. Later, specific requests of interest can be looked at individually for problems in different service stages, or a group of requests can be summarized through a statistical analysis. Server Performance Advisor (SPA), available from the Microsoft Download Center, is a server management and diagnosis tool that employs server instrumentation designed for end-to-end tracing.

The Unified Event Provider Model and APIs

On Windows Vista, we introduce a new set of event provider APIs that are easier to use while offering more features and enhanced security options. The new APIs are also used to write to the Event Viewer, uniting trace and event logs into one consistent API set. This section describes the model and the APIs in more detail. The differences between the new APIs and the existing ones will be pointed out as necessary.

In order to become an ETW provider, a software component needs to register with ETW through the EventRegister API. EventRegister requires a GUID called ProviderId that uniquely identifies the provider. Any software entity (application, shared DLL, or driver) can register as a provider since a provider does not have to be bound to any OS entity. Registration is typically done at the entry point into the component: a DLL attach routine or a driver entry, for example. A registration handle is returned to be used in subsequent logging API calls. Finally, an EventUnregister call is made at the end of the provider execution.

With the existing provider APIs, providers must supply a callback for enable/disable notification. That is, if a controller enables the provider, the registered callback function is invoked with enable settings. One of the parameters to the callback is a handle to the session that the provider is enabled to. Upon receiving an enable callback, the provider sets a global variable (such as TracingOn) to indicate whether tracing is on or off and stores the session handle as well. Then it uses this session handle, which it obtained from the callback, in logging API calls (conditionally based on the value of TracingOn).

In the new ETW provider model, ETW remembers the enable settings on behalf of providers. In other words, providers register and invoke logging calls without having to check whether they are currently enabled or not. Within the logging API, ETW quickly looks at the enable settings and sends the events to sessions only if they are enabled. If they're not, the logging call is discarded. Hence, enable/disable callback is optional in the new model. However, an enable callback may still be necessary in some scenarios. For instance, instrumentation intended for a state machine construction often requires a snapshot or state rundown events at the beginning and end of traces.

The logging API, EventWrite, takes a registration handle (as opposed to a session handle in the old model). Using a registration handle in the logging calls allows enable settings to be transparent to providers. Since the handle used in logging is an opaque handle in the new model, ETW is now capable of multicasting events. That is, a provider can be enabled to more than one ETW session, which was not possible in the old model where a session handle is used.

ETW provides separate APIs, EventEnabled and EventProviderEnabled, that test if the provider is enabled. Although the logging APIs already check the enable setting before writing events, sometimes providers may require additional work when tracing is enabled. One such scenario involves gathering and constructing informational event data that is not necessarily required for program execution. With these APIs, a provider can find out at any time whether tracing is enabled or not, if there is a need.

As stated previously, users can add variable context data to each event. The logging API employs a scatter/gather mechanism for picking up event-specific data items. Callers pass in additional event data items by constructing an array of data descriptors. A data descriptor is a struct with pointer and size fields. Thus, users add one data descriptor for each data item to log. A macro (EventDataDescCreate) is provided for easy construction of a data descriptor. ETW then copies user-provided content into its session buffers during logging.

In the corresponding event manifest, the layout of an event should be specified through a <Template> tag. A template describes user-specified context data that each event includes. The template can define the layout, which may contain individual data fields, such as integers and strings, or complex data structures such as an array of structs. A template is not required for all events; if no template is specified, that event is expected to have no user-provided data. In the manifest, a template may be shared by multiple events, such as a Start and Stop event having the same context information. When a consumer application encounters an event, it locates an event template through the TDH APIs and decodes the variable event data accordingly. Providers using the old APIs supply layout information through Windows Management Instrumentation (WMI) Managed Object Formats (MOFs).

Figure 3 contains sample provider code that uses the new event APIs for registration and logging. This is a user-mode provider that employs user-mode provider APIs, but a corresponding set of kernel-mode provider APIs are also available. The first event in Figure 3 writes two user-provided data items. One is of type ULONG and the other a NULL terminated WCHAR string. EventDataDescCreate is invoked to construct the appropriate data descriptor array. In addition to the EventWrite API shown here, there are two more logging APIs: EventWriteString and EventWriteTransfer. EventWriteString allows simple logging of a non-manifested string. When EventWriteString is called, ETW marks the header denoting that the event data is a single NULL terminated WCHAR string. If a consumer sees this in the header, it treats the user data as a string, without having to search for the event schema through TDH. EventWriteString enables a quick logging of strings without changing the manifest.

Figure 3 ETW Provider (Close Figure 3)
#include <myevents.h>   // Header generated from manifest.
                        // Contains MyProviderId and event descriptors.

REGHANDLE MyProvRegHandle;
ULONG MyInteger;
PWCHAR MyString;
ULONG MyStringLength;
EVENT_DATA_DESCRIPTOR DataDescriptor[2];

...

// Register the ETW provider.
Status = EventRegister(&MyProviderId,      // ProviderId (GUID)
                       NULL,               // Optional Callback
                       NULL,               // OPtioanl Callback Context
                       &MyProvRegHandle);  // Registration Handle

...

// Construct DataDescriptor and write an event with
// MyInteger and MyString.
EventDataDescCreate(&DataDescriptor[0],    // DataDescriptor
                    &MyInteger,            // Pointer to the data
                    sizeof(ULONG));        // Size of data
EventDataDescCreate(&DataDescriptor[1], &MyString, MyStringLength);

Status = EventWrite(MyProvRegHandle,       // Registration Handle
                    MyEventDescriptor1,    // EventDescriptor
                    2,                     // DataDescriptor array size
                    DataDescriptor);       // DataDescriptor array

...

// Write another event with no user data.
if (EventEnabled(MyProvRegHandle, MyEventDescriptor2)) {
    // Do extra work if enabled and write event.
    ...

    Status = EventWrite(MyProvRegHandle, MyEventDescriptor2, 0, NULL);
}

...

// Unregister the ETW provider.
Status = EventUnregister(MyProvRegHandle);

The EventWriteTransfer and EventActivityIdControl APIs cater to the need of end-to-end tracing instrumentation. As described earlier, end-to-end tracing is an instrumentation methodology targeting server applications that perform different activities simultaneously for a number of user requests. For instance, a request for a script execution on a Web page leaves a client machine and arrives at the network layer on a server. Then it travels through the HTTP driver, IIS, the ASP.NET engine, and maybe an Exchange Server on another machine. The goal of end-to-end tracing is to record all the relevant activities regarding this request through ETW events for later debugging and performance analysis. This requires a unique ID that can identify individual requests. Correlation can be performed during consumption with the help of this unique activity ID.

ETW addresses this need by introducing an ActivityId in each event. Every event logged with the new API automatically picks up the current activity ID stored in the executing thread. The activity ID is displayed in the XML dump in <Correlation ActivityId> tag in the <System> section. The provider can get, set, and create an activity ID for the executing thread using the EventActivityIdControl API. The activity ID can travel with the request across multiple components. Unfortunately, it may not be possible to propagate the activity ID in some cases due to public protocol restriction, design restriction, and so on. The EventWriteTransfer API writes a transfer event, indicating a transfer of activity IDs. In addition to all the parameters of EventWrite, EventWriteTransfer takes two more arguments-ActivityId and RelatedActivityId.

Every event is stamped with the provider ID and assigned an entity called the event descriptor, which defines standard event information and provides further identification and semantics to it. Developers determine the event descriptors for instrumentation points during the instrumentation design phase and write corresponding entries in the event manifest. Then the Message Compiler in a development environment generates event descriptors in a header file from the given event manifest, which in turn are included and used in source files. Programmatically, event descriptor is a struct consisting of the following fields: Id, Version, Channel, Level, Opcode, Task, and Keywords:

typedef struct _EVENT_DESCRIPTOR {
    USHORT Id;
    UCHAR Version;
    UCHAR Channel;
    UCHAR Level;
    UCHAR Opcode;
    USHORT Task;
    ULONGLONG Keyword;
} EVENT_DESCRIPTOR, *PEVENT_DESCRIPTOR;

An event ID is used to uniquely identify an event in a provider. When an event is defined in the manifest, the event ID is the only mandatory entry. Upon encountering an event, a consumer uses its provider ID (GUID) and event ID (USHORT) to locate the manifest for it. Likewise, a version offers events to be changed and augmented in later releases while retaining the same semantics and event ID. Therefore, the event ID and version together with the provider ID can uniquely identify an event.

A channel defines a group of events for a target audience. A channel belongs to one of the four types: admin, operational, analytic, and debug. Events raised into an admin channel are actionable events; upon receiving an event, the administrator should immediately know why the event was raised and what to do about it. Events raised into an operational channel are targeted at high-level monitoring tools and support staff; they offer more detailed context and are more frequent than admin channel events. Events grouped in admin and operational channels are sent to the event log automatically and displayed in the Event Viewer. An analytic channel is for traditional traces that are targeted at expert-level support professionals or detailed diagnosis and troubleshooting tools. A debug channel is to be used for debug messages and contains events that are meant to be consumed by developers. Analytic and debug channel events are not enabled by default. Through channels, events intended for different purposes and audiences can be added with one set of APIs.

When enabling a provider, a controller can specify a level (1-byte integer) and keywords (8-byte bit masks). The level and the keywords are used to add dimensions to ETW instrumentation. The level is designed to enable filtering based on the severity or verbosity of events. The keyword is designed to indicate sub-components in a provider. For instance, developers may divide events into informational events and critical-error events. Also, they can assign different keywords to sub-components of the application. By enabling selectively with different level and keywords, the trace controller can enable providers to log only error events from sub-component B, or all events from sub-components A and C, and so forth. For providers using the old APIs, a keyword is 4 bytes, and filtering by level and keyword has to be done explicitly in the provider code. It is important to note that when a controller enables a particular level, all events with a level value less than or equal (higher or equal severity) to what the controller specified are also enabled. Although levels and keywords can be custom designed and assigned to events by developers, there are predefined levels, as shown in Figure 4.

Figure 4 Severity Levels (Close Figure 4)
<levels>
    <level name="win:LogAlways" symbol="WINEVENT_LEVEL_LOG_ALWAYS"
        value="0" message="$(string.level.LogAlways)"> Log Always
    </level>
    <level name="win:Critical" symbol="WINEVENT_LEVEL_CRITICAL" value="1"
        message="$(string.level.Critical)"> Only critical errors </level>
    <level name="win:Error" symbol="WINEVENT_LEVEL_ERROR" value="2"
        message="$(string.level.Error)"> All errors, includes win:
        Critical </level>
    <level name="win:Warning" symbol="WINEVENT_LEVEL_WARNING" value="3"
        message="$(string.level.Warning)"> All warnings, includes
        win:Error </level>
    <level name="win:Informational" symbol="WINEVENT_LEVEL_INFO"
        value="4"
        message="$(string.level.Informational)"> All informational
        content, including win:Warning </level>
    <level name="win:Verbose" symbol="WINEVENT_LEVEL_VERBOSE" value="5"
        message="$(string.level.Verbose)"> All tracing, including
        previous levels </level>
</levels>

Task and opcode are used to attach additional information to each event. A task specifies a common logical component or task being instrumented. The task often represents the key high-level steps that a component takes to achieve its purpose. An opcode signifies the specific operation being performed at the time the event is written. For example, the Windows kernel provider groups all file I/O operation events into a "FileIO" task. Opcode indicates what the operation was, such as Create, Open, Read, and Write. Unlike ID, version, channel, level, and keyword, task and opcode are only used for adding information; they do not have any impact on controlling instrumentation or locating metadata.

The event descriptors and layouts are specified in the event manifest. Developers write the event manifest when instrumentation is designed. An event manifest is written in XML, in which user-defined channels, tasks, opcodes, levels, and keywords are specified in appropriate XML tags. ETW predefined channels, levels, and opcodes can also be used; the predefined channels, however, refer to global channels and should only be used for admin-targeted events. The different metadata fields are combined for each event and defined in an <Event> tag that is uniquely associated with an event ID. An event can also have a message string that will be displayed with substituted values from the event data when the event is being read by a consumer. Figure 5 shows an XML fragment from a sample event manifest. Later we'll present a step-by-step guideline to event instrumentation design and instrumentation.

Figure 5 Event Manifest Fragment (Close Figure 5)
<provider name="Microsoft-Windows-Kernel-Registry"
    guid="{70eb4f03-c1de-4f73-a051-33d13d5413bd}"
    symbol="RegistryProvGuid"
    resourceFileName="%SystemRoot%\System32\advapi32.dll"
    messageFileName="%SystemRoot%\System32\advapi32.dll">
  <channels>
    <channel name="Microsoft-Windows-Kernel-Registry/Analytic"
        chid="RegistryEvents" symbol="REG_Events" type="Analytic"
        isolation="System">This channel contains registry
        events.</channel>
  </channels>
  <opcodes>
    <opcode value="32" name="CreateKey" symbol="" />

    ...

  </opcodes>
  <keywords>
    <keyword name="CreateKey" symbol="" mask="0x1000" />

    ...

  </keywords>
  <templates>
    <template tid="tid_RegOpenCreate">
      <data name="BaseObject" inType="win:Pointer"
         outType="win:HexInt64" />
      <data name="KeyObject" inType="win:Pointer"
          outType="win:HexInt64" />
      <data name="Status" inType="win:UInt32"
          outType="win:HexInt32" />
      <data name="Disposition" inType="win:UInt32" />
      <data name="BaseName" inType="win:UnicodeString"
          outType="xs:string" />
      <data name="RelativeName" inType="win:UnicodeString"
          outType="xs:string" />
    </template>

    ...

  </templates>
  <events>
    <event value="1" symbol="ETW_REGISTRY_EVENT_CREATE_KEY"
        template="tid_RegOpenCreate" opcode="CreateKey"
        channel="RegistryEvents" level="win:Informational"
        keywords="CreateKey"
        message="$(string.Registry.RegOpenCreate)"/>

        ...

      </events>
    </provider>

      ...

  <localization>
    <resources culture="en-US">
      <stringTable>
        <string id="Registry.RegOpenCreate"
            value="Registry key %6 was created with status %3." />
      </stringTable>
    </resources>
  </localization>

Managed ETW provider APIs will also be available in the Microsoft® .NET Framework 3.5, code-named "Orcas." Managed consumer and controller APIs are currently in the planning stage. System.Diagnostics.Eventing contains the EventProvider class. EventProvider offers methods for all the functionality described above for the native applications. Users need to instantiate the class with a provider GUID and use the class instance to log events. EventDescriptor is a struct equivalent to the event descriptor in the native APIs. Unlike the native case, EventDescriptors are not generated for managed applications. However, a tool capable of generating code aimed at better performance and validation is under consideration. A sample usage of EventProvider is shown in Figure 6.

Figure 6 Managed ETW Provider (Close Figure 6)
using System.Diagnostics.Eventing;
...

static void Main(string[] ArgRead)
{
    int MyInteger;
    string MyString;

    ...

    // Construct event descriptor.
    EventDescriptor Event1 = new EventDescriptor(5, 0, 0, 2, 0, 0, 0);

    // Instantiate event provider.
    EventProvider etwProvider = new EventProvider(
        new Guid("d58c126f-b309-11d1-969e-0000f875a5bc"));

    ...

    // Write an event with MyInteger and MyString.
    etwProvider.WriteEvent(ref Event1, MyInteger, MyString);

    ...
}

In addition, ETW on Windows Vista offers improved security options for providers. By default, any provider can register and write events. However, developers can place restrictions on a GUID so that only authorized users can register a provider with that GUID. A provider can also specify who can enable it. Furthermore, ETW allows a controller to declare a session to be secure, meaning that it receives events only from a certain group of users.

Once the binary with the event provider instrumentation is compiled, the provider is installed. Then users can collect events from the provider using the logman tool, an in-the-box controller application. The logman.exe command below issues two ETW control API calls that start an ETW session called mysession and enable a provider:

> logman start mysession -p <provider name> -o mytest.etl -ets

Session mysession writes events to a file called mytest.etl. There are a number of logman.exe options that customize logging modes, buffer configuration, and so on. Here, <provider name> can be the provider name in the manifest or the GUID that the provider uses to register with using the EventRegister API. If the provider GUID is 11223344-5566-7788-99aa-bbccddeeff00, the actual command line becomes the following:

> logman start mysession
-p {11223344-5566-7788-99aa-bbccddeeff00}
-o mytest.etl -ets

ETW allows a controller to pre-enable a provider before it registers and executes. This lets users collect startup traces for dlls and drivers for which the loading of binary images is not easily controlled. A controller starts a session and enables a provider before the provider begins execution. As soon as the provider registers, it is enabled.

After events are collected as needed, the session can be stopped by using the stop option in logman.exe.

> logman stop mysession -ets

A trace file called mytest.etl should now exist in the same directory in which the logman command was executed. Tracerpt.exe can be used to get the dump of events from this file:

> tracerpt mytest.etl

This command creates a dump file (dumpfile.xml by default) and a text summary file (summary.txt by default). The summary file contains a short summary of event statistics. The dump file contains XML dumps of events, such as the one shown in Figure 2. The data fields from the event header are presented in the <System> section. Process ID, thread ID, processor ID, and CPU usage are given in the <Execution> tag within <System>. Also, the <System> section shows the values of the event descriptor fields of the event. Output file names can be specified by the user as options to the tracerpt.exe command. There are other GUI tools on Windows Vista capable of collecting and viewing traces, which we'll discuss later.

There are already hundreds of event providers installed on Windows Vista. Users can see the list of these providers from PerfMon or the Event Viewer or through a logman.exe command:

> logman query providers

Design and Implementation Guidelines

In this section, we discuss the steps to create the manifest file that defines the events, instrument the code with ETW API calls, and successfully build and install the event provider.

Design the Instrumentation Manifest The instrumentation manifest is an XML-format file that defines the event provider and the events that the event provider logs to ETW. Each event has a standard metadata and a variable event data section. The manifest can either be created by hand or using the manifest generator tool available in the SDK (ecmangen.exe). Here are the steps:

  1. Create an event provider and determine its provider ID, which is a GUID.
  2. Define the logical tasks of the component and the specific instrumentation points within those logical tasks. This defines the tasks and opcodes for the events that the provider writes.
  3. Define the subcomponents that will be logging events and assign a keyword for each subcomponent. This allows controllers to selectively enable events.
  4. Assess the target audiences of events being designed and define channels for each target audience.
  5. Create the different constructs that will define the variable user context data. This is done in the <templates> section.
  6. Put everything together and define events in the <events> section of the manifest. Assign to each event the symbolic identifier, severity, keywords, channel, task, opcode, and template. A localizable event-specific message can be added as well.

Instrument the Code with the ETW API After the events have been defined in the manifest, use the unified APIs to write events to ETW. For native applications, first generate a header file that contains the event definitions from the manifest so it can be included in the provider code. Then use the ETW APIs to write the events designed previously. This is the process:

  1. Run the Message Compiler (mc.exe) on the manifest created above. This generates the header file that contains event descriptor structs for the events, with each struct instance having the same name as the event symbolic identifier. The Message Compiler also produces the resource string file. The generated header file needs to be included in the provider code.
  2. Before writing code to raise events, a provider needs to register with the provider ID defined in the manifest by calling the EventRegister function. A registration handle returned from EventRegister should be stored in order to be used for logging API calls and the corresponding EventUnregister call.
  3. Call EventWrite to log the events defined in the manifest. The event-specific data is passed in through a data descriptor array as depicted in Figure 3. (Developers must make sure that the data fields exactly match the corresponding event templates specified in the manifest.) Pass in the right event descriptor struct defined in the header file with the event.
  4. After logging all the necessary events, call EventUnregister and pass in the registration handle.

Compiling the Instrumented Binary The manifest information that defines the events should be compiled during the build process and attached as a resource into the provider executable binary. All the localizable strings will be taken out of this compiled manifest and placed in a separate resource file that must be compiled and linked separately. When the provider binary is deployed, the manifest information needs to be installed on the system so that both controllers and consumers can find the event provider and event definitions. All the required tools are available natively on Windows Vista or from the SDK. The steps involved follow:

  1. If there are localizable message strings for the events, it may be necessary to run rc.exe to compile the resource file into a .res.
  2. If Visual Studio® is used, the developer can include a pre-build command to run rc.exe to generate the .res compiled resource and include this compiled resource file in the project.
  3. Link the compiled resource file using either cl.exe or link.exe (this is done in Step 2 for users of Visual Studio) and compile the application or driver.
  4. When installing the provider, run the wevtutil.exe tool with the -im option to install the manifest that defines the events. This allows controllers to discover the event providers and also lets consuming tools find the information needed to decode events properly.

Tools and Support

So far we have used logman.exe, a command-line controller, and tracerpt.exe, a general consumer tool. Logman.exe also has an option to schedule collections on remote machines. And Tracerpt.exe is capable of dumping events in CSV format, and it can generate resource-consumption reports using the -report option for some of the known core OS events. Figure 7 shows the hot file table generated from tracerpt.exe after processing and correlating some known core OS events based on a state machine-building technique.

Figure 7 Hot File Report Generated by Tracerpt
Figure 7 Hot File Report Generated by Tracerpt(Click the image for a smaller view)
Figure 7 Hot File Report Generated by Tracerpt
Figure 7 Hot File Report Generated by Tracerpt (Click the image for a larger view)

Reliability and Performance Monitor (RPM) (which includes PerfMon on Windows Vista) offers a graphical interface for collecting traces. Using the Event Trace Sessions interface, users can set up a trace session, pick providers to enable, and start and stop ETW sessions. Figure 8 shows RPM when a user is looking at the list of providers to enable. RPM introduces the Data Collector Set concept, in which all the necessary information regarding event traces, performance counters, and configuration (registry and WMI classes) data collection is combined into a single collection set. Users can create a data collection set with providers enabled for trace and counter data. Once created, the collection set settings are stored in RPM so that users can easily start and stop data collection without having to specify sources every time. This feature supports common diagnostic scenarios, where users can specify the providers to collect traces and counters from for some known diagnostic scenarios.

Figure 8 Event Trace Providers in RPM
Figure 8 Event Trace Providers in RPM(Click the image for a smaller view)
Figure 8 Event Trace Providers in RPM
Figure 8 Event Trace Providers in RPM (Click the image for a larger view)

The Event Viewer offers another means for collecting and viewing events. The new event APIs were designed for events directed to the event log as well, and admin and operational channel events are automatically routed to and displayed in the Event Viewer, to be monitored for important software status. The Event Viewer also expands its functionality for analytic and debug events. The "View" option allows displaying all available analytic and debug channels, and users can right-click to enable trace collection. In Figure 9, a user right-clicked an analytic channel of the registry event provider to enable trace collection.

Sample code is available in software development kits for provider, controller, and consumer APIs. Sample providers using the new provider APIs include manifests for the events that they log. A driver sample is available as well.

Figure 9 Event Viewer Interface for Analytic Event Collection
Figure 9 Event Viewer Interface for Analytic Event Collection(Click the image for a smaller view)
Figure 9 Event Viewer Interface for Analytic Event Collection
Figure 9 Event Viewer Interface for Analytic Event Collection (Click the image for a larger view)

Conclusion

We have presented the unified event provider model and APIs for ETW newly available on Windows Vista. Our goal is to introduce the APIs to developers and provide them with a usage example and general guidelines. Tracing is still an unfamiliar technology to many developers, and we hope this article provides insight into one of the key instrumentation and diagnostic infrastructures on Windows platforms. Within a few years since its introduction as a general tracing technology on the Windows 2000 operating system, a number of components and applications have employed ETW as a basis for their tracing and manageability features, and this trend is likely to continue.

We anticipate a greater number of developers will begin to adopt event instrumentation technology in order to deliver more reliable, better performing, and more easily manageable applications. Instrumentation will not only help these developers debug their applications, but will also help the people who deploy and manage the applications. Furthermore, instrumentation will aid in capacity planning, trend analysis, and finding performance bottlenecks as well. ETW provides a useful tracing infrastructure to enable applications to provide instrumentation to address all of the above scenarios, and we expect this technology to play a fundamental role as the core building block in the endeavor to make applications more manageable and more diagnosable.

Dr. Insung Park is a Development Lead for the Windows Instrumentation Platform Team. He has published a dozen papers on performance analysis, request tracking, instrumentation technology, and programming methodology and support. His e-mail address is insungp@microsoft.com.

Ricky Buch is a Program Manager for the Windows Instrumentation Platform Team. He works on both Event Tracing for Windows and the Performance Counter Library technologies. He can be reached at ricky.buch@microsoft.com.


ASP.NET 2.0: Enforce Web Standards For Better Accessibility

ASP.NET 2.0

Enforce Web Standards For Better Accessibility

Ben Waldron
This article discusses:
  • Adopting Web standards
  • CSS Control Adapters
  • ASP.NET 2.0 Themes and Master Pages
This article uses the following technologies:
ASP.NET 2.0
Code download available at: WebStandards2007_04.exe (318KB)
Contents
Adopting Web Standards
XHTML Flavors
ASP.NET 2.0 as a Web Standards Platform
CSS Control Adapters
Master Pages and Themes
The Petition

I
have to admit that Web standards were not always very important to me. Like many of you, I began writing Web pages at the time the first Web browsers were being released. Back then, I thought closing every HTML paragraph tag would slow me down and consume unnecessary portions of a user's precious 28K of bandwidth. So, I didn't do it. As long as the browser didn't care, neither did I.

Since then, however, I have begun to recognize the importance of Web standards to the future of the Web and of Web-based software in general. Of course, Web standards are about much more than closing HTML tags. They are a critical factor in how well software components can be used in future contexts.

Besides complying with the W3C standards for XHTML, cascading style sheets (CSS), XML, and so forth, Web applications should be search-engine friendly, accessible, have human-readable URLs, and employ other accepted practices that make them easier to find and use. Web standards also require careful attention to the structure of the code behind the sites, making it easier to maintain in the future. In this article I'll explain why everyone, especially those writing server controls, should adhere to Web standards.

Adopting Web Standards

Web standards impact four broad categories: code validity, site navigation, accessibility, and separation of content and design, as shown in Figure 1. If you address the larger concerns in these categories, the job of producing standards-compliant code is largely accomplished. Let's look at each category in turn.

Figure 1 Categorizing Web Standards (Close Figure 1)

Standard | Description
Code validity | Web sites must use valid XHTML and CSS according to W3C standards and be well formed. This will ensure maximum browser support, faster rendering, and code that's easier to understand.
Site navigation | Web sites should have a hierarchic structure with a site map and user-friendly URLs. Users should be able to determine their location within a site based on the URL.
Accessibility | Web sites must be available to impaired users and should be available across a wide range of browsers and devices.
Separation between design and content | Web sites must use a style sheet to define layout and presentation with content represented in XHTML.

Code Validity It is important that controls emit valid XHTML code. This means, among other things, that all tags are closed, that you choose the correct DocType, and that you create alt tags for images. For best results, you should run the output of all Web pages through a validation service to ensure it's compliant. The W3C provides two helpful validation services: validator.w3.org forXHTML and jigsaw.w3.org/css-validator for CSS. These services can be automated as part of your build process or performed manually before code is deployed.

Most browsers are evolving to adhere to Web standards. This is certainly true of Internet Explorer® 7.0 and Mozilla Firefox 2.0, which were released at the end of 2006. So when you stick to the Web standards they enforce, you can be much more confident that your code will render correctly on current browsers.

Site Navigation Many content management systems internalize content and serve up page URLs that give the user no clue about where the page is in relation to the rest of the site. Moreover, dynamically created content tends to generate unfriendly URLs because information such as unique identifiers is often passed in the URL via the query string. To remedy these problems, URLs should be organized according to subject matter and show the user where he is. And you can achieve friendly URLs for dynamically generated content by using URL rewriting, in which an HTTP module maps an unfriendly URL to a friendly URL when the HTTP request is handled. URL rewriting is supported in the Microsoft® .NET Framework. Friendly URLs also make your content more discoverable, and some search engines actually rank pages higher if they have friendly URLs.

Accessibility Building or rendering code that is accessible to people with disabilities is as important to a business as building wheelchair-accessible doors. You can't possibly predict the abilities of all the users who will visit your site. For example, between five and ten percent of all males are colorblind. Fortunately, there are a few easy practices that can make a Web site more accessible. Avoiding the use of tables can help because screen readers, which read the screen to visually impaired users, are confused by table layouts. Accessible forms are important as well. I am guilty of building a two-column table with one column containing the label and the other containing the HTML form element. The correct approach is to define a label with the "for" attribute indicating the ID of the form element that the label describes (see Figure 2).

Figure 2 An Accessible Form (Close Figure 2)
<form id="FormAccessible" runat="server">
    <fieldset>
        <legend>Personal Information</legend>
        <p>
            <label for="TextFirstName">First Name</label>
            <asp:TextBox ID="TextFirstName" runat="server"
                AccessKey="f" TabIndex="0" />
        </p>
        <p>
            <label for="TextLastName">Last Name</label>
            <asp:TextBox ID="TextLastName" runat="server"
                AccessKey="l" TabIndex="1" />
        </p>
    </fieldset>
    <fieldset>
        <legend>Contact Information</legend>
        <p>
            <label for="TextEmail">Email</label>
            <asp:TextBox ID="TextEmail" runat="server"
                AccessKey="e" TabIndex="2"/>
        </p>
    </fieldset>
</form>

Other accessibility approaches include choosing easily distinguishable colors for important items like links and buttons, making sure that pages are readable when font size is increased (and allowing it to be increased), and using the alt attribute to describe images. A nice side-effect of using alt is that the description can be indexed by search engines.

Finally, designing the site to be more device-agnostic means more people can access it. After all, users have cell phones, PDAs, Smartphones, and other devices that access the Web. It is frustrating for them not to be able to access the data they need because the site can't be consumed on a mobile device.

Separation of Content and Design Web pages were around long before XML became the ubiquitous standard it is today. With XML, there is a clear distinction between content and design because XML doesn't easily allow style information to be intertwined with content the way HTML did.

The benefits to this separation are many. Keeping all layout and style information in CSS stylesheets makes it possible to support multiple devices. It makes the emitted XHTML smaller. The XHTML is cleaner and more organized, which is good for editing and debugging. Cleaner XHTML also makes it easier for screen readers to read the contents of a Web page and allows search engines to more accurately index the pages.

XHTML Flavors

Now that I've highlighted the importance of Web standards, let's create some pages that implement them. The first step is to choose the correct document type. The document type definition (DOCTYPE) tells the browser's validator which type of XHTML you are using. XHTML 1.0 comes in three flavors: Strict, Transitional, and Frameset. XHTML 1.1 provides only one flavor that restricts the use of HTML elements that are deemed presentational. The types are listed in Figure 3.

Figure 3 XHTML Document Types (Close Figure 3)

Version | Syntax | Description
XHTML 1.0 Strict | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict //EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"> | Supports only elements and attributes defined in HTML 4.01 and follows XML syntax rules.
XHTML 1.0 Transitional | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional //EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> | Supports some deprecated elements and attributes that are considered presentational (such as <center> and <u>).
XHTML 1.0 Frameset | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Frameset //EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-frameset.dtd"> | Supports the use of frames using the <object> or <iframe> elements. In general, the use of iframes is discouraged.
XHTML 1.1 | <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1 //EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> | Most recent W3C recommendation, removes all elements considered presentational.

It's better to use the XHTML 1.1 or HTML 1.0 Strict DocTypes because they help ensure that all presentational elements migrate away from the XHTML and into style sheets. Note, however, that conforming to these types does not completely ensure that a page is Web standards-compliant because the use of XHTML tables to position content is still permitted; therefore, you can't rely completely on validation since validation won't fail simply because you use tables.

The Transitional DocType should be used when you can't completely eradicate all presentational elements from code. It should be used to signal that the page is not strictly compliant and needs to be refactored at a later time.

Frames are supported by the XHTML 1.0 specification, but were removed for XHTML 1.1. In the future, the use of iframes and objects will likely be replaced by the XFrame specification. Including frames has been generally discouraged because of usability concerns. If you've ever hit the Back button on a Web page that has frames, you know what I mean.

ASP.NET 2.0 as a Web Standards Platform

There were some well-known deficiencies in ASP.NET 1.1. The out-of-the-box controls produced code that would not pass validation. That was largely attributed to the way that ViewState was handled in ASP.NET 1.1-using a hidden input tag that was not contained within a block display, like this:

<input type="hidden" name="__VIEWSTATE"
value="dDwtMTU1NzQzNDgy..." />

This, combined with some other syntactical issues, gave ASP.NET 1.1 an unfortunate reputation for non-compliance.

ASP.NET 2.0 addressed many of the Web standards issues. For example, if you look at the source of ASP.NET 2.0-generated pages, you see that ViewState is now wrapped in a div tag, making it compliant:

<div>
    <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE"
           value dDwtMTU1NzQzNDgy..." />
</div>

Microsoft has made it known publicly that one of the goals of ASP.NET 2.0 is to be more aligned with Web standards and, in fact, ASP.NET 2.0 lets you create compliant Web sites and controls. A great resource for building compliant Web sites is Stephen Walther's MSDN® article "Building ASP.NET 2.0 Web Sites Using Web Standards". This 78-page article goes into great detail on building sites according to Web standards.

CSS Control Adapters

While ASP.NET 2.0 takes a great step forward in terms of compliance, there are still challenges. For instance, some ASP.NET 2.0 controls render as HTML tables when they shouldn't. In some cases, controls like a GridView or DataList should render output within a table since the nature of the control dictates tabular data. Other controls, like the menu control, should not render as a table. Figure 4 shows a portion of an HTML table that is rendered for the menu item of the menu control. The table and its many style attributes make this code less than optimal.

Figure 4 Default Menu HTML Output (Close Figure 4)
<table id="ctl00_menuTop" class="ctl00_menuTop_2"
       cellpadding="0" cellspacing="0" border="0">
  <tr onmouseover="Menu_HoverStatic(this)"
      onmouseout="Menu_Unhover(this)"
      onkeyup="Menu_Key(this)" id="ctl00_menuTopn0">
    <td>
      <table cellpadding="0" cellspacing="0" border="0" width="100%">
        <tr>
          <td style="white-space:nowrap;width:100%;">
            <a class="ctl00_menuTop_1"
               href="http://Msdn.WebStandards/Default.aspx">Main Page</a>
          </td>
          <td style="width:0;">
            <img src="/Msdn.WebStandards/WebResource.axd?d=hnH9bjBrxucw46WA5Xha0Kh9gZ-YF67wSyd4b51QsEc1&amp;t=632962190992968750"
                 alt="Expand Main Page"
                 style="border-style:none;vertical-align:middle;" />
          </td>
        </tr>
      </table>
    </td>
  </tr>
</table>

Because the HTML is rendered as a table when the data in the table is generally not tabular, this code is not considered to be Web standards compliant. Using an HTML table to render a list of menu items is like using a DataSet to tackle a job for which you'd typically use an array or collection of strings. The accepted way to render hierarchical data like this is to use the HTML unordered list, which makes sense when you think about it-a menu really is a series of nested lists.

Fortunately, Microsoft distributes the ASP.NET 2.0 CSS Friendly Control Adapters 1.0 toolkit, which lets you override the default output of controls to remove, replace, or inject HTML when the control is rendered. After several beta iterations, the toolkit was released and can be downloaded from asp.net/CSSAdapters/Default.aspx.

The CSS Control Adapters provide the ability to override the output of any control, whether delivered with ASP.NET 2.0, third-party, or custom, without modifying the source code of the control. As a developer, you do not have the source code for all the controls you use and even if you did, modifying it is not always the best way to override functionality. This is especially true if all you need to do is modify the rendered HTML.

The architecture that drives the control adapters is built on the Adapter or Wrapper design pattern. The abstract class ControlAdapter defines the base functionality for all adapters. The WebControlAdapter class inherits the ControlAdapter functionality while adding methods for rendering individual HTML tags. Classes like the Menu Adapter inherit from the WebControlAdapter to provide functionality specific to a control. Figure 5 shows the class diagram for the control adapters.

Figure 5 ASP.NET 2.0 Control Adapter Classes
Figure 5 ASP.NET 2.0 Control Adapter Classes(Click the image for a smaller view)
Figure 5 ASP.NET 2.0 Control Adapter Classes
Figure 5 ASP.NET 2.0 Control Adapter Classes (Click the image for a larger view)

The implementation of this architecture is straightforward, especially for controls that already have adapters developed for them. Fortunately, most all the ASP.NET controls do have adapters. In the example shown, the MenuAdapter class is used to override the default HTML table layout with an unordered list.

The first step to accomplish this is to create or customize a browser definition file, which is used to detect the browser that is accessing the ASP.NET 2.0 site and provide specific functionality for that browser. This is a good way to provide a customized experience to users coming to the site from mobile devices. For use with CSS Control Adapters, however, the browser definition file is used to signify that compliant browsers should use one or more adapters.

The XML shown in Figure 6 shows a full browser definition file that tells ASP.NET to route all of the MenuControl classes to the MenuAdapter for all browsers that match the described criteria. The criteria describe all W3C-compliant browsers that support JavaScript and CSS. This would match most of the browsers used today. The browser definition file also gives you information you can use for all kinds of other customization of the user experience.

Figure 6 ASP.NET 2.0 Browser Definition File (Close Figure 6)
<browsers>
  <browser refID="Default">
    <controlAdapters>
      <adapter controlType="System.Web.UI.WebControls.Menu"
               adapterType="CSSFriendly.MenuAdapter" />
    </controlAdapters>
  </browser>

  <browser id="W3C_Validator" parentID="default">
    <identification>
      <userAgent match="^W3C_Validator" />
    </identification>
    <capabilities>
      <capability name="browser"              value="W3C Validator" />
      <capability name="ecmaScriptVersion"    value="1.2" />
      <capability name="javascript"           value="true" />
      <capability name="supportsCss"          value="true" />
      <capability name="supportsCallback"     value="true" />
      <capability name="tables"               value="true" />
      <capability name="tagWriter"
        value="System.Web.UI.HtmlTextWriter" />
      <capability name="w3cdomversion"        value="1.0" />
    </capabilities>
  </browser>
</browsers>

The last step in implementing the CSS Control Adapter is to create a class that is derived from the adapter class. Figure 7 shows the example menu adapter that ships with the ASP.NET 2.0 Control Adapter. The class derives from the MenuAdapter class, enumerates through all of the items that make up the controls, and overrides the HTML with the appropriate tags. Shown there is the portion of the class that renders the beginning of the unordered list in the BuildItems method.

Figure 7 Menu Control Adapter HTML Output (Close Figure 7)
Public Class MenuAdapter
        Inherits System.Web.UI.WebControls.Adapters.MenuAdapter
    ...

    Protected Overrides Sub RenderContents(ByVal writer As HtmlTextWriter)
        If (Extender.AdapterEnabled) Then
            writer.Indent = writer.Indent + 1

            BuildItems(Control.Items, True, writer)

            writer.Indent = writer.Indent—1
            writer.WriteLine()
        Else
            MyBase.RenderContents(writer)
        End If
    End Sub

    Private Sub BuildItems(ByVal items As MenuItemCollection, _
            ByVal isRoot As Boolean, ByVal writer As HtmlTextWriter)
        If (items.Count > 0) Then
            writer.WriteLine()

            writer.WriteBeginTag("ul")
            If isRoot Then
                writer.WriteAttribute("class", "AspNet-Menu")
            End If
            writer.Write(HtmlTextWriter.TagRightChar)
            writer.Indent = writer.Indent + 1

            For Each item As MenuItem In items
                BuildItem(item, writer)
            Next

            writer.Indent = writer.Indent—1
            writer.WriteLine()
            writer.WriteEndTag("ul")
        End If
    End Sub
End Class

If you look at the HTML source in Figure 8, you can see that the table from the menu control is replaced with an unordered list. The unordered list is not the only noteworthy topic in the HTML output, however. The output is also noticeably cleaner because it is not cluttered with style and scripting code. All of the positioning and style now resides in the CSS file for the menu and the menu is now considered Web standards-compliant.

Figure 8 Menu Control Adapter HTML Output (Close Figure 8)
<div class="AspNet-Menu-Horizontal">
  <ul class="AspNet-Menu">
    <li class="AspNet-Menu-WithChildren">
    <a href="http://Default.aspx" class="AspNet-Menu-Link">Web Standards</a>
      <ul>
        <li class="AspNet-Menu-Leaf">
          <a href="http://ControlAdapters.aspx">Control Adapters Sample</a>
        </li>
        <li class="AspNet-Menu-Leaf">
          <a href="http://AccessibleForm.aspx">Accessible Form</a>
                    </li>
                </ul>
            </li>
        </ul>
</div>

The CSS Control Adapter comes with adapters for many controls, including for the Tree view, Login, and CreateUserWizard, which represent the controls that notoriously render tables when they should not. It takes only a few minutes to integrate the samples into an existing project and customize to your liking.

Master Pages and Themes

Now that you have the tools to render controls that comply with Web standards, you can make an entire site compliant. The combination of Master Pages and Themes provide a powerful and flexible infrastructure for a Web site.

Master Pages allow Web developers to create flexible templates for Web pages. A powerful tool for creating reusable components across Web pages, Master Pages also provide a way to ensure standards compliance throughout a site. Creating a Master Page that uses the XHTML 1.1 DocType is good way to ensure compliance across all pages within the site. Master Pages are also handy for letting you make all changes to the DocType in one place when, for example, a site migrates from XHTML 1.0 Transitional to Strict.

ASP.NET 2.0 Themes allow you to create multiple designs for the same content, enhancing accessibility. For example, themes can improve accessibility for visually impaired users by letting them increase the font and image size. Placing the ability to switch themes within the Master Page allows users to view the site in a manner that best meets their individual needs.

One technique I find valuable to enable users to switch themes is to define a class that is derived from the Page class. The ability to switch themes is only available very early in the lifecycle of an ASP.NET page-in the OnPreInit method. The class in Figure 9 shows how to capture the theme a user has selected and apply it in the overridden method.

Figure 9 ThemeSelectablePage Class (Close Figure 9)
''' <summary>
'''  Override Page with PreInit method to allow for theme switching
''' </summary>
''' <remarks>"__Theme" represents the RadioButton list of theme</remarks>
Public Class ThemeSelectablePage
        Inherits System.Web.UI.Page
    Protected Overloads Overrides Sub OnPreInit(ByVal e As EventArgs)
        For Each key As String In Me.Request.Form.AllKeys
            If key.IndexOf("__Theme") > -1 Then
                Page.Theme = Me.Request.Form(key).Trim
                Exit For
            End If
        Next
        MyBase.OnPreInit(e)
    End Sub
End Class

This class is looking for a request form variable with a predefined name and passing the value to the theme class. All Web pages that implement this class will provide the infrastructure needed to enable multiple themes. This is a very simple way to add theme switching to all pages on a site.

The Petition

This article discussed why Web standards are so important and looked at some techniques that can be used to create compliant Web sites. It can be challenging to adopt these standards because it is often not the easiest path to take, but it is the best way to ensure that the controls and HTML you build today will be accessible to a broader base of users on a larger number of devices in the future. Browsers are increasingly conforming to these standards and will continue to do so; the burden now falls on the development community to produce Web standards-compliant code.

Ben Waldron is Chief Technology Officer at Pop Art, Inc, an interactive agency that provides a host of Web-based solutions to clients. Ben is based in Portland, Oregon and can be contacted for questions or opportunities via e-mail: Ben.Waldron@PopArt.com.


Editor's Note: Winning Team

Editor's Note
Winning Team

We're always proud of the content we bring you in each issue of MSDN Magazine, but we're especially excited by the wonderful news we got recently. MSDN Magazine has been recognized by the Society for Technical Communication (STC) with several awards. We thought we were dreaming, but then we saw the press release on the Internet, which proves it's true. (The only thing that would prove it even more is if it were on a wiki that was on the Internet. That would be pretty darn infallible.)

All in all, we won three awards, including a "best of show." To quote the release:

SAN FRANCISCO, Jan. 25 /PRNewswire/-MSDN Magazine, The Microsoft Journal for Developers, has been honored with three awards from the Society of Technical Communication.

MSDN Magazine was recognized with a best of show/distinguished award in the reference material category for the MSDN Magazine Web site. MSDN Magazine also received an award of excellenceand an award of merit.

"Earthly Delights: Code Your Applications to Deliver the World with Virtual Earth APIs" by Chandu Thota (September 2006) was recognized with an award of excellence in the scholarly/professional article category.

MSDN Magazine was also honored with an award of merit in the scholarly/professional journals category. Contributors include the MSDN Magazine art and production team and the MSDN Magazine editorial team.

It truly is a great honor to be acknowledged by our peers as the leading magazine in the developer community. It takes a lot of work to put out top-quality content every month, and it's a reflection on our staff and on the authors who contribute to each issue.

So where do we go now that we've filled our trophy case? We've got an ever-expanding Web site that complements the magazine you hold in your hands today. At msdn.microsoft.com/msdnmag, you will find current articles in 11 languages, as well as our up-and-coming MSDN Magazine blog and our maybe-someday-Emmy-winning MSDN Magazine podcasts. If you haven't checked us out in a while, it's definitely time to come take a look.

Our blog, which you can find at blogs.msdn.com/msdnmagazine, lets you hear from the MSDN Magazine editors on a variety of topics. We give you the lowdown on upcoming issues, as well as a look back at some old favorites. We've also got a bunch of discussions threads on other topics, and it's a great place to interact with the editors of MSDN Magazine. We can be surprisingly friendly when people are watching.

If bringing authors into your living room is more your speed, the MSDN Magazine podcast series is going strong at msdn.microsoft.com/msdnmag/podcast. The April 2007 issue podcast features Charles Petzold. We're proud to say we knew Charles way back when-he's been writing for us since 1986! You should listen to it today. It's hard work creating one of those shows and taking out all the bad words (like "Linux"). If you're not reading the magazine right now, there's no better way to spend 30 minutes of your time.

If blogs and podcasts aren't enough and you want to get even closer to the award-winning MSDN Magazine team, be sure to drop by our booth at Microsoft TechoEd later this year and say hi. TechoEd will take place in Orlando on June 4-8, 2007 (microsoft.com/events/teched2007), and it promises to be a terrific event. We will be all over that convention center like socks on feet. We'd love to meet you, and we've got some fun stuff and great giveaways planned. Mention that you saw us in the April 2007 Editor's Note: you won't get any prizes, but we'll think you're really cool. -J.T.

Thanks to the following Microsoft technical experts for their help with this issue: Sunil Agarwal, Tom Archer, Vittorio Bertocci, Simon Calvert, Rohin Dabas, Naren Datha, Inbar Gazit, Matt Gibbs, Vitek Karas, Bertrand Le Roy, Val Menn, Peter-Michael Osera, Jordan Parker, Rob Relyea, Doug Purdy, Jevan Saks, Greg Schechter, Garrett Serack, Radhakrishnan Srikanth, Richard Turner, Nigel Watling, and Jason Wohlgemuth.


Toolbox: WiX Installation, Creating PowerPoint Decks, and More

Toolbox
WiX Installation, Creating PowerPoint Decks, and More
Scott Mitchell

Contents
Create WiX Installation Files
Create PowerPoint Presentations Programmatically
Source Control for Projects Big and Small
Six Degrees of SQL Hacks

Create WiX Installation Files

Most Windows-based applications today are set up using the Windows® Installer, which is an engine for installing, repairing, and removing applications. When installing an application, the relevant information and files are packaged into an MSI file.

There are many open-source and commercial tools for creating MSI files. One popular open-source option, spearheaded by Rob Mensching, is the WiX toolset from Microsoft, which provides command-line tools that process an XML file to generate an MSI file. The command-line environment allows for integration with automated build processes. By using an XML file to specify the installation package's details, this information can be maintained in source control and inspected and modified programmatically. However, command-line tools and XML files are much harder to master.

Creating installation packages using WiX can be as easy as point and click with WiXAware 1.0 by InstallAware Software Corporation. WiXAware is a GUI for creating, editing, validating, building, and installing MSI packages using the WiX toolset. Rather than having to enter the XML source file's content by hand, WiXAware lets you navigate the XML document via a tree, adding new elements and attributes through property windows and dialog boxes. And WiXAware works directly with the native WiX XML source file, so it can still be maintained in source control and edited by other developers or utilities through the raw XML format.

Creating an Installation File with WiXAware
Creating an Installation File with WiXAware (Click the image for a smaller view)
Creating an Installation File with WiXAware
Creating an Installation File with WiXAware (Click the image for a larger view)

WiXAware is much more than just a graphical XML editor, though. It includes a graphical designer that you use to build custom user interface screens, and offers tools for validating installation packages using the Microsoft Internal Consistency Evaluators (ICE) and WiXCop, a validation tool created by Derek Cicerone. At the click of a button, WiXAware will compile and link the XML source file, creating the installation package. The installation package can then be executed with the MSI log output to the Debug window.

Creating MSI installation packages can be accomplished using a variety of tools. If you currently use or are considering using the WiX toolset, check out WiXAware. It simplifies creating, editing, validating, and building the XML source files, thus improving productivity and easing WiX's learning curve.

Price: $499 for a single-user license.

www.wixaware.com

Create PowerPoint Presentations Programmatically

Reporting is an integral part of any business application. Commonly, these reports are delivered through a Web page, PDF document, or Microsoft® Excel® spreadsheet. It's rare for reports to be available as Microsoft PowerPoint® applications, yet that is the document format of choice for managers, the sales staff, and others who use report data in meetings and presentations. These users must often resort to copying and pasting report data into their presentations. So why not give these users the report data directly in PowerPoint?

Aspose.Slides for .NET version 2.6, from Aspose Pty Ltd., provides a library for programmatically creating, opening, reading, modifying, saving, and converting PowerPoint files. It supports all versions from PowerPoint 97 to PowerPoint 2007 and can be used to work with PowerPoint Presentation (PPT), PowerPoint Template (POT), and PowerPoint SlideShow (PPS) files. With the Aspose.Slides for .NET intuitive class library, working with PowerPoint presentations is a breeze. To create a new presentation, simply create an instance of the Presentation class; to open an existing presentation, just pass the file name to the Presentation class's constructor. The various pieces of a PowerPoint presentation-slides, text, tables, pictures, and so on-all have corresponding classes with straightforward members. With a surprisingly scant amount of code, you can retrieve your report data and render it as a PowerPoint presentation. Aspose.Slides for .NET also makes it easy to convert PowerPoint to PDF.

Aspose.Slides for .NET directly generates the binary contents of the PowerPoint presentation and therefore does not have any reliance on Microsoft Office. You can use it in server-based scenarios without having to install Office on the server.

If you have coworkers who copy report data from a Web page into PowerPoint, help them reclaim their time. With Aspose.Slides for .NET you can quickly and easily turn your report data into PowerPoint presentations.

Price: $599.00 for a developer enterprise subscription.

www.aspose.com

Source Control for Projects Big and Small

Every software company needs some form of source control. Source control applications work by providing a centralized repository of code for a particular project or program. Developers can get the latest version of the code, make their contributions, and then commit their changes back to the repository. Without a controlled, centralized code repository, it is all too easy for one developer to accidentally overwrite the changes made by another. Even if you're working on a project by yourself, source control makes sense because the program saves a version of every copy of the software that's checked in. This history of changes provides a backup of the application's source code, allows the developer to retrieve and examine old code, and can be used to roll back the source code to a previous version, if necessary.

Microsoft has long offered developers Visual SourceSafe®, which was last upgraded with the Visual Studio 2005 release. Now, with Visual Studio Team System, Microsoft has released another source control tool, one designed specifically for supporting much larger developer teams.

The TortoiseSVN Interface
The TortoiseSVN Interface (Click the image for a smaller view)
The TortoiseSVN Interface
The TortoiseSVN Interface (Click the image for a larger view)

There is also a healthy third-party and open-source source control market. One such popular open-source offering is Subversion version 1.4.2. Like other such tools, Subversion provides a centralized repository for storing source code and maintains a history of changes. Subversion can be configured to support access to its repositories through HTTP and HTTPS, enabling remote developers to interface with the source code repository. And Subversion also supports both the "check-in, modify, check-out" and "modify and merge" workflows.

Whereas most commercial source control programs provide a GUI to the repositories and Visual Studio integration, Subversion offers neither. It is simply a source control engine and provides only a command-line interface. Fortunately, an additional open-source project-TortoiseSVN-provides a more user-friendly, graphical interface, although there is no Visual Studio integration. However, TortoiseSVN is integrated with the Windows shell. Once you have established a working directory for a particular Subversion repository, you can lock, merge, update, diff, or view statistics for any file by right-clicking on it in Windows Explorer and choosing the appropriate option from the TortoiseSVN context menu. Don't let TortoiseSVN's lack of Visual Studio integration dissuade you from considering Subversion. In fact, soon after I started using Subversion, I found the Windows shell to be a more intuitive interface for working with the underlying source control system than the Visual Studio IDE.

Price: Free (open-source).

subversion.tigris.org and tortoisesvn.tigris.org

Six Degrees of SQL Hacks

Most books on SQL focus on its syntax and semantics, specifying the structure of the SELECT statement, how the GROUP BY clause can be used, and so forth. While such information is helpful for beginners, seasoned database professionals are more interested in scenario-based learning. They want to read about queries and techniques that can help them tackle their difficult problems. What queries can you use to identify overlapping ranges in data? How do you create and maintain an audit table, or compare and synchronize tables? If this is the kind of information you are searching for, check out SQL Hacks, written by Andrew Cumming and Gordon Russell and published by O'Reilly.

SQL Hacks provides 100 hacks that cover a gamut of scenarios. The book's 12 chapters range from SQL-specific hacks that explore how to accomplish common data-related challenges using SELECT, INSERT, UPDATE, or DELETE queries, to ones that look at interoperating with peripheral technologies, such as XML, XSLT, and Web applications. I found the "Number Crunching," "Organizing Data," and "Reporting" hacks the most interesting and applicable to my day-to-day work. Each hack is a terse, cut-to-the-chase look at how to tackle the posed challenge. All of the hacks address real-world challenges, but many are written in a light, humorous tone that improves the book's readability. One hack, for instance, illustrates how to find relationships among data by using SQL to play the "Six Degrees of Kevin Bacon" game (en.wikipedia.org/wiki/Six_Degrees_of_Kevin_Bacon).

Rather than focus on a particular database implementation, the authors decided to extend their hacks to SQL Server™, Oracle, MySQL, Microsoft Access™, DB2, and PostgreSQL. Many examples show executing database queries through some external application. When highlighting a database feature, Perl is typically used. When showcasing a language-specific feature, however, examples are in Perl, C#, Java, and Ruby. I found this variety distracting at times, but does open the book to a wider audience.

If you're already familiar with SQL syntax, but want to learn new techniques, pick up a copy of SQL Hacks.

Price: $29.99.

Send your questions and comments for Scott to toolsmm@microsoft.com.

Scott Mitchell, author of numerous books and founder of 4GuysFromRolla.com, is an MVP who has been working with Microsoft Web technologies since 1998. Scott is an independent consultant, trainer, and writer. Reach him at Mitchell@4guysfromrolla.com or via his blog at ScottOnWriting.NET.

All prices were confirmed at press time and are subject to change. The opinions expressed in this column are solely those of the author and do not necessarily reflect the opinions of Microsoft.

MSDN Magazine does not make any representation or warranty, express or implied with respect to any code or other information herein. MSDN Magazine disclaims any liability whatsoever for any use of such code or other information.


CLR Inside Out: New Library Classes in &quot;Orcas&quot;

CLR Inside Out
New Library Classes in "Orcas"
Mike Downen and Inbar Gazit and Justin Van Patten

Contents
Suite B Support
Elliptic Curve Cryptography
ECDSA
ECDH
Big Integer Support
High-Performance Sets
Pipes
Conclusion

F
or the next version of the Microsoft .NET Framework (which will ship with the next release of Visual Studio® currently code-named "Orcas"), its assemblies have been divided into two groups, internally referred to as "red bits" and "green bits." The red bits include all the libraries that shipped before as part of the .NET Framework 2.0 and 3.0 (such as mscorlib.dll and system.dll). To maintain a high assurance of backward compatibility for Visual Studio "Orcas," changes in the red bits have been greatly limited.

The green bits assemblies are the brand new libraries with additional classes that work on top of the red bits assemblies. Most of the classes listed in this column are in the green bits assemblies (such as system.core.dll), with a few involving limited changes in the red bits assemblies. For more background information, check out Soma Somasegar's blog and Jason Zander's blog.

The classes described in this column are available as part of the January 2007 Visual Studio "Orcas" Community Technology Preview (CTP). The CLR's contributions to the new libraries include:

  • A new add-in hosting model, which was discussed in the last two editions of CLR Inside Out
  • Support for the Suite B set of cryptographic algorithms, as specified by the National Security Agency (NSA)
  • Support for big integers
  • A high-performance set collection
  • Support for anonymous and named pipes
  • Improved time zone support
  • Lightweight reader/writer lock classes
  • Better integration with Event Tracing for Windows® (ETW), including ETW provider and ETW trace listener APIs

In this column, we'll discuss the new Suite B cryptography features, big integer support, the set class, and the pipes capabilities.

Suite B Support

One of our major goals for Visual Studio "Orcas" was to add support for the Suite B set of cryptographic algorithms. In order to support Suite B, a platform has to support:

  • The Advanced Encryption Standard (AES) with key sizes of 128 and 256 bits for encryption
  • The Secure Hash Algorithm (SHA-256 and SHA-384) for hashing
  • The Elliptic Curve Digital Signature Algorithm (ECDSA) using curves of 256-bit and 384-bit prime moduli for signing
  • Elliptic Curve Diffie-Hellman (ECDH) using curves of 256 and 384-bit prime moduli for key exchange/secret agreement

More information about Suite B is available in the overview published by the NSA at www.nsa.gov/ia/industry/crypto_suite_b.cfm.

In addition to these criteria, we also wanted to support Federal Information Processing Standard (FIPS) certified implementations of these algorithms. We've had support for AES (via our RijndaelManaged class) and SHA-256 and SHA-384 (using our SHA256Managed and SHA384Managed classes) since version 1.0 of the .NET Framework, but our implementations were not FIPS-certified.

As a first step, we added managed code wrappers for the FIPS-certified AES, SHA-256, and SHA-384 implementations in Windows. In Visual Studio "Orcas," these appear as the new AesCryptoServiceProvider, SHA256CryptoServiceProvider, and SHA384CryptoServiceProvider classes. All of these are located in the System.Security.Cryptography namespace.

These algorithms are available on all platforms on which the underlying Windows Crypto Service Providers are available-Windows XP and later for AES, and Windows Server® 2003 and later for the SHA algorithms. They generally work exactly like their managed equivalents, so developers should typically be able to swap them in without noticing any changes in behavior. The one difference is that, per the AES specification, AesCryptoServiceProvider only supports a fixed block size of 128 bits.

The CngKey Class

To support our new CNG-based managed cryptography classes, we've added a CngKey class to abstract the storage and usage of CNG keys. CNG keys work similarly to key containers in today's Crypto API (CAPI)—they allow you to store a key pair or a public key securely and refer to it using a simple string name. You can use CngKey objects when working with the ECDsaCng and ECDiffieHellmanCng classes.

You can also use the CngKey class directly to perform many operations, including opening, creating, deleting, and exporting keys. If we don't have a managed API for the operation you want, you can get to the underlying key handle to use when calling Win32® APIs directly.

Elliptic Curve Cryptography

Then we added classes for ECDSA and ECDH. These classes wrap the new Crypto Next Generation (CNG) implementations of these algorithms in Windows Vista™. (For more details on how we support CNG classes, see the sidebar "The CngKey Class.") Elliptic curve cryptography is asymmetric, meaning it uses a public and private key pair for operations, similar to RSA. But elliptic curve algorithms are generally more efficient computationally than traditional public key cryptography algorithms, especially as key sizes grow larger. In fact, the NSA recommends an RSA key size of 15,360 bits to protect a symmetric key of 256 bits, whereas it recommends just a 521-bit elliptic curve key to protect the same 256-bit symmetric key.

More information on the benefits of elliptic curve cryptography is available in the NSA's article "The Case for Elliptic Key Cryptography".

ECDSA

ECDSA works like most signing algorithms: you sign with a private key and verify with a public key. The new ECDsaCng class contains our implementation of ECDSA and follows the same pattern as other managed cryptography classes. Say you want to sign some data, assuming your key pair is stored in a CNG key named MyKey:

// Returns the signature
byte[] SignMyData (byte[] data)
{
   ECDsaCng signingAlg = new ECDsaCng(CngKey.Open("MyKey"));
   return signingAlg.SignData(data);
}

Now, say you want to verify a signature from Jane and her public key is stored in a CNG key named JanesKey. You would do the following:

// Returns whether or not the signature could be verified
bool VerifyJanesSignature(byte[] data, byte[] signature)
{
   ECDsaCng signingAlg = new ECDsaCng(CngKey.Open("JanesKey"));
   return signingAlg.VerifyData(data, signature);
}

The ECDsaCng class also lets you set different options when signing or verifying, mostly through properties on the class. For example, by default the ECDsaCng class uses the SHA-256 hashing algorithm during signing and verification. If you want to use a different algorithm, you simply change the HashAlgorithm property before signing. Let's say you choose to use SHA-512. The signing example for MyKey would now look like this:

// Returns the signature
byte[] SignMyData (byte[] data)
{
   ECDsaCng signingAlg = new ECDsaCng(CngKey.Open("MyKey"));
   signingAlg.HashAlgorithm = CngAlgorithm.Sha512;
   return signingAlg.SignData(data);
}

And the verification example for JanesKey would be this:

// Returns whether or not the signature could be verified
bool VerifyJanesSignature(byte[] data, byte[] signature)
{
   ECDsaCng signingAlg = new ECDsaCng(CngKey.Open("JanesKey"));
   signingAlg.HashAlgorithm = CngAlgorithm.Sha512;
   return signingAlg.VerifyData(data, signature);
}

ECDH

ECDH is a key exchange algorithm, sometimes referred to as a secret agreement algorithm. This algorithm is used to derive a secret key-often used for encryption or decryption with a symmetric algorithm-by exchanging a public key with another party. For example, say two users, Alice and Bob, want to derive a secret key. They both already have an elliptic curve key pair (both public and private keys). In order to derive a secret key using ECDH, they would perform the following steps:

  1. Alice sends her public key to Bob. He must be sure he's really getting the key from Alice and that the key has not been modified, but the key does not have to be sent confidentially.
  2. Bob sends his public key to Alice. Likewise, Alice must be absolutely sure that she's really getting the key from Bob and that the key has not been modified, but again the key does not have to be sent confidentially.
  3. Alice and Bob can now generate the same secret value, given that they each have the other party's public key and their own key pair. This secret value can then be used to generate a secret key for a symmetric algorithm.

The code for this is pretty simple. The following example generates a 256-bit key for Alice, assuming that her key pair is stored in a CNG key named MyKey:

// Returns a 256 bit secret key that is shared with the other party
bool Generate256BitKey(string otherParty)
{
   ECDiffieHellmanCng keyExchAlg =
      new ECDiffieHellmanCng(CngKey.Open("MyKey"));
   byte [] myPublicKey = keyExchAlg.PublicKey.ToByteArray();
   ... // send myPublicKey to other party
   byte[] otherPartysPublicKeyBytes = ...; // get other party's key
   ECDiffieHellmanCngPublicKey otherPartysPublicKey =
      ECDiffieHellmanCngPublicKey.FromByteArray(
         otherPartysPublicKeyBytes, CngKeyBlobFormat.EccPublicBlob);
   return keyExchAlg.DeriveKeyMaterial(otherPartysPublicKey);
}

Alice now has a 256-bit symmetric key that can be used with the other party (Bob) to encrypt and communicate confidential data. I'll stress again, however, that it's important to make sure you're really getting the genuine public key from the other party in a situation like the example above, not from an imposter. This typically requires some form of authentication.

Like the ECDsaCng class, the ECDiffieHellmanCng class used here can be customized through properties. For example, you can use different key derivation functions specifying all of the parameters for the key derivation functions using these properties.

Big Integer Support

Support for integers that can represent very large numbers is another customer request that we have implemented in the next version of the .NET Framework. Have you ever needed a really huge number, one that is so big that it just couldn't fit into any other base-type in the .NET Framework? Now you can use the new BigInteger type, which allows you to use integers of any arbitrary size, up to the limits of available memory. To begin, let's take a look at some scenarios in which you might need to use a really big number.

Astronomical calculations that involve distances in space require the ability to calculate extremely large numbers. Astronomers often calculate the distance in light-years in order to avoid using really big numbers. That's because a light-year is about 5,878,625,373,183.61 miles (or 9,460,730,472,580.8 kilometers) and so the actual number of miles between stars (not to mention feet or inches) is larger than what can be represented with 64 bits.

Big integers are also used in statistics and number theory. While not all mathematicians are necessarily working with integers of this size, consider the Law of Truly Large Numbers (which states that with a sample size large enough, any outrageous thing is likely to happen) and some of the writings by John Edensor Littlewood.

Furthermore, many cryptographic algorithms require the use of very large prime numbers. These numbers are much larger than can be represented by base types today.

The new BigInteger class is in the new System.Numeric namespace. It supports most of the operators you'd expect of a standard numeric type, such as addition (+), subtraction (-), and multiplication (*). The sample code in Figure 1 shows how to do a factorial computation using the BigInteger class. Figure 2 shows an example of the output if you enter 1000.

Figure 2 Factorial Program Output for 1000 (Close Figure 2)
402387260077093773543702433923003985719374864210714632543799910
429938512398629020592044208486969404800479988610197196058631666
872994808558901323829669944590997424504087073759918823627727188
732519779505950995276120874975462497043601418278094646496291056
393887437886487337119181045825783647849977012476632889835955735
432513185323958463075557409114262417474349347553428646576611667
797396668820291207379143853719588249808126867838374559731746136
085379534524221586593201928090878297308431392844403281231558611
036976801357304216168747609675871348312025478589320767169132448
426236131412508780208000261683151027341827977704784635868170164
365024153691398281264810213092761244896359928705114964975419909
342221566832572080821333186116811553615836546984046708975602900
950537616475847728421889679646244945160765353408198901385442487
984959953319101723355556602139450399736280750137837615307127761
926849034352625200015888535147331611702103968175921510907788019
393178114194545257223865541461062892187960223838971476088506276
862967146674697562911234082439208160153780889893964518263243671
616762179168909779911903754031274622289988005195444414282012187
361745992642956581746628302955570299024324153181617210465832036
786906117260158783520751516284225540265170483304226143974286933
061690897968482590125458327168226458066526769958652682272807075
781391858178889652208164348344825993266043367660176999612831860
788386150279465955131156552036093988180612138558600301435694527
224206344631797460594682573103790084024432438465657245014402821
885252470935190620929023136493273497565513958720559654228749774
011413346962715422845862377387538230483865688976461927383814900
140767310446640259899490222221765904339901886018566526485061799
702356193897017860040811889729918311021171229845901641921068884
387121855646124960798722908519296819372388642614839657382291123
125024186649353143970137428531926649875337218940694281434118520
158014123344828015051399694290153483077644569099073152433278288
269864602789864321139083506217095002597389863554277196742822248
757586765752344220207573630569498825087968928162753848863396909
959826280956121450994871701244516461260379029309120889086942028
510640182154399457156805941872748998094254742173582401063677404
595741785160829230135358081840096996372524230560855903700624271
243416909004153690105933983835777939410970027753472000000000000
000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000
Figure 1 Factorial Computation Using BigInteger (Close Figure 1)
using System.Numeric;

static BigInteger Factorial(BigInteger x)
{
    BigInteger result = 1;
    for (BigInteger i = 2; i <= x; i++)
        result *= i;
    return result;
}

static void Main(string[] args)
{
   BigInteger x = BigInteger.Parse(Console.ReadLine());
   Console.WriteLine(Factorial(x).ToString());
}

BigInteger supports many other mathematical operations through static methods on the class. These include:

  • Abs (returns the absolute value for a given BigInteger)
  • Compare (compares two BigIntegers)
  • Divide (returns the quotient of two BigIntegers)
  • DivRem (returns both the quotient and the remainder of two BigIntegers)
  • Equals (returns true if two BigIntegers have the same value)
  • GreatestCommonDivisor (returns the maximum number that is a divisor of both BigIntegers)
  • ModPow (returns one BigInteger raised to the power of another BigInteger modulo a third; the exponent cannot be negative)
  • Pow (returns one BigInteger raised to the power of another; the exponent cannot be negative)
  • Remainder (returns the remainder of dividing one BigInteger by another)

We will be posting more information on using and extending the BigInteger class on the BCL Team's blog at blogs.msdn.com/bclteam.

We should note that there are currently some design trade-offs in BigInteger. It was very important to get this new numeric type into the .NET Framework for Visual Studio "Orcas." Moreover, it was important that this new base-type be available early enough for other green bits features to make use of it. So we decided for this release not to fully support all the printing, formatting, and parsing that the other numeric types support. Thus, only the decimal (D), general (G), and hexadecimal (X) formats are supported.

The BigInteger class is an example of a new base type that we expect many other library types to consume for specialized applications. The next type we'll cover is a new collection that we added due to popular demand: a set class.

High-Performance Sets

The new HashSet class is a high-performance generic collection in the System.Collections.Generic namespace. It is an unordered collection that contains unique elements. HashSet implements all the standard collection methods (such as Add, Remove, and Contains) and provides several set operations (including union, intersection, and symmetric difference).

The following sample code demonstrates the use of HashSet with integers:

HashSet<int> set1 = new
    HashSet<int>();
set1.Add(1);
set1.Add(3);
set1.Add(5);
set1.Add(3); // set1 already contains 3; it isn't added twice
// set1 contains 1,3,5

HashSet<int> set2 = new HashSet<int>(new int[] { 2, 4, 6 });
// set2 contains 2,4,6

set2.UnionWith(set1);
// set2 contains 1,2,3,4,5,6

Notice how after adding four items (1, 3, 5, and 3), set1 ends up containing only three elements (1, 3, and 5). This behavior is by design. HashSet contains only unique elements. Once an item has been added to the set, subsequent attempts to add the same item will not succeed. The Add method actually returns a Boolean value indicating whether or not the item was successfully added to the set. The example above could have been rewritten as follows:

HashSet<int> set1 = new HashSet<int>();
bool added = set1.Add(1); // added is true
added = set1.Add(3); // added is true
added = set1.Add(5); // added is true
added = set1.Add(3); // added is false
// set1 contains 1,3,5

In the first example, set2 is initialized by passing in an array of ints ({ 2, 4, 6}) to the HashSet constructor, which takes a generic IEnumerable as a parameter. If there are duplicate elements in the IEnumerable, they are ignored when initializing the HashSet. For example, if set2 had been initialized with an array containing 2, 4, 2, and 6, the set would still end up containing only three elements (2, 4, and 6) since duplicates are ignored.

The set operations provided by HashSet (UnionWith, IntersectWith, ExceptWith, and SymmetricExceptWith) don't just take another HashSet object as a parameter-they actually take a generic IEnumerable. This means that the set operations can be used with any other collection. Figure 3 shows how the set operations can work with any collection that implements the generic IEnumerable interface. As with the constructor overload that takes an IEnumerable, duplicates are ignored. (The behavior of UnionWith, IntersectWith, ExceptWith, and SymmetricExceptWith is outlined in Figure 4.)

Figure 4 Set Operations (Close Figure 4)

Set Operation | Description | Example
UnionWith(IEnumerable<T> other); | Modifies the HashSet to contain all the elements contained by both the HashSet or other. | // set1 contains 1,2,3,4 // set2 contains 3,4,5,6 set1.UnionWith(set2); // set1 contains 1,2,3,4,5,6
IntersectWith(IEnumerable<T> other); | Modifies the HashSet to contain all the elements contained by both the HashSet and other. | // set1 contains 1,2,3,4 // set2 contains 3,4,5,6 set1.IntersectWith(set2); // set1 contains 3,4
ExceptWith(IEnumerable<T> other); | Removes all elements contained by the HashSet that are contained by other. | // set1 contains 1,2,3,4 // set2 contains 3,4,5,6 set1.ExceptWith(set2); // set1 contains 1,2
SymmetricExceptWith(IEnumerable<T> other); | Modifies the HashSet to contain all elements contained by either the HashSet or other, but not both. | // set1 contains 1,2,3,4 // set2 contains 3,4,5,6 set1.SymmetricExceptWith(set2); // set1 contains 1,2,5,6
Figure 3 Set Operations and Generic IEnumerable (Close Figure 3)
List<int> oddNumbers = new List<int>();
oddNumbers.Add(1);
oddNumbers.Add(3);
oddNumbers.Add(5);

HashSet<int> numberSet = new HashSet<int>();
numberSet.UnionWith(oddNumbers);
// numberSet contains 1,3,5

int[] evenNumbers = { 2, 4, 6, 2, 6, 4, 2, 4, 6, 2 };
numberSet.UnionWith(evenNumbers); // duplicates are ignored
// numberSet contains 1,2,3,4,5,6

numberSet.ExceptWith(oddNumbers);
// numberSet contains 2,4,6

HashSet provides more operations than just the methods noted in the table. These include IsSubsetOf, IsProperSubsetOf, IsSupersetOf, and IsProperSupersetOf. With these, HashSet supports most of the mathematical operations that can typically be performed on sets.

As you can see, getting started using this new collection is quite simple. HashSet fits in nicely with our other generic collections and fills one of the last remaining holes in our collection APIs.

Pipes

The last classes we're going to look at in this column are the pipe classes. For Visual Studio "Orcas," we've added support for both anonymous and named pipes to the .NET Framework. Pipes are used to achieve interprocess communication (IPC) between two or more processes on the same machine or over a network. The new types, which can be found in the System.IO.Pipes namespace, expose nearly all the pipe functionality provided by Windows.

Anonymous pipes are used for communication between a parent process and a child process. These one-way pipes are unnamed and must be used locally on the same machine. Figure 5 shows the code for sending a string from a parent process to a child process.

Figure 5 Anonymous Pipes (Close Figure 5)

Parent Process

using (Process process = new Process())
{
    process.StartInfo.FileName = "child.exe";
    using (AnonymousPipeServerStream pipeStream =
        new AnonymousPipeServerStream(PipeDirection.Out,
            HandleInheritability.Inheritable))
   {
        process.StartInfo.Arguments =
            pipeStream.GetClientHandleAsString();
        process.StartInfo.UseShellExecute = false;
        process.Start();

        pipeStream.DisposeLocalCopyOfClientHandle();
        using (StreamWriter sw = new StreamWriter(pipeStream))
        {
            sw.AutoFlush = true;
            sw.WriteLine(Console.ReadLine());
        }
    }
    process.WaitForExit();
}

Child Process

using (PipeStream pipeStream =
    new AnonymousPipeClientStream(PipeDirection.In, args[0]))
{
    using (StreamReader sr = new StreamReader(pipeStream))
    {
        string temp;
        while ((temp = sr.ReadLine()) != null)
        {
            Console.WriteLine(temp);
        }
    }
}

Named pipes provide much more functionality than anonymous pipes. For starters, they are full duplex and can be used over a network. Named pipes support multiple server instances of a single name, message-based communication, asynchronous I/O, and impersonation. Figure 6 shows how to send two strings from one process to another process using a named pipe.

Figure 6 Named Pipes (Close Figure 6)

Server Process

using (NamedPipeServerStream pipeStream =
    new NamedPipeServerStream("mypipe"))
{
    pipeStream.WaitForConnection();
    using (StreamWriter sw = new StreamWriter(pipeStream))
    {
        sw.AutoFlush = true;
        sw.WriteLine(Console.ReadLine());
        sw.WriteLine(Console.ReadLine());
    }
}

Client Process

using (NamedPipeClientStream pipeStream =
    new NamedPipeClientStream("mypipe"))
{
    pipeStream.Connect();
    using (StreamReader sr = new StreamReader(pipeStream))
    {
        string temp;
        while ((temp = sr.ReadLine()) != null) {
            Console.WriteLine(temp);
        }
    }
}

Named pipes also support message-based communication (demonstrated in Figure 7). This allows a reading process to read varying-length messages sent by the writing process. In Figure 7, two string messages are sent from a server process to a client process. The example includes two helper classes-MessageWriter and MessageReader-that do the actual work of writing and reading string messages to and from the underlying named pipe streams.

Figure 7 Message-Based Communication (Close Figure 7)

Helper Classes

class MessageWriter
{
    NamedPipeServerStream m_pipeStream;
    Encoding m_encoding;
    public MessageWriter(NamedPipeServerStream pipeStream,
        Encoding encoding)
    {
        m_pipeStream = pipeStream;
        m_encoding = encoding;
    }
    public void WriteMessage(string message)
    {
        Byte[] buffer;
        buffer = m_encoding.GetBytes(message);
        m_pipeStream.Write(buffer, 0, buffer.Length);
    }
}
class MessageReader
{
    NamedPipeClientStream m_pipeStream;
    Decoder m_decoder;
    Byte[] m_buffer = new Byte[10];
    Char[] m_charBuffer = new Char[10];
    StringBuilder m_stringBuilder = new StringBuilder();
    public MessageReader(NamedPipeClientStream pipeStream,
        Encoding encoding)
    {
        m_pipeStream = pipeStream;
        m_decoder = encoding.GetDecoder();
    }
    public string ReadMessage()
    {
        // Clear the StringBuilder
        m_stringBuilder.Length = 0;
        // Read the message
        int bytesRead;
        do {
            bytesRead = m_pipeStream.Read(m_buffer, 0, m_buffer.Length);
            int numChars = m_decoder.GetChars(m_buffer, 0, bytesRead,
                m_charBuffer, 0);
            m_stringBuilder.Append(m_charBuffer, 0, numChars);
        } while (!m_pipeStream.IsMessageComplete);
        // Return the message or null if the pipe has been closed
        if (bytesRead != 0) {
            m_decoder.Reset();
            return m_stringBuilder.ToString();
        }
        else return null;
    }
}

Server Process

using (NamedPipeServerStream pipeStream =
    new NamedPipeServerStream("messagepipe",
    PipeDirection.InOut, 1, PipeTransmissionMode.Message))
{
    pipeStream.WaitForConnection();
    MessageWriter mw = new MessageWriter(pipeStream, Encoding.UTF8);
    mw.WriteMessage("Hello World!");
    mw.WriteMessage("Named Pipes Are Cool!");
}

Client Process

using (NamedPipeClientStream pipeStream =
    new NamedPipeClientStream("messagepipe"))
{
    pipeStream.Connect();
    pipeStream.ReadMode = PipeTransmissionMode.Message;
    MessageReader mr = new MessageReader(pipeStream, Encoding.UTF8);
    string message;
    while ((message = mr.ReadMessage()) != null)
    {
        Console.WriteLine(message);
    }
}

As you can see, the new pipe types are first-class managed citizens, making it easy to achieve IPC from within managed code. Anyone familiar with streams should have no trouble getting started using pipes.

Conclusion

The classes discussed in this column are a sampling of the new green bits functionality available in Visual Studio "Orcas." Download the latest CTP at msdn2.microsoft.com/en-us/vstudio/aa700831.aspx to give these types a test drive. And stop by the BCL Team's blog to let us know what you think.

Send your questions and comments to clrinout@microsoft.com.

Mike Downen is a Program Manager focusing on security and deployment on the CLR team. He can be reached via his blog at blogs.msdn.com/CLRSecurity.

Inbar Gazit is a Program Manger focusing on the base class libraries for the CLR team. He can be reached via e-mail at inbar@microsoft.com.

Justin Van Patten is a Program Manager for the base class libraries within the CLR team. He can be reached via the BCL team blog at blogs.msdn.com/bclteam.


Data Points: Disabling Constraints and Triggers

Data Points
Disabling Constraints and Triggers
John Papa

Code download available at: DataPoints2007_04.exe (153KB)
Contents
Disabling Foreign Keys
Disabling Triggers
Synchronizing an Offline/Mobile Database
Cursors and Information Schema Views
Disabling All Foreign Keys
Disabling All Triggers
Disabling All Triggers Quickly
Wrapping Up

C
onstraints are valuable tools for maintaining data integrity in databases. However, there are times when it is convenient to disable one or more of them to perform tasks such as data synchronization with an offline database. When SQL Server™ replication is used to synchronize data between databases, individual objects can be told to withhold enforcement during replication.

For example, when a foreign key constraint is defined using the NOT FOR REPLICATION statement, SQL Server will not enforce the constraint during the replication process. In fact, the NOT FOR REPLICATION statement can be used directly in T-SQL statements that define foreign key constraints, check constraints, identities, and triggers. For those who use SQL Server replication, using the NOT FOR REPLICATION statement on the appropriate objects is a good option. However, if you are instead performing a manual synchronization of your data, another option is to disable the constraints and triggers manually.

Manual synchronization is often performed in online/offline applications where there is a subset of the data to be synchronized and where greater control over how the data is synchronized is required. In this month's column, I will discuss when it might be advantageous to manually disable and enable constraints, what kinds of problems this can solve, and some tricks to help you out.

Disabling Foreign Keys

I do not advocate removing foreign key constraints from a relational database. However, there are times when you may need to temporarily relieve the referential integrity checks of one or more foreign keys, such as when you're performing a large number of inserts and updates on a series of tables and want more accurate results and better performance. Of course, you should only use this technique when it is impractical to perform massive amounts of data updates to an entire database in the proper relational order.

So when might it be a good idea to disable foreign key constraints? Imagine you have a relational database structure that has dozens of tables, all of which are related to each other in some way through foreign key constraints. The application that interacts with this database has an offline/mobile version that communicates with a second instance of the database that likely resides on a laptop. Data changes that are made to the main database might need to be synchronized with the offline/mobile database, and there are several ways to handle this.

One way to synchronize the data is to apply the inserts, updates, and deletes to the offline/mobile database in the order that adheres to the relational structure. For example, insert customers before their respective orders, and orders before their order details. Record deletion would be performed in the opposite direction (from child to parent). This technique, however, can become too complex to implement and maintain on large database structures.

Another technique is to drop the foreign key constraints, synchronize the data, and then recreate the foreign key constraints. While this approach can work, a less dramatic approach is to merely disable the foreign key constraints. Once that's done, the data can be synchronized and then the foreign keys can be enabled once again. The syntax to disable a foreign key is shown here:

-- Disable foreign key constraint
ALTER TABLE Orders
    NOCHECK CONSTRAINT
        FK_Orders_Customers

-- Add a new Order record for a non-existent customer
INSERT INTO Orders (CustomerID) VALUES ('BLAH')

-- Select all orders for the non-existent customer
SELECT * FROM Orders WHERE CustomerID = 'BLAH'

This foreign key enforces that the CustomerID in the orders table be a valid CustomerID from the Customers table. The code disables the foreign key and then inserts an order into the Orders table. The order record that is inserted has a CustomerID that does not exist in the parent Customers table. Because the foreign key is disabled, the integrity check is bypassed and the order record is inserted successfully.

The following code demonstrates how to re-enable the foreign key constraint and then test that it is working. When this code is executed, the order is not inserted because the constraint is enforced. An error message will be returned stating that the insert statement conflicts with the foreign key constraint.

-- Enable foreign key constraint
ALTER TABLE Orders
    CHECK CONSTRAINT
        FK_Orders_Customers

-- Add a new Order record for a non-existent customer
INSERT INTO Orders (CustomerID) VALUES ('BLEH')

When disabling foreign keys, triggers, and other constraints, it is important to make sure that no Data Manipulation Language (DML) operations are being performed on the database during that time period. This must be handled when performing the disabling options manually. If you are instead synchronizing data with SQL Server replication and the NOT FOR REPLICATION statement, this condition is handled automatically.

Disabling Triggers

It's also possible to disable a trigger when necessary. Sometimes when synchronizing data you do not want a trigger to fire. For instance, if you are updating an offline/mobile database (as in the previous example), there may be triggers on several tables that you may not be aware of. These triggers may fire when a row is inserted into their table causing the row to be reinserted into another table. But during a large data synchronization, you might not want those types of inserts to be reinserted. To avoid that eventuality, you can temporarily disable the trigger, like so:

DISABLE TRIGGER MyTriggerName ON MyTableName

Conversely, to enable it, do this:

ENABLE TRIGGER MyTriggerName ON MyTableName

The following code shows a trigger that will fire (and display a message) when one or more rows are inserted or updated in the Region table.

CREATE TRIGGER trRegion_InsertUpdate ON Region
    FOR INSERT, UPDATE
AS
    PRINT ' Trigger is running. '
          +  CAST(@@ROWCOUNT AS VARCHAR(10))
          + ' row(s) affected.'
GO

You can test this trigger by inserting a new region into the Region table, like so:

INSERT INTO Region (RegionID, RegionDescription)
VALUES (5, 'Some New Region')

When this code is executed in a query window, the new region is inserted into the Region table, the trigger fires, and a message is displayed in the messages pane. To disable the trigger, you can execute the following code:

DISABLE TRIGGER trRegion_InsertUpdate ON Region

If you execute this code to update the new region, the data is updated but the trigger will not fire:

UPDATE Region SET RegionDescription = 'A Great Region'
WHERE RegionID = 5

To re-enable the trigger, simply execute the following query:

ENABLE TRIGGER trRegion_InsertUpdate ON Region

Synchronizing an Offline/Mobile Database

Disabling a single trigger or foreign key has its uses, but disabling all of them in one fell swoop can be extremely useful in the offline/mobile database synchronization scenario I described. To do so, you would take the following steps: disable all foreign key constraints, disable all triggers, perform the insert, update and delete operations, enable all foreign key constraints, and finally re-enable all triggers.

By disabling the triggers and foreign key constraints, the order in which the data is modified becomes inconsequential. In a small database like Northwind, this is a minor savings; in a large database with dozens or hundreds of tables and relationships, however, this can be a huge time saver. In addition, without this technique you would have to modify the sequence of the tables in your script to insert/update/delete the data every time the database schema adds new tables and relationships. With the foreign key constraints and triggers disabled, you can just add the tables to the end of your script since order is not important.

It is critical to remember to enable the triggers and foreign keys at the end, regardless of whether the script was successful or not. For example, if your script disables the constraints and triggers and attempts to modify the data, and the attempt fails, you would still want the constraints and triggers to be enabled at the end.

Cursors and Information Schema Views

To disable all foreign keys, you must first gather a list of them and the tables to which each is applied. Here, Cursors and INFORMATION_SCHEMA views can assist you. The view named INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS will return a list of the foreign keys in the current database. To get the name of the table that the foreign key is applied to, use the view called INFORMATION_SCHEMA.TABLE_CONSTRAINTS. The following code joins these two views to return a list of all of the foreign keys and their respective tables.

SELECT
  ref.constraint_name AS FK_Name,
  fk.table_name AS FK_Table
FROM
  INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS ref
  INNER JOIN INFORMATION_SCHEMA.TABLE_CONSTRAINTS fk
  ON ref.constraint_name = fk.constraint_name
ORDER BY
  fk.table_name,
  ref.constraint_name

Disabling All Foreign Keys

Now that you have your list, you can write a script to disable the foreign keys. You can declare and open a cursor, iterate through the list, create the T-SQL command that will disable a foreign key, and execute it for each foreign key (see Figure 1).

Figure 1 Disabling and Enabling All Foreign Keys (Close Figure 1)
CREATE PROCEDURE pr_Disable_Triggers_v2
    @disable BIT = 1
AS
    DECLARE
        @sql VARCHAR(500),
        @tableName VARCHAR(128),
        @tableSchema VARCHAR(128)

    -- List of all tables
    DECLARE triggerCursor CURSOR
        FOR
    SELECT
        t.TABLE_NAME AS TableName,
        t.TABLE_SCHEMA AS TableSchema
    FROM
        INFORMATION_SCHEMA.TABLES t
    ORDER BY
        t.TABLE_NAME,
        t.TABLE_SCHEMA

    OPEN triggerCursor

    FETCH NEXT FROM triggerCursor
    INTO @tableName, @tableSchema

    WHILE ( @@FETCH_STATUS = 0 )
        BEGIN
            IF @disable = 1
                SET @sql = ‘ALTER TABLE ‘ + @tableSchema
                    + ‘.[‘ + @tableName + ‘] DISABLE TRIGGER ALL’
            ELSE
                SET @sql = ‘ALTER TABLE ‘ + @tableSchema
                    + ‘.[‘ + @tableName + ‘] ENABLE TRIGGER ALL’

            PRINT ‘Executing Statement - ‘ + @sql

            EXECUTE ( @sql )
            FETCH NEXT FROM triggerCursor
            INTO @tableName, @tableSchema
        END

    CLOSE triggerCursor
    DEALLOCATE triggerCursor

First, the foreignKeyCursor cursor is declared as the SELECT statement that gathers the list of foreign keys and their table names. Next, the cursor is opened and the initial FETCH statement is executed. This FETCH statement will read the first row's data into the local variables @foreignKeyName and @tableName.

When looping through a cursor, you can check the @@FETCH_STATUS for a value of 0, which indicates that the fetch was successful. This means the loop will continue to move forward so it can get each successive foreign key from the rowset.

@@FETCH_STATUS is available to all cursors on the connection. So if you are looping through multiple cursors, it is important to check the value of @@FETCH_STATUS in the statement immediately following the FETCH statement. @@FETCH_STATUS will reflect the status for the most recent FETCH operation on the connection. Valid values for @@FETCH_STATUS are:

  • 0 = FETCH was successful
  • -1 = FETCH was unsuccessful
  • -2 = the row that was fetched is missing

Inside the loop, the code builds the ALTER TABLE command differently depending on whether the intention is to disable or enable the foreign key constraint (using the CHECK or NOCHECK keyword). The statement is then printed as a message so its progress can be observed and then the statement is executed. Finally, when all rows have been iterated through, the stored procedure closes and deallocates the cursor.

Disabling All Triggers

The stored procedure in Figure 1 will disable or enable all foreign keys in a database. In some cases, you will want to disable all triggers during data synchronization, as well. The pr_Disable_Triggers stored procedure in Figure 2 does just that.

Figure 2 Disabling and Enabling All Triggers (Close Figure 2)
CREATE PROCEDURE pr_Disable_Triggers
@disable BIT = 1
AS
    DECLARE
        @sql VARCHAR(500),
        @tableName VARCHAR(128),
        @triggerName VARCHAR(128),
        @tableSchema VARCHAR(128)

    -- List of all triggers and tables that exist on them
    DECLARE triggerCursor CURSOR
        FOR
    SELECT
        so_tr.name AS TriggerName,
        so_tbl.name AS TableName,
        t.TABLE_SCHEMA AS TableSchema
    FROM
        sysobjects so_tr
    INNER JOIN sysobjects so_tbl ON so_tr.parent_obj = so_tbl.id
    INNER JOIN INFORMATION_SCHEMA.TABLES t
    ON
        t.TABLE_NAME = so_tbl.name
    WHERE
        so_tr.type = ‘TR’
    ORDER BY
        so_tbl.name ASC,
        so_tr.name ASC

    OPEN triggerCursor

    FETCH NEXT FROM triggerCursor
    INTO @triggerName, @tableName, @tableSchema

    WHILE ( @@FETCH_STATUS = 0 )
        BEGIN
            IF @disable = 1
                SET @sql = ‘DISABLE TRIGGER [‘
                    + @triggerName + ‘] ON ‘
                    + @tableSchema + ‘.[‘ + @tableName + ‘]’
            ELSE
                SET @sql = ‘ENABLE TRIGGER [‘
                    + @triggerName + ‘] ON ‘
                    + @tableSchema + ‘.[‘ + @tableName + ‘]’

            PRINT ‘Executing Statement - ‘ + @sql
            EXECUTE ( @sql )
            FETCH NEXT FROM triggerCursor
            INTO @triggerName, @tableName,  @tableSchema
        END

    CLOSE triggerCursor
    DEALLOCATE triggerCursor

The pr_Disable_Triggers stored procedure gets a rowset containing all of triggers in the current database as well as the names of their respective tables (and their schema). Since there is no INFORMATION_SCHEMA.TRIGGERS view, I gather the list of triggers and the related information from a combination of the SQL Server 2005 system tables and INFORMATION_SCHEMA views, where possible.

The sysobjects system table exists in all databases and can be queried to return a list of all triggers or tables. This system table can be joined against itself to get a list of the triggers and the tables that they operate on, as shown in the declaration of the cursor in Figure 2.

If you use this stored procedure on a database such as AdventureWorks where the tables are under specific schemas, you must prefix the name of the table with the schema name. The pr_Disable_Triggers stored procedure handles this situation by also joining to the INFORMATION_SCHEMA.TABLES view, which returns the SCHEMA_NAME for the table.

Once you have written routines that will disable and re-enable the triggers and foreign keys, you can use them in a script that will modify an offline/mobile database to keep it synchronized with a main database. For example, the following script could be used, replacing the comment in the middle with the data manipulation operations.

pr_Disable_Foreign_Keys  1
pr_Disable_Triggers 1
-- Perform data operations
pr_Disable_Foreign_Keys  0
pr_Disable_Triggers 0

Disabling All Triggers Quickly

The pr_Disable_Triggers stored procedure shown in Figure 2 could be modified to execute a slightly different T-SQL command that will disable or enable all of the triggers. There is a T-SQL statement using a flavor of the ALTER TABLE command that will disable or enable all triggers on a table. Using this technique, the pr_Disable_Triggers stored procedure can be modified to execute the following statement for each table to disable all of the triggers on that table:

ALTER TABLE MySchemaName.MyTableName DISABLE TRIGGER ALL

By using this syntax, the cursor's query would gather all of the table names but not need to get the names of the triggers themselves. Thus, the stored procedure could be modified to use this technique to loop through the list of tables and enable or disable all triggers on each table. The modified stored procedure is shown in Figure 3.

Figure 3 Disabling All Triggers with the ALTER TABLE Command (Close Figure 3)
CREATE PROCEDURE pr_Disable_Foreign_Keys
    @disable BIT = 1
AS
    DECLARE
        @sql VARCHAR(500),
        @tableName VARCHAR(128),
        @foreignKeyName VARCHAR(128)

    -- A list of all foreign keys and table names
    DECLARE foreignKeyCursor CURSOR
    FOR SELECT
        ref.constraint_name AS FK_Name,
        fk.table_name AS FK_Table
    FROM
        INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS ref
        INNER JOIN INFORMATION_SCHEMA.TABLE_CONSTRAINTS fk
    ON ref.constraint_name = fk.constraint_name
    ORDER BY
        fk.table_name,
        ref.constraint_name

    OPEN foreignKeyCursor

    FETCH NEXT FROM foreignKeyCursor
    INTO @foreignKeyName, @tableName

    WHILE ( @@FETCH_STATUS = 0 )
        BEGIN
            IF @disable = 1
                SET @sql = ‘ALTER TABLE [‘
                    + @tableName + ‘] NOCHECK CONSTRAINT [‘
                    + @foreignKeyName + ‘]’
            ELSE
                SET @sql = ‘ALTER TABLE [‘
                    + @tableName + ‘] CHECK CONSTRAINT [‘
                    + @foreignKeyName + ‘]’

        PRINT ‘Executing Statement - ‘ + @sql

        EXECUTE(@sql)
        FETCH NEXT FROM foreignKeyCursor
        INTO @foreignKeyName, @tableName
    END

    CLOSE foreignKeyCursor
    DEALLOCATE foreignKeyCursor

Wrapping Up

It is more efficient to perform multiple database operations such as those described in this article from a T-SQL script than it is to execute them all individually from a .NET application. For example, you could gather a list of foreign keys from a .NET application using ADO.NET and then execute the commands to disable each of the foreign keys. This could be extended to get the list of triggers and then disable them. All of these actions would require going back and forth between the .NET code and the database, consuming more resources than if the code were executed entirely on the database server in T-SQL.

Send your questions and comments for John to mmdata@microsoft.com.

John Papa is a senior .NET Consultant with ASPSOFT (aspsoft.com) and a baseball fanatic who spends most of his summer nights rooting for the Yankees with his family and his faithful dog, Kadi. John, a C# MVP, has authored several books on ADO, XML, and SQL Server, and can often be found speaking at industry conferences such as VSLive or blogging at codebetter.com/blogs/john.papa.


Team System: Work Item Tracking

Team System
Work Item Tracking
Brian A. Randell

Code download available at: TeamSystem2007_04.exe (179KB)
Contents
Team Foundation Servers, Team Projects, and Work Items
Getting Connected
Accessing the WIT Store
Working with Queries
Presenting and Editing Work Items
Conclusion

I
n my previous column, I started to describe how you can build a source-code control add-in for Microsoft® Word 2003 using the APIs exposed in Team System. If you examine the check-in dialog exposed by Team Explorer in Visual Studio® 2005, you'll notice that the integrated check-in experience is quite rich. Not only can you check in source files, but you can also associate your check-in with work items, add check-in notes, and validate your check-in against policy. Figure 1 shows the standard check-in dialog with the Work Items option selected.

Figure 1 Team Explorer Integrated Check-In Dialog
Figure 1 Team Explorer Integrated Check-In Dialog(Click the image for a smaller view)
Figure 1 Team Explorer Integrated Check-In Dialog
Figure 1 Team Explorer Integrated Check-In Dialog (Click the image for a larger view)

This is deceptively simple on the surface-in reality, this part of the check-in experience exposes a great number of work item features. For example, you can change the query used to display the work item list. You can drill down into a particular work item and manipulate it using the standard work item UI by double-clicking on it. And you can perform a check-in action, which you use to either associate the work item with the check-in's changeset or perform another action, such as marking the work item as complete.

Before you can implement these features in the add-in, you'll need a good understanding of the work item API. In this column, I'll explain how to build a simple Work Item Explorer (see Figure 2). This sample demonstrates the core operations needed to add work item support to the add-in. Due to space constraints, I'll only cover the core code needed to access the work item services of Visual Studio 2005 Team Foundation Server. I will not detail the sample's Windows® Forms-related code. (All the code is, however, provided in the download.) In the next installment of the Team System column, I'll show you how to add work item support to the add-in.

Figure 2 Simple Work Item Explorer
Figure 2 Simple Work Item Explorer(Click the image for a smaller view)
Figure 2 Simple Work Item Explorer
Figure 2 Simple Work Item Explorer (Click the image for a larger view)

Team Foundation Servers, Team Projects, and Work Items

The work item tracking subsystem is a core feature of a Team Foundation Server installation. In order to use work items, you need at least one team project, which you create using the New Team Project wizard. When using this wizard, you specify the process template to use (a blueprint that describes the structure your team project will be based upon).

Of particular interest to this discussion, the process template you choose defines the default types of work items your team project will initially support. You can choose to customize the existing work item types or add your own. Work items provide a way for your team members to track just about anything related to the team project, including, but not limited to, bugs and tasks.

A default Team Foundation Server installation includes two process templates: Microsoft Solutions Framework for Agile Software Development version 4.0 (MSF Agile for short), and MSF for CMMI Process Improvement version 4.0 (MSF CMMI for short). Figure 3 provides an overview of the default types of work items, what each is for, and with which templates each is included.

Figure 3 Default Work Item Types (Close Figure 3)

Work Item Type | Description | MSF Agile | MSF CMMI
Bug | Communicates that a potential problem exists or has existed in the system. |  |
Change Request | Identifies a proposed change to some part of the product or baseline. |  |
Issue | Documents an event or situation that may block work or is currently blocking work on the product. |  |
Quality of Service Requirement | Documents characteristics of the system, such as performance, load, availability, stress, accessibility, serviceability, and maintainability. |  |
Requirement | Captures and tracks what the product needs to do to solve the customer problem. |  |
Review | Documents the results of a design or code review. |  |
Risk | Identifies any probable event or condition that can have a potentially negative outcome on the project in the future. |  |
Scenario | Records a single path of user interaction through the system. |  |
Task | Communicates the need to do some work. |  |

Each work item is composed of a set of fields. Team Foundation Server scopes fields to the entire server installation. You can create your own custom process template and thus your own custom work items either from scratch or by modifying the templates provided by Microsoft. When defining your own custom work items, you can also define custom field definitions. This, however, is a discussion for another article.

Getting Connected

The add-in currently connects to the Team Foundation Server installation and its version control service via the Connect method in the tfsvcUtil class. In addition, it uses a custom Windows Forms dialog to get the server connection string from the user. You can use that same code with a few slight modifications to include a connection to the work item tracking (WIT) store. First, you need to add a reference to the Microsoft.TeamFoundation.WorkItemTracking.Client.dll assembly. Then, in the tfsvcUtil class, you need add the following Imports statement:

Imports Microsoft.TeamFoundation.WorkItem-Tracking.Client

You must also provide a class-level variable defined to hold a reference to the WIT store:

Private m_tfsWIT As WorkItemStore
And finally, you need to add an additional line of code to the Connect method so it can connect to the WIT store (see Figure 4).

Figure 4 Connecting to the WIT Store (Close Figure 4)
Public Shared Sub Connect( _
    ByVal fqServerName As String, ByVal showLoginUI As Boolean)
  Try
    m_serverName = fqServerName
    If showLoginUI Then
      Dim icp As ICredentialsProvider = New UICredentialsProvider
      m_tfs = TeamFoundationServerFactory.GetServer( _
        m_serverName, icp)
    Else
      m_tfs = TeamFoundationServerFactory.GetServer(m_serverName)
    End If

    m_tfs.Authenticate()

    ' Version Control Server
    m_tfsVCS = CType(m_tfs.GetService( _
      GetType(VersionControlServer)), _
      VersionControlServer)

    ' Work Item Store
    m_tfsWIT = CType(m_tfs.GetService( _
      GetType(WorkItemStore)), WorkItemStore)

    serverValidated = True
  Catch ex As Exception
    Throw New tfsUtilException( _
      String.Format(MSG_UNEXPECTED, "Connect"), ex, m_serverName)
  End Try
End Sub

Assuming a valid TeamFoundationServer reference exists, all it takes to connect to the work item store is to call the TeamFoundationServer's GetService method and pass the type object for the WorkItemStore. If you dig deeper into the Team Foundation Server APIs, you'll discover the DomainProjectPicker class-a hidden gem in the Proxy namespace within the Microsoft.TeamFoundation.dll assembly. This class exposes a GUI that lets you define Team Foundation Server connections and optionally access team projects defined on a server. The sample application uses this class rather than the aforementioned code. The sample app needs references to the following Team Foundation Server assemblies to perform its tasks:

  • Microsoft.TeamFoundation.dll
  • Microsoft.TeamFoundation.Client.dll
  • Microsoft.TeamFoundation.Common.Library.dll
  • Microsoft.TeamFoundation.WorkItemTracking.Client.dll
  • Microsoft.TeamFoundation.WorkItemTracking.Controls.dll

The main form (frmMain.vb) of the sample application has six buttons, a combobox to list the selected team projects, a combobox to list the available work item types for the active team project, and a few labels to provide status information. You need to add the following Imports statements to the top of frmMain.vb:

Imports Microsoft.TeamFoundation.Client
Imports Microsoft.TeamFoundation.Common
Imports Microsoft.TeamFoundation.Proxy
Imports Microsoft.TeamFoundation.Server
Imports Microsoft.TeamFoundation.WorkItemTracking.Client

You must also add code to the click event handler in the Pick Team Foundation Server button, as shown in Figure 5 (note that the GUI management-related code has been left out for space purposes). This code displays the default Connect to Team Foundation Server dialog. The constructor of the DomainProjectPicker class provides an enumeration to control the style of the dialog. For example, you can choose to show the dialog with team project selection turned off. However, for this sample the default behavior is what you want. Figure 6 displays the results of the call to ShowDialog using the default constructor.

Figure 5 Using the DomainProjectPicker Class to Connect (Close Figure 5)
Private Sub btnPickTFS_Click( _
    ByVal sender As System.Object, ByVal e As System.EventArgs) _
    Handles btnPickTFS.Click

  Dim dpp As New DomainProjectPicker()

  Select Case dpp.ShowDialog(Me)
    Case Windows.Forms.DialogResult.OK
      mwis = Nothing
      mprj = Nothing

      mtfs = dpp.SelectedServer()
      Me.lblTFS.Text = mtfs.Name

      Me.cboTeamProjects.DataSource = dpp.SelectedProjects()
      Me.cboTeamProjects.DisplayMember = "Name"
      If dpp.SelectedProjects.Length > 0 Then
        Me.cboTeamProjects.Enabled = True
        Me.btnConnectWIS.Enabled = True
      End If
  End Select
End Sub
Figure 6 Connect Dialog Box
Figure 6 Connect Dialog Box

When you click the Servers button, the dialog displays the built-in Add/Remove Team Foundation Server dialog. If you select one or more team projects and then click OK, the code binds the list of selected projects to the Team projects combobox control.

Accessing the WIT Store

Once you have a valid connection to the server, you can connect to the WIT store. In the click event handler of the Connect to Work Item Store button, you need to use the GetService method as discussed earlier to connect to the WIT store. Once connected, the code initializes a class-level Project reference using the currently selected team project name:

If mtfs IsNot Nothing Then
  mwis = CType(mtfs.GetService( _
    GetType(WorkItemStore)), WorkItemStore)
End If

mprj = mwis.Projects.Item(Me.cboTeamProjects.Text)

Once you have a connection to the WIT store, you can get information from the server. The private method UpdateInfoData, shown in Figure 7, performs this task and displays the data on the main form. The code first gets the selected project's list of work item types as an array of WorkItemTypes objects. It binds the array to the Type combobox, and then the code uses the WIT store's FieldDefinitions collection to get a count of the server's field definitions. Next, the code retrieves the count of stored queries, both public and private, available for the identity connecting to the server. Finally, the code constructs a dynamic Work Item Query Language (WIQL) query to get a list of work items available to the identity connecting. (More details on WIQL in a moment.) Note that the code gets the list of field definitions and the work item count via the WIT store reference, whereas it uses the active team project reference to retrieve the work item types and stored queries.

Figure 7 UpdateInfoData Listing (Close Figure 7)
Private Sub UpdateInfoData()
  If mprj IsNot Nothing Then
    Me.cboWIType.DataSource = mprj.WorkItemTypes
    Me.cboWIType.DisplayMember = "Name"
    Me.cboWIType.Enabled = True

    Me.lblFieldCount.Text = mwis.FieldDefinitions.Count.ToString()

    Me.lblSQCount.Text = mprj.StoredQueries.Count.ToString()

    Dim wiql As String = String.Format( _
      "SELECT [System.Id] FROM WorkItems " & vbCrLf & _
      "WHERE [System.TeamProject] = '{0}'", mprj.Name)
    Me.lblWICount.Text = mwis.QueryCount(wiql).ToString()
  End If

WIQL is a SQL-like language. Full syntax reference is available online at msdn2.microsoft.com/bb130155.aspx. You can execute WIQL queries either as a dynamic WIQL statement or from a stored query. You can publish queries to your Team Foundation Server, making them available as public queries for your entire team or as private queries for your use only.

A valid query returns a collection of WorkItems. The WIT only returns those work items that match the query request and are visible to the identity executing the query based upon any security restrictions. Note is that the WIT does not throw an exception if the resultset is restricted due to security. Each WorkItem type has a collection of Field objects; each field in turn maps to a FieldDefinition object that provides the metadata for that particular field.

Constructing a valid WIQL query requires a valid list of field names using either the field name or its reference name. The field name is the friendly version and, when composed of multiple words, usually has spaces. When used in a WIQL query, a field name with spaces needs to be enclosed in square brackets. The reference name is a fully qualified name of the field that resembles the format used for .NET types. For example, a work item's Title uses the friendly name of Title and the reference name of System.Title. As another example, the Integration Build field's name is Integration Build, and its reference name is Microsoft.VSTS.Build.IntegrationBuild. Its label on the CMMI templates Change Request work item form is Integrated In. You can examine all the field definitions for a particular server accessing the FieldDefinitions of a valid WorkItemStore reference. If you run the sample application and click the Show Fields List, you can view a list of all the fields defined and drill down into each item. You could use this information to help you construct WIQL queries or even build your own query-build UI.

Working with Queries

Each of the two process templates included in the default Team Foundation Server installation contains a set of predefined WIQL queries. The MSF Agile template includes 11, while the MSF CMMI template includes 16. You access stored queries on a per-project basis. Clicking the Show Stored Queries button in the sample application loads a form listing all the stored queries for the active team project. A listbox on the form displays all the stored queries by name. Double-clicking on a query will open a detail form similar to the one shown in Figure 8. If you examine the queries provided by Microsoft, you'll see that they use the reference names when referring to work item fields.

Figure 8 Stored Query Details for the All Tasks Query
Figure 8 Stored Query Details for the All Tasks Query(Click the image for a smaller view)
Figure 8 Stored Query Details for the All Tasks Query
Figure 8 Stored Query Details for the All Tasks Query (Click the image for a larger view)

The detail form has two buttons: one that executes the query and returns the results in a grid and one that returns the results in a list view. The WIT API supports numerous ways to execute a WIQL query. The detail form shown in Figure 8 uses two different techniques to execute the query. If you click the Execute Query (Grid) button, the code preps the WIQL statement and passes it to the frmWIList that actually executes the query. When you click the Execute Query (List) button, the query is prepared differently and executed using the Query method of the stored query's project's store reference (see Figure 9).

Figure 9 Executing a WIQL Query (Close Figure 9)
Private Sub btnExecuteGrid_Click( _
    ByVal sender As System.Object, ByVal e As System.EventArgs) _
      Handles btnExecuteGrid.Click

  Dim wiqlToExecute As String = mSQ.QueryText.ToLower()
  If wiqlToExecute.Contains(MACRO_PROJECT) Then
    wiqlToExecute = wiqlToExecute.Replace( _
      MACRO_PROJECT, "'" & mSQ.Project.Name & "'")
  End If

  Using f As New frmWIList(mSQ.Project.Store, wiqlToExecute)
    f.ShowDialog(Me)
  End Using
End Sub

Private Sub btnExecuteList_Click( _
    ByVal sender As System.Object, ByVal e As System.EventArgs) _
      Handles btnExecuteList.Click

  Dim wiqlToExecute As String = mSQ.QueryText.ToLower()
  Dim params As New Hashtable
  Dim wic As WorkItemCollection = Nothing

  If wiqlToExecute.Contains(MACRO_PROJECT) Then
    params.Add(MACRO_PROJECT.Substring(1), mSQ.Project.Name)
    wic = mSQ.Project.Store.Query(wiqlToExecute, params)
  Else
    wic = mSQ.Project.Store.Query(wiqlToExecute)
  End If

  If wic.Count = 0 Then
    MessageBox.Show("The query didn't return any Work Items.", _
      "Warning", MessageBoxButtons.OK, MessageBoxIcon.Exclamation)
  Else
    Dim wi As WorkItem = wic.Item(0)
    Dim d As New frmAWorkItem(Nothing)
    Dim title As String = _
      String.Format("{0}'s Work Items: {1}", _
      mSQ.Name, wic.Count)

    Using f As New frmListBox(title, wi, wic, "Title", d)
      f.ShowDialog(Me)
    End Using
  End If
End Sub

Both pieces of code prepare the WIQL by checking for a predefined query macro: @project. (Any string in a WIQL statement that is prefixed with an @ sign is considered to be a macro.) The WIT plumbing automatically expands certain built-in macros, such as @today. Your own macros and the @project macro require manual substitution.

Depending on how you execute the query, you can have the WIT API perform macro substitution or you can do it yourself. The code in btnExecuteGrid performs simple manual substitution looking only for the @project macro. However, btnExecuteList lets the WIT API do the heavy lifting. For production code, using the API is recommended because, with the substitution approach, you might inadvertently do an incorrect substitution (such as if you were substituting for @project, but there was an occurrence of a superstring, such as @projected).

If you look at the results of each button click and the APIs used, you'll see that the results you're looking for often force you to go down a particular path. In the sample application, the grid detail form uses a WorkItemResultGrid control (more on this in just a bit) that only supports a WIQL string as input limiting the API choices. The list view, on the other hand, uses standard data binding to hook the WorkItemCollection to a standard Windows Forms ListBox control. Still, the end result is the same-executing the query returns a collection of WorkItem objects.

Presenting and Editing Work Items

If you can execute a WIQL query-either dynamically or from a stored query-you have the ability to view and manipulate work items using the work item object model. At some point, however, you might need to present a UI so the user can enter, edit, or view work item information. In fact, if you perform a check-in using the Team Explorer integrated check-in experience, you'll find that double-clicking on a work item brings up the standard work item editor. This is exactly the feature you want to have in the add-in that you're building.

Microsoft has exposed the main work item experience via user controls within the Microsoft.TeamFoundation.WorkItemTracking.Controls.dll assembly. The sample application uses three of the controls: WorkItemResultGrid (which is visible when clicking the btnExecuteGrid button on the frmStoredQueryDetails.vb form), PickWorkItemsControl (which lets you execute stored queries and searches), and WorkItemFormControl (which is visible when you double-click on a work item from the grid view or list view forms). The WorkItemFormControl control is shown in Figure 10.

Figure 10 WorkItemFormControl
Figure 10 WorkItemFormControl(Click the image for a smaller view)
Figure 10 WorkItemFormControl
Figure 10 WorkItemFormControl (Click the image for a larger view)

These three controls do all the heavy lifting when it comes to searching, viewing, and editing work items. While you can add the controls to your toolbox, the sample application sets up the controls dynamically. There isn't any real design-time experience. The sample application uses the PickWorkItemsControl when you click the Search for Work Items button on the main form.

The control needs a reference to a valid WIT store object and a reference to the active project. From there it does all the hard work. The sample form that hosts the control hooks the control's PickWorkItemsListViewDoubleClicked event to show the work item using the WorkItemFormControl, as shown in Figure 10.

WorkItemFormControl exposes an Item property where you set the active work item that should be displayed. You have full access to each field and its dirty state. You can also reset the work item's values to their original queried values by using the Reset method. But keep in mind that it is up to you to implement your own save and cancel logic.

Figure 11 provides the relevant bits of code necessary to do just that. Note that if the user cancels the editing experience via the Cancel button or by using the window adornments, the code resets the work item. This is an important point if you're using a collection of WorkItems. The control takes changes in the UI and transfers them to the work item immediately. If you don't reset, the work item is dirty, and if it is reopened by the user, it will still have that information.

Figure 11 Handling Save and Cancel (Close Figure 11)
Imports Microsoft.TeamFoundation.WorkItemTracking.Client
Imports Microsoft.TeamFoundation.WorkItemTracking.Controls

Friend Class frmAWorkItem
  Implements IProcessDoubleClick

  Private IsNewWI As Boolean = False
  Private IsCanceled As Boolean = False
  Private wifCtl As WorkItemFormControl = Nothing

  Private Sub frmAWorkItem_FormClosing( _
    ByVal sender As Object, _
    ByVal e As System.Windows.Forms.FormClosingEventArgs) _
    Handles Me.FormClosing

    If wifCtl.Item.IsDirty AndAlso (Not IsCanceled) Then
      Select Case MessageBox.Show( _
        "You made changes to the work item." & vbCrLf & _
        "Would you like to save your changes?", "Save Changes", _
        MessageBoxButtons.YesNoCancel, MessageBoxIcon.Question)

        Case Windows.Forms.DialogResult.Yes
          e.Cancel = (Not Me.Save())

        Case Windows.Forms.DialogResult.No
          If wifCtl.Item.IsDirty OrElse _
            (Not wifCtl.Item.IsValid) Then

            wifCtl.Item.Reset()
          End If

        Case Windows.Forms.DialogResult.Cancel
          e.Cancel = True

      End Select
    End If
  End Sub

  Private Sub btnCancel_Click( _
    ByVal sender As Object, ByVal e As System.EventArgs) _
    Handles btnCancel.Click

    If wifCtl.Item.IsDirty OrElse (Not wifCtl.Item.IsValid) Then
      wifCtl.Item.Reset()
    End If

    IsCanceled = True
  End Sub

  Private Sub btnSave_Click( _
    ByVal sender As Object, ByVal e As System.EventArgs) _
    Handles btnSave.Click

    If wifCtl.Item.IsDirty Then
      Me.Save()
      Me.btnSave.Enabled = False
    End If
  End Sub

  Private Function Save() As Boolean
    Dim badFields As ArrayList = wifCtl.Item.Validate()
    If Not wifCtl.Item.IsValid Then
      Using sw As New IO.StringWriter
        sw.WriteLine("The following fields are invalid: ")
        For Each f As Field In badFields
          If Not f.IsValid Then
            sw.WriteLine(f.Name)
          End If
        Next

        MessageBox.Show(sw.ToString(), "Invalid fields", _
          MessageBoxButtons.OK, MessageBoxIcon.Error)

        MsgBox(sw.ToString())
      End Using

      Return False
    End If

    wifCtl.Item.Save()

    If IsNewWI Then
      Me.Text = "Work Item: " & wifCtl.Item.Id
      MessageBox.Show(String.Format( _
        "New {0} saved. It's ID Is {1}", wifCtl.Item.Type.Name, _
        wifCtl.Item.Id), "Save Work Item", _
        MessageBoxButtons.OK, MessageBoxIcon.Information)
    End If

    Return True
  End Function
End Class

When performing a save operation, you use the IsDirty and IsValid properties to determine if a save is necessary and possible. If the work item is not valid, you use the work item's Validate method to retrieve a list of the invalid fields. This gives you an opportunity to either correct the errors in code or return the user to the UI (which is what the code in Figure 11 does). If the work item is valid, you save the changes to the object and back to the Team Foundation Server using the Save method.

Finally, if you want to be able to create new work items in your application, you need one more piece of data. As part of the code shown earlier in Figure 7, a combobox on the main form is loaded with a list of work items supported by the active project. The combobox control lists the Name of each work item. This is all you really need to spin a work item and open the UI. The following code shows what you need to do:

Dim WIT As String = Me.cboWIType.Text
If WIT IsNot Nothing AndAlso WIT <> String.Empty Then
  Dim wi As WorkItem = mprj.WorkItemTypes(WIT).NewWorkItem()

  Using f As New frmAWorkItem(wi)
    f.ShowDialog(Me)
  End Using
End If

Basically, you pass the string name of the work item type to the WorkItemTypes collection object and call its NewWorkItem method. The code then passes the new work item instance to the frmAWorkItem object's constructor, which binds the new work item to the WorkItemForm control.

Conclusion

The Work Item Tracking subsystem offered by Team Foundation Server exposes a rich object model for integration with your own solutions. Accessing work item information from any application only requires a few lines of code. In addition, when building an application, you can let the built-in controls do the heavy lifting to provide a rich and consistent interface. In the next column, I'll show you how to integrate work item support into the add-in that I started in the previous column.

Send your questions and comments to mmvsts@microsoft.com.

Brian A. Randell is a senior consultant with MCW Technologies LLC. Brian spends his time speaking, teaching, and writing about Microsoft technologies. He is the author of Pluralsight's Applied Team System course and is a Microsoft MVP. Contact Brian via his blog at mcwtech.com/cs/blogs/brianr.


Cutting Edge: Subclassing and Overriding ASP.NET Pages—Part I

Cutting Edge
Subclassing and Overriding ASP.NET Pages—Part I
Dino Esposito

Code download available at: CuttingEdge2007_04.exe (874KB)
Contents
Possible Scenarios
Possible Techniques
Intercepting the Page Handler
Adding New Event Handlers
Getting a Reference to the Control Tree
Page Override with HTTP Handlers
URL Rewriting
Summary

A
client recently approached me and said "we need to enter changes to some ASP.NET pages. Can you help?" Like any consultant would, I promptly replied "Sure. Tell me more." But the client didn't actually have much more to share than the URLs to the pages. In a nutshell, the client had to modify some ASP.NET pages, but didn't have the source code. At first, I saw only trouble in this, but the more I talked to the client, the more I started to see an interesting challenge.

You don't need a button's source code to create a derived class, and you don't need an ASP.NET page's source code to modify its behavior. In Windows programming, you typically hook up low level messages and subclass a window. In ASP.NET, you can try to hook up page events to override both the page behavior and output.

After some brainstorming, I came up with a list of realistic scenarios where you might need to modify the runtime behavior of a page without touching the source code. I also came up with a number of techniques you can use to get the job done. In this two-part article, I'll go through these techniques and show you how to change the user interface and behavior of ASP.NET pages with just read-only access or even no access at all to the source code.

Possible Scenarios

Say a certain group of users run a personalized version of a Web application, and that application requires some remote debugging or profiling. You can't interrupt the service, but you still need to figure out what's going on. Performance counters and built-in tracing capabilities may not be enough. And the deployed pages may not even have tracing instructions incorporated or turned on. If this is the case, at a minimum, you'll need to intervene remotely to inject tracing code.

Another situation where this may be necessary is when temporary changes to a variety of pages are required. For a single page, you can just make a copy of the old page and replace it. Updating several pages, though, can be more difficult and make maintenance significantly more complex.

Yet another scenario is when a corporate policy prohibits write access to the source code. In this case, the source code is available, you can read it, but you can't modify it. You can, however, add extensions to the site.

Finally, the most dramatic situation-one that I really did encounter one time-is when the company runs a Web application for which the source code is no longer available for some reason.

As you can see, there are a number of scenarios in which you may need to modify the runtime behavior of pages without having access to the source code. So how would you proceed?

Possible Techniques

There are a number of techniques that allow you to modify a running ASP.NET page without touching its source code. Figure 1 lists a few approaches. Not all techniques are valid in all scenarios, and some of the techniques can be used together. The various techniques may require you to write a new HTTP module or HTTP handler, or enter changes to the web.config file. In most cases, you'll need to restart the application. In fact, changes to the web.config or global.asax files and additions to the Bin folder or the App_Code folder restart the application automatically.

Figure 1 Modify Pages without Source Code Access (Close Figure 1)

Technique | Implementation
Access the control tree | HTTP module
Modify the page base class | web.config
Control replacement | web.config
Build providers | Assembly
Redirect a page | HTTP handler
Override a page | HTTP handler or URL rewriting

By using HTTP modules, you can hook up any events in the application life cycle as well as capture the HTTP handler used to render out the page. When you hold such a reference, you can manage handlers for page events (such as Page_Load and Page_PreRender) and also add or remove controls, change links to resources and images, and add or remove CSS classes. Likewise, an HTTP handler can be used to replace a single page. And URL rewriting can be used to redirect users to pages that don't even exist when the application is first deployed.

If you're having trouble with a particular control, you can use the <tagMapping> element of the web.config file to automatically redirect the page parser to a new control derived from the old one. I'll explain this technique in my next column. For now, let's focus on tools and techniques you can use to override page-level events and modify the UI of a page on the fly without changing the source code.

Intercepting the Page Handler

The processing of each ASP.NET request causes a series of application-level events, as shown in Figure 2. The request starts with the BeginRequest event and finishes with the EndRequest event. A system object called HttpApplication governs the request processing and ensures that special components-HTTP modules-can register handlers for some of these application-level events. Once authenticated and authorized (and if not resolved through the output cache), the request is assigned its own handler component.

Figure 2 Requests
Figure 2 Requests

Each ASP.NET request needs a special component-the HTTP handler-to be processed. A reference to the HTTP handler in charge of the current request is available when the application lifecycle fires the PostMapRequestHandler event.

Application-level events can be wired up in two ways. You can write handlers in the global.asax file (one of the app's source files) or you can write a custom HTTP module that registers a listener for the specified event. Functionally, the two approaches are comparable. However, if you use HTTP modules, you can reach your goal by simply adding a new component and editing the web.config file in a less intrusive way. In both cases the app is restarted.

The first application event to fire after the page handler has been determined is PostMapRequestHandler. As shown in Figure 3, an HTTP module can register a listener for this event and be notified when the handler is found for each ASP.NET request. If you need to do this for only a few pages, you might want to filter out undesired pages in the module code.

Figure 3 HTTP Module to Listen for the Page Handler (Close Figure 3)
Imports System
Imports System.Web
Imports System.Web.UI
Imports System.Web.Compilation
Imports System.Web.Hosting
Imports System.IO
Imports System.Text

Namespace Samples

    Public Class SubclassModule : Implements IHttpModule

        Private _app As HttpApplication

#Region “IHttpModule”
        Public Sub Dispose() Implements IHttpModule.Dispose
        End Sub

        Public Sub Init(ByVal context As HttpApplication) _
                Implements IHttpModule.Init
            If context Is Nothing Then
                Throw New ArgumentNullException(“[context] argument”)
            End If

            ‘ Cache the HTTP application reference
            _app = context

            ‘ Map the app event to hook up the page
            AddHandler context.PostMapRequestHandler, _
                AddressOf OnPostMapRequestHandler
        End Sub
#End Region
#Region “App Event Handlers”
        Private Sub OnPostMapRequestHandler( _
                ByVal source As Object, ByVal e As EventArgs)
            ‘ Get the just mapped HTTP handler and cast to Page
            Dim pageHandler As IHttpHandler = Nothing
            If TypeOf _app.Context.Handler Is System.Web.UI.Page Then
                pageHandler = _app.Context.Handler
            End If

            ‘ If OK, register an event handler for PreRender
            If Not pageHandler Is Nothing Then
                HookUpPage(pageHandler)
            End If
        End Sub
#End Region

#Region “Helpers”
        Private Sub HookUpPage(ByVal pageHandler As Page)
            ‘ Wire up as many application events as needed
            AddHandler pageHandler.Load, AddressOf OnLoad
            ‘ ...
        End Sub

        Private Sub OnLoad(ByVal sender As Object, ByVal e As EventArgs)
            Dim pageHandler As Page = DirectCast(sender, Page)
            ‘ TODO: add your code here
        End Sub
#End Region

    End Class
End Namespace

An HTTP module is a class that implements two methods: Init and Dispose. The Init method is invoked once, when the application starts up. As in Figure 3, the Init method registers listeners called for each ASP.NET request. Typically, you check the URL of the incoming request within the OnPostMapRequestHandler method. Here's an example:

Sub OnPostMapRequestHandler( _
       ByVal source As Object, ByVal e As EventArgs)
    Dim url as Uri
    url = _app.Context.Request.Url
    If Not url.AbsolutePath.Equals( _
           "/WebSite1/default.aspx") Then
       Return
    End If
    ...
End Sub

An ASP.NET request makes its way through the runtime pipeline together with a system object that represents the context of the ongoing request. This object is an instance of the HttpContext class. Handler, which is one of the properties on the HttpContext object, contains a reference to the HTTP handler object that will serve the request. Unless a custom HTTP handler has been defined for the requested resource, an ASPX request is managed by an instance of a class that inherits from the System.Web.UI.Page class. At this point, you hold a reference to the object that is used to process the page and you can dynamically add new handlers for common page events, such as PreRender and Load. Here's an example:

Private Sub HookUpPage(ByVal pageHandler As Page)
   ' Wire up as many application events as needed
   AddHandler pageHandler.Load, AddressOf OnLoad
    ...
End Sub

More precisely, the object bound to the Handler property is an instance of the dynamically created Page class modeled after the markup you wrote in the ASPX server resource. In other words, the page handler contains an event handler for the page's Load event if you placed a Page_Load method in the codebehind class.

The AddHandler keyword in Visual Basic® (and the += operator in C#) adds a new handler to the chain of delegates bound to the given event. As a result, any code you add to the Handler object executes after any equivalent handlers defined in the ASPX of the page. The code in the OnLoad method referenced above runs after the code in Page_Load.

At this point, what can you do in OnLoad? Just about anything, as long as you know the structure of the page and the elements in its control tree. If you can read the source code of the page, you can easily grab this information. Otherwise, you'll need a dump of the page's control tree. One way to get this is to turn on page tracing and redirect the output to the ASP.NET built-in trace.axd tool. You can turn on tracing via the web.config file, as shown here:

<system.web>
    <trace enabled="true" />
</system.web>

The next step is to invoke the trace.axd utility on the Web site you're working on and take a look at the control tree. Figure 4 shows the utility in action.

Figure 4 Trace.axd in Action
Figure 4 Trace.axd in Action(Click the image for a smaller view)
Figure 4 Trace.axd in Action
Figure 4 Trace.axd in Action (Click the image for a larger view)

As an example, I created a new Web site using the Personal Starter Kit (see Figure 5). I'll show you how to modify some of its pages without touching the source code. In particular, I'll show you how to automatically move the input focus to the user-name field of the login box.

Figure 5 Default Version of Personal Web Site
Figure 5 Default Version of Personal Web Site(Click the image for a smaller view)
Figure 5 Default Version of Personal Web Site
Figure 5 Default Version of Personal Web Site (Click the image for a larger view)

Adding New Event Handlers

If you could edit the source code, you would probably add a line or two to the Page_Load handler and call the method Focus on the reference of the Login control. From the control tree (partially shown in Figure 4), you can figure out that the page contains a Login control named Login1. The following code retrieves a child control in the page tree, moves the input focus to its area, and also resets the title of the page:

Private Sub OnLoad(ByVal sender As Object, ByVal e As EventArgs)
   Dim pageHandler As Page = DirectCast(sender, Page)

   ' Move the input focus to control Login1
   Dim ctl As Control = FindControlRecursive(pageHandler, "Login1")
   ctl.Focus()

   ' Change title of the page
   pageHandler.Title = "This page has been hooked up"
End Sub

The FindControl method on each ASP.NET Control class only lets you peer into the direct children of the control. In no way does it look into the subtree rooted in the control. For this functionality, you'd have to write your own recursive find method, as I've done in my FindControlRecursive method.

Is there a way to avoid having to walk down the control tree for every request of a given page? Not really. You can't cache the control instance because the page tree is rebuilt for each request. And without touching the source code, you can't add extra properties to the codebehind page to expose a given control through a property. Of course, it would be a lot easier if you could make copies of existing source files and replace classes or precompiled assemblies.

Figure 6 shows an altered version of the sample site. The title of the page has been changed via an HTTP module, the input focus has been moved to the login control, and the theme of the site has been changed via external code. To change the theme, you have to wire up the PreInit event:

Private Sub HookUpPage(ByVal pageHandler As Page)
    AddHandler pageHandler.PreInit, AddressOf OnPreInit
    AddHandler pageHandler.Load, AddressOf OnLoad
End Sub

Private Sub OnPreInit(ByVal sender As Object, _
    ByVal e As EventArgs)
    Dim pageHandler As Page = DirectCast(sender, Page)
    pageHandler.Theme = "Black"
End Sub

Figure 6 Altered Personal Web Site
Figure 6 Altered Personal Web Site(Click the image for a smaller view)
Figure 6 Altered Personal Web Site
Figure 6 Altered Personal Web Site (Click the image for a larger view)

Page manipulations require getting a reference to the page's control tree. The Controls collection of the page handler is the outermost collection of controls in the page and is where your navigation in the control's tree begins.

Getting a Reference to the Control Tree

Say your client needs a quick way to add new content, such as late-breaking news, to the site. Holding a reference to the page handler is the key. As a first step, you can add a new handler for the PreRender event in the HookUpPage method of the sample HTTP module:

AddHandler pageHandler.PreRender, AddressOf OnPreRender

In the handler, you first get the page reference and then find the desired point of injection, as in Figure 7. Imagine that you want to provide notice that the Web site is driven by an external component and to do this, you want to add a new label at the very top of each page managed by the HTTP module. The following code snippet sets up a Label control and then adds it to the Controls collection of the page:

Sub AddStaticText(ByVal pageHandler As Page)
    Dim msg As String = ConfigurationManager.AppSettings("StaticMessage")
    Dim lbl As New Label
    lbl.ID = "SubclassModule_Label1"
    lbl.BackColor = Color.White
    lbl.ForeColor = Color.Red
    lbl.Text = String.Format("<div>{0}</div>", msg)
    pageHandler.Controls.AddAt(0, lbl)
End Sub

Figure 7 HTTP Module to Intercept the Page Handler (Close Figure 7)
Private Sub OnLoad(ByVal sender As Object, ByVal e As EventArgs)
    Dim pageHandler As Page = DirectCast(sender, Page)
    If pageHandler Is Nothing Then Return

    ‘ Move the input focus to control Login1
    Dim controlName As String = “Login1”
    Dim ctl As Control = FindControlRecursive(pageHandler, controlName)
    If Not ctl Is Nothing Then ctl.Focus()

    ‘ Change the page title
    pageHandler.Title = “This page has been hooked up”

    ‘ Add a new control tree with postback support
    AddLinkButton(pageHandler)
End Sub

Private Sub OnPreRender(ByVal sender As Object, ByVal e As EventArgs)
    Dim pageHandler As Page = DirectCast(sender, Page)
    If pageHandler Is Nothing Then Return

    ‘ Add static text (no postback)
    AddStaticText(pageHandler)
End Sub

Private Sub AddStaticText(ByVal pageHandler As Page)
    Dim msg As String = ConfigurationManager.AppSettings(“StaticMessage”)
    Dim lbl As New Label
    lbl.ID = “SubclassModule_Label1”
    lbl.BackColor = Color.White
    lbl.ForeColor = Color.Red
    lbl.Text = String.Format( _
        “<div style=’margin:0;width:100%;font-size:20pt’;>{0}</div>”, msg)
    pageHandler.Controls.AddAt(0, lbl)
End Sub

Private Sub AddLinkButton(ByVal pageHandler As Page)
    Dim controlName As String = “Main”
    Dim ctl As Control = FindControlRecursive(pageHandler, controlName)
    If ctl Is Nothing Then Return

    Dim msg As String = ConfigurationManager.AppSettings(“LinkMessage”)
    Dim link As New LinkButton
    link.ID = “SubclassModule_LinkButton1”
    link.ToolTip = “Click to hide the ‘Create account’ button”
    link.BackColor = Color.Blue
    link.ForeColor = Color.Yellow
    link.Text = String.Format( _
        “<hr><div style=’width:100%;font-size:30pt’;>{0}</div><hr>”, msg)
    AddHandler link.Click, AddressOf SubclassModule_LinkButton1_Click
    ctl.Controls.AddAt(0, link)
End Sub

Private Sub SubclassModule_LinkButton1_Click( _
        ByVal sender As Object, ByVal e As EventArgs)
    Dim pageHandler As Page = DirectCast(_app.Context.Handler, Page)
    If pageHandler Is Nothing Then Return

    Dim controlName As String = “Image1”
    Dim ctl As WebControl = DirectCast( _
        FindControlRecursive(pageHandler, controlName), WebControl)
    If ctl Is Nothing Then Return

    ctl.Visible = False
End Sub

The Controls property is an instance of the ControlCollection class and features two methods for adding new control instances: Add and AddAt. Add appends new controls to the collection; AddAt is more flexible, allowing you to specify a 0-based index for the desired position. The label text can contain any HTML markup and can be read (all or in part) from any external resources, including databases, XML documents, and the web.config file.

A label or literal control is good at showing dynamic contents but not if you want to generate and handle postback events. Instead, let's add a link button. If the link button is intended to take the user to a new page, then there's no significant difference with the scenario I just considered. But if the link button has to originate a postback, run some code on the server, and then update the page, you need to reconsider the solution.

As obvious as it may seem, a control added during the pre-rendering stage is not yet part of the control tree when the postback event is processed. The ASP.NET page runtime figures out the target of the postback and raises the event before the pre-rendering stage. The sender of the postback is written in the HTTP request.

The ASP.NET runtime needs to find a match between the sender name and an existing control in order to fire the postback event. For this reason, the link button must be created earlier than the PreRender event. A good time for this to occur is the Load event. Imagine the injected control tree contains some input fields, such as check or textboxes. The state of these controls must be updated with data posted from the client. For the update to take place automatically, these controls must be created no later than the Load event. In this case, the ASP.NET runtime guarantees that the viewstate is restored correctly and any relevant posted data is applied.

In the Load event, you add a new LinkButton control and add a handler for its Click event:

AddHandler link.Click, AddressOf SubclassModule_LinkButton1_Click

When the user clicks on the link, the page will post back and ASP.NET will have all the information it needs to resolve the target of the postback correctly. As a result, the specified event handler is invoked.

So what can you do from within the handler? Most everything or, more precisely, everything you know how to do. But without source markup and the codebehind class environment, simple tasks can prove challenging. Suppose you want to change the status of another control in the page. For instance, say you want to let the user click on the newly added link button and disable another button in the page. Admittedly, this functionality doesn't make a lot of sense when put in these terms. However, taken individually, these represent the tasks you might want to accomplish on hooked and subclassed ASP.NET pages.

The question, then, is how would you retrieve the page object from within the handler? The page object is required for gaining access to the overall control tree and locating the control to work with. A reference to the current HTTP handler-that is, the Page object-is stored in the Handler property of the HttpContext object. You don't normally resort to this sort of trick to get the page reference in the codebehind, but from an HTTP module, this trick is essential:

Sub LinkButton1_Click(sender As Object, e As EventArgs)
    Dim pageHandler As Page = DirectCast(_app.Context.Handler, Page)
    If pageHandler Is Nothing Then Return
    ...
End Sub

You use the page object to start another recursive search to locate the control of choice. For example, say you want to disable the "create account" button shown in Figure 5. A quick analysis of the trace output and the HTML source of the page reveals that there's no real button behind this. The Personal Web site uses a number of real-world techniques to build sites. It deeply leverages the layout capabilities of cascading style sheets and uses a lot of images to provide a more appealing UI. Thus, the create account button is expressed as follows:

<a href="http://register.aspx">
  <asp:image id="Image1" runat="server" />
</a>

In the Controls collection, you'll find a reference to the Image control, but nothing specifically for the <a> tag as it lacks the runat="server" attribute. This is an inherent limitation to the subclassing techniques I'm using here. Subclassing is not like direct programming and should only be used when necessary. Subclassing an unaware page doesn't let you reproduce everything you can do normally in ASP.NET. In ASP.NET, any sequence of markup text not decorated with the runat attribute is treated like plain text and emitted verbatim. Furthermore, contiguous text is grouped in a single LiteralControl, making it even harder for subclassers to find any sort of reference to a particular piece of markup and its server counterpart. One thing you can try is to figure out the ID of the HTML tag and inject some script code that does the work upon loading of the page. Unfortunately, in this case, the <a> tag lacks even the ID attribute, making it virtually impossible to program against on both the server and the client. Disabling the image is possible, but this doesn't stop clicking. A possible alternative is to hide the control:

Dim controlName As String = "Image1"
Dim ctl As WebControl = DirectCast( _
    FindControlRecursive(pageHandler, controlName), WebControl)
If ctl Is Nothing Then Return
ctl.Visible = False

Another technique to manipulate the output of an existing page is to capture the response before it is sent to the browser. From an HTTP module, you hook up the PostRequestHandlerExecute event and assign a custom stream to the Filter property of the HttpResponse object. After this, any text being written to the response stream will pass through your own stream giving you a chance to preview the HTML source and make any required changes, such as modifying DOM and script. A real-world scenario where this technique is useful is changing the URL of a hyperlink inserted as a simple <a> tag in the source page.

Page Override with HTTP Handlers

Subclassing is a good technique for entering limited changes to an existing page without touching the source code. If you can replace a given page with a new page (one with the same name but different content) then, by all means, that is the technique you should use. Subclassing is different from replacing and should be used when you have no simpler alternatives.

To completely change the output of a page without touching the source code, you can redirect the page to a different HTTP handler. The change occurs in the web.config file, like so:

<httpHandlers>
   <add verb="*"
        path="register.aspx"
        type="Samples.Modules.RegisterAspxHandler, Subclass"
        validate="false" />
</httpHandlers>

Any attempt to access register.aspx will be handled by the specified HTTP handler rather than the original page. Needless to say, the HTTP handler is responsible for any output that is shown to the end user. An HTTP handler is a class that implements the IHttpHandler interface. All of the final output is generated inside the ProcessRequest method:

Public Sub ProcessRequest(ByVal context As HttpContext) _
       Implements IHttpHandler.ProcessRequest
   context.Response.ContentType = "text/plain"
   context.Response.Write("Hello World")
End Sub

With such a handler registered in the web.config file, the output of the register.aspx page is a simple "Hello World" message.

An HTTP handler represents the lowest-level tool to serve any ASP.NET requests. Once set up, the HTTP handler code is the only endpoint that executes for the request. For page requests, therefore, a custom HTTP handler means no viewstate, no postback, and no lifecycle events. If the request has to go through the regular lifecycle, you should consider the SetRenderMethodDelegate method on the Page class. The method designates a callback that is invoked to render the page contents to the browser. The method is actually defined on the Control class and is available for server controls to render their content into parent controls. Here's an example:

Private Sub OnPreRenderComplete(object sender, EventArgs e)
   Dim pageHandler As Page = DirectCast(sender, Page)
   If Not pageHandler Is Nothing Then
       Dim method As New RenderMethod(RenderPageContents)
       pageHandler.SetRenderMethodDelegate(method);
   End If
End Sub

Private Sub RenderPageContents( _
        ByVal output As HtmlTextWriter, _
        ByVal container As Control)
   ...
End Sub

The method has been designed for very specific uses by the framework and is not intended for public use. Yet, it is an option to consider if all you need to do is override the page rendering without affecting the page lifecycle. Keep in mind, however, that this technique could cause trouble if the render method delegate was previously set to something else.

URL Rewriting

URL rewriting is a good technique for programmatically redirecting a request to a different URL. Basically, URL rewriting consists of changing the URL of the request early in the application request lifecycle-for example, in BeginRequest. To rewrite the URL, you use the RewritePath method of the HttpContext object:

HttpContext context = HttpContext.Current;
context.RewritePath("newpage.aspx");

The effect is the same as with a classic HTTP 302 redirect, except no new physical request is made by the browser. By embedding the preceding code snippet in an HTTP module (or by simply editing the global.asax file), you can check the target of the current request and decide whether to redirect it to another page and different content.

ASP.NET 2.0 supports the <urlMappings> section for a purely declarative and unconditional URL rewriting.

Summary

Like any other type of application, a Web site is made up of source code-whether codebehind compiled code, markup, or script. The source code base is responsible for the user interface and the behavior of the site. If, at some point, you need to change the user interface and behavior, the easiest approach is editing the source code. However, if this is not possible, you can try the techniques described in this article to achieve your goals.

If you compare Figure 5 and Figure 6, you'll see what an external binary component added to the site can do. All the changes you observe have been obtained by simply adding a component-an HTTP module-and completely ignoring the source files. Keep in mind that making runtime changes introduces a per-request overhead, so you should aim to make the site as flexible and configurable as possible and try to anticipate potential changes.

Send your questions and comments for Dino to cutting@microsoft.com.

Dino Esposito is a mentor at Solid Quality Learning and the author of Programming Microsoft ASP.NET 2.0 (Microsoft Press, 2005). Based in Italy, Dino is a frequent speaker at industry events worldwide. Get in touch with Dino at cutting@microsoft.com and join the blog at weblogs.asp.net/despos.


Security Briefs: Improve Manageability through Event Logging

Security Briefs
Improve Manageability through Event Logging
Keith Brown

Contents
The IT Pro Is a User Too
A Closer Look at the Event Log
Using the Event Log from the .NET Framework
Working with Message Files
Creating a Custom Log
Logging Events at Run Time
Conclusion

T
he three traditional pillars of computing security are confidentiality, integrity, and availability. Often when I speak of availability, I talk about obvious threats such as denial of service attacks, but there's more to it than that. An application that requires high availability must be resilient to abnormal situations. If something goes wrong, an administrator must be able to sort out a solution quickly. Applications designed with management in mind tend to be more highly available, partly because administrators are constantly made aware of the application's health. When something does go wrong, a manageable application will tell the administrator how to fix the problem instead of simply dumping out a stack trace that requires a developer to analyze.

You may have heard about the Dynamic Systems Initiative (DSI) led by Microsoft (microsoft.com/dsi). This long-term vision would have us model applications using tools and languages such as the System Definition Model (SDM) and the Service Modeling Language (SML) to build more easily deployable and manageable applications. The Visual Studio® Team System Distributed System Designer, formerly code-named "Whitehorse" (msdn2.microsoft.com/ms181778.aspx), is a great example of where this initiative is going, but this is just an early glimpse. Developers need practical guidance they can use today to build manageable systems.

The IT Pro Is a User Too

I recently spent some time speaking with David Aiken, a Microsoft developer/evangelist whose passion is building manageable software. He said something that really hit home: "Developers should think of the IT professional as a user of their software." He pointed out how much painstaking effort we undertake when designing the layout of a Windows form or Web page, and posed the question, "Why we don't make the same effort when we're designing an error message to be posted to the event log?"

David has a good point. Not only is the IT professional a user of your software, she's the one who will be operating and managing it for its entire lifetime. Making her job easier is not only important from an availability standpoint, but having the system administrator as an ally will very likely help you sell the next version of your product. Applications that are designed for operations are few and far between today, and this is one area where you can seek a competitive edge.

Over the past several years, I've seen tremendous advances from Microsoft in terms of building more secure software, including tools and blueprints for building threat models. Now, with a tighter leash on security, the focus seems to be shifting towards manageability. With any luck, by the time this article is printed, you'll be hearing about health modeling tools from Microsoft.

In the meantime, I'm going to address some of the lowest-hanging fruit on the management tree: event logging. I'll focus on how you can do a better job of using the event log from the Microsoft® .NET Framework on Windows Server® 2003.

A Closer Look at the Event Log

The Windows® event log was designed with localization and efficiency in mind. For example, a well-designed event found in the Application log should contain just enough prose to explain a problem to the administrator and provide a suggested course of action. The vast majority of this prose should be localized for the reader, who may very well be on a remote machine in a different country. To support reading the log in your own native language, the localizable prose in the message is not actually stored in the log; rather, it is merged with the log locally as you read it. The actual prose is stored in a localized resource DLL on the reader's machine. This also has the effect of keeping the log files smaller and reducing the amount of network traffic required to read them.

Before the .NET Framework came along, this was all very clear to developers because you had to create and register these resource DLLs yourself. A typical message in a resource DLL might look like this: "Failed to connect to database %1 using connection string %2. Check to see that the database is available."

In this example, the only data about the event that would be stored in the event log would be the event ID, source, category, and a few other bits of metadata, along with the arguments supplied when the event was logged: the database name (%1) and the connection string (%2). When you use a tool such as Event Viewer to read an event like this, Windows will load the message resource DLL installed on your machine, look up the localized message based on the event ID, then merge the arguments from the log in order to display the message to you.

The way Windows figures out which resource DLL to use is by looking at the name of the event source in the log and mapping it to a registry key. This registry key contains the path to the resource DLL that should be used to display the message. Figure 1 shows an example of an event source registry key. In the early days, a Win32® programmer would create this key manually, but when the .NET Framework came along, it included an EventLog class to hide details like this.

Figure 1 Event Source Registration
Figure 1 Event Source Registration(Click the image for a smaller view)
Figure 1 Event Source Registration
Figure 1 Event Source Registration (Click the image for a larger view)

Using the Event Log from the .NET Framework

In version 1.x of the .NET Framework, the EventLog class provided very rudimentary access to the event log. The strategy seemed to be to simplify access to the event log to perhaps convince people to actually use it. At install time, when your code is running with high privilege, you can create an event source with a single line of code:

EventLog.CreateEventSource("MyApp",
    "Application")

Then at run time, when your code is running with lower privilege, you can log events without ever having to create a resource DLL:

EventLog.WriteEntry("MyApp", "Testing 123")

It's important to note that there are two steps involved here. Remember that when you create an event source, you're actually creating a registry key. This is a privileged operation, and if you neglect this step, EventLog.WriteEntry will attempt to do it for you. While this may work when you're running a demo as an administrator, it's not going to work in production code when you're running under a normal user account. So be sure to create your event sources during installation when you have permissions to do so!

Normally, if you were to write a message to the event log without having a corresponding resource DLL, you'd get a rather nasty note in the event log, complaining that no description for the event could be found (see Figure 2). The Framework has a unique way of getting around this. The simple call to CreateEventSource wires your event source up with a default message file. This file is called EventLogMessages.dll, and it contains 65,536 event descriptions, each of which consists of the string "%1", a placeholder for whatever string you want to write. While this may have encouraged a few more people to use the event log, it defeated the purpose of having message files in the first place: to allow localized viewing of messages in the reader's language, and to keep the size of the event logs in check.

Figure 2 An Event with No Message File
Figure 2 An Event with No Message File(Click the image for a smaller view)
Figure 2 An Event with No Message File
Figure 2 An Event with No Message File (Click the image for a larger view)

Fortunately, version 2.0 of the .NET Framework remedies this and allows you to gain the full power of the event log by registering a message file when you create your event source. This lets you support localized categories and messages, and it reduces the size of your messages in the log.

Here's the modern way to register an event source using the .NET Framework 2.0:

EventSourceCreationData source =
    new EventSourceCreationData("MyApp", "Application");
source.CategoryCount = 2;
source.CategoryResourceFile  = "path to dll";
source.MessageResourceFile   = "path to dll";
source.ParameterResourceFile = "path to dll";

EventLog.CreateEventSource(source);

Note that while technically you are allowed to have three resource DLLs, most applications roll all three resource types-categories, messages, and parameters-into a single DLL.

Working with Message Files

In order to create a resource DLL for the event log, you'll need to start by creating one or more message files. I'll keep things simple by using a single message file, which is a text file that traditionally carries a .mc file extension. You use a tool called mc.exe, the message compiler, to process these files.

The message compiler keeps track of several variables that you can set at any time in your message file. MessageId is one variable that you'll want to update for each message, while you might set the Severity variable once for an entire section of your message file. If you don't explicitly set a MessageId, the assigned MessageId will be one greater than the previous one. Variable definitions are separated by white space (including line break):

    MessageId=1001 SymbolicName=AppStarted
    Severity=Informational

Message prose itself must be started on a new line and can cover as many lines as you need. It is terminated by a new line with a period, as shown below:

MessageId=1003 SymbolicName=DatabaseConnectionFailed
Language=English
Could not connect to SQL Server database %1. Reason: %2
.
Language=Spanish
No podía conectar con SQL Server database %1. Razón: %2
.

Here I've defined a single message with two localized bits of prose. Before I can compile this message file, I need to define the languages I'm using. So at the top of the message file, I've added a header section with language definitions:

LanguageNames=(English=0x409:MSG00409)
LanguageNames=(Spanish=0xC0A:MSG00C0A)

This defines the symbolic name I'll be using to represent each language along with its locale ID. Since the message compiler produces a separate output file for each language, I've supplied the name it should use for each file. Here's what I get when I run the message compiler from the command line:

$ mc messages.mc
MC: Compiling messages.mc
$ dir
12/16/2006  05:18 AM             1,970 messages.h
12/16/2006  05:13 AM             1,199 Messages.mc
12/16/2006  05:18 AM                74 messages.rc
12/16/2006  05:18 AM               356 MSG00409.bin
12/16/2006  05:18 AM               352 MSG00C0A.bin

As you can see, the message compiler produces a .bin file for each language along with a resource file (messages.rc) that refers to the .bin files. The messages.rc file looks like this:

LANGUAGE 0xa,0x3
1 11 MSG00C0A.bin
LANGUAGE 0x9,0x1
1 11 MSG00409.bin

You can now build a resource DLL with two more steps: running the resource compiler to create a .res file, then using the linker to link that file into a DLL:

$ rc messages.rc
$ link /DLL /SUBSYSTEM:WINDOWS /NOENTRY /MACHINE:x86 messages.res
Microsoft (R) Incremental Linker Version 8.00.50727.42
Copyright (C) Microsoft Corporation.  All rights reserved.

At this point you should have a file called messages.dll that you can use when you create your event source.

There's a lot of nitty-gritty detail you can learn about message files by looking up mc.exe in the MSDN documentation, but there are a few items of particular interest that I'll point out.

First, you should include the localized names of your event categories in your message file. These look like normal messages, except that they are short names and must use consecutive MessageIds that start with the number 1. When you create your event source, you should specify the CategoryCount as the highest category message number. Here are the two category definitions in my sample application (you will likely have more interesting and varied categories for your own application):

;// Categories
MessageId=1 SymbolicName=Administrative
Language=English
Administrative
.
Language=Spanish
Administrativo
.

MessageId=2 SymbolicName=Operational
Language=English
Operational
.
Language=Spanish
Operacional
.

Keep in mind that the message compiler was originally written for use by C programmers writing native Win32 applications. Did you notice the messages.h file that was produced? This file contains a set of constants that C/C++ programmers can use to refer to categories and messages. This is the reason for the SymbolicName variable you see in all of my message and category definitions. You may also have noticed the strange-looking comment above my category list:

;// Categories

Technically, the message compiler only needs the semicolon to delimit a comment, but traditionally you'll see a C style comment following that. The reason for this is because the compiler emits all of these comments into the messages.h file as well.

So what good is the SymbolicName variable to a programmer who uses .NET if it is only used to create a C header file? Frankly, it's not much use as things stand today, but imagine writing a parser that would take a message file as input and produce a class definition in your managed language of choice.

There are many other variables that are typically defined in the header of the message file, but I'm relying on the default definitions. Here are the default message severities that you'll get, and you can use these values in your message definitions:

SeverityNames=(Success=0
               Informational=1
               Warning=2
               Error=3
              )

One other thing about message files that may be confusing at first is the concept of parameters. There are two types of parameters. The most common type are those that you fill with data collected at run time: %1, %2, and so on. The other type is simply a way of factoring out common strings from your message files. For example, early in development, you might not want to hardcode the name of your application in lots of individual messages, so you can factor that name out as a parameter, then use a special syntax, %%N where N is the MessageId of the replacement string from the parameter resource DLL.

Creating a Custom Log

If there's one thing an IT pro really hates, it's an application that makes a lot of noise in the application event log, drowning out the really important messages that well-behaved applications produce from time to time. You shouldn't be writing operational, analytical, or trace/debug messages to the application log! These are more appropriate for a custom application log, and you can easily create one of these in your application's installer by specifying the name of the custom log you wish to create (the code below creates a new log called MyApp) if it doesn't already exist. Note that you'll need a distinct event source name for each log file you create:

EventSourceCreationData source =
    new EventSourceCreationData("MyApp Operational", "MyApp");
source.CategoryCount = 2;
source.CategoryResourceFile  = "path to dll";
source.MessageResourceFile   = "path to dll";
source.ParameterResourceFile = "path to dll";

EventLog.CreateEventSource(source);

Logging Events at Run Time

Just as you may be accustomed to creating a data layer for your application, complete with strongly typed arguments that simplify database access, you should consider creating classes to help you log events. In my sample application, I've created two classes for event logging. The first is for administrative events that will be written to the Application log, and the second is for operational events that will be written to my application's custom log. These classes each expose a method per event, with strongly typed parameters representing the data that each event needs for its parameters. For example, here is the code for an event that records a failure to connect to a database in my custom operational log:

public void DatabaseConnectionFailed(
    string dbName, string connectionString) {
    operationalLog.WriteEvent(
        new EventInstance(1003, 2, EventLogEntryType.Error),
        dbName, connectionString, new StackTrace(1));
}

The WriteEvent method is new for the .NET Framework 2.0. This method allows you to pass arguments that will fill in the parameters in your message file such as %1. Figure 3 shows an example of this event in the log.

Note that this particular method provides a lot of detail, including the database name, the connection string, and a stack trace. That is why this message is destined for my custom operational log. I also send a simpler, more actionable message to the Application log.

Figure 3 A Parameterized Event
Figure 3 A Parameterized Event(Click the image for a smaller view)
Figure 3 A Parameterized Event
Figure 3 A Parameterized Event (Click the image for a larger view)

Conclusion

Applications that are designed for operations are carefully instrumented using technologies such as the event log. By treating the IT pro as a first-class user of your application, you're helping to ensure smooth operation and less downtime. By creating your own custom event log and sequestering your noisier events there, you make room in the Application log for the actionable events that quickly explain to the IT pro how to restore your application to health.

A lot about the existing eventing system, including the .mc message file format, is a bit arcane. Fortunately, there is a completely redesigned eventing system built into Windows Vista™ that will also be part of the upcoming Windows Server code-named "Longhorn." While many of the basic concepts stay the same, there are a lot of cool new features that will allow you to log richer events in a more granular way. Plus, the new files that replace message files contain significantly more metadata for events and are stored as XML, which will make parsing much easier. I'll be covering the new system in an upcoming column, so stay tuned!

Send your questions or comments for Keith to briefs@microsoft.com.

Keith Brown is a co-founder of Pluralsight, a premier Microsoft .NET training provider. Keith is the author of Pluralsight's Applied .NET Security course, as well as several books, including The .NET Developer's Guide to Windows Security, which is available both in print and on the web. Learn more at www.pluralsight.com/keith.


Service Station: WCF Messaging Fundamentals

Service Station
WCF Messaging Fundamentals
Aaron Skonnard

Code download available at: ServiceStation2007_04.exe (161KB)
Contents
XML Representations
The Message Class
Message Versions
Reading and Writing Messages
Typed Message Bodies
Message Lifetime
Message Headers and Properties
Mapping Messages to Methods
Endpoints and Bindings
Bringing It All Together

W
hen you start pulling the layers away from Windows® Communication Foundation, you find a sophisticated XML-based messaging framework that offers a world of possibilities when connecting systems using a variety of protocols and formats. In this month's column, I am highlighting some of the key messaging features we can thank for such flexibility.

This column assumes a basic understanding of the Windows Communication Foundation programming model. If you're new to it, before proceeding you should read my article in the February 2006 issue of MSDN®Magazine.

One of the primary goals of the Windows Communication Foundation messaging architecture is to provide a unified programming model while also allowing flexibility in how data is represented and messages are transmitted. It accomplishes this by building on XML as the data model plus SOAP and WS-Addressing as the messaging framework. However, the fact that Windows Communication Foundation builds on these models doesn't mean you have to use XML 1.0, SOAP, or WS-Addressing when transmitting messages. You'll see that Windows Communication Foundation provides a great deal of flexibility.

XML Representations

Since the early days of XML, the software industry has relied on a little-known specification that provides a standard definition for the data found in an XML document. This specification, referred to as the XML Information Set (InfoSet), defines elements and attributes in terms of the information they contain, in a way that's completely independent of the byte representation. (See www.w3.org/TR/xml-infoset for details.)

The InfoSet specification has made it possible for other XML specifications and APIs to offer a consistent view of the data found in an XML document even though they might represent that data completely differently. Ultimately, the InfoSet provides a common rendezvous point for applications working with XML data, as illustrated in Figure 1. In the end, it's the job of an XML processor to translate between the byte representation and the programming model experience.

Figure 1 The Role of the XML InfoSet
Figure 1 The Role of the XML InfoSet(Click the image for a smaller view)
Figure 1 The Role of the XML InfoSet
Figure 1 The Role of the XML InfoSet (Click the image for a larger view)

Windows Communication Foundation introduces some fundamental enhancements to the System.Xml namespace that make it possible to leverage alternate byte representations-rather than simply literal XML 1.0-when reading and writing XML documents. The main classes of interest here, System.Xml.XmlDictionaryReader and System.Xml.XmlDictionaryWriter, are located in the new System.Runtime.Serialization assembly that ships with the Microsoft® .NET Framework 3.0.

Both the XmlDictionaryReader and XmlDictionaryWriter classes provide static factory methods for creating readers and writers that use text, binary, and MTOM (Message Transmission Optimization Mechanism) representations. For instance, XmlDictionaryReader provides the CreateTextReader, CreateBinaryReader, and CreateMtomReader methods while XmlDictionaryWriter provides the corresponding CreateTextWriter, CreateBinaryWriter, and CreateMtomWriter methods.

Let's look at some examples. Consider this literal XML 1.0 representation of a customer:

<!-- customer.xml -->
<Customer xmlns="http://example.org/customer">
   <Email>bob@contoso.com</Email>
   <Name>Bob</Name>
</Customer>

The following code shows how to read customer.xml into an XmlDocument object and how you can use XmlDictionaryWriter to persist the same XmlDocument object back out to a binary representation on disk:

// read from XML 1.0 text representation
XmlDocument doc = new XmlDocument();
doc.Load("customer.xml");

// write to binary representation
FileStream custBinStream = new FileStream(
    "customer.bin", FileMode.Create);
using (XmlWriter xw = XmlDictionaryWriter.CreateBinaryWriter(
    custBinStream))
{
    doc.WriteContentTo(xw);
}

After this code runs, you'll have a binary representation saved in customer.bin that looks like Figure 2. Even though the binary editor shown in Figure 2 happens to display a sequence of characters, this is actually a binary file and obviously does not look like an XML 1.0 file. Nevertheless, it does represent an XML document (an InfoSet). Still, this binary format is proprietary to version 3.0 of System.Xml and is not compatible with other .NET versions or Web service frameworks.

Figure 2 Binary Representation of Customer XML Document
Figure 2 Binary Representation of Customer XML Document(Click the image for a smaller view)
Figure 2 Binary Representation of Customer XML Document
Figure 2 Binary Representation of Customer XML Document (Click the image for a larger view)

The code sample in Figure 3 demonstrates how you can read the binary representation back into an XmlDocument object and how you can persist it back out to an MTOM representation on disk.The MTOM representation looks very different than the text or binary representations, as you can see in Figure 4. Nevertheless, it represents the same XML document and can be processed using the same System.Xml APIs. The MTOM representation also differs from the binary representation in that MTOM is a W3C Recommendation based on the InfoSet and is widely supported across Web service frameworks. MTOM makes it possible to optimize the transmission of binary elements within XML documents by leveraging multi-part MIME framing.

Figure 3 Read from Binary and Write to MTOM Representation (Close Figure 3)
// read from binary representation
XmlDocument doc = new XmlDocument();
FileStream custBinStream = new FileStream(
    "customer.bin", FileMode.Open);
using (XmlReader xr = XmlDictionaryReader.CreateBinaryReader(
    custBinStream, XmlDictionaryReaderQuotas.Max))
{
    doc.Load(xr);
}

// write to MTOM representation
FileStream custMtomStream = new FileStream(
    "customer.mtom", FileMode.Create);
using (XmlWriter xw = XmlDictionaryWriter.CreateMtomWriter(
    custMtomStream, Encoding.UTF8, 1024, "text/xml"))
{
    doc.WriteTo(xw);
}
Figure 4 MTOM Representation of Customer XML Document
Figure 4 MTOM Representation of Customer XML Document(Click the image for a smaller view)
Figure 4 MTOM Representation of Customer XML Document
Figure 4 MTOM Representation of Customer XML Document (Click the image for a larger view)

We can now come full circle by reading the MTOM representation back into an XmlDocument and by persisting it to a text representation on disk, like this:

// read from MTOM representation
XmlDocument doc = new XmlDocument();
FileStream custMtomStream = new FileStream(
    "customer.mtom", FileMode.Open);
using (XmlReader xr = XmlDictionaryReader.CreateMtomReader(
    custMtomStream, Encoding.UTF8, XmlDictionaryReaderQuotas.Max))
{
    doc.Load(xr);
}

// write to text (XML 1.0) representation
doc.Save("customer.xml");

The resulting custom.xml file should look identical to the original text representation. As you have seen, these System.Xml enhancements offer great flexibility by providing three ways to represent XML for persistence and transmission. Of course, other representations could be added in the future. And although it was Windows Communication Foundation that brought these enhancements to the table, you can make use of them in any .NET Framework 3.0 applications.

Windows Communication Foundation builds on this by letting you choose which XML representation to use when transmitting messages. If you need interoperability, you should choose the XML 1.0 text representation. If you need interoperability along with efficient support for binary payloads, you should choose the MTOM representation. And in scenarios that are .NET-only, the binary representation may provide better performance. The key here is that you actually have a choice.

The Message Class

Another key feature of any messaging framework is the ability to extend message payloads with arbitrary headers. A header is simply extra information traveling with the message, used to implement additional message-processing functionality (like security, reliable messaging, and transactions). In the case of XML messages, this means extending an XML payload with XML headers-both both of these are represented as XML elements framed within a container element. This functionality is exactly what SOAP provides.

The SOAP framework makes it possible to define XML-based protocols that can be used over any transport, but without relying on any transport-specific feature. WS-Addressing is a specification that extends SOAP to provide a transport-neutral mechanism for addressing/routing SOAP messages. Both SOAP and WS-Addressing are based on the InfoSet and permit (but do not require) XML 1.0 syntax to be used when transmitting messages.

The Windows Communication Foundation supports the use of any of the XmlDictionaryReader/Writer representations discussed in the previous section, which allows applications to seamlessly span a range of reach and performance requirements. Windows Communication Foundation models all messages using the Message class shown in Figure 5. As you can see, the Message class essentially models a message body and collections of message headers and properties. The available methods are primarily for creating messages, reading and writing the message body, and manipulating the collections of headers and properties.

Figure 5 System.ServiceModel.Channels.Message Class (Close Figure 5)
public abstract class Message : IDisposable
{
    // numerous overloads for creating messages
    public static Message CreateMessage(...);

    // creates a buffered copy of a message
    public MessageBuffer CreateBufferedCopy(int maxBufferSize);

    // closes a message
    public void Close();

    // returns an XmlDictionaryReader for reading the body
    public XmlDictionaryReader GetReaderAtBodyContents();

    // deserializes the body into a .NET object
    public T GetBody<T>();
    public T GetBody<T>(XmlObjectSerializer serializer);

    // numerous methods/overloads for writing messages
    public void WriteMessage(XmlDictionaryWriter writer);
    public void WriteBody(XmlDictionaryWriter writer);
    public override string ToString();

    // properties
    public abstract MessageHeaders Headers { get; }
    public abstract MessageProperties Properties { get; }
    public MessageState State { get; }
    public abstract MessageVersion Version { get; }
    public virtual bool IsEmpty { get; }
    public virtual bool IsFault { get; }

    ...
}

You create a Message object by calling one of the various static CreateMessage overloads and you dispose of a Message object using IDisposable or by explicitly calling Close. You can create a new Message from scratch, which is typical when sending a message. Or you can create a new Message from a message stream, which is typical when receiving messages.

In the case of creating a message from scratch, you must specify the action, the message version, and the body to use within the message. The action uniquely identifies the intent or semantics of the message. Windows Communication Foundation services rely on the action to dispatch incoming messages to the appropriate method. The message version identifies which versions of SOAP and WS-Addressing to use during transmission, if any. There are a variety of options to choose from when specifying the message version. Let's take a look at these options.

Message Versions

As mentioned, the MessageVersion class lets you specify the versions of SOAP and WS-Addressing you wish to use. You can create a MessageVersion object by calling CreateVersion and supplying an EnvelopeVersion object (to identify the SOAP version) along with an AddressingVersion object (to identify the WS-Addressing version). Here's what this looks like:

MessageVersion version = MessageVersion.CreateVersion(
    EnvelopeVersion.Soap12, AddressingVersion.WSAddressing10);

If you take a look at the EnvelopeVersion class, you'll see that Windows Communication Foundation currently supports three options for SOAP-None, Soap11, and Soap12, as shown here:

public sealed class EnvelopeVersion
{
    public static EnvelopeVersion None { get; }
    public static EnvelopeVersion Soap11 { get; }
    public static EnvelopeVersion Soap12 { get; }
    ...
}

Likewise, if you look at AddressingVersion, you'll see that Windows Communication Foundation currently supports three options for WS-Addressing-None, WSAddressing10, and WSAddressingAugust2004, as shown here:

public sealed class AddressingVersion
{
    public static AddressingVersion None { get; }
    public static AddressingVersion WSAddressing10 { get; }
    public static AddressingVersion WSAddressingAugust2004 { get;}
    ...
}

EnvelopeVersion.None specifies that you don't want to use SOAP during transmission, which also requires you to use AddressingVersion.None. This is a common setting for when you want to leverage Windows Communication Foundation in plain old XML messaging scenarios. Soap11 represents the SOAP 1.1 specification, which is in wide use today, and Soap12 represents the SOAP 1.2 W3C Recommendation (see www.w3.org/TR/soap for links to both specifications).

WSAddressingAugust2004 represents the original WS-Addressing W3C submission from August 2004 (see www.w3.org/Submission/ws-addressing), which is widely supported today. And WSAddressing10 represents the final WS-Addressing 1.0 W3C Recommendation (see www.w3.org/2002/ws/addr).

So, depending on which versions of SOAP and WS-Addressing the other side of the wire supports, you can pick and choose the right version to facilitate interoperability. And in order to make things a little easier for you, the MessageVersion class provides several public properties that return cached MessageVersion objects representing the most common combinations of both specifications (see Figure 6). So instead of calling CreateVersion explicitly, you can simply specify MessageVersion.Soap12WSAddressing10, which happens to be the same as MessageVersion.Default.

Figure 6 Cached MessageVersion Objects (Close Figure 6)
public sealed class MessageVersion
{
    public static MessageVersion Default { get; }
    public static MessageVersion None { get; }
    public static MessageVersion Soap11 { get; }
    public static MessageVersion Soap11WSAddressing10 { get; }
    public static MessageVersion Soap11WSAddressingAugust2004
      { get; }
    public static MessageVersion Soap12 { get; }
    public static MessageVersion Soap12WSAddressing10 { get; }
    public static MessageVersion Soap12WSAddressingAugust2004
      { get; }

    ... //
}

Reading and Writing Messages

When you create a message from scratch, there are a great many overloads that let you specify the body of the message. You can supply the body as an XmlDictionaryReader, XmlReader, or serializable object. You can also supply a MessageFault object for the body in order to create a fault message. Once the message has been created, you can gain access to the body by calling GetReaderAtBodyContents, which returns an XmlReader, or by calling GetBody<T> to deserialize the body into a .NET object.

When you want to write a message, you can write either the entire message or just the body by calling the WriteMessage or WriteBody methods, respectively. Both have overloads that allow you to provide an XmlDictionaryWriter or XmlWriter object. The following example shows how to create a new message that's subsequently written to a file named message.xml:

// load body from customer.xml file
XmlDocument doc = new XmlDocument();
doc.Load("customer.xml");

Message m = Message.CreateMessage(
    MessageVersion.Soap11WSAddressingAugust2004,
    "urn:add-customer", new XmlNodeReader(doc));

FileStream fs = new FileStream("message.xml", FileMode.Create);
using (XmlWriter xw = XmlDictionaryWriter.CreateTextWriter(fs))
{
    m.WriteMessage(xw);
}

In this case, "urn:add-customer" is specified for the action and Soap11WSAddressing2004 for the message version. The body is supplied via an XmlReader. And then WriteMessage is called to write the message out to a text-based XmlDictionaryWriter object. The resulting message.xml looks like this:

<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"
    xmlns:a="http://schemas.xmlsoap.org/ws/2004/08/addressing">
  <s:Header>
    <a:Action s:mustUnderstand="1">urn:add-customer</a:Action>
  </s:Header>
  <s:Body>
    <Customer xmlns="http://example.org/customer">
      <Email>bob@contoso.com</Email>
      <Name>Bob</Name>
    </Customer>
  </s:Body>
</s:Envelope>

Note that the XML namespaces indicate SOAP 1.1 and WS-Addressing from August 2004. It's important to point out that you could easily write the message out to a binary or MTOM representation using a different XmlDictionaryWriter object.

Now, if you wanted to, you could change the version to MessageVersion.Soap12WSAddressing10, and the resulting file would look like this instead:

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope"
    xmlns:a="http://www.w3.org/2005/08/addressing">
  <s:Header>
    <a:Action s:mustUnderstand="1">urn:add-customer</a:Action>
  </s:Header>
  <s:Body>
    <Customer xmlns="http://example.org/customer">
      <Email>bob@contoso.com</Email>
      <Name>Bob</Name>
    </Customer>
  </s:Body>
</s:Envelope>
Notice the newer versions of SOAP and WS-Addressing.

And if you were to change the version to MessageVersion.None, the resulting message would no longer contain a SOAP envelope. Instead, it would simply contain the body:

<Customer xmlns="http://example.org/customer">
  <Email>bob@contoso.com</Email>
  <Name>Bob</Name>
</Customer>

In order to read the message back in, you need to use a different CreateMessage overload that allows you to provide an XmlDictionaryReader for reading the message stream. You also need to specify the correct message version. The code that follows shows an example that reads the last message above with no envelope (version is MessageVersion.None):

FileStream fs = new FileStream("message.xml", FileMode.Open);
using (XmlDictionaryReader reader =
   XmlDictionaryReader.CreateTextReader(
      fs, XmlDictionaryReaderQuotas.Max))
{
    Message m = Message.CreateMessage(reader, 1024,
       MessageVersion.None);

    XmlDocument doc = new XmlDocument();
    doc.Load(m.GetReaderAtBodyContents());
    Console.WriteLine(doc.InnerXml);
}

As you can see, the Message class provides great flexibility around reading and writing messages using different versions of SOAP and WS-Addressing-if using any of them at all. Also, the fact that it builds on XmlDictionaryReader for reading and writing operations means you can use any of the supported XML representations when transmitting messages.

Typed Message Bodies

So far, I've been showing examples of working directly with XML. When you'd rather deserialize XML into .NET objects, you can use one of the supported serializers, such as DataContractSerializer, NetDataContractSerializer, or XmlSerializer. The following class is capable of representing the XML found in customer.xml when used with DataContractSerializer:

[DataContract(Namespace="http://example.org/customer")]
public class Customer
{
    public Customer() { }
    public Customer(string name, string email)
    {
        this.Name = name;
        this.Email = email;
    }
    [DataMember]
    public string Name;
    [DataMember]
    public string Email;
}

Now you can create messages by supplying Customer objects for the body, like this:

Customer cust = new Customer("Bob", "bob@contoso.com");
Message msg = Message.CreateMessage(
    MessageVersion.Soap11WSAddressingAugust2004,
    "urn:add-customer", cust);

Despite using an object to represent the body, the resulting message will look identical to the previous examples.

Now, when reading the body, you can use GetBody<Customer> to produce a new Customer object, as shown here:

Customer c = msg.GetBody<Customer>();
Console.WriteLine("Name: {0}, Email: {1}", c.Name, c.Email);

These versions of CreateMessage and GetBody<T> use DataContractSerializer behind the scenes. There is another version of each method that allows you to explicitly specify the serializer you wish to use.

For more details, see my August 2006 column titled "Serialization in Windows Communication Foundation" (available at msdn.microsoft.com/msdnmag/issues/06/08/ServiceStation).

Message Lifetime

The Message class has been carefully designed to support streaming. As a consequence, the body can only be processed once during the lifetime of a Message object. In order to ensure this, the Message class provides a State property that describes the current state of the object. MessageState defines five possible message states:

public enum MessageState
{
    Created,
    Read,
    Written,
    Copied,
    Closed
}

A Message object starts out in the Created state, which is the only valid state for processing the body. There are a few different ways to process the body: you can read it, write it, or copy it. Calling either GetReaderAtBodyContents or GetBody<T> changes the state to Read. Calling either WriteMessage or WriteBody changes the state to Written. And calling CreateBufferedCopy changes the state to Copied. For example, consider the following example that moves a message through several states:

Customer cust = new Customer("Bob", "bob@contoso.com");
Message m = Message.CreateMessage(
    MessageVersion.Soap11WSAddressingAugust2004,
    "urn:add-customer", cust);
Console.WriteLine("State: {0}", m.State);

Customer c = m.GetBody<Customer>();
// cannot access body from here on
Console.WriteLine("State: {0}", m.State);

m.Close();
Console.WriteLine("State: {0}", m.State);

When you execute the code, this text is printed to the console:

State: Created
State: Read
State: Closed

Once a Message object is no longer in the Created state, any method that needs access to the body will throw an exception. For example, calling GetBody<T> again, after the first call results in an exception. In situations where you need to process the body multiple times, you can create a buffered copy (by calling CreateBufferedCopy) or you can load the body into an XmlDocument or deserialize it into a .NET object for in-memory use.

Needless to say, the State property makes it possible to determine at runtime if the body of a particular Message has already been consumed (when it's no longer in the Created state) and how it was consumed (whether it was read, written, or copied).

Message Headers and Properties

The Message class provides a Headers property to model the collection of headers associated with the body. The following example illustrates how to add a custom header named "ContextId" to a message:

Customer cust = new Customer("Bob", "bob@contoso.com");
Message m = Message.CreateMessage(
    MessageVersion.Default, "urn:add-customer", cust);

m.Headers.Add(
    MessageHeader.CreateHeader(
        "ContextId", "http://example.org/customHeaders",
        Guid.NewGuid()));

When this message is written, it looks like this:

<s:Envelope xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://www.w3.org/2005/08/addressing">
  <s:Header>
    <a:Action s:mustUnderstand="1">urn:add-customer</a:Action>
    <ContextId xmlns="http://example.org/customHeaders"
    >a200f76d-5b83-4496-a035-4f9e70a07959</ContextId>
  </s:Header>
  ...
</s:Envelope>

It's typically the job of intermediaries to process headers found in a message. Such intermediaries often need to store the results of their processing somewhere for future use in the processing pipeline. For this purpose, the Message class also provides a Properties collection for storing arbitrary name/object pairs.

Unlike headers, which are always transmitted in the message, message properties are typically only used during local processing-they may not have any impact on what happens on the wire, although sometimes they do. For example, when using HTTP, Windows Communication Foundation stores the HTTP request details in the Properties collection of the incoming Message object. The details are stored in an object of type HttpRequestMessageProperty, which you can access using httpRequest for the property name. You can also influence the HTTP response details by populating a property named httpResponse with an object of type HttpReponseMessageProperty.

It's common for components within the Windows Communication Foundation channel layer to make heavy use of headers and properties-especially the built-in protocol channels that implement the various WS-* specifications.

Mapping Messages to Methods

Before you can begin sending or receiving messages with Windows Communication Foundation, you must first define a service contract that maps incoming messages to methods. This is accomplished by associating action values with method signatures. For example, the following interface defines the most universal service contract possible for a one-way operation:

[ServiceContract]
public interface IUniversalOneWay
{
    [OperationContract(Action="*")]
    void ProcessMessage(Message msg);
}

This definition associates ProcessMessage with all incoming messages regardless of their action values (thanks to the Action= "*" clause). It's also possible to define a generic request-reply operation using the same technique:

[ServiceContract]
public interface IUniversalRequestReply
{
    [OperationContract(Action="*", ReplyAction="*")]
    Message ProcessMessage(Message msg);
}

You can also associate methods with specific action values by setting the Action property to a specific value like urn:add-customer, as shown here:

[ServiceContract]
public interface ICustomer
{
    [OperationContract(Action="urn:add-customer")]
    void AddCustomer(Message msg);
}

When Windows Communication Foundation receives an incoming message, it looks at the action to determine which method to dispatch to and then it performs any necessary serialization. When using Message for the request/response types, Windows Communication Foundation doesn't perform any serialization-you decide how to process the message. However, when using typed signatures, Windows Communication Foundation automates the process of reading and writing the message bodies behind the scenes using serialization techniques. For example, consider this service contract:

[ServiceContract]
public interface ICustomer
{
    [OperationContract(Action="urn:add-customer")]
    void AddCustomer(Customer cust);
}

In this case, Windows Communication Foundation takes care of deserializing the body of the incoming message into a Customer object before invoking AddCustomer.

There is another serialization shortcut, known as message contracts, for automatically adding headers to a Message object. Message contracts allow you to annotate a class, specifying which fields map to headers versus the body:

[MessageContract(IsWrapped=false)]
public class AddCustomerRequest
{
    [MessageHeader(Name="ContextId",
        Namespace="http://example.org/customHeaders"]
    public Guid ContextId;

    [MessageBodyMember]
    public Customer customer;
}

With this message contract class in place, you can define an operation that uses it as the request type, as shown here:

[ServiceContract]
public interface ICustomer
{
    [OperationContract(Action = "urn:add-customer")]
    void AddCustomer(AddCustomerRequest request);
}

This tells Windows Communication Foundation to automatically map between the ContextId field in the object and the ContextId header in the SOAP message, freeing you from having to manually deal with the Headers collection.

Endpoints and Bindings

In order to wire-up a Windows Communication Foundation service, you need to provide several pieces of information. First, you must tell it what transport and address to use. Second, you must specify what XML representation and message version to expect. Third, you must configure which WS-* protocols to wire-in. And finally, you must provide the mapping between the incoming messages and the service methods. In Windows Communication Foundation, you specify all of these details via endpoints.

An endpoint configuration is simply a combination of an address, binding, and contract. As I already discussed, the contract defines the mapping between messages and methods. The binding specifies the remaining messaging details. It specifies what transport, XML representation, and message version to use during transmission. It also identifies which WS-* protocols should be included when building the channel stack.

A binding specifies a message encoder; this is what actually controls the XML representation and message version details. At run time, the message encoder is the component used by the transport channel to read and write messages to and from a stream. Windows Communication Foundation provides three built-in message encoder implementations: TextMessageEncoder, BinaryMessageEncoder, and MtomMessageEncoder. All of these classes build on the new XmlDictionaryReader and XmlDictionaryWriter classes discussed earlier to support a given XML representation. Each class also comes configured to use a specific message version.

As you may know, Windows Communication Foundation comes with built-in bindings to facilitate common scenarios. Each binding is configured to use a particular message encoder. For example, both BasicHttpBinding and WSHttpBinding are configured to use the TextMessageEncoder. However, the former uses MessageVersion.Soap11 while the latter uses MessageVersion.Soap12WSAddressing10 for the message version. NetTcpBinding, NetNamedPipeBinding, and NetMsmqBinding all come configured to use the BinaryMessageEncoder and MessageVersion.Soap12WSAddressing10.

You can opt to use a different message encoder on a particular binding through its properties or a binding configuration. For example, the following shows how to change BasicHttpBinding's message encoder to MtomMessageEncoder:

BasicHttpBinding basic = new BasicHttpBinding();
basic.MessageEncoding = WSMessageEncoding.Mtom;
... // use binding

When you need more control over the binding configuration, you can define a custom binding using the CustomBinding class:

MtomMessageEncodingBindingElement mtom =
    new MtomMessageEncodingBindingElement(
        MessageVersion.Soap12, Encoding.UTF8);

CustomBinding binding = new CustomBinding();
binding.Elements.Add(mtom);
binding.Elements.Add(new HttpTransportBindingElement());
... // use binding

This technique allows you to configure the precise message version and text encoding details used by the message encoder. In addition to this flexibility, the Windows Communication Foundation architecture lets you write and use custom message encoders-the SDK comes with a sample that shows how to do this. In general, Windows Communication Foundation makes it easy to configure endpoints to use different XML representations and message versions without having any impact on your service code.

Bringing It All Together

Let's look at a final example that brings most of these messaging concepts together. Figure 7 shows how to implement, host, and configure a generic service that uses the IUniversalOneWay contract along with a customized binding and encoder. Figure 8 shows how to implement a client capable of sending messages to the hosted service. The service implementation uses System.Xml to process the incoming message while the client produces a message using a Customer object.

Figure 8 Generic WCF Client (Close Figure 8)
class Program
{
    static void Main(string[] args)
    {
        Console.ReadLine();

        // create and configure an MTOM encoder
        MtomMessageEncodingBindingElement mtom =
            new MtomMessageEncodingBindingElement(
                MessageVersion.Soap12, Encoding.UTF8);

        // create a CustomBinding using MTOM encoder
        CustomBinding binding = new CustomBinding();
        binding.Elements.Add(mtom);
        binding.Elements.Add(new HttpTransportBindingElement());

        // create a ChannelFactory using the custom binding
        ChannelFactory<IUniversalOneWay> cf =
            new ChannelFactory<IUniversalOneWay>(binding);
        IUniversalOneWay channel = cf.CreateChannel(
            new EndpointAddress(
                "http://localhost:8080/customerservice"));

        // create a Message and send through channel
        Customer cust = new Customer("Bob", "bob@contoso.com");
        Message msg = Message.CreateMessage(
            MessageVersion.Soap12, "urn:add-customer", cust);
        channel.ProcessMessage(msg);
    }
}
Figure 7 Generic WCF Service (Close Figure 7)
public class CustomerService : IUniversalOneWay
{
    #region IUniversalOneWay Members
    public void ProcessMessage(Message msg)
    {
        Console.WriteLine("Message received...");
        XmlDocument doc = new XmlDocument();
        doc.Load(msg.GetReaderAtBodyContents());
        Console.WriteLine("Customer: {0} ({1})",
            doc.SelectSingleNode("/*/*[local-name()='Name']").InnerText,
            doc.SelectSingleNode("/*/*[local-name()=
                'Email']").InnerText);
    }
    #endregion
}

class Program
{
    static void Main(string[] args)
    {
        using (ServiceHost host =
            new ServiceHost(typeof(CustomerService),
            new Uri("http://localhost:8080/customerservice")))
        {
            // create and configure an MTOM encoder
            MtomMessageEncodingBindingElement mtom =
                new MtomMessageEncodingBindingElement(
                    MessageVersion.Soap12, Encoding.UTF8);

            // create a CustomBinding using MTOM encoder
            CustomBinding binding = new CustomBinding();
            binding.Elements.Add(mtom);
            binding.Elements.Add(
                new HttpTransportBindingElement());

            // add an endpoint using the custom binding
            host.AddServiceEndpoint(typeof(IUniversalOneWay),
                binding, "");
            host.Open();

            Console.WriteLine("CustomerService is up and running...");
            Console.ReadKey();
        }
    }
}

The Windows Communication Foundation messaging architecture provides a unified programming model that offers significant messaging flexibility. This makes it possible to seamlessly implement plain-old XML services that don't use SOAP, WS-Addressing, or any other WS-* protocol. It also makes it possible to configure the more sophisticated bindings to meet the precise needs of specific integration scenarios when different protocols and versions are required.

Send your questions and comments for Aaron to sstation@microsoft.com.

Aaron Skonnard is a cofounder of Pluralsight, a Microsoft .NET training provider. Aaron is the author of Pluralsight's Applied Web Services 2.0, Applied BizTalk Server 2006, and Introducing Windows Communication Foundation courses. Aaron has spent years developing courses, speaking at conferences, and teaching professional developers. Reach him at pluralsight.com/aaron.


Extreme ASP.NET: Web Deployment Projects

Extreme ASP.NET
Web Deployment Projects
Fritz Onion

Contents
Deployment in ASP.NET 2.0
Web Deployment Projects
Merging Assemblies
Pluggable Configuration Files
Creating Reusable User Controls
Conclusion

W
hen ASP was first released, Web programming was more difficult because you needed IIS to serve your ASP pages. Later, ASP.NET 2.0 and Visual Studio® 2005 made everything easier by introducing the Web site model of development. Instead of creating a new project inside Visual Studio, the Web site model lets you point to a directory and start writing pages and code. Furthermore, you can quickly test your site with the built-in ASP.NET Development Server, which hosts ASP.NET in a local process and obviates the need to install IIS to begin developing. The beauty of the Web site model is that you can develop your Web application without thinking about packaging and deployment. Need another class? Add a .cs file to the App_Code directory and start writing. Want to store localizable strings in a resource file? Add a .resx file to the App_GlobalResources directory and type in the strings. Everything just works; you don't have to think about the compilation and deployment aspect at all.

When you are ready to deploy, you have several options. The simplest choice is to copy your files to a live server and let everything be compiled on-demand (as it was in your test environment). The second option is to use the aspnet_compiler.exe utility and precompile the application into a binary release, which leaves you nothing but a collection of assemblies, static content, and configuration files to push to the server. The third option is to again use aspnet_compiler.exe, but to create an updateable binary deployment where your .as*x files remain intact (and modifiable) and all of your code files are compiled into binary assemblies.

This seems to cover every possible scenario, leaving the developer to focus simply on writing the Web application, with packaging and deployment decisions to be made later when the application is actually deployed. There was a fair amount of backlash against this model, however, especially from developers who were used to their Web projects being real projects, specified in real project files, that let you inject pre-and post-build functions, exclude files from the build process, move between debug and release builds with a command-line switch, and so on. In response, Microsoft quickly introduced the Web Application Project or WAP, initially released as an add-in to Visual Studio 2005, and now included in Visual Studio 2005 Service Pack 1 (SP1), available for download from msdn.microsoft.com/vstudio/support/vs2005sp1.

WAP provides an alternative to the Web site model that is much closer to the Visual Studio .NET 2003 Web Project model. The new WAP model compiles all of the source code files during the build process and generates a single assembly in the local /bin directory for deployment. WAP also makes it much easier to incrementally adopt the new partial class codebehind model introduced in ASP.NET 2.0 because you can now open a Visual Studio .NET 2003 project and only your .sln and .csproj (or .vbproj) files will be modified during the conversion. You can then convert each file and its codebehind class to the new partial class model independently of any other file in the project (by right-clicking on the file in the Solution Explorer and selecting Convert to Web Application), or just leave them using the old model. This is in contrast to converting a Visual Studio .NET 2003 Web project to the Web site model, which converts all files at once and does not support incremental adoption.

Finally, there is a new project type called Web Deployment Projects (the main topic of this column), which introduces myriad additional deployment options for both Web site projects and Web Application Projects. Web Deployment Projects fill the remaining holes in the deployment options for both Web site apps and Web Application Projects and make it possible to implement practically any deployment scenario in a simple and extensible way. To understand exactly what this new project type adds, let's first review what we had before Web Deployment Projects were available.

Deployment in ASP.NET 2.0

When you build an application using the Web site model, you have the option of precompiling the site for deployment. You can access the precompilation utility through the Build | Publish menu in Visual Studio 2005 or directly through the command-line utility aspnet_compiler.exe. Figure 1 shows the interface to this tool exposed by Visual Studio.

Figure 1 Publish Web Site Utility
Figure 1 Publish Web Site Utility(Click the image for a smaller view)
Figure 1 Publish Web Site Utility
Figure 1 Publish Web Site Utility (Click the image for a larger view)

The first decision you have to make when using the publish utility is whether you want your .as*x files to be updatable once deployed (use the "Allow this precompiled site to be updatable" option of -u switch in the aspnet_compiler.exe command-line utility). This decision hinges on whether you want to be able to make minor changes to your pages once deployed without having to go through the entire deployment process again. You may, in fact, want to explicitly disallow any modifications to the deployed pages and require that all modifications go through the standard deployment (and hopefully testing) process, in which case publishing the site as not updatable is the proper choice.

When a site is published as not updatable, it is possible to completely remove all .as*x files and publish only binary assemblies (plus configuration files and static content). However, without the physical files in place, it is impossible for ASP.NET to tell which classes to use for which endpoint requests. For example, if a request comes into your application for Page1.aspx and you have used non-updatable binary deployment, there very well may not be any Page1.aspx file on disk, and there is nothing in the existing configuration files to indicate which class in the collection of assemblies deployed to the /bin directory should actually be the handler for this request. To remedy this, the compilation process will also generate a collection of .compiled files that contain endpoint-to-type mapping and file dependency information in a simple XML format, and these files must be published along with the binary assemblies in the /bin directory of the deployed site. As an example, if you did have a page named Page1.aspx in your application, the aspnet_compiler.exe utility would generate a file named page1.aspx.cdcab7d2.compiled (with the hash code varying) that contained the following XML:

<?xml version="1.0" encoding="utf-8"?>
<preserve resultType="3"
          virtualPath="/SampleWebSite/Page1.aspx"
          hash="8a8da6c5a" filehash="42c4a74221152888"
          flags="110000" assembly="App_Web_aq9bt8mj"
          type="ASP.page1_aspx">
  <filedeps>
    <filedep name="/SampleWebSite/Page1.aspx" />
    <filedep name="/SampleWebSite/Page1.aspx.cs" />
  </filedeps>
</preserve>

The other major decision you have to make when publishing a Web site with this utility is the granularity of the packaging of the generated assemblies. You can either create a separate assembly for each directory in your site or create a separate assembly for each compilable file in your site (.aspx, .ascx, .asax, and so on.) by checking the Use fixed naming and single page assemblies (or -fixednames in the aspnet_compiler.exe command-line utility). This decision is not as obvious as you might think, as each option has its own potential issues. If you elect to not use the -fixednames option, then every time you publish your application a completely new set of assemblies will be generated, with completely different names from the ones published earlier. This means that deployment is trickier because you must take care to delete all of the previously published assemblies on the live server before deploying the new assemblies or you'll generate redundant class definition errors on the next request. Using the -fixednames option will resolve this problem as each file will correspond to a distinctly named assembly that will not change from one compilation to the next. If you have a large site, however, generating a separate assembly for each page, control, and Master Page can easily mean managing the publication of hundreds of assemblies. It is this problem of assembly granularity in deployment that Web Deployment Projects solve in a much more satisfying way, as you will see.

You can also introduce assembly signing into the compilation process to create strong-named, versioned assemblies, suitable for deployment in the Global Assembly Cache (GAC) if needed. You can mark the generated assemblies with the assembly-level attribute AllowPartiallyTrustedCallers using the -aptca option, which would be necessary if you did deploy any assemblies to the GAC and were running ASP.NET at a low or medium level of trust. (Keep in mind that this attribute should only be applied to assemblies that have been shown not to expose any security vulnerabilities, as using it with a vulnerability could expose a luring attack.)

One other detail about publishing your site is that if you do elect to use Web Application Projects instead of the Web site model, the Build | Publish dialog box will look quite different, as shown in Figure 2. Web Application Projects assume that you want to publish the application as updatable .as*x files and precompiled source files (the same model it uses in development), so the binary-only deployment options are not available. This utility is really closer in nature to the Copy Web site utility available with Web sites than it is to the Publish Web Site utility since it involves copying files produced by the standard build process.

Figure 2 Publish Web Utility for Web Application Projects
Figure 2 Publish Web Utility for Web Application Projects(Click the image for a smaller view)
Figure 2 Publish Web Utility for Web Application Projects
Figure 2 Publish Web Utility for Web Application Projects (Click the image for a larger view)

Technically you are not restricted from using binary-only (non-updatable) deployment, even if you are using Web Application Projects. If you think about it, the output of the build of a WAP is a valid Web site, which you can then pass through the aspnet_compiler.exe utility to generate create a binary deployment. You just can't invoke it from the Visual Studio 2005 interface which, fortunately, Web Deployment Projects rectify.

Web Deployment Projects

So what's missing from the existing compilation and deployment options presented so far? Primarily two things: the ability to control the naming of assemblies, especially for deployment purposes, and the ability to consolidate all of the output assemblies into a single assembly for simplified deployment. Web Deployment Projects solve both of these problems. Perhaps even more significantly, however, they also tie up a lot of loose ends in the deployment story that existed with Web site applications and Web Application Projects.

At their core, Web Deployment Projects (available for download at msdn2.microsoft.com/aa336619.aspx) represent just another type of project you add to your solution. Like all Visual Studio project files, Web deployment projects are MSBuild scripts that can be compiled directly in the IDE or run from the command line. Instead of specifying a collection of source code files to compile, however, Web Deployment Projects contain build commands to compile and package Web sites (or Web Application Projects). This means that they will invoke the aspnet_compiler.exe utility (among others) to create a deployment of a particular Web application. Web Deployment Projects are shipped as a Visual Studio add-in package that includes an easy-to-use menu item for injecting new projects and a complete set of property pages to control all of the available settings. To add one to an existing application, right-click on an existing Web site (or Web Application Project) and select the Add Web Deployment Project item as shown in Figure 3. This will add a new .wdproj file containing an MSBuild script to your solution, which will generate a deployment of the application you created it from.

Figure 3 Adding a Web Deployment Project
Figure 3 Adding a Web Deployment Project

Figure 3a

Figure 3b (Click the image for a smaller view)

Figure 3b (Click the image for a larger view)

Figure 3c

Once the Web Deployment Project is part of your solution, you can access the property pages of the project file to control exactly what the project does for you, as shown in Figure 4. The default setting for a new deployment project is to deploy the application in updatable mode, with all the .as*x files intact, and the source files compiled into a single assembly deployed in the top-level /bin directory. These deployment projects work the same regardless of whether the source application is using the Web site model or the Web Application Project model, which means that you can now select either development model without impacting your deployment options. One of the most significant features of Web Deployment Projects is the ability to configure the deployment to be all binary (not updatable) in the form of a single assembly, the name of which you can choose. Using this model of deployment means that you can update your entire site merely by pushing a single assembly to the /bin directory of your live site, and not concern yourself with deleting existing assemblies prior to deploying or dealing with a partially deployed site causing errors. It is still necessary to deploy the .compiled files for the endpoint mappings, but these files only change when you add, delete, or move pages in your site.

Figure 4 Web Deployment Project Property Page
Figure 4 Web Deployment Project Property Page(Click the image for a smaller view)
Figure 4 Web Deployment Project Property Page
Figure 4 Web Deployment Project Property Page (Click the image for a larger view)

Web Deployment Projects provide flexibility in deployment and let you make packaging and deployment decisions independently of how you actually built your Web applications. This independence between development and deployment was partially achieved in the original release of ASP.NET 2.0 with the aspnet_compiler.exe utility, but never fully realized because of the constraints imposed when performing the deployment. With Web Deployment Projects, the separation between development and deployment is now complete, and your decision about how to build your applications will no longer impact your deployment choices.

Merging Assemblies

Much of what Web Deployment Projects provide is just a repackaging of existing utilities exposed via MSBuild tasks and a new interface, but there are also a couple of completely new features included. The most intriguing is the ability to merge assemblies.

When you install Web Deployment Projects, you will find an executable called aspnet_merge.exe in the installation directory (%PROGRAMFILES%\MSBuild\Microsoft\WebDeployment\v8.0, by default). This executable is capable of taking the multi-assembly output of a precompiled site and merging the assemblies into one. This is the utility that is incorporated into your build script if you select the merge option in a Web Deployment Project. As an example of what is possible with this utility, consider the output of a precompiled Web site, run without the updatable switch, shown in Figure 5. The source application for this output contained two subdirectories, a top-level global.asax file, a class defined in App_Code, and a user control. The end result of the compilation is five different assemblies and a collection of .compiled files. If you run the aspnet_merge.exe utility on this directory with the -o switch to request a single assembly output, shown at the bottom of Figure 5, the result is a much more manageable single assembly named whatever you specify.

Figure 5a Using the aspnet_merge.exe Utility
Figure 5a Using the aspnet_merge.exe Utility(Click the image for a smaller view)
Figure 5a Using the aspnet_merge.exe Utility
Figure 5a Using the aspnet_merge.exe Utility (Click the image for a larger view)

Figure 5b (Click the image for a smaller view)

Figure 5b (Click the image for a larger view)

Although the aspnet_merge.exe utility and the corresponding MSBuild task that ship with Web Deployment Projects are new, the underlying technology for merging assemblies has actually been around since the Microsoft® .NET Framework 1.1 in the form of a utility made available from Microsoft Research called ILMerge, the latest version of which is available for download from research.microsoft.com/~mbarnett/ILMerge.aspx. This utility is directly incorporated into aspnet_merge.exe and does all the heavy lifting involved with merging assemblies. If you think about it, the merging of assemblies is a rather complicated task. You need to take into consideration signing, versioning, and other assembly-level attributes, embedded resources, and XML documentation, as well as manage the details of clashing type names, and so on. The ILMerge utility manages all of these details for you, with switches to control various decisions about the process. It also gives you the ability to transform .exe assemblies into .dll assemblies for packaging purposes. As an example, suppose you have three assemblies: a.dll, b.dll, and c.exe which you would like to merge into a single library assembly. As long as there were no conflicts in typenames, the following command line would generate a new library, d.dll with all of the types defined in a.dll, b.dll, and c.exe:

ilmerge.exe /t:library /ndebug /out:d.dll a.dll b.dll c.exe

Pluggable Configuration Files

The other completely new feature that comes with Web Deployment Projects is the ability to create pluggable configuration files. It is a common problem when deploying Web applications to find a way to manage the differences in your configuration files between development and deployment. For example, you may have a local test database to run your site, have another database used by a staging server, and yet another used by the live server. If you are storing your connection strings in web.config (typically in the connectionStrings section), then you need some way of modifying those strings when the application is pushed out to a staging server or to a production machine. Web Deployment Projects offer a clean solution to this problem with a new MSBuild task called ReplaceConfigSections.

This task allows you to specify independent files that store the contents of a particular configuration section independently based on solution configurations. For example, you might create a debugconnectionstrings.config file to store the debug version of our connectionStrings configuration section that looked like this:

<connectionStrings>
    <add connectionString="server=localhost;database=sales;
                           trusted_connection=yes" name="sales_dsn"/>
</connectionStrings>

Similarly, you would then create separate files for each of the solution configurations defined (release, stage, and so on) and populate them with the desired connection strings for their respective deployment environments. For the release configuration, you might name the file releaseconnectionstrings.config and populate it as follows:

<connectionStrings>
    <add connectionString="server=livedbserver;database=sales;
                           trusted_connection=yes"
                           name="sales_dsn"/>
</connectionStrings>

Next, you would configure the MSBuild script added by Web Deployment Projects to describe which configuration sections in the main web.config file should be replaced, and the source files that will supply the content for the replacement. You could modify the script by hand, but there is a nice interface exposed through the property pages of the build script in Visual Studio that will do it for you, as Figure 6 shows. In this case, you are setting the properties for the debug solution configuration, so check the Enable Web.config file section replacement option and specify the section to be replaced along with the file with the contents to replace it:

connectionStrings=debugconnectionstrings.config

Figure 6 Property Page to Set Configuration File Section Replacement
Figure 6 Property Page to Set Configuration File Section Replacement(Click the image for a smaller view)
Figure 6 Property Page to Set Configuration File Section Replacement
Figure 6 Property Page to Set Configuration File Section Replacement (Click the image for a larger view)

You would use this same dialog page to set the configuration replacement for the Release solution configuration (and any others we had defined) with the corresponding files.

When you then run the build script, the ReplaceConfigSections task extracts the contents from any associated config files and replaces the contents of the corresponding configuration section, creating a new web.config file that is pushed to the deployment directory. This configuration file replacement feature means that you can maintain configuration differences between deployment environments in a manageable way with text files that can be versioned under source control, and you don't have to resort to referring to that sticky note reminding you to change the connection string when you deploy. It should be emphasized that this feature works with any section of the configuration file, even custom sections, so if you have differences in other configuration sections (for example, appSettings) you can easily specify those differences with this build task as well.

Creating Reusable User Controls

There is an interesting side application of Web deployment projects that solves a problem that has plagued ASP.NET developers for years-how to create reusable user controls to share across applications. User controls are fundamentally just composite custom controls whose child controls are laid out in an .ascx file. The ability to use the designer for laying out controls and adding handlers is a huge benefit for most developers since it feels almost identical to building a page, except that the resulting .ascx file can be included as a control in any page. The disadvantage has always been that you need the physical .ascx file in the application's directory to actually use it. Techniques for making .ascx controls shareable across applications are available, but they usually involve chores like creating shared virtual directories between applications or harvesting temporary assemblies generated by ASP.NET at request time, and they've never been satisfactory.

The introduction of the aspnet_compiler.exe utility in version 2.0 brought us much closer to a decent solution. With the compiler, you can create a Web site consisting of only user controls and publish the site in non-updateable mode using the compiler to generate reusable assemblies. Once you have the resulting assembly (or assemblies), you can then deploy to any Web application and reference the user control just as you would a custom control (not by using the src attribute as you would for .ascx files). The only disadvantage to this technique is that you either have to accept the randomly named assembly produced by the compilation process or select the fixednames option in the compiler to generate a fixed named assembly for each Master Page in the site (not a single assembly for the entire collection).

Web Deployment Projects provide the final step to create truly reusable user control assemblies. You can take the same Web site consisting exclusively of user controls and add a Web Deployment Project to create a single output assembly with the name of your choice. It's even straightforward to create a signed assembly to deploy to the GAC for sharing controls across multiple applications without redeploying the assembly in each /bin directory.

Conclusion

The release of Web Deployment Projects completes the set of tools for deploying ASP.NET applications in a very satisfying way. It is now possible to deploy your applications in any manner ranging from all source to all binary, with complete control over the generation, packaging, and naming of the binary assemblies. In addition, Web Deployment Projects provide a solution for replacing sections of your configuration files based on your target build, and they solve the problem of distributing reusable user controls. Anyone who is building and deploying ASP.NET applications will undoubtedly find some aspect of Web Deployment Projects compelling enough to begin using them today.

Send your questions and comments for Fritz to xtrmasp@microsoft.com.

Fritz Onion is a co-founder of Pluralsight, a Microsoft .NET training provider, where he heads the Web development curriculum. Fritz is the author of Essential ASP.NET (Addison Wesley, 2003) and Essential ASP.NET 2.0 (Addison Wesley 2006). Reach him at pluralsight.com/fritz.


Foundations: 3D Mesh Geometries

Foundations
3D Mesh Geometries
Charles Petzold

Code download available at: Foundations2007_04.exe (165KB)
Contents
The 3D Viewport
Anatomy of a Mesh Geometry
Algorithmic Mesh Geometries
Accommodating the Consumer
Integrating with XAML
Dependency Properties
Proper Handling of Collections
Seams and Ends
Showcasing the Cylinder
Throw Out the First One

A
mong the classes that contribute to the Microsoft Windows Presentation Foundation, those in the System.Windows.Media.Media3D namespace stand out. These are the classes that are intended to bring three-dimensional graphics to mainstream Windows® applications. As with the Windows Presentation Foundation 2D graphics, 3D graphics are often most conveniently accessed in Extensible Application Markup Language (XAML), but there the similarities pretty much end. 3D graphics programming involves significantly different concepts and conventions. Where 3D and 2D overlap is in the area of brushes: you always cover the surface of a 3D visual with a 2D brush.

Figure 1 shows Hello3D, a 3D version of the traditional "Hello, World" program. If you're running Windows Vista™ or have the Microsoft® .NET Framework 3.0 runtime installed under Windows XP, you can simply use Internet Explorer® to launch the XAML code that produces it to see the image yourself (see Figure 2).

Figure 1 The Hello3D Image
Figure 1 The Hello3D Image

The 3D Viewport

In 3D graphics programming, there are no lines, no Bezier spline curves, no rectangles or ellipses. Every 3D object is a collection of triangles in three-dimensional coordinate space. Triangles are intrinsic to 3D programming because each individual triangle always defines a flat surface, yet collections of triangles can mimic a solid object and even approximate a curved surface. As you get deeper into 3D programming, you'll start seeing everything in your life in terms of triangles.

As Hello3D.xaml shows, 3D views are composed within a Viewport3D element. A 3D scene requires one or more objects of type GeometryModel3D, one or more light sources, and a camera that governs how the 3D object is projected onto a 2D surface, and thus how the viewer sees the scene.

The GeometryModel3D element has three crucial properties: Geometry, Material, and BackMaterial. The Geometry property is set to a MeshGeometry3D element that describes the visual object in terms of coordinate points and triangles. The Material and BackMaterial properties indicate how the fronts and backs of the object are to be colored. In Hello3D.xaml, both properties are set to objects of type DiffuseMaterial. The Material property is a VisualBrush that consists of a TextBlock with the words "Hello, World." The BackMaterial property is simply a red brush. (If you want to see the back of the object, change the camera Position property to "0 0 -5" and LookDirection to "0 0 1.")

Anatomy of a Mesh Geometry

In this column, I want to focus on a particularly important part of this Viewport3D assemblage-the MeshGeometry3D class that defines the actual geometry of the 3D object. This class has four important properties: Positions, TriangleIndices, TextureCoordinates, and Normals. The Positions property is a collection of Point3D objects, which define locations in terms of X, Y, and Z coordinates. In this coordinate system, X coordinates increase to the right, Y coordinates increase going up the screen, and Z coordinates increase coming out of the screen. (This is known as a right-handed coordinate system; if you point the index finger of your right hand toward increasing X values, and your middle finger toward increasing Y values, your thumb points to increasing Z values. The Windows Presentation Foundation 3D system also implements the right-hand rule for rotations: if you point the thumb of your right hand to increasing values on any axis, the curve of your other fingers shows the direction of positive rotation around that axis.)

The Positions property in the Hello3D.xaml example has six Point3D objects separated by commas:

Positions="-2 1 -1, 0 2 0, 2 1 -1, -2 -1 -1, 0 -2 0, 2 -1 -1"

The first three are the coordinates of the top of the object from left to right, and the last three are the corresponding coordinates on the bottom. Notice that the Z coordinates in the center are 0, but -1 on the left and right edges, so the center is to the foreground of the left and right edges.

The Positions property indicates all the vertices of the object. These vertices are certainly instrumental in defining the object, but they don't tell the whole story. Any group of three vertices can be combined in a triangle, and that's what the TriangleIndices collection indicates. TriangleIndices is a collection of integers arranged in triplets. Each group of three integers defines a triangle. The values of the integers are indices into the Positions collection. For example, the first triplet is 0 3 1, which refers to the 3D points (-2, 1, -1), (-2, -1,-1), and (0, 2, 0). That's the triangle at the upper-left corner of the object.

TriangleIndices="0 3 1, 1 3 4, 1 5 2, 1 4 5"

The TriangleIndices collection actually drives the rendering of the object. Any Positions element not referenced by TriangleIndices is ignored (if there are no TriangleIndices, every Point3D triplet in the Positions collection is interpreted to be a triangle).

Each triangle is considered to have a front and a back. When looking at the front of a triangle, the triplet indexes the points in a counter-clockwise direction. If you change the first triplet to 0 1 3, you'll see the upper-left triangle colored red, since you're now looking at the back of the triangle instead of its front.

If a 3D object is to be colored with a solid brush, the Positions and TriangleIndices properties are sufficient. For other types of brushes (gradient brushes or tile brushes), you'll also need TextureCoordinates. Brushes are 2D surfaces that cover 3D objects like shrink wrap. The TextureCoordinates collection indicates a correspondence between the vertices of the 3D object and the coordinates of the 2D brush. This collection contains one 2D point for each 3D point in Positions. These 2D points are relative coordinates (between 0 and 1) with increasing values of Y going down. The point (0, 0) indicates the top-left corner of the brush, and (1, 1) is the bottom-right corner. In Hello3D.xaml, the six 2D points indicate the coordinates of the VisualBrush defined as the Material property of the GeometryModel3D element:

TextureCoordinates="0 0, 0.5 0, 1 0, 0 1, 0.5 1, 1 1"

In effect, each triangle of the 3D object is covered by a triangle of the brush, which might need to be stretched or shrunk to fit.

The Normals property is a collection of vectors in one-to-one correspondence with the Positions collection. Each vertex is considered to face a particular direction, which is indicated by the Normals vector for that vertex. Each point within each triangle reflects light differently based on an interpolation of the vectors at its three vertices. If you don't supply a Normals collection, one is calculated for you based on an average of the normals of the triangles that meet at each vertex shared in your mesh specification.

Algorithmic Mesh Geometries

The classes in the System.Windows.Media Media3D namespace offer no higher-level interfaces beyond MeshGeometry3D. (By the end of this article, you might understand why.) For simple objects with flat sides, such as pyramids and cubes and such, you can hand-code the MeshGeometry3D object. For more complex primitives, particularly those that have curved surfaces, you'll probably choose to generate the vertices and indices algorithmically using a .NET-compliant language. Indeed, if you're ambitious, you might conceive of a whole library of mesh geometries deriving from MeshGeometry3D.

Immediately you run into a brick wall. MeshGeometry3D is sealed; it cannot be inherited. Moreover, MeshGeometry3D itself derives from the abstract class Geometry3D, and you can't derive from Geometry3D because it has a solitary, internal constructor.

Rather than deriving a class from MeshGeometry3D, an alternative approach is to simply define a class that generates a MeshGeometry3D object and exposes that object as a property. You can then define this class as a resource in a XAML file and reference the MeshGeometry3D through a binding.

Figure 3 shows a class named SimpleCylinderGenerator written in C#. (I'll be restricting myself to cylinders here.) In the downloadable code for this column, SimpleCylinderGenerator contributes to a DLL named Petzold.MeshGeometries.

The SimpleCylinderGenerator class has two properties: you use the Slices properties to define how many triangles are used to approximate the curvature of the cylinder. (I appropriated the term "slices" from the documentation of the static Mesh.Cylinder method in the Direct3D class library.) The number of triangles running the length of the cylinder is actually double the Slices property. The top and bottom ends of the cylinder each also require a number of triangles equal to Slices.

The MeshGeometry property creates a MeshGeometry3D object based on the Slices property. The algorithm is hardcoded to create a cylinder that extends one unit along the positive Y axis with a radius of one unit. The idea here is that you can later size this object and move it wherever you want using transforms.

The SimpleCylinderDemo program shows how to use this SimpleCylinderGenerator class. The bulk of this program is the SimpleCylinderDemo.xaml file, which is shown in Figure 4. The root element contains an XML namespace declaration for the namespace and DLL in which the SimpleCylinderGenerator class is defined and associates with that a prefix of pmg (for "Petzold Mesh Geometries").

The Resources section includes an object of type SimpleCylinderGenerator with the key name of "cylinder" and a Slices value of 36. The GeometryModel3D element assigns its Geometry property to a binding of this resource and its MeshGeometry property. A transform changes the size of the cylinder. With some directional lighting and a SpecularMaterial object added to the DiffuseMaterial, the results are shown in Figure 5. You could animate this object by animating any transforms applied to the object.

Figure 5 A Cylinder
Figure 5 A Cylinder

Obviously, the SimpleCylinderGenerator technique works, but I feel that it has a number of flaws and insufficiencies. Recognizing these problems and fixing them will provide several insights into not only mesh geometries in particular, but also some general aspects of Windows Presentation Foundation programming.

Accommodating the Consumer

When I first started defining mesh geometries for objects such as cylinders, I strived above all for symmetry, and SimpleCylinderGenerator reveals my biases: all the triangles are isosceles. But for the triangles running lengthwise along the cylinder, that's actually a problem.

It's often advantageous that a mesh-geometry algorithm produce something useful when the Slices property is small. Try setting the Slices property of SimpleCylinderGenerator to 4 and you get the object shown on the left in Figure 6. (Edges have been enhanced for illustrative purposes.) It's interesting but not quite as useful as the object on the right. The object on the right requires that the triangles running lengthwise be not isosceles triangles, but right triangles, so that each pair of right triangles forms a rectangle.

Figure 6 Slicing a Cylinder
Figure 6 Slicing a Cylinder

SimpleCylinderGenerator does not generate a TextureCoordinates property, which is fine if the cylinder is only going to be covered with a SolidColorBrush. On the other hand, if you want to use any type of GradientBrush, or any type of TileBrush, you'll need a TextureCoordinates property that has been intelligently generated along with the Positions property.

When a cylinder is covered with 2D brush, you probably want the brush to wrap around the cylinder so that the left and right edges of the brush meet at a line I'll call the "seam." This seam should run along the length of the cylinder. That's yet another reason to use right triangles rather than isosceles triangles for defining the mesh.

SimpleCylinderGenerator also reveals another of my biases, one of economy: all the Point3D objects in the Positions collection are unique. Within the for loop that generates the TriangleIndices collection, the modulus operator is used so that some later triangles use points early in the Positions collection. It somehow seemed to me that this might be necessary to "close up" the object, but that's just plain wrong. Again, if you want to cover the cylinder with a brush, you'll need to have duplicate Point3D objects at the seam so that the same location in 3D space maps to both the left side and the right side of the brush.

SimpleCylinderGenerator takes the easiest approach imaginable in defining the Positions collection by making the cylinder have a length and radius of one unit. It then relies on the consumer of the class (which might be you or might be another programmer) to come up with the transforms that will size and position the cylinder appropriately. It's fairly easy to translate the base of the cylinder to another location with a translation transform, but what about placing the top of the cylinder at a particular location as well? You'll need to calculate a rotation transform just for that job.

Give the consumer a break! It makes more sense for a cylinder generator to have properties of Point1 and Point2 that indicate the coordinate positions of the centers of the two ends of the cylinder. You might also want to consider Radius1 and Radius2 properties for independently setting the radii at the two ends. There's an extra bonus with having separate radii properties: if one radius is 0, the algorithm generates a cone. When defining mesh-geometry algorithms, "bonus" primitives are always welcome.

So far we have a nice list of improvements to SimpleGeometryGenerator, but I want to make that list even longer.

Integrating with XAML

To use SimpleCylinderGenerator, you must define the class as a resource and then access it with a binding. It would be much preferable to have a class that instantiates in XAML and integrates right into the Viewport3D markup. But how? It can't derive from MeshGeometry3D or Geometry3D.

Fortunately, there's another way, but it requires a familiarity with the often confusing Media3D namespace and some of the terminology involved in class names. Let's first talk about the difference between visuals and models.

A visual is something that can render itself on the screen. Within the Media3D namespace, that's the abstract Visual3D class. Visual3D objects are children of the Viewport3D.

A model, in contrast, is a description of something that a visual can render. In 3D programming, models include the 3D objects themselves and also the lights that illuminate them. The same models can be shared among multiple visuals. In the Media3D namespace, for example, the model is represented by the abstract Model3D class. Classes that derive from Model3D include GeometryModel3D, Light (which is abstract), and Model3DGroup.

The class that hooks a visual together with a model is ModelVisual3D. ModelVisual3D derives from Visual3D, so it's definitely a visual, but it has a Content property of type Model3D. A model is the content of a visual in much the same way that text or a bitmap is the content of a button.

Amazingly enough, ModelVisual3D is the only class in the entire Media3D namespace that is neither sealed nor abstract, which means that it's available for inheritance with no apparent problems (see Daniel Lehenbauer's blog entry at blogs.msdn.com/478923.aspx and Karsten Januszewski's blog entry at blogs.msdn.com/479924.aspx for more information).

Suppose you code a Cylinder class that derives from ModelVisual3D. You could then specify such an element as a child of a Viewport3D:

<Viewport3D>
    <pmg:Cylinder Point1="0 1 5" Point2="3 2 -4"
                  Radius1="0.25" Radius2="0.125" />
    ...
</Viewport3D>

This seems quite convenient, but it's not quite as simple as this markup would imply. If the Cylinder class derives from ModelVisual3D, then it inherits a Content property of type Model3D (from which GeometryModel3D derives). The Cylinder would have to create an object of type GeometryModel3D to set to its inherited Content property. GeometryModel3D defines a Geometry property, so the Cylinder class would also create a MeshGeometry3D object to set to this Geometry property.

So far, it's not bad, but here's the kicker. GeometryModel3D also has Material and BackMaterial properties. This is how materials get associated with a mesh geometry. To make this scheme work correctly, the Cylinder class has to redefine the Material and BackMaterial properties, so the markup will look like this:

<Viewport3D>
    <pmg:Cylinder Point1="0 1 5" Point2="3 2 -4"
                  Radius1="0.25" Radius2="0.125">
        <pmg:Cylinder.Material>
        ...
        </pmg:Cylinder.Material>
        <pmg:Cylinder.BackMaterial>
        ...
        </pmg:Cylinder.BackMaterial>
    </pmg:Cylinder>
    ...
</Viewport3D>

Now comes the big question: I've indicated that the Cylinder class has properties named Point1, Point2, Radius1, and Radius2. Do you want those properties potentially to be targets of data bindings? I suspect you do. Do you want them to be animatable? Oh, I am sure you do. In either case, these properties must be defined as dependency properties. As you might have heard, everything in the Windows Presentation Foundation is animatable (but only if it's a dependency property).

Dependency Properties

Dependency properties have emerged as one of the most important innovations of the Windows Presentation Foundation. Within Windows Presentation Foundation, properties can be set in a variety of ways. Properties can be set directly on an object in code or XAML. In addition, properties can be set through styles and through data bindings. Some properties can be inherited through a parent-child relationship, and properties can be animated. If a property isn't set at all, it has a default value. Dependency properties are an attempt to make all this variety work correctly with predictable levels of precedence.

Dependency properties require some scary-looking overhead in your classes that involves registering the properties with the system, but after that's finished, you pretty much don't have to worry about how the property is being set. The important thing is that your class gets notified when the property has changed so it can react to those changes.

The Cylinder class, which is among the downloadable code for this article, defines nine dependency properties. Rather than Cylinder deriving directly from ModelVisual3D, I created an intermediate class named ModelVisualBase that might be parent to other classes such as Sphere, Tube, and BunnyRabbit. ModelVisualBase defines Geometry, Material, and BackMaterial dependency properties that add an owner to the same properties defined in GeometryModel3D.

As an example of how dependency properties are implemented in your code, let's look at the Radius1 property of Cylinder. Registering the dependency property involves defining a public static read-only field of type DependencyProperty named Radius1Property, that is, the property name with the word Property appended:

public static readonly DependencyProperty Radius1Property =
    DependencyProperty.Register(
        "Radius1", typeof(double), typeof(Cylinder),
        new PropertyMetadata(1.0, PositionsChanged),
                             ValidateNonNegative);

Notice the arguments to the PropertyMetadata constructor, which indicate the default value, a method in the Cylinder class to call when the value changes, and a method to validate values.

You also include a traditional property definition (also called a "CLR property") that references this static read-only field:

public double Radius1
{
    get { return (double)GetValue(Radius1Property); }
    set { SetValue(Radius1Property, value); }
}

The GetValue and SetValue methods are inherited from DependencyObject. Any class that implements dependency properties must derive from DependencyObject.

It's important to realize that changes to the Radius1 property don't always go through the CLR property. When Radius1 is being animated, for example, the Radius1Property field is accessed rather than the Radius1 property. For that reason, you should avoid doing anything else in the Radius1 property except calling GetValue and SetValue.

The definition of Radius1Property includes references to two methods in the Cylinder class. Because these methods are referenced by a static field, the methods themselves must also be static. The ValidateNonNegative method simply checks that Radius1 isn't being set to a negative value:

static bool ValidateNonNegative(object value)
{
    return (double)value >= 0;
}

If some code does set Radius1 to a negative value, an exception will be thrown, but that's not the responsibility of your class.

The other method is a notification to the class that the property has been changed. Because the property can be changed without the CLR property being accessed, this notification method is your only opportunity to react to this property change. Here's the method I called PositionsChanged that's called when the Radius1 property has been changed:

static void PositionsChanged(DependencyObject obj,
    DependencyPropertyChangedEventArgs args)
{
    Cylinder cyl = (Cylinder)obj;
    cyl.GeneratePositions();
}

The method is static, but the first argument is the actual Cylinder object whose property is being altered. The DependencyPropertyChangedEventArgs object has information about the particular property being changed, its old value, and its new value, but Cylinder ignores that stuff and just calls an instance method: GeneratePositions. This method is responsible for generating the Positions and Normals properties of the MeshGeometry3D object based on the new radius.

Proper Handling of Collections

The Cylinder class defines three separate methods for defining the four collections of the MeshGeometry3D object. GeneratePositions is responsible for the Positions and Normals collections; the two other methods are named GenerateTriangleIndices and GenerateTextureCoordinates. Cylinder's constructor calls all three methods to initialize the object; thereafter they are called in response to changes in the dependency properties. For most properties (Point1, Point2, Radius1, Radius2) only the Positions and Normals collections need to be recalculated. For the Slices property, however, all four collections need to be redone. Similar to the Slices property is another property named Stacks that governs the lengthwise subdivision of the cylinder. In many cases, you can set Stacks equal to 1, which is the default value. But if you're using PointLight or SpotLight to illuminate the cylinder, you'll need to generate much smaller triangles to get the light to work correctly.

When properties such as Point1 and Radius1 are animated, a method like GeneratePositions can be called very frequently, and as fast as possible. Here are some potential problems and solutions.

In particular, the method should avoid memory allocations. The last thing you want is for the CLR garbage collection truck to come rolling through your animation cleaning up the messes you've left behind. If at all possible, any new expressions should refer to structures rather than classes. In the Cylinder class, for example, I have the constructor create objects of type RotateTransform3D and AxisAngleRotation3D that are stored as fields and reused by each call to GeneratePositions.

The collections that you set to the MeshGeometry3D properties are of type Point3DCollection, Vector3DCollection, Int32Collection, and PointCollection. All collections derive from Freezable and include a Changed event that is fired whenever any element of the collection changes. This provides a powerful tool: for example, individual members of these collections can be animated to create deformations of 3D objects. But if you're entirely recalculating the whole collection, you only want a notification to occur when you're finished. For this reason, you should detach the collection from the MeshGeometry3D object and reattach it when you're finished. Here's how GeneratePositions begins and ends:

void GeneratePositions()
{
    MeshGeometry3D mesh = (MeshGeometry3D)Geometry;
    Point3DCollection points = mesh.Positions;
    mesh.Positions = null;
    points.Clear();

        ...

    mesh.Positions = points;
}

The first statement accesses the Geometry property defined by ModelVisualBase and inherited by Cylinder. Notice that the method reuses the Point3DCollection rather than reallocating a new one. The four collections are created by Cylinder's constructor using the number of elements they will contain based on the default Slices property. If Slices increases, the collection size might then be insufficient and a memory allocation will have to take place, but this shouldn't happen very often.

On the other hand, if you need to populate a collection with a large number of elements, then you should create that collection with a constructor indicating the number of elements. Allocating the necessary memory initially helps avoid frequent memory allocations while adding elements to the collection. Thanks to Tim Cahill's blog entry of August 31, 2006 and to Daniel Lehenbauer's summary of November 19, 2006 for these tips.

Seams and Ends

Earlier in this column, I discussed the seam where the left side of the brush meets the right. Where should this seam go? I decided it should go "in the back" of the cylinder, which in the common case is towards the negative Z axis. The only case in which this logic breaks down is for a cylinder along the Z axis, in which case the Cylinder class puts it on the bottom.

You can wrap an ImageBrush around the cylinder, and if the length and circumference of the cylinder match the aspect ratio of the bitmap, there won't be any distortion. One problem, however, involves the two ends of the cylinder. What should happen there? I decided I definitely wanted part of the image to wrap around the top and bottom ends. The proportion of the image to be wrapped would be governed by two additional properties named Fold1 and Fold2, with default values of 0.1 and 0.9 respectively. These indicate the relative brush coordinates where the image folds from the length of the cylinder to the ends.

The ends of the cylinder are approximated by a ring of triangles that share a common point in the center. If you "unfold" the triangles that define the cylinder and spread them out over a rectangular brush, you can visualize the correspondences between the mesh geometry and brush. Figure 7 shows such a correspondence for a 12-slice cylinder and a gradient brush. This correspondence seems reasonable, and if the dimensions of the brush, cylinder, and the "fold" settings are set right, the brush might not be stretched at all.

Figure 7 Brush Mapped to Cylinder
Figure 7 Brush Mapped to Cylinder

But for a bitmap, you might want as few seams as possible. You already have a seam where the two sides of the bitmap meet. You might avoid additional seams on the ends of the cylinder by using a layout shown in Figure 8. Of course, there will be some stretching involved. (In either case, not all of the brush gets on the cylinder, but that's unavoidable unless you create much finer triangles.)

Figure 8 Better Brush Mapping
Figure 8 Better Brush Mapping

I couldn't decide which I like best, so I implemented both. The first option seems to be suitable for DrawingBrush, and the second for ImageBrush, so I defined a TextureType property and a TextureType enumeration with three members: Drawing, Image, and None. The None option suppresses the generation of TextureCoordinates entirely.

Showcasing the Cylinder

Now it's time to put the Cylinder class through its paces. The ImageOnCylinder project defines a cylinder object in XAML and puts a bitmap on its surface:

<pmg:Cylinder x:Name="cyl" Slices="32"
              TextureType="Image" Fold1="0.25" Fold2="0.75">
  <pmg:Cylinder.Material>
    <DiffuseMaterial>
      <DiffuseMaterial.Brush>
        <ImageBrush
         ImageSource="http://www.charlespetzold.com/PetzoldTattoo.jpg"
        />
      </DiffuseMaterial.Brush>
    </DiffuseMaterial>
  </pmg:Cylinder.Material>
  ...

The ImageOnCylinder.xaml file animates the Point1, Radius1, and Radius2 properties, and the cylinder is also rotated slowly around its axis. A snapshot is shown in Figure 9. The program also provides a scrollbar to rotate the camera around the X axis. (When running any of these samples, if you can't see the whole image, trying making the window narrower.) The PolkaDottedCylinder project defines a DrawingBrush with carefully calculated dimensions to avoid distortion, as shown in Figure 10.

Figure 9 ImageOnCylinder
Figure 9 ImageOnCylinder
Figure 10 Polka Dots
Figure 10 Polka Dots

The final example goes back to a SolidColorBrush, but has a total of 10 cylinders, and shows the convenience of precisely defining the locations of the two ends. When two 3D objects meet, they meld rather nicely, as shown in Figure 11. Two scrollbars are provided for viewing the chair from different angles (but don't try sitting on it just yet).

Figure 11 SteelChair
Figure 11 SteelChair

Throw Out the First One

There's an old recommendation for writing software: after you're finished, throw out what you've done and start over, incorporating everything you learned when writing the first version. Of course, this approach isn't always practical, but it worked for me. The Cylinder class I wrote is my fifth or sixth stab at writing a class that generates a mesh geometry, and I now feel I'm ready to move on to the sphere.

Send your questions and comments to mmnet30@microsoft.com.

Charles Petzold is a Contributing Editor to MSDN Magazine and the author of Applications=Code+Markup: A Guide to the Microsoft Windows Presentation Foundation (Microsoft Press, 2006).


.NET Matters: Restart Manager and Generic Method Compilation

.NET Matters
Restart Manager and Generic Method Compilation
Stephen Toub

Code download available at: NetMatters2007_04.exe (160KB)

Q When my application attempts to access a file, I get access-denied errors because the file is in use by another application. In the past, I've used tools from Sysinternals (microsoft.com/technet/sysinternals) to figure out what that other application is, but I'd like to be able to discover this programmatically from within my application. Is there a way to programmatically determine what processes are currently using a specific file?

A I've been asked this question several times, and every time I've refrained from answering it because until very recently, I didn't have an answer. Information on the files currently open by an arbitrary process has been accessible to kernel-mode drivers but until now, no documented user-mode Win32® or Microsoft® .NET Framework APIs that exposed that information have existed. However, with the release of Windows Vista™, and a little ingenuity, it's now possible to determine this information from a user-mode application. Don't go running to the API documentation for an aptly named method, however. Rather, think outside the box a bit, and you'll find yourself staring squarely at the Restart Manager API.

New to Windows Vista, the Restart Manager API was added to reduce the number of system restarts that are required during software installation. The main reason software installations and updates require system restarts is that running processes are using files that the installation needs to access. The Restart Manager APIs address this conflict by allowing an installer to politely shut down Restart Manager-aware applications that are holding onto resources the installer needs to access. And as part of this, the Restart Manager APIs allow an installer to query for processes that are holding onto resources the installer previously registered as needing. The beauty of this, though, is that any application can be considered an "installer," and thus any application can use these Restart Manager APIs to query for other processes that are holding onto specific resources. Thus, I can write a method that accepts as a parameter a path to the file in question, and that method can use the Restart Manager APIs to gather a list of processes using that file.

The .NET Framework 3.0 does not include managed equivalents for the Restart Manager APIs, so I need to provide my own through the magic of P/Invoke. For my purposes, I need access to four functions, all of which are exposed from rstrtmgr.dll, and for all of which P/Invoke definitions are shown in Figure 1 (note that the Restart Manager APIs include many more functions than these four, but these are all I need for this purpose).

Figure 1 P/Invoke Declarations for Key Restart Manager APIs (Close Figure 1)
[DllImport(“rstrtmgr.dll”, CharSet = CharSet.Unicode)]
static extern int RmStartSession(
    out uint pSessionHandle, int dwSessionFlags, string strSessionKey);

[DllImport(“rstrtmgr.dll”)]
static extern int RmEndSession(uint pSessionHandle);

[DllImport(“rstrtmgr.dll”, CharSet = CharSet.Unicode)]
static extern int RmRegisterResources(uint pSessionHandle,
    UInt32 nFiles, string[] rgsFilenames,
    UInt32 nApplications, [In] RM_UNIQUE_PROCESS[] rgApplications,
    UInt32 nServices, string[] rgsServiceNames);

[DllImport(“rstrtmgr.dll”)]
static extern int RmGetList(uint dwSessionHandle,
    out uint pnProcInfoNeeded, ref uint pnProcInfo,
    [In, Out] RM_PROCESS_INFO[] rgAffectedApps,
    ref uint lpdwRebootReasons);

private const int RmRebootReasonNone = 0;
private const int CCH_RM_MAX_APP_NAME = 255;
private const int CCH_RM_MAX_SVC_NAME = 63;

[StructLayout(LayoutKind.Sequential)]
struct RM_UNIQUE_PROCESS
{
    public int dwProcessId;
    public FILETIME ProcessStartTime;
}

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
struct RM_PROCESS_INFO
{
    public RM_UNIQUE_PROCESS Process;
    [MarshalAs(UnmanagedType.ByValTStr,
               SizeConst = CCH_RM_MAX_APP_NAME + 1)]
    public string strAppName;
    [MarshalAs(UnmanagedType.ByValTStr,
               SizeConst = CCH_RM_MAX_SVC_NAME + 1)]
    public string strServiceShortName;
    public RM_APP_TYPE ApplicationType;
    public uint AppStatus;
    public uint TSSessionId;
    [MarshalAs(UnmanagedType.Bool)]
    public bool bRestartable;
}

enum RM_APP_TYPE
{
    RmUnknownApp = 0,
    RmMainWindow = 1,
    RmOtherWindow = 2,
    RmService = 3,
    RmExplorer = 4,
    RmConsole = 5,
    RmCritical = 1000
}

The RmStartSession function is used to create a new Restart Manager session, and the RmEndSession function is used to close a previously started session. RmStartSession returns a handle to the created session, which is then provided to other functions, including RmEndSession, that need to refer to that session.

The other two functions I need deal specifically with resources. The RmRegisterResources function is used to specify a list of files, applications, and services that my installer wants access to, and the RmGetList function returns information about other processes that are using those resources. RmGetList returns that information as an array of RM_PROCESS_INFO structures, each of which contains the process ID of the offending process.

With these interop signatures and structures in place, I can write my method, which is listed in Figure 2. My GetProcessesUsingFiles method accepts a list of file paths and returns a list of System.Diagnostic.Process objects representing any processes using the files at the specified paths. To start the procedure, the method calls the RmStartSession method to let the Restart Manager know we're getting underway; the string key provided as the last parameter to RmStartSession must be unique, so I'm using a GUID.

Figure 2 Using Restart Manager to Detect In-Use Files (Close Figure 2)
public static IList<Process> GetProcessesUsingFiles(
    IList<string> filePaths)
{
    uint sessionHandle;
    List<Process> processes = new List<Process>();

    // Create a restart manager session
    int rv = RmStartSession(out sessionHandle,
        0, Guid.NewGuid().ToString(“N”));
    if (rv != 0) throw new Win32Exception();
    try
    {
        // Let the restart manager know what files we’re interested in
        string[] pathStrings = new string[filePaths.Count];
        filePaths.CopyTo(pathStrings, 0);
        rv = RmRegisterResources(sessionHandle,
            (uint)pathStrings.Length, pathStrings,
            0, null, 0, null);
        if (rv != 0) throw new Win32Exception();

        // Ask the restart manager what other applications
        // are using those files
        const int ERROR_MORE_DATA = 234;
        uint pnProcInfoNeeded = 0, pnProcInfo = 0,
             lpdwRebootReasons = RmRebootReasonNone;
        rv = RmGetList(sessionHandle, out pnProcInfoNeeded,
            ref pnProcInfo, null, ref lpdwRebootReasons);
        if (rv == ERROR_MORE_DATA)
        {
            // Create an array to store the process results
            RM_PROCESS_INFO[] processInfo =
                new RM_PROCESS_INFO[pnProcInfoNeeded];
            pnProcInfo = (uint)processInfo.Length;

            // Get the list
            rv = RmGetList(sessionHandle, out pnProcInfoNeeded,
                ref pnProcInfo, processInfo, ref lpdwRebootReasons);
            if (rv == 0)
            {
                // Enumerate all of the results and add them to the
                // list to be returned
                for(int i; i<pnProcInfo; i++)
                {
                    try
                    {
                        processes.Add(Process.GetProcessById(
                            processInfo[i].Process.dwProcessId));
                    }
                        // in case the process is no longer running
                    catch (ArgumentException) { }
                }
            }
            else throw new Win32Exception();
        }
        else if (rv != 0) throw new Win32Exception();
    }
        // Close the resource manager
    finally { RmEndSession(sessionHandle); }

    return processes;
}

If the session is started successfully, I pass the file paths provided by the user to the RmRegisterResources function (since I only care about files, the parameters dealing with applications and services are ignored). With the resources registered, I call the RmGetList function. The function accepts several parameters, including the session handle, an out parameter that will store the number of processes affected, a parameter detailing the number of elements in the array I'm providing, the array to store returned process information, and a parameter that dictates why a restart is needed (in our case, it's not needed, so RmRebootReasonNone is specified). On this call to RmGetList, I don't know yet how many processes are involved, so I pass 0 into the pnProcInfo parameter. If there are no affected processes, RmGetList will return 0. If it returns the value 234, or ERROR_MORE_DATA, the array I provided is not big enough to store information about the affected processes, which really means that there are affected processes, since the array I passed in was of length 0. But now that I know how many processes are affected, I can call RmGetList again, this time with the correctly sized array. Assuming it returns 0 this time, I loop through the returned RM_PROCESS_INFO structures, using the Process.GetProcessById method and the Process.dwProcessId from each to create a List<Process> collection that I can return to the caller.

This approach to calling RmGetList twice does have a drawback, but it's necessary given the model in which RmGetList is expected to be used. It's possible that between the first call and the second call, the number of processes using the file has changed. As such, even on the second call the array provided might not be big enough to handle all of the information that needs to be returned. In this scenario, I've chosen to throw an exception, but you can modify the method to try calling RmGetList in a loop, either until the array is the correct size or until a certain retry count has been reached.

With this method in hand, it's now a simple task to enumerate all of the processes accessing a specific file, as is demonstrated by the following application. This is a useful tool to compile and leave on your desktop; if you ever have a problem accessing a file, just drag and drop that file onto the app, and it'll tell you what applications/processes are using it:

class Program {
    static void Main(string[] args) {
        IList<Process> processes =
            InUseDetection.GetProcessesUsingFiles(args);
        Console.WriteLine(processes.Count + " processes found:");
        foreach (Process p in processes) {
            Console.WriteLine(p.ProcessName + ": " + p.Id);
        }
        Console.ReadLine();
    }
}

With the resulting Process instances, you can do whatever you want with those processes found to be using your files, including gathering information about them or even killing them. However, think twice (or three times) before doing the latter; that's the whole reason the Restart Manager APIs exist, to politely shut down other applications using resources you want access to while allowing them to save state so they can be restarted without losing any data. Restart Manager is able to shut down applications as well as services. Currently, however, it can only shut down applications that exist in the same user session as the Restart Manager session (so, for example, an application you run can't use Restart Manager to shut down an application being run by another user logged in through Terminal Services). Finally, note that this technique will only return information about processes that are holding an active file handle to the file in question; if an application opened a file, grabbed some data from it, closed the file, and then displayed that data, it may look to you like the file is open by the application, but this procedure won't show that (which is, of course, correct behavior).

For more information on using the Restart Manager APIs, see Daniel Moth's screencast about the subject on Channel 9 at channel9.msdn.com/Showpost.aspx?postid=251492.

Q I've written a generic method, and I understand that this method may be compiled multiple times based on different type arguments provided to it. How can I tell if the method is being reused based on the arguments with which I'm using it, or if it is, in fact, being recompiled for different invocations?

A In short, the native code generated for a generic method is shared when the type arguments are all reference types, and is not shared when any are value types (there are some subtleties here, however, so see blogs.msdn.com/259224.aspx for a bit more information). However, using Visual Studio® 2005, you can pretty easily figure out when, in fact, the generated native code for your managed method is being reused. Simply set a breakpoint in the method and at run time open the Registers debugging window. One of the registers shown will be the EIP register, which is the register storing the address in memory of the next instruction to execute. Every time the breakpoint in your method is hit, check the value of EIP. If the method was only compiled once, every time you record the value of EIP, it'll be the same. And if it ever changes, you know there are multiple instantiations of the generic method.

This is demonstrated by the program shown in Figure 3. I set a breakpoint on the highlighted line and ran the app within Visual Studio. The breakpoint was hit six times, resulting in the following EIP values (these may change from run to run, but the relationships about to be discussed between the values will remain the same):

0073123A
00731297
00731297
00731312
00731297
0073123A

Figure 3 Generic Method to Test Compilation (Close Figure 3)
class Program
{
    static void Main() {
        Test.WriteType<int>();
        Test.WriteType<string>();
        Test.WriteType<object>();
        Test.WriteType<float>();
        Test.WriteType<Test>();
        Test.WriteType<int>();
    }

    class Test {
        public static void WriteType<T>() {
            Console.WriteLine(typeof(T).ToString());
        }
    }
}

As you can see, the values for the 2nd, 3rd, and 5th entries are all the same, since those correspond to the reference type arguments. The 1st and 6th entries are equal but are different from all the others, as the same value type is being used. And the 4th entry is unique as it's the only call using the float value type.

Of course, this technique will only work with JIT compiler implementations that don't support code pitching (the ability to free JIT-compiled code). Thus, it would be ineffective, for example, with code running on the .NET Compact Framework.

Send your questions and comments to netqa@microsoft.com.

Stephen Toub is the Technical Editor for MSDN Magazine.


Netting C++: Mapping Templates to Generics

New information has been added to this article since publication.
Refer to the Editor's Update below.

Netting C++
Mapping Templates to Generics
Stanley Lippman

Code download available at: NettingC++2007_04.exe (165KB)

In this month's column, I'll conclude the port of my ISO-C++ Text Query Language (TQL) application to the Microsoft® .NET Framework and C++/CLI. In particular, I'll drill down into mapping templates and the Standard Template Library (STL) in to the .NET generic facility. Although I worked on the original template implementation for the 3.0 cfront release from Bell Laboratories back in 1991 and have been a strong advocate of their use since, I can't recommend the use of templates within C++/CLI.

Under .NET, the static nature of C++ templates instantiation makes them less than first-class citizens of C++/CLI. In language design, we speak of the primary language entities as exhibiting first-class status. Think of an int: the movement in the 1970s to model abstract data types was an attempt to provide a mechanism so that user-defined types could approach the first-class status of primitive language types such as integer. So it is always legitimate to compare the first- and second class-citizens of a language. And if we do that with a generic and a template object under the .NET Framework, we could say that generics fly first class, and templates unfortunately are not only back in coach, but they're in the last row so they can't even recline.

The problem is that a template is local to the assembly within which it is built. This is because an assembly type is identified not simply by its name, but by its location. Thus, two objects of myTemplate<int> passed between assemblies in a method would be flagged as a type error. So templates cannot be part of the public interface of a type. In addition, they are not language-interoperable.

Generics are tightly integrated into .NET because the underlying intermediate language was extended to explicitly support them. In addition, the runtime automatically manages the instantiation of generics on demand as they are needed. That problem was never solved in C++ due to the nature of the native compilation model in which files are compiled separately and linked together only at run time.

More generally, .NET solves three programming-in-the-large problems that proved intractable in C++ due to the native compilation model: the initialization order of complex global objects (that is, the assurance that an object is constructed before its first use when the use and initialization are in separate files), the management of dynamic heap-allocated memory, and the on-demand instantiation of a parameterized type. These problems, each of which represents a language layer of complexity independent of the actual domain solution, disappear under .NET.

The issue of templates versus generics moves front and center in this column because TQL relies heavily on templates-in particular, on the STL. There are three primary problems to address, and they are all aggravating since this is work we would all prefer not to have to do.

First, my implementation uses the STL vector class. While the STL is not currently available under C++/CLI, the .NET Framework does provide an equivalent generic collection type: List<T>. Declaration-wise, then, I simply have to swap it in. However, I'll have to manually change the usage of each object since the two sets of operations are different. (Alternatively, I could think about writing my own one-to-one mapping of the vector class. I'll consider that later for the STL set associative container.) For example, here are a number of vector declarations:

typedef pair<short,short> location;
typedef vector<location>  loc;
typedef vector<string>    text;
typedef pair<text*,loc*>  text_loc;

To replace vector by List, simply open up the namespace, change the type name, and add the tracking handle (^), since List is a reference class:

using namespace System::Collections::Generic;

typedef pair<short,short> location;
typedef List<location>^   loc;
typedef List<String^>^    text;
typedef pair<text,loc>    text_loc;

Second, as you can see from the previous code sample, this implementation also uses the C++ Standard Library pair<> utility. It is not available with C++/CLI, so I'll have to do something there. It's a pretty simple implementation, so I'll just see if I can clone it. The thing is, nothing maps quite one-to-one between the native and managed platform. Again, that's not surprising, since you need to think of the managed platform as the best thinking on how to make the native platform more successful. (I think the only thing missing right now is equivalent performance.)

Finally, my implementation also uses the STL set class. Unfortunately, there is no equivalent generic collection type, so I'll have to do something. My first thought is to implement a one-to-one mapping of its interface-except I cannot hijack heap allocation under the .NET Framework. But, as you'll see, that turns out to be a less than good idea. Here's an example of the TQL use of set:

class Query {
public:
    // ...

private:

    set<short>*       vec2set( const vector<location>* );
    vector<location>  loc;
    set<short>        *solution;
};

So let's see how to solve the issue with the C++ Standard Library pair<> utility. The pair<> utility is exceedingly trivial. Its value is not so much in the elegance of its design as the fact that it is standardized as part of the C++ Standard Library. However, it does exhibit some template magic that only partially migrates to the .NET Framework and generics. Figure 1 depicts what the ISO-C++ Standard shows us of pair<>'s implementation.

Figure 1 Standard pair<> Implementation (Close Figure 1)
// from Section 20.2.2 Pairs
template <class T1, class T2>
struct pair {
    typedef T1 first_type;
    typedef T2 second_type;

    T1 first;
    T2 second;

    pair();
    pair(const T1& x, const T2& y);
    template<class U, class V> pair(const pair<U, V> &p);
};

template <class T1, class T2>
bool operator==(const pair<T1, T2>& x, const pair<T1, T2>& y);

template <class T1, class T2>
bool operator<(const pair<T1, T2>& x, const pair<T1, T2>& y);

template <class T1, class T2>
pair<T1, T2> make_pair(T1 x, T2 y);

I'll maintain the unencapsulated public nature of pair<> by keeping it a struct. In addition, I'll define it as a value class since I want to minimize its overhead. And, as I indicated at the start of the column, I'll implement it as a generic. The skeleton declaration looks as follows:

generic <class T1, class T2>
value struct pair { ... };

This supports the previous use of pair<> , such as:

typedef List<String^>^    text;
typedef pair<text,loc>    text_loc;

It also supports the sorts of uses illustrated in Figure 2.

Figure 2 Uses of pair<> (Close Figure 2)
int main(array<System::String ^> ^args)
{
    /* generates:
        first: 10, second: 20
        first: hello, second: world
        first: 1024, second: eek!
    */

    pair< int, int > pi( 10, 20 );
    pair< int, int >::first_type t1 = pi.first;
    Console::WriteLine( “first: {0}, second: {1}”,
                        t1, pi.second );

    pair< String^, String^ > ps( “hello”, “world” );
    pair< String^, String^ >::second_type t2 = ps.second;
    Console::WriteLine( “first: {0}, second: {1}”,
                        ps.first, t2 );

    pair< int, String^ > psi =
        pair< int,String^ >::make_pair( 1024, “eek!” );

    pair< int, String^ >::first_type ft = psi.first;
    pair< int, String^ >::second_type st = psi.second;
    Console::WriteLine( “first: {0}, second: {1}”, ft, st );

    return 0;
}

So, let's look at the implementation, the easiest part of which is the data:

generic <class T1, class T2>
value struct pair
{
    typedef T1 first_type;
    typedef T2 second_type;

    T1 first;
    T2 second;

I was holding my breath to see whether the trait pattern carries over into generics. The typedef-ing of the type parameter to a type-transparent name is one of the more elegant template patterns. (I would suspect that you cannot do this in C#. If you can't, that is one neat advantage C++/CLI has over C#.) In this case, first_type and second_type can be used transparently, as I did in Figure 2:

pair< int, String^ >::first_type ft  = psi.first;
pair< int, String^ >::second_type st = psi.second;

The next set of pair elements we need to deal with are the three constructors, and generics provides mixed results in this case.

According to the standard, the default constructor for pair zeroes out the pair of objects, T1 and T2. Value types cannot support a default constructor because the compiler cannot guarantee that it can always intervene to generate the appropriate invocation. In this case, however, that is more than fine since each value type is guaranteed to be zeroed out by the runtime prior to being handed over to you. So, for this one, the .NET Framework is the winner, so to speak. Removing code, in general, is always an improvement.

The two-argument constructor takes a T1 and a T2. This is largely the same within C++/CLI, except for two issues in the following declaration:

pair(const T1& t1, const T2& t2 );

The first issue is const, which the .NET Framework does not recognize. On the C++/CLI design team, one school of thought is if the framework doesn't recognize const and a function with a const argument is passed to the framework, then we can't guarantee const-ness and therefore mustn't support it. The counter argument, which initially prevailed, was that C++ programmers use const as an interface design toggle and we can guarantee this surface-that is, compile-time-behavior. And, it was claimed, this is very important to them.

The problem with the initial design choice is that it was justified by a supposition. It claimed to know what is most important to the hearts and minds of C++ programmers. That's a difficult conversation to navigate. After all, most of us can make plausible assertions about what C++ programmers want-naturally we won't agree but we all believe that we know what is right. Unfortunately, beliefs are what we get passionate about, and they typically gain importance over technical issues. In a sense, our implementation of templates was more a supposition than a technical decision.

The immutability of a value (that is, its const-ness) is not something we tend to argue about, however passionate our nature. As much as C++ programmers want their const, they want their const to remain const even more. That was just something the compiler cannot guarantee given the absence of const within the System namespace-except perhaps by disallowing all calls into that namespace that want to pass a const parameter. This was felt to be unsupportable. So we reversed ourselves and removed support for const.

The second issue in the two-constructor signature is the declaration of the parameterized type name as a reference:

const T2& t2

You have to do this in native programming because of the performance penalty of passing a large user-defined type by value to the function being invoked. Under the .NET Framework, this is not an issue. A value type should not be more than 16 bytes; reference types are all passed by reference. So the revised C++/CLI two-constructor signature looks like this:

 pair( T1 t1, T2 t2 ): first( t1 ), second( t2 ){}.

I would characterize this two-argument constructor as a wash when viewed as moving from one platform to the other.

The third constructor is a template copy constructor (at least in the case where T1 equals U and T2 equals V):

template<class U, class V> pair(const pair<U, V> &p);

This is not supported under C++/CLI generics. Member function generics, in general, are supported, such as this:

generic <class T1, class T2>
value struct pair {
    generic <class U, class V> void foo( U, V ); // ok
    // ...
};

But my attempts to have a generic constructor member function result in a rather inscrutable error message, which left me unsure whether the message was a compiler error or an aspect of the common language runtime (CLR) specification:

error C2061: syntax error : identifier 'pair<T1,T2>'
error C2238: unexpected token(s) preceding ';'

The following shows the two additional syntaxes I tried:

generic <class T1, class T2>
value struct pair
{
    // neither compile ...
    generic <class U, class V> pair<U,V>( int );
    generic <class U, class V> pair( pair<U, V>% );

    // ok: but not quite equivalent
    // note: not considered a copy constructor
    pair( pair<Object^, Object^>% );

    // ...
};

As it turns out, the CLR implementation does not support a generic constructor. (Unfortunately, a clearer error message probably won't make it into the next Visual Studio® release.) Therefore, in scoring this, I would give the point to templates. The entire pair implementation is shown in Figure 3.

Figure 3 The Pair Implementation (Close Figure 3)
generic <class T1, class T2>
public value struct pair {
    typedef T1 first_type;
    typedef T2 second_type;

    T1 first;
    T2 second;

    pair( T1 t1, T2 t2 ): first( t1 ), second( t2 ) {}

    static bool operator ==( pair<T1,T2> x, pair<T1,T2> y )
    {
        return x.first == y.first && x.second == y.second;
    }

    static bool operator<( pair<T1,T2> x, pair<T1,T2> y )
    {
        return x.first < y.first ||
           (!( y.first < x.first) && x.second < y.second );
    }

    static pair<T1, T2> make_pair( T1 x, T2 y )
         { return pair( x, y ); }
};
[ Editor's Update - 3/14/2007: The code in Figure 3 will compile under Visual C++ 2005. However, due to the nature of generics, it shouldn't have, and a fix to the compiler was implemented as part of Visual C++ 2005 SP1 that will prevent this from compiling (SP1 is available for download). For a pair struct like the one implemented in Figure 3, there are several solutions. One is to use templates instead of generics (simply changing "generic" to "template" ). Another is to constrain the generic parameters to implement IComparable<T> methods on IComparable<T> can then be used within the struct definition to compare instances of the generic parameters.]

Finally, I have to deal with the set class abstraction provided by the STL. In the .NET Framework 2.0, there is not a set representation within the Generic collection namespace. I don't see any alternative except to cook up something of my own-which is unfortunate since presumably we'll all end up cooking our own slightly different implementation, and that makes for an awfully big set. Figure 4 shows the ISO-C++ specification of the public interface for set. On one level, I imagine people expect I would implement the full STL specification. That was what I first assumed until I looked at it closely.

Figure 4 ISO/C++ Set Public Specification (Close Figure 4)
template<class Key, class Compare = less<Key>,
         class Allocator = allocator<Key> >
class set {
public:
    // types:
    typedef Key key_type;
    typedef Key value_type;
    typedef Compare key_compare;
    typedef Compare value_compare;
    typedef Allocator allocator_type;
    typedef typename Allocator::reference reference;
    typedef typename Allocator::const_reference const_reference;
    typedef implementation defined iterator; // See 23.1
    typedef implementation defined const_iterator; // See 23.1
    typedef implementation defined size_type; // See 23.1
    typedef implementation defined difference_type;// See 23.1
    typedef typename Allocator::pointer pointer;
    typedef typename Allocator::const_pointer const_pointer;
    typedef std::reverse_iterator<iterator> reverse_iterator;
    typedef std::reverse_iterator<const_iterator> const_reverse_iterator;

    // 23.3.3.1 construct/copy/destroy:
    explicit set(const Compare& comp = Compare(),
                 const Allocator& = Allocator());

    template <class InputIterator>
    set(InputIterator first, InputIterator last,
        const Compare& comp = Compare(), const Allocator& = Allocator());
    set(const set<Key,Compare,Allocator>& x);
    ˜set();
    set<Key,Compare,Allocator>& operator=
        (const set<Key,Compare,Allocator>& x);
    allocator_type get_allocator() const;

    // iterators:
    iterator begin();
    const_iterator begin() const;
    iterator end();
    const_iterator end() const;
    reverse_iterator rbegin();
    const_reverse_iterator rbegin() const;
    reverse_iterator rend();
    const_reverse_iterator rend() const;

    // capacity:
    bool empty() const;
    size_type size() const;
    size_type max_size() const;

    // modifiers:
    pair<iterator,bool> insert(const value_type& x);
    iterator insert(iterator position, const value_type& x);
    template <class InputIterator>
    void insert(InputIterator first, InputIterator last);
    void erase(iterator position);
    size_type erase(const key_type& x);
    void erase(iterator first, iterator last);
    void swap(set<Key,Compare,Allocator>&);
    void clear();

    // observers:
    key_compare key_comp() const;
    value_compare value_comp() const;

    // set operations:
    iterator find(const key_type& x) const;
    size_type count(const key_type& x) const;
    iterator lower_bound(const key_type& x) const;
    iterator upper_bound(const key_type& x) const;
    pair<iterator,iterator> equal_range(const key_type& x) const;
};

Why, for example, would anyone want a reverse iterator for a set, let alone both a const and non-const instance of that? The functional abstraction of a set has little connection with, I would guess, 80 percent of its STL interface. That is, about 80 percent of the members of set are a requirement of the STL, not of the set abstraction. So that's pure STL overhead; there is nothing I really need in that. So I'm going to (virtually) toss it out. My set looks like this:

generic <typename element>
public ref class set : IEnumerable {
public:
    set();

    void add( element );
    bool find( element );
    void remove( element );

    int  count();
    bool empty();
    void clear();

    virtual IEnumerator^ GetEnumerator()
};

And then, of course, I need to swap this set interface in place of the STL set of calls. This is the same problem as moving from vector to List. Space doesn't permit me to walk through any of the implementation details, but you can view that and the entire TQL .NET port in the download from the MSDN®Magazine Web site.

Send your questions and comments for Stanley to purecpp@microsoft.com.

Stanley Lippman began working on C++ with its inventor, Bjarne Stroustrup, in 1984 at Bell Laboratories. Later, Stan worked in feature animation both at Disney and DreamWorks and served as a Software Technical Director on Fantasia 2000. He has since served as Distinguished Consultant with JPL, and an Architect with the Visual C++ team at Microsoft. He is currently Senior Software Engineer at Emergent Game Technologies.


{ End Bracket }: Geopegging

{ End Bracket }
Geopegging
Joshua Trupin

Code download available at: EndBracket2007_04.exe (174KB)

As part of my birthday haul last year, I received a gem of a gift: a new camera. And not just any camera, either. It was a new Nikon D200.

One of the features that really grabbed my interest was its built-in GPS capabilities. If you had a standard serial-port GPS device, you could plug it into the camera via a converter cable and location data would be written into each shot you took.

The problem was that the serial-port GPS receivers I owned were museum pieces that didn't work with the camera, and new GPS devices tend to have USB connections. Arrgh! I decided to take the most foolhardy plunge since I bought fajitas at a Manhattan street fair two years ago. I created my own homebrew solution out of OEM parts. To make a long story short, it actually worked. If you want the heartstopping play-by-play, you can find it at my blog.

Geopegging Adds GPS Location Data to a JPG
Geopegging Adds GPS Location Data to a JPG (Click the image for a smaller view)
Geopegging Adds GPS Location Data to a JPG
Geopegging Adds GPS Location Data to a JPG (Click the image for a larger view)

I've been putting my photos up on my hosting site, trupin.smugmug.com. Smugmug has a nice feature that automatically maps all your tagged photos, and even though it used Google Earth, I liked it. But I knew I could take this idea a step further. Hundreds, perhaps thousands, of photos could soon be turned from throwaway snapshots into vitally important records of time and place. Adding location data to photos is known as geotagging, but my program would extend that term with a slightly more specific concept, "geopegging."

Geopegging, which is now defined as "adding GPS location data to a JPG through a map-based interface," is a concept that took a few steps to drag into reality. First, I had to figure out where the geo data actually went inside a JPG file. There's surprisingly little information about this online, but I used a variety of sources and the power of the Microsoft® .NET Framework to create a simple EXIF editor. If you want to find the camera's lens aperture at the time of the photo, you look at the field tagged 0x9202. It turns out that the GPS data all goes in tags numbered 0 through 8. This didn't fill me with confidence, because I imagined someone choosing the first numbers they could think of, well after the fact. The Framework includes a GetPropertyItem member in the System.Drawing.Image class, so it's easy to extract header data if you know the EXIF tag value.

Next, I wrote a program that hosted Microsoft Internet Explorer® inside a Windows® Forms app so that I could display Virtual Earth™ and allow a user to select the location to be added to a photo. That was a bit easier. The fun part was getting Virtual Earth working within a frame of my desktop app so that it could communicate back and forth with the Windows Forms code in the app. This part involves a bit of JavaScript inside the window plus some tricks (documented in the Virtual Earth online help) that gets the two parts of the program talking to one another.

The trick here is to wire up the WebBrowser control with some script through its DocumentText property. You use AttachEvent to call a script function when the browser control is clicked; this function then calls window.external.YourFunctionName and passes it a string.

There's more complexity, including handling the Virtual Earth control properly, converting values, and writing to the JPG file. I've put my basic geopegging program up as a download and invite you to take a look at it. Now I've got a world full of tagged photos, and I can update them any time I want. I don't need the homemade GPS device at all...but I still use it, if only to show off a bit.

Joshua Trupin is the Executive Editor of MSDN Magazine and TechNet Magazine.